Yes a new/better driver would definitely be a big improvement.
But I need to use technology to compensate for my lack of talent so I upgraded the rear stock VE callipers to a pair of 4 spot Wilwoods. (AP's couldn't fit under my OZwheels). The Stock VE Rears used to get quite hot.
Using the Territory Disc rotor as it sits back little and Aussie Mikes Machine shop made an adapter the Rotor for my VE rear with a nice spigot ring.
Oh and put in some braided lines on the Rear as well.