I have personally seen guys drive their full race cars all over the place on public roads at the Pebble Beach Concours. And I mean P1 LeMans cars and the likes. During that weekend, it seems that the cops just sort of look the other way. Yes, those owners could care less...
I agree that for the average Joe, the liability of driving something built for race only and unregistered on the road leaves you wide open to losing everything.
Any insurance you might have would simply tell you that you're on your own.
And in an accident, you would be facing gross negligence of motor vehicle code, among probably many other bad things including possible loss of the vehicle. And God forbid you hurt someone!