In fact you wouldn't have to go slow at all, it's very flat and smooth, not even one pothole. Except for the bridge--clunk-clunk-clunk... A certain canoe paddler has driven up and down the old railbed nervously, with a 37-point turn at the end... On the hike in, I once had to make way for a Ford Ranger with two fat guys in it, pulling a trailer with a boat.
I thought about it too, my car easily makes it between the two boulders. But it's a gamble. Algonquin Park kind of expects it from big-gut cigarette-smoking fishermen and would just slap them with a fine. Same for paddlers perhaps, but they may react differently, and restrict access even more--does anybody really want to be the one who ruined it for others?
--C.