Having volunteered at a few SCCA/RallyAmerica events, I'm fairly confident there were course marshals at any trail crossings. At the rally events, they would send sweep cars through to open/close the course and would shut down the event if people were in places they shouldn't be. Most of the time we also had a radio relay to communicate with course control if there were any issues and could flag cars mid-stream if needed.
More than 20 years ago my wife & I decided to drive up Mt. Washington in our Honda Civic. After paying at the booth our car failed inspection as it it did not have an option to downshift it into first gear, so we got vouchers to go up in a van. The driver supplemented the usual recording with some extra commentary and she said that supposedly every day there was a driver who refused to drive down their car from Mt. Washington. If I remember correctly they were charging $10 at the time to drive these cars back to the bottom of the hill.
I guess one thing about the auto race is that the participants don't need to deal with other drivers who are just too scared to drive when they get too close to road's edge. Still having watched the video, I think this is really crazy. I don't see how he could have had much grip over the gravel part of the road.