I would never get any work done if I had to wait on the Realtor or broker to arrive. In my area the Realtor boards prohibit the Realtors from being present at the inspections. The key box lease is the only way to go. Less travel time chasing keys down and less interaction with the Realtors.
I have had a few interesting inspections in the college town I live in. I could tell you stories....