The size of the service is probably just fine. One needs to do a load calculation, or take measurements, to be able to say there's a problem.
Indeed, one could argue that supplying 80 amps to a home that only needs, say, 50 amps, is safer than supplying it with 100 amps.
I'd never advise a service change simply because the equipment was 'old.'
Age is a clue, though. Apart from the actual condition of the equipment, it's a suggestion that you look for improper electrical work that has been done over the past fifty years. That's where the dangers are found.
Heck, until recently my place was doing just fine with two circuits, and a 30 amp, 120 volt service. I certainly never blew a fuse ..... and yes, they were the right sizes
Naturally, my old service would not have been able to handle the addition of an electric range, electric water heater, air conditioning, or a electrically heated hot tub.
Yet, even with a fancy new breaker panel, my home's electrical system is unchanged - all I have is a big box with empty spaces. The wiring inside the house hasn't changed a bit since 1940. Two circuits, no ground wire, very few outlets.
In that regard I'm lucky. No flying splices, no 3-prong receptacles, no confused shared neutrals .... even at the light fixtures, there' been little over-lamping (as the insulation hasn't turned to dust). Those are the types of things to focus on.
Arguing with an inspector is like wrestling a pig in the mud, pretty soon you realize the pig enjoys it.
