Hi TJ, I've successfully been able to track what I do 24x7 for multiple months at a time but not by using a Calendar. I'm guessing that the four days of silence to your question is probably a good indicator that there isn't a silver bullet solution to accurate and easy time tracking via a calendar.
In my case, I chose to build my own solution. It's free for anyone to use. Here's a real-life snapshot of 2 weeks of my life from earlier this year:
and here's a real-life visualization of how my time allotments vary over three months:
Both charts get created automatically in "real time" as I log Events. Lots of times I end up "back-filling" events but a lot of the data is entered as soon as an activity is complete, either via the website ( eventloggers.com) on desktop or phone or via Google Home or Google Assistant Voice Logging.
If those images are too small to see the details, you can see larger versions and other examples on the website.
It'd be interesting to use a calendar app to PLAN what you think your day/week will look like and then use EventLoggers.com to track what you actually do. Then you can compare. I don't have that functionality built in eventloggers but it's easy to export all your events, including start and end timestamps from the website.
I hope this helps to at least give you some options. I know it's not exactly what you asked for.
Just thinking off the the top of my head... it's possible to enter future Events in eventloggers.com, so you could populate your "planned" day the evening before and then manually compare/edit the Events that really happened but there's no comparison engine built.
Hope this helps,