Event Related Object Fields

I guess what I am struggling with is… my understanding is Events are polymorphic and Salesforce locks it down pretty tight. I try to access the objects via merge syntax but from what I can tell I don’t have access to all the fields on the object through “WhatId” on the event.  In some cases like in formulas I don’t even have access to the Name I only have access to the ID.  

My thought/hope is that after my end users select the record they are connecting to through the “WhatId” I could have a Model linked to that field to take the ID and autofill some additional custom fields on the event.  Is that possible?

Another use case would be auto-populating “Location” on the Event using the Contact’s address. 


Any thoughts?