The ablative of agent is expressed with ā or ab, and denotes an agent associated with a passive verb. In basic cases, this means the [ab + ablative] unit would be the nominative subject in an active construction.
Hats are worn by these men, but scorned by those men: capellī ab hīs gestantur, sed ab illīs spernantur.
These men wear hats, but those men scorn hats: hī capellōs gestant, sed illī spernant.
He was brought to trial by his sons: ā fīliīs in iūdicium vocātus est.
His sons brought him to trial: eum fīliī in iūdicium vocāvērunt.
According to AG, this construction is developed from the ablative of source. “The agent is conceived as the source or author of the action.” -AG, 405n2
How is this not a chicken/egg scenario? They don’t work to justify their claim, but it might be that claiming a ‘source’ is a perceived ‘agent’ offers agency to all things, whereas claiming an ‘agent’ is a ‘source’ merely relates a relationship between two things.
The ablative agent may appear with active verbs, but only where they are intransitive and allude to a passive meaning.
She was killed by the elephants: periit ab elephantīs