Shane,
You’ll put the function and field of whichever field is the “number” field in the Fields to order records by property of the aggregate model. For example, I have an aggregate Account model that groups by Industry and has an aggregation on Id. If I want to show the records sorted by count high to low, then I put COUNT(Id) DESC into the Fields to order records by property