Oh, there we go. Turns out one of the S3 configuration options got renamed in Mastodon 2.5.0.
I'm using 1.6.1, remember.
Now that I can see new media is working fine, time to upload old media...
@KS It's somewhat funny that the moment you post a lot is when stuff has broken. :P