Apparently the last time WordPress updated itself, something changed. I noticed I was unable to login to the admin page, but the normal blog worked fine.
I tried a bunch of different fixes I found online, but the problem was that the wp-admin/ folder was set to group writeable permissions (0775), rather than owner write only. Changing permissions on the folder to 0755 fixed the problem for me.
I tried resetting my theme as part of the fix. I actually kind of like this theme (the default) so I might keep it for a while.