Fixed the issue -- it seems that the default S3 upload timeout of Mastodon is too low for our self-hosted distributed storage cluster, and an underperforming node today triggered a lot of failure. Increased the timeout for now. Note that attachments may take longer to upload until the performance of the cluster restores.
Our Mastodon media storage seems to be broken still -- uploads fail frequently. Trying to figure out what happened.
Our media storage server experienced an unknown failure last night probably due to the remote-attached Hetzner volume. It has now been recovered.
We have finished migration and upgrade to Mastodon v3.3.0.
We have finished server migration.
We have finished migration of the database server. We will proceed to migrate the main application server in the coming days.
This server is going DOWN for database migration within 30 minutes
This server is going DOWN for database migration within 2 - 3 hours.
We will upgrade to Mastodon 3.3.0 after we migrate to the new server.
We have updated to Mastodon v3.2.1, and as a change in 3.2.1 introduced stricter authorization requirements in AUTHORIZED_FETCH mode, our federation with Pawoo would probably become buggy and unstable. There's nothing we can do to fix this except urging Pawoo to migrate to a version newer than 3.0.0. We do not want our user content to be fetched by some of our blocked instances.
Our server is back running again and everything seems to be fine for now.
This instance is going DOWN for migration RIGHT NOW.
This server is going DOWN for migration within 2 hours.
We will be performing instance migration some time between the next few hours to tomorrow. Expect the instance to be down for some time during the migration process.
SN.Angry.Im has been upgraded to Mastodon v3.2.0.
Future notices and other instance-related messages of SN.Angry.Im will be provided on this account.
Administration account for SN.Angry.Im
