i think ive discovered why the system keeps falling over after 24-48 hours. its been haunting me since earlier this year, and seemed to get worse when the messaging queues exploded in quantity from the 1.5.0 upgrade....
finished the upgrade to 1.5.1. everything should be functional at this point....
i have implemented all sorts of rate limiting at the protocol/database level to keep the server from falling over....
its taken a few days for the traffic to settle down, but i think were synced back up. i have increased messaging workers, and it appears to be helping.
finished upgrading the system to mbin 1.4.1. the upgrade necessitated the install of amqproxy, which has turned out to greatly increase system performance....
i added the shared list of bad mastadon/misskey instances to the defederation list
i have added some connection rate limiting to keep bigger instances from hosing everything. lemmy.world started causing some issues.
automated backup is setup/configured, etc....
finally found an instance type to match resource usage.. need at least 8gb of ram to keep from falling over....
something stupid is happening causing processing of inbound ap messaging to take forever, leaving the system in a constant state of catch-up. cant seem to figure out how to increase consumption throughput (worker count does not solve the issue).
the t3a isnt responsive enough for burst use, so i just went straight t3
this shit was broken, yo. a recent package update broken image uploading to s3. fixed with some acl syntax adjustment....
too hard keeping it running, lets see if mbin stabilizes the message processing issues.
i have established this to serve as my home, and maybe others in this new post-reddit,twitter world. may they rest in peace....