logo Sign In

Post #1551951

Author
Jay
Parent topic
Recent OT.com outage
Link to post in topic
https://originaltrilogy.com/post/id/1551951/action/topic#1551951
Date created
30-Aug-2023, 2:07 PM

Han

The forum was offline for several days due to an error that crashed the app server. I have a monitoring service in place to notify me about these outages since I don’t participate in the forum discussions on a daily basis, but the monitoring service failed for unknown reasons. I’m evaluating new, redundant monitoring services to prevent this issue from recurring.

There is also a script to restart the site in the event of crashes, which has always worked reliably in the past (it’s been triggered numerous times, and it works so quickly no one even knows there was an outage). I’m investigating that as well.

The moderators also contacted me via e-mail, but my inbox is a nightmare at the moment and I’m days behind on responding to messages, so that’s my fault. I became too comfortable with the systems I have in place to handle these issues. I will monitor my OT inbox more closely in the future.

No data was lost and I see no evidence of a security breach. Just a cascade of dumb failures—both electronic and human (me)—leading to a bad outcome.

I apologize for the inconvenience and any concern this may have caused. We’re not going anywhere 😃