moderated Event: Website outage #outage - Tuesday, 21 August 2018 #outage #cal-invite Calendar <main@...>

Website outage #outage

Tuesday, 21 August 2018
6:50am to 7:50am
(GMT-07:00) America/Los Angeles


At approximately 6:50am, the website started to become unresponsive. Internally, the web servers were rebooting because they were running out of database connections. I originally thought that this was a database issue, but after much investigation, I was able to track down the problem. A fix I pushed to the site yesterday afternoon affecting Chat was now causing database connections to leak. I have temporarily disabled the Chat function until I can fix the bugfix. Email delivery was not affected during this time.

I will post a post mortem either later today or tomorrow, after I've reinstated Chat and had some time to consider what, if any, changes to make based on this outage.

Join to automatically receive all group messages.