moderated Event: Website outage #outage - Tuesday, 21 August 2018 #outage #cal-invite
firstname.lastname@example.org Calendar <main@...>
Website outage #outage
At approximately 6:50am, the website started to become unresponsive. Internally, the web servers were rebooting because they were running out of database connections. I originally thought that this was a database issue, but after much investigation, I was able to track down the problem. A fix I pushed to the site yesterday afternoon affecting Chat was now causing database connections to leak. I have temporarily disabled the Chat function until I can fix the bugfix. Email delivery was not affected during this time.
I will post a post mortem either later today or tomorrow, after I've reinstated Chat and had some time to consider what, if any, changes to make based on this outage.