Pages: 1
  Print  
Author Topic: Back from a little downtime.  (Read 9296 times)
Offline (Female) IsmAvatar
Posted on: September 17, 2011, 07:49:52 pm

LateralGM Developer
LGM Developer
Location: Pennsylvania/USA
Joined: Apr 2008
Posts: 877

View Profile Email
We experienced a little downtime recently where most pages displayed either an SMF Database error or the header displayed funky like it didn't load fully. This mostly only affected enigma-related or database-dependent pages, as other pages hosed on the server worked fine.

The prior problem was caused by a database corruption which may be related to the latter problem.
The latter problem was someone wrote the header in a non-html5 compliant way.

We've repaired the database and made the header compliant, so this issue should be fully resolved.

We apologize for any confusion caused during this time, but we're glad to announce that we're back up and going strong.

Special thanks to sirmxe for figuring this out and fixing it.
Logged
Post made September 19, 2011, 08:39:18 am was deleted at the author's request.
Offline (Female) IsmAvatar
Reply #2 Posted on: October 04, 2011, 01:02:37 am

LateralGM Developer
LGM Developer
Location: Pennsylvania/USA
Joined: Apr 2008
Posts: 877

View Profile Email
At around 20:45 (EST) on Oct 1, we began experiencing another similar SMF Database error which took down the entirety of the enigma-dev website. We informed Gary, and he came on at around 1:45 and resolved it immediately.

The issue was that we overflowed our httpd error logs, exceeded our disk quota, and were unable to assign new database pids. The fix was simply to delete the logs, freeing up 30g of space.

This leaves us with 2 pieces of good news, and 1 piece of bad news.
The good news is that it's very easy to fix, and should it happen again, we can fix it much quicker without needing to wait for gary.
The bad news is that, since we didn't look at the logs, we're not really sure what caused them to overflow in the first place, so it could happen again in the semi-near future (probably a couple weeks from now)
The other good news is that if and when it happens again, we'll take a peek in the logs and hopefully be able to fix it for good.
Logged
Pages: 1
  Print