Forum Outage

[Calum]

That should be it now.

Imgur plug and probably emoticons need fixing.

[Calum]

Emoticons fixed

Imgurs working as well.

If there is anything I forgot, post here.

[oldman]

Thanks Calum!

[Zrako]

Great job @Calum , thank you for keeping the forum running

[ricky0115]

Nice work man keep it up!

[admiral323]

your doing a great job.much appreciated

[Calum]

How odd, forum seems to be playing up.

Could be the HDD failing so I will back up the database and monitor it.

We may lose data if it fails.

[NINJA]

@Calum Is everything fixed now? As it seems awfully quiet in here???

[Calum]

@NINJA May do, stats say the site is drawing in over 400,000 hits a month with over a 4000 unique visitors a month.

Think of all that money in ad revenue...that I don't make

[NINJA]

@Calum Missed opportunity there bud!!! :smiling_face_with_open_mouth_cold_sweat: :face_with_tears_of_joy:

[NINJA]

@Calum The forum is going down a lot lately, as I keep getting the 'Down for maintenance message'. This is happening at different times of the day, so I'm not sure if it's a hardware, server or hoster issue? Can you look into it please bud.

[Calum]

@NINJA Oh really? I've not seen this my end.

Is it the 503 Message? With the Blue Screen?

It's pay day this weekend, maybe I ought to upgrade a few things.

[NINJA]

@Calum No just the 'down for maintenance message' - I'll screenshot it for you next time.

[NINJA]

@Calum The site was just down with the blue 503 message.

[Calum]

@NINJA Yeah I know the one. I guess the poor server is just tasked to capacity. I don't think there really is much I can do about that. A faster server maybe.

[NINJA]

@Calum I've not had the blue 503 message for a long while, but the other message is more frequent. You should ask the server company to put more 50p's in the meter slot more regularly!!! :grinning_face_with_smiling_eyes: :smiling_face_with_open_mouth_closed_eyes: :smiling_face_with_open_mouth_cold_sweat: :face_with_tears_of_joy:

[Calum]

@NINJA Yeah okay bud.

It looks like the server has been crashing and rebooting quite a fair bit by the looks of it.

Everytime you seen that error message, that means the server has just died.

There are three threads of this forum running at any one time. These are proxied to a single thread of the Database. It looks like when one thread dies, the proxy load balances to another thread. If you kill all three of them, you'll be facing the blue message whilst you wait for the processes to come back alive.

I think I'm going to have to get some new hardware over the next few weeks.

I will investigate other options in the meantime.

We may lose some data...

[NINJA]

@Calum OK good_2_know that you know what's going on. Hopefully you can get it all sorted and not lose to much???

I just got this message;

ERROR
MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.

Followed by this when I refreshed the page;

[Calum]

@NINJA yeah that is just nginx x. Looks like all three processes died.

[Calum]

Okay this is getting ridiculous.

I've isolated the problem and it sounds like the forum is getting a little too big for its boots.

I don't think the hardware is failing, which is good news. But it does sound like it is bottlenecked. Unfortunately the hardware isn't upgradeable. Especially the parts that it is having problems with.

Ideally I think the solution would be to migrate the database to a different technology. This would result in a slower website, but would be infinitely more reliable.