Web www.M3Forum.net

View Full Version : Down time over the last 24 hours


Sirius
Wed, Aug-26-2009, 11:54:29 PM
I want to apologize about the down time and offer an explanation.

Yesterday around 5pm or so I started to notice a lot of Database error emails. When I jumped on the forum, I found it to be crashed. I contacted our host and asked them to repair our database and scheduled maintenance for 2AM EST Wed morning.

During the maintenance, something went awry and the repair failed. Considering this was around 6AM this morning, they decided to revert to an earlier back up -- instead of contacting me first. I noticed that around 12PM this afternoon and asked the hosting company to revert to the backup they created right before the repair last night and to repair that backup. During that repair process something failed again... but this time we were missing about 1 Gb worth of data.

During the few times I could log in, I noticed a few sections of the forum had lost over five years of posts. Many threads, although listed, lost all of their content and other odd bugs and errors. After getting on the phone with the hosting company, we decided it would be best to load Tuesdays backup. However, this comes with a loss of approximately 24 hours of information.

I do apologize for this, but with the options at hand, this is the best we could pull off.

daytonaviolet
Wed, Aug-26-2009, 11:55:25 PM
downtime sucks :(

Sirius
Wed, Aug-26-2009, 11:59:32 PM
downtime sucks :(

:agree:

I have been dealing with it since yesterday afternoon. :mad:

KILO15
Thu, Aug-27-2009, 12:25:36 AM
Thanks for the hard work bro:beer:

tom @ eas
Thu, Aug-27-2009, 01:28:40 AM
No worries - grateful it was only 24 hours instead of 5 years lost.

deilenberger
Thu, Aug-27-2009, 02:17:45 AM
As we say in the computer field - poop happens.

beebo100
Thu, Aug-27-2009, 02:20:33 AM
thanks for the explanation

calvinf1
Thu, Aug-27-2009, 02:45:51 AM
Thanks for the hard work and explanation.... lets hope everything is ok so that I can get through my work day tomorrow.... I can only handle so much of 46fanatics in one day...

405
Thu, Aug-27-2009, 05:04:28 AM
Thanks for the hard work and explanation.... lets hope everything is ok so that I can get through my work day tomorrow.... I can only handle so much of 46fanatics in one day...

rofl
couldnt have said it better myself.

skelzy
Thu, Feb-18-2010, 05:12:37 AM
Good job rectifying it....You made the right choice....

ApexSpeed
Thu, Jun-21-2012, 11:44:50 PM
No-win situation, it sucks. I've been there. Minimize the damage, get the product available again and move on.


Nice work. http://www.m3forum.net/m3forum/images/icons/icon14.gif

lucad2
Fri, Jun-22-2012, 12:44:35 AM
Thanks for getting it back up and running, I seriously went though withdrawal today

Sirius
Fri, Jun-22-2012, 01:35:39 AM
Well, it was up before I went to bed around 5am EST. Didn't notice it until this afternoon. Sorry about that.

e46jan
Fri, Jun-22-2012, 03:40:18 AM
Lol I started to think my work comp blocked m3f....phew thanks mods

tavenger5
Mon, Jun-25-2012, 05:53:21 PM
Sirius, this exact thing has happened to me in the past with my vbulletin sites.

One thing that will help if a table crashes in the future, the post table in particular, is moving attachments and avatars out of the database and to the file system. The database will be significantly smaller and this will also help with database stress when the site is loaded.

Sirius
Mon, Jun-25-2012, 06:29:13 PM
Yup. The post table has had it's issues many times. Actually, before I opted to restore a backup yesterday, mysqlcheck had been stuck on the post table for over 8 hours trying to repair it. :facepalm:

And all images are on the file system. So, our Db is only 4.9 GB in size right now.

Busabos77
Mon, Jun-25-2012, 06:33:47 PM
I was forced to go to E46 just to browse... Nothing like m3forum... I was sad! LOL!

brichter
Mon, Jun-25-2012, 08:28:07 PM
Wait a minnit, what about the 3000 posts I made during that time? :facepalm:

They're really all gone? :cry:











Good job, 24 hours is better than 5 years. :grouphug:

(Sorry, had to post ***** it up for #500! :rulaiz:)

Sirius
Tue, Jun-26-2012, 03:19:20 AM
^^ Yep. I'll agree to that. Lol.

bmwJohhnyD
Tue, Jun-26-2012, 03:39:16 AM
Good job, 24 hours is better than 5 years. :grouphug:




exactly . but we probably missed about 10 threads of herra frush and poke and stretch stance threads. :facepalm:

oh yeah and a few "what kind of oil should i use threads"

and dont forget the " is this a good deal fread" :rofl3:


anyways. glad it is back!!

tavenger5
Tue, Jun-26-2012, 09:17:24 PM
Yup. The post table has had it's issues many times. Actually, before I opted to restore a backup yesterday, mysqlcheck had been stuck on the post table for over 8 hours trying to repair it. :facepalm:

And all images are on the file system. So, our Db is only 4.9 GB in size right now.

Yack! I've had that happen too. In my case I wasn't allocating enough memory to certain functions of mysql. I would highly recommend putting in an optimization request: https://www.vbulletin.com/forum/forumdisplay.php/14-Server-Configuration

or better yet, hire George (eva2000) to tweak the server. I did and he improved the speed of my sites a good amount.

Fury
Fri, Jul-06-2012, 12:58:20 PM
What happened yesterday? It seemed to be running good after the last fix.

Sirius
Fri, Jul-06-2012, 07:08:51 PM
It was... until we ran out of hard drive space (which mysql really doesn't like). We are switching to two 500GB drives tonight, which will provide plenty of space for a long time.

bmwJohhnyD
Fri, Jul-06-2012, 07:13:47 PM
It was... until we ran out of hard drive space (which mysql really doesn't like). We are switching to two 500GB drives tonight, which will provide plenty of space for a long time.

two 500gb!<@?@!

wtf. i have a 3tb drive full of porn, cant we do better than that?

do we need to start a paypal collection? m3forum can not be contained to a measly raided 500gb!

Sirius
Fri, Jul-06-2012, 08:35:39 PM
The current setup is two 160 GB... lol.

bmwJohhnyD
Fri, Jul-06-2012, 10:59:02 PM
The current setup is two 160 GB... lol.

dang. how much down time we looking at honestly. >12 but <24? :facepalm:

bmwJohhnyD
Sun, Jul-08-2012, 02:12:41 PM
>24.. sigh.

glad to be back.

sorry to complain for something that i dont pay for. i dont really mean it.

HerrSMiG
Wed, Jan-23-2013, 06:38:59 PM
I just assumed Ains took the forum offline after the King's opener so Craig couldn't talk shit.

King Luis
Thu, Jan-24-2013, 07:55:45 PM
I just assumed Ains took the forum offline after the King's opener so Craig couldn't talk shit.

:rofl2:

IFTPC