Search disabled, maintenance, and other changes

[Rawhide]

In order to improve the stability of the current forums, as well as to prepare for an upgrade in the future, I switched the forums off to perform some maintenance tasks.

If these tasks were successful, they were scheduled to take several hours. If unsuccessful, we'd know pretty much immediately and revert to an unchanged state. Unfortunately, due to the extremely large size of the forum database, this process ended taking up far longer than anticipated. In the end, I made the decision to dump the entire search index because it was the only remaining table and would take far too long to finish the maintenance. As such, searching has been disabled.

How much this helps the current server remains to be seen, but it's important to remember that these maintenance tasks performed now will reduce the time needed in the final upgrade by an almost 1:1 ratio, as they need to be performed before or during the upgrade. However, it comes with a caveat. Due to the time taken to perform these tasks, I may need to revise and greatly increase the estimate on how long the final upgrade will take.

The services and settings are still being tuned. I will be monitoring and making further changes. So, you can expect the forum to be switched off or unavailable for shorter periods in the future as the adjustments take place.

If stability and speed is greatly improved, I may start the process of reindexing the forums, in order to reenable search. This will take several days and have the effect of slowing the server right back down during this time.

In addition, there will be future maintenance tasks that may need to be performed in the future. This may require the forums to be switched off again for several hours at a time, but hopefully not for as long a time as the recent tasks, until the final upgrade takes place.

[DeusMortuusEst]

I just wanted to drop by and say thanks for all the hard work you, and the other admins do to keep the forum the great place it is!

[Winfred]

Is it likely that we will lose posts in this future upgrade?

[BobVosh]

Servers are significantly better, thanks for all the hard work!

[Telonius]

::Gasp cough sputter:: Ahhh!!! Thank goodness, I was having some nasty withdrawal symptoms here.

Way to go, Rawhide! Hope the Search function is up soon. Google should work in the interim.

[Slii Arhem]

Alright! It's good to know what was going on for those three days, and that it'll happen again in short bursts before the final upgrade. I was ailing without my PMs there for a while, so I guess it's time to copy the pertinent ones into a document again so I'm not without them for any length of time.

So hey, thanks for the advanced warning on the rest of these. This one wound up blindsiding me and my little group down in freeform, but I'll be sure to check site issues more often from now on so I'm not caught so offguard next time.

And on that note, tata for now.

[pendell]

Thank you very much, rawhide. I know it's been a long work.

I second the desire that search functionality be restored when practicable. We might as well start fresh with a new database. There is no value to keeping old posts around if they cannot be found. Inaccessible posts are no better than deleted posts. I'd rather have a clean database with intact search than a tremendous old database I cannot find anything in.

Also, I have a professional interest in mysql. I'm rather curious as to exactly what problems were had, because I intend to construct a similar database for my company.If there are scaling issues that can be better solved by Oracle or Db2 or SQL Server, I'd really, really like to know about it.

Again, thank you very much for your hard work!

Respectfully,

Brian P.

[Rawhide]

Originally Posted by pendell

I'm rather curious as to exactly what problems were had, because I intend to construct a similar database for my company.

The database was huge, I underestimated how long it would take. It needed to be done, I would have given more warning and better messages had I known. That's pretty much it, there weren't any issues.

[Rising Phoenix]

I'll just chime in and say that the site is running much, much better for me too. Thanks!

[Madara]

I don't know if you guys can, but it would have been nice to have the info in the news section, seeing as we couldn't get to the forum. Otherwise, I think it already feels faster!
Likely a Placebo effect

[Rawhide]

Originally Posted by Madara

I don't know if you guys can, but it would have been nice to have the info in the news section, seeing as we couldn't get to the forum. Otherwise, I think it already feels faster!
Likely a Placebo effect

There was a message left which you could see when you tried to access the forums.

Though, for a couple of hours in the middle, the message did say "backup in progress - 30 mins". But other than that, you had all the information I had.

[CoffeeIncluded]

Oh, GITP, I have missed you so much. Rawhide, is there a place where I can hug the forum? You know what? I don't care.

*Huggles*

[Chess435]

Yay, my mandatory forum vacation is over!

[Diego Havoc]

Me yesterday evening:

Spoiler

Glad to have the forums back up. Thanks for letting us know about the future upgrades.

As for searching, we can always type site:giantitp.com into Google and search that way, so it's not all bad.

[Yora]

I wonder if it helped.

[Tanuki Tales]

I'd like to mention that not more than 20 minutes ago when I was trying to load a thread I got a "500 Internal Server Error" like I would tend to get back before the 3 days down-time.

So I hope that the maintenance helped in the long run, because I'm just noticing slower load times and that one error so far.

[Rawhide]

Originally Posted by Troll Brau

I'd like to mention that not more than 20 minutes ago when I was trying to load a thread I got a "500 Internal Server Error" like I would tend to get back before the 3 days down-time.

So I hope that the maintenance helped in the long run, because I'm just noticing slower load times and that one error so far.

Probably due to the fact that the server had just been rebooted.

[Mynxae]

Originally Posted by Diego Havoc

Me yesterday evening:

Spoiler

Am I the only one who feels this way too?

Hopefully it doesn't take that long next time.

[Anarion]

Well, the upside here is that science has been advanced, and I think we can conclusively say that forum addiction is a thing that exists. *shudder*

In all seriousness, thanks for the update, Rawhide, and I do find the site loading faster, at least right now.

[inexorabletruth]

Well... it's hard to complain about a free service. I'm just glad we're back, and I can go back to DM'ing my games.

I'm sorry the database was such a handful for you, and I'm grateful for the work you do to make this such a great place to post in.

[Ravens_cry]

I can use site search on Google if I really need to find something.
Generally it works almost as well, so no worries.
Thanks for all the hard work you put into this site. It is a real treat to be part of such a great community.

[The Bushranger]

Originally Posted by pendell

I second the desire that search functionality be restored when practicable. We might as well start fresh with a new database. There is no value to keeping old posts around if they cannot be found. Inaccessible posts are no better than deleted posts. I'd rather have a clean database with intact search than a tremendous old database I cannot find anything in.

As Ravens cry points out, Google search makes every post accessable.

Use "site:giantitp.com <search terms>". It's nifty.

[Roland St. Jude]

And so ends the most productive three-day stretch I've had in six years.

I, too, am grateful to Rawhide for all the hard work keeping all the ones and zeros in line around here.

[Anarion]

Originally Posted by Roland St. Jude

And so ends the most productive three-day stretch I've had in six years.

I, too, am grateful to Rawhide for all the hard work keeping all the ones and zeros in line around here.

The real bright side is that you didn't have a single rules violation, scrubbed post, or locked thread in 3 full days. I wonder if that's a record?

[TheFallenOne]

On the other hand, I don't think there was ever a larger portion of the userbase thinking and muttering profanities while looking at the forum

[Deth Muncher]

So if search is down, does the fancy searching trick via Google still work? I'm trying to remember how it works, but it'd be something like giantitp.com:Searchgoeshere or something. Right? Would that still let us search for things on the site without the in-site search?

[Savannah]

You just need "site:www.giantitp.com" (no quotes) before or after your search terms. And, yes, it will still work just fine, as it's using google and not the forum search.

[Deth Muncher]

Originally Posted by Savannah

You just need "site:www.giantitp.com" (no quotes) before or after your search terms. And, yes, it will still work just fine, as it's using google and not the forum search.

Yeaaaaah that was it. I wonder if maybe we could get the Search button back, but just have it redirect to a Google search pre-loaded with that in it? Or is that too much work/too dumb of an idea?

[Xondoure]

Regarding the site search. Is it possible to use google to search a specific thread? I've been trying with little results.

[Rawhide]

Originally Posted by Xondoure

Regarding the site search. Is it possible to use google to search a specific thread? I've been trying with little results.

Yes. Put the thread title in quotes. It's not perfect (you will sometimes get some outside results), and some thread titles are too generic for good results, but it generally works.