-
Search disabled, maintenance, and other changes
In order to improve the stability of the current forums, as well as to prepare for an upgrade in the future, I switched the forums off to perform some maintenance tasks.
If these tasks were successful, they were scheduled to take several hours. If unsuccessful, we'd know pretty much immediately and revert to an unchanged state. Unfortunately, due to the extremely large size of the forum database, this process ended taking up far longer than anticipated. In the end, I made the decision to dump the entire search index because it was the only remaining table and would take far too long to finish the maintenance. As such, searching has been disabled.
How much this helps the current server remains to be seen, but it's important to remember that these maintenance tasks performed now will reduce the time needed in the final upgrade by an almost 1:1 ratio, as they need to be performed before or during the upgrade. However, it comes with a caveat. Due to the time taken to perform these tasks, I may need to revise and greatly increase the estimate on how long the final upgrade will take.
The services and settings are still being tuned. I will be monitoring and making further changes. So, you can expect the forum to be switched off or unavailable for shorter periods in the future as the adjustments take place.
If stability and speed is greatly improved, I may start the process of reindexing the forums, in order to reenable search. This will take several days and have the effect of slowing the server right back down during this time.
In addition, there will be future maintenance tasks that may need to be performed in the future. This may require the forums to be switched off again for several hours at a time, but hopefully not for as long a time as the recent tasks, until the final upgrade takes place.
-
Re: Search disabled, maintenance, and other changes
I just wanted to drop by and say thanks for all the hard work you, and the other admins do to keep the forum the great place it is!
:smallsmile:
-
Re: Search disabled, maintenance, and other changes
Is it likely that we will lose posts in this future upgrade?
-
Re: Search disabled, maintenance, and other changes
Servers are significantly better, thanks for all the hard work!
-
Re: Search disabled, maintenance, and other changes
::Gasp cough sputter:: Ahhh!!! Thank goodness, I was having some nasty withdrawal symptoms here. :smallwink:
Way to go, Rawhide! Hope the Search function is up soon. Google should work in the interim.
-
Re: Search disabled, maintenance, and other changes
Alright! It's good to know what was going on for those three days, and that it'll happen again in short bursts before the final upgrade. I was ailing without my PMs there for a while, so I guess it's time to copy the pertinent ones into a document again so I'm not without them for any length of time.
So hey, thanks for the advanced warning on the rest of these. This one wound up blindsiding me and my little group down in freeform, but I'll be sure to check site issues more often from now on so I'm not caught so offguard next time.
And on that note, tata for now.
-
Re: Search disabled, maintenance, and other changes
Thank you very much, rawhide. I know it's been a long work.
I second the desire that search functionality be restored when practicable. We might as well start fresh with a new database. There is no value to keeping old posts around if they cannot be found. Inaccessible posts are no better than deleted posts. I'd rather have a clean database with intact search than a tremendous old database I cannot find anything in.
Also, I have a professional interest in mysql. I'm rather curious as to exactly what problems were had, because I intend to construct a similar database for my company.If there are scaling issues that can be better solved by Oracle or Db2 or SQL Server, I'd really, really like to know about it.
Again, thank you very much for your hard work!
Respectfully,
Brian P.
-
Re: Search disabled, maintenance, and other changes
Quote:
Originally Posted by
pendell
I'm rather curious as to exactly what problems were had, because I intend to construct a similar database for my company.
The database was huge, I underestimated how long it would take. It needed to be done, I would have given more warning and better messages had I known. That's pretty much it, there weren't any issues.
-
Re: Search disabled, maintenance, and other changes
I'll just chime in and say that the site is running much, much better for me too. Thanks!
-
Re: Search disabled, maintenance, and other changes
I don't know if you guys can, but it would have been nice to have the info in the news section, seeing as we couldn't get to the forum. Otherwise, I think it already feels faster! :smallbiggrin:
Likely a Placebo effect
-
Re: Search disabled, maintenance, and other changes
Quote:
Originally Posted by
Madara
I don't know if you guys can, but it would have been nice to have the info in the news section, seeing as we couldn't get to the forum. Otherwise, I think it already feels faster! :smallbiggrin:
Likely a Placebo effect
There was a message left which you could see when you tried to access the forums.
Though, for a couple of hours in the middle, the message did say "backup in progress - 30 mins". But other than that, you had all the information I had.
-
Re: Search disabled, maintenance, and other changes
Oh, GITP, I have missed you so much. Rawhide, is there a place where I can hug the forum? You know what? I don't care.
*Huggles*
-
Re: Search disabled, maintenance, and other changes
Yay, my mandatory forum vacation is over!
-
Re: Search disabled, maintenance, and other changes
Me yesterday evening:
Glad to have the forums back up. Thanks for letting us know about the future upgrades.
As for searching, we can always type site:giantitp.com into Google and search that way, so it's not all bad.
-
Re: Search disabled, maintenance, and other changes
-
Re: Search disabled, maintenance, and other changes
I'd like to mention that not more than 20 minutes ago when I was trying to load a thread I got a "500 Internal Server Error" like I would tend to get back before the 3 days down-time.
So I hope that the maintenance helped in the long run, because I'm just noticing slower load times and that one error so far.
-
Re: Search disabled, maintenance, and other changes
Quote:
Originally Posted by
Troll Brau
I'd like to mention that not more than 20 minutes ago when I was trying to load a thread I got a "500 Internal Server Error" like I would tend to get back before the 3 days down-time.
So I hope that the maintenance helped in the long run, because I'm just noticing slower load times and that one error so far.
Probably due to the fact that the server had just been rebooted.
-
Re: Search disabled, maintenance, and other changes
Quote:
Originally Posted by
Diego Havoc
Am I the only one who feels this way too?
Hopefully it doesn't take that long next time. :smallbiggrin:
-
Re: Search disabled, maintenance, and other changes
Well, the upside here is that science has been advanced, and I think we can conclusively say that forum addiction is a thing that exists. *shudder*
In all seriousness, thanks for the update, Rawhide, and I do find the site loading faster, at least right now.
-
Re: Search disabled, maintenance, and other changes
Well... it's hard to complain about a free service. I'm just glad we're back, and I can go back to DM'ing my games.
I'm sorry the database was such a handful for you, and I'm grateful for the work you do to make this such a great place to post in.
-
Re: Search disabled, maintenance, and other changes
I can use site search on Google if I really need to find something.
Generally it works almost as well, so no worries.
Thanks for all the hard work you put into this site. It is a real treat to be part of such a great community. :smallsmile:
-
Re: Search disabled, maintenance, and other changes
Quote:
Originally Posted by
pendell
I second the desire that search functionality be restored when practicable. We might as well start fresh with a new database. There is no value to keeping old posts around if they cannot be found. Inaccessible posts are no better than deleted posts. I'd rather have a clean database with intact search than a tremendous old database I cannot find anything in.
As Ravens cry points out, Google search makes every post accessable.
Use "site:giantitp.com <search terms>". It's nifty. :smallsmile:
-
Re: Search disabled, maintenance, and other changes
And so ends the most productive three-day stretch I've had in six years. :smallwink:
I, too, am grateful to Rawhide for all the hard work keeping all the ones and zeros in line around here.
-
Re: Search disabled, maintenance, and other changes
Quote:
Originally Posted by
Roland St. Jude
And so ends the most productive three-day stretch I've had in six years. :smallwink:
I, too, am grateful to Rawhide for all the hard work keeping all the ones and zeros in line around here.
The real bright side is that you didn't have a single rules violation, scrubbed post, or locked thread in 3 full days. I wonder if that's a record?
-
Re: Search disabled, maintenance, and other changes
On the other hand, I don't think there was ever a larger portion of the userbase thinking and muttering profanities while looking at the forum :smalltongue:
-
Re: Search disabled, maintenance, and other changes
So if search is down, does the fancy searching trick via Google still work? I'm trying to remember how it works, but it'd be something like giantitp.com:Searchgoeshere or something. Right? Would that still let us search for things on the site without the in-site search?
-
Re: Search disabled, maintenance, and other changes
You just need "site:www.giantitp.com" (no quotes) before or after your search terms. And, yes, it will still work just fine, as it's using google and not the forum search.
-
Re: Search disabled, maintenance, and other changes
Quote:
Originally Posted by
Savannah
You just need "site:www.giantitp.com" (no quotes) before or after your search terms. And, yes, it will still work just fine, as it's using google and not the forum search.
Yeaaaaah that was it. I wonder if maybe we could get the Search button back, but just have it redirect to a Google search pre-loaded with that in it? Or is that too much work/too dumb of an idea?
-
Re: Search disabled, maintenance, and other changes
Regarding the site search. Is it possible to use google to search a specific thread? I've been trying with little results.
-
Re: Search disabled, maintenance, and other changes
Quote:
Originally Posted by
Xondoure
Regarding the site search. Is it possible to use google to search a specific thread? I've been trying with little results.
Yes. Put the thread title in quotes. It's not perfect (you will sometimes get some outside results), and some thread titles are too generic for good results, but it generally works.
-
Re: Search disabled, maintenance, and other changes
Forums are back, huzzah!
I'm not sure if it matters, but yesterday or the day before I began to receive a different error when I visited to see if the forums were back up. Instead of loading the website with the "forums are offline, etc" message it showed a white Cloudfare page that suggested the system was down (paraphrased).
Don't know if that's a helpful technical report but I'd thought I'd mention it.
Also, another technical report. While trying to use Google to read various threads I noticed that Google Cache wasn't very effective in getting what I wanted. Almost all of the Google Cache version of the pages pulled up showed the "forums are offline, etc" page instead of the Cached page. A few of them did load though, which only made the whole thing stranger.
-
Re: Search disabled, maintenance, and other changes
Quote:
Originally Posted by
Rawhide
Yes. Put the thread title in quotes. It's not perfect (you will sometimes get some outside results), and some thread titles are too generic for good results, but it generally works.
You can narrow down the search further by adding the following search time to Google:
site:giantitp.com/forums
-
Re: Search disabled, maintenance, and other changes
Also, note that you can type -search term to not display any results with the indicated term. Sometimes helps to narrow things down.
-
Re: Search disabled, maintenance, and other changes
Well that was interesting. And by interesting, I mean horrific. :smalltongue:
Thanks for going through all that trouble for us, Rawhide. You're awesome. :smallbiggrin:
-
Re: Search disabled, maintenance, and other changes
I can say this much. The forum is much, much faster and much less prone to errors and server busy signals.
In fact, I haven't had any of either so far!
That in itself deserves a round of applauds:smallsmile:
-
Re: Search disabled, maintenance, and other changes
Quote:
Originally Posted by
Chambers
Forums are back, huzzah!
I'm not sure if it matters, but yesterday or the day before I began to receive a different error when I visited to see if the forums were back up. Instead of loading the website with the "forums are offline, etc" message it showed a white Cloudfare page that suggested the system was down (paraphrased).
Don't know if that's a helpful technical report but I'd thought I'd mention it.
Just wanted to add that I saw this for a while yesterday as well.
Also: YAY FORUMS BE BACK!
-
Re: Search disabled, maintenance, and other changes
Thanks for all your hard work, Rawhide! :smallsmile:
-
Re: Search disabled, maintenance, and other changes
So, did anyone notice the backup?
-
Re: Search disabled, maintenance, and other changes
no. maintenence didnt happen tonight did it?
-
Re: Search disabled, maintenance, and other changes
Quote:
Originally Posted by
Rawhide
So, did anyone notice the backup?
There was no backup, right? I've been righting and posting stuff uninterrupted for the last 45 minutes.
-
Re: Search disabled, maintenance, and other changes
Backup took place between 3:00am to 3:25am, CDT (it's now 3:40am, CDT).
-
Re: Search disabled, maintenance, and other changes
I may have noticed it, had my phone decided to not go on the fritz.
But I would also like to add my thanks, Rawhide. The site loads way faster than it used to for me. And I haven't gotten a single 500 server error yet.
Now if only I could steam to act the same way...
-
Re: Search disabled, maintenance, and other changes
I'll just be honest and admit that the sudden downtime was frustating, mostly because I hadn't noticed any warning prior to it, and that perhaps the first PBP game I've joined that didn't die within five pages is hosted here.
OI still do understand need for maintenance, and this will hopefully prove usefull. Hopefully future observation(something my current, sleep-deprived state is incapable of), will hopefully show the forum to be more reliable accesability-wise.
-
Re: Search disabled, maintenance, and other changes
Quote:
Originally Posted by
Rawhide
Yes. Put the thread title in quotes. It's not perfect (you will sometimes get some outside results), and some thread titles are too generic for good results, but it generally works.
I think if you want to search for threads you can use intitle to get threads with those words. For instance searching for this thread you can do site:giantitp.com intitle:"search disabled".
edit:Just thought of an improvement, add in "-intitle:page" if you want to get one result per thread instead of one for each page of the thread
-
Re: Search disabled, maintenance, and other changes
Quote:
Originally Posted by
Rawhide
As such, searching has been disabled.
I suppose it's a little price to pay and, anyway, the function was far from functioning in a good way.
Google works for threads... a lot less when you search for specific words from a user, but again, it's an acceptable loss.
Thanks for the amazing work!
-
Re: Search disabled, maintenance, and other changes
Backup coming up in just over half an hour. Watch out for it.
-
Re: Search disabled, maintenance, and other changes
Can't actually check from here, but it should be in progress right now.
-
Re: Search disabled, maintenance, and other changes
Wait, we're getting backups without the forums going down now?
-
Re: Search disabled, maintenance, and other changes
Rawhide posted that message a couple of hours ago, hun. :smalltongue: I didn't log on to the site between 9:00 and 9:30 GMT (which is when it usually goes down) so I couldn't say.
-
Re: Search disabled, maintenance, and other changes
Well, the backup completed successfully.
-
Re: Search disabled, maintenance, and other changes
Quote:
Originally Posted by
Killer Angel
I suppose it's a little price to pay and, anyway, the function was far from functioning in a good way.
Google works for threads... a lot less when you search for specific words from a user, but again, it's an acceptable loss.
Thanks for the amazing work!
Understandable to a point, but one shouldnt need to be a programmer just to search the database. Hopefully we can bring that feature back soon...
-
Re: Search disabled, maintenance, and other changes
It's hardly "being a programmer". I should know - I did nothing maths-y or computer-y at uni!
Search strings are really really common and very useful overall. I use them all the time with my general searches. Putting things in quotes, using +term or -term to include or exclude a search term, and site:www.yyy.zzz are vital to getting Google to do what I actually want and not make too many (terrible) assumptions on its own.
Edit: Rawhide, is there a maintenance scheduling thread or something I can subscribe to, so that I don't need to remember to check in here every time I log in? Just like "Gonna be doing maintenance tomorrow, 2 hours or so, 6pm GMT+11".
-
Re: Search disabled, maintenance, and other changes
:smallfrown: But I can't use it to search for people talking about me stalk particular posters find most recent posts by or about a particular person for totally legit and not-at-all self-obsessed reasons.
-
Re: Search disabled, maintenance, and other changes
Oh really? Then my super sneaky post about you in one of the forum's dustier corners shall remain safely hidden! :smallamused:
-
Re: Search disabled, maintenance, and other changes
I'm not going totally nuts not being able to find search, that's good.
Thank you for all your hard work; databases take up 14 layers of the hells, I'm convinced.
-
Re: Search disabled, maintenance, and other changes
Er, excuse me Mr. Rawhide sir, I don't seem to be able to find a list of my previous posts on the forum, is that due to the disabling of Search? Is there some other way of accessing my post record?
-
Re: Search disabled, maintenance, and other changes
Quote:
Originally Posted by
willpell
Er, excuse me Mr. Rawhide sir, I don't seem to be able to find a list of my previous posts on the forum, is that due to the disabling of Search? Is there some other way of accessing my post record?
Yes it is, no there isn't. Sorry.
-
Re: Search disabled, maintenance, and other changes
Quote:
Originally Posted by
willpell
Is there some other way of accessing my post record?
It sure would be great if there were some way to search on the internet...
-
Re: Search disabled, maintenance, and other changes
I recently got a new form of error message, for the first time, and I want to know if its a result of the changes.
Its not that it presented problems, and it was fine after refreshing the page. I just noticed it was a different page and thought I should bring it up.
-
Re: Search disabled, maintenance, and other changes
Quote:
Originally Posted by
Madara
I recently got a new form of error message, for the first time, and I want to know if its a result of the changes.
Its not that it presented problems, and it was fine after refreshing the page. I just noticed it was a different page and thought I should bring it up.
Nah, that's unrelated to the maintenance work.