TLS server move progress thread

Ryushikaze

Deus Admiral Parsimonious, PHD, DDS, MD, JD, OBE
AKA
Tim, Ryu
There was some slowdown at around 10PM EST for awhile. Seems to happen rather consistently each night, as far as folks can tell. Might be some backup process running. Any clues?
 

Tifabelle

Pro Adventurer
AKA
Tifabelle, Nathan Drake, Locke Cole, Kain Highwind, Yamcha, Arya Stark
It was down from like 10pm until yopy fixed it around 3am. He mentioned to me that he'll have a better look at it over the weekend when he has more time. Cause this needs to not happen every damn night :P
 

Cthulhu

Administrator
AKA
Yop
:bigmonster: well, it'll probably keep happening for a while. I made a few tweaks to the server just now, but I don't think they'll be a proper solution. What I did:

* Tune MySQL down, so it uses less memory.
* Change the max timeout from 300 seconds (5 minutes) to 15 seconds. This means your request will time out if it takes more than 15 seconds. This may be too low though, for example if you're on a slow mobile connection or something. I'll up it to 30 to be sure. This does mean, at least in theory, that processes will clear up sooner if they take too long to finish. Plus it'll give you a neat error message sooner, :monster:.
* Was going to change some apache settings so apache uses less memory, but that would reduce the amount of simultaneous users down to 10 or something, so I'll hold off on that for now.

The solution I'm aiming for, and which I hope to get in place in the weekend, is to run PHP in a separate process and to replace the memory-hogging Apache with the more lightweight nginx.
 

Cthulhu

Administrator
AKA
Yop
There was another last night, but this time I was still up so :monster:

My diagnosis: I haven't a clue wtf is going on, :monster:. Will see to trying to optimize the server further today / tomorrow, or just upgrade the server - ad revenue still covers that cost.
 

Dashell

SMILE!
AKA
Sonique, Quexinos, Pinkie Pie, Derpy Hooves
Maybe try another webhost? I have no idea with who but if that's the case can you get your money back?
 

Cthulhu

Administrator
AKA
Yop
Nope, they just supply a virtual server, what you do with it - and how it performs - is all up to whoever operates it, in this case, me :monster:. Apache and MySQL just aren't low-memory bits of software. I'll fiddle with it tomorrow, see what I can fixx.
 

Fangu

Great Old One
Yop, it's fine. Take your time. No pressure. Your effort is appreciated.
Just saying this because I think you're a bit fed up with the web hosting issue atm :P
(Of course you'll never admit it.)
 

Cthulhu

Administrator
AKA
Yop
I won't admit it 'cause it's not true, :monster:. I like screwing around with the server, I just don't like it if / when the site becomes unavailable :monster:.

Hacking at vB or Wordpress now, that's something I don't like doing :monster:
 

Dashell

SMILE!
AKA
Sonique, Quexinos, Pinkie Pie, Derpy Hooves
Yeah I'm not trying to rush you or anything dude, you obviously know way more than I do. I just want to make sure you're not getting screwed.

It does seem to be getting better at least though :monster:
 

Cthulhu

Administrator
AKA
Yop
Bugger me with a stick, :monster:.

Alright, so I broke the site for a bit today. I tried to configure PHP to run as a FastCGI module, i.e. not as a part of Apache, and fucked some stuff up so you could actually download .php files for a while - which is not good.

I stopped the server and tried to fix it, but couldn't get it to work, so I eventually just removed and reinstalled the webserver completely, starting from the ground up.

Followed these instructions. Which... sorta worked.

Restarted the server, 500 / internal server errors ensued. Checked out the error log, turns out the thing ended up in a redirect loop between the public_html folder and some CGI working folder (if that makes any sense), a known issue for Wordpress installations, as is shown in one of the comments of that article. Added an exception for redirecting in the /usr/lib/cgi-bin folder which seems to have fixed that.

Sooo... now PHP scripts are handled outside of the webserver (Apache) process. Next up is diving into those settings and see if there's anything to optimize. Instead of a dozen Apache processes, there now seem to be a dozen PHP processes running. I'll see to fixxing that, and if that don't work either, I'll just order the more expensive hosting thingy.

@you guys, I removed and reinstalled some, please see to trying out stuff and see if there's anything not working properly, k thx bai.
 
Bugger me with a stick, :monster:.

Alright, so I broke the site for a bit today. I tried to configure PHP to run as a FastCGI module, i.e. not as a part of Apache, and fucked some stuff up so you could actually download .php files for a while - which is not good.

I noticed. I was about to preview a personal message I was writing and all I got was a PHP download offer. Fortunately it all seems to work fine now...for the time being.
 

Cthulhu

Administrator
AKA
Yop
Yeah, that was a bit of a security risk, actually, as you could also download the configuration file and determine the MySQL password and such, :monster:. I don't think that's a big problem in this case - there's no PHPMyAdmin configured and you can't access the database from the outside - but still.
 

Fangu

Great Old One
I won't admit it 'cause it's not true, :monster:
The amounts of STFU's I got at #tls tells me otherwise.

Or you knew you were screwing it up and was starting to sweat and chew your nails.

Which of course you wouldn't admit either.


NO U.
 

Cthulhu

Administrator
AKA
Yop
Naw, I was slightly annoyed by the distraction and the repeated stating of the obvious, :monster:. I'm working on it, I know it's b0rked until I tell it's not.


Also, related, Flintlock was able to download the unexecuted .php files for a while after fixxing it, this was probably due to his browser caching the returned files. A clearing of ye cache fixxed that too.
 

Fangu

Great Old One
If you cba (:monster:), you should announce your fiddles up front. Then you won't be bothered by the annoying members trying to help you out with error messages :monster:
 

Cthulhu

Administrator
AKA
Yop
There wasn't supposed to be any downtime until I actually changed stuff, :monster:. But Apache just restarted itself b0rkenly when I installed the php-fastcgi package. Not my fault :monster:

But I should get a virtual machine with the same setup as the TLS server, so I can do practice runs of this kinda stuff.
 

Dashell

SMILE!
AKA
Sonique, Quexinos, Pinkie Pie, Derpy Hooves
Well it hasn't been down in a while so good job on whatever you did :monster:

The only problem I have now is the main page of the forums loads slow, but that's the only one and I have no idea what that is. Maybe it's just me.
 

Cthulhu

Administrator
AKA
Yop
Yeah, I noticed that too. I'm looking at the stats on the page for that one... It might also be Memcached being slow or swapping or something though, I should turn it off for a bit to see if that makes the site slower or faster. It's probably just complex queries that are hard to cache.
 
Top Bottom