Cthulhu
Administrator
- AKA
- Yop
Hi all, as some of you may have noticed (lmao), we've had a few days of downtime.
We went down at around 6 AM CET due to planned maintenance at our host, Leaseweb; they mentioned they needed to do some updates on their network infrastructure, and because of that they would take our server offline.
However, after the maintenance was done at 9 AM, our server did not come back online. I couldn't get the server to restart so I had to contact support. Turns out that both the hard drives in the server had failed and needed to be replaced. Since this was going to take some time, I enabled a maintenance page on Cloudflare, which I put in front of TLS a while ago just in case.
Since this is a dedicated server, it was physical hardware and entirely managed by us; the host took no responsibility for the contents of the broken disks. Thankfully, I had a backup, but it was from nearly 24 hours before the site went down so unfortunately anything added or changed on monday the 5th has been lost.
So yesterday and today there was a lot of back and forth with this support staff to get the disks replaced. I think they have been replaced now and the dedicated server is available again, but... I had lost trust basically, and was fed up having to manage the server myself. It was a lot of complicated configuration, let's just leave it at that.
I went to look for a different host instead; instead of a decidated server, I wanted to move back to virtual hosting. That way, if there's a hardware failure, the systems in place will transparently move our server to different hardware. They can do smarter things with backups too.
I've had a look around and found some Opinions that recommended Hetzner, a German cloud provider with very agreeable rates and services. With just a few clicks and a few seconds waiting I had them set up a new virtual server, triple core CPU, 4GB memory, 80GB disk drive + a 200 GB storage volume,with a very transparent pricing scheme; our total monthly bill has also halved with this.
What is still outstanding is redoing the backups - while the main hard drive, including the forum and front page databases, are backed up now, the 200GB volume is not; that one contains all the files from the website and such. But that can wait a little while. And furthermore there were some other security things I need to fix.
Oh, we're also on a newer version of PHP; the version of Xenforo we had was from 2018 and didn't work with the newer version. I've spent some money on a much-needed update for that one. We're seeing a number of non-breaking but annoying visual issues showing up, we'll need to spend a bit of time to fix these - mostly things like our custom badges and the like, nothing too serious.
Feel free to use this thread to mention issues; particularly look for missing data; there should be nothing missing outside of posts made on monday.
We went down at around 6 AM CET due to planned maintenance at our host, Leaseweb; they mentioned they needed to do some updates on their network infrastructure, and because of that they would take our server offline.
However, after the maintenance was done at 9 AM, our server did not come back online. I couldn't get the server to restart so I had to contact support. Turns out that both the hard drives in the server had failed and needed to be replaced. Since this was going to take some time, I enabled a maintenance page on Cloudflare, which I put in front of TLS a while ago just in case.
Since this is a dedicated server, it was physical hardware and entirely managed by us; the host took no responsibility for the contents of the broken disks. Thankfully, I had a backup, but it was from nearly 24 hours before the site went down so unfortunately anything added or changed on monday the 5th has been lost.
So yesterday and today there was a lot of back and forth with this support staff to get the disks replaced. I think they have been replaced now and the dedicated server is available again, but... I had lost trust basically, and was fed up having to manage the server myself. It was a lot of complicated configuration, let's just leave it at that.
I went to look for a different host instead; instead of a decidated server, I wanted to move back to virtual hosting. That way, if there's a hardware failure, the systems in place will transparently move our server to different hardware. They can do smarter things with backups too.
I've had a look around and found some Opinions that recommended Hetzner, a German cloud provider with very agreeable rates and services. With just a few clicks and a few seconds waiting I had them set up a new virtual server, triple core CPU, 4GB memory, 80GB disk drive + a 200 GB storage volume,with a very transparent pricing scheme; our total monthly bill has also halved with this.
What is still outstanding is redoing the backups - while the main hard drive, including the forum and front page databases, are backed up now, the 200GB volume is not; that one contains all the files from the website and such. But that can wait a little while. And furthermore there were some other security things I need to fix.
Oh, we're also on a newer version of PHP; the version of Xenforo we had was from 2018 and didn't work with the newer version. I've spent some money on a much-needed update for that one. We're seeing a number of non-breaking but annoying visual issues showing up, we'll need to spend a bit of time to fix these - mostly things like our custom badges and the like, nothing too serious.
Feel free to use this thread to mention issues; particularly look for missing data; there should be nothing missing outside of posts made on monday.
, definitely keep 
for consistency.