Site downtime

[Cthulhu]

Hi all,

Apologies for the two days downtime; I got notified that TLS was down yesterday morning. I contacted support of our hosting party, and they indicated there was a problem with the hard drive.

I did several attempt to get back to the server in some way or another, but to no avail. After going back and forth with support for a few times, they finally replaced some hardware. As luck would have it, replacing the cables worked; the hard drive itself was not broken. Phew. Below the support's message:

I would like to inform you regarding server CXOF002.

The disk was recognized in the BIOS, however it did not show up in the
boot menu or in the rescue environment.

Therefore I swapped both the sata cable and the sata port and the server
booted without any issues and is reachable again.

If you have questions, please don't hesitate to ask.

Once I get back home, I'll try and sort out backups properly so I won't have to stress out for two days again. Thank you for your patience.

 

[Cthulhu]

I'm pulling a backup atm btw, so the site might be a bit slow. I don't want to worry about having an outdated backup again,

 

[Lex]

Wooooo we're back *does the dance of joy*

 

[AvecAloes]

Yay, Yop!

 

[Fangu]

Contrary to what we feared, nothing was lost?! Yop that is amazing!!!

A GIANT WOO

 

[Cthulhu]

WOO

 

[Carlie]

I got nostalgic when a "Database Error" appeared on my screen, haven't gotten those in over 2 years, good times.

 

[Fangu]

WOO

Now you're just showing off

 

[Ghost X]

WOO

WOO

Chugga chugga chugga chugga.

 

[Octo]

I'm glad its fixed. I was getting worried. Thanks for your sterling efforts Yop

 

[Ⓐaron]

Glad nothing was lost; thanks for sorting everything out

 

[Cthulhu]

Right. I've been fixxing up things tonight in terms of the backups so I'll be less inclined to sit in a corner and whimper. What I've done is:

* Extend the backup script (see below) to include relevant configuration files - more because it was kind of a pain to get them all set up as they are now, and they could probably use some more tweaking
* Spend $10 a month on a 100 GB Dropbox upgrade
* Install Dropbox onto the server
* Configure the backup script to move done backups to the Dropbox folder
* Configure a cronjob to make a backup automatically; at least a full backup weekly. I should figure out if there's a smart way to create incremental backups of the database.

When the backup is done and moved to the dropbox folder, it'll upload to Dropbox, from which it will get synced with my PC and other PC's I have Dropbox connected to. A backup atm is about 5-6 GB in size, mostly due to big files being hosted (MP3s and such) on TLS. Creating a backup takes about half an hour, and uploading to Dropbox - at the moment - will take about 3 - 4 hours at ~400-500 KB/sec, and atm it doesn't seem to influence website performance. I've set up backups to run at down times, which is about 8 AM my time, or around midnight in the US. Creating a backup should hardly affect forum experience; it's just that when it's created, the database dump program locks the database completely for a few minutes. For now I've set it up to create a backup tomorrow morning, just to check if the cronjob works; after that I'll probably set it to monday mornings or something.

Below is my backup script; there are many backup scripts like it, but this one is mine:

Spoiler: here be bash code

#!/bin/bash
set -e

DATE=$(date +%Y-%m-%d-%H.%M.%S)
CONFFILE=`echo ~/backups/config.${DATE}.tar`
BACKUPFILE=`echo ~/backups/thelifestream.${DATE}.tgz`
SQLFILE=`echo ~/backups/thelifestream.${DATE}.sql.tgz`

echo 'Creating configuration backups...'

tar cvf $CONFFILE /etc/php5/ /etc/mysql/ /etc/vsftpd.conf /etc/vsftpd.chroot_list /etc/apache2

echo 'Creating database backup...'

mysqldump -v -u root thelifestream | gzip -c  > $SQLFILE

echo 'Creating file backup...'

tar czvf $BACKUPFILE /var/www $SQLFILE $CONFFILE

echo 'Moving file backup to dropbox...'

mv $BACKUPFILE ~/Dropbox/tlsbackups

echo "Backup $BACKUPFILE created; dropbox should start uploading nao."

LIKE A BOSS.

 

[Fangu]

bash script.
Oh Yop talk nerdy to me.

 

[Gym Leader Devil]

OH, GOOD, THE TLS LIFETIMER ISN'T EMPTY AFTER ALL. I COULDN'T BE SURE SURE, DAMNED UNCERTAINTY PRINCIPLE.