The big move! The life of a Romanichel...

Changing servers: an overview

Well, it's not as simple as I thought it would be, so I thought an article was in order to explain it to everyone once and for all! This is the story of my life... 3 really rotten weeks of my life anyway... It's for you guys... Enjoy.

 

As you may know, all our services (games, website, teamspeak) were running on a large OVH machine, an SP-128: Xeon E5-1650V3 12 threads, 3.6-3.8Ghz, 128Gio DDR4 RAM, 3x2To RAID 5, all protected against DDoS by Arbor. Not a single DDoS in a year, especially after what we experienced at Online.net !

This OVH server was really cool, but also really expensive... €198 per month. The bill almost ended up hurting all of us: it was time for a change. But downgrading was out of the question!

Well, that's just as well, because in the meantime OVH has released some 'Game' servers that look very interesting. For around €83/month including VAT, they offer: 4790K 4Ghz 4cores 8 threads, 32GB RAM, 2x240GB SSD, and permanent anti-DDoS mitigation... Yum yum! It's not cheap yet, but it's hard to find any equivalent, let alone at this price. What's more, Light-Blue (Admin and Owner of Garry's Mod Brony servers) will be sharing the load on this machine... All good things!

However, as it's out of the question to put the SSDs in RAID 0 (2x faster and infinitely less reliable than RAID 1 for those who don't know), 240GB of usable space is a bit short for us... And it would be a shame to put any load other than games on this beautiful machine. And it would be cool in any case to have several servers, you never know... Right, got it?

So three things to deport to one or more other small servers:

• Websites and TeamSpeak (yes, I also host other sites, including my pro site)
• Seedboxes on another server, which will be separate from the community (Yes, we had so much disk space and so much bandwidth that we took the opportunity to put in a nice seedbox... To seed Linux distributions of course!)

Well, we'll see about the seedboxes later, but we need to move the sites as quickly as possible! On 25 April, no more SP-128, which will be celebrating its first year, and 1 year of uptime (never restarted the beautiful beast).

So for the hosting server, after a long search for a functional and inexpensive VPS, I landed (once again) on Pulseheberg. This hosting company turns out to be the real Free Mobile of hosting... And that's not a compliment, those who are with Free Mobile and love Youtube know what I'm talking about. It's true that the hosting company offers rock-bottom prices, but in my case the service was worthy of Romania, and what's more (still in my case) it was made worse by a loss of data... and a week's work lost, one!

After a week of transferring websites, configuring mail servers, setting up TLS certificates, dealing with various email service providers (including Microsoft, who had fun spamming us and even rejecting our emails...), and on top of all that, wasting an inordinate amount of time submitting a ticket a day to Pulseheberg for problems I'd never encountered with other hosts... I won't list them because they wouldn't fit on the page. I won't go through the list because it wouldn't fit on the page... The main problem was that the server was ultra slow, and didn't even offer 2 effective CPU cores, for the 8 indicated on the paper... They finally put me on a host that was performing, oh yes, yes, certainly, better, but that was shut down and deleted all of a sudden without warning, along with my data and my work. They'd moved me to a host that wasn't dedicated to production! Serious stuff!

And that's where the Romanichel's journey begins.

I was so fucking motivated, I'd done everything, I'd even taken days off... But no, some incompetent bastard had to fuck my server. Obviously, in 1 week, I hadn't set up a backup system, but I'd learnt my lesson: Start with that, when the hosting company only charges 12€/month.

Well, harassed on twitter and via their "customer manager" (well yeah, that's all I had to do, I couldn't go on...), they did try to rebuild the RAID at Pulseheberg, and give me my data... It took them 5 days, 5 fucking days to give me a fucking incomplete image... I got a /etc (basically the program configuration for those who don't know Linux), and nothing from /home, nothing from /var... there was almost nothing to take. Bloody hell... the glands...

So we had to get used to the idea: everything had to be redone... Luckily, I still had the OVH server, remember? The SP-128. It goes down on 25 April, and I didn't delete anything on it. So I hadn't lost EVERYTHING, but 'just' a week's work. In any case, I also had backups of the Mesnie, but they weren't very young...

Obviously, the question arose of changing hosts... But they assured me that the error wouldn't happen again, that it was only the 2nd time it had happened since they started, that they were aware that it was twice too many... I managed to negotiate - among other things - the backup option offered, which seems to me to be more than adequate compensation for the situation. So come on, we're staying with them for web hosting, on one condition: the server has to pass the benchmarks. Well, it didn't... They'd put me back on a host with rotten performance, so I shouted again, and they put me back on a real host... This time, the performance is now excellent for a VPS (almost the same level of CPU as the SP-128, that's saying something), so it's decided, we're staying!

So it's time for a re-migration!

Websites, teamspeak, the whole mess... And the same day, surprise, major packet losses for all teamspeak users, a general disconnection... Pfuah... Fuck, it never stops... Thanks Free.

 

Re ticket, investigation... This time it's their bandwidth supplier, Cogent, known for its troubles with Orange, which is literally the shit, causing losses for everyone, and above all, ranging from 8 to 15% for Orange users, even on fibre... https://twitter.com/UltimateByte/status/720997484357644288

Well, TeamSpeak couldn't stay like that, it had to be changed as quickly as possible, and permanently... I'm not going to transfer TeamSpeak 50 times. So it was time to take over the Game server sooner rather than later!

So I go round TeamSpeak to do the famous Mesnian quest... Yes, you know, like at mass, if you've ever had the misfortune to go... The thing where you don't hear anything about the whole mess, and at the end you clearly hear "And now, the quest"! Well, you didn't understand a word of what was going on, but you know you need a bit of cash in a hurry... We had €35 in donations, we needed €85! With just two generous donations (thanks guys <3), we were there!

Bam, OVH "Game" MC-32 server, we had it! I've never seen a server install packages so quickly... A real killer. Two hours later, the machine was totally ready, custom, and the teamspeak transfer was done! And strangely enough... no one was losing any more packages.

I spent the next two days doing just that:

• transfer game servers as a priority
• update all game server management scripts
• update all game servers and clean them up
• create a table of default ports/collection ids, maps and gamemodes for each server
• do the cronjobs that go with it so that the game servers are always up and running, with an auto reboot + update every night
• put all servers on gametracker
• redo the game server pages and gametracker links...
• to rebuild the TeraGeek website
• transfer databases and functional sites from the community
• to learn more about TLS certificates
• renaming mesnie.org again and again to terageek.org and http to https all over the site...
• to redo the news identically (thanks google cache)
• missing an evening, and a visit from a friend to keep us going
• and doing so many things that I can't even remember them all...

What remains to be done:

• Update all crash screens and escape menus on gmod servers
• The mail servers. At the moment, people can't register on the site because of this, which is a shame... That's why I've put the forum into maintenance. Well, I should be more efficient than ever at doing this super-annoying task, given that I already did it not long ago, but damn, it's so long and tedious to do... Once a year is bad enough, but twice in a month... Expect a return within 48 hours, I think.

Anyway, the hardest part is over, the SP-128 is ready to be closed, there's nothing important left on it...

It's going to be a relatively quiet week, the first in 3 weeks! At last, at last! At last I can finally take a breather!

Let's hope the Pulseheberg VPS holds up... The thing won't understand when it hits 4,000 pages a day to display... Because yes, it happens, averaging 2,000 pages a day over the last 6 months, and up to 1,000 visitors a day (I'm waiting for my medal please).

Good stuffing to you. :)