Avoid the tech-nightmare before Christmas: the yearly upgrade cycle.



  By Mike Resseler, Product Strategy Specialist for Veeam and MVP for System Center Cloud and Datacenter Management


Here comes December, which for most means happy family moments, gifts, and a nice bearded guy traveling around the world in his deer driven sled.
For some of us, this announces the quietest time of the year, and that means happy upgrade moments, patches, and a nice IT guy traveling around the datacenter to apply upgrades.
If you get well organised you should be able to have both… but sometimes a glitch comes and you spend your holiday season with scary crashing servers, bugs, and an exhausted IT pro running around the datacenter…
What side do you want to be on?

The tech-nightmare before Christmas.. and how to avoid it.

The short period between Christmas and New Year is often the ideal time for a yearly maintenance cycle. During this week, many companies are shut or are running with limited staff due to the holiday season. This allows IT professionals hassle-free spare time to do maintenance work. I’ve been in this situation many years working in an internal IT team. All the patches, upgrades and whatever else we couldn’t do during the rest of the year is planned during that week.

There's always enough reasons to explain why we do it during that week; there isn’t enough time during the year; we can’t do it because we're not allowed to bring those services offline or maybe because we don’t have enough resources in our small IT team… the list goes on!

While most of us have looked forward to (finally) tackling a lot of the issues with these updates and patches, we're always very nervous, what if things go wrong? What if an update goes crazy and the service doesn’t start anymore. What if…

Unfortunately, I've been in the situation where this has happened and it hit us pretty hard. Upgrading and patching servers suddenly became fixing and restoring a lot of our services, and consequently missing New Year’s Eve and New Year itself, something that isn’t easy to explain to the family, but as true IT Pro’s, we had to get everything up and running again, and family came second at that time. That time, I was the exhausted IT pro running around the datacenter instead of having family time.

First thing you do afterwards is to review the lessons learned. We don’t want to have the same issues twice. So our first round of analysis was looking at the technical things that went wrong to avoid future failures. We had to look it from a process point of view.

The quick and easy answer to avoid such problems is (of course) test out before implementing. But that’s easier said than done. You might have a test environment where you can try out your upgrades, but even if you've that, it will never be an exact copy of your production environment, which still means that there could be issues. Or you just don’t have a test environment at all, so you don’t have the answer.

In those days, testing out upgrades was an extremely difficult process, required many days of work and a lot of investment. Something, as I already stated, we didn’t have.

Fortunately, a few years ago Santa put some nice gifts under the IT tree, and today there are solutions to these problems.

Christmas, the upgrade, and Greean Santa.

The first component of this solution is called Virtualisation. It is now well deployed in most datacenters and keeps getting bigger. The recent Hyper-V 2012 R2 release just gave you one more reason (a couple of months in advance of Christmas) to go for it! Now, combine this with one of Veeam Backup & Replication’s nicest features, called Virtual Lab, you will have a way to save your family time.

Veeam Virtual Lab allows you to restore your virtual environment (including the one on that shiny new Windows Server 2012 R2) into an isolated network for testing purpose.
This isn't a similar environment to your production… This isn't the same software but on another hardware… This is an actual identical copy of your production environment!

Based on any of your backups, the virtual lab will be running your virtual machines on the same hardware and hypervisor, just in an isolated network to avoid any conflicts.

Why does it matter?
Because this means before you start that scary update on your production environment, you can test all you want in the Veeam Virtual Lab. You can apply the patch/upgrade there, see if it works and document it precisely before running it in production. What if the upgrade doesn't work? Just shut down the lab, restart a new one from a clean copy and try again. No impact on your production, no impact on your backup.

So, for next Christmas, will you try to run you upgrades on production or in a look-alike environment?
Or, will you rather ask the Veeam Greean Santa to pack you a nice Virtual Lab so you can test it calmly, and then go to your New Year dinner relaxed and confident it will be all fine?



Author BIO: Mike Resseler is a Product Strategy Specialist for Veeam. Mike is focused on technologies around Hyper-V and System Center. With years of experience in the field he presents on many occasions on large events such as MMS, TechEd and TechDays. Mike has been an awarded the MVP for System Center Cloud and Datacenter Management since 2010. His major hobby is discussing and developing solid Disaster Recovery scenarios. Additionally, he has enterprise-class experience in Private Cloud architecture, deployment with marked focus on protection from the bottom to the top. He holds certifications in many Microsoft Technologies such as MCITP. Follow Mike on Twitter @MikeResseler or @Veeam