We are pleased to welcome guest blogger Dani Shomron, a SAAS industry veteran. Dani has held development, management and executive roles in multiple verticles and geographies. He holds a BSc. in Computer Science from the Hebrew University and an MSc. in Artificial Intelligence from the University of Edinburgh. Dani is an expert in SaaS Operations and ISVs transition from on-site to on-demand.
One of the biggest appeals of the SaaS model is that upgrades are seamless and transparent. You log off in the evening with one version and the next morning when you log in, voila! you have an upgraded version with all the bug fixes and new features.
What happens in the background is another story. Scary sometimes. A Midnight Summer’s Nightmare.
If you, a SaaS company, have your SLAs in place and adhere to them, your full upgrade windows (i.e. planned downtime) are probably few and far between. Some SLAs allow for a periodic one hour downtime with a week’s notification, but anything longer than one hour will require an earlier notification. You must ensure that the upgrade is performed within that window, or any minute beyond that will start counting against your SLAs. You do not want to extend that window unnecessarily, as your customers are counting on your 24X7 service.
Upon receiving the upgrade instructions from Engineering, you will need to start building a plan. Often, the plan will include dozens, or even hundreds of short steps, and complex dependencies. In order to ensure that the upgrade will be performed correctly and within the allotted time frame you should be able to answer the following questions:
• How long will it take the team to run the upgrade?
• What happens if you need to rollback?
• What is the point of no return, when you must rollback without violating the change window?
A mature SaaS operations group will likely have a Staging environment, closely mimicking production. The best way to go about planning and executing the upgrade is to perform the process on the Staging environment. You will need to carefully design the procedure and run it multiple times until you perfect the process, taking note of how long it takes and practicing the rollback.
This is a tedious process, treasured by engineers as much as they love deciphering undocumented legacy code. But it is essential to ensure success.
So you need to make the process as short as possible and measurable. If you had the tools to automate the procedure you could reduce the time of execution (and of the dry runs on Staging). You could know exactly how long it will take and have the needed repeatability. Repeatability cannot be overrated when encroaching upon the Holy of Holies, under the pressure of timetables and the short temper of the anxious COO/VP Ops.
Since upgrades are an integral part of a SaaS operation and one of the main causes of the gray hair, ulcers, I wanna go home factor, you would do yourself and your customers a great service if you had the tools to automate the entire process.
application service automation