Today is Judgment Day, when Skynet becomes self-aware. It is, apparently, also a very, very bad day for Amazon Web Services.

Lots of people have raised questions today about what Amazon’s difficulties today mean for the future of cloud IaaS. My belief is that this doesn’t do anything to the adoption curve — but I do believe that customers who rely upon Amazon to run their businesses will, and should, think hard about the resiliency of their architectures.

It’s important to understand what did and did not happen today. There’s been a popular impression that “EC2 is down”. It’s not. To understand what happened, though, some explanation of Amazon’s infrastructure is necessary.

http://blogs.gartner.com/lydia_leong/2011/04/21/amazon-outage-and-the-auto-immune-vulnerabilities-of-resiliency/

Amazon.com’s real problem isn’t the outage, it’s the communication

Like many companies running on Amazon’s Web Services, BigDoor has been affected by the AWS outage today. And like most startups, we are braced for bad stuff to happen, and we do our best to learn from the painful stuff. We spent a better part of the day in constant contact via Twitter, emails and phone calls apologizing and updating our more than 250 publishers that were affected. Today has provided plenty of lessons, and because transparency is fast becoming the lifeblood of Seattle’s startup community we thought we’d pass a few along.

http://www.geekwire.com/2011/amazoncoms-real-problem-outage-communication

AWS is down: Why the sky is falling

Amazon Web Services, “the cloud” to many people, has had a significant issue in one of their datacenters since about 1AM Pacific time April 21st. Some huge websites (reddit, quora, foursquare) are all down or significantly impacted. I’ve seen a lot of misinformation which suggests that this is all purely due to the laziness of the affected sites’ engineers, but that isn’t the case. Here’s why:

http://justinsb.posterous.com/aws-down-why-the-sky-is-falling

Standing on the shoulders of giants and stumbling with them – the Amazon AWS outage’s “pain” statistics

Today, at around 1am Pacific Time, Amazon began having major problems with some of their cloud infrastructure: specifically with their EC2, EBS, and RDS offerings. The issues are ongoing, and many of your favorite internet sites or services are probably still down or at reduced functionality because of it.

This kind of outage is one of PagerDuty’s big “moments”; when a big chunk of the services on the internet say: ”Hey PagerDuty, I’m down, so wake someone up to fix me!”

http://blog.pagerduty.com/2011/04/22/standing-on-the-shoulders-of-giants-and-stumbling-with-them-the-amazon-aws-outages-pain-statistics/

Cloud crash has a silver lining

If you are at all interested in cloud computing you would have by now no doubt heard that this has been a dark day in the world of Cloud Computing today. Something has gone terribly wrong in the networking in the Amazon US-East-1 region. That caused storage subsystems to go on the fritz and the rest as they say is history. Some of the very prominent web properties have been down and the volume of chatter on the subject has been deafening. I had to turn the sound on my TweetDeck off a I could no longer stand to hear the constant beeping generated by the tweet arrivals.

http://freedb2.com/2011/04/21/cloud-crash-has-a-silver-lining/

Today’s EC2 / EBS Outage: Lessons learned

Today Britain woke to the news that Amazon Web Services had suffered a major outage in its US East facility. This affected Heroku, Reddit, Foursquare, Quora and many more well-known internet services hosted on EC2. The cause of the outage appears to have been a case of so-called ‘auto-immune disease’. Amazon’s automated processes began remirroring a large number of EBS volumes, which had a knock on effect of significantly degrading EBS (and thus RDS) performance and availability across multiple availability zones. Naturally the nay-sayers were out in force, decrying cloud-based architectures as doomed to failure from the very start. As the dust starts to settle, we attempt to distill some lessons from the outage.

http://agilesysadmin.net/ec2-outage-lessons

Why Twilio Wasn’t Affected by Today’s AWS Issues

Starting early this morning, Amazon Web Services experienced several service problems at one of its east coast datacenters. The outage impacted major sites across the Internet. The number of high profile sites affected by the issue shows both the amazing success of cloud services in enabling the current Internet ecosystem, and also the importance of solid distributed architectural design when building cloud services.

Twilio’s APIs and service were not affected by the AWS issues today. As we’ve grown and scaled Twilio on Amazon Web Services, we’ve followed a set of architectural design principles to minimize the impact of occasional, but inevitable issues in underlying infrastructure.

http://www.twilio.com/engineering/2011/04/22/why-twilio-wasnt-affected-by-todays-aws-issues/

The AWS Outage: The Cloud’s Shining Moment

So many cloud pundits are piling on to the misfortunes of Amazon Web Services this week as a response to the massive failures in the AWS Virginia region. If you think this week exposed weakness in the cloud, you don’t get it: it was the cloud’s shining moment, exposing the strength of cloud computing.

In short, if your systems failed in the Amazon cloud this week, it wasn’t Amazon’s fault. You either deemed an outage of this nature an acceptable risk or you failed to design for Amazon’s cloud computing model. The strength of cloud computing is that it puts control over application availability in the hands of the application developer and not in the hands of your IT staff, data center limitations, or a managed services provider.

http://broadcast.oreilly.com/2011/04/the-aws-outage-the-clouds-shining-moment.html

Amazon’s Trouble Raises Cloud Computing Doubts

As technical problems interrupted computer services provided by Amazon for a second day on Friday, industry analysts said the troubles would prompt many companies to reconsider relying on remote computers beyond their control.

“This is a wake-up call for cloud computing,” said Matthew Eastwood, an analyst for the research firm IDC, using the term for accessing services and information in big data centers remotely over the Internet from anywhere, as if the services were in a cloud. “It will force a conversation in the industry.”

http://www.nytimes.com/2011/04/23/technology/23cloud.html?_r=1

If time will spare itself some more, I believe that I’ll post some additional flows to remotely install Nolio Agents onto the servers so they too can be available for deployment automation, continuous deployment, and other cool stuff IT operations like to get involved in.

The initial workflows available in this pacakge are:

• Prepare GoGrid signature
• Generic API call
• Get server rams list
• Get unassigned public IP
• Get public image list
• Add server

You can download the application service component at http://library.noliosoft.com/library/gogrid/

About Nolio

Nolio, the leader in application release automation and release management, is committed to helping customers meet the growing challenges and complexities of releasing and managing application deployments across the data center – increasing application uptime and reducing IT operation costs. Worldwide customers, enterprises, online services and SaaS businesses rely on Nolio ASAP to automate application deployment, maintenance, remediation and recovery processes for physical, virtual and cloud infrastructures – delivering automation solutions for application-centric IT.

Download the PDF version

Deployment Obstacles: Multiple Instances, Multiple Tiers and Multiple Deployment Events

Cloud technology has considerably reduced the cost of computational power incurred by organizations. The capability of renting potentially limitless computational capacity at very affordable rates has increased the number of cloud based servers an organization can deploy in production and pre-production operations. However, more servers mean that there are more instances which need to be updated, patched, maintained and serviced. And, this in-turn means an increase in the operational costs of deployment and application maintenance.

An increasing number of instances are one of the many factors that stand in the way of error-free and cost-effective ongoing deployment to the cloud. Enterprise scale applications entail multi-tier architecture. Deployment to this type of architecture is traditionally performed separately for each tier. Co-dependency between the tiers turns deployment into a non-trivial operation in which progress in one tier is often delayed until a process in another is complete. This non-linear deployment flow becomes even more difficult to manage when it has to be repeated hundreds of times for each server cluster in the various application data centers and environments. Another factor that further complicates an already complicated context is the number of deployment events that have to be executed during a specified time period. Organizations in verticals focused on online services such as e-commerce, media, finance and social networking are implementing agile development methodologies with short release cycles to perform more effectively in a competitive market and to maintain market leadership. Operation teams in such organizations are executing two or threedeployment events per day. In this type of high-frequency deployment setting, the complexity derived from the factors mentioned above namely multi-instance, multi-environment, and multi-tier configurations, grows exponentially.

In short, the challenging nature of the deployment process in the cloud is a result of several factors. Some are related to the character of the applications deployed, some to development methodologies and others to the number of users served by the application. Cloud technology is neither the only nor the primary reason behind this complexity but it does make matters worse, as the unlimited infrastructure the cloud affords enables applications to grow even more.

Images of the Future

The reason why cloud technology is said to further make matters worse for ongoing/continuous deployment – the abundance of cheap computational power – may conceal a promise to revolutionize application release deployment. This promise is based on achieving a complete separation in the cloud between the physical and the logical aspects of the application. Such separation would enable operational staff to create a kind of “master instance” – one image incorporating all application tiers and logical components. When this master instance becomes operational in the cloud, cloud technology will provide all the processing power required by this single instance application.

The implications of this scenario on ongoing application deployment are extensive. Instead of deploying hundreds of application, database and web server instances separately, the IT operation team will deploy only one image containing a single instance. Cloud technology will be able to provide sufficient computing resources to this all-inclusive one instance, even though this instance may be providing services on a global scale. There is no need to instantiate this image to different cloud based data centers which are distributed geographically according to performance demands and then to individually configure each instance to its environment. In effect, the instance as one copy of an application component among many has ceased to exist.

Images of the Present

Clearly, the above scenario is totally futuristic. Complete separation between the physical layer and the logical layer, between infrastructure and application has not yet left the confines of the laboratory. As of today, the architecture of applications residing on the cloud mirrors the architecture of their earthly counterparts. This means that an application which is deployed on the cloud has the same number of instances, the same data center distribution and the same server configuration, with the same co-dependencies, it would have if it resided in a traditional, non-cloud infrastructure.

Since cloud applications are still reflections of their non-cloud version, they share all of the above mentioned deployment complexities with the latter. However, the cloud makes it economically feasible to deploy applications by means of images. As cloud technology offers a potentially unlimited supply of servers quickly and cheaply, the road is paved to provide new servers with application images every time there is any type of deployment event, regardless of its scale.

At first glance, it appears that this method of performing ongoing deployment saves time and resources as there is no need to directly upgrade, patch, or fix each application instance. Instead, only one image containing an instance of the latest application version is uploaded and then instantiated according to the performance requirements and data center distribution of the application. For example, instead of directly deploying a patch to 300 application server instances, it is possible to upload one image of the patched application server and then, with the press of a button, instantiate it in the cloud.

However, a closer look reveals that although this method is suitable for initial deployment, it is not suitable for ongoing deployment. Firstly, it takes some time to prepare an image, and then additional time to upload it to the cloud. In many cases this process has to be repeated when more than one image needs to be deployed. For example, when an update modifies both the application tier and the storage tier, two images have to be prepared, packaged, and uploaded.

Secondly, after the image is uploaded it takes additional time to prepare copies of the image according to the number of instances required. However, this is just the beginning. When this step is complete and all servers are instantiated in the cloud, they are nothing but a multitude of identical clones. It is then often necessary to individually configure and customize each and every one of them to the environment in which they operate.

It is fairly obvious that this particular work flow is too time consuming and it cannot be an acceptable method for continuous cloud deployment. This is particularly true in organizations that have multiple deployment events on a daily basis, some of which are very minor such as replacing a few lines of code in a configuration file. It is clear that the only way to cope with the time constraints imposed by such scenarios is to update the existing application rather than replacing it with a new instance. An application release deployment automation tool must be utilized in order to consistently execute an application-wide deployment event to the hundred or so instances involved.

Summary

It can be seen that deployment by image replication does not provide a solution to cloud deployment as long as the number of instances in the cloud correlates to the number of servers in a non-cloud platform. The same is true for other factors which make deployment complicated, such as multi-tier architecture and multiple environments. Co-dependency is still an obstacle for manual deployment since cloud instances are correlated to application tiers, so that each instance is either a web server, application server, or data base server, to name but a few. And, because the instances are grouped to different environment clusters in the cloud, they each require their own configuration and customization. The conclusion is that the use of images as a deployment tool does not provide an answer to the challenges imposed by ongoing deployment of enterprise grade applications in the cloud. Multiplicity of instances, environments, and deployment events is still the core of the problematic nature of deployment. The only existing solution that can untangle the complexities triggered by this multiplicity is a dedicated automation tool which, by modeling the application deployment flow and by repetitively applying reusable deployment activities, can streamline the varying deployment events as well as implement a consistent, customizable deployment lifecycle in the organization.

From the blog:

While the delivery of virtual machines is indisputably faster than deploying physical machines, it is often assumed that this also streamlines the process of deploying IT Services (applications). Virtualization can be used as an accelerator for building out highly dynamic cloud based services, however, the fact remains that while provisioning and deploying VMs has been greatly simplified, application deployments are still complicated and dependent on People, Processes and Technology.

Sure, in a perfect world where organizations only run one application on one type of OS, and one version of that application – virtualization can certainly automate the entire process of application delivery.  QA and development environments have been realizing this benefit for years.  However, what happens when organizations begin to accelerate their adoption and begin migrating mission critical, multi-tier applications to the virtual data center?

Unfortunately, we live in an IT service world that is comprised of ever increasing complexity requiring in-depth knowledge of application deployment, application dependencies, application monitoring and a deep understanding of security and regulatory requirements.  Combine this with the myth that applications deployed in the virtual data center can be delivered faster, cheaper, and easier, it is no surprise that some operations teams are struggling to meet these expectations.  How many times has the underlying virtual infrastructure (VMs) been provisioned in minutes, while the actual delivery of services takes hours, days, or in certain cases, weeks.

The full post can be read at http://www.zdnet.com/blog/virtualization/myth-virtualization-increases-the-speed-of-delivering-it-services/2642

Increasing application complexity, the growing volume and frequency of application changes, and the need to execute operations across multiple environments and infrastructures (physical, virtual and cloud) are causing application failures, bottlenecks and delays. IT operations are struggling with the challenges of managing the release and maintenance of data center applications – with quality, on time and on budget. Read more about Nolio ASAP.

Ozzie said that the future for companies’ data hosting will be in a combination of cloud computing and on-premise data centers. In his words, “at some point in time, every major enterprise, every company, every ISV is going to have some blend of software that runs on-premises and some that runs in the cloud, and everyone wants tools that they can use to in essence deploy some apps to part of their organization that might be in the cloud…”

We couldn’t agree more. Our vision is that in the future, every company will have some kind of environment in the cloud. As Daniel Lyons of Newsweek recently said, “People are going to be putting their information not into some device but into some service that lives in the sky.” Lyon added, “Pretty much everyone in the tech industry agrees it’s the future.” For companies, cloud computing makes a lot of sense because it’s cheaper: in some cases, it cuts a company’s costs in half.

Of course, organizations that move to the cloud will need tools to help them deploy and manage their cloud applications.

The layer IT will need to handle in the future is the Application layer: deploying, managing, maintaining and troubleshooting applications in the cloud. Existing automation tools are old and fit traditional data centers. They are system- and infrastructure- centric solutions, while there should be a shift to application-centric solutions that will enable effective and efficient automation.

Nolio is a vendor leading the charge on doing just that. As an innovator of Application Service Automation solutions for physical, virtual and cloud data centers, Nolio’s Application Service Automation is a software platform for designing and executing automated application service workflows across the data center, enabling reliable, effective processes for the deployment of applications in the cloud and for the management of application change.