For the second time in less than two weeks, the internet experienced major problems across a variety of websites today. The reason for this: outages at Microsoft Azure and Amazon AWS cloud services. The consequences of the issues ranged from simply not being able to access certain websites over credit card payments not being able to be processed to hospitals unable to access vital data of their patients. Are we too dependent on major cloud providers?
The Big 3 of the Internet
The internet originated as a network between researchers to share information with each other. In the early days, if you wanted to share information, you would host it on your local machine, to which the other people could connect and download said information. This approach works fine for a limited number of users, but in the modern day, with billions of users wanting access to data around the clock, it would be impractical to provide this data from your local machine. Nowadays we rely on millions of servers in data centers spread around the globe to do this task, and especially in cloud services. Cloud services give companies and regular people alike the possibility to host their data without the need of building up the complex infrastructure required for doing this, both in terms of hardware (acquiring servers and data center space) and software (setting up the network and connecting it to the internet). The major problem with this is that almost 2/3rds of these services are provided by only three companies these days: Google, Microsoft and Amazon.
The consequences of outages
This dependence on just three companies leads to major issues if something goes wrong there. This could be seen twice in the last two weeks, first with a major AWS outage last week, affecting many parts of people's lives, and again today when Azure encountered problems with their service. The problem is that even things that don't seem directly connected to said cloud providers often rely on other resources, which in turn again are hosted by one of the big three, so even services that don't seem affected at first can sometimes not work properly. One example of this are payment networks. If they rely on AWS to process, for example, a credit card transaction and the service is not available due to an outage at Amazon, the transaction can't be processed and you can't buy the product, even though the shop you want to buy from is not directly hosted by AWS. The more worrying part is that parts of our critical infrastructure, like hospitals, also use those cloud services to manage, store and process data relevant for their operations. If they aren't able to access their inventory management systems and check supplies of critical supplies, they could run out, leaving a person without possibly life-saving medication.
What happened?
While it's too early to tell what happened at Azure today, we have an idea of what caused the AWS outage last week: A collision between two programs trying to update a single DNS entry at the same time. This resulted in said entry being blank, causing at first an outage at AWS DynamoDB database services, which in turn cascaded then to many other AWS services as well. This shows that major parts of the internet in its current state can face problems caused by such tiny things as a single DNS entry, showing how vulnerable it is architecturally.
Are we too dependent on major cloud providers?
The big problem with our dependence on the big three is that an issue at one of them affects a sizeable chunk of the internet. While internationally spread cloud providers have huge benefits for the users, their huge market shares result in issues affecting many businesses and people at the same time. And as long as cost is the deciding factor in regard to how IT infrastructure is planned and set up, the big cloud providers have the advantage of being able to offer the best scaling opportunities for what they cost. In an ideal world we would go back to the earlier times with more decentralized infrastructure and companies using their own hardware in local data centers, but since most of the time this is not the most economical solution, the dependence of major parts of the internet on just a few cloud providers will not decrease.
