Amazon is big. In its last financial quarter, it sold $32bn (£25.6bn) worth of stuff worldwide, including $6bn of media, $10bn of sales outside North America, and $23bn of electronics “and other general merchandise”. That “other” category encompasses everything from crucifixes to sex toys, board games to plyboard, and mousemats printed with the faces of obscure TV and Radio personalities.
It has also diversified beyond its simple shopping business: the company will sell you something to be delivered in less than one hour, food from restaurants, and even digital content to be watched on your TV or listened to on your phone. And, of course, it has a hardware business which many other companies would kill for, producing ebook readers and tablets, and single-handedly creating the product category of “smart speaker” with the Echo.
But there’s another chunk of Amazon that you’re less likely to know about. It’s responsible for a full tenth of the company’s revenues, yet its “operating income” – the amount of money it leaves in Amazon’s coffers once expenses are accounted for – dwarfs any other sector, pulling in $861m compared to the $255m Amazon makes in North American sales and the $541m it loses internationally.
The division is Amazon Web Services, or AWS, the section of the company that sells cloud computing services to both the outside world and to Amazon itself. You can buy storage space to hold a huge database, bandwidth to host a website, or processing power to run complex software remotely. It lets companies and individuals avoid the hassle of buying and running their own hardware, while also letting them pay for only what they actually use.
It began as almost a point of principle for Amazon founder, Jeff Bezos, before evolving to become the single most profitable part of the entire company. Now, AWS is moving into the third stage of its life, providing the underpinning for Amazon’s own quest to dominate not just our shopping, but our homes themselves.
What actually is AWS?
The sort of people who use AWS are almost as varied as the people who shop at Amazon. I employed it when I decided to train a neural network to write Guardian leaders: work that would have taken around 50 hours on my laptop took eight when I employed Amazon’s specialised hardware to do the job, and cost around £4.
At the other end of the scale, Netflix uses AWS for almost all its backend infrastructure, storing and streaming its web series from the same servers as its direct competitor, Amazon Instant Video – servers which Amazon owns. More than 35% of all network traffic in North America is Netflix, and all of that ultimately comes from Amazon’s servers.
Its customer list is huge running from Adobe and Airbnb to Yelp and Zapproved, via the UK Ministry of Justice and Nasa’s Jet Propulsion Laboratory. But at the beginning, it had just one customer: Amazon.
Before it was Amazon Web Services, it was just Amazon’s backend technology. Like any large dotcom in the early 00s, Amazon ran its own datacentre, spending millions on servers and software. One SEC filing from May 2001 put its quarterly technology bill at $55m, apparently reduced from $70m through the switch to the open-source operating system Linux. (Linux is still used in Amazon’s servers today, though the scale is somewhat larger: in fact, Amazon actually develops its own version of the operating system, which it offers to AWS customers.)
Then, one day, Bezos issued a mandate. Writing almost a decade later, long-term Amazon employee Steve Yegge remembered it as the archetypal example of Bezos’ micromanagement. This is a chief executive who, Yegge said, “hands out little yellow stickies with his name on them, reminding people ‘who runs the company’ when they disagree with him”; mandates from on high are standard when you work under Bezos.
But this one was different. Every team at Amazon, Bezos said, should begin working with each other in standard, systematic ways, and only those ways. If the ads team needed sales figures from the analytics, they weren’t allowed to ask for them, or send an email over; instead, the analytics team had to build an interface for pulling the sales figures out, share it with the whole company and teach the ads team how to use it. And those interfaces weren’t allowed to be simplistic things built for internal use only; they had to withstand being opened up to developers outside Amazon, too.
In effect, Bezos was asking Amazon to stop behaving like one singular company, and start behaving like hundreds of mini companies all bound together through one shared CEO. That command was the beginning of Amazon Web Services, which launched in July 2002.
Yegge, who left for Google in 2005, wrote that “from the time Bezos issued his edict through the time I left, Amazon had transformed culturally into a company that thinks about everything in a services-first fashion.”
The roots of this transformation were there for everyone to see. The first example of Amazon opening its internal services up to the outside world came in 2000, when the company launched Marketplace, its platform for external retailers to sell their goods through the Amazon website. A bold move, it demonstrated the scope of Amazon’s ambitions – directly competing with another dotcom giant, eBay, even while it was still focusing on matching wits with the old masters of brick and mortar retail. But it also showed the company was already keenly aware about which parts of its business were tightly guarded unique selling points, and which could be opened up to the rest of the world. Amazon wanted to make money selling you everything it could, and make money selling you everything it couldn’t, too.
Some spotted this transformation as it happened, but drew the wrong conclusions. In 2001, venture capitalist Steve Jurvetson argued that Amazon should spin its logistics operation off from its web storefront, letting other companies outsource their warehousing and dispatching to the new firm while Amazon focused on its website business. Instead, Amazon kept its logistics firmly in-house, relentlessly improving efficiencies, until 2006, when it opened that up too. Through Fulfilment by Amazon, companies can now outsource their warehousing and dispatching to Amazon even while Amazon itself continues to own those warehouses.
In 2006, the fruits of that externalisation came to bear when Amazon launched AWS to the outside world.
And it grew
When it launched, AWS was little more than a fancy way to buy space and time on Amazon’s computers. It had spent years in private beta, gradually allowing more and more customers to take advantage of the service-led structure that Amazon had invented for itself, and eventually coalesced into four services: storage, computing, database and internal messaging.
The first two, sold as Amazon S3 (for “Simple Storage Service”) and EC2 (for “Elastic Computing Cloud”), still provide the bulk of Amazon’s offering to customers now. Storage plus computing is the basis of almost everything you would want to do on the internet, from simple stuff like hosting webpages and delivering media to hideously complicated work like training a neural network, crunching regressions on big datasets, or just running a company like Airbnb or Pinboard.
But before AWS, if you wanted to get storage and computing power on the net, you had to hire server time to do it. That meant tracking down a server provider, picking the type of machine you wanted, and paying every month for your stuff to carry on sitting on that machine (usually paying on top for any bandwidth you used to actually get data from the server to your customers).
That leads to a host of predictable problems. If too many people visited your site all at once, it would crash under the load; but if you bought, or leased, a server big enough to handle them all, you’d be paying through the nose for power you weren’t using.
While useful on a day to day basis, that flexibility takes on a whole new meaning when viewed over the course of a company’s life. Scaling up is traditionally one of the hardest things for any startup to do: systems, procedures and products that work for a few users often fail when a company has a few million.
One of the most notorious examples in recent years is Twitter, which found out early in its life that the old-fashioned, server-focused way it was built was simply not compatible with it becoming a worldwide media platform. The first obvious crunch point was the 2010 World Cup, when Twitter would regularly get overloaded and crash to its error page, featuring the notorious “fail whale”. Initially, Twitter tried to “throw machines at the problem”, simply buying more and more servers, but eventually it realised it needed a more complex, ground-up redesign of its entire back-end.
The year after Twitter was founded, another service decided to launch without buying into the old-fashioned model. Dropbox, launched in 2007, probably couldn’t have existed without the existence of Amazon’s storage-for-rent. The company ran its own servers for the nitty gritty of the service, storing metadata about whose files were where in-house, but the actual files themselves – the gigabytes of free space every new user is given, and the terabytes of space paid users could consume – were hosted on S3.
You’ll have heard the name for the new system pioneered by the likes of Dropbox: cloud computing. Amazon’s cloud was one of the first, and is the biggest in the world today, but it’s still far outweighed by all the companies determined to host their own data. Speaking to the Guardian, Gavin Jackson, the head of AWS for the UK and Ireland, described that “inertia” from traditional IT as the company’s biggest competitor. Admins are still reluctant to host their company’s crown jewels on servers they don’t control.
Cloud on the horizon
But enough companies have made the switch that there’s a new problem on the horizon: concentration. Way back in the late 1960s, Arpanet, the predecessor to the internet, was built with a decentralised model: no one server, university or military establishment could take down the network if it was removed. That meant the project, funded by the US department of defence, could theoretically route around damage, making it robust against attack.
More than 50 years on, and the internet has changed beyond all recognition. The physical infrastructure routes through a few chokepoints: submarine and underground cables can be cut, knocking internet access offline for whole countries at a time; major internet service providers are consolidating, allowing action by just a few companies to censor the internet for millions; and a few major providers acting as the phonebook for the internet are another bottleneck, and if they can be forced offline, the entire eastern seaboard can suffer.
Amazon Web Services is reaching the size where it might need to be added to that list. A 2015 outage on the company’s eastcoast datacentre, for instance, caused connection issues for Netflix and Tinder for hours, as well as several of Amazon’s own services.
Amazon’s Jackson argues that it’s wrong to view AWS as anywhere near a single point of failure for the net. “We’re still day one,” he says, pointing to the fact that AWS’s annual run rate – the amount of cash it pulls in each year – is around $10bn. That’s high, but the annual run-rate of the wider internet services market is $3trn. Amazon is a big fish, but swimming in an enormous ocean.
More to the point, he argues, Amazon’s customers choose to use them, unlike most internet chokepoints, which are forced through the basic facts of internet infrastructure. If Amazon’s concentration ever becomes a problem, they’re free to up stakes and move over to another cloud provider such as Microsoft or Google, or go back to the good old-fashioned days of self-hosting.
The future: Alexa
One customer, of course, will likely never leave: Amazon itself. Internally, AWS insists it doesn’t see Amazon as anything other than a large customer. “A very demanding large customer,” Jackson says with a smile, “and a very vocal one.” There’s obvious business reasons for doing so, since some of AWS’s largest customers are direct competitors of the greater Amazon business; Jackson cited Dropbox, Tesco and Netflix as clients that “vote with their wallets” in sticking around (since we spoke, however, Dropbox changed its vote, migrating away from the Amazon cloud).
Even if Amazon is just another AWS customer, AWS isn’t merely a backend for Amazon anymore. The cloud provider has gone through several distinct phases in its history, from wildly ambitious project, to future-proofed data centre, to reliably profitable side-bet. But now it’s moving into a new stage, one which takes it back to the core of Amazon’s business.
Amazon’s hardware division, which dates back to the launch of the first Kindle in 2007, has traced a similar path to AWS, from weird and ambitious to core project.
First, Amazon’s Kindle e-readers grew from an expensive distraction to the dominant platform of a multi-billion pound industry; then they expanded into general-purpose tablet computing, and multi-platform media streaming. It slowly began to eye the sort of platform dominance that had made Apple the biggest company in the world.
But the expensive failure of the Fire Phone, Amazon’s attempt to take on Apple head first, prompted a change in strategy: a slim black cylinder called the Echo. Two years on, the Echo has competition from Google, in the form of the Google Home. But it remains the dominant “smart speaker” on the market, letting users interact with the device by speaking with “Alexa”, Amazon’s virtual assistant.
In a way, the device is just an evolutionary step from phone-based assistants like Siri and Google Now. But it also moves those assistants on by opening up the platform to external developers: anyone can write their own Alexa “skill”, from useful mini-apps like MyChef and Philips Hue’s smart-home hub to utterly useless ones like Egg Facts (“Stay up to date on the latest egg-related trivia with egg facts”).
There are over 7,000 skills on the market now, with more coming every day. And what’s the easiest way to make an Alexa skill? Hosting it in AWS. If you have a small enough userbase, Amazon won’t even charge you for the pleasure, letting budding voice-app developers start small and work their way up, risk-free.
The value of Amazon’s decade of experience shouldn’t be discounted, either. Internet services are notoriously hard to pull off at scale: one look at Apple’s continuing struggles to effectively take on Google with its iCloud software suite should be enough to convince on that count. The Echo has the benefit of piggybacking on a much more stable, mature and stress-tested platform than it would have if Amazon had to build its services from the ground up to support cloud-hosted speech-based apps.
For a sense of the scale of Amazon’s ambitions for Alexa and the Echo, analyst Ben Thompson suggests it’s worth thinking of the speech service as “Amazon’s operating system”. An operating system is the piece of software that sits between complicated hardware and the user, allowing for standardisation for both software developers and the user interface itself. In that way, it’s almost better to think of Alexa as AWS’s operating system: the first thing from Amazon that opens up the power of its cloud service to end users the world over.
Amazon has a tough fight on its hands. Google has been in the web services game much longer, and has much more experience creating – and monetising – consumer software. The battle for the living room feels a lot like the battle for our pockets was a decade ago: there may just be room for two companies, but no more.
But Amazon didn’t come out of nowhere. There’s a reason Alexa was so good, so quickly, and that reason is more than a decade’s experience working in the cloud. Amazon’s latest big bet has already paid off, and now it’s enabling the company’s next big one.
Global reach
23m: Items ordered from Amazon’s sellers on 28 November 2016 – Cyber Monday – up 40% on a year earlier.
43%: Amazon’s share of total US online sales, based on research by Slice Intelligence.
64m: Number of customers signed up for the video-streaming service Prime, according to the consultancy cg42.
5th: Ranking of Amazon’s founder, Jeff Bezos, in the league table of the world’s richest people.
1.3m: Estimated number of servers that comprise AWS, the enormously profitable cloud computing arm, according to analysts at Oppenheimer.