Trending Topics
Trending
Celebrating 20 years of Amazon S3, the technology that “fundamentally changed how organisations thought about data”
Could the launch of Amazon S3* have a more low-key launch announcement? “Earlier today we rolled out Amazon S3, our reliable, highly scalable, low-latency data storage service,” wrote Jeff Barr, at that point a Web Services Evangelist for Amazon and now VP and Chief Evangelist for AWS. “The system was designed to provide a data availability factor of 99.99%; all data is transparently stored in multiple locations,” he added.
This is the acorn from which a gigantic tree came to grow. One that the world now relies on, as we all discover when a branch of AWS fails. After all, there’s still a 0.01% gap between 99.99% and 100%.
To mark the 20th anniversary, we asked six leaders to share their thoughts. People such as Václav Svátek, CEO of ČMIS. “[In] 2005, scaling storage meant a frantic capital expenditure request, weeks of waiting for hardware, and the literal rack-and-stack of SAN/NAS arrays,” he writes below. “S3 flipped that script entirely by introducing the industry to the infinite bucket. It moved storage from the world of hardware procurement into the world of software logic.”
Or, to quote Wasabi Technologies’ Craig Stockdale: “Object storage fundamentally changed how organisations thought about data.”
Our huge thanks to Craig Stockdale, Daniel Essex, Nick Sampson, Henry Zelikovsky, Václav Svátek and Gillian Holloway. To quote Jeff Barr’s original post, “I’ve got to catch a plane to Silicon Valley in a few minutes, or I’d write a lot more.”
Craig Stockdale, Country Managing Director ANZ at Wasabi Technologies
Object storage fundamentally changed how organisations thought about data. Traditional SAN and NAS storage was tightly paired with infrastructure hardware, so organisations had to carefully manage capacity and costs, with data often treated as something to either archive or discard. Object storage introduced a scalable architecture with rich metadata and global accessibility. This enabled organisations to start treating data as a strategic asset that can be analysed and reused across applications and environments to gain business insights.
That change is now critical for AI as organisations ingest, process and move huge volumes of unstructured data quickly. At Wasabi, we found over two-thirds of AI budgets for ANZ companies go to data, storage and compute reflecting how central data infrastructure has become to AI development. As AI pipelines become more data-intensive, high throughput is the number one requirement organisations look for in object storage for AI workloads, reinforcing its role as the foundation for modern AI innovation.
Daniel Essex, Product Manager, Park Place Technologies
Object storage represented a fundamental change in the philosophy of data management. It moved organisations away from thinking about where a file lives to focusing on what a piece of data actually is.
In traditional systems, metadata is very basic – typically limited to a filename, size, and date created. Object storage, however, allows for rich, custom metadata: labels and attributes that can be attached directly to the data itself. This makes data far more searchable and programmable, enabling it to be organised, queried, and used in new ways.
At the same time, object storage scales horizontally. Rather than relying on increasingly expensive hardware, organisations can simply add more commodity nodes as their storage needs grow.
For enterprises, this changed behaviour around data. Instead of routinely “pruning” data to save space, organisations began to retain it. The mindset shifted from a culture of deletion to one of retention – recognising that stored data could later become valuable fuel for analytics and AI.
Nick Sampson, Software Engineering Lead at Fathom
If S3 hadn’t been invented, the internet would likely be a less diverse place, more dominated by incumbents with fewer startups getting off the ground. While on-premises infrastructure still makes economic sense for some, the upfront costs and technical friction of building durable storage are huge.
Managing that complexity and scaling it with growth would have been a major distraction for small teams, pulling focus away from their core product. These teams would have likely been left with less reliable systems and performance trade-offs that would have stifled their ability to scale.
S3 changed the game by replacing these complexities with a simple monthly subscription. Suddenly, a two-person startup had access to the same “11 nines” of durability as a global bank, paying only for the gigabytes they actually used. If that usage increased, you don’t need to scale your hardware, your monthly bill just increases.
Today, S3 is so foundational that a single regional outage can take a huge portion of the internet with it. Simply put, it is critical infrastructure that the digital world depends on.
Henry Zelikovsky, CEO of Softlab360
AWS S3 has indeed been transformative since its release in 2006. It transformed enterprise architecture views on topology of servers and inter-server connectivity. Previously, enterprise technology and application system solutions, from small to large, were placed into private data centers with data storage aligned with dedicated servers with disk space at maximum capacity of server hardware at that time.
As data storage hardware evolved into SAN/NAS, storage capacity increased, as did its cost. Solutions were priced according to infrastructure costs, where they would be hosted. AWS presented an outsourced option to managed private data centers, the cloud version of the data center, with one important addition – networked object-storage, S3. This option was, on average, at a lower cost, but S3’s ability to address data storage by object-type presenting an opportunity to arrange data in data types defined by system architects and data designers.
This arrangement did not require a database or a data warehouse, and yet, in a similar manner, offered access by object-type. AWS S3 API enabled opportunity to select storage capacity and storage responsiveness (speed of storage and retrieval), freedom to design its object-type structure, and assurance that storage is connected to AWS-based machinery where applications would reside.
AWS guaranteed that allocated cloud resources would be maintained 24×7, and data movements would be streamlined in the shortest networked path. Application architects and software developers gained an outsourced, reliable, dynamically expandable data storage with a pay-as-you-go budget option. That was the biggest win.
Today, AWS S3 is integrated with multi-level security features of AWS, and AWS resources for marshaling application traffic to and from different S3 buckets, separating storage into “hot”, more responsive, and “cold”, less-frequently used, with lower cost. In comparison to pre-S3 days, S3 is an effective data storage resource, dynamically sized, with multiple data formats (data object-type) support, priced to usage, and integrated via an API for convenient application use – all the points that created benefits.
Václav Svátek, CEO of ČMIS
Before S3, storage was a physical constraint. If you were a senior engineering leader in 2005, scaling storage meant a frantic capital expenditure request, weeks of waiting for hardware, and the literal rack-and-stack of SAN/NAS arrays. S3 flipped that script entirely by introducing the industry to the infinite bucket. It moved storage from the world of hardware procurement into the world of software logic.
My take on it is simply that they established the standard. What we now call “AWS S3-compatible storage” which is offered by every major player today, including ČMIS. The core difference, and the real kick in the teeth, is that AWS charges customers for a bloody fortune just to download their own data back out of their S3 buckets.
To sum it up, S3 was rather an existential necessity than technological breakthrough and a crucial step that enabled the rapid growth of companies like Netflix, Airbnb, and Dropbox. Big tech as we know it wouldn’t exist without massive, globally available cloud infrastructure, whether delivered by Amazon or another provider. By abstracting the physical layer, S3 allowed organizations to stop worrying about disk failures and RAID configurations and start focusing on data utility. Companies could focus on building products in the new digital world, where data could be accessible simply via API. This made it possible to build faster and more efficiently, but usually at a cost.
Gillian Holloway, EMEA Vice President – Partnerships at Insight
Without Amazon S3, the cloud ecosystem we know today would likely have developed much more slowly and in a far more fragmented way. Many organisations would have continued relying on on-premise storage for longer, and the data platforms that now power analytics and AI would have taken much longer to scale.
At the time, the idea that businesses could store and access vast amounts of data through a simple API, without owning or managing the infrastructure behind it, felt like a real turning point. It fundamentally changed how companies approached infrastructure and, in many ways, helped turn storage from something that limited what they could do into something that enabled them to experiment, build and innovate.
*For anyone who may be wondering, Amazon S3 stands for Amazon Simple Storage Service.
You might also be interested
About The Author
