The History of Data Storage – From the Caves to the Cloud

Data: a critical, life-changing, and fundamental asset that supports humanity’s existence and evolution. For thousands of years (yes, thousands!), data storage solutions have evolved and supported humans by allowing us to “remember” and share knowledge in easy, maintainable, and searchable manners. Data turns into information, which in turn turns into knowledge. The ability to learn from the past and plan for the future is highly influenced by how we manage data in our systems today.

Software engineers are the catalysts of this process: our responsibility is to define and deliver solutions to people’s problems through software engineering – solutions that mostly revolve around data manipulation at a large or small scale. Having understood the importance of persistence in software engineering, you’re ready to bring your solutions’ persistence to the next level.

In this chapter, we will explore the modern era, where databases have become the backbone of our applications and the entire planet. We will cover the following topics:

• Why do databases exist? The history of databases
• Characteristics of Java persistence frameworks
• The cloud’s effect on stateful solutions
• Exploring the trade-offs of distributed database systems – a look into the CAP theorem and beyond

This first chapter provides you with an understanding of the past and current states of data storage technologies, before moving on to more advanced topics. This will give you a better foundation to work from. You will learn how data storage technologies responded to the market’s cloud-shift mentality. Finally, you will become familiar with practices such as Domain-Driven Design (DDD), which perfectly ties in with good persistence development practices, and the challenges faced by distributed data systems that await us in a distributed world, such as the CAP theorem.