-
Book Overview & Buying
-
Table Of Contents
-
Feedback & Rating

Data Engineering with AWS
By :

The data pipeline examples that we have used in this book have been based on common types of transformations and pipelines, but they have been relatively simple examples. As you can imagine, in large organizations, the types of data pipelines that are built can be a lot more complex and may end up processing extremely large sets of data.
In this section, we will examine two examples of more complex data engineering pipelines from two very well-known organizations – Spotify and Netflix. Both of these companies have public blogs that cover software and data engineering, and the details provided about their pipelines in this section have been taken from the public information that's been made available in a variety of blog posts and articles.
Every year, for the past few years, the music streaming service Spotify has used the extensive data they have on their user's...