Part I, Introduction, provides a general introduction to the area of distributed tracing.

Chapter 1, Why Distributed Tracing, frames the observability problem that distributed tracing aims to solve and explains why other monitoring tools fall short when it comes to troubleshooting pathological behavior in complex distributed systems. The chapter includes a brief history of my personal experience with tracing and an explanation of why I felt that writing this book would be a useful contribution to the industry.

Chapter 2, Take Tracing for a HotROD Ride, dives in with an easy to run, hands-on example used to illustrate the core features, benefits, and capabilities of distributed tracing, using Jaeger, an open source tracing platform, the OpenTracing instrumentation, and a demo application HotROD (Rides on Demand).

Chapter 3, Distributed Tracing Fundamentals, reviews the basic operating principles of end-to-end tracing, such as causality tracking and metadata propagation, and various design decisions historically taken by different implementations that affect the types of problems a given tracing architecture is able to solve. It introduces the reader to two different tracing models, the more expressive event model, and more popular span model.

Part II, Data Gathering Problem, is dedicated to discussions about the different ways of getting tracing data out of the applications, through manual and automatic (agent-based) instrumentation, for both RPC-style and asynchronous (for example, using message queues) applications.

Chapter 4, Instrumentation Basics with OpenTracing, provides a step-by-step guide of manually instrumenting a simple "hello, world" style application for tracing, as it is being evolved from a monolith to a microservices-based system. Three parallel sets of examples are provided in popular programming languages: Go, Java, and Python. The chapter teaches the fundamentals of tracing instrumentation using the OpenTracing API; however, the general patterns are applicable to other instrumentation APIs as well. In the final exercises the chapter introduces automatic (agent-based) instrumentation style that requires little, if any, actual code changes in the application.

Chapter 5, Instrumentation of Asynchronous Applications, continues the lessons from chapter 4, and applies them to an "online chat" type of application built around asynchronous messaging using Apache Kafka.

Chapter 6, Tracing Standards and Ecosystem, explores the often confusing ecosystem of the tracing industry at large, including the emerging standards such as OpenTracing, W3C Trace Context, and OpenCensus. It provides a useful taxonomy of how to think about the different commercial and open source projects and their positions in relation to each other.

Chapter 7, Tracing with Service Mesh, uses the service mesh Istio, running on Kubernetes, to trace an application and compare the results with tracing an application that is natively instrumented for tracing via the OpenTracing API. It reviews the pros and cons of each approach.

Chapter 8, All About Sampling, explains why tracing platforms are often required to sample transactions and provides an in-depth review of different sampling techniques, from consistent head-based sampling strategies (probabilistic, rate limiting, adaptive, and so on) to the emerging favorite, tail-based sampling.

Part III, Getting Value from Tracing, talks about the different ways engineers and organization can benefit from adopting a distributed tracing solution.

Chapter 9, Turning the Lights On, gives examples of the core value proposition of tracing, covering features that are commonly available in most tracing solutions; such as service graphs; critical path analysis; performance analysis with trace patterns; latency histograms and exemplars; and the long-term profiling techniques.

Chapter 10, Distributed Context Propagation, steps back to discuss context propagation, a technology that underpins most existing tracing infrastructures. It covers Tracing Plane from Brown University, which implements a general-purpose, tool-agnostic framework for context propagation, or "baggage," and covers a number of useful techniques and tools for observability and chaos engineering that have been built on top of context propagation and tracing.

Chapter 11, Integration with Metrics and Logs, shows how all is not lost for traditional monitoring tools, and how combining them with tracing infrastructure gives them new capabilities and makes them more useful in microservices environments.

Chapter 12, Gathering Insights with Data Mining, begins with the basics of data mining and feature extraction from tracing data, followed by a practical example involving the Jaeger backend, Apache Kafka, Elasticsearch, Kibana, an Apache Flink data mining job, and a microservices simulator, microsim. It ends with a discussion of further evolution of data mining techniques, such as inferring and observing trends, and historical and ad hoc data analysis.

Part IV, Deploying and Operating Tracing Infrastructure, completes the book with an assortment of practical advice to the tracing teams about implementing and operating tracing platforms in large organizations.

Chapter 13, Implementing Tracing in Large Organizations, discusses how to overcome many technical and organizational challenges that often prevent wide adoption of distributed tracing in enterprises or full realization of its value.

Chapter 14, Under the Hood of a Distributed Tracing System, starts with a brief discussion of build versus buy considerations, then goes deep into many technical details of the architecture and deployment modes of a tracing platform, such as multi-tenancy, security, operation in multiple data centers, monitoring, and resiliency. The Jaeger project is used to illustrate many architectural decisions, yet overall the content is applicable to most tracing infrastructures.