这本书解决了什么问题？

系统选型缺乏依据：通过识别不同工具的优缺点，帮助读者在数据库和消息队列等选型中做出更明智的决策。不理解一致性与容错权衡：深入剖析一致性、可伸缩性、容错性和复杂度之间的权衡关系，建立系统设计判断力。分布式理论难以落地：将分布式系统研究与现代数据库构建基石相结合，弥合理论与工程实践的鸿沟。不了解主流在线服务架构：带领读者走到主流在线服务幕后，学习其数据系统架构设计思路。已有系统使用效率低：深入分析读者已在使用的系统，帮助更高效地使用和运维这些系统。

这本书适合哪些读者？

软件工程师与架构师：需要设计和实现数据基础设施，必须理解分布式系统的核心权衡与设计原则。后端开发人员：日常使用数据库和消息队列，希望深入理解底层原理以提升系统设计能力。数据库运维人员：需要了解不同存储引擎和复制机制的优劣，优化系统运维效率。计算机专业高年级学生与研究生：希望将分布式系统理论知识与工业界实践相结合。

这本书应该怎么读？

先读第一部分打基础：第1至4章建立数据系统基础认知，是后续深入阅读的前提。重点精读复制与分区章节：第5、6章是分布式系统的核心，建议结合工作实际反复研读。事务与一致性章节需慢读：第7至9章概念密集，建议配合笔记和图示辅助理解。批处理与流处理按需阅读：第10、11章可根据工作方向选择性深入，不必强求一次读完。结合已有系统对照思考：阅读时对照自己正在使用的系统，思考其设计取舍与改进空间。

读完这本书能收获什么？

建立系统思维：能够从可靠性、可伸缩性和可维护性三个维度评估和设计数据系统。掌握选型能力：理解不同数据模型、存储引擎和查询语言的适用场景，做出明智技术决策。理解分布式核心：深入掌握复制、分区、事务和共识等分布式系统核心概念与实现机制。洞察架构演进：了解主流在线服务的数据架构设计，理解批处理与流处理的融合趋势。避免常见错误：通过学习前人的经验教训，在系统设计中规避常见的分布式陷阱。

E-BOOK 设计数据密集型应用

PDF

设计数据密集型应用

📥 PDF 电子书 · 百度网盘 · 约 68M 📖 南京东南大学出版社 🌐 zh-CN 📅 2,017 年出版

279

下载次数

4.8

用户评分

PDF

数据库

📦 下载本书

📥 立即获取 PDF 👁️ 预览

系统选型缺乏依据：通过识别不同工具的优缺点，帮助读者在数据库和消息队列等选型中做出更明智的决策。
不理解一致性与容错权衡：深入剖析一致性、可伸缩性、容错性和复杂度之间的权衡关系，建立系统设计判断力。
分布式理论难以落地：将分布式系统研究与现代数据库构建基石相结合，弥合理论与工程实践的鸿沟。
不了解主流在线服务架构：带领读者走到主流在线服务幕后，学习其数据系统架构设计思路。
已有系统使用效率低：深入分析读者已在使用的系统，帮助更高效地使用和运维这些系统。

📖 书籍简介

内容简介

书中包含以下内容:
? 深入分析你已经在使用的系统，并学习如何更高效地使用和运维这些系统
? 通过识别不同工具的优缺点，作出更明智的决策
? 了解一致性、可伸缩性、容错性和复杂度之间的权衡
? 理解分布式系统研究，这些研究是现代数据库构建的基石
? 走到一些主流在线服务的幕后，学习它们的架构

作者简介

Martin Kleppmann，是英国剑桥大学的一名分布式系统研究员。在此之前他曾是软件工程师和企业家，在 Linkedin 和 Rapportive 工作过，从事大规模数据基础设施相关的工作。Martin 经常在大会做演讲，写博客，也是开源贡献者。

精彩书评

“这本书太棒了，它在分布式系统理论和实际工程之间的巨大鸿沟上架起了一座桥梁。多希望十年前就能读到这本书，那么这些年来我犯的很多错误就都能避免了。”
——Jay Kreps（Apache Kafka 创始人，Confluent CEO）
“这是一本软件工程师的必读之作。《设计数据密集型应用》是能够连接理论和实践的稀有资料，它能帮助开发者在设计和实现数据基础设施及系统的时候作出明智的决策。”
——Kevin Scoot（微软CTO）

Part I. Foundations of Data Systems
1. Reliable, Scalable, and Maintainable Applications 3
Thinking About Data Systems 4
Reliability 6
Hardware Faults 7
Software Errors 8
Human Errors 9
How Important Is Reliability? 10
Scalability 10
Describing Load 11
Describing Performance 13
Approaches for Coping with Load 17
Maintainability 18
Operability: Making Life Easy for Operations 19
Simplicity: Managing Complexity 20
Evolvability: Making Change Easy 21
Summary 22
2. Data Models and Query Languages 27
Relational Model Versus Document Model 28
The Birth of NoSQL 29
The Object-Relational Mismatch 29
Many-to-One and Many-to-Many Relationships 33
Are Document Databases Repeating History? 36
Relational Versus Document Databases Today 38
Query Languages for Data 42
Declarative Queries on the Web 44
MapReduce Querying 46
Graph-Like Data Models 49
Property Graphs 50
The Cypher Query Language 52
Graph Queries in SQL 53
Triple-Stores and SPARQL 55
The Foundation: Datalog 60
Summary 63
3. Storage and Retrieval 69
Data Structures That Power Your Database 70
Hash Indexes 72
SSTables and LSM-Trees 76
B-Trees 79
Comparing B-Trees and LSM-Trees 83
Other Indexing Structures 85
Transaction Processing or Analytics? 90
Data Warehousing 91
Stars and Snowflakes: Schemas for Analytics 93
Column-Oriented Storage 95
Column Compression 97
Sort Order in Column Storage 99
Writing to Column-Oriented Storage 101
Aggregation: Data Cubes and Materialized Views 101
Summary 103
4. Encoding and Evolution 111
Formats for Encoding Data 112
Language-Specific Formats 113
JSON, XML, and Binary Variants 114
Thrift and Protocol Buffers 117
Avro 122
The Merits of Schemas 127
Modes of Dataflow 128
Dataflow Through Databases 129
Dataflow Through Services: REST and RPC 131
Message-Passing Dataflow 136
Summary 139
Part II. Distributed Data
5. Replication 151
Leaders and Followers 152
Synchronous Versus Asynchronous Replication 153
Setting Up New Followers 155
Handling Node Outages 156
Implementation of Replication Logs 158
Problems with Replication Lag 161
Reading Your Own Writes 162
Monotonic Reads 164
Consistent Prefix Reads 165
Solutions for Replication Lag 167
Multi-Leader Replication 168
Use Cases for Multi-Leader Replication 168
Handling Write Conflicts 171
Multi-Leader Replication Topologies 175
Leaderless Replication 177
Writing to the Database When a Node Is Down 177
Limitations of Quorum Consistency 181
Sloppy Quorums and Hinted Handoff 183
Detecting Concurrent Writes 184
Summary 192
6. Partitioning 199
Partitioning and Replication 200
Partitioning of Key-Value Data 201
Partitioning by Key Range 202
Partitioning by Hash of Key 203
Skewed Workloads and Relieving Hot Spots 205
Partitioning and Secondary Indexes 206
Partitioning Secondary Indexes by Document 206
Partitioning Secondary Indexes by Term 208
Rebalancing Partitions 209
Strategies for Rebalancing 210
Operations: Automatic or Manual Rebalancing 213
Request Routing 214
Parallel Query Execution 216
Summary 216
7. Transactions 221
The Slippery Concept of a Transaction 222
The Meaning of ACID 223
Single-Object and Multi-Object Operations 228
Weak Isolation Levels 233
Read Committed 234
Snapshot Isolation and Repeatable Read 237
Preventing Lost Updates 242
Write Skew and Phantoms 246
Serializability 251
Actual Serial Execution 252
Two-Phase Locking (2PL) 257
Serializable Snapshot Isolation (SSI) 261
Summary 266
8. The Trouble with Distributed Systems 273
Faults and Partial Failures 274
Cloud Computing and Supercomputing 275
Unreliable Networks 277
Network Faults in Practice 279
Detecting Faults 280
Timeouts and Unbounded Delays 281
Synchronous Versus Asynchronous Networks 284
Unreliable Clocks 287
Monotonic Versus Time-of-Day Clocks 288
Clock Synchronization and Accuracy 289
Relying on Synchronized Clocks 291
Process Pauses 295
Knowledge, Truth, and Lies 300
The Truth Is Defined by the Majority 300
Byzantine Faults 304
System Model and Reality 306
Summary 310
9. Consistency and Consensus 321
Consistency Guarantees 322
Linearizability 324
What Makes a System Linearizable? 325
Relying on Linearizability 330
Implementing Linearizable Systems 332
The Cost of Linearizability 335
Ordering Guarantees 339
Ordering and Causality 339
Sequence Number Ordering 343
Total Order Broadcast 348
Distributed Transactions and Consensus 352
Atomic Commit and Two-Phase Commit (2PC) 354
Distributed Transactions in Practice 360
Fault-Tolerant Consensus 364
Membership and Coordination Services 370
Summary 373
Part III. Derived Data
10. Batch Processing 389
Batch Processing with Unix Tools 391
Simple Log Analysis 391
The Unix Philosophy 394
MapReduce and Distributed Filesystems 397
MapReduce Job Execution 399
Reduce-Side Joins and Grouping 403
Map-Side Joins 408
The Output of Batch Workflows 411
Comparing Hadoop to Distributed Databases 414
Beyond MapReduce 419
Materialization of Intermediate State 419
Graphs and Iterative Processing 424
High-Level APIs and Languages 426
Summary 429
11. Stream Processing 439
Transmitting Event Streams 440
Messaging Systems 441
Partitioned Logs 446
Databases and Streams 451
Keeping Systems in Sync 452
Change Data Capture 454
Event Sourcing 457
State, Streams, and Immutability 459
Processing Streams 464
Uses of Stream Processing 465
Reasoning About Time 468
Stream Joins 472
Fault Tolerance 476
Summary 479
12. The Future of Data Systems 489
Data Integration 490
Combining Specialized Tools by Deriving Data 490
Batch and Stream Processing 494
Unbundling Databases 499
Composing Data Storage Technologies 499
Designing Applications Around Dataflow 504
Observing Derived State 509
Aiming for Correctness 515
The End-to-End Argument for Databases 516
Enforcing Constraints 521
Timeliness and Integrity 524
Trust, but Verify 528
Doing the Right Thing 533
Predictive Analytics 533
Privacy and Tracking 536
Summary 543
Glossary 553
Index 559

查看全部↓