123,123,123

內(nèi)容簡(jiǎn)介

　　本書專為想要學(xué)習(xí)并掌握Apache Pulsar的讀者打造，是一本入門圖書。全書共有13章內(nèi)容。第1章和第2章，從生活實(shí)例中引入實(shí)時(shí)消息系統(tǒng)和事件驅(qū)動(dòng)的概念，幫助大家理解Apache Pulsar系統(tǒng)存在的必要性，以及Pulsar的獨(dú)特性和價(jià)值定位。第3章介紹了Pulsar在雅虎的起源故事、設(shè)計(jì)理念與應(yīng)用場(chǎng)景。第4章探究了Pulsar內(nèi)部的主要組件及其工作原理，幫助大家構(gòu)建對(duì)Pulsar的整體架構(gòu)認(rèn)知。第5章到第10章，詳細(xì)介紹了Pulsar消費(fèi)者、生產(chǎn)者的工作原理，應(yīng)用于生產(chǎn)和消費(fèi)消息且與第三方系統(tǒng)交互的框架Pulsar IO，以及Pulsar Functions、分層存儲(chǔ)、Pulsar SQL等Pulsar的獨(dú)特理念和關(guān)鍵特性。第11章與第12章，展示了如何在諸多環(huán)境中部署Pulsar，以及在運(yùn)維Pulsar的過程中需要關(guān)注的指標(biāo)。在第13章中，作者結(jié)合個(gè)人經(jīng)驗(yàn)闡述了Pulsar未來值得探索的領(lǐng)域和方向。本書配圖生動(dòng)形象，實(shí)例豐富，不僅提供了很多代碼示例，還提供了多篇附錄，以作參考。

作者簡(jiǎn)介

　　Jowanza Joseph 是一位就職于 Finicity 的軟件工程師，他負(fù)責(zé)領(lǐng)導(dǎo) Finicity 開放銀行平臺(tái)的網(wǎng)格開發(fā)。Jowanza 曾在多個(gè)項(xiàng)目中使用 Apache Pulsar，并利用基于 Pulsar 構(gòu)建的全托管消息和流處理平臺(tái)實(shí)現(xiàn)了日均處理十億量級(jí)的消息。近十年中， Jowanza 在 Pluralsight 等公司的工作中都使用過Apache Kafka、Akka、Kubernetes 及其他流技術(shù)與消息技術(shù)。他還曾在 Strange Loop、Abstractions、開源峰會(huì)及 O’Reilly 的 Strata Data & AI 大會(huì)上做過技術(shù)分享。蔡正昕，曾在亞馬遜工作， Pulsar 活躍貢獻(xiàn)者。后來加入了 StreamNative，成為 Cloud 工程師。蔡正昕，軟件工程師。曾在亞馬遜，Streamnative等公司任職，從事數(shù)據(jù)處理系統(tǒng)和云服務(wù)控制面的開發(fā)。19年開始關(guān)注Apache Pulsar社區(qū)并參與討論，漏洞修復(fù)和特性開發(fā)。對(duì)Apache Pulsar，Apache BookKeeper，以及在K8S環(huán)境中運(yùn)維有狀態(tài)系統(tǒng)有比較深入的理解。王殿進(jìn)，目前在 StreamNative 負(fù)責(zé) Apache Pulsar 社區(qū)與市場(chǎng)工作。曾在云計(jì)算廠商、SaaS 公司從事過開發(fā)者社區(qū)運(yùn)營(yíng)與用戶增長(zhǎng)工作。徐文濤，曾任Talkdesk 知識(shí)管理工程師，為多個(gè)企業(yè)級(jí) SaaS、CCaaS 與 PaaS 產(chǎn)品撰寫并審核英文文檔。曾任某世界五百?gòu)?qiáng)本地化團(tuán)隊(duì)終審負(fù)責(zé)人，原 KubeSphere 開源容器平臺(tái)本地化、文檔與內(nèi)容策劃負(fù)責(zé)人。曾獲美國(guó) DZone 專欄銀牌作家，人事部 CATTI 二級(jí)口譯、二級(jí)筆譯，聯(lián)合國(guó) UNLPP 二級(jí)筆譯，Linux 基金會(huì)認(rèn)證 Kubernetes 管理員（CKA）、應(yīng)用開發(fā)者（CKAD）與安全專家（CKS）等行業(yè)資格或榮譽(yù)。

圖書目錄

前言 ....................................................................................................xvii
第 1 章　實(shí)時(shí)消息的價(jià)值 ........................................................................ 1
流動(dòng)的數(shù)據(jù) ............................................................................................................1
資源效率 ................................................................................................................3
使用場(chǎng)景 ................................................................................................................4
銀行 .................................................................................................................5
醫(yī)療 .................................................................................................................6
安全 .................................................................................................................7
物聯(lián)網(wǎng) .............................................................................................................8
小結(jié) .....................................................................................................................10
第 2 章　事件流和事件代理 ................................................................... 11
發(fā)布 / 訂閱 ........................................................................................................... 11
隊(duì)列 .....................................................................................................................13
故障類型 ..............................................................................................................15
推送與輪詢 ..........................................................................................................15
為什么需要 Pulsar ................................................................................................16
流和隊(duì)列的統(tǒng)一 ............................................................................................16
模塊化 ...........................................................................................................16
性能 ...............................................................................................................18
小結(jié) .....................................................................................................................19
第 3 章　認(rèn)識(shí) Pulsar ............................................................................ 20
Pulsar 的起源 .......................................................................................................20
Pulsar 的設(shè)計(jì)理念 ................................................................................................21
多租戶 ...........................................................................................................22
跨地域復(fù)制 ...................................................................................................23
性能 ...............................................................................................................27
模塊化 ...........................................................................................................27
Pulsar 生態(tài) ...........................................................................................................28
Pulsar Functions ............................................................................................28
Pulsar IO .......................................................................................................29
Pulsar SQL ....................................................................................................30
Pulsar 成功案例 ...................................................................................................30
雅虎日本 .......................................................................................................30
Splunk ...........................................................................................................31
Iterable ..........................................................................................................32
小結(jié) .....................................................................................................................33
第 4 章　Pulsar 內(nèi)部組件 ..................................................................... 34
Broker ..................................................................................................................34
消息緩存 .......................................................................................................35
與 BookKeeper、ZooKeeper 交互 .................................................................37
Schema 校驗(yàn) ..................................................................................................38
Broker 間的通信 ............................................................................................38
Pulsar Functions 和 Pulsar IO ........................................................................39
Apache BookKeeper .............................................................................................39
預(yù)寫式日志 ...................................................................................................45
消息存儲(chǔ) .......................................................................................................45
對(duì)象 /Blob 存儲(chǔ) .............................................................................................46
Pravega ..........................................................................................................47
Majordodo .....................................................................................................47
Apache ZooKeeper ...............................................................................................48
命名服務(wù) .......................................................................................................49
配置管理 .......................................................................................................49
選主 ...............................................................................................................50
通知系統(tǒng) .......................................................................................................50
Apache Kafka ................................................................................................50
Apache Druid .................................................................................................51
Pulsar Proxy .........................................................................................................52
Java 虛擬機(jī)（JVM） .............................................................................................53
Netty ..............................................................................................................54
Apache Spark .................................................................................................55
Apache Lucene ..............................................................................................55
小結(jié) .....................................................................................................................55
第 5 章　消費(fèi)者 .................................................................................... 56
成為消費(fèi)者意味著什么 ........................................................................................56
訂閱 .....................................................................................................................57
獨(dú)占 ...............................................................................................................60
共享 ...............................................................................................................61
鍵共享 ...........................................................................................................62
故障轉(zhuǎn)移 .......................................................................................................65
確認(rèn) .....................................................................................................................66
單條確認(rèn) .......................................................................................................66
累積確認(rèn) .......................................................................................................67
Schema .................................................................................................................67
消費(fèi)者 Schema 管理 .....................................................................................68
消費(fèi)模式 ..............................................................................................................68
批處理 ...........................................................................................................68
分塊 ...............................................................................................................69
高級(jí)配置 ..............................................................................................................70
延遲消息 .......................................................................................................70
保留策略 .......................................................................................................70
堆積限額 .......................................................................................................71
配置消費(fèi)者 ..........................................................................................................72
回放 ...............................................................................................................73
死信主題 .......................................................................................................75
重試主題 .......................................................................................................76
小結(jié) .....................................................................................................................76
第 6 章　生產(chǎn)者 .................................................................................... 77
同步生產(chǎn)者 ..........................................................................................................77
異步生產(chǎn)者 ..........................................................................................................78
生產(chǎn)者路由 ..........................................................................................................79
輪詢路由 .......................................................................................................79
單分區(qū)路由 ...................................................................................................80
定制分區(qū)路由 ................................................................................................81
生產(chǎn)者配置 ..........................................................................................................83
topicName .....................................................................................................83
producerName ...............................................................................................83
sendTimeoutMs .............................................................................................83
blockIfQueueFull ...........................................................................................84
maxPendingMessages ....................................................................................84
maxPendingMessagesAcrossPartitions ...........................................................84
messageRoutingMode ....................................................................................84
hashingScheme ..............................................................................................84
cryptoFailureAction .......................................................................................85
batchingMaxPublishDelayMicros ..................................................................85
batchingMaxMessages ...................................................................................85
batchingEnabled ............................................................................................85
compressionType ...........................................................................................85
Schema on Write ..................................................................................................86
使用 Schema Registry ....................................................................................86
非持久化主題 ......................................................................................................89
使用場(chǎng)景 .......................................................................................................90
使用非持久化主題 ........................................................................................90
事務(wù) .....................................................................................................................91
小結(jié) .....................................................................................................................92
第 7 章　Pulsar IO ............................................................................... 93
Pulsar IO 架構(gòu) ......................................................................................................93
運(yùn)行時(shí) ...........................................................................................................94
性能考量 .......................................................................................................94
使用場(chǎng)景 ..............................................................................................................95
簡(jiǎn)單事件處理管道 ........................................................................................95
變化數(shù)據(jù)捕獲 ................................................................................................96
考量 .....................................................................................................................97
消息序列化 ...................................................................................................97
管道穩(wěn)定性 ...................................................................................................97
故障處理 .......................................................................................................98
示例 .....................................................................................................................99
Elasticsearch ..................................................................................................99
Netty ............................................................................................................ 100
創(chuàng)建自己的連接器 ............................................................................................. 101
TimescaleDB ............................................................................................... 101
小結(jié) ................................................................................................................... 102
第 8 章　Pulsar Functions .................................................................. 103
流處理 ................................................................................................................ 103
Pulsar Functions 架構(gòu) ......................................................................................... 105
運(yùn)行時(shí) ......................................................................................................... 105
隔離性 ......................................................................................................... 106
在 Kubernetes 上部署 Pulsar Functions 的隔離性 .............................................. 107
使用場(chǎng)景 ............................................................................................................ 107
創(chuàng)建 Pulsar Functions .................................................................................. 108
簡(jiǎn)單事件處理 .............................................................................................. 108
主題清理 ..................................................................................................... 110
主題數(shù)據(jù)統(tǒng)計(jì) .............................................................................................. 113
小結(jié) ................................................................................................................... 115
第 9 章　分層存儲(chǔ) .............................................................................. 116
在云端存儲(chǔ)數(shù)據(jù) ................................................................................................. 117
對(duì)象存儲(chǔ) ..................................................................................................... 118
使用場(chǎng)景 ............................................................................................................ 120
復(fù)制 ............................................................................................................. 120
CQRS .......................................................................................................... 121
災(zāi)難恢復(fù) ..................................................................................................... 122
卸載數(shù)據(jù) ............................................................................................................ 122
Pulsar Offloader ........................................................................................... 122
檢索已卸載的數(shù)據(jù) ............................................................................................. 129
與對(duì)象存儲(chǔ)數(shù)據(jù)交互 ................................................................................... 129
重填主題 ..................................................................................................... 129
使用 Pulsar 客戶端 ...................................................................................... 130
小結(jié) ................................................................................................................... 131
第 10 章　Pulsar SQL ........................................................................ 132
流即表 ................................................................................................................ 133
SQL-on-Anything 引擎 ...................................................................................... 135
Apache Flink: 另一種視角 ........................................................................... 136
Presto/Trino ................................................................................................. 137
Pulsar SQL 的工作原理 ...................................................................................... 138
配置 Pulsar SQL ................................................................................................. 140
性能考量 ............................................................................................................ 142
小結(jié) ................................................................................................................... 143
第 11 章　部署 Pulsar ........................................................................ 144
Docker ................................................................................................................ 144
裸金屬 ................................................................................................................ 146
最低要求 ..................................................................................................... 146
開始安裝 ..................................................................................................... 146
部署 ZooKeeper........................................................................................... 147
啟動(dòng) BookKeeper ........................................................................................ 148
啟動(dòng) Pulsar .................................................................................................. 148
公有云廠商 ........................................................................................................ 149
AWS ............................................................................................................ 151
Azure ........................................................................................................... 152
Google Cloud Platform ................................................................................ 153
Kubernetes ......................................................................................................... 153
小結(jié) ................................................................................................................... 155
第 12 章　運(yùn)維 Pulsar ........................................................................ 156
Apache BookKeeper 指標(biāo) .................................................................................. 157
服務(wù)器指標(biāo) ................................................................................................. 157
Journal 指標(biāo) ................................................................................................ 158
存儲(chǔ)指標(biāo) ..................................................................................................... 159
Apache ZooKeeper 指標(biāo) ..................................................................................... 159
服務(wù)器指標(biāo) ................................................................................................. 160
請(qǐng)求指標(biāo) ..................................................................................................... 161
主題指標(biāo) ............................................................................................................ 161
消費(fèi)者指標(biāo) ........................................................................................................ 163
Pulsar 事務(wù)指標(biāo) ................................................................................................. 163
Pulsar Functions 指標(biāo) ......................................................................................... 164
高級(jí)運(yùn)維技巧 .................................................................................................... 165
攔截器和追蹤機(jī)制 ...................................................................................... 165
Pulsar SQL 指標(biāo) .......................................................................................... 167
指標(biāo)轉(zhuǎn)發(fā) ............................................................................................................ 169
監(jiān)控面板 ..................................................................................................... 169
小結(jié) ................................................................................................................... 170
第 13 章　未來 ................................................................................... 171
編程語言支持 .................................................................................................... 171
擴(kuò)展接口 ..................................................................................................... 171
增強(qiáng) Pulsar Functions .................................................................................. 172
架構(gòu)簡(jiǎn)化 / 擴(kuò)展 ........................................................................................... 175
消息平臺(tái)橋接器 .......................................................................................... 177
小結(jié) ................................................................................................................... 180
附錄 A Pulsar Admin API ................................................................. 181
附錄 B Pulsar Admin CLI ................................................................. 189
附錄 C　跨地域復(fù)制 ........................................................................... 192
附錄 D Pulsar 中的安全、認(rèn)證和鑒權(quán) ............................................... 198
關(guān)于作者 ............................................................................................ 202
封面介紹 ............................................................................................ 202