| 澶辨晥閾炬帴澶勭悊 |
|
kafka瀛︿範(fàn) PDF 涓嬭澆
鏈珯鏁寸悊涓嬭澆錛?/strong>
閾炬帴錛?a target="_blank">https://pan.baidu.com/s/1dFiGtWODstoCKSpn55Pz3g
鎻愬彇鐮侊細(xì)dk23
鐩稿叧鎴浘錛?/strong>
涓昏鍐呭錛?/strong>
1銆並afka綆€浠?/div>
Kafka鏄竴涓紑婧愮殑鍒嗗竷寮忔秷鎭紩鎿?娑堟伅涓棿浠訛紝鍚屾椂Kafka涔熸槸涓€涓祦澶勭悊騫沖彴銆侹akfa鏀寔浠ュ彂甯?璁㈤槄鐨勬柟寮忓湪搴旂敤闂翠紶閫掓秷鎭紝鍚屾椂騫跺熀浜庢秷鎭姛鑳芥坊鍔犱簡(jiǎn)Kafka Connect銆並afka Streams浠ユ敮鎸佽繛鎺ュ叾浠栫郴緇熺殑鏁版嵁(Elasticsearch銆丠adoop絳?
Kafka鏈€鏍稿績(jī)鐨勬渶鎴愮啛鐨勮繕鏄粬鐨勬秷鎭紩鎿庯紝鎵€浠afka澶ч儴鍒嗗簲鐢ㄥ満鏅繕鏄敤鏉ヤ綔涓烘秷鎭槦鍒楀墛宄板鉤璋楓€傚彟澶栵紝Kafka涔熸槸鐩墠鎬ц兘鏈€濂界殑娑堟伅涓棿浠躲€?/div>
2銆並afka鏋舵瀯
鍦↘afka闆嗙兢(Cluster)涓紝涓€涓狵afka鑺傜偣灝辨槸涓€涓狟roker錛屾秷鎭敱Topic鏉ユ壙杞斤紝鍙互瀛樺偍鍦?涓垨澶氫釜Partition涓€傚彂甯冩秷鎭殑搴旂敤涓篜roducer銆佹秷璐規(guī)秷鎭殑搴旂敤涓篊onsumer錛屽涓狢onsumer鍙互淇?jī)鎴怌onsumer Group鍏卞悓娑堣垂涓€涓猅opic涓殑娑堟伅銆?/div>
姒傚康/瀵硅薄 綆€鍗曡鏄?/div>
Broker Kafka鑺傜偣
Topic 涓婚錛岀敤鏉ユ壙杞芥秷鎭?/div>
Partition 鍒嗗尯錛岀敤浜庝富棰樺垎鐗囧瓨鍌?/div>
Producer 鐢熶駭鑰咃紝鍚戜富棰樺彂甯冩秷鎭殑搴旂敤
Consumer 娑堣垂鑰咃紝浠庝富棰樿闃呮秷鎭殑搴旂敤
Consumer Group 娑堣垂鑰呯粍錛岀敱澶氫釜娑堣垂鑰呯粍鎴?/div>
3銆佸噯澶囧伐浣?/div>
Kafka鏈嶅姟鍣?/div>
鍑嗗3鍙癈entOS鏈嶅姟鍣紝騫墮厤緗ソ闈?rùn)鎬両P銆佷富鏈哄悕
Kakfa闆嗙兢闇€瑕佷緷璧朲ooKeeper瀛樺偍Broker銆乀opic絳変俊鎭紝榪欓噷鎴戜滑閮ㄧ講涓夊彴ZK
192.168.218.101 Kafka鑺傜偣1 zk
192.168.218.102 Kafka鑺傜偣2 zk
192.168.218.103 Kafka鑺傜偣3 zk
瀹夎閮ㄧ講濂絢afka涔嬪悗錛屾煡鐪媖afka_2.11-1.1.0/config/server.properties錛岄厤緗互涓嬪唴瀹?/div>
璁劇疆broker.id榪欓噷鏄敤浜庢姇紲ㄩ€変婦錛屾瘡涓妭鐐圭殑鏁板瓧涓嶄竴鏍?/div>
璁劇疆kafka鏃ュ織瀛樻斁鐩綍
璁劇疆鐩戝惉绔彛
浠ヤ笂璁劇疆闄や簡(jiǎn)broker.id 鐩戝惉绔彛ip闇€瑕佷慨鏀癸紝鍏朵綑涓嶇敤淇敼銆?/div>
闄勶細(xì)kafka甯哥敤閰嶇疆
4 鍚姩kafka
/bin/kafka-server-start.sh config/server.properties &
鍦╧8s闆嗙兢涓惎鍔╧afka鐨刾od,鍚姩涓夊彴錛宐roker-0/broker-1/broker-2
1銆乲afka鏈嶅姟鍚姩錛?/kafka-server-start.sh ../config/server.properties
-daemon琛ㄧず闈?rùn)榛樻柟寮忓惎鍔?/div>
2銆佹煡鐪嬫湁鍝簺涓婚錛?./kafka-topics.sh --list –zookeeper chen-1:2181,chen-2:2181,chen-3:2181
3銆佸垱寤簍opic錛?/kafka-topics.sh --create --zookeeper chen-1:2181,chen-2:2181,chen-3:2181 --replication-factor 1 --partitions 1 --topic chen
4銆佹煡鐪媡opic鐨勮緇嗕俊鎭細(xì)./kafka-topics.sh –zookeeper chen-1:2181,chen-2:2181,chen-3:2181 -describe -topic chen
5銆乲afka鐢熶駭鑰呭鎴風(fēng)鍛戒護(hù)錛?/kafka-console-producer.sh --broker-list chen-1:9092 --topic chen 鐢熶駭鑰呯鍙互浜х敓鏁版嵁渚涙秷璐硅€呮秷璐?/div>
6銆乲afka娑堣垂鑰呭鎴風(fēng)鍛戒護(hù)錛?/kafka-console-consumer.sh --bootstrap-server chen-2:9092
--topic chen
./kafka-console-consumer.sh --bootstrap-server chen-3:9092 --from-beginning --topic chen
--from-beginning鍙傛暟鍙互浠庡ご寮€濮嬫秷璐規(guī)暟鎹?/div>
姝ゆ椂榪涘叆zookeeper瀹㈡埛绔紝鍙湅鍒板湪zk涓凡緇忓瓨鍌ㄤ簡(jiǎn)鍜宬afka鐩稿叧鐨勬暟鎹?/div>
__consumer_offsets琛ㄧず璁板綍娑堣垂鐨勫亸縐婚噺
7銆佸垹闄opic錛?/kafka-topics.sh --zookeeper zk鏈嶅姟IP:2181 --delete --topic WordCount錛堟敞錛氫笉鑳界湡姝ｅ垹闄opic鍙槸鎶婅繖涓猼opic鏍囪涓哄垹闄わ紙marked for deletion錛夛紝瑕佸交搴曟妸topic鍒犻櫎蹇呴』鎶妅afka涓笌褰撳墠topic鐩稿叧鐨勬暟鎹洰褰曞拰zookeeper涓笌褰撳墠topic鐩稿叧鐨勮礬寰勪竴騫跺垹闄ゃ€傦級(jí)
5 娑堣垂鑰呯粍錛圕onsumer Group錛?/div>
娑堣垂鑰呯粍鏄疜afka瀹炵幇鍗曟挱鍜屽箍鎾袱縐嶆秷鎭ā鍨嬬殑鎵嬫銆傚悓涓€涓猼opic錛屾瘡涓秷璐硅€呯粍閮藉彲浠ユ嬁鍒扮浉鍚岀殑鍏ㄩ儴鏁版嵁銆?/div>
璁劇疆consumer.properties鏂囦歡
|
| Java1234瀹樻柟緹?5錛?/td> | 
|
| Java1234瀹樻柟緹?5錛?/td> | 838462530 |
鑻忓叕緗戝畨澶?32061202001004鍙?/p>
