Apache Flink集成到CDH 与 测试

[infobox title=”CDH中安装”]
Apache Flink中文官网:https://flink.apache.org/zh/
在https://archive.cloudera.com/csa/1.0.0.0/下载相应csd文件与parcel文件放置在/opt/cloudera/csd/目录与/opt/cloudera/parcel-repo/目录
重启cloudera-scm-server服务，在cm面板进行分发，根据需求进行安装
如需编译安装可从https://archive.apache.org/dist/flink/下载源码进行编译
[hide reply_to_this=”true”]


[/hide]
[/infobox]
[infobox title=”使用自带wordCount程序进行测试”]

flink run -m yarn-cluster -yn 4 -yjm 1024 -ytm 1024 /opt/cloudera/parcels/FLINK/lib/flink/examples/streaming/WordCount.jar --input hdfs://node01:8020/noaa/noaa2020.txt --output hdfs://node01:8020/noaa/tmp/result

[root@node02 ~]# hdfs dfs -tail /noaa/tmp/result
49838,98)
(55001,655)
(60002,1243)
(333,21642)
(21061,319)
(70084,1)
(91106,703)
(0105010490999992020011306504,1)
(69976,303)
(023372fm,303)
(15,33114)
(000399999v0201301v003119999999y009900599,1)
(00501,3430)
(00901999999addma1098301999999remmet078metar,1)
(enat,303)
(130650z,31)
(13006kt,130)
(080v170,11)
(cavok,3638)
(m05,3375)
(m09,1548)
(q0983,450)
(rmk,21514)
(wind,28252)
(700ft,303)
(15012kt,73)
(0110010490999992020011307004,1)
(69983,291)
(023367fm,291)
(12,99784)
(000399999v0201401n002019999999n999999999,9)
(00561,433)
(00941098341addaa101000091ma1999999098311md1810021,1)
(9999remsyn05801049,239)
(26,7046)
(1402,250)
(11056,234)
(21094,237)
(39831,82)
(49834,78)
(58002,283)
(333,21643)
(60005,7803)
(0097010490999992020011307504,1)
(69976,304)
(023372fm,304)
(15,33115)
(000399999v0201401n003119999999y009900599,2)
(00501,3431)
(01001999999addma1098301999999remmet070metar,2)
(enat,304)
(130750z,30)
(14006kt,92)
(cavok,3639)
(m05,3376)
(m10,1300)
(q0983,451)
(rmk,21515)
(wind,28253)
(700ft,304)
(17,373)

ResourceManager Web UI

Apache Flink History Server Web UI

[/infobox]