Apache Flink集成到CDH 与 测试

kamisamak 发布于 2020-07-09 2191 次阅读


[infobox title="CDH中安装"]
Apache Flink中文官网:https://flink.apache.org/zh/
https://archive.cloudera.com/csa/1.0.0.0/下载相应csd文件与parcel文件放置在/opt/cloudera/csd/目录与/opt/cloudera/parcel-repo/目录
重启cloudera-scm-server服务,在cm面板进行分发,根据需求进行安装
如需编译安装可从https://archive.apache.org/dist/flink/下载源码进行编译
[hide reply_to_this="true"]


[/hide]
[/infobox]
[infobox title="使用自带wordCount程序进行测试"]

flink run -m yarn-cluster -yn 4 -yjm 1024 -ytm 1024 /opt/cloudera/parcels/FLINK/lib/flink/examples/streaming/WordCount.jar --input hdfs://node01:8020/noaa/noaa2020.txt --output hdfs://node01:8020/noaa/tmp/result
[root@node02 ~]# hdfs dfs -tail /noaa/tmp/result
49838,98)
(55001,655)
(60002,1243)
(333,21642)
(21061,319)
(70084,1)
(91106,703)
(0105010490999992020011306504,1)
(69976,303)
(023372fm,303)
(15,33114)
(000399999v0201301v003119999999y009900599,1)
(00501,3430)
(00901999999addma1098301999999remmet078metar,1)
(enat,303)
(130650z,31)
(13006kt,130)
(080v170,11)
(cavok,3638)
(m05,3375)
(m09,1548)
(q0983,450)
(rmk,21514)
(wind,28252)
(700ft,303)
(15012kt,73)
(0110010490999992020011307004,1)
(69983,291)
(023367fm,291)
(12,99784)
(000399999v0201401n002019999999n999999999,9)
(00561,433)
(00941098341addaa101000091ma1999999098311md1810021,1)
(9999remsyn05801049,239)
(26,7046)
(1402,250)
(11056,234)
(21094,237)
(39831,82)
(49834,78)
(58002,283)
(333,21643)
(60005,7803)
(0097010490999992020011307504,1)
(69976,304)
(023372fm,304)
(15,33115)
(000399999v0201401n003119999999y009900599,2)
(00501,3431)
(01001999999addma1098301999999remmet070metar,2)
(enat,304)
(130750z,30)
(14006kt,92)
(cavok,3639)
(m05,3376)
(m10,1300)
(q0983,451)
(rmk,21515)
(wind,28253)
(700ft,304)
(17,373)

ResourceManager Web UI

Apache Flink History Server Web UI

[/infobox]

此作者没有提供个人介绍。
最后更新于 2020-07-09