Event Debug Server Status

Version: 6.3.1 (#1466458 built by jenkins on 20190925-0746 git: 56b387edf7c27c5d67c3391b7fa709c082f2114c)
Time now: October 14, 2024 3:24:06 AM +07:00

 |  Configure Logging  |  Threads  |  Thread CPU Time  |  Poor Man's Profiler  |  JMX  |  GC Pause Information

EventCatcherService (com.cloudera.cmf.eventcatcher.server.EventCatcherService)
There are 5000013 documents in the index


Lucene index manager (com.cloudera.cmf.eventcatcher.server.SingleIndexManager)
There are 5000013 events in the index with 10368 deleted events waiting to be expunged.
Last searcher refresh occurred 10.74 secs ago and took 0.000 secs.
Last commit occurred 54.65 secs ago and took 0.004 secs.
Last cleanup occurred 201.00 secs ago and took 3.353 secs.


Event Ingester (com.cloudera.cmf.eventcatcher.server.EventIngester)
Seen 68747 events
Last events: 
  • AvroEventWrapper{attributes={ROLE=[hdfs-DATANODE-9fc5d34330705b6607ef0069e1c77f0c], HOSTS=[datanode4.moe.bigdata.go.th], ROLE_TYPE=[DATANODE], EXCEPTION_TYPES=[java.net.ConnectException, java.net.ConnectException], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[hdfs], SERVICE_TYPE=[HDFS], LOG_LEVEL=[WARN], HOST_IDS=[30986536-d77d-48d8-8fd5-6917445bf616], STACKTRACE=[java.net.ConnectException: Call From datanode4.moe.bigdata.go.th/192.168.239.39 to namenode1.moe.bigdata.go.th:8022 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
	at sun.reflect.GeneratedConstructorAccessor20.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:824)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:754)
	at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1503)
	at org.apache.hadoop.ipc.Client.call(Client.java:1445)
	at org.apache.hadoop.ipc.Client.call(Client.java:1355)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
	at com.sun.proxy.$Proxy20.sendHeartbeat(Unknown Source)
	at org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolClientSideTranslatorPB.sendHeartbeat(DatanodeProtocolClientSideTranslatorPB.java:166)
	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.sendHeartBeat(BPServiceActor.java:516)
	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:646)
	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:847)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
	at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
	at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:686)
	at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:789)
	at org.apache.hadoop.ipc.Client$Connection.access$3600(Client.java:410)
	at org.apache.hadoop.ipc.Client.getConnection(Client.java:1560)
	at org.apache.hadoop.ipc.Client.call(Client.java:1391)
	... 9 more
], SEVERITY=[IMPORTANT]}, content=IOException in offerService, timestamp=1728850809764}
  • AvroEventWrapper{attributes={ROLE=[hdfs-DATANODE-12c9e4112cea25acd51602f1019f29c4], HOSTS=[datanode1.moe.bigdata.go.th], ROLE_TYPE=[DATANODE], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[hdfs], SERVICE_TYPE=[HDFS], LOG_LEVEL=[WARN], HOST_IDS=[f8bc3e01-bdae-4428-8858-572d5e23dd92], SEVERITY=[IMPORTANT]}, content=Problem connecting to server: namenode1.moe.bigdata.go.th/192.168.239.34:8022, timestamp=1728850811075}
  • AvroEventWrapper{attributes={ROLE=[hdfs-DATANODE-f3e08d3125e90218d8495af250c2e2a0], HOSTS=[datanode3.moe.bigdata.go.th], ROLE_TYPE=[DATANODE], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[hdfs], SERVICE_TYPE=[HDFS], LOG_LEVEL=[WARN], HOST_IDS=[ee0391a7-88ed-4fed-ac39-f95e9a63899c], SEVERITY=[IMPORTANT]}, content=Incorrect header or version mismatch from 162.142.125.213:36144 got version 42 expected version 9, timestamp=1728850813525}
  • AvroEventWrapper{attributes={ROLE=[hdfs-DATANODE-dec125281080803bb2b9288dd9aff9c2], HOSTS=[namenode2.moe.bigdata.go.th], ROLE_TYPE=[DATANODE], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[hdfs], SERVICE_TYPE=[HDFS], LOG_LEVEL=[WARN], HOST_IDS=[14d8f5e0-ecc4-4d14-bbe1-89fa8cfde10d], SEVERITY=[IMPORTANT]}, content=Problem connecting to server: namenode1.moe.bigdata.go.th/192.168.239.34:8022, timestamp=1728850816604}
  • AvroEventWrapper{attributes={ROLE=[hdfs-DATANODE-f3e08d3125e90218d8495af250c2e2a0], HOSTS=[datanode3.moe.bigdata.go.th], ROLE_TYPE=[DATANODE], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[hdfs], SERVICE_TYPE=[HDFS], LOG_LEVEL=[WARN], HOST_IDS=[ee0391a7-88ed-4fed-ac39-f95e9a63899c], SEVERITY=[IMPORTANT]}, content=Problem connecting to server: namenode1.moe.bigdata.go.th/192.168.239.34:8022, timestamp=1728850863533}
  • AvroEventWrapper{attributes={ROLE=[hdfs-DATANODE-dec125281080803bb2b9288dd9aff9c2], HOSTS=[namenode2.moe.bigdata.go.th], ROLE_TYPE=[DATANODE], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[hdfs], SERVICE_TYPE=[HDFS], LOG_LEVEL=[WARN], HOST_IDS=[14d8f5e0-ecc4-4d14-bbe1-89fa8cfde10d], SEVERITY=[IMPORTANT]}, content=Problem connecting to server: namenode1.moe.bigdata.go.th/192.168.239.34:8022, timestamp=1728850863568}
  • AvroEventWrapper{attributes={ROLE=[hdfs-DATANODE-12c9e4112cea25acd51602f1019f29c4], HOSTS=[datanode1.moe.bigdata.go.th], ROLE_TYPE=[DATANODE], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[hdfs], SERVICE_TYPE=[HDFS], LOG_LEVEL=[WARN], HOST_IDS=[f8bc3e01-bdae-4428-8858-572d5e23dd92], SEVERITY=[IMPORTANT]}, content=Problem connecting to server: namenode1.moe.bigdata.go.th/192.168.239.34:8022, timestamp=1728850865441}
  • AvroEventWrapper{attributes={ROLE=[mgmt-SERVICEMONITOR-ee5044a537da9044a46e807f90938051], HOSTS=[namenode1.moe.bigdata.go.th], ROLE_TYPE=[SERVICEMONITOR], EXCEPTION_TYPES=[java.net.ConnectException], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[mgmt], SERVICE_TYPE=[MGMT], LOG_LEVEL=[ERROR], HOST_IDS=[b9648f19-c39e-4032-bb42-77596f055418], STACKTRACE=[java.net.ConnectException: Connection refused (Connection refused)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
	at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
	at sun.net.www.http.HttpClient.New(HttpClient.java:339)
	at sun.net.www.http.HttpClient.New(HttpClient.java:357)
	at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1220)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050)
	at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:984)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1564)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
	at com.cloudera.enterprise.UrlUtil.readUrlWithTimeouts(UrlUtil.java:69)
	at com.cloudera.enterprise.HttpBasedJMXJsonInfoRetriever.doQuery(HttpBasedJMXJsonInfoRetriever.java:298)
	at com.cloudera.enterprise.HttpBasedJMXJsonInfoRetriever.getMBeansByDomain(HttpBasedJMXJsonInfoRetriever.java:173)
	at com.cloudera.cmon.firehose.polling.hdfs.HdfsServiceStateFetcher.getNameNodeInfo(HdfsServiceStateFetcher.java:117)
	at com.cloudera.cmon.firehose.polling.hdfs.HdfsServiceStateFetcher.doWorkWithClientConfig(HdfsServiceStateFetcher.java:89)
	at com.cloudera.cmon.firehose.polling.hdfs.HdfsServiceStateFetcher.doWorkWithClientConfig(HdfsServiceStateFetcher.java:41)
	at com.cloudera.cmon.firehose.polling.AbstractCdhWorkUsingClientConfigs.doWork(AbstractCdhWorkUsingClientConfigs.java:45)
	at com.cloudera.cmon.firehose.polling.CdhTask$InstrumentedWork.doWork(CdhTask.java:230)
	at com.cloudera.cmf.cdhclient.CdhExecutor$1.call(CdhExecutor.java:125)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
], SEVERITY=[CRITICAL]}, content=(14 skipped) Failed to collect information on NameNode hdfs-NAMENODE-ee5044a537da9044a46e807f90938051, timestamp=1728850918985}
  • AvroEventWrapper{attributes={ROLE=[hdfs-DATANODE-0fe285ba69be7b2dc5815de987ad25fa], HOSTS=[datanode2.moe.bigdata.go.th], ROLE_TYPE=[DATANODE], EXCEPTION_TYPES=[java.net.ConnectException, java.net.ConnectException], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[hdfs], SERVICE_TYPE=[HDFS], LOG_LEVEL=[WARN], HOST_IDS=[4700eb51-24f5-4c11-924d-4b99f2ae8954], STACKTRACE=[java.net.ConnectException: Call From datanode2.moe.bigdata.go.th/192.168.239.37 to namenode1.moe.bigdata.go.th:8022 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
	at sun.reflect.GeneratedConstructorAccessor18.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:824)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:754)
	at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1503)
	at org.apache.hadoop.ipc.Client.call(Client.java:1445)
	at org.apache.hadoop.ipc.Client.call(Client.java:1355)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
	at com.sun.proxy.$Proxy20.sendHeartbeat(Unknown Source)
	at org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolClientSideTranslatorPB.sendHeartbeat(DatanodeProtocolClientSideTranslatorPB.java:166)
	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.sendHeartBeat(BPServiceActor.java:516)
	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:646)
	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:847)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
	at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
	at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:686)
	at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:789)
	at org.apache.hadoop.ipc.Client$Connection.access$3600(Client.java:410)
	at org.apache.hadoop.ipc.Client.getConnection(Client.java:1560)
	at org.apache.hadoop.ipc.Client.call(Client.java:1391)
	... 9 more
], SEVERITY=[IMPORTANT]}, content=IOException in offerService, timestamp=1728850921035}
  • AvroEventWrapper{attributes={ROLE=[hdfs-DATANODE-12c9e4112cea25acd51602f1019f29c4], HOSTS=[datanode1.moe.bigdata.go.th], ROLE_TYPE=[DATANODE], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[hdfs], SERVICE_TYPE=[HDFS], LOG_LEVEL=[WARN], HOST_IDS=[f8bc3e01-bdae-4428-8858-572d5e23dd92], SEVERITY=[IMPORTANT]}, content=Problem connecting to server: namenode1.moe.bigdata.go.th/192.168.239.34:8022, timestamp=1728850921786}
  • AvroEventWrapper{attributes={ROLE=[hdfs-DATANODE-9fc5d34330705b6607ef0069e1c77f0c], HOSTS=[datanode4.moe.bigdata.go.th], ROLE_TYPE=[DATANODE], EXCEPTION_TYPES=[java.net.ConnectException, java.net.ConnectException], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[hdfs], SERVICE_TYPE=[HDFS], LOG_LEVEL=[WARN], HOST_IDS=[30986536-d77d-48d8-8fd5-6917445bf616], STACKTRACE=[java.net.ConnectException: Call From datanode4.moe.bigdata.go.th/192.168.239.39 to namenode1.moe.bigdata.go.th:8022 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
	at sun.reflect.GeneratedConstructorAccessor20.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:824)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:754)
	at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1503)
	at org.apache.hadoop.ipc.Client.call(Client.java:1445)
	at org.apache.hadoop.ipc.Client.call(Client.java:1355)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
	at com.sun.proxy.$Proxy20.sendHeartbeat(Unknown Source)
	at org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolClientSideTranslatorPB.sendHeartbeat(DatanodeProtocolClientSideTranslatorPB.java:166)
	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.sendHeartBeat(BPServiceActor.java:516)
	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:646)
	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:847)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
	at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
	at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:686)
	at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:789)
	at org.apache.hadoop.ipc.Client$Connection.access$3600(Client.java:410)
	at org.apache.hadoop.ipc.Client.getConnection(Client.java:1560)
	at org.apache.hadoop.ipc.Client.call(Client.java:1391)
	... 9 more
], SEVERITY=[IMPORTANT]}, content=IOException in offerService, timestamp=1728850923802}
  • AvroEventWrapper{attributes={ROLE=[hdfs-DATANODE-228a1f56d12863c198b2f2fcbc5d1f78], HOSTS=[datanode5.moe.bigdata.go.th], ROLE_TYPE=[DATANODE], EXCEPTION_TYPES=[java.net.ConnectException, java.net.ConnectException], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[hdfs], SERVICE_TYPE=[HDFS], LOG_LEVEL=[WARN], HOST_IDS=[b844e16e-38a1-4dc3-bac9-892675e8a5f8], STACKTRACE=[java.net.ConnectException: Call From datanode5.moe.bigdata.go.th/192.168.239.40 to namenode1.moe.bigdata.go.th:8022 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
	at sun.reflect.GeneratedConstructorAccessor20.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:824)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:754)
	at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1503)
	at org.apache.hadoop.ipc.Client.call(Client.java:1445)
	at org.apache.hadoop.ipc.Client.call(Client.java:1355)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
	at com.sun.proxy.$Proxy20.sendHeartbeat(Unknown Source)
	at org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolClientSideTranslatorPB.sendHeartbeat(DatanodeProtocolClientSideTranslatorPB.java:166)
	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.sendHeartBeat(BPServiceActor.java:516)
	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:646)
	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:847)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
	at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
	at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:686)
	at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:789)
	at org.apache.hadoop.ipc.Client$Connection.access$3600(Client.java:410)
	at org.apache.hadoop.ipc.Client.getConnection(Client.java:1560)
	at org.apache.hadoop.ipc.Client.call(Client.java:1391)
	... 9 more
], SEVERITY=[IMPORTANT]}, content=IOException in offerService, timestamp=1728850924139}
  • AvroEventWrapper{attributes={ROLE=[hdfs-DATANODE-dec125281080803bb2b9288dd9aff9c2], HOSTS=[namenode2.moe.bigdata.go.th], ROLE_TYPE=[DATANODE], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[hdfs], SERVICE_TYPE=[HDFS], LOG_LEVEL=[WARN], HOST_IDS=[14d8f5e0-ecc4-4d14-bbe1-89fa8cfde10d], SEVERITY=[IMPORTANT]}, content=Problem connecting to server: namenode1.moe.bigdata.go.th/192.168.239.34:8022, timestamp=1728850924602}
  • AvroEventWrapper{attributes={ROLE=[hdfs-DATANODE-f3e08d3125e90218d8495af250c2e2a0], HOSTS=[datanode3.moe.bigdata.go.th], ROLE_TYPE=[DATANODE], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[hdfs], SERVICE_TYPE=[HDFS], LOG_LEVEL=[WARN], HOST_IDS=[ee0391a7-88ed-4fed-ac39-f95e9a63899c], SEVERITY=[IMPORTANT]}, content=Problem connecting to server: namenode1.moe.bigdata.go.th/192.168.239.34:8022, timestamp=1728850924614}
  • AvroEventWrapper{attributes={ROLE=[hdfs-DATANODE-dec125281080803bb2b9288dd9aff9c2], HOSTS=[namenode2.moe.bigdata.go.th], ROLE_TYPE=[DATANODE], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[hdfs], SERVICE_TYPE=[HDFS], LOG_LEVEL=[WARN], HOST_IDS=[14d8f5e0-ecc4-4d14-bbe1-89fa8cfde10d], SEVERITY=[IMPORTANT]}, content=Problem connecting to server: namenode1.moe.bigdata.go.th/192.168.239.34:8022, timestamp=1728850984954}
  • AvroEventWrapper{attributes={ROLE=[hdfs-DATANODE-f3e08d3125e90218d8495af250c2e2a0], HOSTS=[datanode3.moe.bigdata.go.th], ROLE_TYPE=[DATANODE], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[hdfs], SERVICE_TYPE=[HDFS], LOG_LEVEL=[WARN], HOST_IDS=[ee0391a7-88ed-4fed-ac39-f95e9a63899c], SEVERITY=[IMPORTANT]}, content=Problem connecting to server: namenode1.moe.bigdata.go.th/192.168.239.34:8022, timestamp=1728850984960}
  • AvroEventWrapper{attributes={ROLE=[hdfs-DATANODE-12c9e4112cea25acd51602f1019f29c4], HOSTS=[datanode1.moe.bigdata.go.th], ROLE_TYPE=[DATANODE], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[hdfs], SERVICE_TYPE=[HDFS], LOG_LEVEL=[WARN], HOST_IDS=[f8bc3e01-bdae-4428-8858-572d5e23dd92], SEVERITY=[IMPORTANT]}, content=Problem connecting to server: namenode1.moe.bigdata.go.th/192.168.239.34:8022, timestamp=1728850990023}
  • AvroEventWrapper{attributes={ROLE=[hdfs-DATANODE-12c9e4112cea25acd51602f1019f29c4], HOSTS=[datanode1.moe.bigdata.go.th], ROLE_TYPE=[DATANODE], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[hdfs], SERVICE_TYPE=[HDFS], LOG_LEVEL=[WARN], HOST_IDS=[f8bc3e01-bdae-4428-8858-572d5e23dd92], SEVERITY=[IMPORTANT]}, content=Problem connecting to server: namenode1.moe.bigdata.go.th/192.168.239.34:8022, timestamp=1728851042716}
  • AvroEventWrapper{attributes={ROLE=[hdfs-DATANODE-f3e08d3125e90218d8495af250c2e2a0], HOSTS=[datanode3.moe.bigdata.go.th], ROLE_TYPE=[DATANODE], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[hdfs], SERVICE_TYPE=[HDFS], LOG_LEVEL=[WARN], HOST_IDS=[ee0391a7-88ed-4fed-ac39-f95e9a63899c], SEVERITY=[IMPORTANT]}, content=Problem connecting to server: namenode1.moe.bigdata.go.th/192.168.239.34:8022, timestamp=1728851045863}
  • AvroEventWrapper{attributes={ROLE=[hdfs-DATANODE-dec125281080803bb2b9288dd9aff9c2], HOSTS=[namenode2.moe.bigdata.go.th], ROLE_TYPE=[DATANODE], CATEGORY=[LOG_MESSAGE], EVENTCODE=[EV_LOG_EVENT], SERVICE=[hdfs], SERVICE_TYPE=[HDFS], LOG_LEVEL=[WARN], HOST_IDS=[14d8f5e0-ecc4-4d14-bbe1-89fa8cfde10d], SEVERITY=[IMPORTANT]}, content=Problem connecting to server: namenode1.moe.bigdata.go.th/192.168.239.34:8022, timestamp=1728851046060}


EventMetricsPublisher (com.cloudera.cmf.eventcatcher.server.EventMetricsPublisher)

Currently not running.
12340 runs so far (0 slow), of them 0 reported exceptions.
Last duration: 4 ms. Total duration: 93118 ms.
Last start: October 14, 2024 3:23:53 AM +07:00. Last end: October 14, 2024 3:23:53 AM +07:00.

Event Schema Checker (com.cloudera.cmf.eventcatcher.server.EventSchemaChecker)
Num events without event codes: 0
Num events with unknown event codes: 0
Num events not matching schema: 0

Events not matching schema:


    EventIngester (com.cloudera.cmon.pipeline.Pipeline)
    Pipeline stage summary:
    namedroppedskippedforwardedprocessedsize
    Pipeline Stage tagger-writer000687470
    A total of 68747 messages received. Current rate: 0.63 msg/sec. Max rate: 19.09 msg/sec.

    Pipeline Stage tagger-writer (com.cloudera.cmon.pipeline.PipelineStage)
    Input queue size is 0 (of 10000).
    68747 events processed.
    0 events skipped.
    0 events forwarded.
    0 events dropped.
    Current rate: 0.63 msg/sec. Max rate: 19.09 msg/sec.
    Thread pool size is 2 (max 4).
    First element in queue: null


    Events queryer (com.cloudera.cmf.eventcatcher.server.EventsQueryer)
    Seen 73686 queries
    Last query took 0ms
    Last 20 queries:
    • +__persist_timestamp:[1728850816605 TO 1728851452241]
    • +__persist_timestamp:[1728850816605 TO 1728851462242]
    • +__persist_timestamp:[1728850816605 TO 1728851472243]
    • +__persist_timestamp:[1728850865441 TO 1728851482245]
    • +__persist_timestamp:[1728850865441 TO 1728851492247]
    • +__persist_timestamp:[1728850865441 TO 1728851502249]
    • +__persist_timestamp:[1728850865441 TO 1728851512250]
    • +__persist_timestamp:[1728850865441 TO 1728851522251]
    • +__persist_timestamp:[1728850865441 TO 1728851532253]
    • +__persist_timestamp:[1728850924609 TO 1728851542255]
    • +__persist_timestamp:[1728850924609 TO 1728851552257]
    • +__persist_timestamp:[1728850924609 TO 1728851562258]
    • +__persist_timestamp:[1728850924609 TO 1728851572260]
    • +__persist_timestamp:[1728850924609 TO 1728851582261]
    • +__persist_timestamp:[1728850924609 TO 1728851592263]
    • +__persist_timestamp:[1728850990024 TO 1728851602265]
    • +__persist_timestamp:[1728850990024 TO 1728851612266]
    • +__persist_timestamp:[1728850990024 TO 1728851622268]
    • +__persist_timestamp:[1728850990024 TO 1728851632270]
    • +__persist_timestamp:[1728850990024 TO 1728851642271]


    Lucene index manager (com.cloudera.cmf.eventcatcher.server.SingleIndexManager)
    There are 5000013 events in the index with 10368 deleted events waiting to be expunged.
    Last searcher refresh occurred 10.74 secs ago and took 0.000 secs.
    Last commit occurred 54.65 secs ago and took 0.004 secs.
    Last cleanup occurred 201.00 secs ago and took 3.353 secs.


    Event Debug Server (com.cloudera.enterprise.DebugServer)
    Web server is running? true

    AvroEventStoreServer (com.cloudera.cmf.eventcatcher.server.AvroEventStoreServer)
    Avro NettyServer is running on port 7184
    20 max worker threads


    AvroEventStoreHttpService (com.cloudera.cmf.eventcatcher.server.AvroEventStoreHttpService)
    Avro HttpServer is running on port 7185