0
Ich habe versucht, unter Verwendung von Rinne einen Json-Eintrag in Bienenstock einzufügen, baute ich einen Tisch. Das Waschbecken ist wie folgt konfiguriert -Flume Json Hive Waschbecken
agent1.sinks.sk-json.type = hive
agent1.sinks.sk-json.hive.metastore = dt-s39.campus.ad.bgu.ac.il:3306
agent1.sinks.sk-json.hive.database = blackhole
agent1.sinks.sk-json.hive.partition = sort
agent1.sinks.sk-json.hive.table = blackholetable
agent1.sinks.sk-json.useLocalTimeStamp = true
agent1.sinks.sk-json.serializer = JSON
agent1.sinks.sk-json.channel = c-json
ich die folgende Störung erhalte:
fehlgeschlagen Agent gestartet werden, da Abhängigkeiten nicht in Classpath gefunden wurden. Fehler folgt.
java.lang.NoClassDefFoundError: org/apache/hive/hcatalog/streaming/RecordWriter
at org.apache.flume.sink.hive.HiveSink.createSerializer(HiveSink.java:223)
at org.apache.flume.sink.hive.HiveSink.configure(HiveSink.java:203)
at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
at org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:413)
at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:98)
at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.apache.hive.hcatalog.streaming.RecordWriter
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 13 more
Sollte ich den Serializer HiveJsonSerializer verwenden? https://flume.apache.org/releases/content/1.7.0/apidocs/org/apache/flume/sink/hive/HiveJsonSerializer.html
war ich die JSONSerDe.java https://github.com/cloudera/cdh-twitter-example/tree/master/hive-serdes/src/main/java/com/cloudera/hive/serde
Ich bin ein wenig verloren verwenden soll, hat jemand Erfahrung mit dem und eine Führung geben kann mir?