2016-01-28 11 views
7

Versuchen, XML zu JSON in STORM Trident zu konvertieren Mein Code funktionierte gut in Storm 0.9.1 Version und jetzt aktualisierten wir die neueste Version von Storm 0.10.0 und wir erhalten die folgenden Fehler, wenn Sie irgendeine Idee haben die Ursache bitte lass es mich wissen?Jackson werfen Fehler

11752 [Thread-56-b-1] ERROR b.s.util - Async loop died! 
java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.JavaType.isReferenceType()Z 
    at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.findDefaultDeserializer(BasicDeserializerFactory.java:1522) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] 
    at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.findStdDeserializer(BeanDeserializerFactory.java:167) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] 
    at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:132) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] 
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:403) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] 
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:352) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] 
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:264) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] 
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] 
    at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] 
    at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:461) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] 
    at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:3804) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] 
    at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3698) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] 
    at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2714) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] 
    at com.tda.trident.bolt.HdfsStateConvertor$HdfsFileOptions.execute(HdfsStateConvertor.java:278) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] 
    at com.tda.trident.bolt.HdfsStateConvertor.updateState(HdfsStateConvertor.java:462) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] 
    at com.tda.trident.bolt.HdfsUpdater.updateState(HdfsUpdater.java:15) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] 
    at com.tda.trident.bolt.HdfsUpdater.updateState(HdfsUpdater.java:9) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] 
    at storm.trident.planner.processor.PartitionPersistProcessor.finishBatch(PartitionPersistProcessor.java:98) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] 
    at storm.trident.planner.SubtopologyBolt.finishBatch(SubtopologyBolt.java:152) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] 
    at storm.trident.topology.TridentBoltExecutor.finishBatch(TridentBoltExecutor.java:253) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] 
    at storm.trident.topology.TridentBoltExecutor.checkFinish(TridentBoltExecutor.java:286) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] 
    at storm.trident.topology.TridentBoltExecutor.execute(TridentBoltExecutor.java:360) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] 
    at backtype.storm.daemon.executor$fn__6214$tuple_action_fn__6216.invoke(executor.clj:670) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] 
    at backtype.storm.daemon.executor$mk_task_receiver$fn__6137.invoke(executor.clj:426) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] 
    at backtype.storm.disruptor$clojure_handler$reify__5713.onEvent(disruptor.clj:58) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] 
    at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] 
    at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] 
    at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] 
    at backtype.storm.daemon.executor$fn__6214$fn__6227$fn__6278.invoke(executor.clj:808) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] 
    at backtype.storm.util$async_loop$fn__543.invoke(util.clj:475) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] 
    at clojure.lang.AFn.run(AFn.java:22) [clojure-1.6.0.jar:?] 
    at java.lang.Thread.run(Thread.java:744) [?:1.7.0_45] 

Meine maven Schattenausgabe beim Aufbau von POM ist wie unten gezeigt.

[INFO] --- maven-shade-plugin:1.4:shade (default) @ StormTridentXmlNew --- 
[INFO] Including org.apache.hadoop:hadoop-auth:jar:2.7.1.2.3.2.0-2950 in the shaded jar. 
[INFO] Including org.apache.curator:curator-framework:jar:2.7.1 in the shaded jar. 
[INFO] Including org.apache.storm:storm-hdfs:jar:0.10.0.2.3.2.0-2950 in the shaded jar. 
[INFO] Including org.yaml:snakeyaml:jar:1.11 in the shaded jar. 
[INFO] Including com.hmsonline:storm-jms:jar:0.2.2 in the shaded jar. 
[INFO] Including org.apache.geronimo.specs:geronimo-jms_1.1_spec:jar:1.1.1 in the shaded jar. 
[INFO] Including com.tibco.ems:tibjms:jar:5.1 in the shaded jar. 
[INFO] Including org.springframework:spring-beans:jar:4.1.3.RELEASE in the shaded jar. 
[INFO] Including org.springframework:spring-core:jar:4.1.3.RELEASE in the shaded jar. 
[INFO] Including commons-logging:commons-logging:jar:1.2 in the shaded jar. 
[INFO] Including org.springframework:spring-context:jar:4.1.3.RELEASE in the shaded jar. 
[INFO] Including org.springframework:spring-aop:jar:4.1.3.RELEASE in the shaded jar. 
[INFO] Including aopalliance:aopalliance:jar:1.0 in the shaded jar. 
[INFO] Including org.springframework:spring-expression:jar:4.1.3.RELEASE in the shaded jar. 
[INFO] Including org.springframework:spring-jms:jar:4.1.3.RELEASE in the shaded jar. 
[INFO] Including org.springframework:spring-messaging:jar:4.1.3.RELEASE in the shaded jar. 
[INFO] Including org.springframework:spring-tx:jar:4.1.3.RELEASE in the shaded jar. 
[INFO] Including com.fasterxml.jackson.core:jackson-core:jar:2.6.5 in the shaded jar. 
[INFO] Including com.fasterxml.jackson.core:jackson-annotations:jar:2.6.5 in the shaded jar. 
[INFO] Including com.fasterxml.jackson.core:jackson-databind:jar:2.6.5 in the shaded jar. 
[INFO] Including com.fasterxml.jackson.dataformat:jackson-dataformat-xml:jar:2.6.5 in the shaded jar. 
[INFO] Including com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.6.5 in the shaded jar. 
[INFO] Including org.codehaus.woodstox:stax2-api:jar:3.1.4 in the shaded jar. 
[INFO] Including org.codehaus.woodstox:woodstox-core-asl:jar:4.4.1 in the shaded jar. 
[INFO] Including javax.xml.stream:stax-api:jar:1.0-2 in the shaded jar. 
[INFO] Including org.apache.hadoop:hadoop-client:jar:2.7.1 in the shaded jar. 
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.7.1 in the shaded jar. 
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.7.1 in the shaded jar. 
[INFO] Excluding org.apache.hadoop:hadoop-yarn-client:jar:2.7.1 from the shaded jar. 
[INFO] Including org.apache.hadoop:hadoop-yarn-server-common:jar:2.7.1 in the shaded jar. 
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.7.1 in the shaded jar. 
[INFO] Including org.apache.hadoop:hadoop-yarn-api:jar:2.7.1 in the shaded jar. 
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.1 in the shaded jar. 
[INFO] Excluding org.apache.hadoop:hadoop-yarn-common:jar:2.7.1 from the shaded jar. 
[INFO] Including javax.xml.bind:jaxb-api:jar:2.2.2 in the shaded jar. 
[INFO] Including javax.activation:activation:jar:1.1 in the shaded jar. 
[INFO] Including com.sun.jersey:jersey-client:jar:1.9 in the shaded jar. 
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.7.1 in the shaded jar. 
[INFO] Including org.apache.hadoop:hadoop-annotations:jar:2.7.1 in the shaded jar. 
[INFO] Including org.apache.hadoop:hadoop-common:jar:2.7.1 in the shaded jar. 
[INFO] Including com.google.guava:guava:jar:11.0.2 in the shaded jar. 
[INFO] Including commons-cli:commons-cli:jar:1.2 in the shaded jar. 
[INFO] Including org.apache.commons:commons-math3:jar:3.1.1 in the shaded jar. 
[INFO] Including xmlenc:xmlenc:jar:0.52 in the shaded jar. 
[INFO] Including commons-httpclient:commons-httpclient:jar:3.1 in the shaded jar. 
[INFO] Including commons-codec:commons-codec:jar:1.4 in the shaded jar. 
[INFO] Including commons-io:commons-io:jar:2.4 in the shaded jar. 
[INFO] Including commons-net:commons-net:jar:3.1 in the shaded jar. 
[INFO] Including commons-collections:commons-collections:jar:3.2.1 in the shaded jar. 
[INFO] Including javax.servlet:servlet-api:jar:2.5 in the shaded jar. 
[INFO] Including org.mortbay.jetty:jetty:jar:6.1.26 in the shaded jar. 
[INFO] Including org.mortbay.jetty:jetty-util:jar:6.1.26 in the shaded jar. 
[INFO] Including javax.servlet.jsp:jsp-api:jar:2.1 in the shaded jar. 
[INFO] Including com.sun.jersey:jersey-core:jar:1.9 in the shaded jar. 
[INFO] Including com.sun.jersey:jersey-json:jar:1.9 in the shaded jar. 
[INFO] Including org.codehaus.jettison:jettison:jar:1.1 in the shaded jar. 
[INFO] Including com.sun.xml.bind:jaxb-impl:jar:2.2.3-1 in the shaded jar. 
[INFO] Including com.sun.jersey:jersey-server:jar:1.9 in the shaded jar. 
[INFO] Including asm:asm:jar:3.1 in the shaded jar. 
[INFO] Including log4j:log4j:jar:1.2.17 in the shaded jar. 
[INFO] Including net.java.dev.jets3t:jets3t:jar:0.9.0 in the shaded jar. 
[INFO] Including org.apache.httpcomponents:httpclient:jar:4.1.2 in the shaded jar. 
[INFO] Including org.apache.httpcomponents:httpcore:jar:4.1.2 in the shaded jar. 
[INFO] Including com.jamesmurty.utils:java-xmlbuilder:jar:0.4 in the shaded jar. 
[INFO] Including commons-lang:commons-lang:jar:2.6 in the shaded jar. 
[INFO] Including commons-configuration:commons-configuration:jar:1.6 in the shaded jar. 
[INFO] Including commons-digester:commons-digester:jar:1.8 in the shaded jar. 
[INFO] Excluding commons-beanutils:commons-beanutils:jar:1.7.0 from the shaded jar. 
[INFO] Excluding commons-beanutils:commons-beanutils-core:jar:1.8.0 from the shaded jar. 
[INFO] Excluding org.slf4j:slf4j-api:jar:1.7.10 from the shaded jar. 
[INFO] Excluding org.slf4j:slf4j-log4j12:jar:1.7.10 from the shaded jar. 
[INFO] Including org.apache.avro:avro:jar:1.7.4 in the shaded jar. 
[INFO] Including com.thoughtworks.paranamer:paranamer:jar:2.3 in the shaded jar. 
[INFO] Including org.xerial.snappy:snappy-java:jar:1.0.4.1 in the shaded jar. 
[INFO] Including com.google.protobuf:protobuf-java:jar:2.5.0 in the shaded jar. 
[INFO] Including com.google.code.gson:gson:jar:2.2.4 in the shaded jar. 
[INFO] Including com.jcraft:jsch:jar:0.1.42 in the shaded jar. 
[INFO] Including org.apache.curator:curator-client:jar:2.7.1 in the shaded jar. 
[INFO] Including org.apache.curator:curator-recipes:jar:2.7.1 in the shaded jar. 
[INFO] Including com.google.code.findbugs:jsr305:jar:3.0.0 in the shaded jar. 
[INFO] Including org.apache.htrace:htrace-core:jar:3.1.0-incubating in the shaded jar. 
[INFO] Including org.apache.zookeeper:zookeeper:jar:3.4.6 in the shaded jar. 
[INFO] Including io.netty:netty:jar:3.7.0.Final in the shaded jar. 
[INFO] Including org.apache.commons:commons-compress:jar:1.4.1 in the shaded jar. 
[INFO] Including org.tukaani:xz:jar:1.0 in the shaded jar. 
[INFO] Including org.apache.hadoop:hadoop-hdfs:jar:2.7.1 in the shaded jar. 
[INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar. 
[INFO] Including io.netty:netty-all:jar:4.0.23.Final in the shaded jar. 
[INFO] Including xerces:xercesImpl:jar:2.9.1 in the shaded jar. 
[INFO] Including org.fusesource.leveldbjni:leveldbjni-all:jar:1.8 in the shaded jar. 
[INFO] Replacing original artifact with shaded artifact. 
+0

Bitte posten Sie Ihre pom.xml. Ich würde es gerne versuchen. – Bnrdo

Antwort

0

Die neueste Version von storm ist mit jackson-core-2.3.1.jar in seinem lib-Ordner und meine fette Jar, die ich mit Maven erstellen, hat eine andere Version von Jackson Abhängigkeiten. Deshalb, wenn ich mein Glas im Sturmcluster einreiche, wirft es diese No solche Methodenfehler zur Laufzeit.

ich änderte mein POM, um die gleiche Version von Jackson Abhängigkeiten zu benutzen, was Sturm benutzt und alles funktioniert fein.

+0

Das heißt, Sie wollten eigentlich Jackson 2.6 nicht? – Marco99

+0

Ich brauche Xml zu json Marshalling. Jackson hat diese Funktion in der Version, die STORM intern verwendet, die für mich ausreichen wird. – RAJESH

5

Sieht aus wie Sie haben viele Versionen von jackson jars und Ihr Klassenpfad verweist einige Jackson Jar vor 2.6 zur Laufzeit.

Klasse ResolvedType eingeführt isReferenceType() ab Version 2.6. Siehe unten stehenden Link:

http://googleweblight.com/?lite_url=http://fasterxml.github.io/jackson-core/javadoc/2.6/com/fasterxml/jackson/core/type/ResolvedType.html&ei=ochB8Ipc&lc=en-IN&geid=10&s=1&m=71&ts=1454581842&sig=ALL1Aj7I2EAu3AoKmOu6FKbtFTTuKU02Bg#a-isReferenceType()

Bitte stellen Sie sicher, die richtige Version oder eine höhere zur Laufzeit beziehen. Sie können die älteren Jackson-Gläser entfernen/löschen, wenn sie nicht benötigt werden.

Hoffe, das hilft.

0

Dieses Problem scheint von mehreren Versionen von Jackson-Klassen im Klassenpfad zu kommen. Sie können die unerwünschten Versionen entfernen. Zum Zeitpunkt dieses Schreibens. Die aktuelle Version für jackson isis 2.7.1.