Exception in thread "main" java.lang.NoClassDefFoundError: com/typsichere/config/ConfigFactory

Question

Ich erhalte die obige Fehlermeldung, wenn ich versuche, Funken einreichen Kommentar laufen:

spark-submit --class "retail.DataValidator" --master local --executor-memory 2g --total-executor-cores 2 sample-spark-180417_2.11-1.0.jar /home/hduser/Downloads/inputfiles/ /home/hduser/output/ 

Fehlermeldung :

Exception in thread "main" java.lang.NoClassDefFoundError: com/typesafe/config/ConfigFactory 
at retail.DataValidator$.main(DataValidator.scala:12) 
at retail.DataValidator.main(DataValidator.scala) 
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
at java.lang.reflect.Method.invoke(Method.java:498) 
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738) 
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187) 
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212) 
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126) 
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) 
Caused by: java.lang.ClassNotFoundException: com.typesafe.config.ConfigFactory 
at java.net.URLClassLoader.findClass(URLClassLoader.java:381) 
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) 
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) 
... 11 more 

build.sbt Datei:

name := "sample-spark-180417" 
version := "1.0" 
scalaVersion := "2.11.8" 

libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.1.0" 
libraryDependencies += "com.typesafe" % "config" % "1.3.1" 
libraryDependencies += "org.apache.spark" % "spark-sql_2.11" % "2.1.0" 
libraryDependencies += "org.apache.spark" % "spark-hive_2.11" % "2.1.0" 
libraryDependencies += "mysql" % "mysql-connector-java" % "5.1.42" 
libraryDependencies += "org.scala-lang" % "scala-swing" % "2.10+" 

ich keine maven Abhängigkeiten oder pom.xml-Datei nicht mit.

Dank

Answer 1

Da es kein Fett Glas, wird der Funke Cluster typsichere Glas in seiner Classpath nicht mit.

Senden der Funke Job als ::

spark-submit --jars ./typesafe-***.jar --class "retail.DataValidator" --master local --executor-memory 2g --total-executor-cores 2 sample-spark-180417_2.11-1.0.jar /home/hduser/Downloads/inputfiles/ /home/hduser/output/ 

es, dass Glas in Classpath halten, und die Arbeit einreichen.