Unable Jobs einreichen Cluster (Cluster-Modus)

Question

Spark-Version 1.3.0

Fehler zu entfachen, während Jobs einreichen Cluster in Cluster

./spark-submit --class org.apache.spark.examples.streaming.JavaDirectKafkaWordCount --deploy-mode cluster wordcount-0.1.jar 172.20.5.174:9092,172.20.9.50:9092,172.20.7.135:9092 log 

Renditen Modus zu wecken:

Spark assembly has been built with Hive, including Datanucleus jars on classpath 
Running Spark using the REST application submission protocol. 
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties 
15/04/14 16:41:10 INFO StandaloneRestClient: Submitting a request to launch an application in spark://172.20.9.151:7077. 
Warning: Master endpoint spark://172.20.9.151:7077 was not a REST server. Falling back to legacy submission gateway instead. 
15/04/14 16:41:11 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 
Sending launch command to spark://172.20.9.151:7077 
Error connecting to master spark://172.20.9.151:7077 (akka.tcp://sparkMaster@172.20.9.151:7077), exiting. 

Answer 1

Die Master-Spark-REST-URL befindet sich standardmäßig auf Port 6066. Sie sollten dies also als Master-Endpunkt sehen: spark: //172.20.9.151: 6066.

Wenn Sie die Spark-Webkonsole (http://master:8080) aufrufen, erhalten Sie die Details der verschiedenen Endpunkte Ihres Clusters.