Fehler beim Ausführen des Ziels für das Projekt spark-core_2.11

Question

, Der Vorgang zum Kompilieren des Spark-Codes schlägt auf wenigen Rechnern fehl, während derselbe Quellcode nur wenige andere Maschinen weiterleitet. Bitte überprüfen Sie die Fehler auf Centos (4.10.12-1.el7.elrepo.x86_64) ./build/mvn -X -DskipTests -Dscala.lib.directory =/usr/share/scala -pl Kern kompilieren

INFO] Building Spark Project Core 2.2.2-SNAPSHOT 
[INFO] ------------------------------------------------------------------------ 
[WARNING] The POM for org.apache.spark:spark-launcher_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available 
[WARNING] The POM for org.apache.spark:spark-network-common_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available 
[WARNING] The POM for org.apache.spark:spark-network-shuffle_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available 
[WARNING] The POM for org.apache.spark:spark-unsafe_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available 
[WARNING] The POM for org.apache.spark:spark-tags_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available 
[WARNING] The POM for org.apache.spark:spark-tags_2.11:jar:tests:2.2.2-SNAPSHOT is missing, no dependency information available 
[INFO] ------------------------------------------------------------------------ 
[INFO] BUILD FAILURE 
[INFO] ------------------------------------------------------------------------ 
[INFO] Total time: 0.804 s 
[INFO] Finished at: 2017-12-04T23:18:58-08:00 
[INFO] Final Memory: 43M/1963M 
[INFO] ------------------------------------------------------------------------ 
[ERROR] Failed to execute goal on project spark-core_2.11: Could not resolve dependencies for project org.apache.spark:spark-core_2.11:jar:2.2.2-SNAPSHOT: The following artifacts could not be resolved: org.apache.spark:spark-launcher_2.11:jar:2.2.2-SNAPSHOT, org.apache.spark:spark-network-common_2.11:jar:2.2.2-SNAPSHOT, org.apache.spark:spark-network-shuffle_2.11:jar:2.2.2-SNAPSHOT, org.apache.spark:spark-unsafe_2.11:jar:2.2.2-SNAPSHOT, org.apache.spark:spark-tags_2.11:jar:2.2.2-SNAPSHOT, org.apache.spark:spark-tags_2.11:jar:tests:2.2.2-SNAPSHOT: Failure to find org.apache.spark:spark-launcher_2.11:jar:2.2.2-SNAPSHOT in http://artifact.eng.stellus.in:8081/artifactory/libs-snapshot was cached in the local repository, resolution will not be reattempted until the update interval of snapshots has elapsed or updates are forced -> [Help 1] 

Anmerkung: Die gleichen Quellcode verläuft auf einer anderen Maschine CentOS (3.10.0-514.el7.x86_64) ./build/mvn -DskipTests -Dscala.lib.directory =/usr/share/Scala -pl Kern kompilieren

[INFO] — maven-compiler-plugin:3.7.0:compile (default-compile) @ spark-core_2.11 — 
[INFO] Not compiling main sources 
[INFO] 
[INFO] — scala-maven-plugin:3.2.2:compile (scala-compile-first) @ spark-core_2.11 — 
[INFO] Using zinc server for incremental compilation 
[info] Compile success at Dec 4, 2017 11:17:34 PM [0.331s] 
[INFO] ------------------------------------------------------------------------ 
[INFO] BUILD SUCCESS 
[INFO] ------------------------------------------------------------------------ 
[INFO] Total time: 5.663 s 
[INFO] Finished at: 2017-12-04T23:17:34-08:00 
[INFO] Final Memory: 52M/1297M 
[INFO] ------------------------------------------------------------------------ 

Answer 1

Wenn der Build nur in bestimmten Maschine funktioniert bedeutet dies, dass Sie bereits die Mis Sing Abhängigkeit von Ihrem lokalen Repository .m2. Wenn Sie alle Ihre Verzeichnisse unter .m2 (auf dem Centos-Rechner) entfernen, wird der Build fehlschlagen.

Sie sollten alle fehlenden Abhängigkeiten hinzufügen.

[WARNING] The POM for org.apache.spark:spark-launcher_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available 
[WARNING] The POM for org.apache.spark:spark-network-common_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available 
[WARNING] The POM for org.apache.spark:spark-network-shuffle_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available 
[WARNING] The POM for org.apache.spark:spark-unsafe_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available 
[WARNING] The POM for org.apache.spark:spark-tags_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available 
[WARNING] The POM for org.apache.spark:spark-tags_2.11:jar:tests:2.2.2-SNAPSHOT is missing, no dependency information available