1. Zuhause
  2. Frage und Antwort
  3. Hadoop bei "laufender Job" stecken

Hadoop bei "laufender Job" stecken

2016-09-02 6 views
0

Ich möchte das Hadoop-Wortzahl-Programm von der doc ausführen. Allerdings steckte das Programm bei running jobHadoop bei "laufender Job" stecken

16/09/02 10:51:13 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 
16/09/02 10:51:13 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 
16/09/02 10:51:13 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 
16/09/02 10:51:14 INFO input.FileInputFormat: Total input paths to process : 1 
16/09/02 10:51:14 INFO mapreduce.JobSubmitter: number of splits:2 
16/09/02 10:51:14 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1472783047951_0003 
16/09/02 10:51:14 INFO impl.YarnClientImpl: Submitted application application_1472783047951_0003 
16/09/02 10:51:14 INFO mapreduce.Job: The url to track the job: http://hadoop-master:8088/proxy/application_1472783047951_0003/ 
16/09/02 10:51:14 INFO mapreduce.Job: Running job: job_1472783047951_0003

Und http://hadoop-master:8088/proxy/application_1472783047951_0003/ enter image description here

zeigen und es läuft eine AppMaster auf http://hadoop-slave2:8042, zeigen Sie sich enter image description here

Da es jedoch stuck auf Wordcount, auch sie angesteckt Hive

hive (default)> select a, b, count(1) as cnt from newtb group by a, b; 
Query ID = hadoop_20160902110124_d2b2680b-c493-4986-aa84-f65794bfd8e4 
Total jobs = 1 
Launching Job 1 out of 1 
Number of reduce tasks not specified. Estimated from input data size: 1 
In order to change the average load for a reducer (in bytes): 
    set hive.exec.reducers.bytes.per.reducer=<number> 
In order to limit the maximum number of reducers: 
    set hive.exec.reducers.max=<number> 
In order to set a constant number of reducers: 
    set mapreduce.job.reduces=<number> 
Starting Job = job_1472783047951_0004, Tracking URL = http://hadoop-master:8088/proxy/application_1472783047951_0004/ 
Kill Command = /opt/hadoop-2.6.4/bin/hadoop job -kill job_1472783047951_0004

Das ist nichts falsch mit Select *.

hive (default)> select * from newtb; 
OK 
1 2 3 
1 3 4 
2 3 4 
5 6 7 
8 9 0 
1 8 3 
Time taken: 0.101 seconds, Fetched: 6 row(s)

Also, ich denke, da ist etwas falsch mit MapReduce. Es gibt genug Speicherplatz und Speicher. So, wie man es löst?

Quelle

2016-09-02 Alexander Yau

Antwort

Verwandte Themen

 Verwandte Themen