Bash - Suche basierend auf Zeitstempel in Millisekunden

Ich suche alle Hadoop abgeschlossen (100s) Jobs zwischen einem Zeitintervall. Dieses Zeitintervall ist in Millisekunden.Bash - Suche basierend auf Zeitstempel in Millisekunden

Im Anschluss an das Format:

JobId  State   StartTime  UserName   Queue  Priority  UsedContainers RsvdContainers  UsedMem   RsvdMem  NeededMem     AM info 

job_xxxxxxx SUCCEEDED  1458844667431 default default   NORMAL     N/A    N/A   N/A   N/A  N/A    http://xxxxxxxx:8088/proxy/application_xxxxxxxxxx/jobhistory/job/job_xxxxxxxx 
job_xxxxxxx SUCCEEDED  1459449718363 default default   NORMAL     N/A    N/A  N/A    N/A    N/A  http://xxxxxx.xxxxx.com:8088/proxy/application_xxxxxxxxx/jobhistory/job/job_xxxxx

Im Anschluss ist mein Format:

STARTTIME="Tue Apr 12 10:24:29 EDT 2016" 
ENDTIME="Tue Apr 12 15:24:29 EDT 2016" 

CONVERTTIME_1=`date --date="$STARTTIME" +%s%3N` 
CONVERTTIME_2=`date --date="$ENDTIME" +%s%3N` 

echo $CONVERTTIME_1, $CONVERTTIME_2 

mapred job -list all | sed -n '/$CONVERTTIME_1/,/$CONVERTTIME_2/p' > out

Ausgang: alle Aufträge wie oben in diesen timerange.

Kann jemand bitte helfen, wie man diese bekommt?

AUSGABE mapred Job -Liste alle

mapred Job -Liste alle

job_1457613852865_5163 SUCCEEDED  1459199337140 zzzzzzzzzz uuuuuuuu_critical   NORMAL     N/A    N/A  N/A   N/A  N/A  http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_5163/jobhistory/job/job_1457613852865_5163 

job_1457613852865_4633 SUCCEEDED  1458992402216  zzzyyyy yyyyyy_default  NORMAL     N/A    N/A  N/A    N/A    N/A  http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_4633/jobhistory/job/job_1457613852865_4633 

job_1457613852865_4821 SUCCEEDED  1459078845580  zzzyyyy yyyyyy_default  NORMAL     N/A    N/A  N/A    N/A    N/A  http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_4821/jobhistory/job/job_1457613852865_4821 

job_1457613852865_0322 SUCCEEDED  1457717313217  zzzddd uuuuuuuu_critical   NORMAL     N/A    N/A  N/A   N/A  N/A  http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_0322/jobhistory/job/job_1457613852865_0322 

job_1457613852865_5304 SUCCEEDED  1459254375921  zzzyyyy yyyyyy_default  NORMAL     N/A    N/A  N/A    N/A    N/A  http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_5304/jobhistory/job/job_1457613852865_5304 

job_1457613852865_8744 SUCCEEDED  1460195126188  zzzyyyy yyyyyy_default  NORMAL     N/A    N/A  N/A    N/A    N/A  http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_8744/jobhistory/job/job_1457613852865_8744 

job_1457613852865_3384 SUCCEEDED  1458649020794  zzzyyyy yyyyyy_default  NORMAL     N/A    N/A  N/A    N/A    N/A  http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_3384/jobhistory/job/job_1457613852865_3384 

job_1457613852865_9038 SUCCEEDED  1460291694279  zzzyyyy yyyyyy_default  NORMAL     N/A    N/A  N/A    N/A    N/A  http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_9038/jobhistory/job/job_1457613852865_9038 

job_1457613852865_8487 SUCCEEDED  1460115319590  zzzyyyy yyyyyy_default  NORMAL     N/A    N/A  N/A    N/A    N/A  http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_8487/jobhistory/job/job_1457613852865_8487 

job_1457613852865_8321 SUCCEEDED  1460038991587  dddyyy uuuuuuuu_critical   NORMAL     N/A    N/A  N/A   N/A  N/A  http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_8321/jobhistory/job/job_1457613852865_8321 

job_1457613852865_4661 SUCCEEDED  1458994901619  zzzyyyy yyyyyy_default  NORMAL     N/A    N/A  N/A    N/A    N/A  http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_4661/jobhistory/job/job_1457613852865_4661 

job_1457613852865_1975 SUCCEEDED  1458216683800  zzzyyyy yyyyyy_default  NORMAL     N/A    N/A  N/A    N/A    N/A  http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_1975/jobhistory/job/job_1457613852865_1975

habe ich diese:

#!/bin/bash 


STARTTIME="Tue Apr 12 10:13:01 EDT 2016" 
ENDTIME="Tue Apr 12 10:13:59 EDT 2016" 

start=$(date -d "$STARTTIME" '+%s%3N') 
end=$(date -d "$ENDTIME" '+%s%3N') 

echo "start=$start :: end=$end" 

mapred job -list all | awk -v start="$start" -v end="$end" '$3>=start && $3<=end'

Got einen zusätzlichen Job:

job_1457613852865_9785 SUCCEEDED  1460470436726  yyyyyyyyyy nnnnnnnnnn  NORMAL     N/A    N/A  N/A    N/A    N/A  http://888888.xxxxxxxxxx.com:8088/proxy/application_1457613852865_9785/jobhistory/job/job_1457613852865_9785

Quelle

2016-04-13 sumit

Was ist Ihr Suchmuster und was ist die erwartete Ausgabe? ! – anubhava

#/bin/bash # STARTTIME = "12-Apr-2016 10: 00: 00,012" # ENDTIME = "12-Apr-2016 11: 00: 00,012" STARTTIME = "Di 12. April 10: 24:29 EDT 2016" ENDTIME = "Tue 12. April 15.24.29 EDT 2016" CONVERTTIME_1 = 'date --date = "$ STARTTIME" +% s% 3N' CONVERTTIME_2 =' date --date = "$ ENDTIME" +% s% 3N' echo $ CONVERTTIME_1, $ CONVERTTIME_2 mapred job -list alle | sed -n '/ $ CONVERTTIME_1 /,/$ CONVERTTIME_2/p'> aus Ausgabe: alle Jobs wie oben in diesem Zeitraum. – sumit

Aktualisieren Sie die Frage mit diesem Code. – anubhava

Sie können Ihre Daten int konvertieren o Millisekundenwert und dann verwenden Sie awk, um Ihre Daten zu filtern:

STARTTIME="Tue Apr 12 10:24:29 EDT 2016" 
ENDTIME="Tue Apr 12 15:24:29 EDT 2016" 

start=$(date -d "$STARTTIME" '+%s%3N') 
end=$(date -d "$ENDTIME" '+%s%3N')" 

echo "start=$start :: end=$end" 
mapred job -list all | awk -v start="$start" -v end="$end" '$3>=start && $3<=end'

Quelle

2016-04-13 19:20:08 anubhava

Anubhava Können Sie etwas genauer sein: 1. mapred Job -Liste alle -> dieser Befehl die Ausgabe gibt alle Jobs. 2. Ich konvertiere meine Zeit bereits in Millisekunden. 3. Was soll ich mit dem in Schritt 1 definierten MapRed-Befehl ausführen, um die Daten zu filtern? – sumit

Ich möchte den Code hinzufügen, können Sie mir sagen, wie Sie es einrücken? BTW, Ihr Code hat nicht funktioniert – sumit

# ./test.sh 1460471069000, 1460489069000 Katze test.sh #!/bin/bash STARTTIME = "Tue 12. April 10.24.29 EDT 2016" ENDTIME = "Tue 12. April 15.24.29 EDT 2016 " start = $ (date -d "$ STARTTIME" '+% s% 3N') end = $ (date -d "$ ENDTIME" '+% s% 3N') Echo beginnen $, $ Ende mapred job -list alle | awk -v start = "$ start" -v end = "$ end" '$ 1> = starten && $ 1 <= end' – sumit

Bash - Suche basierend auf Zeitstempel in Millisekunden

Antwort

Verwandte Themen