Parse/Split geschachtelte Single JSON-Array in Logstash

Question

Ich bin auf der Suche nach einem Splitting/Filter für die folgenden JSON-Array. Wir brauchen jeden Wert im Array als einen einzelnen Wert in elastice.

{ "Mot_Temp_Test": { "INT": [ "0", "0", "0", "0", "0", "0", "0", "0", " 0" , "0", "0", "0"]}}

Answer 1

(Dies sind Ergebnisse von Tests, die ich unter Verwendung lief logstash 2.4, wird die Ausgabe rubydebug Codec)

von codec => "json" in Ihrer Eingabe mit Logstash wird Ihr Array tatsächlich als Array sehen. Ich habe deine Ints nummeriert, um sie auseinander zu halten.

{ 
    "Mot_Temp_Test" => { 
     "INT" => [ 
      [ 0] "0", 
      [ 1] "1", 
      [ 2] "2", 
      [ 3] "3", 
      [ 4] "4", 
      [ 5] "5", 
      [ 6] "6", 
      [ 7] "7", 
      [ 8] "8", 
      [ 9] "9", 
      [10] "10", 
      [11] "11" 
     ] 
    }, 
     "@version" => "1", 
     "@timestamp" => "2017-01-20T16:55:42.606Z", 
      "host" => "b5963373fadd" 
} 

Logstash ist nicht gut im Umgang mit Arrays, aber es kann auf sie zugreifen. So können wir eine mutate filter verwenden, um ein Array-Element in ein Feld umzubenennen.

filter { 
    mutate { rename => { "[Mot_Temp_Test][INT][0]" => "int0" } } 
} 

uns gibt:

{ 
    "Mot_Temp_Test" => { 
     "INT" => [ 
      [ 0] "0", 
      [ 1] "0", 
      [ 2] "0", 
      [ 3] "0", 
      [ 4] "0", 
      [ 5] "0", 
      [ 6] "0", 
      [ 7] "0", 
      [ 8] "0", 
      [ 9] "0", 
      [10] "0" 
     ] 
    }, 
     "@version" => "1", 
     "@timestamp" => "2017-01-20T17:08:00.728Z", 
      "host" => "5780e869e09f", 
      "int0" => "0" 
} 

Okay, so sollte dies einfach sein. . .

filter { 
    mutate { 
     rename => { "[Mot_Temp_Test][INT][0]" => "int0" } 
     rename => { "[Mot_Temp_Test][INT][1]" => "int1" } 
     rename => { "[Mot_Temp_Test][INT][2]" => "int2" } 
     rename => { "[Mot_Temp_Test][INT][3]" => "int3" } 
     rename => { "[Mot_Temp_Test][INT][4]" => "int4" } 
     rename => { "[Mot_Temp_Test][INT][5]" => "int5" } 
     rename => { "[Mot_Temp_Test][INT][6]" => "int6" } 
    } 
} 

Aber warten, werden diese Operationen nacheinander behandelt und nach etwas entfernt wird, die Array-Ausfachungen und wir erhalten:

{ 
    "Mot_Temp_Test" => { 
     "INT" => [ 
      [0] "1", 
      [1] "3", 
      [2] "5", 
      [3] "7", 
      [4] "9", 
      [5] "11" 
     ] 
    }, 
     "@version" => "1", 
     "@timestamp" => "2017-01-20T18:48:31.875Z", 
      "host" => "a802749c44fe", 
      "int0" => "0", 
      "int1" => "2", 
      "int2" => "4", 
      "int3" => "6", 
      "int4" => "8", 
      "int5" => "10" 
} 

Der Versuch, dies zu erklären:

filter { 
    mutate { 
     rename => { "[Mot_Temp_Test][INT][0]" => "int0" } 
     rename => { "[Mot_Temp_Test][INT][0]" => "int1" } 
     rename => { "[Mot_Temp_Test][INT][0]" => "int2" } 
     rename => { "[Mot_Temp_Test][INT][0]" => "int3" } 
     rename => { "[Mot_Temp_Test][INT][0]" => "int4" } 
     rename => { "[Mot_Temp_Test][INT][0]" => "int5" } 
     rename => { "[Mot_Temp_Test][INT][0]" => "int6" } 
    } 
} 

Funktioniert nicht genau:

{ 
                  "Mot_Temp_Test" => { 
     "INT" => [ 
      [ 0] "1", 
      [ 1] "2", 
      [ 2] "3", 
      [ 3] "4", 
      [ 4] "5", 
      [ 5] "6", 
      [ 6] "7", 
      [ 7] "8", 
      [ 8] "9", 
      [ 9] "10", 
      [10] "11" 
     ] 
    }, 
                   "@version" => "1", 
                   "@timestamp" => "2017-01-20T18:56:32.608Z", 
                    "host" => "d5b81003f43b", 
    "\"int0\", \"int1\", \"int2\", \"int3\", \"int4\", \"int5\", \"int6\"" => "0" 
} 

Um dies zu erhalten wir arbeiten müssen, eine Reihe von verschiedenen mutieren Filter verwenden:

filter { 
    mutate { rename => { "[Mot_Temp_Test][INT][0]" => "int0" } } 
    mutate { rename => { "[Mot_Temp_Test][INT][0]" => "int1" } } 
    mutate { rename => { "[Mot_Temp_Test][INT][0]" => "int2" } } 
    mutate { rename => { "[Mot_Temp_Test][INT][0]" => "int3" } } 
    mutate { rename => { "[Mot_Temp_Test][INT][0]" => "int4" } } 
    mutate { rename => { "[Mot_Temp_Test][INT][0]" => "int5" } } 
    mutate { rename => { "[Mot_Temp_Test][INT][0]" => "int6" } } 
} 

und Erfolg:

{ 
    "Mot_Temp_Test" => { 
     "INT" => [ 
      [0] "7", 
      [1] "8", 
      [2] "9", 
      [3] "10", 
      [4] "11" 
     ] 
    }, 
     "@version" => "1", 
     "@timestamp" => "2017-01-20T18:21:06.488Z", 
      "host" => "882832d1dd43", 
      "int0" => "0", 
      "int1" => "1", 
      "int2" => "2", 
      "int3" => "3", 
      "int4" => "4", 
      "int5" => "5", 
      "int6" => "6" 
} 

Alles in allem Arrays ist etwas, das logstash nicht gut ist.