Filebeat Multiline Kubernetes Container Logs funktioniert nicht

Question

Hallo, ich habe ein Problem, Kubernetes Container zu analysieren multi Linien mit Dateibeat und Logstash. Kubernetes-Protokolldatei befinden sich in/var/log/containers /*.log und in einer json-Linienstruktur.

Ist etwas über meine Konfiguration falsch? Was habe ich vermisst?

filebeat.yml:

filebeat: 
    # List of prospectors to fetch data. 
    prospectors: 
    - 
     paths: 
     - /var/log/containers/*.log 
     fields: {log_type: containers} 
     ignore_older: 5m 
     symlinks: true 
     json.message_key: log 
     json.keys_under_root: true 
     json.add_error_key: true 
     multiline.pattern: '^\d{4}-\d{2}-\d{2}' 
     multiline.match: after 
     multiline.negate: true 
     document_type: kube-logs 
    registry_file: /var/log/containers/filebeat_registry 
output: 
    logstash: 
    hosts: ["logstash-logging:5044"] 

logstash.conf:

input { 
    beats { 
    port => 5044 
    } 
} 

filter { 
    if [type] == "kube-logs" { 

    date { 
     match => ["time", "ISO8601"] 
     remove_field => ["time"] 
    } 
    json { 
     source => "message" 
    } 

    grok { 
     match => [ "log", "<SOME_PATTERN>" ] 
     overwrite => [ "message" ] 
    } 

} 

Kubernetes Behälter:

{"log":"11:11:17,740 |-INFO in ch.qos.logback.core.joran.action.mapWAR - Attaching appender named [FILE-LOG] to Logger[ROOT]\n","stream":"stdout","time":"2017-05-09T11:11:17.742837362Z"} 
{"log":"11:11:17,740 |-INFO in ch.qos.logback.classic.joran.action.mapWAR - End of configuration.\n","stream":"stdout","time":"2017-05-09T11:11:17.742840277Z"} 
{"log":"11:11:17,741 |-INFO in ch.qos.logback.classic.joran.mapWAR - Registering current configuration as safe fallback point\n","stream":"stdout","time":"2017-05-09T11:11:17.742843277Z"} 
{"log":"\n another line","stream":"stdout","time":"2017-05-09T11:11:17.742846485Z"} 
{"log":"09-May-2017 11:11:17.756 INFO [localhost-startStop-1] org.apache.catalina.startup.mapWAR nice","stream":"stderr","time":"2017-05-09T11:11:17.756924376Z"} 
{"log":"09-May-2017 11:11:17.757 INFO [localhost-startStop-1] org.apache.catalina.startup.mapWAR great","stream":"stderr","time":"2017-05-09T11:11:17.757465828Z"} 

Answer 1

Ich glaube, Sie müssen noch zusammen, um die Linie zu setzen, könnten Sie versuchen, Dies? Verwenden Sie {weil das Protokoll mit {nicht Ihrem Zeitstempelformat beginnt.

filebeat.prospectors: 
- paths: 
    - input.json 
    multiline.pattern: '^{' 
    multiline.negate: true 
    multiline.match: after 

processors: 
- decode_json_fields: 
    fields: ['message'] 
    target: json 

output.console.pretty: true