Skip to content
Snippets Groups Projects
Commit 8335f2bc authored by Arne Øslebø's avatar Arne Øslebø
Browse files

Update dataingestion.md

parent 71a9b22a
No related branches found
No related tags found
No related merge requests found
# Data ingestion # Data ingestion
SOCTools monitors itself which means that there is already support for receiving and parsing the following data: SOCTools monitors itself which means that there is already support for receiving and parsing the data from the following systems:
* Misp * Misp
* Haproxy * Haproxy
* Kibana * Kibana
...@@ -14,7 +14,7 @@ In addtion there is also support for: ...@@ -14,7 +14,7 @@ In addtion there is also support for:
* Suricata EVE logs * Suricata EVE logs
* Zeek logs * Zeek logs
Additional logs can be sent to the SOCTools server on port 6000 using Filebeat. The typical configuration is: Additional logs of this type can be sent to the SOCTools server on port 6000 using Filebeat. The typical configuration is:
``` ```
filebeat.inputs: filebeat.inputs:
...@@ -30,7 +30,7 @@ output.logstash: ...@@ -30,7 +30,7 @@ output.logstash:
loadbalance: true loadbalance: true
``` ```
The extra filed log_type tells Nifi how it should route the data to the correct parser. The following values are currently supported: The extra field log_type tells Nifi how it should route the data to the correct parser. The following values are currently supported:
* elasticsearch * elasticsearch
* haproxy * haproxy
* keycloak * keycloak
...@@ -68,4 +68,4 @@ Assume you have the following log data: ...@@ -68,4 +68,4 @@ Assume you have the following log data:
} }
``` ```
You want to enrich the client IP so you set the attribute enrich_ip1 to the value "/client/ip". To see more example and to see how logs are parsed, take a look at the process group "Data processing"->"Data input"->"SOCTools" in the NiFi GUI. You want to enrich the client IP so you set the attribute enrich_ip1 to the value "/client/ip". To see more examples and to see how logs are parsed, take a look at the process group "Data processing"->"Data input"->"SOCTools" in the NiFi GUI.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment