Telemetry Pipeline
A telemetry pipeline collects, filters, and routes logs and events before they are analysed in the SIEM or SOC.
A telemetry pipeline collects, processes, and distributes security data before it reaches the SIEM or other systems. It controls what data flows where and in what format.
How it works
A pipeline ingests data from multiple log sources. It normalises formats, enriches information, and filters out unnecessary events. It then routes the data to one or more destinations, such as the SIEM, a low-cost archive, or a data lake. This reduces costs and improves data quality. Common tools for this are Cribl, Logstash, or Vector.
For example, a firewall might generate 200 GB of logs daily. Much of this data relates to permitted connections without detection value. The pipeline routes these entries directly to an archive. Only security-relevant events are sent to the SIEM. This reduces licence costs and makes searches faster.
What to look out for
- Filter with care. What seems unimportant today may be missing during an investigation.
- Store raw data in an archive to have it available when required.
- Document every rule in the pipeline. This lets analysts know what data is excluded.
- Monitor the pipeline itself. An outage can stop all data flow.
- Protect the pipeline from manipulation. An attacker in control can erase their tracks.
Benefits
A pipeline makes data storage less dependent on a single SIEM vendor. Data can be distributed to multiple targets for security and operations. A later SIEM migration becomes simpler because the source connections remain.
Switzerland and regulation
The pipeline can mask or remove personal data before forwarding it. This helps implement data minimisation principles of the revFADP (revised Federal Act on Data Protection). It also allows control over keeping certain data within Switzerland.
Typical mistakes
A common mistake is filtering too aggressively to save costs. During an investigation, the very events showing the incident's progression are missing.
How we implement it
We connect your data sources to the SOC using tailored pipelines. Data is stored in Switzerland or the EU.