Data Stream
Overview
The Anchore Data Stream provides a mechanism to stream security data from Anchore Enterprise to external systems for further processing, analysis, and long-term storage. As image vulnerability scans and policy evaluations occur within Anchore Enterprise, the data is captured and written to files. These files are monitored by a sidecar service (such as Fluent Bit). The sidecar service reads the data from the files and forwards the events to external destinations like Splunk, Elasticsearch, or other SIEM platforms.
This feature enables you to integrate:
- Real-time Security Monitoring: Stream vulnerability discoveries and policy violations as they occur
- Centralized Log Management: Aggregate Anchore security data with other infrastructure logs
- Custom Dashboards: Build security dashboards in your preferred analytics platform
- Compliance Reporting: Maintain audit trails of security events for compliance requirements
- Alerting Integration: Trigger alerts based on critical vulnerability discoveries or policy failures
Architecture
The data streaming pipeline consists of three components:
Anchore Enterprise (Reports Worker) → Data Event Files → Fluent Bit Sidecar → External Destination
- Reports Worker: Security data to NDJSON (newline-delimited JSON) files
- Data Event Files: Rotating log files stored on a shared volume, with automatic cleanup of processed files
- Fluent Bit: A lightweight log forwarder that tails the data event files and forwards them to your destination
Data Event Types
The following system data events are streamed:
| Data Event Type | Description |
|---|---|
Image Vulnerability Scan Results | Changes to the vulnerability scan results including CVE IDs, severity, fix availability, and affected packages |
Image Policy Evaluation Findings | Changes to the policy evaluation results including pass/fail status, triggered gates, and findings |
Getting Started
To set up the Data Event Stream integration:
- Configure the Data Stream in Anchore Enterprise
- Deploy Fluent Bit as a sidecar container
- Configure your destination (e.g., Splunk)