Distributed Trace Correlator

A distributed trace correlation engine built during work on the Instana observability platform at IBM. The system ingests high-volume telemetry streams — metrics, logs, and distributed traces — and correlates them into unified service dependency graphs. It uses probabilistic data structures (HyperLogLog, Bloom filters) for efficient cardinality estimation and deduplication at scale. The correlator processes millions of spans per second, identifies service-to-service call patterns, and detects anomalous latency distributions using streaming percentile algorithms. Designed as a set of Go microservices deployed on Kubernetes with Kafka as the message backbone.