At a Glance:
Gigapipe is a lightweight, polyglot observability stack providing drop-in compatible ingestion and querying for logs, metrics, and traces through native Loki, Prometheus, Tempo, and Pyroscope APIs, with data stored in ClickHouse, DuckDB, or S3-backed object storage.
Overview:
Gigapipe is an all-in-one, open-source observability platform for ingesting and querying logs, metrics, traces, and profiling data. It independently implements the Loki, Prometheus, Tempo, and Pyroscope APIs, allowing users to interact with their data using standard query languages like LogQL, PromQL, and TraceQL without requiring any plugins or middleware. The project is designed to be compatible with existing Grafana datasources and also ships with a built-in, zero-dependency explorer. It supports multiple storage backends, including ClickHouse and DuckDB with S3 object storage, giving operators control over their data infrastructure.
Key Decision Points:
Native API Compatibility: The project implements the APIs for Loki, Prometheus, Tempo, and Pyroscope, enabling the use of existing compatible clients and Grafana datasources without custom plugins.
Polyglot Ingestion: Data can be ingested through OpenTelemetry or using native protocols from Grafana, InfluxDB, DataDog, and Elastic, querying it later with any available API.
Storage Backends: Data can be stored using ClickHouse, DuckDB, or an S3-compatible object store through GigAPI, offering flexibility in infrastructure choices.
Built-in Data Explorer: A lightweight, zero-dependency explorer called "view" is included for inspecting logs, metrics, and traces, removing the immediate need for Grafana.
Query Language Support: Users can leverage familiar languages like LogQL for logs, PromQL for metrics, and TraceQL for traces, aligning with existing operator skills.
Core Features:
Multi-API Ingestion: Ingest data using native protocols for Loki, Prometheus, Tempo, Pyroscope, as well as formats from InfluxDB, DataDog, and Elastic.
OpenTelemetry Integration: Officially integrated to consume any log, trace, or metric format from OpenTelemetry instrumentation.
Standard Query APIs: Independently implements the Loki, Prometheus, Tempo, and Pyroscope query APIs for transparent client compatibility.
Built-in Explorer (View): Ships with a zero-dependency, lightweight UI for browsing and exploring log, metric, and trace data.
Multiple Storage Targets: Supports using ClickHouse, DuckDB, and S3 object storage as backend data stores.
Continuous Profiling Support: Implements the Pyroscope API for compatibility with Pyroscope SDK clients and agents.
Use Cases:
Developers can aggregate telemetry from distributed systems using OpenTelemetry and query it with standard tools like Grafana, using its native Loki and Prometheus datasources.
Operators can set up a centralized observability backend that accepts data from multiple vendor formats and stores it in their own S3-backed storage.
Platform engineers can implement a polyglot observability stack that allows querying the same ingested data as logs, metrics, or traces using the appropriate API.
Teams can explore traces with TraceQL and visualize service graphs by connecting Gigapipe to Grafana's Tempo datasource.
Open-Source Alternative Value:
Gigapipe is an open-source project designed to reduce dependency on vendor-controlled observability stacks by independently implementing common APIs and query languages for logs, metrics, and traces. Its transparent, drop-in compatibility with Loki, Prometheus, Tempo, and Pyroscope allows users to operate a unified backend using standard clients and tools. The flexibility to store data in ClickHouse, DuckDB, or S3 gives operators direct control over their data layer, supporting deployment choices that align with existing infrastructure rather than a vendor's managed service requirements.
