Dragonfly

Overview:

Dragonfly is an in-memory data store built for modern application workloads. It is designed as a drop-in replacement for Redis and Memcached, offering full API compatibility without requiring code changes. The project targets developers and teams seeking higher throughput, lower tail latency, and reduced resource consumption compared to legacy in-memory data stores. Dragonfly uses a shared-nothing architecture and novel data structures to scale vertically on modern cloud hardware. It supports both Redis and Memcached protocols, making it suitable for caching, session storage, real-time analytics, and other latency-sensitive data access patterns.

Core Features:

• Redis & Memcached API compatibility: Supports approximately 185 Redis commands and all Memcached commands except cas, enabling direct replacement without code changes.

• Shared-nothing architecture: Partitions keyspace across threads, allowing each thread to manage its own shard for better CPU and memory utilization.

• Novel cache eviction algorithm: When --cache_mode=true is enabled, Dragonfly evicts items least likely to be used in the future near the maxmemory limit, with zero memory overhead for caching metadata.

• Fork-less snapshotting: Performs database snapshots without forking, reducing memory spikes compared to traditional bgsave operations.

• Native HTTP console & Prometheus metrics: Exposes a built-in HTTP interface on the main TCP port, including Prometheus-compatible metrics at :6379/metrics for monitoring.

• Configurable cluster mode: Supports an emulated cluster mode for multi-node setups, configurable via --cluster_mode=emulated.

Use Cases:

• High-throughput caching: Developers can replace Redis or Memcached in caching layers to handle higher QPS (up to 3.8M QPS on large instances) while maintaining sub-millisecond P99 latency.

• Real-time data serving: Suitable for applications needing low-latency access to session state, leaderboards, rate limit counters, or real-time queues.

• Memory-constrained deployments: Teams running on limited resources can benefit from Dragonfly's 30% better memory efficiency compared to Redis in idle state and no visible memory increase during snapshots.

• Multi-protocol environments: System administrators can use Dragonfly as a unified data store for both Redis and Memcached clients, simplifying infrastructure.

Why It Matters:

As an open-source drop-in replacement for Redis and Memcached, Dragonfly offers a high-performance alternative without requiring changes to existing application code. Its shared-nothing architecture and Dash-based hashtable design exploit modern multi-core hardware more efficiently than single-threaded Redis. The project's focus on vertical scalability, memory efficiency, and reduced resource usage makes it a practical choice for teams looking to improve throughput and latency without scaling out infrastructure. Its community-driven development and compatibility with existing tooling (like Prometheus and Grafana) further reduce adoption friction.