SlateDB

Overview:

SlateDB is an embedded storage engine built as a log-structured merge-tree (LSM-tree) that writes data directly to object storage services like S3, GCS, ABS, MinIO, and Tigris. It addresses the need for a database engine that provides bottomless storage capacity, high durability, and straightforward replication by using cloud-native object storage as its primary storage layer. SlateDB is designed for developers building Rust-based applications that require an embedded key-value store capable of handling large, durable datasets without managing local disk limits.

Core Features:

• Object storage-backed SSTs: Writes data as string-sorted tables (SSTs) flushed to object storage, enabling infinite capacity and high durability.

• Configurable write batching: Buffers writes in MemTables and flushes them periodically to object storage, reducing PUT API costs.

• Read performance optimizations: Includes in-memory block caches, local SST disk caches, compression, and bloom filters to reduce read latency and GET API costs.

• Basic key-value API: Supports get, put, and delete operations, with an option to set await_durable to false on put for lower latency at the cost of durability.

• Transactions: Supports transactions for atomic multi-key operations.

• Change data capture (CDC): Provides a built-in CDC mechanism to capture and stream data changes.

Use Cases:

• Developers building embedded Rust applications that need a durable key-value store without managing local disk allocation.

• Data-intensive systems that require replication and durability across cloud environments via object storage.

• Backend services that benefit from low write API costs by batching writes to cloud storage at configurable intervals.

Why It Matters:

SlateDB offers an open-source, embedded storage engine that leverages cloud object storage to provide unlimited capacity and inherent replication, a design that differs from disk-based LSM-trees. Developers gain durability and bottomless storage from the start, with read and write performance optimizations to mitigate object storage latency and costs. Its official bindings for Go, Java, Python, and Node.js/TypeScript also make it accessible beyond Rust-based projects.