DataLens

At a Glance:

DataLens is an open-source business intelligence and data visualization system originally developed at Yandex, deployable via Docker Compose or a Kubernetes Helm chart, and currently supports connecting to ClickHouse and PostgreSQL data sources.

Overview:

DataLens is a business intelligence and data visualization platform designed for building analytical interfaces and exploring data. It was developed and used internally as a primary BI tool at Yandex and is also available as part of the Yandex Cloud platform. The project consists of a Node.js-based user interface, a Python backend for data processing and query generation, a metadata storage service, and an authentication layer. The open-source release focuses on core analytical capabilities, including a local deployment option and native user authentication with role-based access.

Key Decision Points:

• Deployment method: The project supports deployment via Docker Compose for self-hosted environments, with a separate Helm chart option for Kubernetes clusters.

• Data source connectors: The current open-source release includes built-in connectors for ClickHouse, ClickHouse over YTsaurus, and PostgreSQL.

• Authentication model: A native authentication service is included and enabled by default, supporting a role-based system with datalens.viewer, datalens.editor, and datalens.admin roles to control access.

• Project structure: The system is split into distinct services, including a UI proxy (Node.js), a backend query engine (Python), and a metadata storage layer (UnitedStorage), which may influence how self-hosters plan resource allocation and scaling.

Core Features:

• Data visualization and charting: A single-page application UI with a Node.js backend provides interfaces for building charts, with light data post-processing applied on the proxy layer.

• Abstract dataset engine: The backend connects to data sources, generates queries, and performs data processing including formula calculations to produce abstract datasets consumed by the UI.

• Metadata and configuration storage: A dedicated Node.js service called UnitedStorage uses PostgreSQL to persist all DataLens object metadata and configuration.

• User authentication and authorization: A dedicated auth service manages native logins and role-based permissions for datalens.viewer, datalens.editor, and datalens.admin users.

• Workbook export/import workers: A MetaManager service provides workflow workers specifically for exporting and importing workbooks.

Use Cases:

• Data teams and analysts can deploy DataLens locally to build analytical dashboards on top of ClickHouse or PostgreSQL without relying on a cloud-exclusive BI platform.

• System administrators evaluating open-source BI tools can run a proof-of-concept using the Docker Compose deployment to test charting and data modeling on their existing databases.

• Developers or platform engineers can integrate a self-hosted analytical frontend into internal platforms using the Kubernetes Helm chart deployment method.

Open-Source Alternative Value:

DataLens offers a self-hosted alternative for teams that need a BI and visualization layer with native connectors for ClickHouse and PostgreSQL. Its value stems from the availability of its full source code and the ability to deploy the entire stack without mandatory cloud dependencies using Docker or Kubernetes. The inclusion of a built-in authentication layer with distinct user roles and a workbook export/import mechanism provides a practical foundation for internal analytical deployments, though users should note that the open-source version is released with a minimal set of connectors.