Awesome ClickHouse › ETL and Data Processing
GlassFlow for ClickHouse Streaming ETL is an open-source ETL tool designed to facilitate real-time data processing from Kafka to ClickHouse. It offers advanced features such as deduplication with configurable time windows and temporal joins between multiple Kafka topics, enabling efficient and accurate streaming data transformations. The project provides a web-based user interface for easy pipeline management, allowing users to configure and monitor their data pipelines through a convenient graphical interface. Deployment and development are streamlined with Docker-based setups, supporting both production and local development environments. The project includes comprehensive documentation covering installation, usage, pipeline configuration, architecture, and performance benchmarks, making it accessible for users to get started and optimize their data workflows. Additionally, GlassFlow supports community engagement and assistance through GitHub issues, Slack community, and email support. The project is licensed under the Apache License 2.0, promoting open collaboration and usage.
https://github.com/glassflow/clickhouse-etl
PeerDB is a high-performance, PostgreSQL-optimized ETL tool that enables fast, reliable, and cost-effective streaming of data from Postgres to data warehouses, queues, and storage engines, with native integration in ClickHouse Cloud.
YTsaurus is a scalable, fault-tolerant open-source big data platform featuring MapReduce, SQL engine, NoSQL store, and integration with ClickHouse for fast analytics.
Trench is an open-source, production-ready analytics infrastructure built on ClickHouse and Kafka for scalable, real-time event tracking and analytics with GDPR compliance.
Gluten is a middle layer that offloads JVM-based SQL engines' execution, such as Spark SQL, to high-performance native engines like ClickHouse and Velox, leveraging vectorized processing for accele...
Addax is a versatile and extensible open-source ETL tool that supports seamless data transfer between over 20 SQL and NoSQL data sources, including ClickHouse, with easy configuration and deployment options.
DungBeetle is a distributed job server for asynchronously queuing and executing heavy SQL read jobs on MySQL, PostgreSQL, and ClickHouse databases, designed to offload report generation and improve application performance.
ClickBench is a comprehensive and reproducible benchmark designed to evaluate the performance of analytical databases, including ClickHouse, using realistic workloads derived from real-world web analytics data.
DataCap is an integrated software platform for data transformation, integration, and visualization, supporting a wide range of data sources including ClickHouse and other major databases.