getdozer/dozer
Dozer is a high-performance real-time data movement tool leveraging CDC to transfer and transform data efficiently into multiple sinks including Clickhouse.
Awesome ClickHouse › Data Transfer and Synchronization
jitsucom/jitsu
Jitsu is an open-source, self-hosted data ingestion platform designed as an alternative to Segment. It enables modern data teams to collect event data from websites and applications and stream it in real-time to data warehouses or other services. The platform is fully scriptable and aims to simplify the setup of real-time data pipelines, reducing the time required from days to minutes. Jitsu supports multiple SDKs for sending event data, including HTML snippets, React (with Next.js support), an isomorphic NPM package that works in both client and server-side Node.js environments, HTTP API, and a Segment Proxy. This flexibility allows developers to integrate Jitsu easily into various environments and workflows. Jitsu offers a quick start with Docker Compose for easy installation and provides guidance for production-scale deployments. Additionally, there is a cloud version available at use.jitsu.com, which offers a free tier of up to 200,000 events per month and includes a free ClickHouse instance, highlighting its integration with ClickHouse for data warehousing. The platform is built on top of Bulker, an open-source data warehouse ingestion engine, which can also be used independently by users comfortable with low-level APIs. Jitsu emphasizes open-source collaboration and provides comprehensive documentation, including quick start guides, concept explanations, and a catalog of supported destinations for data streaming. Overall, Jitsu is a powerful tool for teams looking to implement real-time data collection and streaming with the flexibility of self-hosting and the benefits of open-source software, backed by strong community support and extensive integration options.
https://github.com/jitsucom/jitsu
Also in Data Transfer and Synchronization
zeromicro/cds
Zeromicro/cds is a Go-based tool for real-time data synchronization from MySQL and MongoDB to ClickHouse, supporting both full and incremental sync with a user-friendly web interface.
apecloud/ape-dts
Ape Data Transfer Suite (ape-dts) is a high-performance, lightweight data migration and replication tool supporting ultra-fast transfers between MySQL, PostgreSQL, Redis, MongoDB, Kafka, and ClickHouse, ideal for disaster recovery and migration scenarios.
housepower/clickhouse_sinker
clickhouse_sinker is a sinker program that efficiently transfers Kafka messages into ClickHouse for real-time data analytics and storage.
nikepan/clickhouse-bulk
ClickHouse-Bulk is a tool that collects many small insert requests and sends them as larger bulk inserts to Yandex ClickHouse servers to improve data ingestion efficiency.
uber/storagetapper
StorageTapper is a scalable real-time MySQL change data streaming, logical backup, and replication service supporting multiple data destinations including experimental Clickhouse integration.
long2ice/synch
Synch is a data synchronization tool that enables full and incremental ETL from PostgreSQL and MySQL databases to ClickHouse, supporting real-time data integration and multiple broker options.
Altinity/clickhouse-sink-connector
Altinity Sink Connector is a scalable tool for replicating data from MySQL, PostgreSQL, and MongoDB to ClickHouse for efficient analytical processing.