jitsucom/jitsu
Jitsu is an open-source, self-hosted data ingestion platform that enables real-time event data collection and streaming to data warehouses, including ClickHouse, as an alternative to Segment.
Awesome ClickHouse › Data Transfer and Synchronization
getdozer/dozer
Dozer is a high-performance real-time data movement tool designed to leverage Change Data Capture (CDC) from various data sources and efficiently move data into multiple sinks. It is significantly faster than traditional solutions like Debezium combined with Kafka, making it an ideal choice for scenarios requiring rapid data synchronization and transformation. Dozer supports stateless transformations natively, allowing users to manipulate data on the fly as it moves between systems. The tool is primarily used for transferring data into data warehouses, with a notable use case involving moving data into Clickhouse, a fast analytical database. This capability enables building data APIs and integrating with large language models (LLMs), enhancing data accessibility and usability for advanced analytics and AI applications. Dozer operates through a simple configuration file that defines connections to source databases and target sinks. Supported sources include popular databases and storage systems such as Postgres, MySQL, Snowflake, Kafka, MongoDB, Amazon S3, and Google Cloud Storage. Some connectors, like Oracle and Aerospike, are available only in the enterprise edition. The tool supports resuming data extraction and offers enterprise-grade features for robust data handling. On the sink side, Dozer supports a variety of databases including Clickhouse, Postgres, MySQL, and Big Query, with Oracle and Aerospike sinks available in the enterprise version. This wide range of supported sources and sinks makes Dozer a versatile solution for real-time data integration across diverse environments. Overall, Dozer is a powerful, efficient, and flexible tool for real-time data movement and transformation, particularly suited for modern data architectures that require fast, reliable, and scalable data pipelines involving Clickhouse and other major data platforms.
https://github.com/getdozer/dozer
Also in Data Transfer and Synchronization
zeromicro/cds
Zeromicro/cds is a Go-based tool for real-time data synchronization from MySQL and MongoDB to ClickHouse, supporting both full and incremental sync with a user-friendly web interface.
apecloud/ape-dts
Ape Data Transfer Suite (ape-dts) is a high-performance, lightweight data migration and replication tool supporting ultra-fast transfers between MySQL, PostgreSQL, Redis, MongoDB, Kafka, and ClickHouse, ideal for disaster recovery and migration scenarios.
housepower/clickhouse_sinker
clickhouse_sinker is a sinker program that efficiently transfers Kafka messages into ClickHouse for real-time data analytics and storage.
nikepan/clickhouse-bulk
ClickHouse-Bulk is a tool that collects many small insert requests and sends them as larger bulk inserts to Yandex ClickHouse servers to improve data ingestion efficiency.
uber/storagetapper
StorageTapper is a scalable real-time MySQL change data streaming, logical backup, and replication service supporting multiple data destinations including experimental Clickhouse integration.
long2ice/synch
Synch is a data synchronization tool that enables full and incremental ETL from PostgreSQL and MySQL databases to ClickHouse, supporting real-time data integration and multiple broker options.
Altinity/clickhouse-sink-connector
Altinity Sink Connector is a scalable tool for replicating data from MySQL, PostgreSQL, and MongoDB to ClickHouse for efficient analytical processing.