jitsucom/jitsu
Jitsu is an open-source, self-hosted data ingestion platform that enables real-time event data collection and streaming to data warehouses, including ClickHouse, as an alternative to Segment.
Awesome ClickHouse › Data Transfer and Synchronization
Percona-Lab/clickhousedb_fdw
The project "clickhousedb_fdw" by Percona-Lab is an open-source Foreign Data Wrapper (FDW) designed to integrate PostgreSQL with ClickHouse, one of the fastest column-store databases. This FDW enables users to perform SELECT and INSERT operations on a ClickHouse database directly from within a PostgreSQL server environment. It supports advanced features such as aggregate pushdown and join pushdown, which significantly enhance query performance by leveraging the computational resources of the remote ClickHouse server for these resource-intensive operations. The project supports multiple PostgreSQL versions, specifically versions 11, 12, and 13, including the Percona Distribution of PostgreSQL. Installation can be done from source or via package managers like yum and apt-get. After installation, users can create the extension in PostgreSQL and configure a foreign server connection to ClickHouse, allowing seamless querying and data manipulation across the two database systems. The FDW facilitates the creation of foreign tables in PostgreSQL that map to tables in ClickHouse, enabling SQL queries on ClickHouse data as if it were local to PostgreSQL. This integration is particularly useful for users who want to combine the analytical power and speed of ClickHouse with the robustness and extensibility of PostgreSQL. The project includes comprehensive documentation covering installation, setup, user guides, release notes, and bug reporting procedures. It is actively maintained, with a focus on improving compatibility and performance. The source code and releases are available on GitHub, and the project encourages community contributions and feedback through its issue tracker. Overall, clickhousedb_fdw is a valuable tool for database administrators and developers looking to harness the strengths of both PostgreSQL and ClickHouse in a unified environment, enabling efficient data analysis and management workflows.
https://github.com/Percona-Lab/clickhousedb_fdw
Also in Data Transfer and Synchronization
getdozer/dozer
Dozer is a high-performance real-time data movement tool leveraging CDC to transfer and transform data efficiently into multiple sinks including Clickhouse.
zeromicro/cds
Zeromicro/cds is a Go-based tool for real-time data synchronization from MySQL and MongoDB to ClickHouse, supporting both full and incremental sync with a user-friendly web interface.
apecloud/ape-dts
Ape Data Transfer Suite (ape-dts) is a high-performance, lightweight data migration and replication tool supporting ultra-fast transfers between MySQL, PostgreSQL, Redis, MongoDB, Kafka, and ClickHouse, ideal for disaster recovery and migration scenarios.
housepower/clickhouse_sinker
clickhouse_sinker is a sinker program that efficiently transfers Kafka messages into ClickHouse for real-time data analytics and storage.
nikepan/clickhouse-bulk
ClickHouse-Bulk is a tool that collects many small insert requests and sends them as larger bulk inserts to Yandex ClickHouse servers to improve data ingestion efficiency.
uber/storagetapper
StorageTapper is a scalable real-time MySQL change data streaming, logical backup, and replication service supporting multiple data destinations including experimental Clickhouse integration.
long2ice/synch
Synch is a data synchronization tool that enables full and incremental ETL from PostgreSQL and MySQL databases to ClickHouse, supporting real-time data integration and multiple broker options.