Awesome ClickHouse › ClickHouse and derivatives

timeplus-io/proton

⭐ 2259 C++ repository created 2023-08-14

Timeplus Proton is a high-performance, lightweight streaming SQL engine and database designed for real-time analytics and ETL at scale. It is built in C++ and leverages the powerful and resource-efficient ClickHouse engine to process millions of rows per second from streaming data sources such as Kafka, Pulsar, Iceberg, and ClickHouse itself. Proton supports advanced streaming features including multi-stream JOINs, Change Data Capture (CDC), UPSERT, LOOKUP, and incremental materialized views, enabling sophisticated stream processing and analytics workflows. Unlike traditional stream processors like Apache Flink or ksqlDB, Timeplus Proton combines stream processing with the ability to store queryable analytical and row-based materialized views within the engine. This allows users to perform real-time analytics directly on streaming data with low latency and high throughput. Proton is optimized for performance using SIMD instructions and can run efficiently on minimal hardware resources, including small cloud instances or even local machines without JVM dependencies. The architecture of Proton integrates ClickHouse as a table store engine within each stream, alongside a Write Ahead Log and other data structures, to unify real-time and historical data processing. This design enables Proton to act as an advanced data pre-processor for ClickHouse, preparing and batching data before ingestion to improve overall system efficiency. Proton supports thousands of SQL functions and can query billions of rows in milliseconds, making it suitable for large-scale data environments. Deployment options include a single binary executable, Docker containers, Docker Compose stacks for Kafka/Redpanda integration, and a managed cloud service. The primary interface is SQL, with additional SDKs and REST APIs available for integration with Python, Java, Go, and BI tools. Proton is the core engine behind Timeplus Enterprise and is positioned as a fast, scalable alternative to existing streaming SQL engines, powered by the mature ClickHouse ecosystem. Keywords: streaming SQL engine, real-time analytics, ETL, ClickHouse, Kafka, Pulsar, Iceberg, CDC, UPSERT, LOOKUP, incremental materialized views, stream processing, high performance, lightweight, C++, SIMD, data pre-processor, multi-stream JOINs, Timeplus Proton, open source, Docker, cloud deployment.

https://github.com/timeplus-io/proton

analyticsc++cdcclickhousecloud-deploymentconfluentcppdata-pre-processordockeretlflink-alternativehigh-performanceicebergincremental-materialized-viewskafkakakfaksqldb-alternativelightweightlookupmulti-stream-joinsopen-sourcepulsarreal-time-analyticsredpandasimdsingle-binarysqlstream-processingstreaming-sqlstreaming-sql-enginetimeplus-protonudfupsert