Awesome ClickHouse › ETL and Data Processing
VulcanSQL is an Analytical Data API Framework designed to facilitate the creation of RESTful APIs from databases, data warehouses, or data lakes, specifically targeting AI agents and data applications. It addresses the challenges faced by data professionals in sharing analytical data efficiently and securely with stakeholders for operational business use cases. The framework simplifies the traditionally complex and time-consuming process of custom API development by automating the transformation of SQL queries into APIs, thereby reducing errors and development time. VulcanSQL supports rapid development and integration by abstracting the complexities of direct database interactions, allowing developers to focus on application logic. It promotes standardization through the use of OpenAPI documents, enabling consistent and interoperable API interactions for AI agents. The framework also emphasizes scalability and maintenance, using a template-driven approach that facilitates easy updates and propagation of changes without extensive manual intervention. Performance is enhanced by leveraging DuckDB as a caching layer, which accelerates query execution and reduces API response times. Deployment is flexible, supporting Docker and command-line setups, with tools to package assets for smooth transitions from development to production. VulcanSQL also offers various data sharing options, integrating seamlessly with common workflow applications and AI agents. The framework is suitable for multiple use cases, including enabling AI agents to interact with data sources, providing customer-facing analytics through dashboards and reports, securely sharing data with external partners, and integrating with internal tools like Zapier and Retools. Comprehensive documentation and examples are available to guide users through installation, API building, data source connections, caching, error handling, validation, privacy, extensions, and deployment. Overall, VulcanSQL streamlines the creation and management of data APIs, making analytical data more accessible and actionable for AI-driven applications and business intelligence.
https://github.com/Canner/vulcan-sql
PeerDB is a high-performance, PostgreSQL-optimized ETL tool that enables fast, reliable, and cost-effective streaming of data from Postgres to data warehouses, queues, and storage engines, with native integration in ClickHouse Cloud.
YTsaurus is a scalable, fault-tolerant open-source big data platform featuring MapReduce, SQL engine, NoSQL store, and integration with ClickHouse for fast analytics.
Trench is an open-source, production-ready analytics infrastructure built on ClickHouse and Kafka for scalable, real-time event tracking and analytics with GDPR compliance.
Gluten is a middle layer that offloads JVM-based SQL engines' execution, such as Spark SQL, to high-performance native engines like ClickHouse and Velox, leveraging vectorized processing for accele...
Addax is a versatile and extensible open-source ETL tool that supports seamless data transfer between over 20 SQL and NoSQL data sources, including ClickHouse, with easy configuration and deployment options.
DungBeetle is a distributed job server for asynchronously queuing and executing heavy SQL read jobs on MySQL, PostgreSQL, and ClickHouse databases, designed to offload report generation and improve application performance.
ClickBench is a comprehensive and reproducible benchmark designed to evaluate the performance of analytical databases, including ClickHouse, using realistic workloads derived from real-world web analytics data.
DataCap is an integrated software platform for data transformation, integration, and visualization, supporting a wide range of data sources including ClickHouse and other major databases.