Awesome ClickHouse › Ops

Altinity/clickhouse-backup

⭐ 1644 Go repository created 2018-09-26

Altinity/clickhouse-backup is a specialized tool designed to facilitate easy backup and restoration of ClickHouse databases. It supports the use of various object storage systems for storing backup files, including cloud providers like AWS, Google Cloud Storage, Azure, Tencent COS, as well as FTP and SFTP. The tool is intended to be run on the same host or Kubernetes pod as the ClickHouse server to ensure access to the necessary data files, as it requires direct access to the ClickHouse data directory for creating backups. It supports backing up entire databases or specific tables, with efficient storage management for multiple backups on the file system. The tool uses streaming compression for uploading and downloading backup data, enhancing performance and reducing storage requirements. Key features include support for the Atomic Database Engine, multi-disk installations, and custom remote storage types through integration with tools like rclone, kopia, restic, and rsync. Incremental backups on remote storage are also supported, allowing for efficient backup management over time. The tool is compatible with ClickHouse versions above 1.1.54394 and primarily supports tables using the MergeTree family of engines, with additional table types supported in newer ClickHouse versions when embedded backup and restore features are enabled. The backup process involves creating hard links to existing data parts using ClickHouse's ALTER TABLE ... FREEZE command, while restoration involves copying these parts to a detached folder and attaching them back to the database. Configuration is flexible, with options to specify remote storage types, concurrency settings for upload and download operations, throttling speeds, and more. The project is maintained by Altinity and has an active community with support channels including Slack and GitHub. It is available as a downloadable binary, Docker image, or can be built from source using Go. Overall, Altinity/clickhouse-backup provides a robust, community-supported solution for managing ClickHouse backups with cloud and local storage integration, designed to ensure data safety and ease of recovery.

https://github.com/Altinity/clickhouse-backup

alter-table-freezealtinityatomic-database-engineawsazblobazurebackupclickhouseclickhouse-backupclickhousedumpcloud-storagedata-safetydockerdumpftpgcsgogoogle-cloud-storageincremental-backupskopiamergetreemulti-diskobject-storagerclonerecoveryresticrestorersyncs3sftptencent-cos