Introduction

In modern data-intensive applications, ensuring database high availability and data integrity is a critical need for every organization. Whether dealing with unexpected hardware failures, software bugs, or natural disasters, a solid backup and recovery strategy is essential to maintaining business continuity. Whaleal Platform (WAP) is designed to meet this demand, offering a robust and flexible data protection solution for MongoDB databases.

Whaleal Platform’s backup strategy combines full backups with incremental backups, ensuring users can quickly restore data to any point in time while optimizing storage and management through flexible strategy settings.

1. Full Backup and Recovery Strategy

Full backup (Snapshot Backup) is a method of taking a snapshot of the entire database at a specific point in time and saving it. It allows for the rapid restoration of the entire database in the event of disaster recovery or large-scale data loss.

1.1 Full Backup Process

Whaleal Platform performs full backups by periodically saving the entire MongoDB database to external storage (e.g., S3). The backup process includes:

• Locking the node: Before performing a full backup, the database node is locked to ensure data consistency during the backup process.
• Compressing data: The backup data is compressed to save storage space.
• Copying to S3 storage: The compressed backup files are copied to external storage systems like S3 for long-term storage and future recovery.

1.2 Full Recovery Process

In the case of severe data loss or database crashes, full recovery is the key to quickly restoring the system. The recovery steps include:

• Downloading the backup data from external storage (e.g., S3).
• Using mongorestore to restore the backup data to the MongoDB database.
• Completely restoring the database to its state at the time of the backup, ensuring data consistency.

1.3 Advantages and Use Cases of Full Backup

• Advantages:
  • High data integrity: Each backup contains all data from the entire database, ensuring consistency during recovery.
  • Fast recovery: Full backups allow for quick restoration when the entire database needs to be restored.
• Use Cases:
  • Large-scale disaster recovery: In cases such as hard drive failure or system disasters, full backups allow for fast restoration of the entire database.
  • Periodic backups: Suitable for scenarios that require regular, comprehensive protection, ensuring complete data availability at specific points in time.

2. Incremental Backup and Recovery Strategy

Incremental backup refers to backing up only the data that has changed since the last backup. Whaleal Platform efficiently records database changes through incremental backups and can restore the database to any point in time as needed.

2.1 Incremental Backup Process

Incremental backup is based on MongoDB’s Oplog (operation log), which records every change in the database. The process is as follows:

• Continuously recording data changes: The Oplog records all database operations (such as inserts, updates, deletions), and these logs can be backed up to separate storage.
• Incremental data storage: Oplog data can be saved periodically and backed up according to user-defined strategies (e.g., daily, hourly).
• Saving storage space: Incremental backups only record changes, significantly reducing the size of backup files.

2.2 Incremental Recovery Process

Incremental recovery involves applying changes recorded in the Oplog after a full recovery. The steps include:

• First, restore the database to a certain point using the full backup.
• Then, apply Oplog data from incremental backups to restore the database to its latest state or a specific point in time.

This method allows for precise recovery of each operation, ensuring data completeness and consistency.

2.3 Advantages and Use Cases of Incremental Backup

• Advantages:
  • Saving storage space: Incremental backups only record changes, significantly reducing storage needs compared to full backups.
  • Flexible recovery: Allows for restoration to any specific point in time, not just the time of the full backup.
• Use Cases:
  • Frequently updated data systems: For applications with frequent data changes, incremental backups can more effectively capture data changes.
  • Precise time-point recovery: When users need to restore to a specific point in time, incremental backups provide flexible recovery capabilities.

3. Combining Both Strategies

The combination of full and incremental backups is the core of Whaleal Platform’s backup and recovery mechanism. By regularly performing full backups and supplementing them with frequent incremental backups, the platform can achieve efficient data protection. Specifically:

• Full backups are primarily used for periodic global data protection, ensuring that the entire system can be quickly restored in the event of a catastrophic failure.
• Incremental backups are used to precisely record and recover each data change, suitable for small-scale data recovery or scenarios requiring restoration to a specific point in time.

This combination not only meets different recovery needs but also optimizes storage resources and reduces the cost of backup operations.

Summary

In conclusion, Whaleal Platform provides a comprehensive backup and recovery solution for MongoDB clusters by integrating full and incremental backups. With its flexibility, efficiency, and scalability, WAP ensures that users can protect their data while minimizing resource usage and maximizing recovery speed. Whether dealing with large-scale disasters or small-scale data changes, Whaleal Platform provides the tools to recover and maintain the health of your MongoDB databases.