Backup and DR
Incremental Backup Explained
Incremental backup is a data backup method designed to reduce storage requirements and bandwidth load and provide the necessary level of data consistency and availability.
In this guide, we overview this concept, how it works, and how it helps save time and money on backup and disaster recovery routines.
What Is Incremental Backup?
This is a type of backup that copies only data that has changed since the previous backup. Unlike a full backup, where the whole data set is copied to the backup storage with every backup job, it allows you to perform a full backup only once in a while.
That is how this backup type gets its name: each backup is an increment for a previous backup.
Further reading Incremental and Differential Backup Comparison
How Incremental Backup Works
To demonstrate the backup process, let’s assume you have made a full backup and that now you would like to to do a weekly backup of any added or changed files:
- Week 1 - first full backup
- Week 2 - added and changed data only since the Week 1 backup
- Week 3 - added and changed data only since the Week 2 backup
This backup type requires less storage space and network load and can be run more often since there are usually a lot of small everyday changes in the data set that should be backed up.
Forever Incremental Backup
It implies that you want incremental backups to be done at regular intervals after the initial full backup was performed. It is very helpful that you don’t need to worry about full backups afterward.
However, with each subsequent backup, the chain of incremental backups becomes larger. As a result, it takes more time and computing capacity for backup software to analyze the full backup and all increments so as to determine the difference between the data on your server/workstation and the data in your backup repository.
This backup type reduces recovery reliability. It also becomes harder and takes longer to recover the whole data set, as it takes time to analyze and recover each backup in the chain.
That is why periodical full backups are highly recommended, in order to start a new sequence of incremental backups. The frequency of full backups depends on your business needs. You may want to conduct it weekly, monthly, or once every couple of months. There is an advanced backup technique to simulate full backups, called a synthetic full backup.
Further reading Synthetic full backup
Versioning
Backup software vendors usually let users set the number of versions of data they would like to keep. Incremental backup implies a full backup and a chain of subsequent increments and, in terms of versioning, every subsequent backup (full or incremental) is a version.
To provide data consistency, you cannot eliminate individual elements of this chain, only the entire series of the full backup and all the increments up to the next full backup. This is why we strongly recommend performing full backups periodically.
Block-level retention policy example
Types of Incremental Backup
There are different backup types that are determined by the size of changes that the software adds to the repository at every subsequent backup. We will overview incremental file-level backup and incremental block-level backup in detail.
Incremental File-Level Backup
In an incremental file-level backup, if a file has been modified, it will be sent to the backup repository to create a new version of it. This backup type is simple to perform and works well for a small data set. However, this method might take longer, because, if the changed files are large, the backup will also be large, which will extend the duration of the backup job and require more storage space.
Incremental Block-Level Backup
In a block-level approach, the backup software performs a more in-depth file analysis and copies only the modified portions of files, making it much faster to upload than using a file-level backup.
Further reading Block-Level Backup Explained
Using an Incremental Backup
Advantages
Compared to the full backup, an incremental type uploads only changed and new files, which allows the reduction of:
- Needed storage space
- Bandwidth consumption
- Computing resources usage
Let’s consider several different scenarios to back up a 5 GB dataset with a 3% daily change rate for one month and see how effective they are and how much storage space you need.
- Weekly full backups only. In this case, you will need to set up only 4-5 backups a month which is quite storage-friendly, especially if your retention policy doesn’t require you to keep several versions of every backup. The main issue here is data reliability. If something malicious happens to your servers or endpoints, you will permanently lose about a week of critical business data. Storage space calculation: 5 * 5 GB = 25 GB
Further reading RTO vs. RPO: Two Means toward the Same End
- Full backups daily. When you set full backups every day, it is quite reliable, isn’t it? However, it is unlikely someone is going to use this scenario as it requires so much storage room, bandwidth, and computing capacity that this backup strategy becomes highly ineffective. Storage space calculation: 22 * 5 GB = 110 GB
- Weekly full backups with incremental backups on weekdays. In this scenario, full backups don’t have to be set very often, and the periods between full backups are covered by incremental sequences. Therefore, this scenario turns out to be the golden mean between saving storage space and providing data reliability. Storage space calculation: (5 * 5 GB) + (22 * 3% * 5 GB) = 28.3 GB
Disadvantages
However, this approach has several drawbacks:
- Longer data restore. It takes more time for backup software to find, analyze, and collect the latest full backup and all subsequent incremental backups.
- Less data consistency and reliability. As we have already mentioned, when you need to recover, the backup software analyzes the whole backup chain. If one element in that chain is corrupt, your data is non-recoverable.
Incremental Backup Best Practices
To make your recovery easier, here are a few tips:
- Perform full backups from time to time. Starting a new full backup sequence is a good way to ensure you can recover data.
- Set a retention policy for your files. A retention policy allows you to choose how long to keep your old files and how many versions of each file you keep. With an incremental backup, it’s recommended to set a policy that allows you to delete old files after a given time, or files that have been deleted from your device.
Further reading Backup Best Practices for MSPs
Summary
Out of all backup types, the incremental backup is one of the most beneficial backup methods as it assures reliability and accessibility of your data, saves a lot of storage space, consumes less bandwidth, and leads to faster backups compared to a full backup scenario. The flaws such as slower restore and dependence on the increments integrity may be addressed by performing the full backup periodically and setting the proper retention policy for your backups.
Simple. Reliable.
