How to remove large unnecessary files from backup storage in order to save space.
When you back up Microsoft 365 data using CubeBackup, the size of the backup can easily balloon, especially for large organizations. For example, one employee might upload many large unimportant videos to SharePoint, or another may keep different copies of VM images or Mac Time machine backups in OneDrive. These files can consume a large amount of space in your backup storage.
CubeBackup for Microsoft 365 allows you to filter out these large unnecessary files from your backups based on your own customized rules. Using CubeBackup, you can set file exclusion rules before the first backup starts to prevent certain files from ever being backed up. You can also use these exclusion rules to remove unnecessary files from the backup storage at a later date.
Prevent files from being backed up before the start of the first backup
After installing CubeBackup on your server and setting it up, you can log into the CubeBackup web console and add your file exclusion rules before the first backup starts.
In the CubeBackup for Microsoft 365 web console, select SETTINGS, and then in the Apps section click *Options at the right of the Drive switch.
In the filter dialog that pops up, you can add multiple file exclusion rules. For a detailed explanation and examples of the exclusion rules, please refer to File Exclusion Rules.
NOTE:
- Once a backup has been started, changes to the exclusion rules will only take effect after the current backup has completed. That is, they will apply to all subsequent backups, but not to the currently running backup.
- If you are unsure whether certain files need to be excluded in the backup, or you are concerned that important files may be mistakenly excluded, it is safer to leave the exclusion rules empty. You can still remove unwanted files after the initial backup is completed.
Remove unimportant files from the backup data
It is not absolutely necessary to set up the file exclusion rules before the first backup. In most cases, administrators may find unnecessary files after backups have already been performed. Don't worry, CubeBackup can still remove these files based on your exclusion rules to free up disk space or cloud storage.
CubeBackup for Microsoft 365 allows you to easily purge files from the backup data using the cbackup filesExclude command. Please follow the detailed instructions appropriate for your operating system.
Log into the CubeBackup console, add the exclusion rules in the SETTINGS > Apps > *Options dialog, then click the "Update settings" button. For detailed information on how to write the file exclusion rules, see File exclusion explanation and examples.
Log into the backup server.
Be sure to stop the CubeBackup service before running this command:
Open a Command Prompt or Windows Powershell as Administrator and run the following command:"C:\Program Files\CubeBackup365\bin\cbsrv.exe" stop # in Command Prompt
& "C:\Program Files\CubeBackup365\bin\cbsrv.exe" stop # in Windows Powershell
Run the cbackup filesExclude list command to list all files that match your exclusion rules.
"C:\Program Files\CubeBackup365\bin\cbackup.exe" filesExclude list <app> <organization-id>
Note:
(1) This command requires special privileges. Please run this command as Administrator.
(2) Depending on the size of the organization's data, this process may take quite a while.Run the cbackup filesExclude remove command to purge files from the backup data. When prompted, double-check that this is the correct user to be deleted. Type "yes" to purge all matched data for that specific organization.
"C:\Program Files\CubeBackup365\bin\cbackup.exe" filesExclude remove <app> <organization-id>
After running this command, you will need to start the CubeBackup service:
Open a Command Prompt or Windows Powershell as Administrator and run the following command:"C:\Program Files\CubeBackup365\bin\cbsrv.exe" start # in Command Prompt
& "C:\Program Files\CubeBackup365\bin\cbsrv.exe" start # in Windows Powershell
In the CubeBackup console, you will now see that the storage status information on the OVERVIEW page has updated.
Log into the CubeBackup console, add the exclusion rules in the SETTINGS > Apps > *Options dialog, then click the "Update settings" button. For detailed information on how to write the file exclusion rules, see File exclusion explanation and examples.
SSH into the backup server.
Before running this command, be sure to stop the CubeBackup service:
sudo /opt/cubebackup365/bin/cbsrv stop
Run the cbackup filesExclude list command to list all files that match your exclusion rules.
sudo -u cbuser /opt/cubebackup365/bin/cbackup filesExclude list <app> <organization-id>
Note:
(1) This command requires special privileges and must be run by the cbuser account.
(2) Depending on the size of the organization's data, this process may take quite a while.Run the cbackup filesExclude remove command to purge files from the backup data. When prompted, double-check that this is the correct user to be deleted. Type "yes" to purge all matched data for that specific organization.
sudo -u cbuser /opt/cubebackup365/bin/cbackup filesExclude remove <app> <organization-id>
After running this command, you will need to start the CubeBackup service.
sudo /opt/cubebackup365/bin/cbsrv start
In the CubeBackup console, you will now see that the storage status information on the OVERVIEW page has updated.
The cbackup filesExclude command will remove the backup data matching the file exclusion rules. However, this command can only work after the CubeBackup service has stopped. This means that you need to stop the docker container before running the command, but that, in turn, makes running the cbackup filesExclude command a bit tricky. For Docker users, please carefully follow the instructions below.
Find the container id for CubeBackup:
docker ps -a
Stop the CubeBackup container:
docker stop <container_id>
Commit the stopped container into a new image (for example, cube/tmp_img):
docker commit <container_id> cube/tmp_img
This command saves the modified container state into a new image.
Start a new docker based on the new temporary image with a bash command.
Please note that you need to mount the volumes exactly as they were in the original CubeBackup docker container so that the new container can properly access the backup data. For example:
docker run --rm -ti \ -v /var/cubebackup_index:/cubebackup_index \ -v /var/cubebackup_data:/cubebackup_data \ --mount source=cube365_cfg,target=/opt/cubebackup365/etc \ --mount source=cube365_data,target=/opt/cubebackup365/data \ --mount source=cube365_log,target=/opt/cubebackup365/log \ cube/tmp_img \ /bin/bash
Now in the bash shell of the new docker, install the sudo command:
yum install sudo -y
Run the cbackup filesExclude command in the docker container.
sudo -u cbuser /opt/cubebackup365/bin/cbackup filesExclude list <app> <organization-id>
Note:
(1) This command requires special privileges and must be run by the cbuser account.
(2) Depending on the size of the organization's data, this process may take quite a while.Run the cbackup filesExclude remove command to purge files from the backup data. When prompted, double-check that this is the correct user to be deleted. Type "yes" to purge all matched data for that specific organization.
sudo -u cbuser /opt/cubebackup365/bin/cbackup filesExclude remove <app> <organization-id>
After the cbackup filesExclude command has successfully completed, type exit to stop and exit the temporary container.
Restart the original CubeBackup container.
docker start <container_id>
In the CubeBackup console, you will now see that the storage status information on the OVERVIEW page has updated.