Expected features in CubeBackup version 3.x

We are currently working on the next major update of CubeBackup, Version 3.0. A lot of improvements will be added in the new version.

1. Parallel backup
CubeBackup will allow you to choose how many accounts can be backed up in parallel. This will greatly improve the backup speed for large organizations who have fast Internet connections.

2. Bandwidth throttling
Users can limit the backup speed in office hours, which can prevent CubeBackup consuming too much network bandwidth in the day time.

3. Run as a service
Most users run CubeBackup on server operating systems, like Windows Server 2012. Making CubeBackup running as a service provides users more flexibilities and convenience.

4. Administrator email notification
Daily/weekly backup statistics, error reports, and other notifications will be sent to G Suite Domain administrators via email. Administrators will get the latest information for the backup without checking logs.

5. Improved log
More information will be added to backup/restore/error logs.

6. Bug fixes
Many bugs will be fixed in the new version.

7. Restoring the whole account
Currently, CubeBackup allows you to restore any files/data for a user when you selected the files and click the “Restore” button. In future versions, Google Apps administrators can restore a whole account with just one click, or even restore all data from one account to another account. This feature might not be available in version 3.0, but should be added to a later release, like version 3.1 or 3.2.

8. Linux version
We got a lot of requests asking for the Linux version. It is still under development and is expected to be released in a few months. The good news is: CubeBackup’s first Linux Version will have all the features as its Windows version.

We are always happy to hear feedbacks from our users, so please feel free to contact us (support@cubebackup.com) if you have any questions.

How to solve the “Authorization failed” problem in the initial configuration

The initial setup of CubeBackup is a little complicated, and some of our users might run into the “Google Apps authorization failed!” error when clicking the “Test” button in the initial configuration of CubeBackup.

auth-failed

If you have carefully followed the instructions at https://www.cubebackup.com/first-time-run.html, but still failed to pass the authorization test, please check the followings steps:
NOTE: The following items are listed in descending order of likelihood.

1. Are the Admin SDK, Drive API, Calendar API, Contact API and Gmail API enabled in Google API console?   A quick way to confirm: all of the above APIs should be listed in API Manager -> Dashboard -> Enabled APIs.

2. The Client Name in your Google App/GSuit  admin console  -> Security -> Advanced settings -> Manage API Client access should be the created Service Account’s client ID (e.g.:10847638222459), not an email address.

domain-auth

3.  Make sure that  “Enable API access” is checked in Google Apps admin console -> Security -> API reference -> API access. Generally, “Enable API access” is checked by default.
enableapi

4. If your computer is running a fresh installation of Windows 7 or Windows Server 2008, and you have not installed all of the updates, the problem may be that you are missing hotfix KB2468871 for the .Net Framework 4.0. Download the hotfix here, and install it. You may be required to restart your computer before the update takes effect.

5. Please ensure that www.googleapis.com  port 443 and imap.gmail.com  port 993 are not blocked by your firewall/proxy.

6. Try CubeBackup on another computer to see if it works there.

Finally, if you’ve tried all of these and the problem persists, please contact us (support@cubebackup.com).

Comparison between cloud backups and on-premise backups

There are several options for Google Apps backup: Spanning and Backupify can backup Google Apps data to another cloud, while CubeBackup can backup your Google Apps data to a local storage inside your company. When you are looking for a backup solution for your Google Apps domain, you might not sure which backup solution is more suitable for your company. So, the comparison of these two different backup strategies might be helpful to you.

cloud-vs-local

Cloud-to-Cloud backup solutions:

Pros:

1. Easy configuration
Cloud-to-Cloud backups employ web interface which is pretty easy to be integrated with Google Apps marketplace.  Only a few steps are required to setup the backup process.
2. Less administration work
All backup jobs are running on backup vendor’s servers. As a Google Apps domain administrator, you can relax with ease in most of the times.
3. A more natural choice
Google Apps is a cloud collaboration platform, with all data stored online. Backing up cloud data to another cloud sounds to be a more natural choice for most companies.

Cons:

1. Higher cost

The pricing for Backupify and Spanning is about $40-50 USD/user/year, which is not so affordable.
2. No physical files accessible
With a cloud-to-cloud backup solution, all backup data are stored in another cloud with a vendor-specific format. Except restoring from the backup, you can almost do nothing with the backup data.  For example, migrating email message backups to another mail system is just impossible.

On-premise  backup solutions:

Pros:

1. More affordable
Comparing to cloud-to-cloud backup solutions, the cost of local backups solutions are much lower.
2. Keep your data in your own hands
All backups are stored in a local storage (a hard disk or a NAS) inside your company’s firewall, totally under your control.
3. Get physical backup files
Backup data are stored locally as physical files, which means that you can easily export these files into another system.  For example, open docs backup files with Microsoft word/OpenOffice, read email messages using Outlook, etc.
4. Keep data from being exposed to any third party
Cloud-to-cloud backups open your data instance onto another cloud vendor which create privacy concerns. Cloud-to-local backups, which only preserve a copy of your data locally, keep you from this security concern.

Cons:

1. Initial configuration is not easy
Local backup solutions require a desktop app to access the local storage. Due to the limitation of Google Apps Marketplace, desktop apps cannot be easily integrated into Google Apps marketplace as web apps do.  To setup a local backup, domain administrators need to create a Google cloud project in Google Developers console and authorize data access to GDrive, Gmail, Contacts and Calendar.
2. A reliable physical server and a local storage are needed
Unlike cloud-to-cloud backup solutions, which are completely disk-free, on-premise backup solutions require a reliable server as well as a large and robust local storage.

Conclusion:

If you do not have enough budget, or you’d like to fully control your backup data, or there is any possibility that your company might migrate your cloud platform from Google Apps to Office 365, an on-premise backup solution, such as CubeBackup, should be your best choice.

If you want an easier and effortless backup solution, you might consider a cloud-to-cloud backup solution.

Initial Configuration Details for CubeBackup

Some customers have been concerned that the initial configuration of CubeBackup is too complicated, and they cannot understand why a simple authorization requires so many operations. In this post, I’d like to discuss OAuth 2.0 Service account and present a more detailed explanation of CubeBackup’s initial configuration.

banner-cartoon1

Although it may seem complicated at first,  the initial authorization process (described here) only consists of two steps:

1.  Create a Service Account in Google Developers Console.

A Google OAuth 2.0 service account allows the administrator of the Google Apps domain to authorize an application to access user data on behalf of users in that domain. This authorization is also referred as “domain-wide authorization” or ” 2 Legged OAuth”. To create a service account, you need to login Google Developer Console with a personal account (e.g.: someone@gmail.com or someone@mycompany.com).  For the purposes of this step, you may consider yourself a developer of Google APIs.

2. Authorize Access to your Google Apps Domain for the Created Service Account.

Next, the Google Apps administrator should enable the service account created in the previous step to access the data in your Google Apps domain.

Q1. What is an OAuth 2.0 Service Account?

OAuth 2.0 enables a third-party application to obtain limited access to an HTTP service. Through OAuth 2.0, the application requests an access token from the specific HTTP service and then attach the token to the subsequent requests to that service. It is a modern, safe authorization standard which protects your credentials from being leaked to anyone else. OAuth 2.0 is the Google’s recommended login method for all third party applications.

A Google OAuth 2.0 service account is a variant of the standard OAuth 2.0, which enables data access to an entire Google Apps domain, not just a single Google account. This is also called 2-Legged OAuth 2.0. More detailed information is available at https://developers.google.com/identity/protocols/OAuth2ServiceAccount#overview.

Q2. Why are CubeBackup users required to create a service account themselves? Shouldn’t this be done automatically?

Service accounts could, of course, be created automatically, or even embedded within CubeBackup itself, but this would negate some very powerful advantages:

I)  By manually creating a service account, users can tailor the account to their own needs by enabling/disabling Google APIs, setting up the request quotas, etc.  They can also monitor API requests statistics in real time using Google Developer Console.

II)  A single service account shared by multiple users might overrun quota limitations. For example, the default Google Drive API quota is 1,000,000,000 requests per day, which should be sufficient for a single organization in most cases. But if this service account is shared by hundreds of organizations, the quota could easily be reached.

III)  Leaving the creation of service account to a third-party is always a potential security risk. The service account owner can monitor your requests, set request quota, and generally interfere with your business.

Q3. Why is CubeBackup’s configuration more complicated, when other web-based apps on Google Apps marketplace seem to easily integrate with Google Apps?

The situation with web-based apps is a little different. They need service account authorization to access Google Apps data, just like desktop apps, but most web-based apps/addons for Google Apps are installed through Google Apps marketplace, as explained by Google:  “When you use Google Apps Marketplace to install an application for your domain, the required permissions are automatically granted to the application. You do not need to manually authorize the service accounts that the application uses.”  

Unfortunately, this straightforward app integration is only available for web-based Google Apps tools, and not for desktop apps.  One of the strengths of CubeBackup is its ability to backup all Google Apps data to local storage, which is simply impossible with web-based approaches, due to the security limitation of web browsers.  The trade-off for this flexibility and power is that service account configuration and authorization must be done manually.

Hotfix for CubeBackup

The new version of CubeBackup, version 2.6.8, has just been released. This is a hotfix version to solve the problems caused by recent changes in Google Drive APIs. We strongly suggest all CubeBackup users upgrade to this version as soon as possible.  We are sorry for the inconvenience, but this upgrade is not optional.

What’s the problem in older versions?

A few days ago, Google made several modifications on its Drive APIs and caused some file-downloading operations in CubeBackup to fail. CubeBackup users might see lots of errors in the log files, such as “Failed on backup of Google Document entry: … Error message: One or more errors occurred. A task was canceled.”, or “Object reference not set to an instance of an object”, etc.  As a result, some Google Drive files were not successfully backed up locally. This problem happened randomly and just started a few days ago.

What’ new in CubeBackup 2.6.8?

This version is just an emergency hotfix for recent Google Drive backup problems. There are no new features added to this version. Due to the changes in Google Drive APIs, every CubeBackup users should upgrade to the new version as soon as possible.  We are sorry for all the trouble. However, as a client app completely based on Google APIs, CubeBackup must be adjusted to even minor changes in Google APIs.

Upgrade instructions:

  • To upgrade to the latest version, please select the “Check for updates” menu item in CubeBackup, then click the “Update” button. You need to exit CubeBackup and restart it for the new version to take effect.    NOTE: Simply closing the main window will not exit the application. To quit CubeBackup, please select “Exit CubeBackup” from the system tray in the right-bottom corner of the screen.
  • Or, you can quit and restart CubeBackup. CubeBackup will check for new versions every time it starts.

What if the upgrade failed?

There are possibilities the upgrade may fail and “Upgrade failed. You can try it again later.” message is given. To solve this problem, you can:

  • Download the CubeBackup hotfix file manually and unzip it into the installation directory for CubeBackup, like “C:\Program Files (x86)\CubeBackup“, to replace the old files.
  • Or, download the new version from www.cubebackup.com  and reinstall it. All the application settings, configurations and user selections in old versions will be kept after the new installation, but you will be asked for the license key again to activate CubeBackup.

For some users, their Drive files backups were incomplete due to the errors in the last few days, how do I fix it?

To retry the backup for a specific user, you can open the local backup folder in Windows Explorer, then delete the folder for that user. For example, if the folder named “jerry@mycompany.com” is deleted, CubeBackup will backup jerry@mycompany.com‘s data completely from the beginning in the next backup cycle.

If you have any problems with this upgrade, please don’t hesitate to contact us.

How to backup Google Apps data to Dropbox

Dropbox is undoubtedly one of the most widely used cloud storage.  It is available on almost all platforms, from Windows, MacOSX, Linux to nearly all mobile operating systems.  Dropbox guarantees that all your files will be protected by 256-bit AES encryption, and all files are transferred through SSL/TLS secure tunnel.  What’s more, compared with Google’s privacy policy, privacy terms of Dropbox are more strict and reasonable – “Your Stuff is yours. These Terms don’t give us any rights to Your stuff except for the limited rights that enable us to offer the Service.  We need your permission to do things like hosting Your Stuff, backing it up, and sharing it when you ask us to… ” . 

As a Google Apps users, you might are seeking methods to backup Google Apps data to Dropbox. Actually, using CubeBackup, it is quite easy to add a backup copy to Dropbox with very little expense.

Step 1:  A Dropbox Pro or Bussiness account.

For small or middle size businesses, whose data in the whole Google Apps domain is less than 1TB (1,000GB),  a Dropbox Pro account is a reasonable choice.  It only costs $99 per year.

For comparatively large organizations, Dropbox for Business might be your best choice. The basic plan for Dropbox Business is 5TB for 5 users, which costs $750 per year. Dropbox for Business starts off with 1TB of space per user, however, you can ask for more space by contacting Dropbox’s sales. Theoretically, you can get as much as possible space from Dropbox for Business as long as you would like to pay for the extra storage. The pricing of Dropbox for Business is higher than Dropbox Pro, but is still much cheaper than other Google Apps cloud-to-cloud backup solutions, such as Backupify or Spanning.

Step 2:  Set the backup location to the Dropbox sync folder.

After installing Dropbox client, a folder named “Dropbox” will be created. If CubeBackup has already been running on your computer, please stop the backup process first, then copy all the backup data to the “Dropbox” folder – This may take a considerably long time if you have tons of backup data.

When the moving of data completed,  open the “Options” dialog of CubeBackup and set the backup location to the new place.

set-backup-location

After this simple configuration, your Google Apps data will be automatically synced to Dropbox cloud storage as they are backed up locally with CubeBackup.

CubeBackup 2.6 released

We are proud to announce the release of CubeBackup version 2.6.  If you are using an old version of CubeBackup, you may have already received a prompt to upgrade. You can easily upgrade to the new version at any time by using the “Check for update” feature.

New features in CubeBackup 2.6

  • Support for secondary domains and domain aliases.

As a Google Apps administrator, you may manage one primary domain along with one or more secondary domains.  CubeBackup 2.6 makes it convenient to manage all backup users in one place, whether they are in a primary domain or secondary domain. Domain aliases are also well supported in the new version.   For example,  in the user management window, domain administrators can manage all users in one list, including user1@primary.com, user2@secondary.com  and  user3@alias.com.

  • Support multiple primary domains.

If you are a Google Apps partner or reseller, you may manage several Google Apps domains for your clients. The new version of CubeBackup allows you to backup multiple domains in one application.

  • Option to back up “Shared with me” files in Google Drive.

Old versions of CubeBackup always back up shared files by default, with no way to disable the feature. For some Google Apps domains, it may cause considerable data duplication and backups could potentially grow to unmanageable sizes. The new version allows you to choose whether or not to backup these shared files.

  • Option to backup “All mail” messages

The domain administrator now can choose whether or not to backup the “All mail” folder/label in Gmail.  Generally, if messages in “All Mail” folder are duplicates which already exist under other folders/labels, including them again would be a waste of time and space.  However, if the “All mail” folder is used to archive mail messages, while the “Inbox” or other labels are reserved for active messages,  the “All mail” option should, of course, be used.

  • Database performance tuning

SQLite database is used in CubeBackup to record the backup metadata. Performance and transactional security are both improved in the new version.

  • Bug fixed

A number of bugs have been fixed in the new version.

 

How to fetch users in secondary domains after the update?

Because secondary domain users were not visible in previous versions of CubeBackup, you need to refresh the user list in order to fetch the users in secondary domains (or domain alias) after the upgrade.

Open the menu item “Settings” -> “User management” , and click the Refresh button. All users, in both primary or secondary domains, will now show up in the users list.  Click the Save button to save all the settings.

user managment

 

From time to time, we receive emails from our customers asking new features. Thank you for all of your suggestions and feedbacks!   The most commonly asked question is:

What to expect in the future versions?

  • Parallel backup

This will be the most important feature in the next version of CubeBackup.   Parallel backup for multiple users can greatly accelerate backup speeds and should be available soon.

  • Google site backup

Google site data backup will also be added in future versions.

  • Restore backup data to another account

Restoring data to another account can be very useful in some circumstance. Look for this feature in future versions

  • Linux version

CubeBackup for Linux will be a service without UI,  and we are confident that Linux administrators will love it! Most features in the Linux version will be almost identical to those in the Windows version.

How to speed up backups for large organizations

Schools or large companies with hundreds of users can easily accumulate terabytes of Google Apps data, including tens of millions of Gmail messages. This presents a serious problem for backup strategies – the initial backup may take months or even longer!

One reason for this is that CubeBackup performs user backups one by one in a non-parallel manner. We plan to include parallel backup algorithm in the next version of CubeBackup, but in the meantime, there are still methods that can be used to greatly speed up large backups by working with the user data in parallel.

Before presenting some of these solutions, I’d like to give an introduction to how backup data is saved locally.  Simply put, all backup data for each user, including the actual Google Apps data and metadata used to record the backup progress for each user, are stored in that user’s own directory. For example, data for user “somone@yourcompany.com”  would be stored in the directory “someone@yourcompany.com” like this:

Folder structure for one user

Folders like “calendar”, “contacts”, “docs”, and “mail” are used to store the actual backup data for Google Apps. There is also a hidden folder named “_config”, which is used to record important metadata, like timestamps for the last backup, the cloud ID of the local backup files, etc. CubeBackup itself only stores general settings for the whole app, all user data, whether actual backup data or metadata to describe a user’s current status, are stored in that user’s own directory.

This design makes the backup data very portable, as well as bringing an extra benefit: several CubeBackup instances can run simultaneously while sharing the same local storage. This means that, for large companies with hundreds of employees, the best way to speed up the backup process is to split the users into separate CubeBackup instances on different computers. Let me illuminate this method with a specific example:

An Example

Assume there is a large Google Apps domain “example.com” with 1000 users, 200 of whom are located in the subdomain “sub.example.com”. There are also several organization units in the main domain (for example, 100 users in marketing 100 in dev, 50 in HR, etc.). The whole structure of the domain looks like this:

domain structure

For such a large domain, a complete initial backup may be so enormous that it would take months to back up all the data locally, which, of course, is unacceptable. In most cases, a professional local storage unit, such as a NAS or SAN, will be used to store the backup data. Usually, these devices have excellent write speeds and are capable of handling very high data throughputs. In this case, you can split the users into different CubeBackup instances running on separate computers, while setting the backup location for each instance to the same NAS/SAN device.  Since CubeBackup can be run on almost any computer, one powerful server and several low-speed desktop computers make a good combination for this strategy.

parallel backup

This parallel backup strategy can speed up the backup process tremendously. Since CubeBackup consumes very little memory and CPU resources, it can run in the background on almost any computer without interrupting the user. This makes it is very easy to add more computers to this kind of parallel backup system.

How many computers should be used to speed up the backup?

In most cases, the bottleneck of the parallel backup system lies in the speed of the NAS/SAN. Since most Gmail message backups are small files, the speed at which these small files can be written to local storage is the key factor in deciding how many computers can be added to the parallel system. Due to the wide spectrum in the performance of different storage devices, it is difficult to predict the optimal number of computers.  We suggest starting small, with 4 computers, and then adding more running instances to the parallel backup system to find the best combination.

Merge all users into the main server after a few backups

Because CubeBackup employs an incremental backup algorithm, subsequent backups after the initial full backup only store new or modified data, which is much less time-consuming.  Therefore, it shouldn’t be necessary to keep so many CubeBackup instances running after the first several backups; the main server should be sufficient to handle most of the load.
final structure

Another option for parallel backup:

In some cases, there is no NAS/SAN available within your company. Instead, each department is responsible for maintaining their own set of backup. In this situation, CubeBackup can be run directly on computers within the department, with backups split according to organizational units, like this:

OU backup

This method does not require high-performance local storage and can use existing PCs for the backup. However, since there is no central data storage, and local storage within each PC is not as stable or reliable as a NAS/SAN, maintenance of the backup data may become a problem.

We plan to add parallel backup and multi-domain support in the next version of CubeBackup.  If you have any suggestions for our product, please let us know!

 

CubeBackup 2.0 is released

Today we are shipping CubeBackup 2.0, a major release that has been in the works for 3 months. There are lots of bug fixes and feature improvements in the new version.

Can I continue to stay with CubeBackup 1.x?

No, you can’t!   Based on Google’s API deprecation statement  http://googleappsdeveloper.blogspot.com/2014/12/reminder-to-migrate-to-updated-google.html  and  https://developers.google.com/identity/protocols/OAuth_ref, several APIs used in CubeBackup 1.x will stop working after April 20, 2015.  As a result, CubeBackup 1.x  will no longer be functional after that date. It is important to upgrade CubeBackup to the new version in order to secure your Google Apps data.  We are sorry for the inconvenience, but this upgrade is not optional.

What’s new in CubeBackup 2.0?

  • Two-Legged OAuth 2.0 (OAuth2.0 service account) for domain-wide authorization.

CubeBackup 1.x uses 2-Legged OAuth1.0 for authorization. When CubeBackup 1.0 was in development, OAuth2.0 was not yet mature enough for Google Apps domain-wide authorization and hardly any code libraries could be found.  However, this situation has changed, and now CubeBackup has been completely moved to OAuth2.0, which offers superior security and flexibility.

  • Migration to the latest Google Apps APIs.

The following APIs are used in CubeBackup 2.0:

    1. Admin SDK Directory API
    2. Google Drive API  v2
    3. Gmail REST API 
    4. Google Contacts API v3
    5. Google Calendar API v3
  • Select users by Organization Unit.

With the new Google Admin Directory API, CubeBackup allows you to select backup users by their organization unit. This feature can be very useful for large companies who might wish to apply different backup policies on different departments.

  • Excellent multi-language support.

CubeBackup has been completely redesigned to support all languages. It can easily handle multiple languages in a single Google Apps domain now.

  • Backup shared files in the Google Drive “Shared with me” folder.

Shared files were not backed up in the old version of CubeBackup.  In CubeBackup 2.0, the “Shared with me” files are fully supported.

  • Fully backup the whole Gmail folder structure.

In CubeBackup 1.x, only messages in “Inbox”, “Sent Mail”, “Starred” and “All Mail” were backed up.  In the new version, the whole mailbox structure, including customized folders/labels, are all included in the backup.

  • Improved performance.

Code optimization and the adoption of the latest Google API has improved backup performance in CubeBackup 2.0.

  • Bug fixed

Many bugs have been fixed in the new version.

Steps to migrate to the new version.

  1. Uninstall CubeBackup 1.x.
  2. Download the latest version of CubeBackup from cubebackup.com.
  3. Install CubeBackup 2.0.

    During the install process, you might be asked to install .Net Framework 4.0 if it’s not preinstalled.   For Windows XP, Windows Server 2003 or Windows Server 2008 users,  KB2468871 hotfix may also be required in some cases. Please visit the installation page to get more information.

  4. Start CubeBackup, and follow these instructions for authorization and configuration.

    The OAuth2.0 Service account (2-Legged OAuth2.0) authorization can be complicated, so please follow the above instructions carefully.

Because significant changes have been made in CubeBackup 2.0, the backup data is not fully compatible with that of CubeBackup 1.x. Google Drive/Docs and Google Contacts backups are unchanged, but Gmail and Calendar backups are now handled in a slightly different way.

When specifying the backup location, there are two options:

A. Delete all data backed up by CubeBackup 1.x and use CubeBackup 2.0 to backup your Google Apps data from the beginning. This gives you a clean start, but may take longer for the first backup. (Recommended)

B. Keep the backup data of CubeBackup 1.x, and specify the same folder as the backup location for CubeBackup 2.0. Google Drive and Contacts data will be backed up incrementally based on the old data, but CubeBackup 2.0 will create a new backup for Gmail and Calendar data. The new data should overwrite the old backup without any problems. This method saves backup time but makes the backup folder a bit more messy.

If you have any problems or suggestions for CubeBackup, please do not hesitate to contact us:  support@cubebackup.com.   Thanks!

 

Price for educational organizations

Google Apps Logo NewRecently, we’ve received a lot of emails inquiring about the pricing for Non-Profit/Educational organizations. Since Google Apps is so popular among students and educators, these users are also very important to us at CubeBackup. We are pleased to offer a discount for Non-Profit/Educational organizations:

$99   USD/year      Up to 50 accounts

$499 USD/year      51-500 accounts

$999 USD/year      501-2000 accounts

$1999 USD/year     2001-5000 accounts

$2999 USD/year     more than 5000 accounts

These prices are heavily discounted and only available to non-profit or educational organizations.

Non-profit and education purchases can be made at https://www.cubebackup.com/pricing-siteprice.html .

Reseller Program: 

We are also looking for new members for our reseller program. If you are a Google Apps reseller/partner and are interested in introducing CubeBackup to your clients, please contact our sales team for more information.