GitLab deleted the wrong database, but when ineffective backup solutions got added to the mix, the site’s system admins had to battle the perfect storm to get the site online. The takeaway from this situation? Choose your backup solutions carefully.

 

backup software
GitLab’s system admins battle the perfect storm of system errors and inefficient backups to get the site back online

 

GitLab, the online tech hub, is facing issues as a result of an accidental database deletion that happened in the wee hours of last night. A tired, frustrated system administrator thought that deleting a database would solve the lag-related issues that had cropped up… only to discover too late that he’d executed the command for the wrong database.

What Went Wrong with GitLabs’ Backups

While the horror of the incident might have been mitigated by the fact that GitLab had not one but five backup methods in place, the problem was that all  of them were discovered to be ineffective. Here’s a quick run-through of the different backup methods GitLab had, and what went wrong with each of them:

  • The LVM snapshot backup wasn’t up-to-date– the last snapshot was manually created by the system admin 6 hours before the database deletion.
  • The backup furnished on a staging environment was not functional– it automatically had the webhooks removed, and the replication process from this source wasn’t trustworthy since it was prone to errors.
  • Their automatic backup solution was storing backups in an unknown location, and to top it, it seemed that older backups had been cleaned out.
  • Backups stored on Azure were incomplete: they only had data from the NFS server but not from the DB server
  • Another solution that was supposed to upload backups to Amazon S3 wasn’t working; so there were no backups in the bucket

 

As a result of these issues, the system admins are struggling to get the 6-hour old backup online. The progress of the data restoration has been closely followed by well-wishers, and many have appreciated the website’s transparency, especially under such duress.

 

How to Identify a Good Backup Solution

It’s certainly freaky that all the five backup solutions that GitLab had were ineffective, but this incident demonstrates that a number of things can go wrong with backups. The real aim for any backup solution, is to be able to restore data with ease… but simple oversights could render backup solutions useless. A backup software should be effective to handle all types of issues and provide enough security. This is why you should watch out for the following traits in any backup solution:

  1. Backup solutions should match your need
    In the case of GitLabs, automatic backups were made once in every 24 hours. Considering the amount of data being added every minute, however, real-time backups would have been perfect for them. While not being the best in terms of data-conservation, the last manual backup was performed by the system admin 6 hours before the crash, and so was the most viable option. Choosing the right backup solution for your need requires the consideration of the frequency of data-addition, the levels of user activity, and the server load.
  2. Backup solutions should allow easy, quick restoration
    The problem with GitLab’s backups stored on its staging environment, was that the replication process was difficult to manage. When you’re already burdened with the responsibility of getting your site back up, you shouldn’t be worrying about the restoration process.
  3. The backup solution should be completely independent of your site… in a known location
    In the GitLab situation, the problem was not knowing the backup destination. This isn’t a problem with WordPress backup solutions,since they usually store backups on your site’s server… or on a personal storage account (such as Dropbox, Drive or Amazon S3). However, this means most of the time, they either require you to access your crashed site for backups… or they store the API key to these accounts on your site (which poses its own problems). Both these options present Catch-22 situations of ‘site is down so need backups, can’t access backups because site is down’. It’s important for you to know all there is to know about your backup destinations.
  4. The backup solution should backup your entire site
    Backups that only contain part of your site (such as GitLabs’ Azure backups) aren’t really reliable when your site goes down. In the case of WordPress backups, some solutions might backup your site except for custom tables (such as those installed by WooCommerce), so you need to be wary of such situations.
  5. You should be able to easily test your backups
    The real problem with all the backup solutions GitLabs had, was that they hadn’t previously tested them… and hence had to give them a hard second look after encountering restoration-related problems. The real concern is that their backups weren’t discovered to be inefficient until they actually needed them. This is why testing backups should be a part of your backup strategy.

 

We’re all human at the end of the day, and the job of a systems admin, especially when overloaded with spam, can never be taken lightly. This is why backups exist– to have an easy ‘undo’ in case there ever is an error, and your site goes down, or data is lost.
We can only hope that things go well for the GitLab team, as they rush to get their data back.

GitLab’s status can be monitored via this Twitter feed. (When this article was published, 73% of the database copy had been made).

Cloud WordPress backups are good when they are independent. They might not be if you’re using your personal cloud storage accounts(for example on Dropbox, Drive or Amazon S3). Read on to know how and why you shouldn’t do it.

 

Are your cloud storage accounts safe?

 

 

We know that following best practices to make WordPress backups means that your backups should not be dependent on your website/server/web host. This means that you must be able access and use your backups without having to access your WordPress site/server/web host. These kind of backups are known as independent backups and are according to best practices of performing WordPress backups.
However, it is easy to think that off-site backups are the same as independent backups. They are not. This is because off-site WordPress backups are not necessarily independent. This is has to do with how WordPress plugins upload backups to your accounts.

 

 

 

WordPress Backups Compromised by API Keys

Plugins which upload your WordPress backups to your Amazon S3, Drive, or Dropbox accounts usually store a copy of your account’s API key on your site. This is what allows those plugins to interact with your accounts, and upload backups. This is part of the setup procedure for many (if not all) backup plugins.

While making automatic uploads to an off-site location is a convenient option, doing so by storing API keys may not be the safest option for you. The simple reason for this is that it is the same as leaving the keys to your bank vault in your living room. The whole point of a vault is to secure whatever you store in there from being burgled. If you leave the keys to the vault, then you have granted access. Backups are also like your most precious possessions. They are what you depend on in your hour of need; hence they must be completely independent of your site.

Continuing from the previous point, if you are using a security key from your Amazon S3 account in multiple locations then your backups may be in trouble even if your site is safe. Even  if one of the sites using that particular security key is hacked then the hacker has access to all the contents on that account.

This is why BlogVault does not ask users for personal accounts but automatically stores multiple copies of backups in different destinations. All these copies are also encrypted; providing your data and additional layer of security. You can access them independent of your web host or WordPress site via your BlogVault dashboard.

 

Limited Storage Space

One of the major attractions of using these storage services as destinations for your WordPress backups is that they offer free storage space. However, if you make backups daily (as you should), and you have large site, then this may not be enough.

This is even more true if you are using the account for reasons other than backups or you are backing up multiple WordPress sites with the same account. Pretty soon you may find yourself paying extra for storage space. So, the economic benefits of not paying for storage may not stand for long; and these economic benefits are anyway diminished when compared to security concerns.

 

Cloud WordPress Backups in Personal Accounts May Equal Personal Data

In case of a hack, losing your backups and your business or blog data may be bad enough but that will certainly not be the end of it. The risk of using a personal storage account is simply too great when you consider that other information you store on the account which may be of a personal nature can also be at risk.

 

Restoring WordPress Backups

All backups have one purpose; restores– to recreate your site using your WordPress backup. Firstly you must have backups to use. Secondly, those backups must be functional and easy to restore. When you are using your personal accounts configured with the backup plugin on your site, both cannot be taken for granted.

The first point has been addressed in the very beginning of this list. As for the second point, even though you may have backup files, if they are altered in any way or are not secure, then using those backups to restore your WordPress site will do more harm than good to your business. BlogVault allows you to Test Restore your backups with a single-click. This way you will not be in doubt.

Even if the files are functional, backups are often uploaded in .zip folders. You may have to spend a considerable amount of time finding the right backup version to restore your site and then upload then .zip folder to your plugin to restore your site. However, this is not possible when your entire site is down because your backup plugin was on your site too. This is why you must be able to access and restore/migrate your backups completely independent of your WordPress site.

On the other hand if you manage to get your site running, then there still may be issues. Restoring a large site takes time and server resources. For this reason, they are, many times cut off. This makes full restores of large sites nearly impossible on some accounts; especially on shared hosting.

 

Cloud WordPress Backups Must Be Independent

If you have not checked your backups because your WordPress site is working fine at the moment, then you may be left with an unwanted surprise when your website goes down in the future.

Use best practices and opt for a service which will provide a comprehensive WordPress backup solution that will keep you worry-free, allowing you to enjoy the ride.

 

Frequent WordPress backups can minimize data loss and thereby greatly help your business. However, they can be resource-intensive and affect your WordPress site performance, if not done right.  

Frequent backups present some obvious advantages which are particularly important for WordPress (WP) sites. Content creation takes some planning, effort and resources. Losing such content may become a major setback for your website. Daily backups minimize data loss in such cases.

Finding secure storage solutions is a real challenge with frequent WordPress backups.
Finding secure storage solutions is a real challenge with frequent WordPress backups.

WordPress sites are dependent on many third party plugins and themes. WordPress site owners are always running the risk of installing software that is not compatible with other plugins or themes on the site or installing those which may have some vulnerabilities. The risk of losing data from frequent updates and third-party software vulnerabilities is mitigated to a degree by having up-to-date backups.

 

Advantages of Frequent Backups

  • Minimize data loss
  • Reduce downtime
  • Retain updates & functionalities on WP sites

 

What are Frequent Backup Options?

Of course real-time backups is the best solution to achieve the goals stated above. Hourly/Daily backups may be the most frequent options apart from that.

 

Challenges with Frequent Backups

Higher frequency of performing backups brings its own complications. Backing up sites not only makes demands on your server resources but also brings up the issue of secure storage of the backups made. To add to the list of issues to consider, tracking whether backups have happened correctly and what has been backed up is not always easy.

 

Backups are Complicated

We have been in the business of premium WordPress backup service for over five years now. A number of things can, and do go wrong with backups. Sometimes when someone opts to backup their site manually, it is as simple as forgetting to perform frequent backups.

Often, WordPress site owners don’t know if backups are happening according to plan. Sometimes not all files are backed up.

In cases where site owners may have backups, restoring sites may not be easy. At other times, site owners who are relying on backups by web hosting services may not be fully aware of backup & storage policies. As a result, there have been times when WordPress site owners find out that there may not be any backups when they need it the most.

 

Resource Intensive

Increased load on your server resources could lead to an increased  site load time or pages crashing. Otherwise, the user experience of visitors to your site may be spoiled because certain elements in the site may not function as intended.

 

Large Sites Offer Their Own Problems

 

Backing up larger sites takes more time & more resources. In such cases it is possible that certain sites may not get backed up at all. This is because hosting services; especially on shared hosting, have policies about the time, and the server-resources that a particular task can take. In such cases although you may have employed a backup solution, your site may have not been backed up at all, or may have been backed up incompletely. In both cases, restoring the site is not possible.

 

Storage Space & Security

Frequent backups lead to multiple copies. Storing these copies securely can be a challenge. Storing backups on your own Dropbox accounts or local storage devices like your PC’s hard drive (HDD) or USB drive is not recommended.

Backups stored locally can become infected with malware as you are constantly browsing and downloading files. Also, HDDS or USB drives have been known to crash. This doesn’t even account for the risks associated with accidents and natural disasters.

Storage may drive up the cost of storing backups as you may have to invest in independent storage solutions.
In all the above cases the real risk is that eventually when you need to restore your site you may not have backups, have incomplete or infected backup files. This is not the optimal scenario for your business. Probably a good way to evaluate a backup solution is to list some scenarios in which you would need to rely on backups, and see if the backup solution in question will give you access to backups and allow you to restore your WordPress site.

 

The Answer?: Backup Service as a Solution

A WordPress backup service like BlogVault will not only take care of storage space and security but make incremental backups. This intelligent approach ensures that even large sites on shared hosting can be completely backed up. Apart from this backups services may also eliminate cache and log files from backups, thereby reducing problems at the time of restores. All of this is done automatically, thereby eliminating the human errors so that you can go about your business without worry.

 

With a WordPress backup service restoring your site is always the goal. When the time comes you will have multiple backups versions; securely stored, from which you can choose. You can also automatically restore your site with a single-click. Of, course a backup service comes with a more premium price tag but with the price you’ll have backups with best practices at your disposal.

 

Storing WordPress backups on your PC can quickly become laborious and the risks outweigh the convenience or economic benefits. Find out why.

Locally storing your WordPress backups means storing them on your PC or desktop. The other option is maybe to store them in an external storage device like a USB drive or or an external HDD/SSD.

 

Saving backups of your WordPress site to your computer seems convenient, but how reliable is it?
Saving backups of your WordPress site to your computer seems convenient, but how reliable is it?

 

In this article let us look at how you can do it, why you may be looking at this option and also answer the question which matters the most– should you do it?

How To Make WordPress Backups Locally

There are 3 ways through which you can download backups to your computer:

  • Manual WordPress Backup Download
  • WordPress Backup Download via cPanel
  • Plugins

 

Manual WordPress Backup Downloads

You can download WordPress files by using an FTP client— eg: FileZilla, CyberDuck. Making a full backup includes backing up files as well as your WordPress site database. To make WordPress database backups you can use phpMyAdmin.

However, once you download your backup files, labeling and organizing them is important. Otherwise it may be impossible to find the desired version when you want to make a restore.

cPanel

Usually web hosts provide a cPanel account to users. Using the tools in cPanel– Create Backup or Backup Wizard, you can download backups. Again these backups are usually .zip files with filenames containing date names. However, that is not enough information when you make regular backups. You may have to spend more time organizing your backups with descriptions to ensure restores are easy.

Plugins

Most WordPress backup plugins; at least all the popular ones, offer the option to download WordPress backups to your computer. However, regardless of the WordPress backup plugin you use, downloadable backup files; especially of the full site, are available in .zip format when you download a full WordPress site backup. On top of that not all plugins give you the option to download individual files. This means we are back to our recurring theme of how downloading and storing backups also means maintaining them.

Storing WordPress Backups Locally

There are some key concerns when thinking of destinations for WordPress backups.

  • Storage space
  • Security
  • Organization
  • Restoration Issues
  • Ease of use

An ideal WordPress backup solution addresses all of these concerns.

Pros and Cons of Storing WordPress Backups Locally

Storage Space

Backups must be made regularly; daily if possible. If you are making regular backups then storage space will become a concern for you. Your PC’s internal HDD will eventually run out. You can solve the problem by investing in an external HDD/SSD, or USB drives dedicated for storing your backups; especially if you have large sites and you make regular backups. If you use USB drives for example you may be forced to make backups once in awhile and and overwrite previous copies. This is not a good solution.

Security of WordPress Backups

Making a backup is a security measure. Which means your backups must be secure. However, storing them on your PC or on a storage device is not the best idea when considering the security of backups.

Malware

Backups stored on a PC may be infected with malware from a few sources. They may either already be on your computer, or your browser may have been infected by a malware from an unsafe site, or your backup files may be corrupted by malware in external storage devices like USB drives or HDD/SSD.

Storage Location

Apart from malware issues, there is the concern of where your backups are stored. Even if you have a dedicated external storage device– HDD/SSD, it may not be enough as they are not reliable. They do have failure rates, and may crash or be infected with malware as they have to connect to your computer at some point. HDDs/SSDs may also stop working due to heat or natural wear and tear. Along with all of these points, if you choose to store backups locally on a hard drive, then your backups are in a single location, this raises the risk of losing them significantly. As a result, they may not serve as the most secure environment for storing your backups.

Organization

Downloaded backups have to be organized if they have to be useful when you have to restore your WordPress site. Consider that your site is down and you have to restore it. If you are left going through all your backup versions one by one trying to make the right decision, then you might spend a lot of time and effort which you could have invested in developing your business ideas.

Restoration Issues

Manual downloads or locally stored backups usually mean manual restores too. This may suit some developers or those who have spent time working on WordPress but for the majority who are business owners, or bloggers who are utilizing the CMS, this may not be a viable option.

Restorations usually have to be done via your cPanel account or via an FTP Client and phpMyAdmin. There are often limits to the size of files that can be uploaded via cPanel or PHPMyAdmin. These restrictions can cause restores to fail. Again, the lack of backup descriptions, and easy options to make restores, together make extra demands of your time and energy. Expending this extra effort may be unnecessary if you utilize a complete WordPress backup service.

Ease of Use

First of all since this is a manual process. If you are following best practices than you have to make backups daily. This can get tiring, and worse, you may forget to make backups at all.

After taking all of the above points into consideration, the answer to this one seems to be clear. Storing WordPress backups locally doesn’t seem to be a great idea. However, there may be a couple of benefits. It is an economical option, and you can be sure that backups are done as making manual backups or downloading them from plugins allows you to keep track  of your backups.

However, even in these cases, you may end up spending on storage devices, or professional help when you need to restore.  Along with those issues, if you account for the time spent doing the work— making, downloading, organizing, and maintaining backups; and the time spent worrying about their safety, then the economical benefits and surety about backups being done seem to be nullified.

Instead choose a professional WordPress backup service like BlogVault, for worry free backups so you can do what you do best.  A premium WordPress backup service  would allow you to easily track backups, makes one-click WordPress restores, and even one-click WordPress migrations; leaving you worry free.

 

Making WordPress Backup to Dropbox seems like an attractive option due to ease of use & low cost.  However, is it the best practice, & will restores be as easy as backups?

Uploading WordPress Backups to Dropbox?

Dropbox is a free, and accessible storage solution for WordPress backups. Storing backups on Dropbox also means that your backups are offsite. This is very important. These points along with the fact that many plugins now exist which upload your backups to Dropbox automatically. This combination of points makes Dropbox a popular option for storing WordPress backups.

 

Dropbox is a popular option for storing off-site WordPress backups
Dropbox is a popular option for storing off-site WordPress backups

 

Plugins to Upload WordPress Backups to Dropbox

Backup & Restore Dropbox, Dropbox Backup by Supsystic, and WordPress Backup to Dropbox are all plugins which backup to Dropbox. BlogVault also supports backups to Dropbox.

Other plugins like Backup Guard, and UpdraftPlus WordPress Backup Plugin provide Dropbox as one of the optional destinations for backing up. IN the case of the the former the option is available only in the PRO version, where as in the case of the later it is an add on.

Advantages of Dropbox Backups

The process is simple. You will need to input your Dropbox login credentials, confirm them and you are done. Some plugins will regularly backup your WordPress site to Dropbox according to the schedule you have set. Tracking this may be another matter altogether.

Apart from the simple process, cost is another factor which  makes Dropbox a seemingly attractive option for backups. Some plugins which allow you to backup your WordPress site to Dropbox are free. Dropbox itself is free upto 2GB so you may feel there are no extra costs with this option.

WordPress Backup to DropBox: Think again!

In order to backup up your WordPress site to Dropbox, plugins will need to store a copy of  your Dropbox account’s API key on the site itself. This means that you are keeping a spare key to your backups on your site. What is the point of leaving a copy of your bank vault’s key in your living room? You might as well have left your valuables in the living room too, right?

Backing up to Dropbox is indeed simple enough. Our WordPress backup plugin offers users the option to upload backup to Dropbox too. Users who know a particular version to be without any problems can download the backup to their Dropbox account. This is not a default option when you use the BlogVault plugin and regular backups are not made to your Dropbox account. We do this because we follow best practices for WordPress backups. Know more about why backups to Dropbox is not safe.

However, if you’re relying on Dropbox only to provide the safety net for your WordPress site then you are in trouble, at least according to our experience.

Dropbox Backups & Restores

Apart from all of these points, there is another issue to making WordPress backup to Dropbox only- restores. After all the entire point of making backups is to empower us when we need to restore our business or blog.

Most WordPress backup plugins zip your files; meaning they download your site in .zip or .gz files. You cannot view .zip or .gz files in Dropbox anyway and you have to download the files to sort them out. In this case Dropbox becomes a temporary storage solution rather than a comprehensive backup solution.

Seemingly simple matters like clutter. Regularly backing up to Dropbox clutters your account. You may not be able to find the files you desire quickly, when you need them. When you have to restore your site, you don’t want to sift through thousands, if not millions, of files.
Tip:
When backing up to Dropbox, ensure that you label the downloaded backups in an organized manner so that you know can categorize different backups. This will be helpful when you have to restore your site.
You need to safeguard your data in a more robust manner to ensure that in your hour of need you know not only know that you have access to backups but also that they are functional. Especially, if you’re running a small business or a popular blog then you might want to look at a more holistic solution for backup and continue making WordPress backup to Dropbox only as an additional step.

Making WordPress Backup to your Google Drive account may mean that you are choosing convenience over efficiency and security. Here’s why.
 

Uploading WordPress Backups to Google Drive

Google Drive presents a convenient option. To begin with it is accessed with your Google account. No multiple logins. Added to this 15 GB of storage space is free to users.

 

Google Drive seems like the perfect vault to store your WordPress backups in.
Google Drive seems like the perfect vault to store your WordPress backups in.

 

You can simply choose among the many plugins which allow you to upload your WordPress backups to Google Drive. UpdraftPlus, BackupGuard, and WP Database Backup are all example of plugins in the WordPress repository which allow you to do just this. However keep in mind in some cases, you may have to pay for an add-on to add Google Drive to your list of backup destinations.
 

Setting Up Google Drive with Your WordPress Backup Plugin

This process may take some steps to get through, but if you follow the documentation of the respective plugins it will be easy. However, the point to keep in mind is that setting up your Drive account with your backup plugins generally means that the plugin stores a ‘client ID’ and ‘client secret’ to your Drive account. This is how the plugin can upload backups to your Drive account. However, this can be a double-edged sword.
 

WordPress Backups to Google Drive: Pros & Cons

Google Drive gives users 15 GB of free storage space. This may prove sufficient if your site is not  large. The economic benefits from using a free plugin and having free storage space cannot be discounted without consideration. Along with this, you can gain access to your Drive account with your Google credentials; no extra logins required.

However, the very same advantages have another face when viewed from the perspective of control, efficiency, and security.
 

WordPress Restores from Google Drive

All backups are about restores. This means making restores must be easy and it must give full control. Backup files uploaded to Google Drive by plugins may not allow for this. It is true that with plugins like UpdraftPlus you can restore directly from your WordPress admin dashboard. However, this may not be enough.

Backups uploaded to Drive are usually in .zip folder; and that makes it very hard for you to find and restore individual files. This is, if your plugin allows for restoration of individual files; which is not always the case.

Restoring individual files has its benefits. Large sites take time to restore. This means more downtime. In other cases your hosting service may limit the time for each action. This is true of most cases, and in such cases your website may have to be manually restored. This is not a burden your business needs.  On the other hand, restoring individual files means that you can avoid all these complications and not suffer the cost from unnecessary downtime. With each passing day this cost continues to increase. For this reason, having more granular control over your backups and restores is important.
 

Are your Backups Secure in Your Google Drive Account?

The other point to consider is that your backups may be vulnerable because a single set of login credentials gives you access to all your accounts. If that is compromised then your backups may be compromised too. The other way is that if your WordPress site is hacked, then that may lead the hackers to your backups since your plugin stored the ‘client ID’ and ‘client secret’ to Drive account.
 

WordPress Backup to Google Drive: Storage Space Issues

In the case that your Google Drive account runs out of space, how will your plugin continue to make backups? You may want to know if you’ll get notifications from the developers of the  backup plugin you use. If this is not the case, then you may not have backups to make restore; which is when you need them the most

While convenience is one factor, uploading your WordPress backups to your Google Drive account may not allow you to practice WordPress backup best practices.
 

No Backup Descriptions

Now let us say that you are following good login practices, using smart passphrases, and following the basic security practices well. You also don’t mind making manual restores. In such a case you may be okay with a plugin which uploads your WordPress backups to your Google Drive account. While this not advisable from a security standpoint, you may still have to contend with another issue– backup descriptions.

As mentioned plugins usually upload your WordPress files in .zip files. The file names may have the date and time when the backups were made but not much else. When you want to manually restore a file you may want a description of what has changed from one backup version to the next. Without this, you may spend a considerable amount of time sifting through files, or spend time organizing backups in your Drive. Either way, you have to invest a considerable amount time and labor.

Tip:
When backing up to Google Drive, ensure that you label the downloaded backups in an organized manner, so you can categorize and differentiate backups. This will be helpful when you have to restore your site.
 

You need to safeguard your data in a more robust manner to ensure that in your hour of need you know not only know that you have access to backups but also that they are functional. Especially, if you’re running a small business or a popular blog then you might want to look at a more complete WordPress backup solution and continue making WordPress backup to Google Drive only as an additional step.