What You Should Takeaway from GitLab’s Database Deletion and Its Backups

Ruby 1 year ago

GitLab deleted the wrong database, but when ineffective backup solutions got added to the mix, the site’s system admins had to battle the perfect storm to get the site online. The takeaway from this situation? Choose your backup solutions carefully.

 

backup software
GitLab’s system admins battle the perfect storm of system errors and inefficient backups to get the site back online

 

GitLab, the online tech hub, is facing issues as a result of an accidental database deletion that happened in the wee hours of last night. A tired, frustrated system administrator thought that deleting a database would solve the lag-related issues that had cropped up… only to discover too late that he’d executed the command for the wrong database.

What Went Wrong with GitLabs’ Backups

While the horror of the incident might have been mitigated by the fact that GitLab had not one but five backup methods in place, the problem was that all  of them were discovered to be ineffective. Here’s a quick run-through of the different backup methods GitLab had, and what went wrong with each of them:

  • The LVM snapshot backup wasn’t up-to-date– the last snapshot was manually created by the system admin 6 hours before the database deletion.
  • The backup furnished on a staging environment was not functional– it automatically had the webhooks removed, and the replication process from this source wasn’t trustworthy since it was prone to errors.
  • Their automatic backup solution was storing backups in an unknown location, and to top it, it seemed that older backups had been cleaned out.
  • Backups stored on Azure were incomplete: they only had data from the NFS server but not from the DB server
  • Another solution that was supposed to upload backups to Amazon S3 wasn’t working; so there were no backups in the bucket

 

As a result of these issues, the system admins are struggling to get the 6-hour old backup online. The progress of the data restoration has been closely followed by well-wishers, and many have appreciated the website’s transparency, especially under such duress.

 

How to Identify a Good Backup Solution

It’s certainly freaky that all the five backup solutions that GitLab had were ineffective, but this incident demonstrates that a number of things can go wrong with backups. The real aim for any backup solution, is to be able to restore data with ease… but simple oversights could render backup solutions useless. A backup software should be effective to handle all types of issues and provide enough security. This is why you should watch out for the following traits in any backup solution:

  1. Backup solutions should match your need
    In the case of GitLabs, automatic backups were made once in every 24 hours. Considering the amount of data being added every minute, however, real-time backups would have been perfect for them. While not being the best in terms of data-conservation, the last manual backup was performed by the system admin 6 hours before the crash, and so was the most viable option. Choosing the right backup solution for your need requires the consideration of the frequency of data-addition, the levels of user activity, and the server load.
  2. Backup solutions should allow easy, quick restoration
    The problem with GitLab’s backups stored on its staging environment, was that the replication process was difficult to manage. When you’re already burdened with the responsibility of getting your site back up, you shouldn’t be worrying about the restoration process.
  3. The backup solution should be completely independent of your site… in a known location
    In the GitLab situation, the problem was not knowing the backup destination. This isn’t a problem with WordPress backup solutions,since they usually store backups on your site’s server… or on a personal storage account (such as Dropbox, Drive or Amazon S3). However, this means most of the time, they either require you to access your crashed site for backups… or they store the API key to these accounts on your site (which poses its own problems). Both these options present Catch-22 situations of ‘site is down so need backups, can’t access backups because site is down’. It’s important for you to know all there is to know about your backup destinations.
  4. The backup solution should backup your entire site
    Backups that only contain part of your site (such as GitLabs’ Azure backups) aren’t really reliable when your site goes down. In the case of WordPress backups, some solutions might backup your site except for custom tables (such as those installed by WooCommerce), so you need to be wary of such situations.
  5. You should be able to easily test your backups
    The real problem with all the backup solutions GitLabs had, was that they hadn’t previously tested them… and hence had to give them a hard second look after encountering restoration-related problems. The real concern is that their backups weren’t discovered to be inefficient until they actually needed them. This is why testing backups should be a part of your backup strategy.

 

We’re all human at the end of the day, and the job of a systems admin, especially when overloaded with spam, can never be taken lightly. This is why backups exist– to have an easy ‘undo’ in case there ever is an error, and your site goes down, or data is lost.
We can only hope that things go well for the GitLab team, as they rush to get their data back.

GitLab’s status can be monitored via this Twitter feed. (When this article was published, 73% of the database copy had been made).

Should You Rely on WordPress Backups by Your Web Hosts?

Suhas 1 year ago

Making WordPress backups with your WordPress hosting service seems like a convenient option. Here’s what you should know about backing up with your web host(s) and why you shouldn’t do it.

Making WordPress backups with your web host may be an option you are considering or are currently following. The idea is instantly attractive as your web host also backs up your WordPress site(s).

However, have you considered why web hosts also provide backups? It is because backups are a basic necessity for most modern day WordPress sites.

Hosting a WordPress site– the act of choosing a host and a plan, may be simple, but maintaining a site and ensuring uptime and quality user experience for visitors to your WordPress site is more difficult. Many things can go wrong with your WordPress site.

 

Your web host's backup might not be as reliable as you think, and might not help you firefight when you really need to.
Your web host’s backup might not be as reliable as you think, and might not help you firefight when you really need to.

WordPress users know that everything from simple updates to hacking may crash your site or cause serious functionality issues. Having a backup can allow you to sort out the issues offline while your users continue to have a good experience and your reputation remains intact.

Running through the characteristics of the ideal WordPress backup solution is a good way to go when you have to evaluate any backup provider. Remember, backups are not for namesakes, you’ll need them at some point. This is true regardless of whether they are made by your web hosting service or not. Which is why backups must held to high standards in all cases.

In this case, let us look at a short checklist of the qualities to look for in a good backup solution:

  • Availability
  • Coverage
  • Frequency
  • Access
  • Storage

And, of course, it all comes down to

  • Restores

This should help you evaluate your backups for functionality, security and use-value.

 

Caveats in WordPress Backups by Web Hosts

While not all web hosts provide WordPress backups, many do. However, even with the ones that do offer backups, there are many caveats attached to the service. The quality of your WordPress backups truly depend on their practices and policies. Let us look at them point by point.

 

Availability of Backups

Some web hosts may offer backups to their basic accounts for an extra fee. However, backups may be included as part of the subscription plan for more advanced plans. SiteGround is a good example. They offers backup services for extra cost the subscribers of their most basic plan– StartUp, but more advanced plans have it included in the service..

When it comes to automatic WordPress backups you also need to be aware of your web host’s policies regarding website size limits. For example, HostGator will backup your WordPress site automatically, if it is less than 10 GB. If not, then automatic backups will not happen. You can only manually  backup your site via cPanel. The onus then, is on you to make, download, organize, and maintain backups. In such cases your backup solution needs to be revisited, because ideally backups must not be an additional responsibility, but must happen automatically.

 

Coverage: What is backed up?

Is your entire site being backed up? A WordPress site consists of files and database. An ideal database must make backups of it all but also give you access to it. This is not a given with all WordPress hosting service. Ask your web host about which parts of the site is backed up beforehand so that you may be prepared with manual backups or other measures when you need them for restores.

 

Frequency of Backups

There really cannot be a golden rule for how frequently you should make backups of your WordPress site. However the general guideline is— frequency of backups = frequency of changes to site. Backups must be done once a day. This will ensure that changes are recorded, and loss of data is minimized in case of a restore. This too is not an ironclad rule. e-Commerce sites may need to backed up more frequently (real-time backups).

Web hosts making WordPress backups may not make backups daily. For example, HostGator makes backups but stores only one copy and overwrites it each time another backup is made; which is only done weekly. This may result in loss of changes and updates.

On the other hand, WP Engine and FlyWheel make daily backups and maintain multiple versions of WordPress backups, but this upgrade in the quality/quantity of backups is also reflected in the price.

 

Access to WordPress Backups

This may seem like a straightforward point but it is not. For example, you can make and access backups with the Create Backup & Backup Wizard tool in cPanel when you have HostGator account. Even though SiteGround does not have a backup service for their most basic plan– StartUp, their site literature mentions that they maintain a backup of all the sites hosted with them. However, this is not accessible to users through the cPanel. In fact, this backup copy isn’t meant for users at all but for technical experts of SiteGround. You may request for this during emergencies, but you cannot be sure of how old this backup maybe. Of course, SiteGround offers Softaculous in its cPanel which can be used to make backups and can also be accessed via your SiteGround cPanel account.

Other web host like Flywheel and WP Engine allow you to access backups through their own dashboard.

 

Storage Backups – Backups are Not Independent

Storage of your WordPress backups is crucial to the security of your backups. The ultimate purpose of backups is restorations. If backups are not securely stored then you may not have them at all to restore your WordPress site in case of emergencies.

Your Web Host Is Not the Ideal Destination for Your WordPress backups

Backups are meant to be your safety net in case something goes wrong with your WordPress site; which can happen for many reasons. If your backups are stored by your web host on your site’s server, then your backups may not serve that purpose. The short version of the explanation for this point is that if your backups if they are stored on your server by your web host, then they are exposed to the same threats as your WordPress site.

Generally your backups may be stored on the same server or in a different location altogether, like an Amazon S3 account. In either of these cases your WordPress backups are not independent of your WordPress hosting service. This means that if you web host is affected for any reason then along with your website, your backups may also be lost.

Even WordPress Hosts Get Hacked

In case your site or server is hacked then you may make the case that your web host stores backups in a completely different location. However, consider a scenario where your web host has been hacked; and this has been known to happen in the past even to the most reputed of hosting services… In such a case, none of the data that belongs to your web host, regardless of location of the infrastructure, is safe.

Natural Disaster

Your WordPress backups must also be your disaster recovery plan. If your web host is affected by a natural disaster and your backups are on their servers, then your backups will be inaccessible.

Backups must be Independent

What this means is that you should be able to access your backups without depending on your web host. In such a case you can always restore your site using your backups no matter what the condition of your web host. This also allows you to easily migrate your site to a new hosting service too, without worrying about the quality of the backup. This is why completely independent backups are needed.

 

Restoring with WordPress Backups from ‘My’ Web Hosts

We can’t stress this enough— backups are about restores. Restoring a WordPress backup must allow for all the same features that you would demand of any other premium backup tool which is considered to be a good experience. The first step to this, is of course ensuring that you have backups from which to restore your website; but as we mentioned, backups with your web host are not independent so this is not a given.

Ways to Restore

cPanel / Tools

One of the way restores can be done, is by using the Backup Wizard tool in cPanel. Generally you cannot restore a Full site backup through the cPanel tools. For this you’ll need to contact your web host’s support. The other way is, if your web host uses a tool like Softaculous like SiteGround does, then you can use that to restore from your WordPress backups.

Auto-restore

Web hosts like WP Engine and Flywheel allow for one-click restores. However, the one problem with this is that there are no descriptions. Although there are dates of when the backups were made, you cannot really track the changes to your site from the last backup.

Differential Restores

Differential restores will not wipe the data on your site but only restore those files from your backup that are not already on your site. This way if the newer posts/files/updates are on your site then they will continue to do.

Most if not all web hosts, wipe the data on your website before restoring from a backup. There will always be a time difference between when a backup was made and when it was restored. This difference may lead to loss of data, since differential restores are not possible with web hosts’ offerings.

Granular Control

Granular control is important since it allows you to restore only a faulty database table or a specific part of your site’s content. In case you downloaded the full site backup, then it is upto you to find the specific table you want to restore. Apart from that downloading or uploading individual WordPress files may be hard, especially for new users because, all backups are .zip files.

Other web hosts like Flywheel and WP Engine, although they offer one-click restores, do not describe the backup versions or allow for restoring individual files or tables. If you want to do this you may have to download a backups version in .zip folder. Extract and choose the files and upload them via an FTP client.

This is obviously not suitable for every circumstance. If you can pinpoint the source of the issue–like a recent update you made to a plugin, you need to restore may that one particular file and not have to spend time restoring the whole site as this can take some time especially if you have a large site.

Test Restores

Backups must be tested before being restored to ensure that they are fully functional. You do not want to find out what may be wrong with your backups once you have restored it on the live site. You may use the staging environment provided by your web host for this. However, if you are a novice, or are not a developer, then this might be difficult for you.

You can check out BlogVault’s Test Restore feature which you can access with a single click from your BlogVault dashboard. This creates a fully functional copy of your site from the backup version you choose. This way you can navigate the copy just like you would your actual site, make sure everything is ticking correctly and then make the restore; all within a matter of minutes.

 

WordPress Backups by Web Hosts Bring Other Worries Too

We have covered how backups by web hosts are not independent. This is important because if you don’t have backups then there is nothing to talk about. However, apart from that glaring miss, there are other big and small worries to which you may have to pay attention.

With automatic backups by web hosts you can’t schedule backups or force backups. There are no backup descriptions (as offered by best-in-class premium WordPress backup plugins like BlogVault). This make organizing backups very difficult.

Also, tracking backups are difficult since you have to login to the cPanel every time to track automatic backups and even to make manual backups. cPanel itself can be a little cluttered and provide an overwhelming experience for new users. The tracking issue may become important to you if your web host has limits on your website size to make backups.

 

WordPress Backups by Web Hosts: The final word

If your backups are not independent, then they don’t fall under the category of ‘following best practices’. So, we cannot recommend this solution it thoroughly. Some web hosts may offer better backup options than others but these options will come at a cost to you. Now that you all the things to consider about backups by web hosts, choose wisely.

 

What Can Go Wrong With Your WordPress Site?

Suhas 1 year ago

While it is easy to be online with a WordPress site, the real task starts after you are online. Do you know all the things that go wrong with your WordPress site? Read on to find out.

Every person wanting to start a blog or a small business has heard the words “you can be online in just 5 minutes”. This is true and this is what makes WordPress popular. However, very few people realize that owning a self-hosted WordPress site is the beginning. There are many things that could go wrong with your site… Right from accidentally deleting files, posts or plugins to a bunch of problems with your hosting provider.

 

A number of things could go wrong with your WordPress site

 

A WordPress site and its web host need to fit well together. Finding the best for your WordPress site might take some trial and error. Even if you do find the option with the least worries there are still many issues you can run into. The key lies in knowing what the potential issues are and finding answers to as many questions as possible from the start.  This is a list of many possible things that can go wrong with your WordPress site.

 

WordPress Host Hardware Issues

The hardware in a web host is one of the most common problems to arise. Everything from overworked hard disks, power surges, heating issues to natural disasters and accidents can cause hardware failures.

Usually, hard disks are said to be the hardware component to fail most frequently. It is not surprising because most hard disks (which are HDDs) rely on moving mechanical parts. This increases not only the probability of wear and tear, but also heating due to friction, and the rate of failure. This is true when compared to the alternative to HDD, the SSD. There are no moving parts, they are silent and reduce chances of heating too, but SSD cards are more expensive and have a high failure rate too.

Heating issues are generally exacerbated by outdated hardware or when there is insufficient cooling infrastructure. On the flip side, if a hosting provider stuffs a room with servers then the cooling infrastructure might prove to be inadequate, automatically heating the hardware as well as the environment. This increases the failure rate in hardware and more likely heating causes performance lags in servers and in turn in your WordPress site.

Something you may not pay attention to, is the location of your web host’s infrastructure and how prone that location is to natural disasters. If your web host is in a location that is prone to flooding, earthquakes or tornadoes then you might want to ask them about the preparations they have made in case of such eventualities. Even cases of heavy storms, lightning has hit data centres causing damage.

Not just natural disasters, even accidents can cause unexpected trouble, such as the freak accident in which an SUV crashed into a building knocking out the power generator of a data centre.

 

Your WordPress Site Is Hacked

WordPress not only the dominant entity in the CMS market now, it is also the fastest growing CMS too. This means that WordPress is big and here to stay for the foreseeable future. This popularity provides hackers a large target.

WordPress is open source software, dependent on plugins and themes and popular. All these points contribute to the CMS being a popular target of hackers.

While vulnerabilities on WordPress core are patched quickly, the security through transparency model means that anyone keeping tabs of WP news knows which vulnerabilities were found, where they were found and what is the patch. This system is just part of the deal when dealing with the open source platform- WordPress.

WordPress, because it depends on plugins and themes to make it extensible is also in an unique position because one of its biggest strengths is also the source of most of its vulnerabilities.

Remember, modern-day hackers are not targeting sites but have bots crawling the net searching for vulnerabilities. If you are not practising basic security practices like updating everything then your WordPress site is at risk.

 

Hosting Provider Issues

While creating a WordPress site may be easy, hosting it can bring up many complications. This is especially true for WordPress sites on shared hosting. On shared hosting, your server might be overloaded if your hosting provider hosts too many sites on your server affecting the performance of your site.

Apart from site performance and uptime you also have to worry about the name server going down, again your hosting provider getting hacked, your account being suspended by your hosting provider, or your hosting provider is going out of business.

 

Natural Disasters & Accidents

Hosting providers even today are affected by natural disasters and accidents. While your web host’s infrastructure may be built with disasters such as earthquakes, floods and tornadoes in mind, it might not be true for all data centres. The best defence, of course, is to ensure that data centres are not built in such locations. However, this is not always possible in the 21st century. The next best option is to be prepared.

This equally true for accidents too. Not only can accidents cause significant damage to your web host, they can also impose significant financial losses to both your web host and you as a WordPress site owner.

The cost of downtime is going up all the time because it not only means the accountable loss in transactions for e-commerce sites but also the more qualitative measure of visitors’ perception of credibility. If not as serious then you could simply lose visitors because there is no destination for them to see and with which to engage.

It is best to plan for a WordPress backup solution that is truly a disaster recovery plan. This means not only reduce or eliminate dependability on your web hosting service, their infrastructure or backups but also protecting your WordPress site from damages caused due to weather which may affect your web host.

 

Software Issues

WordPress is of course an open-source CMS which is extremely popular. This also means that a large number of novices are developing for/on it. Such processes make WordPress extensible and contribute to it is popularity, but also expose it to exploits.

However, along with security scares, bad code on WordPress themes and plugins cause the following compatibility and performance issues:

  • Compatibility with WordPress
  • Compatibility with the theme
  • Compatibility with other plugins installed on the site
  • Proliferation of plugins
    • Security concerns
    • Performance lag

Apart from all these issues, bad code might lead to the dreaded the ‘White Screen of Death’ too. Updating plugins and themes with bad code is one of the reasons for this to occur.

Updating WordPress Plugins & Themes

This means that updating, which is a necessary security step, becomes a serious concern for WordPress site owners. The site may stop being functional and depending on the seriousness of the issue availability of redundancies, your site could be down for hours.

In such cases you have few options that might ease your burden:

  • To start off with the basics making WordPress backups must be the first step of updating your themes & plugins
  • If you’re using a backup service that allows you to test your backups before you restore, then you can you can even use it to test updates before making changes to your live site.
  • Also, in case you make updates to the live site and it doesn’t work out for you, then you can simply restore a backup. This saves time that might have been wasted in figuring out which plugin is at fault for taking your site down.

 

Human Errors

With a self-hosted WordPress site, human errors can occur from two ends- you the WordPress site owner, or the web hosting company.

Site owners

Accidental file deletions

As a site owner, you may delete files, plugins, or even posts. Recovering these may be a difficult job if you do not have them backed up because not all web hosts make WordPress backups and among those that do, not all do it on a daily basis.

Not Renewing Hosting Contract

This seems like a simple enough point and in the modern world with email reminders, it seems like a point that shouldn’t be in this section but it happens often enough for us to not mention it.  In this case, you must know what your web hosting company’s policy is, regarding your data.

Hosting Providers

Accidental file deletions, or rebooting the system has been reported often enough now for it to be part of our checklist to test the efficacy of a given WordPress backup plan. Unlike individual site owners, when a hosting provider runs a script deleting a file or reboots a section of the data centre the scale of the consequence is much bigger. Don’t get me wrong, I don’t mean to underestimate the damage of a single business site losing all its customer and transactions related data. However, generally, errors by hosting providers tend to have a bigger effect in terms of scale than a single WordPress user deleting a post on their site.

 

Data Center Issues

A data centre can be divided into four parts:

  • Building shell
  • IT equipment
  • Electrical Infrastructure
  • Mechanical Infrastructure – Cooling infrastructure

A data centre may face the issue in each of these four sections/parts. Apart from this your data can be threatened when your WordPress hosting service’s data centre itself is hacked or hit by a natural disaster.

The building shell is obviously the first line of defence. It can regulate access and keep the inside equipment safe. The IT equipment is the very business of the data centres – this refers to the servers, storage and communication equipment. Servers and storage can fail either due to wear and tear, heating or power surges, among other causes.

Communication equipment like cables and switches is not easily visualized generally. A single cable not connected properly or knocked off during maintenance can cause a lot of grief. The same can be said of uplink failures, or when network switches fail or undersea cables get cut. A case when a network switch failed and took down four popular web hosting companies, is a good example of how of such issues cause serious enough damage for you consider them a threat to your WordPress site’s uptime.

We mentioned the importance of electrical infrastructure in the previous section. Equally important and closely connected to the electrical infrastructure is the cooling equipment and all the other non-IT equipment that the electricity powers.

If A Data Center Is Hacked?

If a data centre is hacked then your data may be compromised. What is not obvious is that you may not always lose your data to the hacker. There have also been cases when data centres have gone out business because of a single hack. This means even if your site may not be directly compromised, you might still have to find ways to secure your data.

The point to remember is that your data- your website and your backups are at risk even if your site/server is not hacked. Which is why you must have backups which are completely independent of your web host’s data centre.

Power Failures in Data Centers

The power supply is the cornerstone of a good web hosting. If there is adequate and constant power supply is then it powers not only the servers but all the other equipment required to keep the web host running- air handlers/cooling/heating/ventilation, lighting, UPS system and generators, fire suppression systems, alarm systems. Needless to say, a reliable web host must have adequate power backup which is tested and functional. If backups fall short then you might be looking at frequent downtimes which may add up to costing you a significant amount. Asking about your host’s power backup system may be an important factor in your decision-making process when the time comes to choose a web host.

Bad hardware— outdated power backup systems, lack of maintenance, and lack of testing for power failure are all part of reasons why a data centre may experience power outages.

 

Completely Independent WordPress Backups

It is obvious to think— “I have backups. My hosting provider does it for free! I’m safe.” This along with the addition of a moderate financial burden turns most people away from backups. However, ask yourself this— Can I access my WordPress backups when every single point mentioned above does go wrong? If not, then your WordPress backup is not a disaster recovery plan. It is as simple as that. The reason for this is that the functionality and security of your backups are dependent on your web host.

All WordPress backups have one purpose, WordPress restores. For this, you might want to rely on a comprehensive WordPress backup service which is all about restores, BlogVault.

 

Does Your WordPress Host Address These Data Centre Pain Points Effectively?

Suhas 2 years ago

A data centre is a complex entity in WordPress hosting. Do you know the different parts of a data centre, what can go wrong in each of those parts, and how it can affect your WordPress site? Find out.

Many factors in different parts of a data centre and its operations affect the performance of your WordPress sites. This could be due to a number of factors from simple hardware failures, to a breakdown in the power supply.

Different parts of a data center and its operations affect the performance of WordPress sites

Breaking a data centre down broadly will help us to understand these issues, and what can go wrong, in a clear manner.

Parts Of A Data Center

  • Building Shell
  • IT Equipment
  • Electrical Infrastructure
  • Mechanical/Cooling Infrastructure

Operational & Other Issues

  • Human Errors
  • Hacks
  • Natural Disasters & Accident

What Can Go Wrong In Different Parts Of A Data Center?

Building Shell

Generally, little thought is given to the structure which houses the servers and all its accompanying equipment because its layout and design is the first line of defence against any errors. Right from setting up the perimeter as well as the first line of defence, to determining the amount of equipment that can reasonably be stocked in any place the layout of the building is the definitive factor.

The building and how the layout is designed within it can also effectively implement access control protection in the form of magnetic strip cards, registry, etc. These points are crucial to ensuring that your WordPress site is secure.

Access control must be a concern for WordPress site owners looking for hosting services. Otherwise, slip-ups like the one that occurred at Joyent (the case when an operator error rebooted the entire section of compute nodes simultaneously), will be a serious issue with which to contend.

Mistakes are bound to happen even when all the checks are in place because there will human, software or hardware errors. It is just that there are ways to reduce the frequency of such errors. However, you cannot always plan for accidents.

A driver in an SUV fell unconscious, and the vehicle accelerated towards the end of the road, hit a curb going aerial and damaged the wall of building knocking out the generator inside it. The building was owned by Rackspace and as result of the accident, clients had to experience hours of unexpected downtime.

IT Equipment

 

This refers to

  • Servers
  • Storage
  • Communications equipment

Servers

A host of hardware, software and operational issues can cause server failures. Hardware issues usually occur due to overheating, power surges and physical damage caused due to accidents or natural disasters. Software issues occur overtime if there is lack of maintenance or due to malware or viruses. Even if the equipment is not completely damaged such issues can cause your site to lag, delay your site load times, or your site pages may not load at all.

Storage

Hard disks have failure rates and along with heat, natural wear and tear, and power surges all lead to failure. This is true of all hardware equipment in data centres.

Communication Equipment

Communication equipment like a network switch failing can cause serious outages even though it is not an aspect of web hosting we pay much attention to.

Web hosting businesses are facing increasing demands to remain competitive and keep the prices down. At the same time, there is consolidation with a single company owning many brands of web hosting under it. So, downtime from a network switch failure can have a ripple effect, and can affect multiple hosts at the same time.

It is best to diversify your backups in multiple locations to avoid being caught by surprise when facing such situations.

Electrical Infrastructure

While the IT equipment represents the business of the data centre, electrical infrastructure is what allows it function. Electrical infrastructure refers to the power supply and power backup equipment. Much of the claims that data centres make regarding uptimes and site performance depend on the uninterrupted power supply. This means having effective and adequate power backups is crucial.

For a WordPress site owner, this information could help decide the hosting service to host their site on.

Power failures occur when the backup equipment is not tested- if the batteries are functioning and charged, if the power backup system kicks in immediately, etc. Otherwise, sites might go down unexpectedly leading to losses.

Mechanical Infrastructure

Mechanical infrastructure helps regulate the temperature and this plays a crucial role in site performance and determines how dependable your hosting service is. Unregulated temperature can have a serious impact on your site performance.

The rise in temperature can also occur when too many sites are hosted on servers. This overworks the cooling equipment in the data centre, and as a result, fans may fail and exacerbate the problem.

Asking your web host about the access control, power backup and cooling they have could be crucial to know the estimating site’s uptime and performance; especially if you have a large site with many media files.

WordPress Backups Are A Necessity

Apart from this WordPress hosting services face the usual problem of hacking. In this case, even if the vulnerability exploited was not on your site but your data centre is hacked affecting your site, then you could not only lose your site but your WordPress backups as well as any personal/sensitive information which may be stored on your site. Sometimes such losses are irreparable. Not simply because of the impact of the hack which itself may be severe but hacks have forced data centres our of business entirely. In such cases, you may not be able to recover your data at all.

While there are many specialized WordPress hosting services available and the number is growing, it is important that you ensure that your site’s backups are not stored on web host’s servers or equipment. That way you can access your backup even in the case of any such failure. This is simply a good way to make WordPress backups and increase redundancy.

WordPress backups are not a luxury but a necessity. While hosting service have gotten more efficient demand and competition has also grown. This especially true for WordPress hosting. With the growth of WordPress, the number of hackers targeting the platform has also grown. Added to these familiar threats, data centres continue to be affected by natural disasters and accidents.

It may be important to know where the data centres of your WordPress hosting service are located and how prone those locations are to natural disasters. In such cases, you may also want to ask your hosting service the kind of preparations they have in place in case of such eventualities.

Now that you know broadly all the pain points of a data centre and how it can affect your site, opt for a WordPress backup service like BlogVault which secures your backups and diversifies their location effectively. After all, redundancies are useless if they are exposed to the same danger to which your WordPress site is exposed.

Common Mistakes Which Can Crash WordPress Sites

Suhas 2 years ago

We worry so much about performance of servers, PCs and other equipment, that we often forget that human errors have the potential to cause massive damage to WordPress sites.

Owning a self hosted WordPress site is tricky for many reasons. Some of the more entertaining points have to do with human errors. However, they can be just as damaging as data center issues, hacks or natural disasters.

There are many things that can go wrong in data centers due to human intervention, like loosely connected cables after maintenance or other operational issues. However, let us focus on a couple of common and simple ones that are also terrifying to all WordPress users. We have all done it and experienced the cost of it– accidentally deleting files, posts, plugins, users and their content, etc.

 

Human errors can lead to devastating consequences

 

Human errors can occur from two ends- one from system users using the WordPress sites; like admins, editors, and the other from WordPress hosting providers. With both, a common problem which may occur is that of accidental file deletions.

Human Errors by Users of WordPress Sites

Accidental Deletions

Accidentally deleting something on your site is scary. This may include posts, plugins, updates, users and may be even the entire WordPress site itself. When an admin deletes an user, they can also delete all the content authored by the user on the site.

Without backups these may be impossible to recover. We’ve all accidentally deleted files regardless of whether it was on a WordPress site or not. It is that the stakes are much higher when it happens on a live site. So always make backups after you add something to your site and before making updates or customizations.

Forgetting To Renew Hosting Contracts

Another error users make which often leads to WordPress sites going down, is forgetting to renew hosting contracts. This doesn’t seem like an obvious point but it happens often enough to merit a mention here. In such cases you will want to know what your hosting provider’s policy is regarding your data when the contract expires.

Some hosting providers keep data for a week others may store it for different periods of time. If you are too late in approaching your hosting service then you might lose your site as well as its backups.

Human Errors By WordPress Hosts

Accidental File Deletions

Human errors by WordPress hosts may mirror those by users like accidental file deletions but the scale of damage may be much greater. This was very evident when 123-reg knocked off most of their 1.7 million sites offline during a maintenance error.

The result of the error was that some users reported that they had lost access to all their data. A software company; a client of 123-reg at the time said that although the script deleted the servers and the websites of all their customers they could implement their disaster recovery plan because their websites were backed up. At the time, it wasn’t known if websites which were deleted would be restored. 123-reg’s website at the time told users that if they had local backups, then they should use it.

We know of other cases, such as that of  Joyent, when they rebooted all the servers in a section at once. The takeaway is that human errors are a part of most, if not all data centers. However, if you can have a disaster recovery plan that doesn’t rely on your web hosts then you can restore your site with ease and have very little downtime.

Emergency Power Off (EPO)

The Emergency Power Off button is generally used in case of fires or other emergencies to kill the power supply to the data center from a single point and contain the damage. Put in that context the EPO sounds like a good idea.

Guard Against Human Errors with WordPress Backups

However, when the EPO button is highly accessible it makes accidents inevitable. There have been cases of inspectors, delivery persons and even disgruntled employees pushing the button, and causing downtime. All this means that that there can serious unexpected power outages. In such cases, having completely independent WordPress backups with services like BlogVault makes this process easy and worry-free.

Would Your WordPress Site Be Safe in the Face of a Natural Disaster?

Ruby 2 years ago

Data centers have struggled to cope with natural disasters. Hurricane Sandy brought forth enterprising stories of efforts not in the disaster management plan. Do you know how your WordPress site is affected in a natural disaster and how your data center is affected by it?

You may wonder why we may be talking about an issue which may not affect your hosting service or your business as often as other hosting issues. However, not only is the damage to infrastructure, and power supply real, it may also be more severe than other minor glitches which may occur operationally. This means more downtime and more losses.

 

Natural disasters cause data centers much damage and downtime.
Natural disasters cause data centers much damage and downtime.

Research reveals that data centers are still affected by natural disasters with many not operating in conditions which could continue operations after a natural disaster. This means that you if your WordPress hosting provider’s infrastructure is compromised then your sites are too. If you backup with your hosting provider then you may just have lost all your data.

Natural Disasters

You know of the damage Hurricane Sandy wreaked once it reached land. It hit Manhattan, a region densely populated with data centers; and many of them were forced to rely on generators and fuel deliveries. However, others had to allow their data centers to shut down after the power backup was exhausted.

How One Data Center Avoided Power Outage during Hurricane Sandy

Peer1, had its center in Manhattan when Hurricane Sandy hit. However, reportedly, they could not rely only fuel deliveries to help them because their fuel pumps located in the basement were taken out when when water entered them. This made the fuel in their fuel tank inaccessible. Peer1 staff, volunteers and employees reportedly carried small quantities of fuel by hand, up 18 flights of stairs to keep a fuel tank on the floor filled. This meant that an entire night and morning for tiring work to keep the center up and running through the storm. Peer1 survived the storm with some improvised initiatives which were not part of their disaster recovery plan. Their action also meant that their clients did not have to experience outage.

However, this is not a case with all data centers. Hurricane Sandy took down another data center- DataGram, and websites like Gawker, Gizmodo, The Huffington Post and BuzzFeed experienced downtimes.

While the staff of Peer1 took extraordinary measures and were able to think on their feet, you cannot count on such measures to work every time. During Hurricane Sandy some data centers were submerged with the IT equipment severely damaged. Others, like in the Peer1 example protected their servers but were affected due to power supply being cut off. Natural disasters have a domino effect and can adversely impact you data center even though the infrastructure of the data center itself may not be directly harmed.

When data centers began reporting on the the damage caused by the floods some users were naturally disappointed about the level of preparation. If facilities are not designed to operate during floods or earthquakes then there is not a lot you can do in a couple of days or even a week to change that; especially when the intensity is dire.

Accidents

However, sometime even planning is not always helpful. Even if a data center is built with the natural elements in mind, it may not cover against accidents. No, we are not talking about human errors when people reboot or spill drinks or mix up cable connections; but more along the lines of a welding accident which led to an Amazon data center; which was not yet in service, to catch fire. Accidents like that happen and it is not something one can prepare for, although it may have been avoidable.

Not all cases are that clear cut though. It was reported that a person driving a SUV fell unconscious behind the wheel when his blood sugar level dropped. The vehicle continued to accelerate and crashed into a wall and knocking the generator inside it. It was the building Rackspace was using to house power generators for their data center. The unexpected power failure threw a spanner in the works for both Rackspace and their clients. As a result of the accident some users experienced hours of downtime. Rackspace ended up paying US $3.5 million in refunds, reportedly.

What This Teaches Us About WordPress Backups?

One client of Rackspace, following the incident, was quoted as saying, “We’ll work hard to further diversify our systems”. Perhaps this is the best lesson to take away from not only the above story but this article. A disaster recovery plan after all is meant for such circumstances. For all the above mentioned reasons, you’ll need to have a disaster recovery plan which is precisely that. A viable, rehearsed, and reliable plan which can recover your data in the case of a disaster that renders your hosting provider completely inaccessible. Having an effective and independent WordPress backup service like BlogVault protecting your data would be a worry-free solution.

Do you have a disaster recovery plan for your WordPress site?

Ruby 2 years ago

Why do you need it?

Can your business continue to function if you were to lose your data? If your answer is a clear no, then having a disaster recovery plan is a must for you. At some point down the road, your data is going to be in danger. It could be a machine error. It could be a simple human error. It could be a tornado the size of Nebraska. But sooner or later, you’re going to be in a situation where you’re at risk of losing some or all of your data. Some of the common consequences of a disaster –

  • Loss of business/customers
  • Loss of credibility/goodwill
  • Cash flow problems
  • Loss of operational data
  • Financial loss

90% of businesses that lose data from a disaster are forced to shut down within 2 years of the disaster. 50% of businesses experiencing a computer outage will be forced to shut within 5 years. (Source: London Chamber of Commerce). So, having a disaster recovery plan is the best insurance for your business and entire data. But what are the possible reasons behind this ‘disaster’? And how do you deal with them?

It's wise to have a recovery plan for your website
It’s wise to have a recovery plan for your WordPress site

What Can Go Wrong?

Hardware Failure

While we’ve made huge strides in terms of technology, it’s still not perfect. There are bound to be issues now and then. Hard disks, which are the most popular form of storage media, fail more often than you think. The statistical figure indicated is by no means trivial. Other forms of hardware failure can have a similar impact on your business.

Web-hosting Failure

As every site is hosted using one of the providers, a failure on their end undoubtedly spells disaster. Any sort of networking problem can bring down your site. However, this doesn’t pose a big threat to your data. But that’s not the end of it. These hosting providers are a common target of hackers. Once the server is compromised, the hackers have access to all the data that resides on it. The hackers can thus attack 1000s of site by hacking a single provider. Sometimes, hosting providers even suspend your account without prior notice.

Natural Calamities

Natural calamities, though rare, can pose a huge threat to your data. Hurricane Sandy, which hit New York City in 2012, had companies fighting hard to keep their data centers up. It was one of the busiest days for many of them.

WordPress Issues

WordPress, though WP core is known to be stable, has its own share of problems that crop up from time to time. The most common issue that users face is that of version incompatibility. Though WordPress versions are meant to be backward compatible, quite often, a WordPress update ends up breaking a plugin or theme due to incompatibility. Underlying API changes in a new version could also result in breaking parts of your site.

Plugin/ Theme Issues

WordPress is an open platform, inviting a lot of people to develop plugins and themes. Since each plugin and theme is written independently, not all of them follow the same set of coding guidelines and standards. This makes installing new themes and plugins on your site a risky proposition. A new addition may be incompatible with the underlying WordPress version. Some of the changes made by plugins and themes are –

  • Bad database changes
  • Addition of new tables
  • Modification of standard WordPress tables
  • Changing WordPress configuration files
  • Introducing incompatible code
  • Corruption of .htaccess files

This can result in breaking parts of your site or worse, lead to a crash. Upgrading plugins and themes can also lead to similar issues.

Hacks and Vulnerabilities

WordPress core, by itself, is known to be safe and stable. However, plugins and themes added by developers hailing from diverse backgrounds have become game changers when it comes to WordPress security. Plugins and themes together make up the biggest source of  vulnerabilities found in recent times. Popular plugins like MailPoet, W3Total Cache and Super Cache have been exploited to attack thousands of sites. Similarly, themes are also vulnerable to attacks. The TimThumb library included in many themes was exploited to compromise tons of sites.

Hackers are always looking for new ways to launch attacks on WordPress sites. While most hackers look to make quick profits, some do it merely for fun. They can install malware that’s extremely hard to detect and get rid of. They can also wipe out all of your site’s data.

Human Errors

To err is human. But these errors can prove to be very costly. You can delete a single post or the entire database. Ben Congleton of Olark describes in an interview, a case where a human error nearly took down his business.

The reason behind the disaster can vary, but they will all impact you in the same way. They can all potentially take down your site, and thus your business. So what is the best possible plan to recover from a disaster?

Putting Together a Disaster Recovery Plan

Backup, Backup, Backup: the Cornerstone of a Disaster Recovery Plan

Not enough emphasis can be laid on the importance of backups. Taking regular backups of your data is critical for any business. That way if anything untoward happens, you can recover your site in a matter of few minutes. There are multiple options available from which you can choose. However, it is best to opt for a managed offsite backup service like BlogVault that can handle any situation with ease.

Plan for Extended Downtime

Your plan should cover what you will do if the downtime from the disaster is expected to last more than a few days. For instance, there may be a major outage with your hosting provider. You’ll need to identify possible alternatives to host your site.

Emergency Contact

A natural disaster or emergency could cut off all your regular avenues of communication, so adding a communications element to your plan is important as well. Notifying your customers about the downtime is extremely important. However, when you lose data, your customer information is lost too. Hence it is critical that you have a separate emergency contact list, such as all customer email IDs, stored separately in an easily accessible place.

Test the Plan

Do a test run of your disaster recovery plan to make sure that it works when needed. Also ensure that your plan is known to multiple people at your company so that they can spring into action immediately when disaster strikes.

Disasters do happen, and your company’s data is one of its most important assets. When disaster strikes, you need to be sure that you can get your data back quickly, so there is minimal impact to your business. So work on that disaster recovery plan today, in case you already haven’t. Better safe than sorry, right?