19th Apr // 2016
UK hosting and domain name company 123-reg have unintentionally deleted an unknown number of websites from their servers, causing issues for many of their customers. The incident has since been put down to a coding error which “effectively deleted” content from 67 of their 115,000 European servers.
Affected customers include Scottish League Cup winners Ross County FC, who rely on their web operations for ticket and merchandise sales as well as club and match information for fans. No other major clients have come forward at the time of writing but are expected to if sites are not recovered soon.
123-reg have reported that they have commenced a “recovery process” and are working with data recovery experts as they do not have backups of all of the data that has been lost. The company also said, “Our VPS product is an unmanaged service and we always recommend that customers implement backups to safeguard against unexpected issues.” This likely won't be of great help to those customers who have not got backups and are currently unsure if their website will be restored.
This incident highlights the importance of keeping up-to-date backups of your website data, which should be stored in separate and secure locations. When developing a back-up and restoration plan for your data it's important to consider how quickly you'd need to restore service if something went wrong. If you're running business-critical processes which are required to be online 24/7 then one option to protect your data, which we have implemented for clients, is to run real-time replicas of the any databases and file storage remotely from the master server. In the case of any downtime on the main server, a replica can be promoted into a live server in a matter of minutes, reducing any potentially harmful down time.
Tolerance on downtime can vary significantly depending on website content. If you're running an e-commerce site you obviously want to ensure 100% uptime and this may become more sensitive at different times of the year when you're experiencing peak trading. If your website is not transactional then you may be able to withstand several hours of down-time and because of this can use a more cost-effective backup management system. Most hosting providers offer options such as daily backups but make sure to check the fine-print as some providers may take up to 24 hours to return or restore these backups to your server.
One development practice of using version control software like Git when developing a project has the added benefit of backing up website code if set up through a project hosting provider like Github or Bitbucket. We wouldn't recommend relying on this as a main backup system, but it is a nice side-effect of proper code management.
Got a question about website hosting? Contact us!