Configuring Backups and Performing Migrations

Phorge Administrator and User Documentation (Configuration)

Advice for backing up Phorge, or migrating from one machine to another.

Overview

Phorge does not currently have a comprehensive backup system, but creating backups is not particularly difficult and Phorge does have a few basic tools which can help you set up a reasonable process. In particular, the things which needs to be backed up are:

• the MySQL databases;
• hosted repositories;
• uploaded files; and
• your Phorge configuration files.

This document discusses approaches for backing up this data.

If you are migrating from one machine to another, you can generally follow the same steps you would if you were creating a backup and then restoring it, you will just backup the old machine and then restore the data onto the new machine.

Restarting Phorge after performing a restore makes sure that caches are flushed properly. For complete instructions, see Restarting Phorge.

Backup: MySQL Databases

Most of Phorge's data is stored in MySQL, and it's the most important thing to back up. You can run bin/storage dump to get a dump of all the MySQL databases. This is a convenience script which just runs a normal mysqldump, but will only dump databases Phorge owns.

Since most of this data is compressible, it may be helpful to run it through gzip prior to storage. For example:

phorge/ $ ./bin/storage dump --compress --output backup.sql.gz

Then store the backup somewhere safe, like in a box buried under an old tree stump. No one will ever think to look for it there.

Restore: MySQL

To restore a MySQL dump, just pipe it to mysql on a clean host. (You may need to uncompress it first, if you compressed it prior to storage.)

$ gunzip -c backup.sql.gz | mysql

Backup: Hosted Repositories

If you host repositories in Phorge, you should back them up. You can use bin/repository list-paths to show the local paths on disk for each repository. To back them up, copy them elsewhere.

You can also just clone them and keep the clones up to date, or use Add Mirror to have them mirror somewhere automatically.

Restore: Hosted Repositories

To restore hosted repositories, copy them back into the correct locations as shown by bin/repository list-paths.

Backup: Uploaded Files

Uploaded files may be stored in several different locations. The backup procedure depends on where files are stored:

Default / MySQL: Under the default configuration, uploaded files are stored in MySQL, so the MySQL backup will include all files. In this case, you don't need to do any additional work.

Amazon S3: If you use Amazon S3, redundancy and backups are built in to the service. This is probably sufficient for most installs. If you trust Amazon with your data except not really, you can backup your S3 bucket outside of Phorge.

Local Disk: If you use the local disk storage engine, you'll need to back up files manually. You can do this by creating a copy of the root directory where you told Phorge to put files (the storage.local-disk.path configuration setting).

For more information about configuring how files are stored, see Configuring File Storage.

Restore: Uploaded Files

To restore a backup of local disk storage, just copy the backup into place.

Backup: Configuration Files

You should also backup your configuration files, and any scripts you use to deploy or administrate Phorge (like a customized upgrade script). The best way to do this is to check them into a private repository somewhere and just use whatever backup process you already have in place for repositories. Just copying them somewhere will work fine too, of course.

In particular, you should backup this configuration file which Phorge creates:

phorge/conf/local/local.json

This file contains all of the configuration settings that have been adjusted by using bin/config set <key> <value>.

Restore: Configuration Files

To restore configuration files, just copy them into the right locations. Copy your backup of local.json to phorge/conf/local/local.json.

Security

MySQL dumps have no builtin encryption and most data in Phorge is stored in a raw, accessible form, so giving a user access to backups is a lot like giving them shell access to the machine Phorge runs on. In particular, a user who has the backups can:

• read data that policies do not permit them to see;
• read email addresses and object secret keys; and
• read other users' session and conduit tokens and impersonate them.

Some of this information is durable, so disclosure of even a very old backup may present a risk. If you restrict access to the Phorge host or database, you should also restrict access to the backups.

Skipping Indexes

By default, bin/storage dump does not dump all of the data in the database: it skips some caches which can be rebuilt automatically and do not need to be backed up. Some of these caches are very large, so the size of the dump may be significantly smaller than the size of the databases.

If you have a large amount of data, you can specify --no-indexes when taking a database dump to skip additional tables which contain search indexes. This will reduce the size (and increase the speed) of the backup. This is an advanced option which most installs will not benefit from.

This index data can be rebuilt after a restore, but will not be rebuilt automatically. If you choose to use this flag, you must manually rebuild indexes after a restore (for details, see ((reindex))).

Next Steps

Continue by:

• returning to the Configuration Guide.