Link Search Menu Expand Document
Start for Free

Backup & Restore

This page discusses backing up and restoring the Stardog server. For information on backing up individual Stardog databases and restoring them, please see the page on Database Backups.

Page Contents
  1. Overview
  2. Server Backup
    1. Backup to S3
  3. Server Restore
    1. Restore from S3

Overview

Stardog supports backup/restore functionality for the entire system in one step via the stardog-admin server backup and stardog-admin server restore commands. This functionality is convenient when all databases need to be backed up and restored simultaneously.

The server backup requires as much disk space as production data. Best practice is to give a path to an independent storage volume, but not the same volume as the production data. It is possible to crash Stardog by filling the storage volume that contains the production data.

Server Backup

Server backup will back up the entire Stardog server - all databases and associated metadata. Unlike a database backup, which takes a full backup of the database every time it is run, a server backup takes an incremental backup of the data. That way, each time the command is run, only the updates to the databases since the last backup need to be saved.

Server backups are accomplished with the following command:

$ stardog-admin server backup

This command only supports file-based backups. It cannot be used with an S3, for instance.

You can optionally specify the backup location:

$ stardog-admin server backup /path/to/custom/backup/location

If no backup localtion is specified, the stardog-admin server backup command assumes a default location for its output, namely, $STARDOG_HOME/.backup. That default may be overridden by setting the backup.dir server property in your stardog.properties file.

To copy the backups off local disk, we recommend using a tool such as rclone. After setting up rclone, you can use it to send the backups to another server:

$ rclone sync /path/to/backup/location/ sftp:/path/to/other/location/

Consult the rclone docs for the full list of supported storage systems.

Backup to S3

S3 backups use a URL in the following format:

s3:///<bucket name>/<path prefix>?region=<AWS Region>&AWS_ACCESS_KEY_ID=<access key>&AWS_SECRET_ACCESS_KEY=<verySecretKey1>

Example:

stardog-admin server backup "s3:///mybucket/backup/prefix?region=us-east-1&AWS_ACCESS_KEY_ID=accessKey&AWS_SECRET_ACCESS_KEY=secret"

The double quotes around the S3 URL prevent the linux command line from interpreting the ampersand, “&”, as process detach command.

S3 server backup does not currently support on-premises S3 clones.

stardog.properties does support setting a default location for S3 server backups via the backup.location property.

S3 server backup requires a special Stardog license.

S3 server backup requires ubuntu/debian linux that has libcrypto, libcurl, and libssl installed.

S3 server backup is supported / available in safe mode.

Server Restore

You can use the server restore command to restore backups created by server backup. To do so, you must shut down Stardog and set $STARDOG_HOME to an empty home directory. The server restore command will restore the complete server to $STARDOG_HOME. Once complete, you can start the Stardog server.

$ export STARDOG_HOME=/path/to/empty/stardog/home
$ stardog-admin server restore /path/to/server/backup

Server backups do not contain your license file, stardog.properties, or any other additional files or directories created externally under STARDOG_HOME, so you need to back up and restore those files and directories separately.

By default, server restore will restore the latest backup found in the backup directory. The server configuration option backup.keep.last.number.backups that can be set in stardog.properties controls how many backups will be retained. By default, this option is set to 4, and any one of the older backups can be restored by specifying a backup ID in the command:

$ stardog-admin server restore -b 3 /path/to/server/backup

The server backup command prints the ID for the backup created, which is the value that can be passed to the server restore command. The backup IDs correspond to directories under the versions directory of the backup directory. The creation date for these directories will indicate when the corresponding backup was created.

Restore from S3

S3 restore requires an additional parameter, node-id. Basic file system based restores do not need the node-id parameter. The node-id is an identifier that is unique to each server. It is stored in the stardog.node-id file within $STARDOG_HOME. There are two ways to provide the node-id during a restore. One is using the -i command line parameter shown in the examples below. The second way is to copy the old stardog.node-id file into the empty restore destination prior to executing the restore command.

It is reasonable to use the node-id of an existing cluster server to create/populate a new server. But you must erase the $STARDOG_HOME/stardog.node-id created by the restore before starting the new server. Stardog will then create a new and unique stardog.node-id for the new server upon first start.

Backups can be restored directly from S3 by using an S3 URL in the following format:

s3:///<bucket name>/<path prefix>?region=<AWS Region>&AWS_ACCESS_KEY_ID=<access key>&AWS_SECRET_ACCESS_KEY=<verySecretKey1> -i <node-id>

Example:

stardog-admin server restore "s3:///mybucket/backup/prefix?region=us-east-1&AWS_ACCESS_KEY_ID=accessKey&AWS_SECRET_ACCESS_KEY=secret" -i 051d52f1-9b38-48e1-b127-ac82c3b8adaf

The double quotes around the S3 URL prevent the linux command line from interpreting the ampersand, “&”, as process detach command.