Link Search Menu Expand Document
Start for Free

Transaction Logs

This page discusses how to inspect and replay transaction logs in Stardog, enabling point-in-time recovery by combining database backups with transaction log replay.

Page Contents
  1. Overview
  2. Enabling Transaction Logging
    1. Related Configuration Options
  3. Transaction Log Structure
  4. Inspecting Transaction Logs
    1. View Transactions from a Database
    2. View Individual Updates
    3. Export to a File
    4. Filter by UUID Range
    5. Filter by Time Range
    6. Read a Local Log File
  5. Validation and Consistency
    1. How Validation Works
    2. Invalid Log Scenarios
    3. Skipping Validation
  6. Permissions
  7. Point-in-Time Recovery
  8. Point-in-Time Backup Workflow
    1. Reading the anchor UUID from the backup
    2. Reading current UUID from the running Stardog server
    3. Point-in-Time Recovery Workflow
    4. Replay with UUID Filtering
    5. Replay with Time Filtering
    6. Scripted Point-in-Time Recovery
  9. Cluster Considerations
  10. Best Practices

Overview

Starting with version 12.0, Stardog provides commands to inspect and replay transaction logs. Transaction logs record all database modifications, enabling:

  • Point-in-time recovery: Restore a database to any point between backups by replaying transactions on top of a restored backup
  • Cluster synchronization: Transaction logs are used internally to keep cluster nodes in sync

With transaction logging enabled, Stardog writes a sequential list of records of all transactions to disk. This log can be exported and replayed onto a database backup to recover data up to a specific point in time.

Enabling Transaction Logging

Transaction logging is controlled by the transaction.logging database configuration option. By default, it is:

  • Disabled (false) for standalone Stardog servers
  • Enabled (true) for Stardog Cluster nodes (required for replication)

Create a database with transaction logging enabled:

$ stardog-admin db create -o transaction.logging=true -n myDatabase

To enable transaction logging for an existing database, set the option using stardog-admin metadata set. The database must be offline to change this setting:

$ stardog-admin db offline myDatabase
$ stardog-admin metadata set -o transaction.logging=true -- myDatabase
$ stardog-admin db online myDatabase
Option Default Description
transaction.logging false Enable/disable transaction logging
transaction.logging.rotation.size 524288000 (500 MB) Size in bytes at which the log file rotates
transaction.logging.rotation.remove false Whether to delete old log files after rotation
transaction.logging.use.rotated true Whether to use rotated transaction logs

The option transaction.logging.rotation.remove=false only controls whether tx log entries will be kept around beyond one round of rotation. It is not necessary to set it to false for point-in-time restore. By default, after the configured tx log size is reached, old transactions will be pruned when the log rotates for the second time.

Transaction Log Structure

Transaction logs contain records for each phase of a transaction’s lifecycle:

Record Type Description
Started A new transaction has begun, identified by a UUID
Update Data modification within a transaction (additions/removals)
Commit Transaction has been committed
Rollback Transaction has been rolled back
Done Transaction processing is complete

Each transaction is assigned a UUID when it starts. The log maintains a sequential record that preserves the order and relationship between transactions, which is critical for replay validation.

Inspecting Transaction Logs

The stardog-admin tx log command allows you to inspect transaction logs from a running database or from a local log file.

View Transactions from a Database 

$ stardog-admin tx log myDatabase

This displays a human-readable summary of all transactions in the database’s log.

View Individual Updates

By default, the output shows summarized counts of additions and removals per transaction. Use --updates to see individual update records:

$ stardog-admin tx log myDatabase --updates

Export to a File

Export the raw transaction log for later replay:

$ stardog-admin tx log myDatabase --format raw --output backup-txlog.log
Transaction log exported to: backup-txlog.log
Last transaction UUID: 7e1c0c2a-3a48-4f44-8a39-f4e5d2b6cb0a

The command prints the UUID of the last fully committed transaction included in the export. Capture this value and use it with stardog-admin tx log --from-uuid to fetch the next incremental export. For more details see Scripted Point-in-Time Recovery.

Filter by UUID Range

View transactions within a specific UUID range:

$ stardog-admin tx log myDatabase --from-uuid a1b2c3d4-... --to-uuid f9e8d7c6-...

Filter by Time Range

View transactions within a specific time range:

$ stardog-admin tx log myDatabase --from-time 2024-01-15T10:30:00Z --to-time 2024-01-16T10:30:00Z

Read a Local Log File

Read a previously exported log file:

$ stardog-admin tx log --file /path/to/txlog.log

Validation and Consistency

The replay command validates transaction logs by default to ensure database consistency.

How Validation Works

When a database commits a transaction, it records the transaction’s UUID as the “last committed transaction” in its metadata. During replay, Stardog validates that:

  1. The log contains a Started record for the last committed transaction UUID of the database
  2. The log contains a corresponding Done record for that transaction
  3. All subsequent transactions in the log are complete (have both Started and Done records)

This ensures the log maintains continuity with the database state and contains only complete transactions.

Invalid Log Scenarios

Replay validation will fail when:

  • Missing start record: The first transaction in the replay range doesn’t match (or follow) the database’s last committed transaction
  • Incomplete transactions: A Commit or Done record exists without a corresponding Started record after the last committed transaction point
  • Data corruption: The log contains hash sums to ensure integrity and will fail validation if a transaction record is corrupted

Skipping Validation

You can skip validation with --skip-validate, but this is not recommended:

$ stardog-admin tx replay --skip-validate myDatabase txlog.log

Skipping validation may result in an inconsistent database state. Only use this option if you understand the implications and have verified the log contents manually.

Permissions

Transaction log operations require DBMS execute permission [EXECUTE, "admin:<db>"] on the database - exactly the same permissions as backup and restore for a database.

Point-in-Time Recovery

Point-in-time recovery (PITR) combines a database backup with transaction log replay to restore a database to any moment between the backup and the present. It relies on two pieces:

  • A full backup taken at time T₀. The backup records, in its metadata, the UUID of the last committed transaction at the moment the snapshot was taken — the anchor from which replay must start.
  • One or more transaction log slices captured after T₀. Each slice is a contiguous portion of the transaction log identified by its starting and ending UUIDs. Slices may be exported on any cadence, but every commit between T₀ and the recovery target must be covered by exactly one slice. Slices are then replayed in order on top of the restored backup.

To recover to time T, you restore the most recent backup older than T, then replay slices in order until the transaction at T has been applied. Replay is exact at transaction boundaries: passing --to-uuid to the final replay stops the database at that specific transaction.

For a full scripted pipeline — taking a backup, continuously capturing tx-log slices, and restoring on demand: Point-in-Time Recovery tutorial.

Point-in-Time Backup Workflow

The high-level recipe for capturing the data needed for PITR:

  1. Take a full backup: stardog-admin db backup myDatabase. Note the date-versioned destination directory printed by the command.
  2. Extract the anchor UUID — the last committed transaction at the backup snapshot — directly from the backup’s metadata file.
  3. After T₀, repeatedly check the live database’s index.last.tx. Whenever it advances, export a tx-log slice covering the new range.

Reading the anchor UUID from the backup

stardog-admin metadata convert reads the binary metadata.bk file inside the backup directory and prints it in a parseable format. Reading from the file (rather than querying the running database) guarantees the UUID and the backup data are in sync. There could be more transactions committed in the time between the snapshot completing and the script reading it.

$ stardog-admin metadata convert \
        --input-format BINARY --output-format json \
        /path/to/backup/myDatabase/2026-05-10/metadata.bk \
        | jq -r '.["index.last.tx"]'
54a1f110-fa6c-4d71-8b11-820bd9ea01be

Reading current UUID from the running Stardog server

The same value is exposed by metadata get for the live running database. Every time the database is updated, this value advances to the UUID of the latest committed transaction.

Use --output-format json for clean parsing:

$ stardog-admin metadata get -o index.last.tx myDatabase --output-format json \
        | jq -r '.["index.last.tx"]'
9f823c11-77be-4d28-94a1-1d04e8a3aabe

If this UUID equals the UUID at the end of your previous slice, nothing has changed. If it differs, write a new slice covering --from-uuid <previous> to --to-uuid <current>:

$ stardog-admin tx log myDatabase \
        --from-uuid 54a1f110-fa6c-4d71-8b11-820bd9ea01be \
        --to-uuid   9f823c11-77be-4d28-94a1-1d04e8a3aabe \
        --format raw --output txlog-slice-0001.log
Transaction log exported to: txlog-slice-0001.log
Last transaction UUID: 9f823c11-77be-4d28-94a1-1d04e8a3aabe

The export reports the UUID of the last fully-committed (Done) transaction it wrote — useful for confirmation, and the value to feed back as --from-uuid on the next slice.

Point-in-Time Recovery Workflow

  1. Restore from backup: Start by restoring the most recent backup before your target recovery point. Ensure you know the UUID of the last write transaction valid at the time of the backup.

    $ stardog-admin db restore /path/to/backup/myDatabase/2024-01-15

  2. Export the transaction log: If the original database server is still available and has the full transaction log, you may export it from there. Otherwise, use a previously exported log file.

  $ TO_UUID=$(stardog-admin metadata get -o index.last.tx myDatabase --output-format json | jq -r '.["index.last.tx"]')
  $ stardog-admin tx log myDatabase --from-uuid <backup-uuid> --to-uuid "$TO_UUID" --format raw --output txlog-for-replay.log

Note: The exported log should cover all transactions from the backup point onward. This requires that the log file contains the last committed transaction UUID recorded in the database backup. Otherwise, replay validation will fail because some updates may be missing from the log. This validation can be skipped with --skip-validate.

  1. Preview the replay (recommended): Use --dry-run to verify the replay will succeed before applying changes.

    $ stardog-admin tx replay --dry-run myDatabase txlog-for-replay.log

  2. Replay transactions: Apply the transactions to bring the database forward in time.

    $ stardog-admin tx replay myDatabase txlog-for-replay.log

Replay with UUID Filtering

To recover to a specific point, use UUID filtering. Both --from-uuid and --to-uuid are required:

$ stardog-admin tx replay --from-uuid a1b2c3d4-... --to-uuid f9e8d7c6-... myDatabase txlog.log

Replay with Time Filtering

You can also filter by time, but this requires --skip-validate since time-based filtering may not have the matching starting transaction at the beginning of the tx-log (see How Validation Works):

$ stardog-admin tx replay --skip-validate --from-time 2024-01-15T10:30:00Z --to-time 2024-01-16T10:30:00Z myDatabase txlog.log

Time-based filtering with --skip-validate should be used with care. Ensure you understand the transaction sequence to avoid applying an incomplete or inconsistent set of changes.

Scripted Point-in-Time Recovery

For a full, end-to-end scripted pipeline — taking a full backup, anchoring on the backup’s own metadata, continuously capturing incremental tx-log slices, and replaying them on demand — see the Point-in-Time Recovery tutorial.

Cluster Considerations

In Stardog Cluster deployments:

  • Transaction logging is automatically enabled and cannot be disabled
  • Transaction logs are used internally for node synchronization
  • The tx log and tx replay commands work the same way as in standalone mode
  • Point-in-time recovery should be performed when the database is not receiving concurrent writes from other applications. Otherwise replaying might fail, or lead to an unexpected state.

Best Practices

  1. Enable logging for production: Set transaction.logging=true for any database where point-in-time recovery is important

  2. Coordinate with backups: Schedule regular backups and note the backup timestamps. The backup implicitly records the last committed transaction, which serves as your recovery starting point

  3. Test recovery procedures: Periodically test your point-in-time recovery workflow on a non-production system

  4. Monitor log size: Transaction logs can grow large. Plan for adequate disk space and consider the rotation size setting

  5. Use dry-run: Preview replay operations with --dry-run before applying changes to production databases

  6. Maintain log retention: Ensure that transaction logs are retained long enough to cover your desired recovery window

  7. No concurrent user of database: Ensure that no application is using the database while performing point-in-time recovery.