Process Management

This page discusses the details of process management in Stardog.

Page Contents

1. Overview
2. Listing Processes
3. Terminating a Process
4. Reading Process Status
5. Process IDs
6. Process History
7. Security and Process Management

---

Overview

When Stardog is performing long-running operations, there are three main things an administrator is interested in:

• Which operations are taking up resources.
• Learning roughly how long these operations are going to take.
• Cancelling operations if they are taking too long or blocking other operations.

The unified process management API offers a single view on all types of long running operations in Stardog. We use the term Process to refer to a long-running operation in Stardog; we do not refer to OS processes. The unified process management API complements the existing mechanisms for transactions and queries. Processes which can be managed through this API include Queries, Transactions, Database Creation, Backups, Database Optimize, and many more. While some types of processes will report estimated progress, Queries and Transactions will not.

Process management includes support for:

• Listing running processes.
• Terminating running processes.
• Reading the status of a running or recently finished process.
• Choosing the ID of a process started over HTTP, so it can be tracked by a known ID.

Listing Processes

To see all running processes from the command line, use the ps list command:

$ stardog-admin ps list

The results are returned in tabular format:

+--------------------------------------+---------------------------------------------+-------------+-------+-------------------+--------------+-------------+----------------------+------------+
|              Kernel ID               |                 Process ID                  |  Database   | User  |       Type        | Elapsed time |   Status    |       Progress       | Cancelable |
+--------------------------------------+---------------------------------------------+-------------+-------+-------------------+--------------+-------------+----------------------+------------+
| c70d0fa3-c583-4568-93ee-05c46db8b7cd | 8cc95209-a649-4e36-aa6c-a5a9a72832e7        | myDatabase  | admin | Transaction       | 00:03:15.672 | Terminating | N/A                  | Yes        |
| c70d0fa3-c583-4568-93ee-05c46db8b7cd | internal-DatabaseCreation-admin-myDatabase2 | myDatabase2 | admin | Database Creation | 00:00:12.074 | Running     | Parsing data (65.8%) | Yes        |
+--------------------------------------+---------------------------------------------+-------------+-------+-------------------+--------------+-------------+----------------------+------------+

2 process(es) running.

Depending on the type of process, a progress stage might be shown (“Parsing data” in this instance). The Cancelable column says whether ps kill can act on the process at all. Some user actions might spawn more than one process. Usually these processes are related, and canceling one will cancel the other(s). This can be the case for database optimization and other operations.

Terminating a Process

To terminate a running process, simply pass its ID to the process kill command:

$ stardog-admin ps kill 8cc95209-a649-4e36-aa6c-a5a9a72832e7

Killing a process signals it to stop; the process then has to stop itself. ps kill reports success once the signal has been delivered. The process moves to the Terminating status until it reaches a point at which it can check the signal and terminate. An idle process may sit in the Terminating state until it resumes progress. Check ps list to confirm the process has actually disappeared.

Reading Process Status

To see the status of a single process, pass its ID to the ps status command:

$ stardog-admin ps status 8cc95209-a649-4e36-aa6c-a5a9a72832e7

Over HTTP, use GET /admin/processes/{id} (and DELETE /admin/processes/{id} to kill), with the ID URL-encoded in the path. The response is a JSON object with the process id, type, db, user, startTime and status, plus progress for processes that report it. Once the process has finished, the response also includes endTime and, if the process failed or was killed, an errorMessage.

Process IDs

By default, Stardog assigns each process an ID. Transactions use a UUID; most other processes get a short random ID.

A client starting a process over HTTP can choose the ID itself by sending the SD-Process-Id header. This lets the client track or kill the process without first looking it up in the process list. The header is honored by requests that start a process, including queries, updates, transactions, virtual graph imports, database optimize and backups.

$ curl -u admin:admin \
    -H "SD-Process-Id: 3f2c9b1e-6a0d-4c8e-9d7a-2b5f1e4c8a90" \
    --data-urlencode "query=SELECT * { ?s ?p ?o }" \
    http://localhost:5820/myDatabase/query

$ curl -u admin:admin http://localhost:5820/admin/processes/3f2c9b1e-6a0d-4c8e-9d7a-2b5f1e4c8a90

Rules for the header value:

• It is an opaque string of at most 256 printable ASCII characters. Longer values, or values with other characters, are rejected with 400 Bad Request. Leading and trailing whitespace is removed.
• Any printable ASCII character is allowed, including /, ? and %, as long as the ID is URL-encoded when used in /admin/processes/{id}. To use non-ASCII text, percent-encode it yourself; for example, %E2%82%AC is kept as that literal ID.
• When beginning a transaction (POST /{db}/transaction/begin) the value must be a UUID, because it becomes the transaction ID. Other values are rejected with 400 Bad Request.
• For queries, the queryId request parameter takes precedence over the header.
• A request whose ID is already used by a running process fails. Using UUIDs is recommended for all requests, especially in a cluster.

Process History

Stardog keeps finished processes in memory for a while, so a client can read the final status of a process (for example, whether it finished or failed, and why) after it has ended. Finished processes are not shown by ps list, but ps status and GET /admin/processes/{id} return them.

The history is bounded by two server properties. A finished process is removed when either limit is reached:

• process.history.size: the maximum number of finished processes kept (default 64). Set to 0 to disable the history.
• process.history.ttl: how long a finished process is kept (default 10m).

The history is kept in memory on each server and is lost on restart. In a cluster, each node keeps its own history.

Security and Process Management

Any user can always see and kill the processes they submitted themselves, and a superuser can see and kill every process. In between, these permissions widen what a user reaches:

Killing another user’s process takes both actions: EXECUTE on its own is not enough, since a process you are not allowed to see is not one you may kill.

Every process belongs to a database, which is what makes admin:* equivalent to server-wide. The database-scoped form is the one to grant for multi-tenancy, where each database has its own administrator, and both actions are granted automatically to whoever creates the database.

Queries and transactions are processes too, so they appear in ps list for anyone holding these permissions. Acting on a transaction is narrower than ps kill suggests: see Transactions.

An idle transaction is one that is open but not currently executing anything, which is the state most transactions waiting on a client are in. In that case ps kill reports success and moves it to Terminating without ending it: the transaction stays open, keeps its locks, and appears in tx list until the user executes the next action within it. To end an open transaction immediately, use tx rollback, which takes effect immediately.

---