Logging

This page discusses the different types of logging available Stardog.

Page Contents

1. stardog.log
2. Access and Audit Logging
   1. Access Log Configuration
   2. Audit Log Configuration
   3. Analyzing the Access and Audit Log
      1. Event Types
      2. Print all events
      3. Print all events with specified fields
      4. Filter events
      5. Output formats
         1. JSON Output
         2. Delimited Output
3. Slow Query Log

---

stardog.log

Stardog’s server log is the stardog.log which can be found inside your $STARDOG_HOME directory.

Unlike the other logs mentioned in this section (access, audit, and slow query), the stardog.log can be configured via log4j2. A default log4j2.xml can be found in the <stardog-installation-directory>/server/dbms directory. You can copy this file to your $STARDOG_HOME directory to make any modifications you’d like (e.g. log rotation).

$ cp <stardog/installation-directory>/server/dbms/log4j2.xml $STARDOG_HOME

Access and Audit Logging

Audit logging is a superset of the events in access logging. Access logging covers the most often required logging events; you should consider enabling audit logging if you really need to log every server event. Logging generally doesn’t have much impact on performance; but the safest way to ensure that impact is negligible is to log to a separate disk (or to a centralized logging server, etc.).

The important configuration choices are whether logs should be binary or plain text (both based on ProtocolBuffer message formats); the type of logging (audit or access); the logging location (which may be “off disk” or even “off machine”). Logging to a centralized logging facility requires a Java plugin that implements the Stardog Server logging interface; see Java Programming for more information; and the log rotation policy (file size or time).

We walk through how to setup both the Access and Audit log sections below and discuss their configuration in more detail.

An example stardog.properties file containing configuration for access and audit is provided in our stardog-examples Github repository.

Access Log Configuration

The access log logs all connection-based events, such as queries and transactional changes, and authentication events. Connection boundary events, that is open and closing of the actual connection, are not logged.

The access log is disabled by default and should be enabled explicitly in the stardog.properties configuration file by adding:

logging.access.enabled = true

The type of the access log is required. There are two allowed values: text and binary, both based on Protobuf.

• In text logs, entries are serialized as key-value pairs in plain text, and log entries are delimited by the ASCII Start of Text and ASCII End of Text bytes.
• In binary logs, entries are written as size-delimited protobuf binary messages.

We can add the type of access log by adding the following to the stardog.properties:

logging.access.type = text

Here we’ve chosen to set the type of log as text, as opposed to binary.

Optionally, you can specify a custom file name for the access log. By default, the access log is saved as a file named access.log inside the $STARDOG_HOME directory. If desired, a different file name can be provided as follows. An absolute path can be provided to save the file in a different directory.

logging.access.file = my_access.log

It is possible to define a file rotation strategy so that log entries will be split into multiple files. One possibility is to define a limit on file size and start a new file when the size limit is reached. The file that exceeds the limit will be renamed with a unique suffix, and all the subsequent entries will be added to the newly-created file. The size limit is defined in bytes as follows. See the audit log example for defining an alternative way of rotating files based on fixed-time intervals.

logging.access.rotation.type = size
logging.access.rotation.limit = 1000000

The server configuration properties for access logging are documented fully in the server configuration section.

Audit Log Configuration

As noted earlier, the audit log logs all server events and is a superset of the access log. Access logging covers the most often required logging events; you should only consider enabling audit logging if you really need to log every server event. Logging generally doesn’t have much impact on performance; but the safest way to insure that impact is negligible is to log to a separate disk (or to a centralized logging server, etc.).

The audit log is disabled by default and should be enabled explicitly in the stardog.properties configuration file by adding:

logging.audit.enabled = true

Similar to the access log, the type of the audit log is required. There are two allowed values: text and binary, both based on Protobuf.

• In text logs, entries are serialized as key-value pairs in plain text, and log entries are delimited by the ASCII Start of Text and ASCII End of Text bytes.
• In binary logs, entries are written as size-delimited protobuf binary messages.

We can add the type of audit log by adding the following to the stardog.properties:

logging.audit.type = binary

By default, the audit log is saved as a file named audit.log inside the $STARDOG_HOME directory. Optionally, a different file name can be provided as follows. An absolute path can be provided to save the file in a different directory.

logging.audit.file = my_audit.log

Audit logs can be rotated similarly to access logs. Size limits can be used as described above. Alternately, log files can be rotated at fixed time intervals. The interval value should be a positive integer followed by either letter ‘d’ (for days) or letter ‘h’ (for hours). Below is an example to get daily logs:

logging.audit.rotation.type = time
logging.audit.rotation.interval = 1d

Rotated log files can be compressed with gzip if the following option is set:

logging.audit.rotation.compress = true

Compression happens asynchronously. Once the rotation condition is triggered, the audit file will be rotated in an uncompressed format and will be compressed in a background thread. Once the compression completes, the uncompressed rotated file will be deleted. The compressed rotated file will use the .gz file extension.

The number of rotated files can be limited in order to avoid running out of disk space by accumulating rotated files. The following setting, for example, will limit the number of rotated files to 5:

logging.audit.rotation.file.count = 5

Note that the actual audit log is not included in this count. Therefore, if the limit is set to 5, there will be at most 6 log files at any time. When the number of rotated files exceeds this limit, the latest rotated file will be deleted. If this limit is reduced in the configuration at any point, the server needs to be restarted, and the change will take affect the next time the log file is rotated. For example, suppose the file count was set to 10 and there are 10 rotated log files in $STARDOG_HOME directory. The file count is reduced to 5 and the server is restarted. The next time the log file is rotated, the 6 oldest log files will be deleted and there will be 5 rotated log files, including the newly-rotated file.

The server configuration properties for audit logging are documented fully in the server configuration section.

Analyzing the Access and Audit Log

The contents of the access and audit log can be printed using the log print CLI utility. The utility has multiple options (discussed below) to filter and customize the log contents that are outputted and their format.

Event Types

Stardog logs multiple different types of events. The events can be provided to the log print utility if you wish to filter for certain events such as a ‘ChangePassword’ event. These events are listed and described in the table below:

Print all events

To print all events in the audit/access log, provide the absolute path to the audit/access log as the last argument to the log print command:

Print all events with specified fields

One can choose to only include certain fields in the output. Allowed values are:

• Time
• Event
• Database
• User
• IP
• Value
• Flags
• Query

Fields follow the --field option in the log print command:

Filter events

It’s possible to provide filter expressions to the log print command to hide certain events in the output.

• A filter expression is in the form field operator value where field is one of the fields defined in --field option, operator is one of =, <, <=, >, >=, or ~, and value should be a valid value for the field.

  • The operator ~ represents regular expression matching, where value should be a valid Java regular expression and should match the entire value of the given field.

  • Valid values for the Event field can be found in the event types table above.

  • Filter expressions for the Time field should use xsd:date or xsd:dateTime values.

Filter expressions follow the --filter option in the log print command:

Output formats

The audit/access log can be printed in a tabular format (default), JSON, or a delimited format like CSV.

JSON Output

To format the output of the log print command as JSON, pass the --json flag in.

Delimited Output

One can use a delimiter to separate fields in the output. Specify the delimiter of your choice (e.g. , for a CSV) after the -d option.

Slow Query Log

As the name suggests, the slow query log logs all slow queries. A query is considered to be slow if its execution time exceeds the logging.slow_query.time property defined below.

The slow query log is enabled similarly to the access and audit logs:

logging.slow_query.enabled = true

Slow query time is specified as follows. Any query whose execution time exceeds this limit will be logged in the slow query log. The time value should be a positive integer followed by either letter ‘h’ (for hours), letter ‘m’ (for minutes), letter ‘s’ (for seconds), or letters ‘ms’ (for milliseconds). Example intervals are 1h for 1 hour, 5m for 5 minutes, 90s for 90 seconds, and 500ms for 500 milliseconds. If no time is specified in the configuration, the default value of 1 minute will be used.

logging.slow_query.time = 10s

Unlike access or audit logs, the slow query log has a default type for formatting the slow query events, and it is intended to be human-readable. If no log type is specified, this default type will be used. This default format is not suitable for machine-processing and meant to be only for developers to read. For a machine-processable format, the type of the slow query log can be set to binary, similar to the access and audit logs.

logging.slow_query.type = text

The server configuration properties for slow query logging are documented fully in the server configuration section.

---