Configuring the Voicebox Service

Page Contents

1. Running the Voicebox Service
2. Example Configuration
3. Configuration
4. Voicebox Configuration File
   1. LLM Configuration
   2. Choosing a Model
      1. Model Quality and Latency
   3. Anthropic Configuration
   4. Azure AI Foundry Configuration
      1. API Key Authentication
      2. Service Principal (SPN) Authentication
   5. AWS Bedrock Configuration
   6. Google Vertex Configuration
   7. OpenAI Configuration
      1. Basic Authentication
5. Custom Headers
   1. $VOICEBOX_USER Variable
6. Advanced Customization
   1. Configuration Directory
   2. Endpoint and Database Matching
   3. Matching Priority
   4. Example Setup
7. Internal Stardog Endpoint Support
8. JWT Authentication & Token Exchange

---

How to run and configure the Voicebox Service: network routing to Stardog, the environment variables it accepts, the vbx-config.json file and its supported LLM providers, and per-request customization.

Except where a step is marked otherwise, everything on this page applies to both the v1.x and v0.x Voicebox Service — the configuration file, the LLM providers, custom headers, and JWT authentication are the same in both. Storage backends and operational tuning are specific to v1.0.0+ and are covered in Deploying the Voicebox Service.

For what Voicebox is and how to use it, see Voicebox.

Running the Voicebox Service

1. Similar to Launchpad, the Voicebox Service image can be pulled from Stardog’s private Docker registry.

   docker pull stardog-stardog-apps.jfrog.io/voicebox-service:v1.0.0
   

   • Pinning an explicit version tag, as above, is recommended so that restarting or rebuilding never changes the version you are running. The current tag always points to the latest release if you would rather track it.

2. To run the Voicebox Service with Docker, you can use the following command:

   docker run \
   --env-file .env.voicebox-service \
   -p 8000:8000 \
   -v /host/path/to/vbx-config.json:/voicebox-config/vbx-config.json \
   -v voicebox-frames:/var/lib/voicebox/frames \
   stardog-stardog-apps.jfrog.io/voicebox-service:v1.0.0
   

   • .env.voicebox-service can be named anything but contains the configuration for the Voicebox Service. See Configuration for more details.
   • /host/path/to/vbx-config.json is mounted from the host to /voicebox-config/vbx-config.json in the container. This configuration has more LLM specific configuration.
     • .env.voicebox-service should have VBX_CONFIG_FILE=/voicebox-config/vbx-config.json in it.
   • voicebox-frames is a named volume mounted at the frame store path, where the service persists query results. This is required as of v1.0.0 when using the default local-disk frame store: without it, frames are written to the container’s writable layer and lost on restart. Omit it if you configure an S3 or Azure frame store instead. See Deploying the Voicebox Service for sizing, tuning, and the Kubernetes equivalent.
   • The Voicebox Service HTTP server is exposed on port 8000 so Launchpad can communicate with it.

3. Update Launchpad configuration to point to URL of the Voicebox Service.

   VOICEBOX_SERVICE_ENDPOINT=http://<host>:8000
   

4. Assuming you have restarted Launchpad with the Voicebox Service endpoint configured, you should be able to access Voicebox from the Launchpad UI.

Example Configuration

The service is configured by two things: an environment file and a JSON configuration file that the environment file points at. This example runs Voicebox against AWS Bedrock with the default local-disk frame store.

.env.voicebox-service:

# Absolute path, inside the container, to the JSON configuration file below
VBX_CONFIG_FILE=/voicebox-config/vbx-config.json

# Credentials for the LLM provider selected in the configuration file
AWS_ACCESS_KEY_ID=<access_key_id>
AWS_SECRET_ACCESS_KEY=<secret_access_key>
BEDROCK_REGION=us-west-2

That is everything the service requires. Logging, the frame store and the agent limits all have working defaults; set them only when you need to change one.

vbx-config.json:

{
  "default_llm_config": {
    "llm_provider": "bedrock",
    "llm_name": "us.anthropic.claude-haiku-4-5-20251001-v1:0"
  }
}

The rest of this page describes each of these settings. For frame stores, tuning and health probes, see Deploying the Voicebox Service.

Configuration

The following options should be provided as environment variables to the Voicebox Service. Additional environment variables might be needed based on the LLM provider configured. See the Voicebox Configuration File section below for more details.

Voicebox Configuration File

In addition to the environment variables provided to the Voicebox Service, the Voicebox Service also requires you to give it a configuration file in JSON.

See Example Configuration above for a complete file.

The file requires one field, default_llm_config, which selects and configures the LLM provider. Its contents are described in LLM Configuration below. A file can also carry endpoint and database fields to scope it to particular Stardog connections, which is covered in Advanced Customization.

LLM Configuration

The LLM configuration is specified as a JSON object in the Voicebox configuration file with the following fields.

The following LLM providers are supported:

• Anthropic
• Azure AI
• AWS Bedrock
• Google Vertex
• OpenAI

Choosing a Model

Voicebox answers a question by planning and running a sequence of steps rather than making a single call to the LLM, so the model behind it has to be capable of driving that process. A model can be used with Voicebox if it offers all of the following:

• A chat interface. Multi-turn conversations. Text-completion endpoints are not supported.
• Native tool calling through the provider’s API. This is a hard requirement: a model without it fails outright rather than answering less well.
• Reliable multi-step behavior. Answering takes several steps, and the model has to carry all of them through to a final answer.
• A context window of at least 32K tokens, and 128K or more for large or complex knowledge graph schemas.

In practice that means a current-generation model from one of the providers below. This page deliberately does not list model names: providers release and retire models faster than documentation changes, and any model meeting the requirements above works by setting llm_name to whatever identifier your provider uses — a model ID, a deployment name, or an inference profile.

Models without native tool calling cannot be used, including the Llama models that earlier versions of this page listed.

Model Quality and Latency

Voicebox’s answers are bounded by the model behind them. The same question, against the same knowledge graph with the same configuration, will produce different results on different models.

Stronger models plan better, stay closer to your schema, recover from their own mistakes more often, and are less likely to reference classes or properties that do not exist. Smaller and cheaper models respond faster, but are more likely to answer incorrectly, take more attempts to reach a correct answer, or fail to answer at all.

Because answering takes several steps, per-step speed is not end-to-end speed. A weaker model that needs three extra steps to reach a correct answer can be slower and more expensive overall than a stronger model that gets there directly. On smaller, well-labeled schemas with straightforward questions, a fast mid-tier model often performs comparably at lower cost.

We recommend starting with the latest Claude Haiku, which balances speed and accuracy well for most deployments, and evaluating it against questions representative of your own use. Move to a more capable model if you see incorrect or missing answers. Large or complex schemas, ambiguous terminology, and multi-step analytical questions benefit most from a stronger model.

Anthropic Configuration

Voicebox can use Anthropic as an LLM provider, either directly via the Anthropic API or through AWS Bedrock and Azure AI Foundry endpoints that host Anthropic models.

The following configuration options are used with Anthropic in the Voicebox configuration file.

The following environment variables are used with Anthropic.

When server_url points to an Azure AI Foundry endpoint, you can use either ANTHROPIC_API_KEY or AZURE_API_KEY for authentication. If both are set, ANTHROPIC_API_KEY takes priority.

An example LLM configuration for Anthropic looks like this:

{
    "default_llm_config": {
        "llm_provider": "anthropic",
        "llm_name": "claude-haiku-4-5-20251001"
    }
}

Anthropic models can also be used via AWS Bedrock using the bedrock provider.

Azure AI Foundry Configuration

Voicebox can use an Azure AI endpoint deployed within Azure AI Foundry.

The following configuration options are used with Azure LLM in the Voicebox configuration file. Update the AZURE_AI_ENDPOINT in the server URL to point to your endpoint.

Azure AI also supports custom headers via provider_args.headers.

An example LLM configuration for Azure AI Foundry looks like this:

{
    "default_llm_config": {
        "llm_provider": "azure",
        "llm_name": "gpt-5-mini",
        "server_url": "https://my-endpoint.cognitiveservices.azure.com/models"
    }
}

API Key Authentication

The simplest way to authenticate with Azure AI is using an API key.

Service Principal (SPN) Authentication

As an alternative to API key authentication, Voicebox supports authenticating to Azure AI Services using a Service Principal via the OAuth 2.0 client credentials flow. When SPN variables are configured, the API key is not required.

Client Secret Authentication

Certificate Authentication

If both certificate and client secret are configured, certificate authentication takes priority. If neither SPN method is configured, Voicebox falls back to AZURE_API_KEY.

Additional SPN Configuration

Azure Setup for SPN Authentication

1. Create a Service Principal: Go to Microsoft Entra ID → App registrations → New registration. Name it (e.g. voicebox-service) and register. Note the Application (client) ID and Directory (tenant) ID from the Overview page.

2. Create credentials:
   • Option A: Client Secret — Go to Certificates & secrets → Client secrets → New client secret. Set a description and expiration, then copy the secret Value (shown only once).
   • Option B: Certificate — Generate a certificate, upload the public cert to the app registration, and mount the combined private key + cert PEM file into the Voicebox container. Set AZURE_CLIENT_CERTIFICATE_PATH to the mount path.
3. Assign the Cognitive Services User role: Go to your Azure AI Services resource → Access control (IAM) → Add role assignment. Search for Cognitive Services User, select it, and assign it to your Service Principal.

AWS Bedrock Configuration

Voicebox can use a Bedrock endpoint deployed within AWS.

The following configuration options are used with Bedrock LLM in the Voicebox configuration file.

AWS Bedrock allows users to create application inference profiles to track usage and costs when invoking a model. The ARN associated with an inference profile can be used as the llm_name in Voicebox configuration. All LLM calls initiated by Voicebox will be done using this inference profile.

An example LLM configuration for Bedrock looks like this:

{
    "default_llm_config": {
        "llm_provider": "bedrock",
        "llm_name": "us.anthropic.claude-haiku-4-5-20251001-v1:0"
    }
}

The following environment variables are used with Bedrock.

It is also possible to use IAM roles for accessing Bedrock models instead of specifying access keys if Voicebox service is running on an EC2 instance. The IAM role should have the permissions bedrock:Get*, bedrock:List*, bedrock:InvokeModel, bedrock:InvokeModelWithResponseStream. The AWS built-in policy AmazonBedrockLimitedAccess includes these permissions and can be used directly or a new role can be defined with these permissions.

Once the IAM role containing correct permissions is defined, the role can be attached to the EC2 instance where the Voicebox service is running. For the IAM role to take effect none of the environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, or BEDROCK_PROFILE should be set. When everything is configured correctly, in the Voicebox service logs, you should see a message as follows for the role you have defined:

Found credentials from IAM Role: VoiceboxBedrockRole

Google Vertex Configuration

Voicebox can use models hosted at Google Vertex Model Garden as an LLM endpoint.

The following configuration options are used with Google Vertex in the Voicebox configuration file. Replace the Google_Vertex_AI_Project_Name value in the example with your project name.

Note that, provider_args is a JSON object itself. An example LLM configuration for Google Vertex looks like this:

{  
    "default_llm_config": {    
        "llm_provider": "vertex",
        "llm_name": "claude-haiku-4-5",
        "provider_args": {
          "project" : "My Project Name",
          "location": "global"
        } 
    }
}

The following environment variables are used with Google Vertex.

See Google documentation for the details of creating credential files.

OpenAI Configuration

Voicebox can use OpenAI as an LLM endpoint.

The following configuration options are used with OpenAI in the Voicebox configuration file.

OpenAI also supports custom headers via provider_args.headers.

The following environment variables are used with OpenAI.

Basic Authentication

For OpenAI-compatible proxies that require HTTP Basic authentication instead of an API key, you can configure basic_auth in provider_args. When basic_auth is configured, OPENAI_API_KEY is not required.

{
    "default_llm_config": {
        "llm_provider": "openai",
        "llm_name": "gpt-4o",
        "server_url": "https://your-llm-proxy.example.com/v1/",
        "provider_args": {
            "basic_auth": {
                "client_id": "my-client-id",
                "client_secret": "$LLM_CLIENT_SECRET"
            }
        }
    }
}

The client_id and client_secret are Base64-encoded at runtime to produce an Authorization: Basic <token> header. Values support the same $ENV_VAR template substitution used by provider_args.headers — use environment variable references (e.g. $LLM_CLIENT_SECRET) for sensitive values and hardcoded strings for non-sensitive values like client_id.

An optional auth_scheme field can be set within basic_auth to change the authorization scheme (defaults to "Basic").

Custom Headers

The OpenAI and Azure AI providers support custom HTTP headers included in LLM requests. Custom headers are configured via provider_args.headers in the LLM configuration. Here is an example:

{  
    "default_llm_config": {    
        "llm_provider": "openai",
        "llm_name": "gpt-4o-mini",
        "server_url": "https://api.openai.com/v1/",
        "provider_args": {
          "headers" : {
            "OpenAI-Organization": "org-gnSjNrpIz0bb7V1modfLrNof",
            "OpenAI-Project": "$PROJECT_ID"
          }
        } 
    }
}

Header values support Python string template substitution. Variables referenced in the template string (e.g. $PROJECT_ID) should be defined as environment variables on the Voicebox Service. Environment variables are a better choice for sensitive values whereas non-sensitive values can be directly included in the configuration file.

$VOICEBOX_USER Variable

In addition to environment variables, the special variable $VOICEBOX_USER is automatically resolved to the identity of the user making the request. For web users this is the authenticated username. For Public API requests this is the X-Client-Id header value.

This allows LLM requests to carry user identity, which can be useful for auditing and access control at the LLM provider level:

{  
    "default_llm_config": {    
        "llm_provider": "azure",
        "llm_name": "gpt-5-mini",
        "server_url": "https://my-endpoint.cognitiveservices.azure.com/models",
        "provider_args": {
          "headers" : {
            "X-User-Identity": "$VOICEBOX_USER"
          }
        } 
    }
}

Advanced Customization

For deployments with multiple Stardog servers or databases, you can provide different Voicebox configurations for each endpoint and database combination. This allows you to customize LLM settings, enable/disable features, or use different models based on the target Stardog server.

Configuration Directory

Instead of (or in addition to) a single configuration file, you can specify a directory containing multiple configuration files:

Endpoint and Database Matching

Each configuration file can include endpoint and database fields to specify which Stardog connections it applies to:

{
  "endpoint": "https://stardog-prod.example.com:5820",
  "database": "my-database",
  "default_llm_config": {
    "llm_provider": "bedrock",
    "llm_name": "us.anthropic.claude-haiku-4-5-20251001-v1:0"
  }
}

Matching Priority

When a request is made, Voicebox selects the most specific matching configuration:

1. Exact match - Config with matching endpoint AND database
2. Endpoint wildcard - Config with matching endpoint and database: "*"
3. Global wildcard - Config with endpoint: "*" and database: "*"

Example Setup

/voicebox-config/
├── default.json           # endpoint: "*", database: "*"
├── prod-server.json       # endpoint: "https://prod.example.com:5820", database: "*"
└── prod-analytics.json    # endpoint: "https://prod.example.com:5820", database: "analytics"

With this setup:

• Requests to prod.example.com with database analytics use prod-analytics.json
• Requests to prod.example.com with any other database use prod-server.json
• All other requests use default.json

If both VBX_CONFIG_FILE and VBX_CONFIG_DIR are specified, configurations from both sources are loaded. No two configuration files can have the same endpoint and database combination.

If no matching configuration is found for a request, Voicebox will return an error. You should provide a global default configuration (endpoint: "*", database: "*") unless you intentionally want to disable Voicebox for specific endpoints or databases.

Internal Stardog Endpoint Support

When Voicebox makes requests to Stardog servers, it uses server-side connections that may require different network routing than browser-based requests. To support architectures where the Voicebox service container cannot access Stardog on the public endpoint, Launchpad v3.5.0+ allows you to configure an additional internal endpoint for connections.

When an internal endpoint is configured for a connection, Voicebox automatically uses the internal endpoint while browser-based applications (Studio, Explorer, Designer, Knowledge Catalog) continue using the public endpoint. This is particularly useful in scenarios where:

• Stardog is behind a firewall accessible only within a private network
• Different DNS resolution is needed for internal vs. external access
• Network policies restrict container-to-container communication to internal networks

See the SSO Connection Configuration section and individual provider documentation for details on configuring internal endpoints.

JWT Authentication & Token Exchange

For enterprise deployments requiring OAuth-based authentication between Launchpad, Voicebox, and your LLM Gateway, see the dedicated guide:

JWT Authentication with Okta

This guide covers:

• Okta authorization server setup
• On-Behalf-Of (OBO) token exchange configuration
• Launchpad and Voicebox Service environment variables
• Public API JWT authentication

---