Package com.complexible.stardog.index

Class IndexOptions

java.lang.Object
com.complexible.stardog.index.IndexOptions
All Implemented Interfaces:
MetaPropertyProvider
public final class IndexOptions extends Object implements MetaPropertyProvider

Set of options for configuring the low-level indexes.

Since:
0.6
Version:
5.0
Author:
Michael Grove, Evren Sirin, Fernando Hernandez

  • Field Details

    • SIZE

      public static final InfoProperty<Long> SIZE
      Current size of the database. Provided only for information purposes and cannot be modified by users.

    • BULK_LOAD_DICTIONARY_RESERVED_MEM

      public static final Long BULK_LOAD_DICTIONARY_RESERVED_MEM
      THe amount of memory reserved for system operations when bulk load dictionary is being used,

    • BULK_LOAD_DICTIONARY_MEM

      public static final ConfigProperty<Long> BULK_LOAD_DICTIONARY_MEM
      The amount of memory to be used by the bulk load dictionary. If it is set to 0 or a negative value explicitly by the user, the bulk load dictionary will not be used. If the option is not set by the user and bulk_load memory is being used this option will be automatically set based on memory configuration.

    • PERSIST

      public static final ConfigProperty<Boolean> PERSIST
      Option to determine if the contents of the memory database will be persisted on disk. If memory database is persisted on disk, its contents can be reloaded at a later time when Stardog server is restarted. This option is always true for disk databases.

    • SYNC

      @Deprecated public static final ConfigProperty<Boolean> SYNC
      Deprecated.
      To be removed in 5.1, this option no longer has any effect on mem indexes
      Persistence type for a memory database, Setting this option to true indicates the db should be persisted within the commit. Otherwise, the persist will happen asynchronously after the commit is complete. This flag has no effect against disk indexes.

    • INDEX_TYPE

      public static final ConfigProperty<IndexOptions.IndexType> INDEX_TYPE
      Type of the index.

    • CANONICAL_LITERALS

      public static final ConfigProperty<Boolean> CANONICAL_LITERALS
      Specify that non-string typed literals are canonicalized in the database. Improves query and loading performance, but does change literal values to a canonical form. For example, "1"^^xsd:byte is canonicalized to "1"^^xsd:integer. Set this option to false if you require literals to be exactly as specified, rather than canonicalized. The default value is 'true'. note that, this value can only be set at database creation time and cannot be changed at a future time.

    • INLINE_BOOLEANS

      public static final ConfigProperty<Boolean> INLINE_BOOLEANS

    • INLINING_VERSION

      public static final ConfigProperty<Integer> INLINING_VERSION
      Intended for internal use only, not for users. Specifies the version of the procedure used to canonicalize literals.

    • INDEXES_AGGREGATE

      @Deprecated(since="11.0") public static final ConfigProperty<com.complexible.stardog.index.AggregateIndexes> INDEXES_AGGREGATE
      Deprecated.
      Option to specify whether to maintain aggregate indexes.

    • INDEX_STRATEGY

      public static final InfoProperty<com.complexible.stardog.index.IndexStrategy> INDEX_STRATEGY
      Intended for internal system use only, not for users. Index strategy used by the database.

    • DIFF_INDEX_SIZE

      public static final InfoProperty<Integer> DIFF_INDEX_SIZE
      Current size of the differential index. The differential index is used for storing additions and removals separately from the main index so updates to the database can be performed faster. The query answering will take all the data stored in the main index and the differential index so the query answers will not change. There is slight overhead for query answering with differential indexes if the differential index size gets too large. For this reason, the differential index will be merged to the main index when its size reaches the DIFF_INDEX_MAX_LIMIT. There is no benefit of differential indexes if the main index itself is too small. For this reason, the differential index is not used until the main index size reaches DIFF_INDEX_MAX_LIMIT.

    • DIFF_INDEX_MIN_LIMIT

      public static final ConfigProperty<Integer> DIFF_INDEX_MIN_LIMIT
      The limit for the size of the base index when differential indexes will be used. If the main index size is below this limit, all the changes will be applied to the main index.
      See Also:

    • DIFF_INDEX_MAX_LIMIT

      public static final ConfigProperty<Integer> DIFF_INDEX_MAX_LIMIT
      The limit for the size of the differential index. If the differential index size exceeds this limit, it will automatically be merged to the main index.
      See Also:

    • PROGRESS_OUTPUT

      public static final InfoProperty<PrintStream> PROGRESS_OUTPUT
      Internal use only. Use
      invalid reference 
      DatabaseOptions#PROGRESS_MONITOR_ENABLED
      instead.

    • BULK_LOADER_NAMED_GRAPHS

      public static final InfoProperty<Boolean> BULK_LOADER_NAMED_GRAPHS
      Internal use only

    • BULK_LOADER_ESTIMATED_SIZE

      public static final InfoProperty<Long> BULK_LOADER_ESTIMATED_SIZE
      Internal use only

    • BULK_LOAD_MAX_PARSERS

      public static final ConfigProperty<Integer> BULK_LOAD_MAX_PARSERS

    • BULK_LOAD_MAX_HANDLERS

      public static final ConfigProperty<Integer> BULK_LOAD_MAX_HANDLERS

    • BULK_LOAD_RESIZE_HANDLERS

      public static final ConfigProperty<Boolean> BULK_LOAD_RESIZE_HANDLERS

    • BULK_LOADER_OVERWRITE_DATA

      public static final ConfigProperty<Boolean> BULK_LOADER_OVERWRITE_DATA

    • STATS_UPDATE_COUNT

      public static final InfoProperty<Long> STATS_UPDATE_COUNT
      Internal use only. Number of updated keys, added + removed in total.

    • STATS_UPDATE_DB_MIN_SIZE

      public static final ConfigProperty<Long> STATS_UPDATE_DB_MIN_SIZE
      Minimum number of triples that should be in the database for statistics to be updated automatically.

    • DEPRECATED_STATS_UPDATE_DB_MIN_SIZE

      @Deprecated public static final InfoProperty<Long> DEPRECATED_STATS_UPDATE_DB_MIN_SIZE
      Deprecated.
      use STATS_UPDATE_DB_MIN_SIZE instead
      This is the deprecated option replaced with STATS_UPDATE_DB_MIN_SIZE. The declaration exists only to avoid unnecessary warnings for databases created before version 5.2.3.

    • STATISTICS_ON_DB_CREATION

      public static final ConfigProperty<com.complexible.stardog.index.StatisticsStrategy> STATISTICS_ON_DB_CREATION
      Determines how index statistics is computed when a database is bulk loaded. By default it's computed synchronously.

    • AUTO_STATS_UPDATE

      public static final ConfigProperty<Boolean> AUTO_STATS_UPDATE
      Option to turn on automatic statistics updates. When this option is set to true, Stardog will decide when to update statistics as the database is modified through additions and removals and update statistics as needed. If this option is set to false, Stardog will never update the statistics regardless of how much the database is updated.

    • COMPRESSION

      public static final ConfigProperty<IndexOptions.CompressionType> COMPRESSION
      Option which specifies what kind of compression wil lbe used for data written to disk. Default value: LZ4

    • WRITE_CONFLICT_STRATEGY

      public static final ConfigProperty<IndexOptions.WriteConflictStrategy> WRITE_CONFLICT_STRATEGY

    • WAL_ENABLED

      public static final ConfigProperty<Boolean> WAL_ENABLED
      Option which specifies if databases uses a WAL (Write-ahead Log). WAL ensures committed transactions will not be lost if the server crashes without a proper shutdown. Default value: true

    • BULK_LOAD_SST_CHUNK_SIZE

      public static final ConfigProperty<Long> BULK_LOAD_SST_CHUNK_SIZE
      Option which specifies the maximum number of records will be written to index's sst file during bulk loading; After this threshold exceeds bulk load import current sst file and switches to the new one; Default value: 1_000_000_000L

    • BULK_LOAD_VALUE_SIZE_THRESHOLD

      public static final ConfigProperty<Long> BULK_LOAD_VALUE_SIZE_THRESHOLD
      Option specifies the file size at which dictionary values should switch to a new .sst file. Default value: 500_000_000L

    • TX_BULK_LOAD_HARD_THRESHOLD

      public static final ConfigProperty<Long> TX_BULK_LOAD_HARD_THRESHOLD
      Starting at this limit, the NativeBulkUpdater will be used. If the number of quads in the transaction does not exceed this limit then we fall back to the pipelined writer instead.

    • TX_BULK_LOAD_ADAPTIVE_THRESHOLD

      public static final ConfigProperty<Long> TX_BULK_LOAD_ADAPTIVE_THRESHOLD
      Option which specifies at what point Stardog skips memtables writes and use bulk loading code for transactional updates. The mechanism will first buffer quads in memory until it reaches the threshold, at which point it will switch to bulk loading. Note: this was renamed from TX_BULK_LOAD_WRITE_LIMIT in previous versions, the property name is identical Default value: 495_000L
      Match Spark connector default client buffer size - which is usually estimated around 495k quads at the servers side. ...

    • IS_DICTIONARY_VACUUMING_MULTITHREADED

      public static final ConfigProperty<Boolean> IS_DICTIONARY_VACUUMING_MULTITHREADED
      Option which specifies if the dictionary vacuuming process will be multithreaded; Default value: false

    • STATS_UPDATE_RATIO

      public static final ConfigProperty<Double> STATS_UPDATE_RATIO
      Ratio of updated triples to the number of triples in the database that triggers the automatic statistics recomputation in a background thread. This option has no effect if AUTO_STATS_UPDATE is off or the index size is less than STATS_UPDATE_DB_MIN_SIZE. If the ratio is set to 0.1 then adding 100K triples to a database with 1M triples will trigger the automatic stats recomputation. The added and removed triples are counted separately so adding 50K triples removing 50K triples count as updating 100K triples. Default value: 0.1

    • STATS_UPDATE_BLOCKING_RATIO

      public static final ConfigProperty<Double> STATS_UPDATE_BLOCKING_RATIO
      Ratio of updated triples to the number of triples in the database that triggers the automatic statistics recomputation inside the transaction. This option is similar to STATS_UPDATE_RATIO but it is performed synchronously within the transaction instead of a background thread. The client will not get the commit acknowledgement until the stats is recomputed. This option has no effect if AUTO_STATS_UPDATE is off or the index size is less than STATS_UPDATE_DB_MIN_SIZE. Setting this option to a non-positive number (<= 0) will disable blocking updates. Default value: 0

    • CARDINALITY_CACHE_CAPACITY

      public static final ConfigProperty<Integer> CARDINALITY_CACHE_CAPACITY
      The max capacity for the query pattern cardinality cache that is shared across queries to the same database. Default value: 1024

    • STATISTICS_CHARACTERISTIC_SETS_LIMIT

      public static final ConfigProperty<Integer> STATISTICS_CHARACTERISTIC_SETS_LIMIT
      The max number of characteristic sets computed as a part of the statistical summary of the database. More diverse datasets may require a higher number for more accurate query planning. The downside is higher memory footprint and slower planning. This is a soft limit and the actual number of sets can slightly exceed it. Default value: 10000

    • STATISTICS_RANGE_SAMPLE_SIZE

      public static final ConfigProperty<Integer> STATISTICS_RANGE_SAMPLE_SIZE

    • STATISTICS_CHAINS_ENABLED

      public static final ConfigProperty<Boolean> STATISTICS_CHAINS_ENABLED
      Whether chain statistics is enabled or not. Default value: true

    • STATISTICS_OBJECT_JOIN_ENABLED

      public static final ConfigProperty<Boolean> STATISTICS_OBJECT_JOIN_ENABLED
      Whether property object join statistics is enabled or not. Default value: true

    • STATS_ENABLE_ACTIVE_GRAPH

      public static final ConfigProperty<Boolean> STATS_ENABLE_ACTIVE_GRAPH
      Enabling scaling cardinality estimations based on the active graph and DB statistics Default value: true

    • INDEX_WRITER_MERGE_LIMIT

      public static final ConfigProperty<Integer> INDEX_WRITER_MERGE_LIMIT
      Maximum number of triples to keep in memory for merging interleaving additions and removals while querying uncommitted state.

    • STATISTICS_SKETCH_CAPACITY

      public static final ConfigProperty<Integer> STATISTICS_SKETCH_CAPACITY

    • STATISTICS_RARE_PREDICATE

      public static final ConfigProperty<Double> STATISTICS_RARE_PREDICATE

    • LAST_COMMITTED_TX

      public static final ConfigProperty<UUID> LAST_COMMITTED_TX
      Internal use only - The UUID for the last committed transaction.

    • BNODE_PREFIX_ID

      public static final ConfigProperty<String> BNODE_PREFIX_ID
      Internal use only - The bnode prefix id used for only bulk loading at db creation time.

    • OPTIMIZE_VACUUM_DATA

      public static final ConfigProperty<Boolean> OPTIMIZE_VACUUM_DATA

    • OPTIMIZE_VACUUM_DICTIONARY

      public static final ConfigProperty<Boolean> OPTIMIZE_VACUUM_DICTIONARY

    • OPTIMIZE_STATISTICS

      public static final ConfigProperty<Boolean> OPTIMIZE_STATISTICS

    • OPTIMIZE_COMPACT

      public static final ConfigProperty<Boolean> OPTIMIZE_COMPACT

    • OPTIMIZE_LITERAL_INDEX

      public static final ConfigProperty<Boolean> OPTIMIZE_LITERAL_INDEX

    • OPTIMIZE_DISK_SPACE_FORCE

      public static final ConfigProperty<Boolean> OPTIMIZE_DISK_SPACE_FORCE

    • OPTIMIZE_THROTTLE_OVERRIDE

      public static final ConfigProperty<Boolean> OPTIMIZE_THROTTLE_OVERRIDE

    • ASYNC_FLUSH

      public static final boolean ASYNC_FLUSH

  • Method Details

    • isMemory

      public static boolean isMemory(Metadata theOptions)

    • isDisk

      public static boolean isDisk(Metadata theOptions)

    • memory

      public static Metadata memory()
      Options for creating a memory database. Memory databases are only to be used for testing and prototyping. They're not intended for production use.
      Returns:
      default options for a memory database

    • disk

      public static Metadata disk()

    • isPersist

      public static boolean isPersist(Metadata theOptions)
      Return whether or not the PERSIST option is set
      Parameters:
      theOptions - the options
      Returns:
      true if the persist option is set, false otherwise

    • progressOutput

      public static Optional<PrintStream> progressOutput(Metadata theOptions)