Storage-System Service Configuration

Introduction

The Infinite Scale Storage-System service

Default Values

  • Storage-System listens on port 9215 by default.

Configuration

Environment Variables

The storage-system service is configured via the following environment variables:

  • latest

  • 2.0.0

Environment variables for the storage-system service
Name Type Default Value Description

OCIS_TRACING_ENABLED
STORAGE_SYSTEM_TRACING_ENABLED

bool

false

Activates tracing.

OCIS_TRACING_TYPE
STORAGE_SYSTEM_TRACING_TYPE

string

The type of tracing. Defaults to "", which is the same as "jaeger". Allowed tracing types are "jaeger" and "" as of now.

OCIS_TRACING_ENDPOINT
STORAGE_SYSTEM_TRACING_ENDPOINT

string

The endpoint of the tracing agent.

OCIS_TRACING_COLLECTOR
STORAGE_SYSTEM_TRACING_COLLECTOR

string

The HTTP endpoint for sending spans directly to a collector, i.e. http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset.

OCIS_LOG_LEVEL
STORAGE_SYSTEM_LOG_LEVEL

string

The log level. Valid values are: "panic", "fatal", "error", "warn", "info", "debug", "trace".

OCIS_LOG_PRETTY
STORAGE_SYSTEM_LOG_PRETTY

bool

false

Activates pretty log output.

OCIS_LOG_COLOR
STORAGE_SYSTEM_LOG_COLOR

bool

false

Activates colorized log output.

OCIS_LOG_FILE
STORAGE_SYSTEM_LOG_FILE

string

The path to the log file. Activates logging to this file if set.

STORAGE_SYSTEM_DEBUG_ADDR

string

127.0.0.1:9217

Bind address of the debug server, where metrics, health, config and debug endpoints will be exposed.

STORAGE_SYSTEM_DEBUG_TOKEN

string

Token to secure the metrics endpoint

STORAGE_SYSTEM_DEBUG_PPROF

bool

false

Enables pprof, which can be used for profiling

STORAGE_SYSTEM_DEBUG_ZPAGES

bool

false

Enables zpages, which can be used for collecting and viewing in-memory traces.

STORAGE_SYSTEM_GRPC_ADDR

string

127.0.0.1:9215

The bind address of the GRPC service.

STORAGE_SYSTEM_GRPC_PROTOCOL

string

tcp

The transport protocol of the GPRC service.

STORAGE_SYSTEM_HTTP_ADDR

string

127.0.0.1:9216

The bind address of the HTTP service.

STORAGE_SYSTEM_HTTP_PROTOCOL

string

tcp

The transport protocol of the HTTP service.

OCIS_JWT_SECRET
STORAGE_SYSTEM_JWT_SECRET

string

The secret to mint and validate jwt tokens.

REVA_GATEWAY

string

127.0.0.1:9142

The CS3 gateway endpoint.

OCIS_GRPC_CLIENT_TLS_MODE

string

TLS mode for grpc connection to the go-micro based grpc services. Possible values are 'off', 'insecure' and 'on'. 'off': disables transport security for the clients. 'insecure' allows to use transport security, but disables certificate verification (to be used with the autogenerated self-signed certificates). 'on' enables transport security, including server ceritificate verification.

OCIS_GRPC_CLIENT_TLS_CACERT

string

Path/File name for the root CA certificate (in PEM format) used to validate TLS server certificates of the go-micro based grpc services.

OCIS_SYSTEM_USER_ID

string

ID of the oCIS storage-system system user. Admins need to set the ID for the STORAGE-SYSTEM system user in this config option which is then used to reference the user. Any reasonable long string is possible, preferably this would be an UUIDv4 format.

OCIS_SYSTEM_USER_API_KEY

string

API key for the STORAGE-SYSTEM system user.

STORAGE_SYSTEM_SKIP_USER_GROUPS_IN_TOKEN

bool

false

Disables the loading of user’s group memberships from the reva access token.

OCIS_CACHE_STORE
STORAGE_SYSTEM_CACHE_STORE
OCIS_CACHE_STORE_TYPE

string

memory

Store implementation for the cache. Supported values are 'memory' (default), 'redis', 'redis-sentinel', 'nats-js', 'etcd' and 'noop'. See the text description for details.

OCIS_CACHE_STORE_NODES
STORAGE_SYSTEM_CACHE_NODES
OCIS_CACHE_STORE_ADDRESS

[]string

[]

A comma separated list of nodes to access the configured store. This has no effect when the 'memory' store is configured. Note that the behaviour how nodes are used is dependent on the library of the configured store.

STORAGE_SYSTEM_CACHE_DATABASE

string

system

The database name the configured store should use.

STORAGE_SYSTEM_DRIVER

string

ocis

The driver which should be used by the service.

OCIS_DECOMPOSEDFS_METADATA_BACKEND
STORAGE_SYSTEM_OCIS_METADATA_BACKEND

string

xattrs

The backend to use for storing metadata. Supported values are 'xattrs' and 'ini'. The setting 'xattrs' uses extended attributes to store file metadata while 'ini' uses a dedicated file to store file metadata. Defaults to 'xattrs'.

STORAGE_SYSTEM_OCIS_ROOT

string

~/.ocis/storage/metadata

Path for the directory where the STORAGE-SYSTEM service stores it’s persistent data. If not definied, the root directory derives from $OCIS_BASE_DATA_PATH:/storage.

STORAGE_SYSTEM_OCIS_MAX_ACQUIRE_LOCK_CYCLES

int

20

When trying to lock files, ocis will try this amount of times to acquire the lock before failing. After each try it will wait for an increasing amount of time. Values of 0 or below will be ignored and the default value of 20 will be used.

STORAGE_SYSTEM_OCIS_LOCK_CYCLE_DURATION_FACTOR

int

30

When trying to lock files, ocis will multiply the cycle with this factor and use it as a millisecond timeout. Values of 0 or below will be ignored and the default value of 30 will be used.

STORAGE_SYSTEM_DATA_SERVER_URL

string

http://localhost:9216/data

URL of the data server, needs to be reachable by other services using this service.
Environment variables for the storage-system service
Name Type Default Value Description

OCIS_TRACING_ENABLED
STORAGE_SYSTEM_TRACING_ENABLED

bool

false

Activates tracing.

OCIS_TRACING_TYPE
STORAGE_SYSTEM_TRACING_TYPE

string

The type of tracing. Defaults to "", which is the same as "jaeger". Allowed tracing types are "jaeger" and "" as of now.

OCIS_TRACING_ENDPOINT
STORAGE_SYSTEM_TRACING_ENDPOINT

string

The endpoint of the tracing agent.

OCIS_TRACING_COLLECTOR
STORAGE_SYSTEM_TRACING_COLLECTOR

string

The HTTP endpoint for sending spans directly to a collector, i.e. http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset.

OCIS_LOG_LEVEL
STORAGE_SYSTEM_LOG_LEVEL

string

The log level. Valid values are: "panic", "fatal", "error", "warn", "info", "debug", "trace".

OCIS_LOG_PRETTY
STORAGE_SYSTEM_LOG_PRETTY

bool

false

Activates pretty log output.

OCIS_LOG_COLOR
STORAGE_SYSTEM_LOG_COLOR

bool

false

Activates colorized log output.

OCIS_LOG_FILE
STORAGE_SYSTEM_LOG_FILE

string

The path to the log file. Activates logging to this file if set.

STORAGE_SYSTEM_DEBUG_ADDR

string

127.0.0.1:9217

Bind address of the debug server, where metrics, health, config and debug endpoints will be exposed.

STORAGE_SYSTEM_DEBUG_TOKEN

string

Token to secure the metrics endpoint

STORAGE_SYSTEM_DEBUG_PPROF

bool

false

Enables pprof, which can be used for profiling

STORAGE_SYSTEM_DEBUG_ZPAGES

bool

false

Enables zpages, which can be used for collecting and viewing in-memory traces.

STORAGE_SYSTEM_GRPC_ADDR

string

127.0.0.1:9215

The bind address of the GRPC service.

OCIS_GRPC_TLS_ENABLED

bool

false

Activates TLS for the grpc based services using the server certifcate and key configured via OCIS_GRPC_TLS_CERTIFICATE and OCIS_GRPC_TLS_KEY. If OCIS_GRPC_TLS_CERTIFICATE is not set a temporary server certificate is generated - to be used with OCIS_GRPC_CLIENT_TLS_MODE=insecure.

OCIS_GRPC_TLS_CERTIFICATE

string

Path/File name of the TLS server certificate (in PEM format) for the grpc services.

OCIS_GRPC_TLS_KEY

string

Path/File name for the TLS certificate key (in PEM format) for the server certificate to use for the grpc services.

STORAGE_SYSTEM_GRPC_PROTOCOL

string

tcp

The transport protocol of the GPRC service.

STORAGE_SYSTEM_HTTP_ADDR

string

127.0.0.1:9216

The bind address of the HTTP service.

STORAGE_SYSTEM_HTTP_PROTOCOL

string

tcp

The transport protocol of the HTTP service.

OCIS_JWT_SECRET
STORAGE_SYSTEM_JWT_SECRET

string

The secret to mint and validate jwt tokens.

REVA_GATEWAY

string

127.0.0.1:9142

The CS3 gateway endpoint.

OCIS_GRPC_CLIENT_TLS_MODE

string

TLS mode for grpc connection to the go-micro based grpc services. Possible values are 'off', 'insecure' and 'on'. 'off': disables transport security for the clients. 'insecure' allows to use transport security, but disables certificate verification (to be used with the autogenerated self-signed certificates). 'on' enables transport security, including server ceritificate verification.

OCIS_GRPC_CLIENT_TLS_CACERT

string

Path/File name for the root CA certificate (in PEM format) used to validate TLS server certificates of the go-micro based grpc services.

OCIS_SYSTEM_USER_ID

string

ID of the oCIS storage-system system user. Admins need to set the ID for the STORAGE-SYSTEM system user in this config option which is then used to reference the user. Any reasonable long string is possible, preferably this would be an UUIDv4 format.

OCIS_SYSTEM_USER_API_KEY

string

API key for the STORAGE-SYSTEM system user.

STORAGE_SYSTEM_SKIP_USER_GROUPS_IN_TOKEN

bool

false

Disables the loading of user’s group memberships from the reva access token.

STORAGE_SYSTEM_DRIVER

string

ocis

The driver which should be used by the service.

STORAGE_SYSTEM_OCIS_ROOT

string

~/.ocis/storage/metadata

Path for the directory where the STORAGE-SYSTEM service stores it’s persistent data. If not definied, the root directory derives from $OCIS_BASE_DATA_PATH:/storage.

STORAGE_SYSTEM_OCIS_MAX_ACQUIRE_LOCK_CYCLES

int

20

When trying to lock files, ocis will try this amount of times to acquire the lock before failing. After each try it will wait for an increasing amount of time. Values of 0 or below will be ignored and the default value of 20 will be used.

STORAGE_SYSTEM_OCIS_LOCK_CYCLE_DURATION_FACTOR

int

30

When trying to lock files, ocis will multiply the cycle with this factor and use it as a millisecond timeout. Values of 0 or below will be ignored and the default value of 30 will be used.

STORAGE_SYSTEM_DATA_SERVER_URL

string

http://localhost:9216/data

URL of the data server, needs to be reachable by other services using this service.

YAML Example

Note that the filename shown below has been chosen on purpose.
See the Configuration File Naming for details when setting up your own configuration.

  • latest

  • 2.0.0

# Autogenerated
# Filename: storage-system-config-example.yaml

tracing:
  enabled: false
  type: ""
  endpoint: ""
  collector: ""
log:
  level: ""
  pretty: false
  color: false
  file: ""
debug:
  addr: 127.0.0.1:9217
  token: ""
  pprof: false
  zpages: false
grpc:
  addr: 127.0.0.1:9215
  tls: null
  protocol: tcp
http:
  addr: 127.0.0.1:9216
  protocol: tcp
token_manager:
  jwt_secret: ""
reva:
  address: 127.0.0.1:9142
  tls:
    mode: ""
    cacert: ""
system_user_id: ""
system_user_api_key: ""
skip_user_groups_in_token: false
cache:
  store: memory
  nodes: []
  database: system
driver: ocis
drivers:
  ocis:
    metadata_backend: xattrs
    root: ~/.ocis/storage/metadata
    max_acquire_lock_cycles: 20
    lock_cycle_duration_factor: 30
data_server_url: http://localhost:9216/data
# Autogenerated
# Filename: storage-system-config-example.yaml

tracing:
  enabled: false
  type: ""
  endpoint: ""
  collector: ""
log:
  level: ""
  pretty: false
  color: false
  file: ""
debug:
  addr: 127.0.0.1:9217
  token: ""
  pprof: false
  zpages: false
grpc:
  addr: 127.0.0.1:9215
  tls:
    enabled: false
    cert: ""
    key: ""
  protocol: tcp
http:
  addr: 127.0.0.1:9216
  protocol: tcp
token_manager:
  jwt_secret: ""
reva:
  address: 127.0.0.1:9142
  tls:
    mode: ""
    cacert: ""
system_user_id: ""
system_user_api_key: ""
skip_user_groups_in_token: false
driver: ocis
drivers:
  ocis:
    root: ~/.ocis/storage/metadata
    max_acquire_lock_cycles: 20
    lock_cycle_duration_factor: 30
data_server_url: http://localhost:9216/data