Storage-System Service Configuration

Table of Contents

Introduction
Default Values
Caching
Event Bus Configuration
Configuration
- Environment Variables
- YAML Example

Introduction

The Infinite Scale Storage-System service persists and caches user related data that is defined via Infinite Scale. This can be among other data role assignments, user settings and users shares.

Default Values

Storage-System listens on port 9215 by default.

Caching

The storage-system service can use a configured store via the global OCIS_CACHE_STORE environment variable.

Note that for each global environment variable, an independent service-based one might be available additionally. For precedences see Environment Variable Notes. Check the configuration section below. Supported stores are:

Store Type Description

Store Type	Description
`memory`	Basic in-memory store. Will not survive a restart. Usually the default for caches. See the store environment variable for which one is used.
`nats-js-kv`	Stores data using key-value-store feature of NATS JetStream. Usually the default for stores, see the store environment variable for which one is used.
`redis-sentinel`	Stores data in a configured Redis Sentinel cluster.
`noop`	Stores nothing. Useful for testing. Not recommended in production environments.

memory

Basic in-memory store. Will not survive a restart.
Usually the default for caches. See the store environment variable for which one is used.

nats-js-kv

Stores data using key-value-store feature of NATS JetStream.
Usually the default for stores, see the store environment variable for which one is used.

redis-sentinel

Stores data in a configured Redis Sentinel cluster.

noop

Stores nothing. Useful for testing. Not recommended in production environments.

The storage-system service can only be scaled if not using the memory store and the stores are configured identically over all instances!

If you have used one of the deprecated stores of a former version, you should reconfigure to use one of the supported ones as the deprecated stores will be removed in a later version.

Store specific notes

When using redis-sentinel:
The Redis master to use is configured via e.g. OCIS_CACHE_STORE_NODES in the form of <sentinel-host>:<sentinel-port>/<redis-master> like 10.10.0.200:26379/mymaster.
When using nats-js-kv:
- It is recommended to set OCIS_CACHE_STORE_NODES to the same value as OCIS_EVENTS_ENDPOINT. That way the cache uses the same nats instance as the event bus. See the Event Bus Configuration for more details.
- Authentication can be added, if configured, via OCIS_CACHE_AUTH_USERNAME and OCIS_CACHE_AUTH_PASSWORD.
- It is possible to set OCIS_CACHE_DISABLE_PERSISTENCE to instruct nats to not persist cache data on disc.

Event Bus Configuration

The Infinite Scale event bus can be configured by a set of environment variables.

If you are using a binary installation as described in Minimal Bare Metal Deployment or Bare Metal with systemd, the address of the event bus OCIS_EVENTS_ENDPOINT is predefined as localhost address without the need for further configuration, but changeable on demand.
In case of an orchestrated installation like with Docker or Kubernetes, the event bus must be an external service for scalability like a Redis Sentinel cluster or a key-value-store NATS JetStream. Both named stores are supported and also used in Caching and Persistence. The store used is not part of the Infinite Scale installation and must be separately provided and configured.
Note that from a configuration point of view, caching and persistence are independent of the event bus configuration.

Note that for each global environment variable, a service-based one might be available additionally. For precedences see Environment Variable Notes. Check the configuration section below.

Without the aim of completeness, see the list of environment variables to configure the event bus:

Envvar Description

Envvar	Description
`OCIS_EVENTS_ENDPOINT`	The address of the event system.
`OCIS_EVENTS_CLUSTER`	The clusterID of the event system. Mandatory when using NATS as event system.
`OCIS_EVENTS_ENABLE_TLS`	Enable TLS for the connection to the events broker.
`OCIS_INSECURE`	Whether to verify the server TLS certificates.
`OCIS_EVENTS_AUTH_USERNAME`	The username to authenticate with the events broker.
`OCIS_EVENTS_AUTH_PASSWORD`	The password to authenticate with the events broker.

OCIS_EVENTS_ENDPOINT

The address of the event system.

OCIS_EVENTS_CLUSTER

The clusterID of the event system. Mandatory when using NATS as event system.

OCIS_EVENTS_ENABLE_TLS

Enable TLS for the connection to the events broker.

OCIS_INSECURE

Whether to verify the server TLS certificates.

OCIS_EVENTS_AUTH_USERNAME

The username to authenticate with the events broker.

OCIS_EVENTS_AUTH_PASSWORD

The password to authenticate with the events broker.

Configuration

Environment Variables

The storage-system service is configured via the following environment variables. Read the Environment Variable Types documentation for important details. Column IV shows with which release the environment variable has been introduced.

7.0.0

Environment variables for the storage-system service
Name	IV	Type	Default Value	Description
`OCIS_TRACING_ENABLED` `STORAGE_SYSTEM_TRACING_ENABLED`	pre5.0	bool	false	Activates tracing.
`OCIS_TRACING_TYPE` `STORAGE_SYSTEM_TRACING_TYPE`	pre5.0	string		The type of tracing. Defaults to '', which is the same as 'jaeger'. Allowed tracing types are 'jaeger' and '' as of now.
`OCIS_TRACING_ENDPOINT` `STORAGE_SYSTEM_TRACING_ENDPOINT`	pre5.0	string		The endpoint of the tracing agent.
`OCIS_TRACING_COLLECTOR` `STORAGE_SYSTEM_TRACING_COLLECTOR`	pre5.0	string		The HTTP endpoint for sending spans directly to a collector, i.e. http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset.
`OCIS_LOG_LEVEL` `STORAGE_SYSTEM_LOG_LEVEL`	pre5.0	string		The log level. Valid values are: 'panic', 'fatal', 'error', 'warn', 'info', 'debug', 'trace'.
`OCIS_LOG_PRETTY` `STORAGE_SYSTEM_LOG_PRETTY`	pre5.0	bool	false	Activates pretty log output.
`OCIS_LOG_COLOR` `STORAGE_SYSTEM_LOG_COLOR`	pre5.0	bool	false	Activates colorized log output.
`OCIS_LOG_FILE` `STORAGE_SYSTEM_LOG_FILE`	pre5.0	string		The path to the log file. Activates logging to this file if set.
`STORAGE_SYSTEM_DEBUG_ADDR`	pre5.0	string	127.0.0.1:9217	Bind address of the debug server, where metrics, health, config and debug endpoints will be exposed.
`STORAGE_SYSTEM_DEBUG_TOKEN`	pre5.0	string		Token to secure the metrics endpoint
`STORAGE_SYSTEM_DEBUG_PPROF`	pre5.0	bool	false	Enables pprof, which can be used for profiling
`STORAGE_SYSTEM_DEBUG_ZPAGES`	pre5.0	bool	false	Enables zpages, which can be used for collecting and viewing in-memory traces.
`STORAGE_SYSTEM_GRPC_ADDR`	pre5.0	string	127.0.0.1:9215	The bind address of the GRPC service.
`OCIS_GRPC_PROTOCOL` `STORAGE_SYSTEM_GRPC_PROTOCOL`	pre5.0	string	tcp	The transport protocol of the GPRC service.
`STORAGE_SYSTEM_HTTP_ADDR`	pre5.0	string	127.0.0.1:9216	The bind address of the HTTP service.
`STORAGE_SYSTEM_HTTP_PROTOCOL`	pre5.0	string	tcp	The transport protocol of the HTTP service.
`OCIS_JWT_SECRET` `STORAGE_SYSTEM_JWT_SECRET`	pre5.0	string		The secret to mint and validate jwt tokens.
`OCIS_REVA_GATEWAY`	pre5.0	string	com.owncloud.api.gateway	The CS3 gateway endpoint.
`OCIS_GRPC_CLIENT_TLS_MODE`	pre5.0	string		TLS mode for grpc connection to the go-micro based grpc services. Possible values are 'off', 'insecure' and 'on'. 'off': disables transport security for the clients. 'insecure' allows using transport security, but disables certificate verification (to be used with the autogenerated self-signed certificates). 'on' enables transport security, including server certificate verification.
`OCIS_GRPC_CLIENT_TLS_CACERT`	pre5.0	string		Path/File name for the root CA certificate (in PEM format) used to validate TLS server certificates of the go-micro based grpc services.
`OCIS_SYSTEM_USER_ID`	pre5.0	string		ID of the oCIS storage-system system user. Admins need to set the ID for the STORAGE-SYSTEM system user in this config option which is then used to reference the user. Any reasonable long string is possible, preferably this would be an UUIDv4 format.
`OCIS_SYSTEM_USER_API_KEY`	pre5.0	string		API key for the STORAGE-SYSTEM system user.
`STORAGE_SYSTEM_SKIP_USER_GROUPS_IN_TOKEN`	pre5.0	bool	false	Disables the loading of user’s group memberships from the reva access token.
`OCIS_CACHE_STORE` `STORAGE_SYSTEM_CACHE_STORE`	pre5.0	string	memory	The type of the cache store. Supported values are: 'memory', 'redis-sentinel', 'nats-js-kv', 'noop'. See the text description for details.
`OCIS_CACHE_STORE_NODES` `STORAGE_SYSTEM_CACHE_STORE_NODES`	pre5.0	[]string	[127.0.0.1:9233]	A list of nodes to access the configured store. This has no effect when 'memory' store is configured. Note that the behaviour how nodes are used is dependent on the library of the configured store. See the Environment Variable Types description for more details.
`OCIS_CACHE_DATABASE`	pre5.0	string	storage-system	The database name the configured store should use.
`OCIS_CACHE_TTL` `STORAGE_SYSTEM_CACHE_TTL`	pre5.0	Duration	24m0s	Default time to live for user info in the user info cache. Only applied when access tokens has no expiration. See the Environment Variable Types description for more details.
`OCIS_CACHE_DISABLE_PERSISTENCE` `STORAGE_SYSTEM_CACHE_DISABLE_PERSISTENCE`	5.0	bool	false	Disables persistence of the cache. Only applies when store type 'nats-js-kv' is configured. Defaults to false.
`OCIS_CACHE_AUTH_USERNAME` `STORAGE_SYSTEM_CACHE_AUTH_USERNAME`	5.0	string		Username for the configured store. Only applies when store type 'nats-js-kv' is configured.
`OCIS_CACHE_AUTH_PASSWORD` `STORAGE_SYSTEM_CACHE_AUTH_PASSWORD`	5.0	string		Password for the configured store. Only applies when store type 'nats-js-kv' is configured.
`STORAGE_SYSTEM_DRIVER`	pre5.0	string	ocis	The driver which should be used by the service.
`STORAGE_SYSTEM_OCIS_ROOT`	pre5.0	string	/var/lib/ocis/storage/metadata	Path for the directory where the STORAGE-SYSTEM service stores it’s persistent data. If not defined, the root directory derives from $OCIS_BASE_DATA_PATH/storage.
`STORAGE_SYSTEM_OCIS_MAX_ACQUIRE_LOCK_CYCLES`	pre5.0	int	20	When trying to lock files, ocis will try this amount of times to acquire the lock before failing. After each try it will wait for an increasing amount of time. Values of 0 or below will be ignored and the default value of 20 will be used.
`STORAGE_SYSTEM_OCIS_LOCK_CYCLE_DURATION_FACTOR`	pre5.0	int	30	When trying to lock files, ocis will multiply the cycle with this factor and use it as a millisecond timeout. Values of 0 or below will be ignored and the default value of 30 will be used.
`STORAGE_SYSTEM_DATA_SERVER_URL`	pre5.0	string	http://localhost:9216/data	URL of the data server, needs to be reachable by other services using this service.

YAML Example

Note the file shown below must be renamed and placed in the correct folder according to the Configuration File Naming conventions to be effective.
See the Notes for Environment Variables if you want to use environment variables in the yaml file.

7.0.0

# Autogenerated
# Filename: storage-system-config-example.yaml

tracing:
  enabled: false
  type: ""
  endpoint: ""
  collector: ""
log:
  level: ""
  pretty: false
  color: false
  file: ""
debug:
  addr: 127.0.0.1:9217
  token: ""
  pprof: false
  zpages: false
grpc:
  addr: 127.0.0.1:9215
  tls: null
  protocol: tcp
http:
  addr: 127.0.0.1:9216
  protocol: tcp
token_manager:
  jwt_secret: ""
reva:
  address: com.owncloud.api.gateway
  tls:
    mode: ""
    cacert: ""
system_user_id: ""
system_user_api_key: ""
skip_user_groups_in_token: false
cache:
  store: memory
  nodes:
  - 127.0.0.1:9233
  database: storage-system
  ttl: 24m0s
  disable_persistence: false
  auth_username: ""
  auth_password: ""
driver: ocis
drivers:
  ocis:
    root: /var/lib/ocis/storage/metadata
    max_acquire_lock_cycles: 20
    lock_cycle_duration_factor: 30
data_server_url: http://localhost:9216/data