Storage-System Service Configuration
Caching
The frontend service can use a configured store via STORAGE_SYSTEM_CACHE_STORE. Possible stores are:
| Store Type | Description |
|---|---|
|
Basic in-memory store and the default. |
|
Advanced in-memory store allowing max size. |
|
Stores data in a configured Redis cluster. |
|
Stores data in a configured Redis Sentinel cluster. |
|
Stores data in a configured etcd cluster. |
|
Stores data using the key-value-store feature of NATS JetStream. |
|
Stores nothing. Useful for testing. Not recommended in production environments. |
-
Note that in-memory stores are by nature not reboot-persistent.
-
Though usually not necessary, a database name and a database table can be configured for event stores if the event store supports this. Generally not applicable for stores of type
in-memory. These settings are blank by default which means that the standard settings of the configured store apply. -
The storage-system service can be scaled if not using
in-memorystores and the stores are configured identically over all instances. -
When using
redis-sentinel, the Redis master to use is configured viaSTORAGE_SYSTEM_CACHE_STORE_NODESin the form of<sentinel-host>:<sentinel-port>/<redis-master>like10.10.0.200:26379/mymaster.
Deprecated Metadata Backend
Starting with Infinite Scale version 3.0.0, the default backend for metadata switched to messagepack. If the setting STORAGE_SYSTEM_OCIS_METADATA_BACKEND has not been defined manually, the backend will be migrated to messagepack automatically. Though still possible to manually configure xattrs, this setting should not be used anymore as it will be removed in a later version.
Configuration
Environment Variables
The storage-system service is configured via the following environment variables:
| Deprecation Info | Deprecation Version | Removal Version | Deprecation Replacement |
|---|
| Name | Type | Default Value | Description |
|---|---|---|---|
|
bool |
false |
Activates tracing. |
|
string |
|
The type of tracing. Defaults to '', which is the same as 'jaeger'. Allowed tracing types are 'jaeger' and '' as of now. |
|
string |
|
The endpoint of the tracing agent. |
|
string |
|
The HTTP endpoint for sending spans directly to a collector, i.e. http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset. |
|
string |
|
The log level. Valid values are: 'panic', 'fatal', 'error', 'warn', 'info', 'debug', 'trace'. |
|
bool |
false |
Activates pretty log output. |
|
bool |
false |
Activates colorized log output. |
|
string |
|
The path to the log file. Activates logging to this file if set. |
|
string |
127.0.0.1:9217 |
Bind address of the debug server, where metrics, health, config and debug endpoints will be exposed. |
|
string |
|
Token to secure the metrics endpoint |
|
bool |
false |
Enables pprof, which can be used for profiling |
|
bool |
false |
Enables zpages, which can be used for collecting and viewing in-memory traces. |
|
string |
127.0.0.1:9215 |
The bind address of the GRPC service. |
|
string |
tcp |
The transport protocol of the GPRC service. |
|
string |
127.0.0.1:9216 |
The bind address of the HTTP service. |
|
string |
tcp |
The transport protocol of the HTTP service. |
|
string |
|
The secret to mint and validate jwt tokens. |
|
string |
com.owncloud.api.gateway |
The CS3 gateway endpoint. |
|
string |
|
TLS mode for grpc connection to the go-micro based grpc services. Possible values are 'off', 'insecure' and 'on'. 'off': disables transport security for the clients. 'insecure' allows using transport security, but disables certificate verification (to be used with the autogenerated self-signed certificates). 'on' enables transport security, including server certificate verification. |
|
string |
|
Path/File name for the root CA certificate (in PEM format) used to validate TLS server certificates of the go-micro based grpc services. |
|
string |
|
ID of the oCIS storage-system system user. Admins need to set the ID for the STORAGE-SYSTEM system user in this config option which is then used to reference the user. Any reasonable long string is possible, preferably this would be an UUIDv4 format. |
|
string |
|
API key for the STORAGE-SYSTEM system user. |
|
bool |
false |
Disables the loading of user’s group memberships from the reva access token. |
|
string |
memory |
The type of the cache store. Supported values are: 'memory', 'ocmem', 'etcd', 'redis', 'redis-sentinel', 'nats-js', 'noop'. See the text description for details. |
|
[]string |
[] |
A comma separated list of nodes to access the configured store. This has no effect when 'memory' or 'ocmem' stores are configured. Note that the behaviour how nodes are used is dependent on the library of the configured store. |
|
string |
ocis |
The database name the configured store should use. |
|
Duration |
24m0s |
Default time to live for user info in the user info cache. Only applied when access tokens has no expiration. The duration can be set as number followed by a unit identifier like s, m or h. Defaults to '10s' (10 seconds). |
|
int |
0 |
The maximum quantity of items in the user info cache. Only applies when store type 'ocmem' is configured. Defaults to 512. |
|
string |
ocis |
The driver which should be used by the service. |
|
string |
messagepack |
The backend to use for storing metadata. Supported values are 'messagepack' and 'xattrs'. The setting 'messagepack' uses a dedicated file to store file metadata while 'xattrs' uses extended attributes to store file metadata. Defaults to 'messagepack'. |
|
string |
~/.ocis/storage/metadata |
Path for the directory where the STORAGE-SYSTEM service stores it’s persistent data. If not defined, the root directory derives from $OCIS_BASE_DATA_PATH:/storage. |
|
int |
20 |
When trying to lock files, ocis will try this amount of times to acquire the lock before failing. After each try it will wait for an increasing amount of time. Values of 0 or below will be ignored and the default value of 20 will be used. |
|
int |
30 |
When trying to lock files, ocis will multiply the cycle with this factor and use it as a millisecond timeout. Values of 0 or below will be ignored and the default value of 30 will be used. |
|
string |
http://localhost:9216/data |
URL of the data server, needs to be reachable by other services using this service. |
| Deprecation Info | Deprecation Version | Removal Version | Deprecation Replacement |
|---|
| Name | Type | Default Value | Description |
|---|---|---|---|
|
bool |
false |
Activates tracing. |
|
string |
|
The type of tracing. Defaults to '', which is the same as 'jaeger'. Allowed tracing types are 'jaeger' and '' as of now. |
|
string |
|
The endpoint of the tracing agent. |
|
string |
|
The HTTP endpoint for sending spans directly to a collector, i.e. http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset. |
|
string |
|
The log level. Valid values are: 'panic', 'fatal', 'error', 'warn', 'info', 'debug', 'trace'. |
|
bool |
false |
Activates pretty log output. |
|
bool |
false |
Activates colorized log output. |
|
string |
|
The path to the log file. Activates logging to this file if set. |
|
string |
127.0.0.1:9217 |
Bind address of the debug server, where metrics, health, config and debug endpoints will be exposed. |
|
string |
|
Token to secure the metrics endpoint |
|
bool |
false |
Enables pprof, which can be used for profiling |
|
bool |
false |
Enables zpages, which can be used for collecting and viewing in-memory traces. |
|
string |
127.0.0.1:9215 |
The bind address of the GRPC service. |
|
string |
tcp |
The transport protocol of the GPRC service. |
|
string |
127.0.0.1:9216 |
The bind address of the HTTP service. |
|
string |
tcp |
The transport protocol of the HTTP service. |
|
string |
|
The secret to mint and validate jwt tokens. |
|
string |
com.owncloud.api.gateway |
The CS3 gateway endpoint. |
|
string |
|
TLS mode for grpc connection to the go-micro based grpc services. Possible values are 'off', 'insecure' and 'on'. 'off': disables transport security for the clients. 'insecure' allows using transport security, but disables certificate verification (to be used with the autogenerated self-signed certificates). 'on' enables transport security, including server certificate verification. |
|
string |
|
Path/File name for the root CA certificate (in PEM format) used to validate TLS server certificates of the go-micro based grpc services. |
|
string |
|
ID of the oCIS storage-system system user. Admins need to set the ID for the STORAGE-SYSTEM system user in this config option which is then used to reference the user. Any reasonable long string is possible, preferably this would be an UUIDv4 format. |
|
string |
|
API key for the STORAGE-SYSTEM system user. |
|
bool |
false |
Disables the loading of user’s group memberships from the reva access token. |
|
string |
memory |
The type of the cache store. Supported values are: 'memory', 'ocmem', 'etcd', 'redis', 'redis-sentinel', 'nats-js', 'noop'. See the text description for details. |
|
[]string |
[] |
A comma separated list of nodes to access the configured store. This has no effect when 'memory' or 'ocmem' stores are configured. Note that the behaviour how nodes are used is dependent on the library of the configured store. |
|
string |
ocis |
The database name the configured store should use. |
|
Duration |
24m0s |
Default time to live for user info in the user info cache. Only applied when access tokens has no expiration. The duration can be set as number followed by a unit identifier like s, m or h. Defaults to '10s' (10 seconds). |
|
int |
0 |
The maximum quantity of items in the user info cache. Only applies when store type 'ocmem' is configured. Defaults to 512. |
|
string |
ocis |
The driver which should be used by the service. |
|
string |
messagepack |
The backend to use for storing metadata. Supported values are 'messagepack' and 'xattrs'. The setting 'messagepack' uses a dedicated file to store file metadata while 'xattrs' uses extended attributes to store file metadata. Defaults to 'messagepack'. |
|
string |
~/.ocis/storage/metadata |
Path for the directory where the STORAGE-SYSTEM service stores it’s persistent data. If not defined, the root directory derives from $OCIS_BASE_DATA_PATH:/storage. |
|
int |
20 |
When trying to lock files, ocis will try this amount of times to acquire the lock before failing. After each try it will wait for an increasing amount of time. Values of 0 or below will be ignored and the default value of 20 will be used. |
|
int |
30 |
When trying to lock files, ocis will multiply the cycle with this factor and use it as a millisecond timeout. Values of 0 or below will be ignored and the default value of 30 will be used. |
|
string |
http://localhost:9216/data |
URL of the data server, needs to be reachable by other services using this service. |
| Deprecation Info | Deprecation Version | Removal Version | Deprecation Replacement |
|---|---|---|---|
REVA_GATEWAY changing name for consistency |
3.0 |
4.0.0 |
OCIS_REVA_GATEWAY |
| Name | Type | Default Value | Description |
|---|---|---|---|
|
bool |
false |
Activates tracing. |
|
string |
|
The type of tracing. Defaults to "", which is the same as "jaeger". Allowed tracing types are "jaeger" and "" as of now. |
|
string |
|
The endpoint of the tracing agent. |
|
string |
|
The HTTP endpoint for sending spans directly to a collector, i.e. http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset. |
|
string |
|
The log level. Valid values are: "panic", "fatal", "error", "warn", "info", "debug", "trace". |
|
bool |
false |
Activates pretty log output. |
|
bool |
false |
Activates colorized log output. |
|
string |
|
The path to the log file. Activates logging to this file if set. |
|
string |
127.0.0.1:9217 |
Bind address of the debug server, where metrics, health, config and debug endpoints will be exposed. |
|
string |
|
Token to secure the metrics endpoint |
|
bool |
false |
Enables pprof, which can be used for profiling |
|
bool |
false |
Enables zpages, which can be used for collecting and viewing in-memory traces. |
|
string |
127.0.0.1:9215 |
The bind address of the GRPC service. |
|
string |
tcp |
The transport protocol of the GPRC service. |
|
string |
127.0.0.1:9216 |
The bind address of the HTTP service. |
|
string |
tcp |
The transport protocol of the HTTP service. |
|
string |
|
The secret to mint and validate jwt tokens. |
|
string |
127.0.0.1:9142 |
The CS3 gateway endpoint. |
|
string |
|
TLS mode for grpc connection to the go-micro based grpc services. Possible values are 'off', 'insecure' and 'on'. 'off': disables transport security for the clients. 'insecure' allows using transport security, but disables certificate verification (to be used with the autogenerated self-signed certificates). 'on' enables transport security, including server certificate verification. |
|
string |
|
Path/File name for the root CA certificate (in PEM format) used to validate TLS server certificates of the go-micro based grpc services. |
|
string |
|
ID of the oCIS storage-system system user. Admins need to set the ID for the STORAGE-SYSTEM system user in this config option which is then used to reference the user. Any reasonable long string is possible, preferably this would be an UUIDv4 format. |
|
string |
|
API key for the STORAGE-SYSTEM system user. |
|
bool |
false |
Disables the loading of user’s group memberships from the reva access token. |
|
string |
memory |
The type of the cache store. Supported values are: 'memory', 'ocmem', 'etcd', 'redis', 'redis-sentinel', 'nats-js', 'noop'. See the text description for details. |
|
[]string |
[] |
A comma separated list of nodes to access the configured store. This has no effect when 'memory' or 'ocmem' stores are configured. Note that the behaviour how nodes are used is dependent on the library of the configured store. |
|
string |
ocis |
The database name the configured store should use. |
|
Duration |
24m0s |
Default time to live for user info in the user info cache. Only applied when access tokens has no expiration. The duration can be set as number followed by a unit identifier like s, m or h. Defaults to '10s' (10 seconds). |
|
int |
0 |
The maximum quantity of items in the user info cache. Only applies when store type 'ocmem' is configured. Defaults to 512. |
|
string |
ocis |
The driver which should be used by the service. |
|
string |
messagepack |
The backend to use for storing metadata. Supported values are 'messagepack' and 'xattrs'. The setting 'messagepack' uses a dedicated file to store file metadata while 'xattrs' uses extended attributes to store file metadata. Defaults to 'messagepack'. |
|
string |
~/.ocis/storage/metadata |
Path for the directory where the STORAGE-SYSTEM service stores it’s persistent data. If not defined, the root directory derives from $OCIS_BASE_DATA_PATH:/storage. |
|
int |
20 |
When trying to lock files, ocis will try this amount of times to acquire the lock before failing. After each try it will wait for an increasing amount of time. Values of 0 or below will be ignored and the default value of 20 will be used. |
|
int |
30 |
When trying to lock files, ocis will multiply the cycle with this factor and use it as a millisecond timeout. Values of 0 or below will be ignored and the default value of 30 will be used. |
|
string |
http://localhost:9216/data |
URL of the data server, needs to be reachable by other services using this service. |
YAML Example
Note that the filename shown below has been chosen on purpose.
See the Configuration File Naming for details when setting up your own configuration.
# Autogenerated
# Filename: storage-system-config-example.yaml
tracing:
enabled: false
type: ""
endpoint: ""
collector: ""
log:
level: ""
pretty: false
color: false
file: ""
debug:
addr: 127.0.0.1:9217
token: ""
pprof: false
zpages: false
grpc:
addr: 127.0.0.1:9215
tls: null
protocol: tcp
http:
addr: 127.0.0.1:9216
protocol: tcp
token_manager:
jwt_secret: ""
reva:
address: com.owncloud.api.gateway
tls:
mode: ""
cacert: ""
system_user_id: ""
system_user_api_key: ""
skip_user_groups_in_token: false
cache:
store: memory
nodes: []
database: ocis
ttl: 24m0s
size: 0
driver: ocis
drivers:
ocis:
metadata_backend: messagepack
root: ~/.ocis/storage/metadata
max_acquire_lock_cycles: 20
lock_cycle_duration_factor: 30
data_server_url: http://localhost:9216/data# Autogenerated
# Filename: storage-system-config-example.yaml
tracing:
enabled: false
type: ""
endpoint: ""
collector: ""
log:
level: ""
pretty: false
color: false
file: ""
debug:
addr: 127.0.0.1:9217
token: ""
pprof: false
zpages: false
grpc:
addr: 127.0.0.1:9215
tls: null
protocol: tcp
http:
addr: 127.0.0.1:9216
protocol: tcp
token_manager:
jwt_secret: ""
reva:
address: com.owncloud.api.gateway
tls:
mode: ""
cacert: ""
system_user_id: ""
system_user_api_key: ""
skip_user_groups_in_token: false
cache:
store: memory
nodes: []
database: ocis
ttl: 24m0s
size: 0
driver: ocis
drivers:
ocis:
metadata_backend: messagepack
root: ~/.ocis/storage/metadata
max_acquire_lock_cycles: 20
lock_cycle_duration_factor: 30
data_server_url: http://localhost:9216/data# Autogenerated
# Filename: storage-system-config-example.yaml
tracing:
enabled: false
type: ""
endpoint: ""
collector: ""
log:
level: ""
pretty: false
color: false
file: ""
debug:
addr: 127.0.0.1:9217
token: ""
pprof: false
zpages: false
grpc:
addr: 127.0.0.1:9215
tls: null
protocol: tcp
http:
addr: 127.0.0.1:9216
protocol: tcp
token_manager:
jwt_secret: ""
reva:
address: 127.0.0.1:9142
tls:
mode: ""
cacert: ""
system_user_id: ""
system_user_api_key: ""
skip_user_groups_in_token: false
cache:
store: memory
nodes: []
database: ocis
ttl: 24m0s
size: 0
driver: ocis
drivers:
ocis:
metadata_backend: messagepack
root: ~/.ocis/storage/metadata
max_acquire_lock_cycles: 20
lock_cycle_duration_factor: 30
data_server_url: http://localhost:9216/data