Storage-Users Service Configuration

Table of Contents

Introduction
Default Values
Graceful Shutdown
CLI Commands
- Manage Unfinished Uploads
- Manage Trash-Bin Items
Resource Optimisation
Caching
Event Bus Configuration
Configuration
- Environment Variables
- YAML Example

Introduction

The Infinite Scale Storage-Users service

Default Values

Storage-Users listens on port 9157 by default.

Graceful Shutdown

With Infinite Scale, you can define a graceful shutdown period for the storage-users service.

The graceful shutdown period is only applicable if the storage-users service runs as standalone service. It does not apply if the storage-users service runs as part of the single binary or as single Docker environment. To build an environment where the storage-users service runs as a standalone service, you must start two instances, one without the storage-users service and one only with the the storage-users service. Note that both instances must be able to communicate on the same network.

When hard-stopping Infinite Scale, for example with the kill <pid> command (SIGKILL), it is possible and likely that not all data from the decomposedfs (metadata) has been written to the storage which may result in an inconsistent decomposedfs. When gracefully shutting down Infinite Scale, using a command like SIGTERM, the process will no longer accept any write requests from other services and will try to write the internal open requests which can take an undefined duration based on many factors. To mitigate that situation, the following things have been implemented:

With the value of the environment variable STORAGE_USERS_GRACEFUL_SHUTDOWN_TIMEOUT, the storage-users service will delay its shutdown giving it time to finalize writing necessary data. This delay can be necessary if there is a lot of data to be saved and/or if storage access/throughput is slow. In such a case you would receive an error log entry informing you that not all data could be saved in time. To prevent such occurrences, you must increase the default value.
If a shutdown error has been logged, the command-line maintenance tool Inspect and Manipulate Node Metadata can help to fix the issue. Please contact support for details.

CLI Commands

For any command listed, use --help to get more details and possible options and arguments.

To authenticate CLI commands use:

OCIS_SERVICE_ACCOUNT_SECRET=<acc-secret> and
OCIS_SERVICE_ACCOUNT_ID=<acc-id>.

The storage-users CLI tool uses the default address to establish the connection to the gateway service. If the connection fails, check your custom gateway service GATEWAY_GRPC_ADDR configuration and set the same address in storage-users OCIS_GATEWAY_GRPC_ADDR or STORAGE_USERS_GATEWAY_GRPC_ADDR.

Manage Unfinished Uploads

When using Infinite Scale as user storage, a directory named storage/users/uploads can be found in the Infinite Scale data folder. This is an intermediate directory based on TUS which is an open protocol for resumable uploads. Each upload consists of a blob and a blob.info file. Note that the term blob is just a placeholder.

If an upload succeeds, the blob file will be moved to the target and the blob.info file will be deleted.
In case of incomplete uploads, the blob and blob.info files will continue to recieve data until either the upload succeeds in time or the upload expires based on the STORAGE_USERS_UPLOAD_EXPIRATION variable, see the table below for details.
In case of expired uploads, the blob and blob.info files will not be removed automatically. Thus a lot of data can pile up over time wasting storage space.
In the rare case of a failure, after the upload succeeded but the file was not moved to its target location, which can happen when postprocessing fails, the situation is the same as with expired uploads.

Example cases for expired uploads

When a user uploads a big file but the file exceeds the user-quota, the upload can’t be moved to the target after it has finished. The file stays at the upload location until it is manually cleared.
If the bandwith is limited and the file to transfer can’t be transferred completely before the upload expiration time is reached, the file expires and can’t be processed.
If the upload was technically successful, but the postprocessing step failed due to an internal error, it will not get further processed. See the procedure in the Resume Post-Processing documentation for details how to solve this.

The following commands are available to manage unfinished uploads

ocis storage-users uploads <command>

COMMANDS:
   sessions  Print a list of upload sessions

Sessions command

The sessions command is the entry point for listing, restarting/resuming and cleaning unfinished uploads.

There can never be a clear identification of a failed upload session due to various reasons causing them. You need to apply more critera like free space on disk, a failed service like antivirus etc. to declare an upload as failed.

If not noted otherwise, the restart option can also be replaced with the resume option. This changes behaviour slightly.

restart
When restarting an upload, all steps for open items will be restarted, except if otherwise defined.
resume
When resuming an upload, processing will continue unfinished items from their last completed step.

ocis storage-users uploads sessions

NAME:
   ocis storage-users uploads sessions - Print a list of upload sessions

USAGE:
   ocis storage-users uploads sessions [command options]

OPTIONS:
   --id value    filter sessions by upload session id (default: unset)
   --processing  filter sessions by processing status (default: unset)
   --expired     filter sessions by expired status    (default: unset)
   --has-virus   filter sessions by virus scan result (default: unset)
   --json        output as json                       (default: false)
   --restart     send restart event for all listed sessions (default: false)
   --resume      send resume event for all listed sessions (default: false)
   --clean       remove uploads                       (default: false)
   --help, -h    show help

This will always output a list of uploads that match the criteria. See the examples below.

Some additional information on returned information

Offset is the number of bytes the server has already received.
If Offset == Size the server has received all bytes of the upload.
Processing indicates if the uploaded file is currently going through postprocessing.
Scan Date and Scan Result indicate the scanning status.
If Scan Date is set and Scan Result is empty the file is not virus infected.

Command Examples

Command to list ongoing upload sessions

ocis storage-users uploads sessions --expired=false --processing=false

Sessions that are not expired:
Space	Upload Id	Name	Offset	Size	Executant	Owner	Expires	Processing	Scan Date	Scan Result
f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c	5e387954-7313-4223-a904-bf996da6ec0b	foo.txt	0	1234	f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c	f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c	2024-01-26T13:04:31+01:00	false	2024-04-24T11:24:14+02:00	infected: virus A
f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c	f066244d-97b2-48e7-a30d-b40fcb60cec6	bar.txt	0	4321	f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c	f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c	2024-01-26T13:18:47+01:00	false	2024-04-24T14:38:29+02:00

The sessions command can also clear and restart/resume uploads. The output is the same as if run without the --clean or --restart/--resume option. Note that it is recommended to run the command first without the --clean (--processing) option to double check which uploads would get cleaned (restarted/resumed).

Cleans all expired uploads regardless of processing and virus state.

ocis storage-users uploads sessions \
     --expired=true \
     --clean

Resume all uploads that are currently processing and are not virus infected

ocis storage-users uploads sessions \
     --processing=false \
     --has-virus=false \
     --resume

Manage Trash-Bin Items

This command set provides commands to get an overview of trash-bin items, restore items and purge old items of personal spaces and project spaces (spaces that have been created manually). trash-bin commands require a spaceID as parameter. See Listing Space IDs for details of how to get them.

ocis storage-users trash-bin <command>

COMMANDS:
   purge-expired  Purge expired trash-bin items
   list           Print a list of all trash-bin items of a space.
   restore-all    Restore all trash-bin items for a space.
   restore        Restore a trash-bin item by ID.

Purge Expired

Purge all expired items from the trash-bin

ocis storage-users trash-bin purge-expired

The behaviour of the purge-expired command can be configured by using the following environment variables.

STORAGE_USERS_PURGE_TRASH_BIN_USER_ID
Used to obtain space trash-bin information and takes the system admin user as the default which is the OCIS_ADMIN_USER_ID but can be set individually. It should be noted, that the OCIS_ADMIN_USER_ID is only assigned automatically when using the single binary deployment and must be manually assigned in all other deployments. The command only considers spaces to which the assigned user has access and delete permission.
STORAGE_USERS_PURGE_TRASH_BIN_PERSONAL_DELETE_BEFORE
Has a default value of 720h which equals 30 days. This means, the command will delete all files older than 30 days. The value is human-readable, for valid values see the duration type described in the Environment Variable Types. A value of 0 is equivalent to disable and prevents the deletion of personal space trash-bin files.
STORAGE_USERS_PURGE_TRASH_BIN_PROJECT_DELETE_BEFORE
Has a default value of 720h which equals 30 days. This means, the command will delete all files older than 30 days. The value is human-readable, for valid values see the duration type described in the Environment Variable Types. A value of 0 is equivalent to disable and prevents the deletion of project space trash-bin files.

List and Restore Trash-Bins Items

Restoring is possible only to the original location. The personal or project spaceID is required for the items to be restored. To authenticate the CLI tool use:

OCIS_SERVICE_ACCOUNT_SECRET=<acc-secret>
OCIS_SERVICE_ACCOUNT_ID=<acc-id>

The storage-users CLI tool uses the default address to establish the connection to the gateway service. If the connection fails, check the GATEWAY_GRPC_ADDR configuration from your gateway service and set the same address to the storage-users variable STORAGE_USERS_GATEWAY_GRPC_ADDR or globally with OCIS_GATEWAY_GRPC_ADDR.

Export the gateway address if your configuration differs from the default

export STORAGE_USERS_GATEWAY_GRPC_ADDR=127.0.0.1:9142

Print a list of all trash-bin items of a space

ocis storage-users trash-bin list [command options] ['spaceID' required]

The restore option defines the behavior for an item to be restored, when the item name already exists in the target space. Supported options are: skip, replace and keep-both. The default value is skip.

When the CLI tool restores the item with the replace option, the existing item will be moved to a trash-bin. When the cli tool restores the item with the keep-both option and the designated item already exists, the name of the restored item will be changed by adding a numeric suffix in parentheses. The variable STORAGE_USERS_CLI_MAX_ATTEMPTS_RENAME_FILE defines a maximum number of attempts to rename an item.

Restore all trash-bin items for a space

ocis storage-users trash-bin restore-all [command options] ['spaceID' required]

Restore a trash-bin item by ID

ocis storage-users trash-bin restore [command options] ['spaceID' required] ['itemID' required]

Resource Optimisation

See the General Storage Considerations and Settings documentation for details about configuring the STORAGE_USERS_OCIS_MAX_CONCURRENCY environment variable.

Caching

The storage-users service caches stat, metadata and uuids of files and folders via the configured stores.

The storage-users service can use a configured store via the global OCIS_CACHE_STORE environment variable.

Note that for each global environment variable, an independent service-based one might be available additionally. For precedences see Environment Variable Notes. Check the configuration section below. Supported stores are:

Store Type Description

Store Type	Description
`memory`	Basic in-memory store. Will not survive a restart. Usually the default for caches. See the store environment variable for which one is used.
`nats-js-kv`	Stores data using key-value-store feature of NATS JetStream. Usually the default for stores, see the store environment variable for which one is used.
`redis-sentinel`	Stores data in a configured Redis Sentinel cluster.
`noop`	Stores nothing. Useful for testing. Not recommended in production environments.

memory

Basic in-memory store. Will not survive a restart.
Usually the default for caches. See the store environment variable for which one is used.

nats-js-kv

Stores data using key-value-store feature of NATS JetStream.
Usually the default for stores, see the store environment variable for which one is used.

redis-sentinel

Stores data in a configured Redis Sentinel cluster.

noop

Stores nothing. Useful for testing. Not recommended in production environments.

The storage-users service can only be scaled if not using the memory store and the stores are configured identically over all instances!

If you have used one of the deprecated stores of a former version, you should reconfigure to use one of the supported ones as the deprecated stores will be removed in a later version.

Store specific notes

When using redis-sentinel:
The Redis master to use is configured via e.g. OCIS_CACHE_STORE_NODES in the form of <sentinel-host>:<sentinel-port>/<redis-master> like 10.10.0.200:26379/mymaster.
When using nats-js-kv:
- It is recommended to set OCIS_CACHE_STORE_NODES to the same value as OCIS_EVENTS_ENDPOINT. That way the cache uses the same nats instance as the event bus. See the Event Bus Configuration for more details.
- Authentication can be added, if configured, via OCIS_CACHE_AUTH_USERNAME and OCIS_CACHE_AUTH_PASSWORD.
- It is possible to set OCIS_CACHE_DISABLE_PERSISTENCE to instruct nats to not persist cache data on disc.

Event Bus Configuration

The Infinite Scale event bus can be configured by a set of environment variables.

If you are using a binary installation as described in Minimal Bare Metal Deployment or Bare Metal with systemd, the address of the event bus OCIS_EVENTS_ENDPOINT is predefined as localhost address without the need for further configuration, but changeable on demand.
In case of an orchestrated installation like with Docker or Kubernetes, the event bus must be an external service for scalability like a Redis Sentinel cluster or a key-value-store NATS JetStream. Both named stores are supported and also used in Caching and Persistence. The store used is not part of the Infinite Scale installation and must be separately provided and configured.
Note that from a configuration point of view, caching and persistence are independent of the event bus configuration.

Note that for each global environment variable, a service-based one might be available additionally. For precedences see Environment Variable Notes. Check the configuration section below.

Without the aim of completeness, see the list of environment variables to configure the event bus:

Envvar Description

Envvar	Description
`OCIS_EVENTS_ENDPOINT`	The address of the event system.
`OCIS_EVENTS_CLUSTER`	The clusterID of the event system. Mandatory when using NATS as event system.
`OCIS_EVENTS_ENABLE_TLS`	Enable TLS for the connection to the events broker.
`OCIS_INSECURE`	Whether to verify the server TLS certificates.
`OCIS_EVENTS_AUTH_USERNAME`	The username to authenticate with the events broker.
`OCIS_EVENTS_AUTH_PASSWORD`	The password to authenticate with the events broker.

OCIS_EVENTS_ENDPOINT

The address of the event system.

OCIS_EVENTS_CLUSTER

The clusterID of the event system. Mandatory when using NATS as event system.

OCIS_EVENTS_ENABLE_TLS

Enable TLS for the connection to the events broker.

OCIS_INSECURE

Whether to verify the server TLS certificates.

OCIS_EVENTS_AUTH_USERNAME

The username to authenticate with the events broker.

OCIS_EVENTS_AUTH_PASSWORD

The password to authenticate with the events broker.

Configuration

Environment Variables

The storage-users service is configured via the following environment variables. Read the Environment Variable Types documentation for important details. Column IV shows with which release the environment variable has been introduced.

7.0.0

Environment variables for the storage-users service
Name	IV	Type	Default Value	Description
`STORAGE_USERS_SERVICE_NAME`	7.0.0	string	storage-users	Service name to use. Change this when starting an additional storage provider with a custom configuration to prevent it from colliding with the default 'storage-users' service.
`OCIS_TRACING_ENABLED` `STORAGE_USERS_TRACING_ENABLED`	pre5.0	bool	false	Activates tracing.
`OCIS_TRACING_TYPE` `STORAGE_USERS_TRACING_TYPE`	pre5.0	string		The type of tracing. Defaults to '', which is the same as 'jaeger'. Allowed tracing types are 'jaeger' and '' as of now.
`OCIS_TRACING_ENDPOINT` `STORAGE_USERS_TRACING_ENDPOINT`	pre5.0	string		The endpoint of the tracing agent.
`OCIS_TRACING_COLLECTOR` `STORAGE_USERS_TRACING_COLLECTOR`	pre5.0	string		The HTTP endpoint for sending spans directly to a collector, i.e. http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset.
`OCIS_LOG_LEVEL` `STORAGE_USERS_LOG_LEVEL`	pre5.0	string		The log level. Valid values are: 'panic', 'fatal', 'error', 'warn', 'info', 'debug', 'trace'.
`OCIS_LOG_PRETTY` `STORAGE_USERS_LOG_PRETTY`	pre5.0	bool	false	Activates pretty log output.
`OCIS_LOG_COLOR` `STORAGE_USERS_LOG_COLOR`	pre5.0	bool	false	Activates colorized log output.
`OCIS_LOG_FILE` `STORAGE_USERS_LOG_FILE`	pre5.0	string		The path to the log file. Activates logging to this file if set.
`STORAGE_USERS_DEBUG_ADDR`	pre5.0	string	127.0.0.1:9159	Bind address of the debug server, where metrics, health, config and debug endpoints will be exposed.
`STORAGE_USERS_DEBUG_TOKEN`	pre5.0	string		Token to secure the metrics endpoint.
`STORAGE_USERS_DEBUG_PPROF`	pre5.0	bool	false	Enables pprof, which can be used for profiling.
`STORAGE_USERS_DEBUG_ZPAGES`	pre5.0	bool	false	Enables zpages, which can be used for collecting and viewing in-memory traces.
`STORAGE_USERS_GRPC_ADDR`	pre5.0	string	127.0.0.1:9157	The bind address of the GRPC service.
`OCIS_GRPC_PROTOCOL` `STORAGE_USERS_GRPC_PROTOCOL`	pre5.0	string	tcp	The transport protocol of the GPRC service.
`STORAGE_USERS_HTTP_ADDR`	pre5.0	string	127.0.0.1:9158	The bind address of the HTTP service.
`STORAGE_USERS_HTTP_PROTOCOL`	pre5.0	string	tcp	The transport protocol of the HTTP service.
`OCIS_CORS_ALLOW_ORIGINS` `STORAGE_USERS_CORS_ALLOW_ORIGINS`	pre5.0	[]string	[https://localhost:9200]	A list of allowed CORS origins. See following chapter for more details: Access-Control-Allow-Origin at https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Origin. See the Environment Variable Types description for more details.
`OCIS_CORS_ALLOW_METHODS` `STORAGE_USERS_CORS_ALLOW_METHODS`	pre5.0	[]string	[POST HEAD PATCH OPTIONS GET DELETE]	A list of allowed CORS methods. See following chapter for more details: Access-Control-Request-Method at https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Request-Method. See the Environment Variable Types description for more details.
`OCIS_CORS_ALLOW_HEADERS` `STORAGE_USERS_CORS_ALLOW_HEADERS`	pre5.0	[]string	[Authorization Origin X-Requested-With X-Request-Id X-HTTP-Method-Override Content-Type Upload-Length Upload-Offset Tus-Resumable Upload-Metadata Upload-Defer-Length Upload-Concat Upload-Incomplete Upload-Draft-Interop-Version]	A list of allowed CORS headers. See following chapter for more details: Access-Control-Request-Headers at https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Request-Headers. See the Environment Variable Types description for more details.
`OCIS_CORS_ALLOW_CREDENTIALS` `STORAGE_USERS_CORS_ALLOW_CREDENTIALS`	pre5.0	bool	false	Allow credentials for CORS.See following chapter for more details: Access-Control-Allow-Credentials at https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Credentials.
`OCIS_CORS_EXPOSE_HEADERS` `STORAGE_USERS_CORS_EXPOSE_HEADERS`	pre5.0	[]string	[Upload-Offset Location Upload-Length Tus-Version Tus-Resumable Tus-Max-Size Tus-Extension Upload-Metadata Upload-Defer-Length Upload-Concat Upload-Incomplete Upload-Draft-Interop-Version]	A list of exposed CORS headers. See following chapter for more details: Access-Control-Expose-Headers at https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Expose-Headers. See the Environment Variable Types description for more details.
`OCIS_CORS_MAX_AGE` `STORAGE_USERS_CORS_MAX_AGE`	pre5.0	uint	86400	The max cache duration of preflight headers. See following chapter for more details: Access-Control-Max-Age at https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Max-Age. See the Environment Variable Types description for more details.
`OCIS_JWT_SECRET` `STORAGE_USERS_JWT_SECRET`	pre5.0	string		The secret to mint and validate jwt tokens.
`OCIS_REVA_GATEWAY`	pre5.0	string	com.owncloud.api.gateway	The CS3 gateway endpoint.
`OCIS_GRPC_CLIENT_TLS_MODE`	pre5.0	string		TLS mode for grpc connection to the go-micro based grpc services. Possible values are 'off', 'insecure' and 'on'. 'off': disables transport security for the clients. 'insecure' allows using transport security, but disables certificate verification (to be used with the autogenerated self-signed certificates). 'on' enables transport security, including server certificate verification.
`OCIS_GRPC_CLIENT_TLS_CACERT`	pre5.0	string		Path/File name for the root CA certificate (in PEM format) used to validate TLS server certificates of the go-micro based grpc services.
`STORAGE_USERS_SKIP_USER_GROUPS_IN_TOKEN`	pre5.0	bool	false	Disables the loading of user’s group memberships from the reva access token.
`STORAGE_USERS_GRACEFUL_SHUTDOWN_TIMEOUT`	pre5.0	int	30	The number of seconds to wait for the 'storage-users' service to shutdown cleanly before exiting with an error that gets logged. Note: This setting is only applicable when running the 'storage-users' service as a standalone service. See the text description for more details.
`STORAGE_USERS_DRIVER`	pre5.0	string	ocis	The storage driver which should be used by the service. Defaults to 'ocis', Supported values are: 'ocis', 's3ng' and 'owncloudsql'. The 'ocis' driver stores all data (blob and meta data) in an POSIX compliant volume. The 's3ng' driver stores metadata in a POSIX compliant volume and uploads blobs to the s3 bucket.
`OCIS_DECOMPOSEDFS_PROPAGATOR` `STORAGE_USERS_OCIS_PROPAGATOR`	pre5.0	string	sync	The propagator used for decomposedfs. At the moment, only 'sync' is fully supported, 'async' is available as an experimental option.
`STORAGE_USERS_ASYNC_PROPAGATOR_PROPAGATION_DELAY`	pre5.0	Duration	0s	The delay between a change made to a tree and the propagation start on treesize and treetime. Multiple propagations are computed to a single one. See the Environment Variable Types description for more details.
`STORAGE_USERS_OCIS_ROOT`	pre5.0	string	/var/lib/ocis/storage/users	The directory where the filesystem storage will store blobs and metadata. If not defined, the root directory derives from $OCIS_BASE_DATA_PATH/storage/users.
`STORAGE_USERS_OCIS_USER_LAYOUT`	pre5.0	string	{{.Id.OpaqueId}}	Template string for the user storage layout in the user directory.
`STORAGE_USERS_PERMISSION_ENDPOINT` `STORAGE_USERS_OCIS_PERMISSIONS_ENDPOINT`	pre5.0	string	com.owncloud.api.settings	Endpoint of the permissions service. The endpoints can differ for 'ocis' and 's3ng'.
`STORAGE_USERS_OCIS_PERSONAL_SPACE_ALIAS_TEMPLATE`	pre5.0	string	{{.SpaceType}}/{{.User.Username \| lower}}	Template string to construct personal space aliases.
`STORAGE_USERS_OCIS_PERSONAL_SPACE_PATH_TEMPLATE`	6.0.0	string		Template string to construct the paths of the personal space roots.
`STORAGE_USERS_OCIS_GENERAL_SPACE_ALIAS_TEMPLATE`	pre5.0	string	{{.SpaceType}}/{{.SpaceName \| replace " " "-" \| lower}}	Template string to construct general space aliases.
`STORAGE_USERS_OCIS_GENERAL_SPACE_PATH_TEMPLATE`	6.0.0	string		Template string to construct the paths of the projects space roots.
`STORAGE_USERS_OCIS_SHARE_FOLDER`	pre5.0	string	/Shares	Name of the folder jailing all shares.
`STORAGE_USERS_OCIS_MAX_ACQUIRE_LOCK_CYCLES`	pre5.0	int	20	When trying to lock files, ocis will try this amount of times to acquire the lock before failing. After each try it will wait for an increasing amount of time. Values of 0 or below will be ignored and the default value will be used.
`STORAGE_USERS_OCIS_LOCK_CYCLE_DURATION_FACTOR`	pre5.0	int	30	When trying to lock files, ocis will multiply the cycle with this factor and use it as a millisecond timeout. Values of 0 or below will be ignored and the default value will be used.
`OCIS_MAX_CONCURRENCY` `STORAGE_USERS_OCIS_MAX_CONCURRENCY`	pre5.0	int	5	Maximum number of concurrent go-routines. Higher values can potentially get work done faster but will also cause more load on the system. Values of 0 or below will be ignored and the default value will be used.
`OCIS_ASYNC_UPLOADS`	pre5.0	bool	true	Enable asynchronous file uploads.
`OCIS_SPACES_MAX_QUOTA` `STORAGE_USERS_OCIS_MAX_QUOTA`	pre5.0	uint64	0	Set a global max quota for spaces in bytes. A value of 0 equals unlimited. If not using the global OCIS_SPACES_MAX_QUOTA, you must define the FRONTEND_MAX_QUOTA in the frontend service.
`OCIS_DISABLE_VERSIONING`	7.0.0	bool	false	Disables versioning of files. When set to true, new uploads with the same filename will overwrite existing files instead of creating a new version.
`OCIS_DECOMPOSEDFS_PROPAGATOR` `STORAGE_USERS_S3NG_PROPAGATOR`	pre5.0	string	sync	The propagator used for decomposedfs. At the moment, only 'sync' is fully supported, 'async' is available as an experimental option.
`STORAGE_USERS_ASYNC_PROPAGATOR_PROPAGATION_DELAY`	pre5.0	Duration	0s	The delay between a change made to a tree and the propagation start on treesize and treetime. Multiple propagations are computed to a single one. See the Environment Variable Types description for more details.
`STORAGE_USERS_S3NG_ROOT`	pre5.0	string	/var/lib/ocis/storage/users	The directory where the filesystem storage will store metadata for blobs. If not defined, the root directory derives from $OCIS_BASE_DATA_PATH/storage/users.
`STORAGE_USERS_S3NG_USER_LAYOUT`	pre5.0	string	{{.Id.OpaqueId}}	Template string for the user storage layout in the user directory.
`STORAGE_USERS_PERMISSION_ENDPOINT` `STORAGE_USERS_S3NG_PERMISSIONS_ENDPOINT`	pre5.0	string	com.owncloud.api.settings	Endpoint of the permissions service. The endpoints can differ for 'ocis' and 's3ng'.
`STORAGE_USERS_S3NG_REGION`	pre5.0	string	default	Region of the S3 bucket.
`STORAGE_USERS_S3NG_ACCESS_KEY`	pre5.0	string		Access key for the S3 bucket.
`STORAGE_USERS_S3NG_SECRET_KEY`	pre5.0	string		Secret key for the S3 bucket.
`STORAGE_USERS_S3NG_ENDPOINT`	pre5.0	string		Endpoint for the S3 bucket.
`STORAGE_USERS_S3NG_BUCKET`	pre5.0	string		Name of the S3 bucket.
`STORAGE_USERS_S3NG_PUT_OBJECT_DISABLE_CONTENT_SHA256`	5.0	bool	false	Disable sending content sha256 when copying objects to S3.
`STORAGE_USERS_S3NG_PUT_OBJECT_DISABLE_MULTIPART`	5.0	bool	true	Disable multipart uploads when copying objects to S3
`STORAGE_USERS_S3NG_PUT_OBJECT_SEND_CONTENT_MD5`	5.0	bool	true	Send a Content-MD5 header when copying objects to S3.
`STORAGE_USERS_S3NG_PUT_OBJECT_CONCURRENT_STREAM_PARTS`	5.0	bool	true	Always precreate parts when copying objects to S3.
`STORAGE_USERS_S3NG_PUT_OBJECT_NUM_THREADS`	5.0	uint	4	Number of concurrent uploads to use when copying objects to S3.
`STORAGE_USERS_S3NG_PUT_OBJECT_PART_SIZE`	5.0	uint64	0	Part size for concurrent uploads to S3.
`STORAGE_USERS_S3NG_PERSONAL_SPACE_ALIAS_TEMPLATE`	pre5.0	string	{{.SpaceType}}/{{.User.Username \| lower}}	Template string to construct personal space aliases.
`STORAGE_USERS_S3NG_PERSONAL_SPACE_PATH_TEMPLATE`	6.0.0	string		Template string to construct the paths of the personal space roots.
`STORAGE_USERS_S3NG_GENERAL_SPACE_ALIAS_TEMPLATE`	pre5.0	string	{{.SpaceType}}/{{.SpaceName \| replace " " "-" \| lower}}	Template string to construct general space aliases.
`STORAGE_USERS_S3NG_GENERAL_SPACE_PATH_TEMPLATE`	6.0.0	string		Template string to construct the paths of the projects space roots.
`STORAGE_USERS_S3NG_SHARE_FOLDER`	pre5.0	string	/Shares	Name of the folder jailing all shares.
`STORAGE_USERS_S3NG_MAX_ACQUIRE_LOCK_CYCLES`	pre5.0	int	20	When trying to lock files, ocis will try this amount of times to acquire the lock before failing. After each try it will wait for an increasing amount of time. Values of 0 or below will be ignored and the default value of 20 will be used.
`STORAGE_USERS_S3NG_LOCK_CYCLE_DURATION_FACTOR`	pre5.0	int	30	When trying to lock files, ocis will multiply the cycle with this factor and use it as a millisecond timeout. Values of 0 or below will be ignored and the default value of 30 will be used.
`OCIS_MAX_CONCURRENCY` `STORAGE_USERS_S3NG_MAX_CONCURRENCY`	pre5.0	int	5	Maximum number of concurrent go-routines. Higher values can potentially get work done faster but will also cause more load on the system. Values of 0 or below will be ignored and the default value of 100 will be used.
`OCIS_DISABLE_VERSIONING`	7.0.0	bool	false	Disables versioning of files. When set to true, new uploads with the same filename will overwrite existing files instead of creating a new version.
`STORAGE_USERS_OWNCLOUDSQL_DATADIR`	pre5.0	string	/var/lib/ocis/storage/owncloud	The directory where the filesystem storage will store SQL migration data. If not defined, the root directory derives from $OCIS_BASE_DATA_PATH/storage/owncloud.
`STORAGE_USERS_OWNCLOUDSQL_SHARE_FOLDER`	pre5.0	string	/Shares	Name of the folder jailing all shares.
`STORAGE_USERS_OWNCLOUDSQL_LAYOUT`	pre5.0	string	{{.Username}}	Path layout to use to navigate into a users folder in an owncloud data directory
`STORAGE_USERS_OWNCLOUDSQL_UPLOADINFO_DIR`	pre5.0	string	/var/lib/ocis/storage/uploadinfo	The directory where the filesystem will store uploads temporarily. If not defined, the root directory derives from $OCIS_BASE_DATA_PATH/storage/uploadinfo.
`STORAGE_USERS_OWNCLOUDSQL_DB_USERNAME`	pre5.0	string	owncloud	Username for the database.
`STORAGE_USERS_OWNCLOUDSQL_DB_PASSWORD`	pre5.0	string	owncloud	Password for the database.
`STORAGE_USERS_OWNCLOUDSQL_DB_HOST`	pre5.0	string		Hostname or IP of the database server.
`STORAGE_USERS_OWNCLOUDSQL_DB_PORT`	pre5.0	int	3306	Port that the database server is listening on.
`STORAGE_USERS_OWNCLOUDSQL_DB_NAME`	pre5.0	string	owncloud	Name of the database to be used.
`STORAGE_USERS_OWNCLOUDSQL_USERS_PROVIDER_ENDPOINT`	pre5.0	string	com.owncloud.api.users	Endpoint of the users provider.
`STORAGE_USERS_POSIX_ROOT`	6.0.0	string	/var/lib/ocis/storage/users	The directory where the filesystem storage will store its data. If not defined, the root directory derives from $OCIS_BASE_DATA_PATH/storage/users.
`STORAGE_USERS_POSIX_PERSONAL_SPACE_PATH_TEMPLATE`	6.0.0	string	users/{{.User.Username}}	Template string to construct the paths of the personal space roots.
`STORAGE_USERS_POSIX_GENERAL_SPACE_PATH_TEMPLATE`	6.0.0	string	projects/{{.SpaceId}}	Template string to construct the paths of the projects space roots.
`STORAGE_USERS_PERMISSION_ENDPOINT` `STORAGE_USERS_POSIX_PERMISSIONS_ENDPOINT`	6.0.0	string	com.owncloud.api.settings	Endpoint of the permissions service. The endpoints can differ for 'ocis', 'posix' and 's3ng'.
`OCIS_ASYNC_UPLOADS`	pre5.0	bool	true	Enable asynchronous file uploads.
`STORAGE_USERS_POSIX_SCAN_DEBOUNCE_DELAY`	6.0.0	Duration	1s	The time in milliseconds to wait before scanning the filesystem for changes after a change has been detected.
`STORAGE_USERS_POSIX_USE_SPACE_GROUPS`	6.0.0	bool	false	Use space groups to manage permissions on spaces.
`STORAGE_USERS_POSIX_WATCH_TYPE`	6.0.0	string		Type of the watcher to use for getting notified about changes to the filesystem. Currently available options are 'inotifywait' (default), 'gpfswatchfolder' and 'gpfsfileauditlogging'.
`STORAGE_USERS_POSIX_WATCH_PATH`	6.0.0	string		Path to the watch directory/file. Only applies to the 'gpfsfileauditlogging' and 'inotifywait' watcher, in which case it is the path of the file audit log file/base directory to watch.
`STORAGE_USERS_POSIX_WATCH_FOLDER_KAFKA_BROKERS`	6.0.0	string		Comma-separated list of kafka brokers to read the watchfolder events from.
`STORAGE_USERS_DATA_SERVER_URL`	pre5.0	string	http://localhost:9158/data	URL of the data server, needs to be reachable by the data gateway provided by the frontend service or the user if directly exposed.
`STORAGE_USERS_DATA_GATEWAY_URL`	pre5.0	string	https://localhost:9200/data	URL of the data gateway server
`STORAGE_USERS_TRANSFER_EXPIRES`	pre5.0	int64	86400	The time after which the token for upload postprocessing expires
`OCIS_EVENTS_ENDPOINT` `STORAGE_USERS_EVENTS_ENDPOINT`	pre5.0	string	127.0.0.1:9233	The address of the event system. The event system is the message queuing service. It is used as message broker for the microservice architecture.
`OCIS_EVENTS_CLUSTER` `STORAGE_USERS_EVENTS_CLUSTER`	pre5.0	string	ocis-cluster	The clusterID of the event system. The event system is the message queuing service. It is used as message broker for the microservice architecture. Mandatory when using NATS as event system.
`OCIS_INSECURE` `STORAGE_USERS_EVENTS_TLS_INSECURE`	pre5.0	bool	false	Whether to verify the server TLS certificates.
`OCIS_EVENTS_TLS_ROOT_CA_CERTIFICATE` `STORAGE_USERS_EVENTS_TLS_ROOT_CA_CERTIFICATE`	pre5.0	string		The root CA certificate used to validate the server’s TLS certificate. If provided STORAGE_USERS_EVENTS_TLS_INSECURE will be seen as false.
`OCIS_EVENTS_ENABLE_TLS` `STORAGE_USERS_EVENTS_ENABLE_TLS`	pre5.0	bool	false	Enable TLS for the connection to the events broker. The events broker is the ocis service which receives and delivers events between the services.
`STORAGE_USERS_EVENTS_NUM_CONSUMERS`	pre5.0	int	0	The amount of concurrent event consumers to start. Event consumers are used for post-processing files. Multiple consumers increase parallelisation, but will also increase CPU and memory demands. The setting has no effect when the OCIS_ASYNC_UPLOADS is set to false. The default and minimum value is 1.
`OCIS_EVENTS_AUTH_USERNAME` `STORAGE_USERS_EVENTS_AUTH_USERNAME`	5.0	string		The username to authenticate with the events broker. The events broker is the ocis service which receives and delivers events between the services.
`OCIS_EVENTS_AUTH_PASSWORD` `STORAGE_USERS_EVENTS_AUTH_PASSWORD`	5.0	string		The password to authenticate with the events broker. The events broker is the ocis service which receives and delivers events between the services.
`OCIS_CACHE_STORE` `STORAGE_USERS_FILEMETADATA_CACHE_STORE`	pre5.0	string	memory	The type of the cache store. Supported values are: 'memory', 'redis-sentinel', 'nats-js-kv', 'noop'. See the text description for details.
`OCIS_CACHE_STORE_NODES` `STORAGE_USERS_FILEMETADATA_CACHE_STORE_NODES`	pre5.0	[]string	[127.0.0.1:9233]	A list of nodes to access the configured store. This has no effect when 'memory' store is configured. Note that the behaviour how nodes are used is dependent on the library of the configured store. See the Environment Variable Types description for more details.
`OCIS_CACHE_DATABASE`	pre5.0	string	storage-users	The database name the configured store should use.
`OCIS_CACHE_TTL` `STORAGE_USERS_FILEMETADATA_CACHE_TTL`	pre5.0	Duration	24m0s	Default time to live for user info in the user info cache. Only applied when access tokens has no expiration. See the Environment Variable Types description for more details.
`OCIS_CACHE_DISABLE_PERSISTENCE` `STORAGE_USERS_FILEMETADATA_CACHE_DISABLE_PERSISTENCE`	5.0	bool	false	Disables persistence of the cache. Only applies when store type 'nats-js-kv' is configured. Defaults to false.
`OCIS_CACHE_AUTH_USERNAME` `STORAGE_USERS_FILEMETADATA_CACHE_AUTH_USERNAME`	5.0	string		The username to authenticate with the cache store. Only applies when store type 'nats-js-kv' is configured.
`OCIS_CACHE_AUTH_PASSWORD` `STORAGE_USERS_FILEMETADATA_CACHE_AUTH_PASSWORD`	5.0	string		The password to authenticate with the cache store. Only applies when store type 'nats-js-kv' is configured.
`OCIS_CACHE_STORE` `STORAGE_USERS_ID_CACHE_STORE`	pre5.0	string	memory	The type of the cache store. Supported values are: 'memory', 'redis-sentinel', 'nats-js-kv', 'noop'. See the text description for details.
`OCIS_CACHE_STORE_NODES` `STORAGE_USERS_ID_CACHE_STORE_NODES`	pre5.0	[]string	[127.0.0.1:9233]	A list of nodes to access the configured store. This has no effect when 'memory' store is configured. Note that the behaviour how nodes are used is dependent on the library of the configured store. See the Environment Variable Types description for more details.
`OCIS_CACHE_DATABASE`	pre5.0	string	ids-storage-users	The database name the configured store should use.
`OCIS_CACHE_TTL` `STORAGE_USERS_ID_CACHE_TTL`	pre5.0	Duration	24m0s	Default time to live for user info in the user info cache. Only applied when access tokens have no expiration. Defaults to 300s which is derived from the underlaying package though not explicitly set as default. See the Environment Variable Types description for more details.
`OCIS_CACHE_DISABLE_PERSISTENCE` `STORAGE_USERS_ID_CACHE_DISABLE_PERSISTENCE`	5.0	bool	false	Disables persistence of the cache. Only applies when store type 'nats-js-kv' is configured. Defaults to false.
`OCIS_CACHE_AUTH_USERNAME` `STORAGE_USERS_ID_CACHE_AUTH_USERNAME`	5.0	string		The username to authenticate with the cache store. Only applies when store type 'nats-js-kv' is configured.
`OCIS_CACHE_AUTH_PASSWORD` `STORAGE_USERS_ID_CACHE_AUTH_PASSWORD`	5.0	string		The password to authenticate with the cache store. Only applies when store type 'nats-js-kv' is configured.
`STORAGE_USERS_MOUNT_ID`	pre5.0	string		Mount ID of this storage.
`STORAGE_USERS_EXPOSE_DATA_SERVER`	pre5.0	bool	false	Exposes the data server directly to users and bypasses the data gateway. Ensure that the data server address is reachable by users.
`STORAGE_USERS_READ_ONLY`	pre5.0	bool	false	Set this storage to be read-only.
`STORAGE_USERS_UPLOAD_EXPIRATION`	pre5.0	int64	86400	Duration in seconds after which uploads will expire. Note that when setting this to a low number, uploads could be cancelled before they are finished and return a 403 to the user.
`OCIS_ADMIN_USER_ID` `STORAGE_USERS_PURGE_TRASH_BIN_USER_ID`	pre5.0	string		ID of the user who collects all necessary information for deletion. Consider that the UUID can be encoded in some LDAP deployment configurations like in .ldif files. These need to be decoded beforehand.
`STORAGE_USERS_PURGE_TRASH_BIN_PERSONAL_DELETE_BEFORE`	pre5.0	Duration	720h0m0s	Specifies the period of time in which items that have been in the personal trash-bin for longer than this value should be deleted. A value of 0 means no automatic deletion. See the Environment Variable Types description for more details.
`STORAGE_USERS_PURGE_TRASH_BIN_PROJECT_DELETE_BEFORE`	pre5.0	Duration	720h0m0s	Specifies the period of time in which items that have been in the project trash-bin for longer than this value should be deleted. A value of 0 means no automatic deletion. See the Environment Variable Types description for more details.
`OCIS_SERVICE_ACCOUNT_ID` `STORAGE_USERS_SERVICE_ACCOUNT_ID`	5.0	string		The ID of the service account the service should use. See the 'auth-service' service description for more details.
`OCIS_SERVICE_ACCOUNT_SECRET` `STORAGE_USERS_SERVICE_ACCOUNT_SECRET`	5.0	string		The service account secret.
`OCIS_GATEWAY_GRPC_ADDR` `STORAGE_USERS_GATEWAY_GRPC_ADDR`	5.0	string	127.0.0.1:9142	The bind address of the gateway GRPC address.
`OCIS_MACHINE_AUTH_API_KEY` `STORAGE_USERS_MACHINE_AUTH_API_KEY`	5.0	string		Machine auth API key used to validate internal requests necessary for the access to resources from other services.
`STORAGE_USERS_CLI_MAX_ATTEMPTS_RENAME_FILE`	5.0	int	0	The maximum number of attempts to rename a file when a user restores a file to an existing destination with the same name. The minimum value is 100.

YAML Example

Note the file shown below must be renamed and placed in the correct folder according to the Configuration File Naming conventions to be effective.
See the Notes for Environment Variables if you want to use environment variables in the yaml file.

7.0.0

# Autogenerated
# Filename: storage-users-config-example.yaml

tracing:
  enabled: false
  type: ""
  endpoint: ""
  collector: ""
log:
  level: ""
  pretty: false
  color: false
  file: ""
debug:
  addr: 127.0.0.1:9159
  token: ""
  pprof: false
  zpages: false
grpc:
  addr: 127.0.0.1:9157
  tls: null
  protocol: tcp
http:
  addr: 127.0.0.1:9158
  protocol: tcp
  prefix: data
  cors:
    allow_origins:
    - https://localhost:9200
    allow_methods:
    - POST
    - HEAD
    - PATCH
    - OPTIONS
    - GET
    - DELETE
    allow_headers:
    - Authorization
    - Origin
    - X-Requested-With
    - X-Request-Id
    - X-HTTP-Method-Override
    - Content-Type
    - Upload-Length
    - Upload-Offset
    - Tus-Resumable
    - Upload-Metadata
    - Upload-Defer-Length
    - Upload-Concat
    - Upload-Incomplete
    - Upload-Draft-Interop-Version
    allow_credentials: false
    expose_headers:
    - Upload-Offset
    - Location
    - Upload-Length
    - Tus-Version
    - Tus-Resumable
    - Tus-Max-Size
    - Tus-Extension
    - Upload-Metadata
    - Upload-Defer-Length
    - Upload-Concat
    - Upload-Incomplete
    - Upload-Draft-Interop-Version
    max_age: 86400
token_manager:
  jwt_secret: ""
reva:
  address: com.owncloud.api.gateway
  tls:
    mode: ""
    cacert: ""
skip_user_groups_in_token: false
graceful_shutdown_timeout: 30
driver: ocis
drivers:
  ocis:
    propagator: sync
    async_propagator_options:
      propagation_delay: 0s
    root: /var/lib/ocis/storage/users
    user_layout: '{{.Id.OpaqueId}}'
    permissions_endpoint: com.owncloud.api.settings
    personalspacealias_template: '{{.SpaceType}}/{{.User.Username | lower}}'
    personalspacepath_template: ""
    generalspacealias_template: '{{.SpaceType}}/{{.SpaceName | replace " " "-" | lower}}'
    generalspacepath_template: ""
    share_folder: /Shares
    max_acquire_lock_cycles: 20
    lock_cycle_duration_factor: 30
    max_concurrency: 5
    async_uploads: true
    max_quota: 0
    disable_versioning: false
  s3ng:
    propagator: sync
    async_propagator_options:
      propagation_delay: 0s
    root: /var/lib/ocis/storage/users
    user_layout: '{{.Id.OpaqueId}}'
    permissions_endpoint: com.owncloud.api.settings
    region: default
    access_key: ""
    secret_key: ""
    endpoint: ""
    bucket: ""
    put_object_disable_content_sha254: false
    put_object_disable_multipart: true
    put_object_send_content_md5: true
    put_object_concurrent_stream_parts: true
    put_object_num_threads: 4
    put_object_part_size: 0
    personalspacealias_template: '{{.SpaceType}}/{{.User.Username | lower}}'
    personalspacepath_template: ""
    generalspacealias_template: '{{.SpaceType}}/{{.SpaceName | replace " " "-" | lower}}'
    generalspacepath_template: ""
    share_folder: /Shares
    max_acquire_lock_cycles: 20
    lock_cycle_duration_factor: 30
    max_concurrency: 5
    disable_versioning: false
  owncloudsql:
    root: /var/lib/ocis/storage/owncloud
    share_folder: /Shares
    user_layout: '{{.Username}}'
    upload_info_dir: /var/lib/ocis/storage/uploadinfo
    db_username: owncloud
    db_password: owncloud
    db_host: ""
    db_port: 3306
    db_name: owncloud
    users_provider_endpoint: com.owncloud.api.users
  posix:
    root: /var/lib/ocis/storage/users
    personalspacepath_template: users/{{.User.Username}}
    generalspacepath_template: projects/{{.SpaceId}}
    permissions_endpoint: com.owncloud.api.settings
    async_uploads: true
    scan_debounce_delay: 1s
    use_space_groups: false
    watch_type: ""
    watch_path: ""
    watch_folder_kafka_hosts: ""
data_server_url: http://localhost:9158/data
data_gateway_url: https://localhost:9200/data
transfer_expires: 86400
events:
  endpoint: 127.0.0.1:9233
  cluster: ocis-cluster
  tls_insecure: false
  tls_root_ca_cert_path: ""
  enable_tls: false
  num_consumers: 0
  username: ""
  password: ""
filemetadata_cache:
  store: memory
  nodes:
  - 127.0.0.1:9233
  database: storage-users
  ttl: 24m0s
  disable_persistence: false
  username: ""
  password: ""
id_cache:
  store: memory
  nodes:
  - 127.0.0.1:9233
  database: ids-storage-users
  ttl: 24m0s
  disable_persistence: false
  username: ""
  password: ""
mount_id: ""
expose_data_server: false
readonly: false
upload_expiration: 86400
tasks:
  purge_trash_bin:
    user_id: ""
    personal_delete_before: 720h0m0s
    project_delete_before: 720h0m0s
service_account:
  service_account_id: ""
  service_account_secret: ""
gateway_addr: 127.0.0.1:9142
machine_auth_api_key: ""
max_attempts_rename_file: 0