Thumbnails Service Configuration

Table of Contents

Introduction
Default Values
Thumbnailing Performance
Configuration Hints
Configuration
- Environment Variables
- YAML Example

Introduction

The thumbnails service provides methods to generate thumbnails for various files and resolutions based on requests, and is also responsible for presenting images, not only thumbnails to the client. It retrieves the sources at the location where the user files are stored and saves the thumbnails where system files are stored. Those locations have defaults but can be manually defined via environment variables.

Note for developers, more information is available with regards to querying thumbnails in the services section of the Developer Documentation.

Default Values

Thumbnails listens on port 9185 by default.
The default location storing thumbnails is $OCIS_BASE_DATA_PATH/thumbnails

Thumbnailing Performance

Thumbnail generation can consume considerably resources. For thumbnailing, two libraries are available. One library is embedded in the thumbnail service and statically linked where the other one is an external shared library, dynamically linked and must be available via the OS when accessing. The external shared library is not only much faster, but also provides more possible image types that can be converted. While the embedded library is only available with the binary deployment, the external library is built and preconfigured with the container deployment only. Note, if required, you can manually add the shared library to your OS and build Infinite Scale for the binary deployment on your own. To do so, see the developer documentation for more details.

Configuration Hints

File Locations Overview

The relevant environment variables defining file locations are:

Variable Meaning

Variable	Meaning
`OCIS_BASE_DATA_PATH` ⁽¹⁾	Having a default set by the Infinite Scale code, but if defined, used as base path for other services.
`STORAGE_USERS_OCIS_ROOT`	Source files, defaults to ⁽¹⁾ plus path component, but can be freely defined if required.
`THUMBNAILS_FILESYSTEMSTORAGE_ROOT`	Target files, defaults to ⁽¹⁾ plus path component, but can be freely defined if required.

OCIS_BASE_DATA_PATH ⁽¹⁾

Having a default set by the Infinite Scale code, but if defined, used as base path for other services.

STORAGE_USERS_OCIS_ROOT

Source files, defaults to ⁽¹⁾ plus path component, but can be freely defined if required.

THUMBNAILS_FILESYSTEMSTORAGE_ROOT

Target files, defaults to ⁽¹⁾ plus path component, but can be freely defined if required.

Thumbnail Location

It may be beneficial to define the location of the thumbnails to be other than the default (with system files). This is due to the fact that storing thumbnails can consume a lot of space over time which not necessarily needs to reside on the same partition or mount or expensive drives.

Thumbnail Source File Types

Thumbnails can be generated from the following source file types:

png
jpg
gif
tiff
bmp
txt

The thumbnail service retrieves source files using the information provided by the backend. The Linux backend identifies source files usually based on the extension.

If a file type was not properly assigned or the type identification failed, thumbnail generation will fail and an error will be logged.

Thumbnail Target File Types

Thumbnails can either be generated as png, jpg or gif files. These types are hardcoded and no other types can be requested. A requestor, like another service or a client, can request one of the available types to be generated. If more than one type is required, each type must be requested individually.

Thumbnail Query String Parameters

Clients can request thumbnail previews for files by adding ?preview=1 to the file URL. Requests for files with no thumbnail available respond with HTTP status 404. For a detailed list of parameters see: Developer Documentation.

Thumbnail Resolution

Various resolutions can be defined via THUMBNAILS_RESOLUTIONS. A requestor can request any arbitrary resolution and the thumbnail service will use the one closest to the requested resolution. If more than one resolution is required, each resolution must be requested individually.

Example
Type	Resolution
Requested	18x12
Available	30x20, 15x10, 9x6
Returned	15x10

Deleting Thumbnails

As of now, there is no automated thumbnail deletion. This is especially true when a source file gets deleted or moved. This situation will be solved at a later stage. For the time being, if you run short on physical thumbnails space, you have to manually delete the thumbnail store to free space. Thumbnails will then be recreated on request.

Memory Considerations

Since source files need to be loaded into memory when generating thumbnails, large source files could potentially crash this service if there is insufficient memory available. For bigger instances when using container orchestration deployment methods, this service can be dedicated to its own server(s) with more memory.

To have more control over memory and CPU consumption, the maximum number of concurrent requests can be limited by setting the environment variable THUMBNAILS_MAX_CONCURRENT_REQUESTS. The default value is 0 which does not apply any restrictions to the number of concurrent requests. As soon as the number of concurrent requests is reached, any further request will be responded with 429/Too Many Requests and a requesting client can retry at a later point in time.

Thumbnails and SecureView

If a resource is shared using SecureView, the share reciever will get a 403 (forbidden) response when requesting a thumbnail. The requesting client needs to decide what to show and usually a placeholder thumbnail is used.

Configuration

Environment Variables

The thumbnails service is configured via the following environment variables. Read the Environment Variable Types documentation for important details. Column IV shows with which release the environment variable has been introduced.

master + Rolling 6.6.1

Environment variables for the thumbnails service
Name	IV	Type	Default Value	Description
`OCIS_TRACING_ENABLED` `THUMBNAILS_TRACING_ENABLED`	pre5.0	bool	false	Activates tracing.
`OCIS_TRACING_TYPE` `THUMBNAILS_TRACING_TYPE`	pre5.0	string		The type of tracing. Defaults to '', which is the same as 'jaeger'. Allowed tracing types are 'jaeger' and '' as of now.
`OCIS_TRACING_ENDPOINT` `THUMBNAILS_TRACING_ENDPOINT`	pre5.0	string		The endpoint of the tracing agent.
`OCIS_TRACING_COLLECTOR` `THUMBNAILS_TRACING_COLLECTOR`	pre5.0	string		The HTTP endpoint for sending spans directly to a collector, i.e. http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset.
`OCIS_LOG_LEVEL` `THUMBNAILS_LOG_LEVEL`	pre5.0	string		The log level. Valid values are: 'panic', 'fatal', 'error', 'warn', 'info', 'debug', 'trace'.
`OCIS_LOG_PRETTY` `THUMBNAILS_LOG_PRETTY`	pre5.0	bool	false	Activates pretty log output.
`OCIS_LOG_COLOR` `THUMBNAILS_LOG_COLOR`	pre5.0	bool	false	Activates colorized log output.
`OCIS_LOG_FILE` `THUMBNAILS_LOG_FILE`	pre5.0	string		The path to the log file. Activates logging to this file if set.
`THUMBNAILS_DEBUG_ADDR`	pre5.0	string	127.0.0.1:9189	Bind address of the debug server, where metrics, health, config and debug endpoints will be exposed.
`THUMBNAILS_DEBUG_TOKEN`	pre5.0	string		Token to secure the metrics endpoint.
`THUMBNAILS_DEBUG_PPROF`	pre5.0	bool	false	Enables pprof, which can be used for profiling.
`THUMBNAILS_DEBUG_ZPAGES`	pre5.0	bool	false	Enables zpages, which can be used for collecting and viewing in-memory traces.
`THUMBNAILS_GRPC_ADDR`	pre5.0	string	127.0.0.1:9185	The bind address of the GRPC service.
`THUMBNAILS_MAX_CONCURRENT_REQUESTS`	6.0.0	int	0	Number of maximum concurrent thumbnail requests. Default is 0 which is unlimited.
`THUMBNAILS_HTTP_ADDR`	pre5.0	string	127.0.0.1:9186	The bind address of the HTTP service.
`OCIS_HTTP_TLS_ENABLED`	pre5.0	bool	false	Activates TLS for the http based services using the server certifcate and key configured via OCIS_HTTP_TLS_CERTIFICATE and OCIS_HTTP_TLS_KEY. If OCIS_HTTP_TLS_CERTIFICATE is not set a temporary server certificate is generated - to be used with PROXY_INSECURE_BACKEND=true.
`OCIS_HTTP_TLS_CERTIFICATE`	pre5.0	string		Path/File name of the TLS server certificate (in PEM format) for the http services.
`OCIS_HTTP_TLS_KEY`	pre5.0	string		Path/File name for the TLS certificate key (in PEM format) for the server certificate to use for the http services.
`THUMBNAILS_HTTP_ROOT`	pre5.0	string	/thumbnails	Subdirectory that serves as the root for this HTTP service.
`OCIS_CORS_ALLOW_ORIGINS` `THUMBNAILS_CORS_ALLOW_ORIGINS`	6.0	[]string	[*]	A list of allowed CORS origins. See following chapter for more details: Access-Control-Allow-Origin at https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Origin. See the Environment Variable Types description for more details.
`OCIS_CORS_ALLOW_METHODS` `THUMBNAILS_CORS_ALLOW_METHODS`	6.0	[]string	[GET POST PUT PATCH DELETE OPTIONS]	A list of allowed CORS methods. See following chapter for more details: Access-Control-Request-Method at https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Request-Method. See the Environment Variable Types description for more details.
`OCIS_CORS_ALLOW_HEADERS` `THUMBNAILS_CORS_ALLOW_HEADERS`	6.0	[]string	[Authorization Origin Content-Type Accept X-Requested-With X-Request-Id Cache-Control]	A list of allowed CORS headers. See following chapter for more details: Access-Control-Request-Headers at https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Request-Headers. See the Environment Variable Types description for more details.
`OCIS_CORS_ALLOW_CREDENTIALS` `THUMBNAILS_CORS_ALLOW_CREDENTIALS`	6.0	bool	true	Allow credentials for CORS.See following chapter for more details: Access-Control-Allow-Credentials at https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Credentials.
`THUMBNAILS_RESOLUTIONS`	pre5.0	[]string	[16x16 32x32 64x64 128x128 1080x1920 1920x1080 2160x3840 3840x2160 4320x7680 7680x4320]	The supported list of target resolutions in the format WidthxHeight like 32x32. You can define any resolution as required. See the Environment Variable Types description for more details.
`THUMBNAILS_FILESYSTEMSTORAGE_ROOT`	pre5.0	string	/var/lib/ocis/thumbnails	The directory where the filesystem storage will store the thumbnails. If not defined, the root directory derives from $OCIS_BASE_DATA_PATH/thumbnails.
`OCIS_INSECURE` `THUMBNAILS_WEBDAVSOURCE_INSECURE`	pre5.0	bool	false	Ignore untrusted SSL certificates when connecting to the webdav source.
`OCIS_INSECURE` `THUMBNAILS_CS3SOURCE_INSECURE`	pre5.0	bool	false	Ignore untrusted SSL certificates when connecting to the CS3 source.
`OCIS_REVA_GATEWAY`	pre5.0	string	com.owncloud.api.gateway	CS3 gateway used to look up user metadata
`THUMBNAILS_TXT_FONTMAP_FILE`	pre5.0	string		The path to a font file for txt thumbnails.
`THUMBNAILS_TRANSFER_TOKEN`	pre5.0	string		The secret to sign JWT to download the actual thumbnail file.
`THUMBNAILS_DATA_ENDPOINT`	pre5.0	string	http://127.0.0.1:9186/thumbnails/data	The HTTP endpoint where the actual thumbnail file can be downloaded.
`THUMBNAILS_MAX_INPUT_WIDTH`	6.0.0	int	7680	The maximum width of an input image which is being processed.
`THUMBNAILS_MAX_INPUT_HEIGHT`	6.0.0	int	7680	The maximum height of an input image which is being processed.
`THUMBNAILS_MAX_INPUT_IMAGE_FILE_SIZE`	6.0.0	string	50MB	The maximum file size of an input image which is being processed. Usable common abbreviations: [KB, KiB, MB, MiB, GB, GiB, TB, TiB, PB, PiB, EB, EiB], example: 2GB.

YAML Example

Note the file shown below must be renamed and placed in the correct folder according to the Configuration File Naming conventions to be effective.
See the Notes for Environment Variables if you want to use environment variables in the yaml file.

master + Rolling 6.6.1

# Autogenerated
# Filename: thumbnails-config-example.yaml

tracing:
  enabled: false
  type: ""
  endpoint: ""
  collector: ""
log:
  level: ""
  pretty: false
  color: false
  file: ""
debug:
  addr: 127.0.0.1:9189
  token: ""
  pprof: false
  zpages: false
grpc:
  addr: 127.0.0.1:9185
  tls: null
  max_concurrent_requests: 0
http:
  addr: 127.0.0.1:9186
  tls:
    enabled: false
    cert: ""
    key: ""
  root: /thumbnails
  cors:
    allow_origins:
    - '*'
    allow_methods:
    - GET
    - POST
    - PUT
    - PATCH
    - DELETE
    - OPTIONS
    allow_headers:
    - Authorization
    - Origin
    - Content-Type
    - Accept
    - X-Requested-With
    - X-Request-Id
    - Cache-Control
    allow_credentials: true
grpc_client_tls: null
thumbnail:
  resolutions:
  - 16x16
  - 32x32
  - 64x64
  - 128x128
  - 1080x1920
  - 1920x1080
  - 2160x3840
  - 3840x2160
  - 4320x7680
  - 7680x4320
  filesystem_storage:
    root_directory: /var/lib/ocis/thumbnails
  webdav_allow_insecure: false
  cs3_allow_insecure: false
  reva_gateway: com.owncloud.api.gateway
  font_map_file: ""
  transfer_secret: ""
  data_endpoint: http://127.0.0.1:9186/thumbnails/data
  max_input_width: 7680
  max_input_height: 7680
  max_input_image_file_size: 50MB