Thumbnails Service Configuration

Introduction

The thumbnails service provides methods to generate thumbnails for various files and resolutions based on requests. It retrieves the sources at the location where the user files are stored and saves the thumbnails where system files are stored. Those locations have defaults but can be manually defined via environment variables.

Default Values

  • Thumbnails listens on port 9185 by default.

  • The default location storing thumbnails is $OCIS_BASE_DATA_PATH:/thumbnails

Configuration Hints

File Locations Overview

The relevant environment variables defining file locations are:

Variable Meaning

OCIS_BASE_DATA_PATH (1)

Having a default set by the Infinite Scale code, but if defined, used as base path for other services.

STORAGE_USERS_OCIS_ROOT

Source files, defaults to (1) plus path component, but can be freely defined if required.

THUMBNAILS_FILESYSTEMSTORAGE_ROOT

Target files, defaults to (1) plus path component, but can be freely defined if required.

Thumbnail Location

It may be beneficial to define the location of the thumbnails to be other than the default (with system files). This is due to the fact that storing thumbnails can consume a lot of space over time which not necessarily needs to reside on the same partition or mount or expensive drives.

Thumbnail Source File Types

Thumbnails can be generated from the following source file types:

  • png

  • jpg

  • gif

  • tiff

  • bmp

  • txt

The thumbnail service retrieves source files using the information provided by the backend. The Linux backend identifies source files usually based on the extension.

If a file type was not properly assigned or the type identification failed, thumbnail generation will fail and an error will be logged.

Thumbnail Target File Types

Thumbnails can either be generated as png, jpg or gif files. These types are hardcoded and no other types can be requested. A requestor, like another service or a client, can request one of the available types to be generated. If more than one type is required, each type must be requested individually.

Thumbnail Resolution

Various resolutions can be defined via THUMBNAILS_RESOLUTIONS. A requestor can request any arbitrary resolution and the thumbnail service will use the one closest to the requested resolution. If more than one resolution is required, each resolution must be requested individually.

Example
Type Resolution

Requested

18x12

Available

30x20,
15x10,
9x6

Returned

15x10

Deleting Thumbnails

As of now, there is no automated thumbnail deletion. This is especially true when a source file gets deleted or moved. This situation will be solved at a later stage. For the time being, if you run short on physical thumbnails space, you have to manually delete the thumbnail store to free space. Thumbnails will then be recreated on request.

Memory Considerations

Since source files need to be loaded into memory when generating thumbnails, large source files could potentially crash this service if there is insufficient memory available. For bigger instances when using container orchestration deployment methods, this service can be dedicated to its own server(s) with more memory.

Configuration

Environment Variables

The thumbnails extension is configured via the following environment variables:

  • latest

  • 2.0.0

Environment variables for the thumbnails service
Name Type Default Value Description

OCIS_TRACING_ENABLED
THUMBNAILS_TRACING_ENABLED

bool

false

Activates tracing.

OCIS_TRACING_TYPE
THUMBNAILS_TRACING_TYPE

string

The type of tracing. Defaults to "", which is the same as "jaeger". Allowed tracing types are "jaeger" and "" as of now.

OCIS_TRACING_ENDPOINT
THUMBNAILS_TRACING_ENDPOINT

string

The endpoint of the tracing agent.

OCIS_TRACING_COLLECTOR
THUMBNAILS_TRACING_COLLECTOR

string

The HTTP endpoint for sending spans directly to a collector, i.e. http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset.

OCIS_LOG_LEVEL
THUMBNAILS_LOG_LEVEL

string

The log level. Valid values are: "panic", "fatal", "error", "warn", "info", "debug", "trace".

OCIS_LOG_PRETTY
THUMBNAILS_LOG_PRETTY

bool

false

Activates pretty log output.

OCIS_LOG_COLOR
THUMBNAILS_LOG_COLOR

bool

false

Activates colorized log output.

OCIS_LOG_FILE
THUMBNAILS_LOG_FILE

string

The path to the log file. Activates logging to this file if set.

THUMBNAILS_DEBUG_ADDR

string

127.0.0.1:9189

Bind address of the debug server, where metrics, health, config and debug endpoints will be exposed.

THUMBNAILS_DEBUG_TOKEN

string

Token to secure the metrics endpoint.

THUMBNAILS_DEBUG_PPROF

bool

false

Enables pprof, which can be used for profiling.

THUMBNAILS_DEBUG_ZPAGES

bool

false

Enables zpages, which can be used for collecting and viewing in-memory traces.

THUMBNAILS_GRPC_ADDR

string

127.0.0.1:9185

The bind address of the GRPC service.

OCIS_GRPC_TLS_ENABLED

bool

false

Activates TLS for the grpc based services using the server certifcate and key configured via OCIS_GRPC_TLS_CERTIFICATE and OCIS_GRPC_TLS_KEY. If OCIS_GRPC_TLS_CERTIFICATE is not set a temporary server certificate is generated - to be used with OCIS_GRPC_CLIENT_TLS_MODE=insecure.

OCIS_GRPC_TLS_CERTIFICATE

string

Path/File name of the TLS server certificate (in PEM format) for the grpc services.

OCIS_GRPC_TLS_KEY

string

Path/File name for the TLS certificate key (in PEM format) for the server certificate to use for the grpc services.

THUMBNAILS_HTTP_ADDR

string

127.0.0.1:9186

The bind address of the HTTP service.

OCIS_HTTP_TLS_ENABLED

bool

false

Activates TLS for the http based services using the server certifcate and key configured via OCIS_HTTP_TLS_CERTIFICATE and OCIS_HTTP_TLS_KEY. If OCIS_HTTP_TLS_CERTIFICATE is not set a temporary server certificate is generated - to be used with PROXY_INSECURE_BACKEND=true.

OCIS_HTTP_TLS_CERTIFICATE

string

Path/File name of the TLS server certificate (in PEM format) for the http services.

OCIS_HTTP_TLS_KEY

string

Path/File name for the TLS certificate key (in PEM format) for the server certificate to use for the http services.

THUMBNAILS_HTTP_ROOT

string

/thumbnails

Subdirectory that serves as the root for this HTTP service.

OCIS_GRPC_CLIENT_TLS_MODE

string

TLS mode for grpc connection to the go-micro based grpc services. Possible values are 'off', 'insecure' and 'on'. 'off': disables transport security for the clients. 'insecure' allows to use transport security, but disables certificate verification (to be used with the autogenerated self-signed certificates). 'on' enables transport security, including server ceritificate verification.

OCIS_GRPC_CLIENT_TLS_CACERT

string

Path/File name for the root CA certificate (in PEM format) used to validate TLS server certificates of the go-micro based grpc services.

THUMBNAILS_RESOLUTIONS

[]string

[16x16 32x32 64x64 128x128 1920x1080 3840x2160 7680x4320]

The supported target resolutions in the format WidthxHeight e.g. 32x32. You can define any resolution as required and separate multiple resolutions by blank or comma.

THUMBNAILS_FILESYSTEMSTORAGE_ROOT

string

~/.ocis/thumbnails

The directory where the filesystem storage will store the thumbnails. If not definied, the root directory derives from $OCIS_BASE_DATA_PATH:/thumbnails.

OCIS_INSECURE
THUMBNAILS_WEBDAVSOURCE_INSECURE

bool

false

Ignore untrusted SSL certificates when connecting to the webdav source.

OCIS_INSECURE
THUMBNAILS_CS3SOURCE_INSECURE

bool

false

Ignore untrusted SSL certificates when connecting to the CS3 source.

REVA_GATEWAY

string

127.0.0.1:9142

CS3 gateway used to look up user metadata

THUMBNAILS_TXT_FONTMAP_FILE

string

The path to a font file for txt thumbnails.

THUMBNAILS_TRANSFER_TOKEN

string

The secret to sign JWT to download the actual thumbnail file.

THUMBNAILS_DATA_ENDPOINT

string

http://127.0.0.1:9186/thumbnails/data

The HTTP endpoint where the actual thumbnail file can be downloaded.

Environment variables for the thumbnails service
Name Type Default Value Description

OCIS_TRACING_ENABLED
THUMBNAILS_TRACING_ENABLED

bool

false

Activates tracing.

OCIS_TRACING_TYPE
THUMBNAILS_TRACING_TYPE

string

The type of tracing. Defaults to "", which is the same as "jaeger". Allowed tracing types are "jaeger" and "" as of now.

OCIS_TRACING_ENDPOINT
THUMBNAILS_TRACING_ENDPOINT

string

The endpoint of the tracing agent.

OCIS_TRACING_COLLECTOR
THUMBNAILS_TRACING_COLLECTOR

string

The HTTP endpoint for sending spans directly to a collector, i.e. http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset.

OCIS_LOG_LEVEL
THUMBNAILS_LOG_LEVEL

string

The log level. Valid values are: "panic", "fatal", "error", "warn", "info", "debug", "trace".

OCIS_LOG_PRETTY
THUMBNAILS_LOG_PRETTY

bool

false

Activates pretty log output.

OCIS_LOG_COLOR
THUMBNAILS_LOG_COLOR

bool

false

Activates colorized log output.

OCIS_LOG_FILE
THUMBNAILS_LOG_FILE

string

The path to the log file. Activates logging to this file if set.

THUMBNAILS_DEBUG_ADDR

string

127.0.0.1:9189

Bind address of the debug server, where metrics, health, config and debug endpoints will be exposed.

THUMBNAILS_DEBUG_TOKEN

string

Token to secure the metrics endpoint.

THUMBNAILS_DEBUG_PPROF

bool

false

Enables pprof, which can be used for profiling.

THUMBNAILS_DEBUG_ZPAGES

bool

false

Enables zpages, which can be used for collecting and viewing in-memory traces.

THUMBNAILS_GRPC_ADDR

string

127.0.0.1:9185

The bind address of the GRPC service.

OCIS_GRPC_TLS_ENABLED

bool

false

Activates TLS for the grpc based services using the server certifcate and key configured via OCIS_GRPC_TLS_CERTIFICATE and OCIS_GRPC_TLS_KEY. If OCIS_GRPC_TLS_CERTIFICATE is not set a temporary server certificate is generated - to be used with OCIS_GRPC_CLIENT_TLS_MODE=insecure.

OCIS_GRPC_TLS_CERTIFICATE

string

Path/File name of the TLS server certificate (in PEM format) for the grpc services.

OCIS_GRPC_TLS_KEY

string

Path/File name for the TLS certificate key (in PEM format) for the server certificate to use for the grpc services.

THUMBNAILS_HTTP_ADDR

string

127.0.0.1:9186

The bind address of the HTTP service.

OCIS_HTTP_TLS_ENABLED

bool

false

Activates TLS for the http based services using the server certifcate and key configured via OCIS_HTTP_TLS_CERTIFICATE and OCIS_HTTP_TLS_KEY. If OCIS_HTTP_TLS_CERTIFICATE is not set a temporary server certificate is generated - to be used with PROXY_INSECURE_BACKEND=true.

OCIS_HTTP_TLS_CERTIFICATE

string

Path/File name of the TLS server certificate (in PEM format) for the http services.

OCIS_HTTP_TLS_KEY

string

Path/File name for the TLS certificate key (in PEM format) for the server certificate to use for the http services.

THUMBNAILS_HTTP_ROOT

string

/thumbnails

Subdirectory that serves as the root for this HTTP service.

OCIS_GRPC_CLIENT_TLS_MODE

string

TLS mode for grpc connection to the go-micro based grpc services. Possible values are 'off', 'insecure' and 'on'. 'off': disables transport security for the clients. 'insecure' allows to use transport security, but disables certificate verification (to be used with the autogenerated self-signed certificates). 'on' enables transport security, including server ceritificate verification.

OCIS_GRPC_CLIENT_TLS_CACERT

string

Path/File name for the root CA certificate (in PEM format) used to validate TLS server certificates of the go-micro based grpc services.

THUMBNAILS_RESOLUTIONS

[]string

[16x16 32x32 64x64 128x128 1920x1080 3840x2160 7680x4320]

The supported target resolutions in the format WidthxHeight e.g. 32x32. You can define any resolution as required and separate multiple resolutions by blank or comma.

THUMBNAILS_FILESYSTEMSTORAGE_ROOT

string

~/.ocis/thumbnails

The directory where the filesystem storage will store the thumbnails. If not definied, the root directory derives from $OCIS_BASE_DATA_PATH:/thumbnails.

OCIS_INSECURE
THUMBNAILS_WEBDAVSOURCE_INSECURE

bool

false

Ignore untrusted SSL certificates when connecting to the webdav source.

OCIS_INSECURE
THUMBNAILS_CS3SOURCE_INSECURE

bool

false

Ignore untrusted SSL certificates when connecting to the CS3 source.

REVA_GATEWAY

string

127.0.0.1:9142

CS3 gateway used to look up user metadata

THUMBNAILS_TXT_FONTMAP_FILE

string

The path to a font file for txt thumbnails.

THUMBNAILS_TRANSFER_TOKEN

string

The secret to sign JWT to download the actual thumbnail file.

THUMBNAILS_DATA_ENDPOINT

string

http://127.0.0.1:9186/thumbnails/data

The HTTP endpoint where the actual thumbnail file can be downloaded.

YAML Example

  • latest

  • 2.0.0

# Autogenerated
# Filename: thumbnails-config-example.yaml

tracing:
  enabled: false
  type: ""
  endpoint: ""
  collector: ""
log:
  level: ""
  pretty: false
  color: false
  file: ""
debug:
  addr: 127.0.0.1:9189
  token: ""
  pprof: false
  zpages: false
grpc:
  addr: 127.0.0.1:9185
  tls:
    enabled: false
    cert: ""
    key: ""
http:
  addr: 127.0.0.1:9186
  tls:
    enabled: false
    cert: ""
    key: ""
  root: /thumbnails
grpc_client_tls:
  mode: ""
  cacert: ""
thumbnail:
  resolutions:
  - 16x16
  - 32x32
  - 64x64
  - 128x128
  - 1920x1080
  - 3840x2160
  - 7680x4320
  filesystem_storage:
    root_directory: ~/.ocis/thumbnails
  webdav_allow_insecure: false
  cs3_allow_insecure: false
  reva_gateway: 127.0.0.1:9142
  font_map_file: ""
  transfer_secret: ""
  data_endpoint: http://127.0.0.1:9186/thumbnails/data
# Autogenerated
# Filename: thumbnails-config-example.yaml

tracing:
  enabled: false
  type: ""
  endpoint: ""
  collector: ""
log:
  level: ""
  pretty: false
  color: false
  file: ""
debug:
  addr: 127.0.0.1:9189
  token: ""
  pprof: false
  zpages: false
grpc:
  addr: 127.0.0.1:9185
  tls:
    enabled: false
    cert: ""
    key: ""
http:
  addr: 127.0.0.1:9186
  tls:
    enabled: false
    cert: ""
    key: ""
  root: /thumbnails
grpc_client_tls:
  mode: ""
  cacert: ""
thumbnail:
  resolutions:
  - 16x16
  - 32x32
  - 64x64
  - 128x128
  - 1920x1080
  - 3840x2160
  - 7680x4320
  filesystem_storage:
    root_directory: ~/.ocis/thumbnails
  webdav_allow_insecure: false
  cs3_allow_insecure: false
  reva_gateway: 127.0.0.1:9142
  font_map_file: ""
  transfer_secret: ""
  data_endpoint: http://127.0.0.1:9186/thumbnails/data