General Information

Introduction

This document covers general aspects of Infinite Scale like start modes, services, important minimum configuration etc. for a common understanding.

We highly recommend reading this document first before you start setting up your system. Many obstacles can be avoided when knowing the basic concepts. Though it is tempting to just give things a try - which is totally ok, you will quickly realize that you may have to start again from scratch before the setup meets your requirements.

The example commands shown need to be adjusted depending on whether you are using the Binary Setup or a Container Setup.

When using global options on startup, you can always use command line options or environment variables. Run ocis help and see Starting Infinite Scale With Environment Variables for details.

Embedded Supervisor (Runtime)

Infinite Scale has an embedded supervisor for managing the runtime and reducing the memory footprint. In addition, this supervisor takes care that a service will be restarted automatically if it fails. When using an external supervisor like systemd, Kubernetes or others, the embedded supervisor is not needed, the services are managed by the external supervisor.

Deciding the Startup Mode

The following two mode types do not predefine a particular installation method like the manual or container setup. When using Kubernetes, the embedded supervisor is not necessary, the supervisor of the underlying system is used.

Starting Infinite Scale using the embedded supervisor

This mode can be used when scaling is not the primary focus and can be the case if you have a:

Starting Infinite Scale in unsupervised mode

This mode is used when availability, scaling and the adjustment to dynamically changing requirements have a high priority. In this case, an external supervisor like Kubernetes is used to deploy and run Infinite Scale with its services.

Managing Services

Services are built as microservices which can be started, stopped or instantiated. Services Rules documentation has been added to explain some background. Read this carefully to avoid unwanted behavior. For details of each service see the Services documentation.

List Available Services

Just type ocis to get a list of commands and available services.

When typing ocis <service-name> --help, you will get detailed help regarding the specified service.

Manage Instances of Services

Infinite Scale Supervised Services

In supervised mode, all services are started with one command as you can see below in the example when using the binary setup. Note that the services started with the runtime share the same PID.

Start the Infinite Scale Runtime
ocis server
List running services
ocis list
Stopping the Infinite Scale Runtime

In supervised mode, you have to stop the ocis server which also stops all services. See Stopping Infinite Scale for more details.

Unsupervised Services

At any time, you can create unsupervised instances of a service with ocis [service-name] server, for example ocis proxy server. These services are independent of services in supervised mode and have their own PID. The Instances are managed with classic OS methods or e.g. via Kubernetes.

Note that you need configuration for and access to the service instances like with a load balancer when you scale.

Configuring Services

To configure services, see the Services section in the Deployment documentation.

Configuration Rules

Administrators must be aware of the sources, the location and order applied (the configuration file arithmetics). Mismanaging them can be a source of confusion leading to undesired results on the final configuration created and applied.

  • Infinite Scale uses a hierarchical structure for its configuration, where each element overwrites its precedent. These are:

Configurations must be located in the OCIS_CONFIG_DIR path.

Configuration File Naming

The configuration files for Infinite Scale are YAML-based (a human-friendly data serialization language).

The filename to define a config has the following namespace:

ocis.yaml
 or
[service name].yaml

When using ocis.yaml and you configure a service, the topic for the service configuration must be the service name.

You can list the available services names by typing:

ocis list

Special Configuration Settings

Special configuration settings can be the following:

  • Configurations that can only be made with yaml but not with environment variables.

  • yaml configurations that use an OS environment variable for the value.

yaml only settings

There are certain situations where multiple settings using the same keys apply for a configuration set. This can be like in the app-registry service where default apps for mimetypes are defined. A single environment variable cannot hold all that information. For this case, a yaml configuration is the only way possible. Note that both installation methods, binary and container, can deal with yaml configuration files.

Using OS environment variables in yaml files

OS environment variables can be used in yaml config files for Infinite Scale services which will be replaced by the actual value of the environment variable at runtime. This method allows defining a standardized setup, but parameterize it for different use cases.

Example:
web:
  http:
    addr: ${SOME_HTTP_ADDR}

In the example above, the value of the OS enironment variable SOME_HTTP_ADDR will be used at runtime for the key addr.

Default Paths

As you will read below, the config directory and the base directory for storing metadata must be located on POSIX filesystems. Consider for the ease of backup and restore, to keep both directories on the same filesystem.

Note that the term blob is used for file data the user uploads, while metatdata refers to all data that describes the blob.

Environment variable Description

OCIS_CONFIG_DIR

Path to config files.

OCIS_BASE_DATA_PATH

Path to system relevant data.

STORAGE_USERS_OCIS_ROOT

Path to blobs and metadata if POSIX is used.
Derives from OCIS_BASE_DATA_PATH if not set otherwise.
Used if STORAGE_USERS_DRIVER is set to ocis

STORAGE_USERS_S3NG_ROOT

Path to metadata if S3 is used.
Derives from OCIS_BASE_DATA_PATH if not set otherwise.
Used if STORAGE_USERS_DRIVER is set to s3ng.
See Using S3 for Blobs for the S3 configuration.

Configuration Directory

  • The configuration directory has a default location for config files, which must be on a POSIX storage:

    • For container images (inside the container)
      /etc/ocis/

    • For binary releases
      $HOME/.ocis/config/

      • Do not replace $HOME with ~ (tilde).

        • The code does not resolve ~ to the users home directory.

      • Check that $HOME resolves to a valid directory.

        • When using a system user for the runtime, which has no login and therefore no home directory, like in the scenario Setting up systemd for Infinite Scale, you must specify a configuration file location.

  • You can deviate from the default location and define a custom configuration directory on startup using the environment variable OCIS_CONFIG_DIR.

Base Data Directory

Because Infinite Scale does not use a database for storing information like users, groups, spaces, internal data, etc., it saves all this data to a permanent file location. Depending on the system setup, the base directory contains not only the metadata but also blobs. See Filesystems and Shared Storage for more details.

  • When only using a supported POSIX filesystem, blobs and metadata are stored on POSIX.

  • When using S3 and POSIX, blobs are stored on S3, while metadata is stored on POSIX. See the S3 documentation for more details.

Path for System Data

The environment variable OCIS_BASE_DATA_PATH sets the base path in a generic way. It has a default location but can also be manually defined. It is the root for many services which automatically add a subdirectory to that root for storing their data. Some services can manually define that path if necessary. Defining them independently can be required like when using a Container Orchestration setup or recommended for using the search service or thumbnails service.

Default Location

  • The base path has a default location for metadata and service dependent data (see above) which must be on a POSIX storage. If not otherwise defined when using S3, it is also used to store blobs using that path:

    • For container images (inside the container)
      /var/lib/ocis

    • For binary releases
      $HOME/.ocis/

      • Do not replace $HOME with ~ (tilde).

        • The code does not resolve ~ to the users home directory.

      • Check that $HOME resolves to a valid directory.

        • When using a system user for the runtime, which has no login and therefore no home directory, like in the generic binary setup scenario Setting up systemd for Infinite Scale or in the deployment example Small-Scale with systemd, you must specify a base directory location because a system user has no logon and therefore no home directory!

  • You can deviate from the default location and define a custom base directory on startup using the environment variable OCIS_BASE_DATA_PATH.

  • More Important Notes

    When setting the base directory manually, it will be used automatically for the services described above - if they are not otherwise manually defined.
    The location must be used by Infinite Scale exclusively. Writing into this location not using Infinite Scale is discouraged to avoid any unexpected behavior.

Using S3 for Blobs

When using S3 for storing user data (blobs), metadata must reside on POSIX. The environment variable responsible for storaing metadata in S3 is STORAGE_USERS_S3NG_ROOT and derives, if not otherwise defined, from the base directory OCIS_BASE_DATA_PATH. For more details see the section Base Data Directory above.

Read the S3 documentation for more details on how to configure Infinite Scale for the with S3.

Initialize Infinite Scale

Infinite Scale can be run by manually defining the environment like you do when using Container Orchestration. When using the Binary Setup or the Container Setup, you can prepare Infinite Scale for further configuration and recurring starts. After reading The ocis init Command for important details, start the initialisation. To do so, run:

ocis init

You can add command line parameters. To see which ones are available, type:

ocis init --help

Command line parameters are beneficial if you e.g. want to hand over all necessary parameters without getting to any questionnaire or if you want to define the admin password yourself not getting a random one assigned.

The command line option --force-overwrite is only intended for developer usage. If you set this option, your config will be overwritten, your data, if any is present, will persist, but it will not be accessible anymore. This is, among other things, because the issuer (short iss part of openID Connect) will be overwritten.

To reinitialize Infinite Scale, you have to delete your config and your data and start from scratch.

Start Infinite Scale With All Predefined Services

When you type ocis server, the embedded supervisor is automatically used and starts available predefined services automatically. The supervisor starts by default on port 9250 and listens for commands regarding the lifecycle of the supervised services.

To list the started predefined services, type:

ocis list

This will print an output like:

+--------------------+
|      SERVICE       |
+--------------------+
| app-provider       |
| app-registry       |
| auth-basic         |
| auth-machine       |
| auth-service       |
| clientlog          |
| eventhistory       |
| frontend           |
| gateway            |
| graph              |
| groups             |
| idm                |
| idp                |
| nats               |
| notifications      |
| ocdav              |
| ocm                |
| ocs                |
| postprocessing     |
| proxy              |
| search             |
| settings           |
| sharing            |
| sse                |
| storage-publiclink |
| storage-shares     |
| storage-system     |
| storage-users      |
| thumbnails         |
| userlog            |
| users              |
| web                |
| webdav             |
| webfinger          |
+--------------------+

Starting Infinite Scale With Environment Variables

You can use environment variables to define or overwrite config parameters which will be used when starting Infinite Scale like:

PROXY_HTTP_ADDR=0.0.0.0:5555 ocis server

or when using multiple environment variables like:

PROXY_HTTP_ADDR=0.0.0.0:5555 \
PROXY_DEBUG_ADDR=0.0.0.0:6666 \
ocis server

Globally Shared Logging Values

When running in supervised mode (ocis server), it is beneficial to have common values for logging so that the log output is correctly formatted or everything is piped to the same file without duplicating config keys and values all over the place. This is possible using the global log config key with the following example:

ocis.yaml
log:
  level: error
  color: true
  pretty: true
  file: /var/tmp/ocis_output.log
In case of a service overwriting its shared logging config received from the main ocis.yaml file, you must specify all values.

Log Config Keys

These are the necessary log keys and the available values:

log:
  level: [ error | warning | info | debug ]
  color: [ true | false ]
  pretty: [ true | false ]
  file: [ path/to/log/file ] # MUST not be used with pretty = true

Configurations to Access the Web UI

You can easily access Infinite Scale via ownCloud Web with minimal configuration needs. Without going into too much detail, you need to provide the following two environment variables. See also the section about Handling Certificates and Demo Users and Groups.

OCIS_URL

Expects a URL including protocol, host and optionally port to simplify configuring all the different services. Other service environment variables also using an URL still take precedence if set, but will fall back to this URL if not set.

If you need to access Infinite Scale running on a VM or a remote machine via a host name other than localhost or in a container, you must configure the host name with OCIS_URL. The same applies if you are not using host names but an IP address (e.g. 192.168.178.25) instead.
By default, Infinite Scale enforces https for web and client access. If necessary, this can be changed in particular setups to http, which is not recommended for production. For details see TLS for the HTTP Frontend and Proxy Service Configuration.

If you want to reuse an already configured minimized setup for any other address than https://localhost :
PROXY_HTTP_ADDR

When using 0.0.0.0:9200, the proxy will listen to all available interfaces. If you want or need to change that based on your requirements, you can use a different address e.g. to bind the proxy to an interface.

The bind address for PROXY_HTTP_ADDR must be on the same interface where the configured URL from the OCIS_URL environment variable is reachable.
Common reasons binding to a particular IP address

  • Multiple network interfaces configured for specific tasks like web, storage, administration.

  • Binding SSL certificates to IP addresses.

  • …​

Examples

  • PROXY_HTTP_ADDR=127.0.0.0:9200
    This causes Infinite Scale to only bind to the local network interface.

  • PROXY_HTTP_ADDR=0.0.0.0:9200
    This tells Infinite Scale to bind it to all available network interfaces.

Also see the Using the Embedded IDP Service for configuration notes.

Handling Certificates

Certificates are necessary to secure browser access. Infinite Scale can run with embedded self-signed certificates mainly used for testing purposes or signed certificates provided by the admin. To tell Infinite Scale which kind of certificates you are using, the environment variable OCIS_INSECURE is used.

  • When clients have direct access to the Infinite Scale instance:

    • Use signed certificates, see below.

    • When using unsigned certificates, depending on the browser used, errors may get logged like:
      TLS handshake error from IP:PORT: remote error: tls: unknown certificate

  • When using a reverse proxy:

    • Browser access terminates at the webserver and certificates are handled there.

    • You can additionally secure the communication using (un)signed certificates for communication between the reverse proxy and infinite Scale.

Embedded Self-Signed Certificates

In order to run Infinite Scale with automatically generated and self-signed certificates, set OCIS_INSECURE=true.

OCIS_INSECURE=true \
PROXY_HTTP_ADDR=0.0.0.0:9200 \
OCIS_URL=https://localhost:9200 \
ocis server

Provided Signed Certificates

Self-Signed Certificates

If your certificates are self-signed, set OCIS_INSECURE=true like in the example of embedded self-signed certificates above.

Certificates Signed by a Trusted CA

If you have your own certificates already in place, make Infinite Scale use them by adding the following environment variables to the command. Replace the certificates path and file names according to your needs:

OCIS_INSECURE=false \
PROXY_HTTP_ADDR=0.0.0.0:9200 \
OCIS_URL=https://localhost:9200 \
PROXY_TRANSPORT_TLS_KEY=./certs/your-host.key \
PROXY_TRANSPORT_TLS_CERT=./certs/your-host.crt \
ocis server

Default Users and Groups

Default users and groups are only created when you initialize Infinite Scale as first task. The same is true for demo users and groups which need an environment variable to be set on initializing Infinite Scale to get created.

If you have not declared demo user creation during initializing, you can for the time being only empty the Base Data Directory and remove the ocis.yaml file which resets the system. Then you can start from scratch and enable demo user creation.

Admin User

An admin user will be created when running the ocis init command with the following credentials:

Admin user and group created on first ocis start
Username Password Email Role Group

admin

Printed by the output of ocis init

admin@example.org

admin

users

Login to the webinterface with this admin user and change relevant data according your needs or create new users. As an example to reach out the webinterface use https://localhost:9200.

Password Reset for the Admin User

If you have forgotten the password for the admin user or want to change it, run the following command. Note that the admin user must already exist:

ocis idm resetpassword

After running this command and entering a new password, the admin can relogin using the new password.

The password is written into the ocis.yaml file in section idm:.

Demo Users and Groups

Create Demo Users and Groups

You can let Infinite Scale create demo users and groups for testing purposes. Because these demo users and groups can be a significant security issue, you should remove them before going into production or your system is exposed to the outside world, for details see Securing Infinite Scale.

To let Infinite Scale create these demo users and groups for you, start the runtime the very first time with:

IDM_CREATE_DEMO_USERS=true \
ocis init
List of available demo users and groups
Username Password Email Role Groups

einstein

relativity

einstein@example.org

user

users,
philosophy-haters,
physics-lovers,
sailing-lovers,
violin-haters

marie

radioactivity

marie@example.org

user

users,
physics-lovers,
polonium-lovers,
radium-lovers

moss

vista

moss@example.org

space admin

users

richard

superfluidity

richard@example.org

user

users,
philosophy-haters,
physics-lovers,
quantum-lovers

katherine

gemini

katherine@example.org

space admin

users,
sailing-lovers,
physics-lovers,
quantum-lovers

You can now login with one of the demo users created using the OCIS_URL in you browser like https://localhost:9200.

Manage Users and Groups

If you have enabled demo users and groups and you want to manage or delete them, use the web UI, e.g. https://localhost:9200.

Default Ports

See Used Port Ranges at the Services description for details.

Logging

See Logging at the Services description for details.

Using the Embedded IDP Service

See the Special Settings section in the Proxy service for configuration details when using the Infinite Scale IDP service instead of an external IDP.

Using a Reverse Proxy

If you are using a reverse proxy like Traefik and the reverse proxy manages the certificates to secure the client access, you can use extra certificates between the proxy and Infinite Scale, although this is not mandatory. See the section Handling Certificates for more details.

If you want to reuse an already configured minimized setup for any other address than https://localhost :

Maintenance Commands

There are multiple commands available to maintain the Infinite Scale instance. See the Maintenance Commands document for more details.