Policies Service Configuration

Introduction

The Infinite Scale policies service provides a new gRPC API which can be used to check whether a requested operation is allowed or not. To do so, Open Policy Agent (OPA) is used to define the set of rules of what is permitted and what is not.

  • The policies service does not start automatically and must be started manually. For more details see the Start Infinite Scale section.

  • The reason for excluding the policies service from autostart is, that you must have a defined rulset available. If the ruleset is not available, the policies service will prevent other services from starting up.

Policies are written in the Rego query language. The location of the Rego files can be configured via yaml. A configuration using environment variables is not possible.

Default Values

  • The policies service listens on port 9125 by default.

General Information

The policies service consists of the following modules:

  • Proxy authorization (middleware)

  • Event authorization (async post-processing)

  • gRPC API (can be used by other services)

To configure the policies service, three environment variables need to be defined:

  • POLICIES_ENGINE_TIMEOUT

  • POLICIES_POSTPROCESSING_QUERY

  • PROXY_POLICIES_QUERY

Note that each query setting defines the Complete Rules variable defined in the Rego rule set the corresponding step uses for the evaluation. If the variable is mistyped or not found, the evaluation defaults to deny. Individual query definitions can be defined for each module.

To activate the policies service for a module, it must be started with a yaml configuration that points to at least one Rego file that contains the complete rule variable to be queried. Note that if the service is scaled horizontally, each instance should have access to the same Rego files to avoid unpredictable results. If a file path has been configured but the file is not present or accessible, the evaluation defaults to deny.

When using async post-processing via the postprocessing service, the value policies must be added to the POSTPROCESSING_STEPS configuration in the order in which the evaluation should take place. Example: First check if a file contains questionable content via policies. If it looks okay, continue to check for viruses.

For configuration examples, the example policies provided by the Example Policies are used.

Modules

gRPC API

The gRPC API can be used by any other internal service. It can also be used for example by third parties to find out if an action is allowed or not. This layer is already used by the proxy middleware. There is no configuration necessary, because the query setting (complete rule variable) must be part of the request.

Proxy Middleware

The proxy service already includes a middleware which uses the internal gRPC api to evaluate the policies. Since the proxy is in heavy use and every HTTP request is processed here, only simple and quick decisions should be evaluated. More complex queries such as file content evaluation are strongly discouraged.

If the evaluation in the proxy results in a denied outcome, the response will return a 403 Permission Denied with the following response body:

{
    "error":
    {
        "code": "deniedByPolicy",
        "message": "Operation denied due to security policies",
        "innererror":
        {
            "date": "2023-09-19T13:22:20Z",
            "filename": "File",
            "method": "POST",
            "path": "/dav/spaces/some-space-id/Folder/",
            "request-id": "9CFCE925-F9D9-4F26-AB3B-2C1C40A9CD0C"
        }
    }
}

Event Service (Postprocessing)

This layer is event-based and part of the postprocessing service. Since processing at this point is asynchronous, the operations can also take longer and be more expensive, like evaluating the contents of a file.

Defining Policies to Evaluate

Each module can have as many policy files as needed for evaluation. Files can also include other files if necessary. To use policies, they have to be saved to a location that is accessible to the policies service. As a good starting point, take the config directory and use a subdirectory collecting all the .rego files, though any other directory can be defined. The config directory is already accessible by all services and usually is included in a backup plan.

If this is done, it’s required to configure the policies service to use these files:

It is important that all necessary files are added to the list of files the policies service uses. 
policies:
  engine:
    policies:
      - your_path_to_policies/proxy.rego
      - your_path_to_policies/postprocessing.rego
      - your_path_to_policies/util.rego

Once the references to policy files are configured correctly, the _QUERY configuration needs to be defined for the proxy middleware and for the events service.

Setting the Query Configuration

To define a value for the query evaluation, the following scheme is necessary:

data.<package-name>.<complete-rule-variable-name>

  • The keyword data is mandatory and must be present.

  • The package-name is defined in one .rego file like package postprocessing. It is not related to the filename. For more details, see the packages documentation.

  • The complete-rule-variable-name is the variable providing the result of the evaluation.

  • Exact one of the defined files, which is responsible for returning the evaluation result, must contain the combination of <package-name> and <complete-rule-variable-name>.

Proxy

Note that this setting has to be part of the proxy configuration.

proxy:
  policies_middleware:
    query: data.proxy.granted

The same can be achieved by setting the following environment variable:

PROXY_POLICIES_QUERY=data.proxy.granted

Postprocessing

policies:
  postprocessing:
    query: data.postprocessing.granted

The same can be achieved by setting the following environment variable:

POLICIES_POSTPROCESSING_QUERY=data.postprocessing.granted

As soon as that query is configured, the postprocessing service must be informed to use the policies step by setting the environment variable:

POSTPROCESSING_STEPS=policies

Note that additional steps can be configured and their position in the list defines the order of processing. For details see the postprocessing service documentation.

Example Policies

The policies service contains a set of preconfigured example policies. See the deployment examples directory for details. The contained policies disallow Infinite Scale to create certain file types, both via the proxy middleware and the events service via postprocessing.

  • 7.0.0

Rego policies deployment examples directory

Using git version name: v7.0.0

Extend Mime Type File Extension Mapping

In the extended set of the rego query language, it is possible to get a list of associated file extensions based on a mime type, for example ocis.mimetype.extensions("application/pdf").

The list of mappings is restricted by default and is provided by the host system Infinite Scale is installed on.

This list can be extended. To do so, Infinite Scale must be provided with the path to a custom mime.types file that maps mime types to extensions. The location of the file must be accessible by all instances of the policy service. As a rule of thumb, use the directory where the Infinite Scale configuration files are stored. Note that existing mappings from the host are extended by the definitions from the mime types file, but not replaced.

The path to that file can be provided via a yaml configuration or an environment variable. Make sure to replace the OCIS_CONFIG_DIR string by an existing path. See the documentation of the Configuration Directory and the Extended Environment Variables for predefined or manually defined config paths.

POLICIES_ENGINE_MIMES=OCIS_CONFIG_DIR/mime.types
policies:
  engine:
    mimes: OCIS_CONFIG_DIR/mime.types

A good example of how such a file should be formatted can be found in the Apache svn repository.

Event Bus Configuration

The Infinite Scale event bus can be configured by a set of environment variables.

  • If you are using a binary installation as described in Minimal Bare Metal Deployment or Bare Metal with systemd, the address of the event bus OCIS_EVENTS_ENDPOINT is predefined as localhost address without the need for further configuration, but changeable on demand.

  • In case of an orchestrated installation like with Docker or Kubernetes, the event bus must be an external service for scalability like a Redis Sentinel cluster or a key-value-store NATS JetStream. Both named stores are supported and also used in Caching and Persistence. The store used is not part of the Infinite Scale installation and must be separately provided and configured.

  • Note that from a configuration point of view, caching and persistence are independent of the event bus configuration.

Note that for each global environment variable, a service-based one might be available additionally. For precedences see Environment Variable Notes. Check the configuration section below.

Without the aim of completeness, see the list of environment variables to configure the event bus:

Envvar Description

OCIS_EVENTS_ENDPOINT

The address of the event system.

OCIS_EVENTS_CLUSTER

The clusterID of the event system. Mandatory when using NATS as event system.

OCIS_EVENTS_ENABLE_TLS

Enable TLS for the connection to the events broker.

OCIS_INSECURE

Whether to verify the server TLS certificates.

OCIS_EVENTS_AUTH_USERNAME

The username to authenticate with the events broker.

OCIS_EVENTS_AUTH_PASSWORD

The password to authenticate with the events broker.

Configuration

Environment Variables

The policies service is configured via the following environment variables. Read the Environment Variable Types documentation for important details. Column IV shows with which release the environment variable has been introduced.

  • 7.0.0

Environment variables for the policies service
Name IV Type Default Value Description

POLICIES_GRPC_ADDR

pre5.0

string

127.0.0.1:9125

The bind address of the GRPC service.

POLICIES_DEBUG_ADDR

pre5.0

string

127.0.0.1:9129

Bind address of the debug server, where metrics, health, config and debug endpoints will be exposed.

POLICIES_DEBUG_TOKEN

pre5.0

string

Token to secure the metrics endpoint.

POLICIES_DEBUG_PPROF

pre5.0

bool

false

Enables pprof, which can be used for profiling.

POLICIES_DEBUG_ZPAGES

pre5.0

bool

false

Enables zpages, which can be used for collecting and viewing in-memory traces.

OCIS_EVENTS_ENDPOINT
POLICIES_EVENTS_ENDPOINT

pre5.0

string

127.0.0.1:9233

The address of the event system. The event system is the message queuing service. It is used as message broker for the microservice architecture.

OCIS_EVENTS_CLUSTER
POLICIES_EVENTS_CLUSTER

pre5.0

string

ocis-cluster

The clusterID of the event system. The event system is the message queuing service. It is used as message broker for the microservice architecture. Mandatory when using NATS as event system.

OCIS_INSECURE
POLICIES_EVENTS_TLS_INSECURE

pre5.0

bool

false

Whether the server should skip the client certificate verification during the TLS handshake.

OCIS_EVENTS_TLS_ROOT_CA_CERTIFICATE
POLICIES_EVENTS_TLS_ROOT_CA_CERTIFICATE

pre5.0

string

The root CA certificate used to validate the server’s TLS certificate. If provided POLICIES_EVENTS_TLS_INSECURE will be seen as false.

OCIS_EVENTS_ENABLE_TLS
POLICIES_EVENTS_ENABLE_TLS

pre5.0

bool

false

Enable TLS for the connection to the events broker. The events broker is the ocis service which receives and delivers events between the services.

OCIS_EVENTS_AUTH_USERNAME
POLICIES_EVENTS_AUTH_USERNAME

5.0

string

The username to authenticate with the events broker. The events broker is the ocis service which receives and delivers events between the services.

OCIS_EVENTS_AUTH_PASSWORD
POLICIES_EVENTS_AUTH_PASSWORD

5.0

string

The password to authenticate with the events broker. The events broker is the ocis service which receives and delivers events between the services.

OCIS_LOG_LEVEL
POLICIES_LOG_LEVEL

pre5.0

string

The log level. Valid values are: 'panic', 'fatal', 'error', 'warn', 'info', 'debug', 'trace'.

OCIS_LOG_PRETTY
POLICIES_LOG_PRETTY

pre5.0

bool

false

Activates pretty log output.

OCIS_LOG_COLOR
POLICIES_LOG_COLOR

pre5.0

bool

false

Activates colorized log output.

OCIS_LOG_FILE
POLICIES_LOG_FILE

pre5.0

string

The path to the log file. Activates logging to this file if set.

POLICIES_ENGINE_TIMEOUT

pre5.0

Duration

10s

Sets the timeout the rego expression evaluation can take. Rules default to deny if the timeout was reached. See the Environment Variable Types description for more details.

POLICIES_ENGINE_MIMES

pre5.0

string

Sets the mimes file path which maps mimetypes to associated file extensions. See the text description for details.

POLICIES_POSTPROCESSING_QUERY

pre5.0

string

Defines the 'Complete Rules' variable defined in the rego rule set this step uses for its evaluation. Defaults to deny if the variable was not found.

OCIS_TRACING_ENABLED
POLICIES_TRACING_ENABLED

5.0

bool

false

Activates tracing.

OCIS_TRACING_TYPE
POLICIES_TRACING_TYPE

5.0

string

The type of tracing. Defaults to '', which is the same as 'jaeger'. Allowed tracing types are 'jaeger' and '' as of now.

OCIS_TRACING_ENDPOINT
POLICIES_TRACING_ENDPOINT

5.0

string

The endpoint of the tracing agent.

OCIS_TRACING_COLLECTOR
POLICIES_TRACING_COLLECTOR

5.0

string

The HTTP endpoint for sending spans directly to a collector, i.e. http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset.

YAML Example

  • 7.0.0

# Autogenerated
# Filename: policies-config-example.yaml

grpc:
  addr: 127.0.0.1:9125
  tls: null
debug:
  addr: 127.0.0.1:9129
  token: ""
  pprof: false
  zpages: false
events:
  endpoint: 127.0.0.1:9233
  cluster: ocis-cluster
  tls_insecure: false
  tls_root_ca_certificate: ""
  enable_tls: false
  username: ""
  password: ""
grpc_client_tls: null
log:
  level: ""
  pretty: false
  color: false
  file: ""
engine:
  timeout: 10s
  policies: []
  mimes: ""
postprocessing:
  query: ""
tracing:
  enabled: false
  type: ""
  endpoint: ""
  collector: ""