Antivirus Service Configuration

Table of Contents

Introduction
Memory Considerations
Antivirus Configuration
Operation Modes
- Post-processing
- Scaling in Kubernetes
Event Bus Configuration
Configuration
- Environment Variables
- YAML Example

Introduction

The Infinite Scale antivirus service is responsible for scanning files for viruses.

The antivirus service does not start automatically and must be started manually. For more details see the Start Infinite Scale section.
The reason for excluding the antivirus service from autostart is, that it needs an external antivirus service available and configured.

Memory Considerations

The antivirus service can consume considerably amounts of memory. This is relevant to provide or define sufficient memory for the deployment selected. To avoid out of memory (OOM) situations, the following equation gives a rough overview based on experiences made. The memory calculation comes without any guarantee, is intended as overview only and subject of change.

memory limit = max file size x workers x factor 8 - 14

With:

ANTIVIRUS_WORKERS == 1

 50MB file --> factor 14   --> 700MB memory
844MB file --> factor  8,3 -->   7GB memory

Antivirus Configuration

Antivirus Scanner Type

The antivirus service currently supports ICAP and clamAV as antivirus scanners. The ANTIVIRUS_SCANNER_TYPE environment variable is used to select the scanner. The detailed configuration for each scanner heavily depends on the scanner type selected. See the environment variables for more details.

For icap, only scanners using the X-Infection-Found header are currently supported.
For clamav, local Unix Domain Sockets and remote network sockets can be configured either by:
- pointing to a file like /tmp/clamd.socket or
- with a stream socket like tcp://host.docker.internal:3310.

Maximum Scan Size

Several factors can make it necessary to limit the maximum file size the antivirus service will use for scanning. Use the ANTIVIRUS_MAX_SCAN_SIZE environment variable to scan only a given amount of bytes. Obviously, it is recommended to scan the whole file, but several factors like scanner type and version, bandwidth, performance issues, etc. might make a limit necessary.

To prevent Out or Memory errors, ANTIVIRUS_MAX_SCAN_SIZE needs to be set lower than available RAM.

Antivirus Workers

The number of concurrent scans can be increased by setting ANTIVIRUS_WORKERS. Be aware that this will also increase memory usage.

Infected File Handling

The antivirus service allows three different ways of handling infected files. Those can be set via the ANTIVIRUS_INFECTED_FILE_HANDLING environment variable:

delete: (default): Infected files will be deleted immediately, further post-processing is cancelled.
abort: (advanced option): Infected files will be kept, further post-processing is cancelled. Files can be manually retrieved and inspected by an admin. To identify the file for further investigation, the antivirus service logs the abort/infected state including the file ID. The file is located in the storage/users/uploads folder of the ocis data directory and persists until it is manually deleted by the admin via the Manage Unfinished Uploads command.
continue: (not recommended - only for debugging): Infected files will be marked via metadata as infected but post-processing continues normally. Note: Infected Files are moved to their final destination and therefore not prevented from download which includes the risk of spreading viruses.

In all cases, a log entry is added declaring the infection, and handling method and a notification via the userlog service is sent.

Scanner Inaccessibility

In case a scanner is not accessible by the antivirus service like a network outage, service outage, or hardware outage, the antivirus service uses the abort case for further processing, independent of the actual setting made. In any case, an error is logged noting the inaccessibility of the scanner used.

Operation Modes

The antivirus service can scan files during post-processing. on demand scanning is currently not available and might be added in a future release.

Post-processing

The antivirus service will scan files during post-processing. It listens for a post-processing step called virusscan. This step can be added in the environment variable POSTPROCESSING_STEPS. Read the documentation of the postprocessing service for more details.

Scaling in Kubernetes

In kubernetes, ANTIVIRUS_WORKERS and ANTIVIRUS_MAX_SCAN_SIZE can be used to trigger the horizontal pod autoscaler by requesting a memory size that is below ANTIVIRUS_MAX_SCAN_SIZE. Keep in mind that ANTIVIRUS_MAX_SCAN_SIZE amount of memory might be held by ANTIVIRUS_WORKERS number of go routines.

Event Bus Configuration

The Infinite Scale event bus can be configured by a set of environment variables.

If you are using a binary installation as described in Minimal Bare Metal Deployment or Bare Metal with systemd, the address of the event bus OCIS_EVENTS_ENDPOINT is predefined as localhost address without the need for further configuration, but changeable on demand.
In case of an orchestrated installation like with Docker or Kubernetes, the event bus must be an external service for scalability like a Redis Sentinel cluster or a key-value-store NATS JetStream. Both named stores are supported and also used in Caching and Persistence. The store used is not part of the Infinite Scale installation and must be separately provided and configured.
Note that from a configuration point of view, caching and persistence are independent of the event bus configuration.

Note that for each global environment variable, a service-based one might be available additionally. For precedences see Environment Variable Notes. Check the configuration section below.

Without the aim of completeness, see the list of environment variables to configure the event bus:

Envvar Description

Envvar	Description
`OCIS_EVENTS_ENDPOINT`	The address of the event system.
`OCIS_EVENTS_CLUSTER`	The clusterID of the event system. Mandatory when using NATS as event system.
`OCIS_EVENTS_ENABLE_TLS`	Enable TLS for the connection to the events broker.
`OCIS_INSECURE`	Whether to verify the server TLS certificates.
`OCIS_EVENTS_AUTH_USERNAME`	The username to authenticate with the events broker.
`OCIS_EVENTS_AUTH_PASSWORD`	The password to authenticate with the events broker.

OCIS_EVENTS_ENDPOINT

The address of the event system.

OCIS_EVENTS_CLUSTER

The clusterID of the event system. Mandatory when using NATS as event system.

OCIS_EVENTS_ENABLE_TLS

Enable TLS for the connection to the events broker.

OCIS_INSECURE

Whether to verify the server TLS certificates.

OCIS_EVENTS_AUTH_USERNAME

The username to authenticate with the events broker.

OCIS_EVENTS_AUTH_PASSWORD

The password to authenticate with the events broker.

Configuration

Environment Variables

The antivirus service is configured via the following environment variables. Read the Environment Variable Types documentation for important details. Column IV shows with which release the environment variable has been introduced.

master + Rolling 6.6.1

Environment variables for the antivirus service
Name	IV	Type	Default Value	Description
`OCIS_LOG_LEVEL` `ANTIVIRUS_LOG_LEVEL`	pre5.0	string		The log level. Valid values are: 'panic', 'fatal', 'error', 'warn', 'info', 'debug', 'trace'.
`OCIS_LOG_PRETTY` `ANTIVIRUS_LOG_PRETTY`	pre5.0	bool	false	Activates pretty log output.
`OCIS_LOG_COLOR` `ANTIVIRUS_LOG_COLOR`	pre5.0	bool	false	Activates colorized log output.
`OCIS_LOG_FILE` `ANTIVIRUS_LOG_FILE`	pre5.0	string		The path to the log file. Activates logging to this file if set.
`ANTIVIRUS_DEBUG_ADDR`	pre5.0	string	127.0.0.1:9277	Bind address of the debug server, where metrics, health, config and debug endpoints will be exposed.
`ANTIVIRUS_DEBUG_TOKEN`	pre5.0	string		Token to secure the metrics endpoint.
`ANTIVIRUS_DEBUG_PPROF`	pre5.0	bool	false	Enables pprof, which can be used for profiling.
`ANTIVIRUS_DEBUG_ZPAGES`	pre5.0	bool	false	Enables zpages, which can be used for collecting and viewing in-memory traces.
`OCIS_TRACING_ENABLED` `ANTIVIRUS_TRACING_ENABLED`	pre5.0	bool	false	Activates tracing.
`OCIS_TRACING_TYPE` `ANTIVIRUS_TRACING_TYPE`	pre5.0	string		The type of tracing. Defaults to '', which is the same as 'jaeger'. Allowed tracing types are 'jaeger' and '' as of now.
`OCIS_TRACING_ENDPOINT` `ANTIVIRUS_TRACING_ENDPOINT`	pre5.0	string		The endpoint of the tracing agent.
`OCIS_TRACING_COLLECTOR` `ANTIVIRUS_TRACING_COLLECTOR`	pre5.0	string		The HTTP endpoint for sending spans directly to a collector, i.e. http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset.
`ANTIVIRUS_INFECTED_FILE_HANDLING`	pre5.0	string	delete	Defines the behaviour when a virus has been found. Supported options are: 'delete', 'continue' and 'abort '. Delete will delete the file. Continue will mark the file as infected but continues further processing. Abort will keep the file in the uploads folder for further admin inspection and will not move it to its final destination.
`OCIS_EVENTS_ENDPOINT` `ANTIVIRUS_EVENTS_ENDPOINT`	pre5.0	string	127.0.0.1:9233	The address of the event system. The event system is the message queuing service. It is used as message broker for the microservice architecture.
`OCIS_EVENTS_CLUSTER` `ANTIVIRUS_EVENTS_CLUSTER`	pre5.0	string	ocis-cluster	The clusterID of the event system. The event system is the message queuing service. It is used as message broker for the microservice architecture. Mandatory when using NATS as event system.
`OCIS_INSECURE` `ANTIVIRUS_EVENTS_TLS_INSECURE`	pre5.0	bool	false	Whether to verify the server TLS certificates.
`OCIS_EVENTS_TLS_ROOT_CA_CERTIFICATE` `ANTIVIRUS_EVENTS_TLS_ROOT_CA_CERTIFICATE`	pre5.0	string		The root CA certificate used to validate the server’s TLS certificate. If provided ANTIVIRUS_EVENTS_TLS_INSECURE will be seen as false.
`OCIS_EVENTS_ENABLE_TLS` `ANTIVIRUS_EVENTS_ENABLE_TLS`	pre5.0	bool	false	Enable TLS for the connection to the events broker. The events broker is the ocis service which receives and delivers events between the services.
`OCIS_EVENTS_AUTH_USERNAME` `ANTIVIRUS_EVENTS_AUTH_USERNAME`	5.0	string		The username to authenticate with the events broker. The events broker is the ocis service which receives and delivers events between the services.
`OCIS_EVENTS_AUTH_PASSWORD` `ANTIVIRUS_EVENTS_AUTH_PASSWORD`	5.0	string		The password to authenticate with the events broker. The events broker is the ocis service which receives and delivers events between the services.
`ANTIVIRUS_WORKERS`	7.0.0	int	10	The number of concurrent go routines that fetch events from the event queue.
`ANTIVIRUS_SCANNER_TYPE`	pre5.0	string	clamav	The antivirus scanner to use. Supported values are 'clamav' and 'icap'.
`ANTIVIRUS_CLAMAV_SOCKET`	pre5.0	string	/run/clamav/clamd.ctl	The socket clamav is running on. Note the default value is an example which needs adaption according your OS.
`ANTIVIRUS_ICAP_SCAN_TIMEOUT`	5.0	Duration	5m0s	Scan timeout for the ICAP client. Defaults to '5m' (5 minutes). See the Environment Variable Types description for more details.
`ANTIVIRUS_ICAP_URL`	pre5.0	string	icap://127.0.0.1:1344	URL of the ICAP server.
`ANTIVIRUS_ICAP_SERVICE`	pre5.0	string	avscan	The name of the ICAP service.
`ANTIVIRUS_MAX_SCAN_SIZE`	pre5.0	string		The maximum scan size the virus scanner can handle. Only this many bytes of a file will be scanned. 0 means unlimited and is the default. Usable common abbreviations: [KB, KiB, MB, MiB, GB, GiB, TB, TiB, PB, PiB, EB, EiB], example: 2GB.
`ANTIVIRUS_DEBUG_SCAN_OUTCOME`	pre5.0	string		A predefined outcome for virus scanning, FOR DEBUG PURPOSES ONLY! (example values: 'found,infected')

YAML Example

Note the file shown below must be renamed and placed in the correct folder according to the Configuration File Naming conventions to be effective.
See the Notes for Environment Variables if you want to use environment variables in the yaml file.

master + Rolling 6.6.1

# Autogenerated
# Filename: antivirus-config-example.yaml

file: ""
log:
  level: ""
  pretty: false
  color: false
  file: ""
debug:
  addr: 127.0.0.1:9277
  token: ""
  pprof: false
  zpages: false
tracing:
  enabled: false
  type: ""
  endpoint: ""
  collector: ""
infected-file-handling: delete
events:
  endpoint: 127.0.0.1:9233
  cluster: ocis-cluster
  tls_insecure: false
  tls_root_ca_certificate: ""
  enable_tls: false
  username: ""
  password: ""
workers: 10
scanner:
  type: clamav
  clamav:
    socket: /run/clamav/clamd.ctl
  icap:
    scan_timeout: 5m0s
    url: icap://127.0.0.1:1344
    service: avscan
max-scan-size: ""