Background Jobs

Introduction

A system like ownCloud sometimes requires tasks to be done on a regular basis without requiring user interaction or hindering ownCloud’s performance. For that reason, as a system administrator you can configure background jobs (for example, database clean-ups) to be executed without any user interaction.

These jobs are typically referred to as Cron Jobs. Cron jobs are commands or shell-based scripts that are scheduled to periodically run at fixed times, dates, or intervals. To run Cron jobs with ownCloud, we recommend that you use the occ system:cron command.

Use the occ background command set to select which scheduler you want to use for controlling. For more details on how to manage background jobs, refer to Managing Background Jobs.

As an example:

sudo -u www-data ./occ background:cron

Is the same as using the Cron section on your ownCloud Admin page.

Docker Note

If you are using the official docker images you don’t need to take care about the configuration for background jobs, the docker image is already configured to use cron internally. If required, this can also be adjusted by the environment variables OWNCLOUD_CROND_ENABLED and OWNCLOUD_CROND_SCHEDULE.

Cron Jobs

You can schedule Cron jobs in three ways: Cron, Webcron, or AJAX. These can all be configured in the admin settings menu. However, the recommended method is to use Cron. The following sections describe the differences between each method.

There are a number of things to keep in mind when choosing an automation option:

  1. While the default method is AJAX, though the preferred way is to use Cron.
    The reason for this distinction is that AJAX is easier to get up and running. As a result, it makes sense (often times) to accept it in the interests of expediency. However, doing so is known to cause issues, such as backlogs and potentially not running every job on a heavily-loaded system. What’s more, an increasing amount of ownCloud automation has been migrated from AJAX to Cron in recent versions. For this reason, we encourage you to not use it for too long — especially if your site is rapidly growing.

  2. While Webcron is better than AJAX, it has limitations too.
    For example, running Webcron will only remove a single item from the job queue, not all of them. Cron, however, will clear the entire queue.

It’s for this reason that we encourage you to use Cron — if at all possible.

Cron

Using the operating system Cron feature is the preferred method for executing regular tasks. This method enables the execution of scheduled jobs without the inherent limitations which the web server might have.

For example, to run a Cron job on a *nix system every 15 minutes (recommended), under the default web server user (often, www-data or wwwrun) you must set up the following Cron job to call the occ system:cron command:

sudo crontab -u www-data -e
*/15  *  *  *  * /usr/bin/php -f /path/to/your/owncloud/occ system:cron

You can verify if the cron job has been added and scheduled by executing:

sudo crontab -u www-data -l
*/15  *  *  *  * /usr/bin/php -f /path/to/your/owncloud/occ system:cron
You have to make sure that PHP is found by Cron; hence why we’ve deliberately added the full path.

Please refer to the crontab man page for the exact command syntax if you don’t want to have it run every 15 minutes.

There are other methods to invoke programs by the system regularly, e.g., systemd timers

Webcron

By registering your ownCloud cron.php script address as an external webcron service (for example, easyCron), you ensure that background jobs are executed regularly. To use this type of service, your external webcron service must be able to access your ownCloud server using the Internet. For example:

URL to call: http[s]://<domain-of-your-server>/owncloud/cron.php

AJAX

The AJAX scheduling method is the default option.
However, it is also the least reliable. Each time a user visits the ownCloud page, a single background job is executed. The advantage of this mechanism, however, is that it does not require access to the system nor registration with a third party service. The disadvantage of this mechanism, when compared to the Webcron service, is that it requires regular visits to the page for it to be triggered.

Especially when using the Activity App or external storages, where new files are added, updated, or deleted one of the other methods should be used.

Parallel Task Execution

Regardless of the approach which you take, since ownCloud 9.1, Cron jobs can be run in parallel. This is done by running background:cron multiple times. Depending on the process which you are automating, this may not be necessary. However, for longer-running tasks, such as those which are LDAP related, it may be very beneficial.

There is no way to do so via the ownCloud UI. But, the most direct way to do so, is by opening three console tabs and in each one run

sudo -u www-data ./occ system:cron

Each of these processes would acquire their own list of jobs to process without overlapping any other.

Available Background Jobs

A number of existing background jobs are available to be run just for specific tasks.

These jobs are generally only needed on large instances and can be run as background jobs. If the number of users in your installation ranges between 1,000 and 3,000, or if you’re using LDAP and it becomes a bottleneck, then admins can delete several entries in the oc_jobs table and replace them with the corresponding occ command, which you can see here:

  • OCA\\DAV\CardDAV\\SyncJobocc dav:sync-system-addressbook

  • OCA\\Federation\\SyncJobocc federation:sync-addressbooks

  • OCA\\Files_Trashbin\\BackgroundJob\\ExpireTrashocc trashbin:expire

  • OCA\\Files_Versions\\BackgroundJob\\ExpireVersionsocc versions:expire

If used, these should be scheduled to run on a daily basis.

While not exhaustive, these include:

CleanupChunks

The CleanupChunks command, occ dav:cleanup-chunks, will clean up outdated chunks (uploaded files) more than a certain number of days old and needs to be added to your crontab.

There is no matching background job to delete from the oc_jobs table.

ExpireTrash

The ExpireTrash job, contained in OCA\Files_Trashbin\BackgroundJob\ExpireTrash, will remove any file in the ownCloud trash bin which is older than the specified maximum file retention time. It can be run, as follows, using the OCC trashbin command:

sudo -u www-data ./occ trashbin:expire

ExpireVersions

The ExpireVersions job, contained in OCA\Files_Versions\BackgroundJob\ExpireVersions, will expire versions of files which are older than the specified maximum version retention time. It can be run, as follows, using the OCC versions command:

sudo -u www-data ./occ versions:expire
Please take care when adding ExpireTrash and ExpireVersions as Cron jobs. Make sure that they’re not started in parallel on multiple machines. Running in parallel on a single machine is fine. But, currently, there isn’t sufficient locking in place to prevent them from conflicting with each other if running in parallel across multiple machines.

SyncJob (CardDAV)

The CardDAV SyncJob, contained in OCA\DAV\CardDAV\SyncJob, syncs the local system address book, updating any existing contacts, and deleting any expired contacts. It can be run, as follows, using the OCC dav command:

sudo -u www-data ./occ dav:sync-system-addressbook

SyncJob (Federation)

OCAFederationSyncJob

It can be run, as follows, using the OCC federation sync command:

sudo -u www-data ./occ federation:sync-addressbooks

Troubleshooting

Remove Non-Existent Background Jobs

See the Remove Non-Existent Background Jobs section in the general troubleshooting documentation for more details.

Forbidden error for Scanner.php

If you find a Forbidden error message in your log files, with a reference to the Scanner.php file, then you should:

  • Check if you have any shares with the status pending.

  • Configure conditional logging for cron to see more output.