Background Jobs
Introduction
A system like ownCloud sometimes requires tasks to be done on a regular basis without requiring user interaction or hindering ownCloud’s performance. For that reason, as a system administrator you can configure background jobs (for example, database clean-ups) to be executed without any user interaction.
These jobs are typically referred to as Cron Jobs. Cron jobs are commands or shell-based scripts that are scheduled to periodically run at fixed times, dates, or intervals. To run Cron jobs with ownCloud, we recommend that you use the occ system:cron command.
Use the occ background command set to select which scheduler you want to use for controlling. For more details on how to manage background jobs, refer to Managing Background Jobs.
As an example:
sudo -u www-data ./occ background:cronIs the same as using the Cron section on your ownCloud Admin page.
Docker Note
If you are using the official docker images you don’t need to take care about the configuration for background jobs, the docker image is already configured to use cron internally. If required, this can also be adjusted by the environment variables OWNCLOUD_CROND_ENABLED and OWNCLOUD_CROND_SCHEDULE.
Cron Jobs
You can schedule Cron jobs in three ways: Cron, Webcron, or AJAX. These can all be configured in the admin settings menu. However, the recommended method is to use Cron. The following sections describe the differences between each method.
There are a number of things to keep in mind when choosing an automation option:
-
While the default method is AJAX, though the preferred way is to use Cron.
The reason for this distinction is that AJAX is easier to get up and running. As a result, it makes sense (often times) to accept it in the interests of expediency. However, doing so is known to cause issues, such as backlogs and potentially not running every job on a heavily-loaded system. What’s more, an increasing amount of ownCloud automation has been migrated from AJAX to Cron in recent versions. For this reason, we encourage you to not use it for too long — especially if your site is rapidly growing. -
While Webcron is better than AJAX, it has limitations too.
For example, running Webcron will only remove a single item from the job queue, not all of them. Cron, however, will clear the entire queue.
| It’s for this reason that we encourage you to use Cron — if at all possible. |
Cron
Using the operating system Cron feature is the preferred method for executing regular tasks. This method enables the execution of scheduled jobs without the inherent limitations which the web server might have.
For example, to run a Cron job on a *nix system every 15 minutes (recommended), under the default web server user (often, www-data or wwwrun) you must set up the following Cron job to call the occ system:cron command:
sudo crontab -u www-data -e
*/15 * * * * /usr/bin/php -f /path/to/your/owncloud/occ system:cronYou can verify if the cron job has been added and scheduled by executing:
sudo crontab -u www-data -l
*/15 * * * * /usr/bin/php -f /path/to/your/owncloud/occ system:cron| You have to make sure that PHP is found by Cron; hence why we’ve deliberately added the full path. |
Please refer to the crontab man page for the exact command syntax if you don’t want to have it run every 15 minutes.
| There are other methods to invoke programs by the system regularly, e.g., systemd timers |
Webcron
By registering your ownCloud cron.php script address as an external webcron service (for example, easyCron), you ensure that background jobs are executed regularly. To use this type of service, your external webcron service must be able to access your ownCloud server using the Internet. For example:
URL to call: http[s]://<domain-of-your-server>/owncloud/cron.phpAJAX
The AJAX scheduling method is the default option.
However, it is also the least reliable. Each time a user visits the ownCloud page, a single background job is executed. The advantage of this mechanism, however, is that it does not require access to the system nor registration with a third party service. The disadvantage of this mechanism, when compared to the Webcron service, is that it requires regular visits to the page for it to be triggered.
| Especially when using the Activity App or external storages, where new files are added, updated, or deleted one of the other methods should be used. |
Parallel Task Execution
Regardless of the approach which you take, since ownCloud 9.1, Cron jobs can be run in parallel. This is done by running background:cron multiple times. Depending on the process which you are automating, this may not be necessary. However, for longer-running tasks, such as those which are LDAP related, it may be very beneficial.
There is no way to do so via the ownCloud UI. But, the most direct way to do so, is by opening three console tabs and in each one run
sudo -u www-data ./occ system:cronEach of these processes would acquire their own list of jobs to process without overlapping any other.
Available Background Jobs
A number of existing background jobs are available to be run just for specific tasks.
|
These jobs are generally only needed on large instances and can be run as background jobs. If the number of users in your installation ranges between 1,000 and 3,000, or if you’re using LDAP and it becomes a bottleneck, then admins can delete several entries in the
If used, these should be scheduled to run on a daily basis. |
While not exhaustive, these include:
CleanupChunks
The CleanupChunks command, occ dav:cleanup-chunks, will clean up outdated chunks (uploaded files) more than a certain number of days old and needs to be added to your crontab.
There is no matching background job to delete from the oc_jobs table.
|
ExpireTrash
The ExpireTrash job, contained in OCA\Files_Trashbin\BackgroundJob\ExpireTrash, will remove any file in the ownCloud trash bin which is older than the specified maximum file retention time. It can be run, as follows, using the OCC trashbin command:
sudo -u www-data ./occ trashbin:expireExpireVersions
The ExpireVersions job, contained in OCA\Files_Versions\BackgroundJob\ExpireVersions, will expire versions of files which are older than the specified maximum version retention time. It can be run, as follows, using the OCC versions command:
sudo -u www-data ./occ versions:expire
Please take care when adding ExpireTrash and ExpireVersions as Cron jobs. Make sure that they’re not started in parallel on multiple machines. Running in parallel on a single machine is fine. But, currently, there isn’t sufficient locking in place to prevent them from conflicting with each other if running in parallel across multiple machines.
|
SyncJob (CardDAV)
The CardDAV SyncJob, contained in OCA\DAV\CardDAV\SyncJob, syncs the local system address book, updating any existing contacts, and deleting any expired contacts. It can be run, as follows, using the OCC dav command:
sudo -u www-data ./occ dav:sync-system-addressbookSyncJob (Federation)
OCAFederationSyncJob
It can be run, as follows, using the OCC federation sync command:
sudo -u www-data ./occ federation:sync-addressbooksTroubleshooting
Remove Non-Existent Background Jobs
See the Remove Non-Existent Background Jobs section in the general troubleshooting documentation for more details.