The HSM (Hardware Security Module) Daemon (hsmdaemon)

Introduction

The hsmdaemon is a daemon provided by ownCloud to delegate encryption to an HSM (Hardware Security Module). This can be necessary as PHP cannot directly interface with a PKCS11 stack, neither with an API wrapper because none exists, nor via the OpenSSL bindings. Therefore a separate process is needed to decrypt anything with the private key stored in an HSM.

When using hsmdaemon with an HSM, the keys may still be stored on the same physical machine as ownCloud.
For hsmdaemon support, you need ownCloud Enterprise Edition >= 10.2. We recommend consulting with us when deploying storage encryption with an HSM.
Starting with the Encryption App version 1.5.1, HSM can now work with both binary and base64 encoding/decoding. If not otherwise configured, binary is the default.

Running exec() to decrypt the key with a command line command to do the encryption might leak the HSM credentials if the admin lists the currently running processes. To prevent that, an HSM daemon will be used that can open a session to the HSM upon startup.

This daemon will be used by ownCloud to decrypt the current master key upon request. The communication happens via UNIX sockets or TCP sockets and is authorized by a shared token that the daemon stores in the ownCloud database via a REST/JSON route.

ownCloud internally uses OpenSSL to encrypt and decrypt keys and that is extended to support en-/decrypt operations via the new daemon. The current solution encrypts the ownCloud master key with a key from the HSM.

From the technical point of view the Crypt class is extended to handle the key generation in the HSM device and also to get the key from HSM. For the read/write operation on a file, the request goes to the HSM and then, based on the keys fetched from HSM, the files are encrypted or decrypted. The keys are not replaced.

How The HSM Daemon Interacts with ownCloud

Upon startup, the daemon will generate a token and send it to ownCloud via a new REST/JSON route. After connecting with the HSM daemon, an unsophisticated, line-based, protocol is used (every line ends with CRLF):

  1. ownCloud sends the token read from database.

  2. The daemon compares the received token with its token and returns an OK line.

  3. ownCloud then sends the data it wants to decrypt as a Base64-encoded, one-line string.

  4. The daemon returns the decrypted data as a Base64-encoded one-line string.

Doing so ensures that an evil admin will need to wiretap the communication between either the database or the HSM daemon and ownCloud.

Quick Overview

HSM support consists of two core parts:

  1. An actual HSM PKCS11 module.

  2. A hsmdaemon that provides a JWT - protected web API for the PKCS11 stack to generate key pairs and decrypt data.

Deployment Recommendation

We recommend running hsmdaemon on every web server to reduce latency.

Installation

Integrating the hsmdaemon with ownCloud requires 3 steps; these are:

  1. Install a PKCS11 Module

  2. Install and Configure the hsmdaemon

  3. Configure ownCloud

The installation instructions in this guide have been designed to work with ownCloud’s supported operating systems. If you are using a different operating system or distribution, please adjust the instructions to suit your environment.

Install a PKCS11 Module

Install Using a Preconfigured PKCS11 Module

At least one PKCS11 library is necessary. This is typically provided by an HSM vendor. If a PKCS11 library is not available, you can use the software HSM - SoftHSM2.

Initialise the Token

Now we can initialize the token:

sudo softhsm2-util --init-token --slot 0 --label "My token 1"

It will ask for two PINs, an SO and a User pin. See opendnssec for more information. The SO PIN can e.g. be used to re-initialize the token and the user PIN is handed out to the application so it can interact with the token.

Install PKCS11 CLI tools (optional)

To use the PKCS11 API on the CLI, we need to install OpenSC.

Initialise on Debian and Ubuntu

To install OpenSC on Debian and Ubuntu, run the following command:

sudo apt install -y opensc
Initialise on openSUSE and SUSE Linux Enterprise Server

To install OpenSC on openSUSE and SUSE Linux Enterprise Server, run the following command:

sudo zypper install -y --auto-agree-with-licenses opensc
Initialise on Fedora and Red Hat Enterprise Linux and Centos

To install OpenSC on Fedora and Red Hat Enterprise Linux and Centos, run the following command:

sudo yum install --assumeyes opensc

List Tokens

You can list available tokens using the pkcs11-tool by running the following command:

sudo pkcs11-tool --module </path/to/libsofthsm2.so> -l --pin <user-pin> -O
The Module Parameter

The module parameter is either the library provided by the HSM vendor or libsofthsm2 which was installed with SoftHSM 2. If you are using libsofthsm2, the path to libsofthsm2.so for each of the supported distributions is available below.

Distribution

Path

Debian and Ubuntu

/usr/lib/softhsm/libsofthsm2.so

openSUSE and SUSE Linux Enterprise Server

/usr/lib64/pkcs11/libsofthsm2.so

Fedora and Red Hat Enterprise Linux and Centos

/usr/lib64/pkcs11/libsofthsm2.so
See the OpenSC Wiki for more information.

Install and Configure the hsmdaemon

Installing hsmdaemon requires several steps. These are:

  1. Install the hsmdaemon Binary

  2. Copy the Config File

  3. Install the System Service

  4. Configure the PKCS 11 Module Path

  5. Configure Slot and Pin

  6. Test the hsmdaemon

  7. Configure Other Options

Install the hsmdaemon Binary

After you have obtained the hsmdaemon from ownCloud, you need to move the hsmdaemon binary to a directory located in your system path and make the binary executable:

sudo install -m 755 ./hsmdaemon /usr/local/bin/hsmdaemon

Copy the Config File

The default location where hsmdaemon looks for its config file is /etc/hsmdaemon/hsmdaemon.toml. To create it from the example config file available in the provided package, run the following commands:

Create the hsmdaemon configuration directory:

sudo mkdir /etc/hsmdaemon

Copy the example config file, allow only root and users in the root group to read & write the configuration file:

sudo install -m 640 ./hsmdaemon.toml /etc/hsmdaemon/hsmdaemon.toml

Install the System Service

Now that the binary is available and the configuration file is in place, hsmdaemon must be installed as a system service. To do so, run it with the install option as in the example below.

sudo /usr/local/bin/hsmdaemon install
sudo service hsmdaemon start

If it installs successfully, you should see the following console output:

Install HSM Daemon:           [  OK  ]

It should now be running and set to start automatically at boot time.

The daemon is managed using the following three commands:

  • sudo service hsmdaemon start

  • sudo service hsmdaemon stop and

  • sudo service hsmdaemon status

Configure the PKCS11 Module Path

To set the path to the PKCS11 module, update the line below in /etc/hsmdaemon/hsmdaemon.toml, with the appropriate path on your system.

[pkcs11]
# softhsm v2
module = "/usr/lib/x86_64-linux-gnu/softhsm/libsofthsm2.so"

List Available Slots

This command lists the available slots.

sudo hsmdaemon listslots
{"level":"debug","ts":"2019-02-14T09:27:02.068+0100","caller":"hsmdaemon/keymanager.go:27","msg":"initialize pkcs11 module","module":"/usr/lib/softhsm/libsofthsm2.so"}
{"level":"info","ts":"2019-02-14T09:27:02.087+0100","caller":"hsmdaemon/keymanager.go:65","msg":"Slots found","slotIds":[550099622,1989683358,2]}
Available slots:
Slot: 550099622,
    Slot info:
        Description:      SoftHSM slot ID 0x20c9daa6
        Manufacturer ID:  SoftHSM project
        Hardware version: 2.2
        Firmware version: 2.2
        Token present:    yes
        Flags:
    Token info:
        Manufacturer ID:    SoftHSM project
        Model:              SoftHSM v2
        Hardware version:   2.2
        Firmware version:   2.2
        Serial number:      e8ba06bca0c9daa6
        Initialized:        yes
        User PIN init.:     yes
        Label:              oc token without pin
        MaxSessionCount:    0
        SessionCount:       18446744073709551615
        MaxRwSessionCount:  0
        RwSessionCount:     18446744073709551615
        MaxPinLen:          255
        MinPinLen:          4
        TotalPublicMemory:  18446744073709551615
        FreePublicMemory:   18446744073709551615
        TotalPrivateMemory: 18446744073709551615
        FreePrivateMemory:  18446744073709551615
        UTCTime:            2019021408270200
        Flags: CKF_RNG CKF_LOGIN_REQUIRED CKF_RESTORE_KEY_NOT_NEEDED CKF_USER_PIN_COUNT_LOW
Slot: 1989683358,
    Slot info:
        Description:      SoftHSM slot ID 0x7698289e
        Manufacturer ID:  SoftHSM project
        Hardware version: 2.2
        Firmware version: 2.2
See the OpenSC Wiki for more information.

Configure the Slot and Pin

Define which slot to use and if a PIN is needed. Update /etc/hsmdaemon/hsmdaemon.toml with the information gathered in the pkcs11 section as in the example below.

[pkcs11]
# softhsm v2
module = "/usr/lib/x86_64-linux-gnu/softhsm/libsofthsm2.so"
# The user pin supplied when running softhsm2-util --init-token, comment it out
# or leave empty if no pin is necessary
pin = "1234"
# Find your slot id with `sudo hsmdaemon listslots`
slot = 550099622

Test the hsmdaemon

Test Key Generation

If no PIN is supplied, generating a new key might be protected by an operator card that has to be inserted in the HSM. In this case, coordinate testing and final master key generation with your HSM team.

For testing the key generation, run the following example command:

sudo hsmdaemon genkey test

Id: 9bac3719-2b8d-11e9-aeab-0242b5ece4c3, label: test

-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAl1BO4vsI+xDk+x0nccl7
HQhMR/hwfa0+N8fyYNI8yzTTmYDqz9aaF20qG48+mjC0AUEt2kfKo94xM3UeEw4c
st4j1dpRJtmAJThcuN8OH3sa+3MeXWgGuWxjB1lxEEOqax2A6XzllDlbDsogwkOL
hSkUU9AaMRBtF8fASJGtJDP+iXwdb7OsFg78PS1wBAISYSUwk06xY7LwWIxge+hY
4oU+5x4itusdO6rz6kbcJtmUyDUb8DhKnN6OdkhnifUZLBG9HQyTa5OM+BAabbFZ
mTM2gZlUnGKXN7c4kaBPFt1IfjjVYu7pvj3B2uxUf4GywuSuWGWnAy89FqeXteRV
jwIDAQAB
-----END PUBLIC KEY-----

Test Showing Keys

To show an existing key, use the hsmdaemon showkey command with the key’s id as in the following example.

sudo hsmdaemon showkey 9bac3719-2b8d-11e9-aeab-0242b5ece4c3

Test Data Encryption

For testing data encryption, run the following example commands:

The first argument is the Id: value from running the genkey command above. The second is the base64-encoded data to be encrypted.

sudo hsmdaemon encrypt 9bac3719-2b8d-11e9-aeab-0242b5ece4c3 Zm9vYmFy

If successful, you should see output similar to the example below:

{"level":"debug","ts":"2019-03-20T12:43:40.540+0100","caller":"hsmdaemon/keymanager.go:27","msg":"initialize pkcs11 module","module":"/usr/lib/softhsm/libsofthsm2.so"}
{"level":"debug","ts":"2019-03-20T12:43:40.545+0100","caller":"hsmdaemon/keymanager.go:205","msg":"openHSMSession","slotID":858597139}
{"level":"info","ts":"2019-03-20T12:43:40.549+0100","caller":"hsmdaemon/keymanager.go:621","msg":"Fetching private key","keyID":"9bac3719-2b8d-11e9-aeab-0242b5ece4c3"}
{"level":"debug","ts":"2019-03-20T12:43:40.549+0100","caller":"hsmdaemon/keymanager.go:641","msg":"Got uuid","string":"13d34146-4b02-11e9-adbd-0023ae27c404"}
WcezVb2N6bF8wlDooKZcmFn3tZgoIpoFGx6wQetx9sp1nK7JW2Y4OKt7P+0VKKlFO7yXaffVDD2Q6jZZCQukQVRV1zJrwbI9xU3YlOAwJFPP+WM/dZ1vdUwi7L05wq8UpL13LJWlMkvd1eIqKJS7apMnFk2hbnxXP6UKZmI++1tXvqbAc6fwhcB5J+JG6lmS4RwnD+eJC3dq5t00zzdI6vuIM/y3UT7ESklmHl5bKl+N+d6yk6qLxnFnIJweL+M3Tf13+XPNAh5JxZpheJPvN3oL28uX76aizy4BCLnRgQ/ryUQeDF+a4zNF22sMwBh4Pt46KrYGNDZAnQpVzmkrZQ==

Test Data Decryption

For testing data decryption, run the following example commands:

sudo grep "generated keypair" /var/log/hsm.log

You should see output similar to the example below:

{"level":"debug","ts":"2021-06-19T03:10:01.562+0200","msg":"generated keypair","tokenID":"1262668f-d09b-11eb-b283-960000c05f34"}
{"level":"debug","ts":"2021-06-19T03:10:03.043+0200","msg":"generated keypair","tokenID":"1374447f-d09b-11eb-83c8-960000c05f34"}
{"level":"debug","ts":"2021-06-19T03:10:03.710+0200","msg":"generated keypair","tokenID":"13cd3f95-d09b-11eb-83c8-960000c05f34"}
key_id=$(sudo grep "generated keypair" /var/log/hsm.log | head -1 | jq .tokenID -r)
hello="Hello, world!"
echo "$hello" | base64

SGVsbG8sIHdvcmxkIQo=
test_enc=$(sudo ./hsmdaemon encrypt $key_id $(echo "$hello" | base64) | tee /dev/stderr)
ep6Y1aAVAYpAesZ1+sQzzUepjO82o34kjmm63Drmz+6KED4oIBARQkXeW/OoxgUg6kQhQK1thA/3Ww33aaRxIESzVQF598qjXhhEXQ/OGL6BC+3tPclC7ujUZaA7CG1NDkMneLFDd2+Tbax4OM+/w0zhfTMPgT0I1NrH/03owVglbWHjgLZmN/vxpPZKm/lyAV9tI2HW36UjVLEMD2qtPFXqjLU4YjZOVnMdETxQNSCWIVauFw0+VQQ/RiAqiXzRXEgO6YKxOBk0n9IMT6XEH4MkMQTgb9pB12jrNSa9aMHbCvCneEmhd0CHBxPX499EkxxwtoEnXe6PATXsOg3VRA==
sudo -u www-data ./occ  encryption:hsmdaemon:decrypt --keyId $key_id "$test_enc"

decrypted string (base64 encoded): 'SGVsbG8sIHdvcmxkIQo='
sudo tail -5 /var/log/hsm.log
{"level":"debug","ts":"2021-06-20T23:46:11.958+0200","msg":"openHSMSession","slotID":757826573}
{"level":"debug","ts":"2021-06-20T23:46:11.960+0200","msg":"created new session"}
{"level":"debug","ts":"2021-06-20T23:46:11.960+0200","msg":"Got uuid","string":"1262668f-d09b-11eb-b283-960000c05f34"}
{"level":"debug","ts":"2021-06-20T23:46:11.962+0200","msg":"found object","id":"\u0012bf\ufffdЛ\u0011벃\ufffd\u0000\u0000\ufffd_4"}
{"level":"debug","ts":"2021-06-20T23:46:11.963+0200","msg":"Decrypted"}

Results

  • The base64 encoded string matches before encryption and after decryption.

  • The key-id seen in the log with "generated keypair" causes no errors during encryption.

  • The key-id re-appears in the log during decryption ("Got uuid")

Configure Other Options (optional)

For more options see the self-documented default config file hsmdaemon.toml.

During ownCloud config, you might want to run the hsmdaemon service in the foreground to see what is going on. You can do so using the following command (which also shows example console output, formatted for readability):

sudo hsmdaemon
{
    "level": "info",
    "ts": "2019-02-14T09:32:59.081+0100",
    "caller": "hsmdaemon/hsmdaemon.go:146",
    "msg": "Server listening",
    "host": "localhost",
    "port": 8513,
    "version": "0.0.7",
    "build": "2019-02-08T10:47:55+00:00"
}

Configure ownCloud

If anyone accesses ownCloud while encryption is enabled, it will automatically generate the keys. To prevent this shut down the web server until encryption is appropriately configured.

Configuring ownCloud to work with the hsmdaemon requires the following steps:

Generate a Secret for the hsmdaemon REST API

Generate a shared secret to use for the hsmdaemon.

cat /proc/sys/kernel/random/uuid
7a7d1826-b514-4d9f-afc7-a7485084e8de

Use this generated secret for hsmdaemon in /etc/hsmdaemon/hsmdaemon.toml

[jwt]
secret = "7a7d1826-b514-4d9f-afc7-a7485084e8de"

Set the generated secret for ownCloud:

sudo -u www-data ./occ config:app:set encryption hsm.jwt.secret --value '7a7d1826-b514-4d9f-afc7-a7485084e8de'

If the command succeeds, you should see the following console output:

Config value hsm.jwt.secret for app encryption set to 7a7d1826-b514-4d9f-afc7-a7485084e8de

Configure HSM-based Encryption

Enable the HSM mode and enable encryption by running the commands in the following example:

sudo -u www-data ./occ app:enable encryption
sudo -u www-data ./occ config:app:set encryption hsm.url --value 'http://localhost:8513'
sudo -u www-data ./occ encryption:enable

If the commands are successful, you should see the following console output:

encryption enabled

Config value hsm.url for app encryption set to http://localhost:8513

Master key successfully enabled.

Encryption enabled
Default module: OC_DEFAULT_MODULE

Initialize and Check Generated Keys

Now start your web server and log in with any user to initialize the keys, have a look at the output of the hsmdaemon to see key generation and decryption requests. Check that the private key /path/to/data/files_encryption/OC_DEFAULT_MODULE/ is less than 1000 bytes. If it is not, then something is not configured correctly. You have to wipe all keys and reset the database flags for encryption to get a clean start for the ownCloud setup.