Storage pool type: cephfs

CephFS implements a POSIX-compliant filesystem, using a Ceph storage cluster to store its data. As CephFS builds upon Ceph, it shares most of its properties. This includes redundancy, scalability, self-healing, and high availability.

Tip Proxmox VE can manage Ceph setups, which makes configuring a CephFS storage easier. As modern hardware offers a lot of processing power and RAM, running storage services and VMs on same node is possible without a significant performance impact.

To use the CephFS storage plugin, you must replace the stock Debian Ceph client, by adding our Ceph repository. Once added, run apt update, followed by apt dist-upgrade, in order to get the newest packages.

Warning Please ensure that there are no other Ceph repositories configured. Otherwise the installation will fail or there will be mixed package versions on the node, leading to unexpected behavior.

Configuration

This backend supports the common storage properties nodes, disable, content, as well as the following cephfs specific properties:

fs-name

Name of the Ceph FS.

monhost

List of monitor daemon addresses. Optional, only needed if Ceph is not running on the Proxmox VE cluster.

path

The local mount point. Optional, defaults to /mnt/pve/<STORAGE_ID>/.

username

Ceph client ID. Optional, defaults to admin. Use only the ID without the client. prefix.

subdir

CephFS subdirectory to mount. Optional, defaults to /.

fuse

Access CephFS through FUSE, instead of the kernel client. Optional, defaults to 0.

Configuration example for an external Ceph cluster (/etc/pve/storage.cfg)
cephfs: cephfs-external
        monhost 10.1.1.20 10.1.1.21 10.1.1.22
        path /mnt/pve/cephfs-external
        content backup
        username pve-cephfs
        fs-name cephfs
Note Don’t forget to set up the client’s secret key file, if cephx was not disabled.

Authentication

Note If Ceph is installed locally on the Proxmox VE cluster, the following is done automatically when adding the storage.

If you use cephx authentication, which is enabled by default, provide the secret of a dedicated identity with only the capabilities required for this storage. Obtain or create the identity as described in Ceph User Management.
[cephusermgmt]
 The example below uses client.pve-cephfs; do not use client.admin for a storage client.

Export the bare key through SSH to one Proxmox VE node:

# ssh root@<external-ceph-server> \
    'ceph auth get-key client.pve-cephfs' > /root/cephfs.secret

Then pass that file and the identity without its client. prefix to pvesm:

# pvesm add cephfs <name> --monhost "10.1.1.20 10.1.1.21 10.1.1.22" \
    --fs-name cephfs --content backup --username pve-cephfs \
    --keyring /root/cephfs.secret

After pvesm succeeds, remove /root/cephfs.secret because it contains the key in plain text.

When configuring an external CephFS storage in the web interface, paste the secret into the appropriate field. CephFS uses the bare key, not the RBD keyring format with a [client.<USER>] section. Proxmox VE stores the secret at /etc/pve/priv/ceph/<STORAGE_ID>.secret.

If the external cluster rotates this identity, replace the stored secret before remounting the storage. The local cephx migration helper never rotates keys of external clusters.

Storage Features

The cephfs backend is a POSIX-compliant filesystem, on top of a Ceph cluster.

Table 1. Storage features for backend cephfs
Content types Image formats Shared Snapshots Clones

vztmpl iso backup snippets

none

yes

yes[1]

no

[1] While no known bugs exist, snapshots are not yet guaranteed to be stable, as they lack sufficient testing.

See Also