400 Configure ZFS

Use case 

Use the role vbotka.freebsd.zfs to configure ZFS pools and datasets.

Tree 

shell> tree .
.
├── ansible.cfg
├── host_vars
│   └── iocage_06
│       ├── loader.yml
│       └── zfs.yml
├── iocage.ini
├── pb-loader.yml
└── pb-zfs.yml

Synopsis 

On a managed host:
- Create ZFS pools:
  - zroot
- Create and mount ZFS datasets:
  - zroot/export mounted on /export
  - zroot/ports mounted on /usr/ports
  - zroot/src mounted on /usr/src
  - zroot/obj mounted on /usr/obj
  - zroot/poudriere mounted on /usr/local/poudriere

Requirements 

Role vbotka.freebsd.zfs
Role vbotka.freebsd.postinstall

Notes 

The role vbotka.freebsd.postinstall is used to configure /boot/loader.conf.

Note

vbotka.freebsd.zfs is the role zfs in the collection vbotka.freebsd.

vbotka.freebsd_zfs is the role freebsd_zfs in the namespace vbotka.

Known issues 

zpool state=present is not idempotent #10771

The module community.general.zpool cannot create a correct diff. For example:

(Pdb) p vdev_layout_diff
{'before': {'vdevs': [{'type': 'stripe', 'disks': ['/dev/ada2']}, {'type': 'stripe', 'disks': ['/dev/ada3']}]},
 'after': {'vdevs': [{'type': 'stripe', 'disks': ['/dev/ada2', '/dev/ada3']}]}}

This makes the module non-idempotent. It crashes when run repeatedly. For example:

failed: [srv.example.org] (item=iocage) =>
    ansible_loop_var: item
    changed: false
    cmd: /sbin/zpool add iocage /dev/ada2 /dev/ada3
    item:
        key: iocage
        value:
            vdevs:
            -   disks:
                - /dev/ada2
                - /dev/ada3
    msg: |-
        invalid vdev specification
        use '-f' to override the following errors:
        /dev/ada2 is part of active pool 'iocage'
        /dev/ada3 is part of active pool 'iocage'
    rc: 1
    ...

Setting force: true doesn’t help. At the moment, the only workaround is to skip the module if the pool already exists. You will see a warning. For example:

TASK [vbotka.freebsd_zfs : Pools: WARNING | community.general.zpool skipped.] ****
ok: [srv.example.org] =>
    msg: |-
        # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        # WARNING:
        #
        # The module community.general.zpool is not idempotent. It crashes
        # when running repeatedly. Because of the poor quality, the module
        # community.general.zpool will be skipped for pools:
        # ['iocage']
        #
        # Configure the skipped pools manually, if necessary.
        # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

ansible.cfg 

[defaults]
callback_result_format = yaml
display_skipped_hosts = False
gathering = explicit
interpreter_python = auto_silent
log_path = /var/log/ansible.log
deprecation_warnings = False

[connection]
pipelining = true

Inventory iocage.ini 

iocage_06

[iocage]
iocage_06

[iocage:vars]
ansible_user=admin
ansible_become=true
ansible_python_interpreter=auto_silent

host_vars 

host_vars/iocage_06/loader.yml

fp_loader_conf_template: ''
fp_loader_conf:
  - name: vfs.zfs.prefetch.disable
    value: "0"

host_vars/iocage_06/zfs.yml

fzfs_enable: true

fzfs_pools:
  zroot:
    vdevs:
      - disks: [/dev/nda0p5]

fzfs_manage:
  - name: zroot/export
    state: present
    extra_zfs_properties:
      compression: 'lz4'
      mountpoint: /export
  - name: zroot/ports
    state: present
    extra_zfs_properties:
      compression: 'lz4'
      mountpoint: /usr/ports
  - name: zroot/src
    state: present
    extra_zfs_properties:
      compression: 'lz4'
      mountpoint: /usr/src
  - name: zroot/obj
    state: present
    extra_zfs_properties:
      compression: 'lz4'
      mountpoint: /usr/obj
  - name: zroot/poudriere
    state: present
    extra_zfs_properties:
      compression: 'lz4'
      mountpoint: /usr/local/poudriere

Playbook pb-loader.yml 

- name: Configure loader.conf
  hosts: iocage
  gather_facts: true

  tasks:

    - name: Configure loader.conf
      ansible.builtin.import_role:
        name: vbotka.freebsd.postinstall
        tasks_from: loader

Playbook output - loader.conf 

(env) > ansible-playbook -i iocage.ini pb-loader.yml

PLAY [Configure loader.conf] ***************************************************

TASK [Gathering Facts] *********************************************************
ok: [iocage_06]

TASK [vbotka.freebsd.postinstall : Loader: Backup orig /boot/loader.conf] ******
ok: [iocage_06]

TASK [vbotka.freebsd.postinstall : Loader: Configure by sysctl /boot/loader.conf] ***
ok: [iocage_06] => (item={'name': 'vfs.zfs.prefetch.disable', 'value': '0'})

TASK [vbotka.freebsd.postinstall : Loader: Get sysctls settable via loader.] ***
ok: [iocage_06]

PLAY RECAP *********************************************************************
iocage_06                  : ok=4    changed=0    unreachable=0    failed=0    skipped=5    rescued=0    ignored=0   

Note

Reboot if you see the message:

[MESSAGE] Reboot to activate configuration in /boot/loader.conf

Playbook pb-zfs.yml 

- name: Configure ZFS.
  hosts: iocage
  gather_facts: true

  roles:
    - vbotka.freebsd.zfs

Playbook output - Display variables 

(env) > ansible-playbook -i iocage.ini -t fzfs_debug -e fzfs_debug=true pb-zfs.yml

PLAY [Configure ZFS.] **********************************************************

TASK [Gathering Facts] *********************************************************
ok: [iocage_06]

TASK [vbotka.freebsd.zfs : Debug fzfs_debug=true] ******************************
ok: [iocage_06] => 
    msg: |-
        fzfs_role_version: 2.9.0
        ansible_role_name: vbotka.freebsd.zfs
        ansible_facts.architecture: amd64
        ansible_facts.os_family: FreeBSD
        ansible_facts.distribution: FreeBSD
        ansible_facts.distribution_major_version: 15
        ansible_facts.distribution_version: 15.1
        ansible_facts.distribution_release: 15.1-RELEASE
        ansible_facts.python_version: 3.12.13

        fzfs_enable: True
        fzfs_debug2: False
        fzfs_pools_sanity: True
        fzfs_assert_quiet: True
        fzfs_facts_ds: False
        fzfs_facts_recurse: True
        fzfs_mount_all: True
        fzfs_sysctl_tuneables_warning: True

        fzfs_sysctl_conf:
          []

        fzfs_rcconf:
          []

        fzfs_pools:
          zroot:
            vdevs:
            - disks: [/dev/nda0p5]

        fzfs_manage:
          - extra_zfs_properties: {compression: lz4, mountpoint: /export}
            name: zroot/export
            state: present
          - extra_zfs_properties: {compression: lz4, mountpoint: /usr/ports}
            name: zroot/ports
            state: present
          - extra_zfs_properties: {compression: lz4, mountpoint: /usr/src}
            name: zroot/src
            state: present
          - extra_zfs_properties: {compression: lz4, mountpoint: /usr/obj}
            name: zroot/obj
            state: present
          - extra_zfs_properties: {compression: lz4, mountpoint: /usr/local/poudriere}
            name: zroot/poudriere
            state: present

        fzfs_mountpoints:
          []

PLAY RECAP *********************************************************************
iocage_06                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

Playbook output - Configure ZFS 

(env) > ansible-playbook -i iocage.ini pb-zfs.yml

PLAY [Configure ZFS.] **********************************************************

TASK [Gathering Facts] *********************************************************
ok: [iocage_06]

TASK [Sysctl: Include vbotka.freebsd.postinstall sysctl] ***********************
included: vbotka.freebsd.postinstall for iocage_06

TASK [vbotka.freebsd.postinstall : Sysctl: Get sysctls settable via loader.] ***
ok: [iocage_06]

TASK [vbotka.freebsd.zfs : Rcconf: Enable and Start zfs] ***********************
ok: [iocage_06]

TASK [vbotka.freebsd.zfs : Pools: Get zpool status.] ***************************
ok: [iocage_06]

TASK [vbotka.freebsd.zfs : Pools: Sanity: No fzfs_pools disks already in other pools.] ***
ok: [iocage_06]

TASK [vbotka.freebsd.zfs : Pools: Sanity: No fzfs_pools disks already mounted.] ***
ok: [iocage_06]

TASK [vbotka.freebsd.zfs : Pools: Sanity: fzfs_pools disks are unique.] ********
ok: [iocage_06]

TASK [vbotka.freebsd.zfs : Pools: WARNING | community.general.zpool skipped.] ***
ok: [iocage_06] => 
    msg: |-
        # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        # WARNING:
        #
        # The module community.general.zpool is not idempotent. It crashes
        # when running repeatedly. Because of the poor quality, the module
        # community.general.zpool will be skipped for pools:
        # ['zroot']
        #
        # Configure the skipped pools manually, if necessary.
        # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

TASK [vbotka.freebsd.zfs : Facts: Run zpool_facts] *****************************
ok: [iocage_06]

TASK [vbotka.freebsd.zfs : Manage: Sanity all pools available.] ****************
ok: [iocage_06]

TASK [vbotka.freebsd.zfs : Manage: Manage zfs] *********************************
ok: [iocage_06] => (item=zroot/export)
ok: [iocage_06] => (item=zroot/ports)
ok: [iocage_06] => (item=zroot/src)
ok: [iocage_06] => (item=zroot/obj)
ok: [iocage_06] => (item=zroot/poudriere)

PLAY RECAP *********************************************************************
iocage_06                  : ok=12   changed=0    unreachable=0    failed=0    skipped=15   rescued=0    ignored=0   

Playbook output - List pools 

(env) > ansible-playbook -i iocage.ini -t fzfs_facts_pools -e fzfs_debug=true pb-zfs.yml

PLAY [Configure ZFS.] **********************************************************

TASK [Gathering Facts] *********************************************************
ok: [iocage_06]

TASK [vbotka.freebsd.zfs : Facts: Run zpool_facts] *****************************
ok: [iocage_06]

TASK [vbotka.freebsd.zfs : Facts: Debug list pools fzfs_debug=true] ************
ok: [iocage_06] => 
    msg: |-
        fzfs_pools_required: ['zroot']
        fzfs_pools_missing: []
        fzfs_pools_present: ['zroot']
        name: zroot size: 368G free: 352G

PLAY RECAP *********************************************************************
iocage_06                  : ok=3    changed=0    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0   

Playbook output - List datasets 

(env) > ansible-playbook -i iocage.ini -t fzfs_facts_ds -e fzfs_facts_ds=true -e fzfs_debug=true pb-zfs.yml

PLAY [Configure ZFS.] **********************************************************

TASK [Gathering Facts] *********************************************************
ok: [iocage_06]

TASK [vbotka.freebsd.zfs : Facts: Run zpool_facts] *****************************
ok: [iocage_06]

TASK [vbotka.freebsd.zfs : Facts: Run zfs_facts] *******************************
ok: [iocage_06] => (item=zroot)

TASK [vbotka.freebsd.zfs : Facts: Get dictionary of datasets] ******************
ok: [iocage_06]

TASK [vbotka.freebsd.zfs : Facts: Debug list of datasets fzfs_debug=true] ******
ok: [iocage_06] => 
    fzfs_datasets | to_nice_yaml: |-
        zroot:
            zroot: /zroot
            zroot/export: /export
            zroot/iocage: /zroot/iocage
            zroot/iocage/download: /zroot/iocage/download
            zroot/iocage/download/15.0-RELEASE: /zroot/iocage/download/15.0-RELEASE
            zroot/iocage/download/15.1-RELEASE: /zroot/iocage/download/15.1-RELEASE
            zroot/iocage/images: /zroot/iocage/images
            zroot/iocage/jails: /zroot/iocage/jails
            zroot/iocage/jails/log-server-01: /zroot/iocage/jails/log-server-01
            zroot/iocage/jails/log-server-01/root: /zroot/iocage/jails/log-server-01/root
            zroot/iocage/jails/pkg-repo: /zroot/iocage/jails/pkg-repo
            zroot/iocage/jails/pkg-repo/root: /zroot/iocage/jails/pkg-repo/root
            zroot/iocage/jails/repos: /zroot/iocage/jails/repos
            zroot/iocage/jails/repos-devel: /zroot/iocage/jails/repos-devel
            zroot/iocage/jails/repos-devel/root: /zroot/iocage/jails/repos-devel/root
            zroot/iocage/jails/repos/root: /zroot/iocage/jails/repos/root
            zroot/iocage/log: /zroot/iocage/log
            zroot/iocage/releases: /zroot/iocage/releases
            zroot/iocage/releases/15.0-RELEASE: /zroot/iocage/releases/15.0-RELEASE
            zroot/iocage/releases/15.0-RELEASE/root: /zroot/iocage/releases/15.0-RELEASE/root
            zroot/iocage/releases/15.1-RELEASE: /zroot/iocage/releases/15.1-RELEASE
            zroot/iocage/releases/15.1-RELEASE/root: /zroot/iocage/releases/15.1-RELEASE/root
            zroot/iocage/releases/15.1-RELEASE/root@ansible-apache: null
            zroot/iocage/releases/15.1-RELEASE/root@ansible-client: null
            zroot/iocage/releases/15.1-RELEASE/root@ansible-init: null
            zroot/iocage/releases/15.1-RELEASE/root@ansible-nginx: null
            zroot/iocage/releases/15.1-RELEASE/root@ansible-pkg-repo: null
            zroot/iocage/releases/15.1-RELEASE/root@ansible-repos: null
            zroot/iocage/templates: /zroot/iocage/templates
            zroot/iocage/templates/ansible-apache: /zroot/iocage/templates/ansible-apache
            zroot/iocage/templates/ansible-apache/root: /zroot/iocage/templates/ansible-apache/root
            zroot/iocage/templates/ansible-client: /zroot/iocage/templates/ansible-client
            zroot/iocage/templates/ansible-client/root: /zroot/iocage/templates/ansible-client/root
            zroot/iocage/templates/ansible-init: /zroot/iocage/templates/ansible-init
            zroot/iocage/templates/ansible-init/root: /zroot/iocage/templates/ansible-init/root
            zroot/iocage/templates/ansible-init/root@log-server-01: null
            zroot/iocage/templates/ansible-init@log-server-01: null
            zroot/iocage/templates/ansible-nginx: /zroot/iocage/templates/ansible-nginx
            zroot/iocage/templates/ansible-nginx/root: /zroot/iocage/templates/ansible-nginx/root
            zroot/iocage/templates/ansible-pkg-repo: /zroot/iocage/templates/ansible-pkg-repo
            zroot/iocage/templates/ansible-pkg-repo/root: /zroot/iocage/templates/ansible-pkg-repo/root
            zroot/iocage/templates/ansible-pkg-repo/root@pkg-repo: null
            zroot/iocage/templates/ansible-pkg-repo@pkg-repo: null
            zroot/iocage/templates/ansible-repos: /zroot/iocage/templates/ansible-repos
            zroot/iocage/templates/ansible-repos/root: /zroot/iocage/templates/ansible-repos/root
            zroot/iocage/templates/ansible-repos/root@repos: null
            zroot/iocage/templates/ansible-repos/root@repos-devel: null
            zroot/iocage/templates/ansible-repos@repos: null
            zroot/iocage/templates/ansible-repos@repos-devel: null
            zroot/obj: /usr/obj
            zroot/ports: /usr/ports
            zroot/poudriere: /usr/local/poudriere
            zroot/repo: /usr/local/repo
            zroot/src: /usr/src

PLAY RECAP *********************************************************************
iocage_06                  : ok=5    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0   