500 syslog-ng server and syslog-ng clients

Use case 

Configure and run a log server. Configure log clients and test them. Use syslog-ng. Use the jails created in the example 207 Create DHCP jails with auto UUID, iocage_tags, alias and class. The project keys are jail’s aliases.

project:
  logserv_1:
    class: [logserv]
    vmm: iocage_01
  http_1:
    class: [http, logclient]
    vmm: iocage_02
  db_1:
    class: [db, logclient]
    vmm: iocage_02
  http_2:
    class: [http, logclient]
    vmm: iocage_04
  db_2:
    class: [db, logclient]
    vmm: iocage_04

Destroy all jails

(env) > ansible-playbook vbotka.freebsd.pb_iocage_destroy_all_jails.yml \
                         -i iocage.ini \
                         --flush-cache

Create ansible_client templates. Run the play in 202 Create iocage templates. Clone DHCP jails.
```
(env) > ansible-playbook pb-iocage-template.yml -i iocage.ini
```
Create the project. Run the play in 207 Create DHCP jails with auto UUID, iocage_tags, alias and class
```
(env) > ansible-playbook pb-iocage-project-create.yml -i iocage.ini -i hosts
```

Tree 

shell > tree .
.
├── ansible.cfg
├── group_vars
│   ├── all
│   │   └── common.yml
│   ├── logclient
│   │   └── syslog-ng.yml
│   └── logserv
│       └── syslog-ng.yml
├── hosts
│   ├── 01_iocage.yml
│   ├── 02_iocage.yml
│   ├── 04_iocage.yml
│   └── 99_constructed.yml
├── host_vars
│   ├── iocage_01
│   │   └── iocage.yml
│   ├── iocage_02
│   │   └── iocage.yml
│   └── iocage_04
│       └── iocage.yml
├── iocage.ini
├── pb-all-groups.yml
├── pb-logclient.yml
├── pb-logserv.yml
└── pb-test-logclient.yml

Synopsis 

In the inventory group logserv:
- install sysutils/syslog-ng
- configure syslog-ng Server.
In the inventory group logclient:
- install sysutils/syslog-ng
- configure syslog-ng Client.

Requirements 

inventory plugin vbotka.freebsd.iocage
module vbotka.freebsd.service
role vbotka.freebsd.postinstall
jails created in the project 207 Create DHCP jails with auto UUID, iocage_tags, alias and class

Notes 

Quoting syslog-ng - FreeBSD Wiki:

One of the most typical use of syslog-ng is central log aggregation. … It collects log messages on TCP port 514 and saves them to directories and files based on sender host name and current date.

Note

vbotka.freebsd.postinstall is the role postinstall in the collection vbotka.freebsd.
vbotka.freebsd_postinstall is the role freebsd_postinstall in the namespace vbotka.
Please make sure the versions are the same before you switch between them.

ansible.cfg 

[defaults]
gathering = explicit
callback_result_format = yaml
display_skipped_hosts = false
host_key_checking = false

[connection]
pipelining = true

Inventory iocage.ini 

iocage_01 ansible_host=10.1.0.18
iocage_02 ansible_host=10.1.0.73
iocage_04 ansible_host=10.1.0.29

[iocage]
iocage_01
iocage_02
iocage_04

[iocage:vars]
ansible_user=admin
ansible_become=true
ansible_python_interpreter=auto_silent

hosts 

hosts/01_iocage.yml

plugin: vbotka.freebsd.iocage
host: 10.1.0.18
user: admin
cache: true
cache_plugin: ansible.builtin.jsonfile
cache_connection: /var/tmp/inventory_cache
cache_timeout: 3600
cache_prefix: iocage_01_
get_properties: true
inventory_hostname_tag: alias
hooks_results:
  - /var/db/dhclient-hook.address.epair0b

hosts/02_iocage.yml

plugin: vbotka.freebsd.iocage
host: 10.1.0.73
user: admin
cache: true
cache_plugin: ansible.builtin.jsonfile
cache_connection: /var/tmp/inventory_cache
cache_timeout: 3600
cache_prefix: iocage_02_
get_properties: true
inventory_hostname_tag: alias
hooks_results:
  - /var/db/dhclient-hook.address.epair0b

hosts/04_iocage.yml

plugin: vbotka.freebsd.iocage
host: 10.1.0.29
user: admin
cache: true
cache_plugin: ansible.builtin.jsonfile
cache_connection: /var/tmp/inventory_cache
cache_timeout: 3600
cache_prefix: iocage_04_
get_properties: true
inventory_hostname_tag: alias
hooks_results:
  - /var/db/dhclient-hook.address.epair0b

hosts/99_constructed.yml

plugin: ansible.builtin.constructed
compose:
  ansible_host: (iocage_hooks.0 == '-') | ternary(iocage_ip4, iocage_hooks.0)
  iocage_tags: dict(iocage_properties.notes | regex_findall('(\w+)=([\w\-]+)'))
  iocage_classes: iocage_properties.notes | regex_findall('class=(\w+)') 
groups:
  db: "'db' in iocage_classes"
  http: "'http' in iocage_classes"
  logclient: "'logclient' in iocage_classes"
  logserv: "'logserv' in iocage_classes"
keyed_groups:
  - prefix: state
    key: iocage_state
  - prefix: vmm
    key: iocage_tags.vmm

Playbook pb-all-groups.yml 

- name: Display all groups.
  hosts: all

  tasks:

    - debug:
        msg: |
          ansible_host: {{ ansible_host | d('UNDEFINED') }}
          iocage_properties.host_hostuuid: {{ iocage_properties.host_hostuuid | d('UNDEFINED') }}
          iocage_classes: {{ iocage_classes | d([]) | to_yaml }}
          iocage_tags: {{ iocage_tags | d({}) | to_yaml }}

    - debug:
        msg: |
          {% for group in groups %}
          {{ group }}: {{ groups[group] }}
          {% endfor %}
      run_once: true

Playbook output - Display groups 

Flush the cache if you created the project and haven’t refreshed it yet.

(env) > ansible-playbook pb-all-groups.yml -i hosts --flush-cache

PLAY [Display all groups.] *****************************************************

TASK [debug] *******************************************************************
ok: [logserv_1] => 
    msg: |-
        ansible_host: 10.1.0.237
        iocage_properties.host_hostuuid: 84c518c4
        iocage_classes: [logserv]

        iocage_tags: {alias: logserv_1, class: logserv, vmm: iocage_01}
ok: [http_1] => 
    msg: |-
        ansible_host: 10.1.0.144
        iocage_properties.host_hostuuid: 35dd4161
        iocage_classes: [http, logclient]

        iocage_tags: {alias: http_1, class: 'http,logclient', vmm: iocage_02}
ok: [db_1] => 
    msg: |-
        ansible_host: 10.1.0.143
        iocage_properties.host_hostuuid: 4db6ccdc
        iocage_classes: [db, logclient]

        iocage_tags: {alias: db_1, class: 'db,logclient', vmm: iocage_02}
ok: [http_2] => 
    msg: |-
        ansible_host: 10.1.0.176
        iocage_properties.host_hostuuid: 49aa351b
        iocage_classes: [http, logclient]

        iocage_tags: {alias: http_2, class: 'http,logclient', vmm: iocage_04}
ok: [db_2] => 
    msg: |-
        ansible_host: 10.1.0.219
        iocage_properties.host_hostuuid: 6a671f8f
        iocage_classes: [db, logclient]

        iocage_tags: {alias: db_2, class: 'db,logclient', vmm: iocage_04}

TASK [debug] *******************************************************************
ok: [logserv_1] => 
    msg: |-
        all: ['logserv_1', 'http_1', 'db_1', 'http_2', 'db_2']
        ungrouped: []
        logserv: ['logserv_1']
        state_up: ['logserv_1', 'http_1', 'db_1', 'http_2', 'db_2']
        vmm_iocage_01: ['logserv_1']
        http: ['http_1', 'http_2']
        logclient: ['http_1', 'db_1', 'http_2', 'db_2']
        vmm_iocage_02: ['http_1', 'db_1']
        db: ['db_1', 'db_2']
        vmm_iocage_04: ['http_2', 'db_2']

PLAY RECAP *********************************************************************
db_1                       : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
db_2                       : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
http_1                     : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
http_2                     : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
logserv_1                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

group_vars 

group_vars/all/common.yml

ansible_python_interpreter: auto_silent
log_serv: "{{ hostvars.logserv_1.iocage_properties.host_hostuuid }}"

group_vars/logserv/syslog-ng.yml

fp_syslogng: true
fp_syslogng_enable: true
fp_syslogng_conf_template: syslog-ng.conf.server.j2
fp_syslogng_conf:
  header:
    - '@version:4.8'
    - '@include "scl.conf"'
  source:
    s_remote:
      tcp:
        port:
          514
  destination:
    d_remote:
      file: '"/var/log/remote/${HOST}/${YEAR}_${MONTH}_${DAY}.log" create-dirs(yes)'
  log:
    - source: s_remote
      destination: d_remote

group_vars/logclient/syslog-ng.yml

fp_syslogng: true
fp_syslogng_enable: true
fp_syslogng_conf_template: syslog-ng.conf.client.j2
fp_syslogng_conf:
  header:
    - '@version:4.8'
    - '@include "scl.conf"'
  source:
    s_local:
      system:
      internal:
  destination:
    d_network:
      network: '"{{ log_serv }}" transport("udp")'
  log:
    - source: s_local
      destination: d_network

Playbook pb-logserv.yml 

- name: Configure and start Log Server.
  hosts: logserv
  remote_user: admin
  become: true
    
  tasks:

    - name: Install syslog-ng
      when: install | d(false) | bool
      community.general.pkgng:
        name: sysutils/syslog-ng
        use_globs: false

    - name: Configure and start Log Server.
      ansible.builtin.import_role:
        name: vbotka.freebsd.postinstall
        tasks_from: syslog-ng.yml

Playbook output - Log Server 

Install the package if you’re running this play for the first time.

(env) > ansible-playbook pb-logserv.yml -i hosts -e install=true

PLAY [Configure and start Log Server.] *****************************************

TASK [Install syslog-ng] *******************************************************
changed: [logserv_1]

TASK [vbotka.freebsd.postinstall : Syslog-ng: Sanity fp_syslogng_conf is empty.] ***
ok: [logserv_1]

TASK [vbotka.freebsd.postinstall : Syslog-ng: Configure /usr/local/etc/syslog-ng.conf] ***
changed: [logserv_1]

TASK [vbotka.freebsd.postinstall : Rcconf: Configure syslog_ng_enable in /etc/rc.conf] ***
changed: [logserv_1]

RUNNING HANDLER [vbotka.freebsd.postinstall : Start syslog-ng] *****************
changed: [logserv_1]

RUNNING HANDLER [vbotka.freebsd.postinstall : Reload syslog-ng] ****************
ok: [logserv_1]

PLAY RECAP *********************************************************************
logserv_1                  : ok=6    changed=4    unreachable=0    failed=0    skipped=7    rescued=0    ignored=0   

Test the Log Server 

(env) > ssh admin@4b07a142 sudo service syslog-ng status
syslog_ng is running as pid 63344.

(env) > ssh admin@4b07a142 loggen -i -S -n 1 localhost 514
count=1, rate = 100000.00 msg/sec
average rate = 1.95 msg/sec, count=1, time=0.512063, (average) msg size=256, bandwidth=0.49 kB/sec

(env) > ssh admin@4b07a142 sudo cat /var/log/remote/localhost/2025_08_12.log
 Aug 12 01:41:22 localhost prg00000[1234]: seq: 0000000000, thread: 0000, runid: 1754955682, stamp: 2025-08-12T01:41:22 PADDPADD...
 Aug 12 01:42:42 localhost prg00000[1234]: seq: 0000000000, thread: 0000, runid: 1754955762, stamp: 2025-08-12T01:42:42 PADDPADD...

Note

This test is not created dynamically. The Log Server jail name in this test differs from the dynamically created name in the example. (TBD)

Playbook pb-logclient.yml 

- name: Configure and start Log Client.
  hosts: logclient
  remote_user: admin
  become: true
    
  tasks:

    - name: Debug.
      when: debug | d(false) | bool
      ansible.builtin.debug:
        msg: |
          log_serv: {{ log_serv }}
          fp_syslogd_enable: {{ fp_syslogd_enable }}
          fp_syslogng_enable: {{ fp_syslogng_enable }}

    - name: Install syslog-ng
      when: install | d(false) | bool
      delegate_to: "{{ iocage_tags.vmm }}"
      community.general.pkgng:
        name: sysutils/syslog-ng
        use_globs: false
        jail: "{{ iocage_jid }}"
        cached: true

    - name: Stop syslogd.
      vbotka.freebsd.service:
        script: syslogd
        command: stop

    - name: Disable syslogd.
      vbotka.freebsd.service:
        script: syslogd
        command: disable

    - name: Configure and start syslog-ng.
      ansible.builtin.import_role:
        name: vbotka.freebsd.postinstall
        tasks_from: syslog-ng.yml

Playbook output - Log Client 

Install the package if you’re running this play for the first time.

(env) > ansible-playbook pb-logclient.yml -i hosts -i iocage.ini -e install=true -e debug=true

PLAY [Configure and start Log Client.] *****************************************

TASK [Debug.] ******************************************************************
ok: [http_1] => 
    msg: |-
        log_serv: 84c518c4
        fp_syslogd_enable: False
        fp_syslogng_enable: True
ok: [db_1] => 
    msg: |-
        log_serv: 84c518c4
        fp_syslogd_enable: False
        fp_syslogng_enable: True
ok: [http_2] => 
    msg: |-
        log_serv: 84c518c4
        fp_syslogd_enable: False
        fp_syslogng_enable: True
ok: [db_2] => 
    msg: |-
        log_serv: 84c518c4
        fp_syslogd_enable: False
        fp_syslogng_enable: True

TASK [Install syslog-ng] *******************************************************
changed: [http_2 -> iocage_04(10.1.0.29)]
changed: [db_2 -> iocage_04(10.1.0.29)]
changed: [http_1 -> iocage_02(10.1.0.73)]
changed: [db_1 -> iocage_02(10.1.0.73)]

TASK [Stop syslogd.] ***********************************************************
changed: [http_2]
changed: [db_2]
changed: [db_1]
changed: [http_1]

TASK [Disable syslogd.] ********************************************************
changed: [http_2]
changed: [db_2]
changed: [db_1]
changed: [http_1]

TASK [vbotka.freebsd.postinstall : Syslog-ng: Sanity fp_syslogng_conf is empty.] ***
ok: [http_1]
ok: [db_1]
ok: [http_2]
ok: [db_2]

TASK [vbotka.freebsd.postinstall : Syslog-ng: Configure /usr/local/etc/syslog-ng.conf] ***
changed: [db_2]
changed: [http_2]
changed: [db_1]
changed: [http_1]

TASK [vbotka.freebsd.postinstall : Rcconf: Configure syslog_ng_enable in /etc/rc.conf] ***
changed: [db_2]
changed: [http_2]
changed: [db_1]
changed: [http_1]

RUNNING HANDLER [vbotka.freebsd.postinstall : Start syslog-ng] *****************
changed: [http_2]
changed: [db_2]
changed: [http_1]
changed: [db_1]

RUNNING HANDLER [vbotka.freebsd.postinstall : Reload syslog-ng] ****************
ok: [http_2]
ok: [db_2]
ok: [http_1]
ok: [db_1]

PLAY RECAP *********************************************************************
db_1                       : ok=9    changed=6    unreachable=0    failed=0    skipped=7    rescued=0    ignored=0   
db_2                       : ok=9    changed=6    unreachable=0    failed=0    skipped=7    rescued=0    ignored=0   
http_1                     : ok=9    changed=6    unreachable=0    failed=0    skipped=7    rescued=0    ignored=0   
http_2                     : ok=9    changed=6    unreachable=0    failed=0    skipped=7    rescued=0    ignored=0   

Playbook pb-test-logclient.yml 

- name: Test Log Client.
  hosts: logclient
  remote_user: admin
  become: true
    
  tasks:

    - name: Run loggen
      register: out
      ansible.builtin.command: "loggen -i -S -n 1 {{ log_serv }} 514"

    - name: Debug.
      ansible.builtin.debug:
        var: out.stderr

Playbook output - Test Log Client 

(env) > ansible-playbook pb-test-logclient.yml -i hosts

PLAY [Test Log Client.] ********************************************************

TASK [Run loggen] **************************************************************
changed: [http_2]
changed: [db_2]
changed: [http_1]
changed: [db_1]

TASK [Debug.] ******************************************************************
ok: [http_1] => 
    out.stderr: |-
        count=1, rate = 166666.67 msg/sec
        average rate = 1.85 msg/sec, count=1, time=0.540408, (average) msg size=256, bandwidth=0.46 kB/sec
ok: [http_2] => 
    out.stderr: |-
        count=1, rate = 250000.00 msg/sec
        average rate = 2.00 msg/sec, count=1, time=0.500028, (average) msg size=256, bandwidth=0.50 kB/sec
ok: [db_1] => 
    out.stderr: |-
        count=1, rate = 27027.03 msg/sec
        average rate = 1.83 msg/sec, count=1, time=0.545652, (average) msg size=256, bandwidth=0.46 kB/sec
ok: [db_2] => 
    out.stderr: average rate = 2.00 msg/sec, count=1, time=0.500012, (average) msg size=256,
        bandwidth=0.50 kB/sec

PLAY RECAP *********************************************************************
db_1                       : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
db_2                       : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
http_1                     : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
http_2                     : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

Example of the directory at the Log Server.

(env) > ssh admin@4b07a142 sudo ls -lat /var/log/remote/ | sort
drwx------  2 root  wheel   3 Aug 12 01:30 0f8e3961
drwx------  2 root  wheel   3 Aug 12 01:30 33a222bb
drwx------  2 root  wheel   3 Aug 12 01:30 35167ffa
drwx------  2 root  wheel   3 Aug 12 01:30 ef5d35da
drwx------  2 root  wheel   3 Aug 12 01:41 localhost
drwx------  7 root  wheel   7 Aug 12 01:41 .
drwxr-xr-x  3 root  wheel  17 Aug 12 01:30 ..

Note

This example of the directory at the Log Server is not created dynamically. (TBD)