Journald

Last edited

Pithy

journald sits in an event loop, when a message arrives it parses and enriches it based on its source before writing to the journal.

Blog on journald

Journald

Unlike syslog, which typically saves log messages to plain text files, the systemd journal stores messages in a binary format. All message attributes are indexed automatically, which makes the log easier and faster to search.

The journal collects and indexes messages from several sources:

  • /dev/log syslog socket
  • /dev/kmsg kernel messages (replaces klogd)
  • /run/systemd/journal/stdout stdout
  • /run/systemd/journal/socket native journal API
  • kernel audit subsystem

To forward these logs to another system use one of the following:

  • systemd-journal-remote
  • 3rd party alloy/fluent bit
  • old school syslog forwarding

Journal config

Default/base config is stored /etc/systemd/journald.conf, custom configs should go in /etc/systemd/journald.conf.d/*.conf. The base file contains commented out version of every possible option along with their default values.

A few settings

[Journal]
Storage=auto         # writes to /var/log/journal for persistance ONLY if it exists
Storage=persistent   # EXPLICITLY ensure logs survive reboots

SystemMaxUse=2G      # Cap total journal size
MaxRentionSec=3month # Time based pruning regardless of size
MaxFileSec=1week     # How often the active journal file is rotated
ForwardToSylog=no    # If running alongside syslog forwarding remotely

Journalctl

For Linux distributions running systemd, the quickest and easiest way to view logs is to use the journalctl.

Commands

journalctl -u ssh # view specific unit
journalctl -f  # live
journalctl --disk-usage # may need to run with sudo for full
journalctl -b 0 -u ssh # ssh logs in current boot
journalctl --since=yesterday --until=now
journalctl -n 100 /usr/sbin/sshd # last 100 from specific binary

journalctl | vim -             # pipe into vim
journalctl _EXE=/usr/bin/sudo  # specific binay
journalctl --list-boots        # to see all the boot logs

journalctl --priority err      # errors and above only
# priority levels:
# 0=emerg 1=alert 2=crit 3=err 4=warning 5=notice 6=info 7=debug