Systemd journal ingestion

Overview

journal-ingest collects JSON Lines from the systemd journal into per-unit streams. It requires Linux, journalctl, and permission to read the selected journals.

Example configuration

pond_path: logs/journal                  # Required: example pond destination.
collect_kernel: true                    # Default: also collect kernel messages.
journalctl_command: journalctl          # Default: command resolved on the host.
timestamp_field: __REALTIME_TIMESTAMP   # Default: journald microsecond timestamp.
extra_args: []                          # Default: no extra journalctl arguments.

For example, extra_args: [--merge] includes journals from other namespaces available to the host command.

Use and behavior

Install with pond mknod journal-ingest /etc/journal --config-path journal.yaml, then run pond run /etc/journal -- push. status reports the persisted cursor and per-unit counts; pull is a no-op.

The cursor is stored beneath the output path as .journal-cursor. Removing it changes incremental behavior. Default timestamps are journald microseconds. Query collected files using jsonlogs, explicitly casting timestamp and numeric fields as needed.

Full configuration

Settings

FieldAccepted typeRequiredDefaultDescription
collect_kernelbooleannotrueWhether to collect kernel ring buffer as a separate stream (default: true)
extra_argsarray of stringno[]Extra command-line arguments appended to every journalctl invocation. Useful for e.g. ["--merge"] to pull both the system journal and the per-user journal of the invoking user (which is what surfaces user-scope units like pond@water-prod.service to the ingester). Default: empty.
journalctl_commandstringno"journalctl"Command to invoke journalctl (default: "journalctl") Can be customized for non-standard installs
pond_pathstringyesnot specifiedDestination path within the pond (relative to pond root) Example: "logs/watershop"
timestamp_fieldstringno"__REALTIME_TIMESTAMP"JSON field name containing the timestamp (default: "__REALTIME_TIMESTAMP") The value is interpreted as microseconds since Unix epoch.