arcline-project/os-build
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
1618897bc2ba3307e855b483858928a7d7727137
os-build/docs/observability.md
Blake Ridgway a27d3fb313 docs: document disk images, installer, vendor packaging, and secure boot 
- docs/secureboot.md: MOK workflow (generate, build, enroll).
- docs/building.md: image + install quickstart, new outputs, ARCLINE_SIGN.
- docs/observability.md: vendor .debs are now the primary packaging path.
- docs/editions.md: cloud ships a qcow2; qemu test command.
- docs/architecture.md: full pipeline table incl. deploy/install; the
  four follow-up items are now implemented; new "on the horizon" list.
- README: updated feature list, quickstart, and status.
2026-08-21 13:33:17 -05:00

2.7 KiB
Raw Blame History

Observability

The server edition ships a pre-configured, local-only observability stack — the same setup Arcline IT runs in production. It never talks to any external service.

 node_exporter ──► Prometheus ──► Grafana (localhost:3000)
                     │  ▲
   journald ─────► promtail ──► Loki (localhost:3100)
Component Port Role Config source
node_exporter 9100 host metrics Debian package (enabled by default)
Prometheus 9090 metric storage + alerting overlays/server/etc/prometheus/
Grafana 3000 dashboards (auto-provisioned) overlays/server/etc/grafana/
Loki 3100 log storage (14d retention) overlays/server/etc/loki/
promtail 9080 ships journald + /var/log → Loki overlays/server/etc/promtail/

What is pre-configured

  • Prometheus scrapes itself + node_exporter and loads alerting rules (arcline.rules.yml): NodeDown, high load, disk > 85% / critical > 95%, service crash-looping.
  • Grafana auto-provisions the Prometheus/Loki datasources and a starting Node Overview dashboard (node-overview.json) on first start.
  • Loki is single-node filesystem storage with a 14-day retention, matching the journald retention.
  • promtail ships the systemd journal and /var/log to the local Loki.

Packaging note

Prometheus + node_exporter + alertmanager are in Debian main and install with the edition packages. Grafana, Loki, and Promtail are not — but they're now packaged as .debs by toolchain/build-vendor.sh:

make vendor    # → build/debs/grafana_<v>_amd64.deb, arcline-loki_<v>.deb, arcline-promtail_<v>.deb
make iso-server

The debs land in build/debs/ alongside the toolchain, so the next rootfs build picks them up automatically and the configure hook enables grafana-server / loki / promtail when their binaries are present.

  • grafana — the official OSS .deb from dl.grafana.com, vendored as-is.
  • arcline-loki / arcline-promtail — static binaries from the Loki GitHub release, wrapped in minimal debs that install the configs from overlays/server/etc/ and systemd units from toolchain/vendor/.

If the downloads fail (no network / version bumped), the build skips them with a warning — the old ARCLINE_EXTRA_REPOS=1 path (apt.grafana.com) remains as a fallback.

Access

  • Grafana UI: http://<host>:3000 (default admin credentials are set at first boot by the installer — never left at Grafana's defaults).
  • Prometheus: http://<host>:9090.

Both are not exposed by the default-deny firewall; open ports deliberately or access via ssh tunnel.