a27d3fb313
- docs/secureboot.md: MOK workflow (generate, build, enroll). - docs/building.md: image + install quickstart, new outputs, ARCLINE_SIGN. - docs/observability.md: vendor .debs are now the primary packaging path. - docs/editions.md: cloud ships a qcow2; qemu test command. - docs/architecture.md: full pipeline table incl. deploy/install; the four follow-up items are now implemented; new "on the horizon" list. - README: updated feature list, quickstart, and status.
64 lines
2.7 KiB
Markdown
64 lines
2.7 KiB
Markdown
# Observability
|
|
|
|
The server edition ships a pre-configured, **local-only** observability stack —
|
|
the same setup Arcline IT runs in production. It never talks to any external
|
|
service.
|
|
|
|
```
|
|
node_exporter ──► Prometheus ──► Grafana (localhost:3000)
|
|
│ ▲
|
|
journald ─────► promtail ──► Loki (localhost:3100)
|
|
```
|
|
|
|
| Component | Port | Role | Config source |
|
|
|-----------|------|------|---------------|
|
|
| node_exporter | `9100` | host metrics | Debian package (enabled by default) |
|
|
| Prometheus | `9090` | metric storage + alerting | `overlays/server/etc/prometheus/` |
|
|
| Grafana | `3000` | dashboards (auto-provisioned) | `overlays/server/etc/grafana/` |
|
|
| Loki | `3100` | log storage (14d retention) | `overlays/server/etc/loki/` |
|
|
| promtail | `9080` | ships journald + /var/log → Loki | `overlays/server/etc/promtail/` |
|
|
|
|
## What is pre-configured
|
|
|
|
- **Prometheus** scrapes itself + `node_exporter` and loads alerting rules
|
|
(`arcline.rules.yml`): NodeDown, high load, disk > 85% / critical > 95%,
|
|
service crash-looping.
|
|
- **Grafana** auto-provisions the Prometheus/Loki datasources and a starting
|
|
Node Overview dashboard (`node-overview.json`) on first start.
|
|
- **Loki** is single-node filesystem storage with a 14-day retention, matching
|
|
the journald retention.
|
|
- **promtail** ships the systemd journal and `/var/log` to the *local* Loki.
|
|
|
|
## Packaging note
|
|
|
|
Prometheus + node_exporter + alertmanager are in Debian main and install with
|
|
the edition packages. **Grafana**, **Loki**, and **Promtail** are not — but
|
|
they're now packaged as `.debs` by `toolchain/build-vendor.sh`:
|
|
|
|
```bash
|
|
make vendor # → build/debs/grafana_<v>_amd64.deb, arcline-loki_<v>.deb, arcline-promtail_<v>.deb
|
|
make iso-server
|
|
```
|
|
|
|
The debs land in `build/debs/` alongside the toolchain, so the next rootfs
|
|
build picks them up automatically and the configure hook enables
|
|
`grafana-server` / `loki` / `promtail` when their binaries are present.
|
|
|
|
- `grafana` — the official OSS `.deb` from `dl.grafana.com`, vendored as-is.
|
|
- `arcline-loki` / `arcline-promtail` — static binaries from the Loki GitHub
|
|
release, wrapped in minimal debs that install the configs from
|
|
`overlays/server/etc/` and systemd units from `toolchain/vendor/`.
|
|
|
|
If the downloads fail (no network / version bumped), the build skips them with
|
|
a warning — the old `ARCLINE_EXTRA_REPOS=1` path (apt.grafana.com) remains as
|
|
a fallback.
|
|
|
|
## Access
|
|
|
|
- Grafana UI: `http://<host>:3000` (default admin credentials are set at
|
|
first boot by the installer — never left at Grafana's defaults).
|
|
- Prometheus: `http://<host>:9090`.
|
|
|
|
Both are **not** exposed by the default-deny firewall; open ports deliberately
|
|
or access via ssh tunnel.
|