What we are doing now, and a dated log of what is done. Updated and pushed after every step. For the full history, decisions and recovery guide see PROJECT_LOG.md.
Where things stand (2026-10-09):
m5 is on 8f91158.smartconfig 0.5.0. Its scd runs from
/usr/lib/systemd/system, with sc in /usr/sbin.9d43980). Chunk J (steps 3, 4, 5, 5a and 8b98817) is
done and reviewed, and its 13 findings fixed
(reviews/2026-10-12-m5-followups-chunk-j.md;
your picks for J1, J6 and J7, 2026-10-12, all on the recommended
answer). Chunk K, step 6 is on main (47d35a5, 2026-10-16:
secret files shown hidden; M6 step 2’s sc check --tsv can build on
it). Step 7 is on main too (1d6671a, --show-secrets as root).
Chunk K is done and reviewed
(reviews/2026-10-16-m5-followups-chunk-k.md;
your picks for K1 and K7, 2026-10-16, both on the recommended answer).
Chunk L is built: step 8 (ea50fad, the apt hook) and step 9
(9a40d4e, the lab’s --deb checks the upgrade’s rows). Chunk L
is done and reviewed
(reviews/2026-10-16-m5-followups-chunk-l.md;
L1 high, all six fixed). All nine steps of the round are built and
reviewed. Next: the sign-off runs, each waiting for your OK (see
“Still waiting for you”). The round’s “done” for the M6 session comes
after them. The handoff, if this session stops: ~/smartconfig-work/M5FU-RESUME.md.~/smartconfig-work/m6/smartconfig (your call, 2026-10-09;
the plan, draft 3, is ~/smartconfig-work/M6_PLAN.draft3.md). Its
commits land here only after this round, one at a time.sc why, a local model) is not planned yet. It needs your
decisions first: the runtime, the model and its licence, packaging, and
an exception to the one-binary and dependency rules.Where each milestone stands: MILESTONES.md. A progress dashboard as it was on 28 Sep (M1): visuals/progress.html.
The review of the whole plan (2026-10-09, after M5) checked every
plan, roadmap and overview doc and the M6 draft. The write-up is
~/smartconfig-work/project-review-2026-10-09.md. It found three things:
Still waiting for you:
The M5 follow-up round’s sign-off (MILESTONES, “Then, each with
your OK”): all checks; accept-m2, accept-m3 and accept-m4 as
root; make lab-e2e LAB_E2E_ARGS=--deb in both modes (an hour or two
each under TCG: the host’s freezes cost earlier runs their verdict); the
new package installed on this VM, a reboot, and its read-only check.
Say which to run, and when.
grub-default-syntax is a blocker, but a broken /etc/default/grub
makes update-grub fail and keep the old menu. The danger comes
later, when a kernel is removed.init=) falls back to
/sbin/init; only a bad root= breaks the rescue entry./usr/sbin, purge, the hand install, the package’s name, the
command’s name. m5 went out on the recommended answers; say if one
should turn round. The plan itself is not approved.signoff/lab-e2e-22383a0/bios-stall-5-1.txt). On
2026-10-05 it also went down without a shutdown three times, and once
more on 2026-10-06 at about 11:30 UTC, idle (last -x: crash). It
crashed twice more after 2026-10-09 21:57 UTC, and the first crash
damaged this repo’s .git (log, 2026-10-11). Sleep
or power saving on the host is the first thing to look at. M6’s long
lab runs depend on this. On 2026-10-15 it froze three times, 1, 16
and 9 minutes after boot, twice with the guest idle: look at
VBox.log after a freeze (Guru Meditation, VERR_, paused), the
green turtle (Hyper-V), and the vCPUs (log, 2026-10-16).Done (kept for reference): the reboot of this VM on the package
(2026-10-09; the read-only check passed on its first two boots), and
this VM’s /usr/sbin/policy-rc.d (your answer, 2026-10-09, “Move it
aside”: it is /var/backups/smartconfig/policy-rc.d.image, sha256
7a5c733d…; to put it back, sudo mv it to /usr/sbin/policy-rc.d).
What “Now” said while each milestone was built, kept as written; the log below has the details.
M2 plan approved (2026-09-28, “start M2, all my picks”): every recommendation in M2_PLAN.md section 16 and in NEXT_STEPS.md holds.
Overnight run, starting 04:30 IST (23:00 UTC) on 28 Sep, unattended:
make race (4).SC_HOMEs. No install of the watcher,
no reboot, nothing written to /etc, /boot or the real store.Now: M3 is being built (M3_PLAN.md, approved 2026-10-03,
“great go ahead”: all 7 recommendations of its section 14, which also
passes gate G1, scd may run validators as root). 16 steps in five chunks,
one at a time. Done: step 1 (45df7da). Step 2 (95da13b). Step 3 (b665691). Step 4 (efc2ce3): chunk A is built and reviewed
(reviews/2026-10-03-m3-chunk-a.md,
fixes a2e325f). Step 5 (afa9403). Step 6 (5f6068b). Step 7 (c997865): chunk B is built and reviewed
(reviews/2026-10-03-m3-chunk-b.md,
fixes a946f6f). Chunk C (steps 8-11) is built: four agents built the checkers in
separate worktrees, Claude reviewed and integrated each; reviewed
(reviews/2026-10-03-m3-chunk-c.md,
fixes 41c87bf). Timing-bound tests made robust (5b1d5a5). Step 12 (580e1d1). Step 14 (7ccb3d9).
Step 13 is in (a73dfde, 2e4d4ee) and the acceptance script is
committed, not run (2ff038b). Chunk D is reviewed
(reviews/2026-10-03-m3-chunk-d.md,
fixes 2fb037e, plan change C8). Step 15, the docs, is in (see the log).
M3 is built. Now: the sign-off runs (plan section 11; your call,
2026-10-03: close the M2 soak now and sign off, on the snapshot you
already have). S1, S2 and S3 are done, and S4’s final review is
closed (reviews/2026-10-03-m3-final.md,
fixes 297fbc7, bb31f3e, plan change C9, 8 follow-ups in MILESTONES).
accept-m3 passes on the fixed build. M3 is done: tag m3
(2026-10-03); the fixed build runs as scd.
Now: M4 is being built (M4_PLAN.md, approved
2026-10-04, “approved”: all 7 recommendations of its section 10). 13
steps in five chunks, one at a time. Done: steps 1-3, chunk A built and reviewed
(reviews/2026-10-04-m4-chunk-a.md,
fixes 49a0a5b). Done: steps 4-6, chunk B built and reviewed
(reviews/2026-10-04-m4-chunk-b.md,
fixes 40288d8, plan change C4). Done: steps 7-10, chunk C built and reviewed
(reviews/2026-10-04-m4-chunk-c.md,
fixes 683678a, plan change C5). Step 11, the QEMU lab (lab/,
make lab-e2e), is committed (9f29ea5, with its one product fix
55d2bde). The step 11 check on 618e4e9: UEFI passed; BIOS did not in
three runs, none of them a fault of sc (log, 2026-10-05), and the last
one needed a lab fix (bec333b). The second round (on 225e2d6) gave
no verdict either: UEFI stalled before GRUB, and this VM went down
during BIOS (log, 2026-10-05). The lab now retries such a stall
(152ecbe). The step 11 check passed on 56f5359: make lab-e2e
PASS, UEFI and BIOS, clean tree, with boot 5, no retries (log,
2026-10-05). The chunk D review is closed (2026-10-06;
reviews/2026-10-05-m4-chunk-d.md,
plan change C6). Two reviewers: 1 high (narrow), 10 medium, 13 low,
5 cleanups, none of which changed the PASS on 56f5359. Fixes
3115b71 (sc: the 60 s limit, the hangup flag, the login hint),
9d3df9e (lab: verdict classes, teardown, the cache); from the runs
of the fixes 1d25037 and c70d886 (sc: the console writer’s turn,
a host pause as one slice), 64911f4 and 0ccba27 (lab: the second
command line, a SHUTDOWN not lost); the mutation tests fcb1b6d and
a371738 (326 lab tests). make lab-e2e on 22383a0: UEFI PASS
(24m29s, 0 retries) and, in a bios-only rerun of the same tree, BIOS
PASS (45m14s, 2 lab retries, both the host’s pauses); both with boot 5
and dirty=no. Open items are in the write-up; the host’s pauses are
yours (below). A check-in every 30 min resumes work after a usage
limit (your ask); no VM is up. Step 12 is in (3e72281):
scripts/accept-m4.sh (make accept-m4), the non-boot parts as root,
nothing real written; checked as a user, not run as root yet. Its run
is part of sign-off S1 (with all checks and make lab-e2e) and needs
your OK. Step 13 is in (b722bf4): the docs (README “When the
machine does not boot”, MILESTONES, PROJECT_LOG, CLAUDE.md; see the
log). All 13 steps are built. The chunk E review is closed
(reviews/2026-10-06-m4-chunk-e.md,
plan change C7): 1 high (the README’s GRUB password recipe), 4 medium,
12 low, 6 cleanups, all fixed. Fixes 654ee7f (sc status: the undo is
the newest blocker before a newer error; the menu promise only while
its flag is set) and 4b91d97 (accept-m4.sh: the runtime units stay
loaded, so step 2 can read their journal; the real store read-only; a
bad verdict; more). M4 is built and reviewed. Sign-off S1
passed (your OK, 2026-10-06 14:22 UTC, “yes start S1”; log): all
checks, make accept-m4 as root PASS (19f2200), and make lab-e2e
PASS in both modes on 2c58f8b, one sc build (cafca1512050):
BIOS 44m18s and UEFI 37m19s, no retries, boot 5, clean tree. On the
way: an ovmf update made the lab’s reference image stale, and make
lab-image was found gone since 9d3df9e (back in b9761a9).
Sign-off S2 passed (your OK and your snapshot, 2026-10-06 19:45
UTC, “took the snapshot”; log). The M4 build (cafca1512050, the one
S1 tested), the two boot units, the drop-ins and 42_smartconfig are
installed on this VM, update-grub has run; the reboot’s verdict ok,
flag clear, 23.5 s, sc off the critical chain, the menu hidden, sc
status healthy. You reset the two boots after it (a black desktop;
your answer). Sign-off S3 passed (your OK and a fresh snapshot,
2026-10-06 21:05 UTC, “Yes, taken — go”; your “s3 done” and two
photos, 2026-10-07 06:27 UTC; log): the bad line in /etc/fstab (row
053fdf, scd: blocker); two failed boots, both verdict bad, each with
the login screen up over emergency mode (on a desktop gdm takes the
screen, so the way in is the next boot’s menu); the rescue boot with
the report above “Press Enter” (your photos); your sc restore fb5cf4;
and the boot after it: verdict ok, flag clear, /etc/fstab back
(sha256 9d71ab60…), sc status healthy, exit 0. New on a desktop,
for S4 (log): a red [FAILED] grub-initrd-fallback.service right above
the report (Ubuntu’s unit, which cannot write grubenv on the read-only
root), sc boot seen twice in each failed boot (harmless here), and
the README’s “stops in emergency mode”. S4: the final review is
closed (your OK, 2026-10-07 06:45 UTC, “go ahead with S4”;
reviews/2026-10-07-m4-final.md, plan
change C8): three reviewers, 2 high, 6 medium, all fixed, and 13 low
(69ea585 … d9c8817, log); the rest is the M4 follow-up list in
MILESTONES. A lab run on 87c6636 was stopped at your word (09:31 UTC)
and does not count (log). S4 is closed and M4 is done: tag m4
(your OK, 2026-10-07 13:20 UTC, “yes, go ahead after it passes”; no
snapshot, your call; log). make lab-e2e on 59203a3, clean tree,
one sc build (1dce09f5ff12): UEFI PASS (19m35s) and, in a BIOS-only
rerun of the same tree after a [lab] ssh timeout, BIOS PASS (19m59s);
no retries, boot 5. accept-m4 as root PASS on that build, and that
build is installed on this VM (sc, both units, 42_smartconfig,
update-grub; the S1 build is kept as sc-2c58f8b). The annotated tag
m4 is on 59203a3 and pushed. No VM is running.
(The reboot check of that install is folded into the follow-ups’ one,
below: no reboot came between.)
Now: M5, the package (your call, 2026-10-07: M5 starts after the
follow-ups either way). The plan, for your read: M5_PLAN.md;
its questions run on the recommended answers until you say otherwise.
Its steps, drafted in ~/smartconfig-work/fu-scratch/wt-m5, land one at
a time, each with its checks, CI and a worklog line. Where it stands
(2026-10-09): steps 1-8 built and reviewed
(reviews/2026-10-08-m5.md); of the sign-off
(step 9), the lab’s --deb run passed in both modes on 8f91158, and
this VM runs that package (installed over the hand install, rebooted,
upgraded; log, 2026-10-09), and the remove and the install again
passed (2026-10-09, after your policy-rc.d answer: moved aside).
M5 is done: tag m5 on 8f91158 (2026-10-09; your answer “Decide
later” on the license tagged it on the package the lab tested). This VM
runs 0.5.0, built at the tag. The cleanup is done (log). Next: M6.
The M3 and M4 follow-ups: done and signed off (2026-10-08). Chunks
F, G and H, each reviewed. make lab-e2e LAB_E2E_ARGS=--grub-password
on 643cec3 (clean, sc 22a34f3df9b3): UEFI PASS (29m19s, 1 retry)
and, in a BIOS-only rerun of the same tree after host stalls cost two
runs their verdicts, BIOS PASS (24m21s, 2 retries); run files in
~/smartconfig-work/signoff/lab-e2e-643cec3/. As root (Claude, your OK
of 2026-10-07): accept-m2, accept-m3 and accept-m4 PASS on
d686341 with that build (accept-m4 fixed there: the hardened test
units ran sc from /home, which ProtectHome= hides), then that build
installed: scd Type=notify (startup rescan 413 ms), the three units
enabled, systemd-analyze verify clean, grub-script-check ok, the
rescue entry in grub.cfg, sc status exit 0, no failed units; the
59203a3 build kept as /var/backups/smartconfig/sc-59203a3. Log
~/smartconfig-work/signoff/signoff-fu-20261008T035149Z-d686341.log.
At the next session after a reboot (yours to start: a reboot ends
this session): the first boot with the follow-ups’ units. Read-only:
its verdict ok, the flag clear, sc-boot-seen and sc-boot-ok done
under their sandbox, scd active (Type=notify), sudo sc status exit
0, no failed units; log them.
The M3 and M4 follow-ups, as planned (your call, 2026-10-07: “if you have finished everything from v1 to v4 then only start the M5 plan”, then “All follow-ups”). Every open item of the M3 and M4 follow-up lists and the actionable “Open from the reviews” notes in MILESTONES, one at a time, each with its tests, checks, commit, worklog line, push and CI; where an item has a real choice, you pick. Out: M4 follow-up 8 (LUKS, a non-goal), and the items that are yours (below). Order, severity first, in three chunks, each closed by a review:
cfb704e): M3 1 (unit drop-ins, the one medium),
2 and 3 (sshd: ListenAddress, a drop-in with its main file), 4 (raw
lines for -v), 5 (Ctrl-C or SIGTERM to sc check), 7 (a check’s row
lookup), 8 (cosmetics).sc status and restore (done 2026-10-07, reviewed:
reviews/2026-10-07-m4-followups-chunk-g.md,
fixes 0a55306): M4 1 (a restore under an
unmounted /boot), 2 (a file replaced by a symlink), 9 (a torn verdict
line), 7 (cosmetics); the notes: the mount hint for /usr/local and
/boot, SIGHUP before signal.Notify, edits recorded after the
healthy boot’s row, the report held until sc returns. Your picks
(2026-10-07): M4 1 refuses (nothing written, “mount it first”); the
verdict waits for scd’s startup rescan (scd Type=notify once the
rescan is done, sc-boot-ok after it); the console prints its header
at once and the changes when ready; the early SIGHUP stays a
documented limit. M4 4 (soft-reboot, chunk H) is documented as a
limit (systemd 255 has no soft-reboot count).a1c1228): M4 3 (GRUB_TOP_LEVEL),
4 (soft-reboot), 5 (the units’ hardening, the test knob), 6 (the test
gaps); the notes: lab check 1.8, the GRUB password recipe in the lab,
the lab’s own items, the cmd/sc flake.make lab-e2e in both modes, accept-m2,
accept-m3 and accept-m4 as root (accept-m2 too, as scd.service
changed) and one install of the new build, the reboot and its
read-only check; then M5. Your calls (2026-10-07): Claude runs these
itself, with sudo (“Permissions are granted to you”); a step that
still needs you (auto mode refusing sudo) is skipped, written under
“Still waiting for you” with what is left, and the work goes on; after
the follow-ups M5 starts either way.
The install puts in scd.service too (Type=notify since 32cc097):
sc first, then the unit, daemon-reload, restart scd.M2 follow-ups done; the soak (S6) runs (2026-10-03). M2 is done
(tag m2). All 8 follow-ups from the final review are in (log,
2026-10-03), and the build of b6ab3cc runs as scd since 14:27 UTC (the
m2 binary is kept as sc-m2). Earlier that day the VM came back from
the pre-S5 snapshot; the m2 install, the apt upgrade and the
post-upgrade manifest were redone (S5’s results stand, log 2026-10-02).
The soak (S6) is closed (your call, 2026-10-03, after 5.6 h instead
of 24; log). It passed on what it saw. Next: your VM settings and snapshot, then the M3 plan
for your approval. The soak check, once scd has run about 24 h on this
build: uptime, restarts, memory, log volume,
unexplained rows, and /etc against manifest-post-s5apt.txt in
/var/backups/smartconfig. (Kernel 7.0.0-38 has run since 2026-10-05.)
To remove the watcher: README “Watch every change”; keep the
store (sc-m1 still reads it; copying changes.db.m1-backup back would
drop every M2 row). One thing at a time: no review runs while the next
step is built (your call, 2026-09-30).
Done at the tag: m1 on df1a378 (pushed); backups in
/var/backups/smartconfig (binary sc-m1, store, /etc and /boot/grub,
manifest and checksums), verified; recovery guide points to sc-m1.
Decisions already made for M2 (2026-09-27, “go with your picks”):
/etc minus the exclude list; the tiers only decide how
loudly to alert./etc: authorized_keys files and /boot/grub; not whole
.ssh folders (they hold private keys).690ab95 to 020c205.847e14a, 994f16c.main, public repo
https://github.com/NikhilSah27/SmartConfig, GitHub Pages site
https://nikhilsah27.github.io/SmartConfig/. f5d81bc.2980474.sc: 35 real-world checks (bad inputs, setuid
restore, deleted files, immutable files, corrupt blobs, 8 parallel
snapshots). No bugs; two test-script mistakes corrected.9df9dd3..<name>.sc-tmp-* in /etc/logrotate.d; tested ~, .dpkg-tmp and
.swp suffixes and none avoids it. Impact is one skipped duplicate stanza
if logrotate runs during a restore’s few milliseconds. Left as is.ReadWithMeta now opens with O_NOFOLLOW|O_NONBLOCK and
requires the opened file to be the one lstat saw. New test reproduces a
swapped file, a symlink and a FIFO; it fails on the old code (read the
wrong file, followed the symlink, blocked on the FIFO). c720f22.store.Restore now inserts its row, renames the file while
holding the write lock, then commits (rolled back if the write fails). A
watcher that sees the rename and takes the lock finds the restore row
instead of an unexplained change. Tests pause inside the restore to prove
it and fail on the old order. ec15ece.sudo ./scripts/smoke.sh PASS, /etc/hosts unchanged./tmp to ~/smartconfig-work/ on the VM,
because /tmp is wiped at every boot.c5b693e.sc snapshot now compares with the latest row under the write
lock, so a snapshot taken during a restore no longer records a spurious
extra row. The new test fails on the old code. 09a7dbd.1e6e91a.c0d3839.a517893.3277111.e7420bb.sc snapshot records what is on disk when it commits (re-reads
if the file changed meanwhile) and takes its timestamp under the lock.
f944a1c.sc now removes prepared temp files,
prints one line and exits 1; tested with the real binary. f4a0948.41dd177.sc at a safe point with a truthful
one-line message; SIGHUP is handled, SIGQUIT prints no stack trace, and
signals a script ignores stay ignored. Tested in-process and with the real
binary; the binary tests fail on the round-3 handler. 4be4bb7.423e9dc.348ec67.2dee5bb.151b7fd.m1 on df1a378 and pushed the tag./var/backups/smartconfig (root only): sc-m1, the
store, /etc and /boot/grub, manifest and checksums. Verified: the
archive has 3145 entries including fstab and grub.cfg; sc-m1 reads the
real store.ec9a58a).pre-commit, commit-msg and
pre-push hooks refuse the private denylist (kept outside the repo, mode
0600) and the shapes of crypt hashes ($6$, $y$), private keys, psk=
values and GitHub tokens. Tested with 11 cases, including a commit that
bypassed the commit hooks being refused at push. The hooks are local and not
in the repo.make race (question 4) 2a915c1 and the CLAUDE.md edits (question
8) f4c79be.20330e4: govulncheck v1.8.0 found 3
standard-library vulnerabilities with Go 1.22.2 (2 reachable on Linux) and
none with 1.26.8. The recovery guide no longer sets GOTOOLCHAIN=local.9a746c3)
(TestSnapshotDoesNotRetryCommit, TestInterruptAfterRenameStopsRetries),
so a slow disk cannot fail them. Mutation checks: with the retry bugs put
back, both fail (6 attempts, want 1).270b095). The slow disk exposed a real M1 race: after a first Ctrl-C the
restore printed “interrupted (file not changed)”, and a second Ctrl-C before
sc exited printed another line. Now the first to claim the end prints, the
other stays silent, and a finished command also claims it. A new real-binary
test holds sc open after its line (a hook built only with -tags sctest,
absent from the shipped binary) and fails with two lines without the fix.a1a6419: history is ordered by insert order (rowid), not by
the wall-clock timestamp, so a clock stepping back cannot make an older row
the newest. Two new tests (store and sc log) fail on the old queries.
Checks clean as user, race and root.cb7a635: it missed its
window 5 of 5 times on tmpfs (so probably on a CI runner’s fast disk too); a
pause built only into the signal tests’ binary now makes it catch the window
on the first try.34da00e: sc init creates changes.db mode 0600 and
tightens an existing 0644 one; the journal gets the same mode.
Mutation-checked. Checks clean as user, race and root.69e8f9b: schema version 1 adds kind and target and
the index changes_path_seq, migrated in one IMMEDIATE transaction;
an M1 database with rows is copied to changes.db.m1-backup (0600)
first; a newer schema or a WAL store is refused with one line. The
concurrent-first-open test now holds all 4 opens until each has read
version 0, so it races on every run. Mutation checks: 6 of 6 guards
caught. Checks clean as user, race and root. The real store is not
touched (gate G1).m1-frozen taken (you, 2026-09-30).3e266cc after you added the workflow scope to the token:
gofmt, vet, tests as user and root, race, static build, on every push
to main and every pull request.ee0ddea: the step 3 backup read changes.db
with a second descriptor, whose close dropped the process’s SQLite
locks; with several processes opening an M1 store at once, 13 of 20
failed mid-migration and an index could be corrupted. The copy now
comes through SQLite itself. After: 0 of 80 failed, integrity ok. Two
new tests use real processes and fail on the old code.397877a: a symlink at the backup’s name no
longer counts as a backup.b3d3f8e: a store upgraded by a newer sc while this one
waits for the lock is refused, not set back.7ba575b: new package internal/scope with the glob language
of plan 7.2 (**, * with leading dots, classes, escapes, braces);
every pattern validated at compile time. Mutation checks: 14 of 14
caught, two after adding cases. Checks clean as user, race and root.b5ad961: the default scope (plan 7.3, embedded), Recorded,
Tier, FingerprintOnly, login homes. Against this VM’s real /etc
and /boot/grub (read-only listing): 323 watched directories, 1,143
files, 234 links, tiers 342/119/66/850, fingerprint-only exactly
machine-id and the three host keys; the same as the plan’s prototype.
Mutation checks: 24 of 24 caught. Checks clean as user, race and root.78b8cc7, your call after reviewing the excluded list:
/etc/alternatives recorded (firewall families at tier 3), TLS private
keys, credstore and PPP secrets fingerprint-only at tier 2. On this VM:
327 directories, 1,147 files, 380 links. The new tests fail 14 times on
the old scope.80bc523: ssh-keygen -A writes a private host key
as NAME.XXXXXXXXXX first; that name is now fingerprint-only.310bc1f: systemd generator and run-parts
directories added to block D, so scripts they run (foo.disabled,
sedAbC123) are recorded.\** and [**] accepted. Test gaps closed.1aca688, your call on the review’s two questions:
~/.ssh/rc, ~/.ssh/environment and /etc/ssh/sshrc recorded at
tier 2; dropbear host keys, WireGuard, apt auth and LUKS key files
fingerprint-only. None exist on this VM today.4034618: fsutil.ReadState reads files and symlinks
through the parent directory without following any symlink, refuses
directories, FIFOs, sockets, devices and big files without opening
them, and says whether the path changed during the read;
SymlinkAtomic and RemoveFile. sc snapshot now also refuses a file
in a symlinked directory. M1 fsutil tests unchanged. Mutation checks:
15 of 15 caught; the new tests found a trailing-slash bug while the
step was written. Checks clean as user, race and root; builds for
arm64 too. Plan note C10 (API reuse).5e9bb25: rows have kinds (file, link, digest, deleted);
Record batches up to 50 observations with dedup, stamp re-check under
the lock, proof-of-absence rows and computed intents; Snapshot is
built on it, so sc snapshot records symlinks as links and
fingerprint-only paths as digests. A writer killed mid-transaction
after a cache spill leaves only committed rows. Mutation checks: 19 of
20 caught, one equivalent. Checks clean as user, race and root. Plan
note C11 (computed intents only for watcher rows).83c706c: restore handles links (atomic symlink, owner set
first) and deletions (removes the file after a pre-restore row); it
refuses fingerprint-only paths, digest rows and unsafe directories
(symlinked, missing, or owned by another user) before writing
anything. A watcher recording during a restore adds no row after it.
Mutation checks: 15 of 16 caught, one equivalent. Checks clean as
user, race and root.3a3dcf5: the CLI understands row kinds: link and
fingerprint lines in sc snapshot, link/deleted/digest in sc log,
link targets in cat and diff, and the four restore lines of plan
5.6. No command shows a fingerprint-only path’s content. Mutation
checks: 12 of 12 caught. Checks clean as user, race and root.d388381: make m1-compat builds sc-m1 from the m1 tag
without touching the checkout and proves it still reads a migrated
store, restores file rows, refuses every new row kind with one line
and changes nothing. The test fails when pointed at the M2 binary, so
it tells them apart. Checks clean as user, race and root.cae8992: a fingerprint-only file
reached through a symlinked directory (/proc/self/root/etc/machine-id)
was stored with its content; the rule now also checks the resolved path.go test was
piped through tail, which hid its exit status. It was fixed before
any push; checks now run with pipefail.make m1-compat passes.e410264: new package internal/watch with the inotify
layer (syscall package only): nonblocking instance whose Close
unblocks Read, directory-only watches that refuse symlinks, and an
event parser. Mutation checks: 9 of 9 caught; 5 runs in a row pass.
Checks clean as user, race and root.e6b1c32: the watcher. Two inotify readers, one debounced
worker recording through the store with the stamp check, proof of
absence for “did not exist”, a startup rescan with “while not
watching” and a one-line baseline summary, log lines without content,
panic recovery, the single-instance lock and the SC_HOME-in-a-root
refusal. Writer matrix and structural tests pass; 5 runs in a row and
the race detector clean. Mutation checks: 17 of 20 caught, 3
equivalent; one test was found not to test what it claimed and fixed.
Plan note C12.836a2a9: overflow (a real kernel overflow in the test),
periodic and rate-limited rescans, root changes (passwd, late roots, a
root moved away and back), the watch limit, the free-space floor,
user-file limits, the dirty-set bound, store-error backoff, and
TestKillDuringBaseline. The race detector found a real race in the
worker’s wait computation; fixed. One race-mode run of the whole suite
failed in internal/watch with output not kept; the next 9 race runs,
5 normal runs and the root run passed. The chunk D review is asked to
look for flakiness. Mutation checks: 14 of 14. Plan note C13.1beb8f3: as uid 1000, a user could make the
root watcher store a root-only file through a directory named
authorized_keys and a symlink swap. Nothing below .ssh entries is
recorded now, and the watcher never reads through a symlink below a
root. The attack is a permanent root-only test.962a118 and 3cbbd2d: an endless
“moved” loop, a crash on a moved directory over the dirty bound, new
directories left unwatched by a racing rescan, missing deleted rows
when paths change type, rescan loops on busy hosts, and a hang after a
panic. All fixed with tests that fail when the fix is reverted.39ae262: sc watch with --root, runContext, exit 0 on
SIGTERM (the real binary is tested), journald priority prefixes only
when JOURNAL_STREAM names stderr. Mutation checks: 6 of 6. Checks
clean as user, race and root.833a899: scripts/scd.service (plan 9) with
TestUnitFile, and scripts/accept-m2.sh (plan 12, 19 steps, make
accept-m2). The acceptance run has not been run: it is yours.51a4b0a: README (sc watch, install by hand, scope),
CLAUDE.md layout, PROJECT_LOG (rescue binary, resume prompt).sudo make accept-m2), would have restored root’s own
authorized_keys under nested sudo, and could stop itself with kill
-STOP 0: both fixed e865164 before it was ever run. sc watch: SIGPIPE
no longer kills it silently, SIGHUP rescans instead of stopping, fatal
lines are logged at err d56237f. Docs warn that the M2 sc upgrades an
M1 store on first use 0bdf909. Plan note C14. Checks clean as user (5
watcher runs), race and root; make m1-compat passes.d20ae38, your OK):
gofmt, vet, go test ./... as user and as root, make race all clean;
file bin/sc says statically linked; make m1-compat passes.
make accept-m2 PASS, all 19 steps (baseline 14 s after the restart,
integrity_check ok). Its test paths, runtime unit and scratch store
were gone afterwards.sudo ./scripts/smoke.sh PASS. /etc/hosts
is back byte for byte (same sha256, 0644 root:root), no scratch store
left./var/backups/smartconfig present; store-m1.tar lists,
sc-m1 runs. New: manifest-pre-s2.txt and sha256-pre-s2.txt,
/etc and /boot/grub just before the install.changes.db.m1-backup
made, integrity ok, user_version 1; sc-m1 still reads the migrated
copy. The real store was untouched; the copy was deleted.bin/sc from 595b440 (sha256 a0bee866…f307, static)
to /usr/local/sbin/sc and scripts/scd.service;
systemd-analyze verify clean; systemctl enable --now scd.
Journal: watching /etc, /boot/grub, 2 .ssh dirs (329
directories); baseline 1530 first seen, 0 changed, 0 deleted, 6.9 s;
no warnings, 0 restarts, 21 MB.changes.db.m1-backup kept; sc-m1 reads it./etc against the manifest: only scd.service and its
multi-user.target.wants link are new; nothing else changed./etc/ssh/sshd_config (written in
place, same inode), no sc typed: one row, T2 changed (fa203e) at
journald warning; sc diff 868df4 shows the old version.chmod -x /etc/grub.d/41_custom, then +x: two mode-only rows,
mode 0755->0644 and back, T1, content unchanged.systemctl mask rsync.service (disabled, inactive), then unmask: a
did not exist row, a link row created (sc cat prints
/dev/null), then deleted; T1.systemctl stop scd, a comment line added to /etc/hosts, start:
baseline: 0 first seen, 1 changed (400 ms), the row says changed
while not watching, T3 at notice.sc restore of both files with scd running: a pre-restore and a
restore row each, no journal line and no automatic row in the next
8 s. Both files back byte for byte (/etc/hosts sha256 as before),
sshd -t ok.
After: /etc against manifest-pre-s2.txt differs only by the scd
unit and its link; integrity ok, 1545 rows (11 new, all expected);
scd active, 0 restarts, no err lines.systemd-run --on-active=30 systemctl reboot, your OK):
reboot.target
reached./etc against manifest-pre-s2.txt differs only by the scd
unit, its link and cups/subscriptions.conf{,.O} (rewritten by cupsd
at 22:15, excluded by default.scope line 87). Side result: the store
was open under a running scd at the hard reset and is intact
(integrity ok, 1545 rows).critical-chain, 116
ms in blame (the exec only); nothing is ordered after it except
shutdown.target and multi-user.target. scd active, 0 restarts,
25 MB; baseline 0 first seen, 0 changed, 0 deleted (3.1 s).~/smartconfig-work/signoff/):
/etc/hosts edited
and /etc/sc-s5-test.conf created, page cache dropped, scd started
and kill -9ed right after its watching line, before its baseline
line. systemd restarted it 10 s later (NRestarts=1). The killed
instance had already committed the new file as first seen; the new
one recorded /etc/hosts as changed while not watching and nothing
twice (2 new rows). Undone afterwards (sc restore, rm)./etc and
/boot/grub it changed only /boot/grub/grub.cfg (manifest diff), and
scd recorded it once (cc5cff). No burst, so the kill could not
land in one.logcheck-database (190
conffiles under /etc/logcheck, no maintainer scripts, no service)
installed, then purged with kill -9 after the first change line.
Install: 190 did not exist plus 190 created rows, each newest row
matches the disk. Purge: killed after 100 of the 190 deletions; the
restart recorded the other 90 as deleted while not watching:
exactly 190 rows, every newest row matches the disk, no two identical
automatic rows in a row, integrity ok, NRestarts=2. /etc is the
same as after the upgrade (the 3 other files in /etc/logcheck
belong to rsyslog, gpg-agent and cracklib-runtime).scd.service.journalctl -f | grep -m N only ends when
journalctl writes again, so after a one-second burst it hung until
its timeout and the first kill never fired. The scripts now poll.first seen row, which
gets no journal line of its own (plan 6, 7.6); if scd is killed
before its baseline summary, not even the count is logged. The row is
in the store. The existence-flag alerts deferred to M3 (plan 15)
should cover this case.git clone from GitHub at 49c6d13:
gofmt, vet and go test ./... clean, bin/sc statically linked. CI
green on the same commit (user, root, race, static build).9cee575: a user renaming ~/.ssh in a loop
held back every rescan except the startup one.3349449: snapd’s user/*.wants/snap.* links are now
excluded.go test -count=5
./internal/watch/... passes; the scope test against this VM’s real
/etc listing passes.sudo journalctl, retention, how to remove the watcher),
PROJECT_LOG (status, VM state, recovery guide now points at
/usr/local/sbin/sc with sc-m1 as fallback), NEXT_STEPS (2.4 ticked,
soak deferred). Tag m2.m2 installed: /usr/local/sbin/sc replaced by the build of tag
m2 (sha256 03fbc30d…, static; was a0bee866… from 595b440), scd
restarted: baseline 0 first seen, 0 changed, 0 deleted (2.2 s),
integrity ok, 2121 rows, 15 MB. CI green on 112bceb (tag m2).clocksource: Long readout interval
warnings (the host holds the vCPUs off for up to 5.6 s), several have
rtkit starvation; 9 vCPUs on an i7-13700HX host; GNOME on Wayland with
software rendering (VirtualBox graphics adapter, no 3D). Suggested on
the host, before the soak: 4 vCPUs and the VMSVGA graphics controller./etc and /boot/grub match
manifest-pre-s5.txt except cups/subscriptions.conf{,.O} (cupsd,
excluded).m2 reinstalled (your yes): built at tag m2 in a clean checkout,
sha256 03fbc30d…, the same binary as before the rollback; installed to
/usr/local/sbin/sc, scd restarted: baseline 0 first seen, 0 changed,
0 deleted (555 ms), 0 restarts, 13 MB; store integrity ok, 1545 rows.systemctl restart systemd-timesyncd stepped it to
07:32 UTC (GitHub’s Date header agrees). Commits f592ece and
cc5e620 carry committer times 6 h 24 min early.apt upgrade rerun (your yes; the same 31 as your “all 31”): 25
upgraded, 8 new (kernel 7.0.0-38, not booted yet), 6 mesa packages held
back by phasing. Across /etc and /boot/grub only
/boot/grub/grub.cfg changed, recorded once (862cd3, T1).
manifest-post-s5apt.txt and sha256-post-s5apt.txt saved again in
/var/backups/smartconfig. scd 0 restarts, 11 MB; store integrity ok,
1546 rows.5b57aaf: New checked SC_HOME
against every root, login .ssh roots too. A user who swapped ~/.ssh
for a symlink to a directory holding SC_HOME, between the lstat and that
check, failed the start (reproduced with a test hook); 5 such failures
in 10 min leave the unit failed (StartLimitBurst=5). Now New checks
only the scope’s roots; the startup rescan skips such a .ssh with a
log line, as later rescans already did. New test
TestSSHRootSwappedAtStartup. Build, gofmt, vet, go test ./... as
user, and internal/watch and cmd/sc as root: clean. Not installed:
scd still runs the m2 build.
make race is flaky on this VM, with and without the change:
TestRescanUnderBusyEvents (“3 rescans for one request”) failed in 1
of 3 full race runs of the watch package on the parent commit and 3 of
7 with the change, and never alone (6 of 6 pass each way);
TestUserFileLimits timed out once in 7, never alone (8 of 8 each
way). Load on a CPU-starved VM; CI is the reference.bd8fad9: a rescan marked
stored paths only under usable roots, so after mv ~/.ssh ~/.ssh.old
its files kept present rows, and a new ~/.ssh gave changed rows
with no deleted row first. A login root that is missing, a symlink or
not a directory is now kept as gone: the rescan marks its stored paths
and the worker reads them below it (absent, or the symlink refused).
Moved away while watching: deleted (found by rescan); a new .ssh
then gives created. While scd is down: deleted while not watching.
A scope root that is gone still gets no rows, since a separate /boot
may only be unmounted (mount tracking is M4); a root that only fails
with another error (an NFS home’s EACCES) is not taken as gone.
New tests TestLoginRootGone, TestRootGoneAtStartup.
.ssh in a loop now gets a deleted and a
created row per file, about one pair a minute (the created row
waits UserFileGap): at most 4 files per login, no new objects. The
general rate limit is follow-up 8.go test ./... as user, internal/watch
and cmd/sc as root; the root tests 5 times under race: clean.
make race: only the known TestRescanUnderBusyEvents flake (4
rescans this time). Not installed.c0cfe77: renameat2(RENAME_EXCHANGE)
of watched directories a and b gives MOVED_FROM a, MOVED_TO b,
MOVED_FROM b, MOVED_TO a. The second MOVED_FROM removed, by path, the
watches MOVED_TO b had just added for a’s old directory, so it and its
subdirectories stayed unwatched until the hourly rescan. A directory
moved away is now walked again when another directory is found at its
name. New test TestDirExchange (raw renameat2, amd64 and arm64:
the syscall package has no wrapper, and x/sys would be a new
dependency): edits in both swapped trees come from events, no rescan.
Checks: build, gofmt, vet, go test ./... as user, internal/watch
and cmd/sc as root, the test 5 times under race, make race: all
clean. Not installed.465b30d: when an
overflow lost a directory’s move away and a new directory was made at
its name, the rescan watched the new one and left the old one’s watch
filed under the same name. A name created in the old directory then
entered the new one’s listing, and a later file of that name was
recorded first seen instead of did not exist and created
(reproduced). walk now removes another watch it finds under the name
it watches. The review’s other symptom (a subdirectory moved away in
the old directory unwatching the new one’s) was already covered by
c0cfe77’s re-walk. New test TestOverflowDropsStaleWatch (a real
overflow; also checks that every watch is its path’s). Checks: build,
gofmt, vet, go test ./... as user, internal/watch and cmd/sc as
root, the test 5 times under race, make race: all clean. Not
installed.d76a295: a walk merged
every name of the old listing into the new one (to keep names an event
added while it listed), so names of files removed long ago stayed, and
a file made later under such a name lost its proof of absence: first
seen instead of did not exist and created (reproduced). A name
from the old listing is now kept only if it is still there (lstat,
only for names the new listing lacks); one that exists is never
dropped, since a rename over it would then look like a create. New test
TestWalkTrimsListing; TestProofOfAbsence and the writer matrix
still pass. Checks: build, gofmt, vet, go test ./... as user,
internal/watch and cmd/sc as root, the proof tests 5 times under
race, make race: all clean. Not installed.2a8b686: Record
stores a new object before its transaction finds the path moved, and
keeps it. The home-file limit counted rows only, so a user who kept
rewriting a file in their .ssh got about ten retries 50 ms apart
(the Quiet of the tests), each leaving an object of up to 64 KiB,
before every row: reproduced, 11 tries in 1.5 s. A new object stored
for a home file whose record finds it moved now counts like a row, so
the retry waits UserFileGap: 1 try. Such orphans are still kept
(nothing is pruned in M2); they are now bounded like rows. New test
TestUserFileOrphansLimited. Checks: build, gofmt, vet, go test
./... as user, internal/watch and cmd/sc as root, the user-file
tests 5 times under race, make race: all clean. Not installed.7463e5d: kill -9
during sc restore can leave .<base>.sc-tmp-<random> next to its
target, and the scope excludes those names, so nothing said so. A walk
now logs one older than 10 minutes once per run, at warning: stale
temp file PATH (written TIME), left by an interrupted sc restore; not
recorded, remove it. Reported, not removed: it may hold the only copy
of what the restore was writing, a fresh one may be a restore at work,
and a removal in /etc is the admin’s call. MILESTONES no longer says
the watcher clears them. New test TestStaleTempReported. Checks:
build, gofmt, vet, go test ./... as user, internal/watch and
cmd/sc as root, the test 5 times under race: clean.
make race failed in TestRescanUnderBusyEvents (a 30 s wait timed
out; “7 rescans” on a rerun). Alone under race, back to back, it
failed 4 of 4 and then 2 of 4 on 6c1f306 (before today’s
follow-ups) and exactly the same on this commit: the VM’s CPU
starvation (pressure up to 13% this hour), not the changes. CI passes
it. Worth making less timing-bound before M3.4ec437a: a digest row’s id was
the first 6 hex of sha256(path, ts, fingerprint) and the journal prints
it, so a journal reader could test guesses at a short fingerprint-only
secret (reproduced: the id followed the formula). Digest rows now get 6
random hex digits with the same uniqueness check; file, link and
deleted rows keep the formula (M1 ids still reproduced), and existing
rows keep their ids. Plan 5.2 updated, change C15. New test
TestDigestIDRandom. Checks: build, gofmt, vet, make m1-compat,
internal/store and cmd/sc as root: clean.
TestOverflow, TestOverflowDropsStaleWatch,
TestRescanUnderBusyEvents) timed out in make test and make race
at CPU pressure up to 15%. Run back to back at normal load, the whole
watch package passes on 6c1f306 (37.8 s) and on this code (44.9 s,
8 more tests), and TestOverflow alone passes 10 of 10 on both at
0.6 s. gnome-shell averages over half a CPU (software rendering).064bf68 (your pick): every change of
a system file got a row, so one rewritten every second meant about
86,000 rows, objects and journal lines a day. Each system file now has
a row budget: 20 rows, one more earned every 5 min. With none left the
next row waits, then records the newest state with ` (rate-limited);
the first wait logs PATH changes constantly: recording it at most
every 5 min, the newest state at warning. Budgets back to full are
dropped at each rescan, so the next flood warns again. .ssh files keep
their 60 s rule; restore rows are not counted (written by sc). A file
rewritten without pause now gets about 300 rows a day. Plan 6.3 and
C16, README, MILESTONES updated. New test TestPathRateLimit; the test
setup gives other tests a budget of 1000 (TestRestoreWhileWatching
changes one file 20 times and waited 5 min for the 21st row).
accept-m2.sh checked by reading: no file gets more than a few
watcher rows (its restores are sc rows; its 20,000 chmods run while
scd is stopped). Checks: build, gofmt, vet, go test ./… as user,
internal/watch, internal/store and cmd/sc as root, make race`:
all clean. Not installed.b6ab3cc (M2 plus follow-ups 1-8; static, sha256 396f84cb…, clean
tree) to /usr/local/sbin/sc; the m2 binary kept as
/var/backups/smartconfig/sc-m2 (sha256 03fbc30d…). Unit unchanged
since m2, store schema unchanged (user_version 1). scd restarted:
baseline 0 first seen, 0 changed, 0 deleted (368 ms), 0 restarts, 13 MB;
integrity ok, 1546 rows. The m2 build’s run before it (about 7 h, 2 of
them paused): 4.5 s CPU, 12.7 MB peak, 0 restarts, one row
(grub.cfg, the upgrade). PROJECT_LOG: VM state and recovery guide now
name the new build and sc-m2.b6ab3cc. The VM settings (4 vCPUs,
VMSVGA) and the snapshot are still open on your side.sc edit, sc check,
checks in scd after a recorded change, a built-in file graph; no store
change; 16 steps in five chunks. Probed on this VM first (fabricated
files in ~/smartconfig-work/m3plan/; the real /etc only read), which
changed the design: a stock system already fails some validators (pwck
exit 2, 8 of 121 udev rule files), so only findings an edit adds are
blamed on it; netplan generate --root-dir still asks for a
daemon-reload, so the generator binary is used; findmnt --verify
accepts a misspelt option and reports a missing disk as an error even
with nofail, so our rules set the severity. Waiting for you: the 7
questions in section 14.check: findings, rules, runner, 45df7da: new package
internal/check. Finding and Severity; the rules table (empty: each
checker adds its rules with its step); Runner, the one place that
starts a validator (fixed tool directories, never $PATH; argument list,
no shell; LC_ALL=C and a fixed PATH; stdin /dev/null; 10 s timeout
that kills the process group; 64 KiB output cap; a missing tool is a
result, not an error); Scratch, the private copy under
$SC_HOME/tmp. Found while testing: os/exec reports a leftover
child only when the tool exits 0, so the group is killed after every
run. A mutation (kill the tool only, not its group) fails
TestRunTimeoutKillsGroup. Checks: build, gofmt, vet, go test ./...
as user, the package as root and 3 times under race, static: clean.check: file graph, 95da13b: default.graph and its
parser (check, apply, mode lines; first match wins), with the
scope’s glob language exported as scope.Glob. The built-in graph
holds only its header until the checkers arrive; TestDefaultGraph
then requires sample paths per checker that the scope records and that
are not fingerprint-only. Plan change C1 (your OK): no with line,
since M3 checks only the file that changed. Checks: build, gofmt, vet,
go test ./..., both packages under race: clean.check: fstab, b665691: Checks.Check (graph lookup,
scratch copy, sorted findings) and the fstab checker: sc’s own rules on
every line, then findmnt --verify --tab-file on the scratch copy.
Six rules with explanations. Severities follow what
systemd-fstab-generator does (probed, plan A14, change C2): blocker
only for a local filesystem without nofail/noauto; /, swap and
network filesystems are errors; nofail/noauto lines warnings.
Tests: 19-line fabricated fstab against a fake machine; findmnt outputs
captured here as golden files (user and root); a hanging findmnt; the
real findmnt where installed. Checks: build, gofmt, vet, go test
./..., the package as root and twice under race, static: clean.check: baseline diff, efc2ce3: Added(before, after)
gives what an edit added: a finding is the same when its rule, severity
and text are, wherever its line moved; the same finding once more, or
with a higher severity, is added. Worst. 11 cases. Checks: build,
gofmt, vet, go test ./...: clean. Chunk A (steps 1-4) is built; its
review is next.a2e325f. Three
were high: the baseline diff hid a newly broken line when the file
already had a bad one; a UUID in the wrong letter case passed; findmnt’s
word on devices sc cannot look up was dropped. Also: false blockers on
working type and option spellings, stdout and stderr read apart, a
validator that did not really run is now a note, and leftovers are
killed while the tool is still a zombie. Plan change C3. Checks: build,
gofmt, vet, go test ./..., the package as root and 3 times under
race, static: clean.sc check, afa9403: the command of plan 6.1: a table
(severity, file, line, rule, one sentence), -v for explanations and
the validators’ own lines, exit 2 for a blocker or an error. No
argument: every regular file the graph has a checker for
(Graph.Files); an id checks that saved version. What could not be
checked is a note: line. Plan change C4: a user without write access
to $SC_HOME gets a private scratch directory, so a readable file is
checked without sudo. Tried read-only on the real /etc/fstab as a
user (scratch in a temp SC_HOME): no problems, one note (not root,
types not compared). A test found a crash on a pattern with no fixed
directory (**/name); fixed before the commit. Checks: build, gofmt,
vet, go test ./..., check and cmd/sc as root, race on the new
tests, make m1-compat: clean.store: Replace, 5f6068b: a file and its row written in
one commit under the write lock (restore’s commit, now shared as
commitWrite), for sc edit. Saves the state before unless the newest
row holds it; one row of the new origin edit; a new file gets a did
not exist row first; ErrFileChanged when the path is no longer the
version read; the restore refusals apply. Tests: rows, modes, undo by
restore, the three races, refusals, 20 rounds against a running watcher
(no extra row), and sc-m1 lists and restores an edit row. Checks:
build, gofmt, vet, go test ./..., store and cmd/sc as root,
race on store and the watcher test, make m1-compat: clean.sc edit, c997865: the command of plan 6.2. The editor
opens a copy; the result is checked against the old content and only
what the edit added counts; a new blocker or error is explained and
asked about (edit again, save anyway, quit; end of input quits, exit
2); saved through store.Replace with the file’s mode and owner. One
sc edit at a time (a lock); a file that changed on disk meanwhile is
not written and the edited version is kept under $SC_HOME/tmp/kept-*;
copies an interrupted sc edit left are removed by the next one.
Tests drive it with a script as editor: clean save, unchanged, quit,
end of input, edit again, save anyway, warning only, an old blocker not
blamed, new file, changed on disk, refusals, the lock, a failing or
missing editor, an editor with arguments. Checks: build, gofmt, vet,
go test ./..., store and cmd/sc as root, race on the new tests
and store, static: clean. Chunk B (steps 5-7) is built; its review
is next.a946f6f. Two were high and lost
the user’s work: sc edit deleted the edited copy when the save
failed, and a Ctrl-C from inside the editor killed sc and orphaned the
editor. Now the copy is kept after any failure and its path printed,
and SIGINT/SIGQUIT are the editor’s while it runs. Also: sc check
exits 1 when a file could not be read; ids as sc cat takes them; a
deleted row before a creation after an unrecorded deletion; 80-column
lines. Plan change C5. Checks: build, gofmt, vet, go test ./...,
store, check and cmd/sc as root (one timeout of the M2 test
TestWatchSIGHUPRescans under load; 8 of 8 alone), race on the new
tests, make m1-compat: clean.check: sshd, 4a3451c (parallel agent, reviewed and
integrated here): sshd -t -f on the scratch copy. One sshd-invalid
blocker per line of this file sshd names; the text names the keyword,
never the value; a problem in an included file is a note; deprecation
notices with exit 0 are nothing. A HostKey line naming a key that does
not exist, when sshd has no other, is a finding (the server would not
start; the agent’s choice, kept). No host key a user may read, a
missing /run/sshd or a missing file is a note, never a clean run. 20
golden outputs from OpenSSH 9.6p1 on fabricated files. Checks: gofmt,
vet, the package as user and as root (the real-sshd test included) and
under race, static: clean. Step 8 (sudoers) follows when its agent
reports.d4ad4e9: CI’s root pass failed TestSshdReal: on
a runner where ssh.service never ran there is no /run/sshd, and sshd
(as root) rightly does not finish, which the checker reports as a note.
The test assumed the dev VM’s /run/sshd; it now expects the note
there. Checked as root with /run hidden in a private mount namespace.check: sudoers, c588cdd (parallel agent, reviewed and
integrated here): visudo -c -f on the scratch copy; one
sudoers-syntax finding per message about this file, text of ours
(never the line): syntax errors and an alias defined twice are
blockers, a bad Defaults option or a missing include an error, unused
or undefined aliases warnings. Problems in included files and anything
visudo could not read are notes. sudoers-mode, changed in review:
error only where sudo ignores the file (root does not own it, others or
a non-root group may write it, per sudoers(5)); any other mode than
0440 is a warning (sudo reads it, only visudo -c complains). Note
from the agent: sudo 1.9.15 recovers from a syntax error by dropping
the rest of the line; the explanation says so. Checks: gofmt, vet, the
package as user, as root and as root without /run, under race: clean.check: systemd units and grub, ff952e7 (parallel
agent, reviewed and integrated here): systemd-analyze verify on the
scratch copy (keeps the unit’s name): a skipped line is
unit-unknown-key (warning), a missing Exec command unit-exec-missing
(error), anything that stops the unit loading or starting unit-syntax
(error); other units’ problems and failed man lookups are ignored.
/etc/default/grub with sh -n and grub.cfg/custom.cfg with
grub-script-check: blockers at the line named. Checked as root here
too (the agent could not): clean.check: plain rules, 7aeb144 (parallel agent,
reviewed and integrated here): nsswitch-no-files, preload-missing-lib,
flag-nologin, flag-sshd-not-to-be-run, hosts-no-localhost, each
reading the file as glibc 2.39 does (the agent read glibc’s sources).
Changed in review, on the agent’s own warning: systemd alone is not a
local source for passwd: (it gives root and nobody, not /etc/passwd);
my brief had it wrong. Left for later, from the agent: a bad action in
any nsswitch line makes glibc reject the whole file.go test ./... as user, the check
package as root, as root without /run, and under race: clean.41c87bf except one cleanup. Two were
high: harmless systemd remarks (User=nobody) were reported as “the
unit does not load” (new unit-notice warning), and a bad nsswitch
action, which makes glibc reject the whole file, went unnoticed (new
nsswitch-invalid blocker; 13 cases checked against glibc 2.39 by
bind-mounting fabricated files in a private mount namespace). Also:
sudoers names sudo skips, the mode of a saved version, the walk depth
of brace patterns. sc check with no argument now checks what scd
records (10 files here; snapd’s generated units are left out, as in
M2). Plan change C6. Checks: build, gofmt, vet, go test ./..., the
check package as root and under race: clean.5b1d5a5: TestRescanUnderBusyEvents generated
files without end, so a slow worker hit the dirty bound for real and its
(correct) rescans failed the test; it now makes 60 files, below the
bound, and a mutation that counts rescan marks still fails it (15
rescans). The overflow tests and TestWatchSIGHUPRescans get 60 s
upper bounds (a full kernel queue to work through; sc watch’s 10 s
rescan gap). Watch package: three race runs in a row clean; make
race clean.watch: check after a recorded change, 580e1d1: after an
automatic file row, the worker queues the path for one checker
goroutine (a bounded set, 100 paths) and never waits. The checker
compares the newest content with the content before the first queued
change and logs only what was added (T1 /etc/fstab: check: blocker
RULE, line N: SENTENCE (ID) at err; errors at warning; warnings at
notice; 5 lines at most), and check: ok again when a reported file is
clean. No validator output reaches the journal. Not checked: deleted,
link and digest rows, restore and edit rows, the startup baseline. A
full queue drops the path with one line. Tests: the lines and their
priorities, no repeat, ok again, quiet cases, a slow validator does
not hold up rows, the queue bound (3 times, and under race). Checks:
build, gofmt, vet, go test ./..., watch and cmd/sc as root,
watch under race: clean. Not installed (gate G2).sc scope, 7ccb3d9 (built while the step 13 agents
work; no shared files): explains a path, one property a line: recorded
or not and the deciding scope line (or the excluded directory above
it, sc’s own directory, or outside every watched directory), tier and
its line, fingerprint-only and its line, checker and when a change
applies. Uses the machine’s scope (login .ssh roots, SC_HOME left
out). scope.Explain is tested to agree with Recorded, Tier and
FingerprintOnly. Scope lines go on their own line (80 columns).
Checks: build, gofmt, vet, go test ./..., the new tests as root:
clean.sc check --as PATH FILE, c53180f (plan change C7): checks a
candidate as if it were at PATH, before it is copied there; rules about
the file on disk at PATH do not apply. It lets the acceptance run give
every checker fabricated content as root with the real validators,
without writing a broken fstab, sudoers, netplan or passwd anywhere.check: netplan and udev, a73dfde (parallel agent,
reviewed and integrated here): netplan merges the file with the
machine’s other netplan files in a scratch root and runs netplan’s
generator with --root-dir (traced as root here: it starts only
systemctl is-system-running and writes only under the scratch root);
netplan-invalid blocker, texts never repeat a value (Wi-Fi
passwords). The Runner takes the generator’s full path from a
one-entry list. udev: udevadm verify --no-style, udev-invalid
(error) and the new udev-notice (warning). Checks: gofmt, vet, the
package as user, as root and under race: clean. Step 13b (passwd,
group, sysctl) follows when its agent reports.check: passwd, group and sysctl, 2e4d4ee (parallel
agent, stopped at the usage limit before it reported; its worktree
survived). It had left one mutation of its own self-check applied in
sysctl.go: restored from its backup, and the mutation confirmed to
fail a test; its temporary capture test removed. Reviewed here without
its report: pwck/grpck run read-only on the copy and a made-up shadow
file (the real hashes are never read; traced as root: no writes, no
lock), severities from glibc’s parsing, missing home directories are
no finding, quoted lines never reach a finding; passwd-root,
passwd-shell-missing. sysctl --dry-run traced as root: no writes.
The real passwd, group, sysctl.conf and sysctl.d files check clean.
Checks: gofmt, vet, the package as user, as root and under race.scripts/accept-m3.sh, 2ff038b: committed now that every rule it
names exists; its step 1 (sc check --as, all checkers) passes as a
user. The rest needs root and writes a test unit under
/etc/systemd/system: sign-off S1, with your OK.2fb037e
(reviews/2026-10-03-m3-chunk-d.md,
plan change C8). One reviewer agent, 10 findings, each reproduced or
tested here first.
passwd-root was a false blocker on stock Ubuntu: nss-systemd
supplies root when no line does. Shown with a fabricated passwd
bind-mounted in a private mount namespace: getent gives uid 0 and
sudo still works. It is now a warning there; a root line with another
uid stays a blocker.sc check --as no longer waits on a FIFO.sc scope checks the roots with lstat, as scd does.Explain is built on decide; one no-login list and one
PathError helper.Checks: gofmt and vet clean; all tests pass as a user, as root and
under race. A stray sc binary from go build ./cmd/sc got into the
fix commit; it was taken out before the push, and /sc is now
ignored.
make accept-m3; a new section, “Check before
it breaks”, with sc check (<id>, --as, -v, exit codes),
sc edit, sc scope, what each checker runs, and scd’s check
lines. The example output comes from a made-up fstab.accept-m3.sh added to the layout.Every relative link resolves.
b6ab3cc build from 14:27 to 20:04 UTC:
/etc and /boot/grub against manifest-post-s5apt.txt: only
cups/subscriptions.conf{,.O} changed (cupsd rewrites them; the
scope leaves them out).Not covered: apt-daily-upgrade did not run (skipped: the VM reports
no AC power), and the deliberate apt upgrade was S5’s rerun, before
this build. A longer run and an apt run under scd come with M3’s
install.
make build fmt vet m1-compat on 36fa076; the tests last
passed as a user, as root and under race on 2fb037e (only docs
changed after); CI green.sudo ./scripts/accept-m3.sh, with the real scd stopped:
6db9415, which also makes fail() print its newlines.sc check on the real system: no problems in 24
files. scd’s check line and “ok again” for the test unit; sc edit
through quit, edit again and save anyway with exactly 2 rows;
sc scope. The real fstab, sudoers, sshd, grub, nsswitch, hosts,
passwd, group, sysctl, netplan and sudoers.d files are unchanged.468c633 (Go code as in
2fb037e; sha256 8e96663f…) is installed as /usr/local/sbin/sc.
The b6ab3cc build it replaces is kept as
/var/backups/smartconfig/sc-b6ab3cc (sha256 396f84cb…); to roll
back, install it again and restart scd. The unit file did not change.
scd restarted at 20:15:47 UTC: baseline 0 first seen, 0 changed,
0 deleted (1.1 s), 16 MB.
sudo sc check: no problems found in 24 files, exit 0. There is
nothing to review.sudo sc edit /etc/fstab, where the editor adds a data disk
with a mistyped UUID (made up). sc showed fstab-source-missing as a
blocker on the new line, with its explanation (a 90 s wait, then
emergency mode with root locked), and the prompt. Answering q
gave “not saved” and exit 2. fstab kept its sha256 (9d71ab60…), no
row was added and no copy left behind.PermitRootLogn no appended in place to the real
/etc/ssh/sshd_config, as nano saves. Within about 1 s scd logged
the row (c89eb3, T2) and check: blocker sshd-invalid, line 132
at err. sudo sc check showed the same, exit 2. sc restore 2043f5
put the file back: sha256 64325541… as before, 0644 root:root,
sshd -t clean, sc check clean, ssh.service never restarted.297fbc7:
297fbc7 and bb31f3e: 6 of 7 medium findings, among them
a relative SC_HOME giving a false clean and a timed-out validator
giving “ok again”; plus 7 low ones and a cleanup, among them “ok
again” at a restore (the S3 gap).make m1-compat passes; CI green.accept-m3.sh, with cases for the new rules, passed again as root
on bb31f3e with scd stopped and started again around it.sc check as root on the real system: no problems in 24 files.m3 (your yes: “install and tag”). The fixed build
(code of bb31f3e, sha256 cf5ba078…) is installed as
/usr/local/sbin/sc; the S2 build is kept as
/var/backups/smartconfig/sc-m3pre. scd restarted at 21:25 UTC:
baseline 0 first seen, 0 changed, 0 deleted, 14 MB. sudo sc check:
no problems in 24 files. The annotated tag m3 is on c253cfd and
pushed.ro fstab=no systemd.unit=rescue.target
SYSTEMD_SULOGIN_FORCE=1, from a generated GRUB entry, gives a root
shell with a read-only root in about 40 s. It also works under
Secure Boot.sc restore after a remount fixed a bad fstab, and the next boot
was healthy (an end-to-end script passed in 408 s).sulogin-lockedpwd.patch.~/smartconfig-work/m4lab, its overlays rebased onto relative
paths.$SC_HOME/boots, so
sc-m1, sc-m2 and sc-m3 keep reading the store), and an automated
QEMU owner scenario (make lab-e2e). Waiting for your approval.init=/bin/bash
last; the backup binaries listed as they are now.mode=ro where access(2) says sc may not write. Every
write returns “remount it read-write first”./run for root),
and the store is untouched. Tested by chmod as a user with the
existing crash helper; a mutation gives the lab’s error 776. Checked
by hand with a SIGKILLed Python writer: one note line, the committed
rows, and a self-repair once writable.sc check.
$SC_HOME and $TMPDIR read-only, the copies go to /run
(root) or the runtime directory, then /dev/shm, and are removed.
With none writable, one line names the places tried.49a0a5b).
sc log | head left a full copy
in /run per run). They are now tracked private directories, removed
on signals and swept after an hour, and SIGPIPE is caught.unshare --mount);
migrations on the copy; a journal race; messages; the copy’s
location; two film lines.internal/boot keeps $SC_HOME/boots (seen and verdict lines,
append-only, trimmed past 1 MiB). The hidden sc boot seen and
sc boot verdict ask systemctl through the M3 runner.scripts/sc-boot-seen.service runs early, outside local-fs.target,
so a boot that fails on a disk is still seen. It waits only for the
store’s filesystem (RequiresMountsFor): the lab prototype’s
Before=local-fs.target would write under a separate /var’s mount
point.scripts/sc-boot-ok.service runs after multi-user.target.systemd-analyze verify is clean for both. They are not installed;
that is sign-off S2.sc status.
--console caps time at 10 s and output at 20 lines, but judges
every change, a fix found while testing.40288d8, plan change C4). The reviewer also booted the units
in a throwaway QEMU guest.
sc-boot-seen’s 10 s timeout killed it on slow
boots, so the failed boot left no trace. It no longer holds up
sysinit and gets 90 s.scripts/42_smartconfig. It adds the “SmartConfig
rescue” GRUB entry: the newest kernel with an initrd, root= as
10_linux gives it, and the lab’s rescue arguments. It also adds the
smartconfig_pending menu flag. btrfs and ZFS roots get no entry.
Tested on fabricated /boot trees; grub-script-check accepts the
output.sc boot seen sets
smartconfig_pending=1 in grubenv, and a healthy verdict unsets it.
It is left alone where grubenv is not GRUB’s 1024-byte block, such as
a separate /boot that is not mounted yet. Tested with a fake and with
the real grub-editenv.scripts/smartconfig-rescue.conf
goes into rescue.service.d and emergency.service.d and runs sc
status --console before the shell. systemd-analyze verify accepts
both services with it.+i/+a precheck. sc restore and sc edit
refuse an immutable target, or one in an immutable or append-only
directory, before writing anything, and name the chattr to run.
Tested as root with chattr.
683678a, plan change C5).
sc status said “normal” because only the
service is activating there. Fixed.root=, a whole-boot ordering test.9f29ea5). lab/ runs the M4
owner scenario in a VM under TCG, for UEFI and BIOS: a healthy boot,
an fstab edit, the broken boot, GRUB’s menu and the rescue entry, sc
status --console and sc restore in the rescue shell, the healthy
boot after. Dev only: no sudo, no KVM, not in CI. make lab-image
once, then make lab-e2e; make lab-test runs its 157 unit tests.
55d2bde): on BIOS the console getty
can start next to emergency.service and hang up the console; sc
status --console then died by SIGHUP and the report was lost. It now
drops SIGHUP in sc’s own handler (not SIG_IGN, which the commands
it runs would inherit; that leak failed TestWatchSIGHUPRescans
first). The lab now fails 2.5 if sc: interrupted by hangup shows.grep -a); both commits were scanned again and are clean.make
lab-test passes.618e4e9): UEFI PASS, BIOS not yet.
618e4e9 came first, from a BIOS run that ended INCONCLUSIVE at
1.0: ssh lost the reboot command (exit 255) and the lab gave up. It
now asks the guest and sends the command once more only when no
reboot is queued.618e4e9: PASS in 30m51s, no retries, clean tree, with
boot 5; boot 2 ended as outcome a.goal3=multi-user, which sign-off needs
once). In boot 5 the guest kernel stopped at smpboot: x86: Booting
SMP configuration: for 18 minutes, before any userspace; the lab
gave up after 900 s (5.1, [lab]). The same step took 0.1 s in the
boots before it. A TCG stall, seen once; the lab does not retry it,
and that is left so unless it comes back.bec333b): with no shell line in boot 2, 2.5 takes the
block above the login prompt and holds it to the golden as
strictly; a wrong report or none is still an [M4] failure. Three
tests from that run’s console text; that run’s whole boot 2,
replayed, now gives 2.5 a note and no failure.sudo first. It is Ubuntu’s race (plan A2), and sc’s report
came through; whether the report or the README should say so is
open.make lab-test passes (166 tests). No Go file changed.225e2d6): no verdict; the lab now
retries a stall (152ecbe).
last says crash) and came back at
17:18. The same happened at 04:54 (BIOS run 1 of the first round)
and at 06:00. A whole make lab-e2e needs the VM up for over an
hour.152ecbe): a stall is a [lab] retry, once per boot,
only where sc has no part in the boot yet: before GRUB ran its
configuration (no observer line, no menu for a whole
BUDGET_MENU), or in the kernel before Run /init in a boot that
should reach a login prompt. Anywhere else it stays INCONCLUSIVE.
Each stall leaves evidence/stall-*.txt (QEMU’s CPU time over
5 s, its registers twice, the gaps in the lab’s own running), so
the next one says what stood still.style=[hidden].
After a reset later than GRUB, Ubuntu’s recordfail is still 1 and
no style is set, so a retry after a panic would have failed x.1
as [M4] on UEFI. It now expects style=[] there (sc’s flag block
must change nothing without the flag).make lab-test passes (174 tests); the failed run’s boot 1,
replayed, is a stall retry; the registers command and the CPU time
reader were tried against this QEMU. No Go file changed.56f5359): make lab-e2e PASS.
dirty=no, with boot 5. Boot 2
ended as outcome b (goal3=multi-user, which sign-off needs
once). One warning, 2.5: scd was already running when the report
was made.dirty=no, with boot 5. Boot 2
ended as outcome a. Two warnings, 3.1 and 4.1: no observer line on
the VGA screen, as the design expects where the menu wipes them.~/smartconfig-work/signoff/lab-e2e-56f5359/
(result, log, ledger, mux log per mode) and the run’s own log
beside it.3115b71 (sc: status --console stops itself
after 60 s, since systemd ends the rescue unit, shell and all, at
90 s; the hangup flag set earlier; a login hint in the emergency
report) and 9d3df9e (lab: ssh loss and host pauses never an [M4]
result, product faults never INCONCLUSIVE, one-shot signals, the
cache must be the lab’s, one lab per user, the recordfail rule by
attempt).make lab-e2e on 85e9f59: FAIL, both modes, no retries.
UEFI 3.6: the rescue report on ttyS0 ended mid-row; sc gave each
console one 2 s write. BIOS 3.4: the rescue kernel’s first
“Command line:” lost one byte on serial (“oot=UUID=”), after the
host stood still 379 s; its second print was whole. The host was
loaded by the test agents beside the run.1d25037: a console keeps its turn while it takes bytes or its
queue drains (5 s stalled, 10 s at most). 64911f4: the lab reads
the kernel’s second command line where the first lost a byte;
check_decision’s bios notes were lost (found by a test agent).64911f4: tests as a user, as root and under race;
make lab-test (196).make lab-e2e on the fixed tree.fcb1b6d).
lab/e2e.py (a check’s condition removed
or loosened; 261 on the fixed code, 115 of the first 158 had
survived make lab-test) now all have a test: execute() end to
end with the VM and the flow stubbed (exit codes, NOTRUN, the
ledger, teardown, the summary line); one fault at a time in every
check from 0.5 to 5.1, on the text and facts of the passing run of
56f5359; the glue (boot()’s flag and limits, flow(),
wait_healthy_end, panicked, wait_for, wait_ssh, the
preflight). lab/test_e2e.py only, +1873 lines; make lab-test
196 -> 283, three runs in a row, 12 s each; go test ./cmd/sc/ ok.64911f4: test_wait_for_and_panics
expected a Retry from a panic and got None. 64911f4 had given
FakeRun a wait_for of its own (the fake console then had no
expect), which shadowed the real E2E.wait_for the test drives.
The new tests give FakeCon an expect, so that override is
removed and both tests run the real method; each still fails when
its code path is broken (the panic branch; the second command line).fcb1b6d (scratch-a/mutate.py -j 6 --fast
on copies of clone-a): 260 killed, 0 survived. The 261st,
“check_cmdline ignored”, no longer applies: 64911f4 rewrote that
text.make lab-e2e.a371738).
lab/vm.py, lab/console.py,
lab/serialmux.py and the guest scripts (clone-b/.rv/mut.py:
123 Python, 23 shell, of which 68 and 13 had survived) all have a
test now: 123 of 123 and 23 of 23 killed on a371738. The one
shell mutant that still passed, “hashes-after cached”, added a bare
: and changed nothing; it is replaced by the block dropped.SC_LAB_CACHE and XDG_RUNTIME_DIR point at temp dirs): the pinned
image, the key, the reference image (three 0444 files, the qcow2
last, a failed qemu-img check, the kernel pin, panics retried),
QEMU as a child (own session, PR_SET_PDEATHSIG, quit then
SIGKILL) with a stand-in script, Qmp on a fake unix-socket server,
SSH_OPTIONS, SeedServer on 127.0.0.1, running_qemu’s uid,
render_seed, fill_template, classify_boot, parse_menu,
pick, report_block, expect, the mux’s send timeout and wait;
install.sh’s step rcs and not-root refusal, facts.sh’s timeout
per part (the mode’s limit), a part’s stdin, a failing sc cat.
lab/test_lab.py and cmd/sc/unit_test.go only, +998 lines;
make lab-test 283 -> 325, three runs in a row, 15 s each;
go test ./cmd/sc/ ok; gofmt and vet clean.test_the_first_signal_is_the_
only_one (from 9d3df9e) is load-bound. With four CPU burners
beside it, 3 of 20 runs had the child hang past the 60 s limit
(SIGHUP twice, SIGTERM once); once, with go test beside
make lab-test, the SIGINT case ended “SystemExit 143” with no
teardown. The same child script run directly under the same load:
0 hangs in 120. Not understood yet; make lab-test alone passes.
Also test_the_loop_measures_its_wakes (from 152ecbe) failed
once in the repo, right after a go test loop: it expects the
late wake to measure exactly “0.2 s”. One go test ./cmd/sc/ of
14 failed right after three lab-test runs; its output was not
kept, and 13 later runs passed. In the repo after the merge:
make lab-test three runs in a row ok (22 s), go test ./cmd/sc/ ok.make lab-e2e on fc47ea3 FAILed; two fixes.
(Log: ~/smartconfig-work/signoff/lab-e2e-20261005T233510Z-fc47ea3.log;
runs 20261005T233511Z-uefi-fc47ea3, 20261006T001025Z-bios-fc47ea3.)
UEFI INCONCLUSIVE at 5.2 only; BIOS 5.2 the same and 2.5 FAIL. Every
other row passed, boot 2 outcome b, no retries.
0ccba27): the guest powered off, QEMU
sent SHUTDOWN (qmp.log +2110.886) and exited 0, the lab said “no
SHUTDOWN after poweroff (QEMU exited)” at once. The QMP reader runs
the listeners before an event is in events; e2e’s listener is
mux.mark, which since 9d3df9e drains up to 1 s, and the serial
loop was reconnecting (marks.log: SHUTDOWN#24 at +2111.96). In that
second wait_event’s abort (QEMU not alive) returned None. Now an
abort waits for the reader’s EOF (5 s at most) and looks once more.
Test: a fake QMP sends SHUTDOWN and closes under a slow listener with
abort true; it failed on the old code. make lab-test 326, three
runs; go test ./cmd/sc/ ok.c70d886): no line of the report on ttyS0 in boot 2.
Boot 2’s journal (the kept disk booted once through an overlay,
TCG, deleted after) shows the host pause: no entry between
monotonic 107.868 and 118.201, both guest clocks jumped 10.3 s
(mux.log “+545.63 gap 10.0 s”), and emergency.service’s job began
at 107.825, so sc status --console ran across it. writeConsole
measured the stall in wall-clock time; the slice over the pause,
with ttyS0’s queue busy with systemd’s flood, counted as a 10 s
stall and ttyS0 was given up with nothing written (tty1 took the
report, so no stdout fallback). Ruled out on a pty: Write gives
ErrDeadlineExceeded, not EAGAIN, and the old writer waited the
whole 5 s. Now each slice counts at most consoleStep (500 ms)
towards stall and turn: one pause is one slice. Test
TestWriteConsoleHostPause failed on the old writer (0 of 1260
bytes). make build fmt vet test race, the root pass, make
lab-test ok. Not changed: the report text, the goldens.make lab-e2e on c70d886.make lab-e2e PASS on 22383a0, both
modes; write-up and plan change C6.
~/smartconfig-work/signoff/lab-e2e-20261006T011750Z-22383a0.log):
UEFI PASS in 24m29s, 0 retries, boot2=b goal3=multi-user, boot 5,
dirty=no, through host pauses of 167 and 318 s. BIOS INCONCLUSIVE
at 5.2 only: the lab’s cat boot_id over ssh timed out (180 s)
across a 384 s host pause; every [M4] row passed.LAB_MODES=bios on the same tree (log ...T020514Z-22383a0-bios.log):
BIOS PASS in 45m14s, 2 lab retries, boot2=b goal3=multi-user,
boot 5, dirty=no. Boot 3: a menu-missed retry. Boot 5: the first
real stall retry, the kernel at smpboot for 900 s;
evidence/stall-5-1.txt shows mux gaps of 250, 364, 400, 400 and
265 s and a 5 s CPU sample spanning 265 s: the host stopped this
VM for most of 24 minutes. Neither retry is sc’s. Both runs on the
same commit and the same sc (a0502d6461db); evidence in
~/smartconfig-work/signoff/lab-e2e-22383a0/.docs/reviews/2026-10-05-m4-chunk-d.md: the 29 findings
and item 6 with their fix commits, the three verification runs,
the mutation work (261 of 261 scenario, 123 of 123 and 23 of 23
machinery, the check_decision notes bug), and the open items:
the host’s pauses, an ssh retry across a pause, systemd’s lines
missing on ttyS0, the mux’s 1 s drain, two load-bound tests, one
unkept go test failure, facts.sh pipe statuses (B14), the
hangup window before signal.Notify (B5), and the README’s and
the registry’s stale “2.5 is a W in c”.docs/M4_PLAN.md): the 60 s limit of
sc status --console, the console writer’s turn (5 s stalled,
10 s at most, a host pause one slice), the login line in the
emergency report, the hangup flag from main; replaces C4’s “no
time cap” and chunk C’s 2 s write.scripts: accept-m4.sh, not
built yet (build it in the repo with its checks; its run as root
needs the user’s OK, sign-off S1).4ce59a3: lab/README.md and the registry text in
lab/e2e.py say F in every outcome, as check_emergency_report
does; make lab-test 326 ok three runs, go test ./cmd/sc/ ok).
This worklog’s “Now” corrected: step 12 is not built yet, and the
30-minute check-in does run.scripts/accept-m4.sh (3e72281), make accept-m4:
the parts of M4 that need no reboot, as root on this VM, bin/sc on a
throwaway SC_HOME, nothing real written. The two boot units as
runtime units with a copy of grubenv bound over the real one (seen and
the flag, then this boot’s verdict, which must be ok, and no flag);
sc status and --console on a made-up fstab with a missing disk,
recorded from a private mount namespace; restore’s chattr +i/+a
refusals; a read-only store on a tmpfs and a copy with a hot journal;
grub-mkconfig on copies of /etc/grub.d with and without
42_smartconfig, in private mount namespaces; sc status on the real
store; the real files’ checksums. Checked as a user: bash -n, the
status, read-only and hot-journal sections against bin/sc (boots lines
written by hand, the fstab row made in a test database: unprivileged
user namespaces are off on Ubuntu 24.04), the grub.cfg checks on this
machine’s kernel and root, both runtime units through systemd-analyze
verify, make build fmt vet test. Not run as root: that is S1 and
needs the user’s OK. This VM had gone down again before this session
(about 11:30 UTC, idle, last -x: crash); the check-in was recreated.make accept-m4, lab-test and lab-e2e;
a section “When the machine does not boot (M4)”: what the rescue
path gives, the rescue report as the lab’s UEFI run of 22383a0
printed it, what to type, the emergency-mode login prompt and the
rescue shell’s quiet console (chunk D item 6), the install by hand
and its removal, sc status’s exit codes, sc on a read-only root,
the chattr refusal, where it does less, and the security note with
a GRUB superuser recipe (--unrestricted on 10_linux’s default
entry only), marked as not tested in the lab yet.internal/boot and the scripts in the layout.6ce3263..8f071ed). Two read-only reviewer agents, no root,
no QEMU: A on accept-m4.sh, B on the docs. Reports in
~/smartconfig-work/review-m4e/ (their harness refused the file
writes; the session saved the returned text unchanged). Write-up
reviews/2026-10-06-m4-chunk-e.md.
GRUB_SAVEDEFAULT, GRUB_DISABLE_SUBMENU or ZFS, and its check did
not show which entry boots. Now: use it only with GRUB_DEFAULT=0
alone, check the one marked line, do not reboot otherwise, how to
undo; the conffile prompt; what the password closes.sc status gave the undo for the error’s file. 654ee7f:
the newest blocker, else the newest error (plan 3.1), and the menu
promise only while its flag is set (B8); “60 s” for “1m0s”.4b91d97: RemainAfterExit=yes, plus the real store read
read-only in a namespace (A2), grubenv compared byte for byte, the
made units checked, os-prober off through /etc/default/grub.d, a
sturdier cleanup, exact initrd, wider checksums, a bad verdict with
a fake systemctl (2b).go test ./... as a user and as
root, make race, make lab-test (326); the new tests fail on the
old code; the script’s new parts replayed as a user.accept-m4.sh as root, and make lab-e2e on the fixed
build: both are S1, which needs the user’s OK.c70d886 (2026-10-06 00:15 UTC),
unnoticed: every red run failed only
test_an_event_sent_before_qemu_exits_is_not_lost_to_abort (lab,
from 0ccba27), in CI’s root pass. The test asked whether QMP had
closed right after wait_event returned the SHUTDOWN, which can be
before the reader reads EOF: a race in the test, not in the lab.
Reproduced here 1 in 25 beside four CPU burners; 0a1b71f asks after
the reader’s EOF: 0 in 25 under the same load, make lab-test 326 ok,
CI green on 0a1b71f. Lesson: check gh run list after each push.go test ./... as a user and as
root, make race, make lab-test (326), CI green (0a1b71f), and
make m1-compat.make accept-m4 as root on 19f2200: PASS, the script’s first
root run (~/smartconfig-work/signoff/accept-m4-20261006T142311Z-19f2200.log).
The boot units took 19 and 35 ms; this boot’s verdict is ok
(failed-units=0); the bad verdict kept the flag; sc status and
--console gave the blocker and sc restore of the good version;
the chattr refusals; the read-only and hot-journal stores;
42_smartconfig under grub-mkconfig adds only its part, and this
machine’s grub.cfg is what grub-mkconfig makes today; this machine’s
store, read-only: no problem in its 3 newest files, exit 0. Nothing
left behind (no runtime unit, mount, work dir or private dir; the
real grubenv has no flag).make lab-e2e starts on the commit of this entry.make lab-e2e on 2c58f8b (clean tree, sc cafca1512050):
BIOS PASS in 44m18s (0 retries, boot2=b, boot 5). UEFI was
INCONCLUSIVE at P.5 in 1 s: ovmf was upgraded by apt at 06:34 UTC
today (2024.02-2ubuntu0.9 to 0.10), after the reference image was
made, and the lab refuses another firmware under the old variables.make lab-image did nothing: the chunk D fixes (9d3df9e) had
dropped its rule, and plain provision would have kept the image
anyway (its name does not depend on the firmware). So, without a
commit (one build for both modes): vm.py provision --force, 216 s,
then UEFI alone on the same commit: PASS in 37m19s (0 retries,
boot2=b, boot 5). Evidence in
~/smartconfig-work/signoff/lab-e2e-2c58f8b/; logs
lab-e2e-20261006T142402Z-2c58f8b.log and
lab-e2e-20261006T151513Z-2c58f8b-uefi.log.b9761a9: the lab-image rule back, with LAB_FORCE=1 for an
ovmf update, in P.5’s advice and the lab README; checked by running
it and make lab-test.~/smartconfig-work/signoff/pre-s2-20261006T194841Z.txt):
boot 2c1e65ee, up since 19:37 UTC; the boot before it ended at
17:38 UTC without a shutdown (last -x: crash), whether your
power-off for the snapshot or the host is yours to say. BIOS, GPT.
Boot times of the last seven boots 12.4 to 33.7 s (this one 28.8 s).
grubenv empty, GRUB_DEFAULT=0, hidden menu, timeout 0; no failed
units. New in /var/backups/smartconfig: sc-m3 (cf5ba078, the
build scd ran since M3), manifest-pre-m4s2.txt and
sha256-pre-m4s2.txt (/etc and /boot/grub).bin/sc as built for S1
(cafca1512050, 2c58f8b, clean; no code or script change since),
scd restarted on it (baseline 0/0/0); both units enabled,
systemd-analyze verify clean; both drop-ins; 42_smartconfig,
update-grub exit 0, grub-script-check ok. grub.cfg: the default
is still entry 0, Ubuntu; “SmartConfig rescue” last, on
7.0.0-38 with ro fstab=no systemd.unit=rescue.target
SYSTEMD_SULOGIN_FORCE=1; the menu block only under
smartconfig_pending=1./etc and /boot/grub against the manifest differ by
exactly grub.cfg, 42_smartconfig, the two units, their two links
and the two drop-ins with their directories; scd recorded each.
sudo sc status: no healthy boot recorded yet, exit 0.~/smartconfig-work/signoff/post-s2-20261006T203734Z.txt).
systemd-run --on-active=120 systemctl reboot, clean
shutdown at 19:54:39 UTC (boot 2c1e65ee).276489a3): ok. Verdict ok (local-fs active, no
emergency or rescue, 0 failed units), grubenv empty after it.
23.5 s to “Startup finished” (graphical.target at 22.6 s
monotonic), inside the 12.4-33.7 s of the seven boots before.
sc-boot-seen done at 5.2 s; sc-boot-ok starts after
multi-user.target, 0.6 s.last -x). The next
boot (75d27954): verdict ok, 38.1 s; you logged in at 19:57:44 and
DING (desktop icons) started 24 times in 90 s (2 on a normal boot),
the black-desktop symptom of M2’s S4; the journal ends 19:59:23, no
shutdown (last -x: crash). The boot after it (94783975, now):
verdict ok, 37.6 s, DING 2, you logged in at 20:30.grub-editenv fsyncs
grubenv (strace on a copy: O_TRUNC, write 1024, fsync), so each
ok verdict was on disk before the reset.sc-boot-seen 847 ms,
sc-boot-ok 521 ms, scd 107 ms in blame. graphical.target came
at 22.6, 37.4 and 36.7 s monotonic on the three boots (12.1 to
33.6 s on the six before); the extra in the last two is
plymouth-quit-wait (13.7 s) and NetworkManager, after an unclean
end each./etc and /boot/grub against manifest-pre-m4s2.txt: the S2
files only, and cups/subscriptions.conf{,.O} (cupsd; out of
scope, as in M2’s S4). scd running, 0 restarts; sc status: last
healthy this boot, nothing changed since, exit 0. No failed units.grub-common.service at 10.2, 15.3 and 14.7 s, before each reset
(as in the lab, A6); you reported no menu.~/smartconfig-work/signoff/pre-s3-20261006T210624Z.txt):
boot 94783975 ok, grubenv empty, /etc/fstab 446 bytes, sha256
9d71ab60…, the same as its newest row fb5cf4; sc status
healthy, exit 0. Checked first on a copy: sc check --as
/etc/fstab gives the blocker, exit 2.UUID=3f6c1e2a-9b7d-4c1e-8f2a-5d6e7f8a9b0c
/mnt/backup ext4 defaults 0 2, sha256 1ab60193…. scd at once:
row 053fdf, “check: blocker fstab-source-missing, line 10”; sc
status lists it and the undo sc restore fb5cf4, exit 2. No
daemon-reload: the owner reboots.systemd-run --on-active=120 systemctl reboot);
you drive the console.~/smartconfig-work/signoff/pre-s3-rescue-20261007T055831Z.txt):
two boots after 94783975 (ended 21:09): 928e5ac3 (21:16 to
01:20, then gone without a shutdown, last -x: crash, the host
again) and f11e5522 (05:41). grubenv smartconfig_pending=1;
/etc/fstab still the bad one (sha256 1ab60193…); sc status:
this boot f11e5522 (emergency), root read-write, scd running;
systemctl is-system-running: maintenance; the desktop came up all
the same (both boots). No rescue boot yet.~/smartconfig-work/signoff/post-s3-20261007T062818Z.txt, photos in
~/smartconfig-work/signoff/s3-photos/).
928e5ac3, f11e5522): verdict bad both
(local-fs=inactive emergency=active, newest row 1566, the bad
edit). Each waited 90 s for the missing disk (the device timed out
at 92.4 and 93.9 s), reached emergency.target at 95.2 and 94.8 s,
and started gdm about 1.5 s later: on this desktop the login screen
comes up over the emergency shell and the report printed there (the
tty1-versus-gdm question of plan section 7: gdm). 1 min 42 s and
1 min 40 s to “Startup finished”. f11e5522 ended with a clean
reboot at 06:14:05.7ce2c935, your photos): “SmartConfig rescue” from
the menu, and the report above “Press Enter” as in the README: this
boot (rescue), root read-only; last healthy 94783975; failed since
2 boots, “a mount failed, emergency mode”; scd not running; 053fdf
blocker fstab-source-missing, line 10; the five commands; “The menu
shows once more”. You ran mount -o remount,rw / and sc restore
fb5cf4: “restored /etc/fstab from fb5cf4 (mode 0644 root:root),
previous state saved as 8f7c6e”; then the rest. As designed it has
no seen line and no verdict (/var/lib read-only), and no journal
(volatile on a read-only root); last -x has its shutdown.8438a7f4, Ubuntu’s entry by its command line):
verdict ok at 9.9 s (local-fs=active, newest row 1568, the
restore), grubenv empty after it, 9.95 s to “Startup finished”, no
failed units, is-system-running running. /etc/fstab 446 bytes,
0644 root:root, sha256 9d71ab60… (= fb5cf4). sc log
/etc/fstab: 8f7c6e pre-restore (518 bytes, the bad file) and
c8c9bd “restored from fb5cf4”. sc status: healthy this boot,
nothing changed since, exit 0. scd running, 0 restarts; sc still
cafca151….[FAILED] Failed to start grub-initrd-fallback.service
right above the report (photo). Ubuntu’s unit is in
rescue.target.wants on this install (not in the lab’s cloud
image) and runs grub-editenv … unset initrdfail, which cannot
write on the read-only root. Harmless: the boot after ran it ok.
The README’s rescue section could say to expect it.sc boot seen ran twice in each failed boot (2.3 and 5.0 s; two
seen lines in boots), once in every normal boot and in the lab’s
failed boot. sysinit.target waited 90 s for local-fs there; a
second start job for it in that window starts the finished oneshot
again. Harmless here: both before the verdict, and sc status
counts boots by id (“2 boots”). Whether the unit wants
RemainAfterExit=yes is for the final review.maintenance). The next boot’s menu is the way
in, as plan section 8 says (“M4 does not depend on emergency
mode”); the README could say so.m4), with your OK.~/smartconfig-work/review-m4final/findings-{a,b,c}.md.
81319d8: A boot
integration, B the rescue-time CLI and its data, C security and the
docs. Read-only here, no QEMU; the S2 and S3 journals read as root.69ea585: a stopped sole console held sc status --console, and
the shell, past systemd’s 90 s (the fallback write to stdout,
the same console, had no limit). Now stdout only when no console
opens, and one 80 s deadline for every write.4a8f7b4: an fstab first seen after the line (scd’s baseline,
no healthy boot yet) was “new”, and the undo moved it aside;
without a healthy boot a blocker in the oldest row listed was
missed. Now new only with proof of absence, else compared with
the first version, else “fix it by hand”.8c6f30e (A1, C2): “exit” in the rescue shell after the remount
gave an “ok” verdict that cleared the flag and hid the next
undo. Both units now need a command line without fstab=no. A
whole-boot systemd-analyze verify test came with it.580edca (A2): the rescue entry keeps the default entry’s options
but quiet splash (a nomodeset was lost).734b670 (B3, B4): a deleted flag file is no error; no store in
rescue says to mount /var, not sc init.580edca masks Ubuntu’s grub-initrd-fallback in the rescue boot:
S3’s red FAILED line.03ec9ca: a boot with a verdict is not seen again; the flag is set
before the seen line, and by a bad verdict; the boots file ends a
torn line before the next, skips one unended, and refuses a
symlink.d9c8817: the 60 s stop removes the /run copies; the undo’s
paths are shell-quoted.grep -s; removal order.go test ./... as a user and as
root, make race, make lab-test (326) where the lab or scripts
changed, make m1-compat. Every new test fails on the code before
its fix. CI green on each.make lab-e2e on the fixed tree (next, no sudo).accept-m4 as root, the install, the tag: your word.make lab-e2e on 87c6636 (log
~/smartconfig-work/signoff/lab-e2e-20261007T091132Z-87c6636.log,
run ~/.cache/smartconfig-lab/runs/20261007T091134Z-uefi-87c6636/)
stopped by SIGTERM to its process group after 20 min, in UEFI mode:
INCONCLUSIVE, “interrupted (signal, exit 143)” at 3.5.fstab=no. Not a sign-off run; it is to be
rerun.~/smartconfig-work/review-m4final/.m4 (your OK, 13:20 UTC,
“yes, go ahead after it passes”; then “i don’t need snapshot, go
ahead” and “don’t wait start the work. Approved”).
c95d3ab6); HEAD and
origin/main both 59203a3, clean; no QEMU; CI green.make lab-e2e on 59203a3, detached, LAB_REQUIRE_CLEAN=1 (log
~/smartconfig-work/signoff/lab-e2e-20261007T131334Z-59203a3.log):
dirty=no, sc
1dce09f5ff12 (run 20261007T131337Z-uefi-59203a3).[lab] ssh exit 255 while
install.sh ran (24.2 s; the lab’s ssh gives up after about 15 s
without an answer). The guest’s console was quiet at its login
prompt, no panic or reboot. This VM’s desktop was busy in that
minute (Chrome: WebGL and H.264 encoding in the journal). Not sc
(run 20261007T133312Z-bios-59203a3).lab-e2e-20261007T133745Z-59203a3-bios.log): PASS, 19m59s,
boot2=a, boot 5, 0 retries, dirty=no, the same sc (run
20261007T133747Z-bios-59203a3).~/smartconfig-work/signoff/lab-e2e-59203a3/ (both
modes, and the inconclusive BIOS run as bios-inconclusive-*).sudo sh ~/smartconfig-work/signoff/step3-m4-59203a3.sh
(log step3-m4-20261007T142427Z-59203a3.log). It runs
scripts/accept-m4.sh itself, not make accept-m4 (which rebuilds
bin/sc), and checks before and after it that HEAD is 59203a3,
the tree is clean and bin/sc is 1dce09f5ff12.
~/.config/git/ignore, which ignores .claude/settings.local.json.
The script now runs git as the user.accept-m4: PASS, exit 0. Step 7 noted that this machine’s
grub.cfg was not what grub-mkconfig made today (the S1
42_smartconfig was still installed); step 8: healthy, exit 0.c95d3ab6, scd pid 903 with 0 restarts, both units
enabled, grubenv empty, no failed units, /usr/local/sbin/sc the
S1 build cafca151…. New in /var/backups/smartconfig:
sc-2c58f8b (that build), manifest-pre-m4inst.txt and
sha256-pre-m4inst.txt (/etc and /boot/grub).update-grub./usr/local/sbin/sc is 1dce09f5ff12; scd restarted on
it (pid 13382, 0 restarts); both units enabled, systemd-analyze
verify clean; grub-script-check ok. grub.cfg: Ubuntu still
entry 0, “SmartConfig rescue” last, on 7.0.0-38 with
systemd.mask=grub-initrd-fallback.service ro fstab=no
systemd.unit=rescue.target SYSTEMD_SULOGIN_FORCE=1; grubenv empty./etc and /boot/grub against the manifest: exactly grub.cfg,
42_smartconfig and the two units (the drop-ins were already
these). scd recorded the four. sudo sc status: healthy, exit 0.
No failed units.m4 is on 59203a3, the commit the installed
build was made from, and pushed.1246469 (chunk F). A new
checker, unitdropin, for /etc/systemd/system/NAME.d/*.conf.
systemd-analyze verify loads NAME by name, with a scratch directory
first on SYSTEMD_UNIT_PATH, where the candidate takes the place of
the drop-in of the same name (probed on this VM, systemd 255: the
scratch copy wins, and a trailing “:” keeps the stock path).
ExecStart= without the empty one before
it is the drop-in’s; a problem the unit had before is not.sshd.service.d on Ubuntu)
systemd names the unit ssh.service, so the alias is followed
through the unit path’s symlink; a template’s drop-in is said twice
(template and instance), now one finding. verify checks only the
first command of each Exec… list (systemd’s limit): an added
ExecStartPre= after one with - is not looked at.unit-dropin-orphan (warning). A prefix
drop-in (foo-.service.d) gets a note, not a check.TestUnitDropIn (11 cases, a fake verify that answers by
the drop-in’s size), TestUnitAlias, TestUnitDropInRealVerify
(journald, real systemd). 8 mutations of the new code, each caught.
accept-m3.sh has three new cases (run as a user here with this
build: as expected; as root at the end, with your run).go test ./... as a user and as root, make
race, make m1-compat. The M4 rescue drop-ins on this VM check
clean with the new build.ListenAddress, done e9133ca (chunk F). A new
rule of sc’s own, sshd-listen-missing (warning): a ListenAddress
whose literal address no interface here has. sshd -t binds nothing
and passes it.
ssh.socket,
which has FreeBind=yes, so the bind does not fail at boot; SSH just
takes no connections there, and none at all when that is the only
address. The review’s “gone at the next boot” holds for an sshd that
binds for itself.TestSshdListen (12 cases), TestSshdListenReal (every
address of this machine, then 192.0.2.7). 11 mutations, each caught;
the one that first survived (Unmap) got a case.TestRulesTable: the explanation had six
lines (at most five, for the console). Shortened; all checks then
passed: gofmt, vet, go test ./... as a user and as root, make
race, make m1-compat.ef2989d (chunk
F). sshd -t now reads the drop-in where sshd does: in a scratch copy
of sshd_config whose Include names a scratch copy of the drop-in
directory (the candidate in place of its namesake).
AuthorizedKeysCommand in a
drop-in with its user in sshd_config was a false blocker alone and
is clean now. A Match a drop-in leaves open ends with the drop-in,
and a second Subsystem sftp passes: no 9.6 setting makes another
file’s line fail, so that path is tested with a fake sshd.sshd_config does
not include it, a file cannot be read, or another file stops sshd.TestSshdTogether (real sshd on fake /etc/ssh trees, 9
cases), TestSshdTogetherOther; 9 mutations, each caught.go test ./... as a user and as root, make
race, make m1-compat.6fb78f0, fixed in 762909e. CI’s root pass
failed TestSshdTogether: its runner has no /run/sshd, so sshd -t
as root adds a note to a clean run, and the case that falls back to
the drop-in alone is one. This VM has /run/sshd; a reproduction in
a private mount namespace was refused by Claude Code’s safety check, so
CI was the test: green on 762909e.56c5d31. The race pass of follow-up 4’s checks failed
TestSudoersVisudoBroken (“run visudo: context deadline exceeded”):
with other tests running alongside, more than its 200 ms passed before
the start, and exec refuses a done context. Now that is a TimedOut
result, a note like any timeout; TestRunTimeoutBeforeStart (1 ns)
gave the error before. Checks are run alone from now on.c77b060
(chunk F). A note is sc’s sentence; what a validator said behind it
(which may quote a file) is Report.Said: sc check -v prints it,
without -v a line says -v shows it; sc edit prints it under the
note. About 15 notes changed in 9 checkers.
Said; the sshd
together check keeps the first run’s lines only when it keeps its
result (a test with a fake sshd says them once).notesSaid in the check tests (the old notes’ raw text is
asserted as said lines), TestCheckCLISaid, TestEditSaid; 8
mutations, each caught (one only after a test was extended).go test ./... as a user and as root, make
race (uncached), make m1-compat.84747b8 (follow-up 4), fixed in a43a7bf. CI’s root pass:
when the sshd check inside sshd_config needed no second run, sshd’s
words behind a note (“did not finish checking”, no /run/sshd on the
runner) were not handed to Said. A real bug, which this VM cannot
show (it has /run/sshd); TestSshdTogetherSaid (a fake sshd) failed
before the fix. The check package passed as a user, as root and under
race; CI is the full check.aca4ab5
(chunk F). The validator has its own process group, which a Ctrl-C at
the terminal does not reach, and sc ended by the signal without its
cleanups: a hung validator ran on, its scratch copy stayed an hour.
check.Stop kills every validator group not yet reaped and removes
every scratch directory; sc’s signal handler calls it where it removes
its temp files (scd at its second signal only). A group leaves the set
before it is reaped, so its id cannot be another’s.
TestSignalStopsValidator (SIGINT and SIGTERM to sc check,
a fake findmnt with a child: both gone, no scratch, sc dies by the
signal within 3 s), TestStop (running, scratch, started after).
4 mutations, each caught. Full checks passed.8923fd7 (chunk F).
checkPath listed the path’s whole history (0.1 s at 20,000 rows) to
find two rows. Store.Around gets them by index search: the change by
its id (found while writing it: ids are unique, a UNIQUE index, so the
old loop’s handling of a repeated id was never needed), and the row
before the first queued change through AsOf. TestPathQueryPlans
asserts both SQLite plans (no scan, no sort); TestAround; 5
mutations, each caught. Checks: gofmt, vet, go test ./... as a user
and as root, make race (uncached), make m1-compat.c4c4b2b (chunk F; all of chunk F’s
follow-ups are now in: 1-5, 7, 8; 6 was done in M4).
/etc/default/grub: dash puts an unterminated quote at the end of
the file, past its last line. sc now names the line that leaves a
quote open read on its own (with the next line’s quotes read as one
file, the mistake seems a line later), unless the next such line
closes it (a value over two lines). The real dash’s test: line 2,
not 4. “end of file unexpected” goes on the last line./etc/fstab: the captured findmnt output shows a heading per line
in the file’s order (two “none” for two swap lines), not one per
mount point: the k-th heading is the k-th line when each has one;
else a message naming one line’s source picks it.TestShQuoteOpen, TestFstabHeadings; 7 mutations, each
caught. Full checks passed.4f90bbc in
~/smartconfig-work/review-m3fu-f/): A the internal/check changes, B
the CLI, signals, scd’s checker, accept-m3.sh and the docs. No code
change until it is closed. Your picks for chunk G and M4 4 are in “Now”.cfb704e). Two reviewers on 4f90bbc: A (internal/check) 8
findings, one high: a drop-in for a template alias (autovt@.service.d,
getty@’s alias on Ubuntu) was not checked, so the autologin drop-in
without its ExecStart= reset passed; B (the CLI, signals, scd, docs)
nothing high, the worst a stop signal that left a scratch copy when
sc check had more files (8 of 20 runs). All fixed, with a test each
(each failing tests the reviewers wrote reran first); 28 of 30 fixes,
undone one at a time, made a test fail (two are reasoned only, named in
the write-up). Full checks passed (race uncached).
sc status and restore), with your picks in “Now”.0d22229
(chunk G, your pick: refuse). sc restore writes nothing when the
path is under the deepest /etc/fstab mount point that the mount
table does not list (a separate /boot in the rescue shell), and says
mount /boot first, exit 1; without an fstab or a mount table it goes
ahead. Tests: TestRestoreUnmountedMount (refused, file unchanged;
mounted, restored), TestUnmountedMount (the deepest point, escaped
names, no mount table). Full checks passed.17f4a34 (chunk G).
sc status calls it an error (“now a symlink to X, not checked”) with
the healthy version’s restore; a link that was a link stays as it was.
TestStatusFileNowLink, which fails without the change. Full checks
passed.4adc132 (chunk G). The
next append ends a fragment a crash left with “ #torn”, and Read
skips it: a verdict cut in its row id can no longer pull the healthy
boot’s line back. Old boots files have no mark and read as before.
The new case in TestReadTornLines failed before. Full checks passed.284c381 (chunk G). The console’s
rows have their date (the lab’s goldens and test reports too); the
row -1 healthy boot has its own title; sc check and sc status make
no $SC_HOME/tmp; a WAL store on a read-only root gets sc’s own
“sc needs delete” (a copy was tried and dropped: sc refuses WAL
anyway); FS_IOC_GETFLAGS is only for the ports sc builds for (mips
and sparc were never built: the review’s case cannot happen). Tests
for each; make lab-test 326 and the full checks passed.845b395 (chunk G; the chunk E review’s
note). sc status puts mount <point> in its undo when the file is
under an unmounted mount point of fstab (a separate /boot), as sc
restore now refuses there. TestStatusUndoMounts. Full checks
passed (a first run’s root pass used the system’s Go 1.22, as the path
to the toolchain was taken outside the repo; rerun right).7fdb494 (chunk G, your pick for the
chunk C review’s note). A rescue report not ready after 2 s shows its
header (This boot, Last healthy, Failed since, scd), then the rest or
the stop. Not at once, as asked: a report in time stays one write,
which other console lines cannot cut, and the M4 lab matches it whole;
a hung one still shows the header long before the 60 s stop.
TestConsoleStatusHeaderFirst. Full checks passed.32cc097 (chunk G, your
pick for the chunk B review’s note). scd is Type=notify and sends
READY=1 once its startup rescan is recorded and logged; sc-boot-ok
is ordered after scd.service. scd spells out systemd’s default
dependencies but the target’s After=, so boot does not wait for the
rescan; a rescan that hangs fails the start after 5 minutes. An older
sc under this unit would never be ready: the README says to install
sc first. TestReadyAfterBaseline, TestSdNotify, the unit tests.
Full checks passed; the first run’s root pass lost one fake
systemd-analyze to a VM pause (11.6 s against the 10 s limit, 3 of 3
reruns pass), so fake machines get a minute, b7a89e6.2ae67e8 (chunk G, your
picks: M4 follow-up 4 and the chunk D note stay as limits). The README
says a soft-reboot is the same boot to sc (systemd 255 has no count of
them), so a failure in it brings no menu; the SIGHUP window (3 to 5 ms
before signal.Notify) was already said in main, and the rescue
shell starts even if the report dies. Docs only. Chunk G’s items are
all in; next, its review.0a55306. Two reviewers in clones of 0fcabd1: A (boot integration)
found 2 medium real-machine bugs (scd never ready while a startup path
is held back: under the free-space floor or with a store error systemd
restarted it every 5 minutes and each verdict waited 5 minutes; READY
before a startup path an event touched), 2 that would have failed the
sign-off (accept-m2 step 18 on sc-boot-ok’s After=scd.service; lab
2.5 when the getty comes between a slow report’s header and its rest),
the late header and a torn line at a trim. B (status and restore) found
low ones: a covered mount taken as mounted, nested mount points, a
symlinked mount point, the console’s symlink row without “error”, the
row -1 boundary’s label, fstab blocking its own undo, a file mount
point, two titles, and test gaps. All fixed, each with a test; 22 of 22
fixes undone one at a time fail a test. Full checks passed (the race
pass once lost test_the_loop_measures_its_wakes, the load-bound lab
test chunk D noted, a chunk H item; the rerun passed). Chunk G is
done; chunk H is next. accept-m2 joins the sign-off runs
(scd.service changed).GRUB_TOP_LEVEL, done af23c2e (chunk H). The rescue
entry boots the kernel GRUB_TOP_LEVEL names, as 10_linux puts it
first, when it has an initrd; else the newest with one, and a word on
stderr. TestGrubScriptTopLevel (fails without the change). Full
checks passed.8c5cc40 (chunk H). Both
boot units: NoNewPrivileges=yes, ProtectHome=yes; sc-boot-seen
also PrivateNetwork=yes (sc-boot-ok asks systemd over its socket).
Ubuntu’s sysinit units (systemd-resolved, -timesyncd, nftables)
set up such namespaces this early. 42_smartconfig takes no
SC_GRUB_BOOT; the tests rewrite its boot=/boot line. Checked by
systemd-analyze verify and TestBootUnits; that a broken boot still
gets its seen line with them is for the lab run after chunk H. Full
checks passed.270c3c5 (chunk H). TestBootUnits
refuses sc-boot-seen ordered after sysinit.target/basic.target
and any binding or conflict on sc-boot-ok;
TestStatusConsoleCapWarnings pins the 5-file cap with room to spare
and no undo for warnings. Each of the six mutations (four unit lines,
the cap, the undo’s severity) and the row -1 fallback’s fails a test.
Full checks passed.f0bf189 (chunk H; the final review’s A8).
facts.sh normal prints sc-boot-seen’s Before= (seen-before);
1.8 fails on a target in it but shutdown.target, or an unread one.
This VM’s installed unit: grub-common.service shutdown.target
sc-boot-ok.service grub-initrd-fallback.service. Lab tests for each
case; undoing the check fails one. Full checks and make lab-test
passed.b092a79 (chunk H). A read over ssh
that ran out of time while the host stood still runs once more; a
mark does not wait out its drain while serial reconnects; facts.sh’s
filtered parts keep the command’s status; the wake test takes any late
wake. Explained, no change: systemd’s lines missing on ttyS0 (v255
turns its console output off once a Type=idle unit stops waiting
while a unit has the console: emergency.service and its shell). Not
reproduced: the signal test’s hang (0 of 135 runs beside four CPU
burners). A test for each change, each failing without it. Full
checks and make lab-test passed (a first run stopped at gofmt on
the test’s stubs).818dace (chunk H; the
chunk E note). --grub-password adds checks 6.1-6.5 after boot 5: the
README’s recipe scripted as root (one --unrestricted line,
Ubuntu’s), the default boot asks for nothing, the rescue entry from
the menu asks for the user and the password (serial in UEFI, the VGA
screen and qcodes in BIOS) and boots with them (fstab=no, its shell
reboots), Ubuntu from the menu asks for nothing and is ok. Lab tests
for the recipe’s judging, the two prompts and the summary tag; full
checks and make lab-test passed. Then make lab-e2e
LAB_E2E_ARGS=--grub-password on 818dace (clean tree), log
~/smartconfig-work/signoff/lab-e2e-20261007T212317Z-818dace.log,
run files in signoff/lab-e2e-818dace/: PASS both modes, UEFI
27m03s (outcome b), BIOS 26m06s (outcome a), 0 retries, grubpw=yes.
It also ran chunk H’s hardened boot units through the broken boot 2
(seen line and menu as before) and the new check 1.8 (seen’s
Before= has no target but shutdown.target). The README now says
the lab runs the recipe.cmd/sc flake, found and fixed e37758b (chunk H; the chunk D
note). A hunt (~/smartconfig-work/fu-scratch/h7/hunt.sh: three
make lab-test, then go test ./cmd/sc/, ten times) failed at once,
because it ran under nohup: SIGHUP ignored is inherited, sc keeps an
ignored hangup (by design), and TestConsoleStatusOutlivesHangup,
TestWatchSIGHUPRescans and the lab’s signal test (via
TestLabPython) failed every time. The earlier unkept failure fits
that. TestMain un-ignores SIGHUP in the test process, so children
start with the default; the lab test’s child resets it. Under nohup
those tests pass now, and the hunt then ran 10 of 10 clean. Full
checks passed. Chunk H’s items are all in; next, its review.a1c1228. Two reviewers in clones of 100da93, no high finding
and no sign-off false pass. A (GRUB script, units, tests): the units’
comments (the early-namespace precedent was wrong; PrivateNetwork=
keeps file-system sockets), a blocklist test that let ordering and
sandbox keys through, .old kernels. B (the lab): four medium ones in
the --grub-password stage, each a failure in the wrong class (a
prompt in 6a/6c a [lab] timeout, 6a’s verdict not awaited before the
flag, a retried 6c expecting a menu, 6.4’s waits outside ran_out),
and low ones (a dead VGA watch, 1.8’s docstring, facts.sh dropping
output, 11 surviving mutations, the README). All fixed with tests;
28 of 28 fixes undone one at a time fail a test. Full checks and
make lab-test passed. Chunk H is done; next, the sign-off runs:
make lab-e2e LAB_E2E_ARGS=--grub-password, then accept-m2,
accept-m3, accept-m4 and the install as root.526f14b: INCONCLUSIVE twice, not yet
passed (2026-10-08). make lab-e2e LAB_E2E_ARGS=--grub-password,
logs ~/smartconfig-work/signoff/lab-e2e-20261008T000839Z-526f14b.log
and lab-e2e-20261008T005326Z-526f14b.log. No [M4] failure. UEFI lost
its verdict both times to the host (ssh exit 255 during host stalls of
12-181 s; then a slow-udev retry and no kernel in 600 s; this VM had
about 0.6-1 GB free, no swap, load 16-25, snapd busy). BIOS failed
both times at the same place: 6.2’s cat
/proc/sys/kernel/random/boot_id over ssh, exit 255 after about 31 s,
right after 6.1 (the recipe and update-grub). In 818dace the same
step passed; this looks systematic, not load: next, read that run’s
serial and ssh logs (~/.cache/smartconfig-lab/runs/*bios-526f14b),
fix, rerun. Waiting on it: the root steps
(~/smartconfig-work/signoff/signoff-fu-526f14b.sh: accept-m2/m3/m4
and the install; its WANT is this build) and the reboot. The M5 plan
draft is in ~/smartconfig-work/fu-scratch/m5/M5_PLAN.draft.md.1cadd8c. Why bios lost 6.2
twice: right after the recipe’s update-grub, ssh to the busy guest on
the loaded host got no answer to 3 keepalives (ServerAliveInterval=5,
15 s) and gave up, exit 255, its message hidden by LogLevel=ERROR;
the path itself is the same as in 818dace (which passed), so this
is the environment. ServerAliveCountMax=12 (60 s; each call keeps its
own timeout), and boot_id (a read) runs once more after ssh’s own
failure. Lab tests for both, each failing without its change. Full
checks and make lab-test passed. Next: the sign-off lab run again./usr/sbin/sc; the store kept on purge; the hand install’s known
copies moved aside; the package named smartconfig.643cec3, both
modes, --grub-password), accept-m2/m3/m4 as root and the install
(d686341, the same sc); details in “Now”. The root steps ran from
this session with your OK; auto mode allowed them this time.sc version, done 50e1128. sc version prints the
package version, the commit (Go’s own stamp, -dirty for a changed
tree) and Go’s version. scripts/version.sh: 0.N.0 at the tag mN,
else 0.N.99+git<commit time>.<sha7>, from the commit alone (the
lab’s P.2 builds again and compares); make build passes it with
-X main.version. TestVersion, TestVersionScript (a git repo of
its own: no tag, at m4, after it). Full checks passed./usr/sbin/sc and make deb, done 9c25edd and
7f70233. The units, the rescue drop-in, the README, the lab and
accept-m4 name /usr/sbin/sc (Q2’s recommended answer). make deb
builds dist/smartconfig_<version>_amd64.deb with dpkg-deb
--root-owner-group (Q1: no new tool): sc in /usr/sbin, the units
and drop-ins in /usr/lib/systemd/system, 42_smartconfig a conffile,
docs, control and md5sums; reproducible (two builds of a commit match).
TestBuildDeb. Full checks, make lab-test and make deb passed. The
hand install on this VM is unchanged until the package replaces it.e710ce2. postinst,
prerm, postrm as dh_installsystemd writes them (enable, mask,
purge), scd restarted on every configure (start would leave a hand
install’s old scd running), update-grub on install and remove, the
flag unset on remove, the store kept on purge and said (Q3). preinst
on a first install moves the README’s hand install aside as
NAME.dpkg-old (Q4), disabling its units first. Tests: each script’s
calls on install, upgrade, a unit the owner disabled, a failing
update-grub, remove and purge; the takeover with a foreign unit left
alone. Full checks passed (twice: the restart came from rereading the
takeover, after the first run).922e5cc. CI fetches the
tags, runs make deb, prints its control and contents, and keeps it as
the deb artifact (2.7 MB; the upload action pinned by commit, as the
others). The plan gains Q6: the license its copyright file names.--deb, done 7c4e466. The lab installs the
package of the tree in its clean guest (dpkg -i), holds 0.4 to the
package’s own files, runs the M4 scenario on it, then D.1 (a later
build over it: scd restarted, store and boots kept), D.2 (dpkg -r:
sc gone, no rescue entry, flag unset, store kept) and D.3 (dpkg -P:
42_smartconfig gone, store kept and said). 42_smartconfig now adds
nothing without sc (a removed package leaves it, a conffile).
Tests: install.sh’s deb mode, D.1-D.3 against canned answers (each
failure), 0.4’s package variant, the tags; the GRUB script without
sc. Full checks and make lab-test (350) passed. Next: make
lab-e2e LAB_E2E_ARGS=--deb, both modes.--deb run on 1c5850f: UEFI PASS, BIOS INCONCLUSIVE; a
lab fix, 7bbaa82 (2026-10-08). make lab-e2e LAB_E2E_ARGS=--deb,
log ~/smartconfig-work/signoff/lab-e2e-20261008T050330Z-1c5850f-deb.log,
run files in ~/smartconfig-work/signoff/lab-e2e-1c5850f/. UEFI passed
everything, deb=yes (24m06s): 0.4 to 0.7 on the package’s own files,
the M4 scenario on it, D.1 (the +lab1 build over it, scd restarted),
D.2 (dpkg -r) and D.3 (dpkg -P). BIOS lost its verdict at 1.1, a
[lab] error: the reboot gave its two guest RESETs 13 ms apart, but the
host stood still 2.5 s while QMP’s reader marked the serial log for the
second, past the 2 s wait, and it came as a reset of its own. Fixed: a
QMP round trip after the wait. Tests for it; full checks and make
lab-test (353) passed. Found while reading for M5 step 8: the plan’s
step 7 also asks for the takeover of a hand install in the lab, which
only TestDebPreinstHandInstall covers. Next: that, as D.4.4df3912. After
the purge, the lab puts in the README’s hand install up to M4 from the
m4 tag’s own files (the package’s sc in /usr/local/sbin, scd
running from it), then dpkg -i of the package: each of the 7 files
must be NAME.dpkg-old, not executable, and named; scd restarted from
/usr/sbin/sc; the units from /usr/lib, enabled; the drop-ins from
/usr/lib only; one rescue entry; sc in root’s PATH /usr/sbin/sc.
A hand install that does not come up is a [lab] error. --deb now
needs the m4 tag in the clone. Tests for each check (each undone
fails one), stage_hand, stage_debs. Full checks and make lab-test
(356) passed. Next: step 8, the README.d222530. README “Install (M5)”: make
deb, apt install, sc version; the first verdict at the next boot,
a failed update-grub, an upgrade, the takeover of a hand install,
remove and purge (the store kept). The hand-install and removal blocks
of the watcher and the rescue path point to it; the Layout lists name
the package’s scripts. Full checks passed. Next: the M5 review (two
reviewers), then the sign-off.--deb run on 82e4168: PASS, both modes (2026-10-08).
make lab-e2e LAB_E2E_ARGS=--deb, log
~/smartconfig-work/signoff/lab-e2e-20261008T062524Z-82e4168-deb.log,
run files in ~/smartconfig-work/signoff/lab-e2e-82e4168/. UEFI 27m01s,
BIOS 32m59s, no retries, dirty=no, deb=yes: the M4 scenario on the
package, D.1-D.3, and D.4 for the first time (all 7 hand files aside,
scd from /usr/sbin/sc, one rescue entry). BIOS chained its five double
RESETs in time; the new QMP round trip was not needed this run. Next:
the M5 review.82e4168: A on the package (with the real dpkg on a fake
root), B on the lab’s --deb mode. Two highs, both fixed: a remove and
install again lost the rescue entry (preinst took the package’s own
42_smartconfig for a hand install), and the lab never showed
postinst’s update-grub (install.sh ran its own). The fixes:
abort-install (5e9f1e3);8284709);9ab3586);37207e9);update-grub, plus D.5,
a remove and install again (99d91ba);accept-m4 step 7 (8cd9c8d);84f9959).Not code fixes: A6, DPKG_ROOT, is plan C6; A11, the name clash with
Ubuntu’s sc spreadsheet, is plan question 7, for you. Full checks and
make lab-test (364) passed; every mutation the reviewers saw survive
now fails a test. Next: the sign-off, make lab-e2e LAB_E2E_ARGS=--deb
on the fixed tree, then the install on this VM.
4b34d12, both modes (2026-10-08).
make lab-e2e LAB_E2E_ARGS=--deb, log
~/smartconfig-work/signoff/lab-e2e-20261008T084346Z-4b34d12-deb.log,
run files in ~/smartconfig-work/signoff/lab-e2e-4b34d12/. UEFI
29m36s, BIOS 30m04s, no retries, dirty=no, deb=yes: the M4
scenario on the package (0.5 now holds postinst’s own update-grub),
D.1-D.5. make deb here gives the lab’s package byte for byte
(781246c8c6b6).~/smartconfig-work/signoff/signoff-m5-4b34d12.sh, log
signoff-m5-20261008T094446Z-4b34d12.log. apt-get install of the
package over the follow-ups’ hand install: the takeover is right
(all 7 files .dpkg-old, not executable; the units from /usr/lib,
enabled; the drop-ins from /usr/lib; one rescue entry; dpkg
--verify clean; rows kept; no new failed unit; backups in
/var/backups/smartconfig: sc-d686341, manifest-pre-m5.txt,
sha256-pre-m5.txt, failed-pre-m5.txt). It stopped on:
/usr/sbin/policy-rc.d answers 101 to
everything (deb-systemd-invoke says so), so the hand scd ran on
its moved-aside sc. Restarted by hand (systemctl restart scd):
now /usr/sbin/sc, Type=notify, baseline in 589 ms. The lab’s
image has no policy-rc.d.sudo sc status exits 2 after the takeover: it lists the hand
install’s /etc/systemd/system units and drop-ins as “error:
deleted” (the package’s own units under /usr/lib stand in for
them) and suggests sc restore of a drop-in, which would undo the
takeover. Until the next healthy boot only, but wrong advice.Not run yet: the remove and the install again (they wait on the
policy-rc.d, under “Still waiting for you”). Next: fixes for both (a
postinst that says when scd still runs another sc; sc status not
calling a deleted /etc override with a unit or drop-in of the same
name under /usr/lib an error), the lab holding sc status after the
takeover, then the rest of the sign-off.
8bd34c3, d81e2d5 (docs
a47df47; plan C7, C8). postinst says when scd was not restarted (its
MainPID the same, or none while enabled) and prerm when scd still runs
after the stop, each with the command; policy-rc.d is still honoured.
sc status does not call a deleted unit or drop-in of
/etc/systemd/system an error when one of the same name in a later
unit directory stands in (deleted; /usr/lib's in use); a mask is
none. Tests for each, each undone fails one; on this VM’s store the
fixed sc status exits 0 (as root, read only). Full checks passed. The
lab’s guest has no policy-rc.d, and its hand install is not there at a
healthy boot, so neither shows in the lab: this VM is where they are
checked. Next: the lab’s --deb run on this tree, then this VM: an
upgrade to it (postinst’s note under the policy-rc.d), with the remove
and install again still waiting on the policy-rc.d.~/smartconfig-work/signoff/post-m5-reboot-20261009T170300Z.txt). The
first boots with the M5 package (4b34d12) and the follow-ups’ units.
This VM went down twice without a shutdown: its journal stops at
2026-10-08 10:57 UTC (that killed the lab’s BIOS run on 8f91158 at
21m29s, no verdict; UEFI had passed) and again 2026-10-09 06:50 UTC,
about a minute into the next boot. Both boots since then, b9b41a6b
and 8d0112f3: verdict ok (failed-units=0), the flag clear,
sc-boot-seen and sc-boot-ok done (ProtectHome=yes,
NoNewPrivileges=yes), scd active from /usr/sbin/sc
(Type=notify, baseline 2.0 s), sudo sc status exit 0, no failed
units, as before M5. Startup 13.7 s; sc is not on the critical chain.
sc boot seen ran twice in b9b41a6b, harmless (as in S3). Next: the
BIOS rerun of the lab’s --deb run on 8f91158, then the upgrade on
this VM.8f91158, both modes (2026-10-09).
make lab-e2e LAB_E2E_ARGS=--deb, one sc build (4fa140edc6fb),
dirty=no, deb=yes, boot 5, D.1-D.5 PASS. UEFI 30m00s, no retries
(2026-10-08, log lab-e2e-20261008T100457Z-8f91158-deb.log); its BIOS
run died with this VM at 21m29s, so BIOS ran again alone on the same
commit (checked out detached): PASS, 22m19s, 1 lab retry (a 2.0 s gap
between VGA polls before boot 3’s menu), boot 2 outcome b (log
lab-e2e-20261009T171037Z-8f91158-deb-bios.log). Run files in
~/smartconfig-work/signoff/lab-e2e-8f91158/. make deb here gives
the lab’s package byte for byte (c8d176d1c042).8f91158, all checks pass
(2026-10-09, as root; script
~/smartconfig-work/signoff/signoff-m5-upgrade-8f91158.sh, log
signoff-m5-upgrade-20261009T173440Z-8f91158.log). apt-get install
of that package over 4b34d12’s. With this VM’s policy-rc.d, postinst
now says so: “smartconfig: scd was not restarted (does
/usr/sbin/policy-rc.d forbid it?): sudo systemctl restart scd”, and scd
kept its old pid until that restart (now /usr/sbin/sc,
Type=notify). 22 checks ok, among them dpkg --verify clean, the
units and drop-ins from /usr/lib, systemd-analyze verify,
grub-script-check, one rescue entry, no flag, rows kept (1586),
sc status exit 0 (the takeover’s deleted /etc units are no error
now), no new failed unit. From 4b34d12 to 8f91158 only sc status
and the maintainer scripts changed, no boot unit or boot path, so the
reboot check above stands for this build. Left of M5’s sign-off: the
remove and the install again on this VM, which wait on your
policy-rc.d answer; then the tag m5.8f91158, dpkg --verify clean), scd from
/usr/lib/systemd/system running /usr/sbin/sc, the rescue path from
the package, the policy-rc.d, and the builds kept in
/var/backups/smartconfig (sc-d686341, sc-59203a3, sc-2c58f8b
new in the list); its recovery steps start from /usr/sbin/sc.
Checked read only on this VM: /etc/fstab and /etc/hosts still the
originals, sudo sc status exit 0, no failed unit. MILESTONES says
where M5 stands. Still waiting: your policy-rc.d answer, for the
remove and install again./var/backups/smartconfig/policy-rc.d.image; script
~/smartconfig-work/signoff/signoff-m5-reinstall-8f91158.sh, log
signoff-m5-reinstall-20261009T180716Z-8f91158.log). The package of
8f91158 (c8d176d1c042), the one the lab passed. apt-get remove:
prerm stopped scd itself (0.5 s, no note), dpkg rc, /usr/sbin/sc,
the units and the drop-ins gone, the three units masked, no rescue
entry, no flag, grub-script-check ok, the store, the conffile and the
hand install’s seven NAME.dpkg-old kept, no failed unit: 25 checks
ok. apt-get install again: no takeover (not a first install),
postinst restarted scd itself (no note), dpkg --verify clean,
42_smartconfig the package’s, then the 17 installed checks, sc
status exit 0 among them: 22 ok. scd’s startup rescan found the one
file changed while it was stopped (/boot/grub/grub.cfg, postrm’s
update-grub, row 0677f5), and it recorded postinst’s update-grub
as it ran (207478); rows 1586 to 1588. Against the manifest from
before M5 only the takeover, the wants/ links now to /usr/lib, the
package’s 42_smartconfig and churn of cups and snap; no mask is
left. The remove took 2m26s: four daemon-reloads of 4-6.5 s and
about 1m45s of update-grub on a busy VM (load 4.9; motd-news,
packagekit, apt-daily started meanwhile; no gap over 22 s in the
journal). With this, plan step 9 is done: the lab in both modes, the
install over the hand install, the reboot, an upgrade, the remove and
the install again. Next: the tag m5 on 8f91158.m5 (2026-10-09, annotated, on 8f91158, the
commit the lab passed; pushed). Checked first in a clone at the tag:
make deb VERSION=0.4.99+git95.20261008095800.8f91158 gives the tested
package byte for byte (c8d176d1c042, sc 4fa140edc6fb), and make
deb there gives 0.5.0 (0087c1219e54, twice), whose files, modes and
owners are the same; only control’s Version, the changelog’s first
line, sc’s version string (the same size) and so md5sums differ.
Found on the way: a build in a git worktree stamps no vcs info into
sc (go version -m: (devel), no vcs.revision, even with
-buildvcs=true), so it is not the clone’s bytes; the lab and make
deb here run in the checkout, where it is stamped. Then this VM, as
root (script ~/smartconfig-work/signoff/signoff-m5-release-0.5.0.sh,
log signoff-m5-release-20261009T182604Z-0.5.0.log): apt-get install
of 0.5.0 over the tested build, postinst restarted scd itself (no
policy-rc.d now), sc version 0.5.0 (8f91158122ad, go1.26.8), dpkg
--verify clean and the 17 installed checks ok, sc status exit 0
among them, rows kept (1588). MILESTONES and PROJECT_LOG say M5 is
done. Next: the cleanup, then M6 (its plan, for your read).~/smartconfig-work/fu-scratch/wt-m5 and wt-revg are removed:
wt-m5’s drafts (3f4c8fb and the wip commits before it; main has the
steps as they landed) stay reachable as the local ref
refs/archive/wt-m5, not pushed; wt-revg was at 0fcabd1 (in main),
and its four untracked probe tests from the chunk G review are copied
to fu-scratch/revg-probes/. dist/ keeps three builds: 4b34d12’s
(the first install here), 8f91158’s (the one the lab passed) and
0.5.0 (the tag); the eight older builds are gone (each is rebuilt
from its commit, in a clone).~/smartconfig-work/project-review-2026-10-09.md./etc/shadow and the network files with Wi-Fi keys are stored with
their content.sc log escaping limit, fixed in c56fe24;/usr/local;auto;sc log, the header not counted).TestLabPython.restore.go’s note
on the dirfd restore.sshd-socket-generator writes sshd_config’s Port and
ListenAddress into ssh.socket at daemon-reload. So a new port or
address needs systemctl daemon-reload, then
systemctl restart ssh.socket (openssh-server’s README.Debian).
default.graph said only systemctl restart ssh.TestDefaultGraphSSHApply, failed on the old text first.go test ./... clean; the build is static.sc prune;The plan is in MILESTONES, “M5 follow-ups (the round of 2026-10-09)”: nine steps in chunks I to L.
Fixed on the way: the installer’s network file is
/etc/cloud/cloud.cfg.d/90-installer-network.cfg (recorded with its
content, tier 4). MILESTONES’ open list named it under
/etc/netplan/.
sc status and the free-space floor (2026-10-09,
chunk I).
sc status adds two
lines: Store: only N MiB free under <home> (scd's floor: 256 MiB);
and that scd holds back changes that need new content until there is
room.watch.DefaultFloorBytes, used by
scd and by sc status.TestStatusStoreUnderFloor: under, at the floor, statfs
failing, the console. Four mutations (no call, >= to >, no
console guard, the wrong directory) each failed it.go test ./... clean.sc prune.sc prune (2026-10-09, chunk I).
--older-than, 90 days by default), then the stored versions no row
uses that are over an hour old.sc status compares with);--yes does not ask. If the boots
file cannot be read, nothing is pruned.Record and commitWrite (restore, sc edit) put their blob again
under the lock;VACUUM, because it would renumber the rowids.store and sc prune, each with its tests.Record and of a Restore;make m1-compat);99999999999999h overflowed into range,
and is refused now.go test ./... and make race clean.sc status was still right after it.cef26b5, 11
findings, two high:
sc status and the rescue report compare
with when no boot line kept them.cef26b5. On the fixes
6 pass. The 7th replays a writer of an earlier release; sc prune
now refuses while scd runs one.9d43980:
sc snapshot is a manual row;sc;sc edit a duplicate “before” row, against M3’s
rule (TestReplaceWhileWatching).go test ./..., make race and
make m1-compat clean.docs/reviews/2026-10-09-m5-followups-chunk-i.md.a4571cb).
sc restore writes a file with a checker,
it checks the version against the file as it is now, as sc edit
does.--force skips the check.mutating is set only for the write.restorecheck_test.go:
--force, the healthy version;lab/e2e.py’s read-only refusal check and
the acceptance scripts’ restores are unaffected: their versions add
nothing, or have no checker.go test ./... clean./etc/default/grub.d/*.cfg.grub.d checked (2026-10-09, chunk J, dc6c3da).
/etc/default/grub.d/*.cfg after
/etc/default/grub, and the cloud image keeps its GRUB settings
there. sc checked only /etc/default/grub (the M6 pilot’s G08).sh -n and
sc’s own rules run on each.TestShSyntaxGrubD (an unclosed quote, spaces around =;
names update-grub does not source are not checked), and
TestDefaultGraph with a grub.d sample. Both failed before the
fix.go test ./... clean.sc help <unknown>.sc help <unknown> (2026-10-09, chunk J, 943207e).
TestHelpUnknown: help of a command, --help, and
help help still exit 0. Both mutations (no leftover check, cobra’s
help back) failed it.go test ./... clean.2584d47).
m6/M6-RESUME.md).check.CheckVersion takes a row’s mode and owner, and
sudoers-mode judges those.TestCheckModeOnlyChange (scd: chmod 0666 after the start gives
the error line) failed before the fix;TestCheckVersionMode;go test ./... and make race clean.8b98817).
sc restore’s check judged the version by the file’s
mode on disk now, but a restore writes the row’s mode and owner. A
world-writable sudoers drop-in put back over a 0440 one gave no
question.check.CheckVersion and
its row’s mode and owner.TestRestoreChecksMode failed before the fix.go test ./... clean.f48e3c0, and the repo’s repair (2026-10-11).
f48e3c0 was pushed, and
again before the boot of 2026-10-11 20:51 UTC (last -x: crash).FETCH_HEAD
was empty, and the reflogs and this file’s last entry ended in NULs.main was origin/main, and the working tree
matched a fresh clone except for this file’s NUL tail.git fetch
brought them back. This file was checked out again, and the NULs
were stripped from the reflogs. git fsck is clean. The .git
from before is in ~/smartconfig-work/fu-scratch/crash-20261009/.~/smartconfig-work/m6/smartconfig) was not
hit: git fsck is clean and its two commits are there.~/smartconfig-work/review-m5fu-j/run1-crashed/.~/smartconfig-work/review-m5fu-j/findings.md), none high. Each
finding was read in the code, and its 14 failing review tests failed
again here.
sc status and sc check <id> judge a
version without its mode; J3, NAME=$NAME word and NAME+= pass
the grub.d check.--force, and exit 2.sc status would call “error: deleted” asks,
unless --force.--force.
store.RestoreRefusal now runs those refusals first, in Restore’s
order (Restore runs the same code).TestRestoreRefusalsFirst, TestRestoreNotFullyChecked
and TestRestoreNoCheckerNoScratch each failed before the fix.make build fmt vet test race lab-test m1-compat clean
(~/smartconfig-work/fu-scratch/checks-j-fix1.log).sc restore X > file waited with its prompt in the file,
and in a while read id; do sc restore $id; done < list loop it took
the next line for its answer. sc prune’s question did the same.TCGETS). Without one they read nothing, write nothing, name the
flag that does not ask (--force, --yes) on stderr, and exit 2.
--help and the README say so.TestRestoreNoTerminal and TestPruneNoTerminal failed
before the fix; TestIsTerminal (a buffer, a pipe and /dev/null are
not terminals, a pseudo-terminal is).make build fmt vet test race lab-test m1-compat clean
(~/smartconfig-work/fu-scratch/checks-j-fix2.log).sc restore GOOD, asked
when GOOD had an error the file now lacked, or when the file was now
missing or a link (all of GOOD’s problems counted as added); without
a terminal it then exited 2. The README said it asks nothing.
healthyLine, which status’s boundary code became. Any other
version is checked as before.--force does not ask.accept-m2.sh’s removals are of /etc/sc-accept.d/
files, which no checker reads, and every lab restore is of the
rescue version.TestRestoreRescueUndoNotAsked and
TestRestoreRemovalAsks failed before the fix.make build fmt vet test race lab-test m1-compat clean
(~/smartconfig-work/fu-scratch/checks-j-fix3.log).8b98817 taught scd and sc restore to
judge a version by its row’s mode and owner, but not sc status or
sc check <id>. A sudoers drop-in made world-writable, which sudo
then ignores, was “no problem found” in the rescue report (no undo,
exit 0), and sc check of that row exited 0, while scd logged an
error and sc restore asked.statusProblem checks the row and the version it is
compared with by their own rows’ mode and owner (CheckVersion),
and so does sc check <id> for a file row. sc check --as still
judges no mode: its file is not installed with its own.TestStatusAndCheckJudgeRowMode failed before the fix.make build fmt vet test race lab-test m1-compat clean
(~/smartconfig-work/fu-scratch/checks-j-fix4.log).CheckVersion and its
row’s mode and owner, and the checker remembers a result by content,
mode and owner.TestCheckAfterByRowMode failed before the fix.make build fmt vet test race lab-test m1-compat clean
(~/smartconfig-work/fu-scratch/checks-j-fix5.log).GRUB_CMDLINE_LINUX_DEFAULT=$GRUB_CMDLINE_LINUX_DEFAULT
nomodeset (a value with $ was never judged) and
GRUB_CMDLINE_LINUX_DEFAULT+=" nomodeset" (bash’s, not sh’s). dash
runs nomodeset, or the whole += word, as a command, and
update-grub stops. Both are now blockers; a value of plain
characters, $NAME and ${NAME} is judged.GRUB_TIMEOUT=0 GRUB_TIMEOUT_STYLE=hidden sets both, but was
a blocker. Assignments in a row are now fine; a word after them that
is not one is still a blocker.grub.d/.50-old.cfg) was checked,
though sh’s *.cfg skips it. The graph now gives no checker to a
name with a leading dot that only a wildcard matched, in every
directory: sh, glob(3), systemd, udev and sudo all skip such names.
The scope still records them. netplan’s own note for a dot-file is
now the graph’s “no checker reads this file”.TestShAssignments (6 cases more, each checked with dash
as before), TestShSyntaxGrubD and TestGraphHiddenNames failed
before the fix. TestNetplanDotFile now expects no checker (the
first full run caught it:
~/smartconfig-work/fu-scratch/checks-j-fix6-run1-netplan-fail.log).make build fmt vet test race lab-test m1-compat clean
(~/smartconfig-work/fu-scratch/checks-j-fix6.log).sc help <command> <word>.sc help restore abc123
said unknown command "restore abc123". sc help help had lost
cobra’s text on what it is for.unknown command, exit 1; a
command sc has, then a word it does not know, says restore has no
command "abc123" (see: sc help restore), exit 1. sc help help
says what it does.TestHelpUnknown (the new messages, sc help boot seen,
and the -h line of a command’s help, which the review’s mutation
H2 showed nothing pinned) failed before the fix.make build fmt vet test race lab-test m1-compat clean
(~/smartconfig-work/fu-scratch/checks-j-fix7.log).TestRestoreAnswers, TestRestoreCtrlCAtQuestion,
TestCheckVersionModeBits, and its guards and no-scratch tests as
TestRestoreUnchecked and TestRestoreCheckCannotRun). New:
TestRestoreChecksOwner and TestCheckRowOwners (rows recorded with
an owner a test cannot give a file), TestRestoreWarningNotAsked,
TestRestoreRemovalNotAsked, TestStatusBeforeByRowMode,
TestCheckRestoreModeOkAgain, and atTerminal with one end a file.~/smartconfig-work/fu-scratch/j-mut/: the runner mutate.py,
results-pass3.txt, results-pass4-R10-full.txt).make build fmt vet test race lab-test m1-compat clean
(~/smartconfig-work/fu-scratch/checks-j-fix8.log).4ab3489, and the fix (2026-10-13).
TestRestoreCtrlCAtQuestion: it
looked for sc’s private scratch directory under TMPDIR, but as root
sc makes it in /run first. The test came from the review, which ran
only as a user; the local checks did too.make build fmt vet test race lab-test m1-compat, and the
root pass locally (sudo -n works here), clean
(~/smartconfig-work/fu-scratch/checks-j-fix8b.log).docs/reviews/2026-10-12-m5-followups-chunk-j.md:
J1-J13, none high, all fixed; the two things left as they were (a
link row over a checked file asks nothing; a version over a link or
a missing file is compared with nothing).mutating and on the signal contract say
that a restore or prune is a mutating command only past its question.
MILESTONES: chunk J closed, its items say what restore does now, and
Open items 3 and 11 are done; the link row joins the Open list.make build fmt vet test race lab-test m1-compat and the
root pass clean (~/smartconfig-work/fu-scratch/checks-j-fix9.log).47d35a5).
secret GLOB: /etc/{shadow,gshadow},
NetworkManager’s system-connections, netplan and the installer’s
90-installer-network.cfg. Their content is stored as before.internal/redact): shadow’s and gshadow’s
password field (the lock marks ! and * stay), key=value and
key: value pairs whose key names a secret (not -flags or
-type), a keyfile’s [*secrets] sections, YAML values on the
lines after a secret key, and PEM private-key bodies.sc cat (<hidden>), sc diff (<hidden N>, one
numbering for both sides, so a changed password shows; a change only
in blanks next to a hidden value is said, not called equal), a note
on stderr for both, and what a validator said about such a file in
sc check -v, sc edit and sc restore. sc scope names the line.
sc restore writes the real content.~/smartconfig-work/fu-scratch/k6/wip-step6-*.patch.hunter22, a fake PEM body, the marker after psk=) and a
comment naming the PEM header; its denylist of real values found
nothing. You committed it with --no-verify (2026-10-16)./etc/shadow as root
(auto mode refused reading it); the unit tests cover each format.make build fmt vet test race lab-test m1-compat and the
root pass clean (~/smartconfig-work/fu-scratch/checks-k6.log,
run 2).--show-secrets as root.--show-secrets, as root (2026-10-16, 1d6671a).
sc cat, sc diff and sc check take --show-secrets.
As root, secret files are shown as stored, and what a validator said
as it said it. Anyone else is refused before anything is read
(sc: --show-secrets needs root, exit 1). sc edit and sc restore
have no such flag and keep hiding. The stderr note on hidden values
now names the flag.TestShowSecrets (refused as a user for all three; shown
as root; the next run hides again). Four mutations (the flag ignored
by cat/diff, by check’s lines, no root check, the check inverted):
all killed.make build fmt vet test race lab-test m1-compat and the
root pass clean (~/smartconfig-work/fu-scratch/checks-k7.log).~/smartconfig-work/review-m5fu-k/findings.md.
TestReview* fail again
(review-m5fu-k/scratch/verify-main-session.log), and K1’s paths
were checked with sc scope.sc restore can still put any of them back.sc diff marks values as in both versions, old or new, with no
numbers, so it no longer tells which values are equal.7d8fec1).
? key / : value, a flow mapping’s
value on the next line (K2); a one-line private key (K3); vpnc keys
with blanks, mka-cak, 802-1x.pin (K4); PGP key blocks (K5). A
key in a comment leaves nothing open (K10).sc diff
shows a change in it (TestCatDiffSecrets now says so).~/smartconfig-work/fu-scratch/k-mut/fix1.py, fix1-results.txt).make build fmt vet test race lab-test m1-compat and the
root pass clean (~/smartconfig-work/fu-scratch/checks-k-fix1.log).c404bd1).
sc restore can
still put any of these back./etc/cloud,
/etc/fstab, ifupdown, wpa_supplicant, sssd, nslcd, swanctl,
OpenVPN, msmtp, ssmtp, MySQL’s debian.cnf, Samba and Let’s
Encrypt.*.secret, rsyncd.secrets,
krb5.keytab, ipsec.secrets, sasl_passwd, exim’s
passwd.client, authinfo, chrony.keys, BIND’s keys, OpenVPN’s
key and auth files, Let’s Encrypt’s private keys, opasswd.key value lines;
OpenVPN’s inline key blocks; more key names (authtok, token, …pass,
…pw, …_private). A cloud-init chpasswd: mapping is hidden whole.~/smartconfig-work/fu-scratch/k-mut/fix2.py, fix2-results.txt).make build fmt vet test race lab-test m1-compat and the
root pass clean (~/smartconfig-work/fu-scratch/checks-k-fix2.log).0df7f46,
a5078a6).
c16f948 (pushing disabled, no sudo), each leaving a patch
(~/smartconfig-work/k-fix/: briefs, patches, notes). Claude read
each patch, applied it here, ran the checks and committed.0df7f46): sc diff marks a value
<hidden> when the other version hides it too, else <hidden, old>
or <hidden, new> (redact.Sides); one note for each path that hid
anything.a5078a6): sc scope says what holds for
a path scd does not record if sc snapshot saves it; sc cat --help
and the README warn that sc cat ID > FILE writes markers; tests for
the review’s surviving mutations, 81 of 82 killed (the survivor is
unreachable).[ too. Four mutations of these, all killed.make build fmt vet, the tests and the race pass with a
40-minute timeout (the agents’ runs hit go test’s 10 minutes under
their shared load), lab-test, m1-compat and the root pass, clean
for each (~/smartconfig-work/fu-scratch/checks-k-fix3.log,
checks-k-fix4.log).accept-m4.sh’s sc cat sha checks of
/etc/fstab assume it has no secret option (true here and in the
lab); chunk L, which reworks the lab’s --deb mode, settles it.docs/reviews/2026-10-16-m5-followups-chunk-k.md. K1-K13, one high,
all fixed. MILESTONES: chunk K closed, step 6’s list as it is now.
Chunk K is closed. Next: chunk L, step 8 (the apt hook).ea50fad).
/etc/apt/apt.conf.d/50smartconfig, a second
conffile: DPkg::Pre-Invoke and Post-Invoke run sc apt begin and
sc apt end, guarded ([ -x ] and || true), so a missing or
failing sc neither stops apt nor prints anything. apt-config -c
reads it.internal/aptrun, /run/smartconfig/apt-run): the
pid and start time of the apt above the hook’s shells, and when the
run began. scd reads it for each change it records; a mark whose apt
is gone or whose pid was reused tags nothing. sc apt opens no store
and is root only. Only the system’s scd reads it: one with --root
(the tests) does not, so an unattended upgrade cannot tag test rows.apt, the computed intent and the run, e.g.
changed (apt run 2026-10-16 07:05:12 UTC). They are scd’s rows
(store.Watched): the checker reports what they add, sc prune may
drop them, a snapshot equal to one is a row of its own. sc-m1 lists,
shows and restores them (make m1-compat gains the case).~/smartconfig-work/fu-scratch/k-mut/step8.py, step8-results.txt).make build fmt vet, the tests and race pass (40-minute
timeout), lab-test, m1-compat, make deb and the root pass clean
(~/smartconfig-work/fu-scratch/checks-l8.log).--deb mode checks the upgrade’s rows.--deb upgrade through apt (2026-10-16,
9a40d4e).
DPkg::Pre-Invoke is apt’s; D.1’s dpkg -i never ran
the hook. D.1 now runs apt-get install of the later build.42_smartconfig, a conffile the old package left unmodified, so dpkg
replaces it with no question, inside the run.apt with the run,
that apt’s mark is gone after the run, and that a change made after
it is auto; six new fault cases, one problem each.accept-m4.sh, facts.sh, e2e.py 1.9
and 4.6) read sc cat --show-secrets: fstab is a secret file since
chunk K. Each runs as root with this tree’s sc, so the flag is
always there.make lab-e2e --deb is a sign-off run (your OK).make lab-test (364 tests) and bash -n accept-m4.sh
(~/smartconfig-work/fu-scratch/checks-l9.log); no Go changed.c898df1, and the fix (2026-10-16).
TestLabGuestFacts (Go) runs the lab’s facts.sh
against a made-up sc whose cat read the id as its first word;
step 9 put --show-secrets there. Step 9’s checks ran only make
lab-test, since no Go had changed: a Go test reads that shell script.make build fmt vet, the tests and race pass, lab-test,
m1-compat and the root pass clean
(~/smartconfig-work/fu-scratch/checks-l9b.log). Every step runs the
whole suite before its push.~/smartconfig-work/review-m5fu-l/findings.md.
Runner named that fork, so the mark was always
stale and no row was ever tagged. Shown with the real apt-get 2.8.3
in a sandbox (no package database touched). The lab’s D.1 would have
failed on it. The tests missed it: no fork in the fake process tree,
and one test took its expected pid from Runner itself.review-m5fu-l/scratch/verify-main-session.log).51c8659).
Runner takes the parent when the process above the shells
has its parent’s name (apt’s fork), one step only. A test runs the
hook line from a real fork that system()s it, as apt does.sc’s output to /dev/null (a downgrade
keeps the conffile; 0.5.0’s sc printed “unknown command” twice into
each apt run). This step’s log entry said the hook printed nothing;
it did when sc failed.begin with
no apt, a created file’s rows, and D.1’s commands; the README says
only apt’s runs are tagged.lab-test, m1-compat, make
deb and the root pass clean
(~/smartconfig-work/fu-scratch/checks-l-fix1.log).6022f33).
lab-test, m1-compat and the
root pass clean (~/smartconfig-work/fu-scratch/checks-l-fix2.log).docs/reviews/2026-10-16-m5-followups-chunk-l.md. L1-L6, one high, all
fixed. MILESTONES: chunk L closed, step 8 says what it does now.
Chunk L is closed, and with it every step of the round. Next:
the sign-off runs, each with your OK.