diff --git a/docs/handover.md b/docs/handover.md index d631320..bb95aa8 100644 --- a/docs/handover.md +++ b/docs/handover.md @@ -1,6 +1,6 @@ # ROV Autonomous Inspection System — Project Handover **Project Name:** Argonaut 3 -**Version:** 2.25 +**Version:** 2.26 **Date:** 13 July 2026 **File:** `docs/handover.md` (repo-tracked, no version number in filename — see Version History for versioning) @@ -24,6 +24,16 @@ recovery (`2a3e577`) surfaced a second gap — gate-mode committed recovery ente `RETURN_TO_SAFE` and could never leave it. Fixed with a new permanent arrival interface, verified live end-to-end on the bench (`bf815cc`). See §15f and Changelog, 13 Jul entries. +⚠ **CORRECTED later the same session (13 Jul 2026)** — the condition-clear-commits-to-recovery +entry trigger described immediately above (`2a3e577`) is **KNOWN-WRONG**, not merely +mode-aware-and-working. It produced a contradictory live state: W1 showing GREEN ("Systems +nominal") while the vehicle was simultaneously in `RETURN_TO_SAFE` ("Recovering to safe zone"). +Corrected intent recorded in `docs/design_intent_register.md` v1.4 (`255096f`): committed +recovery must be triggered by fault PERSISTENCE or RECURRENCE, never by a condition clearing — +GREEN is unconditionally a working state. The safe-zone-reached arrival EXIT mechanism +(`bf815cc`) is unaffected and still correct; only the ENTRY trigger needs rework. This is the +**top-priority next-session task** — see §0 NEXT and §15g. + > Full narrative for each prior version is in Version History below — not repeated here. --- @@ -54,19 +64,22 @@ verified live end-to-end on the bench (`bf815cc`). See §15f and Changelog, 13 J end-to-end: kill mavros → RED + "Heartbeat Lost", restart → recover to GREEN. Four commits `86b2728`/`1b0050d`/`dbe9515`/`3409ae5` — full detail in Changelog below. §15d (was OPEN, SAFETY-CRITICAL) is resolved. -- **Mode-aware failsafe recovery (13 Jul 2026)** — `failsafe_monitor` now subscribes to - `/rov/mode/profile` and branches condition-clear recovery on `health_role` (`2a3e577`): gate - (AUV) commits `HOLD_AND_RECOVER`→`RETURN_TO_SAFE` on condition-clear instead of returning to - NORMAL; advisory (ROV) holds and waits for an explicit operator resume (placeholder topic). - This surfaced a new gap once the mode system was actually live — gate-mode `RETURN_TO_SAFE` - had no exit, a terminal trap. Closed by a new PERMANENT interface, - `/rov/nav/safe_zone_reached` (std_msgs/Bool), that transitions the FSM to NORMAL on arrival - (`bf815cc`). Verified live end-to-end on the bench: published the event by hand, watched the - FSM release `RETURN_TO_SAFE`→NORMAL and the `motion_controller` recovery broadcast stop. The - topic/subscription are permanent; today's publisher is a manual bench command only — the - real navigation-raised event and a temporary W1-contextual operator button are both still to - come. See §15f, DIR-7 addendum, and the new Parked Design Item in - `docs/design_intent_register.md`. +- **Mode-aware failsafe recovery — PARTIALLY correct, entry trigger KNOWN-WRONG (13 Jul 2026, + corrected later the same day).** `failsafe_monitor` correctly subscribes to + `/rov/mode/profile` and reads `health_role` (`2a3e577`) — this mechanism is confirmed working + and correct, unaffected by the correction below. The safe-zone-reached arrival EXIT + mechanism — a new PERMANENT interface, `/rov/nav/safe_zone_reached` (std_msgs/Bool), that + transitions gate-mode `RETURN_TO_SAFE` to NORMAL on arrival (`bf815cc`) — is also confirmed + working and correct, verified live end-to-end on the bench (published the event by hand, + watched the FSM release `RETURN_TO_SAFE`→NORMAL and the `motion_controller` recovery + broadcast stop). **However, the ENTRY trigger built the same day (`2a3e577`: gate commits + `HOLD_AND_RECOVER`→`RETURN_TO_SAFE` ON CONDITION-CLEAR) is KNOWN-WRONG** — corrected in + `docs/design_intent_register.md` v1.4 (`255096f`) after it produced a contradictory live + state (W1 GREEN "Systems nominal" while the vehicle sat in `RETURN_TO_SAFE` "Recovering to + safe zone"). Corrected intent: committed recovery must be triggered by fault PERSISTENCE or + RECURRENCE, never by condition-clear — GREEN is unconditionally a working state. Rework to + the count-based model is the **top-priority next-session task** — see §0 NEXT, §15g, and the + DIR-7 addendum in `docs/design_intent_register.md`. - W6 Altitude Widget — deployed. NO SONAR state confirmed (expected — no Ping2 connected). Commit `0640353`. - Backup system — push model, both Pis → server. RPi4: 00:00 daily. RPi5: 00:30 daily. @@ -124,22 +137,41 @@ verified live end-to-end on the bench (`bf815cc`). See §15f and Changelog, 13 J to be read directly off `/rov/failsafe` every time rather than watched in the service log. Should log state transitions and cause on each one. Minor, but cost real diagnostic time. See §15f. +- **W1 safe-zone-reached button — PAUSED, not abandoned (13 Jul 2026).** Supporting + infrastructure is built and committed: `rov_api`'s `POST /nav/safe_zone_reached` endpoint and + the `external/rov-failsafe-state` data lake variable it depends on (`2def2b1`); a pre-edit + backup of the W1 widget was made per CREATE-THEN-BACKUP + (`widgets/w1_system_health_indicator.json.2026-07-13.bak`). The widget edit itself was not + started — the button would surface FSM state reached via a trigger now known to be wrong (see + the mode-aware recovery correction above), so it waits on that rework. See §15g. +- **Stray nested colcon workspace caused recurring `rov_mission` build failures — root-caused + and fixed (13 Jul 2026).** `build/`/`install`/`log` directories had been created inside + `src/rov-autonomy` on RPi5 (from `colcon build` being run from inside the repo checkout + instead of the workspace root), colliding with the real workspace at `/data/ros2_ws` and + causing builds to resolve `mode_profiles.yaml` through a stale nested `install/` tree. This + had recurred 2-3 times. Fixed by removing the stray directories (gitignored/untracked — local + filesystem cleanup only, no git change). **Durable guard: `colcon build` must only ever be + run from `/data/ros2_ws`, never from inside `src/rov-autonomy`.** See §15g. **PARKED (design decisions deferred, priority order — moved here from CLAUDE.md 11 Jul 2026):** 1. ~~**Doc restructure**~~ — DONE this session (11 Jul 2026): this document, `CLAUDE.md`, and `docs/design_intent_register.md` restructured/updated; all docs moved into repo `docs/`. -2. **Mode-dependent failsafe recovery** — AUV (`health_role: gate`) commits to recovery on - condition-clear, does not auto-resume; ROV (`advisory`) holds and hands the - continue-or-recover decision to the operator. **Essentially complete (13 Jul 2026):** - `failsafe_monitor` now reads the mode profile and branches on `health_role` (`2a3e577`), and - gate-mode `RETURN_TO_SAFE` now has a real exit via the new permanent - `/rov/nav/safe_zone_reached` interface (`bf815cc`), verified live end-to-end on the bench. - **Remaining:** the event's publisher — navigation cannot yet raise it, so the immediate next - step is disposable bench scaffolding, a temporary W1-contextual operator button (shown only - while W1 reads the recovering/`RETURN_TO_SAFE` state, backed by a `rov_api` endpoint), to be - deleted outright once navigation publishes the real GPS/EKF-derived event. Advisory mode's - operator resume is also still a placeholder topic pending the proper `rov_api` endpoint. See - DIR-7 addendum + new Parked Design Item, `docs/design_intent_register.md`. +2. **Mode-dependent failsafe recovery — CORRECTED intent (13 Jul 2026), rework pending.** + `failsafe_monitor` correctly reads the mode profile and branches on `health_role` + (`2a3e577`) — that mechanism stands. But `2a3e577`'s ENTRY trigger (gate commits + `HOLD_AND_RECOVER`→`RETURN_TO_SAFE` **on condition-clear**) is **KNOWN-WRONG**: it produced a + contradictory live state (W1 GREEN while the FSM sat in `RETURN_TO_SAFE`). **Corrected intent** + (`docs/design_intent_register.md` v1.4, `255096f`): GREEN is unconditionally a working state; + committed recovery is triggered by fault **persistence** (config-driven raw failure count, + transient tier) or **recurrence/hard-persistence** (RED that doesn't clear, immediate commit) + — never by condition-clear. All thresholds are config-wizard-driven via the resolved mode + profile, like `health_role`. The safe-zone-reached arrival EXIT mechanism + (`/rov/nav/safe_zone_reached`, `bf815cc`) is unaffected and still correct — it still + completes a committed recovery; only entry into one was wrong. **This rework is the + top-priority NEXT item** (see §0 NEXT). The W1-contextual operator button (supporting + `rov_api` endpoint + data lake variable already built, `2def2b1`) is PAUSED pending this + rework — see Open Items above and §15g. Advisory mode's operator resume is also still a + placeholder topic pending the proper `rov_api` endpoint. 3. **Auto-recovery** — `failsafe_monitor` should attempt to re-establish the FC link itself, escalating/holding RED only if it cannot. Real-world dives have no operator to restore comms. **Intent now recorded** in DIR's Parked Design Items (11 Jul 2026) — implementation still open. @@ -155,21 +187,31 @@ verified live end-to-end on the bench (`bf815cc`). See §15f and Changelog, 13 J a dedicated design session. **NEXT (priority order, carried from §12):** -1. **Safe-zone-reached arrival button (bench scaffolding)** — temporary W1-contextual operator - button, shown only while W1 reads the recovering/`RETURN_TO_SAFE` state, backed by a new - `rov_api` endpoint that publishes `True` to `/rov/nav/safe_zone_reached`. Disposable — - deleted outright once navigation can raise the real GPS/EKF-derived arrival event on the - same (permanent) topic. See §15f, DIR-7 addendum, and the new Parked Design Item. -2. **W3 widget code**: update label to "RETURN TO SAFE", set `FASTAPI_HOST` = +1. **FSM entry-trigger rework — count-based fault-persistence model (TOP PRIORITY, 13 Jul + 2026).** `failsafe_monitor`'s condition-clear-commits-to-recovery entry logic (`2a3e577`) is + KNOWN-WRONG (see `docs/design_intent_register.md` v1.4, DIR-7 addendum, `255096f`). Rework + to: GREEN unconditionally a working state; committed recovery triggered by fault persistence + (transient tier — config-driven raw failure count, no time window) or hard-persistence (RED + that doesn't clear — immediate commit); all thresholds config-wizard-driven via the resolved + mode profile, like `health_role`. The safe-zone-reached arrival EXIT mechanism (`bf815cc`) + needs no rework — only the entry trigger. +2. **Safe-zone-reached arrival button (bench scaffolding) — PAUSED, waiting on item 1.** + `rov_api`'s `POST /nav/safe_zone_reached` endpoint and the `external/rov-failsafe-state` data + lake variable it depends on are built and committed (`2def2b1`); a pre-edit W1 widget backup + was made per CREATE-THEN-BACKUP. The widget edit itself is not started — the button would + surface FSM state reached via the known-wrong trigger above, so it is meaningless until item + 1 lands. Disposable once navigation can raise the real GPS/EKF-derived arrival event on the + same permanent topic. See §15g, DIR-7 addendum, and the Parked Design Items. +3. **W3 widget code**: update label to "RETURN TO SAFE", set `FASTAPI_HOST` = `http://192.168.1.101:8081`, then test W3 end-to-end (button → /abort → FSM, already verified server-side). -3. **Stage 2b design decisions** (deferred pending Grant): return budget "shortest path to +4. **Stage 2b design decisions** (deferred pending Grant): return budget "shortest path to safety" concept; W7 backup notice (Option A confirmed: server read-only JSON, RPi5 polls outbound). -4. **W2 testing**: test with live mission via `POST /mission/start` (backend now available) +5. **W2 testing**: test with live mission via `POST /mission/start` (backend now available) + loaded plan. -5. **Ping2 sonar purchase** (~$280 USD): required for W6 full end-to-end test. -6. **Pre-field hardening pass**: UFW + RPi5 SSH password (bundled, triggered before remote +6. **Ping2 sonar purchase** (~$280 USD): required for W6 full end-to-end test. +7. **Pre-field hardening pass**: UFW + RPi5 SSH password (bundled, triggered before remote ops/field). **DATA LAKE VARIABLES (19 total, published by cockpit_bridge):** @@ -200,6 +242,8 @@ verified live end-to-end on the bench (`bf815cc`). See §15f and Changelog, 13 J | Hash | Description | |---|---| +| `255096f` | docs(dir): correct inverted DIR-7 in-mission-recovery intent — count-based fault-persistence model, GREEN unconditional working state | +| `2def2b1` | rov_api: POST /nav/safe_zone_reached endpoint; cockpit_bridge: external/rov-failsafe-state variable | | `bf815cc` | failsafe_monitor: /rov/nav/safe_zone_reached permanent interface closes gate-mode RETURN_TO_SAFE terminal trap | | `464e17e` | rov_full.launch.py: wire mode_profile_loader in (was built 4e48dc2, never launched) + respawn | | `2a3e577` | failsafe_monitor: mode-aware recovery (health_role gate/advisory) per DIR-7 addendum | @@ -229,6 +273,7 @@ verified live end-to-end on the bench (`bf815cc`). See §15f and Changelog, 13 J | Version | Date | Changes | |---|---|---| +| 2.26 | 13 Jul 2026 | **Same-day correction, extending v2.25's record.** The mode-aware recovery built earlier today (`2a3e577`) was found to be internally contradictory: its gate-mode entry trigger commits to `RETURN_TO_SAFE` ON CONDITION-CLEAR, which produced a live state that should not be possible — W1 showing GREEN ("Systems nominal") while the vehicle sat in `RETURN_TO_SAFE` ("Recovering to safe zone"). Corrected in `docs/design_intent_register.md` v1.4 (`255096f`): GREEN is unconditionally a working state; committed recovery is triggered by fault PERSISTENCE (transient tier — config-driven raw failure count, deliberately no time window) or immediate hard-persistence (RED that doesn't clear), never by condition-clear; all thresholds config-wizard-driven via the resolved mode profile like `health_role`. Survives unaffected: `health_role` mode-awareness (`2a3e577`'s subscription mechanism) and the safe-zone-reached arrival EXIT mechanism (`bf815cc`) — only ENTRY into committed recovery was wrong. `2a3e577`'s entry logic is now flagged KNOWN-WRONG pending rework, which is the new top-priority NEXT item, ahead of everything previously listed. Also recorded: (1) the W1 safe-zone-reached button is PAUSED, not abandoned — its `rov_api` endpoint (`POST /nav/safe_zone_reached`) and the `external/rov-failsafe-state` data lake variable are built and committed (`2def2b1`), a pre-edit W1 widget backup was made per CREATE-THEN-BACKUP, but the widget edit itself was not started because the FSM state it would surface is reached via a known-wrong trigger; (2) a recurring `rov_mission` build failure (recurred 2-3 times) was root-caused to stray nested `build`/`install`/`log` directories inside `src/rov-autonomy` on RPi5 (from `colcon build` run from the wrong directory), colliding with the real workspace at `/data/ros2_ws` — fixed by removing the stray directories (gitignored, local cleanup only, no git change); durable guard recorded: `colcon build` only ever from `/data/ros2_ws`. §0 Confirmed Working, Open Items, PARKED, NEXT, Recent Commits all updated; new §15g narrative section. Companion DIR bump: `docs/design_intent_register.md` v1.4 — see that changelog for the full corrected-intent detail, not repeated here. | | 2.25 | 13 Jul 2026 | **Two-part fix session, both surfaced by putting the 11 Jul mode-aware work under load.** (1) `mode_profile_loader` was found never to have been wired into `rov_full.launch.py` despite being committed 7 Jul (`4e48dc2`) — `/rov/mode/profile` had zero publishers at runtime since then, so the entire operating-mode profile system had been dark and `failsafe_monitor`'s `health_role` had been silently running on its gate default the whole time. Fixed by adding the node to the launch file with respawn (`464e17e`), verified live (publisher count 0→1). (2) With the mode system now actually publishing, `failsafe_monitor`'s mode-aware recovery (`2a3e577`, same session — subscribes to the resolved profile, branches condition-clear recovery on `health_role`) surfaced a second gap: gate-mode committed recovery entered `RETURN_TO_SAFE` and could never leave it, because that branch deliberately ignores further condition-clear so recovery runs to completion, and nothing signalled completion. Fixed by a new PERMANENT interface, `/rov/nav/safe_zone_reached` (std_msgs/Bool), that transitions the FSM `RETURN_TO_SAFE`→`NORMAL` on arrival when gate-mode (`bf815cc`), verified live end-to-end on the bench (published the event by hand, watched the FSM release and the `motion_controller` recovery broadcast stop). Today's publisher is temporary bench scaffolding only — real navigation-raised arrival and a temporary W1-contextual operator button are both still to come; see the new Parked Design Item in `docs/design_intent_register.md` v1.3. §0 Confirmed Working, PARKED, and NEXT updated; new §15f narrative section. **Two pre-existing `failsafe_monitor` defects found this session (not caused by it):** `flag_manual_abort` is a one-way latch never reset, permanently shadowing all lower-priority handling (including this session's own new logic) after any single manual abort — field-deployment concern, needs its own session; and `failsafe_monitor` produced no log output in `journalctl` this session, costing diagnostic time. Both added to §0 Open Items and §15f. Companion DIR bump: `docs/design_intent_register.md` v1.3 adds the mode-aware safe-zone-reached Parked Design Item — see that changelog for detail. | | 2.24 | 11 Jul 2026 | **§15d resolved.** Vehicle health / FC-comms-loss fault (discovered 9 Jul, flagged OPEN/SAFETY-CRITICAL in v2.22) is fixed as of four commits today: `86b2728` (unconditional RED on comms loss + `fcu_ever_connected` latch + cause text), `1b0050d` (FSM recovery fix — `RETURN_TO_SAFE` was a terminal trap), `dbe9515` (new `external/rov-failsafe-cause` variable), `3409ae5` (W1 widget shows state + cause text). Verified live end-to-end. Related fault found+fixed same session: `cockpit_bridge` was publishing `failsafe_state` (FSM, 3+) not `assessment_state` (0/1/2) to `rov-failsafe` — the actual cause of that day's "widgets show no data," fixed in `d854641`, compounded by a restart-storm wedging port 9001 (→ CLAUDE.md's one-restart-then-verify-9001 rule). §0 Confirmed Working and Recent Commits updated; PARKED list updated to reflect intent now recorded in the DIR (mode-dependent recovery, auto-recovery, dedicated cause field, AMBER runtime-response — all still open on implementation, closed on design intent). Companion DIR bump: `docs/design_intent_register.md` v1.2 adds the two DIR-7 addenda and a Parked Design Items section — see that changelog for detail, not repeated here. | | 2.23 | 11 Jul 2026 | **Doc restructure.** This document is now the single source of truth for current state (Quick Status Board gains PARKED and NEXT subsections, folding in the list previously buried in CLAUDE.md's SESSION LOG and §12's Next Priorities). New **Changelog** section (below) replaces CLAUDE.md's SESSION LOG — same entries, same append-only format, moved here. The embedded Design Intent Register (formerly here, ~200 lines) is **removed** — full DIR now lives only in `docs/design_intent_register.md`; this document keeps a short pointer + summary of which DIRs govern current work. All docs moved into the repo under `docs/` (git-tracked, no version number in filenames — see CLAUDE.md Documentation Discipline). Fixed: duplicate `## 15d` heading (second instance renamed `15e`); §15's "Recording control" locked-decision row corrected from "DIR-9 — always-on" to DIR-9a (ARM-bounded); duplicate 8-variable data-lake table in §9 replaced with a pointer to §0's canonical 19-variable table; stale "Widget JSON not in Gitea" row in §16 marked resolved (W6 committed per `785ec56`, already reflected in §12). No new facts asserted beyond what was already recorded elsewhere in this document, CLAUDE.md, or the DIR — this pass is consolidation, not new findings. | @@ -367,6 +412,44 @@ context needed. Full trigger-rule text lives in `CLAUDE.md`. produced no log output at all in journalctl this session — state had to be read directly off /rov/failsafe every time. Should log state transitions and cause. Minor but cost real diagnostic time this session. +- 2026-07-13 — CORRECTED (later same session): the DIR-7 in-mission-recovery intent + implemented earlier today (2a3e577) was inverted. It had gate-mode AUV commit to recovery + ON CONDITION-CLEAR, which produced a contradictory live state: W1 GREEN ("Systems nominal") + while the vehicle was simultaneously in RETURN_TO_SAFE ("Recovering to safe zone"). Corrected + intent recorded in docs/design_intent_register.md v1.4 (255096f): GREEN is unconditionally a + working state, a cleared fault returns to GREEN and the mission continues/resumes, and + committed recovery is triggered by fault PERSISTENCE or RECURRENCE, never by clearing. Three + fault tiers recorded: uncontrollable (power loss etc., out of FSM scope, adjacent to shelved + DIR-6); transient/self-resetting (system attempts restore, returns to GREEN and resumes on + success, commits to RETURN_TO_SAFE only once a raw config-driven failure count is exceeded, + deliberately no time window); hard/persistent (RED that doesn't clear, commits immediately). + All thresholds are config-wizard-driven via the resolved mode profile, like health_role, not + hardcoded. Survives unaffected: health_role mode-awareness (2a3e577's subscription + mechanism) and the safe-zone-reached arrival EXIT mechanism (bf815cc) — only ENTRY into + committed recovery was wrong, the exit mechanism is unchanged and correct. 2a3e577's + condition-clear entry logic is now KNOWN-WRONG pending rework — new top-priority NEXT item, + ahead of everything previously listed. New Parked Design Item: adaptive restart-timing + (monitor own restore count/duration per fault, weigh future attempts against observed + history instead of a fixed time) — depends on the count-based retry machinery above existing + first, design-captured not build-specified. +- 2026-07-13 — W1 safe-zone-reached button work PAUSED, not abandoned. Supporting + infrastructure built and committed: rov_api POST /nav/safe_zone_reached endpoint (mirrors + the existing /abort pattern exactly) and the new external/rov-failsafe-state data lake + variable it depends on (surfaces FailsafeStatus.failsafe_state, distinct from the existing + external/rov-failsafe assessment integer) — both in 2def2b1. Pre-edit backup of the W1 + widget made per CREATE-THEN-BACKUP: widgets/w1_system_health_indicator.json.2026-07-13.bak. + The widget edit itself was deliberately not started — the button would surface FSM state + (rov-failsafe-state == 3 / RETURN_TO_SAFE) reached via the trigger just found to be + known-wrong above, so it would be meaningless until the FSM rework lands. +- 2026-07-13 — Root-caused a recurring rov_mission build failure (recurred 2-3 times across + sessions): stray nested build/, install/, and log/ directories existed inside + src/rov-autonomy on RPi5, created when colcon build was at some point run from inside the + repo source directory instead of the actual workspace root, /data/ros2_ws. These collided + with the real workspace and caused rov_mission builds to fail resolving mode_profiles.yaml + through a stale nested install/ tree. Fixed by removing the stray directories from RPi5 — + gitignored and untracked, so this was a local filesystem cleanup only, no git change. + Durable guard: colcon build must only ever be run from /data/ros2_ws, never from inside + src/rov-autonomy. Recorded so this is not reintroduced. --- @@ -1171,6 +1254,87 @@ MAVLinkRouter is the confirmed stable router. Do not recommend or use MAVP2P. --- +## 15g. DIR-7 recovery-intent correction, W1 button paused, and a build-workspace footgun — 13 Jul 2026 (later same session) + +### DIR-7 in-mission-recovery intent corrected — condition-clear entry trigger was inverted + +Later the same day as §15f, the DIR-7 "In-mission recovery is mode-dependent" addendum was +found to be internally contradictory as implemented. `2a3e577`'s gate-mode entry trigger — +commit to `RETURN_TO_SAFE` ON CONDITION-CLEAR — produced a live state that should not be +possible: **W1 showing GREEN ("Systems nominal") while the vehicle was simultaneously in +`RETURN_TO_SAFE` ("Recovering to safe zone")** — a working-state light next to an +active-recovery action. + +**Corrected intent** (full detail: `docs/design_intent_register.md` v1.4, DIR-7 addendum, +commit `255096f` — pointer only here, not duplicated): +- **GREEN is unconditionally a working state.** A fault that clears returns the vehicle to + GREEN and the mission continues or resumes. A cleared condition must **never** trigger or + commit a recovery. +- Committed recovery is triggered by a fault **persisting or recurring**, never by it clearing. +- **Three fault tiers:** (1) **uncontrollable** — power loss or anything that disables the + safety logic itself, out of scope for the FSM entirely, adjacent to the shelved DIR-6 + Emergency Surface concern; (2) **transient/self-resetting** — e.g. a comms blip or sensor + re-sync; the system attempts restore, returns to GREEN and resumes on success, and only + commits to `RETURN_TO_SAFE` once a **raw, config-driven failure count** (no time window, no + fixed duration) is exceeded; (3) **hard/persistent** — a RED that does not clear, commits to + `RETURN_TO_SAFE` immediately. +- All thresholds (the acceptable failure count, and which specific faults are transient vs + hard) are **configuration-driven**, set in the setup wizard and read from the resolved mode + profile the same way `health_role` is — never hardcoded. + +**What this means for the code right now:** +- **KNOWN-WRONG, pending rework (top-priority next-session task):** `2a3e577`'s + condition-clear-commits-to-recovery entry logic (the `health_role == HEALTH_GATE` / + `HOLD_AND_RECOVER` branch in `failsafe_monitor._apply_failsafe_priority`'s "all clear" + block). This is what produces the contradictory GREEN-while-recovering state above, and must + be reworked into the count-based tier model. +- **Survives, confirmed correct, no rework needed:** `failsafe_monitor` subscribing to + `/rov/mode/profile` and reading `health_role` (`2a3e577`'s mode-awareness mechanism) — this + is the right way to know which mode's rules apply, and is unaffected by the correction. Also + survives: the safe-zone-reached arrival EXIT mechanism and its permanent + `/rov/nav/safe_zone_reached` topic (`bf815cc`) — it still correctly completes a **committed** + `RETURN_TO_SAFE` recovery on arrival. Only the trigger for **entering** committed recovery + was wrong; how a genuinely-committed recovery **exits** is untouched and correct. + +**§0 Confirmed Working, PARKED, and NEXT updated accordingly** — mode-aware recovery is no +longer described as "essentially complete"; the FSM entry-trigger rework is now the top NEXT +priority, ahead of the W1 button (which is paused, not the immediate next step — see below). + +### W1 safe-zone-reached button — PAUSED, not abandoned + +The supporting infrastructure for the temporary W1-contextual operator button is built and +committed: the `rov_api` endpoint `POST /nav/safe_zone_reached` (mirrors the existing `/abort` +pattern exactly) and the new `external/rov-failsafe-state` data lake variable it depends on +(surfaces `FailsafeStatus.failsafe_state`, distinct from the existing `external/rov-failsafe` +assessment integer), both in commit `2def2b1`. A pre-edit backup of the W1 widget was made per +CREATE-THEN-BACKUP: `widgets/w1_system_health_indicator.json.2026-07-13.bak`. + +**The W1 widget edit itself was deliberately not started.** The button would surface FSM state +(specifically, `rov-failsafe-state == 3` / `RETURN_TO_SAFE`) that the DIR-7 correction above +just established is reached via a known-wrong trigger — the button would be meaningless, +possibly misleading, until the FSM correctly represents recovery state per the corrected +model. This work is paused, waiting on the FSM rework above, not abandoned or deprioritised +for any other reason. + +### Stray nested colcon workspace — recurring rov_mission build failures root-caused + +A recurring build failure (recurred 2-3 times across sessions) was root-caused this session: +stray nested `build/`, `install/`, and `log/` directories existed inside `src/rov-autonomy` on +RPi5 — created at some point when `colcon build` was run from inside the repo source directory +instead of the actual workspace root, `/data/ros2_ws`. These nested directories collided with +the real workspace, and caused `rov_mission` builds to fail resolving `mode_profiles.yaml` +through a stale nested `install/` tree rather than the real one. + +**Fix:** the stray nested `build/`, `install/`, and `log/` directories were removed from RPi5. +They are gitignored and untracked, so this was a **local filesystem cleanup only — no git +change**, and does not appear in any commit. + +**Durable guard:** `colcon build` must **only ever** be run from `/data/ros2_ws` (the actual +workspace root) — **never** from inside `src/rov-autonomy` (the repo source checkout within +it). Recorded here so this is not reintroduced. + +--- + ## 15b. Session 6 Lessons Learned — RPi5 Base Stack Install ### SSH on Ubuntu 24.04 Server (cloud-init image)