rov-autonomy/CLAUDE.md

# CLAUDE.md — Rules for Claude on the Argonaut 3 Project
**Version:** 3.7
**Updated:** 11 July 2026

**Changelog 3.7 (doc restructure):** This file is now rules-only. Current build state, open
items, and the session-by-session changelog moved to `docs/handover.md` — removed from here:
the live data-lake variable table, the FastAPI endpoint live/deferred lists, the full
ardupilot_manager FD-exhaustion narrative, and the SESSION LOG (now the handover's Changelog
section; trigger rules for what to log stay here, entries do not). All project docs (handover,
DIR, UI design, failsafe design, sim/ML plan, mission/recording lifecycle) moved into the repo
under `docs/`, git-tracked — corrected the MUST-NOT bullet and Documentation Discipline section
that said docs never go to Gitea; that policy is now reversed. Reference table updated to `docs/`
paths, filenames carry no version number (the `Version:` field inside each file is authoritative).

**Changelog 3.6:** Added measurement-before-diagnosis, step-back-on-break,
one-restart-then-verify-9001, and Cockpit widget re-import rules to Working Style.
Corrected the Documentation discipline note that incorrectly claimed CLAUDE.md is never
committed to Gitea — it is tracked and committed like any other repo file.

**Changelog 3.5:** Added `[VSC]` terminal label. Added SESSION LOG section + write triggers
(bridges Claude Code work back to claude.ai chat, which has no native access to Claude Code
sessions).

---

**This file is rules only — how Claude works.** Current build state, open items, and the
session changelog live in `docs/handover.md`. Design intent ("why") lives in
`docs/design_intent_register.md`. Do not look for state here.

---

## PRIME DIRECTIVES — These override everything else

### 1. Research before acting. Always.
Before any technical decision, code change, or command:
- Read project docs (`docs/handover.md`, `docs/ui_design.md`, `docs/design_intent_register.md`,
  this file) and search past chat history
- Read source code and official docs before acting; GitHub source/issues are more
  authoritative than official docs for Cockpit/BlueOS specifics
- Confirm facts from sources, not from memory

If uncertain: **search first, then act**. Never guess.

### 2. Time and tokens are finite and valuable
- **Time wasting is FORBIDDEN. Token wasting is FORBIDDEN.**
- Short replies. No preamble padding. To the point.
- Never repeat an approach that has already failed
- Never ask for information already in the project docs
- Never issue a command without knowing what it will do and why

### 3. Facts only. Never assumptions.
- Do not proceed based on what "should" work
- Do not assume port, routing, or API behaviour — measure it
- Do not infer system state — verify it before issuing commands
- If a claim cannot be verified from a source, say so explicitly
- Never present a prior session's confirmed state as the current running state

---

## Working Style

### One command at a time
Give one command, wait for feedback, then give the next. Never batch commands unless asked.

### Measurement before diagnosis (facts only)
Every claim about system state must be backed by a read-only command output in the same
exchange. No diagnosis from theory. If a cause is asserted, the measurement that proves it
must be shown first. If state is unknown, the only permitted next action is the command
that measures it — not a fix built on assumption. Enforcement phrase: if the operator says
"measurement?" and none was shown, the rule was broken — go measure before proceeding.

### Step back to the last action when something breaks
When something that was working breaks, return to the last action taken. If it worked
before that action and not after, that action caused it — revert or step back from there
before investigating anything else. Do not open new diagnostic threads before ruling out
the most recent change.

### One restart, then verify — never restart-storm
Never issue repeated `systemctl restart argonaut.service` in quick succession. Rapid
cycling leaves cockpit_bridge wedged (9001 never binds; node runs but serves nothing).
Always: one restart, wait ~15s, then verify with `ss -tlnp | grep 9001` before any further
action. Only one restart is in flight at a time.

### Cockpit widget re-import — full quit after JS changes
When a Cockpit DIY widget's JS changes (not just html/css), a delete + re-import inside the
running app can leave the old script's setInterval timer alive in the background, so old
and new timers both write to the DOM and fields flicker. After re-importing a widget whose
JS changed, fully quit and relaunch Cockpit (not just close the window — the Electron
process survives a close). html/css-only changes do not need this.

### Lead with context and corrections BEFORE instructions, never after.

### Copy-paste friendly commands
Commands inside terminal blocks must be immediately copy-pasteable and suitable for the
terminal in use (Git Bash on the laptop, bash on the Pis/server). Never include:
- Placeholders like `<YOUR_IP>` or `path/to/file`
- Hashed `#` comments inside the command block
- Options that require editing before running

### Code comments
When writing code (Python, config, scripts), always include full inline comments
explaining what each section does and why. This is a standing user preference.
NOTE: this applies to CODE files only — NOT to terminal command blocks, which must
stay clean and copy-paste ready with no inline comments.

### Terminal labels
Every command block is preceded by a plain-text terminal label above the block.
Grant pastes terminal output including the shell prompt — use these to identify
which device the output came from:

| Label | Shell prompt | Device | IP |
|---|---|---|---|
| `[SERVER]` | `grant@symbytech:~$` | Dell PowerEdge R210 | 192.168.1.175 |
| `[RPi5]` | `ubuntu@rov-brain:~$` | Raspberry Pi 5 autonomy brain | 192.168.1.101 |
| `[RPi4]` | `pi@blueos:~$` | Raspberry Pi 4 BlueOS gateway | 192.168.1.100 |
| `[LAPTOP — Git Bash]` | `grant@BB8 MINGW64 ...` | Windows field laptop BB8 | — |
| `[VSC]` | — | VS Code integrated terminal on BB8 (local Windows session at `C:\Users\grant\Desktop\Autonomous ROV System`) | — |
| `[BROWSER]` | — | Web browser on laptop | — |
| `[COCKPIT]` | — | Cockpit native desktop app on laptop | — |

`[VSC]` is a **local Windows session on BB8, NOT a Remote-SSH session to rov-brain**, unless
explicitly stated otherwise. Do not assume `[VSC]` == RPi5 working copy.

Never put the label inside the bash block.

> The BlueOS KVM VM (`blueos-vm`) was decommissioned once RPi4/RPi5 hardware became
> operational. `[VM]` is no longer a valid label. Any VM artefacts remaining on the
> server are for reference only.

### Verify before modifying
Always read a file before modifying it. Always check current state before changing it.
State what you observed before proposing a change.

### File operations
Read the exact file first, then `str_replace` with exact matching text. Remove old
versions before writing new ones. Complete files, not fragments — no partial code.

### No guessing at code
When writing code that interfaces with an external system:
1. Find the official documentation or source code
2. Find a working example from the official repo or community
3. Build on the confirmed example
Never invent API calls, message formats, or connection strings.

### Own mistakes once
Acknowledge an error once and fix it. No repeated apologies, no grovelling.

### KISS
Recommend the simple correct path, not a menu of options.

---

## CREATE-THEN-BACKUP (non-negotiable)
When any new component (device, drive, service, config, OS install) is created or
configured, a backup must be put in place immediately as part of that work — not
deferred. Backups and foundational architecture (storage, recovery, monitoring) are
prerequisites, prioritised BEFORE new feature/build work. The RPi4 SD failed with no
backup because this was deferred — do not repeat.

---

## Project-Specific Rules

### Cockpit DIY widgets
- Cockpit is **v1.18.2** (native desktop app mandatory — browser/Lite extension does
  NOT support DIY widgets)
- JSON format with `html`/`css`/`js` string fields
- CSS hard constraints: no `body {}`/`:root {}` global blocks, no `@keyframes`, no
  unicode in JS strings, no `position: fixed`, no global tag selectors
- No `async/await` in widget JS — use `.then()/.catch()`
- Use `var` not `let`/`const` in widget JS for compatibility
- Correct live-value getter: `window.cockpit.getDataLakeVariableData(variableId)`
  (CONFIRMED v1.18.2). `getDataLakeValue()` and `listenToDatalakeVariable()` do NOT
  exist in DIY widget scope. `getAllDataLakeVariablesInfo()` is metadata-only.
- Widget JSON files in the repo use underscore naming: `w6_altitude.json` (matches
  W0–W5). The Cockpit-loaded widget is stored by internal config, not filename, so the
  repo filename is the canonical reference.

### Data injection into Cockpit data lake
- ONLY working external-injection method is the Generic WebSocket Connections feature
  (Settings → General). Production path is the `cockpit_bridge` ROS2 node (rov_mission,
  WebSocket :9001).
- Message format: `variable-name=value` (one per send). Data lake key: `external/{name}`.
- ALWAYS use hyphens — Cockpit normalises underscores to hyphens in IDs.
- HTTP POST to mavlink2rest does NOT work for data lake injection. MAVLink UDP injection
  from external scripts does NOT work. Tested and confirmed — do not suggest them.
- Current full variable list (which ones exist, source topic, meaning) is in
  `docs/handover.md` §0 — not repeated here. It changes as flags/telemetry are added;
  keeping one copy is the point.

### FastAPI backend (rov_api)
- `rov_api` package provides the HTTP-to-ROS2 bridge on port 8081. Runs as
  `argonaut-api.service` (manual-start in dev, same policy as argonaut.service).
- Architecture: uvicorn/FastAPI in the main thread, an embedded rclpy node
  (`RovApiNode`) spinning in a daemon background thread. HTTP handlers call thread-safe
  methods on the node. Do NOT call `rclpy.spin()` more than once.
- **W3 abort path is vehicle-layer:** publishes Bool(true) to `/rov/mission/abort`, which
  failsafe_monitor latches as `flag_manual_abort` → FSM NORMAL → HOLD_AND_RECOVER. This is
  DIR-5 return-to-safe, NOT the mission ABORT service. Do not confuse the two.
- **Python dependencies:** fastapi + uvicorn installed system-wide on RPi5 via
  `sudo pip3 install fastapi uvicorn --break-system-packages --ignore-installed typing_extensions`.
  The `--ignore-installed typing_extensions` flag is required — the Debian-managed
  typing_extensions has no pip RECORD file and blocks the install otherwise.
- Which endpoints are live vs. deferred is current build state — see `docs/handover.md`
  §0 / Phase Task Status, not here.

### Deploy files (version-controlled)
- systemd units and launch wrappers live outside the workspace but are version-controlled
  in the repo `deploy/` folder (copies; authoritative running copies at deploy locations).
- `deploy/argonaut.service` → `/etc/systemd/system/argonaut.service`
- `deploy/argonaut-stack.sh` → `/usr/local/bin/argonaut-stack.sh`
- `deploy/argonaut-api.service` → `/etc/systemd/system/argonaut-api.service`
- `deploy/argonaut-api.sh` → `/usr/local/bin/argonaut-api.sh`
- `deploy/README.md` documents locations, pip deps, and install procedure.
- `/home/ubuntu/ros2_ws` is a symlink to `/data/ros2_ws` — both wrappers rely on it.

### Recording architecture (mechanism — intent is DIR-9a, `docs/design_intent_register.md`)
- The recorder manages a `ros2 bag record` subprocess — MCAP format, zstd compression,
  500MB bag splitting, 10GB free-space retention on `/data`. Bags to
  `/data/bags/dive_<timestamp>/` on NVMe.
- **`foxglove_bridge` is visualisation only — it does NOT record MCAP.** Correct this
  wherever the old "MCAP via foxglove_bridge" claim appears.
- **Storage lever (no reliable in-field server backup):** LOWER PUBLISH RATES on heavy
  streams (keep topics present for debug) is the primary lever; ARM-bounded runs
  self-limit; retention cap tunable; NVMe expansion later. Topic/rate review is open.
- Current implementation status (recording_manager vs. governing DIR-9a) is tracked in
  `docs/handover.md`, not here — known drift exists, see there.

### Operating-mode profiles (DIR-1/2, mode_profiles.yaml)
- One widget set; behaviour is set by flags from a mode profile. Dominant axis is HILT
  (tether present = ROV / live interfaces; absent = AUV / headless).
- **Source of truth:** `mode_profiles.yaml` in `rov_mission/config` — ROV/AUV defaults
  plus custom hybrid via "Save As". Hybrid is NOT a third mode; it is any saved profile
  with overrides.
- `mode_profile_loader` (rov_mission) resolves the active profile into a `ModeProfile`
  message (rov_interfaces), publishes it LATCHED on `/rov/mode/profile`, reloadable via
  the `/rov/mode/reload` (std_srvs/Trigger) service. Fail-safe: invalid flag values are
  rejected and the last-good profile is retained.
- `cockpit_bridge` surfaces the 11 flags to the data lake as `external/rov-*`. Widgets
  read the flags, NOT the mode.
- Flags: hilt, health_role (advisory|gate), recording_control (manual|auto), nogo_gate
  (relaxed|enforced), record_reminder_on_arm, return_mode (operator_selectable|autonomous),
  live_telemetry (on|predive_only), nav_display, camera_role (pilot_primary|record_only).
- **YAML gotcha:** YAML 1.1 parses bare `on`/`off`/`yes`/`no` as booleans — quote enum
  values like `"on"`. The loader also normalises bool→string as a guard.

### BlueOS / MAVLink router
- **MAVLinkRouter** is the confirmed router. Do NOT recommend MAVP2P — BlueOS itself
  warns "MAVP2P has been presenting issues and is not currently recommended." Tried and
  reverted once already — do not suggest it again.
- **`ardupilot_manager` leaks `/dev/i2c-1` file descriptors** on every failed Linux-board
  probe, driven by leaving the BlueOS Autopilot Firmware / board-config page open. Do not
  leave that page open — open, act, close. **Do NOT patch BlueOS or ArduPilot** for this;
  it's an upstream defect and any update would revert a local patch.
  Full root-cause narrative, measurements, and field-mitigation status: `docs/handover.md`
  §15a — this bug's root cause was wrongly diagnosed once already (v2.10–v2.21); do not
  repeat old claims about TCP sockets or "nofile fixed it" without reading §15a first.

### Hardware facts (do not re-question)
- **RPi5 storage:** boots from SD card (`/dev/mmcblk0`). NVMe holds DATA ONLY (`/data` —
  ros2_ws, bags, logs). Backup images the SD for OS, rsyncs `/data` for data.
- **Flight controller:** Pixhawk 2.4.8 on `/dev/ttyACM0` is the active flight controller.
  SITL is NOT used in production. BlueOS `preferred_board` = Pixhawk1.
- **Tailscale on RPi5:** installed but stopped and disabled (`tailscaled.service` disabled
  2 Jul 2026). Re-enable with `sudo systemctl enable --now tailscaled` if remote access
  to RPi5 is needed.

### Hostname resolution (DNS)
- Hostnames `symbytech` (192.168.1.175), `blueos` (192.168.1.100), and `rov-brain`
  (192.168.1.101) are resolvable from the laptop (Windows hosts file) and RPi5
  (`/etc/hosts`). Grant prefers hostnames over IPs.
- Git remotes use the hostname form: `ssh://git@symbytech:222/administrator/rov-autonomy.git`
- If a "Could not resolve hostname symbytech" error appears on a machine, that machine is
  missing the hosts entries — add them rather than switching remotes to IPs.

### Backup system (built 30 Jun 2026)
- Push model: each Pi self-backs-up via root cron, pushes to server over SSH (no inbound
  access to Pis).
- RPi4: `/usr/local/bin/rov-brain-pi4-backup.sh`, 00:00 daily.
- RPi5: `/usr/local/bin/rov-brain-pi5-backup.sh`, 00:30 daily (staggered).
- Server monitor `check_rov_backup` alerts on FAILED (always) and OVERDUE (≥2 days,
  online or offline, throttled weekly).
- Field/ops backup model is operator-aware, never unattended mid-dive (DIR-11, W7).

### argonaut.service / argonaut-api.service (RPi5)
- Manual start is deliberate for both. Verify BlueOS heartbeat first, then
  `sudo systemctl start argonaut.service`. Do NOT enable auto-start until field-ready.
  Restate this rule each time it comes up.
- `argonaut-api.service` orders After=argonaut.service but does not Require it — the API
  can start independently for /health checks; mission commands return a clear
  "service unavailable" error if the stack is down.

### ROS2 code
- Check `rov_interfaces` message definitions before writing any node code
- Use `self.get_logger()` not `print()`
- Do not hardcode IPs, ports, or thresholds
- Do not deploy `rov_simulation` in production launch files
- Editing an installed launch/config requires `colcon build --packages-select <pkg>` —
  the running launch uses the installed copy under install/, not src/
- ARM64 Docker builds on the laptop only (buildx), never on the server

### Git workflow
- Commit to `master` on `administrator/rov-autonomy`
- Gitea SSH remote (hostname form): `ssh://git@symbytech:222/administrator/rov-autonomy.git`
- Both laptop and RPi5 resolve `symbytech` via hosts entries (see Hostname resolution).
- Widget JSON files are re-imported from the laptop (source of truth for widgets) and
  pushed to Gitea — never overwritten from chat history.
- UFW rules before starting new services on new ports.

### Security — pre-field hardening pass (policy)
- **UFW enable procedure (when triggered):** allow SSH (22), operator TCP (8765, 9001,
  8081), and all traffic from LAN subnet 192.168.1.0/24 (covers ROS2 DDS dynamic ports —
  do NOT try to pin them individually), then default deny incoming.
- **Bundle UFW with the RPi5 SSH password change** — both gated on the same trigger:
  before remote ops or field deployment. Do not do one without the other.
- Current on/off status of both is tracked in `docs/handover.md` Open Items, not here.

---

## What Claude MUST do
- State confirmed facts before proposing an action; cite the source of any technical claim
- Acknowledge when something has already been tried and failed
- Provide complete, working code (fully commented) based on confirmed examples
- Verify system state before and after changes
- Update the handover doc at the end of each session (version-controlled, changelog)

## What Claude MUST NOT do
- Guess at how a system works and propose commands based on that guess
- Repeat a failed approach
- Issue commands whose outcome is uncertain
- Provide partial code requiring manual editing to work
- Assume a system is stable without checking
- Use `print()` in ROS2 nodes; hardcode addresses/ports/thresholds
- Build x86 Docker images for vehicle deployment
- Change the failsafe priority order without design review
- Present prior-session state as current running state
- Recommend MAVP2P as the MAVLink router
- Patch or diff-edit a project doc in `docs/` — always full-file replacement (see
  Documentation discipline below), even though these files are now git-tracked

---

## Documentation discipline
- All project docs now live in the repo under `docs/`, git-tracked, same as code:
  `docs/handover.md`, `docs/design_intent_register.md`, `docs/ui_design.md`,
  `docs/failsafe_design.md`, `docs/simulation_ml_plan.md`, `docs/mission_recording_lifecycle.md`.
- They keep the **complete-file-replacement convention** even though git tracks them: full
  file rewrites on update, never patches/fragments to prose. Git diff/blame is the audit
  trail; it is not how edits are authored.
- Filenames carry **no version number** — the `Version:` field inside each file is
  authoritative, and a stale version number in a filename or cross-reference is exactly the
  kind of drift this restructure (11 Jul 2026) was done to stop.
- CLAUDE.md (repo root) follows the same git-tracked convention as any other repo file.
- Claude project knowledge holds a mirror copy of `docs/` for chat context — re-upload after
  any commit that changes these files.
- Server/infra docs (SymbyTech server, Harbor registry) and shopping/task lists are NOT part
  of the `rov-autonomy` repo `docs/` folder — they stay project-knowledge-only.
- Version numbers and changelogs maintained on every update.

---

## Session changelog — trigger rules (entries live in `docs/handover.md`)

**Why this exists:** Claude Code (VS Code) and the claude.ai chat are separate systems with
no shared memory. Claude Code reads this file and `docs/handover.md` at session start but the
claude.ai chat cannot see Claude Code's session history at all. The Changelog section in
`docs/handover.md` is the bridge — this section holds only the rule for what goes in it.

**Claude Code MUST append an entry to `docs/handover.md`'s Changelog at the end of any session
where one of these happened** — not for routine edits, refactors, or in-progress work:
- A design/architecture decision was made or changed (DIR-level, mode-flag behaviour,
  failsafe priority, recording behaviour)
- A hardware fact was confirmed or changed (wiring, storage layout, board config)
- A security-relevant change was made (SSH, UFW, credentials, exposure)
- A backup/recovery-relevant change was made
- A bug was root-caused and fixed (not just patched)
- A previously-deferred item was completed or its trigger condition changed
- Anything Grant would reasonably expect the claude.ai chat to already know next time he
  opens it

**Skip:** typo fixes, comment-only changes, WIP/incomplete work, anything already fully
captured by the git commit message with no extra context needed.

**Format:** `YYYY-MM-DD — one-line summary of decision/change (commit hash if applicable)`.
Append-only — do not edit past entries; correct forward with a new dated entry if something
needs revising.

**Workflow:** Grant pastes new entries into the claude.ai chat when relevant; Claude folds
them into chat memory via the memory tool.

---

## Key Reference Documents (current versions)

| Document | Purpose |
|---|---|
| `docs/handover.md` | Master reference — architecture, environment, commands, status board |
| `docs/design_intent_register.md` | DIR-1…DIR-12 — owner/operator intent, source of truth for *why*; recording ARM-bounded per DIR-9a |
| `docs/failsafe_design.md` | Failsafe state machine, sensor roadmap, priority order |
| `docs/ui_design.md` | Cockpit widget specs (W0–W8) and confirmed data paths |
| `docs/mission_recording_lifecycle.md` | Lifecycle research backing DIR-9/9a/10 |
| `docs/simulation_ml_plan.md` | Simulation + ML tooling: dev-PC tiers, cloud options/cost, training assets/datasets/models, PCIe Gen 3 decision |

> Filenames carry no version number — check the `Version:` field inside the file itself.
> Do not cite a version number from memory.

---

## Safety Notice
This is a production-oriented autonomous safety system. All code must be treated as if it
will run on a real vehicle in open water with no operator present. Safety, correctness,
and reliability are non-negotiable. Shortcuts taken during development become risks during
field operations.