151 lines
12 KiB
Markdown
151 lines
12 KiB
Markdown
# Argonaut 3 — Simulation & ML Tooling: Hardware, Cloud & Training-Asset Plan
|
||
**Version:** 1.0
|
||
**Date:** July 2026
|
||
**Status:** Scoping reference. Companion to the underwater-simulation research report. Registers the compute/platform decisions that report deferred to hardware, plus training-asset sourcing and the PCIe Gen 3 option. Nothing here is purchased or committed — this is the options board for scoping, not an implementation order.
|
||
|
||
> **Scope note.** KISS applies to *implementing a chosen path*, not to scoping. This document deliberately lays out options and trade-offs so a decision can be made on facts, not narrowed prematurely.
|
||
|
||
---
|
||
|
||
## 1. The two tracks and where they converge
|
||
|
||
- **Track 1 — Control / navigation / mission (simulation-in-the-loop).** Bench-test the ROS2 autonomy stack (`mission_executor`, failsafe, nav, recording gate) against a simulated vehicle + world, running the *same* ArduSub firmware (as SITL) and MAVROS the real vehicle uses. **CPU-bound. Modest GPU.**
|
||
- **Track 2 — Perception / machine learning.** Generate photorealistic, auto-labelled synthetic imagery of structures and train feature-recognition models (VDMs, horizontals, K/X nodes, anodes). **GPU-bound. Needs a real RTX GPU.**
|
||
- **Convergence (the real bottleneck).** Track 2's trained detections become landmarks for **structure-relative visual localization** against the known CAD model — the actual unsolved foundational problem. A simulator lets this convergence be developed and validated before pool/field time.
|
||
|
||
The two tracks have different hardware centres of gravity. That is why hardware, not software, is the gating decision.
|
||
|
||
---
|
||
|
||
## 2. Simulator → hardware → stage → intent
|
||
|
||
| Simulator | Track | ROS2 Jazzy | Runs on | Real intent |
|
||
|---|---|---|---|---|
|
||
| **Gazebo Harmonic + `ardupilot_gazebo` (ArduSub SITL)** | 1 | Native | Laptop / dev PC (CPU-led, modest GPU) | Nav/mission/failsafe bench with real firmware + MAVROS, no hardware |
|
||
| **Stonefish** | 1 (higher-fidelity hydro) | Wrapper | Dev PC | Better hydrodynamics/sensor sim when Gazebo is too coarse |
|
||
| **HoloOcean / UNav-Sim (Unreal 5)** | 2 | Python / bridge | Dev PC **with RTX** | Photorealistic labelled synthetic imagery for perception training |
|
||
| **Isaac Sim + OceanSim** | 2 (+ RL) | Native (Isaac ROS) | Dev PC **with RTX 4080+** | Large-scale synthetic data, domain randomization, RL if pursued |
|
||
|
||
Track 1 runs on hardware already owned (a laptop, once it isn't fighting a VM). Track 2 is what forces a GPU purchase-or-rent decision.
|
||
|
||
---
|
||
|
||
## 3. Why the past SITL failures do NOT block this
|
||
|
||
Recorded facts, not memory:
|
||
|
||
- SITL failed in **two under-resourced hosts**: the server BlueOS VM (66–70% CPU on 2 vCPUs, May 2026) and the **RPi4**, where on 22 Jun 2026 a *fresh SD card* still showed `connected: false` — proving the SD was **not** the root cause. The real cause was RPi4 resource strain + a half-connected Pixhawk making BlueOS thrash between real board and SITL.
|
||
- The laptop's own pain was separate: Windows `VirtualMachinePlatform`/`vmmem` consuming ~4 GB RAM / 55% CPU.
|
||
|
||
None of that is intrinsic to SITL. On an adequate dev PC with nothing competing for the FCU, **SITL + Gazebo is the standard, well-supported bench** (`bluerov2_gz` / `orca4` reference stacks). SITL is *software in place of* the Pixhawk — in simulation the physical Pixhawk is not used at all; that is the point.
|
||
|
||
---
|
||
|
||
## 4. Dev PC — minimum / recommended / high-end
|
||
|
||
VRAM is the gating spec. 16 GB is the practical sweet spot; 24 GB+ buys headroom and RL capacity. All three tiers run Track 1 (Gazebo) comfortably — the tiers are set by the Track 2 GPU load.
|
||
|
||
| Tier | CPU / RAM | GPU (VRAM) | ~USD (box) | What it unlocks |
|
||
|---|---|---|---|---|
|
||
| **Minimum** | 6–8 core (Ryzen 7 / i5-13600K), 32 GB | RTX 4060 Ti **16 GB** or RTX 4070 12 GB | $1,200–1,600 | Gazebo comfortably; HoloOcean/UNav-Sim OK; YOLO/SAM fine-tuning fine. Isaac Sim runs but is *below its stated RTX 4080 minimum* (unsupported, may glitch). |
|
||
| **Recommended** | 8+ core (Ryzen 9 / i7-14700K), 64 GB | RTX 4080 Super 16 GB or RTX 5070 Ti / 5080 **16 GB** | $2,000–2,800 | Meets Isaac Sim's stated minimum. Comfortable synthetic-data generation + training across all Track 2 sims. The sensible target if buying once. |
|
||
| **High-end** | 12+ core (Ryzen 9 / i9 / Threadripper), 64–128 GB | RTX 4090 24 GB / RTX 5090 **32 GB** (or RTX 6000 Ada 48 GB pro) | $3,500–6,000+ | Fast iteration, large Replicator/synthetic batches, RL training, big scenes. Overkill until Track 2 is a daily activity. |
|
||
|
||
Hard facts behind the tiers:
|
||
- **Unreal Engine 5** (HoloOcean/UNav-Sim): playable from ~6 GB VRAM; Epic/Puget recommend a strong RTX card with ample VRAM for Nanite/Lumen work.
|
||
- **Isaac Sim (current docs):** minimum **RTX 4080, 16 GB VRAM**, 32–64 GB RAM; **RT cores required — A100/H100 are NOT supported**; 8 GB VRAM is insufficient for complex scenes.
|
||
- **ML fine-tuning** (YOLO / SAM2 / Grounding DINO): 12–16 GB VRAM comfortable; 24 GB generous.
|
||
|
||
**OS:** Ubuntu (native ROS2 Jazzy + Isaac/Gazebo) is the least-friction path; dual-boot Windows if the laptop's other duties need it. Do **not** run the GPU sim inside a Windows VM — that is a repeat of the `vmmem` problem.
|
||
|
||
**The R210 server cannot host this.** 1U, no slot/power/thermal budget for a modern GPU. It stays a services box. A GPU dev PC is a *separate* machine.
|
||
|
||
---
|
||
|
||
## 5. Cloud GPU — alternative / complement to a dev PC
|
||
|
||
**Critical caveat first:** cloud GPU is a **headless notebook/batch** resource — excellent for *training* and for *batch synthetic-data generation* runs, but a poor fit for **interactive** Unreal/Gazebo GUI work (needs a display server / pixel-streaming and is fiddly). So cloud **complements** a local dev PC (which you still want for interactive Track 1 + Track 2 dev); it does not fully replace one.
|
||
|
||
### Free facilities (no/low cost, good for training + learning)
|
||
| Platform | Free GPU | Allowance | Credit card | Best for |
|
||
|---|---|---|---|---|
|
||
| **Kaggle Notebooks** | P100 16 GB or 2×T4 | ~30 GPU-hr/week, 9-hr sessions, background exec | No | Most reliable free tier; fine-tuning small–mid models |
|
||
| **Google Colab** | T4 16 GB | ~15–30 GPU-hr/week (dynamic), ≤12-hr sessions | No | Quick experiments, LoRA/QLoRA, SD |
|
||
| **Lightning AI** | T4/L4 up to A100 (interruptible) | ~80 GPU-hr/mo credits, persistent IDE | No | Structured PyTorch dev, checkpoint-friendly |
|
||
| **Paperspace Gradient** | M4000 8 GB | 6-hr sessions, unlimited restarts | No | Light notebooks |
|
||
| **SageMaker Studio Lab** | T4 | 4-hr sessions | No | ⚠ AWS closing new signups **30 Jul 2026** |
|
||
| **Hyperscaler trial credits** | varies | GCP $300/90d, Azure $200/30d, AWS Activate (startups) | Yes | Short benchmarking bursts; GPU quota often refused on new accounts |
|
||
|
||
### Paid on-demand ($/GPU-hr, mid-2026 snapshot — verify live at deploy)
|
||
| GPU (VRAM) | Cheapest (Vast.ai / RunPod community) | Mid (Lambda / RunPod secure) | Use |
|
||
|---|---|---|---|
|
||
| RTX 4090 (24 GB) | ~$0.31–0.34 | ~$0.59 | Perception training, synthetic-data batch, ≤13B inference |
|
||
| A100 (40/80 GB) | ~$0.60–1.07 | ~$1.29–1.99 | Larger training / fine-tuning |
|
||
| H100 (80 GB) | ~$1.03–2.25 | ~$2.49–3.29 | Only if scaling large / RL |
|
||
|
||
Cheapest providers: **Vast.ai, RunPod, TensorDock, Lambda**. Hyperscalers (AWS/GCP/Azure) run 2–5× more and price whole VMs, not naked GPUs.
|
||
|
||
**Practical read:** a solo developer can do the entire perception-training track on **Kaggle/Colab free + occasional RunPod/Vast RTX 4090 at ~$0.33/hr** with *zero* capex, and defer or skip the dev-PC GPU. The dev PC earns its cost only when interactive sim work becomes frequent. This is a genuine fork worth deciding on cost, not reflex.
|
||
|
||
---
|
||
|
||
## 6. Models & data for training (paid + free)
|
||
|
||
Three distinct things get called "models." All three are needed.
|
||
|
||
### A. 3D assets to build simulated scenes (drop into the sim)
|
||
- **Free:** **Fab** (Epic's marketplace; **Quixel Megascans now free**), **Poly Haven** (CC0), **Sketchfab** (CC-filtered), **BlenderKit** (free tier), **NVIDIA SimReady / Omniverse** assets.
|
||
- **Paid:** **TurboSquid**, **CGTrader**, Fab premium.
|
||
- **Key point:** the actual structure CAD (EBM/NWD → IFC → glTF/GLB) is already owned — that is the irreplaceable asset. These libraries only fill the *surroundings* (seabed, marine growth, debris, props) for domain randomization.
|
||
|
||
### B. Datasets to train/transfer-learn on (free, real underwater)
|
||
| Dataset | Content | Relevance |
|
||
|---|---|---|
|
||
| **SubPipe** | Submarine pipeline inspection — RGB + side-scan sonar + DVL + INS, annotated | Closest real structure-inspection set; matches sensor suite direction |
|
||
| **COU (Common Objects Underwater)** | ~10k instance-segmented man-made objects | Man-made object seg, AUV-oriented |
|
||
| **TrashCan / Seaclear / Brackish / SUIM** | Debris, marine life, segmentation | General underwater vision pre-training / augmentation |
|
||
| **MIMIR-UW** | *Simulated* pipe inspection with labels | Sim-to-real reference |
|
||
|
||
> **Honest finding:** **no public dataset covers offshore jacket features (VDMs, K/X nodes, anodes).** That gap is exactly why the CAD-model + synthetic-data pipeline is the path — synthesize labelled jacket imagery from the owned CAD, pre-train/transfer from the sets above, then fine-tune on real field captures as they accumulate.
|
||
|
||
### C. Base ML models to fine-tune (free / open weights)
|
||
- **Ultralytics YOLO** (v8/v11) — detection + segmentation, light enough for on-vehicle ONNX/TFLite later.
|
||
- **SAM2** (Meta) — promptable segmentation; underwater-adapted variants exist (e.g. SAM-U).
|
||
- **Grounding DINO** — open-vocabulary detection, useful for bootstrapping labels.
|
||
|
||
These are the *starting weights*; you fine-tune them on synthetic (from A/CAD) + real (from B and field) data.
|
||
|
||
---
|
||
|
||
## 7. PCIe Gen 3 on the RPi5 (data drive) — option, logged
|
||
|
||
Kept brief per direction. This is future-proofing, not a current need.
|
||
|
||
| | Fact |
|
||
|---|---|
|
||
| How | `dtparam=pciex1_gen=3` in `/boot/firmware/config.txt`, reboot |
|
||
| Throughput | PCIe 2.0 x1 ≈ ~500 MB/s → PCIe 3.0 x1 ≈ ~900 MB/s (~2×) |
|
||
| Certification | RPi5 is **officially certified only at Gen 2**; Gen 3 is **enabled but not validated** by Raspberry Pi |
|
||
| Scope | Affects the **NVMe data drive** (`/data`) only — boot is on SD, so a bad link corrupts bags/logs, not boot |
|
||
|
||
**Pros:** ~2× NVMe throughput; one-line, reversible; free headroom for when camera streams are recorded to MCAP.
|
||
**Cons:** uncertified — some drives show link dropouts/corruption at Gen 3; **current benefit ≈ nil** (MCAP is nav/MAVLink only, a trickle Gen 2 already over-serves); the drive holds recorded dive data.
|
||
|
||
**Decision:** **Defer.** Revisit at **Phase 3 camera-to-MCAP integration**, and only enable after a stability soak on the specific drive (the official RPi SSD is the likeliest to be stable at Gen 3). Not worth touching for the current nav-only workload.
|
||
|
||
---
|
||
|
||
## 8. Effort read (solo developer)
|
||
|
||
- **Track 1 (Gazebo bench):** near-term, achievable, **needs no purchase to start** — the current laptop runs it once it isn't hosting a VM. Highest-value near-term win: bench-test mission/failsafe logic without hardware.
|
||
- **Track 2 (perception + training):** the harder, longer track. Start on **free cloud (Kaggle/Colab) + occasional rented RTX 4090**; buy the **Recommended dev PC** only when interactive sim/data-gen becomes frequent.
|
||
- **Convergence (structure-relative localization):** the genuine multi-year problem. Sim de-risks it; it is not a quick win.
|
||
- **RL:** research-grade. Defer until perception + localization are working.
|
||
|
||
---
|
||
|
||
## Changelog
|
||
| Version | Date | Change |
|
||
|---|---|---|
|
||
| 1.0 | July 2026 | Initial. Simulator→hardware→stage mapping; dev-PC tiers (min/rec/high); cloud free + paid options with mid-2026 pricing; 3D-asset/dataset/base-model sourcing; PCIe Gen 3 decision (defer). Corrects the record on why past SITL failures do not block a dev-PC SITL+Gazebo bench. |
|