12 KiB
Argonaut 3 — Simulation & ML Tooling: Hardware, Cloud & Training-Asset Plan
Version: 1.0 Date: July 2026 Status: Scoping reference. Companion to the underwater-simulation research report. Registers the compute/platform decisions that report deferred to hardware, plus training-asset sourcing and the PCIe Gen 3 option. Nothing here is purchased or committed — this is the options board for scoping, not an implementation order.
Scope note. KISS applies to implementing a chosen path, not to scoping. This document deliberately lays out options and trade-offs so a decision can be made on facts, not narrowed prematurely.
1. The two tracks and where they converge
- Track 1 — Control / navigation / mission (simulation-in-the-loop). Bench-test the ROS2 autonomy stack (
mission_executor, failsafe, nav, recording gate) against a simulated vehicle + world, running the same ArduSub firmware (as SITL) and MAVROS the real vehicle uses. CPU-bound. Modest GPU. - Track 2 — Perception / machine learning. Generate photorealistic, auto-labelled synthetic imagery of structures and train feature-recognition models (VDMs, horizontals, K/X nodes, anodes). GPU-bound. Needs a real RTX GPU.
- Convergence (the real bottleneck). Track 2's trained detections become landmarks for structure-relative visual localization against the known CAD model — the actual unsolved foundational problem. A simulator lets this convergence be developed and validated before pool/field time.
The two tracks have different hardware centres of gravity. That is why hardware, not software, is the gating decision.
2. Simulator → hardware → stage → intent
| Simulator | Track | ROS2 Jazzy | Runs on | Real intent |
|---|---|---|---|---|
Gazebo Harmonic + ardupilot_gazebo (ArduSub SITL) |
1 | Native | Laptop / dev PC (CPU-led, modest GPU) | Nav/mission/failsafe bench with real firmware + MAVROS, no hardware |
| Stonefish | 1 (higher-fidelity hydro) | Wrapper | Dev PC | Better hydrodynamics/sensor sim when Gazebo is too coarse |
| HoloOcean / UNav-Sim (Unreal 5) | 2 | Python / bridge | Dev PC with RTX | Photorealistic labelled synthetic imagery for perception training |
| Isaac Sim + OceanSim | 2 (+ RL) | Native (Isaac ROS) | Dev PC with RTX 4080+ | Large-scale synthetic data, domain randomization, RL if pursued |
Track 1 runs on hardware already owned (a laptop, once it isn't fighting a VM). Track 2 is what forces a GPU purchase-or-rent decision.
3. Why the past SITL failures do NOT block this
Recorded facts, not memory:
- SITL failed in two under-resourced hosts: the server BlueOS VM (66–70% CPU on 2 vCPUs, May 2026) and the RPi4, where on 22 Jun 2026 a fresh SD card still showed
connected: false— proving the SD was not the root cause. The real cause was RPi4 resource strain + a half-connected Pixhawk making BlueOS thrash between real board and SITL. - The laptop's own pain was separate: Windows
VirtualMachinePlatform/vmmemconsuming ~4 GB RAM / 55% CPU.
None of that is intrinsic to SITL. On an adequate dev PC with nothing competing for the FCU, SITL + Gazebo is the standard, well-supported bench (bluerov2_gz / orca4 reference stacks). SITL is software in place of the Pixhawk — in simulation the physical Pixhawk is not used at all; that is the point.
4. Dev PC — minimum / recommended / high-end
VRAM is the gating spec. 16 GB is the practical sweet spot; 24 GB+ buys headroom and RL capacity. All three tiers run Track 1 (Gazebo) comfortably — the tiers are set by the Track 2 GPU load.
| Tier | CPU / RAM | GPU (VRAM) | ~USD (box) | What it unlocks |
|---|---|---|---|---|
| Minimum | 6–8 core (Ryzen 7 / i5-13600K), 32 GB | RTX 4060 Ti 16 GB or RTX 4070 12 GB | $1,200–1,600 | Gazebo comfortably; HoloOcean/UNav-Sim OK; YOLO/SAM fine-tuning fine. Isaac Sim runs but is below its stated RTX 4080 minimum (unsupported, may glitch). |
| Recommended | 8+ core (Ryzen 9 / i7-14700K), 64 GB | RTX 4080 Super 16 GB or RTX 5070 Ti / 5080 16 GB | $2,000–2,800 | Meets Isaac Sim's stated minimum. Comfortable synthetic-data generation + training across all Track 2 sims. The sensible target if buying once. |
| High-end | 12+ core (Ryzen 9 / i9 / Threadripper), 64–128 GB | RTX 4090 24 GB / RTX 5090 32 GB (or RTX 6000 Ada 48 GB pro) | $3,500–6,000+ | Fast iteration, large Replicator/synthetic batches, RL training, big scenes. Overkill until Track 2 is a daily activity. |
Hard facts behind the tiers:
- Unreal Engine 5 (HoloOcean/UNav-Sim): playable from ~6 GB VRAM; Epic/Puget recommend a strong RTX card with ample VRAM for Nanite/Lumen work.
- Isaac Sim (current docs): minimum RTX 4080, 16 GB VRAM, 32–64 GB RAM; RT cores required — A100/H100 are NOT supported; 8 GB VRAM is insufficient for complex scenes.
- ML fine-tuning (YOLO / SAM2 / Grounding DINO): 12–16 GB VRAM comfortable; 24 GB generous.
OS: Ubuntu (native ROS2 Jazzy + Isaac/Gazebo) is the least-friction path; dual-boot Windows if the laptop's other duties need it. Do not run the GPU sim inside a Windows VM — that is a repeat of the vmmem problem.
The R210 server cannot host this. 1U, no slot/power/thermal budget for a modern GPU. It stays a services box. A GPU dev PC is a separate machine.
5. Cloud GPU — alternative / complement to a dev PC
Critical caveat first: cloud GPU is a headless notebook/batch resource — excellent for training and for batch synthetic-data generation runs, but a poor fit for interactive Unreal/Gazebo GUI work (needs a display server / pixel-streaming and is fiddly). So cloud complements a local dev PC (which you still want for interactive Track 1 + Track 2 dev); it does not fully replace one.
Free facilities (no/low cost, good for training + learning)
| Platform | Free GPU | Allowance | Credit card | Best for |
|---|---|---|---|---|
| Kaggle Notebooks | P100 16 GB or 2×T4 | ~30 GPU-hr/week, 9-hr sessions, background exec | No | Most reliable free tier; fine-tuning small–mid models |
| Google Colab | T4 16 GB | ~15–30 GPU-hr/week (dynamic), ≤12-hr sessions | No | Quick experiments, LoRA/QLoRA, SD |
| Lightning AI | T4/L4 up to A100 (interruptible) | ~80 GPU-hr/mo credits, persistent IDE | No | Structured PyTorch dev, checkpoint-friendly |
| Paperspace Gradient | M4000 8 GB | 6-hr sessions, unlimited restarts | No | Light notebooks |
| SageMaker Studio Lab | T4 | 4-hr sessions | No | ⚠ AWS closing new signups 30 Jul 2026 |
| Hyperscaler trial credits | varies | GCP $300/90d, Azure $200/30d, AWS Activate (startups) | Yes | Short benchmarking bursts; GPU quota often refused on new accounts |
Paid on-demand ($/GPU-hr, mid-2026 snapshot — verify live at deploy)
| GPU (VRAM) | Cheapest (Vast.ai / RunPod community) | Mid (Lambda / RunPod secure) | Use |
|---|---|---|---|
| RTX 4090 (24 GB) | ~$0.31–0.34 | ~$0.59 | Perception training, synthetic-data batch, ≤13B inference |
| A100 (40/80 GB) | ~$0.60–1.07 | ~$1.29–1.99 | Larger training / fine-tuning |
| H100 (80 GB) | ~$1.03–2.25 | ~$2.49–3.29 | Only if scaling large / RL |
Cheapest providers: Vast.ai, RunPod, TensorDock, Lambda. Hyperscalers (AWS/GCP/Azure) run 2–5× more and price whole VMs, not naked GPUs.
Practical read: a solo developer can do the entire perception-training track on Kaggle/Colab free + occasional RunPod/Vast RTX 4090 at ~$0.33/hr with zero capex, and defer or skip the dev-PC GPU. The dev PC earns its cost only when interactive sim work becomes frequent. This is a genuine fork worth deciding on cost, not reflex.
6. Models & data for training (paid + free)
Three distinct things get called "models." All three are needed.
A. 3D assets to build simulated scenes (drop into the sim)
- Free: Fab (Epic's marketplace; Quixel Megascans now free), Poly Haven (CC0), Sketchfab (CC-filtered), BlenderKit (free tier), NVIDIA SimReady / Omniverse assets.
- Paid: TurboSquid, CGTrader, Fab premium.
- Key point: the actual structure CAD (EBM/NWD → IFC → glTF/GLB) is already owned — that is the irreplaceable asset. These libraries only fill the surroundings (seabed, marine growth, debris, props) for domain randomization.
B. Datasets to train/transfer-learn on (free, real underwater)
| Dataset | Content | Relevance |
|---|---|---|
| SubPipe | Submarine pipeline inspection — RGB + side-scan sonar + DVL + INS, annotated | Closest real structure-inspection set; matches sensor suite direction |
| COU (Common Objects Underwater) | ~10k instance-segmented man-made objects | Man-made object seg, AUV-oriented |
| TrashCan / Seaclear / Brackish / SUIM | Debris, marine life, segmentation | General underwater vision pre-training / augmentation |
| MIMIR-UW | Simulated pipe inspection with labels | Sim-to-real reference |
Honest finding: no public dataset covers offshore jacket features (VDMs, K/X nodes, anodes). That gap is exactly why the CAD-model + synthetic-data pipeline is the path — synthesize labelled jacket imagery from the owned CAD, pre-train/transfer from the sets above, then fine-tune on real field captures as they accumulate.
C. Base ML models to fine-tune (free / open weights)
- Ultralytics YOLO (v8/v11) — detection + segmentation, light enough for on-vehicle ONNX/TFLite later.
- SAM2 (Meta) — promptable segmentation; underwater-adapted variants exist (e.g. SAM-U).
- Grounding DINO — open-vocabulary detection, useful for bootstrapping labels.
These are the starting weights; you fine-tune them on synthetic (from A/CAD) + real (from B and field) data.
7. PCIe Gen 3 on the RPi5 (data drive) — option, logged
Kept brief per direction. This is future-proofing, not a current need.
| Fact | |
|---|---|
| How | dtparam=pciex1_gen=3 in /boot/firmware/config.txt, reboot |
| Throughput | PCIe 2.0 x1 ≈ ~500 MB/s → PCIe 3.0 x1 ≈ ~900 MB/s (~2×) |
| Certification | RPi5 is officially certified only at Gen 2; Gen 3 is enabled but not validated by Raspberry Pi |
| Scope | Affects the NVMe data drive (/data) only — boot is on SD, so a bad link corrupts bags/logs, not boot |
Pros: ~2× NVMe throughput; one-line, reversible; free headroom for when camera streams are recorded to MCAP. Cons: uncertified — some drives show link dropouts/corruption at Gen 3; current benefit ≈ nil (MCAP is nav/MAVLink only, a trickle Gen 2 already over-serves); the drive holds recorded dive data.
Decision: Defer. Revisit at Phase 3 camera-to-MCAP integration, and only enable after a stability soak on the specific drive (the official RPi SSD is the likeliest to be stable at Gen 3). Not worth touching for the current nav-only workload.
8. Effort read (solo developer)
- Track 1 (Gazebo bench): near-term, achievable, needs no purchase to start — the current laptop runs it once it isn't hosting a VM. Highest-value near-term win: bench-test mission/failsafe logic without hardware.
- Track 2 (perception + training): the harder, longer track. Start on free cloud (Kaggle/Colab) + occasional rented RTX 4090; buy the Recommended dev PC only when interactive sim/data-gen becomes frequent.
- Convergence (structure-relative localization): the genuine multi-year problem. Sim de-risks it; it is not a quick win.
- RL: research-grade. Defer until perception + localization are working.
Changelog
| Version | Date | Change |
|---|---|---|
| 1.0 | July 2026 | Initial. Simulator→hardware→stage mapping; dev-PC tiers (min/rec/high); cloud free + paid options with mid-2026 pricing; 3D-asset/dataset/base-model sourcing; PCIe Gen 3 decision (defer). Corrects the record on why past SITL failures do not block a dev-PC SITL+Gazebo bench. |