Annie Bhalla · Dennis Rotondi · Riccardo Giubilato · Kai O. Arras · Wolfgang Stuerzl
University of Stuttgart · German Aerospace Center (DLR)
Every location on a planetary surface looks the same.
What if the robot could tell them apart by what the rock is made of?
The Problem
Planetary rovers must navigate and map unknown terrain autonomously — a process called SLAM (Simultaneous Localisation and Mapping). As the rover moves, small errors in motion estimation accumulate. The only way to correct this drift is loop closure: recognising a location the robot has visited before, then snapping the map back into consistency.
The failure mode has a name: perceptual aliasing. On planetary terrain — craters, regolith, scattered rocks — visually distinct locations appear identical to RGB cameras. A system that cannot tell the difference cannot close the loop, and a map that cannot close the loop drifts until it is useless.
The Hypothesis
Rocks that look identical in RGB often have different mineral composition — and minerals have distinctive spectral signatures in the infrared. Hyperspectral cameras capture tens to hundreds of narrow wavelength bands per pixel, effectively encoding what a surface is made of rather than how it appears.
HyperLoop is the first framework to integrate hyperspectral imaging into the loop closure module of a 3D Gaussian Splatting SLAM system. The idea: pair every RGB descriptor with a spectral one. When both agree a location has been seen before, trust it. When they disagree, reject it.
Architecture
HyperLoop extends LoopSplat — a 3DGS-based dense visual SLAM system with explicit loop closure — by adding a parallel hyperspectral branch.
- ›SIFT keypoints → 128-dim local descriptors
- ›K-Means visual codebook (K = 8)
- ›VLAD aggregation → global descriptor
- ›NetVLAD (baseline) or SIFT-VLAD (proposed)
- ›Ximea IMEC sensor · 15 bands · 600–860 nm
- ›Spatial downsampling + per-pixel spectral normalisation
- ›PCA compression: 15D → 6D (95% variance retained)
- ›BoSW (TF-IDF histogram) or PCAK-VLAD (residuals)
- ›Strict-AND: both modalities must independently exceed threshold
- ›Weighted: β·S_RGB + (1-β)·S_HSI, β = 0.4
- ›Thresholds set adaptively (top-30% within-submap similarity)
- ›Accepted candidates → geometric check → Pose Graph Optimisation
Decision-level fusion is chosen because the HSI sensor (40°×20° FOV) covers only a central sub-region of the RGB frame (84°×84°). Operating on per-frame descriptors avoids the need for precise extrinsic calibration between the two sensors.
Dataset — MMOTS, DLR
Data was collected at the Moon-Mars Outdoor Test Site (MMOTS) of DLR, Oberpfaffenhofen — a purpose-built planetary analog environment with regolith-like soil, scattered rocks, and craters.
| Sequence | Frames | Duration | Distance | Role |
|---|---|---|---|---|
| MMOTS-WS | 1,072 | 133.9 s | 8 m | Primary evaluation |
| MMOTS-LL | 1,385 | 172.9 s | 15 m | Vocabulary training |
| MMOTS-RL | 945 | 119.1 s | 18 m | Vocabulary training |
| MMOTS-US | 1,540 | 186.0 s | 24.5 m | Vocabulary training |
| SS-Sweden | 2,100 | 262.2 s | 123.3 m | Spectral diversity |
Results
| Exp. | Configuration | ATE RMSE |
|---|---|---|
| E1 | LoopSplat default (unmodified) | 3380.89 cm |
| E2 | NetVLAD, outdoor-tuned | 8.81 cm |
| E4-B | SIFT-VLAD + BoSW, Strict-AND | 5.04 cm |
| E3 | SIFT-VLAD only (best overall) | 4.95 cm |
The 43% improvement from E2 to E4-B comes entirely from swapping NetVLAD for SIFT-VLAD. Best fusion (E4-B) stays within 1.8% of best RGB-only (E3). Hyperspectral fusion neither helps nor hurts — understanding why is the central finding.
The Key Finding
The Red-NIR spectral range (600–860 nm) provides limited discriminability for planetary analog terrain. The sensor was designed for agricultural vegetation — its range yields almost no material-specific information for rocks and regolith.
Planetary minerals — pyroxene, olivine, feldspar, phyllosilicates — have diagnostic absorption features only in SWIR (900–2500 nm). The framework is ready. The field needs the right sensor.