Nearly 200 Degrees of View: Testing SOTA Depth Estimation Models on Fisheye
Researchers from Habr's ML hub created WideDepth, a benchmark for evaluating depth estimation models on fisheye images. Testing 14 state-of-the-art models revealed that monocular models are particularly sensitive to ultra-wide fields of view, with errors increasing by over 160% when going from 120° to 195°. The benchmark enables controlled comparison of models across different FOVs, stereo baselines, and camera orientations.
The WideDepth benchmark was created to evaluate depth estimation models on fisheye images, addressing the lack of suitable real indoor fisheye datasets. It is based on 101 office scenes captured with a terrestrial laser scanner, providing ground truth depth with millimeter accuracy, and offers images with horizontal FOVs of 120°, 140°, 165°, and 195°, stereo baselines from 20 to 300 mm, and both horizontal and vertical camera orientations. Tests on 14 SOTA models showed that monocular models like Depth Anything V2 and PatchFusion see their AbsRel error jump by 166% and 160% respectively when FOV increases from 120° to 195°, while Depth Pro is more robust with only a 9% increase. For stereo matching, the equirectangular projection proved significantly better than cubemap, reducing EPE error from 4.50 to 1.07 pixels. Fine-tuning the compact BGNet model on fisheye data improved the percentage of pixels with disparity error over 3 pixels from 24.3% to 9.3%, a 62% improvement. The benchmark also includes a training set of 18,000 outdoor stereo pairs captured with handheld sensors to test model adaptation to the fisheye domain.
- Abbreviations
- FOV = Field of View — поле зрения
- AbsRel = Absolute Relative Error — абсолютная относительная ошибка
- EPE = End-Point Error — ошибка конечной точки
- SOTA = State of the Art — современный уровень
- RGB = Red Green Blue — цветовая модель
- LiDAR = Light Detection and Ranging — лазерное сканирование
- CUDA = Compute Unified Device Architecture — архитектура NVIDIA
- ROS = Robot Operating System — операционная система для роботов
Source: Habr — хаб ML —
original