How robots see: depth cameras, stereo, time of flight, LiDAR and learned vision. How accurate depth must be, which sensor fits which job, whether better AI makes sensors matter less, and who supplies them, with a source for every fact.
Visitors to the map ask the same three things about robot eyes: does a robot need an expensive depth camera, which vision system is best, and will better AI make sensors matter less. The 2025 to 2026 record gives a split answer. Learned models now produce depth that their authors compare favourably with common depth cameras: NVIDIA's FoundationStereo turns two plain images into depth (Wen et al., arXiv, 17 Jan 2025), and VGGT, which infers depth, point maps and camera poses from one or many images, won the CVPR 2025 best paper award (CVPR, 13 Jun 2025). At the same time the money and the volumes go to sensors: Cognex agreed to buy RealSense for about $500 million in September 2026 (Cognex release, 22 Sep 2026), Hesai shipped 239,273 robotics lidars in 2025 (Hesai, SEC exhibit, 24 Mar 2026), and Figure added a camera in each palm of Figure 03 (Figure, 9 Oct 2025). Shiny, transparent and black objects remain the classic failure: in a warehouse cell, a learned depth completion model raised grasp success on black packages from 28% to 91.6% (authors' figures, Zhou et al., arXiv, Mar 2026).
How accurate must depth be for manipulation, and when is a cheap camera enough?
The gripper sets the requirement: "if the point cloud data is highly accurate, a mechanical gripper with narrow tolerances can be utilized", otherwise a suction cup with more compliance may be needed (Zivid knowledge base, bin picking requirements); Zivid's blog gives the example of a suction gripper "that has 5mm of tolerance" where lower-quality data can be handled (company view, Zivid blog).
Fine tasks on low-cost hardware: ALOHA learned tasks such as slotting a battery with 80 to 90% success from 10 minutes of demonstrations, using four Logitech webcams and no depth sensor; the authors note that such tasks "typically requires high-end robots, accurate sensors, or careful calibration" (Zhao, Kumar, Levine, Finn, arXiv, 23 Apr 2023).
UMI collects demonstrations with a hand-held gripper that has "a GoPro as the only sensor and recording device", with side mirrors for implicit stereo (Chi et al., arXiv, 15 Feb 2024).
Which sensing technology for which job?
The three common depth-camera methods: stereo compares two views a known distance apart, structured light projects a known pattern, time of flight times reflected light; many lidars use time of flight too (Michelle Sun, Core Matter, 6 Oct 2026).
In the first Amazon Picking Challenge (2015), "3D sensing in some form was employed by 22 teams, 20 of which using structured light or time-of-flight sensors" (Correll et al., arXiv, 21 Jan 2016).
Mobile robots and humanoids add lidar: Unitree lists "Depth Camera+3D LiDAR" for the G1 at a price of US$13.5K (Unitree); RoboSense sold 303,000 robotics lidars in 2025 and names lawn mowers, delivery and humanoid robots as drivers (RoboSense, 15 Jan 2026).
Can learned models replace depth hardware?
Depth Anything V2 (Jun 2024) trains a monocular depth model on synthetic images and pseudo-labelled real images, in sizes from 25M to 1.3B parameters (Yang et al., arXiv, 13 Jun 2024); Depth Anything 3 (Nov 2025) predicts consistent geometry from any number of views and reports gains over VGGT of 44.3% in camera pose accuracy and 25.1% in geometric accuracy on its own benchmark (Lin et al., arXiv, 13 Nov 2025).
FoundationStereo is trained on 1 million synthetic stereo pairs for zero-shot stereo depth (arXiv, 17 Jan 2025); Fast-FoundationStereo "can run over 10x faster than FoundationStereo while closely matching its zero-shot accuracy" (Wen, Dewan, Birchfield, arXiv, 11 Dec 2025); both are packaged in NVIDIA's Isaac ROS stereo depth packages for robots (Isaac ROS documentation).
VGGT infers camera parameters, point maps, depth maps and 3D point tracks from one to hundreds of views, "reconstructing images in under one second" (Wang et al., arXiv, 14 Mar 2025), and won the CVPR 2025 best paper award (CVPR, 13 Jun 2025); Meta's MapAnything regresses metric 3D geometry and cameras in a single feed-forward pass (Keetha et al., arXiv, 16 Sep 2025).
Will better AI make sensors matter less or more?
Tesla on its robotaxi and humanoid stack: "You can have safe, comfortable and affordable autonomy with just cameras." (Ashok Elluswamy, Tesla Q2 2026 call, 22 Jul 2026, transcript via Mitrade).
Waymo's sixth-generation driver carries "13 cameras, 4 lidar, 6 radar, and an array of external audio receivers" (Waymo, 19 Aug 2024).
Figure 03's "new camera architecture delivers twice the frame rate, one-quarter the latency, and a 60% wider field of view per camera", and each hand gets a palm camera so the model keeps "visual awareness even when the main cameras are occluded" (company claims, Figure, 9 Oct 2025).
Shiny, transparent and dark objects: what works?
The problem is old: in 2015 "Objects with reflective covers are challenging for many depth sensors" (Correll et al., arXiv, 21 Jan 2016); transparent objects "often appear as noisy or distorted approximations of the surfaces that lie behind them" (Sajjan et al., ClearGrasp, arXiv, 6 Oct 2019).
Learned depth repair works in the lab: ClearGrasp raised grasp success on transparent objects from 64% to 86% with suction and from 12% to 72% with a parallel-jaw gripper, using RealSense cameras (ClearGrasp); DREDS simulates an active stereo camera on 130K synthetic images and generalises to real specular and transparent objects (Dai et al., arXiv, 7 Aug 2022).
Black items: "Black packages absorb infrared light", leaving holes in time-of-flight depth; Any2Full was deployed in a warehouse cell "handling thousands of packages daily" and raised grasp success on black packages from 28% to 91.6% (authors' figures, Zhou et al., arXiv, Mar 2026).
Where do cameras go on a robot (head, wrist, external), and why?
Head cameras give a view of the surroundings; "Wrist cameras can give a closer view of a grasp, although the hand or object can still block them." (Michelle Sun, Core Matter, 6 Oct 2026).
Placement was already mixed in 2015: 3D sensors sat on the head (8 teams), the end effector (6), the arm (6) and the torso (3) (Correll et al., arXiv, 21 Jan 2016).
ALOHA uses front, top and two wrist cameras (arXiv, 2023); UMI uses a wrist-mounted 155-degree fisheye GoPro only (arXiv, 2024); Figure 03 adds a camera in each palm (Figure, 9 Oct 2025).
What do perception failures cost in pilots and deployments?
In 2015 the best picking robot managed about 30 sorts an hour with a 16% failure rate, against about 400 for a person; "Lighting in the convention hall also proved to be a problem for some teams", and the authors conclude that system integration remained a fundamental challenge (Correll et al., arXiv, 21 Jan 2016).
Asked what broke most often, a Formic deployment engineer told Michelle Sun: "His answer was sensors."; dirt on sensors is a named failure source, and "keeping sensors usable is part of deployment" (Core Matter, 6 Oct 2026).
A camera wiring error cost Generalist AI "an entire day of debugging" at a new warehouse (Andy Zeng, X, 13 Apr 2026).
Supply and cost: who makes the sensors, how much is Chinese, and where are prices going?
Depth cameras: Orbbec (Shenzhen) had 29.0% of the global market for robot 3D visual perception by 2025 revenue (Frost & Sullivan figure in Orbbec's listing documents), with 2025 revenue of RMB 941 million, and applied for a Hong Kong listing on 24 Sep 2026 (36Kr, 29 Sep 2026); Mech-Mind (Beijing) listed in Hong Kong on 1 Sep 2026 and claims about 22.1% of a narrower AI and 3D vision robot component market (company claim via an investor, Unite.AI); RealSense said at its spin-out that its cameras are "embedded in 60 percent of the world’s AMRs and humanoid robots" (company claim, PCWorld, 11 Jul 2025).
Where the cameras are made: Intel's regulatory page lists China or Thailand as country of origin for the RealSense D400 models and Thailand for the D555 (Michelle Sun, Core Matter, 30 Jul 2026, updated 17 Aug 2026).
Robotics lidar is a Chinese volume business: Hesai shipped 239,273 robotics lidars in 2025, up 425.8% (Hesai, SEC exhibit, 24 Mar 2026); RoboSense sold 303,000, up 1,141.8% (RoboSense, 15 Jan 2026).