A person can walk into a room and almost instantly understand that there is a table, a door, a cup, another person and a clear path between the furniture. We rarely think about how much processing happens in that moment: the eyes provide images, the brain estimates distance, recognizes objects, separates moving things from static ones and keeps track of where we are.
A robot has to solve similar problems in a very different way. It may use cameras instead of eyes, LiDAR for range, radar, ultrasonic sensors, an IMU, wheel encoders and force sensors in its fingers. No single sensor provides the whole picture. Robotic “vision” emerges when software fuses those signals into a digital model of the surrounding space.
So a robot does not merely look at the world. It continuously measures it.
A camera sees an image, but that is not enough
A normal camera delivers pixels with brightness and color. A human sees a chair and quickly infers its shape, size and approximate distance. A computer initially sees only numbers.
An AI model has to determine which pixels belong to the chair, the floor, a person or a shadow, then estimate motion and free space. That is the job of computer vision. NVIDIA Isaac ROS combines perception, localization and mapping and shows how modern robots connect cameras with AI models on-device.
A standard camera also has a fundamental limitation: the image is flat. It does not contain a ready-made value saying that a chair is 1.8 meters away. Depth must be inferred or measured.
Two “eyes” create depth
Stereo cameras work in a way that resembles human binocular vision. Two cameras see the scene from slightly different positions, and the computer compares corresponding features to estimate distance.
The result is a depth map: pixels can carry not only color but also an approximate distance to the visible surface. That lets a robot distinguish a wall three meters away from a box one meter away and plan movement accordingly.
LiDAR measures space with light
LiDAR sends light pulses and measures how long they take to return from surfaces. Thousands of such measurements form a point cloud — a three-dimensional description of nearby geometry.
A robot can use that cloud to identify floors, walls, vehicles, obstacles and edges. Isaac Sim can simulate cameras, 2D and 3D LiDAR, radar and other sensors before a physical robot is even built.
Why robots also use radar
Radar uses radio waves and is particularly useful for measuring range and motion. It provides an independent sensing channel when cameras struggle with lighting, fog or other visual conditions.
A household robot may use a much simpler sensor stack. A robot vacuum does not need to identify the brand of a chair; it only needs to know that the area ahead is blocked.
A robot can sense what it cannot see
An IMU — inertial measurement unit — usually contains accelerometers and gyroscopes. It tells the robot about acceleration, rotation and tilt. The machine can detect that it is turning, has been pushed or is beginning to lose balance.
Encoders in motors and joints report how far a wheel or limb has moved. The robot therefore maintains information about both the outside world and its own body.
Touch is also part of perception
A robotic hand cannot rely on vision alone. Tactile and force sensors can detect whether an object is already gripped, is starting to slip or is being squeezed too hard.
Without that feedback, a robot might visually recognize an egg perfectly and still crush it.
The key is sensor fusion
Cameras make mistakes. LiDAR has limitations. IMUs drift over time and wheels can slip. Robotics therefore combines sensors rather than trusting one source.
A camera may identify a person, a depth camera estimates distance, LiDAR confirms the obstacle, radar measures motion, the IMU reports the robot’s own rotation and encoders track the wheels. Software combines those signals into the most likely interpretation of the scene.
How does a robot know where it is?
One of the core technologies is SLAM — Simultaneous Localization and Mapping. The robot builds a map while estimating its own position on that map.
As it moves, it detects landmarks, estimates motion and gradually constructs a representation of the environment. That is why a robot vacuum can display a recognizable floor plan, while a more advanced mobile robot can maintain a 3D map of doors, stairs, corridors and temporary obstacles.
Spot shows how machine perception differs from ours
Boston Dynamics’ Spot uses multiple stereo depth cameras around its body. According to the company’s specifications, those cameras help build a model of nearby space and avoid obstacles.
Yet even such a system has blind spots. Thin cables, transparent glass and certain surfaces can be difficult for geometry-only perception. Boston Dynamics has explored visual foundation models partly because geometric sensing alone does not always provide enough semantic understanding.
Why a robot can miss an obvious obstacle
Darkness, strong backlighting, mirrors, glass, reflective surfaces, fog, rain and very thin objects can all create problems. Camera layouts also leave small blind zones. Boston Dynamics explicitly documents limits of Spot’s obstacle avoidance.
Safe autonomy therefore depends on uncertainty handling rather than the assumption that “AI sees everything.” If the system is not confident that a path is safe, it should slow down or stop.
Seeing an object and understanding it are different
Computer vision may label a table, a cup, a person and a door. A general-purpose robot needs more: a cup can be grasped by its handle, a door may open, a person may suddenly step sideways, and a wet floor changes the safe movement strategy.
This is where perception becomes physical AI: what the robot sees must be connected to what it can and should do.
Does a robot see the world the way we do?
No. For a person, a room is made of colors, objects, familiar faces, experience and meaning. For a robot, the same room can simultaneously be an RGB image, a depth map, a 3D point cloud, semantic labels, localization coordinates and a navigability map.
A human sees a chair. A robot may see an object at X/Y/Z coordinates, semantic class “chair,” and a region through which its route planner should not pass.
How future robots will see
The biggest progress will not come only from higher-resolution cameras. Multimodal AI models will combine visual input, language, maps, tactile feedback and action planning into a more unified representation.
The physical world, however, is much less predictable than text on a screen. Slippery floors, transparent doors, falling objects and unexpected human movement make reliable robotics much harder than image description alone.
So how does a robot see the world?
Not in one way. Cameras provide color and shape. Stereo or depth cameras provide distance. LiDAR provides geometry. Radar adds motion and range information. IMUs report the robot’s own movement. Encoders track wheels and joints. Touch sensors measure contact with objects.
AI interprets those signals, while sensor fusion and SLAM combine them into a shared model of reality.
A robot’s true “eyes” are not a single camera but the entire system of sensors, algorithms and AI that continuously turns the physical world into a digital model accurate enough for the next action.
