Abstract

In vegetated environments, such as forests, exploratory robots play a vital role in navigating complex, cluttered environments where human access is limited and traditional equipment struggles. Visual occlusion from obstacles, such as foliage, can severely obstruct a robot’s sensors, impairing scene understanding. We show that “peering,” a characteristic side-to-side movement used by insects to overcome their visual limitations, can also allow robots to markedly improve visual reasoning under partial occlusion. This is accomplished by applying core signal processing principles, specifically optical synthetic aperture sensing, together with the vision reasoning capabilities of modern large multimodal models. Peering enables real-time, high-resolution, and wavelength-independent perception, which is crucial for vision-based scene understanding across a wide range of applications. The approach is low cost and immediately deployable on any camera-equipped robot. We investigated different peering motions and occlusion masking strategies, demonstrating that, unlike peering, state-of-the-art multiview three-dimensional vision techniques fail under these conditions because of their high susceptibility to occlusion. Our experiments were carried out on an industrial-grade quadrupedal robot. However, the ability to peer is not limited to such platforms but potentially also applicable to bipedal, hexapod, wheeled, or crawling platforms. Robots that can effectively see through partial occlusion will gain superior perception abilities, including enhanced scene understanding, situational awareness, camouflage breaking, and advanced navigation in complex environments.

DISCUSSION

Exploratory field robots have significantly expanded our ability to investigate environments that are too dangerous, remote, or inaccessible for humans. These robots enhance safety, reduce costs, and enable continuous data collection in challenging conditions. They are widely used in environmental monitoring, agriculture, mining, disaster response, space exploration, underwater research, and infrastructure inspection. By performing tasks like mapping terrain, assessing hazards, and gathering scientific data, field robots play a crucial role in advancing research, improving operational efficiency, and supporting safer decision-making across many applications. In vegetated environments, such as forests, exploratory robots play a vital role in navigating complex, cluttered environments where human access is limited and traditional equipment struggles. Equipped with advanced sensing, mapping, and mobility capabilities, these robots can move through undergrowth, around obstacles, and beneath forest canopies to gather information for biodiversity monitoring, wildfire risk assessment, conservation planning, tracking wildlife, surveillance, and studying forest health indicators such as soil conditions, canopy density, and the spread of plant diseases or parasites. Visual occlusion from obstacles like vegetation or debris at disaster sites can severely block a robot's sensors, causing conventional systems like cameras and LiDAR to yield incomplete data.

This compromises accuracy and hampers scene understanding and navigation. We show that severe occlusion creates such significant data gaps that even advanced machine learning techniques like vision reasoning fail. However, we demonstrate that "peering," a behavior insects use to overcome vision limitations, can also enable robots to significantly enhance their vision reasoning under partial occlusion.

The statistical analysis in (64) makes three key findings of synthetic aperture sensing for occlusion removal: First, its efficacy is highly dependent on the occlusion level. It performs optimally at approximately 50% occlusion. For occlusions significantly above this threshold, effective removal becomes more difficult. Conversely, for cases far below it, the application of synthetic aperture sensing is often unnecessary. Second, the size of the SA does not need to be very large (64). A minimal baseline is one that yields a disparity equal to the projected occluder size. Third, the visibility gain depends on the number of integrated samples. This number does not need to be extremely high, but must not be undersampled (64). In scenarios where occlusion masking successfully removes occluder pixels, a higher sample count increases the statistical probability of observing the background through gaps in the occluding foreground. Consequently, more samples are beneficial in such cases. However, with an adaptive SA approach (86,87) that takes more samples in sparse regions and fewer in dense ones, this number can be significantly reduced.

Our experiments were carried out on a quadrupedal robot. However, the ability to peer is not limited to such platforms, but is also applicable to bipedal, hexapod, wheeled, or crawling platforms. The peering motion can be achieved either by moving a camera payload, by moving the robot's entire frame (e.g., during navigation), or both. While sensor fusion can further enhance performance, multi- spectral and omnidirectional vision offer solutions for a wide range of applications. Adaptive sampling in response to local occlusion conditions would lead to enhanced results. All of this must be investigated in the future. For practical systems, all computations must be performed onboard within the robot's processing framework, rather than being separated into online recording and offline pre-processing as we did for experimental simplicity. Vision reasoning can be implemented either using smaller, locally deployed LMMs (such as Qwen) running directly on the robot, or via remote API calls from the robot to large-scale models, such as the latest versions of ChatGPT, Gemini, Claude, Kosmos, or LLaMA. Currently, computing SA integral images takes milliseconds on our hardware (e.g., 17ms to process 300 images at 1080^2px resolution). However, performing reasoning on these images using ChatGPT takes considerably longer, typically 10-20s per SA integral, depending on the server load. Local model deployment would significantly reduce this latency.

An alternative approach might be to apply vision reasoning directly to the raw input images, bypassing the integration step. This may seem practical when dealing with perspectives containing significant occlusion gaps. As illustrated in Fig. S2 of the Supplementary Materials, this approach can only detect background objects visible through sufficiently large gaps. While using multiple images may increase the number of identifiable objects, those that are heavily occluded in all views remain undetectable. Synthetic aperture integral imaging overcomes this limitation. Moreover, it enables the rapid computation of a focal stack (a series of images at varying synthetic focal distances). Applying vision reasoning to this focal stack allows object identification without prior knowledge of their distances or manual focal adjustment, as also demonstrated in Fig. S2 of the Supplementary Materials. Although processing image sequences increases vision reasoning times, small focal stacks (e.g., 10 layers) can be analyzed at moderate speeds of a few seconds (15-25s in our case, depending on server load).

Replacing peering with a static array of camera sensors (65) would severely undersample the SA signal. It would also result in large, complex payloads, require extremely high bandwidth to transmit multiple parallel video streams, and prevent adaptive sampling in response to local occlusion conditions. Active sensors like LiDAR and radar can also overcome occlusion but suffer from drawbacks such as low spatial resolution, a lack of multispectral data (e.g., color, temperature), and, for adequate resolutions, longer processing times for tasks like pulse compression and point-cloud registration. Peering enables real-time and high-resolution perception (which is essential for vision- based scene understanding) that is low-cost and immediately deployable on any camera-equipped robot.

The potential impact of this technology is significant. Robots that can effectively "see through" partial occlusion would possess superior perception capabilities including: scene understanding, situational awareness, and navigational capabilities in complex environments. This opens a wide range of applications. For instance, camouflage breaking is not only a subject of computer vision and image processing (88) but also a motion-parallax-based hunting strategy used by predators (89,90). This effect can also be achieved through peering, as demonstrated in simulation (see Fig. S3 in the Supplementary Materials).

We can learn peering strategies for robotics by studying evolved animal behaviour. This, however, creates a bidirectional exchange where robotics research and animal behaviour studies can stimulate and build upon one another. Whether peering serves functions beyond distance estimation and camouflage breaking in unoccluded environments – such as detecting camouflaged prey through occluding foliage – remains unexplored today.

(pdf here but I get not wanting to click it: https://arxiv.org/pdf/2511.16262v3)