Arizona researchers turn surroundings into virtual 3D sensing screen

University of Arizona researchers have developed a 3D imaging approach that turns surrounding matte surfaces into a virtual screen, allowing machines to measure reflective and diffuse objects in the same scene with greater accuracy.

The work comes from the university’s Computational 3D Imaging and Measurement Lab at the Wyant College of Optical Sciences and has been published in Nature Communications. The system addresses a long-running problem in 3D sensing: most imagers perform well on either diffuse matte surfaces or specular reflective surfaces, while real-world environments usually contain both.

The research team uses a laser scanner to capture the full scene, including matte, glossy, and reflective objects. Algorithms then separate diffuse and specular surfaces. Once the diffuse portions of the scene are identified, they can be used as a virtual screen for deflectometry, a measurement technique that reconstructs reflective surfaces by analysing how projected patterns are distorted in reflection.

That approach removes the need for large physical screens normally required in deflectometry setups. In applications such as inspecting painted car bodies, conventional systems may require tunnel-like screen assemblies to cover wide angular ranges. By using the room or surrounding objects as part of the measurement system, the Arizona method gives 3D sensing a more flexible route into complex environments.

The hardware also includes a neuromorphic event camera, which captures changes in the scene at very high temporal resolution instead of recording full frames at fixed intervals. That allows the system to measure moving mixed-reflectance scenes at high frame rates and across large brightness differences, from dim surfaces to intense reflections.

Autonomous vehicles, robotic surgery systems, industrial inspection cells, and biomedical imaging platforms all face the same basic sensing challenge: real environments do not present clean optical targets. Surgical scenes may contain moist tissue, fluids, instruments, and skin. Factory inspection lines may include matte packaging, polished metal, glass, plastics, and coated surfaces. Vehicles must interpret dark underpasses, bright glare, rain, glass, road signs, cyclists, pedestrians, and small obstacles in the same perception stack.

That wider direction is already visible in industrial perception hardware. Ouster’s native-colour Rev8 lidar family and Aeva’s use of embedded DSP IP for 4D lidar processing both show how sensing is moving from raw data capture into richer sensor-side interpretation.

The Arizona prototype has been demonstrated at tabletop scale, but the underlying method is intended to be scalable across applications ranging from small biomedical structures to larger rooms and buildings. For electronics design, the most important point is the convergence of optics, event-based sensing, computation, and embedded processing. Better 3D vision is no longer only a question of adding a higher-resolution camera. It increasingly depends on combining the right sensor physics with algorithms that understand how real surfaces behave.