PolarVSR: A Unified Framework and Benchmark for Continuous Space-Time Polarization Video Reconstruction

Polarimetric imaging aims to acquire surface polarization characteristics, such as the Degree of Linear Polarization (DoLP) and the Angle of Polarization (AoP). In mainstream Division-of-Focal-Plane (DoFP) color polarization imaging, reconstructing polarization parameters from captured mosaic arrays remains a challenging inverse problem. Existing DoFP cameras are also limited by hardware bottlenecks and often cannot provide high-frame-rate acquisition, which restricts the use of polarimetric imaging in dynamic video tasks. These limitations motivate the joint enhancement of spatial and temporal resolution.To this end, we propose the first space-time polarization video reconstruction architecture. The proposed method performs unified spatiotemporal modeling of polarization directions and uses a polarization-aware implicit neural representation to achieve continuous and high-fidelity upsampling. By analyzing temporal variations in polarization parameters, we further introduce a flow-guided polarization variation loss to supervise polarization dynamics. In addition, we establish the first large-scale color DoFP polarization video benchmark to support this research direction. Extensive experiments on the proposed benchmark demonstrate the effectiveness of the proposed method.

We introduce PV, the first large-scale color polarization video dataset collected using a FLIR BFS-U3-51S5PC-C camera. PV covers diverse indoor and outdoor scenes. For indoor setups, the camera is mounted on a rigid stand, and the objects are placed on a motorized turntable. The turntable is operated at three speed levels, and the light sources are adjusted to create different illumination conditions. The objects cover a wide range of materials, including polarizers, plastics, and frosted surfaces. For outdoor scenarios, some scenes are recorded using a tripod, such as in a zoo, where the activities of different animals are captured and variations in fur textures produce noticeable polarization differences. We also collect driving sequences using an in-vehicle setup under daytime and nighttime road conditions, which are useful for downstream tasks such as object recognition. The camera operates at 75 FPS. In total, the dataset consists of 65 scenes and 117550 frames, with sequence lengths ranging from 200 to 2000 frames.

Computational complexity is evaluated on 200x200 mosaic arrays using an NVIDIA RTX 4090 GPU under demosaicking and 2x frame interpolation.