POLAR: A Portrait OLAT Dataset and Generative Framework for Illumination-Aware Face Modeling

CVPR 2026 (Oral)
Zhuo Chen1,2†, Chengqun Yang1†, Zhuo Su2*, Zheng Lv2, Jingnan Gao1, Xiaoyuan Zhang2, Xiaokang Yang1, Yichao Yan1*
1MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University, 2PICO.
( † denotes equal contribution, * denotes corresponding author. )
Paper arXiv Code (Under internal review) Dataset (Uploading)

Paper Video

POLAR captures high-resolution OLAT facial data with diverse subjects and expressions, from which we synthesize large-scale HDR-relit portraits. POLARNet further learns to generate per-light OLAT responses from a single portrait, enabling scalable and physically consistent relighting under arbitrary HDR environments.

Abstract

Interpolate start reference image.

Face relighting aims to synthesize realistic portraits under novel illumination while preserving identity and geometry. However, progress remains constrained by the limited availability of large-scale, physically consistent illumination data.

To address this, we introduce POLAR, a large-scale and physically calibrated One-Light-at-a-Time (OLAT) dataset containing over 200 subjects captured under 156 lighting directions, multiple views, and diverse expressions. Building upon POLAR, we develop a flow-based generative model POLARNet that predicts per-light OLAT responses from a single portrait, capturing fine-grained and direction-aware illumination effects while preserving facial identity.

Unlike diffusion or background-conditioned methods that rely on statistical or contextual cues, our formulation models illumination as a continuous, physically interpretable transformation between lighting states, enabling scalable and controllable relighting. Together, POLAR and POLARNet form a unified illumination learning framework that links real data, generative synthesis, and physically grounded relighting, establishing a self-sustaining “chicken-and-egg’’ cycle for scalable and reproducible portrait illumination.

The POLAR Dataset

The POLAR dataset provides multiple modalities supporting both relighting and general face modeling tasks:

  • Raw OLAT captures. Each subject is recorded under 156 illuminations across 32 views and 16 expressions at 4K resolution.
  • Lighting annotations. Each OLAT image has calibrated light directions (θ, φ) for illumination reconstruction.
  • Alpha mattes. Per-pixel alpha maps are provided for precise compositing and boundary preservation.
  • Relit portraits. From captured OLATs, we generate both uniformly lit and HDR-relit portraits.
Relit portrait examples showing improved realism and illumination consistency.
We evaluate the effect of diffuse–specular separation and light-cone sampling range, showing that both refinements improve shading realism and highlight consistency.
Age, gender, and skin-tone statistics over 200 volunteers.
We performed statistical analysis on the age, gender, and skin tone over 200 volunteers.

Comparison with Existing OLAT-style Face Datasets

Accessibility indicates whether the dataset is open-source, closed, or partially open-source.

Dataset Subjects Views Expressions Frames Resolution Light Types Accessibility
Total Relighting 70 6 9 10.6M 4K OLAT + HDR-relit Closed
Adobe Data 59 4 5-15 1.2M 1K OLAT + HDR-relit Closed
NetFlix Data 67 36 - 12M 4K OLAT + HDR-relit Closed
ICT-3DRFE 23 2 15 14K 1K Gradient / Polarized Open
Dynamic OLAT <10 4 1 603K 4K OLAT Partial
Goliath-4 4 (public) 100+ - >1M Mixed Relightable captures Partial
BecomingLit 15 16 23 221K 3K OLAT Open
FaceOLAT 139 40 4 5.5M 4K OLAT Open
POLAR (ours) 220 32 16 28.8M 4K OLAT + HDR-relit Open

Method

Interpolate start reference image.

Given a uniformly lit portrait, the encoder–decoder pair \( (\mathbf{E},\mathbf{D}) \) maps both the input and its target OLAT image into latent space. Latent Bridge Matching learns a continuous, direction-conditioned transport between these endpoints, supervised by the velocity field loss \( \mathcal{L}_{\mathrm{LBM}} \). A conditional U-Net predicts the latent drift \( v{\theta}(z_t, t, c_{\text{dir}}) \) using the encoded light direction as input. During inference, a single forward step transports the latent \( z_u \) toward the illumination-specific latent \( z_l \), enabling efficient generation of per-light OLAT responses for all calibrated directions. These synthesized OLATs can be linearly composed to render realistic relighting under arbitrary HDR environments.

Quantitative comparison with state-of-the-art image-based relighting methods.

Methods LPIPS ↓ PSNR ↑ SSIM ↑
SwitchLight 0.168 20.69 0.84
IC-Light 0.314 18.47 0.702
DreamLight 0.175 19.87 0.79
POLARNet (ours) 0.115 22.12 0.82

Delighting and Generalization

We analyze challenging illumination cases and show that POLARNet remains robust beyond constrained capture settings.

Delighting for Non-uniform Illumination

POLARNet assumes a uniformly lit input portrait. If one side of the face is much brighter, this lighting bias can leak into predicted OLAT responses and reduce relighting quality.

We introduce a delighting step that first restores an approximately neutral-light portrait before OLAT prediction, leading to more direction-consistent OLATs and cleaner relit results under strong shadows.

Delighting comparison under strong directional illumination.

Generalization to Challenging Attributes

During data capture, subjects are asked to remove glasses, heavy make-up, and accessories to reduce uncontrolled specular reflections.

Even with these constraints, the model generalizes well to challenging real-world cases, preserving fidelity while maintaining physically consistent relighting.

Generalization examples with accessories and appearance variations.

OLAT and Relighting Results

From left to right: ground-truth OLAT, generated OLAT, and relit portraits under a rotating HDR environment composed from generated OLAT responses.

Ground-truth OLAT

Generated OLAT

Relighting under Rotating HDR

In-the-Wild Results

In-the-wild results rendered under different HDR environment maps.

Citation

@article{chen2025polar,
  title={POLAR: A Portrait OLAT Dataset and Generative Framework for Illumination-Aware Face Modeling},
  author={Chen, Zhuo and Yang, Chengqun and Su, Zhuo and Lv, Zheng and Gao, Jingnan and Zhang, Xiaoyuan and Yang, Xiaokang and Yan, Yichao},
  journal={arXiv preprint arXiv:2512.13192},
  year={2025}
}