Cloud4D: estimating cloud properties at a high spatial and temporal resolution

Conference item

There has been great progress in improving numerical weather prediction and climate models using machine learning. However, most global models act at a kilometer-scale, making it challenging to model individual clouds and factors such as extreme precipitation, wind gusts, turbulence, and surface irradiance. Therefore, there is a need to move towards higher-resolution models, which in turn require high-resolution real-world observations that current instruments struggle to obtain. We present Cloud4D, the first learning-based framework that reconstructs a physically consistent, four–dimensional cloud state using only synchronized ground-based cameras. Leveraging a homography-guided 2D-to-3D transformer, Cloud4D infers the full 3D distribution of liquid water content at 25 m spatial and 5 s temporal resolution. By tracking the 3D liquid water content retrievals over time, Cloud4D additionally estimates horizontal wind vectors. Across a two-month deployment comprising six skyward cameras, our system delivers an order-of-magnitude improvement in space-time resolution relative to state-of-theart satellite measurements, while retaining single-digit relative error (< 10%) against collocated radar measurements.

Authors: Lin, J; Gryspeerdt, E; Clark, R

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Neural Information Processing Systems Foundation
• Publication date: 2026-05-01
• Acceptance date: 2025-09-18
• Event title: 39th Annual Conference on Neural Information Processing Systems (NeurIPS 2025)
• Event location: San Diego, California, USA and Mexico City, Mexico
• Event website: https://neurips.cc/Conferences/2025
• Event start date: 2025-11-30
• Event end date: 2025-12-05
• Language: English