Time-Aware Auto White Balance in Mobile Photography

Mahmoud Afifi*, Luxi Zhao*, Abhijith Punnappurath, Mohamed A. Abdelsalam, Ran Zhang, and Michael S. Brown

arXiv, 2025

AI Center-Toronto, Samsung Electronics

Timestamp and geolocation, combined with capture metadata, provide strong cues for estimating scene illuminants in smartphone camera white balancing. Our method leverages this data, along with color information, using a lightweight learnable model (~5K parameters) that runs efficiently on a flagship mobile DSP (0.25 ms) and CPU (0.80 ms), achieving high accuracy. We also introduce a large dataset (~3.2K raw images) from the S24 Ultra, containing ground-truth illuminants (neutral and user-preference-based) and capture metadata. 

* Equal contribution 

arXiv | Code & Data  

CCMNet: Leveraging Calibrated Color Correction Matrices for Cross-Camera Color Constancy

Dongyoung Kim, Mahmoud Afifi, Dongyun Kim, Michael S. Brown, and Seon Joo Kim

arXiv, 2025

AI Center-Toronto, Samsung Electronics in collaboration with Yonsei University

By leveraging pre-calibrated color correction matrices (CCMs) existing in camera ISPs, we generate a compact camera fingerprint embedding to adapt our method to new cameras. Our method achieves state-of-the-art performance in color constancy across diverse cameras, while remaining lightweight. 

arXiv | Project Page | Code 

Multispectral Demosaicing via Dual Cameras

SaiKiran Tedla*, Junyong Lee*, Beixuan Yang, Mahmoud Afifi, and Michael S. Brown

arXiv, 2025

AI Center-Toronto, Samsung Electronics in collaboration with York University

We present a method for multispectral (MS) image demosaicing designed for dual-camera setups, leveraging co-captured high-fidelity RGB images to guide the reconstruction of lower-fidelity MS images. Alongside, we provide a large dataset of paired RGB and MS mosaiced images with ground-truth demosaiced outputs.

* Equal contribution 

arXiv | Code & Data 

Color Matching Using Hypernetwork-Based Kolmogorov-Arnold Networks

Artem Nikonorov*, Georgy Perevozchikov*, Andrei Korepanov, Nancy Mehta, Mahmoud Afifi, Egor Ershov, and Radu Timofte

arXiv, 2025

In collaboration with the University of Würzburg, Samara University, and IITP RAS

We introduce cmKAN, a powerful color-matching framework that employs a hypernetwork to generate spatially adaptive weights for controlling KAN’s nonlinear splines. cmKAN achieves state-of-the-art color matching between diverse source and target distributions in supervised and unsupervised settings. 

* Equal contribution 

arXiv | Code & Data  

What Do You See? Enhancing Zero-Shot Image Classification with Multimodal Large Language Models

Abdelrahman Abdelhamed*, Mahmoud Afifi*, and Alec Go

arXiv, 2024

Google

With some prompt engineering, multimodal LLMs (e.g., Gemini) can perform zero-shot image classification. However, they may not consistently produce accurate target dataset labels. Our approach leverages multimodal LLMs & cross-modal embedding encoders to produce initial class prediction feature & image description feature alongside image feature, improving zero-shot image classification accuracy without the need for dataset-specific prompts. Our method outperforms prior methods across various datasets, achieving a 6.8% increase in accuracy on ImageNet.

* Equal contribution 

arXiv | Code & Data

Optimizing Illuminant Estimation in Dual-Exposure HDR Imaging

Mahmoud Afifi, Zhenhua Hu, and Liang Liang

ECCV 2024

Google

Utilizing the chromatic distortion present between long and short exposure frames of HDR photography, we introduce a compact guiding feature for illuminant estimators. Processed by just ~300 learnable parameters, our method achieves results that match or surpass previous methods relying on thousands or even millions of parameters.

Paper | Supp. Materials | arXiv | Poster

Rawformer: Unpaired Raw-to-Raw Translation for Learnable Camera ISPs

Georgy Perevozchikov, Nancy Mehta, Mahmoud Afifi, and Radu Timofte

ECCV 2024

In collaboration with the University of Würzburg

Rawformer, an unsupervised Transformer-based encoder-decoder model for raw-to-raw mapping, enables the utilization of learnable camera ISP trained on a specific camera's raw images to render raw images taken by new cameras with different characteristics.

Paper | Supp. Materials | arXiv | Code

Auto White-Balance Correction for Mixed-Illuminant Scenes

Mahmoud Afifi, Marcus A. Brubaker, and Michael S. Brown

WACV 2022

York University 

Mixed/single-illuminant scene white balancing does not necessarily require illuminant estimation. Instead, the problem could be bounded by a small set of predefined white-balance settings. Given that, we locally blend a set of small images rendered with different white-balance settings to generate the final corrected image.

Paper | Supp. Materials | arXiv | Code & Data | Poster | Presentation | Talk | Patent Application

Improving Single-Image Defocus Deblurring: How Dual-Pixel Images Help Through Multi-Task Learning

Abdullah Abuolaim, Mahmoud Afifi, and Michael S. Brown

WACV 2022

York University 

Jointly learning to predict the two DP views from a single blurry input image improves the network’s ability to learn to deblur the image. Generating high-quality DP views can be used for other DP-based applications, such as reflection removal.

PDF | Supplementary Materials | arXiv | Code & Dataset | Patent Application

Cross-Camera Convolutional Color Constancy

Mahmoud Afifi, Jonathan T. Barron, Chloe LeGendre, Yun-Ta Tsai, and Francois Bleibel  

ICCV 2021 (Oral Presentation)

Google Research

A self-calibration method for cross-camera color constancy through the lens of transductive inference: additional (unlabeled) images are provided as input to the model at test time, which allows the model to calibrate itself to the spectral properties of the test-set camera during inference.

Paper | Supp. Materials | arXiv | Code | Poster | Presentation | Presentation (PDF) | Talk | Google @ ICCV'21

HistoGAN: Controlling Colors of GAN-Generated and Real Images via Color Histograms

Mahmoud Afifi, Marcus A. Brubaker, and Michael S. Brown

CVPR 2021 

York University 

HistoGAN is the first work to control colors of GAN-generated images based on features derived directly from color histograms. Our method learns to transfer the color information encapsulated in histogram features to the colors of a GAN-generated images (HistoGAN) or real input images (ReHistoGAN). As color histograms provide an abstract representation of image color that is decoupled from spatial information, our HistoGAN and ReHistoGAN are less restrictive and suitable across arbitrary domains.

Paper | Supp. Materials | arXiv | Code & Dataset | Colab (histogram loss) | Poster | Presentation | Talk

Learning Multi-Scale Photo Exposure Correction

Mahmoud Afifi, Konstantinos G. Derpanis, Björn Ommer,  and Michael S. Brown

CVPR 2021 

SAIC, Samsung Research America in collaboration with Heidelberg University

A single coarse-to-fine deep learning model with adversarial training to correct both over- and under-exposed photographs.

Paper | Supp. Materials | arXiv | Code & Dataset | Poster | Presentation | Talk | Samsung Research Post | Patent

CIE XYZ Net: Unprocessing Images for Low-Level Computer Vision Tasks

Mahmoud Afifi, Abdelrahman Abdelhamed, Abdullah Abuolaim, Abhijith Punnappurath, and Michael S. Brown

TPAMI 2021

York University 

Learning accurate camera-rendering linearization gives a significant improvement for different computer vision tasks (e.g., denoising, deblurring, and image enhancement).

Paper | arXiv | Supp. Materials | External Link | Code & Dataset 

Semi-Supervised Raw-to-Raw Mapping

Mahmoud Afifi and Abdullah Abuolaim

BMVC 2021

York University 

A semi-supervised method to map between two different camera-raw spaces. Training requires an unpaired set of images besides a very small set of paired images taken by these two camera models. 

arXiv | Dataset | Presentation 

Deep White-Balance Editing

Mahmoud Afifi and Michael S. Brown

CVPR 2020 (Oral Presentation)

SAIC, Samsung Research America

A multi-task deep learning model for post-capture white-balance correction and editing

Paper | Supp. Materials | arXiv | Code | Presentation | Talk | CVPR Daily Magazine | Samsung Research Post | Samsung Newsroom | Patent

Modeling Defocus-Disparity in Dual-Pixel Sensors

Abhijith Punnappurath, Abdullah Abuolaim*, Mahmoud Afifi*, and Michael S. Brown

ICCP 2020

York University 

A symmetry property of dual-pixel kernels for unsupervised depth estimation

* Equal contribution 

Paper | Code & Dataset | Talk

System and Method for Color Matching

Atima Lui, Nyalia Lui, Mahmoud Afifi, and Ariadne Bazigos

US Patent 2020

My Nudest Inc

A system for analyzing user input, combining user's image(s) and query responses to provide tailored color outputs. Through color correction and comparison to predetermined color identifiers, it delivers accurate results and product recommendations.

Patent | PDF

Interactive White Balancing for Camera-Rendered Images

Mahmoud Afifi and Michael S. Brown

CIC 2020 (Oral Presentation)

York University 

A simple method to link the nonlinear white-balance correction functions, introduced in our CVPR'19 work, to the user's selected colors to allow interactive white-balance manipulation

arXiv | Code | Presentation

Improving Color Space Conversion for Camera-Captured Images via Wide-Gamut Metadata

Hoang Le, Mahmoud Afifi, and Michael S. Brown

CIC 2020 (Oral Presentation)

York University 

Having additional wide-gamut metadata, available during color space conversion, greatly assists in constructing a locally weighted color mapping function to convert between color gamuts. 

Paper | Code & Dataset

What Else Can Fool Deep Learning? Addressing Color Constancy Errors on Deep Neural Network Performance

Mahmoud Afifi and Michael S. Brown 

ICCV 2019 

York University 

Deep learning models can be fooled by white-balance errors and their accuracy can be improved by augmented images with different white-balance settings.

Paper | Supp. Materials | arXiv | Project Page | Code | Colab

Color Temperature Tuning: Allowing Accurate Post-Capture White-Balance Editing 

Mahmoud Afifi, Abhijith Punnappurath, Abdelrahman Abdelhamed, Hakki Can Karaimer, Abdullah Abuolaim, and Michael S. Brown 

CIC 2019 (Oral Presentation)

Best paper award

York University 

With a small modification to existing camera ISPs, we can achieve accurate post-capture white balance editing by embedding a set of mapping coefficients in the JPEG metadata.

Paper | Project Page | Code | Presentation | Patent Application

Image Recoloring Based on Object Color Distributions

Mahmoud Afifi, Brian Price, Scott Cohen, and Michael S. Brown.

Eurographics 2019 (Short Papers)

York University in collaboration with Adobe Research

A fully automated image recoloring without the need for target/reference images

Paper | Project Page | External Link | Code | Supp. Materials | Presentation | Fast-Forward Video | Adobe Research Post

When Color Constancy Goes Wrong: Correcting Improperly White-Balanced Images

Mahmoud Afifi, Brian Price, Scott Cohen, and Michael S. Brown 

CVPR 2019

York University in collaboration with Adobe Research

The first work to directly address the problem of incorrectly white-balanced images; requires a small memory overhead and it is fast.

Paper | Project Page | Supp. Materials | Demo | Video | Code | Dataset | Adobe Research Post | Patent

Sensor-Independent Illumination Estimation for DNN Models 

Mahmoud Afifi and Michael S. Brown 

BMVC 2019 (Oral Presentation)

York University 

Learning a new canonical space in an unsupervised manner allows us to train a single deep model on multiple camera sensors and perform accurate illuminant estimation for images captured by new unseen camera sensors in the inference phase.

Paper | Project Page | Supp. Materials | arxiv | Code | Presentation | Talk | Patent

Tensor Methods for Group Pattern Discovery of Pedestrian Trajectories

Abdullah Sawas*, Abdullah Abuolaim*, Mahmoud Afifi, and Manos Papagelis

MDM 2018

Best paper award

York University 

Efficient discovery of evolving groups of pedestrians; a new group pattern is introduced in the journal version.

* Equal contribution

Project Page | Poster | Journal Version

AFIF4: Deep Gender Classification Based on AdaBoost-Based Fusion of Isolated Facial Features and Foggy Faces

Mahmoud Afifi and Abdelrahman Abdelhamed 

JVCI 2019

Assiut University 

Gender classification can be improved using different facial features; a user study validates our finding.

arxiv | Dataset

11K Hands: Gender Recognition and Biometric Identification Using a Large Dataset of Hand Images

Mahmoud Afifi

MTA 2019

Assiut University 

Hand images can be used for gender recognition and biometric identification; a large dataset of hand images enables us to train our two-stream deep model.

arxiv | Project Page & Dataset | Code

MPB: A Modified Poisson Blending Technique

Mahmoud Afifi and Khaled F. Hussain

CVM 2016

Assiut University 

Bleeding artifacts caused by Poisson image editing can be reduced by a simple two-stage blending approach.

Paper | Project Page | Code | Video

Conference version: Paper | Project Page | Code | Video