Peyman Milanfar

I lead the Computational Imaging/ Image Processing team in Google Research. My team develops core imaging technologies that are used in a number of products at Google.

One of these technologies is RAISR (Rapid and Accurate Image Super-Resolution): Given an image, we wish to produce an image of larger size with significantly more pixels and higher image quality. With pairs of example images, we train a set of filters (i.e., a mapping) that when applied to a given image that is not in the training set, will produce a higher resolution version of it. The work was highlighted in a Research Blog post. The technology was launched for G+ photos G+ Photos worldwide; and also as part of the MotionStills app .

Another is Turbo Denoising for camera pipelines and other imaging applications. We produced a single-frame denoiser that is (1) fast enough to be practical even for mobile devices, and (2) handles content dependent noise that is typical for real camera captures. For realistic camera noise, our results are competitive with BM3D, but with nearly 400 times speedup. This technique allowed us to speed up denoising algorithm by two orders of magnitude, while producing quality that is state of the art. As a side benefit, less noisy images compress better and lead to smaller file sizes.

Another is Style Transfer which is a process of migrating a style from a given image to the content of another, synthesizing a new image which is an artistic mixture of the two. Our algorithm extends earlier work on texture-synthesis, while aiming to get stylized images that get closer in quality to ones produced by Convolutional Neural Networks. The proposed algorithm is fast and flexible, being able to process any pair of content + style images .

My team also works on more theoretical questions. For instance, in RED (Regularization by Denoising) we proposed a new way to use the denoising engine in defining the regularization for any inverse problem. RED is an explicit image-adaptive Laplacian-based regularization functional, making the overall objective functional clear and well-defined. With a complete flexibility to choose the iterative optimization procedure for minimizing the above functional, RED is capable of incorporating any image denoising algorithm, treat general inverse problems very effectively, and is guaranteed to converge to the globally optimal result. As examples of its utility, we test this approach and demonstrate state-of-the-art results in the image deblurring and super-resolution problems.

A bit about my background: Prior to joining Google, I was a Professor of Electrical Engineering at UC Santa Cruz from 1999-2014. I was also Associate Dean for Research at the School of Engineering from 2010-12. From 2012-2014 I was on leave at Google-x, where I helped develop the imaging pipeline for Google Glass. I received my undergraduate education in electrical engineering and mathematics from the University of California, Berkeley, and the MS and PhD degrees in electrical engineering from MIT. I hold 11 US patents, several of which are commercially licensed. He founded MotionDSP in 2005. I've been keynote speaker at numerous technical conferences including Picture Coding Symposium (PCS), SIAM Imaging Sciences, SPIE, and the International Conference on Multimedia (ICME). Along with my former students, I won several best paper awards from the IEEE Signal Processing Society.

I am a Distinguished Lecturer of the IEEE Signal Processing Society, and a Fellow of the IEEE "for contributions to inverse problems and super-resolution in imaging."

Please visit my public website, for the most up to date list of my publications, cv, etc.

Google Publications

Previous Publications

  •  

    Global Image Denoising

    Hossein Talebi Esfandarani, Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 23 (2014), pp. 755-768

  •  

    A Tour of Modern Image Filtering: New Insights and Methods, Both Practical and Theoretical

    Peyman Milanfar

    IEEE Signal Process. Mag., vol. 30 (2013), pp. 106-128

  •  

    Blind Deconvolution Using Alternating Maximum a Posteriori Estimation with Heavy-Tailed Priors

    Jan Kotera, Filip Sroubek, Peyman Milanfar

    CAIP (2) (2013), pp. 59-66

  •  

    Estimating Spatially Varying Defocus Blur From A Single Image

    Xiang Zhu, Scott Cohen, Stephen Schiller, Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 22 (2013), pp. 4879-4891

  •  

    How to SAIF-ly Boost Denoising Performance

    Hossein Talebi Esfandarani, Xiang Zhu, Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 22 (2013), pp. 1470-1485

  •  

    Removing Atmospheric Turbulence via Space-Invariant Deconvolution

    Xiang Zhu, Peyman Milanfar

    IEEE Trans. Pattern Anal. Mach. Intell., vol. 35 (2013), pp. 157-170

  •  

    Symmetrizing Smoothing Filters

    Peyman Milanfar

    SIAM J. Imaging Sciences, vol. 6 (2013), pp. 263-284

  •  

    Deconvolving PSFs for a Better Motion Deblurring Using Multiple Images

    Xiang Zhu, Filip Sroubek, Peyman Milanfar

    ECCV (5) (2012), pp. 636-647

  •  

    Improving denoising filters by optimal diffusion

    Hossein Talebi Esfandarani, Peyman Milanfar

    ICIP (2012), pp. 1181-1184

  •  

    Patch-Based Near-Optimal Image Denoising

    Priyam Chatterjee, Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 21 (2012), pp. 1635-1649

  •  

    Robust Multichannel Blind Deconvolution via Fast Alternating Minimization

    Filip Sroubek, Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 21 (2012), pp. 1687-1700

  •  

    Robust flash denoising/deblurring by iterative guided filtering

    Hae Jong Seo, Peyman Milanfar

    EURASIP J. Adv. Sig. Proc., vol. 2012 (2012), pp. 3

  •  

    Action Recognition from One Example

    Hae Jong Seo, Peyman Milanfar

    IEEE Trans. Pattern Anal. Mach. Intell., vol. 33 (2011), pp. 867-882

  •  

    Face Verification Using the LARK Representation

    Hae Jong Seo, Peyman Milanfar

    IEEE Transactions on Information Forensics and Security, vol. 6 (2011), pp. 1275-1286

  •  

    Iteratively merging information from a pair of flash/no-flash images using nonlinear diffusion

    Hae Jong Seo, Peyman Milanfar

    ICCV Workshops (2011), pp. 1324-1331

  •  

    Patch-based locally optimal denoising

    Priyam Chatterjee, Peyman Milanfar

    ICIP (2011), pp. 2553-2556

  •  

    Practical Bounds on Image Denoising: From Estimation to Information

    Priyam Chatterjee, Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 20 (2011), pp. 1221-1233

  •  

    Removing Motion Blur With Space-Time Processing

    Hiroyuki Takeda, Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 20 (2011), pp. 2990-3000

  •  

    Restoration for weakly blurred and strongly noisy images

    Xiang Zhu, Peyman Milanfar

    WACV (2011), pp. 103-109

  •  

    Superfast superresolution

    Filip Sroubek, Jan Kamenick√Ĺ, Peyman Milanfar

    ICIP (2011), pp. 1153-1156

  •  

    A no-reference image content metric and its application to denoising

    Xiang Zhu, Peyman Milanfar

    ICIP (2010), pp. 1145-1148

  •  

    Automatic Parameter Selection for Denoising Algorithms Using a No-Reference Measure of Image Content

    Xiang Zhu, Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 19 (2010), pp. 3116-3132

  •  

    Fundamental limits of image denoising: Are we there yet?

    Priyam Chatterjee, Peyman Milanfar

    ICASSP (2010), pp. 1358-1361

  •  

    Is Denoising Dead?

    Priyam Chatterjee, Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 19 (2010), pp. 895-911

  •  

    Learning denoising bounds for noisy images

    Priyam Chatterjee, Peyman Milanfar

    ICIP (2010), pp. 1157-1160

  •  

    Nonlinear kernel backprojection for computed tomography

    Hiroyuki Takeda, Peyman Milanfar

    ICASSP (2010), pp. 618-621

  •  

    Training-Free, Generic Object Detection Using Locally Adaptive Regression Kernels

    Hae Jong Seo, Peyman Milanfar

    IEEE Trans. Pattern Anal. Mach. Intell., vol. 32 (2010), pp. 1688-1704

  •  

    Visual saliency for automatic target detection, boundary detection, and image quality assessment

    Hae Jong Seo, Peyman Milanfar

    ICASSP (2010), pp. 5578-5581

  •  

    A Non-Parametric Approach to Automatic Change Detection in MRI Images of the Brain

    Hae Jong Seo, Peyman Milanfar

    ISBI (2009), pp. 245-248

  •  

    An Adaptive Nonparametric Approach to Restoration and Interpolation for Medical Imaging

    Hiroyuki Takeda, Peyman Milanfar

    ISBI (2009), pp. 666-669

  •  

    Clustering-Based Denoising With Locally Learned Dictionaries

    Priyam Chatterjee, Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 18 (2009), pp. 1438-1451

  •  

    Detection of human actions from a single example

    Hae Jong Seo, Peyman Milanfar

    ICCV (2009), pp. 1965-1970

  •  

    Generalizing the Nonlocal-Means to Super-Resolution Reconstruction

    Matan Protter, Michael Elad, Hiroyuki Takeda, Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 18 (2009), pp. 36-51

  •  

    Image denoising using locally learned dictionaries

    Priyam Chatterjee, Peyman Milanfar

    Computational Imaging (2009), pp. 72460

  •  

    Optimal Registration Of Aliased Images Using Variable Projection With Applications To Super-Resolution

    M. Dirk Robinson, Sina Farsiu, Peyman Milanfar

    Comput. J., vol. 52 (2009), pp. 31-42

  •  

    Super-Resolution Without Explicit Subpixel Motion Estimation

    Hiroyuki Takeda, Peyman Milanfar, Matan Protter, Michael Elad

    IEEE Transactions on Image Processing, vol. 18 (2009), pp. 1958-1975

  •  

    A generalization of non-local means via kernel regression

    Priyam Chatterjee, Peyman Milanfar

    Computational Imaging (2008), pp. 68140

  •  

    Deblurring Using Regularized Locally Adaptive Kernel Regression

    Hiroyuki Takeda, Sina Farsiu, Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 17 (2008), pp. 550-563

  •  

    On Iterative Regularization and Its Application

    Michael R. Charest, Peyman Milanfar

    IEEE Trans. Circuits Syst. Video Techn., vol. 18 (2008), pp. 406-411

  •  

    Spatio-temporal video interpolation and denoising using motion-assisted steering kernel (MASK) regression

    Hiroyuki Takeda, Peter van Beek, Peyman Milanfar

    ICIP (2008), pp. 637-640

  •  

    Using local regression kernels for statistical object detection

    Hae Jong Seo, Peyman Milanfar

    ICIP (2008), pp. 2380-2383

  •  

    Video denoising using higher order optimal space-time adaptation

    Hae Jong Seo, Peyman Milanfar

    ICASSP (2008), pp. 1249-1252

  •  

    Kernel Regression for Image Processing and Reconstruction

    Hiroyuki Takeda, Sina Farsiu, Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 16 (2007), pp. 349-366

  •  

    Mask Design for Optical Microlithography-An Inverse Imaging Problem

    Amyn Poonawala, Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 16 (2007), pp. 774-788

  •  

    Multi-Scale Statistical Detection and Ballistic Imaging Through Turbid Media

    Sina Farsiu, Peyman Milanfar

    ICIP (3) (2007), pp. 537-540

  •  

    Multiframe demosaicing and super-resolution of color images

    Sina Farsiu, Michael Elad, Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 15 (2006), pp. 141-159

  •  

    Robust Kernel Regression for Restoration and Reconstruction of Images from Sparse Noisy Data

    Hiroyuki Takeda, Sina Farsiu, Peyman Milanfar

    ICIP (2006), pp. 1257-1260

  •  

    Shape Estimation from Support and Diameter Functions

    Amyn Poonawala, Peyman Milanfar, Richard J. Gardner

    Journal of Mathematical Imaging and Vision, vol. 24 (2006), pp. 229-244

  •  

    Statistical and Information-Theoretic Analysis of Resolution in Imaging

    Morteza Shahram, Peyman Milanfar

    IEEE Transactions on Information Theory, vol. 52 (2006), pp. 3411-3437

  •  

    Statistical performance analysis of super-resolution

    M. Dirk Robinson, Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 15 (2006), pp. 1413-1428

  •  

    Super-Resolution Imaging: Analysis, Algorithms, and Applications

    Michael K. P. Ng, Tony F. Chan, Moon Gi Kang, Peyman Milanfar

    EURASIP J. Adv. Sig. Proc., vol. 2006 (2006)

  •  

    Video-to-Video Dynamic Super-Resolution for Grayscale and Color Sequences

    Sina Farsiu, Michael Elad, Peyman Milanfar

    EURASIP J. Adv. Sig. Proc., vol. 2006 (2006)

  •  

    Bias minimizing filter design for gradient-based image registration

    M. Dirk Robinson, Peyman Milanfar

    Sig. Proc.: Image Comm., vol. 20 (2005), pp. 554-568

  •  

    Improved spectral analysis of nearby tones using local detectors

    Morteza Shahram, Peyman Milanfar

    ICASSP (4) (2005), pp. 637-640

  •  

    Local detectors for high-resolution spectral analysis: Algorithms and performance

    Morteza Shahram, Peyman Milanfar

    Digital Signal Processing, vol. 15 (2005), pp. 305-316

  •  

    Multidimensional Integral Inversion, with Applications in Shape Reconstruction

    Annie A. M. Cuyt, Gene H. Golub, Peyman Milanfar, Brigitte Verdonk

    SIAM J. Scientific Computing, vol. 27 (2005), pp. 1058-1070

  •  

    On the resolvability of sinusoids with nearby frequencies in the presence of noise

    Morteza Shahram, Peyman Milanfar

    IEEE Transactions on Signal Processing, vol. 53 (2005), pp. 2579-2588

  •  

    The Generalized Eigenvalue Problem for Nonsquare Pencils Using a Minimal Perturbation Approach

    Gregory Boutry, Michael Elad, Gene H. Golub, Peyman Milanfar

    SIAM J. Matrix Analysis Applications, vol. 27 (2005), pp. 582-601

  •  

    Variable projection for near-optimal filtering in low bit-rate block coders

    Yaakov Tsaig, Michael Elad, Peyman Milanfar, Gene H. Golub

    IEEE Trans. Circuits Syst. Video Techn., vol. 15 (2005), pp. 154-160

  •  

    Advances and challenges in super-resolution

    Sina Farsiu, M. Dirk Robinson, Michael Elad, Peyman Milanfar

    Int. J. Imaging Systems and Technology, vol. 14 (2004), pp. 47-57

  •  

    Fast and robust multiframe super resolution

    Sina Farsiu, M. Dirk Robinson, Michael Elad, Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 13 (2004), pp. 1327-1344

  •  

    Fundamental performance limits in image registration

    M. Dirk Robinson, Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 13 (2004), pp. 1185-1199

  •  

    Imaging below the diffraction limit: a statistical analysis

    Morteza Shahram, Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 13 (2004), pp. 677-689

  •  

    Shape from moments - an estimation theory perspective

    Michael Elad, Peyman Milanfar, Gene H. Golub

    IEEE Transactions on Signal Processing, vol. 52 (2004), pp. 1814-1829

  •  

    Trained detection of buried mines in SAR images via the deflection-optimal criterion

    Russell B. Cosgrove, Peyman Milanfar, Joel Kositsky

    IEEE T. Geoscience and Remote Sensing, vol. 42 (2004), pp. 2569-2575

  •  

    Fast Local and Global Projection-Based Methods for Affine Motion Estimation

    M. Dirk Robinson, Peyman Milanfar

    Journal of Mathematical Imaging and Vision, vol. 18 (2003), pp. 35-54

  •  

    Fast and robust super-resolution

    Sina Farsiu, M. Dirk Robinson, Michael Elad, Peyman Milanfar

    ICIP (2) (2003), pp. 291-294

  •  

    Fundamental performance limits in image registration

    M. Dirk Robinson, Peyman Milanfar

    ICIP (2) (2003), pp. 323-326

  •  

    Optimal framework for low bit-rate block coders

    Yaakov Tsaig, Michael Elad, Gene H. Golub, Peyman Milanfar

    ICIP (2) (2003), pp. 219-222

  •  

    Reconstruction of Convex Bodies from Brightness Functions

    R. J. Gardner, Peyman Milanfar

    Discrete & Computational Geometry, vol. 29 (2003), pp. 279-303

  •  

    A statistical analysis of diffraction-limited imaging

    Peyman Milanfar, Ali Shakouri

    ICIP (1) (2002), pp. 864-867

  •  

    A computationally efficient superresolution image reconstruction algorithm

    Nhat Nguyen, Peyman Milanfar, Gene H. Golub

    IEEE Transactions on Image Processing, vol. 10 (2001), pp. 573-583

  •  

    Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement

    Nhat Nguyen, Peyman Milanfar, Gene H. Golub

    IEEE Transactions on Image Processing, vol. 10 (2001), pp. 1299-1308

  •  

    An Efficient Wavelet-Based Algorithm for Image Superresolution

    Nhat Nguyen, Peyman Milanfar

    ICIP (2000), pp. 351-354

  •  

    A model of the effect of image motion in the Radon transform domain

    Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 8 (1999), pp. 1276-1281

  •  

    Preconditioners for regularized image superresolution

    Nhat Nguyen, Gene H. Golub, Peyman Milanfar

    ICASSP (1999), pp. 3249-3252

  •  

    Two-dimensional matched filtering for motion estimation

    Peyman Milanfar

    IEEE Transactions on Image Processing, vol. 8 (1999), pp. 438-444

  •  

    Motion from Projections: A Forward Model

    Peyman Milanfar

    ICIP (2) (1998), pp. 695-699

  •  

    A moment-based variational approach to tomographic reconstruction

    Peyman Milanfar, William Clement Karl, Alan S. Willsky

    IEEE Transactions on Image Processing, vol. 5 (1996), pp. 459-470

  •  

    On the hough transform of a polygon

    Peyman Milanfar

    Pattern Recognition Letters, vol. 17 (1996), pp. 209-210

  •  

    Reconstructing polygons from moments with connections to array processing

    Peyman Milanfar, George C. Verghese, William Clement Karl, Alan S. Willsky

    IEEE Transactions on Signal Processing, vol. 43 (1995), pp. 432-443

  •  

    Modeling and Estimation for a Class of Multiresolution Random Fields

    Peyman Milanfar, Robert R. Tenney, Robert B. Washburn, Alan S. Willsky

    ICIP (3) (1994), pp. 397-401

  •  

    Moment-Based Geometric Image Reconstruction

    Peyman Milanfar, William Clement Karl, Alan S. Willsky

    ICIP (2) (1994), pp. 825-829

  •  

    Reconstructing Binary Polygonal Objects from Projections: A Statistical View

    Peyman Milanfar, William Clement Karl, Alan S. Willsky

    CVGIP: Graphical Model and Image Processing, vol. 56 (1994), pp. 371-391