Peyman Milanfar
 Research Area(s)
 Machine Intelligence
 Machine Perception
One of these technologies is RAISR (Rapid and Accurate Image SuperResolution): Given an image, we wish to produce an image of larger size with significantly more pixels and higher image quality. With pairs of example images, we train a set of filters (i.e., a mapping) that when applied to a given image that is not in the training set, will produce a higher resolution version of it. The work was highlighted in a Research Blog post. The technology was launched for G+ photos G+ Photos worldwide; and also as part of the MotionStills app .
Another is Turbo Denoising for camera pipelines and other imaging applications. We produced a singleframe denoiser that is (1) fast enough to be practical even for mobile devices, and (2) handles content dependent noise that is typical for real camera captures. For realistic camera noise, our results are competitive with BM3D, but with nearly 400 times speedup. This technique allowed us to speed up denoising algorithm by two orders of magnitude, while producing quality that is state of the art. As a side benefit, less noisy images compress better and lead to smaller file sizes.
Another is Style Transfer which is a process of migrating a style from a given image to the content of another, synthesizing a new image which is an artistic mixture of the two. Our algorithm extends earlier work on texturesynthesis, while aiming to get stylized images that get closer in quality to ones produced by Convolutional Neural Networks. The proposed algorithm is fast and flexible, being able to process any pair of content + style images .
My team also works on more theoretical questions. For instance, in RED (Regularization by Denoising) we proposed a new way to use the denoising engine in defining the regularization for any inverse problem. RED is an explicit imageadaptive Laplacianbased regularization functional, making the overall objective functional clear and welldefined. With a complete flexibility to choose the iterative optimization procedure for minimizing the above functional, RED is capable of incorporating any image denoising algorithm, treat general inverse problems very effectively, and is guaranteed to converge to the globally optimal result. As examples of its utility, we test this approach and demonstrate stateoftheart results in the image deblurring and superresolution problems.
A bit about my background: Prior to joining Google, I was a Professor of Electrical Engineering at UC Santa Cruz from 19992014. I was also Associate Dean for Research at the School of Engineering from 201012. From 20122014 I was on leave at Googlex, where I helped develop the imaging pipeline for Google Glass. I received my undergraduate education in electrical engineering and mathematics from the University of California, Berkeley, and the MS and PhD degrees in electrical engineering from MIT. I hold 11 US patents, several of which are commercially licensed. He founded MotionDSP in 2005. I've been keynote speaker at numerous technical conferences including Picture Coding Symposium (PCS), SIAM Imaging Sciences, SPIE, and the International Conference on Multimedia (ICME). Along with my former students, I won several best paper awards from the IEEE Signal Processing Society.
I am a Distinguished Lecturer of the IEEE Signal Processing Society, and a Fellow of the IEEE "for contributions to inverse problems and superresolution in imaging."
Please visit my public website, for the most up to date list of my publications, cv, etc.
Google Publications

The little Engine that Could: Regularization by Denoising (RED)
Yaniv Romano, Michael Elad, Peyman Milanfar
ArXiv (2016) (to appear)
Previous Publications

Global Image Denoising
Hossein Talebi Esfandarani, Peyman Milanfar
IEEE Transactions on Image Processing, vol. 23 (2014), pp. 755768

A Tour of Modern Image Filtering: New Insights and Methods, Both Practical and Theoretical
IEEE Signal Process. Mag., vol. 30 (2013), pp. 106128

Blind Deconvolution Using Alternating Maximum a Posteriori Estimation with HeavyTailed Priors
Jan Kotera, Filip Sroubek, Peyman Milanfar
CAIP (2) (2013), pp. 5966

Estimating Spatially Varying Defocus Blur From A Single Image
Xiang Zhu, Scott Cohen, Stephen Schiller, Peyman Milanfar
IEEE Transactions on Image Processing, vol. 22 (2013), pp. 48794891

How to SAIFly Boost Denoising Performance
Hossein Talebi Esfandarani, Xiang Zhu, Peyman Milanfar
IEEE Transactions on Image Processing, vol. 22 (2013), pp. 14701485

Removing Atmospheric Turbulence via SpaceInvariant Deconvolution
Xiang Zhu, Peyman Milanfar
IEEE Trans. Pattern Anal. Mach. Intell., vol. 35 (2013), pp. 157170

Symmetrizing Smoothing Filters
SIAM J. Imaging Sciences, vol. 6 (2013), pp. 263284

Deconvolving PSFs for a Better Motion Deblurring Using Multiple Images
Xiang Zhu, Filip Sroubek, Peyman Milanfar
ECCV (5) (2012), pp. 636647

Improving denoising filters by optimal diffusion
Hossein Talebi Esfandarani, Peyman Milanfar
ICIP (2012), pp. 11811184

PatchBased NearOptimal Image Denoising
Priyam Chatterjee, Peyman Milanfar
IEEE Transactions on Image Processing, vol. 21 (2012), pp. 16351649

Robust Multichannel Blind Deconvolution via Fast Alternating Minimization
Filip Sroubek, Peyman Milanfar
IEEE Transactions on Image Processing, vol. 21 (2012), pp. 16871700

Robust flash denoising/deblurring by iterative guided filtering
Hae Jong Seo, Peyman Milanfar
EURASIP J. Adv. Sig. Proc., vol. 2012 (2012), pp. 3

Action Recognition from One Example
Hae Jong Seo, Peyman Milanfar
IEEE Trans. Pattern Anal. Mach. Intell., vol. 33 (2011), pp. 867882

Face Verification Using the LARK Representation
Hae Jong Seo, Peyman Milanfar
IEEE Transactions on Information Forensics and Security, vol. 6 (2011), pp. 12751286

Iteratively merging information from a pair of flash/noflash images using nonlinear diffusion
Hae Jong Seo, Peyman Milanfar
ICCV Workshops (2011), pp. 13241331

Patchbased locally optimal denoising
Priyam Chatterjee, Peyman Milanfar
ICIP (2011), pp. 25532556

Practical Bounds on Image Denoising: From Estimation to Information
Priyam Chatterjee, Peyman Milanfar
IEEE Transactions on Image Processing, vol. 20 (2011), pp. 12211233

Removing Motion Blur With SpaceTime Processing
Hiroyuki Takeda, Peyman Milanfar
IEEE Transactions on Image Processing, vol. 20 (2011), pp. 29903000

Restoration for weakly blurred and strongly noisy images
Xiang Zhu, Peyman Milanfar
WACV (2011), pp. 103109

Superfast superresolution
Filip Sroubek, Jan Kamenický, Peyman Milanfar
ICIP (2011), pp. 11531156

A noreference image content metric and its application to denoising
Xiang Zhu, Peyman Milanfar
ICIP (2010), pp. 11451148

Automatic Parameter Selection for Denoising Algorithms Using a NoReference Measure of Image Content
Xiang Zhu, Peyman Milanfar
IEEE Transactions on Image Processing, vol. 19 (2010), pp. 31163132

Fundamental limits of image denoising: Are we there yet?
Priyam Chatterjee, Peyman Milanfar
ICASSP (2010), pp. 13581361

Is Denoising Dead?
Priyam Chatterjee, Peyman Milanfar
IEEE Transactions on Image Processing, vol. 19 (2010), pp. 895911

Learning denoising bounds for noisy images
Priyam Chatterjee, Peyman Milanfar
ICIP (2010), pp. 11571160

Nonlinear kernel backprojection for computed tomography
Hiroyuki Takeda, Peyman Milanfar
ICASSP (2010), pp. 618621

TrainingFree, Generic Object Detection Using Locally Adaptive Regression Kernels
Hae Jong Seo, Peyman Milanfar
IEEE Trans. Pattern Anal. Mach. Intell., vol. 32 (2010), pp. 16881704

Visual saliency for automatic target detection, boundary detection, and image quality assessment
Hae Jong Seo, Peyman Milanfar
ICASSP (2010), pp. 55785581

A NonParametric Approach to Automatic Change Detection in MRI Images of the Brain
Hae Jong Seo, Peyman Milanfar
ISBI (2009), pp. 245248

An Adaptive Nonparametric Approach to Restoration and Interpolation for Medical Imaging
Hiroyuki Takeda, Peyman Milanfar
ISBI (2009), pp. 666669

ClusteringBased Denoising With Locally Learned Dictionaries
Priyam Chatterjee, Peyman Milanfar
IEEE Transactions on Image Processing, vol. 18 (2009), pp. 14381451

Detection of human actions from a single example
Hae Jong Seo, Peyman Milanfar
ICCV (2009), pp. 19651970

Generalizing the NonlocalMeans to SuperResolution Reconstruction
Matan Protter, Michael Elad, Hiroyuki Takeda, Peyman Milanfar
IEEE Transactions on Image Processing, vol. 18 (2009), pp. 3651

Image denoising using locally learned dictionaries
Priyam Chatterjee, Peyman Milanfar
Computational Imaging (2009), pp. 72460

Optimal Registration Of Aliased Images Using Variable Projection With Applications To SuperResolution
M. Dirk Robinson, Sina Farsiu, Peyman Milanfar
Comput. J., vol. 52 (2009), pp. 3142

SuperResolution Without Explicit Subpixel Motion Estimation
Hiroyuki Takeda, Peyman Milanfar, Matan Protter, Michael Elad
IEEE Transactions on Image Processing, vol. 18 (2009), pp. 19581975

A generalization of nonlocal means via kernel regression
Priyam Chatterjee, Peyman Milanfar
Computational Imaging (2008), pp. 68140

Deblurring Using Regularized Locally Adaptive Kernel Regression
Hiroyuki Takeda, Sina Farsiu, Peyman Milanfar
IEEE Transactions on Image Processing, vol. 17 (2008), pp. 550563

On Iterative Regularization and Its Application
Michael R. Charest, Peyman Milanfar
IEEE Trans. Circuits Syst. Video Techn., vol. 18 (2008), pp. 406411

Spatiotemporal video interpolation and denoising using motionassisted steering kernel (MASK) regression
Hiroyuki Takeda, Peter van Beek, Peyman Milanfar
ICIP (2008), pp. 637640

Using local regression kernels for statistical object detection
Hae Jong Seo, Peyman Milanfar
ICIP (2008), pp. 23802383

Video denoising using higher order optimal spacetime adaptation
Hae Jong Seo, Peyman Milanfar
ICASSP (2008), pp. 12491252

Kernel Regression for Image Processing and Reconstruction
Hiroyuki Takeda, Sina Farsiu, Peyman Milanfar
IEEE Transactions on Image Processing, vol. 16 (2007), pp. 349366

Mask Design for Optical MicrolithographyAn Inverse Imaging Problem
Amyn Poonawala, Peyman Milanfar
IEEE Transactions on Image Processing, vol. 16 (2007), pp. 774788

MultiScale Statistical Detection and Ballistic Imaging Through Turbid Media
Sina Farsiu, Peyman Milanfar
ICIP (3) (2007), pp. 537540

Multiframe demosaicing and superresolution of color images
Sina Farsiu, Michael Elad, Peyman Milanfar
IEEE Transactions on Image Processing, vol. 15 (2006), pp. 141159

Robust Kernel Regression for Restoration and Reconstruction of Images from Sparse Noisy Data
Hiroyuki Takeda, Sina Farsiu, Peyman Milanfar
ICIP (2006), pp. 12571260

Shape Estimation from Support and Diameter Functions
Amyn Poonawala, Peyman Milanfar, Richard J. Gardner
Journal of Mathematical Imaging and Vision, vol. 24 (2006), pp. 229244

Statistical and InformationTheoretic Analysis of Resolution in Imaging
Morteza Shahram, Peyman Milanfar
IEEE Transactions on Information Theory, vol. 52 (2006), pp. 34113437

Statistical performance analysis of superresolution
M. Dirk Robinson, Peyman Milanfar
IEEE Transactions on Image Processing, vol. 15 (2006), pp. 14131428

SuperResolution Imaging: Analysis, Algorithms, and Applications
Michael K. P. Ng, Tony F. Chan, Moon Gi Kang, Peyman Milanfar
EURASIP J. Adv. Sig. Proc., vol. 2006 (2006)

VideotoVideo Dynamic SuperResolution for Grayscale and Color Sequences
Sina Farsiu, Michael Elad, Peyman Milanfar
EURASIP J. Adv. Sig. Proc., vol. 2006 (2006)

Bias minimizing filter design for gradientbased image registration
M. Dirk Robinson, Peyman Milanfar
Sig. Proc.: Image Comm., vol. 20 (2005), pp. 554568

Improved spectral analysis of nearby tones using local detectors
Morteza Shahram, Peyman Milanfar
ICASSP (4) (2005), pp. 637640

Local detectors for highresolution spectral analysis: Algorithms and performance
Morteza Shahram, Peyman Milanfar
Digital Signal Processing, vol. 15 (2005), pp. 305316

Multidimensional Integral Inversion, with Applications in Shape Reconstruction
Annie A. M. Cuyt, Gene H. Golub, Peyman Milanfar, Brigitte Verdonk
SIAM J. Scientific Computing, vol. 27 (2005), pp. 10581070

On the resolvability of sinusoids with nearby frequencies in the presence of noise
Morteza Shahram, Peyman Milanfar
IEEE Transactions on Signal Processing, vol. 53 (2005), pp. 25792588

The Generalized Eigenvalue Problem for Nonsquare Pencils Using a Minimal Perturbation Approach
Gregory Boutry, Michael Elad, Gene H. Golub, Peyman Milanfar
SIAM J. Matrix Analysis Applications, vol. 27 (2005), pp. 582601

Variable projection for nearoptimal filtering in low bitrate block coders
Yaakov Tsaig, Michael Elad, Peyman Milanfar, Gene H. Golub
IEEE Trans. Circuits Syst. Video Techn., vol. 15 (2005), pp. 154160

Advances and challenges in superresolution
Sina Farsiu, M. Dirk Robinson, Michael Elad, Peyman Milanfar
Int. J. Imaging Systems and Technology, vol. 14 (2004), pp. 4757

Fast and robust multiframe super resolution
Sina Farsiu, M. Dirk Robinson, Michael Elad, Peyman Milanfar
IEEE Transactions on Image Processing, vol. 13 (2004), pp. 13271344

Fundamental performance limits in image registration
M. Dirk Robinson, Peyman Milanfar
IEEE Transactions on Image Processing, vol. 13 (2004), pp. 11851199

Imaging below the diffraction limit: a statistical analysis
Morteza Shahram, Peyman Milanfar
IEEE Transactions on Image Processing, vol. 13 (2004), pp. 677689

Shape from moments  an estimation theory perspective
Michael Elad, Peyman Milanfar, Gene H. Golub
IEEE Transactions on Signal Processing, vol. 52 (2004), pp. 18141829

Trained detection of buried mines in SAR images via the deflectionoptimal criterion
Russell B. Cosgrove, Peyman Milanfar, Joel Kositsky
IEEE T. Geoscience and Remote Sensing, vol. 42 (2004), pp. 25692575

Fast Local and Global ProjectionBased Methods for Affine Motion Estimation
M. Dirk Robinson, Peyman Milanfar
Journal of Mathematical Imaging and Vision, vol. 18 (2003), pp. 3554

Fast and robust superresolution
Sina Farsiu, M. Dirk Robinson, Michael Elad, Peyman Milanfar
ICIP (2) (2003), pp. 291294

Fundamental performance limits in image registration
M. Dirk Robinson, Peyman Milanfar
ICIP (2) (2003), pp. 323326

Optimal framework for low bitrate block coders
Yaakov Tsaig, Michael Elad, Gene H. Golub, Peyman Milanfar
ICIP (2) (2003), pp. 219222

Reconstruction of Convex Bodies from Brightness Functions
R. J. Gardner, Peyman Milanfar
Discrete Computational Geometry, vol. 29 (2003), pp. 279303

A statistical analysis of diffractionlimited imaging
Peyman Milanfar, Ali Shakouri
ICIP (1) (2002), pp. 864867

A computationally efficient superresolution image reconstruction algorithm
Nhat Nguyen, Peyman Milanfar, Gene H. Golub
IEEE Transactions on Image Processing, vol. 10 (2001), pp. 573583

Efficient generalized crossvalidation with applications to parametric image restoration and resolution enhancement
Nhat Nguyen, Peyman Milanfar, Gene H. Golub
IEEE Transactions on Image Processing, vol. 10 (2001), pp. 12991308

An Efficient WaveletBased Algorithm for Image Superresolution
Nhat Nguyen, Peyman Milanfar
ICIP (2000), pp. 351354

A model of the effect of image motion in the Radon transform domain
IEEE Transactions on Image Processing, vol. 8 (1999), pp. 12761281

Preconditioners for regularized image superresolution
Nhat Nguyen, Gene H. Golub, Peyman Milanfar
ICASSP (1999), pp. 32493252

Twodimensional matched filtering for motion estimation
IEEE Transactions on Image Processing, vol. 8 (1999), pp. 438444

Motion from Projections: A Forward Model
ICIP (2) (1998), pp. 695699

A momentbased variational approach to tomographic reconstruction
Peyman Milanfar, William Clement Karl, Alan S. Willsky
IEEE Transactions on Image Processing, vol. 5 (1996), pp. 459470

On the hough transform of a polygon
Pattern Recognition Letters, vol. 17 (1996), pp. 209210

Reconstructing polygons from moments with connections to array processing
Peyman Milanfar, George C. Verghese, William Clement Karl, Alan S. Willsky
IEEE Transactions on Signal Processing, vol. 43 (1995), pp. 432443

Modeling and Estimation for a Class of Multiresolution Random Fields
Peyman Milanfar, Robert R. Tenney, Robert B. Washburn, Alan S. Willsky
ICIP (3) (1994), pp. 397401

MomentBased Geometric Image Reconstruction
Peyman Milanfar, William Clement Karl, Alan S. Willsky
ICIP (2) (1994), pp. 825829

Reconstructing Binary Polygonal Objects from Projections: A Statistical View
Peyman Milanfar, William Clement Karl, Alan S. Willsky
CVGIP: Graphical Model and Image Processing, vol. 56 (1994), pp. 371391