Douglas Eck

Douglas Eck

Co-Authors

I’m a research scientist working on Magenta, a research project exploring the role of machine learning in the process of creating art and music. Primarily this involves developing new deep learning and reinforcement learning algorithms for generating songs, images, drawings, and other materials. But it's also an exploration in building smart tools and interfaces that allow artists and musicians to extend (not replace!) their processes using these models. Started by me in 2016, Magenta now involves several researchers and engineers from the Google Brain team as well as many others collaborating via open source. Aside from Magenta, I'm working on sequence learning models for summarization and text generation as well new ways to improve AI-generated content based on user feedback.

I have also worked on music recommendation for Play Music, involving both both learning from audio and learning from how users consume music. In the audio domain, the main goal is to transform the ones and zeros in a digital audio file into something where musically-similar songs are also numerically similar, making it easier to do music recommendation. This is (a) user-dependent: my idea of similar is not the same as yours and (b) changes with context: my idea of similarity changes when I make a playlist for jogging versus making a playlist for a dinner party. I might choose the same song (say "Taxman" by the Beatles) but perhaps it would be the tempo for jogging that drove the selection of that specific song versus "I like the album Revolver and want to add it to the dinner party mix" for a dinner party playlist.

Before joining Google in 2010, I was an Associate Professor in Computer Science at University of Montreal. I helped found the BRAMS research center (Brain Music and Sound; www.brams.org) and was involved at the McGill CIRMMT center (Centre for Interdisciplinary Research in Music Media and Technology; www.cirmmt.org). Aside from audio signal processing and machine learning, I worked on music performance modeling. What exactly does a good music performer add to what is already in the score? I treated this as a machine learning question: Hypothetically, if we showed a piano-playing robot a huge collection of Chopin performances--- from the best in the world all the way down to that of a struggling teenage pianist---could it learn to play well by analyzing all of these examples? If so, what’s the right way to perform that analysis? In the end I learned a lot about the complexity and beauty of human music performance, and how performance relates to and extends composition.

Google Publications

Previous Publications

  •  

    Acoustic Space Sampling and the Grand Piano in a Non-Anechoic Environment: a recordist-centric approach to musical acoustic study

    B. Leonard, G. Sikora, M. De Francisco, Douglas Eck

    129th Audio Engineering Society (AES) Convention, London (2010)

  •  

    Acoustic Space Sampling and the Grand Piano in a Non-Anechoic Environment: a recordist-centric approach to to musical acoustic study

    B. Leonard, G. Sikora, M. De Francisco, Douglas Eck

    129th Audio Engineering Society (AES) Convention, London (2010)

  •  

    An Infinite Factor Model Hierarchy Via a Noisy-Or Mechanism

    A. Courville, Douglas Eck, Y. Bengio

    Neural Information Processing Systems Conference 22 (NIPS'09) (2010)

  •  

    Automatic identification of instrument classes in polyphonic and poly-instrument audio

    P. Hamel, S. Wood, Douglas Eck

    Proceedings of the 10th International Conference on Music Information Retrieval (ISMIR 2009)

  •   

    Probabilistic Models for Melodic Prediction

    Jean-Francois Paiement, Samy Bengio, Douglas Eck

    Artificial Intelligence Journal, vol. 173 (2009), pp. 1266-1274

  •  

    Steerable Playlist Generation by Learning Song Similarity from Radio Station Playlists

    F. Maillet, Douglas Eck, G. Desjardins, P. Lamere

    Proceedings of the 10th International Conference on Music Information Retrieval (ISMIR 2009)

  •  

    Towards a musical beat emphasis function

    M. Davies, M. Plumbley, Douglas Eck

    Proceedings of IEEE WASPAA, New Paltz, NY (2009)

  •   

    A Distance Model for Rhythms

    Jean-Francois Paiement, Yves Grandvalet, Samy Bengio, Douglas Eck

    International Conference on Machine Learning (ICML) (2008)

  •   

    A Generative Model for Rhythms

    Jean-Francois Paiement, Samy Bengio, Yves Grandvalet, Doug Eck

    Neural Information Processing Systems, Workshop on Brain, Music and Cognition (2008)

  •  

    A generative model for rhythms

    {J.-F.} Paiement, Y. Grandvalet, S. Bengio, Douglas Eck

    ICML '08: Proceedings of the 25th International Conference on Machine Learning (2008)

  •  

    Automatic generation of social tags for music recommendation

    Douglas Eck, P. Lamere, T. Bertin-Mahieux, S. Green

    Neural Information Processing Systems Conference 20 (NIPS'07) (2008)

  •  

    Autotagger: A Model For Predicting Social Tags from Acoustic Features on Large Music Databases

    T. Bertin-Mahieux, Douglas Eck, F. Maillet, P. Lamere

    Journal of New Music Research, vol. 37 (2008), pp. 115-135

  •  

    On the use of Sparse Time Relative Auditory Codes for Music

    P-A. Manzagol, T. Bertin-Mahieux, Douglas Eck

    Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR 2008)

  •   

    A Generative Model for Distance Patterns in Music

    Jean-Francois Paiement, Yves Grandvalet, Samy Bengio, Douglas Eck

    NIPS Workshop on Music, Brain and Cognition (2007)

  •  

    A Supervised Classification Algorithm For Note Onset Detection

    A. Lacoste, Douglas Eck

    EURASIP Journal on Applied Signal Processing, vol. 2007 (2007), pp. 1-13

  •  

    Autotagging music using supervised machine learning

    Douglas Eck, T. Bertin-Mahieux, P. Lamere

    Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR 2007)

  •  

    Beat Tracking Using an Autocorrelation Phase Matrix

    Douglas Eck

    Proceedings of the 2007 International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE Signal Processing Society, pp. 1313-1316

  •  

    Can't get you out of my head: A connectionist model of cyclic rehearsal

    H. Jaeger, Douglas Eck

    Modeling Communications with Robots and Virtual Humans, Springer-Verlag (2007)

  •  

    Using 3D Visualizations to Explore and Discover Music

    P. Lamere, Douglas Eck

    Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR 2007)

  •  

    Aggregate Features and AdaBoost for Music Classification

    J. Bergstra, N. Casagrande, D. Erhan, Douglas Eck, B. Kégl

    Machine Learning, vol. 65 (2006), pp. 473-484

  •  

    Beat Induction Using an Autocorrelation Phase Matrix

    Douglas Eck

    The Proceedings of the 9th International Conference on Music Perception and Cognition (ICMPC9), Causal Productions (2006), pp. 931-932

  •  

    Finding Long-Timescale Musical Structure with an Autocorrelation Phase Matrix

    Douglas Eck

    Music Perception, vol. 24 (2006), pp. 167-176

  •  

    Predicting genre labels for artists using FreeDB

    J. Bergstra, A. Lacoste, Douglas Eck

    Proceedings of the 7th International Conference on Music Information Retrieval (ISMIR 2006), pp. 85-88

  •  

    Probabilistic Melodic Harmonization

    J.-F. Paiement, D. Eck, S. Bengio

    Advances in Artificial Intelligence: 19th Conference of the Canadian Society for Computational Studies of Intelligence, Canadian AI, Lecture Notes in Computer Science, Springer-Verlag (2006), pp. 218-229

  •  

    Probabilistic Melodic Harmonization

    {J.-F.} Paiement, Douglas Eck, S. Bengio

    Canadian Conference on AI, Springer (2006), pp. 218-229

  •  

    A Graphical Model for Chord Progressions Embedded in a Psychoacoustic Space

    J.-F. Paiement, D. Eck, S. Bengio, D. Barber

    International Conference on Machine Learning, ICML (2005)

  •  

    A Probabilistic Model for Chord Progressions

    J.-F. Paiement, D. Eck, S. Bengio

    International Conference on Music Information Retrieval, ISMIR (2005)

  •  

    A Probabilistic Model for Chord Progressions

    {J.-F.} Paiement, Douglas Eck, S. Bengio

    Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR 2005), London: University of London, pp. 312-319

  •  

    A graphical model for chord progressions embedded in a psychoacoustic space

    {J.-F.} Paiement, Douglas Eck, S. Bengio, D. Barber

    ICML '05: Proceedings of the 22nd international conference on Machine learning, ACM Press, New York, NY, USA (2005), pp. 641-648

  •  

    Editorial: New Research in Rhythm Perception and Production

    Douglas Eck, S. K. Scott

    Music Perception, vol. 22 (2005), pp. 371-388

  •  

    Finding Meter in Music Using an Autocorrelation Phase Matrix and Shannon Entropy

    Douglas Eck, N. Casagrande

    Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR 2005), London: University of London, pp. 504-509

  •  

    Frame-Level Audio Feature Extraction using AdaBoost

    N. Casagrande, Douglas Eck, B. Kégl

    Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR 2005), London: University of London, pp. 345-350

  •  

    Geometry in Sound: A Speech/Music Audio Classifier Inspired by an Image Classifier

    N. Casagrande, Douglas Eck, B. Kegl

    Proceedings of the International Computer Music Conference (ICMC) (2005), pp. 207-210

  •  

    Music Perception, Guest Editor, Special Issue on Rhythm Perception and Production

    Douglas Eck, S. K. Scott

    Music Perception, vol. 22 (3) (2005)

  •  

    A Machine-Learning Approach to Musical Sequence Induction That Uses Autocorrelation to Bridge Long Timelags

    Douglas Eck

    The Proceedings of the Eighth International Conference on Music Perception and Cognition (ICMPC8), Causal Productions, Adelaide (2004), pp. 542-543

  •  

    Biologically Plausible Speech Recognition with LSTM Neural Nets

    A. Graves, Douglas Eck, N. Beringer, J. Schmidhuber

    Proceedings of the First Int'l Workshop on Biologically Inspired Approaches to Advanced Information Technology (Bio-ADIT) (2004), pp. 127-136

  •  

    Kalman filters improve LSTM network performance in problems unsolvable by traditional recurrent nets

    J.A. Pérez-Ortiz, F. A. Gers, Douglas Eck, J. Schmidhuber

    Neural Networks, vol. 16 (2003), pp. 241-250

  •  

    DEKF-LSTM

    F.A. Gers, J.A. Perez-Ortiz, Douglas Eck, J. Schmidhuber

    Proceedings of the 10th European Symposium on Artificial Neural Networks, ESANN 2002

  •  

    Finding Downbeats with a Relaxation Oscillator

    Douglas Eck

    Psychological Research, vol. 66 (2002), pp. 18-25

  •  

    Finding Temporal Structure in Music: Blues Improvisation with LSTM Recurrent Networks

    Douglas Eck, J. Schmidhuber

    Neural Networks for Signal Processing XII, Proceedings of the 2002 IEEE Workshop, IEEE, New York, pp. 747-756

  •  

    Improving Long-Term Online Prediction with Decoupled Extended Kalman Filters

    J.A. Pérez-Ortiz, J. Schmidhuber, F.A. Gers, Douglas Eck

    Artificial Neural Networks -- ICANN 2002 (Proceedings), Springer, Berlin, pp. 1055-1060

  •  

    Learning Context Sensitive Languages with LSTM Trained with Kalman Filters

    F.A. Gers, J.A. Pérez-Ortiz, Douglas Eck, J. Schmidhuber

    Artificial Neural Networks -- ICANN 2002 (Proceedings), Springer, Berlin, pp. 655-660

  •  

    Learning Nonregular Languages: A Comparison of Simple Recurrent Networks and LSTM

    J. Schmidhuber, F.A. Gers, Douglas Eck

    Neural Computation, vol. 14 (2002), pp. 2039-2041

  •  

    Learning The Long-Term Structure of the Blues

    Douglas Eck, J. Schmidhuber

    Artificial Neural Networks -- ICANN 2002 (Proceedings), Springer, Berlin, pp. 284-289

  •  

    A Network of Relaxation Oscillators that Finds Downbeats in Rhythms

    Douglas Eck

    Artificial Neural Networks -- ICANN 2001 (Proceedings), Springer, Berlin, pp. 1239-1247

  •  

    A Positive-Evidence Model for Rhythmical Beat Induction

    Douglas Eck

    Journal of New Music Research, vol. 30 (2001), pp. 187-200

  •  

    Applying LSTM to Time Series Predictable Through Time-Window Approaches

    F. A. Gers, Douglas Eck, J. Schmidhuber

    Artificial Neural Networks -- ICANN 2001 (Proceedings), Springer, Berlin, pp. 669-676

  •  

    Dynamics and Embodiment in Beat Induction

    Douglas Eck, M. Gasser, Robert Port

    Rhythm Perception and Production, Swets and Zeitlinger, Lisse, The Netherlands (2000), pp. 157-170

  •  

    Meter Through Synchrony: Processing Rhythmical Patterns with Relaxation Oscillators

    Douglas Eck

    Ph.D. Thesis, Indiana University, Bloomington, IN (2000)

  •  

    An Exploration of Representational Complexity via Coupled Oscillators

    T. Chemero, Douglas Eck

    Proceedings of the Tenth Midwest Artificial Intelligence and Cognitive Science Society, MIT Press, Cambridge, Mass. (1999)

  •  

    Learning Simple Metrical Preferences in a Network of Fitzhugh-Nagumo Oscillators

    Douglas Eck

    The Proceedings of the Twenty-First Annual Conference of the Cognitive Science Society, Lawrence Erlbaum Associates, New Jersey (1999)

  •  

    Meter as Mechanism: A Neural Network Model that Learns Metrical patterns

    M. Gasser, Douglas Eck, R. Port

    Connect. Sci., vol. 11, no. 2 (1999), pp. 187-216

  •  

    Perception of Simple Rhythmic Patterns in a Network of Oscillators

    Douglas Eck, M. Gasser

    The Proceedings of the Eighteenth Annual Conference of the Cognitive Science Society, Lawrence Erlbaum Associates, New Jersey (1996)

  •  

    Representing Rhythmic Patterns in a Network of Oscillators

    M. Gasser, Douglas Eck

    The Proceedings of the International Conference on Music Perception and Cognition, Lawrence Erlbaum Associates, New Jersey (1996), pp. 361-366