Abstract: In order to cope for the difficult problem of long term dependencies in sequential data in general, and in musical data in particular, a generative model for distance patterns especially designed for music is introduced. A specific implementation of the model when considering Hamming distances over rhythms is described. The proposed model consistently outperforms a standard Hidden Markov Model in terms of conditional prediction accuracy over two different music databases.