Abstract: This paper explores a large margin approach to learning a linear transform for dimensionality reduction. The method assumes a trained Gaussian mixture model for the each class to be discriminated and trains a linear transform with respect to the model using stochastic gradient descent. Results are presented showing improvements in state classification for individual frames and reduced word error rate in a large vocabulary speech recognition problem after maximum likelihood training and boosted maximum mutual information training.