Abstract: We present Visibility Based Preconditioning (VBP) a new technique for efficiently solving the linear least squares problems that arise in bundle adjustment. Using the camera-point visibility structure of the scene, we describe the construction of two preconditioners. These preconditioners when combined with an inexact step LevenbergMarquardt algorithm offer state of the art performance on the BAL data set, with 3-5x reduction in execution time over currently available methods while delivering comparable or better solution quality