Abstract: We propose a novel framework for reconstructing homogenous, transparent, refractive height-fields from a single viewpoint. The height-field is imaged against a known planar background, or sequence of backgrounds. Unlike existing approaches that do a point-by-point reconstruction – which is known to have intractable ambiguities – our method estimates and optimizes for the entire height-field at the same time. The formulation supports shape recovery from measured distortions (deflections) or directly from the images themselves, including from a single image. We report results for a variety of refractive height-fields showing significant improvement over prior art.