We consider a multidimensional search problem that is motivated by questions in
contextual decision-making, such as dynamic pricing and personalized medicine.
Nature selects a state from a d-dimensional unit ball and then generates a sequence
of d-dimensional directions. We are given access to the directions, but not access
to the state. After receiving a direction, we have to guess the value of the dot
product between the state and the direction. Our goal is to minimize the number of
times when our guess is more than ϵ away from the true answer. We construct a
polynomial time algorithm that we call Projected Volume achieving regret
O(dlog(d/ϵ)), which is optimal up to a logd factor. The algorithm combines a volume
cutting strategy with a new geometric technique that we call cylindrification.