Ad Exchange platforms connect online publishers and advertisers and facilitate
selling billions of impressions every day. We study these environments from the
perspective of a publisher who wants to find the profit maximizing exchange to sell
his inventory. Ideally, the publisher would run an auction among exchanges.
However, this is not possible due to technological and other practical
considerations. The publisher needs to send each impression to one of the exchanges
with an asking price. We model the problem as a variation of multi-armed bandits
where exchanges (arms) can behave strategically in order to maximizes their own
profit. We propose mechanisms that find the best exchange with sub-linear regret
and have desirable incentive properties.