Reducing Marketplace Interference Bias Via Shadow Prices

Abstract

Marketplace companies rely heavily on experimentation when making changes to the design or operation of their platforms. A fundamental challenge in marketplace experimentation is dealing with interference. For instance, consider a ride-hailing platform experimenting with a demand-side price discount. The platform performs a randomized control trial (RCT), or A/B test, where some demand units are offered the discounted price while others are offered the undiscounted price. Because the treated units are more likely to book rides as a result of the discount, they reduce the total supply available to all demand-side units, including the control units. This interference between treatment and control units causes the Stable Unit Treatment Value Assumption (SUTVA) to fail, and consequently induces bias in the standard estimator used to evaluate the value generated by the treatment.