Strategic Experimentation with Exponential Bandits

This paper studies a game of strategic experimentation with
two-armed bandits whose risky arm might yield a payoff only after
some exponentially distributed random time. Because of free-riding,
there is an inefficiently low level of experimentation in any
equilibrium where the players use stationary Markovian strategies
with posterior beliefs as the state variable. After characterizing
the unique symmetric Markovian equilibrium of the game, which is in
mixed strategies, we construct a variety of pure-strategy
equilibria. There is no equilibrium where all players use simple
cut-off strategies. Equilibria where players switch finitely often
between the roles of experimenter and free-rider all lead to the
same pattern of information acquisition; the efficiency of these
equilibria depends on the way players share the burden of
experimentation among them. In equilibria where players switch
roles infinitely often, they can acquire an approximately efficient
amount of information, but the rate at which it is acquired still
remains inefficient; moreover, the expected payoff of an
experimenter exhibits the novel feature that it rises as players
become more pessimistic. Finally, over the range of beliefs where
players use both arms a positive fraction of the time, the
symmetric equilibrium is dominated by any asymmetric one in terms
of aggregate payoffs.