Discussion Papers

Undiscounted Bandit Games

Abstract:

We analyze continuous-time games of strategic experimentation with two-armedbandits when there is no discounting. We show that for all specifications of priorbeliefs and payoff-generating processes that satisfy some separability condition, the unique symmetric Markov perfect equilibrium can be computed in a simple closed form involving only the expected current payoff of the risky arm and the expected full-information payoff, given current information. The separability condition holds in a variety of models that have been explored in the literature, all of which assume that the risky arm’s expected payoff per unit of time is time-invariant and actual payoffs are generated by a process with independent and stationary increments. The separability condition does not hold when the expected payoff per unit of time is subject to state-switching.

JEL classification: C73, D83
Keywords: Strategic Experimentation, Two-Armed Bandit, Markov-Perfect Equilibrium

Full text in pdf format:

520c.pdf

Who goes first? Strategic Delay and Learning by Waiting

Abstract:

This paper considers a "war of attrition" game in which agents learn about an uncertain state of the world through private signals and from their peers. I provide existence and uniqueness results for a class of equilibria that satisfy a \full-participation" condition, and show that asymmetries in the distribution of information can lead to excessive stopping and an oversupply of information relative to the social optimum.

Full text in pdf format:

500.pdf

Learning faster or more precisely? Strategic experimentation in networks

Abstract:

The paper analyzes a dynamic model of rational strategic learning in a network. It complements existing literature by providing a detailed picture of short-run dynamics in a game of strategic experimentation where agents are located in a social network. We show that the delay in information transmission caused by incomplete network structures may induce players to increase own experimentation efforts. As a consequence a complete network can fail to be optimal even if there are no costs for links. This means that in the design of networks there exists a trade-off between the speed of learning and accuracy.

Key Words: Strategic Experimentation, Networks, Learning

JEL codes: C73, D83, D85

Full text in pdf format:

485.pdf

Strongly Symmetric Equilibria in Bandit Games

Abstract:

This paper studies strongly symmetric equilibria (SSE) in continuous-time games of strategic experimentation with Poisson bandits. SSE payoffs can be studied via two functional equations similar to the HJB equation used for Markov equilibria. This is valuable for three reasons. First, these equations retain the tractability of Markov equilibrium, while allowing for punishments and rewards: the best and worst equilibrium payoff are explicitly solved for. Second, they capture behavior of the discrete-time game: as the period length goes to zero in the discretized game, the SSE payoff set converges to their solution. Third, they encompass a large payoff set: there is no perfect Bayesian equilibrium in the discrete-time game with frequent interactions with higher asymptotic efficiency.

Keywords: Two-Armed Bandit, Bayesian Learning, Strategic Experimentation,

Strongly Symmetric Equilibrium.

JEL Classification Numbers: C73, D83.

Full text in pdf format:

469.pdf

Breakdowns

Abstract:

We study a continuous-time game of strategic experimentation in which the players try to assess the failure rate of some new equipment or technology. Breakdowns occur at the jump times of a Poisson process whose unknown intensity is either high or low. In marked contrast to existing models, we find that the cooperative value function does not exhibit smooth pasting at the efficient cut-off belief. This finding extends to the boundaries between continuation and stopping regions in Markov perfect equilibria. We characterize the unique symmetric equilibrium, construct a class of asymmetric equilibria, and elucidate the impact of bad versus good Poisson news on equilibrium outcomes.

Keywords: Strategic Experimentation, Two-Armed Bandit, Bayesian Learning,
Poisson Process, Piecewise Deterministic Process, Markov Perfect Equilibrium,
Differential-Difference Equation, Smooth Pasting, Continuous Pasting.
JEL Classification Numbers: C73, D83, O32

Full text in pdf format:

396.pdf

Strategic Experimentation with Private Payoffs

Abstract:

We consider two players facing identical discrete-time bandit problems with a safe and a risky arm. In any period, the risky arm yields either a success or a failure, and the first success reveals the risky arm to dominate the safe one. When payoffs are public information, the ensuing free-rider problem is so severe that the equilibrium number of experiments is at most one plus the number of experiments that a single agent would perform. When payoffs are private information and players can communicate via cheap talk, the socially optimal symmetric experimentation profile can be supported as a perfect Bayesian equilibrium for sufficiently optimistic prior beliefs. These results generalize to more than two players whenever the success probability per period is not too high. In particular, this is the case when successes occur at the jump times of a Poisson process and the period length is sufficiently small.

JEL classification: C73, D83.

Keywords: Strategic Experimentation, Bayesian Learning, Cheap Talk, Two-Armed Bandit, Information Externality.

Full text in pdf format:

387.pdf

Continuous Time Contests

Abstract:

This paper introduces a contest model in which each player decides when to stop a privately observed Brownian motion with drift and incurs costs depending on his stopping time. The player who stops his process at the highest value wins a prize. Applications of the model include procurement contests and competitions for grants. We prove existence and uniqueness of the Nash equilibrium outcome, even if players have to choose bounded stopping times. We derive the equilibrium distribution in closed form. If the noise vanishes, the equilibrium outcome converges to - and thus selects - the symmetric equilibrium outcome of an all-pay auction. For two players and constant costs, each player’s profits increase if costs for both players increase, variance increases, or drift decreases. Intuitively, patience becomes a more important factor for contest success, which reduces informational rents.
Keywords: Contests, all-pay contests, silent timing games.

Full text in pdf format:

376.pdf

Gambling in Contests

Abstract:

This paper presents a strategic model of risk-taking behavior in contests. Formally, we analyze an n-player winner-take-all contest in which each player decides when to stop a privately observed Brownian Motion with drift. A player whose process reaches zero has to stop. The player with the highest stopping point wins. Contrary to the explicit cost for a higher stopping time in a war of attrition, here, higher stopping times are riskier, because players can go bankrupt. We derive a closed-form solution of the unique Nash equilibrium outcome of the game. In equilibrium, the trade-off between risk and reward causes a non-monotonicity: highest expected losses occur if the process decreases only slightly in expectation.
Keywords: Discontinuous games; Contests; Relative performance pay; Risktaking behavior
JEL classification: C72; C73; D81

Full text in pdf format:

375.pdf

Experimentation in Two-Sided Markets

Abstract:

We study optimal experimentation by a monopolistic platform in a two-sided market framework. The platform provider faces uncertainty about the strength of the externality each side is exerting on the other. It maximizes the expected present value of its profit stream in a continuous-time infinite-horizon framework by setting participation fees or quantities on both sides. We show that a price-setting platform provider sets a fee lower than the myopically optimal level on at least one side of the market, and on both sides if the two externalities are of approximately equal strenght. If the externality that one side exerts is sufficiently weaker than the externality it experiences, the optimal fee on this side exceeds the myopically optimal level. We obtain analogous results for expected prives when the platform provider chooses quantities. While the optimal policy does not admin closed-form representations in general, we identify special cases in which the undiscounted limit of the model can be solved in closed form.
Keywords: Two-Sided Market, Network Effects, Monopoly Experimentation, Bayesian Learning, Optimal Control
JEL classification: D42, D83, L12

Full text in pdf format:

365.pdf

Dynamic Coordination via Organizational Routines

Abstract:

We investigate dynamic coordination among members of a problem solving team who receive private signals about which of their actions are required for a (static) coordinated solution and who have repeated opportunities to explore different action combinations. In this environment ordinal equilibria, in which agents condition only on how their signals rank their actions and not on signal strength, lead to simple patterns of behavior that have a natural interpretation as routines. These routine spartially solve the team’s coordination problem by synchronizing the team’s search efforts and prove to be resilient to changes in the environment by being expost equilibria, to agents having only a coarse understanding of other agents’ strategies by being fully cursed, and to natural forms of agents’ overconfidence. The price of this resilience is that optimal routines are frequently not optimal equilibria.

January 2011

Full text in pdf format:

355.pdf

Strategic Learning in Teams

Abstract:

This paper analyzes a two-player game of strategic experimentation with three-armed exponential bandits in continuous time. Players face replica bandits, with one arm that is safe in that it generates a known payoff, whereas the likelihood of the risky arms’ yielding a positive payoff is initially unknown. It is common knowledge that the types of the two risky arms are perfectly negatively correlated. I show that the efficient policy is incentive-compatible if, and only if, the stakes are high enough. Moreover, learning will be complete in any Markov perfect equilibrium with continuous value functions if, and only if, the stakes exceed a certain threshold.

Keywords: Strategic Experimentation, Three-Armed Bandit, Exponential Distribution, Poisson Process, Bayesian Learning, Markov Perfect Equilibrium
JEL Classification: C73, D83, O32

July 2010

Full text in pdf format:

333.pdf

Technology Adoption, Social Learning, and Economic Policy

Abstract:

We study a two-player dynamic investment model with information externalities and provide necessary and sufficient conditions for a unique switching equilibrium. When the public information is sufficiently high and a social planer therefore expects an investment boom, investments should be taxed. Conversely, any positive investment tax is suboptimally high if the public information is sufficiently unfavorable.We also show that an investment tax may increase overall investment activity.

Keywords: Information Externality, Strategic Waiting, Delay, Information Cascade, Investment Boom, Optimal Taxation

JEL Classification: D62, D83

February 2010

Full text in pdf format:

306.pdf

Screening and Merger Activity

Abstract:

In our paper targets, by setting a reserve price, screen acquirers on their (expected) ability to generate merger-specific synergies. Both empirical evidence and many common merger models suggest that the difference between high- and low-synergy mergers becomes smaller during booms. This implies that the target’s opportunity cost for sorting out relatively less fitting acquirers increases and, hence, targets screen less tightly during booms, which leads to a hike in merger activity. Our screening mechanism not only predicts that merger activity is intense during economic booms and subdued during recessions but is also consistent with other stylized facts about takeovers and generates novel testable predictions.

Keywords: Takeovers, Merger Waves, Defense Tactics, Screening

JEL Classification: D21, D80, L11.

August 2009

Full text in pdf format:

270.pdf

Strategic Experimentation with Poisson Bandits

Abstract:

We study a game of strategic experimentation with two-armed bandits where the risky arm distributes lump-sum payoffs according to a Poisson process. Its intensity is either high or low, and unknown to the players. We consider Markov perfect equilibria with beliefs as the state variable. As the belief process is piecewise deterministic, payoff functions solve differential-difference equations.  here is no equilibrium where all players use cut-off strategies, and all equilibria exhibit an ‘encouragement effect’ relative to the single-agent optimum. We construct asymmetric equilibria in which players have symmetric continuation values at sufficiently optimistic beliefs yet take turns playing the risky arm before all experimentation stops. Owing to the encouragement effect, these equilibria Pareto dominate the unique symmetric one for sufficiently frequent turns. Rewarding the last experimenter with a higher continuation value increases the range of beliefs where players experiment, but may reduce average payoffs at more optimistic beliefs. Some equilibria exhibit an ‘anticipation effect’: as beliefs become more pessimistic, the continuation value of a single experimenter increases over some range because a lower belief means a shorter wait until another player takes over.

Keywords: Strategic Experimentation, Two-Armed Bandit, Poisson Process, Bayesian Learning, Piecewise Deterministic Process, Markov Perfect Equilibrium, Differential-Difference Equation

JEL Classification: C73, D83, O32

May 2009

Full text in pdf format:

260.pdf

Negatively Correlated Bandits

Abstract:

We analyze a two-player game of strategic experimentation with two-armed bandits. Each player has to decide in continuous time whether to use a safe arm with a known payoff or a risky arm whose likelihood of delivering payoffs is initially unknown. The quality of the risky arms is perfectly negatively correlated between players. In marked contrast to the case where both risky arms are of the same type, we find that learn- ing will be complete in any Markov perfect equilibrium if the stakes exceed a certain threshold, and that all equilibria are in cutoff strategies. For low stakes, the equilib- rium is unique, symmetric, and coincides with the planner's solution. For high stakes, the equilibrium is unique, symmetric, and tantamount to myopic behavior. For inter- mediate stakes, there is a continuum of equilibria.

Keywords: Strategic Experimentation, Two-Armed Bandit, Exponential Distribution, Poisson Process, Bayesian Learning, Markov Perfect Equilibrium
JEL Classification: C73, D83, O32
August 2008

Full text in pdf format:

243.pdf