Lecture 5 (Dutta): Dynamic Programming 4

Original article (link) posted: 13/10/2005

First we looked at two more properties of the value function, supermodularity and differentiability.
Then, we examined the model with slightly different assumptions; deterministic transition function, unbounded reward function, unit discount factor, and finite horizon DP.

Result (Supermodularity)
Suppose action space is subset of R. If the reward function is supermodular and transition function is action-dependent, then the optimal action correspondence is monotone (the largest action in the optimal set is monotone).

Result (Differentiability)
Suppose reward function is differentiable on S and either
a) transition function is action-dependent
b) transition function has a density which is differentiable on S
then, V is differentiable on int S

Deterministic model
Just a special case of the stochastic case

Unbounded reward
Some kind of bound conditions are needed

No discounting
Continuity of discounting case and no discounting case (See Dutta (1995))
It is hard to derive the value function without discounting (Long-run average payoff). So, we can first solve the value function in the discounting case and make a discount factor go to unity to solve it.

Finite horizon DP (continuity of V)
The value function of a finite DP problem with T remaining period, V(T) converges (in sup norm) to that of infinite horizon model as T goes to infinite.

Finite horizon DP (continuity of h)
Under continuity and compactness assumptions, there exists a sequence of optimal action policies h(t), h(t-1),… If h(T) converges to some policy, say h, as T goes to infinity, then h is a stationary optimal policy for the corresponding infinite DP problem.


3G Auctions in UK: Q&A

The last chapter of Paul Klemperer's famous auction book provides non-technical answers to the FAQs on spectrum auctions, especially on the 3G auction that he (and Ken Binmore) designed and implemented in the UK.

The chapter titled "Were Auctions a Good Idea?" is based on his newspaper article, "The Wrong Culprit for Telecom Trouble" appeared in Financial Times (Nov. 26, 2002) where he listed three typical critics against 3G auctions:
The critics assume three things, that 
1: the telecoms companies paid more for the licenses than they thought the licenses were worth
2: this expenditure has reduced investment in 3G
3: it destroyed the telecoms companies' market value
Paul answers to each question as follows:
1. It is now hard to imagine any company voluntarily paid billions for a 3G license. But volunteer they all did - and they celebrated their victories. (...) Like any other market, an auction simply matches willing buyers and willing sellers - it cannot protect them against their own mistakes.
2. The auction fees are history; they have (almost) all been paid in full, cannot be recouped by cutting investment, and make no difference to its profitability. (...) Investment in 3G, as in anything else, is primarily motivated by attractive returns in the future - not by money spent in the past.
3. All in all, it seems a stretch to blame $100bn of shareholder misery on $100bn of 3G license fees.
Then, he concludes the article by saying:
There is nothing special about an auction: it is just another market. Buying houses or shares at the peak of a housing or stock-market boom does not entitle anyone to compensation. Why should we make an exception for the phone companies?
The main effect of the license fees was simply to transfer $100bn from shareholders around the world to certain European governments. This was both equitable, since the companies were buying a public asset that they valued this highly at the time, and efficient, since such a lump sum transfer is much more efficient than most forms of taxation. Efficient, equitable and voluntary government funding is not easy to find: perhaps we should be more enthusiastic about it.


Theory Seminar "Fudenberg and Levine"

Original article (link) posted: 08/10/2005

Fudenberg and Levine (2005) "A Dual Self Model of Impulse Control"

The paper proposes a "dual-self" model for a single agent decision making. In their model, the patient long-run self and a sequence of myopic short-run selves who share the same preferences over the stage-game outcome play games to decide some dynamic decision. In each period, the long-run self moves first and chooses the utility function of the myopic self possibly with some reduction in utility ("self-control"). After seeing this "self-control" level, the short-run player takes the final decision.
The model gives a unified explanation for a number of empirical regularities related to self-control problems and a value for commitment in decision problems, including the apparent time-inconsistency that has motivated models of hyperbolic discounting and Rabin's paradox of risk aversion in the large and small. The base version of the model is consistent with Gul-Pesendorfer axioms.

The paper is quite interesting and the presentation by Professor Levine was really nice. He illustrated many experimental results and explain how their theory can explain them. However, it was bit unclear for me to see their marginal contribution in the literature of behavioral Economics. It seems that there are many alternative theories which can also explain those experimental results. I must check the papers in this field at least to some extent...

Interesting Papers on Reference

Gul and Pesendorfer (2001) "Temptation and Self-Control" Econometrica, 69
: An axiomatic approach of temptation preference
Gul and Pesendorfer (2004) "Self-Control and the Theory of Consumption" Econometrica, 72
: Generalization of the 2001 paper with multi-period decision aiming for the application to macro economics
Krusell and Smith (2003) "Consumption-Savings Decisions with Quasi-Geometric Discounting" Econometrica, 71
: Multiple equilibria in hyperbolic models
Laibson (1997) "Golden eggs and hyperbolic discounting" QJE, 112
: Revival of non-exponential discounting preference
Laibson (2001) "A Cue-Theory of Consumption" QJE, 116
: Cue-theory
O'Donogue and Rabin (1999) "Doing It Now or Latter" AER, 89
: Early or latter decision
Rabin (2000) "Risk Aversion and Expected-Utility Theory: A Calibration Theorem" Econometrica
: Rabin's paradox
Thaler and Shefin (1981) "An Economic Theory of Self-Control" JPE, 89
: A pioneering work of self-control


Contract Theory Conference

I organized the 4th Japan-Taiwan Contract Theory Conference on December 4, 2010 at my institute, National Graduate Institute for Policy Studies (GRIPS), which turned out to be extremely successful. We had 49 participants, including 18 guests from Taiwan; enjoyed listening to six intriguing talks and discussion. I am grateful for all who kindly attended this event and helped me setting it up.
The following is the conference program with my brief comments.

First session: 10:05--11:55

Chair: Chia-Hui Chen (Academia Sinica)
Speaker 1: Chien-Lung Chen (National Tsing-Hua University) 
  • How Can Agents Be Better-Off with Pay Caps?  
Speaker 2: Mami Kobayashi (Kinki University)  
  • Market Liquidity and Capital Structure of Financial Institution

Comments: The first talk is about optimal wage scheme under pay caps (in addition to minimum wage constraints), and the second is about incentive issues on corporate finance and governance. Theoretical robustness and the connection to actual labor and financial markets were discussed.                                              

Second session: 13:10--15:00
Chair: Jong-Rong Chen (National Central University)
Speaker 1: Chi-Hsiang Liu (National Central University)
  • Promotion, Relative and Individual Performance Pay
Speaker 2: Kohei Kawamura (University of Edinburgh)
  • Eliciting Information from a Large Population
Comments: Both talks showed frontier works on contract theory. The first one is considerable extension of promotion model and the second is to combine the ideas of cheap talk and (sequential) Bayesian learning. The interpretation and potential applications of their results were discussed.

Third session: 15:20―17:10
Chair: Pao-Chih Cheng (National Central University)
Speaker 1: Hsiang-Ling Shen (National Taiwan University)
  • Sponsored Search Auctions under Two Search Engines
Speaker 2: Takanori Adachi (Nagoya University)
  • Political Accountability, Electoral Control and Media Bias (with Yoichi Hizen)
Comments: Two speakers talked about their preliminary works which are both very new and strongly connected to real life interests: sponsored search auctions and media bias. Since some parts of their models were not well shaped yet, ideas for further improvement were discussed and proposed.


Lecture 4 (Dutta): Dynamic Programming 3

Original article (link) posted: 06/10/2005

In the last class, we checked the strong relationship between the dynamic optimization problem and the functional equation. In this class, we examined continuity, monotonicity and concavity of the value function.

Assumptions (for continuity)
(A1) State Space: Borel subset of a metric space
(A2) Action Space: Metric space
(A3) Reward Function: Bounded, measurable and continuous
(A4) Transition Function: Weakly continuous
(A5) Feasibility Correspondence: continuous

Theorem (Maitra 68)
Under (A1)-(A5), the value function V is bounded and continuous function. Furthermore, there is a stationary Markovian policy, h, that is optimal.
Step1 Bellman operator T maps bounded and continuous function (denoted by C(S)) back into the same space.
Step2 C(S) is a complete metric space.
Step3 T is a contraction.
Step4 There is a measurable selection h from the Bellman equation.
Note) To prove Step1, we use the Maximum Theorem. For the other part of the proof, we rely on the established results.

Assumptions (for monotonicity)
(B1) State Space: A subset of R^n
(B3)=(A3) + increasing on S (State Space)
(B4)=(A4) + First Order Stochastically Increases on S
(B5)=(A5) + A higher state has a larger feasible set

Theorem (Monotonicity)
Under (B1)-(B5), the value function is bounded, continuous and increasing on S.

Note) Step2-4 are almost same as before. So, we essentially need to check only Step1.

Theorem (Strict Monotonicity)
If we have a strictly increasing reward function, then V is also strictly increasing.
It is easy to see from the definition of the functional equation.
You should not try to restrict the space to the set of strictly increasing functions, because it is not complete. (the limit may not be a strictly increasing function)

Assumptions (Concavity)
(C1)=(B1) + S is convex
(C2) Action Space: A subset of R^m and convex
(C3)=(B3) + reward function is concave
(C4)=(A4) + transition function is "concave"
(concavity here is defined in terms of Second Order Stochastic Dominance)
(C5)=(B5) + graph is convex

Theorem (Concavity)
Under (C1)-(C5), the value function is bounded, continuous, and concave.
With strict concavity of the reward function, V is bounded, continuous, and strictly concave. In this case, h becomes a continuous function.


Theory Seminar (Sigurdsson)

Original article (link) posted: 05/10/2005

Sigurdsson (2005) "Auctions as Mechanism: An Application to Bankruptcy Reorganization" Job Market Paper

In the paper, he proposes the new mechanism of reorganization of firms in cases of bankruptcy. As a matter of fact, a firm that files for bankruptcy must either liquidate under Chapter 7 of the Bankruptcy Code or reorganize under Chapter 11. The distribution follows the absolute value priority rule (APR) which states that no creditor shall receive any value until all claims senior to his have been paid in full. Legal scholars have proposed several mechanisms as alternatives to the current system of judicially supervised bargaining, widely believed to be costly, lengthy, and to result in inefficient capital structures and violations of the APR. A major drawback of those proposals is their reliance on cash payments.
In the mechanism he proposes, the entire reorganized firm is sold in a cash auction and the proceeds are distributed to creditors according to the APR. He mentions 4 advantages of his mechanism. That is, the mechanism
1) implements the APR for far more general capital structures than the simple debt and equity structures assumed in previous mechanisms.
2) all but eliminates the need for cash payments and therefore works under tight financial constraints.
3) offers the advantage of familiarity over its more novel competitors in an auction.
4) allocates ownership efficiently when creditors do not agree on the firm's value.

It seemed that the participants of the seminar liked his paper. Hope he will get a good job!!
By the way, his main advisor is Eric Maskin. I heard a rumor that he did not take a student, but it should be wrong. I would like to talk to him about my research too.


Auction theory: Myerson and mechanism design approach

Continued from the previous two articles, we still focus on Introduction of the auction book:

After referring early literature on auction theory, the authors introduce mechanism design theory and explain Myerson's path-breaking contribution.

The early work of Vickrey, Wilson, and Milgrom was largely focused on an equilibrium analysis and comparison of standard auction formats. Myerson led the development of mechanism design theory, which enables the researcher to characterize equilibrium outcomes of all auction mechanisms, and identify optimal mechanisms - those mechanisms that maximize some objective, such as revenues. His first application was to auctions.
Myerson (1981) determined the revenue-maximizing auction with risk-neutral bidders and independent private information. He also proved a general revenue equivalence theorem that says that revenues depend fundamentally on how the items are assigned - any two auction formats that lead to the same assignment of the items yield the same revenues to the seller.
The trick in Myerson's analysis was recognizing that any auction can be represented as a direct mechanism in which bidders simultaneously report their private information and then the mechanism determines assignments and payments based on the vector of reports. For any equilibrium of any auction game, there is an equivalent direct mechanism in which bidders truthfully report types and agree to participate. Hence, without loss of generality we can look at incentive compatible and individually rational mechanisms to understand properties of all auction games. Incentive compatibility respects the fact that the bidders have private information about their values; individual rationality respects the bidders voluntary participation decision. This key idea is known as the revelation principle (Myerson, 1979).
The following is a brief summary of other related papers.

Myerson and Satterthwaite (1983)
  • applied revelation principle
  • proved the general impossibility of efficient bargaining
Related Papers
  • Cramton, GIbbons, and Klemperer (1987) showed that efficiency becomes possible when the traders jointly own the items
  • Wilson (1993) also showed efficient mechanisms when the roles of buyers and sellers are not fixed ex ante

Cramton, Gibbons, and Klemperer, "Dissolving a Partnership Efficiently," Econometrica, 1987.
Myerson, "Incentive Compatibility and the Bargaining Problem," Econometrica, 1979.
Myerson, "Optimal Auction Design," Mathmatics of Operations Research, 1981.
Myerson and Satterthwaite,  "Efficient Mechanism for Bilateral Trading," Journal of Economic Theory, 1983.
Wilson, "Design of Efficient Trading Procedures," in Friedman and Rust ed., The Double Auction Market: Institutions, Theories, and Evidence, Addison-Wesley Publishing Company, 1993.