St. Petersburg paradox

[gusl]

Imagine a game in which you:

* win $2 with probability 1/2
* win $4 with probability 1/4
* win $8 with probability 1/8
* win $16 with probability 1/16
* win $32 with probability 1/32
and so on, ad infinitum.

Assume also that your payoff will be determined and paid up instantly.

How much would you pay to play this game?

The expected value of the game is
1/2 * 2 + 1/4 * 4 + 1/8 * 8 + ... =
1 + 1 + 1 + ... = infinity

St. Petersburg paradox

A naive decision theory using only this expected value would therefore suggest that any fee, no matter how high, would be worth paying for this opportunity. In practice, no reasonable person would pay more than a few dollars to enter. This seemingly paradoxical difference led to the name St. Petersburg paradox. [huh? why this name? -GL]

And I think this behavior is perfectly rational.

Your answer to the question above is a measure of your risk-aversity. But we should also investigate the effects of a lower-payoff vs a higher payoff game.

A game paying twice as much as the above game:
* win $4 with probability 1/2
* win $8 with probability 1/4
* ...
is obviously a better game to play (almost twice as good), and yet, the expected value of the game is also infinity.

My solution would be to make a transformation on the probabilities that makes the sum converge. The idea is that we have a "horizon", and very remote probabilities should count for less. But a 1/8 probability should count almost the same as a 1/2 probability. Any concrete solutions?

One problem is that you can always make a game with payoffs that make this sum diverge: just make the payoffs proportional to 1/p. This is not too problematic however: IMHO, the real problem is unbounded utility. Founding utility on human happiness is a good way out (I think).

Comments

[marknau.livejournal.com]

You don't need to link utility with happiness. You just need to posit that utility doesn't necessarily scale linearly with increased funds.

That, plus the fact that the game is not actually going to pay out anything beyond the 18th term or less.

[gustavolacerda.livejournal.com]

You're right, of course. Linking it with happiness is like using an atom bomb to kill a fly.

I've been thinking that maybe median utility is a better thing to optimize than expected utility in one-shot games... but that is obviously problematic too.

[easwaran.livejournal.com]

Yes - it tells you not to take a bet with a 1/3 chance of paying $1,000,000 but a 2/3 chance of costing $1.

[spoonless.livejournal.com]

Any concrete solutions?

We had a good discussion on this in mathematics a few years ago, but it would be pretty tough to try and find the thread at this point.

You're in the right direction when you mention founding utility on human happiness. The issue here is how huge the sums of money you need to include with extremely unlikely probability are so ridiculously large that they couldn't possibly matter. From a utility standpoint, a 100-billion dollars doesn't matter a million times more than a $100,000... because the sum is so large that we couldn't possibly make use of it to give us that much more happiness. And in practice, nobody could ever offer us the chance to play this game since there is always going to be a cutoff to the largest amount they could pay out. Even if the utility were exactly proportionate to the monitary value of huge sums, you would still have to set the cutoff at, say a few trillion dollars, or at the total amount of money in the world. If you set a cutoff around here, then you can caluclate exactly how much it would be worth it for you to play... and it's actually not worth that much. (For instance, if the cutoff were 1-trillion dollars, then you should only be willing to pay $40 to play since 2^40 = 1trillion.) If the person offering you the game doesn't convince you they have more than that to pay you, then it isn't worth $40.

[en-ki.livejournal.com]

"Expectation" is pretty clearly a poor word for "probability-weighted mean" when you're looking at only a few trials, and I've always considered utility to be the core solution to the problem. After all, why would anyone ever buy lottery tickets? (Because there is utility even if you don't win: the dream of winning, and the excitement of the moment of decision.)

Consider also, though, that human beings will make this sort of calculation using approximate math, not exact math, and so it's quite likely that probabilities less than maybe 1% are ignorable, and the human idea of the expectation is really just $6 or $7. If I were to think about how I would make that decision, ignoring the mathematician in my brain, that's how I would do it.

[gustavolacerda.livejournal.com]

After staring at it and thinking about it, I'd probably pay $4 or $5 for a one-shot of the first game.

[spoonless.livejournal.com]

I think it would be worth around $25 to me, assuming I knew the person could pay me an infinite amount of money. I don't see how it being one-shot or as-many-as-you-want-shot could make any difference. Expected value, IMO, is always the best decision, as long as you take the utility of the money into account (and the cost of doing business, thrill of the risk, and a couple other technicalities.).

Out of curiousity, would you be willing to play the game with me on the opposite side? I'm willing to pay you $8 if you're willing to flip a coin 10 times, and pay me $1024 if it comes up heads all ten, $512 if it's only (the first) 9 in a row, etc. If you'd only pay $5 for the infinite game, then surely you would accept an extra $3 from me to play a game even more in your favor, eh?

[gustavolacerda.livejournal.com]

In multiple-shot games, unlikely events no longer become so unlikely.

Why would you not pay me $9? Is the game worth $10 to you?

[easwaran.livejournal.com]

I've been thinking about this stuff - just gave a talk about some of it in Canberra last week in fact. I'll probably post about it soon.

The problem with bounded utility is that there aren't really any good arguments that there is such an upper bound. And it would be somewhat odd for there to be such a bound - what's the ratio of this utility to the utility of $100?

Actually, if you really can get a game with St. Petersburg payout in terms of utility, I think you should take it too.

There is some sense in which miniscule probabilities really seem like they should be basically ignored. But it's really unclear what that sense is.

[gustavolacerda.livejournal.com]

Actually, if you really can get a game with St. Petersburg payout in terms of utility, I think you should take it too.

hm... I think you should have used the counterfactual mood. The idea of infinite utility is absurd to me.

[easwaran.livejournal.com]

There's no infinite utility involved - just finite utilities in a certain distribution. No matter what the outcome of the game, only a finite amount of utility is achieved. It has to be unbounded of course.

Is unbounded utility really absurd? It seems fairly implausible, but I think I'd need an argument that it's absurd. Because it also seems implausible that there's some finite amount that is the upper bound. (Even if it's not achieved.)

[gustavolacerda.livejournal.com]

Unbounded utility is biologically impossible. While I can't give you a bound myself, I claim that they exist: just like the laws of biology constrain how fast an athlete can run (there must be bounds based on the properties of muscles, energy output, aerodynamics, etc), so do the laws of neurology constrain how happy one can be (never mind that happiness can't be directly observed).

In any case, I'm a descriptivist, so the concept of utility is only useful as a chunk for the purposes of explaining behavior at a high level. My escape is to say that the notion of utility breaks down: it only exists in the mind of the observer (and some rational agents, at the object level). People don't always do what makes them happy.

Am I being coherent?

[easwaran.livejournal.com]

That's plausible that there's a biological limitation. I'm just saying that it's not plausible that there's a conceptual limitation.

Also, I'm not convinced that expected utility is a good enough description of behavior to be a descriptivist about it. I'd rather be normative about it.

What you say sounds coherent to me, but it shouldn't make you think that unbounded utility is absurd, any more than a person traveling at unbounded speed is absurd. (Pre-relativity, at least.)

[(Anonymous)]

I think your comments are ill-conceived, Gustavo. Have you read any of the relevant literature?

[gustavolacerda.livejournal.com]

who is this?

[(Anonymous)]

I would prefer not to disclose the name others use to address me. Of course, but perhaps of little use to you, I will disclose the name I call myself: me. Kidding aside, I suggest that you read the relevant literature before you make wild, so-called "intuitive" claims.

[gustavolacerda.livejournal.com]

Since I don't know who you are, the burden is on you to show me that you are worth my time.
You could do this by, e.g. saying something substantial, rather than making handwaving criticisms.

[(Anonymous)]

I was unaware that your time is worth anything. Indeed, I do not intend to show you I am worth your time. I moreover don't intend to associate myself with your quibbling discussions. I believe what I wrote earlier corresponds to the fact of the matter, and I stand by it. Please take my criticisms seriously, accordingly writing something substantive.

[gustavolacerda.livejournal.com]

Can you point me to any relevant literature? Until then, what is wrong with speculating on my blog?

What good does it do to say "I think your comments are ill-conceived. Go read!"? Would you expect anyone to say "Oh yeah, what was I thinking?! Thank you very much!"

And why would you prefer to not identify yourself?

[(Anonymous)]

The reason I prefer not to identify myself is that I anticipated that you would react hostilely. Nothing is wrong with speculation, but you are hardly speculating. As I indicated, you are quibbling. I would be happy to point you to reading materials that would inform your discussions, provided I can be assured that you have taken my criticisms seriously.

[gustavolacerda.livejournal.com]

Why should I react hostilely? Have I shown any evidence of being intolerant to criticism in the past?

http://www.ask.com/reference/dictionary/ahdict/38480/quibble

quibble
intransitive verb: -bled, -bling, -bles.

1. To evade the truth or importance of an issue by raising trivial distinctions and objections.
2. To find fault or criticize for petty reasons; cavil.

I am sincerely curious about where I'm quibbling in this post.

I'm afraid I can't yet pass any judgement on your criticisms, since all you've said so far was "you are wrong!".

[(Anonymous)]

Look, I am done here. Stop responding. It's good to see that you have familiarized yourself with the definition of quibble. With practice, perhaps you will learn to apply the definition when you review your posts. I wish you all the best. Don't fall asleep on your armchair.

[twinofmunin.livejournal.com]

it's his journal, troll. if it bothers you, don't read it. your comments so far have all been worthless and your wording tastes of that of someone who is trying too hard.

[gustavolacerda.livejournal.com]

I'd like to see more of your talks online, btw... Are you into that "logical learning" stuff of Fitelson's, as an alternative to logical omniscience? I think this would blend quite well with cognitive modeling.

[easwaran.livejournal.com]

I'll post a link when I finally do.

Anyway, I'm interested in some sort of solution of logical learning. Branden's stuff is a good approximation to model learning of specific pieces of logical information, but it still requires omniscience about a lot of logic (everything built up truth-functionally from the uninterpreted sentences).

[gustavolacerda.livejournal.com]

I would like to compare these different solutions for logical omniscience to the ultimate solution: cognitive models, as in ACT-R.

[marknau.livejournal.com]

Just for kicks, I made the following model:
1) Assume the payer will not actually pay more than 2^23
2) Assume total utility is equal to sqrt(present value of lifetime earnings)

Then the proper price for the first game is about $10-$11 for most reasonable values of PVLE.

[gustavolacerda.livejournal.com]

If you eliminate the first assumption, I think you'll still get approximately the same thing.

Making the utilities increase as sqrt(2^n) = 2^(n/2) (instead of increasing as 2^n) is enough to make the expected utility converge, isn't it?

[marknau.livejournal.com]

Yes, I agree.

I made the first assumption because I was doing it arithmetically rather than bothering to solve an infinite series.