Search Results

Thursday, April 23, 2009

Don't Let Me Be Misunderstood

“I am not ashamed to confess I am ignorant of what I do not know.” - Marcus Tullius Cicero

You may have noticed Maxwell's comment on my last article, about the shortcomings of making predictions based on rather limited models. This post originally started as a comment addressing some of his points, but after considering it, I think there's a larger idea here that deserves a full post of its own. As always, I leave it to you, our dear readers, to be the final judges.



Your point about altruism is absolutely correct - and somewhat of a sore point between ('experimental') economists and psychologists. The argument, at least, from the rather biased point of view of the economist goes like this:

Economist: "If we model payoffs in X way, and we model actions in Y way, then, we can make prediction Z."

Psychologist: "Well, I tried that, and we don't get Z, we get Z'. Therefore, your theory must be wrong."

Economist: "Actually, you thought you were testing people with payoffs X and actions Y, but you were actually testing people with payoffs X', and actions Y'. If we alter our model, so that it's talking about X' and Y', we do get (at least something much closer to) Z'. In fact, here's an exact measure of Z-Z', and X-X'. So shut up."

Yes, it's a childish-sounding argument. But, unfortunately, this is often what happens when economists pretend to be psychologists and vice versa, and often in any situation where individuals from different backgrounds meet.

You're correct in saying that the Prisoners' Dilemma, when it just takes into account time served, give us 'bad' predictions. But, if instead of time served, we were able to come up with some measure of 'overall value,' that included time served, what type of facility the time was served in, whether you'll get shanked as punishment for snitching, loyalty to your partner, and so on.

In this case, assuming that we've properly included all the relevant costs and benefits, we would be perfectly justified in performing some calculation, and coming to some conclusion based only on the numbers.

Which brings me to the larger idea of this post: understanding what, exactly, economic theory and the assumptions it must always be based on, actually mean.

This goes along with the idea that Maxwell put forth a few months ago: that claims that 'science has been wrong before' are incorrect, and that a proper statement of the claim would be, 'science has made incorrect predictions before.' It's important to understand the difference between process and product, whether in relation to science, economics, government/political systems, or any other complex field of study.

The field of economics, like essentially all modern sciences, is based at least partly in mathematics. Math gives us an incredibly powerful set of tools to work with, and a convenient shorthand for talking about problems, but we have to be able to translate everything we do from 'economics' to 'mathematics' and back again. Thus, while math is one of our chief tools, it is not necessarily the 'basis' of the field, but merely a tool used by it.

Economics is based, then, on an axiom and a definition.

First, we assert that, given a set of options - where this set can be almost anything imaginable - individuals will choose certain of those options. That is, for each individual, given a set of options, there is a set of choices they will make.

Most commonly, we think of this in terms of 'bundles of goods.' When you go to the supermarket, there are 'bundles' such as (+12 eggs, -$1.35), (+1 gallon skim milk, -$1.89), (+nothing, -nothing), and so on. Given these options, which may include things like the time of day, how hungry you are, and anything else that may or may not influence your decisions, you make certain choices.

Secondly, we define 'preferences' as some function that takes in a set of 'options' and spits out a set of 'choices.' Thus, we claim that there is some 'mapping' from (all the possible bundles of goods, which include all the relevant information about everything in the state of the world) to (some subset of that set) that describes how you actually behave, and we call this your 'preferences' (or, your 'preference relation,' and so on).

So that's it. Essentially all of economics consists of some 'model' of the above, which we then use to try and answer some question we consider interesting about the world.

How does this relate to the 'translation' to math mentioned earlier? Well, it turns out that, under certain conditions, we can come up with a mathematical representation of preferences as a a normal mathematical function that takes a bunch of numbers representing an option (X) and gives us a number that, in some sense, tells us 'how much you like X.'

This is, of course, the infamous concept of 'utility' - the idea that, inside each of us, there's a part of our brain that's constantly evaluating our options and plotting them on a line somewhere, and directing us to simply grab the highest one. This has, over the history of the field of economics, led to a number of arguments, much confusion, and conclusions that, while technically correct, are often useless.

The point is that these numbers, our 'utility,' don't really mean anything in any real sense. They are merely a mathematical convention, a container that we must put our ideas in before we can use the tools provided by mathematics. Of course, once we 'do math,' we have to translate the results back into 'economics' - unpacking the container, if you will.

Let's return to the argument between the Economist and the Psychologist again. The economist, in his model, says that if individuals have actions X, and preferences Y, and the 'game' they play is set up according to a set of rules, G, then we should see them do Z. Often, these games are designed to be like the Prisoners' Dilemma - we actually try to find places where, given a model that seems reasonable, the conclusions we come to don't match what we see in the real world. From this, we find ways to change and grow our models: is there a problem with what we included in X, Y or G? Is there something about the way we're coming up with Z that's wrong? Are we actually seeing Z, but not recognizing it, because we were expecting it in a different shape/form?

This type of thinking leads us to be able to say more useful things then just, 'The model (X, Y, G, Z) is wrong.' One way to do this is to ask, 'how wrong is the model?' If we can come up with a rule for saying when it's 'really wrong' versus when it's 'not too wrong,' we might be able to figure out where the problems are. Maybe Z is very different from Z', but it turns out if we change X a little bit, we get something very close to Z'.

Which is, essentially, what the Economist and Psychologist are arguing about. The Psychologist says, "Well, there's obviously something wrong here," and the Economist responds, "Yeah, I noticed. Here, tweak this knob, and see if it works."


To wrap up, it's important to understand that there's a difference between process and product. The prediction that 'people will all throw litter on the ground' is correct, if we've set up our model properly. The concepts that this conclusion is based on are axiomatic - they are essentially true by definition. The fact that the prediction differs from reality is important - it's crucial, in fact, if we ever want to be able to say anything 'useful' in the sense of answering interesting questions about the world.

- Tiro

PS - My apologies to Maxwell - I really don't mean to sound like I'm disagreeing with you, or disregarding the issues you mentioned. You're absolutely right!


Maxwell Evans said...

that was kinda trippy.

I'll write an article responding soon, probably, but I'd like to point out something rq:

My earlier article "science has been wrong before too" wasn't taking issue with the "been wrong" it was taking issue with the use of the word "science."

You can't say "science has made incorrect predictions," either, because science doesn't make predictions. Science is a method used by scientists, and much like economists make predictions, not economics, scientists make predictions, not science.

However, I think your purpose of pulling on that article was essentially spot on - the difficulties (in the physical sciences or in economics) lie in the applications, more than with the methodology.

A scientist says "if X, then Z," and designs an experiment to simulate X (or X and Y) as rigorously as possible.

The difficulty with economics (it seems to me), and where it gets into trouble with philosophers, is that designing a rigorous experiment to test X in economics is so difficult that you almost always end up simulating X' instead, which generates the result Z', which gets the philosopher on your back.

Marcus Tullius Tiro said...

Precisely. And almost always, X' ends up being some simplified version of X.

Most of the situations we study are so mind-bendingly complex, that we can't actually reproduce them - or if we could, there wouldn't actually be a lot of value, in that we'd spend all our time modeling, and none of it predicting, etc.

Interestingly, one of the most popular methods of dealing with this is to come up with some 'model,' derive some mathematical conclusions about the model, and then try to see if the results are borne out in the 'real world.'

It's like one of my advisers once told me - I was concerned about a particular decision-making problem we were studying, and how the methods we were using to derive conclusions were nothing like what happens when 'real people' make such decisions. He told me that it doesn't matter: all that matters is whether or not we end up with a 'good model' - one that yields the same results as reality - whether or not the model 'looks like' the real process. It doesn't matter, for example, that people don't solve a bunch of simultaneous differential equations when playing pool, if a model that does gives us good predictions about how they'll play.