Beating the Crowd

Suppose you are participating in an auction for a box. Nobody knows exactly what is in the box, but they are bidding on it for various reasons; some think they can make money, some might have sentimental reasons, and others might just be having fun. You yourself are trying to make money. You asked around a bit, did some research, and think that you have a good estimate for the value of the box. You think it’s worth $10,000 and decide to bid $9,000.

You won.

How do you feel? Great? You made money (on average, you think)! But thinking about it a little more gives you pause. The rest of the world bid for this box and your bid beat all of them. Every single person who did their own research, and presumably thought just as hard as you, decided to bid less. Could it really be that you know more than all of them? You open the box, and find a $100 bill inside.

It’s a bit of a contrived example, but it illustrates the point nonetheless. You might have had a reasonable confidence interval in isolation. But the very fact that you won the bid, that you were more aggressive than the rest of the world, shifts the probability of your decision actually being good vastly against you. Professional traders think very hard about this. It’s why firms like CitSec pay for order flow from brokers like Robinhood.¹ It’s very simple: if you had to win an auction, you should want to win it against a random retail “trader” rather than a professional hedge fund.

The question, then, is whether the world is usually correct with their hunches. Statistician Francis Galton tried to find out through a well-known experiment. He asked a group of people to independently guess the weight of a displayed ox. The average guess turned out to be within a couple of pounds of the correct weight. This seems surprising, but is logically consistent: in the absence of other information or opinions, one would have to assume the average human guesses any quantity correctly. To say anything else implies an opinion that the mean human would cognitively overestimate or underestimate the quantity, and this is equivalent to an opinion on human cognitive bias for this specific problem. Like anything, there are exceptions everywhere; my claim is merely that in isolation of any other opinion, a crowd’s guess of a quantity/decision tends to be pretty good.

These two concepts are very powerful even beyond isolated price discovery settings, and you can find applications everywhere. Off the top of my head, here are some examples ranked in severity.²

• You are in line at Disneyland for a ride. There’s an announcement: the ride is temporarily out of commission, but the park expects it to be operational again “very soon.” You don’t mind waiting a bit, but in front of you people groan and start to leave the line. How long will the wait be?
• You’ve been trying to get a reservation at a hot new restaurant for a while, but never could. But today, you finally managed to snag a 7 pm reservation on a Friday night. Did you just get lucky, or is the restaurant less popular for some reason?
• You accept a prestigious job at a company out of undergrad. A competitive tech or trading firm, perhaps. You think to yourself, you were in the top 0.1% that beat the other 99.9% out of the candidate pool. Another way to look at it is that, out of the group the company was willing to hire, you are probably near the bottom - i.e. you could not find anything better.
• You start dating someone very attractive. The most attractive you’ve ever had a chance with. Why are they in the relationship?

When can you go against the crowd, and be a winner without the curse? There are several dynamics that can give you good reason to do this. I’ve listed out the top three.

The first, and the most common dynamic, is one that is decided by an informational edge. In the Disneyland example, your reaction should change depending on how much you know about the park itself. If you are an annual passholder and you’ve been to this ride many times, and know that delays often get resolved quickly, you would be reasonable to stay. If you have never been to the park and do not know anything about it, it may be better to follow the wisdom of the crowd and assume that the market is being reasonable when deciding that waiting is not worth it. Either way, your action is grounded in your own prior knowledge but ultimately depends on whether you think you know more than others.

The second is defined by the difference in each party’s willingness. Let’s pick the restaurant example. If the night of the reservation is Christmas, a hypothesis is that the average person is more likely to be at home with family. They are not willing to eat out at the restaurant for a reason that you are aware of. Your preferences are different. Though this seems to be closely tied to an informational edge, the difference is that every party has the same amount of information. It’s just that due to their personal preference, they choose not to take the same decision as you.

This leads us to the third exception: differing situations. The dating and marriage market is the quintessential illustration of this. Why is your significant other dating you? One important aspect is that there’s some inertia: no one else has the specific set of interactions you’ve shared and the (presumably) demonstrated emotional and physical connection. Both parties have made the decision that it’s not worth throwing the relationship away to take a gamble in the dating market again. Optimally stopped. This works because like all things, this is not an isolated auction setting. There definitely exists someone out there that is better than you in all aspects; but they are not you. They don’t have the same experiences, the same history, nor the same circumstances to be able to make the same romantic trade you are making.

So what can we do?

An important thing to keep in mind is that all of these listed exceptions and more intertwine with each other. They’re always pushing and tugging on your decisions, but lead to the same result: decisions should never be made in isolation. In practice I keep a two-step checklist: make the decision in a vacuum, then color that decision in the light of what others are doing. Every time I skip step two, I end up regretting it.