The illusion of skill and the reality of an efficient market

You probably know the saying that the house always wins. This is true for games of chance like roulette or coin-flipping, where the probability of every possible outcome is known beforehand.

If you flip a coin multiple times, and each time it lands heads you win 0.97$ and every time it lands tails you lose 1$, over the long run, you are going to have a bad time. We know this, because we can assign a definite probability to the coin landing either heads or tails (50/50). Any profit obtained from a series of bets like this, is based on chance and nothing more.

However, in sports betting the true probabilities for every outcome aren’t knowable in the same sense as the probability of a coin landing on either heads or tails is. This opens the door to exploiting market inefficiencies, mistakes made by the bookkeepers trying to assess the true underlying probabilities.

It also opens the door to something called the illusion of skill.

For this blogpost, I have been following eight professional football tipsters from a popular betting site, pyckio.com, for one year. Before I started following them, they accumulated an impressive 6.7% yield in 11,233 bets altogether.

pyckio_1123yield

The main question though is if those results were due to chance or if their yield was due to the tipsters systematically exploiting market inefficiencies within the odds. Are we looking at exceptionally lucky or exceptionally skilled bettors?

Here are the individual betting records for every tipster before I started tracking their bets:

pyckio_tipsters_b5_hochkant

Okay. Now before we get to the results how these tipsters performed in the last year, I’d like you to think about what you expect their yields to be:

How many tipsters will have returned a profit? How do past and future yield correlate? What will the combined yield of all tipsters be?

Take a second to think about it, maybe write your answers down or make a note on your phone.

But before we get to the results, a few words about the dataset I used.

Pyckio and the tipsters

I downloaded the data directly from pyckio.com. Pyckio is a website, where you can place bets against real life odds and track your betting record.

They use a ranking algorithm, which tries to eliminate chance as a factor in the betting returns of every tipster as much as possible. You can read more about it on their site. If your ranking is above a certain level (4.25 out of 5), you are considered a PRO tipster, meaning others will have to pay you to see your tips.

On February 5th 2017 pyckio modified their ranking algorithm. Some PRO’s lost their status, some other tipsters gained the status of a PRO tipster.

The tipsters analysed for this article were all PROs as of that day. Pyckio has a very transparent policy and lets you download the bets their PRO tipsters have made. I kept downloading the betting records for a year and stopped following a tipster as soon as he lost his status as a PRO.

The eight PRO tipsters, however, were using different staking strategies. Some tipsters used on average 8-10 units per bet, some only 1 or 3. Aggregated to one betting record, this skews the overall yield towards the yields from the tipsters with the highest stakes.

To make the betting records comparable and to be able to add them all together to an overall betting record I divided the profits (or losses) by the stakes made for every bet. This way, every bet was treated as if only one unit per bet was staked.

Now, this added up to 11,233 bets made until (and including) February 5th 2017 with a yield of 6.7%, without any bias towards high staking tipsters.

You have already seen it, but here it is again:

pyckio_1123yield

In contrast to this, let’s look at the following table. It was posted a while ago by the official pyckio account on twitter and shows the aggregated yield of all their registered members:

pyckio_longshot

This underlines the favourite-longshot bias within the odds, but it also shows that on average, bettors don’t do much more than replicating chance. The aggregated yield is almost exactly the usual margin used by Pinnacle (all bets at pyckio are placed with Pinnacle odds).

This will have to be the benchmark for any professional tipster: outperforming random chance.

So, how did the PRO’s perform from February 6th 2017 until February 6th 2018?

 

The results

The aggregated yield of all tipsters for that year was -2.2% in 3687 bets. The tips any tipster continued to provide as a regular member after losing his status as a PRO, were not included in this sample. The tips we are looking at are only tips that had to be paid for.

pyckio_alltipsters_after5

This is very close to the margin used by Pinnacle. Overall, even the professional tipsters didn’t seem to be able to replicate much more than chance. Did the past yield of a tipster indicate anything at all? Let’s look at the correlation between each tipster’s yield:

pyckio_tipsters_tabelleKorrelation

The correlation is 0.00077. In other words: the past yield of a tipster, at least in this sample here, does not predict his future success at all.

This is how the graph looks like, attached to the previous betting record of all eight tipsters:

pyckiotips5

And here for every individual tipster:

pyckio_tipsters_after5_

You can check out the current profiles of all eight tipsters here: yanschen, magdalenatrip, icooperi, emarques, motaliz, winsports, elvizen, enriquechafer

 

So…are all tipsters trying to scam you?

No, I don’t think that any of these tipsters were actively trying to scam anyone. Especially pyckio seems to be a platform that is genuinely interested in differentiating in between what is luck and what is skill. They even made their data available to Joseph Buchdahl for his article Is Sports Betting Completely Random? where he analysed over one million bets placed by over 6,000 bettors, concluding that what bettors are doing is exactly what would happen by chance.

Imagine those bettors substituted by 6,000 cats. And let those cats place one million random bets. What Buchdahl’s findings show is that the betting returns would not differ in any way. A few cats would make huge profits, most cats however would face losses.

But what makes bettors believe that they personally can beat the sportsbooks at setting the odds, something that nobody seems to be able to do?

At the beginning of this blogpost I described a bet, where you flip a coin and if it lands heads, you win 0.97$ and if it lands tails you lose 1$.

This is an example taken from a blogpost Flipping coins and the importance of betting at the highest odds by @zorba138.

In it, he simulates how profits and losses might look like for 100 bettors over a series of 10,000 of these coin flips:

zorba

This graph is from zorba138’s blog. I highlighted the line where the bettors would break even (black).

6 out of the 100 simulated bettors would have made a profit, eventhough you can clearly see the overall edge the market has over these bettors.

A betting market without any inefficiencies still allows for some bettors to continuously make profits, even over a long period of time.

It is easy though, to look at a simulated profit curve and realize that its profits are only due to chance.

However, a lot of bettors, even though they may pay lip service to the concept of survivorship bias, fail to recognize it when evaluating their own betting record.

And with “a lot of bettors” I mean you and me, too.

 

The illusion of skill and how to (maybe) overcome it

The illusion of skill is the disconnect between knowing that there is something statistically no one can do and the feeling that you personally can do it.

Unlike a simulated profit curve or a cat placing random bets, actual betting contains a lot of tasks involving skill.

Bettors may build their own prediction model, distinguish between relevant and irrelevant news, do tactical analyses of the teams, interpret market movements etc., eventually leading to their decision which bet to place.

Performing these tasks gives any bettor the immediate feeling of understanding. This understanding is much more compelling than the abstract concept of survivorship bias.

In his article for the New York Times Don’t Blink! The Hazards of Confidence, Daniel Kahneman writes about the illusion of skill and how to distinguish overconfidence from true expert knowledge.

He writes that there are two questions you should ask yourself:

  • “Is the environment in which the judgment is made sufficiently regular to enable predictions from the available evidence?”

While Kahneman goes on to say the answer is “No.” for stock pickers, I think the answer to this question is probably “Yes.” for sports bettors. In football, there are only three possible outcomes and there are clearly defined rules to the game. Predictions are possible, and sportsbooks seem to be exceptionally good at it.

  • “Do the professionals have an adequate opportunity to learn the cues and the regularities?”

The answer to this question depends on the quality of feedback a bettor receives. Betting returns as a feedback, however, are terrible. As we saw earlier, the past yields of the PRO tipsters evaluated did not correlate at all with the yield the tipsters accomplished in the future.

In an earlier post, I simulated a perfect market without any inefficiencies for a sample of 170 games. I used the implied probabilities from the bookmakers to simulate those 170 games and let five public prediction models bet against the bookmakers’ odds. The results were that the perfect market only got identified as having the best probabilities in 20% of the simulations (In which case, none of the models would have returned a profit.)

This graph shows how often each prediction model was identified as the best model in percent, according to betting returns. As you can see, the feedback received for the predictions is almost completely random.

image1(1)

As mentioned earlier, the assumption in sports betting is that unlike in games of chance like roulette, profits are possible for some skilled bettors that can assess the real underlying probabilities of a game better than the sportsbooks are able to do.

Looking exclusively at profits returned from betting does not answer that question. It’s an incredibly hard question to answer. But still, if you are able to express your predictions in probabilities, it is possible to assess the quality of those predictions with proper scoring rules. Scoring rules measure the error in predictions. I wrote another post on how they work and why they are important here, if you want to know more.

Using the Ranked Probability Score on our 170 games sample had the perfect betting odds identified as the best out of the six models in 70% of the simulations.

Here are the results compared to using betting returns as feedback:

diagramme

Although this is done on a really small sample of 170 games, using a scoring rule to evaluate the predictions does a lot better than betting returns. I plan on doing another simulation like this at the end of the season with more games which I will link to here.

Still, in 30% of the simulations, the Ranked Probability Score resulted in false positives. So, if you decide to use a scoring rule to evaluate your betting record, you should be aware of that.

However, using scoring rules upgrades the feedback you get for your predictions from complete randomness to a strong indicator of quality.

 

 

If you have any questions or feedback, get in touch with me on twitter @fussbALEXperte

 

 

Advertisements

3 thoughts on “The illusion of skill and the reality of an efficient market

  1. Wow! Absolutly awesome!!! You answered the questions I was asking myself for a while. Thank you! Way to go!

    By the way I love your drawings and reimaginations of other brand images. Waiting for more!

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s