The Ref Was Right, How to Talk About Probability in March Madness, also in Average April, May Too…
The sun comes up in the east and sets in the west, you may have different words an for these things and ascribe them different meanings but the underlying physical thing is extremely consistent. In the world of null hypothesis testing, you should be completely comfortable rejecting the null. Coin flips are interesting, they are wobbly. I once had a large room of people flip coins, after the aggregated first round of flips (>500) we had a definite preference for heads, the second round had a less intense preference toward tails, surely after another round we would have settled in around 50/50, but with some wobbliness.
In many ways it does a great disservice to teach probability from a stochastic basis. For the most part, there are no coin flip models in life or research, but there are also not many interesting models like the sun either. Proving that fire is hot is not a particularly interesting insight. Yep, it is hot. The interesting stuff comes in degrees of belief for events that are neither random nor determined. This is the heart of the Bayesian turn which animates everything from chess rankings to political polling. Basically, what if instead of thinking of the world as a bunch of coin-flips, what happens when we take our priors seriously and change them with care?
Occam’s Razor would suggest that the explanation with the smallest number of assumptions is most likely correct. At the same time, there is the problem of heuristic availability — that you may simply lack the framework for understanding what is happening and thus might accept an explanation that is seemingly simple but utterly wrong because you can’t see all of your ridiculous assumptions. Conspiracy theories take this to the next level by making your bad, simplistic model pleasurable.
Consider hard core anti-democracy activists committed to the idea of a stolen 2020 election: a universal perfect organization of leftists have orchestrated an undetectable program. Without even getting to the question of undetectability, how on earth could an organization of tens of thousands, if not hundreds of thousands of people from all backgrounds keep such a secret? Let’s say we have 10 people, and each has an 80% of keeping a secret. There is a non-trivial chance that they could pull it off, around 10%. If you try to calculate this with your handy MacOS calculator for 1000 people the result is “not a number.” Similarly, finding a cache of ultra rare merchandise in a storage locker should have tipped the reporter off to another whopper long before the subject of the story messed up with caller id. In a wonderful, long, video about speed running a video game, without overtly saying that a speed runner cheated, the mathematician arrived at the idea that to get such odds from random number generators one would need to be the luckiest human ever, by some substantial margin.
Just because there is some asymptotically low probability that something could be true doesn’t mean that we must act as if it is true. This is among the greatest dangers for the turn toward speculative journalism — treating near zero probability outcomes has the chance to make them more likely. False balance often requires the abuse of probability to fit a coin-flip like model of both sides. In the case of all three stories: perfecting a conspiracy, chancing upon a pot of gold, or hitting a hot streak on a gaming table, when a probability is near zero we should be willing to act as if it is not true. Why does academic writing get so weird? If you are required to never reach a conclusion because some incredibly unlikely event may have some alternative explanation your writing will be warped. Consider the case of a digital non-linear video editor — loading all the clips into a digital computer changed how television was made and how it looks. While it was a perfectly reasonable point in academic world to accuse this insight of the dreaded “technological determinism” it isn’t particularly helpful. Surely an industrial factor with thousands of optical printers and scissors+tape specialists could have made a real time non-linear analog editor a physical possibility but it would be simply ridiculous.
Letting it Ride
Next week the annual basketball festival will begin. Basketball games have referees. Much to the consternation of fans, at the end of big games refs “swallow their whistles.” Frustrated fans bluster that the game should be called the same during every second like a civil law system where the textual interpretation of a foul should always produce a whistle. Unwatchable basketball would be the result of universal enforcement. Worse if enforcement were mechanical the game would devolve into a series of gestures to produce fouls. If you like Chris Paul initiating contact when his team is in the bonus to generate artificial free throws, do advocates for uniform officiating have a treat for you.
Think about the end of a big time college basketball game, the blue team is down three running down the floor to take one last circus shot to tie the game. The shooter pushes off at half court — a foul, but a foul that is almost never called. Why? Because the shot is highly unlikely to go in the basket. It is better aesthetically for the shot to go up, not for a ref to intervene. Aesthetically? What does beauty have to do with this? Games are a set of discourses around randomness. There is no reason why any of our games should exist in the first place. Uniformity can be ugly, at the same time, a complete lack of consistency is a real mess.
If the defender rocked the shooter at half court with a flagrant foul the call would not be made, a gentle push is not. In the early part of a game, when there are many intervening acts set to take place, refs might be more aggressive in calling because the probability of the game is still floating and there are good reasons to shape the game flow. This is why winners frequently point to the totality of the game rather than individual calls. If your team loses on a low-probability shot with glance of a foul you should consider why your team was playing defense near the hoop when the three was clearly going up, why the defender was allowed to catch the ball in stride, or why their Center was allowed to catch a long lob and hand-off for the shot. Your team had ten ways to stop this.
Refs swallow the whistle when the call would be ugly or dramatically effect the distribution of late game probability. Substantial research indicates that non-subjective calls do not decrease late in games. At the end of one Super Bowl, a team repeatedly threw the same out pass toward the front of the end zone, between five and six yards up. The defense was excellent, the chance of making the catch was near zero. If the ref had blown the whistle on contact at five yards and an inch the change from the prior probability would have been extreme. Making the call would have been the wrong choice. Sports fans have ways of thinking about math that allow them to respect prior probabilities.
Juxtaposing the ways that sports fans handle prior probability and the ways that journalists are made to abuse probability by obliterating priors helps us see what is really at stake here: it is good to know things. Conspiracy theories make bad assumptions pleasurable. Ill conceived media literacy curricula tell students to be smart to not believe anything. A sports fan will know which team is winning, you know when someone is telling you a monster fib about their good fortune. This is confirmed by experimental psychological research, people who know nothing are better than random, but not as good as experts or direct measurements of the team.
Epistemic humility is important, there are so many ways of thinking and so much of how we perform knowing is about appearing to be omniscient. Tetlock was extremely persuasive in Expert Political Judgement, that there is a strong negative correlation between extreme views and predictive accuracy. People who have extreme views do a bad job of integrating information. It gets better. Extreme outcome prediction is actually correlated with low predictive accuracy. Folks who pick black swans aren’t predicting at all, they just say the same extreme thing all the time.
There is a strong expectation that everyone will know everything, that our cognitive heuristics, like frequentest probability will hold. After all we have the google machine, how can you ever not know anything? Did you not allow the algorithm to help you educate yourself in what the algorithm thinks is important?
Sporting events are one of the only times when adults talk about math in a robust empirical way. All of the picking, in-game probabilities, and rankings are occasions for folks to learn better ways of thinking. Abandoning prior probability or choosing to replace priors with coin flips is not humility, it is corrosion.