The objective, the conservative, the hard nosed.  All these labels usually confer opinions of soundness of judgment and critical thinking upon those who bear them.

But, what if the subjective is more powerful and often more right than monolithic objective truth?

There is a field of statistics called Bayesian statistics deemed to be subjective by many. Most financial statistics and risk analysis uses the frequentist approach. Frequentists use historical data poured into a model to generate a probability distribution. This probability distribution is then assumed to reflect not only the past, but also the future probabilities.

The way we measure risk in finance with forms of price volatility is deeply flawed. Risk is assumed to be an absolute number or factor existing out there waiting to be found like Planck’s constant.  This pursuit or faith in an absolute numerical representation of risk. Finance and economics are ultimately social activities. There are no absolute truths for a given point in time.

Battle of the Frequentists and the Bayesians

The counter approach to frequentists is Bayesian inference. An important book about Bayesian history has come out. Its importance isn’t in pushing the science of Bayesian statistics forward, but rather in explaining the power of an idea so controversial and yet so useful it couldn’t be killed.

The Theory That Would Not Die by Sharon Bertsch McGrayne tells the tale of how Bayes theorem has quietly changed the world by solving some of the most vexing problems.  McGrayne’s research is only surpassed by her ability to tell an excellent story. Some of the amazing stories involving Bayesian analysis include:

• How Bayes helped win D-Day and may have shortened WWII by 2 years.

• The RAND think tank predicted the chances of a nuclear bomb accident with a sample size of zero in 1945.

• How Bayes theory found a Russian sub in the real Hunt for Red October in 1968?

• How the navy started looking for its lost nuclear bomb off the coast of Spain in 1966. The bomb was found 260 yards away from the Bayesian guess. The original search field was 140 square miles of ocean.

• Why and how was Bayes theory was almost killed by academics in the US?

• Hints that the multi-billion dollar Renaissance Technology Hedge fund is filled with Bayesians.

The power of Bayesian analysis should resonate with any financial professional. Here is a quote from Madansky a researcher challenged with finding the future rate of nuclear accidents when the sample size was zero:

“if you are willing to admit a shred of disbelief, you can let Bayes theorem work…Bayes is the only other theology that you can go to. It’s just sort of natural for this particular problem. At least that’s how I felt back then.”

Please note the term theology above. For statisticians there are 2 strong camps the frequentists and the Bayesian’s. These belief systems are at strong odds with one another. Almost all financial risk methods are based on frequentist approaches. Moody’s had the CDO pricing model that assumed all house prices in the US couldn’t go down because they never had.

Dead Turkeys are frequentists

The frequentist approach would fall foul of Taleb’s turkey model in which a turkey is fed and cared for 999 days. The frequentist turkey never predicts he will be at dinner on day 1,000 as the prior samples allow for no such possibility or event.

Bayesian analysis allows for counterfactuals, speculation and conjecture. This is a good thing for truth can be stranger than and past experience. How many 2009 risk models had the US losing its AAA status built into them as a scenario in the foreseeable future (5-7 years)?

The work using Bayes inferences and scenarios from the RAND institute went on to influence the design of safety protocols for the US nuclear arsenal. I like the idea of a tool like Bayesian inference being used for safety design. My hope is that such thought would go into designing rules and regulations for economic systems. As it is the thinking and thought behind banking rules is often crude or overly frequentist.

The statistician Tukey also makes an appearance in the fight to keep Bayes theorem alive and applied.  At one point he articulates the value of subjectivity and multiple views of truth, something finance and risk management need to acknowledge as worthwhile and valid.

Tukey explains calling objectivity an “heirloom” and “a fallacy… Engineers are not expected to design identical bridges or aircraft. Why should statisticians be expected to reach identical results from examinations of the same set of data?”

If you manage money, risk or have to make predictions, I would strongly suggest reading this gripping book, The Theory That Would Not Die. After completing it, if you are like me you will run to your browser to find Bayesian tools and tips. Your objective view of the world may never look the same again.