SyStemS thinking

hi al t to i uar hen act gw in ng: nKi

ow Kn

ft le id vo a ns ur t

Kno

ef id l avo rns tu
win

he gw

n to

the SerieS finale on thinking anD DeciSion making, thiS article exploreS when actuarieS ShoulD uSe a heuriStic or SyStemS analySiS methoD of DeciSion making. By neil cantle anD Dave ingram

iD you know that FedEx drivers rarely make left turns? Both UPS and FedEx announced back in 2007 that they found that minimizing left turns from their delivery drivers routes actually saved time and money. You can imagine what they had to go through to make that decision. First, someone had to have the outside-the-box idea, then they studied traffic flow and accident information and then someone probably built a model to simulate a large number of deliveries with and without left turns. Finally, they took that research and turned it into a simple bit of standard operating procedure. There were four steps to that process. First, someone noticed that there was a problem with routes with left turns. Second, they studied the system of traffic flows and accidents and spotted patterns that suggested an underlying mechanism at work. Third, they built a statistical simulation model of the problem. Fourth, they turned what they learned into a simple rule of thumb or heuristic. The best actuarial work will usually go through a similar pattern. The actuarial control cycle was developed to ensure that actuaries remember to challenge their models as new data becomes available. Good actuaries are respected for their deep knowledge of how things work and their ability to model them. This merging of the real and statistical worlds to produce usable rules of thumb for non-actuaries is what makes good actuaries valuable. While the best actuaries may have long been wrapping their statistical models with deep insights up front and simplifying communications at the back end, researchers have been developing those processes into fields all their own. What actuaries now need to consider is whether we need to learn from their research and incorporate the findings into professional actuarial work.

June/July 2012

the actuary

25

StatiSticS are not enough It is hard to imagine running an insurance company without actuaries; however, many years ago people tried. But they struggled; at least some of them did, to understand the right amount to charge and how much they needed to hold onto to give reasonable assurance that they could pay out future claims and obligations. With the help of actuaries and actuarial methods, the answers to those questions about rates and, reserves became somewhat more routine and with those answers, some insurers have survived for hundreds of years. People also traded options and futures contracts for thousands of years without the benefit of statistics. Since the application of statistics to finance, the perceived safety of trading of derivatives and the volumes of such trades have increased by orders of magnitude. Statistics in general, and actuarial techniques in particular, have proven to be immensely powerful methods to find and exploit situations where either combining or separating risks can be advantageous. Statistics is all about data: collecting it, organizing it, analyzing it and interpreting it. In the hands of actuaries, studies of mortality, health, natural disasters, motor accidents, etc., have all yielded to statistical analyses permitting the individual risks to be seen as sufficiently predictable that companies are prepared to accept them for a suitable premium. Centuries of insurance business have been based upon the notion that risk events become somewhat predictable when viewed in sufficient numbers. A particular strength of statistical methods has been that people can focus on the outcomes of a process without

necessarily needing to know how it works. This, of course, makes studying the problem much simpler. Much of the world is highly complex, so statistical approaches have provided a way to make progress in understanding even where a detailed knowledge of the problems mechanics is absent. A mathematician might say that statistics can be used to describe the behavior of phenomena when we do not know the actual equation that defines that behavior.

Much of the criticism leveled at statistics comes from the fact that it applies tests to prove whether a hypothesis of the world is true or not. Depending upon the level of certainty required, there is a feeling that you can pretty much prove anything is true. Again, this largely comes back to the fact that statistics tend to make big predictions about outcomes but do not explain the mechanism by which they occur, giving people little substance to buy in to. So even with the benefit of actuarial- and statistics-based financial math, actuaries are always complainingcomplaining of lack of voice, lack of influence in the affairs of the firms that seem to fundamentally rely upon their expertise. In another part of the financial services world, many institutions placed very heavy reliance on financial quants, some of whom were literally rocket scientists wielding their statistical models. As we all know, this story did not end well. The rocket scientists and their models missed the big picture of the flawed mortgages deep inside of their incredibly complex structures because they were only looking at the outcomes and not what was really happening underneath.

neeD to unDerStanD The apparent success of statistical approaches has arguably focused a generation of practitioners on modeling outcomes and not everyone remembers the assumptions that are being made in applying statistical methods to a problem. Whilst studying the trend in outcomes is a powerful way to understand something, it does not provide insight into why the outcome is that. This lack of explanatory power can lead to the results being hard to understand and causing people to miss the fact that the model has ceased to be a suitable representation of the event being studied. This can be particularly dangerous when the models are making predictions about rare events so you cannot directly observe conflicting evidence to highlight that the model is wrong. The language of statistical analysis is beguilingly obvious and yet great care is actually needed to make sure the underlying assumptions are understood and that the messages from studying the model are actually interpreted correctly. One of the most common mistakes is that people assume correlated factors are actually linked in some way. In fact, correlation makes no statement at all about whether two variables are connected.

unDerStanDing why Other sciences found that complex behaviors and outcomes are not necessarily the result of complex rules and that the interactions between factors are crucially important to the understanding of such phenomena. This is somewhat at odds with the statistical world where outcomes are all that seem to matter. Applying a systems approach to understanding a problem requires the modeler to first appreciate the whole, before apply-

26

the actuary

June/July 2012

Kno

ef id l avo rns tu
win

he gw

n to

ing reductionist methods to replicate it. This discipline of remaining open-minded about what the driving processes are and then updating ones beliefs in the model is a good way to prevent the model itself being seen as unassailable truth. An unhealthy belief in models arguably contributed to the recent crisis and serves as a good reminder that understanding why the model is producing a particular outcome is a very important part of the modeling process. In two recent articles in The Actuary, we described two different approaches to thinking and decision making Heuristics (The Evolution of Thinking, Feb./ March 2012) and Systems Analysis (Systems Thinking, April/May 2012)that are, we believe, the answers to these seemingly opposite problems of underreliance and overreliance on quants. The approach that is fundamental to actuarial science and to quantitative finance is rooted in a new branch of mathematics, perhaps the youngest branch of that very old tree, statistics.

ics to stock markets while economists provided the rationale. But there are (at least) two major flaws to this advance in the scientific and mathematical approach to understanding the future. First, the bulk of humanity has not had the requisite amount of math training to understand

in schools. In the RDM process a decision maker will identify potential solutions to a problem and then evaluate the characteristics of the outcomes under those solutions to find the optimal choice. Herbert Simon (1957) identified a major flaw in the RDM approach. There is no natural lim-

SatiSficing meanS concluding the deciSion proceSS when a SatiSfactory outcome can be found from one deciSion alternative.
any of this. And, in that bulk resides most of the top executives of the very companies who must rely on the quantitative analysis. But while the quants were advancing their ability to do analyses of dizzyingly complex financial transactions, psychologists were studying how the bulk of humanity makes their decisions. In The Actuary article, The Evolution of Thinking, (Feb./March 2012), the work of Gerd Gigerenzer was cited explaining the Fast and Frugal Heuristic (FFH) as the best representation of how the other 99 percent think. With FFH, people will naturally form decision rules where they automatically evaluate thousands of possible clues and quickly isolate a few that are all that is needed to find a reasonable solution to most problems. Another psychologist, Gary Klein, in his book Sources of Power, describes studies of real people making life-and-death decisions. The process he finds is called Natural Decision Making (NDM). When he dissects the NDM process as it is practiced by true experts in fields as diverse as firefighting and aviation, he finds that the process is usually fairly similar, but not at all similar to the Rational Decision Making (RDM) that is frequently taught it to the analysis needed to reach the optimal conclusion. Simon suggested that RDM required unbounded rationality, or, in other words, potentially infinite data and infinite analysis. Actuarial methods are ultimately a part of an RDM process. The unbounded rationality idea is easily seen in the frequent calls of actuaries for more data and more time to complete the analysis. Simon went on in his work to develop the idea of satisficing as a decision criteria instead of optimizing. Satisficing means concluding the decision process when a satisfactory outcome can be found from one decision alternative.

the language of riSk Statistics allows for a mathematical expression of the unknown future. Its most powerful application is in the determination of an expected value of some future set of possible financial outcomes of an indeterminate agreement between parties. As the applications of statistics were expanded in the financial world, statistics also naturally became the language of risk. The main thrust of economics over the past 50 to 75 years has been to translate the ideas of how the world of commerce works into the language of statistics so that the powerful tools of statistical calculus could be applied. At some point, scientists crossed over into the financial world and started to apply models from quantum phys-

a Different approach The findings of Kleins studies line up well with Simons satisficing. The experts studied by Klein spent their time in a pattern that was directly opposed to the RDM model. Under RDM, the decision maker needs to spend most of his or her time studying the alternative solutions. Klein found that in real life, most experienced decision makers spent al-

June/July 2012

the actuary

27

most all of their time studying the problem. As they studied that problem, they were constantly eliminating possible solutions as unsatisfactory. They usually only considered one potential solution at a time. When these experienced decision makers ran out of time or were satisfied that they had stud-

his comment was a backhanded slam at the total inaccuracy of the models. Actuarial models have had big misses as well. In the United Kingdom, actuarial appraisals of annuity liabilities were found to be short by a large fraction in the early part

types of incidents that are just not visible to the standard analysis techniques. Outside-the-box thinking is usually based upon Systems Thinking. It seems to be so unusual because most people do not try to understand systems most of the time. In most situations, people just assume that tomorrows weather will be the same as yesterdays. It is computationally more efficient for a brain to spot patterns and trends than to think too deeply about why it is happening that way, so it is not surprising that most people default to studying problems that way. As we pointed out in The Actuary article Systems Thinking (April/May 2012), statistical models are by themselves insufficient to capture the range of possibilities of a complex adaptive system. Systems Thinking can provide the insights that allow ac-

outSide-the-box thinking iS uSually baSed upon SyStemS thinking.

ied all important aspects of the problem, they then decided to use the single solution that was still under consideration, one that would work. These two approaches, FFH and NDM, as practiced by Kleins experts appear to fit well into Kahenemans Thinking: Fast and Slow. The Heuristics are the fast thinking and the NDM approach the slow thinking. But RDM, which includes most actuarial analysis, is outside of these two common systems of thinking and decision making. Klein found that only very inexperienced decision makers used an RDM approach. This discussion gives a compelling description of why decision makers may not seem to be listening to actuaries. of this century. In the United States, actuarial models of variable annuity guarantees drastically understated the cost of out-ofthe-money guarantees in the run up to the dot-com collapse. Other insurance models of natural catastrophe risks have proved to be inadequate to anticipate the frequency and severity of catastrophes that have hit insured zones in the past 10 years. Nassim Talebs book, The Black Swan, is about those

Heuristic Process
EXISTING HEURISTIC

Big miSSeS anD outSiDe the Box One of the reasons why decision makers seem to not be listening to actuaries is the fact that like the financial pricing models of mortgage securities, actuarial work sometimes misses the mark. Not by a small amount. In the phrasing of David Viniar of Goldman Sachs, a 25 standard deviation event. Many people take that to mean that Viniar does not understand how ridiculously remote a 25 standard deviation event would be. But it is actually more likely that he does understand statistics and

NEW HEURISTIC

UNEXPECTED INFORMATION

STATISTICAL ANALYSIS

SYSTEMS THINKING

28

the actuary

June/July 2012

Kno

ef id l avo rns tu
win

he gw

n to

tuaries and other quantitative analysts to look around the corner of the situation that they are modeling.

with by a Heuristic. So the existing Heuristic may well include consulting an actuary for some statistical analysis. For actuaries to advance from the situation of dissatisfaction with their role in decision making, actuaries need to understand how our work can be used to reform Heuristics and NDM processes, and we need to avoid wrong turns in our statistical analyses that can be reduced by using systems analysis.

upDating the actuarial thinking cycle Recognizing that the modern world is increasingly complex and learning from the insights of those who have studied complex adaptive systems, it is a healthy reminder that we must remain open-minded to see what is in front of us. Integrating techniques more formally into actuarial work that can help to make sense of the underlying mechanisms of problems will help to ground models more clearly in reality and therefore enhance the role of actuaries in explaining their models and demonstrating their value. It will also help users to understand the deficiencies of models so that they appreciate when the model can be used and when it cannot. Some actuarial work could be identified as specifically targeted to updating a Heuristic (or NDM process). (See image on page 28.) While most business decisions are based upon a set of existing Heuristics and NDM processes, in the areas where actuaries usually work, there is some recognition that statistical analysis can produce superior results than a simple Heuristic. This is usually because the number of potentially important variables is much larger than can be dealt

But the profession cannot afford to continually be 10 years late to applying the latest, best thinking to our work. This new work in the area of Heuristics, NDM and Systems Analysis can be used to improve actuarial processes that already reflect these ideas in perhaps a less developed manner. Actuaries usually figure out when they shouldnt be taking left turns. Systems Analysis provides the tools for more consistently finding the right route and the reason for avoiding the left turns. NDM and Heuristics provide ways for actuaries to better communicate their findings to others. A
neil cantle, aSa, fia, m.a., is principal and consulting actuary with Milliman, Inc., London. He can be contacted at [email protected]. David ingram, fSa, cera, frm, prm, is executive vice president for Willis Re Inc. He can be contacted at [email protected]. ResouRces: Gerd Gigerenzer Rationality for Mortals (2008) Daniel Kahneman, Thinking Fast, Thinking Slow (2011) Gary Klein, Sources of Power: How People Make Decisions (1998) Gary Klein, Streetlights and Shadows (2009) Herbert Simon, A Behavioral Model of Rational Choice (1957)

when to avoiD left turnS One system that actuaries were late to understand was the market system. Actuaries had little need for understanding the shortterm fluctuations of market prices and tended to model market-traded instruments ignoring those fluctuations. However, a vast science grew up that was based almost solely upon the study of those fluctuations in market prices. Coincidentally, Financial Economics itself has been seen to have a vast shortcoming in its appreciation of the market system. An example of current Systems Thinking applied to problems that actuaries encounter is the Plural Rationalities discussion (recently featured in The Actuary, The Changing Seasons of Risk Attitudes, Feb./March 2011). Both risk attitudes of businesses and risk in the business environment are seen to be a part of an interdependent complex adaptive system.

June/July 2012

the actuary

29