LITMIR.BIZ

CHAPTER 3 How Do We Know What Works?

      Leaders get out in front and stay there by raising the standards by which they judge themselves—and by which they are willing to be judged.

      —FREDRICK SMITH, CEO, FEDEX

      The first principle is that you must not fool yourself, and you are the easiest person to fool.

      —RICHARD P. FEYNMAN, NOBEL PRIZE–WINNING PHYSICIST

      According to some risk management surveys, organizations are very often satisfied with their risk assessment and risk management methods. For example, a survey by the major consulting firm Deloitte in 2012 found that 72 percent of organizations rate themselves as “extremely effective” or “very effective” at managing risks (up slightly from 66 percent in 2010). In other words, a majority believe their risk management is working. But, as the quote by Feynman above tells us, we are easy to fool.

A harder question to answer is, “What is the evidence for the belief that it works?” For any firm that hasn't asked that question before, it should be an immediate priority. If the firm can't answer that question, then it has no reason to believe that efforts to manage risks are working or, for that matter, are even focusing on the right risks. The standard must be some objective measure that could be verified by other stakeholders in the organization or outside auditors.

      Most (69 percent according to the HDR/KPMG survey) don't even attempt to measure whether risk management is working. Of those who say they do measure risk, most (63 percent) are merely using a survey of staff with questions such as, “How would you rate the effectiveness of risk management?” It may not be obvious now, but there are ways to measure risk management objectively even though such measurements are uncommon.

      This chapter will describe the difficulties in conducting measurements of risk management and some solutions for overcoming them. But first, to highlight the importance of measuring risk management, let's look at one example involving the health and safety of large numbers of people.

ANECDOTE: THE RISK OF OUTSOURCING DRUG MANUFACTURING

      In 2007, I was asked to speak at a conference organized by the Consumer Health Products Association (a pharmaceutical industry association). The event organizers were specifically interested in my contrarian views on common risk management methods. After my keynote, I was asked by the event organizers to attend another session on a new risk management method for outsourcing drug manufacturing and provide my comments to the audience. They thought it would be interesting if I could start a conversation by offering an on-the-spot evaluation of the new method.

      To control costs, this large pharmaceutical manufacturer was more frequently outsourcing certain batch processes to China. Virtually all of this manufacturer's competition were doing the same. But although the costs were significantly lower, they had a concern that batches from China might have additional quality control issues over and above those of batches manufactured here in the United States. These concerns were entirely justified.

Earlier that year there had already been several widely publicized product safety incidents with goods produced in China. In June, there was a toxin found in toothpaste and lead found in toys produced in China. Then there was tainted pet food that killed as many as 4,000 pets. There was even the disturbing case of “Aqua Dots,” the children's craft-beads that stuck together to make different designs. The coating of these beads could metabolize in the stomach to produce gamma-hydroxybutyrate—the chemical used in date-rape drugs.

So, clearly, assessing the risk of outsourcing was a major area of interest at the conference, and the room was at capacity. The presenter—a very respected chemical engineer—began to describe a risk assessment method based on a subjective weighted score.¹ In it, several “risk

Скачать книгу