“You are smarter than your data. Data do not understand causes and effects; humans do.”- Judea Pearl, The Book of Why: The New Science of Cause and Effect  <h2>Understanding the ‘Why’ to navigate ambiguities in decision-making</h2>In the complex world of business, the decision-making process can often feel like trying to find your way through fog. It's hard to be sure you're choosing the right path when everything seems unclear. This article emphasises a powerful tool to cut through that fog: Understanding the ‘Why’ behind things, which is a bit like using a map in unknown territory. By looking closely at the reasons behind business challenges, as scientists do when they are solving mysteries, we can make better decisions. Through real-life examples and stories, we'll see how figuring out the ‘Why’ can light up the way forward in business. <h2>A case study: Beyond sales team ‘right-sizing’</h2>Consider the case of one of our clients (operating in a B2B environment). They faced the dilemma of a ‘bloated’ sales team that was trained for years to handle client servicing (everything from order delays to payment follow-ups). The market was projected to grow slowly at 7% year on year. However, the client wanted to grow faster and profitably. A simple approach would be to reduce excess salespeople and improve profitability. Right?Wrong! Analysing this problem with existing data risked taking actions that would be detrimental to the company’s existence! The sales team, in this case, played a crucial role in ensuring that the client’s customers were served on time, every time, in a highly competitive market.The understanding we built here was: “If the market is highly competitive, then order servicing is a business-critical action and cannot be diluted.” Having the sales team involved in order servicing end-to-end was a feature, not a bug!Hence, what was needed was a rethink of the market the client was operating in, not ‘rightsizing’ the sales team. We developed a strategy to retain the customer service skills of the existing sales team and reskill top performers for an untapped high-growth market segment. We then used data from the untapped market to determine how to build a moat for the client, thereby enabling profitable growth.It worked! The client is in the process of redesigning their entire sales strategy. Taking a step back allowed us to prevent an action that could have jeopardised a business -critical feature of the sales operations!  <h2>The quest for ‘why’</h2>This way of working on business problems (taking a step back and building a deep understanding of the underlying causes) is inspired by science. We call it Causal Thinking (you can call it – the quest for ‘Why’). There are many pitfalls to not following this approach. Here’s one example of the many disasters in the business world where decision-making has been purely data-driven without understanding the ‘Why’.  <h2>The 2008 Financial Crisis</h2>A devastating blow to the global economy was exacerbated by an overreliance on sophisticated algorithms used by rating agencies and trading desks at major banks. These algorithms, intended to assess the risk associated with various financial instruments, were critically flawed. They mistakenly rated complex mortgage-backed securities (MBS), many of which were essentially junk bonds, as AAA - the highest possible credit rating. This misrepresentation of risk encouraged a widespread investment in these toxic assets, leading to a massive bubble. When the bubble burst, it triggered a cascade of financial failures as the actual value of these securities became apparent. This crisis highlighted the dangers of relying too heavily on algorithmic decision-making in financial markets, underscoring the need for more robust oversight and a deeper understanding of the risks posed by complex financial instruments. The movie ‘The Big Short’ illustrates how some predicted the crash by delving into the story behind MBS, unlike the major banks, which missed the signs (or chose to ignore them).David Spiegelhalter, in his book, The Art of Statistics: Learning from Data, writes, “Senior managers simply did not realize the frail basis on which these models were built, losing track of the fact that models are simplifications of the real world – they are the maps, not the territory. The result was one of the worst global economic crises in history.” Contrast this with the lifesaving work done by Dr. John Snow amidst the cholera outbreak of the 1850s in London. Dr. John Snow, an English physician, is often considered one of the fathers of modern epidemiology for his pioneering work during the cholera outbreaks in London in the mid-19th century. His most famous research is associated with the 1854 cholera outbreak in the Broad Street area of Soho, London.Snow did not believe in the then-dominant miasma theory, which suggested that diseases such as cholera spread through the air as a form of ‘bad air’. Instead, he proposed that cholera was spreading through contaminated water – a new theory.Dr. Snow had evidence contrary to the miasma theory, which allowed him to propose a new one. For instance, if the ‘miasma’ spread through the air, why were there pockets of no disease within a cluster of high disease prevalence? How could air be causing the disease if everyone in the same cluster and prominent pockets was breathing the same contaminated air, yet those pockets were completely healthy?Thus began Dr. Snow’s meticulous case-tracking and mapping of cholera cases. He identified a public water pump on Broad Street as the epicentre of the outbreak.By gathering data on where the victims of cholera lived and from where they got their water, Dr. Snow was able to convincingly argue that those who drank water from the contaminated Broad Street pump were much more likely to develop and die from cholera, and those who were drinking from safe supplies in the same vicinity as the Broad Street pump were safe, even though they were in the vicinity.  He famously had the pump handle removed, and the outbreak quickly subsided.Dr. Snow's work didn't immediately change public health policies. Still, over time, his theory, backed by a strong evidence-based approach, helped shift the scientific community's perspective to recognise the importance of sanitation, clean water, and the germ theory of disease. His mapping and data analysis methods remain foundational in epidemiology today, illustrating the importance of building a sound theory, gathering empirical evidence, and applying scientific methods to public health.The key point is Dr. Snow started with a causal framework, a theory of ‘Why’ and then worked his way to meticulously investigate data to determine whether it was correct - NOT the other way around! <figure class="image"><img style="aspect-ratio:1692/1030;" src="https://web-assets.vectorconsulting.in/storage/editor_uploads//screenshot-2024-05-13-at-44453-pm_1715598908.png" width="1692" height="1030"></figure><h2> </h2><h2>Data as a compass, not the map</h2>Just like Dr. John Snow used data to trace the cause of cholera back to a single water pump, data can illuminate the actual reasons behind what we observe, helping us validate or invalidate the causal story. A striking illustration from science is how observations during a solar eclipse validated one of Einstein's revolutionary predictions that gravity can bend light. This concept reshaped our understanding of the universe, but it wasn't universally accepted until data from a solar eclipse showed that light from stars curves around the sun, precisely as Einstein had predicted. This historical moment underscores a critical lesson: Data is most powerful when used to test our theories and hypotheses, not when viewed in isolation. Just gathering data without a clear idea of what you're looking for can be like finding pieces of different puzzles mixed - you might have a lot of pieces, but they don't help you see the whole picture.In business, this means that before delving into data, we should start with a theory or a hypothesis about why things are happening. Then use data to test our theory and guide us towards the right solutions and decisions. This approach ensures that we're not just collecting data for data's sake but using it as a tool to challenge and refine our understanding, leading to clearer insights and better outcomes.  <h2>A simple guide to building a causal viewpoint</h2>Let’s circle back to our case on the sales team ‘rightsizing’ and analyse it using simple causal analysis steps: 1. Clarify the problem you’re trying to fix (an exact definition of the problem)  <ul><li>Good Precision: “Growth has slowed down to the market rate of about 7%, while the cost of sales (including sales team opex) has gone up 1 percentage point.”</li><li>Poor Precision: ”We have a large sales force.”  </li></ul>2. Do NOT explain the problem – let the problem stand on its own <ul><li>“Cost of sales has gone up because we’ve hired people faster that market growth.” </li></ul> 3. Start with a hypothesis - a simple explanation of ‘why this might be happening’ <ul><li>“Share of business at regular customers is growing at market rate, while the share of business at irregular customers is stagnant or declining as these customers are switching to other types of input raw materials, or RM (this is where the bulk of new sales recruits have been deployed).” </li></ul> 4. Try to invalidate the hypothesis by counterexamples or any other variable impacting the outcome <ul><li>Switching to other RM is irrelevant, as this is true only for a few customers.</li><li>The import of input RM has increased, impacting the client’s customer base.</li><li>Overall consumption of the input RM remains steady at 7% for ‘irregular’ customers as well. </li></ul> 5. If the hypothesis still stands, check for effects of the hypothesis being true; if not, build a new hypothesis The hypothesis does not stand the test of invalidation in this case. A new hypothesis is needed! <ul><li>New Hypothesis: Imported RM is cheaper and of similar quality, allowing ‘irregular’ customers to switchCheck for effects if the new hypothesis is true:</li><li>‘Irregular’ customers are still consuming the same RM at a steady pace.</li><li>Switched customers are fine with the increased lead time of supply from imports.</li><li>Sales team’s efforts are rejected primarily due to the price differential.</li><li>If the sales team can match price with imports, some of the irregular customers switch back. </li></ul> 6. Build a direction of solutionThe new hypothesis stands validated. It is now time to brainstorm solutions.<ul style="list-style-type:disc;"><li>Since competing on price alone is a recipe for continuous margin pressure, this direction is ruled out.</li><li>There will always be imported RM flooding the domestic market for regular SKUs; there is no way of stopping this.</li><li>Customers already prefer our RM for non-regular SKUs.</li><li>The ONLY way to sustain existing demand and grow FASTER than the market is to DEVELOP more consumption occasions – i.e., convert users of other types of RM! A tall ask, but it’s possible. Our client therefore began redeploying the sales team, and the ‘bloated’ team no longer needed to be ‘rightsized’. In fact, we might need to hire more!</li></ul>These simple steps need rigorous analysis and can be quite challenging to implement without practice. It’s easy to find quick fixes using existing data. However, building a habit of thinking causally can have immense bottom-line benefits for companies – by avoiding erroneous decision-making and by ignoring irrelevant variables. This approach allows companies to focus on the right problem and the right explanation for it. So, happy thinking!

Key Takeaways

Ask the ‘WHY’  How causality can play a crucial role in solving business problems

Ask the ‘Why’: How causality can play a crucial role in solving business problems

“You are smarter than your data. Data do not understand causes and effects; humans do.”- Judea Pearl, The Book of Why: The New Science of Cause and Effect  <h2>Understanding the ‘Why’ to navigate ambiguities in decision-making</h2>In the complex world of business, the decision-making process can often feel like trying to find your way through fog. It's hard to be sure you're choosing the right path when everything seems unclear. This article emphasises a powerful tool to cut through that fog: Understanding the ‘Why’ behind things, which is a bit like using a map in unknown territory. By looking closely at the reasons behind business challenges, as scientists do when they are solving mysteries, we can make better decisions. Through real-life examples and stories, we'll see how figuring out the ‘Why’ can light up the way forward in business. <h2>A case study: Beyond sales team ‘right-sizing’</h2>Consider the case of one of our clients (operating in a B2B environment). They faced the dilemma of a ‘bloated’ sales team that was trained for years to handle client servicing (everything from order delays to payment follow-ups). The market was projected to grow slowly at 7% year on year. However, the client wanted to grow faster and profitably. A simple approach would be to reduce excess salespeople and improve profitability. Right?Wrong! Analysing this problem with existing data risked taking actions that would be detrimental to the company’s existence! The sales team, in this case, played a crucial role in ensuring that the client’s customers were served on time, every time, in a highly competitive market.The understanding we built here was: “If the market is highly competitive, then order servicing is a business-critical action and cannot be diluted.” Having the sales team involved in order servicing end-to-end was a feature, not a bug!Hence, what was needed was a rethink of the market the client was operating in, not ‘rightsizing’ the sales team. We developed a strategy to retain the customer service skills of the existing sales team and reskill top performers for an untapped high-growth market segment. We then used data from the untapped market to determine how to build a moat for the client, thereby enabling profitable growth.It worked! The client is in the process of redesigning their entire sales strategy. Taking a step back allowed us to prevent an action that could have jeopardised a business -critical feature of the sales operations!  <h2>The quest for ‘why’</h2>This way of working on business problems (taking a step back and building a deep understanding of the underlying causes) is inspired by science. We call it Causal Thinking (you can call it – the quest for ‘Why’). There are many pitfalls to not following this approach. Here’s one example of the many disasters in the business world where decision-making has been purely data-driven without understanding the ‘Why’.  <h2>The 2008 Financial Crisis</h2>A devastating blow to the global economy was exacerbated by an overreliance on sophisticated algorithms used by rating agencies and trading desks at major banks. These algorithms, intended to assess the risk associated with various financial instruments, were critically flawed. They mistakenly rated complex mortgage-backed securities (MBS), many of which were essentially junk bonds, as AAA - the highest possible credit rating. This misrepresentation of risk encouraged a widespread investment in these toxic assets, leading to a massive bubble. When the bubble burst, it triggered a cascade of financial failures as the actual value of these securities became apparent. This crisis highlighted the dangers of relying too heavily on algorithmic decision-making in financial markets, underscoring the need for more robust oversight and a deeper understanding of the risks posed by complex financial instruments. The movie ‘The Big Short’ illustrates how some predicted the crash by delving into the story behind MBS, unlike the major banks, which missed the signs (or chose to ignore them).David Spiegelhalter, in his book, The Art of Statistics: Learning from Data, writes, “Senior managers simply did not realize the frail basis on which these models were built, losing track of the fact that models are simplifications of the real world – they are the maps, not the territory. The result was one of the worst global economic crises in history.” Contrast this with the lifesaving work done by Dr. John Snow amidst the cholera outbreak of the 1850s in London. Dr. John Snow, an English physician, is often considered one of the fathers of modern epidemiology for his pioneering work during the cholera outbreaks in London in the mid-19th century. His most famous research is associated with the 1854 cholera outbreak in the Broad Street area of Soho, London.Snow did not believe in the then-dominant miasma theory, which suggested that diseases such as cholera spread through the air as a form of ‘bad air’. Instead, he proposed that cholera was spreading through contaminated water – a new theory.Dr. Snow had evidence contrary to the miasma theory, which allowed him to propose a new one. For instance, if the ‘miasma’ spread through the air, why were there pockets of no disease within a cluster of high disease prevalence? How could air be causing the disease if everyone in the same cluster and prominent pockets was breathing the same contaminated air, yet those pockets were completely healthy?Thus began Dr. Snow’s meticulous case-tracking and mapping of cholera cases. He identified a public water pump on Broad Street as the epicentre of the outbreak.By gathering data on where the victims of cholera lived and from where they got their water, Dr. Snow was able to convincingly argue that those who drank water from the contaminated Broad Street pump were much more likely to develop and die from cholera, and those who were drinking from safe supplies in the same vicinity as the Broad Street pump were safe, even though they were in the vicinity.  He famously had the pump handle removed, and the outbreak quickly subsided.Dr. Snow's work didn't immediately change public health policies. Still, over time, his theory, backed by a strong evidence-based approach, helped shift the scientific community's perspective to recognise the importance of sanitation, clean water, and the germ theory of disease. His mapping and data analysis methods remain foundational in epidemiology today, illustrating the importance of building a sound theory, gathering empirical evidence, and applying scientific methods to public health.The key point is Dr. Snow started with a causal framework, a theory of ‘Why’ and then worked his way to meticulously investigate data to determine whether it was correct - NOT the other way around! <figure class="image"><img style="aspect-ratio:1692/1030;" src="https://web-assets.vectorconsulting.in/storage/editor_uploads//screenshot-2024-05-13-at-44453-pm_1715598908.png" width="1692" height="1030"></figure><h2> </h2><h2>Data as a compass, not the map</h2>Just like Dr. John Snow used data to trace the cause of cholera back to a single water pump, data can illuminate the actual reasons behind what we observe, helping us validate or invalidate the causal story. A striking illustration from science is how observations during a solar eclipse validated one of Einstein's revolutionary predictions that gravity can bend light. This concept reshaped our understanding of the universe, but it wasn't universally accepted until data from a solar eclipse showed that light from stars curves around the sun, precisely as Einstein had predicted. This historical moment underscores a critical lesson: Data is most powerful when used to test our theories and hypotheses, not when viewed in isolation. Just gathering data without a clear idea of what you're looking for can be like finding pieces of different puzzles mixed - you might have a lot of pieces, but they don't help you see the whole picture.In business, this means that before delving into data, we should start with a theory or a hypothesis about why things are happening. Then use data to test our theory and guide us towards the right solutions and decisions. This approach ensures that we're not just collecting data for data's sake but using it as a tool to challenge and refine our understanding, leading to clearer insights and better outcomes.   <h2>A simple guide to building a causal viewpoint</h2>Let’s circle back to our case on the sales team ‘rightsizing’ and analyse it using simple causal analysis steps: 1. Clarify the problem you’re trying to fix (an exact definition of the problem)  <ul><li>Good Precision: “Growth has slowed down to the market rate of about 7%, while the cost of sales (including sales team opex) has gone up 1 percentage point.”</li><li>Poor Precision: ”We have a large sales force.”  </li></ul>2. Do NOT explain the problem – let the problem stand on its own <ul><li>“Cost of sales has gone up because we’ve hired people faster that market growth.” </li></ul> 3. Start with a hypothesis - a simple explanation of ‘why this might be happening’ <ul><li>“Share of business at regular customers is growing at market rate, while the share of business at irregular customers is stagnant or declining as these customers are switching to other types of input raw materials, or RM (this is where the bulk of new sales recruits have been deployed).” </li></ul> 4. Try to invalidate the hypothesis by counterexamples or any other variable impacting the outcome <ul><li>Switching to other RM is irrelevant, as this is true only for a few customers.</li><li>The import of input RM has increased, impacting the client’s customer base.</li><li>Overall consumption of the input RM remains steady at 7% for ‘irregular’ customers as well. </li></ul> 5. If the hypothesis still stands, check for effects of the hypothesis being true; if not, build a new hypothesis The hypothesis does not stand the test of invalidation in this case. A new hypothesis is needed! <ul><li>New Hypothesis: Imported RM is cheaper and of similar quality, allowing ‘irregular’ customers to switchCheck for effects if the new hypothesis is true:</li><li>‘Irregular’ customers are still consuming the same RM at a steady pace.</li><li>Switched customers are fine with the increased lead time of supply from imports.</li><li>Sales team’s efforts are rejected primarily due to the price differential.</li><li>If the sales team can match price with imports, some of the irregular customers switch back. </li></ul> 6. Build a direction of solutionThe new hypothesis stands validated. It is now time to brainstorm solutions.<ul style="list-style-type:disc;"><li>Since competing on price alone is a recipe for continuous margin pressure, this direction is ruled out.</li><li>There will always be imported RM flooding the domestic market for regular SKUs; there is no way of stopping this.</li><li>Customers already prefer our RM for non-regular SKUs.</li><li>The ONLY way to sustain existing demand and grow FASTER than the market is to DEVELOP more consumption occasions – i.e., convert users of other types of RM! A tall ask, but it’s possible. Our client therefore began redeploying the sales team, and the ‘bloated’ team no longer needed to be ‘rightsized’. In fact, we might need to hire more!</li></ul>These simple steps need rigorous analysis and can be quite challenging to implement without practice. It’s easy to find quick fixes using existing data. However, building a habit of thinking causally can have immense bottom-line benefits for companies – by avoiding erroneous decision-making and by ignoring irrelevant variables. This approach allows companies to focus on the right problem and the right explanation for it. So, happy thinking!

Ask the ‘WHY’: How causality can play a crucial role in solving business problems

Learn how understanding the 'why' behind business decisions can lead to smarter strategies and better outcomes. Discover the role of causal thinking in solving problems and driving growth.