Learning Under Uncertainty: Science Meets Startups!

In the worlds of science and startups, learning under uncertainty is a crucial component. Both fields aim to maximize impact and make informed decisions while they are confronted to considerable unknowns. This blog explores the scientific method and the startup method, highlighting their similarities and differences, particularly in their approaches to dealing with different sources of uncertainty. It aims to provide initial insights into the question, drawing from multiple readings listed at the end of this article (which I highly recommend exploring to form your own opinion on the topic), as well as from my personal experience in scientific research and building a startup from scratch :)

What is the Scientific Method?

The scientific method is a systematic approach to robustly improve our understanding of the natural world despite of different sources of uncertainty. It consists in making hypothesis about the real world, experiment them against the real world, and conclude about their veracity on the real world. It typically involves the following steps:

1. Hypothesis: Formulate potential explanations about a natural phenomena. Scientific hypothesis can always be tested against the real world to verify/falsify them. It is usually interesting to collect initial observation or beliefs to formulate relevant hypothesis for the community. Example: Water boils at 100 degree Celsius in Munich.
2. Experimentation: Design and conduct experiments to test the hypothesis against the real world. The results of the experiment should bring information to accept/refute the hypothesis, and should be reproducible to ensure their reliability. Example: Heat water until it boils in Munich. Repeat the experiment 10 times.
3. Conclusion: Confirm/refute the hypothesis based on the experimental results. Ideally, the conclusion should logically derive from the observed results, and should be presented with confidence estimates. Example: Water boils at ~100 degree Celsius in Munich with small variations.

Given the scientific conclusion of one cycle, this step-by-step approach is usually repeated to discover new knowledge (Example: A next hypothesis could be the air pressure affects the boiling temperature of water in Munich). The scientific method presented above might vary a bit depending on the fields of applications but usually contains these important steps.

In the scientific context, the epistemic uncertainty is composed of all the uncertainty that can be reduced with the collection of experimental data (Example: the boiling temperature in Munich), while the aleatoric uncertainty is composed of all the uncertainty that cannot be controlled during the experimentation phase (Example: the meteo in Munich) [2]. Hence, the ultimate goal of the scientific method is to reduce the epistemic uncertainty about our understanding of the real world, while accounting for the aleatoric uncertainty which arise from the inherent randomness in natural processes.

What is the (Lean) Startup Method?

According to Eric Ries in the book “Lean Startup” [1], a startup is a human institution designed to robustly create new products or services under conditions of extreme uncertainty. The (lean) startup method consists in navigating this uncertainty through a cycle of building, measuring, and learning. The key components are:

1. Build: Define key assumptions of the startup idea and develop a Minimum Viable Product (MVP) to test them quickly and with minimal resources. Example: Design a new design of Bavarian Lederhosen for the Oktoberfest.
2. Measure: Collecting data on the MVP’s performance using actionable metrics to test the key assumptions. In contrast with vanity metrics which give good feelings but do not really measure success, actionable metrics are capable to show clear cause and effects. Example: Measure the sales of the new design of Lederhosen.
3. Learn: Analyze the data to validate or invalidate the initial assumptions of the MVP, and decide whether to pivot to another different idea or persevere to fine-tune the idea. Example: The new design of Lederhosen does not match customer expectations because of its modern style.

The (lean) startup method emphasizes rapid experimentation and iteration to mitigate risks and adapt to changing conditions quickly (Example: A next MVP could be to design more traditional clothes for the Oktoberfest)

In the startup context, the epistemic uncertainty refers to the unknowns about the market or customer preferences which can be reduced by building new features and measuring their effects (Example: the new design meets customers expectations), while aleatoric uncertainty might involve unpredictable economic factors or random events affecting market conditions (Example: potential customers do not have the financial power to buy products due to the economy situation).

How do the scientific and startup methods compare?

Based on the description of the scientific and the startup methods, we can arguably identify important similarities between these two worlds. Nonetheless, it is important to nuance their similarities with their differences in the way they are practically applied.

Similarities

1. Confrontation of ideas and real-world: Both methods rely heavily on experimentation to test scientific or business hypotheses/assumptions against the real world.
2. Iterative Approach: Both methods rely on quick and efficient feedback loops to continuously learn and adapt with wasting time in unfruitful directions.
3. Rational decisions: Decisions are based on rational derived from empirical data or logic rather than intuition or speculation.
4. Management of Uncertainty: The management of porjects with extreme uncertainty is an important component in both methods. Since both scientific and startup contexts are subject to extreme uncertainty on their outcomes, both methods aim to systematically reduce epistemic uncertainty and manage aleatoric uncertainty over time.

Differences

1. End Goals: The major difference between the two methods is the objective they chase. While the scientific method seeks to generate new knowledge and understand fundamental principles, the startup method aims to create viable products and achieve business success.
2. Public vs Private Works: On one hand, scientific research usually recommends to make the discovered knowledge public, thus facilitating verification of the work by research peers. On the other hand, startups usually aim to keep parts of the knowledge secret to protect the unique selling points of the company. This is a critiacal difference since the publication of scientific articles is sometimes considered as part of the scientific methods.
3. Nature of Uncertainty: In scientific research, uncertainty often stems from a lack of knowledge about natural phenomena (epistemic uncertainty) and inherent randomness in natural processes (aleatoric uncertainty). In startups, epistemic uncertainty arises from unknowns about customer and market behavior, while aleatoric uncertainty involves external economic conditions and unforeseen events.
4. Iteration Speed: The iteration speed is mostly defined by the subject of inquiry of the scientific or business idea. For example, testing (scientific or business) hypothesis of drugs on humans in clinical trials is significantly slower than testing (scientific or business) hypothesis of AI algorithms which can be done immediately.
5. External Communication: Scientific research aims to communicate complete and verified discoveries e.g. via research papers and reproducible experiments. In contrast, startups often imitate confidence and skills to follow well-known saying “Fake it until you make it”. While it can sometimes help to faster realize the startup ambition [5], it also sometimes translate in exaggerated marketing statements [7].
6. Learning Robustness: In contrast with science where it is a necessary condition to make logical conclusions based on data or proofs, startups can sometimes sell what they believe they can do in the future instead of what they concretely and currently have in the present. While it can be useful to accelerate iterations or when it is hard to collect enough validations, this startup behavior can be dangerous and lead to misconduct [6]. Successful startups usually manage to robustly test their hypothesis without scarifying efficiency.

Conclusion

Both the scientific method and (lean) startup methods provide robust frameworks for learning under uncertainty. By embracing hypothesis, experimentation, rational analysis within feedback loops both scientists and entrepreneurs can navigate the unknowns in their respective fields. Understanding these methods’ nuances helps optimize strategies for achieving impactful research and innovative business ideas in the face of uncertainty.

References

1. Ries E.. The Lean Startup.
2. Crawford S., et al. Peer Review and the Changing Research Record. Journal of the American Society for Information Science.
3. Tversky, A., et al. Judgment under Uncertainty: Heuristics and Biases. Science.
4. Charpentier B. Uncertainty Estimation on Independent and Non-Independent Data.
5. Afia Ibnat. Why “fake it till you make it” works. Blog.
6. https://www.imperial.ac.uk/business-school/ib-knowledge/entrepreneurship-innovation/the-dark-side-startups-when-fake-it-till-you-make-it-goes/
7. *What startups were very hot, but failed to live up to the hype? Forum.*