opinions-philosophy-leanBusinessIsNotScience

I've been hearing that the "lean" entrepreneurial methodology is the application of the scientific method to market-opportunity identification.

It's not.

It's the application of trial-and-error to market-opportunity identification. This is a good idea, but trial-and-error alone is not enough to constitue the "scientific method". The scientific method also requires other components, including but not limited to design of experiments to test hypotheses, design for repeatability, reasoned interpretation of experiments, publication, peer review, an interplay between theory and experiment, and an unhurried push towards consensus.

Sorry to be pedantic here, but as we saw with Marxism, because science has an authority associated with it in our culture, ideologies that call themselves "scientific" sometimes lead supporters to attack criticism using misplaced appeals to authority. So i'm a little touchy about bestowing the mantle of "science" on things.

What the scientific method would really look like if applied to market-opportunity identification

There is a hypothesis floating around the entreprenurial community that perhaps there might be a market for a certain type of product that fulfills a certain function. The seasoned advisor of an entreprenur suggests that she check it out. They design a product to test this hypothesis. The product is introduced to the market, and it fails. The entreprenur suggests to her mentor (who is also her funder) that they try something else. But her mentor is fixated on this hypothesis. So they try again, although the entreprenur expects to fail.

No expense is spared; the entreprenur knows that if the result is that no one buys it, if the product is not clearly the best possible product of its kind, her mentor will simply insist that she try again, saying, "It's not clear if a market didn't exist, or if the product just wasn't good enough".

The product is introduced to the market -- it's a hit! It's flying off the shelves! Wait a minute; an overly enthusastic marketer thought up a really cute name for the product and a few quick polls have found that almost everyone is buying it as a gag gift due to the funny name. Angrily, the entreprenur pulls the product and fires the marketer; after all, upon publication, the reviewers would say, "This doesn't tell us if there was a real market opportunity for this type of product, or just for the NAME of your particular product. Due to trademark law, not everyone can use that name; your experiment isn't repeatable." In science, it is better to have a repeatable experiment that shows a small result than to have an unrepeatable experiment that shows a large result.

The entreprenur launches a similar product under a different name (carefully marketed to a totally different crowd of people, lest the company's name recognition for their previous product contaminate the results). The product sells a reasonable amount. It seems the mentor was right; there is a market for this product. The entreprenur and the mentor begin discussing how to write the results up. In the process of discussing the results with each other and with colleagues, they realize that the product actually could be used in another way totally different from originally intended. They interview some customers and find that, indeed, many customers are using the product in the unintended fashion. The unexpected purpose of the product is merely that it generates electricity at low cost and with no pollution. Drat! If they published this the reviewers would say, "It's unclear if there is actually a market for what you are saying, or if people are just buying this as a method of producing electricity. Everyone knows that there is a market for low-cost green energy, so that is not a sufficiently interesting result to be publishable in the top journals".

The entreprenur pulls the product, and carefully redesigns it so that is cannot be used for produce electricity. The redesigned product is introduced again (again carefully making sure that none of the company's name recognition from the previous products inadventently boost sales of this product), and it sells anemically -- but steadily. The entreprenur and her mentor write up the results for publication. The publication includes a careful and complete description of all of the important marketing and sales data that were gathered. It carefully cites and thanks all of the competitors who have introduced similar products in the past.

The publication is submitted to a journal, where is it reviewed by the entreprenur's competitors. The reviewers don't like it. It seems that the contours of the product's shape is just very friendly while also making it look elegant and hip. The reviewers question whether, possibly, people are buying the product just because it looks cool. They ask the entreprenur to release a new product that has similar functionality but is uglier.

The entreprenur thinks this is hogwash but she complies, pulling the existing product, redesigning it to be uglier, and re-launches it. The product sells just as well (anemic, but steady). The results are sent back to the journal, which agrees to publish the paper.

The paper is published, initially to great interest, but there is a problem. The entreprenur's competitors introduce similar products, and they don't sell well at all. There are cries that the experiment was not replicable; even whispered accusations of accounting fraud. The entreprenur invites her competitors to her lab to be trained on exactly how to produce and market the product. After months of training, some of the competitors are finally able to produce almost a perfect clone of the product, which, coupled with a similar marketing, sells almost as well as the original. The entreprenur is vindicated.

Still, questions remain. There are still other alternative uses for the product; it is well-known that the product can be used for propping open doors. In the analysis in her publication, the entreprenur estimated the demand for a product the size of hers for the propping-open-doors market and subtracted this from her sales figures in order to estimate the demand arising from the target market. However, the question of how to correctly estimate the demand for the propping-open-doors market has never really been properly addressed, and even the entreprenur admits that her estimate in this regard is mostly just a guess. "It just has to be tiny," she says, "when have you ever heard of someone buying a product not designed to prop open doors solely for that purpose?" By and large, the business community agrees, however there are doubters, and some of the doubters are very prominent and well-regarded businessmen. "Usually it is not a problem however there are exceptions," says esteemed CEO and serial entreprenur Steve Jobs. "This product happens to have a stand that is rubbery and door-stop shaped. Once in my company we made something with a stand like that and found that people were buying it just for its door-stop like properties. Of course, i pulled the product immediately and released one with a new stand, and I didn't even report that trial. To do otherwise would have just been sloppy science. Let's just say I wouldn't tell my VC friends that this entreprenur is worth taking a chance on.". Worse, the entreprenur is still dogged by shadowy accusations. A follow-on product is selling well, although competing products are not selling at all, and it is said that our protagonist used her charisma and personal connections to secure strategic partnership and co-branding agreements with certain large distributors and celebrites, possibly creating demand for the product through advertising where no natural demand exists -- another case of an unrepeatable experiment.

So, for now the question is considered unresolved. Does the market exist or not? It is still an open question. The business community has not yet reached consensus. The business community may take decades to conclude whether or not this market exists for decades -- and that's ok, because what's important to the business community is that when a conclusion is reached, that conclusion is unlikely to be wrong. For now, other entreprenurs, when drawing up business plans, are not permitted to simply assume that there is any demand for products of this type, despite the existance of multiple extant product of this type that are sellling. At this point, any entreprenur who openly/blindly assumed that this market exists would have trouble attracting funding.

What point am i making here?

The scientific method bestows authority because it is the most reliable way we know for discovering empirical truth. For this to work, the scientific community must be willing to say, "Yes, it looks like X is true, and yes, if I had to bet, I'd bet that X is true, but the truth or falsity of X really hasn't been established yet with the level of rigor required to call the result scientific". There is a distinction between something that is likely to be true, and something which is conclusive (although even conclusions, if empirical, have a level of uncertainly remaining).

The "lean" entrepreneurial methodology is focused on getting results. The results of lean product introduction experiments do not lead to reliable conclusions about markets, because they do not satisfy the rigor of hypothesis testing, repeatability, publication, peer review, integration with theory, and unhurried consensus. That's okay, because business decisions can and should be made now, in the presence of risk, rather than a decade from now, after all uncertainties have been resolved.

If a product succeeds or fails in the marketplace, this may be enough information for the company that introduced it to make a decision of what to work on next, but it will usually not be enough information to conclusively determine anything. If lean entrepreneurialism is thought to be science, there is a danger that practitioners will mistakenly argue that the evidence provided by such experiments are conclusive, which will lead to incorrect assumptions being accepted as proven facts; this could deter companies from following a vision despite a few failures, as pressure is put on them to pivot because of the incorrect assumption that the existence or non-existence of a market has been conclusively determined via experiment. There is also a danger that proponents of lean will argue that it is good because it is like science and science is good.