Bold Hypotheses

 

In my last post I mentioned the importance of making bold hypothesis in product management.  I feel it makes sense to explain a bit more what I understand by bold hypothesis and why they’re important

I feel that the metric driven start up world often downplays the role of hypothesis in the road of discovering product/market fit. My sense is that there is a general misconception which assumes that correct hypotheses are a direct consequence of orderly arrangement and analysis of data. I would argue that is seldom the case.

I think most scientist would agree that the framing of a hypothesis is the most difficult part of science research, and where the greatest degree of ability and inventive is needed. I also think that they would agree that despite the prominence of data and statistically driven research no method has yet been found to create hypothesis by rule. Often, there is a long deductive journey from hypothesis to something that can be tested by observation.  In order to come up with valuable hypothesis you need to be able to take intellectual risks. You will usually need bold hypotheses to come up with great discoveries.

Let me share an example of what I see as a bold hypothesis: It was assumed since the time of the Greeks (perhaps even before?) that the celestial bodies moved in circular orbits. This claim was backed by a observation but  was also highly influenced by theological notions. It was believed that celestial bodies were either gods or specially created by gods, and hence were perfect. Perfection was represented aesthetically by their spherical form and the circular orbit.

As observation improved it became apparent that the circular form did not properly describe the orbit of the planets. The data clearly stated that the orbits were Not circular but the data did not clearly say that it was elliptical either. For any modern trained eye the data clearly depicts an elliptical orbit, but back then there was no room to imagine that could be the case.

Since imagining anything other than a circular orbit was so difficult, one of the intermediate hypotheses was the notion of the epicircle which looks like this:

 

As we now know, this hypothesis did not make the cut until eventually Kepler proposed the elliptic orbit. But this did not fully account for certain irregularities in the data, so Newton then improved on it explaining that actually the gravitational pull between celestial bodies explained why the orbits were not perfect.

My point with all of this is that data supports models, but creativity and the intellectual force to question the most fundamental assumptions is what really drives bold hypothesis.  And these bold hypothesis are what truly moves knowledge forward.

In the case of our own industry, I think the misconception in web (and mobile) start-ups comes from confusing ease of test with ease of producing hypothesis. Often enough, this leads to tactical improvements which are heavily data driven and not to core feature improvements that open or create new markets.

The problem with early data driven hypotheses is that it is likely to leave you optimizing a local maxima.  To find not the peak of the current mountain but the highest mountain available, you need to be able to question the assumptions by which you analyse the data and then you need to be creative enough to think about alternative, more simple models that could account for the data at hand.
Creativity drives hypothesis prior to the data, and only after the hypothesis has tested will the data validate or eliminate your hypothesis.

How I approach product management in early stage start ups

I want to share the main points I always keep in mind when working on product management. Most of these are common knowledge to the lean start up / customer development crowd as they are inspired by those initiatives, but I wanted to see what you guys make of it.

1)  Your main responsibility is to decide what not to include. This is the key to fast iterations, and fast iterations are the most sensible path to finding product/market fit because it helps you built the institutional knowledge you need at a faster rate. You see, when you start your company you really don’t know what it is you don’t know. Figuring out what to finally build requires a corpus of institutional knowledge that needs to be unique to your company and unique to your product.

2) The main criteria for what to include is to pick the riskier items first. For example, if you are trying to build the next Facebook on top of voice notes (as opposed to pictures) probably all you need to start testing your idea is a feed of some sort with the voice notes of all your friends. A profile with my education background or an event invite system is not something you need in order to test your idea. You are starting a company because you think there is a different and better way of doing things. That new way of doing things will be your riskier items. Test those first.

3)  Never lose sight of the product development cost structure. Since fast iterations are the key, the main item of the cost structure is time. Hence, you need to have a good sense of how long things take to get done. This is particularly difficult to get right in software development but every marginal improvement in accuracy on your end has a disproportional positive impact in your ability to launch a product that will work. Early product development ROI is a function of time spent per iteration  x positive delta in institutional knowledge. The more visibility you have on the time it takes to iterate, the more visibility you will have on the ROI of your iterations.

4) Before product/market fit use metrics to test Bold hypothesis. This will increase your chances of maximizing institutional knowledge per iteration. By bold hypothesis I mean educated guesses about users intent that will result in new features or removing features. In contrast, light hypothesis are changes that clarify and optimize current usage. Examples include changing the order of the items on the navigation bar, testing colors and copy and optimizing viral hooks. After market fit, you should make way for more light hypothesis but never abandoned the bold ones. You should always ask yourself: If I where to start this from scratch, how would I build it ?

5) Simplicity is key. You need to make sure that if people don’t use a feature it’s because they don’t like it, not because they don’t understand it or they can’t find it. Simplicity has an aesthetics quality to it, but it does not mean optimizing for aesthetics. In fact, people often will bundle aesthetic approval with product approval and what you need first is product validation.

I would be interested in learning what are the key product development guidelines that you always keep in mind.