Bold Hypotheses


In my last post I mentioned the importance of making bold hypothesis in product management.  I feel it makes sense to explain a bit more what I understand by bold hypothesis and why they’re important

I feel that the metric driven start up world often downplays the role of hypothesis in the road of discovering product/market fit. My sense is that there is a general misconception which assumes that correct hypotheses are a direct consequence of orderly arrangement and analysis of data. I would argue that is seldom the case.

I think most scientist would agree that the framing of a hypothesis is the most difficult part of science research, and where the greatest degree of ability and inventive is needed. I also think that they would agree that despite the prominence of data and statistically driven research no method has yet been found to create hypothesis by rule. Often, there is a long deductive journey from hypothesis to something that can be tested by observation.  In order to come up with valuable hypothesis you need to be able to take intellectual risks. You will usually need bold hypotheses to come up with great discoveries.

Let me share an example of what I see as a bold hypothesis: It was assumed since the time of the Greeks (perhaps even before?) that the celestial bodies moved in circular orbits. This claim was backed by a observation but  was also highly influenced by theological notions. It was believed that celestial bodies were either gods or specially created by gods, and hence were perfect. Perfection was represented aesthetically by their spherical form and the circular orbit.

As observation improved it became apparent that the circular form did not properly describe the orbit of the planets. The data clearly stated that the orbits were Not circular but the data did not clearly say that it was elliptical either. For any modern trained eye the data clearly depicts an elliptical orbit, but back then there was no room to imagine that could be the case.

Since imagining anything other than a circular orbit was so difficult, one of the intermediate hypotheses was the notion of the epicircle which looks like this:


As we now know, this hypothesis did not make the cut until eventually Kepler proposed the elliptic orbit. But this did not fully account for certain irregularities in the data, so Newton then improved on it explaining that actually the gravitational pull between celestial bodies explained why the orbits were not perfect.

My point with all of this is that data supports models, but creativity and the intellectual force to question the most fundamental assumptions is what really drives bold hypothesis.  And these bold hypothesis are what truly moves knowledge forward.

In the case of our own industry, I think the misconception in web (and mobile) start-ups comes from confusing ease of test with ease of producing hypothesis. Often enough, this leads to tactical improvements which are heavily data driven and not to core feature improvements that open or create new markets.

The problem with early data driven hypotheses is that it is likely to leave you optimizing a local maxima.  To find not the peak of the current mountain but the highest mountain available, you need to be able to question the assumptions by which you analyse the data and then you need to be creative enough to think about alternative, more simple models that could account for the data at hand.
Creativity drives hypothesis prior to the data, and only after the hypothesis has tested will the data validate or eliminate your hypothesis.

How I approach product management in early stage start ups

I want to share the main points I always keep in mind when working on product management. Most of these are common knowledge to the lean start up / customer development crowd as they are inspired by those initiatives, but I wanted to see what you guys make of it.

1)  Your main responsibility is to decide what not to include. This is the key to fast iterations, and fast iterations are the most sensible path to finding product/market fit because it helps you built the institutional knowledge you need at a faster rate. You see, when you start your company you really don’t know what it is you don’t know. Figuring out what to finally build requires a corpus of institutional knowledge that needs to be unique to your company and unique to your product.

2) The main criteria for what to include is to pick the riskier items first. For example, if you are trying to build the next Facebook on top of voice notes (as opposed to pictures) probably all you need to start testing your idea is a feed of some sort with the voice notes of all your friends. A profile with my education background or an event invite system is not something you need in order to test your idea. You are starting a company because you think there is a different and better way of doing things. That new way of doing things will be your riskier items. Test those first.

3)  Never lose sight of the product development cost structure. Since fast iterations are the key, the main item of the cost structure is time. Hence, you need to have a good sense of how long things take to get done. This is particularly difficult to get right in software development but every marginal improvement in accuracy on your end has a disproportional positive impact in your ability to launch a product that will work. Early product development ROI is a function of time spent per iteration  x positive delta in institutional knowledge. The more visibility you have on the time it takes to iterate, the more visibility you will have on the ROI of your iterations.

4) Before product/market fit use metrics to test Bold hypothesis. This will increase your chances of maximizing institutional knowledge per iteration. By bold hypothesis I mean educated guesses about users intent that will result in new features or removing features. In contrast, light hypothesis are changes that clarify and optimize current usage. Examples include changing the order of the items on the navigation bar, testing colors and copy and optimizing viral hooks. After market fit, you should make way for more light hypothesis but never abandoned the bold ones. You should always ask yourself: If I where to start this from scratch, how would I build it ?

5) Simplicity is key. You need to make sure that if people don’t use a feature it’s because they don’t like it, not because they don’t understand it or they can’t find it. Simplicity has an aesthetics quality to it, but it does not mean optimizing for aesthetics. In fact, people often will bundle aesthetic approval with product approval and what you need first is product validation.

I would be interested in learning what are the key product development guidelines that you always keep in mind.



These are the slides from my SELA 2010 conference at Stanford Univ.

Thoughts around innovation

Here are some ideas I have been playing around with that I thought might be of interest to anyone trying to build a new product or website:

Always focus on the tasks users are trying to accomplish, not the functions of the product or services currently being used.
It makes no sense to build a site framed under the premise that you are trying to enter the social networking space or that you are trying to build a better facebook. You have to think about the tasks that people undertake on such sites (sharing photos with friends, finding long lost friends, etc) and find a way to make that task better.

Making a task “better” means making it faster, in fewer steps or cheaper. If users take pics on their mobile, then sync the phone with their computer and then upload the picture from the desktop to the social network, having a one click button that posts the image directly from the phone makes the task of sharing pictures easier and hence better.

Not all innovation opportunities are created equal.
One way of thinking of the iPod is as a device that helps people carry with them as much music as possible. The main tasks that the iPod does for you are (a) help you mobilize your current music library easily, (b) allows you to store a lot of music in a small mobile device and (c) provides you with the navigation tools to find the songs you want as fast and as easy as possible. Now, if you ask someone how important memory is as a feature users are likely to say 10 out of 10. But if you ask them how satisfied they are with the current memory offer, they might say 8 out of 10. An iPod killer with 1 terabyte of memory is not really a game changer; saying memory is important does not equal saying that the more memory, the better.

The fewer the tasks you differentiate by, the better.
Explaining a user what your product or site does is very difficult. You understand it, your investors understand it, but users are far less patient. Moreover, even if users understand it, they are very difficult group of people to impress (very few people  said WOW! when they went to twitter for the first time… most of us needed the critical mass of tweet streams to jump on board). If you want to have a chance at users remembering your product or site, pick one feature or task you are helping users with and make sure you make it significantly easier to do.

Some recommended reading:

Something Really New: Three Simple Steps to Creating Truly Innovative Products
by Denis J. Hauptly

What Customers Want: Using Outcome-Driven Innovation to Create Breakthrough Products and Services

by Anthony Ulwick

First look at Reframeit (and how it can ease information overload on comments)

Reframeit , in their own words,  is “a browser extension that lets you comment right next to any text or image that you find on the web”.   Comments on news sites and blogs are clearly set up in an inefficient way.  By being placed at the end of the whole text, the comments either revolve around the main idea or if they dwell on different issues the comments quickly turn into a set of silo and discontinued discussions.
Threaded comments are a way of dealing with this issue, allowing commentators to set aside sub conversations. Yet, the individual who makes the comment pivots the conversation and if someone does not agree with the way the issue has been framed she or he may very well start another “pocket” of comments which can in effect be quite similar to the first.
Reframeit does the trick with a Firefox extension allowing you to comment on precise sections of the text.  This way the discussion revolves around specific sets of content. In a way, it works like crowd sourced notes on a text.  Image Reframeit integrated with the Kindle; you could for example have your whole class make notes on a given textbook.
In short, Reframeit allows us to better contribute and digest the stream of comments that adds up to the overloaded information available on the Web. There is already more to read that we can have; adding up the comments made on top of that enormous mass of text only increases the information overload. Hence, tackling information overload on the comments is both reasonable as well as useful.
One of the lessons reaffirmed with the Reframeit experience is that we are going to have to come up with innovative user interfaces to deal with the information overload.  I am hesitant as to how far browser extensions can take us, but it is clearly worth the try.

Excerpts and the new online news ecosystem

I recently read a interesting article on the nyt titled “Copyright Challenge for Sites That Excerpt”. Here are what I found to be the ideas of the article:

Generally, the excerpts have been considered legal, and for years they have been welcomed by major media companies, which were happy to receive links and pass-along traffic from the swarm of Web sites that regurgitate their news and information.

But some media executives are growing concerned that the increasingly popular curators of the Web that are taking large pieces of the original work — a practice sometimes called scraping — are shaving away potential readers and profiting from the content

“A lot of news organizations are saying, ‘We’re not willing to accept the tiny fraction of a penny that we get from the page views that these links are sending in,’ ” said Joshua Benton, the director of the Nieman Journalism Lab at Harvard

The legal disputes are emblematic of a larger question that has emerged from the Internet’s link economy. The editors of many Web sites, including ones operated by the Times Company, post excerpts from competitors’ content from time to time. At what point does excerpting from an article become illegal copying?

After reading it I remained wondering if people reading aggregated excerpts at the Huffington Post are getting their news there instead that on the New York Times and whether Ariana Hufington could do without the New York Times (that is considering she is indeed undermining the economic foundations of traditional news papers and traditional – data gathering, fact checkers – journalist).

To me it does not makes sense to do without the aggregated excerpts so  the question is what we can do to either make the New York Times sustainable under these new environment  or at least make the “traditional journalists” viable under these new conditions.
I will be getting my head around these issues.

Original article here

Seth Godin’s Five pillars of success

Here are the five pillars of success according to Seth Godin:

The five pillars of success

  1. See (really see) what’s possible
  2. Know specifically what you want to achieve
  3. Make good decisions
  4. Understand the tactics to get things done and to change minds
  5. Earn the trust and respect of the people around you

- Original post

The first point is tricky: if you indeed know what’s possible and what’s not, you will have a hard time innovating. If you don’t have a clue as to what is possible and what’s not, than the odds are you will end up holding to a thin branch at the edge of a 10 story building high cliff. So, what should we take away from this first point? Well, if you are looking to infuse technological innovation into a specific industry, you better know darn well the limits of technology.  Is the tools you use that you need to understand and master. You need to know how to use them and envision how it can affect the world around you.