Experimentation Matters – A Book Review
I recently read the book Experimentation Matters: Unlocking the Potential of New Technologies for Innovation by Thomke
In summary, the book argues that experimentation is the key to innovation and that successful companies put processes into place to encourage it. Aside from the writing style which is quite pedantic, the author makes several key points:
1. Anticipate and Exploit Early Information through Front-loaded Innovation Processes – by performing experiments early on in the development process (front-loading), one can build a better product faster rather than performing testing at the end of the process.
2. Experiment Frequently but do not overload your organization – more repetitive testing early-on in the development cycle can save money but there is a limit to how many experiments can be run.
3. Integrate new and traditional technologies to unlock performance – new experimental technologies enhance existing processes but rarely replace traditional processes altogether.
4. Organize for rapid experimentation – experiments must be done with rapid feedback or the benefit of learning is lost.
5. Fail early and often but avoid “mistakes” – early failures are desirable because they generate learning. Mistakes are defined as a misuse of resources that generate little or no learning.
6. Use projects as experiments – the book discusses several examples in which a company put out a tool that lets customers experiment on their own. This generates tremendous amounts of learning and innovation because now the company has empowered customers to create their own innovations based on the company’s tool.
I particularly like #5 because so often people judge the result of an experiment by whether or not it becomes a sellable product when the real value is in the learning. Sometimes we learn more from the failures than the successes. The goal of experimentation is to generate learning and if the project accomplishes that than it’s a success.
The author goes on to outline factors that affect learning by experimentation:
1. Fidelity of experiments – the degree to which a model and its testing conditions represent a final product, process, or service under actual use conditions.
2. Cost of experiments – the total cost of designing, building, running, and analyzing an experiment, including expenses for prototypes, laboratory use, and so on.
3. Iteration Time – the time from planning experiments to when the analyzed results are available and used for planning another interaction.
4. Capacity – the number of same fidelity experiments that can be carried out per unit time.
Strategy – the extent to which experiments are run in parallel or series.
5. Signal-to-noise ratio – the extent to which the variable of interest is obscured by experimental noise.
6. Type of experiment – the degree of variable manipulation (incremental versus radical changes); no manipulation results in observations only.
Finally, the author reminds us of the realities of new technologies:
1. Technologies are limited by the processes and people that use them.
2. Organizational interfaces can get in the way of experimentation.
3. Technologies change faster than behavior.
The only drawback in reading the book is the length. The author has a knack for stretching out a good story. What could be told in three or four pages ends up as forty pages or more. Nevertheless, the points are important for those working in Emerging Technologies. I recommend skimming it.
If you are working with Emerging Technologies, I would like to talk with you further. Please contact me at hall.martin@ni.com.
Best regards,
Hall T. Martin
In summary, the book argues that experimentation is the key to innovation and that successful companies put processes into place to encourage it. Aside from the writing style which is quite pedantic, the author makes several key points:
1. Anticipate and Exploit Early Information through Front-loaded Innovation Processes – by performing experiments early on in the development process (front-loading), one can build a better product faster rather than performing testing at the end of the process.
2. Experiment Frequently but do not overload your organization – more repetitive testing early-on in the development cycle can save money but there is a limit to how many experiments can be run.
3. Integrate new and traditional technologies to unlock performance – new experimental technologies enhance existing processes but rarely replace traditional processes altogether.
4. Organize for rapid experimentation – experiments must be done with rapid feedback or the benefit of learning is lost.
5. Fail early and often but avoid “mistakes” – early failures are desirable because they generate learning. Mistakes are defined as a misuse of resources that generate little or no learning.
6. Use projects as experiments – the book discusses several examples in which a company put out a tool that lets customers experiment on their own. This generates tremendous amounts of learning and innovation because now the company has empowered customers to create their own innovations based on the company’s tool.
I particularly like #5 because so often people judge the result of an experiment by whether or not it becomes a sellable product when the real value is in the learning. Sometimes we learn more from the failures than the successes. The goal of experimentation is to generate learning and if the project accomplishes that than it’s a success.
The author goes on to outline factors that affect learning by experimentation:
1. Fidelity of experiments – the degree to which a model and its testing conditions represent a final product, process, or service under actual use conditions.
2. Cost of experiments – the total cost of designing, building, running, and analyzing an experiment, including expenses for prototypes, laboratory use, and so on.
3. Iteration Time – the time from planning experiments to when the analyzed results are available and used for planning another interaction.
4. Capacity – the number of same fidelity experiments that can be carried out per unit time.
Strategy – the extent to which experiments are run in parallel or series.
5. Signal-to-noise ratio – the extent to which the variable of interest is obscured by experimental noise.
6. Type of experiment – the degree of variable manipulation (incremental versus radical changes); no manipulation results in observations only.
Finally, the author reminds us of the realities of new technologies:
1. Technologies are limited by the processes and people that use them.
2. Organizational interfaces can get in the way of experimentation.
3. Technologies change faster than behavior.
The only drawback in reading the book is the length. The author has a knack for stretching out a good story. What could be told in three or four pages ends up as forty pages or more. Nevertheless, the points are important for those working in Emerging Technologies. I recommend skimming it.
If you are working with Emerging Technologies, I would like to talk with you further. Please contact me at hall.martin@ni.com.
Best regards,
Hall T. Martin
posted by Hall T. Martin at 9:41 AM
0 Comments:
Post a Comment
<< Home