Category Archives: Prototyping

Prototyping Fidelity and the Design Turing Test

Posted on by
Reply

Prototyping is a topic I have been involved with for 30 years [c.f. 2, 3]. Prototyping technologies and conceptual advances have changed dramatically over the years, but the two reasons we must prototype have not, and will never change.

    1. Prototyping encourages user involvement and joint ownership of projects.
    2. Prototyping facilitates the mutual and concurrent learning processes of users and designers.

Numerous conceptual advance have been made to redefine and assist with modeling and designing systems for consumers and users. One of the most important concepts is related to the coarseness of the prototype. This coarseness is called prototype fidelity.  I would like to draw on the concept of prototype fidelity and relate it to our earlier research.

Adaptive Design and Prototype Fidelity

In earlier papers we identified three levels of prototyping. Level 1 was input and output design. Level 2 prototyping was called heuristic design and it involved the addition of functional operations and limited interactions to input and output design. Level 3 prototyping was called adaptive design. A prototype built using adaptive design was an actual working systems that was improved forever in an adaptive iterative process. The concept of adaptive design was articulated by Peter Keen over 35 years ago and defined as a process of learning and evolution [4].

I like the concept of prototype fidelity as discussed by Catani and Biers much better than the levels of prototyping [1]. Prototype fidelity is a clean, easy to understand concept. An excellent article by Laura Busche in 2014 captures the idea of prototyping fidelity and illustrates why developing a low fidelity prototyping is the first step in user-oriented design.

The gold standard for a prototype is to develop a high fidelity prototype that passes for what I like to refer to as the Design Turing Test (DTT).

Although we do not yet have a specific means for determining the particular characteristics that make a user interface high or low fidelity, we can loosely define fidelity by analogy to the Turing test. To the extent that a person using the prototype cannot distinguish it from the final system, the prototype is high fidelity. If the prototype can readily be distinguished from the service, then fidelity is low. [5]

In essence, prototype fidelity refers to the degree to which the prototype looks and functions like an actual system or product. We will refer to the levels of fidelity, the degree to which a prototype reaches the Design Turing Test hurdle, as the Lo-Fi prototype, the Hi-Fi prototype and the Functional working prototype. High fidelity, functional prototypes should pass the DTT.

Lo-Fi Prototypes: A low fidelity prototype could be a sketch of the input and output screens or a process diagram of how the product works. It can also be a thought experiment with a brief description. The key is communicate the essence of the concept.

Low fidelity prototypes can be done using pencil and with a drawing program, however, I am such a terrible artist that I often turn to Grafio for Alpha prototypes. My favorite tools for low fidelity prototyping are paper and pencil, any tablet drawing program, PowerPoint, Visio, and Grafio in most situations. A new on-line and desktop tool called Creately shows great promise as a low fidelity prototyping tool.

Hi-Fi Prototypes: A higher fidelity prototype would include input and output mock up screens and it should also illustrate hierarchical and navigational relationships between the various screens. High fidelity prototypes are usually developed using wire frames. A wire frame is just an image or picture of a tablet, a smart phone or a computer screen.

My turn-to tools for high fidelity prototyping are InVision for realistic wire frame mock ups, and PowerPoint, and Excel.  You must pay for the more advanced features of inVision, but you can try it out on a small project to see how it works.  I also like Prototyping on Paper or POP. This is one of the easiest tools for taking pictures of paper and pencil sketches and then making them linkable.

If tangible products are being designed computer aided design (CAD) tools can be used. Free versions of these tools are reviewed here.

Functional Prototypes: The highest fidelity prototype is supposed to act and look like the real system or product, even though it may eventually be re-coded or rebuilt using different technologies. Functional prototypes are developed using a technology that represents the flow and dynamics of the screens, as well as supporting the background processes that support the application. Functional prototypes should usually be capable of passing the prototyping Turing test, in that the person using the prototype should not be able to distinguish it from the final system.

I do not have a favorite tool for Functional prototypes. They are typically implemented in a native language such as Xcode for IOS iPhone apps and in Java-like languages for Android apps. I usually have a developer implement Alpha prototypes. They are rarely static and are continuously redesigned in an iterative fashion.

Example of Lo-Fi and Hi-Fi Prototyping

The following figure illustrates the low fidelity and high fidelity prototypes for a system for searching and rescuing a lost climber in a remote location. Figure 1 is a very terrible drawn low fidelity prototype using a pen and pencil. My collaborator took that drawing and developed a wire frame iPad mock up using inVision.

The first version of the high fidelity prototype developed in InVision was not the idea that I tried to convey. My drawing was terrible. So I used Grafio to draw a more refined diagram, a higher fidelity, but still at low fidelity prototype, that is illustrated in Figure 2. My collaborator then went on to develop the high fidelity prototype that is illustrated in Figure 3.

The  high fidelity prototype in Figure 3 is a dynamic wire-frame that was developed using InVision. If you click on a name, for example “Kate”, you can get her health and movement status. All of the members of the rescue have hot links to their movement and health status. This application would eventually be programmed in XCode or Java, depending on the smart phone platform.

The key take away from the back and forth process of developing low fidelity and high fidelity prototypes is that developing a prototype facilitated the mutual and concurrent learning processes for both of us. It was an iterative on-going reciprocal learning process.

As illustrated in Figure 4, the prototyping process is non-linear and iterative. The process can be low fidelity, high fidelity, low fidelity, fidelity, and then the creation of an actual  functioning prototype that passes the design Turing test. Or the process can begin with a low fidelity and then go directly to the development of the functional prototype. The order is determined by the complexity of the application and the degree to which the designer and user understand each other.

Conclusion

The key to design is to get the users, consumers, designer, developers and management on the same page. Prototyping encourages learning and exploration. The ideal prototypes passes the Turing Design Test. It has never been easier to develop virtual world prototypes. There are numerous 3D printing tools and digital tools for designing products that clear the Design Turing Test hurdle. Take advantage of them. It is a prototype or perish world.

Prototyping Process

Figure 4: Prototyping is a Continuous Process

 

Additional Material on prototyping concepts  and prototyping tools

References

  1. Catani, Michael B., and David W. Biers. “Usability evaluation and prototype fidelity: Users and usability professionals.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting. Vol. 42. No. 19. SAGE Publications, 1998
  2. Cerveny, Robert , G. Lawrence Sanders and Edward J. Garrity. “The application of prototyping to systems development: A rationale and model.” Journal of Management Information Systems (1986): 52-62.
  3. Cerveny, Robert P., Edward J. Garrity, and G. Lawrence Sanders. “A problem-solving perspective on systems development.” Journal of Management Information Systems (1990): 103-122.
  4. Peter G. W. Keen. 1980. Adaptive design for decision support systems. SIGMIS Database 12, 1-2 (September 1980), 15-25. DOI=http://dx.doi.org/10.1145/1017654.1017659
  5. “Virzi, Robert A. “What can you learn from a low-fidelity prototype?.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting. Vol. 33. No. 4. SAGE Publications, 1989, pp 224.
Figure 1

Figure 1: Low Fidelity Version 1

Figure 2

Figure 2: Low Fidelity Version 2

Figure 3

Figure 3: High Fidelity

Figure 3

Figure 3: High Fidelity

Selecting the best start-up idea and project feasibility

Posted on by
2

This past year I have had numerous teams that had two or three startup ideas. Some of the teams want me to make the decision on what project to select. Most of them, however, just want some guidance on what questions to ask.

In its simplest form, the questions should relate to the economic, market, financial, operational and technical feasibility aspects of the idea, along with the characteristics of the founder(s). Here is a list of 14 questions to consider when weighing-in on an idea for a start-up (Figure 1).

Startup Potential

Figure 1: Questions related to Start-Up Potential

Questions related to market and business sustainability

Many investors and founders look at the size of the market and the potential profits as the critical criterion for investing in a startup. But a startup must be able to capture part of that market and reach those customers through a marketing campaign. In order to become a viable sustainable business, funds are needed to launch the business, regardless of whether they come from the founders, family, friends or investors. If the startup costs for the business are low relative to the availability of funds then the business may eventually exist as a sustainable entity. The enemy is the burn rate or burning cash and not having funds to pay the bills. Long term sustainability is very difficult unless there is some way to obtain recurring revenues in the form of complementary products and services and refreshing the product line through R&D.

  • Q1.  How large is the market?
  • Q2.  Will the business be able to capture some of the market and realize a profit?
  • Q3.  Can the customers be readily identified and reached?
  • Q4.  Can funds be secured for starting the company?
  • Q5.  Are the startup cost relatively low?
  • Q6.  Is the time till the company is sustainable short?

Questions related to building the product or service

These questions relate to manufacturability, which is the ability of the startup to make the product and in the case of services, to set up processes that will be used to deliver a service.  These questions are related to the ability of a start-up to develop a viable supply chain.   A good indication of degree of manufacturability is whether or not a realistic prototype can be built. The key is to be able to obtain raw materials, components and people and to design processes for delivering a product or service. Scalable business are desirable because they can grow and contract with changes customer preferences and disturbances in the economy.  When a product or service involves emerging technologies then research and development will be an important driver of manufacturability and product design.  Products and services requiring high levels of initial investment of R&D requirements are inherently risky and contribute to cash burn, even though they can be the ticket to hyper-profitability.

  • Q7.  Manufacturability: Is the product or service manufacturable?
  • Q8.  Can a prototype or mockup of a product or service be built?
  • Q9.  Can employees be secured with the necessary expertise and skills?
  • Q10. Are the materials and components available to build the product or service?
  • Q11. Is the business easily scaled for growth and retrenchment?
  • Q12. Are there minimal initial R&D requirements for manufacturability and product design to get the business going?

Questions related to founders and team composition

There is some evidence that the composition of the startup team will have a positive impact on survival. But the evidence is confusing. Diverse teams can in some instances improve firm performance in a very competitive environment. But sometimes, teams that are homogeneous can act quicker when they are well-aligned on strategic decisions. It appears that the team composition should depend on the characteristics of the industry and the product being developed.

Many pundits and academics have identified their own typology of  what founding teams should look like (e.g. go here for blog discussions, here for Steve Blank’s typology and go here for recent academic research). A high-tech startup team might include a founder that is a visionary/strategist, a technologist/engineer founder, a designer/prototyper, and a marketer/hustler (Figure 2). It also appears that founder teams with about 3 or 4 members are more successful than a solo team or teams with more than 4 members.

If the founders are familiar with the product then many of the questions related to the market and the manufacturability of the product or service are answered or at least can be addressed. But sometimes, individuals and companies have to jump to a very dissimilar product or service in order to survive. Hard work, and learning about and exploring new product lines by prototyping can lead eventually to familiarity and insight. And of course, ignorance that is fueled by enthusiasm has fueled many inventions. Never count out enthusiasm, determination and hard work.

  • Q13. Do the founders have the right mix of expertise and do the team members complement each other?
  • Q14. Are the founders enthusiastic and willing to work hard?

How to use the questions

The questions are set up so that if you answered “yes” rather than “no” to a question, there would be less project risk. Lower risk usually means that the project is feasible. But that does not mean that risk aversion is the goal. There is usually a trade-off between the level of risk and potential returns. There are numerous examples where high risk projects produced numerous millionaires and hyper-profitability.

I had thought about developing a simple scoring model where you could compare different projects using the questions, but I think that would also be a mistake. Selecting the project with the highest score might eliminate a project that could change the world or perhaps just lead to financial security.

It is important that we not to eliminate ideas too early by letting preconceived biases get in the way of creativity (see “How to Let 999 flowers Die”).  Sometimes it takes a while to get people to understand what you are trying to do because you have not been a good communicator.  Sometimes, however, it takes a long time for people like me to “get it.”

The march to innovation should be in between a turtle’s and a rabbit’s pace. You have to give the idea time to mature, but not move too slowly because the idea will wilt on the vine. I suspect a kindergartner pace would work best. Just don’t spend too much time pondering the questions and don’t just race from question to question.  Ponder, race, prototype, ponder, race, prototype, ponder race prototype ….

Typology

Figure 2: Potential Areas of Expertise for Founders

Using Thought Experiments to Develop Innovative Products and Business Models

Posted on by
1

The most intriguing aspect of Walter Isaacson’s biography on Einstein was his numerous discussions of thought experiments. Einstein conducted many, many thought experiments. Thought experiments helped Einstein to understand and disentangled complex concepts and to develop theories.

“He made imaginative leaps and discerned great principles through thought experiments rather than by methodical inductions based on experimental data. The theories that resulted were at times astonishing, mysterious, and counterintuitive, yet they contained notions that could capture the popular imagination: the relativity of space and time, E=mc2, the bending of light beams, and the warping of space. [2007, W. Isaacson, p. 6]”

“Over the years, he would picture in his mind such things as lightning strikes and moving trains accelerating elevators and falling painters, two-dimensional blind beetles crawling on curved branches, as well as a variety of contraptions designed to pinpoint, at least in theory, the location and velocity of speeding electrons. [2007, W. Isaacson, p. 27]”

Thought experiments need to be rhetorical in a good way.  A good thought experiment should be artful, eloquent, effective and persuasive in conveying the ideas, but not too pretentious or bombastic There is a downside to rhetoric in that the innovator, the scientist or the storyteller may be too good in developing  a colorful and vivid description at the expense of gathering facts and testing the veracity of the theory.  Sometimes thought experiments can actually confuse more than illuminate. The thought experiment may not work because of differences in the eyes of the beholder, because it needs more refinement or because it is invalid. Here are some guidelines for developing your own though experiment in order to tell a clear and cogent story.

  1. You need to be able to explain your concept in 3 or 4 sentences.
  2. You should eventually write your ideas in a paragraph.
  3. You need to use pencil and paper to illustrate your innovation. (Tablets are also Ok, but the interface should not get in the way of your creativity.)
  4. You should try to present your thought experiment on 1 page. It should be self-explanatory.
  5. You should present your thought experiment to many people.
  6. You should refine your thought experiment over and over after you receive feedback.
  7. You should sometimes return to your original iterations and ignore some of the feedback.
  8. You should let your ideas incubate. Layoff the idea for several days and do this often.
  9. You should be obsessed with the project and the compulsive about the details.
  10. Go back to 1, 2, 3, 4, 5, 6, 7, 8 or 9. Remember this is a nonlinear iterative process

I believe that thought experiments are the genesis of all innovations and creative process. This includes new products and new business models. Sometimes scientific thought experiments are verified by real-world experiments, data collection, and mathematical proofs.  The innovator progressively validates a thought experiment with the pitch and the business plan. The plan and the pitch are the refinement and incarnation of the entrepreneur’s thought experiment. The ultimate validation is when the business or the innovation goes live.

Figures 1 and 2 illustrates the migration of a thought experiment involving tracking individuals, pets and expensive assets. The initial drawing and narrative was started on Sunday evening and 3.2 was completed by 7am on Tuesday. Feedback from several individuals helped to refine the drawing.  Most of the time was spent futzing around with the interfaces of the two apps and locating symbols and graphics. The idea was derived from several student projects and from watching my son crack our Wi-Fi password in less than an hour using Backtrack. I think similar cracking code could be implemented in a very small unobtrusive device.  CPU, memory, Wi-Fi, cellular and GPS chipsets are shrinking in size and price. This technology has undoubtedly been developed in some form. But, there is always room for improvement and a thought experiment can be used to drive that process.

Here are some additional references on thought experiments:

  • Walter Isaacson’s 2007 biography entitled Einstein: His Life and Universe, Simon Schuster is one of my favorite books. It does a good job of describing how thought experiments influenced Einstein’s ideas.
  • As expected Wired Magazine has an interesting twist on thought experiments by Greta Lorge
  • Great animations of thought experiments can be found at Brain Pickings. BTW some of these animations don’t really help me to understand the ideas.
  • Very nice overview of thought experiments can be found at the Stanford Encyclopedia of Philosophy
  • Horowitz and Massey present a detailed discussion of how thought experiments have influenced scientific reasoning and philosophy.
  • Here is an interesting discussion of Einstein’s chasing a beam of light thought experiment by John Norton and go here for more of his philosophy of science work.
  • Another good philosophy book on the what, how and whys of thought experiments was written by Roy Sorenson.
  • Here is an illustration of a brute force method for cracking WiFi access points. This type of tool is available on a number of free digital forensics and penetration testing tools such as Backtrack. No, we are not safe from intrusions.

Figure 1: Thought Experiment 1.0

Kidnapping 1.0

Figure 2: Thought Experiment 3.2

Kidnapping 3.2