Step 6.1

Test beta prototype with target users under controlled lab conditions.

Primary findings

Secondary findings

Primary findings

Barriers

Data collection methods affect the quantity and quality of feedback. For example, less tech savvy patients interact less with touch screens or tablets. 
Case study findings
(View full citation)

Users, and in particular consumers, are often not able to articulate their need for really innovative products, because their thinking is framed by the products currently on the market. Also, reactions of potential users when confronted with highly innovative new product concepts are not always reliable, as they may find it difficult to imagine the use of these innovative products in their daily lives.
Literature review.
(View full citation)

Carriers

Studying user identities and individual motivations is a useful application of researching open innovation.
Literature review findings
(View full citation)

Complaint and issue feedback databases store and quantify issues with devices and processes in a more standard method than human resource led initiatives, which are less empirical and more likely to give employees qualms about sharing.
Postmarket questionnaire results
(View full citation)

Guidelines for effective Beta Test program management: 1) Product design should be frozen prior to Beta testing. 2) Effective Beta testing programs recognize the full set of benefits possible (as shown in Figure 1). 3) Maximum utility of Beta testing is derived from recognizing the segmentation of the market, and devising ways of making the Beta tests as close approximations to real-world experiences. 4)As a general control rule, grow the number of Beta sites over time; begin with sophisticated customers who have good relations with the company.
Literature review, analysis of twenty-one programs, and four in-depth field investigations.
(View full citation)

Teams may meet with customers directly in focus group settings, to achieve a shared team understanding of who the customer is and what the customer wants.
Literature Review — 1969  — 1994.
(View full citation)

Methods

As ExxonMobil's NPD process progresses, marketing activities scale up from limited sampling with key customers to much more significant work in trials with customers using the new product.
Industry experience.
(View full citation)

Conduct market research, business analysis, prototype development and trials to reduce uncertainty, acquire information and ultimately increase new product development success.
Literature Review
(View full citation)

Include consumers in Beta prototype testing activities
Survey.
(View full citation)

It is important to collect and assess market and competitive information in order to understand customer needs, wants and specifications for the product; to know customers' price sensitivity; to understand customers' purchase decisions; and to learn about competitors' strategies, strengths and weaknesses.
Case studies conducted on over 600 new product launches in the U.S. and in China.
(View full citation)

Prototype testing can be used to uncover and correct problems. However, the small samples used may not be representative of the entire population. Information may be more useful for troubleshooting than for predicting success.
Literature Review
(View full citation)

Six general themes for NPD best practice are described: (1) Instill a strategic, long-term orientation toward NPD. (2) Have a formal portfolio management process. (3) Implement a formal NPD process supported by a discipline to adhere to this process. (4) Conduct market research proactively. (5) Use cross-functional teams. (6) Utilize standardized criteria and metrics.
Generated from published studies on benchmarking data.
(View full citation)

Techniques for gathering voice of the customer information include: use of customer complaints, internal market research, focus groups, one-to-one interviews, phone interviews, contextual inquiry, customer behavior studies, and perceptual mapping.
Literature review and case studies
(View full citation)

Usually with beta testing, working prototypes are placed with selected customers in order to test the influence of 'environmental factors', as well as the level of customer satisfaction. The results from these tests are used in order to refine the product further and to eliminate 'bugs'.
Literature review.
(View full citation)

Utilization of market research early in the new product development process, and continuing throughout the entire development phase is critical to ensuring success.
Survey data.
(View full citation)

Voice of the Customer Information as a Best Practice for the NPD process: 1) Market and buyer behavior studies are a valuable source of information for planning the market launch. 2) Market research as a tool to help define the product. 3) The customer or user ought to be an integral part of the Development process. 4) Identification of customers or users real or un-articulated needs and their problems, is considered fundamental to voice-of-the-customer research, and should be a key input to product design. 5) Working with highly innovative users or customers.
A quantitative survey of 105 business units, supported by team's experience in NPD modeling, consultation, application and analysis.
(View full citation)

Tips

Pharmaceutical companies are heavily scrutinized and collect extensive testing data which is useful for incremental development. Producers in other less regulated disciplines (devices, clinical care procedures) could benefit from such a well-documented R&D process.  
Postmarket questionnaire results
(View full citation)

Involvement of customers with strong past ties will result in the development of products with higher competitive performance than those that involve customers with no or few past ties. However, this practice is most effective when developing incremental products, rather than highly innovative products.
Survey of 137 new product development projects.
(View full citation)

Overall, product quality, relative to competitors (as defined by the customer), was ranked as the most important factor in how the product performed in the market. Product "value" compared with that of competitors and the clinical effectiveness of the product were then ranked as the next most important aspects, respectively.
Survey results from medical device manufacturers.
(View full citation)

To improve short-term competitive performance of products in the marketplace, projects to develop incremental new products should involve homogeneous groups of customers.
Survey of 137 product development projects.
(View full citation)

Secondary findings

Carriers

To mitigate power inequalities and transcend the different cognitive and cultural worlds of participating stakeholders, an independent facilitator should guide the process and fulfill an intermediary role as a knowledge broker.
Source: Sperling & Ashby (2001); Hargadon (2002). In: Klerkx, Laurens & Leeuwis, Cees (2007)

Methods

Beta testing — Usability testing done by customers at the manufacturer's facility as an input to the design process.
Source: Prasse (1991). In: Dolan, RJ & Matthews, JM (1993)

Customers can evaluate the product's interface with existing operations and feedback can benefit supplying firms because such feedback alerts sellers to buyer's perceptions of salient product attributes and reduces market uncertainty.
Source: Stump, Athaide & Joshi (2002) p. 444.. In: Koufteros, X., Vonderembse, M. & Jayaram, J. (2005)

For demand articulation, dialogue between end-users and producers of knowledge and information, as well as other relevant stakeholders, should take place throughout the innovation process.
Source: Douthwaite et al (2001). In: Klerkx, Laurens & Leeuwis, Cees (2007)

Listening to the customer's needs early in the product development process has been identified as being critical for eventual market success.
Source: Griffin & Hauser, 1993; Hauer & Clausing, 1988; Nayak & Chen, 1993. In: Datar, S., Jordan, C.C., Kekre, S., Rajiv, S. & Srinivasan, K. (1997)

Marketing-R&D interface theory suggests that without the interface between these two functions, marketing knowledge would be underused, potentially resulting in unsuccessful products.
Source: Griffin & Hauser (1992); Gupta, Raj, & Wilemon (1986); Song & Dyer (1995); Song & Parry (1997). In: Li, T. & Calantone, R. J. (1998)

One way to integrate the “voice of the customer” is to implement quality function development (QFD). QFD emphasizes identifying customer needs and mapping them to specific product characteristics. A series of interaction matrices translates customer needs into process step specifications.
Source: Hauser & Clausing (1988) and Griffin (1992). In: Spivey, W.A., Munson, J.M., & Wolcott, J.H. (1997)

The market research methods most frequently cited by Fortune 500 companies include: focus groups (used by 68% of companies surveyd); limited roll out (42%); concept tests (26%); show tests and clinics (19%); attitude and usage studies (19%); conjoint analysis (15%); Delphi (9%); quality function deployment (9%); home usage tests (9%); product life-cycle models (8%); and synectics (8%).
Source: Mahajan and Wind, (1992). In: May-Plumlee, T. & Little, T.J. (2006)

Tips

Among computer and medical equipment manufacturers, successful new products incorporated greater use of market information in the NPD process, while failed products used less.
Source: Ottum & Moore, 1997. In: Suwannapron, P., Speece, M (2003)

Companies must interact with customers when technology is experiencing rapid change.
Source: Day & Wensley (1988). In: Li, T. & Calantone, R. J. (1998)

The greater the customer demand, the greater the need for integration of marketing knowledge and R&D knowledge in order to fill the gap between customer requirements and product offerings.
Source: Gupta, Raj & Wilemon (1986). In: Li, T. & Calantone, R. J. (1998)

Understanding customer needs and preferences is a key factor in new product success.
Source: Cooper and Kleinschmidt, 1995, 1996. In: Joshi, A. W. & Sharma, S. (2004)