PulseUX Blog

Should McKinsey Design Retract Its “Business Value of Design” Report and Related MDI Index?

By Charles L. Mauro CHFP and Paul W. Thurman DBA, MS

Two weeks ago my research partner, Paul Thurman, and I published an analysis of the McKinsey Design “Business Value of Design” (BVD) report and MDI Index. That analysis articulated the numerous research problems with the report and put forward the opinion that its findings were, from a research perspective, both inaccurate and unsupportable. For any of you who have not read the original report and then the critical analysis, you should consider doing so as the combination is both insightful and, at the same time, deeply concerning. Should McKinsey retract its BVD report? Here are the issues…you decide.

Critical Analysis: Why McKinsey’s “The Business Value of Design” Report Ultimately Misstates the Value of UX Design and Design Overall

Charles Mauro CHFP

By Charles L. Mauro CHFP and Paul W. Thurman DBA, MS

Recently, there has been an ongoing fascination with McKinsey’s acquisition of design agencies and how the renowned management-consulting firm would turn a profit from such investments. One would expect such an integration of high design and management consulting by a world-class firm like McKinsey to produce industry changing insights and expertise options. However, if the recently published and highly publicized McKinsey Design industry report “The Business Value of Design” (hereafter referred to as BVD) and associated McKinsey Design Index (hereafter referred to as MDI) is any indication, the grand vision may need reconsideration. It appears, based on critical analysis, that the breathless excitement presented by McKinsey Design in the BVD release video, claiming to establish for the first time “individual design actions that leaders have taken…tied back to business performance” is not only inaccurate, but unsupportable. Sadly, given the business gravitas of McKinsey and its obvious financial resources, it is clear that this a major missed opportunity for McKinsey Design, and design overall.

Important Note: Prior to publication of this review, the authors attempted to contact McKinsey Design BVD and MDI authors, but received no response. It was our intention to seek clarification and elaboration on the findings and supporting data before publication of this review. We welcome a response to this analysis by the McKinsey Design research team. The business value of design is of value to all who engage in the theory and practice of design and design science.

Understanding Costs And Risks For HFE Usability Studies — Part 1: Testing In-House

Charles Mauro CHFP

MedDeviceOnline-Author-Image — *Image courtesy of MedDeviceOnline*

Executing HFE studies internally is often far more complex and time-consuming than drug delivery device development teams realize. MUS recently published part one of a two-part article detailing the factors that need to be considered by device development teams in order to produce high-quality usability testing data appropriate for decision-making and HFE optimization while maximizing cost-effectiveness.

The U.S. Food and Drug Administration (FDA) is increasing demands^[i],[ii] for drug delivery device developers to demonstrate the human factors engineering (HFE) effectiveness of their devices and related labeling systems. Therefore, rigorous HFE usability testing is required to ensure a given medical device meets HFE standards prior to submission for FDA approval.

Read Full Article

Keep an eye out for part two of this series in which we will describe in detail how device development teams can best understand and manage the costs of high-quality usability testing, should they decide to retain an independent testing agency.

BREAKING NEWS From Mauro USABILITY Science – 12/13/21

Charles Mauro CHFP

For IMMEDIATE Release: Geoffroy B. Zose Ph.D., the CEO of NPE Corporation, spoke last week at the annual meeting of the International Society for Human Factors Science In The Design of Global Package Distribution Systems held in Copenhagen. He indicated that his firm’s recent migration from the human-based, dead-reckoning global navigation and package delivery system to Google Maps point-to-point navigation interface was causing serious concern based on a recent usability test conducted with a professionally recruited and validated package delivery crew last month in Norway. He summarized the recent findings from the aforementioned summative usability testing sequence as follows:

October 2018 Panels and Speeches by Charles L. Mauro

Charles Mauro CHFP

Charles L. Mauro CHFP Delivers Keynote Speech at HFES International Annual Meeting 2018 in Philadelphia. Charles L. Mauro CHFP was the invited keynote speaker at the recent Human Factors and Ergonomics Society (HFES) International Annual Meeting held in Philadelphia, PA on October 2^nd, 2018. The speech titled: “Human factors engineering innovations in the age of Apple v. Samsung: How HF/E practice can add major value to the intellectual property assets of corporations and academic institutions” was well-received by the audience made up of HFE professionals from both academia and industry. The keynote talk focused on a detailed analysis of the far-reaching impact of the recent Supreme Court decision and its impact on device design IP theory and practice. The talk explored the use of neuroscience-based testing methods to determine infringement and to provide rigorous research-based support for IP litigation.

Charles L. Mauro CHFP Invited to Speak on Prestigious Panel at Design Law 2018 Conference in Washington, DC. It is no secret that design patent law is in a major period of upset and reformation. The dramatic changes are the direct result of the recent Supreme Court decision in several intellectual property cases, including Apple v. Samsung. The prestigious panel, including Christopher J. Renk of Banner & Witcoff Ltd., John Froemming of Jones Day and Nika F. Aldrich of Schwabe, Williamson & Wyatt, was focused on the complex issues of damage calculations once infringement has been found by the jury. Mr. Mauro covered theory and practice applied to testing the impact of visual design on consumer decision making and related research.

(Photo by D. Barnes)

Recent Research: Real-World Usability Testing

Charles Mauro CHFP

MUS Executes Complex Multi-City Study to Explore Disparities in Actual Use of Complex Drug Delivery Devices vs. Expected HFE Performance Based on Summative FDA Study Data.

There is an interesting phenomenon occurring in the actual real-world use of more complex drug delivery devices by patients and HCPs. The problem is there is a wide disparity between how the device performs during summative HFE testing for FDA submission and the surprising number of reported usability problems recorded after the device actually gets into the hands of patients in real-world settings. This disparity is reason for concern because a dramatic decrease in HFE performance by a device can signal significant loss of clinical effectiveness. To address this question MUS has recently executed a complex multi-city HFE field analysis to determine the source of such disparities and to aid major clients in understanding the structure and basis for identifying and resolving such outcomes during formative and summative studies. The learnings are being applied to updated and more robust summative testing methods that will deliver to the FDA more reliable HFE performance data. Request more information.

Apple v. Samsung Update

Charles Mauro CHFP

Apple-Samsung-Court-Image

Back to Square One The landmark Apple v. Samsung matter is far from over. It is a story both tragic and hopeful. Here is my summary of the case as it currently stands on appeal before Judge Koh in California.

Involvement of The Professional Design Community and Related Amicus Brief Some of you may have signed the Supreme Court amicus brief in the Apple v. Samsung case that Charles Mauro co-wrote together with the legal team at Orrick. Thank you for your support.

Impact of Our Design Amicus Brief In October of 2016, the Supreme Court heard oral arguments from Apple, Samsung, and from the US Department of Justice as a neutral party. All justices read our amicus brief signed by numerous IDSA members and other world-class designers including Dieter Rams, Norman Foster, Terance Conran, fashion designer Calvin Klein and design heads of many leading design studios across the globe. The contents of the brief were referenced in the proceedings. The amicus received very wide exposure in the design and business press. In fact, the brief has become required critical reading for design students at leading design programs on a global basis.

What Happened At The Supreme Court Hearing During the actual proceedings in Washington DC, quite unexpectedly, Apple agreed with Samsung on the primary question before the court. This was a major surprise to everyone who worked to blunt the attack by Samsung. In short, the two parties agreed that it would be conceptually possible for damages in design patent cases to be calculated on less than the total profits obtained from the sale of the entire product. As you may recall, the total profits method for calculating damages has been in place since the late 1890s and was the only point which Samsung had asked the court to rule on.

The Not-So-Surprising Outcome Left with no meaningful dispute between the parties (a very rare event), the justices voted 3 months later in their final ruling that damages could be calculated based on the relevant component part of designs to which the design patent had been applied instead of to the total profits of the entire product. The ruling by the Supreme Court essentially gutted the prior damages legal framework and with it a good deal of design rights IP on a global basis. However, the ruling was no surprise given that both Apple and Samsung agreed on the key legal question. This ruling left the global design profession in disbelief.

Why Professional Usability Testing Matters

Charles Mauro CHFP

How Do Development Teams Produce World-Class Products? For consumer-facing hardware or software to have high levels of usability and UX quality, they must undergo rigorous professional usability testing before design freeze. Such professional testing must make use of a wide range of mockups, simulations, and functional prototypes. It turns out that such testing is a fundamental requirement of all consumer-facing hardware and software systems that will eventually deliver a high degree of consumer engagement. One can often predict the potential usability and UX performance of a hardware device or software interface simply by examining the development process and the degree to which professional human factors science and unbiased professional usability testing has been applied during development.

Investors Don’t Link Poor Usability With High-Risk Usability and feature/function engagement are criteria that are rarely investigated by institutional investors when assessing the relative risk of making an investment decision. Defective usability has played a major role in the failure of numerous recent high profile products including the now infamous SNAP Spectacles. A few simple questions by investors during SNAP’s recent IPO roadshow would have likely revealed a failure to employ human factors research and professional usability testing during development of the SNAP Spectacles. More generally, when management is asked whether or not their products have undergone professional user testing and feature/function engagement testing they will often respond in the affirmative. Many executives wrongly assume that running a survey, or obtaining feedback from a few employees or a small group of consumers constitutes professional usability testing. It does not.

Design Does Not Replace Professional Usability Testing Professional usability testing is not the same as UX Design, Industrial Design or Design Thinking which are concept generation frameworks. Professional usability testing is a concept verification methodology. In order to have confidence that what has been developed will be usable and engaging one must apply a meaningful measure of marketing science. Professional usability testing is, first and foremost, a science-based process that is designed to produce unbiased and robust data on products and software user experience performance. In professional usability testing programs, there is always a need for statistical confidence that can aid management in making objective decisions about the usability and feature/function engagement of hardware or software before design freeze. Continue Reading…

How 4 Simple Usability Heuristics Could Have Saved SNAP Glasses

Charles Mauro CHFP

Wall Street And Hardware SNAP is the quintessential example of spinning gold from code. This is not to take away from SNAP’s apparent success as a virtual social media platform and possible business performance. Although the quarterly reports are, in real cash flow terms, terrifyingly low for such a staggering market cap, and marketers are not flocking to SNAP as predicted. We will leave that issue to the spreadsheet wizards at the investment banks that pumped SNAP stock to a heady IPO. Apparently during the pre-IPO roadshow, SNAP indicated that hardware would be a major part of its value proposition going forward. This was a big ask by the investment banks and a big promise on the part of SNAP. Based on a detailed usability and feature/function allocation analysis it is clear that for the most part SNAP Spectacles (Glasses) failed due to poor usability and a flawed feature/function design. Here is what SNAP could have done to dramatically improve the success of this potentially groundbreaking product concept.

Why SNAP Spectacles Failed: A Detailed Professional Usability Heuristics Analysis

Charles Mauro CHFP

Spinning gold from code This is an object lesson in why hardware design is exceedingly difficult for software businesses. It is always an interesting exercise to watch a software-based and cloud-delivered business, even a very clever one with a massive user base and a seemingly successful business model, attempt hardware design. There is a certain arrogance that comes from developing a successful business based essentially on code, server farms and prodigious levels of capital looking for the next Facebook. Such business models, when they are successful, can be a lot like minting money without the attendant risks associated with producing actual physical products. With the right combination of usability, features, social media exposure and a large measure of luck, seemingly ridiculous ideas turn into billion dollar corporations backed by Wall Street and investors. Such is the basic trajectory of SNAP. Continue Reading…