How past product experiences shape your UX
Understanding, measuring, and applying skill transfer in user research
If you’re a Mac person and you’ve ever needed to use Windows, you probably had a bad time. That's not a swipe at Microsoft: the same holds in reverse. But why?
Many habits don’t transfer from one environment to the other. On Windows, closing an application window quits the application, whereas macOS separates these functions. Between the two, the buttons to minimize a window are on opposite corners. In short, you need different cues and behaviors to accomplish even simple tasks.
Yet there are many other experiences where prior habits and knowledge will actually give you a boost. As UX professionals, we need a systematic way to understand how those past experiences can influence interactions with new products.
This article examines skill transfer, how it manifests as positive, neutral, or negative, and how it affects learning and performance. We’ll talk about how product development teams can promote the most effective kind of transfer, and discuss ways UX Researchers can measure transfer to help toward that goal.
What is transfer?
If you’ve worked in UX for any length of time, you’ve heard the term "intuitive" used to describe our ideal outcome. But what exactly does it mean for an interface to be intuitive?
My favorite definition comes from Jef Raskin, a human-computer interaction (HCI) expert who led the early development of the Apple Macintosh computer. According to Raskin, an intuitive interface leverages users' pre-existing skills and knowledge, allowing them to transfer what they already know from other contexts to effectively interact with the new interface. What this means is that our initial exposure to new products is never a true blank slate. We bring to it all our prior knowledge and experiences gained from interacting with other products, both similar and dissimilar.
We can operationalize this definition using a concept from educational psychology: transfer of learning. It describes how prior knowledge or experience in one setting affects learning and performance in another. There are three main types.
Positive transfer occurs when prior knowledge or experience helps an individual understand and perform better in a new context. This is common when the contexts are related or similar. Examples:
Many successful NFL quarterbacks, such as Colin Kaepernick, Dan Marino, and Troy Aikman, were also talented baseball pitchers. Both roles require the ability to throw a ball quickly and accurately.
Learning a Romance language like French after having studied Latin, as French directly descends from Latin and shares many linguistic roots.
Google Sheets and Microsoft Excel have similar interfaces and functionality. If proficient in one, learning the other is likely to be relatively quick.
JASP, a free and open-source R-based statistical package, mimics the appearance of the more expensive SPSS. For those who may have used SPSS in graduate school but can no longer afford it, picking up JASP will come quickly.
Neutral (or zero) transfer describes when prior knowledge has little or no impact on learning in a new context, often due to a lack of similarity between the two contexts. Examples:
As Michael Jordan famously learned, the explosive power, vertical jumping ability, speed and endurance required in basketball don’t translate cleanly to the unique demands of baseball.
Learning Arabic won’t be any easier for Latin students, as the two languages share virtually no common linguistic roots or structures.
Experience with Microsoft Excel or SPSS won’t give much of an advantage or disadvantage to those learning other specialized applications like Photoshop or Final Cut Pro, since they serve different purposes with different interfaces.
Negative transfer, then, is when prior knowledge or experience hinders learning and performance in a new context. This happens primarily when habits and behaviors rewarded in one context are counterproductive in the other.
We’ve already mentioned the differences between macOS and Windows as an illustrative case here, but a few more examples include:
Learning to drive in the United States and then driving in the United Kingdom, where traffic flows on opposite sides of the road, can lead to negative transfer of driving habits.
Learning to type on a Dvorak keyboard layout after being proficient with the standard QWERTY keyboard layout. Although the Dvorak layout positions the keys more efficiently, learning a new arrangement amounts to relearning to type.
Encountering false cognates (words that appear similar but have different meanings) when learning a new language can cause negative transfer from one's native language.
In UX, we strive to promote positive transfer while minimizing negative transfer. By understanding the past experiences and mental models of users, we can tailor our designs to align with their expectations, making our products easier to learn and use.
Promoting positive transfer
Product teams can leverage two key strategies to promote positive transfer and enhance user experiences:
The first is designing for familiarity and consistency.
The goal is to intentionally incorporate design patterns, conventions, and interaction models that are likely to be familiar to a significant portion of the target user base. In principle, this is similar to one of Jakob Nielsen's usability heuristics, “Consistency and Standards.” This can be as simple as adopting recognizable icons for common actions like save, print, and undo/redo in productivity tools and office suites.
The second strategy is to provide effective training and support.
For complex products requiring training, focus on helping users build mental models that facilitate positive transfer. Explicitly highlight connections with familiar concepts and processes from other contexts. For example, if your team were building a video editing tool, you might relate key functions to similar tools users may have used before, like Microsoft Word's track changes for editing.
Incorporate hands-on exercises that mirror real-world scenarios, to allow users to practice applying their developing skills in a controlled environment, such as having them edit a sample video project.
The goal of an “intuitive” interface becomes concrete and achievable when teams intentionally design for familiarity.
Measuring transfer in user research
One of the best ways UX Researchers can contribute to the goal of promoting positive transfer is by measuring it well. To do that, researchers have a few key considerations during study preparation, design, and analysis.
For a start, we want to recruit participants with relevant experiences.
During the screening process, researchers should ask about experience using similar products or tools. For example, if you’re studying the usability of a smart home device, it might be valuable to ask candidates about their experiences with other smart home devices, voice assistants, or chatbots. Depending on your goals, it could be useful to recruit participants with both low and high familiarity.
The example above highlights a few best practices when asking about relevant experience.
If you’re working in a complex space, it’s sometimes necessary to include definitions and explanations when a question includes unavoidable technical jargon. Further, we ask about a specific time frame, since participants may have extensive, but dated and no longer relevant experience. Instead of asking participants to rate their own proficiency, which might often be unreliable, it may be better to ask about frequency of use. And since inattentive responders can compromise your data quality, it’s typically helpful to include a distractor choice, like the fictitious CRM ZenConnect.
As you move into study design, a few choices will depend on your goals:
Align the tasks: If measuring the effect of training or onboarding, ensure that the tasks participants perform align with the content covered in the training materials. For instance, if examining the helpfulness of an introductory onboarding flow, follow-up tasks should include scenarios similar to those covered.
Measure both contexts: To get a stronger measure of transfer effects, assess participants' performance in both the original or baseline context and the new context within the same study. For example, in a redesign validation study, test performance on both the original or baseline design and the new design variation.
Prefer behavioral measures: Relying solely on self-reported performance measures can lead to inaccurate conclusions. Instead, aim to collect objective performance data. In an unmoderated usability study, for example, participants could be asked to rate their confidence in successfully completing a task, while their actual success is verified through URL validation or knowledge-based validation questions.
During analysis, researchers have a few tools to quantify transfer effects.
Although correlations and statistically testing means through t-tests or ANOVAs can help, my recommendation is a regression. The resulting coefficient of determination or R² shows how much performance on the product we’re building is predicted or explained by performance from another context. Interpreting a single result in isolation can be challenging, but accumulating multiple coefficients of determination allows for a better understanding of the relative strength and consistency of the relationship.
By carefully considering these factors during study design, execution, and analysis, UX Researchers can generate valuable insights into the role of transfer in shaping user experiences with products and interfaces.
The bottom line
Transfer of learning describes the phenomenon where prior knowledge and experiences influence our ability to learn and perform in new contexts. There are three types: positive, where prior knowledge enhances performance; neutral, where prior knowledge has no significant impact; and negative, where prior knowledge hinders performance.
This article discussed a few key ways product teams and the UX Researchers who support them can promote positive transfer:
Leverage familiar design patterns, metaphors, and interaction models aligned with users' prior experiences.
Provide targeted training and support that explicitly connects new concepts to users' existing knowledge.
Design research studies that accurately capture and measure transfer effects across different user segments.
Analyze study data to quantify the impact of transfer and identify areas for optimization.
When stakeholders and colleagues request a more "intuitive" design, they often mean one that utilizes positive transfer. By operationalizing this definition and implementing the theory and strategies discussed here, teams can more reliably create intuitive interfaces.
ANOTHER THOUGHT…
The single-item usability scale to rule them all?
Is the best UX rating scale also the shortest?
If you've ever needed a way of measuring perceived usability, you've probably used the System Usability Scale (SUS) or newer, shorter instruments like the SUPR-Q or UMUX-LITE. But you may be less familiar with the Adjective Rating Scale, a single-item questionnaire developed by Bangor, Kortum, and Miller in 2009. It asks just one question: "Overall, I would rate the user-friendliness of this product as:" and then provides 7 adjective responses, from "Worst Imaginable" to "Best Imaginable."
Originally developed as an adjunct to the SUS to help practitioners interpret its results, there may be a good case for using it by itself. According to a recently published article in the journal Human Factors, it could do the job of much longer scales. The authors concluded that it "was as good as, and sometimes significantly better than, all other measures of perceived usability considered here at representing the difference in usability between a good-usability system and a poor-usability system."
If you're curious to read more, check out the full article (paywalled) on Human Factors. And let me know about your experiences using it.
Drill deeper
These are the broad considerations to keep in mind when measuring transfer in your product experiences. If you’re looking for a partner to help you operationalize and measure transfer in your context, Drill Bit Labs offers the expertise you need.
Our offerings include:
Custom user research projects that deliver actionable insights
Tailored training sessions, from the basics of research methods, to deep dives on measuring transfer
Advisory services to align strategy, process, and impact
Connect with us to discuss your upcoming projects or ongoing UX needs.