The complex science of a simple quiz
Immortalised by teen magazines and Buzzfeed, the humble quiz has matured into a powerful online tool for personalising recommendations, building customer loyalty and even recruiting new hires. So how do you design the perfect set of questions? Perspective Careers speaks to the experts.
When shopping search engine Lyst began recruiting experts for its Sneaker Intelligence Unit, it created a 16-question game instead of a traditional job listing.
In one week, the page saw 24,000 visitors — 1,000 per cent more than the company expected from a traditional application in that time frame.
“We had to get creative in how to reach out to sneaker enthusiasts,” says Communications Director Katy Lubin.
Lyst is one of a growing fleet of e-commerce companies that are utilising quizzes and customer feedback surveys to provide personalised recommendations, build customer loyalty and recruit new hires. But building an effective quiz is not as simple as it seems.
The goal of a quiz determines its design
The most effective quizzes are designed with a forum, customer and goal in mind.
The amount of time invested in a quiz should be proportionate to what the customer receives, says Matt Field, co-founder and president of predictive analytics platform MakerSights. “When you ask people for information, you enter into a contract of mutual benefit,” he says. “If there isn’t a good reason [to take the quiz], you will probably get lower participation and less valuable insight.” Customer feedback surveys, for example, should be mobile-friendly and take 90 seconds or less to complete.
Online lingerie company True&Co.’s 17-question quiz is designed to help customers find bras that fit. To determine the initial set of questions, founder Michelle Lam began with interviews and focus groups. Over time, A/B testing showed which questions were most important for making recommendations and how many to ask.
While shorter quizzes tend to have higher completion rates, for True&Co. longer is better. Its bra fit survey asks relatively intimate details about a customer’s breast shape and fit concerns. While it does lower the completion rate, it translates into a higher conversion-to-order rate, says Lam. When the reward is finding a well-fitting bra, the customer is willing to invest more time.
As True&Co.’s algorithms have improved, the company has been able to better connect the dots between seemingly unrelated details. The team learned, for example, that consumers with feline references in their email address were more likely to buy red lingerie than consumers with dog or bird references.
To build a product recommendation quiz, Keith Bendes, vice president of brand partnerships at interactive experience agency Float Hybrid, takes a rather different approach. He begins by looking at the products and assigning each one attributes to gauge the personality of the person likely to buy them. Finally, he builds a “decision tree” to map out the path of each question. A three-question quiz with three options for each question leads to 27 separate possible outcomes.
Deciding the right questions
Subscription-based jewellery service Rocksbox offers three-piece jewellery sets personalised to customers’ tastes. The process begins with a six-question quiz. Questions include, “Which metal styles do you wear?” and “Which [ring/necklace/bracelet/earring] styles do you prefer?”
Decreasing the amount and complexity of questions resulted in a 6 per cent increase in conversions, says founder and CEO Meaghan Rose. She found that vague or nuanced questions such as, “What style are you?” weren’t helpful because customers either weren’t sure or their actual behaviour didn’t match the answers they gave.
“It’s a really bad question because it caused a moment of anxiety or pause in the sign-up process — it feels judgmental,” Rose says. “[Customers thought], ‘What does ‘boho’ even mean?’ It’s hard to categorically say you don’t wear something.” Its quiz is now limited to binary questions that narrow down thousands of products into hundreds.
Images can be more effective than text
Many of the sources we spoke to said that images are better than worded questions because they are easier to interpret, faster to respond to and result in more useful data. A respondent might have a hard time defining her style as “feminine” or “edgy”, but can quickly see an outfit and know if she’d wear it.
“A human’s ability to digest and react to a fully fleshed-out image versus a verbal prompt is much more natural,” says Field from MakerSights. He adds that a “thumbs up” or “thumbs down” can quickly establish the scale of general consensus, while a five-star rating can give more depth to a rating.
In 2018, online personal styling service Stitch Fix, which onboards customers with an approximately 40-question survey, added an optional, image-based game called Style Shuffle, which prompts customers to give a thumbs up or thumbs down to different items without seeing price or fit. The game has improved Stitch Fix’s personalisation capabilities and informed inventory choices, says Chief Algorithms Officer Eric Colson. More than 75 per cent of the brand’s 2.9 million active clients have since used Style Shuffle, resulting in an increase in revenue among active clients and an increase in engagement for paying and non-paying clients.
“My advice is to not assume you’ll get it right the first time,” Colson says. “It’s not always obvious which questions will be valuable and better ideas often come later. Also, be sure to be ready to use the information you learned. If clients are providing information, they expect it to be used.”