CASRO Tech: Making the Case for Online Dial Testing



Amidst all the conceptual talk of research opportunities with Big Data, social media and mobile taking place last week at the CASRO Technology & Innovation conference, there were a handful of presentations focused on real-world methods and results. One such presentation was delivered by HSN’s Elizabeth Merrick and Dialsmith’s David Paull. The two teamed up to present on the benefits and outcomes of HSN’s use of Dialsmith’s online tools for Moment-to-Moment testing with its Insider Community.

Watch the recorded version of this presentation here (no sign-in required / running time: 14:31)

Here are the key takeaways from the presentation:

  • Per David, “Relying on recall memory alone is flawed.” He backed this statement up by referencing the role that two key issues play when it comes to memory and recall: 1) interference theory; and 2) the misinformation effect. Both lead to reliability problems with recall memory that can be mitigated by complimenting recall with moment-to-moment.
  • Per Elizabeth, “Using Perception Analyzer Online allowed HSN to test dozens of host video clips that offline would’ve required more than 30 focus group sessions.”
  • Per Elizabeth, “The Perception Analyzer Online testing revealed key moments of viewer feedback that allowed HSN to understand the impact that a familiar host has on a viewer. We learned that while more and less familiar hosts eventually got to the same rating level, more familiar hosts started out the segment with a faster boost while less familiar hosts got there at a much more gradual pace. This is important to know because if we’re running a shorter segment, or need to get to the value quicker, a more familiar host will better accomplish that. However, there is a higher cost to HSN for more familiar hosts and when we can reduce costs and use less familiar hosts we do so and that has a positive impact on the bottom line.”
  • Per Elizabeth, “Slidermetrix provided HSN with a streamlined and effective way to gather moment-to-moment feedback within HSN’s community environment.”
  • Per Elizabeth, “Using Slidermetrix in conjunction with community chat allowed us to pinpoint something that participants were having trouble articulating during group discussion. We always show multiple product close-ups during a segment and what we learned through the Slidermetrix moment-to-moment ratings was that we were causing viewer fatigue with too many close-ups. The first and even second close-up rated well, but by the time we got the third and forth the ratings dropped significantly. Once we learned that through this quantitative method we were able to discuss it specifically with participants where they were then able to better articulate the issue to us. That is something we were unable to learn through group discussion alone.”
  • Per Elizabeth, “As Customer Experience Management becomes a central force in business we are going to need to better understand each moment, the ups and downs, the joys and disappointments that all lead up to a final outcome – whether that’s a sale, a recommendation, or even a decision to end  a contract.”
  • Per Elizabeth, “You typically think of dial testing as a tactical, media-centric methodology, but HSN is first and foremost a retailer.  We drive sales in addition to viewership with this type of research. Online really opens up possibilities, so think big.”
  • Per David, “Online moment-to-moment is platform agnostic. New tools allow you to seamlessly redirect and pass-back respondents between the moment-to-moment element and other survey platforms.”

We’ll be posting up a recorded version of this presentation soon so stay tuned. Until then, you can find some interesting nuggets and tips from Elizabeth as well as from a host of other experienced research pros versed in both online and offline Moment-to-Moment in this free downloadable eBook.