FPG Research-on-Research: Fast Data vs. Deeper Insights
As researchers, we are constantly faced with the challenge of balancing how fast we need data with the need for valuable depth of insight. These everyday scenarios can be in the following ways:
1. I am more interested in getting data fast than I am for deep insight
2. I am interested in getting data fast, but also need some level of increased insight
3. I am more interested in a higher level of detail and insight than speed
A researcher may come across all of these scenarios on a regular basis, so it is important to understand which options are available and how each can maximize efficiency and effectiveness.
Each of the three data collection methods collected respondent feedback on alternative packaging designs. The consistent measure across all three methods was overall preference, obtained in a different way for each method:
Gauge Mobile App – swipe selection
FPG Polling™ – radio button next to image
Qualtrics – drag and drop image preferred
Additional questions were included in both FPG Polling™ and Qualtrics, and shared the following:
Strength of preference
Fit with brand
Other measures were included with Qualtrics only:
Visual impression – mark up exercise
Attribute preference – drag and drop under package design that best represents brand
Open ended question – how packaging makes one feel about brand
The paired comparisons were set up as follows:
Package A vs. Package B (bottled water product)
Package C vs. Package D (snack product)
Package E vs. Package F (soft drink product)
The purpose of this research was to determine if three diverse quantitative data collection methods could be employed to collect consistent data – with the decision for method dependent upon the specific need for speed and/or added insight.
- Whether the respondent was tasked with clicking a radio button, swiping left/right, or dragging and dropping to indicate preference, all three methods provided comparable and consistent results.
- In essence, the same decision on the winning package designs would have been made regardless of data collection method.
- All three methods are viable options for measuring preference – with two of the options providing the ability to collect additional insight.
- In addition, the strength of preference was consistent across all three methods.
- There was a clear winner in each paired comparison and this was noted to the same degree across all three methods.
- Moving beyond preference ratings, this Research on Research study has shown that additional data and insight can be collected efficiently and effectively – depending on specific client needs.
- Viable options are available whether the decision is to collect a small amount of additional data or a large amount and at each phase a client may be in the evaluation process.
- Including interactive exercises (Qualtrics) and qualitative feedback in your research provides additional in-depth insight and diagnostics that help separate or distinguish options. As an example, the mark up exercise allowed the generation of a heat map that clearly indicated which design was more effective in drawing attention to the brand name. This could serve as a tie-breaker if designs are closely matched on other measures.
- These methods could also be employed in succession to cost effectively and efficiently cull down a large number of options. As opposed to running many designs through a long detailed survey process, a more efficient option would employ both Gauge and Polling in the initial phases to cull the designs down to a small number for focused optimization. An example of how that could be implemented:
- Gauge Mobile App – initiated in the first phase to cull down a large number of designs. “Winners” go to next phase – FPG Polling™.
- FPG Polling™ – in the second phase we add high level measures that will serve to further reduce or separate options.
- Qualtrics – the third phase takes smaller set of “winners” from the second phase to focus on optimizing and collecting deep insight. This final phase will be more efficient as it is more focused on optimizing the winning designs.