Validation of Virtual Shopping Data

One measure that has been suggested as a way to validate virtual store results is the correlation of in-store market shares to virtual store market shares.

For a recent study, In Context Solutions obtained in-store sales data for the products in a category for 40 different units of analysis. Some of these were sub-categories, some Brands, and some at a SKU level. When the share of the category dollar sales is calculated for each of the 40 units and correlated with the corresponding share of sales in the virtual store sample the correlation is 0.99.

This is exceptionally high and so we broke it down into the compnent groups thinking perhaps it is related to the sub-category levels being easier to get correct and the SKU level being harder. This is, in fact, true, however, while the correlation for the sub-categories is 0.99, for brands it is still 0.90 and for the SKU level it is 0.91, so they are all very strong correlations.

So, based on this measure, virtual store clearly does a good job of providing sales results that are highly correlated with in-market sales data.

This is not an isolated result. While I was at General Mills we conducted dozens of studies and when we looked at this measure for those studies we consistently saw correlations at 0.9 or higher.

The Advantages of Internet Virtual Store Research

Last time, I discussed the reasons for using 3D images in virtual store testing. This time I want to discuss why the Internet as the research mode is important.

One of the primary issues new users face for doing virtual store research in 3D is the upfront costs. If they want to create a virtual store using 3D images for all of the reasons discussed in the last installment, there are significant costs to create the category of interest and the environment of interest in 3D. Once you have these images created they can be reused for subsequent studies making those studies less costly.

Most vendors who use 3D images must conduct their interviews in-person as the software they use can't fit over the Internet. So another cost is that of in-person interviews for each study. The typical cost per interview in-person is $75 or more, while over the internet they can be done in the $8-$12 range depending on the specs for most studies. So, the primary advantage of the Internet mode is a substantial cost reduction. Most studies cost half as much on the internet than they would with in-person interviews.

Another issue created by the in-person interviews is a reduction of sample size. Many vendors recommend virtual store sample sizes of 100-300 per cell for their studies. This can be dangerous, depending on the number of products in the category of interest. The sample should be large enough to capture the distribution of hwo each of the products turns in the store. That is to say ideally, all products would be purchased by at least 1 respondent. In real stores, all of the products on the shelf will move at some velocity or the retailer wouldn't carry it. We need to avoid having too small of a sample to allow this distribution of sales to be captured. If the size is too small some products will not be purchased and this can lead to bad estimates and decisions.

So, to really make Virtual Store simulation useful for widespread scalable use in research a 3D online tool is necessary.

Virtual Store Research - In Context Solutions

I'll start my blog with this disclaimer. I own a marketing research supplier company with a few partners. We specialize in virtual store research. So, I have a vested interest in this space. What I also have is an excellent understanding of virtual store simulations and how they can be utilized by retailers, manufacuters, marketing services companies, and research companies to better understand how and why people shop the way they do.

I started this company because of the experiences I had with other virtual store vendors. I was at General Mills in their consumer insights function, managing their in-store testing, and brought in virtual store testing as a method to reduce the cost and time of doing this kind of research. The other vendors of virtual store research fall into one of two camps. One set provides a rich 3D virtual store simulation that allows respondents to move through a virtual store and shop, but the interviews have to be done in person. This makes the research projects about twice as expensive as they would be if they could be done online. The other set of vendors can do the virtual store online, but their interfaces are 2D for the shopping and this can lead to some issues for studying many aspects of a store environment.

Our company, In Context Solutions, provides the rich 3D simulation, but can do it over the internet providing the best of both worlds. This allows us to gather data over the internet to make it more scalable and cost effective, but still provide the best possible simulation of a store. If you want more information about us, please see our website at www.incontextsolutions.net.

I plan to use this blog to share some of what I have learned about how to do virtual store research and what some of the issues might be. I welcome your input and comments and will be happy to respond to them.

Virtual store research - Why is 3D important?

2D Virtual Shelf Set

3D Virtual Shelf Set

You might ask why 3D is important or necessary in virtual store research? The reason is many of the shopper marketing levers we want to study are interacted with in three dimensions. These include free standing displays, end caps, coupon dispensers, signage such as shelf blades, aisle layouts, and many others are shopped from multiple angles and in different sequences in the real store and in order to fully capture how people will behave when faced with these 3D parts of the shopper environment requires a 3D interface.

While at General Mills I had the opportunity to do side-by-side research using 2D and 3D vendors. There are two additional areas were the results differ.

The first difference is on measures of awareness and purchase, usually the key measures for packaging or new product research. The reason for this difference is the 2D interface requires the respondent to scroll the shelf set to see products shelved down the aisle. As the products scroll onto the screen the respondent is forced to look at them to see if they are the product they are looking to buy. This forced exposure to the packages causes them to more often notice new products or packaging or signage in the shelf set. In a real store, or a 3D simulation of a store the shopper usually faces down the aisle and walks down the aisle until they reach a visual clue like the red of the boxes of Coca Cola or the Yellow of the boxes of Cheerios and finds their product from its relative location to the visual marker. This process takes just a few seconds and they usually don't look at the rest of the products in the aisle as they walk past them and consequently don't notice the new products, packages, or signage.

The second difference in the data between 2D and 3D is things shelved on the top and bottom shelves don't pay a penalty in a 2D shelf set as all items are shown in roughly the same view. In a real store and in a 3D virtual store it is much harder to see things that are not at eye level. This greatly impacts the purchases of items on the bottom or top shelf or in a bunker.

These differences mean that the data is less accurate for any of these situations. While the 2D interfaces will work for small categories with the products of interest being shelved at eye-level, many situations will cause the data to be less than ideal. This is why our platform is a fully 3D interactive environment to allow the shopper to shop as they would in a real store.