Katrina and Rita: Understanding the Storms' Impact on Market Research

By NPD Analytic Director, J.D. Deitch

While our concerns remain foremost with the people who suffered the fury of Hurricanes Katrina and Rita – including those in our Houston office and several employees with strong ties to the affected regions – our obligation to clients and Retailer Partners necessitates understanding the storms’ effects on our data collection. Our Research Sciences experts have assessed the impact and created this explanation of our thinking on the matter and the steps we are taking to ensure the continuity and integrity of our methods and minimize the impact on our products and clients.

Consumer sampling and participation in our surveys

Potential impact
In light of what so many have had to endure over the past weeks, we would hardly expect completion of our surveys to be a top priority for our consumer panelists. But to exclude panelists as a matter of course based on a set of ZIP codes or other regional identifiers presumes they are both unable and unwilling to participate in our studies. As we saw with the September 11 tragedy just a few years ago, and indeed as we’re seeing now, these assumptions may not be valid.

Were we to exclude the whole of an affected region, we would be exchanging one bias for another. For while we can identify those who live in the general region, conditions within these areas vary considerably, and we have no way of accurately identifying which areas and panelists are most affected. We have also noted the extent to which people can and do remain in touch electronically. The Internet has been an important, and in some cases, the primary, means of communication since before the storms made landfall. It is therefore quite plausible that people are receiving and opening our study invitations – maybe not on their own computers, but they can access their e-mail nonetheless. We also know that in an effort to reestablish some semblance of normalcy, people often try to resume their routines in difficult times. In short, while there are many who remain profoundly affected, there are others who are less so.

Our plan
Ultimately, as a research company, we prefer to know rather than guess. In this instance, where there are shades of gray in terms of sensitivity and impact, we believe the best thing we can do is to continue on course with our study invitations. We will never be able to understand and control for the hurricanes’ impact without being able to measure it, and we cannot measure without data. So we will monitor our samples diligently, and with no small amount of compassion for those who may have much more important things to deal with.

Consumer projections and controlling for changes in
response rates

Potential impact
If we continue inviting panelists to participate in our studies, then our ability to manage the effects of the hurricanes comes down to the robustness of our consumer projection systems. Each NPD division that tracks domestic markets employs a projection system that controls and balances on nine different regions within the U.S, and nearly all of these have further controls within region that balance on Metropolitan Statistical Area (MSA).

As we’ve seen, there have been shifts in population due to the hurricanes. In the case of Rita, the shift was fairly short in duration: for the most part, Houston residents evacuated for about one week and returned home. While this migration was indeed extraordinary, it was also largely temporary. In the case of Katrina however, the displacement has lasted longer and will manifest itself differently in our projections. Some evacuees have returned. Some may still be away from home but might be projected accurately since they remain in a similarly populated area in the region (we group MSA into three tiers based on size – non-MSA/rural, small, and large). And there are others who remain outside their home MSA or region.

The main issue becomes the permanence of their dislocation. Our projection targets are determined based on U.S. Census population figures, and therefore any changes in the residence for the evacuees would first need to be accounted for there, in order for us to change our projection targets. Given our best estimates of population movement, the change, though likely very small, might indeed be perceptible. We would therefore accurately capture the purchase, but would risk categorizing it in the wrong geography.

Finally, there is one scenario we simply cannot control for: if those affected by the hurricanes disproportionately fail to respond to our study invitations. Our projection systems can handle this numerically, but this presumes these individuals behave in a way that is systematically similar to those not affected by the hurricanes. This might be the case. And it might not.

Our plan
Our intention is to monitor where our evacuated panelists have landed, but to continue projecting these individuals based their last-known permanent residence. To the extent possible, our projection systems are managing the change, but we do not know how many individuals have permanently moved, nor has the Census Bureau altered its figures to account for any hurricane-related migration.

Given all the data we have available, we estimate that the storms affected approximately 3 million people who live in the New Orleans-Metairie-Kenner, Baton Rouge, Beaumont-Port Arthur and Gulfport-Biloxi MSAs along the affected coastline. Those who went to Texas or Arkansas would have remained in the same region, but those who went to Mississippi or Florida would have left and, therefore, we would apply their purchases to their home region/MSA. We can say this with certainty: if purchasing trends do change slightly in the surrounding areas, these changes are surely real and reflect actual, though perhaps (and, we hope for those affected) temporary shifts in behavior.

To keep track of any population shifts, all NPD divisions will be carefully monitoring their projection systems in the coming months for even small disruptions in trends. Any issues that arise will be thoroughly investigated by a working group of NPD’s Research Science and Client teams to determine appropriate courses of action.

Receiving point-of-sale data

Potential impact
Obviously stores that are closed due to the hurricanes will not record any sales, and this lack of reporting will indeed reflect the true geographical retail environment. Likewise, we might expect stores in areas with a significant influx of evacuees (e.g., Houston) to possibly show greater than expected sales. Again, were this to happen, it would be a true chronicle of the retail landscape.

In terms of our ability to receive point-of-sale (POS) data, we think the only problem we might face – one we believe is unlikely – is the potential disruption in data transmission from an individual store to its company’s central data repository. Given the implications this would have for the retailer’s supply chain, we expect problems of this nature to be addressed quite quickly.

Our plan
Our Retail Business Group will remain in contact with our Retailer Partners to monitor the flow of POS data. Instances of potential disruption will be evaluated and addressed in conjunction with NPD’s Research Sciences group and the affected NPD division.

Projecting POS data

Potential impact
Certain NPD divisions take their raw POS data feeds and project them in an effort to capture total retail volume for a given industry. One way they do this is by using projected consumer data to create a total industry estimate, then developing POS projection factors to make the POS total equal the consumer total. When POS projections are calculated this way, they will reflect the quality of the consumer data, and we will be facing the issues we just discussed above.

Conclusion

We hope we have clearly explained our thinking about the hurricanes, as well as our broader perspective on managing unexpected events that may have an impact on our business. At the end of the day, we rely on some fundamental principles when faced with uncertainty: we contemplate, we investigate, we estimate, and we communicate. Our ‘bias’ as a company is to know rather than guess. Having said this, we must ultimately accept the fact that we will never know with absolute certainty how many people have been truly affected. But by analyzing the data and applying solid principles of market research, we will be able to gather the information we need to make informed choices and steer a course that minimizes the negative effects on our clients.