Online Surveys in IP Cases

On March 28, 2011, LJS Legal Research Director Sara Parikh presented at the 2011 Intellectual Property Law Spring Seminar.

This white paper focuses on the correct use of Online Surveys in IP Cases, with information on proper phone, in-person and online survey methodology, and concludes with a checklist for designing and conducting IP litigation surveys.

Sponsored by the Intellectual Property Law Section of the State Bar of Michigan, in cooperation with the Institute of Continuing Legal Education

Online Surveys in IP Cases

I. Research Methodologies

A. Introduction

Once challenged as "hearsay," surveys are now routinely accepted, and often expected, as evidence in intellectual property and false advertising disputes. Survey experts draw upon a range of market research methodologies when designing and conducting surveys for such disputes. Telephone surveys and mall intercept studies are the most frequently employed, but the use of online surveys in intellectual property (IP) litigation is beginning to gain acceptance.

This paper begins with a discussion of telephone surveys, mall intercepts and online surveys with respect to their typical applications, characteristics, and limitations. It then focuses on the use of online surveys in the market research industry generally, followed by a discussion of recent submission of online surveys in Lanham Act disputes. Finally, the article provides a checklist of things to consider when designing and executing any type of survey for use in intellectual property and false advertising disputes.

B. Background

The field of commercial market research really took off in the mid-20th century as companies began to see the value of talking to their customers and potential customers. The pioneers in the industry were typically trained in the social and behavioral sciences, with expertise in research methodologies and statistical techniques. This training and orientation still grounds the profession today.

The industry standards for market research are built upon enduring social scientific principles such as transparency, sampling theory, and objective measurement, tabulation and analysis.

In the early years of market research, surveys were typically conducted by telephone or face to face (even door to door), as those were the best means of reaching people and of securing a representative sample of the population. Both methods could be sampled through pure "randomization" or by selecting a number of sample spots to represent the population. Beginning in the 1970s, the growth of the shopping mall (replacing the "Village Green" as a public gathering spot), provided a more efficient way of reaching consumers in a populated shopping environment. Research facilities were established in shopping malls across the country to accommodate the growing demand for consumer surveys.

In the past ten years or so, there has been a significant shift in both communications technology and consumer shopping patterns.The traditional landline telephone survey is not necessarily as representative as it once was due to the rise in voice mail and call screening, and the recent growth of cell-phone only households. Further, with the growth of the freestanding big box store and the rise in Internet usage in general, and online shopping in particular, visitation to shopping malls has plateaued and even declined in recent years. As a result, the telephone survey and mall intercept study have been augmented (but not replaced) by online surveys. Today, the market research industry relies heavily on telephone surveys, mall intercepts, and online surveys, each of which is discussed in greater detail below.

C. Telephone Surveys

Characteristics. Telephone surveys of consumers are often conducted by calling both listed and unlisted telephone numbers using computer-generated telephone exchanges to represent all households with landline telephones. Business to business research is also often conducted by telephone. Business to business studies, or consumer studies with a narrow universe, might draw upon a list of targeted companies or households provided by one of the nation's many sampling companies. Assuming the sample from which the respondents are drawn is representative of the universe, telephone studies are projectable to the larger population. Telephone surveys, like intercept studies, are also administered by trained interviewers who are skilled in proper survey administration, probing and recording verbatim responses.

Limitations. As noted above, in recent years there has been a decline in response rates in telephone surveys, which can impact their projectability. Further, with the rise in cell-phone only households, purely landline surveys are no longer as representative as they once were. Landline-only studies can under-represent younger adults, as well as low-income populations, who are most likely to only use and have a cell-phone. Researchers today sometimes supplement landline sample with cell-phone sample to better represent the population. Finally, one of the central limitations of the telephone survey is the inability to gauge respondent reaction to visual stimuli. It is possible to solve this by conducting a hybrid study in which the respondent is sent the stimuli ahead of time (by mail or email) or asked to log on to a dedicated URL to review the stimuli during the telephone interview. However, studies that require exposing participants to visual stimuli are usually conducted in person or online.

D. Mall Intercepts

Characteristics. In a mall intercept study, an interviewer will approach a consumer shopping at the mall and ask them the relevant screening questions to ensure that they are qualified for the study. If they qualify and agree to the interview, the respondent is typically escorted to a private interviewing room to complete the interview. Because they are conducted in person, the mall intercept study is well-suited to gauging consumer reaction to visual stimuli. Further, because it is conducted in a true shopping environment, the mall intercept study is a good technique for replicating the marketplace in which consumer goods are purchased. Finally, mall intercept studies, like telephone surveys, are administered by trained interviewers who are skilled in proper interviewing techniques.

Limitations. Though shopping malls are a good place to find most consumers, the mall intercept study is not always feasible or appropriate when trying to reach certain populations (e.g., professionals or small niche markets). Further, because mall intercept studies are not truly random samples, they are not considered to be probability studies. However, researchers have developed a number of practices to make mall intercept studies more representative of the population. For example, rather than using one or two mall locations, the study will be conducted in a number of different mall locations that distribute across the relevant geographic area. Further, the researcher might establish sub-quotas by select demographic criteria (e.g., age, gender, etc. ) to better represent the population. With these kinds of controls in place, survey researchers, companies, and the courts usually consider mall intercept studies to be sufficiently representative to extrapolate the results to the target population.

E. Online Surveys.

Characteristics. In an online survey, the potential participants are sent an email inviting them to participate in the study. The email invitation includes a link to the online survey, which is embedded with a unique PIN to ensure that each respondent only completes the survey once. Online surveys are either conducted with respondents who are members of an "opt-in" panel, or with members from a targeted email list.

• Opt-In Panels. There are a number of companies that have built large-scale online panels of consumers and/or businesses. Panel members volunteer to participate and agree to be contacted for future interviews, usually filling out a "Profile Questionnaire" that captures basic personal data (e.g., age, gender, income). These data can then be used to target particular panel members for a study. While most established panel companies use a number of techniques to build representative panels, the panels vary in terms of how representative they are of the population.
• Targeted Lists. In addition to using established panel companies, online surveys are often conducted with members of a well-defined population in which a list of members is available (e.g., all alumni of a college, or all members of a particular professional association). List members are invited by email to participate in the study and provided with a dedicated link that takes them to a survey. Online surveys conducted with these kinds of targeted lists can be projectable to the population, assuming the list is representative of the full universe.

Whether it is based on an opt-in panel or a targeted list, the online survey can be a good way to reach groups of hard to find respondents. Further, the online survey allows for testing reaction to visual stimuli, including photographs of products, as well as print, video and even audio advertisements. However, if the respondent needs to look at or experience (e.g., touch, feel, taste, etc.) the actual product in person, online exposure to a visual image is obviously not sufficient. Further, when showing a visual stimulus online, consideration should be given to variation in the respondents' monitor size, color settings, and screen resolution.
Limitations. Though Internet usage has grown dramatically over the past 10 years or so, certain populations are under-represented online. For example, while Internet usage in the U.S. is almost universal among the young and the college-educated, lower income and older adults are less likely to be online. Further, because most online surveys use opt-in panels, there is a greater potential for self-selection bias. Participants not only need to agree to participate in the study, they also must have opted-in to the panel. Finally, unlike telephone and mall intercept studies, because online surveys are self-administered, there is no interviewer present to clarify or probe.

II. Online Surveys in IP Litigation

A. Background

As Internet penetration has grown, the business community and market research industry have rapidly embraced the use of online surveys in traditional market research. Today, online surveys account for about 50% of industry research dollars, up from about 30% in 2004.

Nevertheless, there has been a lag in the use of online surveys in IP litigation. This lag can be partially explained by the frequent need to show a visual stimulus. Further, until Internet penetration and usage reached a certain critical mass, survey experts were reluctant to rely on online surveys because Internet users were not truly representative of the population. However, while still relatively uncommon, more online surveys are being submitted as evidence in IP cases today, perhaps due to a number of factors:

The Internet itself has become a major marketplace for certain goods and services, making it the appropriate venue to conduct certain studies

• Today Internet usage in the U.S. approaches universality
• Panel companies have become more aggressive and sophisticated about building large and representative panels
• Online surveys have become a mainstay of the business community and market research industry

B. Case Studies

In the past four years, a small number of online surveys have been submitted in Lanham Act disputes, including both likelihood of confusion and false advertising cases. Importantly, I did not find any examples of online surveys submitted in trademark dilution cases. Dilution surveys tend to be conducted by telephone or mall intercept studies. Because dilution surveys are intended to capture top of mind association, the presence of a trained interviewer, who can control the timing and pace of the study, is a particularly important consideration when designing and conducting a dilution survey.

Citation	Case	Type of Online Survey
2007 WL 4239237 (E.D. Michigan)	Citizens Banking Corp. v. Citizens First Bancorp.	Likelihood of Confusion
242 F.R.D. 315 (E.D. Pennsylvania)	Merisant Co. v. McNeil Nutritionals.	False Advertising
2008 WL 755065(D. Kansas)	University of Kansas v. Larry Sinks, et al.	Likelihood of Confusion
2010 WL 5095676 (D. Minn)	3M Company v. Pradeep Mohan	Likelihood of Confusion
2011 WL 70562 (S.D.N.Y)	Pom Wonderful v. Orange Juice USA, Inc.	False Advertising

An analysis of these cases finds the following:

• The surveys represent a range of methodologies. For example online surveys to measure likelihood of confusion have utilized the "Eveready" design, the "Squirt" design, and the "Array" test.
• The online surveys have been conducted with both businesses and consumers.
• Opposing counsel routinely criticizes the use of the online survey methodology, among other things.

Despite opponent's criticism of the use of online surveys, the courts have generally not rejected the use of online surveys as a rule.

In fact, in none of the cases did the court criticize the online methodology per se. Instead, the courts evaluated these online surveys with the same lens that they use to evaluate other types of surveys for litigation. Rather that the specific method used, the courts tend to weigh more traditional factors. For example, as they do when evaluating a mall intercept study or a telephone survey, the court will evaluate the online survey for:

• Relevant universe
• Sample size
• Representativeness of the sample
• Replicating marketplace conditions
• Use of non-leading methods and questions
• Transparency in method and reporting

When designing an online survey for use in Lanham Act cases, it is important to consider the basic protocols that apply to any survey.

III. General Survey Protocols

A. Conclusions

The introduction of online surveys into IP litigation does not signify a radical transformation of the field. Counsel and their survey experts will still need to select the appropriate method for the issue at hand. Online surveys will be subject to the same standards of measurement and the same criteria as traditional telephone and mall intercept surveys. In light of this, it is helpful to review those criteria.

Twenty years ago, counsel and their survey experts worked without a clear road map when designing and conducting surveys for submission in Lanham Act cases. In the intervening years, as more surveys have been submitted into evidence, some general protocols have emerged, which apply to nearly all types of disputes. Importantly, these protocols do not only apply to surveys submitted in the court system. They originated in and reflect the protocols for the market research industry in general. The checklist that follows delineates the primary factors that should be considered when designing and executing any type of IP survey.

B. Checklist for Designing and Conducting a Survey for Use in IP Litigation

• Select the appropriate universe, including geographic area and target market
• Draw a sample that sufficiently represents that universe
• Replicate marketplace conditions in the survey methodology
• Double-blind survey, so that neither the interviewer nor the respondent is aware of the sponsor of the study
• Show any stimuli in context
• Develop non-leading questions
• Un-biased coding and tabulations
• Transparency in reporting, with complete description of methodology, sample size and frame, questions asked and survey responses.

Presented March 28, 2011 at 2011 Intellectual Property Law Spring Seminar
Sponsored by the Intellectual Property Law Section of the State Bar of Michigan, in cooperation with the Institute of Continuing Legal Education
Sara Parikh, Ph.D., Leo J. Shapiro & Associates