Sample Design

Researchers use a dual-frame methodology of landline and cellular random digit dial (RDD) samples to obtain representative samples of Americans 18 years of age and older in the continental United States who have access to landline or mobile phones. Both samples are provided by Survey Sampling International, LLC (SSI) according to PSRAI specifications.

Numbers for the landline sample are drawn with equal probabilities from active blocks (area code + exchange + two-digit block number) that contain three or more residential directory listings. The cellular sample is not list-assisted, but was drawn through a systematic sampling from dedicated wireless 100-blocks and shared service 100-blocks with no directory-listed landline numbers.

Contact Procedures

Live interviews are conducted with respondents. As many as seven attempts are made to contact every sampled telephone number. Sample is released for interviewing in replicates, which are representative subsamples of the larger sample. Using replicates to control the release of sample ensures that complete call procedures are followed for the entire sample. Calls are staggered over times of day and days of the week to maximize the chance of making contact with potential respondents. Each phone number receives at least one daytime call when necessary.

For the landline sample, interviewers ask to speak with the youngest adult male or female currently at home based on a random rotation. If no male/female is available, interviewers ask to speak with the youngest adult of the other gender. This systematic respondent selection technique has been shown to produce samples that closely mirror the population in terms of age and gender when combined with cell interviewing.

For the cellular sample, interviews are conducted with the person who answered the phone. Interviewers verified that the person was an adult and in a safe place before administering the survey.

Weighting and Analysis

Weighting is generally used in survey analysis to compensate for sample designs and patterns of non-response that might bias results. Samples are weighted to match national adult general population parameters. A two-stage weighting procedure is used to weight dual-frame samples.

The first stage of weighting corrects for different probabilities of selection associated with the number of adults in each household and each respondent's telephone usage patterns.[1] This weighting also adjusts for the overlapping landline and cell sample frames and the relative sizes of each frame and each sample.

This first-stage weight for the ith case can be expressed as:

Where SLL = size of the landline sample
SCP = size of the cell phone sample
ADi = Number of adults in the household
R = Estimated ratio of the land line sample frame to the cell phone sample frame

The equations can be simplified by plugging in the values for SLL and SCP. Additionally, we will estimate of the ratio of the size of landline sample frame to the cell phone sample frame R = 0.70.

The second stage of weighting balances sample demographics to population parameters. The sample is balanced to match national population parameters for sex, age, education, race, Hispanic origin, region (U.S. Census definitions), population density, number of adults in household, and telephone usage. The basic weighting parameters come from a special analysis of the Census Bureau's Annual Social and Economic Supplement (ASEC) that includes all households in the continental United States. The population density parameter is derived from Census data. The telephone usage parameter comes from an analysis of the July-December 2011 National Health Interview Survey. [2]

Weighting is accomplished using Sample Balancing, a special iterative sample weighting program that simultaneously balances the distributions of all variables using a statistical technique called the Deming Algorithm. Weights are trimmed to prevent individual interviews from having too much influence on the final results. The use of these weights in statistical analysis ensures that the demographic characteristics of the sample closely approximate the demographic characteristics of the national population.

The survey's margin of error is the largest 95% confidence interval for any estimated proportion based on the total sample— the one around 50%. For example, the margin of error for the September 2012 entire sample is ±3.7 percentage points. This means that in 95 out every 100 samples drawn using the same methodology, estimated proportions based on the entire sample will be no more than 3.7 percentage points away from their true values in the population. It is important to remember that sampling fluctuations are only one possible source of error in a survey estimate. Other sources, such as respondent selection bias, questionnaire wording and reporting inaccuracy, may contribute additional error of greater or lesser magnitude.

[1] i.e., whether respondents have only a landline telephone, only a cell phone, or both kinds of telephone.

[1] Blumberg SJ, Luke JV. Wireless substitution: Early release of estimates from the National Health Interview Survey, July-December, 2011. National Center for Health Statistics. Jul 2012.