Reason Rupe Poll Methodology

Sampling Frame

Researchers use a triple-frame methodology to obtain representative samples of Americans 18 years of age and older with landline or mobile phones. Three components comprise the sampling frame: 1) a Random Digit Dialing (RDD) sample for landline phones, 2) a RDD sample for mobile phones, and 3) a listed sample from directory-listed landlines. Potential duplicate numbers are removed. Sampling frames are constructed to optimize the representativeness of the poll (age, race/ethnicity, region), cost, and fieldwork considerations.

There are benefits to using a triple-frame methodology by combining an RDD landline sample, a directory-listed sample, and an RDD mobile phone sample. This ensures the representation of both individuals with listed and unlisted phones and those with only landlines, only mobile phones, or both.  The proportions of RDD landline, RDD mobile, and listed numbers in the sampling frame are based on expected response and cooperation rates which vary across different characteristics, such as phone type, economic status, and race/ethnicity. Combining samples improves the representation of particular groups that can often be underrepresented in sampling frames. The listed sample provides groups of randomly selected phone numbers from likely African-American, Latino, and Asian households using tract density and surname directory-listed methodology, and a randomly selected sample from directory-listed numbers.

Another method of dealing with potential underrepresentation of particular groups in the sample is to weight those groups to be sure their impact on the overall results reflects their proportion in the population. The Reason-Rupe poll applies weights according to gender, age, ethnicity, and Census region based on the most recent US Census data.

Sample Details

Mobile RDD sample: RDD mobile phone samples are generated using dedicated mobile phone 1000-series telephone blocks (e.g. 215-653-7xxx). As required by law all mobile phone numbers are dialed manually.

Landline RDD sample:  To construct the initial sample frame from which RDD samples are then generated, residential exchanges and working blocks are determined. A block is defined as working if one or more listed residential numbers are found within that block. Within any given block there are 100 possible two-digit combinations that form the suffix in a complete telephone number. All exchanges and working blocks are then ordered by county, and within each county by region, metro area, and non-metropolitan area with exchanges and working blocks in ascending order. Telephone numbers generated for each county are proportional to the counties’ share of telephone numbers in the US.

Listed household sample: Directory-listed samples are obtained from GENESYS Sampling, which helps improve sample efficiency and response rates and to ensure a demographically representative sample. GENESYS Sampling compiles all directory-listed numbers, and then samples them to create smaller samples for purchase.  Random phone numbers are drawn from this database using similar techniques used for RDD databases. Listed random samples draw from each county so that the share of numbers drawn from that county are proportional to that county’s share of listed numbers in the U.S.  Additional information from other databases is appended to these files to provide information about the numbers’ associated households. This information allows researchers to increase representation of often under-sampled groups such as African-American, Latino, and Asian households. Listed samples are then compiled to include random samples of likely African-American, Latino, and Asian households using tract density and surname directory-listed methodology, and a random sample of directory-listed numbers.

Combining Samples

The proportions of RDD landline, RDD mobile, and listed samples used are based on the usage of landline and mobile phones, listed and unlisted numbers, and expected response rates across groups. Replicates from these three samples are released iteratively beginning first with RDD mobile and RDD landline samples. At specified intervals, replicates from the listed samples are released into the CATI system.

Randomizing respondents within households.  Interviewers who reach an individual on a landline phone use the “recent-birthday method” to randomize selection within the household. This means they ask if they can speak with an individual in the household 18 years of age or older who had the most recent birthday. The recent-birthday method is not used for mobile phone interviews, since most Americans do not share mobile phones. Interviewers do ask mobile phone respondents if they are 18 years of age or older.

Sources of Potential Total Survey Error

Particular groups who are systematically more difficult to access by phone than other groups introduce non-response bias into survey sampling. For example, low-income respondents may be more likely to work multiple jobs or work evenings or weekend hours when many telephone surveys are conducted. For this reason, the Reason-Rupe poll makes five attempts to contact working residential numbers and complete an interview. Calls are varied over times of the day and days of the week to maximize the probability of contacting each potential respondent. Coverage error can result from the sampling frame not including those in the population without phones. However, it is estimated that only about 2% of the population lacks access to a phone. Other sources of error may result from survey question wording, question order, question response options, interviewer error, timing, etc. The Reason-Rupe poll aims to minimize measurement error by rotating response options when possible, training interviewers, and vetting the questionnaire.

Survey Administration

Live interviewers use CATI technology to conduct survey interviews with respondents. Calls are made between 9:00 a.m. to 9:00 p.m. each day of the week. Since not all numbers in the sampling frame are working non-business numbers, (i.e. business numbers, non-working numbers, faxes, etc) up to five attempts are made to achieve contact with non-business working numbers.