Survey Methods

Survey Question Development

The total survey consisted of 118 questions. Thirty-three of these questions tested for respondents’ civic knowledge, while the remaining questions secured information on their public philosophy (39 questions), civic behavior (29), and demographics (17). Drs. Kenneth Dautrich, Richard Brake, and Gary Scott coordinated the development of these questions through a rigorous process of independent consultation, validity analyses, and scholarly review.

Thirteen of the 33 knowledge questions are taken from previous ISI surveys developed by ISI faculty advisors from universities around the country. Nine of the civic knowledge questions are taken from the U.S. Department of Education’s 12th grade NAEP test, and six from the U.S. naturalization exam. Two new knowledge questions were developed especially for this new survey and three are drawn from an “American History 101” exam posted online by www.InfoPlease.com.

Additional consultation and scholarly review concerning survey question development and final formulation were secured from members of the National Civic Literacy Board, and members of ISI’s professional staff.

Interview Technique and Sample Size

The research approach used to conduct this survey features a national random-digit-dial (RDD) sampling design. A total of 2,508 American adults were included in the sampling. They were interviewed by telephone from April 17 to May 10, 2008.

The margin of error for the sample of 2,508 adults is +/- 2.0 percentage points at the 95% level of confidence. When reporting on subgroups of adults (e.g., men, women, college graduates, etc.), the sampling error is higher. The sampling and interview methodology was designed by Dr. Kenneth Dautrich at the University of Connecticut.

Survey Population

The telephone survey data can be taken to represent a probability sample of all individuals who reside in households with residential telephone service in the United States.

Randomized Sample Selection

The RDD telephone component generated random samples of telephone households in the United States. Within each telephone household, one respondent was chosen utilizing the modified Trodahl/Carter in-house selection technique. We asked for the youngest male first, then if not available, the youngest female. This technique removes the control of the person answering the telephone deciding who participates in the survey.

For this study, Braun Research, Inc., was commissioned to conduct the telephone data collection utilizing RDD sampling through Survey Sampling International (SSI). Using SSI’s standard RDD methodology, a sample was drawn with a sample size equal to the number of completed interviews.

Weighting

In order to make appropriate projections to the survey population, weight has been applied to the data. Weight represents a compound probability that adjusts a sample to match the population characteristics of the civilian noninstitutionalized population of the United States. The weighting process adjusts for error inherent in the sampling methodology. The frame of the general population was aligned to the national population, as taken from the 2006 American Community Survey, and a weight was applied based on age, gender, education, and race. The full weighting calculations are detailed on the opposite page.

Analyses and Report Writing

The various diagnostics and statistical analyses of the raw data matrix, including statistical inferences based upon multiple regression analyses, were independently conducted and then jointly corroborated by Dr. Kenneth Dautrich at the University of Connecticut and Dr. Gary Scott at ISI. ISI visiting fellow Terence Jeffrey provided the technical writing for this report.