How you plan to use the information gathered should inform the decisions you make in designing, conducting, and analyzing the survey, as well as presenting its findings. Some possible objectives of your survey might be to inform local policymakers, provide community education on certain issues or programs, or enhance citizen participation in the policy-making process by:

gathering information on residents' attitudes and opinions

measuring behaviors and population characteristics

soliciting community reactions to policy proposals or solutions

having residents assess the effectiveness of programs, facilities, and services

informing people about new or planned public services

making residents more aware of community problems and their effects

providing residents access to the public policy-making process

providing an opportunity for communities to influence public decisions

In local government or in your community, an important outcome of conducting a survey might be increased commitment to the survey process, findings, and subsequent recommendations. By bringing citizens into the public decision-making process in the early stages, commitment to the results of the process-and the psychological satisfaction of participating in public decision making in a new and different way-can be obtained. For example, one result of recent planning surveys conducted in communities across Washington State was increased citizen excitement with the planning process.

Below we present a framework for organizing a survey and some factors to consider during its design and implementation. Specific applications to face-to-face, telephone, mail, and drop-off and pick-up survey methods are discussed. We wish to stress that conducting a survey is technical at all stages. Persons using the survey interview technique to gather information for the first time should consult an expert to assist in developing the survey design, writing questions, and planning the most appropriate implementation strategies (Table 1). Experienced local government officials, community group members, and local Cooperative Extension agents can design and implement community surveys if commitment to the survey process exists and time is taken to ensure the quality of the process. Quality assurance again may require consulting with an experienced surveyor.

 

TABLE 1.
Places to Find Survey Technical Assistance

Community College or University Political Science, Sociology, or Rural Sociology Departments

Washington State Partnership for Rural Improvement (Spokane, Yakima Valley, and Skagit Valley Community Colleges)

County Planning Departments/Commissions

County Cooperative Extension Offices

Regional Council of Governments

State Department of Community Development

Private Survey Consultants

 

WHY DO A SURVEY?

Surveys are used to collect information about a population. Usually, questions are asked to a subset or sample of the community's population and the responses generate either numeric (quantitative) or written (qualitative) information about the topics under examination. Quantitative and qualitative information are usually requested in the same survey, providing support and context for understanding the survey's findings. Quantitative information comes in numbers and answers questions like how many, how much, or to what extent. Qualitative information comes in written words where the respondent expresses his or her thoughts on a particular subject. Both types of information need to be interpreted, and the resources for interpretation are different (see Step 5: Analysis of Data).

The process of conducting a survey starts with asking, and more importantly, answering yes to the following questions:

Do you have specific questions?

Can people answer your questions?

Do you know exactly who you want to survey?

Is there a time deadline for completing the survey?

Do you know how the findings will be used?

The answers to these fundamental questions will help you avoid some of the pitfalls often encountered when doing a survey. They also will help you decide whether a survey is appropriate. Oftentimes, when reviewing these questions you will conclude that a data collection method other than the survey might better meet your specific information needs. Perhaps you can't identify specific questions-a little library work and several informal conversations could provide alternatives to doing a survey. If time is short, a survey is not a practical way to obtain information. The answers to these questions also might lead you to conclude that a survey should be combined with one or more additional data collection methods to gain insight into the topic(s) being investigated. We advocate using a multiple method strategy for gathering information ( Michael Butler and Howell 1980). The greater depth of insight provided when combining survey data with other types of data, such as already compiled reports, can enrich subsequent programming or policy development.

Surveys are one of several ways of collecting information about your community. There is a lot of already published information about your community that is available if you know how to look for it, ranging from census data collected by the national government to special reports compiled by local economic development agencies, utility companies, and even school and hospital districts (Table 2). In addition, if it isn't important to know how the majority of community members feel about an issue, perhaps selecting a limited number of "knowledgeable" persons for a focus group interview can provide the information you desire (Michael Butler and DePhelps 1994; Patton 1990).

 

TABLE 2.
Where to Find Published Information About Your Community

Public, Community College, and University Libraries

County Cooperative Extension Office

Local Historical Society

City Hall

Chamber of Commerce

Local Government Offices

 

SURVEY FUNDAMENTALS

The six steps presented below provide an outline for using surveys in your community. Because of the volume and technical nature of information available, we only provide an overview of the survey process. Each step may require technical expertise to ensure that your survey does not "break down." Each step is critical to the successful completion of the survey and none should be overlooked. There are many books available through your local library (public, community college, or university) that provide detailed information about survey research; several good sources are provided in the references at the end of this circular. Your county Cooperative Extension agent, local planner, or planning agency may also be useful resources in helping you find out what type of technical assistance is available in your community. For the discussion below, please keep in mind that the purpose of your survey should determine your design choices.

Step 1: Selecting the Sample

Sampling refers to the process of selecting a subset of persons who are representative of the community population so that you can capture the variety of behaviors, attitudes, or beliefs of the community without having to include everyone directly in the survey. Sampling is one of the most important and frequently misunderstood aspects of doing a survey. Sampling can be a very technical endeavor. Because sampling is so important to the survey process, it is discussed here in some detail. In the end, remember that the purpose of your survey should determine your sampling strategy.

When selecting your sample, you might decide that rather than individual persons, you're interested in surveying households or other community institutions. A key question for you to ask is: Do you have a list of all of the possible people who you need to get information from? Many community lists are available from local government offices, utility companies, school districts, or voter precincts.

Selecting your survey sample can take one of two forms-convenience or random assignment. Convenience samples are selected without regard to the community's population. You might decide to survey the people who cross Main Street and First during the lunch hour on Wednesday, or the people who come out of the local grocery store on Thursday afternoon, because those are the best times for you (and your survey budget doesn't permit mailed surveys). Or, you believe that all business people feel the same way about skateboarding on sidewalks downtown and so you only survey members of the Rotary (a purposive sample). In neither case are you worried about whether your sample, and therefore your survey, is representative of the community or community subgroup, e.g., business people.

Random assignment, on the other hand, refers to a technical selection process in which each person in the community (in technical terms, the population) has the same (or a known) opportunity to be selected as a survey respondent. In a survey of the general population (a survey not targeted at specific subgroups of the community, such as women residents, residents over age 60, residents of certain neighborhoods, etc.), you would like your sample to accurately reflect your community. Accuracy is important because it permits you to generalize from your sample to the entire community. This is because through random assignment, your sample, and therefore your survey's findings, are representative of the whole community.

The size of the sample depends on how accurate your results need to be, and on your budget. Survey professionals will tell you that you can buy better accuracy; in general, however, there are certain minimum limits that should be observed. Samples of fewer than 30 persons are generally too small to be useful; experienced surveyors tend to regard 100 or so as the minimum sample size when the population is large. At the other end, the maximum practical size of surveys tends to be around 1,000. In general, your sample can be smaller if the population is relatively homogeneous (similar), only rough estimates of what the community thinks or feels are needed, there are few major commitments to be based on the survey information, and your budget and/or time constraints provide additional limits on your survey. If differences are important, the attitudes of several subsets of the population are relevant, and the potential consequences of using the survey in public decision making are costly, selecting your sample becomes critical to the success of your survey.

For example, single adults, families, and senior citizens are each, as a group, likely to have very different perspectives regarding the need to have after-school activities available to school-age children. If the purpose of your survey is to find out what your community members think about after-school programs, you need to include enough single adults, families, and senior citizens to be accurate. If you are not concerned about community subgroups, a sample size of 100 is fine! If you need to be more accurate in terms of special groups (single adults, families, senior citizens), your sampling strategy becomes more complex. One common contingency is stratification, determining which characteristics of the population (i.e., which subgroups) are key to your survey and then dividing the survey population into such groups or strata. Random selection would then proceed within each of the strata. Stratification and other sampling methods are discussed in greater detail by Dillman (1978) and Fowler (1988).

Step 2: Wording the Questions

The first activity in writing a question is to identify the types of information needed. There are basically four different types of information that can be collected using a survey. First, there is information about attitudes, or what people say they want. For example, you may need to know the extent to which local residents favor the use of specific methods of collecting recyclable materials in your community. Attitude questions could be used to learn whether residents are "for" or "against" recycling different materials. Second, information can be collected about the beliefs of people. For example, do residents think that recycling household waste will improve the environment. This is information about what people think is true. A third type of information is about the behavior of people, for example, the frequency which residents place recyclable materials at the collection center. Fourth, information can be collected about the attributes of people, or what they are. Some common attributes collected in surveys are the respondent's gender, age, and status as a homeowner or renter. Using the four kinds of information in supportive ways can provide a great deal of insight about the population being surveyed.

There are four types of question structure that can be used when writing a survey: (1) open-ended, (2) close-ended with ordered response categories, (3) close-ended with unordered response categories, and, (4) partially close-ended (Figure 1) (Dillman 1978). Each question structure has a specific purpose.

Open-ended questions have no answer choices provided. Instead, respondents are asked to answer the question in their own words. Besides allowing respondents to express themselves freely, this type of question can be used to elicit precise information.

Close-ended questions with ordered response categories provide a gradation of responses for a single concept. For example, the gradation might range from strongly agree to strongly disagree. They tend to be specific and restrict respondents thinking to a very narrow aspect of an issue or situation in a limited way.

Close-ended questions with unordered response categories ask for information about alternate choices, such as the location of a community swimming pool. These questions do not limit the respondents response to gradations of a single concept.

Partially close-ended questions are used when there are many possible responses but the survey is only addressing a few of them, such as the use of specific community services-tennis courts, basketball courts, and specific programs at the recreation center. By asking the respondent "Are there any other services that you use?" you can avoid listing all of the possible services that are available in the community or forcing respondents into categories that they do not fit. Often, questions of this type simply contain an "Other ..................." category.

Asking questions also requires technical knowledge. Vague terms in questions or response categories, the use of double negatives, assuming too much knowledge, writing cryptic questions, and the use of broad categories to overcome objections are only a few potential problems often found in survey questions. A short list of bad and better questions is provided in Figure 2. A general rule of thumb is to keep your questions and response categories short and simple.

Step 3: Constructing the Questionnaire

Getting responses to questions requires that the questionnaire be presented in a way that entices the respondent to fill out the survey. Would you feel like answering your questionnaire? Poor copying (off-centered or blurry pages), dark paper making the type hard to read, or not identifying the survey's sponsor can damage the credibility of your survey. The expression "the medium is the message" is true in the construction of your survey questionnaire (e.g., its size, shape, and appearance), especially if it is being given to the respondent without the interviewer administering it such as with a mail survey.

Just as important as being visually pleasing, is where different types of questions appear in the questionnaire. Frequently we find questionnaires beginning with requests for demographic information (age, marital status, and so on). These types of questions, being rather personal, belong at the end of the questionnaire and should be preceded with a justification statement. For example, "Finally, we would like to obtain some additional information to help with the statistical analysis." If the respondent is told that the purpose of the survey is assessing residents' needs for specific community services, questions about services should be at the beginning of the questionnaire. The more controversial the questions, such as how to pay for the services needed, can be raised after the respondent has made the effort to answer the initial questions. Discovering "irrelevant" or controversial and possibly offensive questions at the beginning of the questionnaire may lead the respondent to discard it.

FIGURE 3.
Survey Checklist

use booklet form (long sheets of paper folded once in the middle, printed on both sides on white or off-white paper) the front page is a cover page (title, your logo, directions for responding, name and address of the sponsor) the back page is "clean" (it offers little more than an invitation for comments) the first question pulls in the respondent (e.g., easily understood, interesting, applicable to everyone, neutral) questions are in some order (descending importance based on your purpose-in this case respondent attributes such as age, occupation, level of education, etc., are last) the questionnaire layout should be consistent (e.g., all close-ended responses to the right of the question) the questionnaire should be "conversational" (similar questions grouped together, transitional statements like "Next we would like you to assesS the quality of services currently available" are used, there is a flow between sections, objectionable questions are positioned after less objectionable ones) the questionnaire should be pre-tested before it is administered

The checklist in Figure 3, based on research conducted to enhance survey response rates for mailed surveys, is useful to consider when drafting your survey questionnaire. Whether you mail your survey or administer it face-to-face, a "clean" document is easiest to answer.

Figure 4 provides an example of a well-designed questionnaire.

Step 4: Implementing the Survey

An important decision that needs to be made is how to conduct your survey. Four survey interview approaches are available: (1) face-to-face, (2) telephone, (3) mail, or (4) drop-off and pick-up. Your decision as to which method to use depends upon the purpose and timeline of your research, the distance of respondents, and the resources available (e.g., some methods cost more, some are more labor intensive). Each approach has benefits and drawbacks, some of which are listed below.

Face-to-face interviews provide in-depth, person-to-person exchanges. This is the most useful method when your respondents are close by and not likely to respond to other approaches (perhaps they are unable to read or have no telephone), but it is also the most costly. Volunteers can be used to conduct the survey, but extensive training and monitoring are needed to maintain consistency and quality in the interviews.

Telephone interviews permit the benefits of face-to-face interviewing at a lower cost, and when conducted from a single location, permit the monitoring of multiple interviewers. The disadvantages to telephone interviewing include sampling limitations (e.g., some households don't have telephones), intrusiveness of this method (some people will not respond over the telephone), questionnaire constraints (you cannot use visual aids), and appropriateness of sensitive questions.

For mail questionnaires, the chief advantages are relatively low costs, minimal staff and facilities are needed for conducting the survey, access to widely dispersed samples is provided, and respondents can complete the survey at their convenience. Disadvantages include difficulty in gaining the cooperation of some groups, the interviewer is not involved in data collection, and you need to have good mailing addresses for the sample.

Drop-off and pick-up questionnaires have the advantage of reaching all respondents and allowing time for questionnaire completion. Disadvantages include the interviewer not being involved in data collection and requiring volunteers to help drop and retrieve the questionnaires (in smaller communities collection sites can be predetermined and noted to respondents).

Step 5: Analysis of the Data

Once you have received responses to your survey, you need to make some order out of the many, possibly diverse statements (qualitative data) or numbers (quantitative data) gathered. Data analysis refers to the set of technical (statistical) procedures that help summarize attitudes and activities and make comparisons among them relatively simple. Personal computers and user-friendly statistical software packages provide many opportunities to analyze community data with a bit of technical assistance. Data analysis generally has one of three purposes-to describe the data you have collected, to explore new questions about relationships among the issues surveyed, or to find evidence that helps explain any patterns found in the data. The type of data analysis that you do should match the purpose of your survey.

One note of caution: it is unlikely that everyone will complete your survey regardless of the approach you use. As a result, nonresponses (and nonresponse rates) must be a part of your analysis. A general rule of thumb for dealing with nonresponses is never assume that the people who did not respond are just like the people who did respond. A major source of error in survey findings comes from not dealing with nonresponse rates and assumptions made about people who didn't respond. This is another area where technical assistance can be helpful.

Step 6: Presenting the Findings

Presentation requires understanding your audience, both in terms of what they value and how well-informed they are regarding the topics of your survey. Some of the potential audiences for your survey findings are:

• elected officials (municipal, county, state)
• government agencies, program managers, and staff
• advisory boards and committees
• community groups (PTAs, church groups, homeowners associations)
• local business groups
• the media

The key to effective presentation is separating the findings from the methodology. Some useful reporting formats include: technical reports, memoranda, news releases, presentations at public meetings or staff workshops, and videotapes. The use of charts, diagrams, or other visual aids is suggested regardless of reporting format. Fowler (1988) is a good source for information about data presentations.

SUMMARY

Surveys can provide useful information to enlighten policy-making and program development and evaluation. Surveys also provide an important means for public involvement in community decision making and educating people about issues and services available.

Conducting a survey is a technical endeavor and often requires assistance from people who are familiar with survey fundamentals such as sampling procedures and questionnaire design. Communities may want to team with university-based researchers to design their surveys. County Cooperative Extension agents are a good source of information and an excellent connection to survey researchers in land grant universities.

It is important to be clear as to the purpose of the survey and to develop well-stated questions that people can answer. Carefully thinking through the use of the findings when designing a survey is also important. Lastly, using appropriate means of communicating with different audiences and users of the findings is critical to assure that the investment in data collection has been worthwhile. Achieving success in the survey process is based upon carefully executing each step ranging from deciding about how to meet your information needs to presenting the survey findings.

 

REFERENCES

Dillman, Don A. 1978. Mail and Telephone Surveys: The Total Design Method. New York: John Wiley.

Fowler, Floyd J., Jr. 1988. Survey Research Methods (Rev. Ed.). Newbury Park: Sage Publications, Inc.

Michael Butler, Lorna and Colette DePhelps. 1994. Focus Groups: A Tool for Understanding Community Perceptions and Experiences. Community Ventures: Partnerships in Education and Research Series, WREP 0128. Pullman, WA: Washington State University.

Michael Butler, Lorna and Robert E. Howell. 1980. Community Needs Assessment Techniques. Coping With Growth Series, WREP 44. Corvallis, OR: Western Rural Development Center, Oregon State University.

Patton, Michael Quinn. 1990. Qualitative Evaluation and Research Methods. Second Edition. Newbury Park: Sage Publications, Inc.

 

Community Ventures: Partnerships in Education and Research Circular Series Topics

---

Community Surveys

Sondeo or Rapid Reconnaissance

Delphi Technique

Focus Group Interview Technique

Volunteers as Partners in Community Action

Techniques for Successful Public Involvement

Diverse Partners in Planning and Decision Making

Using Case Studies

Organizing for On-Farm Research and Education

Interest-Based Participation Through Negotiation

Role of Government Agencies in Public Decision Making

Citizen Advisory Systems

---

Community Ventures Series Coordinators:

Lorna Michael Butler

Colette DePhelps

Kelsey Gray

---

Community Ventures Manuscript Reviewers

Extension invites participation in its programs and offers them to all people without discrimination.

WREP0132 March 1995 $1.00

---

TOP