The delphi was designed for non-interacting groups. Non-interacting groups can include groups whose members are geographically distant, groups whose members tend to clash, or groups in which status differences might affect decision making. The delphi is useful when the issue facing the community is a "hot button" topic. The delphi procedure, as a social technology, does require some expertise to conduct.

This circular provides an overview of the delphi procedure to help local government officials and staff, community group members, and Cooperative Extension agents decide if this particular technique can be useful in community or regional decision making. First, the strengths and weaknesses of the delphi are discussed and an overview of the delphi procedure is provided. This is followed by an illustration of one particular use of the delphi and examples of different rounds of the procedure, including scoring techniques and the provision of feedback, are provided to illustrate how the delphi is applied. The circular concludes with a summary of tasks potentially addressed by the delphi and types of policy participation obtained by applying this procedure.

---

Delphi Technique Fundamentals

---

The delphi, devised in the 1950s, was first used as a procedure for prediction.1 More recently, the delphi has been used in a variety of different forums including land-use planning, regional policy making in areas such as transportation, social service programming in education and health care, and in organizational restructuring. These applications stem from one of the delphi's main objectives-to obtain a reliable consensus of opinion from a group of "experts."

Strengths. The delphi is used when it is important to have pooled judgment, following the maxim "two heads are better than one." The delphi has three important strengths. First, anonymous, individual responses are at the heart of the procedure. The delphi is used to bring participants together without bringing them into the same room, avoiding the costs and hassles of traveling to and from meetings; traveling 100 miles into a neighboring county can be disruptive, even in good weather! In addition, by not bringing the participants together in the same room, the delphi reduces the effect of dominating individuals (the status problem) and allows the group to share responsibility. Shared responsibility is a tonic for developing consensus. Shared responsibility also promotes satisfaction through participation in and ownership of the resulting decision(s).

Second, controlled feedback through several rounds of the procedure reduces direct confrontation and the disadvantages that conflict leads to-quickly accepting or dismissing other opinions, focusing on personalities rather than the issues at hand, or closing-off discussion of novel or different ideas are recognized as serious problems for interacting groups.

Third, statistical group response, or the tallying of each participant's valuation of the delphi responses, ensures that each person's opinion is reflected in the final response. This contributes to the shared responsibility for not only the outcome of the delphi, but also in the process that eventually provides the outcome. These three features of the delphi make it a useful procedure for developing ownership and consensus, or at minimum, finding out whether there is any common ground for anchoring consensus.

Weaknesses. The delphi also has three important limitations. The first limitation is that participants must have written communication skills. Because the delphi is grounded in written communication, it is critical that all group members can read and write. Second, the delphi is labor intensive and time consuming. If time is short, the delphi cannot be used. The procedure, particularly if mailed questionnaires are used, can take 45 days to administer over a 12-week period from decision-to-go to the "final report." As a result, the delphi is often administered in conjunction with other meetings, e.g., regularly scheduled task force meetings, advisory board meetings, PTA meetings; the delphi "homework" helps to preserve continuity between meetings (Figure 1).

 

Tip:

Be brief and deal with the delphi at the end of the scheduled meeting, after regular business has been conducted. The next round of questionnaires also can be handed out at this time, rather than being mailed. And, if participants have access to a computer and a modem, electronic mail can also be used to facilitate timeliness (Table 1).

Figure 1
Delphi Instructions

The third limitation of the delphi is the need for highly motivated participants. The quality of responses depends upon the interest and motivation of the participants, particularly since there is no one to make sure the delphi questionnaires are completed and mailed back.

---

Delphi Mechanics

---

The second limitation noted above began by describing the delphi as a labor intensive procedure. It is now time to provide an overview of the delphi, task by task, round by round. Commitment to the process is critical to its success, and commitment is required by both the participants and the person (or group) responsible for keeping the procedure moving. If a single person (or monitor) is used to facilitate the procedure, that person will need to have the time to analyze the responses, develop the questionnaires, and generally keep the process moving. Community work groups can be developed to monitor the process; in this case it is still a good idea to have one person who is responsible for facilitating the process: scheduling analysis meetings; making sure that all materials needed are available; and having some degree of control over the questionnaires and letters of instruction.

Preliminary Steps. The delphi requires three preliminary steps. First, the purpose of the delphi must be clear so that the initial question (the delphi question) can be developed. The delphi question must be able to get the information that is desired from the participants (known as panelists). Since the participants respond on their own, the delphi question must be stated clearly. Delbecq (1975) suggests the following three probes to help you develop and focus your delphi question.

• Why are you interested in this particular delphi?
  
  
• What do you need to know that you don't know now?
  
  
• How will results from the delphi influence decision making once the procedure is completed?

Second, you need to determine who should participate in the delphi as a panelist. Because you are seeking expert judgments, it is important to identify "experts." Experts differ by situation, so it is important to have a pre-determined set of qualifications desired of panelists. If a community group is nominating experts, the group should first decide on the desired qualifications. In many communities, it is possible to pull a work group together representing different community interests who can then develop the delphi question and nominate experts. This same group might continue to work together throughout the delphi, analyzing the information and later drafting or reviewing the delphi's findings.

Third, how many panelists should actually participate in the delphi? If the group of experts is fairly homogeneous (sharing similar opinions) then ten to fifteen panelists will be enough. If there are diverse interests present among the experts, then the size of the group will need to be increased to ensure balance. For most community-oriented delphis, 30 is about as large a group as you will want to have. Keep in mind, the delphi is a labor intensive procedure; the greater the number of panelists, the greater the information load, both for the panelists to consider and for the monitor to analyze, so it is important to carefully consider the number of panelists. Once you have a list of potential panelists, personally contact each member of this group and discuss the delphi, its expected outcomes and uses, and ask for that person's commitment to the process. Most people are pleased to be considered experts and will be happy to participate.

Delphi Mechanics. The delphi is basically a series of questionnaires addressing a particular topic. Consensus is developed through several iterative steps. First, panelists work independently, then their individual efforts are summarized by the monitor, then the summaries are fed back to the panelists for more work and so on. The mechanics of the procedure are described below.

Each panelist receives a copy of the delphi question and a set of instructions on how to respond (keep the instructions short and to the point), what the response time is (ten working days is sufficient), and where to send the completed information (usually via a stamped, self-addressed envelope that you provide). Figure 2 shows a typical set of instructions for the initial questionnaire. This is known as Round 1.

Figure 2.
Initial Questionnaire

---

---

Once the information is returned, it is analyzed. If a work group of community residents is analyzing the responses, the monitor should make sure that adequate workspace is provided. The multipurpose room in a school if it has tables, a conference room in city hall, or wherever the Rotary meets are possible workspaces. Each item on each of the returned questionnaires needs to be considered separately; the individual questionnaires can be cut (if they are not written on both sides); or each idea on each questionnaire can be transferred to a 5x7 card. Then, all like ideas are clustered. The cluster is given a name indicating what idea it represents. This part of the analysis can be conducted by individual members of the work group.

Next, all of the named clusters that work group members have identified must be organized; similar clusters (or like ideas) are grouped together. The entire work group participates. The reorganized clusters now include all of the responses received in the initial round; one person (the monitor) creates a single master list of responses. This master list then becomes the questionnaire for Round 2. The new questionnaire now highlights areas of agreement and disagreement, and indicates the level of consensus. Consensus, or the number of panelists "saying" similar things, is an outcome captured in the clustering process.

The Round 2 questionnaire is then distributed to the panelists, thanking them for their initial responses and once again including instructions on how to respond in this round, when to respond, and a self-addressed stamped envelope. Typically, Round 2 asks the panelists to review the items and then prioritize, rank, argue for or against, and/or clarify the items on the questionnaire. Priority voting space is usually indicated to the left of each item. Space for comments is provided on the right side of the page or under each item. Space for new items is provided at the end of the questionnaire. Figure 3 shows an example of this questionnaire.

Analysis of Round 2 requires a tallying of "votes" (the ranking of items) and a summary of new comments. This ranking then becomes the basis for Round 3. The Round 3 questionnaire goes out like the previous round asking panelists to make a judgment of the importance of the items. In this way, closure is provided and areas of common ground and potential barriers are identified (barriers are found in the areas of disagreement).

Analysis of Round 3 then permits the monitor or community work group to draft the findings of the delphi. A fourth round may be necessary. A summary of the results should go to each of the respondents along with a final thank-you letter.

Fifty-seven recommendations were received in the initial round of the delphi process. Please indicate how you would rank each of the following recommendations using the scale provided below. Add any new recommendations in the space provided at the end of this questionnaire.

Ranking:

4	Highly relevant recommendation to SANDAG
3	Relevant recommendation
2	Insignificantly relevant recommendation
1	No relevance; should not be a recommendation
	One or more people from Mexico should be on the SANDAG board or at the meetings to provide needed input from the Mexican perspective.
	SANDAG needs to more actively seek meaningful participation from Mexican municipal and state representatives.
	Make Tijuana an official member of SANDAG.
	Tijuana should have official representation on the SANDAG Board of Directors.
	SANDAG should assist all jurisdictions in transborder affairs matters.
	SANDAG should act as a liaison agency with Mexico for the region's smaller cities.
	All SANDAG maps should show comparable features (data from comparable years) for both sides of the border.
	SANDAG should provide an institutional mechanism for bringing "south of the border" views into regional planning, and should participate in regional planning efforts conducted "south of the border."

Case Study: Mexican-American Border-Related Issues: Using the Delphi to Come to Consensus

This example of using a delphi took place between October 1989 and February 1990. The delphi was used to develop recommendations for a regional solution to a volatile public issue. This case is used to illustrate how a delphi procedure can be modified and still achieve its purpose-to obtain a reliable consensus of opinion from a group of "experts." The example parallels the previous discussion.

Purpose. The delphi was used in conjunction with an ongoing task force to permit continued learning while developing a set of recommendations that the panelists could agree upon. The delphi was used in part to reduce conflict (anonymous responses) surrounding a "hot button" issue (illegal border crossings), allowing the task force members to be frank in the interactive group setting while still sharing in the responsibility of developing serious recommendations.

Qualifications. The task force was made up of persons who were knowledgeable and interested in the particular issue. Because this was a regional government-sponsored task force, officials and staff representing the different member jurisdictions of the council of governments (local and county governments) and representatives of border-focused state and national agencies. In addition, a number of other interested persons attended all of the task force meetings (university faculty, businesspersons, representatives of nonprofit community institutions, and additional staff from the municipal and county agencies that dealt specifically with border issues).

Number of Respondents. The task force consisted of about 15 members. The additional attendees added about ten persons to the group. When the delphi started, which was several weeks into task force's meetings, the co-chairs asked everyone to participate. The delphi "panel" had 25 members, although not all panelists participated in each round.

Mechanics. The initial questionnaire was distributed at the end of a regularly scheduled meeting (Figure 2).

The delphi "question" yielded 57 responses, or items, which were typed as a single list and sent to all panelists. In the second round panelists ranked the recommendations in importance and added new recommendations or rephrased existing ones (Figure 3). Figure 4 shows the first page of results. Note how detailed the information about the analysis is, and that the questionnaire items are reordered and listed according to ranked score. In the third round, the ranking of recommendations continued, and the 57 items were clustered into 30 items. Three new items were added at the end of round 1 and showed up at the end of the Round 2 questionnaire (not illustrated). Clarifications, as noted in the description of Round 3 analysis, are indicated by underlined text (Figure 5).

---

---

Finally, in Round 4 the panel was asked to rank the relevance of the recommendations to the regional council of governments, to other levels of government, or as "no action." The results of Round 4 are shown in Figure 6; there were a total of 42 recommendations at the end of the delphi.

---

What Did the Delphi Do? The nature of the recommendations and the sometimes contentious debates surrounding certain issues during meetings, particularly those related to (1) legal and illegal immigration (e.g., access to housing and health services, crime) and (2) questions of administrative jurisdiction, illustrate the difficulty of achieving expert consensus. Indeed, oftentimes not only did officials from different levels of government disagree, but officials from the same level disagreed. Yet the delphi allowed for a set of recommendations to be obtained in an orderly manner, and with a certain degree of consensus. The lowest scoring recommendations showed the disagreements-where there was no consensus-and where potential barriers to future policy actions might arise.

In this example, the delphi was run in conjunction with a task force, but it allowed the panelists time to reflect on the information that was discussed at the twice monthly meetings. The weaknesses of interacting groups were avoided by the delphi, and the monitor, in this case a staff planner with the regional council of governments collected and analyzed all of the information in between rounds.

---

Summary

---

The Delphi technique is a group technique for idea generation and consensus building. It requires technical expertise to administer and implement, but can provide excellent results. Some of the tasks that the delphi can help a community address are:

• determining priorities, setting goals, or establishing future directions;
  
  
• designing other needs assessment strategies;
  
  
• improving service delivery;
  
  
• evaluating programs or alternative plans; or
  
  
• aggregating judgments or views of special interests or opposing groups.

The delphi allows these tasks to be accomplished because it allows for the participation of community residents-both as experts and as analysts-in several rounds of "policy thinking," including:

References

---

Delbecq, Andre L. and others. 1975. Group Techniques for Program Planning. Glenview, IL: Scott, Foresman and Co.

Moore, Carl. 1987. Group Techniques for Idea Building. Newbury Park, CA: Sage Publications.

Michael Butler, Lorna. 1992. Participatory Group Approaches to Situational Assessment: Nominal Group Process. Pullman, WA: Cooperative Extension.

Michael Butler, Lorna. 1992. Participatory Group Approaches to Situational Assessment: Delphi Technique. Pullman, WA: Cooperative Extension.

---

Community Ventures: Partnerships in Education and Research Circular Series Topics

---

Community Surveys

Sondeo or Rapid Reconnaissance

Delphi Technique

Focus Group Interview Technique

Volunteers as Partners in Community Action

Techniques for Successful Public Involvement

Diverse Partners in Planning and Decision Making

Using Case Studies

Organizing for On-Farm Research and Education

Interest-Based Participation Through Negotiation

Role of Government Agencies in Public Decision Making

Citizen Advisory Systems

---

Community Ventures Series Coordinators:

Lorna Michael Butler

Colette DePhelps

Kelsey Gray

---

Community Ventures Manuscript Reviewers

Extension invites participation in its programs and offers them to all people without discrimination.

WREP0131 March 1995 $1.00

Top