Background:
HAVA, NIST, and usability
The 2002 Help America Vote
Act (HAVA) assigns the National Institute of Standards and Technology (NIST) a
key technical role in assisting with nationwide improvements for voting systems.
(See http://vote.nist.gov .) In particular, NIST provides
research support to the Technical Guidelines Development Committee (TGDC)
established by HAVA to develop voluntary voting system guidelines which are
then delivered to the Election Assistance Commission. This paper describes some of NIST’s work improving the
usability and accessibility of voting systems in response to the TGDC’s
resolutions #08-05 Usability Guidance for Instructions, Ballot Design, and
Error Messages and #06-05 Accommodating a Wide Range of Human Abilities. (See http://vote.nist.gov/Official%20w-o%20signature.pdf
for the full text.)
These resolutions recognize
that all voting systems (not just the equipment designated as "accessible")
must be designed for a large range of abilities. Voters may
·
be inexperienced,
·
arrive at the polling
place tired and stressed,
·
have limited English
proficiency or low literacy, or
·
have reading, learning,
or cognitive disabilities.
A
good overview of how readability of information can be enhanced for people with
cognitive disabilities through clear, concise language can be found in a
publication of the National Center for the
Dissemination of Disability Research (NCDDR 2003).
Ballot
language is important
Clear instructions are a
necessary part of the voting process whether voters are working with paper or
electronic ballots.
Voters are exercising an
important right as citizens. If
they cannot understand how to use their voting materials, they may not be
successful in voting for the candidates and positions of their choice.
They may make mistakes that
invalidate their ballots. They may vote for candidates or positions that are
not the ones they meant to vote for.
They may be intimidated by unclear or insufficient instructions and give
up without voting. Indeed, they
may choose not to try to vote.
Language and design are
critical factors in the usability of any document, whether that document is on
paper or on a computer screen. This
is as true of ballots as of any other document. Further, usability is as important as security and
reliability in providing a successful experience to voters.
Little
research exists on ballot language
In 2002, R. Michael Alvarez
wrote:
Unfortunately, I am aware of no research on voting
instructions… I have encountered no academic research on this important
aspect of ballot design. This
research vacuum needs to be filled, quickly.
The authors of this paper
have begun to fill that research vacuum.[2]
This paper is a report of the initial steps in research on voting instructions.
Relevant
research can be found in many related disciplines
Although little has been
done to study the language of instructions for voting, a large body of relevant
research exists on how people read, on the effectiveness of writing in plain
language, and on how to present instructions that people follow successfully.
We can draw best practices for writing instructions from
many sources in fields such as:
· cognitive psychology
· human-computer interaction
· information design
· linguistics
· reading
·
technical writing
For reviews of relevant
research before 1980, see (Felker, et al., 1980; Felker, et al., 1981). For a review of work done in the 1980s, see (Carroll, 1990;
Redish, 1993). For a more recent
review of work in information design and writing, see (Schriver, 1997). Also
see resources at www.plainlanguage.gov,
www.plainlanguagenetwork.org,
and www.usabilityprofessionals.org/usability_resources.
Many
instructions to voters violate best practices
A recent study (Redish,
2005) shows that many ballots in the United States include instructions that
are more likely to hinder than to help voters.
Methodology
We reviewed
· more than 100 paper ballots
· sample ballots on three Direct Recording Electronic
voting machines (DREs)
·
the online demonstration
of voting on the web site of a fourth DRE
The paper ballots came from
all 50 states and the District of Columbia. They encompassed elections from 1998, 2000, and 2004, represented
a wide range of voting situations.
Some of the paper ballots can be found at http://vote.nist.gov/ballots.htm .
In working with the DREs, we also reviewed
·
the instructions for
voters on how to move through the ballot
·
messages to voters that
might appear as they are voting (for example, asking the voter to confirm a
change in straight party voting from one party to another party)
· system messages that might appear to the voter or to a
poll worker when something goes wrong
We reviewed all of these
instructions and messages in light of best practices which we drew from the
research in the disciplines cited above.
Results
Most, if not all, ballots,
both on paper and on screen, violate at least some of the best practices. In this paper, we have room to show only
a few examples from our study. See
(Redish, 2005) for the complete description of our analysis of the gap between
best practices and typical ballot instructions. See (Redish, 2006) for preliminary work towards what might be
guidance for vendors of voting equipment and/or for election officials.
Note that the examples of
poor practice in this paper come from actual DRE or paper ballots used in
United States elections. We do not
identify the specific election or equipment because our purpose is not to
single out any particular election official or any particular equipment vendor
for poor practice. Our purpose is
to use these examples to help all election officials and vendors improve the
instructions they give to voters.
Best Practice 1.
Tell people about consequences before they are likely to act.
Research on instructions
shows that many people act as soon as they see something that seems relevant. They do not read further to see if there
are conditions, constraints, or consequences for those actions (Carroll, 1990;
Redish, 1988, 1993).
Consider Example 1A. It is the screen that voters on one DRE
see when they select VOTE but have undervoted in at least one contest.
The research on how people act
predicts that many voters will press Confirm when they get to the second
message without reading either the note or the instructions on how to return to
previous pages of the ballot.
Best Practice 1 could
easily be put into the voting guidelines the TGDC and NIST are developing. It would be simple to test and confirm that
a DRE meets this guideline.[3]
X Example 1A (poor practice)
Example 1B shows what the
same screen would look like if it followed best practices on how people act.
Ö Example 1B (good
practice)
Best Practice
2.
Put the context before the action in each instruction
Other research shows how
important a similar principle is even on the sentence level. Dixon (1987) investigated what people
did with instructions that had two parts: contextual information and an
instruction to act.
Here is one of many examples
from Dixon's study:
Order: context first, then action:
This will be a picture of a wine glass. Draw a triangle on top of an upside-down T.
Order: action first, then context:
Draw a triangle on top of an upside-down T. This will be a picture of a wine glass.
Dixon was most interested
in what people did with the action first order. Two hypotheses are possible:
· Readers put the action into a mental buffer and wait
for the context before acting.
· Readers guess at the context and act without waiting
for the context.
Results showed that people
guessed and did not wait. Note how similar this result is to the work of
Carroll on how users jump to act.
From the order context
first, then action, most readers drew a wine glass as in Figure 1a. From the order action first, then
context, many readers drew a Christmas tree as in Figure 1b.
Figure 1. Figure 1a is what most people draw when the information is given with the context first, "This will be a wine glass." Figure 1b is what many people draw when the information is given with the action first, "Draw a triangle on top of an upside down T."
Studies of how people
behave in conversations (Clark and Haviland, 1975) and of problems readers have
in understanding technical, scientific, and academic writing (Gopen and Swan,
1990) similarly stress the importance of putting context first.
Many ballot instructions,
however, put the action before the context, in violation of the best practice
we draw from this research.
Compare Example 2A with
Example 2B. Compare Example 2C
with Example 2D.
In each example, we have indicated which part is the action
and which part is the context by putting boxes saying "Action" or
"Context" over the first word of the relevant part of the sentence.
X Example 2A (poor practice)
Ö Example 2B (good practice)
X Example 2C (poor practice)
Ö Example 2D (good practice)
Best Practice
3.
Use familiar, common words. Avoid
technical or specialized words that users are not likely to understand.
This is a basic "plain
language" guideline. Many
research studies have found that words that have high frequency of use in
everyday language are understood better than low-frequency words. Short, familiar words are recognized
faster and read faster.
Technical words that are
not necessary
Many voters are likely to
be confused by the technical terminology that some DREs use.
Compare Example 3A with
Example 3B.
X Example 3A (poor practice)
Your electronic ballot is activated.
What does this really mean
for the voter? Example 3B makes the same point from the voter's point of view
in language that is more meaningful to voters.
Ö Example 3B (good practice)
You may now start to vote.
Metaphors that may not
work well
"Navigate" in the
meaning of move around in a web site or computer program is a metaphor based on
boating. Many voters may not
understand that metaphorical use.
Compare Example 3C with
Example 3D.
X Example 3C (poor practice)
Navigate forward through the ballot.
Theofanos and Redish (2003)
found that half of the people in their study of web users did not click on the link
"skip navigation," even though that is what they wanted to do. Participants in this study were
experienced web users who are blind and who regularly use a screen reader to
listen to web sites. Clicking on
"skip navigation" would allow them to get directly to the content and
not listen to the same information at the top of every web page. Many sites include a link with these
words, and these users must have heard the phrase many times. However, they did not understand the
word "navigation" in the meaning of getting somewhere on a web site. Their words for what they wanted to do
were "get directly to the content" or "skip to the
content."
Will all voters, some of
whom are computer novices, understand "navigate" in this context? Example
3D is a clearer way to convey the same message.
Ö Example 3D (good practice)
Move to the next ballot page.
Legal language that is
outdated
Legal documents in the
United States tend to be very conservative in their language. Writers use old documents as models,
without updating the language in them.
But, one of the most basic
findings of linguistic research is that language changes over time. As Redish explained with examples in a
1985 paper, what was clear, plain language hundreds of years ago is not clear,
plain language today (Redish, 1985).
Compare Example 3E with
Example 3F.
Example 3E is a case in
point. We no longer use "such" in the way it is used in this ballot.
such candidate as you desire
Example 3F is contemporary
English for the same message.
Ö Example 3F (good practice)
the candidate you want
Best Practice 4.
Put instructions in logical order: First task, first; last task, last.
This practice is relevant
to both paper and electronic ballots, but the issue comes up most seriously on
typical paper ballots. On most
paper ballots, the voter gets several instructions at the top of the page. Most of these top-of-the-page
instruction blocks violate many best practices.
Example 4A actually has
several features that make it better than many other paper ballots: Each
instruction is in its own paragraph with a little space between
paragraphs. The instructions are
numbered. The order in each
instruction is context first, action second; so it does well on Best Practice
2.
However, the two
instructions on how to actually vote are separated by the basic instruction on
what to use. And the instruction on what to do if you make a mistake is also in
the middle. If erasures on ballots
are a serious problem, the instruction should stand out more. When people come back to look for what
to do if they make a mistake, having the instruction at the end of the list
rather than in the middle will make it easier to find.
X
Example 4A (poor practice)
In Example 4B, we have only
reordered the instructions so that the general instruction about what to use
comes first, the two instructions about how to vote come one after the other,
and the instruction about what to do if there is a problem comes last.
Ö Example 4B (good practice)
Notice, however, that two
of the instructions here might raise other issues:
· Must voters use a "lead" pencil? Do voters
today know what that means? What type of pencil might they have that is not a
"lead" pencil?
· The instructions about writing in a name, say to write
the "complete name." But what does "complete" mean here?
Does the voter have to know a middle name or middle initial?
In voting, issues like
these are not trivial. In some
jurisdictions, a name that does not meet the local legal definition of
"complete" is not counted as a legitimate write-in vote. Yet, the instruction itself is not
complete because it does not define "complete" for the voter.
Further
research is needed
The examples in this paper illustrate
just four of many best practices for instructions that our review found are
violated in most ballots today. Even
our preliminary study shows that language in voting instructions could be greatly
improved to help voters by following research-based best practices. However, those best practices are
derived from research with other types of documents in other contexts. We still need to study how they apply to
ballots. We need further research
on two sets of questions:
Question
1: How important are these best
practices to a successful experience for voters?
By suggesting that each of
the best practices that we derived from research is relevant to voting
instructions, we are, in effect, predicting that violating the best practice
hinders voters and applying the best practice helps voters. The next logical research step is to
find out if these predictions are valid.
The best method for that
research is usability testing – having representative voters work in
realistic voting settings with realistic ballots – where we vary the
wording and presentation of language in controlled ways.
Question
2: Do voters have difficulty
understanding words that are special to voting?
A specific area for further
research is the set of words that are special to voting. Ballots include many words that are not
part of common, everyday English or that are used on ballots in special
meanings. These special words
include
|
·
candidate ·
cast a ballot ·
contest / race ·
early cast ·
mixed ticket |
·
non-partisan ·
partisan ·
party ·
split ticket ·
straight ticket |
Research with actual voters
is needed to know what voters think these words mean and to explore ways of
explaining them.
References
Alvarez,
R. M., 2002, Ballot Design Options, paper prepared for Human Factors
Research
on Voting Machines and Ballot Designs: an Exploratory Study. http://www.capc.umd.edu/rpts/MD_EVote_
Alvarez.pdf
Carroll,
J. M., 1990, The Nurnberg funnel: designing minimalist instruction for
practical computer skill. Cambridge,
Mass.: MIT Press. [This is a review of the many studies that Carroll and his
colleagues did at IBM's T. J. Watson Research Lab through the 1980s.]
Clark,
H. and Haviland, S., 1975, Comprehension and the given-new contract. In R.
Freedle (Ed.) Discourse production and comprehension, Hillsdale, NJ:
Lawrence Erlbaum, 1-40.
Dixon,
P., 1987, The processing of organizational and component step information in
written directions. Journal of Memory and Language, 26, 24-35.
Felker, D. B., et al, 1981, Guidelines for Document Designers, Washington, DC: American Institutes for Research.
Felker, D. B., et al, 1980, Document Design:
A Review of the Relevant Research (with chapters on psycholinguistics,
cognitive psychology, instructional research, readability, human factors, and
typography), Washington, DC:
American Institutes for Research.
National Center for the Dissemination of Disability Research
(NCDDR), 2003, Making Materials
Useful for People with Cognitive Disabilities, Research Exchange, 8 (3). http://www.ncddr.org/du/researchexchange/v08n03/2_materials.html
Gopen,
G. D. and Swan, J. A., 1990, The science of scientific writing. American
Scientist, 78, 550-558.
Redish,
J. C., 2006, Guidelines for Writing Clear Instructions and Messages for
Voters and Poll Workers, report to
NIST, 2/21/06. http://vote.nist.gov/032906PlainLanguageRpt.pdf
Redish,
J. C., 2005, Review of the Gap Between Instructions for Voting and Best
Practice in Providing Instructions,
report to NIST, 12/9/05. http://vote.nist.gov/instructiongap.pdf
Redish,
J. C., 1993, Understanding readers. In C. Barnum and S. Carliner (Eds.), Techniques
for Technical Communicators, NY:
Prentice Hall, 14-41. [This is a review of research from the 1980s.]
Redish,
J. C., 1988, Reading to learn to do. Technical Writing Teacher, 15 (3), 223-233. (Reprinted in IEEE Transactions on
Professional Communication, 1989,
December, 32 (4), 289-293.)
Redish,
J. C., 1985, The Plain Language Movement. In S. Greenbaum (Ed.), The English
Language Today, Oxford: Pergamon
Press, 125-138.
Schriver,
K., 1997, Dynamics in Document Design,
NY: John Wiley & Sons.
Theofanos,
M. F. and Redish, J. C., 2003, Guidelines for accessible and usable web sites:
Observing users who work with screen readers, Interactions, X (6), November-December, 38-51.