INFORMATION COLLECTION TOOLS – ADVANTAGES AND DISADVANTAGES

Information Advantages Disadvantages

Collection Tools
Observation  Collect data where and when  Susceptible to observer bias
an event or activity is  Hawthorne effect – people
occurring usually perform better when
 Does not rely on people’s they know they are being
willingness to provide observed
information  Does not increase
 Directly see what people do understanding of why people
rather than relying on what behave the way they do
they say they do
Document Review  Relatively inexpensive  Information may be
 Good source of background inapplicable, disorganized,
information unavailable or out of date
 Unobtrusive  Could be biased because of
 Provides a “behind the selective survival of information
scenes” look at a program  Information may be incomplete
that may not be directly or inaccurate
observable  Can be time consuming to
 May bring up issues not collect, review, and analyze
noted by other means many documents
Interviews  Useful for gaining insight  Susceptible to interview bias
and context into a topic  Time consuming and expensive
 Allows respondents to compared to other data
describe what is important to collection methods
them  May seem intrusive to the
 Useful for gathering quotes respondent
and stories
Focus Groups  Quick and relatively easy to  Susceptible to facilitator bias
set up  Discussion can be dominated or
 Group dynamics can provide sidetracked by a few individuals
useful information that  Data analysis is time consuming
individual data collection and needs to be well planned in
does not provide advance
 Is useful in gaining insight  Does not provide valid
into a topic that may be information at the individual
more difficult to gather level
information through other  The information is not
data collection methods representative of other groups

1
Surveys and  Administration is  Survey respondents may not
Questionnaires comparatively inexpensive complete the survey resulting in
and easy even when low response rates
gathering data from large  Items may not have the same
numbers of people spread meaning to all respondents
over wide geographic area  Size and diversity of sample will
 Reduces chance of evaluator be limited by people’s ability to
bias because the same read
questions are asked of all  Given lack of contact with
respondents respondent, never know who
 Many people are familiar really completed the survey
with surveys  Unable to probe for additional
 Some people feel more details
comfortable responding to a  Good survey questions are hard
survey than participating in to write and they take
an interview considerable time to develop
 Tabulation of closed-ended and hone
responses is an easy and
straightforward process
Town Hall Meetings  Can gather large amount of  Organizing the event takes time
and Other Large data at one time and resources
Group Events  Allows respondents to  Definitely need to have a draw
describe the issues that are to get people to attend in the
important to them form of incentives
 Provides a venue where  Need to have access to people
people can build on each with good facilitation skills
others’ knowledge  Need to have “ducks in a row”
to ensure attendance at event
Case Studies  Fully depicts people’s  Usually quite time consuming
experience in program input, to collect information, organize
process, and results and analyze it
 Powerful way of portraying  Represents depth of
program to outsiders information rather than
breadth
Illustrated  Fun to do and easier to get  Takes some technological
Presentations – people involved because it skill/expertise to “produce” the
Photo Voice, Power doesn’t seem “researchy” presentation
Voice  Especially useful as a way to  Need to have good facilitation
get people of different skills given that these methods
cultures involved or people are group work processes
who are more visual than  May not speak to stakeholders
verbal who prefer more quantitative
 Powerful way to represent approach to data collection and
data – “A picture is worth a may not be appropriate in

2
 thousand words.” situations where numbers are
needed
Skits,  An alternative approach to  Takes some courage for people
Dramatizations, linear models of evaluation who aren’t accustomed to
Other Visual that may not fit well cross- public display
Representations culturally  May not speak to stakeholders
 Useful methods when part of who prefer more quantitative
the evaluation purpose is to approach to data collection and
move toward some form of may not be appropriate in
action situations where numbers are
 Fun, innovative approach to needed
evaluation that causes
people to view program
through another perspective

SOURCES

Finn, J., & Jacobson, M. (2008). Just Practice: A Social Justice Approach to Social Work.
Peosta, IL: eddie bowers publishing.

Jacobson, M., Pruitt Chapin, K., & Rugeley, C. (2009). Toward Reconstructing Poverty
Knowledge: Addressing Food Insecurity through Grassroots Research Design and
Implementation. Journal of Poverty, 13(1), 1-19.

Russ-Eft, D., & Preskill, H. (2001). Evaluation in Organizations. New York: Basic Books.