Guide To Data Collection - CommCare
Guide To Data Collection - CommCare
Guide to Data
Collection
WWW.DIMAGI.COM
Contents
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
How to Define Your Project Objectives
Identify Your Project Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Review & Confirm Your Project Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Organize Your Project Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Focus on Your Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
How to Identify Your Data Requirements
Categorize Your Data Needs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Describe Your Data Needs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Summarize Your Data Needs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
How to Determine Your Method of Data Collection
How Data Requirements Inform Method Selection . . . . . . . . . . . . . . . . . . . . . . 12
Account for Environmental Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Storage & Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Summary of Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
How to Organize Your Data Collection Plan
Two Approaches to a Data Collection Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Additional Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Why Use a Data Collection Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Ready to Go With Data Collection? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Thank you!
This guide would not have been possible without help from our partners at Oikoi (formerly AgImpact),
ProMujer, Terres des hommes, Save the Children, and Dimagi employees and alumni.
Introduction
G E T T I N G S TA R T E D W I T H D ATA C O L L E C T I O N
3
1
How to Define
Your Project Objectives
4
referrals per day for patients in critical condition problem by including stakeholders from all levels,
without any documented medical history. For the and you will experience higher participation
doctors, and thus for Luk, the real priority was in the project over time. This is something Luk
urgent and clear. realized on that same project in Ghana:
5
des hommes’ Integrated eDiagnostic Approach compliance purposes. Include these, as well. You
(IeDA) project in Burkina Faso, recommends might even know of other items your program
understanding where your project fits within the supervisors or partner organizations want to
priorities of the organizations you are working keep track of. If you incorporate the needs of all
with: your partners into the foundation of your project,
you are more likely to avoid conflicting objectives
“Ministries of Health have many competing
later on.
priorities,” said Mr. Foutry. “We found it to be
extremely important to take this into account Organizing your objectives around your partners’
and understand that IeDA was one project needs and accounting for their other priorities
among many for them. Try to understand the will provide you with reliable guardrails (and
players, the dynamics within the MoH (Ministry investment ) as you build out your data collection
of Health), and who could be champions for the plan.
project within the MoH. If you understand their
needs and goals, you can work to make your
project fit with them. Connect your project to Organize Your Project
their other initiatives.” Objectives
Once you establish your key project objectives,
the next step is to outline how you will achieve
What else might my stakeholders
them. What results do you need to prove in
require?
order to call your project a success? When
You often have performance benchmarks to reach working with governmental organizations, such
in order to receive the next round of funding. Build as USAID, the map of this journey is called a
these into your objective. Government agencies “logical framework” or “results framework.” In
might require certain data be measured for essence, they are project to-do lists that make
6
you ask the question of each aspect of your and sub-results. A few example intermediate
solution: Does this get me closer to achieving my results could be:
project objective?
• Intermediate Result #1 (I.R. 1): “Percentage
A results framework places your project of households sprayed with insecticide.”
objectives at the top of the diagram and maps
• Intermediate Result #2 (I.R. 2): “Percentage
out each of the intermediate results that will add
of households using insecticidal bed nets.”
up to their success. In the example below, we
Additional sub-results would follow that can
will use a malaria campaign to illustrate how
break down each intermediate result into further
this works. Our strategic objective (SO) is to
detail, including additional objectives to achieve
“Decrease malaria-related child mortality rates.”
and the tools we would use to achieve them.
From there, we break down the components of
the objective into all its intermediate results (I.R.)
Your organization might not need a results The method of data collection you choose will
framework to justify your program for the be the way you measure the success of many
purposes of funding or additional support. of these intermediate and sub-results. There will
However, a results framework still establishes even be cases where the approach you take will
your project’s objectives as the core focus of aid in improving program outcomes themselves.
your program and helps you think through all Mapping out your full set of objectives will help
the ways you can review their progress and you to align your approach to data collection
ultimately achieve success. and service delivery with the core needs of the
project.
7
Focus on Your Objectives
The process of turning clear and urgent problems project’s lifespan to evaluate their decisions.
into precise project objectives is a crucial step Including considerations from partners will also
before developing your data collection program. help to ensure your project is given the proper
Your project objectives should be at the core of attention and resources required for success.
your entire program, and they should be clearly
It can be hard to get a project moving in the right
defined and written down in a document such
direction. But once you have a clear destination,
as a results framework. Anyone on your team
it is easier to determine whether the next tool or
should be able to understand your project’s
initiative will take you closer to or further away
approach from reading your results framework,
from where you want to be.
which they should reference throughout the
8
2
How to Identify Your
Data Requirements
9
times will help you optimize individual and team patient has actually received treatment.
performance. When you ask a patient if she has ever
received medical treatment, and she replies,
“no,” you don’t need to ask about vaccinations,
Describe Your Data Needs medication, or any other medical treatment.
This will require your workers to know all the
Once you have organized a list of your data
combinations of questions they might need
requirements by category, flesh out their
to ask (or a tool that can be programmed to
attributes and characteristics. There are
filter those questions for them).
numerous questions you can ask to help with
this: There are many more questions you can ask
to help describe the characteristics of your
• Are you searching for quantitative data
you can record or qualitative insights from
sources close to the subject of your analysis?
You don’t have to collect just one type, but the
characteristics of the data you are collecting
will be integral to selecting the right method
of data collection and the questions you ask
later on.
10
variables, but as with everything else, they will
Summarize Your Data Needs
depend on your project’s objectives. Marcos
Lavandera, health analyst at Pro Mujer, a With the myriad methods of data collection,
woman’s development organization in Latin developing a clear, written summary of all the
America, explained that for his project that variables you need will help keep you focused.
focused on women’s healthcare in Mexico, his Differentiating between program performance
entire team took part in the process of defining metrics and worker performance metrics will help
their data’s characteristics. keep your data organized. The characteristics of
your variables will help determine the method
“We had our program director, health analyst,
and features of data collection that will work
and medical director all working together to
best for your program. It won’t surprise you to
make sure we looked at the data from all the
know that data is the most important piece of
possible perspectives,” Lavandera said.
any data collection program, so a comprehensive
All of the characteristics you define will help you
understanding of the data you need to collect is
later, as you determine the right data collection
vital.
method for your program.
11
3
How to Determine Your
Method of Data Collection
How Data Requirements know you only have a one-off survey, depending
on the scale, the cost of setting up a mobile
Inform Method Selection
data collection program (including the platform,
The data you want to collect should inform devices, and data plans) probably doesn’t make
everything about your program – especially the as much sense as paper.
method of data collection you intend to use. The
characteristics of those data will mean different
things for that decision, and each method has What Is the Scale of Your Data
different strengths and weaknesses. This could Collection Program?
be simple, like quantitative inputs might be tough While much information (both qualitative and
to collect from a focus group, but qualitative quantitative) has been collected on paper or
information (like quotes) could be much easier through interviews for a long time, it can be
that way. hard to scale these programs. Many national
health organizations have struggled with
Here are a few questions worth asking to
growing successful pilots because they didn’t
determine what your data requirements mean
have a structure or tool that could handle such
for the method of data collection you select:
large demand. In this case, tools like mobile
data collection and Interactive Voice Response
(IVR) could help you reach significantly larger
populations.
12
Do Your Data Require Technical need to account for where you are collecting
Inputs? data and whom you are collecting them from.
Pro Mujer recommends beginning this effort
Sometimes, organizations need to collect
with the beneficiaries. By putting them first, you
information like GPS coordinates, video, or even
make sure they are the ones experiencing the
fingerprints. In these instances, you require an
greatest impact. Understand the data they have
additional tool to capture the data. If you’re
and the environment they live in to best provide
working with paper, that could mean copying
the services you hope to offer.
down information from a GPS-enabled device,
tracking timestamps from a video camera, or Here are a few questions to consider when
keeping track of photos taken of fingerprints. examining your project’s environmental factors:
With a mobile data collection tool, some of these
• What are the languages spoken by the
features might be included.
people involved (both data collectors and
beneficiaries)?
Account for Environmental • What is the reading level of your typical field
Factors worker or beneficiary?
Once you have listed, organized, and described • What is the level of mobile connectivity in the
all the variables you need to collect, you still region? Is there WiFi available to workers?
13
• How familiar are your data collectors
with mobile devices? What is their level
of digital literacy?
‘No, no, no,’ she said. ‘Do you see that ant hill
in the distance? That is where we get our
reception. Every day, at five o’clock, I will go
stand on that ant hill and hold my phone up in
14
Storage & Security As you consider the storage and privacy of your
data, ask yourself the following:
Certain sectors lean more heavily on this
• Does the dataset need to be de-identified
consideration than others. For some beneficiary
before sharing?
populations and projects such as those working
• Do I need to protect certain data after they
with HIV patient data, privacy concerns may
are collected?
be much more important than others. Thus,
• Who can have access to the data?
understanding where the data you collect go is
• How long can the data be stored?
vitally important.
All of these considerations will be specific to
The actual considerations will depend on the
the industry or sector that you work in, while
sector you are working in. For instance, projects
others will depend on local laws or even partner
in the public health domain might require you
organizations’ codes of conduct. Make sure you
to consider patient confidentiality and HIPAA
are familiar with the requirements of all parties
compliance. The FDA has shared guidelines for
involved before you decide how to collect your
the use of electronic health record data that may
data.
be helpful for your project.
15
Summary of Techniques
There are so many different methods of
collecting data available today. The following
are a selection of some of the more popular
approaches we have seen:
Surveys
Interviews can be based on a common set of Observational data collection is much more
questions, like a survey, but they allow for more low-touch. The idea is that in the absence of an
flexibility in the responses. The organic insights opportunity to interact with your subject (either
gleaned from interviews can give you answers due to distance, scale, time, or other reasons for
to questions you didn’t even think to ask. their inaccessibility), you can observe them to
collect certain types of information.
• Individual: Individual interviews are
relatively self-explanatory. An individual • Firsthand: Firsthand observation allows
with experience in the topic you are curious for the data collector to directly observe
about answers questions from the data and gather notes on an individual or group
collection team. Individual interviews can without interacting with the subject(s).
be a good way to start other types of data This approach is often used when direct
collection programs, especially when you are observation is available but not direct
trying to figure out the right question to ask contact either for reasons of inaccessibility
on something like a survey. or fears of bias.
• Focus groups: Focus groups are like group • Documents/records: Often used when
interviews, often including a stimulus information is needed from the past,
for discussion. Focus group leaders will document review allows for secondhand
prompt the group with a set of questions observation of sources when firsthand
or statements and gather reflections and observation is unavailable.
feedback from the group. It’s a good way
to quickly compare reactions or information
from representative sources and an
opportunity to see how group dynamics
might affect an individual’s reaction.
How to Choose
As you can see, selecting the method of data collection that is right for your program is entirely
dependent upon your program objectives and data requirements. For qualitative data, you might lean
more heavily on individual interviews and focus groups. For hard quantitative insights, look in the
direction of surveys (mobile, paper-based, or otherwise). Some programs might call for a hybrid or
combination of approaches to collect all the information you need.
4
How to Organize Your
Data Collection Plan
18
of different types of flow diagrams–from (e.g. quantitative data vs qualitative data) and
organizational hierarchies to application follows through from how they are collected (e.g.
workflows and information flow diagrams. An paper forms vs mobile device) to where they are
information flow diagram is most often what stored and how they are shared from the bottom
we use when mapping how information flows to the top of your organization (e.g. reporting
through an existing data collection process. It presentation vs online dashboard).
typically starts with what data are being collected
This is a very basic version of an information flow diagram based on a typical CommCare project.
This is the most basic version of an information information flow diagram are (1) What are the
flow diagram. Data from beneficiaries is major milestones that occur in this process? And
collected by community health workers (or other (2) What are the major component types (e.g.
data collectors) using a mobile data collection actions/activities, documents, decisions, etc.)?
app that wirelessly sends data to the cloud. The answers to these questions are like the
There, they are accessed by a program manager pieces to your puzzle. Once you collect them all,
or analyst on a desktop platform. Of course, this start with the outside and work your way in. In
version doesn’t include what type of data they other words, begin with your data source and
are or how that program analyst shares reports your final output and then fill out the pieces in-
with their superiors, funders, or the government. between.
However, that is exactly the type of information
The beauty of an information flow diagram
that is covered in more complex information flow
is that you can read the same diagram from
diagrams.
bottom-to-top and notice different things about
Two key questions to ask when designing your your data collection process than if you review
19
top-to-bottom. The change in perspective will does not quite measure up to an information
help reveal things about the way your data are flow diagram in terms of viewing your program’s
used and potential means for improving them. strengths and weaknesses from a high level,
but it’s great for organizing detailed notes on
Bonus tip: We like to use a platform called Draw.
how each variable is collected, who has access
io to create our workflow maps. Check it out
to what, and even how it might be analyzed. In
here!
fact, you can often find some insightful trends
by reviewing each row in the chart together. For
Data Collection Plan Outline instance, in the chart above, you can see how
the source of your data might differ between
The other way we often analyze the data
data points #1, #2, and #3, by reading across
collection process is by filling out a data
row #2 (“source of data”).
collection plan outline. It helps organize each
variable you are collecting by source, method of One reason we like the Americorps version of
collection, timeline, where it is stored, and how it this outline is that it ends with “How will the
is analyzed and shared. data be used for program improvement?” It is
a good reminder that, regardless of your data
Compared to the information flow diagram, which
collection program objectives, you can always
looks to map data through your entire program
examine your results in a way to improve your
in a visual way, a data collection plan outline
final output. Addressing that question early and
typically summarizes relevant characteristics of
deliberately is a good way to make it a habit and
each variable in a table or chart. This approach
improve the sustainability of your program.
The first few rows of a data collection plan outline from Americorps.
20
Additional Considerations Why Use a Data Collection
As you develop your plan, it’s not uncommon to
Plan?
begin to consider aspects of your program you The most important reason to use frameworks
hadn’t thought about before. This is an intentional like workflow maps and data collection plan
aspect of the process. It’s much better to head outlines is that they help you to understand
into the design and implementation phase of the stakeholders, data sources, and points of
your program aware of these facets, than it is to connection that will reveal areas for improvement.
retroactively build them into a program.
For example, in an analysis of the time between
the collection of data and the submission of
that data to the server, a careful observation of
Timelines
the clinical workflow helped the Dimagi Data
Often, these planning frameworks don’t include
Science team determine that 75% of CommCare
the dimension of time. Consider ways you
users were using their application as an offline
might incorporate it to account for how often
data collection tool. By understanding how the
your frontline workers will head into the field to
data flowed in these low- to no-connectivity
collect data or how often you will lead them in
environments, it was then possible to optimize
refresher training sessions.
surveys, flows, and general user experience.
21
Ready to Go With on correctly, should improve the sustainability of
your program as a whole.
Data Collection? If, after following this process, you find that
A well-organized data collection plan is the mobile data collection is the right avenue for
culmination of all your other work in clarifying your program, we have some good news for you.
your project objectives, defining your data We have developed a comprehensive guide to
requirements, and selecting a method of data take you from the process of selecting the right
collection. This plan will serve as the basis of your mobile data collection tool through designing
efforts to actually design and build a program your surveys, building your tool, and ultimately
that precisely serves the needs of your project maintaining its sustainability. This guide is a
and its beneficiaries. It will help when you bring collection of advice from our partners and Global
on new team members and when you need to Services team, aimed at increasing the use and
apply for new funding. The structure that your impact of mobile data collection tools worldwide.
data collection plan provides will help at every To learn more, check out our Guide to Mobile
step of the way from here on out and, if acted Data Collection.
What’s next?
Visit www.dimagi.com/commcare for more personalized
advice from the team who built the world’s most powerful
mobile data collection platform:
Track data
Build smarter data collection apps that allow
you to collect and track data over time.
Work offline
No signal? No problem. Build a data
collection app that works offline.
Empower end-users
Collect quality data by building an app that
guides your workforce to do their best work.
22