The Complete Guide to

Mobile Data Collection

Table of Contents
Introduction 03

1. How to Choose the Right Mobile Data Collection Tool 04

Mobile vs Paper 04

Understanding Data Requirements 05

Implications of a Project’s Environment 07

How to Design Your Mobile Surveys 09

Identify Your User Stories 09
Design the Structure of Your System 10
Develop Your Content Design 10
Clarify Your Delivery Design 12
Mode of Communication 13

How to Build Your Mobile Data Collection App 14

Plan for Development 14
Draft a Versioning Plan 14
Build Your Outline 15
Additional Features 15
Working with Multiple App Builders 16

How to Test Your Mobile Data Collection App 17

QA Process 17
Usability Testing 20
Pilots 21

How to Deploy Your Mobile Data Collection Software 22

Choose the Right Device 22
Prepare Your Devices 23
Develop Phone Usage Policies 24

6. How to Train Your Team in Mobile Data Collection 25

Choose the Right Method of Training 25
Deliver the Training 26
Reinforce Your Training 26

How to Sustain Your Mobile Data Collection Program 27

Support System 27

Supervisors 28
Incentives for App Usage 28
User Feedback Mechanisms 29

Thank you!

his guide would not have been possible without help from our partners at Oikoi (formerly AgImpact), ProMujer, Terres des hommes, Save the Children, and
T

Dimagi employees and alumni.

Introduction

Getting started with Mobile Data Collection

Mobile data collection is a method of compiling qualitative and quantitative information with the help of a mobile

device (e.g. mobile phone, tablet, etc.). This approach has been proven to increase the speed and accuracy of data

collection, service delivery effectiveness, and program staff performance. Features of mobile data collection, such

as decision support, form logic, and checklists, improve the quality of the data collected, while also ensuring

adherence to data collection and care protocols.

While it is an effective medium, it is also important to treat mobile data collection the same as any other data

collection program. To make sure you’re ready, check out our guide to data collection.

3
 How to Choose the Right Mobile Data
Collection Tool

When it comes to implementing mobile data collection, there is no one-size-fits-all solution.

Depending on the circumstances, what works for one program may not work for yours, so understanding the core
benefits of these systems can help you determine which features might help your program and how.

“We used to run paper-based monitoring and evaluating

systems, which were inefficient and painful to manage.
Mobile data collection made our lives easier”
Prachi Patel, Manager of Technology for Development, CMS India

Compare Mobile Apps with Paper-Based Solutions

While it may be the standard method of data collection at many organizations, there are significant issues
associated with a paper-based data collection approach. The most prominent challenges include:

High error rates

Human error, such as poor handwriting or typos, mean the data collected is incorrect or illegible

Slow reporting or data entry delays

Users need additional time to return to a computer and manually enter data

A lack of flexibility in deploying programmatic changes

Each update to a paper form requires reprinting and redistribution

Disruptions to beneficiary interactions

Frontline workers often spend as much time navigating their paper forms and guides as they do actually
interacting with beneficiaries.

4
 Difficulties in supervision

Supervisors of paper-based programs spend significant time contacting and following up with frontline

workers to check their status rather than analyzing their performance and offering advice.

With the proper implementation, each of these issues can be minimized with the use of a mobile data collection
tool. Such mobile apps can validate data upon entry to ensure higher accuracy. They can incorporate complex
mobile form features – such as skip logic – to help collect data more efficiently. Updates and programmatic
changes can be deployed to the field without the need for printing and distributing new paper surveys. Frontline
workers can reference counsel built into the mobile app to advise them through particularly challenging
assessments as they speak with beneficiaries. Supervisors can securely access relevant data, even doing so
instantly when the worker has a mobile connection.

Depending on the issues your program faces, the switch to mobile data collection can both improve your program
and lead to significant cost savings. One World Bank study found that the average survey cost was reduced by 71.3%
when adopting a mobile data collection tool over paper. Frontline workers have also indicated that as a result of
their use of a mobile data collection app, their credibility in the community increased. Given these benefits, mobile
data collection could be an answer for your program, and outlining your data requirements should be your first step
to selecting the right tool.

“Mobile Data Collection gave us the ability to control the

experience of our field based data collectors, which
drastically reduced human error. With paper-based forms,
you can’t really make your field staff follow a strict protocol”
Bhawna Mangla, Senior Research Associate, Monitoring & Evaluation, Seghal Foundation

Understand Your Data Requirements Before Selecting a Mobile Data

Collection Tool
Mobile data collection tools have immense potential but there isn’t a single solution that will work for everyone. As
you might expect, different tools offer different features.

Dimagi’s User Engagement Manager Marshall Daly recommends evaluating your data needs before you start looking
at different market offerings.

“Like any other data collection program, start with the data
you want, and then find a tool that allows you to collect that
data,”
Joe Daly

5
Even after identifying your data requirements, selecting the right platform is often a challenge, and there are several
points you should consider throughout the selection process:

What format do you need to present your data in?

Certain data formats require certain features in order to capture them. Your data needs will determine
whether you are collecting quantitative or qualitative data, but also whether you will be recording GPS
coordinates, audio, video, or other formats that will require the appropriate functionality from your mobile
data collection app.

Will you need to update your data?

Often, programs require that workers revisit data sources (beneficiaries, locations, etc.) to collect updated
information. If this is the case for your program, it is helpful to have a platform that allows you to link and
review multiple data entries with a feature such as case management. For example, a worker tending to a
tuberculosis patient using a paper-based data collection tool might have to carry around several forms for
the full context of that patient’s medical history. Instead, mobile case management provides that same
worker with easy access to the patient’s medical history and can even help pre-populate forms with
previously collected data, such as name, date of birth, and treatment history.

Who has access to your data?

Just as important as the way data is collected are the people collecting and managing it. Unlike paper-
based systems, which do not allow multiple actors to update or review data, some mobile solutions allow
for multiple relevant and predefined users to view and update data from a single source. Various parties may
need to act on or review the most up-to-date information to fulfill their responsibilities, so a feature like
case sharing, which allows for efficient and secure data sharing, can be exactly what your program needs.

How do you need to share your data?

One of the biggest questions around data collection, in general, is what you will do with all of the data you
collect. Are you going to export your data to a dedicated analytics platform? Or do you need an easily
accessible dashboard for multiple stakeholders to review your data in real time? These types of
considerations help determine whether you need a simple export to Excel, more complex data visualization
features, or even full platform integrations with specialized reporting software, such as Tableau or Spotfire.

As always with the sharing of data, consider any privacy and security needs associated with patient
information. Many tools offer various levels of security, so be sure to check whether your chosen platform
will be compliant with all relevant data privacy regulations.

Not only does the type of data you collect affect the tools and features you need to collect it, but its
characteristics and those of its environment will also be important considerations when deciding on the
right mobile data collection application for your team.

6
Consider the Implications of Your Program’s Environment

Mobile data collection offerings each have their own strengths and weaknesses in different aspects, such as price
and maintainability. Beyond the features that deal with what data you collect, look at how the structure of your
program or the characteristics of the region you work in might affect your platform needs.

How large is your data collection program?

The number of end users in your program is one of the most important considerations for deciding on a
mobile data collection tool, because not all platforms are built for scale. If you expect your program to grow,
ask your prospective technology vendor if they have any current projects at scale. This will give you a sense
of whether a nation-wide project, for example, is possible with their data collection tool. Our
recommendation is to invest in a tool that can grow with your program so you are prepared at any stage.

You should also be aware that certain tools limit the number of end users for lower-tier subscriptions, but
may offer strong supervisory functionality for larger deployments on higher- tier accounts.

7
 How experienced is your team with mobile data collection?

Some mobile data collection platforms offer simple premade surveys. Others are built to support extremely
complex workflows but require a high level of technical competence. There are so many questions to ask to
determine the level of complexity you will need to account for:

Does your platform require custom development by an engineer

Are all required features available in a self- service mode?

Does it require any coding experience?

Do you have an individual on staff who knows how to build and maintain an application on the platform you
have selected? If not, does the platform offer training opportunities or have clear and easy-to-understand
documentation on how it can be used?

Be aware that even the best platforms require extra effort and experience in order to extract their full value.

Indeed, in some cases, organizations assemble a team to build a custom solution, often built upon an
existing open source platform. This can mean that the platform is built precisely to the needs of the
program. However, as the complexity of the platform grows, so too does the complexity of setup and
maintenance. At Dimagi, we have a team of about 15 developers working full time on the platform, including
2-3 engineers fully dedicated to keep our production environments online, up-to-date, and bug-free for our
users.

Will you have an internet connection while in the field?

Once you understand the factors in your control, it is time to consider the context of where you will be
deploying your program–including what is out of your control. If you are deploying in a low- or no-
connectivity environment, look for a system that includes offline functionality. Many platforms will allow you
to record data on the device and upload when connectivity returns. Some even allow for offline case
management, meaning you can store data from prior visits on the device to access and update even without
access to the internet.

What languages do your end users and beneficiaries speak?

Your team and beneficiaries might speak a range of languages. If this is the case for your program, you
should look for a tool that allows you to easily translate a given form. Some tools go even further, allowing a
user to select a language for their application depending on their audience. If the beneficiaries cannot read,
some tools also allow for audio playback.

Why does this matter?

Mobile data collection tools on their own are not silver bullets. They cannot fix a poorly- designed program, and a
solution designed for a basic survey will not aid your complex case management program. Indeed, you will not get
the full value of a complex mobile tool without a technically competent program manager, and in the wrong
environment, mobile tools can even be a detriment to frontline workers.

However, combining the considerations shared above around your data, workers’ needs, and project environment
with the right mobile data collection application will improve the accuracy and speed of your data collection, help
your team do its best work, and ultimately, amplify the impact of your program.

8
 How to Design Your Mobile Surveys

The process of designing your app involves both

(1) defining who will use your app

(2) determining how to best structure the app with the right modules and features.

Identify Your User Stories

Your first step is to identify the users and user stories to build your app around. These should be centered around
the biggest pain points and opportunities for impact you identified in scoping – not just the loudest voice in the
room.

For example, will your enumerators need to register new cases to a list? Will your supervisors need to provide a
specific set of feedback to field workers at regular intervals? Look at the specific interactions and flows of your
current data collection program to see what each of these use cases (or user stories) is for your program.

Below are a few key questions to consider while developing this list of user stories. These questions should help
open your eyes to the potential impact of your tool:

How will my app change existing processes and roles?

How will those changes affect each player in the system
Are those changes addressing pain points and improving workflows, or just complicating things?

Validate your user stories and the answers to these questions with your team before you start designing the
technical structure of your solution to avoid building an app that is misaligned with your project and end users’
needs. Are these experiences they would like to be improved? Can your tool provide feedback to help? Is your
proposed solution going to conflict with any other responsibilities they hold? They should be able to answer these
questions in a way that indicates whether your new mobile data collection tool will be well received and regularly
used.

9
Design the Structure of Your System
Once you validate your user stories, it is time to translate them into your app’s module and form structure. This is
important to do before you start building to ensure your vision is possible with the set of features at your disposal.
Essentially, it is about confirming whether your chosen platform can do precisely what you need it to.

For example, let’s say your design starts with this user story:

The field worker registers all new program participants in the app.

A core requirement could be:

The field worker can search the list of existing participants to ensure they do not register anyone twice, avoiding
duplicates in the system.

While these user stories are an important start, you still need to map out how they actually look in your app: What
module and form structure and/or features will these experiences rely on?

Approach this task systematically by starting with a simple table like the one below. By translating the user stories
you have defined and validated into individual requirements, you answer the question of what you need your app to
do. Those requirements then become a summary of modules (groups of forms), individual forms, and features,
which define how your app performs tasks:

“Search Only”

properties in the

Case List


Module filtering

Minimize

Duplicates

(Registration

From Case List)*

*Usability feature highlight: Minimize Duplicates (or Registration From Case List) allows you to place your registration form at the bottom of your
participant case list, so the user can go from searching the exist- ing cases to registering a new one seamlessly.

Develop Your Content Design

By now, you know what data you need to collect, but the way you actually solicit it might still be in question.
Content design is about developing a survey or assessment that generates responses to best inform the goals of
your project. This involves setting up clear questions that avoid bias, maintain consistency in phrasing, and are
culturally appropriate.

Ask objective questions, like “what effect has this program had?” Not “how has this program helped you?”

Offer objective measures, such as “0-10 kg,” “10-30 kg,” and “30-50 kg.” Not, “light” and “heavy.”

Adapt your questions to the region, speaking in terms of “dollars” in the United States, but “francs” in Mali.

10
Spending a bit of time planning and strategically designing the content (e.g. questions, surveys, etc.) that will be
used in your project will help ensure that you follow a systematic process collecting data from your target
beneficiaries, from the beginning to the end of the data collection period. It is not just about putting all the
questions you can think of in the survey. With a consistent, well-thought-out design, results that emerge from data
collection will be cleaner and more easily compared with results collected at a different time point or even by other
programs and projects with similar objectives.

The primary components of content design are avoiding bias and cultural adaptation, and there are even some
widely-used, validated surveys available that could be exactly what you need.

Avoid bias

Avoiding bias is one of the hardest jobs of a survey developer, as the way you ask a question will impact the
answer you receive.

To capture the full range of possible answers, the phrasing of your questions should remain neutral, which
can be easier said than done. Depending on the topic, it can require extensive knowledge of the subject,
including public perception, power dynamics, and even controversies within the field.

A widely-used Catalog of Biases in Questionnaires highlights many of the ways you

may be unintentionally biasing your data. From problems with phrasing to leading questions, there are many
ways you could be designing the content of your surveys that will result in unreliable, messy data. For
instance, in a poorly designed question, “How often do you exercise?” could be answered by “regularly” or
“occasionally,” which are vague responses. A better option, providing precise and quantifiable answers, would
be:

“How often do you exercise?”

[ ] Twice a week or more often

[ ] Once a week

[ ] Less than once a week

Another area to pay close attention to is the phrasing of questions that are intended to elicit more
subjective responses (i.e. opinions, feelings, 14 or beliefs). The way you pose the question can influence
beneficiaries’ responses in ways that may be unintended.

Cultural differences

Context matters. Cultural references and word selection in survey questions may lead to variability in
interpretations of the questions when applied to different populations.

A common example that could increase the accuracy of your data collection would be if your project
targeted beneficiaries in rural Tanzania, you may consider including a multiple choice question inquiring
about the beneficiaries’ languages spoken at home. The choices could include Kiswahili, English, and
potentially other dialects based on prior knowledge of the population you are working with. And of course, it
is helpful to have the survey itself available in the predominant language spoken by the respondents you are
conducting the survey with.

An example that might have an effect on project outcomes comes from Ethiopia, where the Red Cross logo
was used as a hospital icon, but locals interpreted it as the symbol for a butcher.

Considerations like these should be balanced with the overall goal of the program, as a culturally-specific
reference may elicit a reliable answer from one beneficiary, but if the sample is ever expanded, then you
may run into issues.

The University of Michigan’s Institute for Social Research outlined a few ways to consider cultural references
in survey questions, explaining that you can

(1) Translate your questions

(2) Ask different questions

(3) Employ a mixed approach

11
 How to Approach Cultural Differences in Survey Design

Translating questions

Asking the same questions and translating can mean literally translating a question, but often involves
techniques such as decentering (i.e. removing all cultural references entirely) or including anchoring vignettes
(asking respondents to assess both for themselves and a hypothetical person). This is the best way to be able
to compare data later on.

Asking different questions

This can be a little tricky, because instead of standardizing the question, you must standardize the responses.
The opposite of decentering, it requires you to find appropriate examples for each population to ensure
relevance.

Mixed approach

Finally, a mixed approach could involve a standard question (e.g. “What language do you speak at home?”), but
varied responses depending on cultural context (in Madagascar, Malagasy and French, but in Afghanistan,
Pashto and Dari).

Validated surveys

Why recreate a survey instrument when there may be one that already exists that is well- aligned with your
project objectives? There are databases of existing, validated surveys that would allow you to reliably assess
and compare your results with those of other projects in the broader community. For example, RAND Health
provides free online access to their surveys covering a variety of health topics.

In sum, the content of your survey is the primary determinant of the data you will collect, so extra effort
should be placed on ensuring it is unbiased but relevant to your audience.

C larify Your Delivery Design

How you deliver a question is just as important as how you phrase your question. Determining the optimal delivery
method is all about how to best structure and disseminate your survey. The survey structure and mode of
communication are the primary considerations in this category. This means that once you have organized your user
stories and workflows, you need figure out the proper sequence to put them in.

Survey structure

The structure of a survey, including the sequencing of questions and their available responses, will affect the
data it provides you. Here are a few considerations for how you might

recognize and solve these situations:

Do certain questions depend on others?

If you have questions that will only make sense within the context of a previous answer, you can use skip
logic (also known as display conditions) to control when they appear. Deciding which questions should
appear (or disappear) depending on the answer to a prior question (or questions) is important for collecting
clear, consistent data. For example, if you ask the question, “Are you taking any medications?” and the
respondent says “Yes,” you can use skip logic to trigger the question “Which medications are you taking?”
Otherwise, you can skip that question if the respondent answers “No.”

W hich questions are required?

Often, with paper forms, you will find that important fields are left completely blank. This can render the
whole submission useless. The ability to deliver surveys electronically has made it possible to reduce the
potential of incomplete datasets.

12
 As a general rule, it is good to make every question on your survey required, especially if each one is
required in understanding the overall results.

However, there are some items that it may make sense to keep as optional. For instance, if you include an
open-text question to solicit any additional comments or feedback on the survey, the respondent may not
necessarily have any comments to provide. Therefore, it is reasonable to not require a response.
Alternatively, you might find that including “Don’t Know” or “Refused” for certain questions adds some
clarity to why some responses are left blank. Find the right balance of required questions to make sure you
are capturing all of the data that are essential to your project, while also ensuring that the survey is not too
long or cumbersome to complete.

What type of answers are you expecting?

When developing a structured-entry survey, you may want to consider whether a given question accepts
only one response or multiple responses. While paper forms cannot enforce the number of responses that a
respondent can provide, an electronic survey can. Furthermore, you may want to restrict the type of
responses that can be accepted. For instance, you may want to accept only integers for an item inquiring
about the respondent’s age or restrict phone numbers to ten digits. By using what are called validation
conditions, you can ensure your enumerators correct mistakes on site that they might not otherwise notice
until it is too late.

Mode of communication
Are enumerators reading questions off a paper form or are respondents answering directly on a mobile device? This
consideration is focused mostly on the user experience and how it can affect the reliability of their responses.

For instance, if you have a short survey, and your respondents have access to smartphones, you may want to
consider an SMS-based data collection program. However, if you are unsure of your audience’s access to mobile
phones, you may unknowingly restrict and bias your sample, based on the type of person who has access to a
mobile phone in that population (e.g. male heads of households).

On the other hand, if you have many open-ended questions that require longer text responses, you may want to
consider collecting responses verbally and/or entering responses via tablet or computer. Understanding how each
mode of communication is used in your target location (and the norms and power dynamics associated with those
types of interactions) will help you account for any bias as you design the delivery of your surveys and programs.

13
How to Build Your Mobile Data Collection App

Once you understand the structure of your app, it’s time to begin the building phase. This includes a clear
understanding of how you will approach development, an outline of what functionality each version of the app will
incorporate, and some key features of mobile data collection tools you’ll want to be sure to include.

Plan for Development

The next step is about determining how you will build your app. Keep your team organized and focused on their
assigned workstreams. One app development methodology that we found can help with this is Agile (also known as
Scrum). Agile’s principles of incremental development and constant iteration help ensure you are staying focused on
your project priorities, and validating along the way. JIRA is also a good option for project management software to
support agile teams and processes.

Whatever methodology you choose, make sure roles, responsibilities, and processes are very clear to the team, and
that everyone agrees. Then establish mechanisms to ensure your team sticks to them.

Draft a Versioning Plan

A tough realization for many organizations in the build phase is that not everything will always fit in the first version
of your app. Think carefully about what is time sensitive or a priority and the level of effort associated with including
certain criteria. You need to map out a versioning plan to identify what modules and features will be included in
each version of your app.

However, versioning plans should not just be focused on what is possible to include in any particular version.
Consider which features or flows make the best introduction to the tool for new users and which might be too
confusing for them to take on from the start. Sometimes it makes sense to introduce a certain feature set that you
know the user will be able to understand quickly and wait until they are comfortable before introducing new
functionality. For instance, if your users have never had a mobile device before, it might make sense to start with
text-only inputs and withhold functionality like image capture until they become more familiar with the basic
functionality of the device.

14
Build Your Outline and Fill It In
Now it’s time to build! With the overall structure and workflow that you designed and validated in the Design phase,
start integrating the more detailed content and logic into your forms. What are the unbiased questions you wrote
when you designed your surveys? Add them in! Are there any documents, videos, or messages your app will need to
share at certain times? Put those in, too!

Most versioning plans will start with a prototype for some initial testing. One option is to build your prototype in the
data collection platform you plan to use. This approach ensures your workflows are truly possible with the set of

features available to you, but it also requires that you know how to build on the platform before starting the
process. Alternatively, you can also use a tool like Moqups, which allows you to build interactive mockups that look
and feel like a real, functioning app.

Once this prototype is designed, validate it with your end users: Does your design actually address the workflow
challenges you identified during scoping? Did you understand the participant registration process correctly, or does
your design have gaps?

Explore Additional Features

If you are using CommCare, our form builder offers additional features beyond
the basics (e.g. skip logic/display conditions and validation conditions), which
you might consider adding to your app for an improved user experience:

In Your Form
Calculations in hidden values: Use hidden values (a special question type) in your form to carry out
calculations that are not visible to the user. For example, calculate a patient’s age in years and months
based on the date of birth question asked during patient registration, or calculate the next visit date
based on today’s date
Define default values: Pre-load an answer to a question in your form by loading a case property, text, or a
calculation into the default value field. This helps the user out by suggesting a response but allowing
them to edit it in the form.

In a Module Case Lis

Icons in the case list: Use conditional icons in the case list to visually highlight important information
about each case
Registration from the case list: Make the registration form directly accessible from a case list, so the user
can go from searching for a patient to registering a new one seamlessly
“Search Only” properties in the case list: Even if you only display a few properties on your case list, you
can still search by others (e.g. District) that will be important to your program. “Search Only” properties
give the user more search and filtering options without overcrowding the phone screen.

15
Learn to Work With Multiple App Builders
Sometimes, programs are fortunate enough to have multiple people working on their app. Unfortunately, this can
also lead to confusion and versioning issues. Our team has a few suggestions when it comes to working with
multiple app builders:

Make sure to define clear workstream owners and divide up the app building
accordingly:

Most tools do not allow multiple builders at once, so be clear about who will be working in the platform

and when.

Make and document builds as frequently as possible:

Quick, but incremental updates will make it easier to spot and solve any bugs that crop up. Check to see

if your platform allows you to control versions to see who made what updates and when.

Set clear communication mechanisms with your team:

Daily or every-other-day stand- up meetings for quick check-ins are helpful.

Incorporate consistent feedback mechanisms for your users:

This will help keep a centralized database of feedback so that all app builders are aware of their users’

frustrations and suggestions. This is especially important when your users and app development team

are in different locations and those conversations cannot happen face to face.

16
 How to Test Your Mobile Data Collection App

Receiving consistent feedback throughout the app building process (in both the design and build phases) is vital to
building an app that truly solves the problems relevant to your users and beneficiaries and helps you avoid any
surprises when you test the final product.

This can be feedback from your end users to understand which workflows make sense and which they are confused
by. Or it can be feedback from your tech team explaining how and why certain aspects of the way you built your app
just aren’t quite working. Pilots also remain a great way to collect feedback on the entire process of developing and
deploying your program before launching more widely.

QA Process
The quality assurance (QA) process can often be the most headache-inducing phase of testing, but at the end of
the day, you need to make sure the app you built actually works. This means dedicating time to QA, documenting
every use case, workflow, and feature of your application, and organizing a test plan to make sure you cover it all.

Here is our advice to organizing the most frustrating, but necessary stage of testing:

Build a Dedicated QA Period Into Your Timeline – and Hold Yourself to It

When you lay out your app building timeline, make sure to dedicate time for testing and bug- fixing. You may
be able to get a simpler app to a high level of reliability in a week or two, but a complex one can take
months to explore fully. If you are doing QA right, you should always feel like there is more to do.

Ideally, QA takes place just before user testing or deployment, at a time when the app is frozen for building
and as many team members can jump in as possible. Having non-app-builders test is a great idea, too. A
good rule of thumb is that whoever built the feature should not be the only one to test it in QA.

App Documentation Is Key

Writing test plans is hard. And time-consuming. And sometimes even boring. However, it’s vital to the
success of your app. Before end users start relying on it, you need a systematic way of assuring that your
app does what it’s supposed to – you simply will not find all of the bugs with exploratory or ad-hoc testing.

17
The simplest key to making sure you do not forget important tests when conducting QA – and that you’re
testing the right things – is to have clean technical specifications for your app, like workflow diagrams and
models of your case structure. Invest time and energy upfront in some truly informative documents, ideally
before you even begin designing and building your app.

Then, when it comes time for that crunched QA you promised yourself, you can hit the ground running
because you already know what to test. This documentation is especially helpful when you need to revisit
this process later – for instance, when you introduce a new feature to your app.

The following is a bare-minimum short list of

types of documentation, which can give you a great head-start on your QA process. You can work in design
programs like draw.io, drop a bunch of shapes into a PowerPoint slide, or use traditional pen and paper. Just
make sure your docs are readable, shareable, and get saved somewhere for reference down the line.

List of Use Cases

Think of use cases as your end users’ list of demands. Here are a few examples
“As a front-line worker, I need to register new clients and keep track of their age, address, and pregnancy
status.
“As a midwife, I need to be able to know which of my clients have missed follow-up appointments and
see them at the top of my visit list.
“As a supervisor, I need a list of all clients who have been referred to secondary facilities in the past
month.”

These ideas may be scattered in different formats around your requirement documents, field notes, and
design specs, but taking the time to list them out in this format before building both makes it very easy to
test these capabilities during QA and ups the chances that you build the right functionality the first time
around.

High-level Workflow Diagrams

Depending on the details of your project, these may be from the perspective of your user, your

supervisors, or someone else entirely. You may be mapping paths between forms within the app or real-life
visits at different facilities. Whatever the specifics, it’s important that you have first laid out the complex
processes visually. Similarly to the list of use cases, your team and stakeholders should have been aligned
on the app before you built it, so you can easily leverage the pathways on these diagrams to create QA tests.

Case Structure and Case ‘States’

Diagramming the ‘architecture’ of your app is essential. Before you can be sure it works properly, you need
to deeply understand how it stores and transmits data.

Start with the case structure. At a high level, map out what case types you’re using and what their
relationship is to one another (Parent? Child? Neighbor?). In a separate set of diagrams, think through where
in your app these cases open and close and the key list of case properties that most impact the workflows.
To put it another way, which properties will have the worst consequences if they’re not working right? Focus
on identifying these, and you will know where to focus your QA.

Make Your Test Plan Write Itself

Once you start building your app, learn to rely heavily on available tools, such as Case List reports in
CommCare. You will quickly realize how useful it is to visualize case relationships to see all the places they
are affected.

If your documentation is extensive and up-to- date, you will find that QA tests start to generate naturally.
Each one of those ‘As a ___, I need to ____’ statements is a solid use case and a test to run, custom-written
for a test user.

18
 Each case opening/closing/pivoting point is a test, though it needs a bit more help and can be more subtle
to troubleshoot. These tests will read more like: “Enroll a female client, age 28, who is pregnant. Search for
her case on the case list report and verify that date of birth, gender, and pregnancy status are set correctly.”

QA Is Never Over

Though QA ‘cycles’ may end, QA is really a state of mind: Hold your applications to a higher standard by
developing strong habits around testing as you build, even as early as the design phase. Your app is doing an
important job, helping a frontline worker do hers. When you find bugs, it means she doesn’t have to!

For Your Consideration...

An application handles immunization for newborns, family planning appointments for female clients age 12 and
above, and prenatal visits for pregnant women.

The registration form intakes clients of all ages and genders, opens a client case, and saves case properties to it:
date of birth (‘dob’), gender, pregnancy status, mailing address, phone plan, and favorite food. Which of these case
properties sounds most important to track in your list of use cases and in QA, given that it needs to identify which
clients need which type of services?

If we later discover that the favorite food property is malfunctioning, it would probably have less of an impact on
the app than pregnancy status. Think through these ‘pivot points’ in your app, and list out the case properties that
define them.

It is a given that resources and timelines will always be tight, but prioritizing the components of your app will help
you outline a QA test plan that will make the most of your time.

19
Usability Testing
Usability (or ‘user acceptance’) testing is the process of putting a functioning app in front of your end users and
seeing what they think. The concept is quite simple, but the execution is a bit more nuanced.

From selection bias to asking leading questions, there are many ways that your approach to collecting feedback can
lead your users to give you the answers you want. As tempting as this can be, you need to make sure that the
feedback you collect is as unbiased as possible. Here are six quick tips on how to make sure you keep it clean:

Identify the right users

Make sure you have a representative sample of your users to ensure that everyone who ends up using the
app is accounted for
High- & low-performin
High & low digital literac
Rural and urban

Ask open-ended, rather than leading, yes/ no questions

Just like the questions you ask in the app, the goal of user testing is to uncover clear, unbiased feedback on
how you might improve your platform.

Observe, don’t demo

You want to see the users interact with the system in as realistic an environment as possible, which means
you are not there to hand-hold. See what flows they get caught up on and which go smoothly. This will be
representative of when the users you might never meet get their hands on your app.

C onsolidate observations and feedback

Organizing your feedback in a structured way helps to get a clear picture of the trends. It also ensures that
you can easily compare and contrast the feedback with that of future iterations of the app.

Sort through the noise

D evelop a clear idea of your priorities and how the feedback you receive aligns with them. You cannot

incorporate every change request, so have a decision-making process that all stakeholders agree on, and
stick to it.

a tor enough time in your project plan to incorporate feedback

F c

Q A your app thoroughly again before deploying.

Along with testing the effectiveness of the system you built in the eyes of your users, usability testing helps
determine how the technology fits into your overall project objectives. This phase is a great opportunity to review
everything from your data requirements to your user stories, ensuring that your new mobile data collection tool
delivers on everything you expected from it.

For six more tips, check out this guide to usability testing from our global services team.

20
Pilots

Pilots are the time and place where we get our technology right and ensure we are developing something useful for
the user. They are when we establish and test processes that are required for the long-term success of a program.
The tension is that pilots are meant to be both a special, focused program effort that may differ from the final
program, as well as a process to get us ready to scale and succeed in that same final program.

Here are three key considerations for this phase:

Pilots Should Be Understood as Special

Pilots, by their role in testing and iteration, require extra attention and support. They are a

time to stress test our theory of why this program will be effective, and we want to give it the best chance
to succeed. This means the pilot users or sites might be unique in terms of geography, user types,
connectivity, or other factors.

For this and other reasons, pilots may not then be fully representative of the experience for a diverse set of
users at scale. However, we can be intentional about articulating what makes those pilot users special and
strive to get good representation where feasible.

One danger for pilots on large projects is that we become very good at running one site, but not at the entire
project. This risk can be mitigated by proactively thinking about the various dimensions of scale that need to
be considered and understanding the broader sets of users, geographies, and infrastructure earlier on.

Pilots Are Not Just for Testing Apps

Just as pilots are not only for proving a program’s viability with more users, pilots are not just for apps. They
can test so much more, like training methods, supervision performance, and reporting processes. All aspects
of a program can benefit from a bit of focused iteration and testing before going big. For example, in many
projects, when we roll out a new customized reporting tool, we may again start with a small group of pilot
users within the larger pilot cohort to get feedback and iterate before rolling it out more broadly.

Pilots Are Not Just for the Start of Projects

Even if your program has been successful for some time, you can always work on improving it. But at the
same time, you don’t want to just introduce wholesale changes. In our spirit of continuous learning, pilot
users or sites can serve as the “beta testers” for new app content, integrations, or processes that are
introduced throughout the course of the project.

The CRS ReMiND project in India uses this approach to pilots quite well. The project had 10 ASHAs in the
initial cohort of CommCare users and showed that they would experience increased productivity and
efficiency at scale. So, when the app was scaled to over 250 users, those ten initial ASHAs served as the
pilot for various iterations and phases of the project in the months that followed, as well as key advisors on
the program’s direction.

Why Does Testing Matter?

Testing is a key way to ensure that after selecting a tool, designing the structure of your app, and building a
prototype, you have still placed the end users and beneficiaries at the center of the process. Check your
assumptions against reality to see whether the solutions you have developed will have the impact you hope for.

When your end users and beneficiaries are the focus of and play a role in your process, you ensure a higher quality
application with workflows that address the reality on the ground. Most importantly, with more engaged users on
well-designed workflows, your application will see higher usage and ultimately higher impact.

21
 How to Deploy Your Mobile Data
Collection Software
Your app might be built and the first version tested, but before you can press “deploy,” there are few final pieces to
iron out. You need to know what devices your app will run on and the network they’ll use to transmit data. Your
workers need to know what they can use the devices for and how to refill their data when they run out. Cover off on
these final preparations to make sure the mobile data collection app that you have worked so hard on actually
works on a mobile device.

Choose the Right Device

Your app should not only function on its device – it should thrive.

Problems like slowness and app crashes should already be addressed during design and testing. However, while
smaller projects might be able to design their application for a specific device model, apps for larger scale
programs may need to work for a wider range of devices. In one of our projects, which spans seven Indian states,
different states teams use different phone models based on their procurement process. In all, there are five
different phone models used in the seven states to run the same application.

Within the first few state rollouts, we realized the importance of selecting the right device. We started facing
application usage challenges due to poor quality devices. One device we had selected was quickly running out of
battery, while another device slowed performance after only a few months of use. To maintain a basic standard, we
developed a list of 42 requirements that each device needed to fulfill for that project – a process we now
recommend for most projects that plan to scale beyond their initial launch.

A few basic things to keep in mind while creating such a list for a project:

Depending on the nature of data that is collected, decide between a feature phone and a smartphone.
With the reduction in smartphone prices, most of our partners select smartphones, but we’ve made a
preliminary guide to help with this decision.
In smartphones, it is important to know which operating system will support your data collection tool.
For instance, CommCare is supported only on Android devices, and we have tested it across a range of
devices and summarized our experience with each.
The more complex and customized your application, the more important it is that you perform these
tests with your own app to make sure you find the right device(s) for your program.

22
 Internal memory and battery life are also important to keep in mind while choosing a device. We put
together a checklist to help our partners make the right decision while deciding on a device.

Prepare Your Devices

As the number of users increases, the complexity of device preparation increases exponentially. Smaller projects
might not have as much trouble preparing their devices for the field, but preparing a device for field use is still a
tedious job and there are many pitfalls you can encounter.

When a new device is opened, it takes a few minutes for the initial setup. Often, there is a mandatory operating
system update, which adds a few more minutes. Once the device is ready, the mobile data collection app needs to
be installed.

For projects with only a few users, this is not too much of an issue. However, for projects with hundreds (or
thousands) of users, this process can take weeks, with each device requiring up to 45 minutes for complete setup.
Therefore, it is important to budget both time and resources for device preparation accordingly. To avoid last-
minute scrambling, large programs should identify a specific person to lead the activity who is accountable for
delivering prepared devices to the training/rollout team.

Find the Right Data Plan

Without a doubt, you will want to make sure that the data plan you choose is with a provider who has good
coverage in your project area and will cover the data required to transmit the number of forms your users collect.

Which providers work in your project area?

There are two quick ways to find out which network providers you should explore: The best way is to check
with the end users or mobile shopkeepers who live and work in the area. If that’s not an option, check an
app or platform, like OpenSignal, that can provide different service providers’ coverage maps.

What are the providers’ billing schemes?

We’ve found the best option to be providers who bill by kilobyte of data rather than by time. Just make sure
their rates are competitive and you know how much data each new SIM is pre- loaded with. Also, compare
their pre-paid and post-paid plans. Pre-paid plans might offer a discount, but post-paid plans will help
make sure your team does not run out of data in the field.

How much data do my forms use?

A good rule of thumb for a single form is around 7-10kb. However, depending on the platform, you can
access more reliable estimates. In CommCare, go to “Submit History” > “View Form” > “Raw XML” and save
in a text editor. Check the file size to see how large a form is. Then all you have to do is multiply that form
size by the number of forms you expect a frontline worker to submit in a day and again by the number of
days per month you expect them to work in order to have a final estimate of the data required to submit
your forms.

What else might you use data for?

The focus of your data plan is usually on form submissions, but you will also need data in order to cover
other things like updates to your app, pulling down case data, multimedia capture, and other background
services on the phone. At the end of the day, many CommCare projects comfortably use monthly data plans
of 100 MB or less.

Once you have a provider and plan that works for you, there are a few things worth asking for that might
make life a bit easier for your team. For instance, see if you can procure a set of SIM cards in a series. Or,
ask if the provider offers an option for closed-user groups, so you can set up free calling (and/or SMS)
between the users and staff of your program. There are no guarantees that these options will be available,
but the least you can do is ask!

23
Develop Phone Usage Policies
Don’t skip this step. Phone usage policies are crucial, and we recommend them for all of our projects. These
policies determine device handling, including what should be done in case of lost or broken devices and theft.

They might also cover how much mobile data the device will have or what apps users are allowed to install.

Precisely what these policies cover will depend on the nature of the project, but they will help in setting
expectations with your users to avoid confusion and disputes when an incident does occur.

Once your devices are ready for action, it’s time to introduce them to your team.

24
 How to Train Your Team in
Mobile Data Collection

We’ll be straight with you: Actually launching your program can be the hardest part. In fact, 70% of the World Bank’s

information and communications technologies for development (ICT4D) programs have failed. And finding out why
can be a tricky test. Fortunately, we have the experience of over 500 projects to help inform the answer.

For one, digital literacy is often low among the frontline workers in the regions where mobile data collection tools
are introduced.

Another reason is that the interpretation of technology varies from person to person and depends on the context of
the use case: The way an experienced ICT program manager sees a particular data collection app is likely different
from how the community health worker will approach the same solution.

Use the information you have about your audience and environment to transform your new tool into a successful
program. Realize it is not the app that will solve all your problems: It is the people who use it who matter most.

Choose the Right Method of Training

Effective adult learning involves specific techniques. Experiential learning, an understanding of a topic’s importance,
and freedom to learn in their own way are a few methods that are helpful during trainings. Keep in mind your
participants’ backgrounds while choosing which of these techniques is the best fit.

First, your workers need to know how the app will help them do their job better. Explain to them how it will help
them reach more beneficiaries or consult them faster. Show them results from the pilot or data from similar efforts
and challenge them to beat it. You can’t expect your workers to just accept the challenges that might come with
the new way of working – you have to empower them.

Second, everyone learns in a different way. According to Malcolm Knowles, an adult educator, there are three main
approaches to learning:

Visual:

Visual learners prefer to be shown how the app works. Walk them through each screen, show them how the
features work, and make sure to keep asking, “Do you see how this works?”

Auditory:

Auditory learners prefer to have the app explained to them. Talk them through the features of the app and
what they stand to gain from using it.

25
 Tactile:

Tactile learners like to get their hands on the app. You can still talk them through workflows and features,
but do so while they have the app running so they can experience it as it’s explained to them.

Just like you know your workers’ level of digital literacy and objectives, you should know what type of learners they
are. Alternatively, try for a training approach that blends all three methods.

Deliver the Training

We cannot stress enough the importance of the connection between trainer and trainee. Our trainings have been
most successful when participants have been able to interact with the trainer without inhibition.

Building this connection can be tough, and it requires you to help the trainees overcome the usual dynamic and feel
open enough to ask questions and share their thoughts. Our field managers often use some of these tricks (and
they’ve often found bringing some candy helps to sweeten the sessions).

You should expect your approach to trainings to change the more you learn about your audience. Just like your app,
this is an iterative process. For more advice on how to run your training sessions, check out these tips from our
global services team.

Reinforce Your Training

Post-training reinforcement helps a great deal in retention of newly-learned skills and knowledge by filling any gaps
that remained from the initial training. Two ways we reinforce our training are:

Repetition:

We conduct refresher trainings at regular intervals to cover both the things that our workers have forgotten
and our new workers. It also provides a good opportunity to notify workers of new features or for workers to
share advice and feedback.

Resources:

We develop support materials to guide workers as they use the application in real time. Whether it’s a one-
pager that frontline workers can quickly reference on the go, a comprehensive user manual for figuring out
the more complex aspects of the application, or an in-app training module, the idea is to provide your team
with the tools they need to troubleshoot any issues on their own. You can also complement these materials
with best practices and tips from other workers for those users looking to go above and beyond. For more
advice on how to compile a training guide for your app, check out our guide.

Give yourself and your team the best shot at successfully launching your mobile data collection program by
arming them with the tools and training they need. Tailor your approach to their needs, just like you did with
the app, and they will be better for it.

And remember that you won’t always get it right the first time. Never be afraid to switch up your approach
to training: Add more visual aids, toss in an icebreaker to get everyone loose, or shorten your user manual
into a one-pager. Work with your users to figure out the right balance for them.

26
 How to Sustain Your Mobile
Data Collection Program
You can’t assume your work is done after the last training session.

To ensure all the effort you put into the implementation of the program pays off, you must set up a system to
support your users. They should have proper resources and supervision, incentives to continue their usage, and, if
they have thoughts or suggestions about the program, opportunities to be heard.

Support System
The importance of a good support system to sustaining a mobile data collection program cannot be overstated.
Ensuring that responsive channels exist for users to flag issues they might face – related to the application, device,
or data – is important to build trust in the system.

Create a Manual to Serve As a First Point of Reference

In-depth user manuals (printed, digital, or otherwise) for each section of your application are useful
resources and should be the first point of reference for most users. These manuals can contain information
about application features,

how to access them, and (depending on your needs) programmatic guidelines.

Identify Support Channels and Encourage Engagement

Make your users aware of and encourage them to use the support channels you give them. The end user
training before launch is a good first opportunity to introduce them to what support is available. From user
manuals to support personnel, training sessions, and refresher courses, your program should develop a
variety of ways to offer your workers support and sustain their usage. It is equally important to build
capacity among the local partners so that they can solve basic issues that might come up and offer avenues
to escalate those issues, should they not have the expertise to solve them.

Designate a Support Person for Scale

he scale of the support needed is another factor to consider: A single designated support person can handle
a 50-user pilot, but a national-level project with thousands of users might require something more robust
like district-level help desks or a central call center. In both cases, the designated support person(s) should
have:

27
 A good working knowledge of the applicatio
Admin rights to look into the system backen
Strong communication skills

Track Response Rate on Issues Raised

Furthermore, you should track and respond to the issues your users raise. Delays in communication or
repeated failure to resolve their problems will cause users to lose trust in the program. Key metrics, such as
number of open issues at each level and average time taken to resolve, should be closely monitored to
ensure that the support system is running smoothly.

If you want your workers to use your platform, you need to make sure they know how and

address the questions that come up. A well- executed and diverse support system should ensure your team
is supported and always performing at its best.

Supervisors
A clear view of the entire process, including both program and worker performance metrics, is crucial for your
supervisors to keep the program running. With paper forms, most of their time would be spent on tracking down
data and following up with users. Now that their program runs on a mobile device, this information should already
be at their fingertips, allowing them to focus their efforts on supporting the workers who need the most help and
optimizing usage.

With CommCare, we like to use Worker Activity Reports to achieve this. They allow supervisors to see broad-stroke
details of the work that the end users are doing, including the number of forms submitted per user per day to help
measure activity on the application. Your supervisors should be trained so that they are able to access and monitor
the reports themselves, giving them a direct window into their team’s work.

Whatever platform you use, the metrics used to monitor worker performance should be designed as per the nature
of the project and agreed upon between workers and supervisors prior to implementation. For instance, one project
might require a user to submit only a single form per day, whereas another might require five. In some projects, the
number of form submissions might not be reflective of their work at all. While gathering requirements for the
project, care should be taken to also find out how the supervisors currently monitor the workers and where the
existing gaps lie, so monitoring reports can be designed accordingly.

Incentives for App Usage

If users are not given sufficient incentive to use the application, then you should expect usage to decline once the
novelty of the program wears off. There are a number of ways to ensure your team continues to use your tool for the
duration of your program:

Save them time and effort

Reducing the amount of time and effort required from a user is a great way to ensure sustained usage. If
they understand that it is easier for them to fulfill their reporting requirements via the mobile app compared
to using paper forms, they will be more than happy to keep using the app.
Send them reminders:

The implementation of reminders and nudges from supervisors into the app will help keep users on task
and aware of their responsibilities. Whether you do it with a simple SMS or during weekly or monthly check-
ins, it is always a good idea to let your team know what they need to do next.

Show them their impact:

When users understand the impact they have, they are more likely to continue to participate. Show them
the number of children they helped vaccinate or the number of institutional deliveries they carried out.
Depending on your team’s mindset, comparative performance reports could also be a way to instill a sense
of healthy competition.

28
 Give them feedback:

Regular feedback and engagement through supervisors is another way to motivate your users. The
immediate availability of a wide range of data should allow your supervisors to extract insights on the users’
strengths and weaknesses. For instance, a user might have a high number of form submissions (good!), but
may be limiting themselves to a small geographic area (can be improved!). Keep feedback positive, including
tips on how to improve performance.

Closing the feedback loop by offering tools to improve their performance will help ensure that your team
continues to use your application. And the more reasons you give your users to pick up the app, the better.

User Feedback Mechanisms

Yes, massive effort went into the development and implementation of your mobile data collection tool. But to
sustain the usage and impact of your program, it is not enough to just build and release a great application.

Careful thought needs to go into your systems of support, the work of your supervisors, incentives for continued
usage, and how you collect and address feedback.

While there is no one-size-fits-all approach that can be followed to ensure the sustainability of your program,
follow these basic principles to give your team the best chance. Offer help to users who need it in a way that works
for them. Ensure your supervisors can see how their workers are doing, including how they might improve. Build in
mechanisms to help your workers perform at their best. And touch base with them and their supervisors often for
helpful insights to optimize your program.

We have seen many times that a project develops a great app, introduces it to its team, and successfully launches
their data collection program, only to see it end after the pilot phase. We believe all that hard work should not go to
waste, and by maintaining your app, and supervising and supporting your team, while also listening to what they
have to say, you can help your program continue its important work for as long as possible.

What’s next?
Visit www.dimagi.com/commcare for more personalized advice from the
team who built the world’s most powerful mobile data collection platfrom:

Track data

NutriCare Application Build smarter data collection apps that

allow you to collect and track data over
time.

Work offline

No signal? No problem. Build a data

Start Saved collection app that works offline.

Emp wo er end-users

Collect quality data by building an app

Incomplete Sync with Server

that guides your workforce to do their

You last synced with the server

at 11:30:21 AM
best work.

S tart small and scale

Invest in a data collection app that will

keep pace with your growth.

29