CartONG – 23 boulevard du musée, 73000 Chambéry – France
www.cartong.org | [email protected]
A D VA N C E D XL S F OR M S C OD I N G
Table of contents
I. Sorry, what’s your name again? .............................................................................. 1
I.1. Same street name, different towns ................................................................... 2
I.2. Beauty in simplicity........................................................................................... 2
I.3. Familiar faces ................................................................................................... 2
II. First things first ..................................................................................................... 3
III. Patterns, everywhere ........................................................................................... 3
III.1. The wizardry explained .................................................................................. 5
III.2. VIP Guest ....................................................................................................... 5
III.3. More patterns ................................................................................................ 7
III.4. Alternative for “If other, specify” .................................................................. 8
III.5. Caveat ............................................................................................................ 8
IV. Calculations: dealing with blanks .......................................................................... 9
V. Testing, testing, testing…. ...................................................................................... 9
VI. Conclusion .......................................................................................................... 10
By now, many adepts of mobile data collection know the basics about how to code XLS forms,
with good online resources for support.
However, past the basics of the syntax needed to make a form work, not that much is available
and it’s exactly that gap I’ll try to address here. So let’s assume you have coded a couple forms
already and they work. But now you have to code a long one, a half-dozen (paper) pages, full
of tiny multiple choices and tables with a complicated survey-flow pattern… how to make this
faster? This will be the first part of two posts on this thematic.
I. Sorry, what’s your name again?
When coding a long form, a naming convention can go a long way in streamlining the process.
There isn’t one “best” answer to this, however it boils down to:
“Do what you want, but do it for a reason.”
The fewer decisions you make without really having some reasoning behind it, the better your
coding method will be. The following tips have all been set up with this in mind. Therefore, feel
free to take them whole, or adapt to needs & preferences.
I.1. Same street name, different towns
I started some time ago assigning similar names to the “list name” items (that make up the
multiple choice items in the “choice” tab) and to the prompt’s name in the “survey” sheet. Some
differentiators can used, such as add “L” at the end. This way, you can easily edit any given list
choice without having to constantly lookup the main survey sheet (and the other way around).
You then know that to edit the relationHosts prompt, you need to edit the relationHostsL list in
the choice tab. If you place the different list choice in alphabetical order, it also makes it easier
to find it in the choice tab (which is why you would put the differentiators at the end).
The exception to this would be very common choice lists shared by multiple prompts, such as
Yes/No, Increase/Decrease/Stable, etc.
I.2. Beauty in simplicity
When making up a new variable name, I try to be both concise & descriptive. For example, let’s
say we want to assign a name a prompt about vulnerable groups of internally displaced people
by shelter type. There are many ways to name this:
vulnerable_groups_displaced_people
vulnerableGroupIDP
VULNGRP_IDP
Question_5_2A
Of course, the first one is the most descriptive, but isn’t concise. What if you have that question
nested inside a group of questions that is related to Protection and Security, which is itself nested
inside a group of vulnerability factors? If you’re being consistent, you might have named the
groups “protection_security” and vulnerability_factors, so in the database you have a column
name such as “vulnerability_factors/protection_security/vulnerable_groups_displaced_people”.
There are limits to the length that can be assigned to those columns, and they are database-
specific. Bumping into those can lead to problems further down the road.
The very last option of naming the prompt after the question number (probably issued in the
paper form), though commonly used, has many shortcomings:
Adding/removing questions means the numbering will have to be adjusted (useless
work)
It forces you to go back and forth between the XLS form, the paper form & the output
to know what means what
Finally, your XLS form isn’t self-contained anymore: to really know how the form is
structured, you almost need to refer to this other document (word or printed form).
From my perspective, the 3rd option will work best, especially if the expression “Vulnerable
groups” comes up often: in no time VLNGRP will be synonymous to “vulnerable_group”. It strikes
a balance between being descriptive enough to be recognizable at first glance, while not being
too cumbersome.
I.3. Familiar faces
In the choice list, I tend to favor using numbers as values instead of letters. There are pro &
cons to this, however I like the objectivity that comes with assigning numbers to each options:
a 5 is a 5, whereas the choice “Religious Minorities” could have a number of “values” assigned
to it, endless combinations of camel-case or underscore separators, capital letters, etc.
Another potential benefit of this (although it can certainly be possible using letters as values as
well) is to use the same value for very common options that appear in many lists:
-88 for “Others”
-99 for “Do not know”
0 for “No”, 1 for “Yes”
You can establish similar conventions for any common value in your survey
The upside being that when coding constraint, relevant conditions or “If other, please specify”
prompts, the value to check for will always be the same. No need to go check the choice list for
that, ever. Common relevant/constraint can also be copied & pasted, because the action to take
if the answer is “Other” or “No” are often similar.
On the other hand, there is nothing wrong with using readable names for those values instead
of numbers – as explained above, there are pros and cons to both. Depending who will be
working with the output, and what your analysis set-up is, you might decide that it is better for
you to have those value readable right in the output file –especially if your analysis system isn’t
programed and a real person will have to manually interpret the data directly from the server
output.
II. First things first
For a short form, it might make sense to just code everything prompt by prompt. However, for
a long or complex form, the best structure to adopt or the kind of relevant / constraints /
calculation that should be set might not always be obvious: this might come either through
testing or through the training sessions of the end user (if that’s possible in your situation). It
might be more efficient to code those advanced elements progressively, as the form matures.
I would suggest keeping it to the labels, hints, prompt name & types, and adding already a little
bit of structure (grouping questions either for the presentation on the phones if you have larger
screen, or because they are thematically related, etc).
III. Patterns, everywhere
When coding a more complex survey, some common patterns arise such as questions about
rankings:
What are your top 3 three priorities in order of importance?
Food
Lodging
WASH
Clothing
Health
No other priorities
In a paper form, this would translate into small boxes with numbers 1-2-3 written next to a list
of priorities. However, with ODK Collect, there isn’t necessarily just one way to approach such a
prompt. For ease of analysis, I tend to prefer avoiding multiple choices when possible, as they
are generally more difficult to deal with when comes the time to setup the analysis. My inclination
in this case would be to code 3 select_one prompts, with the choice list being the list above for
each.
Logically, if a respondent expresses not having any other needs at the 2nd prompt, you wouldn’t
want the next one to show up. They also shouldn’t be able to select the same need twice
A few things to note:
To code any similar group of question, I can simply copy-past all 3 prompts and then use “Find
& Replace”:
Copy the full 3 rows of the men’s priority questions and past them further down.
Select the 3 questions you just pasted, then ctrl+F
In that box, simply replace the common root for the men’s prompt by the common root
for the woman’s prompt & hit “Replace all”:
Which will the yield:
Pay attention to the number of replacement (Excel will tell you how many replacements have
been made). You want to make sure you selected only the cells in which you want excel to find
& replace: in our case, the 3 rows we just copied. If Excel tells you it made 110 replacements,
you might want to ctrl + z this and retry!
The “name” column have been automatically taken care of, because they all share a common
root, NEEDPRIORW, that replaced the NEEDPRIORM for men. Because we selected the
relevant & constraints columns as well when we hit ctrl+F, Excel replaced those as well.
Setting up the same pattern for various different groups would take a few minutes at most,
whereas coding each of those questions one by one can quickly become tedious and error-prone.
Even if those ranking questions do not refer to the needPriority list of the original prompt, all
that is needed then is to replace the list name as well and you can even use the find & replace
method for that, too. The relevant and constraints conditions are still correctly codded, assuming
you followed the previous advice of consistently using “0” as a value for “No” (or in that case,
“No other needs”)
III.1. The wizardry explained
As an aside, it might be worthwhile explaining how the previous “constraint” works (the
“relevant” pattern is well-documented on the net already, so we’ll skip that part). It is meant to
prevent the user from selecting the same option twice, as you can’t have “Food” as both your
first and second priority. The notation:
not(.=${NEEDPRIORW_1}) and not(.=${NEEDPRIORW _2})
The “.” before the equal signs, in all XLS forms, means “the current prompt”
A CONSTRAINT expression must always evaluate to TRUE if the user is to be allowed to
go to the next prompt
In case the current prompt (“.”) is equal to NEEDPRIORW_1, then that will evaluate to
“TRUE”. Since we don’t want the user to be able to proceed to the next prompt in that
case, we inverse its value by wrapping it into a not() statement. If the current prompt is
different from the NEEDPRIORW_1, then the expression would yield “FALSE” and after
negation it will evaluate to “TRUE”, and the user will be able to proceed.
For NEEDPRIORW_3, we want the value to be different from both the 1 st and the 2nd
prompt in the series. We therefore added the “and” statement between the 2 conditions:
if any of the condition is not met (so the selection is equal to either the 1st or 2nd prompt)
then the whole expression will evaluate to “FALSE” and the user will have to change his
selection.
As those constructs get more complex, it might be worthwhile to test them more thoroughly, to
ensure they work as intended. For example, making a mistake here might prevent the user from
selecting different answers between the 3 prompts – definitely not what we want!
III.2. VIP Guest
Those who coded forms directly in XML can definitely recognize the value of the “choice filter”
column in XLS forms, which allows to easily filter the choice being offered in a prompt, based on
a previous selection. For example, you wouldn’t want to show the full list of 196 countries if the
user already told you he comes from North America.
However, for some types of lists such as villages in a remote region, it can be hard to ensure
you have an exhaustive list, therefore it would be preferable to always allow the user to select
“Other” and then offer a text input for the missing value.
In this example, we have a list of social workers, based in different region (“prefectures”). We
want to use the choice filter to show only the workers in the selected region of operation,
however we need to offer some flexibility in case there’s a new hire or a replacement. For this
to work, we must first add one entry in the social worker list in the “choice” tab. We add “Other”
(“Autre”) option and assign it the value -88. In the column “prefecture”, where the choice filter
operates, we also put “-88” as a value (or any value you usually assign to your “Other”
statements).
In the main “Survey” tab, we have 2 prompts of relevance to this: the INT_Prefecture prompt,
which is the region of operation that the user first selects. This is stored in the variable
INT_Prefecture. The prompt INT_NameWS is the prompt where the social worker is selected.
As can be seen in the choice_filter column, the selection is based on the choice made in the
INT_Prefecture prompt. The form then takes that value, and looks into the “ts” list_name,
and shows all those for which the “prefecture” column value matches the INT_Prefecture
value. But because we also added “or prefecture = -88”, it will also show any value in the “ts”
list_name that has value -88. This is our “Other” option.
And the result is that we can both filter the list of workers according to the prefecture of activity
AND always offer the option to enter another name, regardless of the prefecture selected:
The selection will filter answer according to our filter value, as well always allowing “Other”
value.
III.3. More patterns
Another way that using those patterns can lead to faster coding is when setting up the rather
common “If other, please specify”. When coding a long form, it can be advantageous to setup a
“generic” prompt of that type, without setting reference to any specific variable. It can then be
copy-pasted as needed, while filling in the prompt name to which it refers only at the end:
I try to always append the suffix _OTHER to this type of question, as that allows to quickly
identify them. I will generally use as a core part the same name as the question it refers to. For
example, if the men’s question about need priority did allow for “If other”, I would name it
NEEDPRIORM_OTHER.
I also have only set the generic envelop for the relevant condition for this question. In that case,
if any of the need priority is answered by “Other priority” (which value, as always, is -88), then
I want the prompt to appear.
The above can then be copy-pasted any number of times and to be functional it only requires
adding a complete name and the references to the prompts in the relevant conditions. Proceeding
this way is much faster than manually typing each and every such prompts in the form and the
benefits add up the longer and more complex the form gets.
III.4. Alternative for “If other, specify”
There is an alternative as well for every question which needs to offer a text input option for
“Other” types of answers: one must simply append “or_other” in the “type” column, as shown
below:
In the list choice, there is no need to add the “Other” option as this will be done automatically
by the form:
In the end, the output file in your platform of choice will automatically add a column for the
“Other” text field, just like that would be the case with the manual coding option described
previously.
III.5. Caveat
This is an interesting alternative because it is really quick: it allows you to basically forget
everything about the “If other” option, and let the form handle it. However, there will be some
defaults choices that you’ll have to live with, should you choose that option:
You cannot decide what the label in the list of choices for the user will be: it will say
“Other”. If you have a form in multiple languages, you will have to instruct your user that
this means “Autre”, “Otro” or “Ostatni”.
Same for the coded value of the option: the value will be “other” in small case. If you
have (wisely) opted for some sort of system in the assignment of your value, you might
have to take this into account. In that case, I would opt to make it uniform and always
assign “other” instead of “-88” to all choices that mean “Other”. Remember, “Do what
you want, but do it for a reason”.
The name of the additional columns will always be “nameOfTheQuestion_other”, with
“_other” being automatically added – this would be convenient with our system which is
good.
Finally, we are forced to have one specific “other” column for each question. In the previous
example (by coding manually the constraints), we had a system that would show only one
“Other” column if ANY of the 3 questions have “Other” selected. This is because while we want
to know if some choices are missing in the list, we might not be interested in knowing specifically
if it applies to the 1st, 2nd or 3rd question: perhaps it is sufficient for us to know that the user
wanted to specify something we haven’t listed.
IV. Calculations: dealing with blanks
Performing in-form calculation can be very handy. This can help the enumerator double-check
that the value do indeed add-up, or even code constraints in the range that is acceptable for a
series of related questions. However, one problem is that if ANY of the prompt referred to in the
calculation is left empty, then the result will be blank.
You may get around this by forcing an answer (“required” column), however it is wiser at times
not to make all prompts mandatory – when answer cannot always be expected to be available.
There is a workaround for this, and it involved using IF statement to assign the value “0” to any
unanswered question. Consider the following:
For every variable involved in the calculation, the calculation tests to see if the value is
a number equal or greater than 0:
o If(${A1A_totAvant}>=0, ${A1A_totAvant},0)
o If it is, then it will use that value in the calculation
If(${A1A_totAvant}>=0, ${A1A_totAvant},0)
If not, then it will use 0 as a value for the calculation
If(${A1A_totAvant}>=0, ${A1A_totAvant},0)
This works, because for the XLS form if an answer is skipped (let’s say that the number A1A_fled
isn’t available), then the “value” assigned to that skipped question is blank, and blank will always
be considered “less” than any other numeric value. Because a calculation can’t be performed on
a blank value (a blank value isn’t 0, it is simply nothing), we must assign a value 0 if we want
the rest of the calculation to be performed anyways.
Note that this does NOT assign a value in the original prompt – this is only to allow the calculation
to be performed, so that a meaningful note can be showed to the user.
V. Testing, testing, testing….
Regardless of the platform on which your survey will ultimately be hosted, you will most likely
be better off testing your form more often than less. Depending on your level of comfort with
XLS form, things can get complicated. Catching problems early on, while they are fewer in
numbers and easier to track, will always be easier than to code a whole form and then have to
debug problems after problems.
One option for quick testing of forms in development is to use the offline converter (available at
https://opendatakit.org/use/xlsform/ where all converter options are listed). When you submit
an XLS form to the converter, it will tell you if any syntax is wrong, if there’s a typo in one of
the list_choice or value you used in your constraints/relevant conditions, etc. The messages are
generally very informative.
Another option is using the Kobo platform:
It takes directly the XLS form used for coding, instead of requiring a prior conversion to
XML before being uploaded to the server
It can be tested directly in the browser, in a page-format that allows to have a global
view of the form (instead of the page-by-page display on the phones)
Opening an account is free, therefore it makes sense to have one, even if it isn’t meant as a
development platform. It is worth nothing, however, that the behaviour of the form in webform
can be slightly different than on the phones. The webform also doesn’t allow to have a feel of
how the form will behave with the smaller display of the phones actually used for the survey. It
should therefore be used while the form is in development, but it doesn’t replace testing the
form with the actual devices that it will be used on.
VI. Conclusion
Before the XLS form, everything had to be coded in XML, which was more complicated and
slower. Now that XLS forms have greatly reduced the complexity and time needed to code a
survey, it is time to develop coding practices that maximize this benefit, and hopefully the ones
above can serve as a good starting point to the development of a personal coding system that
is accurate and effective.
