Difference between revisions of "SurveyCTO Programming Work Flow"

Jump to: navigation, search
 
(28 intermediate revisions by the same user not shown)
Line 1: Line 1:
<onlyinclude>While it is possible to code a SurveyCTO [[Questionnaire Design|questionnaire]] in many different ways, the time it will take and the quality of the final product depends on how the work is planned. This article outlines an approach common among experienced SurveyCTO programmers at DIME. </onlyinclude>
<onlyinclude>While it is possible to code a SurveyCTO [[Questionnaire Programming|questionnaire]] in many different ways, the time it will take and the quality of the final product depends on how the work is planned. This article outlines an approach common among experienced [[SurveyCTO Programming|SurveyCTO Programmers]] at DIME. </onlyinclude>


== Read First ==
== Read First ==
*Do not start from the beginning of the [[Survey Pilot|survey]] and work your way through one question at the time. That takes longer and usually results in less well structured [[Questionnaire Design|questionnaires]] that are much more error prone.  
*Do not start from the beginning of the [[Survey Pilot|survey]]; instead, work your way through one, question at a time. The former approach takes longer and usually results in less well structured [[Questionnaire Programming|questionnaires]] that are much more error prone.


== Paper First ==
== Paper First ==


All surveys should always start with creating a paper questionnaire. See [[Questionnaire Design]] and [[Survey Pilot]] for instructions on how to develop and test a paper based survey. When there is no time to create and pilot the paper based version of the questionnaire, a questionnaire should still be created in Word or Excel before starting to code the survey in SurveyCTO code. Skipping this step almost always leads to issues with data quality.  
All [[Survey Pilot|surveys]] should start with creating a paper [[Questionnaire Design|questionnaire]]. When there is no time to create and pilot the paper version of the '''questionnaire''', it should still be created in Word or Excel before starting to code the '''survey''' in SurveyCTO. Skipping this step almost always leads to issues with data quality.  


The important part of doing a paper questionnaire before starting to code is that it allows the project team to focus on what question to ask and how to ask them. The questionnaire coding can be implemented to any format, display and functionality decided on in the paper questionnaire. By focusing on content first and programming implementation only afterwards, the quality of the data we collect gets a higher priority than when the questionnaire is set up in a way which is technically convenient to program. If the SurveyCTO code is implemented before or in parallel to developing the content, then the code always takes focus away from the questionnaire and negatively impacts the way a question is asked and answered.
Creating a paper '''questionnaire''' before starting to code allows the project team to focus on what questions to ask and how to ask them. The [[Questionnaire Programming|questionnaire coding]] can be implemented in any format, display, and functionality decided on in the paper '''questionnaire'''. By focusing on content first and programming afterwards, data quality is a higher priority than if the '''questionnaire''' is set up in a way which is technically convenient to program. If the SurveyCTO code is implemented before or in parallel to developing the content, then the code always takes focus away from the '''questionnaire''' and negatively impacts the way a question is asked and answered.


== Pseudo Code ==
== Pseudo Code ==
When programmers are not exactly sure how all the details of a coding project will come together, they write pseudo code, a plain language description of what their code is supposed to accomplish. It is very rare that anyone starting to code a SurveyCTO [[Questionnaire Programming|questionnaire]] knows how all modules will be coded, so writing pseudo code is the best place to start.


Writing pseudo code is a method that all computer scientists use, before starting to write code, when they are not exactly sure how all details of that code will be in the end. It is very rare that anyone starting to code a SurveyCTO questionnaire knows how all modules will be coded, so writing pseudo code is the best place to start.
Starting to write pseudo code is simple. Start by describing at a high level what the code will do (often in bullet points) in plain English. After all functions of the code have been described at a high level, more details are added. As details are added, the language also changes from plain English to something that resembles code more and more. Computer scientists very often write pseudo code with pen and paper or on a white board.


Writing pseudo code is simple to start. Start by describing at a high level what the code will do (often in bullet points) in plain English. After all functions of the code has been described at a high level, more details are added. As details are added, the language also changes from plain English to something that resembles code more and more. Computer scientists very often write pseudo code with pen and paper or on a white board.
The next subsection gives a quick introduction on how to write pseudo code in relation to a SurveyCTO '''questionnaire'''.
 
The next subsection gives a quick introduction on how to write pseudo code in relation to a SurveyCTO questionnaire.


=== Listing the modules ===
=== Listing the modules ===
 
The highest level a [[Questionnaire Programming|questionnaire]] can be described on are the modules. Therefore, start your '''questionnaire''' by writing a list of them in the order that you want the modules to be asked:
The highest level a questionnaire can be described on are the modules. So start by writing a list of the modules in your questionnaire. Write them in the order that you want them to be asked:


{| class="wikitable"  style="margin-left:  0px; margin-right: auto; border: none;"
{| class="wikitable"  style="margin-left:  0px; margin-right: auto; border: none;"
Line 33: Line 31:


=== Adding groups and repeat groups ===
=== Adding groups and repeat groups ===
The second highest level of detail is adding all the groups of questions (groups) and groups of repeated questions ([[SurveyCTO Repeat Groups and Rosters|repeat groups]]) that will control the flow of the [[Questionnaire Programming|questionnaire]]. Some groups are used only to change the appearance of questions, and those groups do not need to be listed here. The groups that should be mentioned in pseudocode are the ones that are relevant when a condition or skip pattern is met. Take a look at the examples of the groups "Savings" and "Access to Maternal Health" below for a reference to groups that use skip patterns.


The second highest level of detail is adding all the groups of questions (groups) and groups of repeated questions (repeat groups) that will control the flow of the questionnaire. Some groups are used only to change the appearance of questions, and those groups do not need to be listed here. The groups that should be mentioned in pseudocode are the ones that are relevant when a condition or skip pattern is met. Take a look at the examples of the groups Savings and Access to Maternal Health below for a reference to groups that use skip patterns.
When you add a '''repeat group''' or a group that uses a skip pattern, also add what will determine the number of repeats or what will cause the group to be skipped. We are still writing pseudo code so it is perfectly fine to write in plain English, although some code is also good.
 
When you add a repeat group or a group that uses a skip pattern, also add what will determine the number of repeats or what will cause the group to be skipped. We are still writing pseudo code so it is perfectly fine to write in plain English, although some code is also good.


{| class="wikitable"  style="margin-left:  0px; margin-right: auto; border: none;"
{| class="wikitable"  style="margin-left:  0px; margin-right: auto; border: none;"
Line 52: Line 49:
|}
|}


If you look at the pseudo code above, you will see that the number of times the Employment module is repeated depends on a variable that is asked in the Household Roster module. This is the purpose of writing pseudo code. We want to catch as early as possible anything that will cause a problem while we are still working on a high level, before we have spend additional time working on details. If we first spend a week working out the details of every module and figure out later that we need to change the structure of the questionnaire, then there is a risk that we have to re-do our work.
If you look at the pseudo code above, you will see that the number of times the "Employment" module is repeated depends on a '''variable''' that is asked in the "Household Roster" module. This is the purpose of writing pseudo code. We want to catch as early as possible anything that will cause a problem while we are still working on a high level, before spending additional time working on details. If we first spend a week working out the details of every module and figure out later that we need to change the structure of the '''questionnaire''', then there is a risk that we have to re-do our work.


The final pseudo code for these modules should contain the correct order:
The final pseudo code for these modules should contain the correct order:
Line 71: Line 68:


=== Adding more specific references ===
=== Adding more specific references ===
 
From here on, it is an iterative process of adding a few details at a time. Focus on adding [[SurveyCTO Repeat Groups and Rosters|repeat groups]], groups with skip patterns and any '''variables''' that they reference. You should also add references such as being able to include labels from previous modules. For example, in the pseudo code below we most likely want to be able to reference the names listed in the "Household Roster" in the "Employment" section. We might also want to be able to have all women aged 16-45 listed as answer options in any question in the "Access to Maternity Health Care" section. Repeat this step and add a bit more detail in each repetition, and each time make sure that the structure still does what you intend to do.  
From here on, it is a iterative process of adding a few details at the time. Focus on adding repeat groups, groups with skip patterns and any variables that they reference. Also add references such as being able to include labels from previous modules. For example, in the pseudo code below we most likely want to be able to reference the names listed in the household roster in the employment section. We might also want to be able to have all women aged 16-45 listed as answer options in any question in the Access to Maternity Health Care section. Repeat this step and add a bit more detail in each repetition, and each time make sure that the structure still suits what you intend to do.  


{| class="wikitable"  style="margin-left: 0px; margin-right: auto; border: none;"
{| class="wikitable"  style="margin-left: 0px; margin-right: auto; border: none;"
Line 98: Line 94:


=== When is Pseudo Code Complete? ===
=== When is Pseudo Code Complete? ===
There is no clear rule for when pseudo code is completed, but the more experienced a coder is, the more detail is included in their pseudo code. For a beginner, it might be difficult to predict what added detail will require an update to the structure. It is possible to take a break in pseudo coding and code up the structure that you have in the pseudo code already. Do some experimentation to test logic that you are unsure of, and then return to your pseudo code.


There is no clear rule for when a pseudo code is completed, but the more experienced a coder gets the more detail is included in their pseudo code. For a beginner it might be difficult to predict what added detail will require an update to the structure the more detailed it gets. It is possible to take a break in pseudo coding and code up the structure that you have in the pseudo code already. Do some experimentation to test logic that you are unsure of, and then return back to working on your pseudo code.
The point of pseudo code is to get an idea of the details that are needed for the flow of the [[Questionnaire Programming|questionnaire]]. When you have your structure (sometimes called "'''questionnaire''' skeleton") then you can move on to implementing your pseudo code in actual [[SurveyCTO Choice Lists|SurveyCTO]] [[Repeat Groups and Rosters in SurveyCTO|code]].


The point of pseudo code is to get an idea of the details that are needed for the flow of the questionnaire. When you have your structure (sometimes called "questionnaire skeleton") then you can move on to implementing your pseudo code in actual SurveyCTO code.
The more time you spend on pseudo coding, the easier the actual coding will be. Spend at least a few days on this. Write your first pseudo code and then sleep on it and review it with fresh eyes the next day. Even better, when you have a final draft, discuss your pseudo code with someone else working on the project. Asking someone to proofread your pseudo code is just as useful as asking someone to proofread your text documents.
 
The more time you spend on pseudo coding the easier the actual coding will be. Spend some time for at least a few days on this. Write up your first pseudo code and then sleep on it. Review it with fresh eyes the next day. Or even better, when you have a final draft, discuss your pseudo code with someone else in their project. Proofreading pseudo code is just as useful as asking someone to proofread your text documents.


== Start programming ==
== Start programming ==
If you have created a paper [[Survey Pilot|survey]] and written your pseudo code, then you have reduced the risk of running into time consuming issues while [[SurveyCTO Programming|programming]] as much as possible, and can begin coding. However, there are often unexpected issues that even the most experienced coders run into. In those cases, it might be helpful to go back to the paper '''survey''' or the pseudo code and see if you can come up with a solution that way.


You have now reached the point when you can start coding. If you both have first created a paper survey and written your pseudo code, then you to the best degree possible have reduced the risk of running in to time consuming major issues while programming. However, there are often unexpected issues that even the most experienced coders run in to. In those cases, it might be helpful to go back to the paper survey or the pseudo code and see if you can come up with a solution that way.
There are two ways to '''program''' in SurveyCTO. Most programmers who start by writing pseudo code usually find it easiest to implement it using the Excel method. Since we strongly recommend that everyone conducting large household '''surveys''' use the Excel method, we will assume that through the rest of this article.
 
There are two ways to do [[SurveyCTO Programming|programming in SurveyCTO]]. Most programmers that start by developing pseudo code usually find it easiest to implement pseudo code using the Excel method. Since, we also strongly recommend everyone writing large household surveys to use the Excel method, we will assume that through the rest of this article.


=== Template Forms ===
=== Template Forms ===
When you start coding, never start with an empty excel sheet. SurveyCTO provides [[SurveyCTO Form Settings|form templates]] that have the basic settings already in place which is what most programmers usually start with. Other '''forms''' are also used, but more so as inspiration or code examples that are copied and pasted into your [[Questionnaire Programming|questionnaire]].


When you start coding, never start with an empty excel sheet. SurveyCTO provides template forms that have the basic settings already in place. There is a template with the just the basic settings. That is what most programmers usually start with. The other forms are also used a lot, but maybe more often to used as inspiration or code examples that are copied and pasted into your questionnaire.
To create a new '''form''' using a '''form template''', log in to your [[SurveyCTO Server Management|server]] and go to the design tab. Then click <code>'''start new'''</code> right at the top of the section ''Your Forms''. After giving the '''form''' [[SurveyCTO Form Settings#Questionnaire_name_and_id|a name and an id]], you can select which '''form template''' that you want to use. If this is a '''form''' for which you eventually will collect real data on, remember to [[SurveyCTO Form Settings#Encryption|encrypt]] it.
 
To create a new form using a form template, log in to your server and go to the design tab. Then click <code>'''start new'''</code> right at the top of the section ''Your Forms''. After giving the form [[SurveyCTO Form Settings#Questionnaire_name_and_id|a name and an id]] (remember to be plan ahead when creating the id), you can select which form template that you want to use. If this is a form for which you eventually will collect real data on, remember to [[SurveyCTO Form Settings#Encryption|encrypt]] your form.


=== What to program first? ===
=== What to program first? ===
When you [[SurveyCTO Programmaning|program in SurveyCTO]], do not start with the first question and program your way to the last question. You always want to code from a high level down to the details. This is what you have prepared for in your pseudo code, but even if you don't have psuedo code, this is still the best approach.


When you program, do not start with the first question and program your way to the last question. You always want to code from high level to small detail. This is what you have prepared for in your pseudo code, but even if you do not have pseudo code, you should still start with high level and work your self to details.
Start by creating the [[Repeat Groups and Rosters in SurveyCTO |repeat groups]] and skip groups that you have listed in your pseudo code. Then create the '''variables''' that the repeat counts and skip patterns depend on. Make sure that there is at least one '''variable''' in each '''repeat group''' and skip group. After that is completed, upload your [[Survey Pilot|survey]] and test your code thoroughly for the first time. Even if you have spent a lot of time on your pseudo code, you still want to test that all your assumed programming logic works as intended before you spend time working on details.
 
Start with creating the repeat groups and skip groups that you have listed in your pseudo code. Then create the variables that the repeat counts and skip patterns depend on. Then make sure that there are at least one variable in each repeat group and skip group. After that is completed, upload your survey and test you code thoroughly for the first time. Even if you have spent a lot of time on your pseudo code, you still want to test that all your assumed programming logic works as intended before you spend time working on details.


=== Filling in the details ===
=== Filling in the details ===
 
After you have tested your structure, you can start to fill in the details module by module. Start with the module that includes the most complex task you want to execute, but if that module depends on any other module, then you must code the "parent module" first. In each module, keep approaching your code in the same way as before, i.e. from a high level to the details.
When you have tested your structure, then you can start to fill in the details module by module. Start with the module that includes the most complex thing you want to implement, but if that module depends on any other module, then you must do that module first. In each module, keep approaching your code in the same way as before, i.e. from high level to detail.


== Back to parent topic ==
== Back to parent topic ==
Line 132: Line 123:


== Additional Resources ==
== Additional Resources ==
Please add here any articles related to this topic, with a brief description and link
*DIME Analytics (World Bank), [https://osf.io/8e7bj Introduction to SurveyCTO]
*DIME Analytics (World Bank), [https://osf.io/2nepd Advanced SurveyCTO Programming]


[[ Category: Questionnaire Programming ]]
[[ Category: Questionnaire Programming ]]

Latest revision as of 19:07, 2 August 2023

While it is possible to code a SurveyCTO questionnaire in many different ways, the time it will take and the quality of the final product depends on how the work is planned. This article outlines an approach common among experienced SurveyCTO Programmers at DIME.

Read First

  • Do not start from the beginning of the survey; instead, work your way through one, question at a time. The former approach takes longer and usually results in less well structured questionnaires that are much more error prone.

Paper First

All surveys should start with creating a paper questionnaire. When there is no time to create and pilot the paper version of the questionnaire, it should still be created in Word or Excel before starting to code the survey in SurveyCTO. Skipping this step almost always leads to issues with data quality.

Creating a paper questionnaire before starting to code allows the project team to focus on what questions to ask and how to ask them. The questionnaire coding can be implemented in any format, display, and functionality decided on in the paper questionnaire. By focusing on content first and programming afterwards, data quality is a higher priority than if the questionnaire is set up in a way which is technically convenient to program. If the SurveyCTO code is implemented before or in parallel to developing the content, then the code always takes focus away from the questionnaire and negatively impacts the way a question is asked and answered.

Pseudo Code

When programmers are not exactly sure how all the details of a coding project will come together, they write pseudo code, a plain language description of what their code is supposed to accomplish. It is very rare that anyone starting to code a SurveyCTO questionnaire knows how all modules will be coded, so writing pseudo code is the best place to start.

Starting to write pseudo code is simple. Start by describing at a high level what the code will do (often in bullet points) in plain English. After all functions of the code have been described at a high level, more details are added. As details are added, the language also changes from plain English to something that resembles code more and more. Computer scientists very often write pseudo code with pen and paper or on a white board.

The next subsection gives a quick introduction on how to write pseudo code in relation to a SurveyCTO questionnaire.

Listing the modules

The highest level a questionnaire can be described on are the modules. Therefore, start your questionnaire by writing a list of them in the order that you want the modules to be asked:

1. Pseudo Code
  • Employment
  • Household Roster
  • Savings
  • Access to Maternal Health

Adding groups and repeat groups

The second highest level of detail is adding all the groups of questions (groups) and groups of repeated questions (repeat groups) that will control the flow of the questionnaire. Some groups are used only to change the appearance of questions, and those groups do not need to be listed here. The groups that should be mentioned in pseudocode are the ones that are relevant when a condition or skip pattern is met. Take a look at the examples of the groups "Savings" and "Access to Maternal Health" below for a reference to groups that use skip patterns.

When you add a repeat group or a group that uses a skip pattern, also add what will determine the number of repeats or what will cause the group to be skipped. We are still writing pseudo code so it is perfectly fine to write in plain English, although some code is also good.

2.1 Pseudo Code
  • Employment
    • Repeat. One repeat for each household member over the age of 15
  • Household Roster
    • Repeat. Ask how many people are in the household then repeat over that number to ask about name, age, etc.
  • Savings
    • Group with skip pattern. Ask if the household has savings. If so, ask savings questions
  • Access to Maternal Health
    • Group with skip pattern. If there is at least one female between the age of 16 and 45 in the household, then ask this module to the household head.

If you look at the pseudo code above, you will see that the number of times the "Employment" module is repeated depends on a variable that is asked in the "Household Roster" module. This is the purpose of writing pseudo code. We want to catch as early as possible anything that will cause a problem while we are still working on a high level, before spending additional time working on details. If we first spend a week working out the details of every module and figure out later that we need to change the structure of the questionnaire, then there is a risk that we have to re-do our work.

The final pseudo code for these modules should contain the correct order:

2.2 Pseudo Code
  • Household Roster
    • Repeat. Ask how many people are in the household and then repeat over that number to ask about name, age, etc.
  • Employment
    • Repeat. One repeat for each household member over the age of 15
  • Savings
    • Group with skip pattern. Ask if the household has savings. If so, ask savings questions
  • Access to Maternal Health
    • Group with skip pattern. If there is at least one female between the age of 16 and 45 in the household, then ask this module to the household head.

Adding more specific references

From here on, it is an iterative process of adding a few details at a time. Focus on adding repeat groups, groups with skip patterns and any variables that they reference. You should also add references such as being able to include labels from previous modules. For example, in the pseudo code below we most likely want to be able to reference the names listed in the "Household Roster" in the "Employment" section. We might also want to be able to have all women aged 16-45 listed as answer options in any question in the "Access to Maternity Health Care" section. Repeat this step and add a bit more detail in each repetition, and each time make sure that the structure still does what you intend to do.

3. Pseudo Code
  • Household Roster
    • hh_num_members: Ask how many people in the household
    • Repeat over hh_num_members
      • hhm_name: ask about household members name
      • hhm_age: ask about household members age
      • hhm_sex: ask about household members gender
  • Employment
    • Repeat over hh_num_members
      • Inside repeat, have a group restricted to (hhm_age > 15)
        • inside group, ask about employment
  • Savings
    • savings: Do you have any savings?
    • Group. skip if savings = no
      • savings questions
  • Access to Maternal Health
    • Have a calculate that is yes if any household member is (hhm = female and hhm_age > 15 and hhm_age < 45)
    • Group. Skip this group if the calculated variable above is not yes

When is Pseudo Code Complete?

There is no clear rule for when pseudo code is completed, but the more experienced a coder is, the more detail is included in their pseudo code. For a beginner, it might be difficult to predict what added detail will require an update to the structure. It is possible to take a break in pseudo coding and code up the structure that you have in the pseudo code already. Do some experimentation to test logic that you are unsure of, and then return to your pseudo code.

The point of pseudo code is to get an idea of the details that are needed for the flow of the questionnaire. When you have your structure (sometimes called "questionnaire skeleton") then you can move on to implementing your pseudo code in actual SurveyCTO code.

The more time you spend on pseudo coding, the easier the actual coding will be. Spend at least a few days on this. Write your first pseudo code and then sleep on it and review it with fresh eyes the next day. Even better, when you have a final draft, discuss your pseudo code with someone else working on the project. Asking someone to proofread your pseudo code is just as useful as asking someone to proofread your text documents.

Start programming

If you have created a paper survey and written your pseudo code, then you have reduced the risk of running into time consuming issues while programming as much as possible, and can begin coding. However, there are often unexpected issues that even the most experienced coders run into. In those cases, it might be helpful to go back to the paper survey or the pseudo code and see if you can come up with a solution that way.

There are two ways to program in SurveyCTO. Most programmers who start by writing pseudo code usually find it easiest to implement it using the Excel method. Since we strongly recommend that everyone conducting large household surveys use the Excel method, we will assume that through the rest of this article.

Template Forms

When you start coding, never start with an empty excel sheet. SurveyCTO provides form templates that have the basic settings already in place which is what most programmers usually start with. Other forms are also used, but more so as inspiration or code examples that are copied and pasted into your questionnaire.

To create a new form using a form template, log in to your server and go to the design tab. Then click start new right at the top of the section Your Forms. After giving the form a name and an id, you can select which form template that you want to use. If this is a form for which you eventually will collect real data on, remember to encrypt it.

What to program first?

When you program in SurveyCTO, do not start with the first question and program your way to the last question. You always want to code from a high level down to the details. This is what you have prepared for in your pseudo code, but even if you don't have psuedo code, this is still the best approach.

Start by creating the repeat groups and skip groups that you have listed in your pseudo code. Then create the variables that the repeat counts and skip patterns depend on. Make sure that there is at least one variable in each repeat group and skip group. After that is completed, upload your survey and test your code thoroughly for the first time. Even if you have spent a lot of time on your pseudo code, you still want to test that all your assumed programming logic works as intended before you spend time working on details.

Filling in the details

After you have tested your structure, you can start to fill in the details module by module. Start with the module that includes the most complex task you want to execute, but if that module depends on any other module, then you must code the "parent module" first. In each module, keep approaching your code in the same way as before, i.e. from a high level to the details.

Back to parent topic

This article is a part of the topic Questionnaire Programming.

Additional Resources