Difference between revisions of "SurveyCTO Programming Work Flow"
| (12 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
<onlyinclude>While it is possible to code a SurveyCTO [[Questionnaire | <onlyinclude>While it is possible to code a SurveyCTO [[Questionnaire Programming|questionnaire]] in many different ways, the time it will take and the quality of the final product depends on how the work is planned. This article outlines an approach common among experienced [[SurveyCTO Programming|SurveyCTO Programmers]] at DIME. </onlyinclude> | ||
== Read First == | == Read First == | ||
*Do not start from the beginning of the [[Survey Pilot|survey]]; instead, work your way through one question at a time. The former approach takes longer and usually results in less well structured [[Questionnaire | *Do not start from the beginning of the [[Survey Pilot|survey]]; instead, work your way through one, question at a time. The former approach takes longer and usually results in less well structured [[Questionnaire Programming|questionnaires]] that are much more error prone. | ||
== Paper First == | == Paper First == | ||
All [[Survey Pilot|surveys]] should | All [[Survey Pilot|surveys]] should start with creating a paper [[Questionnaire Design|questionnaire]]. When there is no time to create and pilot the paper version of the '''questionnaire''', it should still be created in Word or Excel before starting to code the '''survey''' in SurveyCTO. Skipping this step almost always leads to issues with data quality. | ||
Creating a paper '''questionnaire''' before starting to code allows the project team to focus on what questions to ask and how to ask them. The [[Questionnaire Programming|questionnaire coding]] can be implemented in any format, display and functionality decided on in the paper '''questionnaire'''. By focusing on content first and programming afterwards, data quality is a higher priority than | Creating a paper '''questionnaire''' before starting to code allows the project team to focus on what questions to ask and how to ask them. The [[Questionnaire Programming|questionnaire coding]] can be implemented in any format, display, and functionality decided on in the paper '''questionnaire'''. By focusing on content first and programming afterwards, data quality is a higher priority than if the '''questionnaire''' is set up in a way which is technically convenient to program. If the SurveyCTO code is implemented before or in parallel to developing the content, then the code always takes focus away from the '''questionnaire''' and negatively impacts the way a question is asked and answered. | ||
== Pseudo Code == | == Pseudo Code == | ||
When programmers are not exactly sure how all the details of a coding project will come together, they write pseudo code, a plain language description of what their code is supposed to accomplish. It is very rare that anyone starting to code a SurveyCTO [[Questionnaire Programming|questionnaire]] knows how all modules will be coded, so writing pseudo code is the best place to start. | |||
Starting to write pseudo code is simple. Start by describing at a high level what the code will do (often in bullet points) in plain English. After all functions of the code have been described at a high level, more details are added. As details are added, the language also changes from plain English to something that resembles code more and more. Computer scientists very often write pseudo code with pen and paper or on a white board. | |||
The next subsection gives a quick introduction on how to write pseudo code in relation to a SurveyCTO '''questionnaire'''. | |||
The next subsection gives a quick introduction on how to write pseudo code in relation to a ''' | |||
=== Listing the modules === | === Listing the modules === | ||
The highest level a [[Questionnaire | The highest level a [[Questionnaire Programming|questionnaire]] can be described on are the modules. Therefore, start your '''questionnaire''' by writing a list of them in the order that you want the modules to be asked: | ||
{| class="wikitable" style="margin-left: 0px; margin-right: auto; border: none;" | {| class="wikitable" style="margin-left: 0px; margin-right: auto; border: none;" | ||
| Line 32: | Line 31: | ||
=== Adding groups and repeat groups === | === Adding groups and repeat groups === | ||
The second highest level of detail is adding all the groups of questions (groups) and groups of repeated questions ([[SurveyCTO Repeat Groups and Rosters|repeat groups]]) that will control the flow of the [[Questionnaire Programming|questionnaire]]. Some groups are used only to change the appearance of questions, and those groups do not need to be listed here. The groups that should be mentioned in pseudocode are the ones that are relevant when a condition or skip pattern is met. Take a look at the examples of the groups "Savings" and "Access to Maternal Health" below for a reference to groups that use skip patterns. | |||
The second highest level of detail is adding all the groups of questions (groups) and groups of repeated questions ([[SurveyCTO Repeat Groups and Rosters|repeat groups]]) that will control the flow of the [[Questionnaire | |||
When you add a '''repeat group''' or a group that uses a skip pattern, also add what will determine the number of repeats or what will cause the group to be skipped. We are still writing pseudo code so it is perfectly fine to write in plain English, although some code is also good. | When you add a '''repeat group''' or a group that uses a skip pattern, also add what will determine the number of repeats or what will cause the group to be skipped. We are still writing pseudo code so it is perfectly fine to write in plain English, although some code is also good. | ||
| Line 51: | Line 49: | ||
|} | |} | ||
If you look at the pseudo code above, you will see that the number of times the "Employment" module is repeated depends on a '''variable''' that is asked in the "Household" | If you look at the pseudo code above, you will see that the number of times the "Employment" module is repeated depends on a '''variable''' that is asked in the "Household Roster" module. This is the purpose of writing pseudo code. We want to catch as early as possible anything that will cause a problem while we are still working on a high level, before spending additional time working on details. If we first spend a week working out the details of every module and figure out later that we need to change the structure of the '''questionnaire''', then there is a risk that we have to re-do our work. | ||
The final pseudo code for these modules should contain the correct order: | The final pseudo code for these modules should contain the correct order: | ||
| Line 70: | Line 68: | ||
=== Adding more specific references === | === Adding more specific references === | ||
From here on, it is an iterative process of adding a few details at a time. Focus on adding [[SurveyCTO Repeat Groups and Rosters|repeat groups]], groups with skip patterns and any '''variables''' that they reference. You should also add references such as being able to include labels from previous modules. For example, in the pseudo code below we most likely want to be able to reference the names listed in the "Household Roster" in the "Employment" section. We might also want to be able to have all women aged 16-45 listed as answer options in any question in the "Access to Maternity Health Care" section. Repeat this step and add a bit more detail in each repetition, and each time make sure that the structure still does what you intend to do. | From here on, it is an iterative process of adding a few details at a time. Focus on adding [[SurveyCTO Repeat Groups and Rosters|repeat groups]], groups with skip patterns and any '''variables''' that they reference. You should also add references such as being able to include labels from previous modules. For example, in the pseudo code below we most likely want to be able to reference the names listed in the "Household Roster" in the "Employment" section. We might also want to be able to have all women aged 16-45 listed as answer options in any question in the "Access to Maternity Health Care" section. Repeat this step and add a bit more detail in each repetition, and each time make sure that the structure still does what you intend to do. | ||
| Line 97: | Line 94: | ||
=== When is Pseudo Code Complete? === | === When is Pseudo Code Complete? === | ||
There is no clear rule for when pseudo code is completed, but the more experienced a coder is, the more detail is included in their pseudo code. For a beginner, it might be difficult to predict what added detail will require an update to the structure. It is possible to take a break in pseudo coding and code up the structure that you have in the pseudo code already. Do some experimentation to test logic that you are unsure of, and then return to your pseudo code. | |||
The point of pseudo code is to get an idea of the details that are needed for the flow of the [[Questionnaire Programming|questionnaire]]. When you have your structure (sometimes called "'''questionnaire''' skeleton") then you can move on to implementing your pseudo code in actual [[SurveyCTO Choice Lists|SurveyCTO]] [[Repeat Groups and Rosters in SurveyCTO|code]]. | |||
The point of pseudo code is to get an idea of the details that are needed for the flow of the [[Questionnaire | |||
The more time you spend on pseudo coding, the easier the actual coding will be. Spend at least a few days on this. Write your first pseudo code and then sleep on it and review it with fresh eyes the next day. Even better, when you have a final draft, discuss your pseudo code with someone else working on the project. | The more time you spend on pseudo coding, the easier the actual coding will be. Spend at least a few days on this. Write your first pseudo code and then sleep on it and review it with fresh eyes the next day. Even better, when you have a final draft, discuss your pseudo code with someone else working on the project. Asking someone to proofread your pseudo code is just as useful as asking someone to proofread your text documents. | ||
== Start programming == | == Start programming == | ||
If you have created a paper [[Survey Pilot|survey]] and written your pseudo code, then you have reduced the risk of running into time consuming issues while [[SurveyCTO Programming|programming]] as much as possible and can begin coding. However, there are often unexpected issues that even the most experienced coders run into. In those cases, it might be helpful to go back to the paper '''survey''' or the pseudo code and see if you can come up with a solution that way. | If you have created a paper [[Survey Pilot|survey]] and written your pseudo code, then you have reduced the risk of running into time consuming issues while [[SurveyCTO Programming|programming]] as much as possible, and can begin coding. However, there are often unexpected issues that even the most experienced coders run into. In those cases, it might be helpful to go back to the paper '''survey''' or the pseudo code and see if you can come up with a solution that way. | ||
There are two ways to '''program | There are two ways to '''program''' in SurveyCTO. Most programmers who start by writing pseudo code usually find it easiest to implement it using the Excel method. Since we strongly recommend that everyone conducting large household '''surveys''' use the Excel method, we will assume that through the rest of this article. | ||
=== Template Forms === | === Template Forms === | ||
When you start coding, never start with an empty excel sheet. | When you start coding, never start with an empty excel sheet. SurveyCTO provides [[SurveyCTO Form Settings|form templates]] that have the basic settings already in place which is what most programmers usually start with. Other '''forms''' are also used, but more so as inspiration or code examples that are copied and pasted into your [[Questionnaire Programming|questionnaire]]. | ||
To create a new '''form''' using a '''form template''', log in to your [[SurveyCTO Server Management|server]] and go to the design tab. Then click <code>'''start new'''</code> right at the top of the section ''Your Forms''. After giving the '''form''' [[SurveyCTO Form Settings#Questionnaire_name_and_id|a name and an id]], you can select which '''form template''' that you want to use. If this is a '''form''' for which you eventually will collect real data on, remember to [[SurveyCTO Form Settings#Encryption|encrypt]] | To create a new '''form''' using a '''form template''', log in to your [[SurveyCTO Server Management|server]] and go to the design tab. Then click <code>'''start new'''</code> right at the top of the section ''Your Forms''. After giving the '''form''' [[SurveyCTO Form Settings#Questionnaire_name_and_id|a name and an id]], you can select which '''form template''' that you want to use. If this is a '''form''' for which you eventually will collect real data on, remember to [[SurveyCTO Form Settings#Encryption|encrypt]] it. | ||
=== What to program first? === | === What to program first? === | ||
When you [[SurveyCTO Programmaning|program in SurveyCTO]], do not start with the first question and | When you [[SurveyCTO Programmaning|program in SurveyCTO]], do not start with the first question and program your way to the last question. You always want to code from a high level down to the details. This is what you have prepared for in your pseudo code, but even if you don't have psuedo code, this is still the best approach. | ||
Start | Start by creating the [[Repeat Groups and Rosters in SurveyCTO |repeat groups]] and skip groups that you have listed in your pseudo code. Then create the '''variables''' that the repeat counts and skip patterns depend on. Make sure that there is at least one '''variable''' in each '''repeat group''' and skip group. After that is completed, upload your [[Survey Pilot|survey]] and test your code thoroughly for the first time. Even if you have spent a lot of time on your pseudo code, you still want to test that all your assumed programming logic works as intended before you spend time working on details. | ||
=== Filling in the details === | === Filling in the details === | ||
| Line 127: | Line 123: | ||
== Additional Resources == | == Additional Resources == | ||
*DIME Analytics (World Bank), [https://osf.io/8e7bj Introduction to SurveyCTO] | |||
*DIME Analytics (World Bank), [https://osf.io/2nepd Advanced SurveyCTO Programming] | |||
[[ Category: Questionnaire Programming ]] | [[ Category: Questionnaire Programming ]] | ||
Latest revision as of 19:07, 2 August 2023
While it is possible to code a SurveyCTO questionnaire in many different ways, the time it will take and the quality of the final product depends on how the work is planned. This article outlines an approach common among experienced SurveyCTO Programmers at DIME.
Read First
- Do not start from the beginning of the survey; instead, work your way through one, question at a time. The former approach takes longer and usually results in less well structured questionnaires that are much more error prone.
Paper First
All surveys should start with creating a paper questionnaire. When there is no time to create and pilot the paper version of the questionnaire, it should still be created in Word or Excel before starting to code the survey in SurveyCTO. Skipping this step almost always leads to issues with data quality.
Creating a paper questionnaire before starting to code allows the project team to focus on what questions to ask and how to ask them. The questionnaire coding can be implemented in any format, display, and functionality decided on in the paper questionnaire. By focusing on content first and programming afterwards, data quality is a higher priority than if the questionnaire is set up in a way which is technically convenient to program. If the SurveyCTO code is implemented before or in parallel to developing the content, then the code always takes focus away from the questionnaire and negatively impacts the way a question is asked and answered.
Pseudo Code
When programmers are not exactly sure how all the details of a coding project will come together, they write pseudo code, a plain language description of what their code is supposed to accomplish. It is very rare that anyone starting to code a SurveyCTO questionnaire knows how all modules will be coded, so writing pseudo code is the best place to start.
Starting to write pseudo code is simple. Start by describing at a high level what the code will do (often in bullet points) in plain English. After all functions of the code have been described at a high level, more details are added. As details are added, the language also changes from plain English to something that resembles code more and more. Computer scientists very often write pseudo code with pen and paper or on a white board.
The next subsection gives a quick introduction on how to write pseudo code in relation to a SurveyCTO questionnaire.
Listing the modules
The highest level a questionnaire can be described on are the modules. Therefore, start your questionnaire by writing a list of them in the order that you want the modules to be asked:
| 1. Pseudo Code |
|---|
|
Adding groups and repeat groups
The second highest level of detail is adding all the groups of questions (groups) and groups of repeated questions (repeat groups) that will control the flow of the questionnaire. Some groups are used only to change the appearance of questions, and those groups do not need to be listed here. The groups that should be mentioned in pseudocode are the ones that are relevant when a condition or skip pattern is met. Take a look at the examples of the groups "Savings" and "Access to Maternal Health" below for a reference to groups that use skip patterns.
When you add a repeat group or a group that uses a skip pattern, also add what will determine the number of repeats or what will cause the group to be skipped. We are still writing pseudo code so it is perfectly fine to write in plain English, although some code is also good.
| 2.1 Pseudo Code |
|---|
|
If you look at the pseudo code above, you will see that the number of times the "Employment" module is repeated depends on a variable that is asked in the "Household Roster" module. This is the purpose of writing pseudo code. We want to catch as early as possible anything that will cause a problem while we are still working on a high level, before spending additional time working on details. If we first spend a week working out the details of every module and figure out later that we need to change the structure of the questionnaire, then there is a risk that we have to re-do our work.
The final pseudo code for these modules should contain the correct order:
| 2.2 Pseudo Code |
|---|
|
Adding more specific references
From here on, it is an iterative process of adding a few details at a time. Focus on adding repeat groups, groups with skip patterns and any variables that they reference. You should also add references such as being able to include labels from previous modules. For example, in the pseudo code below we most likely want to be able to reference the names listed in the "Household Roster" in the "Employment" section. We might also want to be able to have all women aged 16-45 listed as answer options in any question in the "Access to Maternity Health Care" section. Repeat this step and add a bit more detail in each repetition, and each time make sure that the structure still does what you intend to do.
| 3. Pseudo Code |
|---|
|
When is Pseudo Code Complete?
There is no clear rule for when pseudo code is completed, but the more experienced a coder is, the more detail is included in their pseudo code. For a beginner, it might be difficult to predict what added detail will require an update to the structure. It is possible to take a break in pseudo coding and code up the structure that you have in the pseudo code already. Do some experimentation to test logic that you are unsure of, and then return to your pseudo code.
The point of pseudo code is to get an idea of the details that are needed for the flow of the questionnaire. When you have your structure (sometimes called "questionnaire skeleton") then you can move on to implementing your pseudo code in actual SurveyCTO code.
The more time you spend on pseudo coding, the easier the actual coding will be. Spend at least a few days on this. Write your first pseudo code and then sleep on it and review it with fresh eyes the next day. Even better, when you have a final draft, discuss your pseudo code with someone else working on the project. Asking someone to proofread your pseudo code is just as useful as asking someone to proofread your text documents.
Start programming
If you have created a paper survey and written your pseudo code, then you have reduced the risk of running into time consuming issues while programming as much as possible, and can begin coding. However, there are often unexpected issues that even the most experienced coders run into. In those cases, it might be helpful to go back to the paper survey or the pseudo code and see if you can come up with a solution that way.
There are two ways to program in SurveyCTO. Most programmers who start by writing pseudo code usually find it easiest to implement it using the Excel method. Since we strongly recommend that everyone conducting large household surveys use the Excel method, we will assume that through the rest of this article.
Template Forms
When you start coding, never start with an empty excel sheet. SurveyCTO provides form templates that have the basic settings already in place which is what most programmers usually start with. Other forms are also used, but more so as inspiration or code examples that are copied and pasted into your questionnaire.
To create a new form using a form template, log in to your server and go to the design tab. Then click start new right at the top of the section Your Forms. After giving the form a name and an id, you can select which form template that you want to use. If this is a form for which you eventually will collect real data on, remember to encrypt it.
What to program first?
When you program in SurveyCTO, do not start with the first question and program your way to the last question. You always want to code from a high level down to the details. This is what you have prepared for in your pseudo code, but even if you don't have psuedo code, this is still the best approach.
Start by creating the repeat groups and skip groups that you have listed in your pseudo code. Then create the variables that the repeat counts and skip patterns depend on. Make sure that there is at least one variable in each repeat group and skip group. After that is completed, upload your survey and test your code thoroughly for the first time. Even if you have spent a lot of time on your pseudo code, you still want to test that all your assumed programming logic works as intended before you spend time working on details.
Filling in the details
After you have tested your structure, you can start to fill in the details module by module. Start with the module that includes the most complex task you want to execute, but if that module depends on any other module, then you must code the "parent module" first. In each module, keep approaching your code in the same way as before, i.e. from a high level to the details.
Back to parent topic
This article is a part of the topic Questionnaire Programming.
Additional Resources
- DIME Analytics (World Bank), Introduction to SurveyCTO
- DIME Analytics (World Bank), Advanced SurveyCTO Programming