Difference between revisions of "Questionnaire Translation"

Jump to: navigation, search
 
(77 intermediate revisions by the same user not shown)
Line 1: Line 1:
<onlyinclude>Careful translation of all survey instruments into the local language(s) is an essential step in conducting a high quality survey. All the hard work put into questionnaire design can be for naught if the translation is poor quality. </onlyinclude>
<onlyinclude>'''Questionnaire translation''' refers to the process of translating [[Field Surveys|survey instruments]] (or questionnaires) into all local languages spoken in the study area. It is a vital component of a high quality [[Primary Data Collection|survey]]. Incomplete or inaccurate translations can, at best, result in loss in nuance intended by the [[Impact Evaluation Team|impact evaluation team]] (or research team), and at worst, completely change the meaning of a question. It can undo all the work that goes into [[Questionnaire Design|designing]] carefully-worded questionnaires.</onlyinclude>


== Read First ==
== Read First ==
*It is very important to translate all survey instruments into all local languages spoken in the study area, and have agreed-upon written translations for all questions.
* The [[Impact Evaluation Team|research team]] should set aside sufficient time for each of the steps involved in translation- '''forward translation''', '''back translation''' and '''reconciliation''', as well as '''validation by enumerators'''.
*Set aside time for '''back-translation''', and discuss translations in [[Enumerator Training]]. Poor translation can, at best, result in loss in nuance intended by the research team in carefully worded questions, and at worst, can completely change the meaning of a question.  
* Questionnaires often go through multiple stages of translation and back translation. Therefore, '''version control''' is important, as it involves maintaining copies of each version during the translation process, so that no information is lost.  
*However, having numerators translate on the go is '''not recommended''', as consistency and accuracy of translation can vary across enumerators in ways that can influence responses.
* Keep in mind that having enumerators translate on the go is not recommended either, as consistency and accuracy of translation can vary across enumerators.  
*Therefore, '''version control''' is important, particularly when multiple translations are involved.  


''Note: This section assumes that the working language of the research team is different than the local language(s) in the study area.''
''Note: This section assumes that the working language of the research team is different from the local language(s) in the study area.''


== Guidelines ==
== Guidelines ==
The [[Impact Evaluation Team|research team]] must keep complete, '''back-translated''' versions for all questions in the survey instrument, and share these with enumerators during [[Enumerator Training|enumerator training]]. '''Back translation''' refers to the process of translating the local language version of a questionnaire back to the language of the research team. This is important to guarantee consistency and helps avoid '''enumerator effects''', which arise because each enumerator shared a different translation of the same question with different respondents.


===Why translate?===
=== Translation process ===
You should have full, back-checked, translations for all questions in the survey instrument. This is important to guarantee consistency. If enumerators translate on the fly, it is difficult to ensure that each question is asked the same way to each respondent, and increases the likelihood of significant Enumerator Effects.
Lack of proper '''due diligence''' during the translation process can cause delays, confusion and field errors, and can potentially lead to inaccurate [[Primary Data Collection|data collection]]. The '''research team''' must remember that effective or accurate translation takes time. '''Best practices ''' suggest that the '''research team''' should allocate at least two weeks for the initial translation, and at least two more weeks for editing and reviewing the translations, and these should be mentioned in the [[Timeline of a Survey Pilot|survey timeline]].


=== Translation Process ===
Researchers often rely on a [[Piloting Survey Content|content-focused pilot]] to gain valuable feedback on the questionnaire. After finalizing and agreeing upon the [[Questionnaire Design|contents]] of the questionnaire, they can initiate the process of translation which has the following steps:
# '''Forward translation''': This refers to the translation from the language of the research team to the local language(s). This process can take a few days to a few weeks, depending on the complexity of the questions in the survey instrument.
# '''Back translation''': Once researchers receive the first translation, they send only the local language version to a different translator, who translates it back to the language of the research team.
# '''Reconciliation''': This refers to the process of comparing the original questionnaire to the '''back translated''' questionnaire. In this stage, researchers should highlight any discrepancies, and categorize them as either minor (like wording issues), or significant (like changes in the meaning of a question).
In practice, the research team can organize a meeting with the two translators to discuss all discrepancies and agree on a final translation. During this meeting, someone from the research team should keep track of all questions where significant content concerns were noted. This will allow the research team to discuss these with local counterparts in the next stage to ensure that the final wording reflects the original intent.


Poor-quality translation will cause delays, confusion, field errors, and potentially inaccurate data. Good translation takes time - budget at least two weeks, and expect to made edits and corrections for significantly longer.  
For example, consider a questionnaire to be '''fielded''' (shared with respondents) in Malawi to a Chichewa-speaking population. The research team finalizes an English version of the questionnaire. They then hire a translator to  '''forward-translate''' from English to Chichewa. A second translator then  '''back-translates''' from Chichewa to English. The research team then compares the original and the '''back-translated''' versions, and discusses discrepancies in the two versions to '''reconcile''' them.


Once the content of the questionnaire is finalized (after [[Survey Pilot|content-focused pilot]]), follow these steps for translation:  
=== Translators ===
# '''Forward translation''' : from the language of the research team to the local language(s). This process that could take a few days to a few weeks, depending entirely on the complexity of the survey instrument.  
Accurate translation requires significant time and skill. The following are the global '''best practices''' that the [[Impact Evaluation Team|research team]] should follow for hiring translators:
# '''Back translation''': when you receive the first translation, send the local language version only to a second translator, and have it translated back to the language the questionnaire was originally written in.  
# '''Sector-specific knowledge.''' The '''research team''' should only work with a professional translator who has sector-specific knowledge and experience in translating surveys.  
# '''Reconciliation''': compare the original questionnaire to the back-translated questionnaire. Highlight any discrepancies. It is a good idea to categorize these as either minor wording issues, or significant content concerns. Organize a meeting with the two translators, to discuss all discrepancies and agree on a final translation. Keep track of all questions where significant content concerns were noted, and discuss with local counterparts to ensure the final wording reflects the original intent.  
# '''Mother tongue.''' The mother tongue of the forward translator should be the local language. Similarly, the mother tongue of the back translator should be the language of the research team.  
# '''Technical terminology.''' Researchers should cross-check all technical terminology with local counterparts who are well-versed in the subject area of the project. For example, for agricultural surveys, this would involve double-checking translations of specific technologies with local agricultural officers and other experts.  


''Example'': a questionnaire to be fielded in Malawi to a Chichewa-speaking population. The research team finalizes an English version of the questionnaire. It is then '''forward translated''' from English to Chichewa. The Chichewa version is then provided to a second translator for '''back translation''' from Chichewa to English. The two English versions are compared, and the Chichewa is discussed and refined, to '''reconcile''' all discrepancies between the back translation and the original version.
Sometimes the '''research team''' might be tempted to assign the job of translation to people who are not professional translators. Given below are the alternatives along with  the reasons for not hiring them as translators:
* '''Survey firm manager'''. The manager of the [[Survey Firm|survey firm]] has a range of responsibilities, and they oversee the overall process of conducting a [[Field Surveys|survey]]. They would neither have sufficient time, nor the sector-specific knowledge required for efficient translation. However, it would be a good idea to have a person within the research team who can coordinate with the professional translator.
* '''Government officials from relevant departments'''. This can include someone like the local agricultural officer we mentioned earlier. While they may have the relevant sector-specific knowledge, they may not be able to give enough time to the task of translation, unless they are released from their primary duties. These local officials are better suited for the purpose of cross-checking translations of technical words, and ensure that the context is not lost in translation.
* '''Local consultant without translation experience'''. Translation is a challenging process, and we have already discussed the harms of an incomplete or inaccurate translation. Being fluent in the relevant languages is not a sufficient requirement, because the task of translation requires written fluency as well.


=== Who should translate? ===
Finally, the '''research team''' must ensure that the '''back translator''' has no professional or personal relation to the '''first translator'''. This is important because  power dynamics between the two translators could affect decisions made during the '''reconciliation''' stage. For example, if one translator is an employee of the other translator or significantly lower in government hierarchy, it might result in a compromise which will affect the quality of the translation.
Good translation requires significant time and skill. Best practice is to work with a professional translator with sector-specific knowledge, and experience translating surveys. Double-check all technical terminology with local counterparts well-versed in the area of the project (for example, for agricultural surveys, double-check translations of specific technologies of interest with agricultural extension officers or other trainers).  


The mother tongue of the forward translator should be the local language. The mother tongue of the back translation should be the language of the research team.
===Version control===


Commonly suggested, but problematic, options for translators:
The [[Impact Evaluation Team|research team]] can expect to make several corrections and refinements to the questionnaire throughout the translation process. This can also involve performing more than one [[Survey Pilot#Stages of a Survey Pilot|data-focused pilot]] and changing the questionnaire that is used in the [[Survey Pilot|survey pilot]].
* the survey firm manager: unlikely to have sufficient time / attention / sectoral knowledge to dedicate to the task
* government counterparts: likely to have the right sectoral knowledge, but unless released from other duties, unusual to be able to devote sufficient time
* local consultant without translation experience: translation is challenging; fluency in the relevant languages is not always sufficient (and do not assume that oral fluency implies written fluency, which is necessary for this translation task)


The back translator should have no relation to the first translator. Beware of power dynamics between the two translators, that could affect decisions made during the reconciliation (e.g. if one translator is an employee of the other, or significantly lower in government hierarchy).
For instance , consider a study that aims to evaluate the impact of providing mid-day meals in government schools in India. The study could end up with low response rates because respondents did not interpret a question correctly. Low response rates can in turn result in [[Sample Size and Power Calculations#Read First|underpowered calculations]]. In this case, the '''research team''' should work with field coordinators and translators to draft a more accurate version that respondents can understand clearly.  


===Version Control===
It is also a good idea to discuss the errors in translation during [[Enumerator Training|enumerator training]] as another level of check on the final translation. Enumerators can then provide feedback based on their own experience, a process called '''validation'''. It is also important to have at least one bilingual staff member to carefully record all corrections that are made throughout the different stage. This last step is also useful for the purpose of [[Data Documentation|data documentation]], which is important to create [[Reproducible Research|reproducible research]].


Expect to make lots of corrections and refinements to translations through the [[Survey Pilot|data-focused pilot]] and [[Enumerator Training]]. It is a good idea to discuss all questions with discrepancies during [[Enumerator Training]] as a triple-check of the final translation. Throughout this process, it is essential to have at least one bilingual staff member that can carefully record all translation corrections.  
Editing and refining the translations several times can often lead to '''version control issues'''. To avoid these, the '''research team''' should use the paper version of the questionnaire used for '''enumerator training''' should print the questions in the original language, as well as all local languages, side-by-side. This makes it easier to make a note of all required changes without worrying about inconsistency across different versions.


To avoid version control issues, we recommend that the paper version of the questionnaire used for [[Enumerator Training]] includes the original language and all translations, side-by-side. Notes on all needed corrections or refinements can then be easily incorporated into all languages, without worries of different versions.
=== Oral interviews ===


=== What about cases when oral translation is required, i.e. no written language? ===
For oral interviews, such as [[Computer-Assisted Personal Interviews (CAPI)|computer-assisted personal interviews (CAPI)]], the [[Impact Evaluation Team|research team]] will need accurate oral translations, rather than written translations. The '''research team''' should keep a few extra things in mind in this case.
# '''Extend training.''' Extend the duration of the [[Enumerator Training|enumerator training]] to spend more time on oral translations.
# '''Cross check translations.''' During the training process, researchers can ask more than one person to suggest translations for each question. Everyone can then discuss each suggestion, and agree on a final version as a group.
# '''Pen-and-paper training.''' It is a good practice to provide enumerators with printed copies of the questionnaire. When printing these, the '''research team''' can leave some space on the paper version of the questionnaire. Enumerators can use this space to write the translated versions of each question.
# '''Mock interviews.''' Enumerators should also spend extra time on mock interviews and focus on collecting feedback on the quality of the translation.


[[Enumerator Training]] will need to be extended to spend significant time on translation. During training, have several people suggest translations for each question, then discuss and agree as a group. Leave space on the paper version of the questionnaire for enumerators to write the transliterated versions of the questions. Spend extra time doing mock interviews, with focused feedback on the translation.
== Related Pages ==
 
[[Special:WhatLinksHere/Questionnaire_Translation|Click here for pages that link to this topic]].
== Back to Parent ==
This article is part of the topic [[Questionnaire Design]]


== Additional Resources ==
== Additional Resources ==
*DIME Analytics’ guidelines on [https://github.com/worldbank/DIME-Resources/blob/master/survey-instruments.pdf survey design and pilot]
* DIME Analytics (World Bank), [https://osf.io/hxyc3/ Survey design and pilot]
 
* DIME Analytics (World Bank), [https://osf.io/63uv9/ Survey guidelines]
[[Category: Primary Data Collection]] [[Category: Questionnaire Design]]
[[Category: Research Design]]
[[Category: Primary Data Collection]]

Latest revision as of 14:52, 13 April 2021

Questionnaire translation refers to the process of translating survey instruments (or questionnaires) into all local languages spoken in the study area. It is a vital component of a high quality survey. Incomplete or inaccurate translations can, at best, result in loss in nuance intended by the impact evaluation team (or research team), and at worst, completely change the meaning of a question. It can undo all the work that goes into designing carefully-worded questionnaires.

Read First

  • The research team should set aside sufficient time for each of the steps involved in translation- forward translation, back translation and reconciliation, as well as validation by enumerators.
  • Questionnaires often go through multiple stages of translation and back translation. Therefore, version control is important, as it involves maintaining copies of each version during the translation process, so that no information is lost.
  • Keep in mind that having enumerators translate on the go is not recommended either, as consistency and accuracy of translation can vary across enumerators.

Note: This section assumes that the working language of the research team is different from the local language(s) in the study area.

Guidelines

The research team must keep complete, back-translated versions for all questions in the survey instrument, and share these with enumerators during enumerator training. Back translation refers to the process of translating the local language version of a questionnaire back to the language of the research team. This is important to guarantee consistency and helps avoid enumerator effects, which arise because each enumerator shared a different translation of the same question with different respondents.

Translation process

Lack of proper due diligence during the translation process can cause delays, confusion and field errors, and can potentially lead to inaccurate data collection. The research team must remember that effective or accurate translation takes time. Best practices suggest that the research team should allocate at least two weeks for the initial translation, and at least two more weeks for editing and reviewing the translations, and these should be mentioned in the survey timeline.

Researchers often rely on a content-focused pilot to gain valuable feedback on the questionnaire. After finalizing and agreeing upon the contents of the questionnaire, they can initiate the process of translation which has the following steps:

  1. Forward translation: This refers to the translation from the language of the research team to the local language(s). This process can take a few days to a few weeks, depending on the complexity of the questions in the survey instrument.
  2. Back translation: Once researchers receive the first translation, they send only the local language version to a different translator, who translates it back to the language of the research team.
  3. Reconciliation: This refers to the process of comparing the original questionnaire to the back translated questionnaire. In this stage, researchers should highlight any discrepancies, and categorize them as either minor (like wording issues), or significant (like changes in the meaning of a question).

In practice, the research team can organize a meeting with the two translators to discuss all discrepancies and agree on a final translation. During this meeting, someone from the research team should keep track of all questions where significant content concerns were noted. This will allow the research team to discuss these with local counterparts in the next stage to ensure that the final wording reflects the original intent.

For example, consider a questionnaire to be fielded (shared with respondents) in Malawi to a Chichewa-speaking population. The research team finalizes an English version of the questionnaire. They then hire a translator to forward-translate from English to Chichewa. A second translator then back-translates from Chichewa to English. The research team then compares the original and the back-translated versions, and discusses discrepancies in the two versions to reconcile them.

Translators

Accurate translation requires significant time and skill. The following are the global best practices that the research team should follow for hiring translators:

  1. Sector-specific knowledge. The research team should only work with a professional translator who has sector-specific knowledge and experience in translating surveys.
  2. Mother tongue. The mother tongue of the forward translator should be the local language. Similarly, the mother tongue of the back translator should be the language of the research team.
  3. Technical terminology. Researchers should cross-check all technical terminology with local counterparts who are well-versed in the subject area of the project. For example, for agricultural surveys, this would involve double-checking translations of specific technologies with local agricultural officers and other experts.

Sometimes the research team might be tempted to assign the job of translation to people who are not professional translators. Given below are the alternatives along with the reasons for not hiring them as translators:

  • Survey firm manager. The manager of the survey firm has a range of responsibilities, and they oversee the overall process of conducting a survey. They would neither have sufficient time, nor the sector-specific knowledge required for efficient translation. However, it would be a good idea to have a person within the research team who can coordinate with the professional translator.
  • Government officials from relevant departments. This can include someone like the local agricultural officer we mentioned earlier. While they may have the relevant sector-specific knowledge, they may not be able to give enough time to the task of translation, unless they are released from their primary duties. These local officials are better suited for the purpose of cross-checking translations of technical words, and ensure that the context is not lost in translation.
  • Local consultant without translation experience. Translation is a challenging process, and we have already discussed the harms of an incomplete or inaccurate translation. Being fluent in the relevant languages is not a sufficient requirement, because the task of translation requires written fluency as well.

Finally, the research team must ensure that the back translator has no professional or personal relation to the first translator. This is important because power dynamics between the two translators could affect decisions made during the reconciliation stage. For example, if one translator is an employee of the other translator or significantly lower in government hierarchy, it might result in a compromise which will affect the quality of the translation.

Version control

The research team can expect to make several corrections and refinements to the questionnaire throughout the translation process. This can also involve performing more than one data-focused pilot and changing the questionnaire that is used in the survey pilot.

For instance , consider a study that aims to evaluate the impact of providing mid-day meals in government schools in India. The study could end up with low response rates because respondents did not interpret a question correctly. Low response rates can in turn result in underpowered calculations. In this case, the research team should work with field coordinators and translators to draft a more accurate version that respondents can understand clearly.

It is also a good idea to discuss the errors in translation during enumerator training as another level of check on the final translation. Enumerators can then provide feedback based on their own experience, a process called validation. It is also important to have at least one bilingual staff member to carefully record all corrections that are made throughout the different stage. This last step is also useful for the purpose of data documentation, which is important to create reproducible research.

Editing and refining the translations several times can often lead to version control issues. To avoid these, the research team should use the paper version of the questionnaire used for enumerator training should print the questions in the original language, as well as all local languages, side-by-side. This makes it easier to make a note of all required changes without worrying about inconsistency across different versions.

Oral interviews

For oral interviews, such as computer-assisted personal interviews (CAPI), the research team will need accurate oral translations, rather than written translations. The research team should keep a few extra things in mind in this case.

  1. Extend training. Extend the duration of the enumerator training to spend more time on oral translations.
  2. Cross check translations. During the training process, researchers can ask more than one person to suggest translations for each question. Everyone can then discuss each suggestion, and agree on a final version as a group.
  3. Pen-and-paper training. It is a good practice to provide enumerators with printed copies of the questionnaire. When printing these, the research team can leave some space on the paper version of the questionnaire. Enumerators can use this space to write the translated versions of each question.
  4. Mock interviews. Enumerators should also spend extra time on mock interviews and focus on collecting feedback on the quality of the translation.

Related Pages

Click here for pages that link to this topic.

Additional Resources