Difference between revisions of "Procuring a Survey Firm"
| (26 intermediate revisions by 6 users not shown) | |||
| Line 1: | Line 1: | ||
This article covers the procurement process for the data collection agency. | <onlyinclude>This article covers the procurement process for the data collection agency. </onlyinclude> | ||
== Read First == | == Read First == | ||
Responsibility for survey firm procurement varies. There are three primary modes: survey firm contracted by the research team (World Bank or University), survey firm contracted by the government (or other implementing partner), or direct hire of data collectors. In the first case, the research team will manage the full selection process. In the second case, the IE team role will be limited to technical assistance on developing the terms of reference and designing a scoring matrix. The third case is much more rare, and typically only employed for pilots, survey audits, or small-scale qualitative data collection. This article focuses on the first case, in which the research team manages the full procurement process. | |||
== Guidelines == | == Guidelines == | ||
=== Who will collect the data? === | === Who will collect the data? === | ||
Many World Bank-funded surveys are conducted by a private [[Survey Firm]]. Other options include: | |||
1. Government agency / Ministry | 1. Government agency / Ministry | ||
| Line 21: | Line 21: | ||
* Cons: Procurement challenge (many individual consultants), full responsibility for logistics, requires much more time/effort from research team | * Cons: Procurement challenge (many individual consultants), full responsibility for logistics, requires much more time/effort from research team | ||
== Procurement Timeline== | |||
{| class="wikitable" style="margin-left: | {| class="wikitable" style="margin-left: 10px; margin-right: auto; border: none;" | ||
|- | |- | ||
! Stage | ! Stage | ||
| Line 60: | Line 59: | ||
|- | |- | ||
|} | |} | ||
For guidelines and examples, refer to [[Survey Firm TOR|Terms of Reference (TOR)]] | |||
=== Terms of Reference === | === Terms of Reference === | ||
| Line 65: | Line 66: | ||
Be sure that expectations and standards are clearly spelled out, with potential consequences. Otherwise even if you detect fraud may not be able to do anything about it! | Be sure that expectations and standards are clearly spelled out, with potential consequences. Otherwise even if you detect fraud may not be able to do anything about it! | ||
=== Troubleshooting === | === Troubleshooting === | ||
* If you find problems with observable quality or representativeness of data, for example from a | * If you find problems with observable quality or representativeness of data, for example from a '''survey audit'''. | ||
** Examples: Enumerators not visiting households / falsifying data, Enumerators falsifying information to shorten interviews, Field Teams dropping households that weren’t actually unavailable (just difficult to get to, or not available on first visit) | ** Examples: Enumerators not visiting households / falsifying data, Enumerators falsifying information to shorten interviews, Field Teams dropping households that weren’t actually unavailable (just difficult to get to, or not available on first visit) | ||
** Clearly specified protocols & standards in TORs are the mechanism for dealing with this. If you have laid out consequences for fraudulent activities in the TORs, those consequences can be called into effect here, and the firm will be given a warning, and if the problem continues the contract can be cancelled. | ** Clearly specified protocols & standards in TORs are the mechanism for dealing with this. If you have laid out consequences for fraudulent activities in the TORs, those consequences can be called into effect here, and the firm will be given a warning, and if the problem continues the contract can be cancelled. | ||
| Line 77: | Line 76: | ||
** Examples: Enumerators supposed to be paid by day, instead paid by questionnaire; enumerators paid much less than promised or salary withheld | ** Examples: Enumerators supposed to be paid by day, instead paid by questionnaire; enumerators paid much less than promised or salary withheld | ||
** It is very difficult for the research team to intervene in this case, as there is no formal relationship with the sub-contractors. It can only formally be dealt with if there are observable consequences for survey protocols or data quality, violating the TORs. | ** It is very difficult for the research team to intervene in this case, as there is no formal relationship with the sub-contractors. It can only formally be dealt with if there are observable consequences for survey protocols or data quality, violating the TORs. | ||
** Though hard to address, it is important to know when these problems are happening. | ** Though hard to address, it is important to know when these problems are happening. It's a good idea for field coordinators to build relationships with enumerators to understand how the work is going and identify issues. | ||
* Know average survey costs in the context you will work in. If a proposal seem | * Know average survey costs in the context you will work in. If a proposal seem too cheap to be true, it probably is. | ||
* Be aware this is potentially a repetitive game, in some markets survey firms have limited competition. | * Be aware this is potentially a repetitive game, in some markets survey firms have limited competition. | ||
| Line 88: | Line 87: | ||
== Back to Parent == | == Back to Parent == | ||
This article is part of the topic [[ | This article is part of the topic [[Primary Data Collection]] | ||
== Additional Resources == | == Additional Resources == | ||
* | *DIME Analytics' [https://github.com/worldbank/DIME-Resources/blob/master/Survey%20firm%20Technical%20Review%20_%20Scoring%20Matrix.xlsx Survey Firm Technical Review Scoring Matrix] | ||
*DIME Analytics' [https://github.com/worldbank/DIME-Resources/blob/master/survey-firms.pdf Working with Survey Firms] | |||
[[Category: | [[Category: Research Design]] | ||
[[Category: Primary Data Collection]] | |||
Latest revision as of 12:42, 1 June 2021
This article covers the procurement process for the data collection agency.
Read First
Responsibility for survey firm procurement varies. There are three primary modes: survey firm contracted by the research team (World Bank or University), survey firm contracted by the government (or other implementing partner), or direct hire of data collectors. In the first case, the research team will manage the full selection process. In the second case, the IE team role will be limited to technical assistance on developing the terms of reference and designing a scoring matrix. The third case is much more rare, and typically only employed for pilots, survey audits, or small-scale qualitative data collection. This article focuses on the first case, in which the research team manages the full procurement process.
Guidelines
Who will collect the data?
Many World Bank-funded surveys are conducted by a private Survey Firm. Other options include:
1. Government agency / Ministry
- Pros: enumerators have sector knowledge; may be logistically simpler if project is paying for survey
- Cons: perceived as independent? willing to survey control sites? Quality controls? performance incentives? HH survey experience?
2. National Statistics Office
- Pros: Often high capacity
- Cons: IE surveys are not in typical scope of work (focus on nationally representative surveys), busy with existing surveys, may not be interested in small-scale contracts
3. Directly hire enumerators
- Pros: Highest degree of flexibility and control over the process
- Cons: Procurement challenge (many individual consultants), full responsibility for logistics, requires much more time/effort from research team
Procurement Timeline
| Stage | Minimum time required |
|---|---|
| Due diligence: research local survey firm options | 2 weeks |
| Prepare detailed Terms of Reference (TORs) | 2 weeks |
| Publish request for Expression of Interest (rEOI) | 1 day |
| Firms submit expression of interest (EOI) | 3 weeks |
| Shortlist firms based on EOI | 1 day |
| Publish TOR and call for proposals | 1 day |
| Shortlisted firms submit technical and financial proposals | 3 weeks |
| Evaluation of technical then financial proposals | 1 week |
| Negotiations and award of contract to selected firm | 1 week |
| Contract published and signed | 1 week |
For guidelines and examples, refer to Terms of Reference (TOR)
Terms of Reference
The Survey Firm TOR specifies scope of work, responsibilities, required activities, and deliverables. Developing a detailed TOR is essential!
Be sure that expectations and standards are clearly spelled out, with potential consequences. Otherwise even if you detect fraud may not be able to do anything about it!
Troubleshooting
- If you find problems with observable quality or representativeness of data, for example from a survey audit.
- Examples: Enumerators not visiting households / falsifying data, Enumerators falsifying information to shorten interviews, Field Teams dropping households that weren’t actually unavailable (just difficult to get to, or not available on first visit)
- Clearly specified protocols & standards in TORs are the mechanism for dealing with this. If you have laid out consequences for fraudulent activities in the TORs, those consequences can be called into effect here, and the firm will be given a warning, and if the problem continues the contract can be cancelled.
- If you become aware of logistical (typically sub-contractor) problems
- Examples: Enumerators supposed to be paid by day, instead paid by questionnaire; enumerators paid much less than promised or salary withheld
- It is very difficult for the research team to intervene in this case, as there is no formal relationship with the sub-contractors. It can only formally be dealt with if there are observable consequences for survey protocols or data quality, violating the TORs.
- Though hard to address, it is important to know when these problems are happening. It's a good idea for field coordinators to build relationships with enumerators to understand how the work is going and identify issues.
- Know average survey costs in the context you will work in. If a proposal seem too cheap to be true, it probably is.
- Be aware this is potentially a repetitive game, in some markets survey firms have limited competition.
- Field coordinators should communicate progress and problems with research team often
- Be careful with asking for things that are not clearly specified in the TORs
Back to Parent
This article is part of the topic Primary Data Collection
Additional Resources
- DIME Analytics' Survey Firm Technical Review Scoring Matrix
- DIME Analytics' Working with Survey Firms