Difference between revisions of "Procuring a Survey Firm"

Jump to: navigation, search
 
(26 intermediate revisions by 6 users not shown)
Line 1: Line 1:
This article covers the procurement process for the data collection agency.  
<onlyinclude>This article covers the procurement process for the data collection agency. </onlyinclude>




== Read First ==
== Read First ==


Survey firm procurement processes vary across IE projects. Depending on the counterparts involved and funding allocation, either the IE team or the implementing agency will be in charge of recruiting the survey firm. If the latter, the IE team may have little visibility on the selection process and should only assist in providing technical feedback. However, if the IE team is in charge of funding the data collection, then it will be responsible for the totality of the selection process.
Responsibility for survey firm procurement varies. There are three primary modes: survey firm contracted by the research team (World Bank or University), survey firm contracted by the government (or other implementing partner), or direct hire of data collectors. In the first case, the research team will manage the full selection process. In the second case, the IE team role will be limited to technical assistance on developing the terms of reference and designing a scoring matrix. The third case is much more rare, and typically only employed for pilots, survey audits, or small-scale qualitative data collection. This article focuses on the first case, in which the research team manages the full procurement process.


== Guidelines ==
== Guidelines ==


=== Who will collect the data? ===
=== Who will collect the data? ===
Most DIME surveys are conducted by a private [[Survey Firm]]. Other options include:  
Many World Bank-funded surveys are conducted by a private [[Survey Firm]]. Other options include:  


1. Government agency / Ministry
1. Government agency / Ministry
Line 21: Line 21:
* Cons: Procurement challenge (many individual consultants), full responsibility for logistics, requires much more time/effort from research team
* Cons: Procurement challenge (many individual consultants), full responsibility for logistics, requires much more time/effort from research team


=== Procurement Process ===
== Procurement Timeline==


{| class="wikitable" style="margin-left: auto; margin-right: auto; border: none;"
{| class="wikitable" style="margin-left: 10px; margin-right: auto; border: none;"
|+ Figure 1: Procurement Process
|-
|-
! Stage  
! Stage  
Line 60: Line 59:
|-
|-
|}
|}
For guidelines and examples, refer to [[Survey Firm TOR|Terms of Reference (TOR)]]


=== Terms of Reference ===
=== Terms of Reference ===
Line 65: Line 66:


Be sure that expectations and standards are clearly spelled out, with potential consequences. Otherwise even if you detect fraud may not be able to do anything about it!
Be sure that expectations and standards are clearly spelled out, with potential consequences. Otherwise even if you detect fraud may not be able to do anything about it!
[https://github.com/worldbank/DIMEwiki/blob/master/Topics/Survey_Firm_Procurement/DIME_Survey_Firm_TOR_Template.pdf Survey Firm TOR Template]


=== Troubleshooting ===
=== Troubleshooting ===


* If you find problems with observable quality or representativeness of data, for example from a [[Survey Audit]]
* If you find problems with observable quality or representativeness of data, for example from a '''survey audit'''.
** Examples: Enumerators not visiting households / falsifying data, Enumerators falsifying information to shorten interviews, Field Teams dropping households that weren’t actually unavailable (just difficult to get to, or not available on first visit)
** Examples: Enumerators not visiting households / falsifying data, Enumerators falsifying information to shorten interviews, Field Teams dropping households that weren’t actually unavailable (just difficult to get to, or not available on first visit)
** Clearly specified protocols & standards in TORs are the mechanism for dealing with this. If you have laid out consequences for fraudulent activities in the TORs, those consequences can be called into effect here, and the firm will be given a warning, and if the problem continues the contract can be cancelled.
** Clearly specified protocols & standards in TORs are the mechanism for dealing with this. If you have laid out consequences for fraudulent activities in the TORs, those consequences can be called into effect here, and the firm will be given a warning, and if the problem continues the contract can be cancelled.
Line 77: Line 76:
** Examples: Enumerators supposed to be paid by day, instead paid by questionnaire; enumerators paid much less than promised or salary withheld
** Examples: Enumerators supposed to be paid by day, instead paid by questionnaire; enumerators paid much less than promised or salary withheld
** It is very difficult for the research team to intervene in this case, as there is no formal relationship with the sub-contractors. It can only formally be dealt with if there are observable consequences for survey protocols or data quality, violating the TORs.
** It is very difficult for the research team to intervene in this case, as there is no formal relationship with the sub-contractors. It can only formally be dealt with if there are observable consequences for survey protocols or data quality, violating the TORs.
** Though hard to address, it is important to know when these problems are happening. IT's a good idea for field coordinators to build relationships with enumerators to understand how the work is going and identify issues.  
** Though hard to address, it is important to know when these problems are happening. It's a good idea for field coordinators to build relationships with enumerators to understand how the work is going and identify issues.  


* Know average survey costs in the context you will work in. If a proposal seem to cheap to be true, it probably is…
* Know average survey costs in the context you will work in. If a proposal seem too cheap to be true, it probably is.


* Be aware this is potentially a repetitive game, in some markets survey firms have limited competition.  
* Be aware this is potentially a repetitive game, in some markets survey firms have limited competition.  
Line 88: Line 87:


== Back to Parent ==
== Back to Parent ==
This article is part of the topic [[Survey Firm Procurement]]
This article is part of the topic [[Primary Data Collection]]
 


== Additional Resources ==
== Additional Resources ==
* list here other articles related to this topic, with a brief description and link
*DIME Analytics' [https://github.com/worldbank/DIME-Resources/blob/master/Survey%20firm%20Technical%20Review%20_%20Scoring%20Matrix.xlsx Survey Firm Technical Review Scoring Matrix]
 
*DIME Analytics' [https://github.com/worldbank/DIME-Resources/blob/master/survey-firms.pdf Working with Survey Firms]
[[Category: Survey Firm Procurement ]]
[[Category: Research Design]]
[[Category: Primary Data Collection]]

Latest revision as of 12:42, 1 June 2021

This article covers the procurement process for the data collection agency.


Read First

Responsibility for survey firm procurement varies. There are three primary modes: survey firm contracted by the research team (World Bank or University), survey firm contracted by the government (or other implementing partner), or direct hire of data collectors. In the first case, the research team will manage the full selection process. In the second case, the IE team role will be limited to technical assistance on developing the terms of reference and designing a scoring matrix. The third case is much more rare, and typically only employed for pilots, survey audits, or small-scale qualitative data collection. This article focuses on the first case, in which the research team manages the full procurement process.

Guidelines

Who will collect the data?

Many World Bank-funded surveys are conducted by a private Survey Firm. Other options include:

1. Government agency / Ministry

  • Pros: enumerators have sector knowledge; may be logistically simpler if project is paying for survey
  • Cons: perceived as independent? willing to survey control sites? Quality controls? performance incentives? HH survey experience?

2. National Statistics Office

  • Pros: Often high capacity
  • Cons: IE surveys are not in typical scope of work (focus on nationally representative surveys), busy with existing surveys, may not be interested in small-scale contracts

3. Directly hire enumerators

  • Pros: Highest degree of flexibility and control over the process
  • Cons: Procurement challenge (many individual consultants), full responsibility for logistics, requires much more time/effort from research team

Procurement Timeline

Stage Minimum time required
Due diligence: research local survey firm options 2 weeks
Prepare detailed Terms of Reference (TORs) 2 weeks
Publish request for Expression of Interest (rEOI) 1 day
Firms submit expression of interest (EOI) 3 weeks
Shortlist firms based on EOI 1 day
Publish TOR and call for proposals 1 day
Shortlisted firms submit technical and financial proposals 3 weeks
Evaluation of technical then financial proposals 1 week
Negotiations and award of contract to selected firm 1 week
Contract published and signed 1 week

For guidelines and examples, refer to Terms of Reference (TOR)

Terms of Reference

The Survey Firm TOR specifies scope of work, responsibilities, required activities, and deliverables. Developing a detailed TOR is essential!

Be sure that expectations and standards are clearly spelled out, with potential consequences. Otherwise even if you detect fraud may not be able to do anything about it!

Troubleshooting

  • If you find problems with observable quality or representativeness of data, for example from a survey audit.
    • Examples: Enumerators not visiting households / falsifying data, Enumerators falsifying information to shorten interviews, Field Teams dropping households that weren’t actually unavailable (just difficult to get to, or not available on first visit)
    • Clearly specified protocols & standards in TORs are the mechanism for dealing with this. If you have laid out consequences for fraudulent activities in the TORs, those consequences can be called into effect here, and the firm will be given a warning, and if the problem continues the contract can be cancelled.
  • If you become aware of logistical (typically sub-contractor) problems
    • Examples: Enumerators supposed to be paid by day, instead paid by questionnaire; enumerators paid much less than promised or salary withheld
    • It is very difficult for the research team to intervene in this case, as there is no formal relationship with the sub-contractors. It can only formally be dealt with if there are observable consequences for survey protocols or data quality, violating the TORs.
    • Though hard to address, it is important to know when these problems are happening. It's a good idea for field coordinators to build relationships with enumerators to understand how the work is going and identify issues.
  • Know average survey costs in the context you will work in. If a proposal seem too cheap to be true, it probably is.
  • Be aware this is potentially a repetitive game, in some markets survey firms have limited competition.
  • Field coordinators should communicate progress and problems with research team often
  • Be careful with asking for things that are not clearly specified in the TORs

Back to Parent

This article is part of the topic Primary Data Collection

Additional Resources