We sought to answer some of these in the context of the Swabhimaan programme using remote telephone-based surveys. Swabhimaan is an ongoing evaluation that tests the delivery of maternal and adolescent nutrition and health services via women’s self-help groups and their higher-order federations. The programme is led by DAY-NRLM and State Rural Livelihoods Missions of Bihar, Chhattisgarh and Odisha, with technical support from UNICEF India. Our team supports programme process monitoring and evidence generation from the programme’s treatment sites. International Institute for Population Sciences (IIPS) has conducted baseline and midline evaluations for Swabhimaan.
This is the first in a series of blog posts where we share our experience of using techonology to adapt process monitoring systems in an on-going impact evaluation during times of crisis. We will also share our findings on the status of maternal and adolescent health & nutrition services, insights on how adaptations are unfolding in the field and their implications for the programme.
In this post we focus on how we navigated contextual challenges of collecting data remotely from Bihar and Chhattisgarh, both states where interviews were conducted in Hindi. Through this we hope to contribute to the growing body of work on conducting phone-based surveys in harder to reach areas that have low connectivity.
What were our questions?
We interviewed 369 respondents involved in interventions across five sites; Jalalgarh and Kasba in Bihar, Bastar block in Chhattisgarh and Koraput Sadar, and Pallahara in Odisha. These intervention blocks are also amongst DAY-NRLM’s poorest resource blocks where the programme seeks to reach the most vulnerable through the existing SHG networks.
We set out to enquire about the following:
- What is the status of essential services, and women collectives- led activities for maternal and adolescent health and nutrition?
- What are the new roles undertaken by district and state-level functionaries to ensure activities under the programme do not come to a standstill?
- Whether activities are being conducted as per government decrees/advisories?
- What is the status of entitlement schemes provided by allied departments?
- What is the status of food security in the community?
Adapting existing process monitoring systems
Monthly progress reports were fed into the programmes’ established MIS system. These were collected from the field via community resource persons known as Poshan Sakhis or ‘nutrition guides’, and Kishori Sakhis or ‘adolescent guides’ who reported on these activities using paper-based formats. Reports were then collated at block and district administrative levels before being shared with the state. The last MPR was collected between March and April 2020.
We chose to use telephone-based surveys most of the cadre have access to phones but are not always literate. Further, since they are already familiar with the process of submitting monthly progress reports and regularly in touch with block-level staff, we didn't expect many cases of non-response.
Interviews were conducted with two kinds of programme implementers using two methods
- Telephonic interviews with Community Cadre: Poshan Sakhis, Kishori Sakhis, Village Organisation functionaries and Cluster Level Federation functionaries involved in various aspects of programme implementation like promoting positive nutrition behaviours, linking beneficiaries to schemes and loans nutri-based livelihoods, organising health and entitlement camps and social drives in the local community
- Link-based self-administered questionnaires with Programme staff: State Rural Livelihood Mission Staff and consultants at block, district and state levels
Block-level MIS consultants and a team of trained data collectors conducted interviews based on a bilingual interview schedule. A random, representative sample of village organisations was drawn from each state for round one.
What we learnt about telephonic data collection
Collecting valid phone numbers. In many cases, respondents’ phone numbers had either changed or were invalid. Given that numbers change frequently, these must be checked and updated regularly before preparing the master database to ensure the interview process runs smoothly.
Connectivity mapping. This must be done beforehand. In Bihar, two villages- Bareta and Sanjheli- which lie in low connectivity areas had to replaced, during the data collection period because respondents from here could not be reached. To deal with this, the possibility of providing respondents with sim cards for networks that have better connectivity, or covering costs of these is being considered. Alternate ways including paper-based data collection from low connectivity areas while adhering to physical distancing norms are also being considered.
Timely information shared with respondents. Prior to the interview, all supervisors must inform respondents on the purpose of the interview and the time-frame for when they can expect a call. This was part of ethical considerations to ensure respondents were in a position to understand and provide informed answers. Prior appointments also save time as data collectors can connect and complete interviews in the first attempt.
Respondents without their own phones. Most respondents in Bastar didn’t have their own phones and the phones were usually with their husbands. Appointments thus had to be made depending on when the husband was at home. In a few cases, certain men were suspicious and insisted on answering on behalf of their wives. In such cases, the background and purpose of data collection were re-explained with an emphasis on why they needed information directly from the women.
Interviewing rural respondents during kharif season. Most of our intervention sites are rice-growing areas. The onset of monsoon meant that families would leave for their fields by 8:30 am and return at 8:00 pm except for an hour in between for lunch. All interviews had to be scheduled within a limited time frame between 7:00 am – 8:30 and after 9:00 pm. Data collectors had to work outside conventional office timings. This was a challenge for block-level staff who didn't own laptops or computers for data entry and were dependent on computers provided at the block office.
Power cuts & phones without balance. Cases of phones being switched off were surprisingly high, particularly in Bastar. This was unexpected because programme staff and supervisors who were also involved in data collection had been in regular touch with respondents. We later found that because of continuous power cuts during data collection week many had to travel to neighbouring villages to recharge and switch on their phones. Further, many couldn’t receive calls because the phones didn’t have balance and mobile shops were closed. Some also reported not having enough money to top-up their phones regularly because of income lost during the lockdown. We were able to contact most respondents at later dates, and some were also contacted via their neighbours' phone
Comprehending regional dialects. While all respondents in Bihar understood Hindi, many spoke Surjapuri which has influences of Assamese, Bengali and Maithli, as their first language. Similarly, our respondents in Bastar also spoke either Halbi or Gondi. This made it challenging for the Delhi-based data collectors to understand responses. They were then oriented and prepared to deal with this.
Contextualising programmatic language. Official terms used for a number of public goods, services and interventions had to be referred to by colloquially used terms. These were discussed and noted after pre-testing. A glossary of local terms was added to the data collector’s manual, to ensure questions were correctly understood.
Paying attention to signs of confusion/ hesitation/discomfort. While the questionnaire administered was objective, signs of hesitation or confusion from the respondent such as prolonged silences or cross-questions from the respondent, for instance, became cues to either probe further or to move to a different section. A provision note of these qualitative nuances has been added at the end of each section. As a team, we made note of and discussed these observations during our daily stocktaking meetings.
Addressing requests/ expectations. The team was careful to ensure that data collectors did not raise respondents' expectations. In cases where a few requests were made, it was clarified that information would be passed on to the block level staff as was done earlier during the programme reporting process. The objective and limitations of the data collection process were re-iterated at these times.
Getting data validated by government functionaries. As data collection was being managed by an external knowledge management team, government validation of the data was essential to build ownership over the process monitoring system. This also ensures that data could be used to take initiate appropriate corrective action. Officials and state programme managers from State Rural livelihoods Missions were thus closely involved in tool design, and finalising the methodology. All factsheets were also validated by district staff.
Building capacities of block and district level consultants. While data collection for the first round was supported by an external team, consecutive rounds will be incrementally handed over to block and district programme staff who normally manage programme MIS. They were thus included in all training sessions. Block and district offices will also involve more resources in data collection tasks. These staff will then be capacitated.
Such adaptations in programmes are critical to ensure the continuity of interventions and to support community-led monitoring systems to reach the desired outcomes of the programme.
Abhishek Saraswat, Data Analyst, International Institute for Population Sciences (IIPS)
Bharati Sahu is a programme consultant for Swabhimaan currently supporting implementation in Bastar
Monica Shrivastav is a Consultant - State support and training
Neha Abraham is a Junior Professional Officer- Knowledge Management & Documentation
snhe3pph2m|00102FE5A592|RoshniDB|TBL_Blog|Description|AFE8B6DD-6A53-4744-AF22-64CEDE6C1BDF