ENSURING DATA QUALITY

Every year, ASER procedures to ensure data quality are reviewed and strengthened. Three key elements are involved: training of field teams, monitoring of the survey while it is in the field, and recheck of data already collected. In recent years more than 40% of all sampled villages have been monitored, rechecked or both. For a detailed report on our quality control framework, click here.

Training

ASER has a 3-tier training process. The kickoff to each year's survey is the National Workshop for all ASER central and state teams, where tools, formats and procedures are extensively reviewed, practiced and finalized. Two important components comprise a field pilot, during which ASER teams conduct the survey in nearby villages exactly as though it were the final exercise; and mock training sessions, where ASER staff plan and practice training on all aspects of the survey.

Following the National Workshop, a state level training is organized in every state, led by the respective ASER team members. These workshops are usually supported by the Pratham State head and by a member of the ASER central team. At the state level training workshops, Master Trainers are trained in all aspects of the survey. Master Trainers - typically more than 1,000 each year -  are drawn from Pratham network and from participating partner organizations, and are responsible for training the volunteers who actually conduct the survey in each district. As in the National Workshop, these sessions focus both on the actual tools and procedures involved and on thinking about effective ways to communicate these topics to ASER volunteers.

Finally, Master Trainers organize a training workshop for volunteers at the district level. District trainings usually last three days, and comprise classroom sessions, field practice, and a quiz. A detailed report on ASER's robust training model can be found here.

Monitoring

Starting with ASER 2011, the ASER survey in each district is planned for two consecutive weekends wherever possible, as opposed to a single weekend as was done previously. In ASER 2012 the survey in most districts was conducted over two weekends, allowing ASER Master Trainers to personally monitor the survey in 3-4 villages - more than 10% of the total sample. In addition, starting with ASER 2011, a call centre was set up in some states to monitor the progress of the survey and the activities of master trainers on a daily basis. These procedures help to quickly identify areas requiring corrective action.

In ASER 2012, about 28% of all villages were monitored by Master Trainers while the survey was in process.             

Recheck

 ASER data recheck procedures have expanded over the years. Currently, four processes are implemented.                

SMS recheck:  A summary of district level data is transmitted via SMS and uploaded on a common portal, enabling ASER Centre staff to assess data quality in real time and identify locations where additional checking is warranted.

Phone and desk recheck:  Respondent households are contacted by telephone, a process that enables the quick identification of villages where the survey was not conducted correctly. In addition, Master Trainers review survey formats received for all surveyed villages.

Master Trainer field recheck:  Based on the information obtained from the desk and phone recheck process, villages are identified for field recheck. In each such village, 50% of all surveyed households are rechecked and key parameters of the survey are verified: sampling, selection of children and testing.

Process audit:  In 2008 and again in 2012, external teams conducted a process audit in selected states. Audit teams observed state and district trainings as well as the actual survey in a sample of villages, in order to assess whether procedures are implemented as envisaged.