Data Collection and Management

Data Collection

PRG has expertise in data collection for survey research, randomized controlled trials, needs assessments, implementation evaluations, and qualitative research. Our extensive data collection experience covers a range of policy and programming areas, including mental and behavioral health, public health, education, children and family services, housing, workforce development, and disaster recovery. Achieving high quality and trustworthy data is the foundation for producing excellent research. PRG staff ensure that the data collection and processing procedures are implemented with the highest levels of precision. We strive to remain innovative, responsive, and work in close consultation with our clients at every phase of the data collection process to ensure exceptional data quality.

PRG staff are skilled in identifying the most appropriate methods of data collection for each project, while remaining flexible and responsive to our clients’ needs and capacity for data collection, as well as the specific characteristics of the study population. We employ an array of quantitative and qualitative data collection methods, including surveys, focus groups, in-depth interviews, participant observation, and obtaining large third-party datasets from a range of federal and regional agencies and institutions. PRG offers the following data collection services:

+ Instrument development and design – PRG develops surveys, discussion guides, screening tools, and numerous other types of data collection instruments that are intuitive, appropriate, and comprehensible for each target group. Our in-depth instrument development process includes thorough literature reviews, extensive pre-testing with members of the target population, and a rigorous internal review process prior to the start of data collection.

+ Sample design and weighting

+ Paper questionnaires – Administered in person or by mail; we use scanning software and manual data entry to process paper questionnaires.

+ Web-based questionnaires – Administered through email or in person; our web-based survey software is adaptive and highly sophisticated, allowing for completion on PC, tablet, or smartphone, and with audio capabilities to assist respondents with limited literacy.

+ Telephone interviews – PRG uses in-house staff for smaller phone surveys or interviews and contracts with a phone bank for larger phone surveys.

+ CATI/CAPI capabilities – PRG staff can code surveys into applicable scripting code for the application of computer-assisted personal or telephone interviews.

+ Focus groups, in-depth interviews, and participant observations

+ Translation

+ Mixed-mode surveys – PRG employs mixed-mode data collection methods to increase response rates, for example, offering respondents the option to submit data in person, by mail, online, or by phone.

+ Tracking and collecting data from hard-to-reach populations – Our analysts have years of experience implementing evidence-based strategies to successfully maintain contact with and collect data from hard-to-reach populations. These efforts are essential to maximize study retention and minimize non-response bias (i.e., to retain the study’s internal validity). For research projects that demand rigorous follow-up, PRG regularly achieves response rates higher than 85%.

+ Use of Third-Party Datasets – PRG has over ten years of experience in using third-party datasets to assess program outcomes or answer research questions. The use of these datasets can significantly reduce costs of data collection, maintain high response rates, and expand the analytical scope. Some examples of dataset types that PRG has used recently: a state public health STD/HIV monitoring database, child and family services database; an administrative database from a state-wide college system; state-wide unemployment insurance wage records; state-wide standardized academic test scores.

Data collection strategies are documented for each client in an Administration Plan, which outlines individual roles and responsibilities for data collection, data verification, a data collection timeline, and if needed, specification of strategies to retain participants and improve response rates. PRG staff regularly reviews and monitors data to ensure that all administration protocols are being observed and that data are being collected with integrity.

Data Quality

Data quality is at the heart of PRG’s core mission to conduct exceptional evaluations and research for our clients and the people they serve. PRG employs the following strategies to ensure data quality and accuracy for each unique client and project:

+ Standardized project-specific guides – Each project has a written evaluation/survey/assessment plan, administration plan, and analysis plan that establishes strict guidelines for measurement, data collection, data management, analysis, and reporting.

+ Training – Whether data are collected by PRG staff or by our clients or community members, PRG provides comprehensive training and ongoing monitoring of data collection procedures for each person tasked with data collection.

+ Data entry – PRG staff members are trained in the proper use of relevant software used for data entry processes. PRG analysts and principals are responsible for quality assurance and fidelity of data.

+ Structured protocols – PRG has documented data entry, quality assurance, and cleaning protocols to establish the accuracy of data, including the identification of patterns of inaccuracies and anomalies, such as out-of-range and missing data, and unusual distributions of scores.

+ Project management responsiveness – Each project has a PRG Research Analyst assigned as the point of contact and day-to-day project manager; the analyst conducts regular telephone meetings and in-person site visits with clients.

Data Security

PRG regularly develops protocols, plans, and instruments that meet the needs of our clients and funding agencies. Security protect the privacy and confidentiality (or anonymity) of participants. Our staff observes strict protocols for handling confidential data. We implement the following management, operational, and technical security controls in data storage:

+ PRG staff members receive training in management and use of confidential data, as well as training in Human Subjects (IRB), HIPPAA, and FERPA requirements.

+ Use of secure, locked storage facilities, use of password-protected servers behind a firewall, and shredding of project-related paper output.

+ All online data collection is done using the latest encryption technology, and data are hosted on a secure server with a state of the art firewall.

+ Confidential paper data are stored in locked filing locations, with specific protocols established for retrieving and returning such data.

Some Examples of Data Collection and Management Experience:

+ PRG conducted multiple state-wide needs assessments of Persons Living with HIV/AIDS. We designed a questionnaire in English and Spanish and developed a comprehensive data collection plan, utilizing peer coordinators to generate a representative sample across the state.

+ PRG was funded by the Office of Adolescent Health to implement a longitudinal, randomized controlled trial to evaluate the effectiveness of a sexual education curriculum designed to reduce risky sexual behavior in African-American adolescents ages 14-18. We developed a questionnaire to gather information on study participants’ knowledge, belief of risk of harm, attitudes, self-efficacy, and behaviors related to safe sex practices. Our staff administers the questionnaire to 850 study participants at four data collection points (baseline, post-program, six-months post program, and 12 months post program); using evidence-based strategies, thus far we have retained over 87% of study participants at six-month follow-up and over 90% of study participants at 12-month follow-up.

+ Under a sub-contract with the National Opinion Research Center (NORC) at the University of Chicago, PRG conducted a qualitative evaluation of a project which sought to enhance community resilience in the face of disaster by building existing local knowledge, resources, and networks. PRG conducted site visits, structured interviews, and focus groups to assess pilot sites to identify implementation facilitators and barriers, as well as best practices and lessons learned to disseminate to future implementation sites.

+ PRG was a lead partner in the assessment of the Catholic Charities Deepwater Horizon Oil Spill Case Management Program. Catholic Charities conducted a case management program with the aim to help families affected by the oil spill in southeast Louisiana move toward recovery and long-term financial stability. PRG conducted several focus groups with case managers, clients, and members of the Consumer Council to draw generalized inferences about the case management services, specifically whether they helped to develop the economic self-sufficiency of participating clients.

+ PRG was contracted by Market Umbrella to conduct surveys at three farmers’ markets throughout New Orleans, using questionnaires developed by Market Umbrella. Target populations for the different surveys were market customers, vendors, and neighbors. PRG drafted brief data collection protocols, trained data collectors, collected the data, and provided Market Umbrella with results.