PDG Program Performance Evaluation

by Kate Sutton, Research Specialist in Qualitative Methods (Buffett Early Childhood Institute) – written for the PDG quarterly eNewsletter

The Program Performance Evaluation (PPE) was launched in February 2019 to evaluate the effectiveness of the state’s Preschool Development Grant Birth-5 (PDG B-5) in aligning and enhancing state systems. A secondary goal involved assessing the effectiveness of the individual project implementation. The PPE addressed the following questions: 1) How are program activities being implemented?; 2) What are the small wins and/or barriers to progress?; 3) What are the unexpected consequences?; 4) How are projects individually and collectively helping to meet Nebraska’s goal to align and improve state systems?

The evaluation included the creation of a logic model, monthly online surveys, and key informant interviews. Seventeen grant projects (nested in 5 grant-mandated activities), were included in the evaluation. Grant project leads responded monthly to an online Qualtrics survey, which captured their timeline and implementation steps completed, project implementation supports and barriers, planned activities for the next month, and perceived unexpected consequences. The PPE Team presented summary survey feedback at the monthly Nebraska Leadership Team meetings. To complement the monthly online survey, the PPE team conducted interviews with PDG project leads and other PDG leadership members to identify perceptions of project success and challenges, and progress related to systems-level changes. Eleven interviews were conducted at the mid-point (August – October) and 18 interviews were conducted at the end of the grant term (January – February). 

The formation of the Program Management team and Nebraska Leadership Team through the Preschool Development Grant increased communication, collaboration, and cooperative efforts among partners. The comprehensive Needs Assessment facilitated greater understanding of the strengths, needs, and gaps in the systems serving young children, families, and ECE providers. Finally, the development of the Nebraska State Early Childhood Strategic Plan has aligned partners around a shared vision for systems change and alignment in the state. Feedback from project leads included barriers to progress. Many project leads mentioned it was a challenge to get the work done in the restricted timeline of the grant. The short timeframe didn’t allow for proper planning or the ability to ‘build capacity’ before implementation. Other implementation barriers included delays in formal notification, Activity 5 delays, late timing of federal guidance, inadequate staffing, and challenges related to spending and the budget. The most notable unintended consequence was related to the organizational capacity to do the work. The proposed work was more complex than anticipated and it exposed internal challenges for each organization and project.  While PDG B-5 increased collaboration, communication, and understanding among many key leaders and stakeholders, the work to align and enhance state systems continues. More detailed findings will be available in the final report to DHHS, available after the end of the grant term on February 29, 2020.

About Nebraska Children

Nebraska Children's mission is to maximize the potential of Nebraska’s children, youth, and families through collaboration and community-centered impact.