Adult Education and Family Literacy Act Program

Assessing Quality

Assessing Quality of Professional Development
Participants are surveyed after all workshops, conference presentations and webinars to evaluate and provide feedback on the activity or event. In regional workshops, changes are made to future workshops based on feedback from participants. Programs review student data to determine if training impacted student outcomes and to determine areas where additional training may be needed. The annual training plan includes state priorities, input from surveys about training needs and advice from a statewide professional development advisory committee. Survey results are reviewed with the professional development advisory committee to determine how training is best delivered, either face to face or online webinars. Workshops conducted on college and career readiness are formatted for online courses which include a post-test to determine if learning objectives were met.

The IPDAE website provides methods to measure and assess the involvement and effectiveness of online modules, webinars and regional workshops through data reports, evaluations and surveys. All IPDAE workshop and webinar registrations are online. Once an individual creates an account in the IPDAE portal, a record is kept of all professional development activities they attend. This record is available by county so administrators can review and make local professional development decisions. In addition, DCAE reviews comments submitted by participants.

Assessment of Programs Receiving Federal Grant Funds
DCAE uses various methods of assessment during the annual and ongoing program evaluation process. This evaluation includes site monitoring visits, desk monitoring, program improvement processes and performance and financial compliance reviews. The results of these processes determine program improvement actions.

DCAE implements a Quality Assurance System that ensures student performance improvement, financial accountability, program quality and regulatory compliance of local providers in accordance with federal laws and regulations, state statutes and rules and the provisions of an approved grant award.

The monitoring component of the Quality Assurance System is risk-based. Risk assessment is used to evaluate variables associated with workforce education grants and to assign a rating for levels of risk to the FDOE and DCAE associated with each provider. In completing a risk assessment, certain factors are identified which may affect levels of risk for each agency. A risk matrix is completed for each provider.

To determine the monitoring strategies appropriate for each provider, an annual risk assessment is conducted by the quality assurance team. A range of monitoring strategies includes conference calls, improvement plans, desktop self-assessments and grant reviews. Additional comprehensive strategies are implemented, such as onsite visits, for providers deemed to be at higher risk.

Program Accountability
DCAE established minimum benchmarks for eligible providers. Each eligible provider is required to demonstrate the ability to meet the benchmarks in the initial grant application and during the grant period. DCAE developed an accountability system to measure student enrollment, progression and performance. These requirements are applicable to all three competitive grant-funded programs: Adult General Education, Integrated English Literacy and Civics Education and Corrections Education.

The accountability system established by DCAE requires an eligible provider to demonstrate a program’s size, scope and quality.  It requires an applicant to establish a minimum level of instructional service necessary to improve literacy and workforce preparation in the lives of working-age adults who lack basic literacy skills, a diploma and/or English language skills. The DCAE requires providers to establish minimum enrollment targets and meet the enrollment targets during the grant period. Periodic verification is conducted to determine agency progress on meeting their enrollment target. Providers must offer instruction for a minimum of 10 hours per week for at least 32 weeks for each WIOA eligible program offered by the agency. Periodic updates are required according to a set program schedule. DCAE establishes a maximum funding level an applicant may request based upon the number of students served.

Program accountability is measured through the following WIOA outcome measures:

• Measurable skill gains (MSG) are reported to DCAE during regularly established survey windows as required by the National Reporting System. This is based on the lowest functioning level in which the participant earned at least one eligible literacy completion point, earned a diploma or GED® or exited and enrolled in postsecondary career and technical education by the end of the reporting year.
• Credential Attainment Rate measures student attainment of a secondary school diploma, recognized equivalent or enrolled in postsecondary education or career training within one year of exit.
• Employment Rate (second quarter after exit) measures the number of exiters during the reporting period who are employed during the second quarter after exit divided by the number of exiters during the reporting period.
• Median Earnings (second quarter after exit), for all exiters in a core program, the report of the wage that is at the midpoint between the highest and lowest wage earned in the second quarter after exit.
• Employment Rate (fourth quarter) for the number of exiters during the reporting period who are employed during the fourth quarter after exit divided by the number of exiters during the reporting period.

During the spring of 2020, DCAE will negotiate state performance with the Office of Career, Technical and Adult Education (OCTAE) and all local providers will be expected to meet the state target for each measure in order to develop a more rigorous accountability system. Local provider performance will be analyzed and reports provided to share information. 

Program Improvement Plans
DCAE conducts annual performance evaluations of local provider performance on the WIOA measures discussed above through a data-driven accountability system for adult education programs. If a program does not meet state performance targets, a Program Improvement Plan is required. The DCAE staff reviews the improvement plans to identify opportunities to improve local practices. DCAE provides supports such as targeted professional development, teacher training and site visits.

In addition, DCAE takes the following actions to continually improve the quality of the programs:

• New Director Training

The annual new director training provides new directors with information such as federal and state guidelines, data collection, NRS reporting and resources needed to administer their programs.

• TABE and CASAS Training/Trainers

Through on-going training, a network of trained professionals ensures the uniform administration and reporting of assessments used for determining federal level gains.

• Data Reporting and Program Improvement Training

Data reporting and program improvement training provides training on the fundamentals of the NRS. Topics include monitoring, data analysis and collection, types of data and measures, assessments, data quality and related information.

• Technical Assistance

Adult education calls and webinars are available to inform adult educators of program changes, to report announcements and provide opportunities for practitioners to ask questions of the DCAE staff. DCAE updates and posts technical assistance papers on the FDOE website.

• Teacher Training

Teacher training provides information and resources to support instruction in the areas of ABE, ESOL, GED® preparation, college and career readiness, career awareness and planning, career pathways, reading and math instructional strategies and adults with disabilities.