Adult Education and Family Literacy Act Program

The CDE assesses the quality of providers through quarterly and annual evaluations. A major focus of these evaluations is to measure the effectiveness of state and local providers in attaining the core indicator performance levels negotiated with the ED.

These evaluations also measure continued progress and improvement of the goals and objectives of the considerations in Section 231(e).

The CDE reviews (1) strategies, processes, and barriers to attaining the performance levels; and (2) quantitative and qualitative data to evaluate the progress and improvement of the programs.

The evaluations (1) collect local provider and student performance measures, (2) determine the level of student improvement, (3) identify program quality, and (4) determine the extent to which populations identified are served.

Results of the evaluations provide (1) relevant information about the effectiveness of adult education programs, (2) characteristics of the learners participating in the literacy programs, (3) analyses of learner gains and identification of emerging needs, and (4) the impact of local providers in meeting their identified performance standards.

Pursuant to Section 212 of the AEFLA, each agency must provide student progress measures obtained from all students who have attended at least 12 hours of instruction in programs receiving sections 225, 231 and 243 federal supplemental funds.

Documented progress of student performance measures must include at a minimum:

·       literacy skill level improvements in reading, writing, and speaking the English language, problem solving, numeracy, and other literacy skills;

·       placement in, retention in, or transition into postsecondary education, training, unsubsidized employment, or career advancement; and

·       a secondary school diploma or its equivalent.

 All agencies are required to maintain individual student records for all students who have attended 12 hours of instruction. Each record must contain (1) student identification and demographic information; (2) attendance rates; (3) years of schooling and placement level at program entry; (4) initial learning goals; (5) specified pre– and post–testing student information; (6) entry and update records; and (7) other specified information necessary.

Monitoring and Program Evaluation

 As part of the CDE’s Federal Program Monitoring (FPM) process, the Adult Education Office conducts annual reviews to ensure local agencies provide effective and high quality services consistent with WIOA Title II legislation. The FPM process also evaluates fiscal and legal areas of responsibility. In addition, for on–going assessment purposes:

1.   The CDE requires all agencies to submit quarterly reports that reflect student participation levels and progress.

2.   The CDE conducts an annual comprehensive qualitative program survey. This survey is required of all participating agencies and involves practitioner focus groups and interviews of both teachers and students. The results provide recommendations for state level planning and development activities, identify best practices and emerging needs, and help focus professional development and training to ensure effective instructional programs for targeted populations.

3.   The CDE presently incorporates the Core Performance Follow–up Survey system to track student outcomes in the areas of obtaining or retaining employment, as well as transitioning to postsecondary education or training. Under WIOA, the CDE in collaboration with the EDD will create an accountability system to track and report the employment follow up required by the new law.

Evaluating Professional Development

Beginning in 2001, CDE contracted with the American Institutes for Research (AIR) to serve as its provider of statewide professional development through the California Adult Literacy Professional Development Project (CALPRO). AIR is one of the world's largest behavioral and social science research and evaluation organizations, and it has considerable expertise in adult learning, technical assistance (TA), and professional development (PD) for adult educators. Regardless of the topic, creating products and services for CALPRO, AIR’s researchers and TA experts to determine what the research says are best practices effective for adult education, and AIR works with external subject matter experts and field practitioners to inform, create, and refine the research-based products will be appropriate for California adult education contexts.

When delivering a training, CALPRO staff, together with its network of external consultants, modifies its products, as needed, to continue to improve them, while staying true to the research.

 In order to assess professional development related to reading instruction, CALPRO offers a six-month Evidence-based Reading Instruction (EBRI) Institute that is extremely comprehensive. Cohorts of practitioners continually assess their students’ ability in the four components of reading, and adjust instruction based on student data, using the new skills and concepts they have gained from the Institute. The Institute, as with other CALPRO Community of Practice professional development opportunities, encourages teachers to reflect on their practice, implement new skills and knowledge with their students, and then refine their practice based on how students respond. CALPRO continually refines its EBRI Institute based on the experiences of its practitioners enrolled in it, while staying true to the underlying research.

For professional development specific to the needs of adult learning, CALPRO’s Instructor Competencies Self-Assessment (ICSA) and Individual Professional Development (PD) Plan. Revised and updated in April 2016, the ICSA is based on a nationally validated, research-based set of Adult Education Teacher Competencies (AETC), which AIR developed for an OCTAE initiative. These 17 competencies span four distinct domains and identify the knowledge and skills that every adult educator needs to have as the foundation for effective instruction. Adult educators may take the ICSA, rate themselves in three areas, and receive results instantly. Their results are provided together with an individual PD plan containing numerous targeted resources specific to each competency, available through CALPRO, other State Leadership Projects and/or LINCS. Teachers are instructed to develop a professional development plan, implement the plan and re-assess their performance. Teachers can take the ICSA as many times as they wish and compare their results and plans over time, as their job priorities and skills change. Additionally, CALPRO has a professional development module and a research brief based on these same AETC competencies. As well, CALPRO offers training in Understanding the Adult Learner in both face-to-face and in two online formats. Finally, all CALPRO trainings are developed using principles of effective adult learning so that the instructors and administrators pursuing the professional development will be effectively engaged.

In working with paid personnel employed by WIOA-funded agencies, CALPRO’s professional development is intended to serve individual educators as well as a higher, program level. For example, the ICSA mentioned above can be taken voluntarily and accessed on CALPRO’s Web site. A database collects responses and allows individual teachers to go back to reexamine their results over time and look for teacher change. If “quality of professional development” is best determined by teacher change of practice, the ICSA and its PD plan encourages teachers to re-take the self-assessment as their students’ needs change and teachers’ priorities in professional development evolve. At the program level, every professional development offering is designed to include strong implementation emphasis, in which teachers reflect on their implementation and then refine their practice.  For CALPRO’s more extensive professional development offerings, such as the Professional Learning Communities Institute, the Leadership Institute for new and aspiring administrators, the Regional Communities of Practice, and the Training of Trainers Institute, there is a strong emphasis on understanding the impact of the professional development as adult educators implement their new knowledge and skills in their program and instructional contexts. CALPRO refines its institutes based on participants’ needs while still adhering to research-based practices.

As part of the 2020–23 WIOA, Title II: AEFLA request for application process, the CDE will require that agencies submit an annual progress report which will include a section related to professional development plans for their institution. The agency’s plan should identify PD goals for its teachers, based on students’ needs and agency priority initiatives; explain how the agency will accomplish these goals; and provide evidence of impact on teacher change and student learning.

CALPRO disseminates information promptly about effective models, research-based and promising practices through many vehicles, such as an annual comprehensive professional development module delivered through various means, including face-to- face, blended, and online formats; multiple formats of online professional learning; an annual research brief, and a web-based Research Archive. In all of its training, CALPRO seeks to understand how effective its professional development is by understanding how teachers’ practice changes. All CALPRO trainings encourage adult educators to reflect on their work and the concepts and skills acquired in the training and then apply it in their instructional contexts.

The CDE funds additional State Leadership Projects through the AEFLA grant: Comprehensive Adult Student Assessment Systems (CASAS), which provides assessment and data & accountability and Outreach and Technical Assistance Network (OTAN), which provides distance learning and instructional technology.

CASAS disseminates information through its annual CASAS Summer Institute, statewide regional network meetings covering all ten CDE regions, TOPSpro Enterprise network meetings. CASAS also provides targeted technical assistance to improve programs, data quality, and National Reporting System (NRS) performance on persistence, educational functioning levels (EFLs), and core performance indicators for entering and retaining employment, entering postsecondary education and training programs, and obtaining a HSD or a HSE. CASAS also provides a variety of online training sessions, statewide face-to-face trainings, and an online tool that presents California NRS adult learner data at the state and local agency levels, where agencies can compare local performance with state goals, other local agencies, and counties in AEFLA programs.

OTAN disseminates information through a multitude of face-to-face and online workshops, conference presentations, and by producing videos that demonstrate teaching with technology and technology integration lesson plans. All videos are archived on OTAN’s website. OTAN recently piloted a Community Model of Online Learning to increase regional access to high-quality online math curriculum for adult learners. The Online Teaching Academy (OTAC) assists instructors in becoming competent online teachers and mentors using Moodle and other instructional technology. It also hosts the Technology Integration Mentor Academy (TIMAC) training, a year-long professional development project where participants to become mentors and increase the effective use of technology in classrooms. The Technology and Distance Learning Symposium rotates each year between north and south geographic locations in the state.