U.S. flag

An official website of the United States government

Dot gov

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Https

Secure .gov websites use HTTPS
A lock () or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Located in:
  • III. Operational Planning Elements

    The Unified or Combined State Plan must include an Operational Planning Elements section that supports the State’s strategy and the system-wide vision described in Section II(c) above.  Unless otherwise noted, all Operational Planning Elements apply to Combined State Plan partner programs included in the plan as well as to core programs.  This section must include—

III. b. 4. A. Assessment of Core Programs

Describe how the core programs will be assessed each year based on State performance accountability measures described in section 116(b) of WIOA.  This State assessment must include the quality, effectiveness, and improvement of programs broken down by local area or provider.  Such state assessments should take into account local and regional planning goals.

Current Narrative:

Title I and Title III

CalJOBSSM provides a unified and streamlined intake and case management system that enables co-enrollment across programs, while reducing the duplication of services provided to AJCC clients. The system currently serves as California’s case management system and official system of record for federally required data for the following programs:

  • Title I Adult
  • Title I Dislocated Worker
  • Title I Youth
  • Title III Wagner-Peyser
  • Jobs for Veterans State Grant
  • Trade Adjustment Assistance
  • National Dislocated Worker Grant

To ensure compliance with the performance accountability measures outlined in WIOA Section 116, the EDD issued Workforce Services Directive WSD19-03, which provides performance guidance related for all programs using the CalJOBSSM system.

Additionally, Workforce Services Directive WSD19-06 provides AJCC staff with information regarding how to track services provided to individuals and employers in the CalJOBSSM system, and provides a link to the data element(s) each CalJOBSSM activity code maps to in the Participant Individual Record Layout.

To ensure the quarterly and annual participant data submitted to the DOL is accurate, the EDD issued Workforce Services Directive WSD18-02, which outlines the process AJCC staff must take to correct data in the CalJOBSSM system. The EDD informs all AJCC staff of the year-end reporting schedule to ensure all supplemental data is entered, and that performance data is reviewed prior to the annual report submission.

Title II

The CDE AEO provides assessment and accountability services for the Title II program through TOPSpro Enterprise. TOPSpro Enterprise is a database designed to accurately measure progress, mastery of skills, and competencies needed to both complete, and advance one or more Educational Functioning Levels.  It automates scoring, collects student demographic data, tracks agency and individual student performance, generates reports, and aggregates data for state and federal year-end reports.

Features of the data system and relevant processes include the following:

  • TOPSpro Enterprise is used to collect and report all student progress and outcome measures, and for collecting information for federal and state annual reporting.
  • The system provides student, class, and program reports that enable local providers to have immediate access to the data for targeting instruction for continuous program improvement.
  • The local data is submitted quarterly and annually to the CDE for monitoring and aggregation into state and federal reports.
  • TOPSpro Enterprise records each student’s goals on entering a class, as well as their educational outcomes.
  • Assessment may be formal (e.g., a written test), or informal (e.g., teacher observation of student performance through a check competencies mastered).

The data collected consists of measurable skill gains in the following programs areas: English Language Acquisition, Adult Secondary Education, and Adult Basic Education. The data collection process begins with program staff at agencies funded by the AEFLA inputting the data on a daily basis at each site during the program year. Each week the data collected from AEFLA funded agencies is aggregated at a statewide level. The annual data aggregation and data validation begins August 1st of each year. The purpose of the annual data aggregation and validation process is to compile state and federal year-end reports due annually, by October 1st.

Performance measures submitted annually on October 1st includes WIOA Title II indicators of Measurable Skill Gains, credential attainment, employment after 2nd and 4th quarter, and median earnings.

The CDE also conducts an annual comprehensive qualitative program survey. This survey is required of all participating agencies and involves practitioner focus groups and interviews of both teachers and students. The results provide recommendations for state level planning and development activities, identify best practices and emerging needs, and helps focus professional development and training to ensure effective instructional programs for targeted populations. The results of the survey also assist in the development of the annual narrative submitted on December 31st.

Title IV

The DOR utilizes a case management system known as AWARE. In addition to WIOA data reporting, the AWARE system has a financial component utilized for federal reporting requirements. The system contains consumer data, case notes, and information regarding goods/services for consumers.

Data are collected and inputted in-house by VR staff located statewide in 13 geographic districts. The DOR continues to train staff on the current processes, and new processes as needed, and use AWARE to collect WIOA data. State VR systems/agencies collect and report summary data in a federally mandated format called the RSA Case Service Report. The RSA-911 report is submitted quarterly for the preceding quarter of the fiscal year by each state’s vocational rehabilitation agency.

The RSA-911 report contains a record for each participant, including VR applicants and students with disabilities receiving pre-employment transition services. It is feasible that a participant may appear more than once in one year, if their case was opened on two or more separate occasions. The federal RSA-911 report aggregates many variables of outcomes-related information, including demographics, disability, interventions, and reason for closure, employment status, sources of financial support, and more. The values of certain fields (e.g. income, hours worked per week, etc.) are reported both status at application and status at closure.

Additionally, RSA-911 quarterly data is consolidated into the annual WIOA Statewide and Local Performance Report (ETA 9169), submitted by October 1 of each year to RSA; this annual report includes five additional, computed data elements related to program performance measures and expenditures for that preceding federal program year.

The RSA assesses the quality, and effectiveness of DOR outcome-related information in relation to the State performance accountability measures described in WIOA Section 116(b), and coordinates guidance and technical assistance to the department to initiate and monitor program improvements. The DOR also uses the information from the quarterly and annual reports internally to self-assess performance and develop strategic initiatives.

Regional Evaluation

The CWDB developed Regional Coordination and Alignment Indicators as a framework for assessing regional implementation progress in achieving the main objectives of the State Plan, which are demand-driven skills attainment, upward mobility and equity for individuals with barriers to employment, and system alignment.  As part of their Regional Plan modifications, each RPU provided a self-assessment of regional coordination using the following Regional Coordination and Alignment Indicators:

Demand Driven Skills Attainment Indicators

  • Indicator A: Region has a team that jointly convenes industry and is led by Industry Champions.
  • Indicator B: Region has shared industry sector focus and pools/shares resources to meet demand in the region.
  • Indicator C: Region has a process to communicate industry workforce needs to supply side partners.
  • Indicator D: Region has policies supporting equity and strives to improve job quality.

Upward Mobility and Equity for Individuals with Barriers to Employment Indicators

  • Indicator E: Region has shared target populations of emphasis.
  • Indicator F: Region deploys shared/pooled resources to provide services, training, and education to meet target population needs.
  • Indicator G: Region utilizes shared/common case management and capacity building strategies such as co-enrollment, professional development, to develop shared responsibility for providing services and ensure quality outcomes.

System Alignment Indicators

  • Indicator H: Region has shared/pooled administrative systems or processes to achieve administrative efficiencies and program outcomes.
  • Indicator I: Region has created formalized structures for decision-making.
  • Indicator J: Region has developed a process for evaluating performance that includes, but may not be limited to:
  • Qualitatively evaluating progress towards meeting regional industry and occupational demand.
  • Tracking the number of industry-recognized credentials and apprenticeships.
  • Aligning negotiated performance measures to regional Indicators.
  • Using the Regional Coordination and Alignment Indicators to evaluate progress.

Each RPU determined how it scored on each of the indicators and provided a narrative rationale to justify the scare as part of the self-assessment. For each of the indicators, RPUs could select from the following four options:

  • No progress made a this time
  • Learning/Experimenting
  • Operationalizing/Doing
  • Growing/Expanding/Scaling

The purpose of the self-assessment was to determine progress made to date within each RPU and to assess in aggregate all RPU efforts that are currently underway. The initial self-assessment will be used as a baseline for subsequent assessments throughout the duration of Regional Plan implementation.