WIOA State Plan Common Elements

Requested Revision (this text has been altered since original submission) The State’s Approach to Program Data Integration and Interoperability (Corresponding to Section III (b)(6) of the planning guidance) California is dedicated to developing a roadmap towards greater data integration and interoperability and is researching centralized and federated methods to track, share, manage and report performance data over the medium term but is doing this in a manner that appreciates the complexity of the task at hand. As pointed out in chapter 2, California provides workforce and education services through a myriad of largely decentralized service delivery structures, including 11,000 K-12 schools, over 1,000 school districts, 1,000 charter schools, 113 community colleges (in 72 community college districts), 58 County Welfare Departments, 58 County Offices of Education, 47 Local Boards overseeing 190 One-Stops, and more than half a dozen state departments and agencies. Not all programs operating in this largely decentralized service delivery network have the same eligibility requirements, data needs, or program goals. Some of the relevant data systems are operated by local government. Some are operated by state government. All have existing case management, data-collection, and reporting legacy systems which have been designed with both their program specific needs and their client population characteristics in mind. Moreover, all these programs have existing relationships with vendors and many are party to legally binding contracts for the provision of case management, data collecting, and reporting services. Furthermore, in the absence of a federal mandate for states’ to have a comprehensive integrated intake and case management system, efforts toward interoperability among State Plan partners will necessarily rely on voluntary participation which will depend on partners seeing value for their programs in joining in such collective efforts. Given this context, the state is approaching the matter of data-sharing and the building of interoperable data systems with the following principles in mind: • Form meets function. The technological architecture for interoperable data-systems should be crafted to serve the policy objectives of the programs they are designed for and should not unduly constrain or predetermine the policy choices of program administrators and operators in a way that limits the capacity for policy innovation. • Data-sharing and data integration efforts make the most sense where there is a commonality of interest, need, or purpose and a set of shared goals. Current and future efforts to develop data-sharing agreements or, where appropriate, move towards data-integration will proceed on the basis of value-added partnership such that all partners gain something from the partnership, such as the development of an ad hoc committee comprised of Title I and III local operators and users of third party systems and the state level case management system (CalJOBS) to make informed decisions and to share information about how to move toward the possibility of further integration. • Agreements will need to recognize and take into account the varied needs of different programs and client populations, the varying privacy requirements of different programs, recognition of data-ownership by program operators, and the need to work collaboratively to craft shared solutions that serve both the programs being operated, and more importantly, the members of the public receiving services. • Any data-sharing and data integration will be developed in order to meet state and federal privacy and security standards as well as those of each participating agency.

Operating from the foregoing policy perspective, the State Board has created the “Data-Sharing and Performance Accountability Workgroup” to assist the Governor in aligning technology across core programs and One-Stop mandatory partners with the goal of improving service delivery to individuals.

Representatives from all WIOA core programs, the community college system, K-12 education, the ETP, and DSS (TANF) have all participated in the workgroup, with representatives meeting with State Board staff more than a twenty times either collectively or program to program. To date, the workgroup has done all of the following: • exchanged information about common data elements that support assessment and evaluation • exchanged information about data systems in-use and extant performance reporting processes • shared information on WIOA performance metrics, reporting requirements, regulations, and guidelines • shared information on other performance reporting requirements in state law including: o SB 1402 (T. Lieu, Chapter 361, Statutes 2012) which requires performance reporting for CCCCO Economic and Workforce Development grants operating under the DWM framework; and o AB 2148 (K. Mullin, Chapter 385, Statutes of 2014) which requires the creation of an Internet-based, annual workforce metrics dashboard that includes information on participant outcomes from community college career technical education, ETP programs, WIA and WIOA Title I Adult, Youth, Dislocated Workers, and Title II Adult Education programs, as well as Trade Adjustment Assistance, and state apprenticeship programs. • discussed challenges to implementing WIOA performance requirements • reviewed approaches to data sharing and workforce performance reporting systems, (including dashboards) in other jurisdictions • developed a forum to explore ways to integrate data systems to facilitate intake and service delivery and to track participation and performance across programs State plan partners are now beginning to map a way forward and have started to express their preferred approaches to achieving WIOA’s vision for achieving data integration and interoperability, including alternative centralized, federated, and hybrid approaches. In the coming months, the State Board will continue to convene this workgroup to plot a way forward using the principles outlined above with the goal of developing a data-sharing governance structure that facilitates compliance with federal requirements. As part of this process, the State Board is working with EDD, SBE/CDE, DOR, CCCCO and other state plan partners to develop an agreement that does the following: • secures access to, and ensures the use of, the base wage file for use in all performance reporting relevant to WIOA employment and wage performance metrics as required under proposed regulations o Core program partners have been made aware that use of the base wage file is required under the federal regulations. o Each core program will need to negotiate a data-sharing agreement with EDD to make proper use of the base wage file. • secures access to, and ensures the use of, data on credential attainment, skills gains, and degree and credential completion for use in all performance reporting relevant to human capital investment • ensures all relevant state and federal privacy requirements are met, including HIPPA and FERPA requirements, and all other relevant state and federal laws • lays the groundwork for developing, where appropriate, common intake processes and integrated or interoperable performance reporting systems where agreements are consistent with the principles outlined above. Requested Revision continued Since the submission of the State Plan in March of 2016, continued efforts in this area have included the following: • The State Board’s continued discussion with WIOA Implementation Workgroup partner agencies about how to best convene, facilitate, and broker a comprehensive data-sharing agreement that leads to an integrated and interoperable data system, both on the front end (common intake and case management) and on the back end (federated data software to produce state and federal performance reports as well as facilitate customized program evaluation using net-impact analysis). The principals of this Workgroup will meet in early June to review work conducted by State Board staff in this area. • The beginnings of a formalized stakeholder process for identifying data systems, data sharing opportunities, and solutions for developing a flexible governance structure through the operation of a federated data model. State Board staff have been briefing all the State Plan partners and various other State Government entities, including Finance and the Legislature on possible ways forward and are preparing formal MOUs to fund staffing to develop a negotiated data governance structure that will serve at the foundation of the state’s approach to integration.

o WIOA partners are expected to contribute staff with policy, performance, information technology, legal and fiscal expertise to help discuss and troubleshoot the development of an integrated and interoperable data system. o Discussion with the California Department of Technology is on-going to scope feasibility of developing a federated state level data sharing solution. The State Board will likely enter into a project management contract with CalTech to shepherd the data-sharing stakeholder process. Additionally, the State Board is working to build greater capacity for cross-system assessment of the education and workforce systems through its participation with partners EDD, CCCCO, and SBE/CDE in the State Workforce and Education Alignment Project (SWEAP) funded by the National Skills Coalition in order to receive technical assistance for the development of data tools that gives state partners and policymakers better data to assess the extent to which relevant workforce and education programs are having measurable labor market impacts for those receiving services. Data tools being developed through this project include the following: • the cross-system metrics dashboard required by AB2148 which includes WIOA title I and Title II, ETP incumbent work training, state-approved apprenticeships, community college career technical education, and Trade Adjustment Act • the community college CTE evaluation data system “LaunchBoard” which may provide the platform for a career pathway evaluator data tool that examines whether people with varying degrees of needs have access to the right programs and range of services to earn credentials and/or move into jobs • supply and demand reports required annually by SB 118 (T. Lieu, Chapter 361, Statutes 2012) that compare numbers of community college program completers to number of jobs openings in in-demand occupations To assess the quality, effectiveness and improvement of the core programs, the State Board will continue to meet with the state plan partners on a regular basis to ensure continuing collaboration and communication, overcome competing or inconsistent priorities, and check in on progress towards meeting goals.

Update on Data-Sharing Efforts

Federal and state law direct the California Workforce Development Board (CWDB) to develop a Workforce Innovation and Opportunity Act (WIOA) State Plan. Federal law also directs the CWDB to develop strategies for technological improvements to facilitate access to and improve the quality of services and activities provided through the workforce system. Federal law also directs the CWDB to develop strategies for aligning technology and data systems across WIOA partner programs, which include federally funded adult education, vocational rehabilitation, and job and career services programs operated by various state and local government entities. Assembly Bill 2148 (K. Mullin, Chapter 385, Statutes of 2014) similarly directs the CWDB to create an Internet-based, annual workforce metrics dashboard that includes information on participant outcomes and labor market impacts from community college career technical education, Employment Training Panel (ETP) programs, Workforce Investment Act (WIA) and WIOA Title I Adult, Youth, Dislocated Workers, and Title II Adult Education Programs, as well as Trade Adjustment Assistance, and state apprenticeship programs. Federal law also directs the State to conduct program assessment and evaluation of relevant workforce programs and to invest WIOA discretionary funds for this purpose.

Under the WIOA State Plan required by federal law the CWDB is working with State Plan partners to address the foregoing statutory mandates. Since the submission of the State Plan in 2016, the California Workforce Development Board has been working with its State Plan partners on a data-sharing pilot project.

The Cross-System Analytics and Assessment for Learning and Skills Attainment (CAAL-SKILLS) data initiative is an interagency and multi-departmental effort to pool participant and program performance data across workforce, education, and human service programs and funding streams. CAAL-SKILLS will use common performance measures to examine participating program outcomes by region, provider, service, demographics, and industry. The project will develop the capacity to evaluate and assess participating programs efficacy, allowing program administrators and policymakers access to actionable data so that programs can be designed to improve program participant outcomes. CAAL-SKILLS is intended to meet the statutory requirements of AB 2148 (K. Mullin, Chapter 385, Statutes of 2014) and AB 1336 (K. Mullin, Chapter 211, Statutes of 2017) and WIOA 116(e) evaluation and assessment requirements.

Participating departments include CWDB, DSS, ETP, DIR-DAS, EDD, CCCCO, CDE, SBE and DOR. The following information provides an overview of progress to date.:

November 2016 - January 2017

1. Orientation meeting with the Governor’s Office

2. Project ‘kick-off’ meeting with the Executive Steering Committee

3. Steering Committee Members assigned Program Leaders to serve as the single-point-of project contact and representative of their organization

4. Orientation Meeting with the Department of Finance

5. Orientation Meetings with Program Leaders

February - April 2017

1. Project Charter was approved by the Executive Steering Committee

2. Established the CWDB Project Team, including: Technical Project Manager (MOU with the Department of Technology), Chief Technical Architect (Consultant), Senior Technical Analyst (Consultant), and Technical Analyst (Employee: New Hire)

3. Selected vendor to complete the Preliminary Assessment of Data Availability and Data Quality

4. Refined the Project Participation Agreement (A.K.A. Interagency Agreement) based on CAAL-Skills partner feedback

5. Completed the following activities associated with the Analysis of Data Availability and Data Quality:

   1. Drafted Analysis of CAAL-Skills Data Requirements

   2. Completed Data Availability and Quality Analysis Phase Kick-off meetings with DSP

   3. Distributed the Data Availability and Quality Analysis Questionnaire to DSP

   4. Provided Data Availability and Quality Analysis Questionnaire overviews to DSP

   5. Submitted Data Availability and Quality Analysis questionnaires to Data Sharing Partners

   6. Partners completed questionnaires and completed review of questionnaires and prepared for partner interviews.

6. Completed the Project Management Plan and associated sub-plans

7. Key Technical Documents have been written, including the:

   1. Technical Requirements Analysis

   2. General Technical Plan

   3. CAAL-Skills Pilot Technology Alternatives Analysis

   4. CAAL-Skills Pilot Hosting

   5. Location Alternatives Analysis

8. Received Agency Information Officer and Information Security Officer support for the recommended Pilot Hosting and Technology Alternatives.

9. Completed Requirements Analysis for AB 2148 Dashboard (Version 2.0)

10. Met with Steering Committee Members (individually) to preview and seek their support of the Pilot Technology and Hosting Environment Alternatives.

11. Received Steering Committee direction/decision on the CAAL-Skills Pilot Technology and Hosting Alternatives.

12. Completed procurement documentation (for equipment, software and services).

May - July 2017

1. Completed procurements and received CAAL-Skills equipment (server, UPS and desktop)

2. Completed software installation and configuration

3. Drafted the Detailed Technical Plan

4. Completed the Data Availability and Quality Analysis

5. Completed the draft Data Sharing Agreement

6. Completed analysis of data available from pilot counties

7. Completed draft Concept of Operations

8. Completed Labor Agency Legal review of the Data Sharing Agreement

9. Created Data Requirements Specifications for Data Sharing Partners

August - October 2017

1. Completed Data Requirement Specification review meetings with Partners

2. Received Partner cost and schedule estimates to fulfill the requirements in the Data Requirement Specification

3. Completed CAAL-Skills orientation with the California Department of Corrections and Rehabilitation

4. Managed agreement with EDD Internal Partners (Contracts & Fiscal) on CAAL-Skills Data File Preparation agreement for EDD WSD, LMID, UI-Tax and ETP.

5. Updated Interagency (Partnership) Agreements and received internal approval of updates to include cost and schedule for Data Formatting and File Preparation, for partners including: CCCCO, DSS, DOR and DIR-DAS.

6. Received internal approval of agreement with CASAS to create data file for CDE.

7. Completed Draft Technology Recovery Plan

8. Conducted Data Sharing Agreement orientation meeting with the partner Attorneys

9. Received list of statutory constraints from pilot partner Attorneys.

10. Began Market Research

November 2017 - January 2018

1. The following project and program management plans are complete:

   1. Technology Recovery Plan

   2. System Security Plan

   3. Human Resources Management Plan

   4. Data Governance Plan

   5. Quality Management Plan

   6. Change Management Plan

   7. Concept of Operations

   8. Program Practice Documents

2. Successfully completed the Information Security & Privacy Assessment of CAAL-Skills

3. Received IT Certification and Accreditation approval from the Information Security and Privacy Office

4. Interagency Agreement Amendments or other agreements, to complete the work of preparing the CAAL-Skills Data File, have been completed for the following: CCCCO, ETP, DOR, EDD (Workforce Services, UI Tax & LMID), DIR-DAS and CDE (All partners except DSS).

5. The Data Sharing Agreement has been established for the following: CCCCO, ETP, DOR, EDD (Workforce Services, UI Tax & LMID) and DIR-DAS (All partners except DSS and CDE).

6. Test Data Files have been received from CCCCO, ETP and DOR

   Next Steps & Planned Activities:

1. Receive Test Data Set from:

   1. EDD - Workforce Services

   2. EDD - LMID

   3. EDD - UI-Tax

   4. DSS for Pilot Counties

   5. DSS Statewide and CDE

2. Validate partner test data files

3. Request and receive production data file from partners

4. Complete the Data Sharing Agreement with CDE and DSS

5. Confirm DSS-preferred contracting approach for Pilot-County data

6. Determine level of CDCR level of engagement for the CAAL-Skills Pilot

7. Complete Technology Practice Documents

8. Complete Staff Management Plan

9. Refine the Concept of Operations and Practice Documents as needed

10. Complete Market Research to identify what other States doing and assess technology alternatives