Adult, Dislocated Worker, and Youth Activities

Governor’s Set Aside Funding: Wisconsin expects to utilize 15% of the Title I set aside funds for the administrative purposes (as allowed under WIOA Sections 128 and 133) and the remaining portion for both: required statewide activities and allowed statewide activities that support the strategic goals set forth in this four-year WIOA Combined State Plan. The state will designate specific initiatives for priority spending based on regional sector strategy priorities and partnership initiatives with local and state agencies. Wisconsin will also look to invest resources in initiatives which may include the following:

Work-Based learning initiatives with a focus on achieving wage increases for workers. These initiatives will be evaluated by measuring associations between the specific initiative and measurable skills gains, credential attainment, and employment outcomes (rates and wages). The focus is to determine which work-based learning initiatives are most strongly associated with good jobs and/or career advancement opportunities.

Apprenticeship strategies for both youth and registered programs to enhance current state efforts including expanding the bridging between the two programs. Evaluation starts with quarterly tracking of co-enrollment between the two Apprenticeship Programs (Youth Apprenticeship and Registered Apprenticeship) and WIOA Title I Programs (Adult, Youth, and Dislocated Worker).  Results are used operationally and for research purposes. Operationally, co-enrollment statistics along with identification of workforce development areas (WDA) with high and low co-enrollments will be used to set co-enrollment targets. Research staff will further investigate the areas with high co-enrollment to identify and share strategies that may increase participation in apprenticeship opportunities. Deeper analysis will investigate the diversity of these co-enrollments by both demographics and employment influencers. 

Coordinated business services. Both qualitative and quantitative data will be collected for coordinated business services evaluation. Qualitative data—regarding perceptions of how well coordinated business services are meeting the needs of the local economy—will be collected from relevant stakeholders (for example, local Workforce Development Boards (WDB), local economists, local businesses, Chambers of Commerce). Specific emphasis will be placed on declining industries and high-demand industries (including those in Hot Jobs Projections on WisConomy—developed by DWD's Bureau of Workforce Information and Technical Support (BWITS) staff; Science, Technology, Engineering, Art, and Mathematics (STEAM); Artificial Intelligence (AI); and generative AI).

Quantitative data will compare the distribution of industries of the businesses served with the distribution of industries in the local economy. The analysis is multi-part. First, it will compare the types of industries that participants in different WIOA programs find and retain jobs. It will then compare the distribution of participants in each industry to the distribution of the industries of the businesses served. Finally, measures of job retention, repeat business customers, opportunities for good jobs, provision of WIOA funded work-based training, industry representation on the Governor's Council on Workforce Investment (CWI) and local WDBs, and types of business services used will also be considered.

Associations between these measures, usage of coordinated business services, and employment outcomes—combined with the other quantitative and qualitative information—will provide feedback for strategic planning and direction for continuous improvement of coordinated business services. For example, if one industry is under-represented in the businesses served, targeted outreach to that industry could be employed.

Expanded partnerships with the Wisconsin Department of Corrections (DOC) to increase job center efforts inside of prisons, support instructional needs for inmates, and to seek innovative ways to provide training to incarcerated clients. Evaluation includes tracking the number of programs inside prisons and the number of individuals in each facility using these programs. Further, the employment outcomes will be measured post-exit and evaluated against the training services, service delivery method, and timeline of service provision. These outcomes will be compared against similar individuals who did not participate in the programs. A feedback loop with local employers, trainers, staff, and participants (before, during, and after program participation) will be established to identify instructional needs that are based on the participants' interests, skills, abilities, the local job market, and workforce needs. Findings will be used to create innovative solutions to better provide training to future participants.

As part of a continuous improvement process, these three evaluations (participation counts, comparing employment outcomes to a comparable population, and customer feedback) will be repeated regularly to determine if innovated solutions improve employment and training service delivery. This will guide further innovation to help improve employment services for justice-involved individuals.

Identifying and responding to skill gap/job quality issues to address workforce development needs. Evaluation is two-fold. First, evaluation efforts seek to identify and respond to skill gap/job-quality issues. Second, the process is then evaluated to determine if it is addressing workforce development needs. Identification and responding to skill gap/job quality issues includes: (1) analysis of resumes and job postings on Job Center of Wisconsin and (2) interviewing job seekers, employers, local workforce boards, and trainers. Program staff will use findings along with Hot Jobs Projections on WisConomy to identify skills gaps and provide insight into whether the workforce development needs are due to the skill gaps, lack of good jobs, or both.

When skill gaps need to be addressed, program staff will then work with local WDBs, employers, job seekers, and trainers to ensure WIOA Title I participants have access to trainings to develop skills for good jobs that are high demand and local. These trainings will be assessed to determine if they lead to good jobs. When the lack of good jobs needs to be addressed, program staff will work with local WDBs, employers, and job seekers to identify strategies to improve job quality to meet the good jobs criteria.

Measuring changes in the duration job postings are open on Job Center of Wisconsin for each local area, coupled with surveys to local employers and job seekers will provide a feedback loop to inform whether this process is effective or if additional efforts are required to improve the evaluation process of identifying and responding to skill gaps/job quality. 

Services to special populations and groups, including homeless populations, foster care clientele, opioid addicted clientele, and targeted barriered military veterans and their spouses. This may include pilot programming. Evaluation will first assess whether targeted outreach efforts increase services to the identified special population. It will then consider whether services targeted for these special populations are associated with either similar or improved employment and training outcomes relative to a comparable group of participants that received standard Title I Adult, Dislocated Worker, and Youth services. Qualitative feedback on pilot programs will be solicited from program participants and frontline staff to provide a feedback loop for customer-centered improvements to pilot programs.

Expanded use of artificial intelligence (AI) and/or virtual reality (VR) in the provision of services to job seekers and employers. Evaluation includes both quantitative and qualitative methods. Qualitative data from job seekers and employers will be collected regarding perceptions of the technology, particularly related to service provision and skills gained (when appropriate). Employment outcomes and time in program will be measured and analyzed relative to a comparable population. Together, this information will be used to provide a regular feedback loop to continuously improve the use of these technologies. Further, evaluations will assess if AI and/or VR technology improves training outcomes, employment rates, and/or wages.

Addressing workforce needs for advancing green/renewable energy. Evaluation will track training needs required by employers to address skills gaps in these areas. Trainings implemented will be evaluated by the employment outcomes of their participants and the employers' assessment of the sufficiency of the training.

Addressing workforce needs for broadband expansion and other infrastructure development. Related to broadband access, we will assess the broadband and data literacy needs of job seekers, workers, and employers. Based on the assessment, efforts will expand broadband access, provide data literacy training and technical assistance, and/or implement other strategies that address job seekers' barriers to connecting to good jobs due to limits in broadband accessibility. We will assess if implemented strategies improve employment outcomes for job seekers who reported issues in access to broadband technology.

Related to addressing workforce needs to develop the infrastructure to provide broadband access and digital literacy training, evaluation will track training needs required by employers to address skills gaps in these areas. Trainings implemented will be evaluated by the employment outcomes of their participants and the employers' assessment of the sufficiency of the training.

Increase understanding of impacts of regional sector strategy priorities and partnership initiatives. Identify difficulties in data sharing for the purpose of better assessing regional sector strategies. DWD will evaluate overall data sharing processes: between divisions within the Department, as well as external WIOA partners. Based on findings, WIOA partners may create a Statewide Data Governance team to better manage data at the regional and state level.

Overall program evaluation of the Title I Adult, Dislocated Worker, and Youth activities will focus on indicators of diversity, inclusion, and equity (such as demographics, employment influencers, and program outcomes) by comparing program participants to a comparable population within the labor force. Associations between programs/services and employment outcomes will also be analyzed. The aim will be to determine what programs/services have the most influence on improved measurable skills gains, credential attainment, and employment outcomes (rates and wages), with a specific focus on identifying which programs lead to good jobs and/or career advancement opportunities. Further analysis will be conducted determine if improved employment outcomes lead to family-sustaining wages.

To incorporate continuous improvement, enrollment counts by demographics and employment influencers (for example, homeless, foster care, opioid addicted, military veterans and their spouses, etc.) will be evaluated so program staff can identify when there are discrepancies between the population served and the comparable population in the labor market. Additionally, targeted outreach efforts to specific populations will be evaluated to understand the outreach methods' effectiveness in increasing services to the identified population and to provide feedback useful in determining whether the method could be applied to another specified population. Operational staff will use evidence-based technical assistance from the Department of Workforce Development (DWD)'s Workforce Data Integration System (WDIS) team and findings from the Worker Connection evaluation to help inform and implement these metrics and a continuous improvement process with the aim to better reach, engage, and serve marginalized populations.

DWD WDIS establishes a Learning Agenda every four years. This is a strategic plan for evidence building. The learning agenda includes research goals evaluating all its service delivery programs, including Title I Adult, Dislocated Worker, and Youth programs. Relevant to Title I programs, the learning agenda questions investigate:

• The impact of individual program services (and their interactions) on employment outcomes.
• The impact of program staff training on employment outcomes.
• Quality of service delivery and customer service and the influence of technology.
• Cost-effectiveness of programs.
• How to use local labor market information to improve service delivery.
• Establishing feedback staff and customer feedback loops on the accessibility and engagement in services.
• Effectiveness of targeted outreach for program services.
• Diversity of the program staff and participants relative to local labor market conditions.
• The impact of diversity within program participation and employment outcomes.

To address the learning questions more specifically, each year an Evidence Plan is established. The Evidence Plan develops an operational structure to address each year's priorities from the Learning Agenda by providing a plan for what data and methodology will be used to address specific sub-questions related to the broad questions on the Learning Agenda. Challenges and a dissemination strategy are outlined as well. Results from these evaluations are used to drive evidence-based decision-making regarding program policy, training of service providers, use of funding, continuous improvement, and for the provision of technical assistance when performance measurements are inadequate.

Altogether, the learning agenda and evidence plan provide a means to not only measure and monitor performance of the Title I Adult, Dislocated Worker, and Youth programs, but also build evidence to improve service delivery, program efficiencies, and improve program outcomes.