WIOA State Plan Common Elements

STATE EVALUATION PROJECTS 

Legislative Services Agency (LSA) Reviews. As discussed in 4.A., Indiana’s LSA conducts an annual review of workforce programs on a rotating five-year cycle. Although the 2023 report focused on non-core programs, the LSA reviews cover each WIOA core program at least once every five years. For example, Wagner Peyser and WIOA Youth were reviewed in 2021; Adult, Dislocated Worker, and Vocational Rehabilitation were reviewed in 2020; and Adult Education reviewed in 2019. These reviews offer insight into each workforce program that can be utilized by all stakeholders to ensure success of each program and can be accessed at: https://iga.in.gov/publications/2023/agency-reports.  

DWD EVALUATION PROJECTS 

Evaluation Peer Learning Cohort. In 2023, Indiana was selected to participate in the 2023-2024 U.S. Department of Labor Evaluation Peer Learning Cohort (EvalPLC). Through this initiative, Indiana learned from peer states, experienced evaluators, and state and national subject matter experts. Indiana’s participation focused on capacity and foundation building for conducting formal evaluations of workforce programs. Indiana’s EvalPLC project team was designed to include a specific group of cross-divisional DWD and LWDA representatives with broad ranging WIOA experience touching all WIOA core programs. Individually, these representatives hold subject matter expertise in state and local strategy, policy, program operations, service delivery, data science, performance and quality, and partner collaboration. Experience with this project will allow DWD to build and solidify a framework for continued collaboration in these areas, better positioning DWD to design, initiate, and complete formal evaluation projects. 

WIOA Title I Evaluations. Following the completion of the EvalPLC work, DWD will identify opportunities to conduct evaluations that are focused on various program strategies identified throughout this Plan. To better inform these strategies, DWD will conduct Title I evaluations in coordination with local workforce boards and other state agencies, where applicable. In prior years, DWD has focused evaluation efforts primarily on training services and employment. We are looking at opportunities to both expand on this work and also consider evaluation in other areas such as barriers to employment; effectiveness in service delivery; and diversity, equity, inclusion, and accessibility (DEIA).

DWD anticipates proposing an extension of the QUEST Dislocated Worker Grant (DWG) to use funds for an evaluation of the grant’s various strategies, which may include an analysis of the benchmark “payment point” model. The model was utilized as an alternative funding strategy to disburse grant funds with an aim to increase participation and co-enrollment across Title I (standard and QUEST) and Title II. The proposed evaluation may also include an implementation review of PIVOT, formerly called the Workforce Recommendation Engine, which aims to reduce duplication of services and make coordinated recommendations to UI claimants through technology and data.

Wagner Peyser (WP) Service Delivery Model Analysis. DWD is also focusing efforts on evaluating its Wagner Peyser program as we work to implement the Final Rule. Beyond assessing compliance against any new requirements, we are also looking to enhance our WP service delivery model overall. In early 2024, DWD initiated work with the National Association of State Workforce Agencies (NASWA)’s Workforce Information Technology Support Center (WITSC) to assist with assessing Indiana’s WP service delivery model.* The project includes conducting a current-state assessment and gap analysis, with the end deliverable being recommendations for evolving DWD’s service delivery model. The goal of this work is to ensure a customer centric model with services delivered in the most effective and efficient manner.

Note: This project also involves analysis of the Reemployment Services and Eligibility Assessment (RESEA) program, which is undergoing a concurrent formal Evaluation pursuant to RESEA requirements.

Title I Equal Opportunity and Nondiscrimination Data Analysis. Indiana’s State Equal Opportunity (EO) Officer participates in DWD’s WIOA Title I annual compliance reviews. The State EO Officer conducts annual compliance monitoring of each LWDA’s progress in implementing and administering WIOA’s Equal Opportunity and Nondiscrimination provisions, part of which includes an analysis of data and subsequent investigation, as necessary, of significant differences in populations receiving services. DWD conducts analysis of the standard deviation of U.S. Census workforce data compared to the population of WIOA enrolled participants the LWDA served in the prior program year.  

The purpose of this analysis is to identify if the LWDA is serving protected groups in its population to the level expected given the demographics of the communities it serves. The expectation is that the LWDA will investigate if the analysis identifies a large difference between numbers of people accessing WIOA Individualized Career Services (ICS) or Training Services (TS) in the LWDA’s WorkOne offices, and numbers in the community.  

The formula used to conduct this EO data analysis identifies the standard deviation (SD) which is a measurement for how likely an outcome is due to random chance. The State EO Officer would expect the breakdown of individuals the LWDA serves to be representative of the breakdown of individuals in their communities. For the formula, a zero indicates the expected outcome. We can assume that small changes (or a small SD) are due to random chance and that large changes are due to outside factors, such as potential discrimination. For this analysis, if the SD is greater than +2.0, we can assume that there is something influencing this difference because there is a less than 5% probability that this difference is due to random chance. 

The State EO Officer’s data analysis process includes review, discussion, and comparison with each LWDA as to the results of both state analysis and any analysis conducted locally. The following information and table below show an example of DWD’s analysis for PY22 on a single local area. This LWDA’s local results showed a high standard deviation, or the older population (55+) potentially being underserved in WIOA services. Similarly, the State EO Officer’s analysis also showed the same category as having a high standard deviation or potentially being underserved.  

The left side of the chart provides the standard deviation between the region’s PY2022 WIOA participants with the region’s overall labor force as indicated by census estimates of employed and unemployed individuals. The right side of the chart compares the region’s PY2022 WIOA participants with census estimates of numbers of unemployed individuals in the region. For this analysis, if the SD is greater than +2.0, we can assume that there is something influencing this difference because there is a less than 5% probability that the difference is due to random chance. For more information on how this analysis was conducted and what standard deviation is, refer to the appendix.  

Key takeaways from the above chart:  

1. Black/African Americans participated in WIOA at a higher rate than expected when compared to both labor force and unemployment census data.  

2. Older populations (55+) participated in WIOA at a lesser rate than expected when compared to both labor force and unemployment census data.  

3. Individuals with disabilities participated in WIOA at a higher rate than expected when compared to both labor force and unemployment census data.  

Federal Measures Dashboard. As discussed throughout this section, DWD also maintains a Federal Quarterly Performance Measures dashboard that enables staff and workforce partners to quickly analyze years’ worth of Local Area Reports in one location. This can also be viewed on the DWD Performance Portal at https://www.in.gov/dwd/performance/. This dashboard provides support to DWD’s assessment and evaluation efforts. 

VOCATIONAL REHABILITATION SERVICES EVALUATION

The Indiana VR program has developed a series of Tableau dashboards that are available to VR leadership and field management level staff. The data provides up-to-date information to support evaluation of compliance with federal and program standards, as well as review of progress on priorities and initiatives. Data can be reviewed at a statewide, regional, and individual staff performance level. Some examples include evaluation of rapid engagement efforts through review of the length of time between application and eligibility, and between eligibility and plan development; status of order of selection releases including the percent of cases entering into service plans; evaluation of discretionary grant and model demonstration projects; participant service spending trends; customer satisfaction; outcome data; state plan goals; pre-ets trends; and a variety of other case statistics.

The VR program is conducting an evaluation of the Supported Employment Plus (SE+) model carried out through the Disability Innovation Fund Subminimum Wage to Competitive Integrated Employment model demonstration grant. This discretionary grant was awarded to Indiana in late 2022 by the Rehabilitation Services Administration. Indiana University, Indiana Institute on Disability and Community is serving as the contracted evaluator. The SE+ model establishes fidelity of supported employment services, and wraps around interventions of peer support services, family engagement, and enhanced benefits and work incentives counseling to individuals with disabilities pursuing competitive, integrated employment. VR is also collaborating with Mathematica who is serving as national SWTCIE evaluator under contract with RSA. VR provides regular reports of progress to the assigned RSA SWTCIE state liaison, along with regular updates to the Indiana Commission on Rehabilitation Services.

Additionally, VR is engaged with Public Consulting Group to evaluate provider performance incentive payments implemented in August 2023. The evaluation will examine the effectiveness of incentive payments in increasing the quality and quantity of employment outcomes for VR participants served through community rehabilitation programs. Outcome data regarding the incentive payments is shared with stakeholders approximately quarterly, including through presentation to the Commission on Rehabilitation services.

Finally, VR has also engaged PCG to support the development of the Comprehensive Statewide Needs Assessment. Survey instruments and focus groups have been developed and distributed to obtain feedback from individuals with disabilities, staff, and stakeholders regarding areas of improvement and service gaps. RSA-911 data will also be reviewed to assess VR performance over the last three years to identify areas of strength and areas that could be improved. The CSNA will be completed by January 2025 and will be shared with RSA. The Commission on Rehabilitation Services has provided input on the CSNA process and survey instruments and will continue to receive updates on the CSNA progress. They will also be provided with an opportunity to identify themes and priorities upon receipt and review of all data collected through surveys, focus groups, and RSA-911 data analysis.