(C) Previous Assessment Results

Beginning with the state plan modification in 2018, and for subsequent state plans and state plan modifications, provide the results of an assessment of the effectiveness of the core programs and other one-stop partner programs and Combined State Plan partner programs included in the plan during the preceding 2-year period. Described how the State is adapting its strategies based on these assessments.

NOTE: The original draft guidance released in August, 2015 requested assessment results from “the preceding 2-year period,” which Idaho’s working group interpreted as the two years preceding the implementation of the Combined State Plan. This guidance was later updated to clarify that the instructions were in reference to the 2018 modification. Idaho’s workforce partners chose to keep the original content that had already been developed, as it provided a historical context for program performance under prior legislation and may be helpful for future reference. This section will be updated for the two-year modification in 2018.

The following assessment results reflect the actual performance of Idaho’s workforce programs at the program level. The performance of each program is directly affected by, and thus reflects, the opportunities, barriers, strengths, and weaknesses identified in the analysis in Section (II) of this plan. As a result, these strategies are based on the factors that currently affect program performance. Therefore, implementing the strategies identified in Section (II) should inherently lead to improved performance outcomes.

For example, attracting and retaining qualified program staff should lead to higher levels of participation, retention, and outcomes for participants. Expanding services to rural communities may initially decrease certain aspects of program performance, as individuals in these communities have a higher likelihood of facing multiple barriers to employment and education. However, over the long-term, this strategy should improve outcomes as these communities build the capacity to support their residents through continued economic growth.

Title I-B - Youth, Adult, Dislocated Worker

The performance reports for the previous two program years for each of the youth, adult and dislocated worker programs are included below. Under the Workforce Investment Act, states were considered to have met the performance goals for Title IB if the actual results are within 80% of the negotiated goal.

Idaho’s performance results across Title IB programs indicate very high-quality programs. For state ranking data available for Program year 2013, Idaho ranks among the best performing states in the nation for many measures. The PY 2013 national ranking is listed along with the description for each of the performance measures.

Youth Program

The primary performance measures for youth under the Workforce Investment Act were:

1. Placement in Employment or Education of those who are not in post-secondary education or employment (including the military) at the date of participation. (2^nd in the nation)
2. Attainment of Degree or Certificate of those enrolled in education at date of participation or any point during the program. (15^th in the nation)
3. Literacy and Numeracy Gains of those out-of-school youth who are basic skills deficient. (44^th in the nation)

For PY 2013, the Youth goals were achieved with the exception of the Literacy and Numeracy Gains for out-of-school youth. After a concerted effort to encourage greater participation among out-of-school youth, this measurement increased significantly to exceed the negotiated performance goal for 2014.

Table 19: Previous Assessment Results for Title IB Youth Programs - WIA

The primary performance measures for youth under the Workforce Innovation and Opportunity Act were:

1. Placement in Employment or Education of those who are not in post-secondary education or employment (including the military) at the date of participation. (2^nd in the nation)
2. Attainment of Degree or Certificate of those enrolled in education at date of participation or any point during the program. (15^th in the nation)
3. Literacy and Numeracy Gains of those out-of-school youth who are basic skills deficient. (44^th in the nation)

The WIOA Youth program transitioned to 100% out-of-school youth in PY2015, which is a much harder population to serve. As a result, the performance measures have dropped. Idaho hopes to recalibrate the baseline for serving youth under WIOA.

Table 20: Previous Assessment Results for Title IB Youth - WIOA

Adult Worker Program

The primary performance measures for adults are:

1. Entered Employment Rate for those who are not employed at the date of participation. (2^nd)

2. Employment Retention Rate for those who are employed in the first quarter of the exit quarter. (14^th)

3. Average Earnings for those adult participants who are employed in the first, second, and third quarter after the exit quarter. (20^th)

Idaho achieved or exceeded the performance goals for its adult program for both PY 2013, PY 2014, and 2015.

Table 21: Previous Assessment Results for Title IB Adult Programs

The primary performance measures for adults under the Workforce Innovation and Opportunity Act are:

1. Percentage of program participants who are in unsubsidized employment during the second quarter after exit

2. Percentage of program participants who are in unsubsidized employment during the fourth quarter after exit

3. Median earnings of program participants who are in unsubsidized employment during the second quarter after exit from the program.

4. Percentage of program participants who obtain a recognized postsecondary credential, or a secondary school diploma or its recognized equivalent during participation in or within 1 year after exit from the program.

Table 22: Continued: Assessment Results for Title IB Adult Programs

Dislocated Worker Program

The primary performance measures for dislocated workers are the same as for adults.

1. Entered Employment Rate for those who are not employed at the date of participation. (9^th)

2. Employment Retention Rate for those who are employed in the first quarter of the exit quarter. (22^nd)

3. Average Earnings for those adult participants who are employed in the first, second, and third quarter after the exit quarter. (15^th)

Idaho achieved or exceeded the performance goals for its dislocated worker program for both PY 2013, PY 2014, and PY2015.

Table 23: Previous Assessment Results for Title IB Dislocated Worker Programs - WIA

The primary performance measures for dislocated workers under the Workforce Innovation and Opportunity Act are:

1. Percentage of program participants who are in unsubsidized employment during the second quarter after exit

2. Percentage of program participants who are in unsubsidized employment during the fourth quarter after exit

3. Median earnings of program participants who are in unsubsidized employment during the second quarter after exit from the program.

4. Percentage of program participants who obtain a recognized postsecondary credential, or a secondary school diploma or its recognized equivalent during participation in or within 1 year after exit from the program.

Table 24 - Previous Assessment Results for Title IB Dislocated Worker - WIOA

Title II - Adult Education and Family Literacy

The Title II performance accountability measures are based on the following primary indicators of performance described in section 116(b)(2)(A) of WIOA:

1. The percentage of program participants in unsubsidized employment in the 2nd quarter after exit;
2. The percentage of program participants in unsubsidized employment in the 4th quarter after exit;
3. Median earnings of program participants who are employed during the 2nd quarter after exit;
4. The percentage of program participants who obtain a recognized postsecondary credential, or a secondary school diploma or its recognized equivalent during participation in or within 1 year after exit;
5. The percentage of program participants achieving measurable skill gains; and
6. Effectiveness in serving employers

The Agency administering Title II Program (Idaho Career &Technical Education) is required to negotiate performance targets with the Office of Career, Technical, and Adult Education at the US Department of Education each year. The table below shows the target and actual performance for Program Year PY15 (July 1, 2015 - June 30, 2016), and PY16 (July 2016 - June 2017).

Table 25: Previous Assessment Results for Title II, Adult Education Programs

Measurable Skill Gain Assessment

The overall skill gain rate for Idaho was 38% for PY16, compared to 33% in PY15. Despite not meeting the PY16 Educational Functioning Level (EFL) performance goals, Idaho’s Adult Education programs did make significant improvements from PY15. The overall rate for measurable skill gain varied from 32% to 45% among local programs in PY16.

Two related trends emerged in the PY15 annual reports, which helped identify some significant causes in performance decline. These trends include a heightened level of staff turnover, and a significant decline in post-testing rates (defined as the percent of students who were pre-tested and who received a post-test, thus allowing us to report a skill gain). The post-testing rate declined in PY15. Generally, a decline in post-testing rates indicates an issue with retaining students. These two trends may have been related in that the decline in post-testing activity could have been the result of process interruptions caused by staff turnover. Another possible cause is the improvement in Idaho’s economy. As the economy improves, it becomes harder to retain students until post-testing, and becomes harder to retain teaching staff, who leave for better opportunities.

For those programs whose performance was unusually low (compared to prior years), the State required a Program Improvement Plan, which was implemented in the winter 2016-2017. The increase in EFL gains in PY16 likely resulted from program improvement efforts; including targeted professional development towards quality instruction, ongoing curriculum refinement, and improved student retention.

Post-Exit Outcomes

The post exit outcomeswere not included in Idaho’s PY16 reporting requirements, as it has been a year to establish baseline data for future performance. Title II targets and actual performance for PY15 post-exit outcomes are described in the following section:

Table 26 - Post-Exit Outcomes, Title II, Adult Education

Entered and Retained Employment

Idaho exceeded the “Enter Employment” target for PY15. The target was 41%, and actual performance was 42%. The program did not meet the “Retain Employment” target for PY15. The target was 69% and the program achieved 66%. Despite having not met the target of 69% in PY15, the actual performance of 66% still reflects a significant improvement of actual performance year-over-year.

Obtain GED/HSE

Idaho exceeded its target for the GED outcome in PY15, at 87%. This is a significant improvement over performance in PY14, which was 76%, as well as PY13, which was 82%. The low performance during the prior two years was largely attributed to the change from the 2002 Series GED to the 2014 Series GED. As predicted, performance in this indicator improved to pre-transition levels once programs were able to stabilize and normalize the changes from this transition.

Post-Secondary Education & Training

In PY15, 32% of Adult Education students who entered the program with a high school credential (or earned a high school equivalent while enrolled), went on to enroll in a post-secondary institution within 18 months of leaving the program. This goal surpassed prior performance under every year of the cohort-reporting model. It is worth noting that the portion of United-States educated students with a high school diploma or equivalent increased significantly statewide in PY15. Even while the overall statewide enrollment decreased, the number of students entering the program with a high school diploma or higher increased from 514 in PY14 to 699 in PY15. When taken into account with the increased performance in this indicator, the program served more students who qualified for the post-secondary indicator, and did a better job of transitioning those students into college.

Title III - Wagner-Peyser

The performance reports for the previous program years for the Wagner-Peyser program are included below. Under the Workforce Investment Act, Title III programs are considered to have met the performance goals if the actual results are within 80% of the negotiated goal. Idaho’s performance results indicate a very high quality Wagner-Peyser program.

The primary performance measures for the Wagner-Peyser program prior to WIOAare:

1. Entered Employment Rate for those who are not employed at the date of participation.

2. Employment Retention Rate for those who are employed in the first quarter after the exit quarter.

3. Average Earnings for those adult participants who are employed in the first, second, and third quarter after the exit quarter.

Idaho achieved or exceeded the negotiated performance goals for its employment services program for both PY 2013, PY 2014, and PY 2015. There is an upward trend for the employment measures, while the six month average earnings actually decreased.

Table 27: Previous Assessment Results for the Wagner Peyser Program

The primary performance measures for Wagner-Peyser under the Workforce Innovation and Opportunity Act are:

1. Percentage of program participants who are in unsubsidized employment during the second quarter after exit
2. Percentage of program participants who are in unsubsidized employment during the fourth quarter after exit
3. Median earnings of program participants who are in unsubsidized employment during the second quarter after exit from the program.

Table 28: Previous Assessment Results for WIOA Title III, Wagner Peyser Program

Title IV - Vocational Rehabilitation

Title IV Vocational Rehabilitation programs are using a phased-in approach to set levels of performance for all primary performance indicators under this Combined State Plan. RSA-TAC-18-01 released January 18, 2018 provides guidance on the requirements for these indicators for PYs 2018 and 2019, noting that the same required levels of performance for PYs 2016 and 2017 be used. The level of performance for PYs 2016 and 2017 were baseline therefore baseline levels of performance will continue to be collected for PYs 2018 and 2019 to be used in establishing initial levels of performance for future negotiation. As a result, no performance levels are being reported for the following Primary Performance Indicators:

• Employment in the 2^nd quarter
• Employment in the 4^th quarter
• Median earnings
• Credential attainment
• Measurable skill gains

OAA Title V - Senior Community Service Employment Program

SCSEP is measured by six core performance measures. Core indicators are subject to goal setting and corrective action. Goals are established annually by the Department of Labor. Performance level goals for each core indicator must be agreed upon the Department of Labor and the grantee before the start of each program year (PY). The Department will annually evaluate and make for public review, information on the performance of each grantee.

SCSEP Actual Performance

The SCSEP Six Core performance measures and a description of each are listed below.

• Community Service: The total number of hours of community service provided by participants divided by the number of hours of community service funded by the grant.
• Entered Employment: The number of participants who are employed divided by the number of participants who exit.
• Employment Retention: The number of participants who are employed divided by the number of participants who exit.
• Average Earnings: Total earnings in the second and third quarters after exit; divided by the number of employed participants who exited.
• Service Level: Total number of participants served divided by a grantee’s authorized number of positions.
• Service Most in Need: Average number of employment barriers per participant. Barriers include having a severe disability; frail; age 75 or older; meet the eligibility requirements related to age for, but do not receive, benefits under title II of the Social Security Act (42 U.S.C. 401 et seq.); live in an area with persistent unemployment; have limited English proficiency; have low literacy skills; reside in a rural area; veteran; have low employment prospects; have failed to find employment after utilizing services provided under title I of the Workforce Investment Act of 1998 (29 U.S.C. 2801 et seq.); or Are homeless or at risk for homelessness.

Table 29: Previous Assessment Results for the SCSEP Program (Senior Employment)

.

SCSEP Strategies to Improve Performance Measures:

Community service and Service Level: The U.S. Department of Labor determines every year how many positions will be available for the SCSEP on a national and state level. After negotiations and or funding changes, sometimes positions will get modified. The assessment results above show that the modified positions were changed in the middle of the third quarter of PY 2016 to reflect the awards made to national grantees by the 2016 competition and changes to Equitable Distribution based on the latest Census data. Fields in the Quarterly Performance Reports cannot be accurately reported for the third and fourth quarters of PY 2016 or for the final end-of-year QPR for PY 2016.

The entered employment measure increased from 32.1% to 39.1%. Through the use of specialized Participant Assistants and Employment Training Coordinators (ETC), the service provider Experience Works (EW) was able to identify specific job opportunities for participant, job goals for referrals, and job development activities.

The employment retention measure dropped from 53.8% to 36.4%. To correct this, EW is focusing on Workforce Re-Entry training, not only how to get a job, but also how to keep the job. EW is also coaching participants how to decide whether a given job and/or employer is the right fit for them vs. taking “any” job even if it seems apparent that it doesn’t completely match the participants employment plan.

The average earnings met the goal in PY 15 and PY16. SCSEP Participants provided 42,999 community service hours in PY 15 compared to the 40,040 of hours provided in PY16.

The most in need performance measure increased from 2.71% to 2.74% average employment barriers per participant. As positions open EW will prioritize participants with barriers to employment first. EW will also ensure that priority of service is given to veterans and ensure positions are filled in rural areas where employment opportunities are limited. By recruiting for participants through their relationships with social service agencies, EW has been moving towards meeting almost the target goal.

Trade Adjustment Assistance

The Trade Adjustment Assistance (TAA) program does not have state negotiated performance measures, but compares its performance measures to the national performance goals.

The Trade Adjustment Assistance program reports two sets of measures. Trade Act Measures are defined by the Trade Adjustment Assistance Act and Common Measures are uniform measures used by multiple federal workforce programs. The difference between the two sets of measures relates to the quarters in which the performance is tracked. Idaho WIOA workforce programs use common measures.

1. Entered Employment Rate for those obtaining employment in the first (Trade Act) or second (Common) quarter after exit.
2. Employment Retention Rate for the next two quarters after exit for the same individuals who obtained employment above.
3. Six-Months Average Earnings for the two quarters reported above.

The performance results for the previous four fiscal years for the Trade Adjustment Assistance program are compared to the national performance goals.

Idaho well exceeded the national goals for entered employment rate for all years. The retention rate was slightly lower in fiscal years 2014 and 2016.

Table 30: Previous Assessment Results for Trade Adjustment Program

Except for fiscal year 2013, Idaho’s six-month average earnings were below the national goals and the national results. However, it should be noted when comparing Idaho’s wages to the nation’s there is a significant difference between the wage markets. The annual mean wage is the United States for all occupations is $47,230 while the same wage in Idaho is $39,770. The difference between the national six-month average earnings goal and one-half of the average annual wage is 78%. In Idaho, TAA exiters earn 90% of the average Idaho annual wage. Idaho’s wage performance for the TAA program is proportionally greater than the expected results.

Increasing wages for the Idaho workforce is definitely a priority and directly corresponds to expanding services to rural communities.

Veterans’ Employment

2018 Modification Update:

Currently there are no performance measures. It is estimated new measures will be in place for PY 2018. This state plan will be updated when the new measures are made known.

2016 Original Submission for VETS follows:

USDOL Veterans’ Employment and Training Service (VETS) has established thirteen performance measures which focus on the effectiveness of the services delivered to veterans at two different levels of the workforce system: a) via staff funded through the Jobs for Veterans State Grants; and b) the general statewide level of labor exchange service to veterans provided by the One-Stop system.

The performance targets are very similar to those under WIA for Title IB programs (Adult, Dislocated and Youth) and Title III (Wagner-Peyser Employment Services), such as: entered employment, employment retention and average earnings at six months. However the targets are calculated specific to veterans’ populations. Unlike the WIA Title IB and Title III performance measures, acceptable Veterans performance measures must match or exceed the negotiated levels of performance.

The VETS program provides two distinct sets of services, which are each measured against specific performance indicators: the Local Veterans Employment Representatives (LVERs) and Disabled Veterans Outreach Program specialists (DVOPs). The LVERs outreach to employers on behalf of veteran jobseekers and ensure that veterans have priority access to all of the services in the One-Stop center. The DVOPs serve only qualified veterans or eligible spouses of qualified veterans who are one or more of the following: disabled veteran, special disabled veteran, homeless, long-term unemployed, a recently released offender, lacking a high school diploma or equivalent, or low-income. The DVOP staff identify and provide primarily intensive case management services directly to those veterans with barriers to employment and special workforce needs.

JVSG Grants-Based Measures (DVOP only)

Because the DVOP staff are responsible for direct intensive services to veterans, the first set of measures reflect the performance of that staff only. These measures are listed below: The first three measures include all veterans and eligible persons served by the DVOPs. Measures five through seven include only the disabled veterans served by DVOPs.

1. DVOP Intensive Services - the proportion of total individual veterans (participants) served by DVOP specialists who received Intensive services

2. Veterans’ Entered Employment Rate (VEER) Weighted - the weighted percentage count of veterans not employed at participation who’ve entered employment following intensive services

3. Veterans’ Employment Retention Rate (VERR)

4. Veterans’ Average Earnings (VAE) (at Six-Months)

5. Disabled Veterans’ Entered Employment Rate (DVEER)

6. Disabled Veterans’ Employment Retention Rate (DVERR)

7. Disabled Veterans’ Average Earnings (DVAE) (at Six Months)

Table 31: Previous Assessment Results for the Veteran’s Employment Programs, JVSG

JVSG Grants-Based Measures

There is a marked improvement across the board from program year 2013 to program year 2014. Several measures were short of the negotiated targets in PY 2013, but all targets were exceeded in PY 2014. Notably, the percent of veterans receiving intensive services by DVOPs increased by 17 percentage points

Labor Exchange Services for Veterans

The second set of performance measures reflect the extent that veterans are served by the state’s labor exchange as a whole. These veterans may have been served by LVERs, DVOPs or other One-Stop system staff or they may simply have accessed the labor exchange system on their own. These measures are listed below. Again, the first three measures (8-10) refer to all veterans and eligible persons served and the second three measures (11-13) include only disabled veterans served.

8. Veterans’ Entered Employment Rate (VEER)

9. Veterans’ Employment Retention Rate (VERR)

10. Veterans’ Average Earnings (VAE) (Six-Months)

11. Disabled Veterans’ EER (DVEER)

12. Disabled Veterans’ ERR (DVERR)

13. Disabled Veterans’ AE (DVAE) (Six Months)

Table 32: Previous Assessment Results for the Veteran’s Employment Program, VETS

VETS Labor Exchange for Veterans

As with the DVOP measures, a couple fell short of the PY 2013 targets but all measures in PY 2014 exceeded the negotiated targets.

Unemployment Insurance

The Unemployment Insurance program has approximately 30 reports that reflect various aspects of the program performance. However in the context of the One-Stop service delivery system, the Unemployment Insurance program will be assessed by evaluating its performance in service delivery to claimants. The cores measure for services to claimants in the UI program are: “All First Payments 14/21-day Timeliness” and “Nonmonetary Determinations 21-day Timeliness.” These performance measures reveal the timeliness of processing and paying UI claimants’ claims.

The standard for first unemployment insurance payments made within 14 days is 87%. Idaho not only exceeds the standard, but ranks among the highest in the nation in its percent.

A second core measure is the timeliness of nonmonetary determinations. A nonmonetary determination is a written notice to the worker and other interested parties which advises of the worker’s eligibility with respect to acts or circumstances which are potentially disqualifying. The standard for these decisions to be made is 80% within 21 days. Idaho does not currently meet the standard of 80% with its highest percent coming in fiscal year 2015 at 75.4%. However it is a significant improvement over two previous years which were below 70%.

Table 33: Previous Assessment Results for Unemployment Insurance
First Payments within 14 Days

Table 34: Previous Assessment Results for Unemployment Insurance
Nonmonetary Determinations within 21 Days

The state has adapted its unemployment insurance program service delivery strategy in recent years. Beginning with a pilot program in FY 2014, Idaho began to centralize its unemployment insurance processing. The results of the pilot program were significant cost savings and standardization of procedures and policy interpretation. The cost savings have been invested in technology for a new unemployment insurance system.

The centralization strategy is also expected to produce higher quality staff through consistent training and therefore high quality service to claimants throughout the state, including those in rural areas.