Perkins Career and Technical Education Program

(d) Accountability for Results

1. Identify and include at least one (1) of the following indicators of career and technical education program quality—

A.The percentage of CTE concentrators (see Text Box 6 for the statutory definition of a CTE concentrator under section 3(12) of Perkins V) graduating from high school having attained a recognized postsecondary credential;

B. The percentage of CTE concentrators graduating high school having attained postsecondary credits in relevant career and technical education programs and programs of study earned through a dual or concurrent enrollment program or another credit transfer agreement; and/or

C. The percentage of CTE concentrators graduating from high school having participated in work-based learning. (Section 113(b)(2)(A)(iv)(I) of Perkins V).

Indiana plans to track and report on all three measures of program quality.  The state feels that each indicator represents a critical measure of program quality and tracking all three will provide the most complete picture. 

The measures will be calculated in the following manner:

Postsecondary Credential Attainment Numerator:  The unduplicated number of CTE concentrators in the reporting cohort who attained a postsecondary credential.  A postsecondary credential will be defined as a promoted industry certification or state issued license that counts as an Indiana’s Graduation Pathways Postsecondary Readiness Competency, or a postsecondary certificate, technical certificate, or degree.

Postsecondary Credit Attainment Numerator: The unduplicated number of CTE concentrators in the reporting cohort who earned at least 9 postsecondary credits in courses that map toward a postsecondary certificate or degree program.

Work-Based Learning Participation Numerator:  The unduplicated number of CTE concentrators in the reporting cohort who participated in at least 75 hours of a work-based learning experience where the student has a work record in a position(s) aligned to the student’s career pathway on their graduation plan. Examples of a work-based learning that would fit this definition include: job site placement and internship; apprenticeship/pre-apprenticeship; State Earn and Learn; clinical or practicum; school-based enterprise; or simulated work environment. 

Denominator: The unduplicated number of CTE concentrators in the reporting cohort.

2. Include any other measure(s) of student success in career and technical education that are statewide, valid, and reliable, and comparable across the State. (Section 113(b)(2)(A)(iv)(II) of Perkins V) Please note that inclusion of “other” program quality measure(s) is optional for States.

The Office of CTE staff will use FY 2021 as an opportunity to explore additional measures of student success. One possibility is a persistence measure that would measure the percent of CTE participants that go on to earn concentrator status. A more formal definition would be provided in future updates of the Perkins V plan. 

3. Provide the eligible agency’s measurement definition with a numerator and denominator for each of the quality indicator(s) the eligible agency selects to use.

N/A

4.Provide on the form in Section V.B, for each year covered by the State plan beginning in FY 2020, State determined performance levels or each of the secondary and postsecondary core indicators, with the levels of performance being the same for all CTE concentrators in the State. (Section 113(b)(3)(A)(i)(I) of Perkins V)

5. Describe the procedure the eligible agency adopted for determining State determined levels of performance described in section 113 of Perkins V, which at a minimum shall include

A. A description of the process for public comment under section 113(b)(3)(B) of Perkins V as part of the development of the State determined levels of performance (see Text Box 7 for the statutory requirements for consultation on State determined performance levels under section 113(b)(3)(B) of Perkins V);

B. An explanation for the State determined levels of performance that meet each of the statutory requirements in Text Box 8; and

C. A description of how the State determined levels of performance set by the eligible agency align with the levels, goals and objectives other Federal and State laws, (Section 122(d)(10) of Perkins V).

Response captured below in (6).

6. As part of the procedures for determining State determined levels of performance, describe the process that will be used to establish a baseline for those levels.

Secondary CTE Participant: Any student who completes not less than one Career and Technical Education program or program of study. 

Secondary CTE Concentrator:  Any student who has completed at least two advanced courses in a single Career and Technical Education program or program of study.

Postsecondary CTE Participant:  A postsecondary CTE student is any postsecondary student that is enrolled in a designated CTE program regardless of whether or not they completed secondary CTE.

Postsecondary CTE Concentrator: A postsecondary CTE Concentrators will be any postsecondary CTE participant who earns at least 12 credits within an approved CTE pathway; or has completed a postsecondary program. 

Indiana’s state levels of performance are determined using the statutory requirements established in Perkins V with a few clarifications. These include that academic proficiency is based upon the student’s first attempt on the state assessment. This holds true for the postsecondary performance levels as well. 

The state-determined levels of performance were established by first converting individual program year’s data to the statutory requirements of Perkins V. A table of the data from the last four years, along with the calculated 3 year average, is included in Appendix 4. One final step was to review the data on 3-year averages to look for any anomalies. The anomalies that were discovered are discussed below. 

The baseline levels for all indicators, except 2S1, 2S2 and 2S3, were determined by taking the 3-year average.  Indicators 2S1, 2S2 and 2S3 utilized the performance levels for the 2018-2019 cohort.  The performance levels for this cohort were significantly lower due to the state transitioning from End of Course Assessments to the Indiana Statewide Testing for Educational Progress (ISTEP). The use of a single cohort’s performance as the baseline, mirrors the process used by the Department of Education to establish the performance levels for ESSA. 

A review of the past performance also identified a concern with 3S1: Post Program Placement. Over the last three years there has been a 6-8% decrease in performance. This trend follows an all-time high level of performance in the 2014-15 Cohort.  There are some reporting errors that are causing the 2018-19 cohort level of performance to be at 64%. The Office of CTE is working to resolve the reporting issues. The baseline level for 3S1 was determined by excluding the most recent year’s performance from the 3 year average.  

Once the baseline levels were determined, a goal for the FY 2024 program year was established for each performance indicator. The goal was determined by comparing Indiana’s performance to national performance and the performance of neighboring states. The total growth needed to meet the established goal by FY 2024 was divided by four to help establish a percent increase goal for each year of the plan.  This process will help to ensure continuous improvement over the span of the Perkins V plan.  

In addition to being available for public comment along with the rest of the Perkins V plan and the state’s WIOA combined plan, the performance level goals were shared through an email with eligible recipients. The email was shared with eligible recipients by January 31, 2020. 

The two performance indicators that most align with the WIOA combined plan are the postsecondary credential attainment and post program placement. These performance indicators are included in the strategic goals for the state. Indiana plans to require all talent development programs to track and monitor all of the WIOA performance levels. The purpose of collecting this data is for comparison purposes to help identify the best uses of state resources. 

Prior to the third year of the Perkins V plan, the state of Indiana will need to reevaluate all eligible performance levels. The changing of the concentrator definition, program of study requirements and alignment and the possible transition to a college entrance exam as the state’s academic assessment for ESEA may necessitate resetting baseline levels and performance targets. 

7. Provide a written response to the comments regarding State determined performance levels received during the public comment period pursuant to section 113(b)(3)(B) of Perkins V. (Section 113(b)(3)(B)(iii) of Perkins V). As part of the written response, include a description of any the changes made to the State determined performance levels as a result of stakeholder feedback.

State-determined levels of performance were shared with stakeholders prior to January 31, 2020. A written response will be prepared after public comment has been received. 

8. Describe how the eligible agency will address disparities or gaps in performance as described in section 113(b)(3)(C)(ii)(II) of Perkins V in each of the plan years, and if no meaningful progress has been achieved prior to the third program year, a description of the additional actions the eligible agency will take to eliminate these disparities or gaps. (Section 122(d)(11) of Perkins V). As part of the written response, and pursuant to the Report of the Senate Committee on Health, Education, Labor, and Pensions (HELP), the eligible agency could indicate that it will analyze data on the core indicators of performance to identify gaps in performance, explain how they will use evidence-based research to develop a plan to provide support and technical assistance to eligible recipients to address and close such gaps, and how they will implement this plan. The eligible agency is not required to submit a new State plan prior to the third program year in order to address this requirement.

During the 2019-2020 program year, the state of Indiana will use the CLNA process as the primary mode for reviewing and auditing the performance of eligible recipients. At the secondary level, disaggregated data have been provided to each eligible recipient for each of the performance indicators. The Office of CTE plans to conduct a thorough review of each eligible recipients CLNA and Local Application not only ensure alignment, but also, to identify performance gaps. Identified gaps will be documented to help the state determine what technical assistance and support is needed. Career Specialists, who specialize in specific technical assistant areas, will analyze the results of this review and be responsible for developing a strategic plan for providing professional development and technical assistance to the state and individual recipients through a variety of formats. 

The review process for CLNA will also help the Office of CTE to identify and prepare for any necessary changes to the CLNA process in two years.  One possibility is to require jointly developed CLNA by secondary and postsecondary recipients on a regional basis. 

The Eligible Agency and the Office of CTE are committed to better utilizing InTERS, Indiana’s extensive CTE data collection portal, to improve data informed decision making and monitoring program quality through high quality outcomes. Over the course of the next two years, Office of CTE staff will develop a real time auditing and monitoring system using InTERS reports or a data dashboard. This auditing and monitoring process will be used to identify individual schools that may require a more intensive on-site monitoring visit and/or be in need of targeted technical assistance. InTERS allows us the opportunity to drill down beyond the district level and to monitor the performance of individual schools and individual programs of study within a school. Having this type of data will be critical to improving the consistency and quality of programs of study. 

Indiana will still make use of the required improvement plans for eligible recipients based upon local agreed upon levels (AULs) that are set via negotiation with each local recipient. In past years, Indiana has required schools to create an emergency improvement plan to require immediate action to address poor performance.  The emergency improvement plans were followed by a school improvement plan as a part of the Local Application.