During the early stages of QEP development, an assessment sub-committee was formed to create an assessment plan for each QEP initiative. The assessment plan was guided by logic models indicating specific program inputs, outputs, and outcomes for each objective. Subsequently, the assessment sub-committee met on September 14th and 19th, 2023 to create an assessment plan based on objective logic model outcomes. From those meetings, a detailed assessment measures checklist was created, which can be seen by clicking the "Assessment Checklist" link in the column to the right. The checklist indicates the data-responsible party and includes formative and summative assessment measures per SACSCOC standard 7.2. It also indicates which measures are quantitative, qualitative, and mixed-methods.

Pre-Term Orientation

For the first initiative - revise pre-term orientation in accordance with best practices -  targeted outcomes are as follows:

• 90% of first-time-in-college (FTIC) students who start the virtual orientation complete it.
• 100% of FTIC students who complete virtual orientation complete the FTIC survey.
• 70% of fall FTIC students and 50% of spring FTIC students who complete the virtual orientation enroll in classes for the upcoming term. (The current rate is approximately 63% for fall and 44% for spring terms.)
• Students are familiar with Collin College policies.
• Students are familiar with how to connect with others virtually or on campus.
• Students are familiar with formal support resources.
• Students are familiar with Collin College’s values and expectations.
• Students are familiar with Collin College’s financial resources.
• Students understand how to use Collin College’s technological platforms, online resources, and email.
• Students report that orientation prepared them for their first year in college.

Currently, orientation data are collected in the software platforms Comevo and Caspio. Reports are generated from these platforms and stored on the local hard drives of Student Engagement personnel. However, once the QEP moves into the implementation phase, the oversight of Collin College’s pre-term orientation will shift to the proposed First-Year Experience (FYE) Office. This office will construct a new database to capture orientation data, including orientation and FTIC survey completion percentages. In addition, the FYE Office will compare pre-term orientation completion data with student enrollment data stored in Collin College’s ZogoTech data warehouse and business intelligence system to establish what percentage of orientation completers enroll in classes for the upcoming term. Additionally, during the orientation, students will be given an orientation quiz to establish their familiarity with Collin College policies, values and expectations, formal support resources, financial resources, technological platforms, online resources and email, and how to connect with others virtually and on campus. Finally, FTIC students will be given an opportunity through a separate survey to indicate whether orientation prepared them for their first year in college, which will be administered by the proposed FYE Office. 

First-Year Seminar

For the second initiative - require a First-Year Seminar course for FTIC students across the district who are TSI not-college-ready - targeted outcomes are as follows: 

• Students will be able to develop an academic plan by identifying education and career goals.
• Students will be able to demonstrate and apply basic study skills.
• Students will be able to demonstrate and apply critical thinking skills.
• Students will be able to demonstrate and apply basic communication skills. 
• Students will complete their first term in good academic standing.
• Students will develop a strong sense of belonging virtually and/or on campus.
• Students will have confidence about their academic preparedness moving into their second year.
• Students will complete their first year in good academic standing.

Assessment of the first four outcomes of this initiative will be captured via course assignments, the grades of which will be housed in course Canvas modules through Collin College’s eLearning Center. Through these assignments, students will demonstrate the competency or ability that is listed in the outcome. Additionally, the student outcomes of completing their first term and first year in good academic standing will be measured using students’ grade point averages (GPA) following their first and second terms, which are stored in Collin College’s ZogoTech system. Students are considered in good academic standing if their cumulative GPA is above a 2.0 and they are not on academic warning, probation, or suspension. Furthermore, the outcomes of students developing a strong sense of belonging and having confidence about their academic preparedness will be measured via student responses on FTIC surveys which will be administered throughout the year by the proposed FYE Office. 

Early Alert System

Currently, Collin College utilizes an early alert system called EARS (Early Alert Reporting System), which utlizes Microsoft Teams for data collection and student tracking. However, there are notable shortcomings with this format which will be addressed by implementing a new early alert system with more robust student tracking and data collection options. For this initiative - identify and implement an early alert system - targeted outcomes are as follows:

• Intervention responders successfully reach student reported through early alerts.
• Faculty feel confident and prepared to use the early alert system.
• Student acts on recommendations from intervention responders.
• Student passes the course in which they received the early alert, or withdraws with no academic penalty.
• Faculty do not feel inconvenienced by early alert system.
• Faculty and staff perceive early alerts as effective.

For the first outcome, the early alert system will track who among faculty-referred students are successfully contacted by intervention responders. (This will need to be a key feature of whatever early alert system is selected for implementation.) For the second outcome, faculty will report their level of confidence and preparedness about using the selected early alert system on a post-training survey. End-of-term faculty surveys will also capture faculty perceptions and attitudes about the convenience of using the early alert system, as well as its perceived effectiveness. In addition, students who receive an early alert referral will indicate on surveys conducted by the proposed FYE office whether they acted on the recommendations from intervention responders. Lastly, following the term in which a student receives an early alert referral, course grades from Collin College’s ZogoTech system will be used to establish the rate at which referred students pass their course(s) or withdraw with no academic penalty. 

First-Year Experience Office

For the fourth initiative - establish and maintain a centralized division/department for first-year student initiatives - targeted outcomes are as follows:

• Form strong partnerships with other student-serving Collin College constituencies.
• Establish district-wide practices and procedures for collecting FTIC data.
• Implement targeted intervention programs.
• Centralize administration for FTIC intervention programs.
• Improve fall-to-fall persistence for underserved FTIC program participants relative to non-program participants.
• Improve academic success for underserved FTIC program participants relative to non-program participants.

The first four outcomes will be assessed through annual program review, which will be completed by the Manager of First-Year Experience and submitted to the Institutional Research Office (IRO) for review. The program review will follow the IRO template for service unit review. The strength of constituent partnerships will be measured by the partnerships resources table in the program review template, which asks “what partnerships and partner resources are established by the unit, and how are they valuable?” Assessment of district-wide practices and procedures for collecting FTIC data will be captured in the program review under the section titled “Why do we do the things we do? Unit relationship to the college mission & strategic plan,” as well as the section titled “How do we impact student outcomes?” The implementation and centralization of targeted intervention programs will also be detailed in the program review template. Additionally, FTIC student persistence and academic success will be measured using enrollment and performance data stored in Collin College’s ZogoTech system. This data will be compared to data collected by the FYE Office to show a relationship between FTIC student persistence and academic performance and engagement with FYE Office initiatives. 

Targeted Interventions Programs

For the fifth initiative - implement targeted intervention programs, including engagement, mentorship, and leadership programs - programs will be created to target certain FTIC student populations (e.g. first-generation, minority male, developmental education, etc.) based on needs assessment and institutional data. Targeted outcomes are as follows:

• Program-eligible students are successfully contacted/reached about participating in a specific program
• Program-eligible students opt into the program, or do not opt out of the program
• Students have greater understanding of college-level expectations
• Students feel more prepared for college-level work
• Students have a better college experience than initially expected
• Students feel supported by the college
• Students enroll for subsequent term
• Program contributed to students' successful transition to college life

The first outcome will be measured with data captured from a customer relations management (CRM) platform designed for mass communication (e.g. Signal Vine, Salesforce, MyEmma, Mailchimp, etc.). This data will show how many program-eligible students receive communications about a specific program and how many of them engage with the communication mode. For example, the CRM platform will be able to track how many program-eligible students open an email inviting them to participate in a specific program, and how many of them do not. This will provide insight to the effectiveness of program advertising and communication. The second outcome will be measured by capturing data in a newly constructed data base in the proposed FYE office showing how many program-eligible students opt into programs, or do not opt out of them. This will be similar to the database constructed to capture orientation data. Student attitudes and perceptions about the intervention programs, including feelings about preparedness for college-level work and support from the college, as well as perceptions about college-level expectations, overall college experience, and program contributions to their successful transition to college life will be captured via student responses on FTIC surveys which will be administered throughout the year by the proposed FYE Office. Finally, the outcome of students enrolling in a subsequent term will be measured using enrollment data from Collin College’s ZogoTech system.

FTIC Co-Enrolled Courses

For the sixth initiative - implement FTIC co-enrolled courses in connection with the first-year seminar and the first developmental level of integrated reading and writing and math - targeted outcomes are as follows:

• Students form strong social relationships
• Students are comfortable in classroom environments
• Accumulation of more credit hours during students’ first year
• Higher rates of enrollment persistence among first-year participants
• Higher rates of academic success (A, B, and C grades) among first-year participants
• Lower rates of withdraw/fail grades during students’ first year.
• High rates of satisfaction with college among first-year participants

For the outcomes of students forming strong social relationships, feeling comfortable in classroom environments, and high rates of satisfaction with college among learning community participants, these will be measured via student responses on FTIC surveys which will be administered throughout the year by the proposed FYE Office. The remaining outcomes involving accumulation of more credit hours, higher rates of enrollment persistence, higher rates of academic success, and lower rates of withdraw/fail grades will be measured using data stored in Collin College’s ZogoTech system at the conclusion of each term. 

Increase Use of Formal Academic Support Systems

For the seventh initiative - increase students’ use of formal academic support systems - targeted outcomes are as follows:

• FTIC students are successfully contacted/reached
• FTIC students are able to identify formal academic, and physical and mental health support resources
• FTIC students are aware of how to access these resources
• FTIC students make use of formal academic, and physical and mental health support resources during their first year

Similar to the assessment approach used to evaluate targeted intervention programs, the outcome of students being successfully notified of a specific formal academic support resource will be measured with data captured from a CRM platform. The remaining outcomes will be measured using data captured via student responses on FTIC surveys which will be administered throughout the year by the proposed FYE Office. 

In total, there are eleven different methods of measuring desired QEP outcomes, and they include class assignment data from Canvas, CRM platform data, early alert system data, faculty training and end-of-term survey data, FTIC pre- and post-survey data, FYE office and intervention program data, orientation quiz data, program reviews submitted to IRO, and data housed in Collin College’s ZogoTech system. There are only three different data-responsible parties, and they include the eLearning Center, the proposed FYE Office, and the Institutional Research Office.