Examples of Best Practices in SAS Program Assessments
Departments and programs across the School of Arts and Sciences have implemented and reported on a broad array of assessments of their majors and minors: there are many best practices we could cite as examples of evidence-based decision-making. In the reports for academic year 2015-16, many departments implemented changes to their programs based on that assessment evidence. Below please find a few notable examples. Full program reports are available here. (NOTE: Only for those with access to Sakia site)
Criminal Justice: Applying assessments to program from start to finish
Criminal Justice has set standards for student achievement of its program goals at both introductory and culminating levels of its required curriculum, and looks for evidence of improvement as students progress through the major. At both ends of the sequence, assessment results have suggested modifications to course design and delivery to improve future student performance.
In the Criminal Justice introductory course, students are presented with case scenarios and prompted to interpret them. This requires them to demonstrate achievement of the program goals for (1) critical analysis of ethical issues, and (2) the ability to apply concepts and theories in specific criminal justice situations. The results of this assessment have revealed variation in student performance across different sections of the course. In AY 2014-15, the poorest student performance was in the section of an instructor teaching the course for the first time. Sharing the assessment results with this instructor and providing the instructor support helped to narrow the student performance gap. The next year, the assessment results showed that students enrolled online achieved less satisfactory results overall than did those in face-to-face sections. The department is now exploring the reasons for this.
The Criminal Justice goals were assessed at the other end of the program sequence as well. A capstone-level rubric defines desired outcomes for the goals at or near completion: this was used to score final papers in multiple 400-level courses for research design and critical thinking skills, critical command of criminal justice theory, and knowledge of the institutions and policies of the criminal justice system. Overall, results were very strong, though weaknesses were found in student achievement of research methods goals. This confirms past observations about the disadvantage of having to rely on research methods courses offered in other departments, which led to Criminal Justice to develop its own methods course. Instructors involved in its design will develop an appropriate assessment for the course, and follow-up assessments in the 400-level courses will explore how effectively this close-the-loop action helps to close the gap in student learning outcomes over time.
Geography: Identifying steps for improvement through curriculum mapping
The Geography Department has a 3-year assessment cycle of successive review of each of its three major tracks. In AY 2015-16, they developed a curriculum map for the track in Global Cultures, Economics and Society, aligning all required and elective courses of the program curriculum with specific program goals, and developing rubrics to assess student learning outcomes for those goals. The three learning goals for this track were directly assessed in a 400-level capstone-level course required of all Geography majors. In a dedicated class session, students completed an essay writing assignment designed to prompt for their ability to synthesize and examine critically a range of contemporary issues, and identify and apply appropriate analytic tools and models.
All the essays were scored by a single reviewer, using the program rubrics. Most students demonstrated satisfactory or better achievement of the goals, but the results for one program goal suggested room for improvement, with few students performing above a satisfactory level. In response, the department scheduled a meeting the following semester for the instructors teaching those courses identified in the curriculum map as specifically addressing this goal.
Comparative Literature: Augmenting the culminating (Capstone) experience to improve outcomes and assessment
Comparative Literature has generated multi-year assessment results from a culminating experience for all its majors. A senior capstone workshop was created to support student performance near program completion, and serve as a site for assessment of the content knowledge learning goals for the major. Mandatory for all graduating seniors not completing an Honors Thesis, this one-credit course requires the completion of a research paper, and a reflective essay on the student’s experience of the major. Results of these direct and indirect assessments have led to changes in the department’s advising structure; revisions to the requirements for the senior research paper; and changes to the content and delivery of the workshop itself, with the aim of further improving student outcomes, clarifying expectations for the work submitted, and prompting more useful feedback from students.
Classics: Involving majors in the program assessment process
The Classics department has taken the approach of involving their graduating seniors actively in their program assessment process. They developed a set of benchmarks for student performance on their program learning goals at or near completion, and have used the resulting rubrics in direct assessments of a sample of student work from their upper-level courses. Graduating seniors are asked to submit a set of essays representative of their work in the major, and a panel of faculty members score student attainment of the desired outcomes in this work. The department uses the results of these assessments both to inform their curriculum revision processes and to better understand the match between their own expectations and those of their students. In response to the results, Classics has implemented a new strategy to increase student participation in study-abroad and other co-curricular options that are closely aligned with student achievement of their program learning goals.
Kinesiology and Health: Measuring student preparation for professional success by linking a capstone-level co-curricular experience to achievement program learning goals
A program learning goal for Kinesiology and Health is that at graduation, all students will be prepared to immediately enter relevant careers, and be qualified for further graduate studies. To this end, seniors complete a capstone-level professional internship in an approved professional agency relevant to their specific major option (one of three health sciences tracks, or Sport Management). Upon the completion of this work-study experience, internship supervisors complete an online survey that includes specific questions on student achievement of the learning goals for the major. The survey also gathers information on the number of years of experience with internships; the number of interns from Rutgers and other universities that were supervised; how Rutgers majors perform relative to all interns, and to professional standards of practice; and other general feedback on the interns and their readiness for employment in the field. This calibration of ESS student outcomes with those of comparable programs, in addition to direct evaluation of student performance in a professional setting, is valuable information in the department’s analysis of the degree to which their students are fulfilling the expectations for achievement in each program track.