The short-term goal of this IRT is to build, test, and validate an interactive and customizable statistical report (CSR) template database to be used to improve the Organic Chemistry courses. The mission of the Organic Chemistry Teaching team (Professors Williams, Roth, Le, Sethi, Roychowdhury, and Zhang), in collaboration with the Vice-Chair of the Undergraduate Program and the Chair of the department of Chemistry and Chemical Biology (CCB), is to improve student learning and student outcomes, student experience, and instructor experience. To date, manually assembled assessment reports have provided much value within and across our organic chemistry courses.
These reports demonstrate - to students and instructors - alignment of learning goals, lecture content, and homework with assessment items and assigned grades. The proposed reports represent an organizational framework upon which the Team plans to demonstrate improved student learning across sections and over time. CSRs will streamline the generation of reports that combine assessment item details, learning goals, and standard assessment data with item response theory based statistical methods (item characteristic curves, etc.).
The student team will consist of three students with knowledge of organic chemistry, computer science, and/or statistics. Students will work under the supervision of professors L. Williams and K-P. Le and collaboratively with the Organic Team and Professor Marc Muniz (CCB), in consultation with Professors Warmuth (CCB Vice-Chair) and Brennan (CCB Chair). This project will require the student team to collaboratively create a suitable template that will generate a CSR. They will combine their expertise and background to create an algorithm that accepts critical input (learning goals, assessment questions, output from R-based statistical item response theory). Statistical data can be generated by free statistical software packages. For example, we use https://shiny.cs.cas.cz/ShinyItemAnalysis/.
The projects will include two main phases and outcomes as follows: Phase 1: CSRs that target the teaching faculty who own the exams: Students will code, test, and validate the report template. The template will generate reports that present each item from an assessment along with the associated learning goals, qualitative descriptors, and key statistic attributes (item characteristic curves, difficulty, discrimination, etc.). The report output as email-ready html or PDF format for faculty and a simplified version for students. Phase 2: CSR database suitable for datamining report items meta-analysis: Students will code, test, and validate a report database aim that will enable the Team to mine report data, e.g. learning goals, question style (e.g. multiple choice vs open answer), item difficult and discrimination, and other data in the CSR templates.