Assessment Workshop, Oberlin College, July 8, 1999: Description

Purpose/Setup of the Workshop

Much of the AIRE is targeted toward curriculum development projects that will increase research experiences in the classroom. We would also like to evaluate the effectiveness of these curricular innovations. To this end, we invited Dr. Elaine Seymour, Director of Ethnography and Evaluation Research, Bureau of Sociological Research, University of Colorado, Boulder, Colorado to come to Oberlin College and run a workshop on assessment. Dr. Seymour developed the background materials and acted as facilitator for the day long workshop.

Background Materials

A number of background materials were passed out by Dr. Seymour for the workshop. These include:

Template for a Common Course Description Structure

Annotated Bibliography

The Procedure

I. Defining Success

Faculty members were all asked to individually write answers to complete the statement; "I will know I am successful (with the research part of my curriculum) when÷.."

As a group the faculty then went through these and collated them into categories. These have been posted under the title Success Indicators.

II. Activities List

Faculty members listed and then discussed the activities that they plan to do in their courses. More specifically, faculty wrote answers to the following question (see Template, 2b). What research or research-like activities are you embedding in the course (whole class, small group, lab and field) that reflect the aspects of the research process that you particularly want the students to experience? Faculty also discussed whether there are common components for a "research experience". Many felt that there are common components including that one a) measures things, b) you note what happens when you change things, c) you deal with data: how to figure things out and then do it, and d) you review previous scholarship.

III. Gains/Learning Objectives embedded in research experiences (Template, #3)

Faculty worked individually to list their objectives and measures of success and then went back to their list of research activities to make sure that each activity was associated with an objective and that each objective had an activity associated with it.

IV. How do you analyze success (Template, #4) and Designing assessments (Template, #5).

Faculty discussed both of these issues. Some possibilities/comments that were brought up include

-Group work: have them confidentially grade and rate each other on certain criteria.

-Ask students questions to ascertain where they are and share the results with the class. Then test again at intervals.

-Carry forward: ask the next faculty down the line what skill they would like students to have, e.g. in the second class it could be tested whether this was achieved.

-If you are making changes, give signposts, i.e. where we are, why we are doing what we are doing.

-Focus groups: These are time/labor intensive. These need to be run by non-students, far away from the academic department. For every 1.5 hour focus group it will take 6-7 hours to transcribe the data and a couple of hours of computer time and another 4-8 hours to code and analyze the data.

-Pre-test/post-test: These could be of skills, analysis of an article, what they think.

V. What are the costs?

We ended with a discussion of what the costs are for curriculum development and the integration of more research-like activities into classes. We also discussed ways to keep track of what those costs are.