The Purpose
The purpose of the evaluation strategy described below is to enable participating faculty from a number of different disciplines to collectively develop a plan by which to monitor what their students gain from their new research-embedded curricula.

    1.  The primary task is to develop a set of assessment strategies that give individual faculty good feedback on the degree to which their aspirations for student learning gains are being realized by their new courses.1
    2.  A secondary aim is to document the nature of the work required to implement the new curriculum offerings in a way that:
         --is comparable across disciplines
         --assists subsequent curriculum developers in the AIRE project to structure their proposals clearly, comprehensively, and with student learning assessment as a central focus.

The Process

A. Developing course descriptions that include criteria for success and strategies by which to judge success (in comparable format) across disciplines:

    1.  The first round of 20 curriculum designers forms working groups of two or three faculty each. Groups may be (by participants' choice) in the same or related disciplines, or represent different disciplines.
         The role of the group members is to help each other through the process of restructuring their original proposal for new research-embedded courses into a form that enables the design of student learning assessments appropriate to each teacher's curriculum goals and that documents the implementation of their new courses.

    2.  Each curriculum developer is asked to redraft the description of their new curriculum using the attached template: by cutting/pasting, filling in blanks, and amplifying their initial descriptions as needed.

    3.  They each distribute their restructured curriculum description to the other group members, read each other's drafts, raise questions, and offer suggestions for clarification and amplification to the authors in their group. They then circulate the revised course descriptions (now in a common format).

    4.  On the basis of these second drafts, the group then meets:
         --to discuss the criteria for student success that the documents contain, and
        --to select ways to assess the degree to which students demonstrate progress in each kind of learning that each course developer is seeking.

There are a number of existing resources to help with this process. A list of handbook materials on classroom assessment is attached. The new Field-tested Learning Assessment Guide (FLAG) web-site is also a guide to the types of classroom assessment available and gives practical examples developed by faculty, evaluators, and researchers (http://www.wcer.wisc.edu/nise/cl1).

    5.  Individual faculty begin to work out ways to monitor (assess) student learning that they can build into their overall assessment strategy for the class. They should seek to address (by individual or group assignments, tests, or other forms of assessment) the attributes they wish to see gained by students as a result of taking the new class. The frequency or weight (including grading) that they give to each assignment should reflect their importance to the teacher's learning goals.

    6.  The final version of their curriculum design (now containing examples of the kinds of assessment strategies they intend to use) should be checked one more time with their own group, then circulated to the whole group, the Project Director, and the evaluator.

    7.  The whole group meets to discuss and further develop
        --some of the assessment ideas they have collectively generated
        --ways to collect and compare over time the results of using particular forms of assessment that are shared within or across disciplines.
 
B. Keeping track of how new courses work out in practice, what changes are made, and why.

The whole group should discuss by what acceptable means they can record and share the progress of their new courses and the achievements of their students, both with their own (first round) cohort, and with the next two cohorts.

Some ways to do this include:
    --Keeping a weekly electronic log (lab notebook) in a common format. These are collected by either the Project Director or the evaluator.
    --A recorded telephone discussion with the evaluator once or twice each semester.

The whole cohort receives a digest of common issues appearing in these records, and meets two or three times over the course of the year to discuss their experiences, discoveries, changes made, successes, and areas of difficulty. Faculty in the next round of curriculum development should be invited to meet with the first cohort and all materials (including assessment data) should be shared with them.

C. Collecting Data on Student Learning Gains

Faculty classroom assessment methods that reflect their own particular criteria for student success are the primary means by which faculty will show what their students have been enabled to accomplish by means of their new courses. Because development of a set of assessment strategies that match their student learning objectives is often an unfamiliar professional task, and because faculty will find that they share particular learning objectives for their research-embedded courses, it is important to develop these collegially, and to discus ways to document them for each cohort of developers.

However, the group may also wish to consider some other commonly-used ways to assess their progress:

    1.  Pre- and Post- tests of student knowledge, skills, or attitudes that exemplify the gains thought to be available to undergraduates through research experiences of the types offered in this initiative.
         Developing one or more pre/post-tests that establish student gains in areas that transcend particular courses requires agreement upon common testable elements, test administration, and collection/compilation and analysis of results.

    2.  Student assessment of their own learning gains (an alternative to traditional classroom evaluations)at mid and/or end points in each semester (see evaluator's example for undergraduate
"modular" chemistry attached). This instrument is available on a web-site (http://www.wcer.wisc.edu/salgains/instructor).

    3.  The setting up of "carry-forward" experiments to check for learning and skills retention and transfer into subsequent classes. (The evaluator will discuss variations in the use of this technique.)

    4.  Student focus group interviews exploring their learning gains concerning the research elements of the new courses (conducted by a qualified, non-particpating colleague, or by the evaluator).

___________________________
¹This is particularly useful to individual faculty for inclusion in their professional portfolio, as well as forming the basis of the collective record of work done by the project participants over the three years.