AIRE ASSESSMENT WORKSHOP
EDUCATIONAL GOALS, OUTCOMES, AND TOOLS

Educational Goals need to be specified as measurable outcomes.\

Types of Measurable Outcomes include:
1. Attitudes.
Involve students' beliefs about the value of the course material and their consequent feelings about the material.

2. Intentions.
Involve plans for using the material or seeking similar experiences in the future.

3. Changes in knowledge.
Involve the development of expertise in the area of the course material, a change in the way the student thinks about the material.

4. Skill acquisition.
Involves the internalization of procedures that are productive of knowledge, its application, integration, and communication.

Assessment Tools Include:
1. Self-Report.
Self-report tools require individuals to evaluate their own knowledge, skill level, attitudes, or intentions.

2. Instructor Assessment.
Instructor assessment requires evaluating student performance.

AIRE ASSESSMENT WORKSHOP, example 1
EXAMPLES OF
ASSESSMENT TOOLS FOR EACH GOAL

These were included as examples for the workshop participants to rate the workshop.

A. Examples of Self-report assessments for each of the outcomes:
1. Intentions
Rate the likelihood or your using one of the assessment tools that you learned about today in your course next semester.
1 2 3 4 5
Very Unlikely Somewhat Likely Very Likely

2. Attitudes
Rate the importance of assessing student outcomes for your own teaching effectiveness.
1 2 3 4 5
Unimportant Somewhat Important Very Important

3. Knowledge
Rate the amount that you learned about assessment tools.
1 2 3 4 5
Learned Very Little Learned Some Learned Very Much

4. Skills
Rate your ability to write an assessment tool for one of your course goals.
1 2 3 4 5
Unable Able (With Assistance) Able

B. Examples of Instructor assessments for each of the outcomes:
1. Intentions and Attitudes
Track the usage of student outcome assessment in participants’ courses.

3. Knowledge
Test participants' acquisition of the information provided in the workshop.

4. Skills
Evaluate the assessment tools that the participants construct during the workshop.

IMPORTANT Points to keep in mind when using all assessment tools.
1. Use multiple items to assess each outcome.
2. Use multiple types of assessments for each outcome.
3. Take a pre- and post- measure whenever possible, to provide the greatest amount of information.
4. Make use of a relevant comparison group when possible.

AIRE ASSESSMENT WORKSHOP, example 2
ASSESSMENT TOOLS FOR A COMPONENT OF A COURSE
Introduction of group research projects into the course, Psychology of Close Relationships

Goals:
1. To enhance appreciation of psychological research in understanding relationship processes and finding solutions for relationship problems.

2. To increase intentions of doing further work involving social science research.

3. To impart elementary research skills, e.g., in archival research, hypothesis construction and operationalization, sampling, statistical analysis, and interpretation of data.
4. To impart knowledge, especially by doing research, about close relationship processes, and the boundary conditions, or limits, of scientific generalizations about these processes.

Student Attitudes (Measured before the course and at the end of the course).
Use the following scale to rate how much you agree/disagree with each of the following statements.
1 2 3 4 5 6 7
Extremely disagree Extremely agree

1. Research about close relationships provides insight into reasons for interpersonal problems. _____

2. Research about close relationships can help people improve their relations with others. _____
3. Research about close relationships does not tell us anything more than common sense does. _____

4. It is impossible to make generalizations from research on close relationships because everyone is different. _____

5. Knowledge about close relationships is better acquired from experience than from research. _____

Intentions (Measured at the course's end).

1. I am planning to take social or behavioral science courses in the future. _____

2. I am thinking about majoring in social or behavioral science. _____

3. I am planning to take at least one course in which I can learn more about research methods. _____

4. I will probably do some kind of independent research or research assistance before graduating from Oberlin. _____

5. I will probably look for a job or internship in which I can get experience doing research before graduating from Oberlin. _____

Knowledge Acquisition (Measured at the course's end)

1. Please rate how well you could explain each of the following ideas to a friend. Please also rate how much your learning about each idea has increased your understanding about close relationships, including your own.
1 2 3 4 5 6 7
I could not explain I could explain
it at all. it extremely well.
1 2 3 4 5 6 7
It has not increased It has increased
my understanding at all. my understanding a great deal.
Could explain Increased understanding

a. Attachment theory _______ _______
b. Social exchange theory _______ _______
c. Relationship schemas _______ _______
d. Metacommunication _______ _______
e. Constructive/destructive conflict_______ _______
f. Other (Please specify) _______ _______

2. Please estimate the contribution that each of the following course components has made to your learning about close relationships.
___ % from lectures
___ % from class discussions
___ % from reading assignments
___ % from presentations
___ % from research project
___ % from ______________
3. Please indicate how much you learned about your group research topic, relative to other topics that we covered in this course.
1 2 3 4 5 6 7
Very much less Very much more

4. Please rate the importance of each of the following in what you learned from your group research project.
1 2 3 4 5 6 7
Not at all important Extremely important
a. Doing the library research _____
b. Designing the study _____
c. Collecting the data _____
d. Analyzing the data _____
e. Interpreting data _____
f. Doing the presentation _____
g. Writing the report _____
h. Other ___________
Third party evaluation of knowledge and skill acquisition (Me)

1. Evaluation of the research project.
a. Was the research question/hypothesis developed from the research literature?
b. Were reliable and valid measures used to address the question (test the hypothesis)?
c. Was an adequate sample selected for the purposes of the research?
d. Was data collection systematic?
e. Were appropriate statistics used to analyze the data?
f. Were results precisely and concisely described?
g. Were results made relevant to the research literature?
h. Were results qualified by the constraints of the method?

2. Final exams: Knowledge of project-related questions versus non-project-related questions.

ASSSESSMENT WORKSHOP, example 3
DEVELOPING ASSESSMENT TOOLS FOR A COURSE

General Introduction to Science

Course Description: This hypothetical course examines the applications of science to current real world problems. Topics include financial decision-making, conservation of environmental resources, and disease prevention. In addition, this course will introduce the tools that scientist use to understand their world. Students will design experiments, collect and analyze data, interpret and present their results.

Course Goals:
1. Learn about science's potential to be applied to real world problems.
2. Learn about scientific procedures for addressing research questions and testing hypotheses.
3. Learn to use SPSS and Excel to analyze and graph data.
4. Learn to evaluate information through critical thinking.
5. Increase interest in applied science.
For each goal, develop measurable outcomes.