How will I know when I am successful?
(Faculty discussion at AIRE Assessment Workshop, Jul 1999)
Students show intrinsic interest in the discipline and/or
research and engagement with the work/discipline.
Individual faculty members said they knew they were successful when:
Students ask more questions in class.
There is an improved classroom dynamic.
Students are involved and committed to the material covered in class.
Students show enthusiasm for the research process.
The students take a deeper interest in the subject matter (greater enthusiasm,
want to pursue it more).
The student's sense of connectedness to other and to the discipline increases.
The student's interest in the field increases; there is a feeling of excitement
and wonder.
Students tell me they will pursue independent research partially because
of the course.
Students go on to take another math or science course that they were not
planning on taking.
Students show more interest in the life of the intellect.
Students appreciate how many disciplines are important (physics, comp
science, chemistry, math, engineering overlap).
When students are fully engaged in the work.
When students want to learn and do more with their topic.
Hypothesis: An increase/change in interest will lead to
greater persistence in the discipline, choice to do research, choice of
discipline-based career. Students will want to learn more and will enjoy
the learning.
Students make connections, integrate knowledge/ideas and
create new ideas from these. They show depth in their analysis of problems
and understanding of complexity.
Individual faculty members said they knew they were successful when:
Students actually achieve/reach original conclusions/ideas in their research.
Students become comfortable integrating information from the literature.
Students learn to integrate various concepts and information.
New thoughts/ideas emerge as the result of discussion and as a result
of research activities.
Students say "Ah Ha!"
Students demonstrate the ability to analyze problems and issues at a deeper
and more complex level.
Students can argue from data, defend their conclusions,
critically evaluate the research of others, see logical errors, see flaws/strengths
in own/others work.
Individual faculty members said they knew they were successful when:
Students can defend their conclusions.
When students pinpoint errors/inadequacies in my research design/hypothesis.
When students can critically evaluate someone else's research (i.e. find
flaws and good aspects).
When students can detect logical fallacies.
Students show they can "think like a scientist". They question
and show skepticism.
Individual faculty members said they knew they were successful when:
Students understand the implications of inherent constraints in the world
that involve inevitable trade-offs.
Students doubt/question their preconceptions and assumptions.
Students begin thinking like scientists. In particular, they ask "What
is the evidence for that assertion, etc". They show skepticism.
Students can understand and identify on an exam the nature of science
as a way of knowing versus other ways of knowing. They appreciate the
benefits of answering a question using science. They can differentiate
good science from pseudo science or non-science.
They are comfortable with and appreciate uncertainty, complexity
and questioning.
Individual faculty members said they knew they were successful when:
Students become comfortable with asking questions related to unknowns
(hypothesis generation).
Students appreciate the complexities that go into designing a new material.
Students show understanding of the scientific process.
They ask questions, can generate hypotheses, design an experiment, evaluate
data and redesign an experiment.
Individual faculty members said they knew they were successful when:
Students can turn an idea into a research topic and design.
Students can design an experiment with proper controls.
When most students identify the many reasons why their planned experiments
did not work/did not answer their experimental hypothesis.
When most students develop their own reasonable/testable research hypothesis.
Students handle data like a scientist. They show competence
in entering data, reading data, analyzing data, graphing data, collecting
data. They can use the computer for all of these. They recognize a reasonable
measurement. They can ballpark/estimate a measurement.
Individual faculty members said they knew they were successful when:
Students can enter data and calculate inferential statistics using SPSS.
Calculate descriptive statistics using Excel.
Students can translate a graph into words.
Students can generate follow up research ideas from a presentation of
the literature.
Students can prepare a simple graph using Excel.
Students can enter data on Excel.
Students can read off a graph and make reasonable conclusions about what
it says for the material.
Students can make a measurement with a reasonable feel for uncertainty
and accuracy of the measurement.
Students can correctly answer questions on an exam regarding the basic
logic of inferential statistics.
Students can correctly compute the mean, median, and standard deviation
of a data set.
Students can examine the frequency distribution of a data set and correctly
determine whether the sample was drawn from a normal distribution.
Students have good presentation skills; they show confidence
and professionalism.
Individual faculty members said they knew they were successful when:
Students become comfortable with data presentation.
When students can clearly present experimental findings.
The student's self confidence in their scientific ability increases.
Students write in a professional style/quality.
Individual faculty members said they knew they were successful when:
Students can write an APA-style results section.
Students can write papers approaching professional caliber.
Students can write in scientific style.
Students report that they are well prepared for the next
class or stage in their career.
Individual faculty members said they knew they were successful when:
A student returns after her first year of graduate school and says she
had no trouble with the theory.
Students engage with complex data and find patterns (independently).
They think creatively about data, observations, literature sources.
Individual faculty members said they knew they were successful when:
Students think creatively and independently.
Students muck around in primary sources and find direction without direct
professorial intervention-cope with "open-endedness" of historical research
and analysis.
Students read the literature to improve their hypothesis or their experimental
techniques.
Students show understanding of lab/computer techniques
well enough to trouble shoot.
Individual faculty members said they knew they were successful when:
Students can troubleshoot problems.
Students understand the technique they are using well enough so they can
figure out what to do if something goes wrong.
Students can turn a qualitative statement of a problem
or observation into a quantitative statement/model. Also the reverse;
be able to turn quantitative results into statements of findings, arguments.
Individual faculty members said they knew they were successful when:
Students can turn "English" questions into quantitative analyses and perform
the appropriate tests.
Students can successfully model a problem on their own or in groups, demonstrated
via a report, etc.
Students are aware that mathematical modeling is an iterative process.
Students can take a real world problem, construct a mathematical model
of it, use software to solve the mathematical model, and use the solution
to say something useful about the original problem.
Students carry forward into the future. They may go on
to do an independent project and work collegially with the professor,
get into graduate programs, and/or get research jobs/careers.
Individual faculty members said they knew they were successful when:
The student and I give a joint talk or publish a paper together.
Students successfully apply what they have learned elsewhere; on the job,
in grad school.
Students go on to do independent research or honors work.
|