Things to Consider when Choosing Method
Choosing the appropriate methods is a critical step to accurately assess student learning. Here are some things to keep in mind when choosing from the wide range of methods available:
Choose methods that yield data that can be used for improvements or changes
Example of inappropriate method choice:
- SLO: Student completing the program will demonstrate competence in conducting research.
- Method: Measure student participation rate in the senior research project.
- Problem: Participation does not equate to competence. A method that captures the quality of student research project would be more appropriate.
Choose methods that actually address the corresponding student learning outcome
Example of inappropriate method choice:
- SLO: Student completing the program will demonstrate level of mastery of key concepts that is comparable to students of similar program at other CSU campuses.
- Method: Measure student mastery of key concepts using final exams developed collaboratively by the course instructors.
- Problem: Locally developed exam does not yield information that can be easily compared with student performance at other campuses. An externally validated test or an exam agreed upon by faculty at multiple campuses would be more appropriate.
Choose methods that are sustainable overtime
Example of inappropriate method choice:
- SLO: Student completing the program will demonstrate confidence in conducting research.
- Method: Interview every student about their self-perception of research self-efficacy; analyze interview transcripts for themes.
- Problem: Depending on the number of students, interview and subsequent analyses could be very time consuming and may not be sustainable overtime. Using a validated questionnaire on research self-efficacy, supplemented by interviewing a small, representative sample of students, would be more appropriate.
Choose instruments that are valid and reliable
Example of inappropriate method choice:
- SLO: Student completing the program will demonstrate confidence in conducting research.
- Method: Measure student self-perception of research self-efficacy using part of the exit survey of science students.
- Problem: The exit survey items are not specifically targeted at research self-efficacy, but student self-concept in science learning in general.
Use of externally validated questionnaire designed for research self-efficacy would be more appropriate.
Choose multiple methods for each student learning outcome
Example of inappropriate method choice:
- SLO: All SLOs in the program
- Method: Ask students to write final essays, and develop different grading rubrics to assess every SLO.
- Problem: Developing and using rubrics can be time consuming, and achieving interrater reliability can be tricky. Using only the rubric approach misses out on many other simpler assessment opportunities. Diversifying the types of methods (quantitative and qualitative) used is more appropriate.
Choose methods that are meaningful for both faculty and students
Example of inappropriate method choice:
- SLO: Student completing the program will demonstrate critical thinking skills.
- Method: Ask students to voluntarily take an hour-long exam on critical thinking at the end of course completion that does not provide any immediate benefit to the students.
- Problem: Without any incentives or meaningful explanations, the students who volunteer are likely to be a skewed sample of the students in the course. In addition, they may not take the exam seriously enough to demonstrate their actual skill level. Provide reasons why students should take the exam, or choose a method that is more meaningful to the students.
(adapted from University of Central Florida Assessment Handbook
)
