Things to Consider when Choosing Method
Choosing the appropriate methods is a critical step to assess accurately a unit’s performance. Here are some things to keep in mind when choosing from the wide range of methods available:
- Choose methods that yield data that can be used for improvements or changes
Example of inappropriate method choice:
- PO: Student receiving advising will be able to register for classes independently.
- Method: Measure student participation rate in advising.
- Problem: Participation does not equate to competence. A method that captures the student actual behavior/competency would be more appropriate.
- Choose methods that correspond to the unit performance or service
Example of inappropriate method choice:
- PO: The Divisions of Academic Affairs and Student Affairs will receive appropriate IT support to integrate and standardize campus-wide advising system.
- Method: Measure student retention rates.
- Problem: The method does not measure the role that was performed (support developing the advising system), it measures a result of the advising system.
- Choose methods that actually address the corresponding performance outcome
Example of inappropriate method choice:
- PO: Students at CSUF receive similar employment seeking support to that at other CSU campuses.
- Method: Measure student satisfaction using survey developed by the appropriate CSUF unit.
- Problem: Locally developed survey does not yield information that can be easily compared with student experience at other campuses. An externally validated survey or a survey that is collaboratively developed by the CSU community would be more appropriate.
- Choose methods that are sustainable overtime
Example of inappropriate method choice:
- PO: Student receiving advising will demonstrate confidence in navigating the college system.
- Method: Interview every student about their self-perceived ability to navigate the college system; analyze interview transcripts for themes.
- Problem: Depending on the number of students, interview and subsequent analyses could be very time consuming and may not be sustainable overtime. Using a validated questionnaire, supplemented by interviewing a small, representative sample of students, would be more appropriate.
- Choose multiple methods for each performance outcome
Example of inappropriate method choice:
- PO: Students can access relevant policies and protocols in an easy and clear manner.
- Method: Analyze hits to website pages that contain policies and protocols.
- Problem: While this measure will document that policy information was accessed, it does not measure if the policy information was “clear.” Adding a communication section to the Customer Service Center Satisfaction survey could measure if policy information is communicated clearly.
