Data Collection Methods

Adapted from University of Hawai`i at Mānoa

Note: The information and resources contained here serve only as ideas that departments may consider as they develop their assessment plans.

Ideally, a program collects both direct and indirect evidence of student learning. Direct evidence, which is required, can reveal what students have learned and to what degree, but it does not provide information as to why the student learned or did not learn. The why is valuable because it can guide faculty members in how to interpret results and make improvements. Indirect evidence can be used to answer why questions. Programs can collect both direct and indirect evidence of student learning to gain a better picture of their students.

Types of Direct Data-Collection Methods

Direct Methods	Examples
Licensure or certification	Nursing program students’ pass rates on the NCLEX (Nursing) examination.
National exams or standardized tests	a) Freshmen and seniors’ scores on the Collegiate Learning Assessment (CLA) or Collegiate Assessment of Academic Proficiency (CAAP) b) Senior-level biology students’ scores on the GRE Subject Test on Biology.
Embedded testing or quizzes	a) Students’ pass rates on the German 202 final exam (students in all sections of German 202 take the same final exam). b) Two questions from the History 151 final exams are scored by a team of faculty members, and results are used for program-level assessment.
Embedded assignments	The program selects course assignments (“signature assignments”) that can provide information on a student learning outcome. Students complete these assignments as a regular part of the course and instructors grade the assignments for the course grade. In addition, the assignments are scored using criteria or a scoring rubric and these scores are used for program-level assessment. Examples: a) Course instructor and an outside faculty member apply a rubric to evaluate case studies written by students in BUS 301. b) A team of faculty members apply a rubric to evaluate videos of students’ oral presentations given in Oral Communication Focus courses.
Grades calibrated to clear student learning outcome(s)	Professors give grades based on explicit criteria (i.e., a departmentally approved rubric that is directly related to particular learning outcomes. (See also “embedded testing or quizzes” and “embedded assignments.”)
Portfolios	A collection of student work such as written assignments, personal reflection, and self-assessments. Developmental portfolios typically include work completed early, middle, and late in the students’ academic career so growth can be noted. Professional portfolios include students’ best work and aim to show the students’ highest achievement level.
Pre- post-tests	When used for program assessment, students take the pre-test as part of a required, introductory course. They take the post-test during their senior year, often in a required course or capstone course. Example: Students in Speech 151 and Speech 251 take a multiple-choice test. The semester that Speech majors and minors graduate, they make an appointment to take the same test.
Employer’s or internship supervisor’s direct evaluations of students’ performances	Evaluation or rating of student performance in a work, internship, or service-learning experience by a qualified professional.
Observation of student performing a task	Professor or an external observer rates each students’ classroom discussion participation using an observation rubric.
Culminating project: capstone projects, senior theses, senior exhibits, senior dance performance	Students produce a piece of work or several pieces that showcase their cumulative experiences in a program. The work(s) are evaluated by a pair of faculty members, a faculty team, or a team comprised of faculty and community members.
Student publications or conference presentations	Students present their research to an audience outside their program. Faculty and/or external reviewers evaluate student performance.
Description or list of what students learned	Students are asked to describe or list what they have learned. The descriptions are evaluated by faculty in the program and compared to the intended student learning outcomes. Example: After completing a service-learning project, students are asked to describe the three most important things they learned through their participation in the project. Faculty members evaluate the descriptions in terms of how well the service-learning project contributed to the program outcomes.

Types of Indirect Data-Collection Assessment Methods

Indirect Methods	Example
Student surveys	Students self-report via a questionnaire (online, or paper) about their ability, attitudes, and/or satisfaction. E.g., Students answer questions about their information literacy competence via an online questionnaire.
Alumni surveys	Alumni report their perceptions via a questionnaire (online, telephone, or paper). E.g., alumni answer questions during a telephone survey about the importance of particular program learning outcomes and whether they are pertinent to their current career or personal life.
Employer surveys	Potential employers complete a survey in which they indicate the job skills they perceive are important for college graduates. Note: if the survey asks employers to directly evaluate the skills, knowledge, and values of new employees who graduated from Brockport, the survey can be considered a direct method of evaluating students.
Focus group interviews	Face-to-face, one-to-many discussions or question/answer session. E.g., A graduate student lead a focus group of 4-5 undergraduate students who were enrolled in Math 111. The graduate student asked the undergraduates to discuss their experiences in the course, including difficulties and successes.
Grades given by professors that are not based on explicit criteria directly related to a learning outcome	Grade point averages or grades of students in a program. E.g., 52% of the students in Written Communication courses received an “A,” “A+” or “A-” grade.
Job placement data	The percent of students who found employment in a field related to the major/program within one year.
Enrollment in higher degree programs	The number or percent of students who pursued a higher degree in the field.
Maps or inventories of practice	A map or matrix of the required curriculum and instructional practices/signature assignments.
Transcript analysis or course-taking patterns	The actual sequence of courses (instead of the program’s desired course sequence for students).
Institutional/Program Research data	Information such as the following: Registration or course enrollment data Class size data Graduation rates Retention rates Grade point averages Specific examples: a) Number of sections with wait-listed students during registration. b) Percent of seats filled by majors. c) Number of students who dropped a course after first day of classes. d) Average enrollment in Writing Intensive sections by course level.