Student Evaluations Bibliography: SETs Bibliography

American Sociological Association (2019). “Statement on Student Evaluations of Teaching.” This statement asserts that “student feedback should not be used alone as a measure of teaching quality. If used in faculty evaluation processes, it should be considered as part of a holistic assessment of teaching effectiveness.” It provides a detailed list of actions that can be taken by institutions to improve the assessment of teaching, with examples from a number of colleges and universities. 

This statement has been endorsed by the following professional organizations: 
American Anthropological Association, American Dialect Society, American Folklore Society, American Historical Association, American Political Science Association, Archeological Institute of America, Association for Slavic, East European, and Eurasian Studies, Association for Theatre in Higher Education,  Canadian Sociological Association, Dance Studies Association, International Center of Medieval Art, Korean American Communication Association, Latin American Studies Association, Middle East Studies Association, National Communication Association, National Council on Family Relations, National Council on Public History, Rhetoric Society of America, Society for Cinema and Media Studies, Society for Classical Studies, Society for Personality and Social Psychology, Society of Architectural Historians, and Sociologists for Women in Society. https://www.asanet.org/sites/default/files/asa_statement_on_student_evaluations_of_teaching_feb132020.pdf

Association of Departments of Foreign Language. (2001). “ADFL Statement of Good Practice: Teaching, Evaluation, and Scholarship “Programs and procedures should maintain a locally appropriate balance between teaching and scholarship and reflect a well-considered sense of what constitutes good practice in teaching and evaluation.”
https://www.adfl.mla.org/Resources/Policy-Statements/ADFL-Statement-of-Good-Practice-Teaching-Evaluation-and-Scholarship

Hansen, W.L. (2014). “Rethinking the Student Course Evaluation: How A Customized Approach Can Improve Teaching and Learning.” Liberal Education (AACU), vo. 100, no. 3. Argues that traditional SETs are “ill-suited” to assess courses and instructors and student-centered evaluations of learning have stronger long-term benefits for both students and instruction. https://drive.google.com/drive/u/0/folders/1a5408xvRfopiYr9Yw0I4if0m8wvNd2ip 

Lawrence, J. W. (2018). “Student Evaluations of Teaching Are Not Valid.” AAUP, May-June 2018. Summarizes recent research on the limitations of using SETs and concludes that other strategies, including institutional-level changes, have a better chance of improving student learning outcomes and ensuring student success.
https://www.aaup.org/article/student-evaluations-teaching-are-not-valid#.YVctZ5rMKUk

General Studies and Bibliographies

Balasubramanian, A. (2003). “From the Student’s View: Student Course Evaluations.” Vanderbilt University Center for Teaching. Presents informal qualitative responses from eight students about their perceptions of student evaluations of teaching. https://drive.google.com/drive/u/0/folders/1a5408xvRfopiYr9Yw0I4if0m8wvNd2ip

Borch, I.; Sandvoll, R.; and Risør, T. (2020). “Discrepancies in purposes of student course evaluations: what does it mean to be ‘satisfied’?” Educational Assessment, Evaluation, and Accountability, vol. 32, pp. 83-102. Found that “students expected all evaluations to be learning oriented and were surprised by the teaching focus in surveys. This discrepancy caused a gap between students’ expectations and the evaluation practice.” Recommends dialogue-based evaluation methods as a promising alternative or supplement to SETs.
https://link.springer.com/article/10.1007/s11092-020-09315-x 

Homan, M.; Key, E., and Kreitzer, R. (2021). “Evidence of Bias in Standard Evaluations of Teaching”. A collectively peer-compiled Google doc of research studies on implicit bias in student evaluations. An extensive bibliography focusing particularly on bias based on gender, race, and perceived “difference.”
https://docs.google.com/document/d/14JiF-fT--F3Qaefjv2jMRFRWUS8TaaT9JjbYke1fgxE/edit

Falkoff, Michelle. (2018). “Why We Must Stop Relying on Student Evaluations of Teaching.” Chronicle of Higher Education 25 April 2018. Reviews the APSA study and concludes “Moving beyond reliance on student evaluations may take more time and effort, but it will also help us ensure that we are helping instructors succeed while eliminating the possibility that bias will play a role in making or breaking their careers.” . https://www.chronicle.com/article/why-we-must-stop-relying-on-student-ratings-of-teaching/

Heller, Janet Ruth (2012). “Contingent Faculty and the Evaluation Process.” College Composition and Communication, vol. 64, no. 1, pp. A8-A12. “If colleges involve instructors in shaping policies and evaluation procedures, non-tenure-track faculty will feel more at home in higher education, they will waste less energy combating bureaucratic discrimination, and they will become better teachers.”
https://www.jstor.org/stable/23264922?seq=4#metadata_info_tab_contents

Jirovec, R. K., Ramanatham, C.S., and Alvarez, A. R. (1998). “Course Evaluations: What Are Social Work Students Telling Us About Teaching Effectiveness?” J. Social Work Education, Spring/Summer 1998, vol.34, no. 2, pp. 229-236. Study of over 5000 student evaluations collected from 1991-93 showed a strong positive relationship between ratings of skills reflecting course organization, rapport with students, and fair grading.   https://www.jstor.org/stable/23043771

Koyama, Dennis (2020). “Research on Student Ratings of Instruction: Implications for Teachers and Professional Development Programs. In Innovations in Educational Leadership and Continuous Teachers’ Professional Development, ed. Osama Al Mahdi. CSMFL Publications, pp. 1-38. A review of research, focusing on studies dealing with class size, GPA, gender, and faculty rank, and discussing responses that may indicate implicit bias. https://books.csmflpublications.com/wp-content/uploads/2020/10/IEL_1_ChapterPreview.pdf

Kreitzer, R.J., and Sweet-Cushman, J. (2021). “Evaluating Student Evaluations of Teaching: A Review of Measurement and Equity Bias in SETs and Recommendations for Ethical Reform. J. Academic Ethics (January 2021). A meta-study reviewing over 100 studies of SETs concludes that “scholars using different data and different methodologies routinely find that women faculty, faculty of color, and other marginalized groups are subject to disadvantage in SETs.” Makes recommendations for the “judicious use of SETs as part of faculty evaluation. https://link.springer.com/article/10.1007/s10805-021-09400-w 

Lee, Theresa Man Ling (2021). “What to do with student evaluations of teaching?” Academic Matters (Journal of the Ontario Confederation of University Faculty Associations), 8 March 2021. Reports on litigation against Canadian universities for implicit bias in teaching evaluations, and concludes that By getting rid of a practice that is found to sustain systemic discrimination against members of minority and equity-seeking groups, the academic community will be taking an important step toward correcting its own injustices. Not only will this buttress our respect for each other as colleagues, it will earn respect from society more broadly.”
https://academicmatters.ca/what-to-do-with-student-evaluations-of-teaching/

Luo, M.N. (2020). “Student Response Rate and Its Impact on Qualitative Evaluation of Faculty Teaching.” The Advocate, vol. 25, no. 2, Spring-Summer 2020. Analyzing student evaluations of teaching in a large college of education, this study found that courses with lower response rates tend to rate faculty as less effective teachers, and offers suggestions for how such bias might be addressed.  
https://newprairiepress.org/advocate/vol25/iss2/6/

Sánchez, Tarquino, et al. (2020). “Students’ Evaluation of Teaching and Their Academic Achievement in a Higher Education Institution of Ecuador.” Frontiers in Psychology 6, March 2020; “The results revealed that SET was not related to academic achievement, once the effect of previous academic achievement was controlled. From these results, it follows that the use of SET as a measure of teachers’ effectiveness for making administrative decisions remains controversial.”
https://www.frontiersin.org/articles/10.3389/fpsyg.2020.00233/full

Stroebe, Wolfgang. (2020) “Student Evaluations of Teaching Encourages Poor Teaching and Contributes to Grade Inflation: A Theoretical and Empirical Analysis. Basic and Applied Psychology, vol. 42 no. 4, pp. 276-294. “This article reviews research that shows that students (a) reward teachers who grade leniently with positive SETs, (b) reward easy courses with positive SETs, and (c) choose courses that promise good grades. The study also shows that instructors want (and need) good SETs.”
https://www.tandfonline.com/doi/full/10.1080/01973533.2020.1756817

Wines, W. A., and Lau, T. J. (2006). “Observations on the Folly of Using Student Evaluations of College Teaching for Faculty Evaluation, Pay, and Retention Decisions and Its Implications for Academic Freedom.” William and Mary Journal of Women and the Law, vol. 13, no. 1, October 2006. “This paper concludes that the use of non-validated student evaluations alone, without any other criteria for teaching effectiveness, raises substantial problems in faculty retention and promotion decisions. It also suggests that such an approach in the right case might violate academic freedom and the First Amendment.”  
https://scholarship.law.wm.edu/wmjowl/vol13/iss1/4/. 

Other Perceived Bias Factors

Cruz-Boone, C.; Boschini, D.; Sherman, T. (2015). “Evaluating Adjunct Professors for Promotion: A Case Study Approach to Review Adjunct Student Evaluation of Teachers over Time.” Journal of Leadership, Equity, and Research, vol. 2, no. 1. Follows an evaluation of one instructor and concludes that “in some instances, educational leaders in higher education should shift their thinking about the way contingent faculty are evaluated, promoted, and included in academic institutions.”
https://journals.sfu.ca/cvj/index.php/cvj/article/view/15

Langen, J. M. (2011). “Evaluation of adjunct faculty in higher education institutions.” Assessment and Evaluation in Higher Education, vol. 36, no. 2, pp. 185-196. Reviews studies of the evaluation of adjunct faculty members and how those evaluations are used in employment decisions. Concludes that adjunct faculty should be evaluated using a teaching portfolio and not on SETs alone. 
https://research.ebsco.com/c/e266yq/viewer/pdf/icfwtwlqun?route=details 
​​​​​

Ahmad, T. (2018). “Teaching evaluation and student response rate.” PSU Research Review vol. 2, no. 3. Discusses student skepticism in participating in online evaluations of teaching and suggests ways lecturers can initiate more student engagement through effective communication. 
https://www.emerald.com/insight/content/doi/10.1108/PRR-03-2018-0008/full/html

Asare, S., and Daniel, B.K. (2017). “Factors influencing response rates in online student evaluation systems: A systematic review approach.” Conference paper. “We carried out a systematic review of the literature and identified four key factors that are likely to influence response rates in online student evaluation. These include survey development, survey delivery, survey return and survey completion. We argue that these factors are critical for improving online response rates.” https://www.researchgate.net/publication/324274298_Factors_influencing_response_rates_in_online_student_evaluation_systems_A_systematic_review_approach

Berk, R. A. (2012). “Top 20 Strategies to Increase the Online Response Rates of Student Rating Scales.” International Journal of Technology in Teaching and Learning, vol. 8, no. 2, pp. 98-107. Written at the beginning of the shift to online evaluations, this article reviews the research and extant technology for gathering online evaluation data and offers strategies to increase the response rate. 
https://www.emerald.com/insight/content/doi/10.1108/PRR-03-2018-0008/full/html

Chapman, D. D., and Joines, J.A. (2017). “Strategies for Increasing Response Rates for Online End-of-Course Evaluations.” International Journal of Teaching and Learning in Higher Education, vol. 29, no. 1, pp. 47-60. Reviews the experiences of shifting to online evaluations at a large public university and the skepticism about the validity of a small number of responses. Describes efforts by faculty to better communicate with students and the development of an FAQ document to help faculty increase response.
https://files.eric.ed.gov/fulltext/EJ1136018.pdf

Fike, David S., et al. (2010). “Online vs. Paper Evaluations of Faculty: When Less is Just as Good.” Journal of Effective Teaching vol. 10, no. 2, pp. 42-54. This study compared two methods of student evaluations of faculty, online versus the traditional paper format. Although the student response rate for online evaluations was lower, evaluation scoring patterns were similar for both methods. The findings suggest that conducting faculty evaluations online may be a suitable alternative to the traditional, paper-based approach. 
https://uncw.edu/jet/articles/vol10_2/fike.pdf

Guder, F., and Malliaris, M. (2013). “Online Course Evaluations Response Rates.” American Journal of Business Education vol. 6, no. 3, May/June 2013. Looks at low rates of return in business classes and concludes that reminder emails and faculty discussion of the importance of the information collected help improve response rate
 https://core.ac.uk/download/pdf/48605218.pdf

Hativa, N. (2013). “Answers to Faculty Concerns about Online Versus In-class Administration of Student Ratings of Instruction (SRI).” In Student Ratings of Instruction: A Practical Approach to Designing, Operating, and Reporting (Oron Press). Provides research-based answers to four major faculty concerns about online evaluation of teaching effectiveness. 
https://uedocs.ucsc.edu/sites/default/files/uedocs/Online_vs_paper_evals.pdf

He, J., and Freeman, L. A. (2020). “Can we trust teaching evaluations when response rates are not high? Implications from a Monte Carlo simulation.” Studies in Higher Education vol. 46, no 9, pp. 1932-1948. Uses statistical modeling to confirm that lower response rates are less likely to be reliable in measurement of student evaluation of teaching, and that faculty and administrators should be “skeptical” in relying on SET information when there is a low rate of return.
https://research.ebsco.com/c/e266yq/viewer/pdf/4zbtibq4kj?route=details 

Ogden, D.T., and Ogden, J.R. (2018). “Suggestions to Increase Course Evaluation Response Rates in Online Courses.” J. Higher Education Theory and Practice vol. 18, no. 4, pp. 10-15. Summaries research on online evaluation results and the resultant skepticism of their worth, and suggests both methods to use and methods to avoid in compiling usable and reliable information from such tools. 
https://research.ebsco.com/c/e266yq/viewer/pdf/4jlyljwgov?route=details 

Stanny, C. J, & Arruda, J. E. (2017). “A Comparison of Student Evaluations of Teaching with Online and Paper-Based Administration.” Scholarship of Teaching and Learning in Psychology, vol. 3, no. 3, pp. 198-207. This study examined SETs for 364 courses taught in three consecutive fall terms. As expected, response rates (and average rating) declined when forms were administered online instead of face-to-face. The authors discuss how reviewers should interpret findings and suggest that reviewers consider additional sources of information when they evaluate teaching quality. 
https://psycnet.apa.org/record/2017-26724-001

Thomas, Rebecca Arlene, UPCEA (University Professional and Continuing Education Association (2021). “Evaluating Online Teaching: Interviewing Instructors with 10+ Years Experience.” 3 May 2021. Examines differences between evaluating online and in-person instruction and offers key takeaways and questions to help institutions and faculty adjust their evaluation processes for online instruction. 
https://upcea.edu/evaluating-online-teaching-interviewing-instructors-with-10-years-experience/

American Association of University Professors (2020). “Principles for Higher Education Response to COVID-19.” 13 March 2020. Lays out 22 principles to guide universities and faculties in equitable practices during emergencies. Insists that institutional decisions should take place in consultation with appropriate faculty decision-making bodies. 
https://www.aft.org/sites/default/files/covid19_aft-aaup_guidanceprinciples_031320.pdf

Boysen, Guy A.(2020).  “Student Evaluations of Teaching During the COVID-19 Pandemic.” Public Pre-print, Scholarship of Teaching and Learning in Psychology, July 2020. Provides comparative results of student evaluation scores from before and during the pandemic. DO!: 10.1037/stl0000222. https://www.researchgate.net/publication/343222555_Student_Evaluations_of_Teaching_During_the_COVID-19_Pandemic

Garris, Christopher, and Bethany Fleck (2020). “Student evaluations of transitioned-online courses during the COVID-19 pandemic.” Scholarship of Teaching and Learning in Psychology. Online First Publication 19 October 2020. “The transition was overall evaluated negatively, specifically that the courses became less enjoyable, less interesting, decreased in learning value, facilitated less attention and effort, and incorporated less cultural content after transitioning online. On a positive note, courses were perceived as becoming more flexible to students' needs after transitioning online. Evaluations of courses transitioning to online were consistently predicted by online self-efficacy, emotional well-being, computer anxiety, online student engagement, and student perceptions of instructor confidence with transitioning online.”  
https://psycnet.apa.org/fulltext/2020-77535-001.pdf

Lederman, Doug (2020). “Evaluating Teaching During the Pandemic.” Inside Higher Ed 8 April 2020. “Provides examples of how colleges and universities are altering evaluation strategies in the wake of the pandemic.” 
https://www.insidehighered.com/digital-learning/article/2020/04/08/many-colleges-are-abandoning-or-downgrading-student-evaluations

Leshchinskaya, Isana, and Claudio Piani (2021). “Student Evaluation of Teaching: COVID-19 Considerations for Validity and Fairness." ": AACU Liberal Education Blog, 7 August 2020. “While it is recommended that formal administration and usage of student evaluations be put on hold, it is still important for students to be able to provide feedback to their instructors regarding their learning experience in the new remote environment. A formative assessment of the instructor by the students—conducted several times throughout the semester at the end of each instructional unit in an anonymous manner—could be extremely useful for improving pedagogy in this new environment.” 
https://www.aacu.org/blog/student-evaluation-teaching-covid-19-considerations-validity-and-fairness

Supiano, Beckie (2021). “Teaching: How Should Colleges Use Course Evaluations in a Pandemic?” Chronicle of Higher Education, 11 February 2021. Describes how one university adjusted its practices, and shares some strategies for enacting change. 
https://www.chronicle.com/newsletter/teaching/2021-02-11
 

Center for Teaching Excellence, University of Iowa. “Student Evaluation of Teaching (SET) Guidelines” (2021).  A set of guidelines developed by a faculty review committee that concluded “The SET is one of several mechanisms for improving teaching and learning and should not be the only data source for evaluating instruction.”
https://www.celt.iastate.edu/teaching/assessment-and-evaluation/student-evaluation-of-teaching-set-guidelines-and-recommendations-for-effective-practice/

Hanson, W. Lee (2014). “Rethinking the Student Course Evaluation: How a Customized Approach Can Improve Teaching and Learning.” Liberal Education (AACU), vol. 100, no. 23, Summer 2014. “To equip students to practice their learning long after they graduate, individual faculty members must find new ways of combining course content, design, and pedagogy in order to engage students in their own learning more fully. They must give greater emphasis to what and how students learn, and what they can do with their learning after acquiring it. Above all, this means focusing on what students are learning and how their intellectual development is being stimulated in the process. Instituting a proficiency-based approach to instruction and learning offers an effective means of accomplishing these objectives.“
https://www.aacu.org/publications-research/periodicals/rethinking-student-course-evaluation

Rowan, Susan, DDS, et al. (2017). “Should Student Evaluation of Teaching Play a Significant Role in the Formal Assessment of Dental Faculty? Two Viewpoints.” J. Dental Education, vol. 81, no. 11, pp. 1362-1372. A point/counterpoint article summarizing the arguments for and against the use of formal SETs in assessing teaching and pointing to the differences between student satisfaction and student achievement in such ratings. 
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5812723/

Schwartz, D., and Muller, E. (2021). “A Summary and Analysis of Written Guidelines for Evaluating Teaching Effectiveness at UNC-Chapel Hill.” Prepared for the Committee on Appointments, Promotion, and Tenure. A comprehensive overview of the ways in which teaching effectiveness is measured by the various colleges, departments, and schools at UNC-CH, with summative guidelines and examples of how different measures can be used for a comprehensive assessment of teaching effectiveness.
https://provost.unc.edu/policies-committees/evaluating-teaching-effectiveness/

Teaching Effectiveness Committee, Auburn University (2018-2019.) “Review on the use of student evaluations of teaching (SETs), with recommendations for the assessment of teaching faculty for promotion, tenure, and annual review. Report of a blue-ribbon faculty committee that concludes reviews a university-wide effort to move from summative to formative evaluation of teaching, with recommendations for a multipart assessment model that does not rely on SETs alone.
http://www.auburn.edu/administration/governance/senate/agendas/2018-2019/Jan/TEC_rpt_SETsFinal(1).pdf

University of Wisconsin Student Learning Assessment (2021). “Best Practices and Sample Questions for Course Evaluation Surveys.”  Provides extensive resources for creating and revising survey questions, improving response rates, and a resource bank of sample questions for assessing student learning.
https://assessment.provost.wisc.edu/best-practices-and-sample-questions-for-course-evaluation-surveys/