Authors retain the copyright without restrictions for their published content in this journal. HSSR is a SHERPA ROMEO Green Journal.
Publishing License
This is an open-access article distributed under the terms of
STUDENT SURVEY FOR HIGHER EDUCATION QUALITY – CHALLENGES TO DESIGN AND ANALYSE DATA
Corresponding Author(s) : Oleksandr O. Torbas
Humanities & Social Sciences Reviews,
Vol. 8 No. 2 (2020): Education Special Issue
Abstract
Purpose of the study: The objective of this study was to present useful practical advice on the process of designing questionnaires for student surveys for higher education quality, as well as conducting such survey through the analysis of results of such survey that was held in National University "Odesa Law Academy".
Methodology: This research is based on the results of an anonymous online survey among 142 students of the Faculty of Advocacy of the National University "Odesa Law Academy", which was conducted in April 2017. The survey consisted of 29 questions. All questions were divided into 3 separate groups: assessment of academic discipline, assessment of lectures, and assessment of practical classes. Each group consisted of open-ended and closed-ended questions. Closed-ended questions were evaluated on a 10-point scale, where 1 is the lowest score, and 10 is the highest.
Main Findings: The research findings demonstrate that to increase the response rate of student it's better to reduce the number of questionnaires and questions in them. Also, the authors stated that the online form of survey is better than paper form (in aspects of accessibility, anonymity, and convenience for students). It was found that the vast majority of students' responses were received during the first days of the survey, which indicates that it is impractical to be conducting such a survey over a long period. It was found that in response to closed-ended questions most students chose the highest mark (10 out of 10), so developers of questionnaires have to choose another scale for closed-ended question assessment which has some real references. Also, it was stated that only a fraction of open-ended questions will contain really useful information that can be further used to improve educational services.
Application of this study: This research can be used by administration staff and teachers of universities as a guideline in conducting similar surveys.
Novelty/Originality of this study: The novelty of this research lies in the attempt of combining relatively easy in implementation but quite effective in usage rules of conducting student’s survey on higher education quality in one paper.
Keywords
Download Citation
Endnote/Zotero/Mendeley (RIS)BibTeX
- Artino, A.R. Jr., La Rochelle, J.S., Dezee, K.J., & Gehlbach, H. (2014). Developing questionnaires for education research: AMEE Guide No. 87. Medical Teacher, 36:6, 463-474. https://doi.org/10.3109/0142159X.2014.889814 DOI: https://doi.org/10.3109/0142159X.2014.889814
- Balam, E. M., &Shannon, D. M. (2010). Student ratings of college teaching: a comparison of faculty and their students. Assessment & Evaluation in Higher Education, 35(2), 209–221. https://doi.org/10.1080/02602930902795901 DOI: https://doi.org/10.1080/02602930902795901
- Bennett, L., &Nair, C. S. (2011). Web-based or paper-based surveys: a quandary? Student Feedback, 119–131. https://doi.org/10.1016/B978-1-84334-573-2.50007-0 DOI: https://doi.org/10.1016/B978-1-84334-573-2.50007-0
- Coffey, M.,& Gibbs, G. (2001). The Evaluation of the Student Evaluation of Educational Quality Questionnaire (SEEQ) in UK Higher Education. Assessment & Evaluation in Higher Education, 26(1), 89–93. https://doi.org/10.1080/02602930020022318 DOI: https://doi.org/10.1080/02602930020022318
- Douglas, J., Douglas, A., &Barnes, B. (2006). Measuring student satisfaction at a UK university. Quality Assurance in Education, Vol. 14, Issue: 3, 251-267. https://doi.org/10.1108/09684880610678568 DOI: https://doi.org/10.1108/09684880610678568
- El Hassan, K. (2009). Investigating substantive and consequential validity of student ratings of instruction. Higher Education Research and Development, 28(3), 319–333. https://doi.org/10.1080/07294360902839917 DOI: https://doi.org/10.1080/07294360902839917
- Grammatikopoulos, V., Linardakis, M., Gregoriadis, A., et al. (2014) Assessing the students’ evaluations of educational quality (SEEQ) questionnaire in Greek higher education. HIGHER EDUCATION, 70(3), 395-408. https://doi.org/10.1007/s10734-014-9837-7 DOI: https://doi.org/10.1007/s10734-014-9837-7
- Green, M. F., Marmolejo, F., & Egron-Polak, E. (2012). The internationalization of higher education: Prospects. In D. K. Deardorff, H. de Wit, J. D. Heyl, & T. Adams (Eds.), The SAGE handbook of international higher education (pp. 439-457). https://doi.org/10.4135/9781452218397.n24 DOI: https://doi.org/10.4135/9781452218397.n24
- Karlberg, C. (2015). The survey fatigue challenge: understanding young people’s motivation to participate in survey research studies. Lunds Universitet, Department of Psychology.
- Marsh, H. W. (1982). SEEQ: A reliable, valid, and useful instrument for collecting students’ evaluations of university teaching. British Journal of Educational Psychology, 52(1), 77–95 https://doi.org/10.1111/j.2044-8279.1982.tb02505.x DOI: https://doi.org/10.1111/j.2044-8279.1982.tb02505.x
- Marsh, H. W. (1986). Applicability paradigm: students’ evaluations of teaching effectiveness in different countries. Journal of Educational Psychology, 78(6), 465–473. https://doi.org/10.1037//0022-0663.78.6.465 DOI: https://doi.org/10.1037//0022-0663.78.6.465
- Marsh, H. W. (1987). Students’ evaluations of university teaching: Research findings, methodological issues, and directions for future research. International Journal of Educational Research, 11(3), 253–388. https://doi.org/10.1016/0883-0355(87)90001-2 DOI: https://doi.org/10.1016/0883-0355(87)90001-2
- Marsh, H. W., &Dunkin, M. J. (1992). Students’ evaluations of university teaching: a multidimensional perspective. In R. P. Perry & J. C. Smart (Eds.). Effective teaching in higher education: Research and practice, 241–320.
- Marsh, H. W., &Hocevar, D. (1991). The multidimensionality of students’ evaluations of teaching effectiveness: the generality of factor structures across academic discipline, instructor level, and course level. Teaching and Teacher Education, 7(1), 9–18. https://doi.org/10.1016/0742-051X(91)90054-S DOI: https://doi.org/10.1016/0742-051X(91)90054-S
- McCann, S., &Gardner, C. (2014). Student personality differences are related to their responses to instructor evaluation forms. Assessment & Evaluation in Higher Education, 39(4), 412–426. https://doi.org/10.1080/02602938.2013.845647 DOI: https://doi.org/10.1080/02602938.2013.845647
- NaUKMA (2016). Survey report on the quality of teaching in NaUKMA (in Ukrainian). NaUKMA. http://www.ukma.edu.ua/index.php/osvita/fakulteti/78-education/3379-tsentr-zabezpechennia-iakosti-osvity
- O’Neill, M.A., &Palmer, A. (2004). Importance-performance analysis: a useful tool for directing continuous quality improvement in higher education. Quality Assurance in Education, Vol. 12 Issue: 1, 39-52, https://doi.org/10.1108/09684880410517423 DOI: https://doi.org/10.1108/09684880410517423
- Phan, L.H., & Childs, K. (2017). Student surveys of teaching & learning quality. An issues paper to contextualize current practice in higher education. The University of Queensland, Australia.
- Porter, S. R., Whitcomb, M. E., &Weitzer, W. H. (2004). Multiple surveys of students and survey fatigue.New Directions for Institutional Research.2004 (121), 63-73. https://doi.org/10.1002/ir.101 DOI: https://doi.org/10.1002/ir.101
- Puzziferro, M.,& Shelton, K. (2008). A model for developing high-quality online courses: integrating a systems approach with learning theory. Journal of Asynchronous Learning Networks, 12(3), 119-136. https://doi.org/10.24059/olj.v12i3-4.1688 DOI: https://doi.org/10.24059/olj.v12i3-4.1688
- Ramsden, P. (1991). A performance indicator of teaching quality in higher education: The Course Experience Questionnaire. Studies in Higher Education, 16:2, 129-150. https://doi.org/10.1080/03075079112331382944 DOI: https://doi.org/10.1080/03075079112331382944
- Rienties, B. (2014). Understanding academics’ resistance towards (online) student evaluation. Assessment & Evaluation in Higher Education, 39(8), 987–1001. https://doi.org/10.1080/02602938.2014.880777 DOI: https://doi.org/10.1080/02602938.2014.880777
- Risquez, A., Vaughan, E., &Murphy, M. (2015). Online student evaluations of teaching: what are we sacrificing for the affordances of technology? Assessment & Evaluation in Higher Education, 40:1, 120-134, https://doi.org/10.1080/02602938.2014.890695 DOI: https://doi.org/10.1080/02602938.2014.890695
- Rowley, J. (2003) Designing student feedback questionnaires. Quality Assurance in Education, Vol. 11 Issue: 3,142-149, https://doi.org/10.1108/09684880310488454 DOI: https://doi.org/10.1108/09684880310488454
- Ryan, T. (2015). Quality assurance in higher education: a review of the literature. Higher Learning Research Communications, 5(4). https://doi.org/10.18870/hlrc.v5i4.257 DOI: https://doi.org/10.18870/hlrc.v5i4.257
- Schindler, L., Puls-Elvidge, S., Welzant, H., & Crawford, L. (2015). Definitions of quality in higher education: Asynthesis of the literature. Higher Learning Research Communications,5(3),3-13. https://doi.org/10.18870/hlrc.v5i3.244 DOI: https://doi.org/10.18870/hlrc.v5i3.244
- Student experience survey (2017). 2016 Student Experience Survey National Report (2017). Quality Indicators for Learning and Teaching. https://www.qilt.edu.au/docs/default-source/gos-reports/2017/2016-ses-infographic.pdf?sfvrsn=84ede33c_2
- USAID New Justice (2017). Methodology for independent, external, on-site assessment of legal education quality in Ukraine. USAID New Justice Program. https://newjustice.org.ua/wp-content/uploads/2017/09/ NJ_Methodology-for-Legal-Education-Quality-Assessment_ENG_May-24_2017_final.pdf
- Watkins, D., &Thomas, B. (1991). Assessing teaching effectiveness: an Indian perspective. Assessment & Evaluation in Higher Education, 16(3), 185–198. https://doi.org/10.1080/0260293910160302 DOI: https://doi.org/10.1080/0260293910160302
- Williams, J. (2015). The National Student Survey should be abolished before it does any more harm.The Guardian. https://www.theguardian.com/higher-education-network/2015/aug/13/the-national-student-survey-should-be-abolished-before-it-does-any-more-harm
- Williams, R., &Brennan, J. (2004). Collecting and using student feedback Date: A guide to good practice.York, UK: Higher Education Academy.
References
Artino, A.R. Jr., La Rochelle, J.S., Dezee, K.J., & Gehlbach, H. (2014). Developing questionnaires for education research: AMEE Guide No. 87. Medical Teacher, 36:6, 463-474. https://doi.org/10.3109/0142159X.2014.889814 DOI: https://doi.org/10.3109/0142159X.2014.889814
Balam, E. M., &Shannon, D. M. (2010). Student ratings of college teaching: a comparison of faculty and their students. Assessment & Evaluation in Higher Education, 35(2), 209–221. https://doi.org/10.1080/02602930902795901 DOI: https://doi.org/10.1080/02602930902795901
Bennett, L., &Nair, C. S. (2011). Web-based or paper-based surveys: a quandary? Student Feedback, 119–131. https://doi.org/10.1016/B978-1-84334-573-2.50007-0 DOI: https://doi.org/10.1016/B978-1-84334-573-2.50007-0
Coffey, M.,& Gibbs, G. (2001). The Evaluation of the Student Evaluation of Educational Quality Questionnaire (SEEQ) in UK Higher Education. Assessment & Evaluation in Higher Education, 26(1), 89–93. https://doi.org/10.1080/02602930020022318 DOI: https://doi.org/10.1080/02602930020022318
Douglas, J., Douglas, A., &Barnes, B. (2006). Measuring student satisfaction at a UK university. Quality Assurance in Education, Vol. 14, Issue: 3, 251-267. https://doi.org/10.1108/09684880610678568 DOI: https://doi.org/10.1108/09684880610678568
El Hassan, K. (2009). Investigating substantive and consequential validity of student ratings of instruction. Higher Education Research and Development, 28(3), 319–333. https://doi.org/10.1080/07294360902839917 DOI: https://doi.org/10.1080/07294360902839917
Grammatikopoulos, V., Linardakis, M., Gregoriadis, A., et al. (2014) Assessing the students’ evaluations of educational quality (SEEQ) questionnaire in Greek higher education. HIGHER EDUCATION, 70(3), 395-408. https://doi.org/10.1007/s10734-014-9837-7 DOI: https://doi.org/10.1007/s10734-014-9837-7
Green, M. F., Marmolejo, F., & Egron-Polak, E. (2012). The internationalization of higher education: Prospects. In D. K. Deardorff, H. de Wit, J. D. Heyl, & T. Adams (Eds.), The SAGE handbook of international higher education (pp. 439-457). https://doi.org/10.4135/9781452218397.n24 DOI: https://doi.org/10.4135/9781452218397.n24
Karlberg, C. (2015). The survey fatigue challenge: understanding young people’s motivation to participate in survey research studies. Lunds Universitet, Department of Psychology.
Marsh, H. W. (1982). SEEQ: A reliable, valid, and useful instrument for collecting students’ evaluations of university teaching. British Journal of Educational Psychology, 52(1), 77–95 https://doi.org/10.1111/j.2044-8279.1982.tb02505.x DOI: https://doi.org/10.1111/j.2044-8279.1982.tb02505.x
Marsh, H. W. (1986). Applicability paradigm: students’ evaluations of teaching effectiveness in different countries. Journal of Educational Psychology, 78(6), 465–473. https://doi.org/10.1037//0022-0663.78.6.465 DOI: https://doi.org/10.1037//0022-0663.78.6.465
Marsh, H. W. (1987). Students’ evaluations of university teaching: Research findings, methodological issues, and directions for future research. International Journal of Educational Research, 11(3), 253–388. https://doi.org/10.1016/0883-0355(87)90001-2 DOI: https://doi.org/10.1016/0883-0355(87)90001-2
Marsh, H. W., &Dunkin, M. J. (1992). Students’ evaluations of university teaching: a multidimensional perspective. In R. P. Perry & J. C. Smart (Eds.). Effective teaching in higher education: Research and practice, 241–320.
Marsh, H. W., &Hocevar, D. (1991). The multidimensionality of students’ evaluations of teaching effectiveness: the generality of factor structures across academic discipline, instructor level, and course level. Teaching and Teacher Education, 7(1), 9–18. https://doi.org/10.1016/0742-051X(91)90054-S DOI: https://doi.org/10.1016/0742-051X(91)90054-S
McCann, S., &Gardner, C. (2014). Student personality differences are related to their responses to instructor evaluation forms. Assessment & Evaluation in Higher Education, 39(4), 412–426. https://doi.org/10.1080/02602938.2013.845647 DOI: https://doi.org/10.1080/02602938.2013.845647
NaUKMA (2016). Survey report on the quality of teaching in NaUKMA (in Ukrainian). NaUKMA. http://www.ukma.edu.ua/index.php/osvita/fakulteti/78-education/3379-tsentr-zabezpechennia-iakosti-osvity
O’Neill, M.A., &Palmer, A. (2004). Importance-performance analysis: a useful tool for directing continuous quality improvement in higher education. Quality Assurance in Education, Vol. 12 Issue: 1, 39-52, https://doi.org/10.1108/09684880410517423 DOI: https://doi.org/10.1108/09684880410517423
Phan, L.H., & Childs, K. (2017). Student surveys of teaching & learning quality. An issues paper to contextualize current practice in higher education. The University of Queensland, Australia.
Porter, S. R., Whitcomb, M. E., &Weitzer, W. H. (2004). Multiple surveys of students and survey fatigue.New Directions for Institutional Research.2004 (121), 63-73. https://doi.org/10.1002/ir.101 DOI: https://doi.org/10.1002/ir.101
Puzziferro, M.,& Shelton, K. (2008). A model for developing high-quality online courses: integrating a systems approach with learning theory. Journal of Asynchronous Learning Networks, 12(3), 119-136. https://doi.org/10.24059/olj.v12i3-4.1688 DOI: https://doi.org/10.24059/olj.v12i3-4.1688
Ramsden, P. (1991). A performance indicator of teaching quality in higher education: The Course Experience Questionnaire. Studies in Higher Education, 16:2, 129-150. https://doi.org/10.1080/03075079112331382944 DOI: https://doi.org/10.1080/03075079112331382944
Rienties, B. (2014). Understanding academics’ resistance towards (online) student evaluation. Assessment & Evaluation in Higher Education, 39(8), 987–1001. https://doi.org/10.1080/02602938.2014.880777 DOI: https://doi.org/10.1080/02602938.2014.880777
Risquez, A., Vaughan, E., &Murphy, M. (2015). Online student evaluations of teaching: what are we sacrificing for the affordances of technology? Assessment & Evaluation in Higher Education, 40:1, 120-134, https://doi.org/10.1080/02602938.2014.890695 DOI: https://doi.org/10.1080/02602938.2014.890695
Rowley, J. (2003) Designing student feedback questionnaires. Quality Assurance in Education, Vol. 11 Issue: 3,142-149, https://doi.org/10.1108/09684880310488454 DOI: https://doi.org/10.1108/09684880310488454
Ryan, T. (2015). Quality assurance in higher education: a review of the literature. Higher Learning Research Communications, 5(4). https://doi.org/10.18870/hlrc.v5i4.257 DOI: https://doi.org/10.18870/hlrc.v5i4.257
Schindler, L., Puls-Elvidge, S., Welzant, H., & Crawford, L. (2015). Definitions of quality in higher education: Asynthesis of the literature. Higher Learning Research Communications,5(3),3-13. https://doi.org/10.18870/hlrc.v5i3.244 DOI: https://doi.org/10.18870/hlrc.v5i3.244
Student experience survey (2017). 2016 Student Experience Survey National Report (2017). Quality Indicators for Learning and Teaching. https://www.qilt.edu.au/docs/default-source/gos-reports/2017/2016-ses-infographic.pdf?sfvrsn=84ede33c_2
USAID New Justice (2017). Methodology for independent, external, on-site assessment of legal education quality in Ukraine. USAID New Justice Program. https://newjustice.org.ua/wp-content/uploads/2017/09/ NJ_Methodology-for-Legal-Education-Quality-Assessment_ENG_May-24_2017_final.pdf
Watkins, D., &Thomas, B. (1991). Assessing teaching effectiveness: an Indian perspective. Assessment & Evaluation in Higher Education, 16(3), 185–198. https://doi.org/10.1080/0260293910160302 DOI: https://doi.org/10.1080/0260293910160302
Williams, J. (2015). The National Student Survey should be abolished before it does any more harm.The Guardian. https://www.theguardian.com/higher-education-network/2015/aug/13/the-national-student-survey-should-be-abolished-before-it-does-any-more-harm
Williams, R., &Brennan, J. (2004). Collecting and using student feedback Date: A guide to good practice.York, UK: Higher Education Academy.