Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

The association between student ratings of specific instructional dimensions and student achievement: Refining and extending the synthesis of data from multisection validity studies

  • Published:
Research in Higher Education Aims and scope Submit manuscript

Abstract

This synthesis extends part of Peter Cohen's well-known examination of the association between student ratings of instruction and student achievement. The results of increasing the number of specific instructional dimensions considered and of preserving more of the information in the evaluations by multiple coding into these dimensions are compared with Cohen's findings. Extending the analysis further by changing the unit of analysis and the method of averaging correlations generally decreases the size of the associations. The instructor's preparation and organization, clarity and understandableness, stimulation of interest, motivation of students to reach high standards, encouragement of discussion and openness to others' opinions, and elocutionary skills consistently explain 10% or more of the variance in student achievement regardless of the method of averaging or the unit of analysis used. The size of the association between an instructional characteristic and student achievement, as one indicator of the characteristic's importance, is compared with three other indicators (extent to which students say the characteristic is important, extent to which faculty say it is important, and size of its correlation with the overall evaluation of the instructor).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abrami, P. C., Cohen, P. A., and d'Appolonia, S. (1988). Implementation problems in meta-analysis.Review of Educational Research 58(2): 151–179.

    Google Scholar 

  • Aleamoni, L. M. (1972). Illinois Course Evaluation Questionnaire (CEQ): Results interpretation manual (Form 66 and Form 32). Report No. 331. Urbana-Champaign, Ill.: Measurement and Research Division, Office of Instructional Resources, University of Illinois.

  • Bangert-Drowns, R. L. (1986). Review of developments in meta-analytic method.Psychological Bulletin 99(3): 388–399.

    Google Scholar 

  • Benton, S. E., and Scott, O. (1976). A comparison of the criterion validity of two types of student response inventories for appraising instruction. Paper read at the annual meeting of the National Council on Measurement in Education.

  • Bolton, B., Bonge, D., and Marr, J. (1979). Ratings of instruction, examination performance, and subsequent enrollment in psychology courses.Teaching of Psychology 6(2): 82–85.

    Google Scholar 

  • Braskamp, L. A., Caulley, D., and Costin, F. (1979). Student ratings and instructor self-ratings and their relationships to student achievement.American Educational Research Journal 16(3): 295–306.

    Google Scholar 

  • Bryson, R. (1974). Teacher evaluations and student learning: A reexamination.Journal of Educational Research 68(1): 11–14.

    Google Scholar 

  • Centra, J. A. (1973). The Student Instructional Report: Comparisons with alumni ratings; item reliabilities; the factor structure. SIR Report No. 3. Princeton, N.J.: Educational Testing Service.

    Google Scholar 

  • Centra, J. A. (1977). Student ratings of instruction and their relationship to student learning.American Educational Research Journal 14(1): 17–24.

    Google Scholar 

  • Chase, C. I., and Keene, J. M., Jr. (1979). Validity of student ratings of faculty. Indiana Studies in Higher Education No. 40. Bloomington, Ind.: Bureau of Evaluative Studies and Testing, Division of Research and Development, Indiana University (ERIC Document Reproduction Service No. ED 169 870).

  • Cohen, P. A. (1980). A meta-analysis of the relationship between student ratings of instruction and student achievement. Unpublished doctoral dissertation, University of Michigan.

  • Cohen, P. A. (1981). Student ratings of instruction and student achievement.Review of Educational Research 51(3): 281–309.

    Google Scholar 

  • Cohen, P. A. (1986). An updated and expanded meta-analysis of multisection student rating validity studies. Paper read at the annual meeting of the American Educational Research Association.

  • Cohen, P. A. (1987). A critical analysis and reanalysis of the multisection validity meta-analysis. Paper read at the annual meeting of the American Educational Research Association.

  • Cohen, S. H., and Berger, W. G. (1970). Dimensions of students' ratings of college instructors underlying subsequent achievement on course examinations.Proceedings of the 78th Annual Convention of the American Psychological Association 5:605–606.

    Google Scholar 

  • Cooper, H. M. (1984).The Integrative Research Review: A Systematic Approach. Beverly Hills, Calif.: Sage Publications.

    Google Scholar 

  • Costin, F. (1978). Do student ratings of college teachers predict student achievement?Teaching of Psychology 5(2): 86–88.

    Google Scholar 

  • d'Apollonia, S., and Abrami, P. C. (1987). An empirical critique of meta-analysis: The literature on student ratings of instruction. Paper read at the annual meeting of the American Educational Research Association.

  • d'Apollonia, S., and Abrami, P. C. (1988). The literature on student ratings of instruction: Yet another meta-analysis. Paper read at the annual meeting of the American Educational Research Association.

  • Dowell, D. A., and Neal, J. A. (1982) A selective review of the validity of student ratings of teaching.Journal of Higher Education 53(1): 51–62.

    Google Scholar 

  • Doyle, K. O., Jr. (1981). Validity and perplexity: An incomplete list of disturbing issues in instructional evaluation research.Instructional Evaluation 6(1): 23–25.

    Google Scholar 

  • Doyle, K. O., Jr., and Crichton, L. I. (1978). Student, peer, and self evaluation of college instruction.Journal of Educational Psychology 70(5): 815–826.

    Google Scholar 

  • Doyle, K. O., Jr., and Whitely, S. E. (1974). Student ratings as criteria for effective teaching.American Educational Research Journal 11(3): 259–274.

    Google Scholar 

  • Elliott, D. N. (1950). Characteristics and relationships of various criteria of college and university teaching.Purdue University Studies in Higher Education 70:5–61.

    Google Scholar 

  • Ellis, N. R., and Rickard, H. C. (1977). Evaluating the teaching of introductory psychology.Teaching of Psychology 4(3): 128–132.

    Google Scholar 

  • Endo, G. T., and Della-Piana, G. (1976). A validation study of course evaluation ratings.Improving College and University Teaching 24(2): 84–86.

    Google Scholar 

  • Feldman, K. A. (1976a). Grades and college students' evaluations of their courses and teachers.Research in Higher Education 4(1): 69–111.

    Google Scholar 

  • Feldman, K. A. (1976b). The superior college teacher from the students' view.Research in Higher Education 5(3): 243–288.

    Google Scholar 

  • Feldman, K. A. (1977). Consistency and variability among college students in rating their teachers and courses: A review and analysis.Research in Higher Education 6(3): 223–274.

    Google Scholar 

  • Feldman, K. A. (1978). Course characteristics and college students' ratings of their teachers: What we know and what we don't.Research in Higher Education 9(3): 199–242.

    Google Scholar 

  • Feldman, K. A. (1979). The significance of circumstances for college students' ratings of their teachers and courses.Research in Higher Education 10(2): 149–172.

    Google Scholar 

  • Feldman, K. A. (1983). Seniority and experience of college teachers as related to evaluations they receive from students.Research in Higher Education 18(1): 3–124.

    Google Scholar 

  • Feldman, K. A. (1984). Class size and college students' evaluations of teachers and courses: A closer look.Research in Higher Education 21(1): 45–116.

    Google Scholar 

  • Feldman, K. A. (1986). The perceived instructional effectiveness of college teachers as related to their personality and attitudinal characteristics: A review and synthesis.Research in Higher Education 24(2): 139–213.

    Google Scholar 

  • Feldman, K. A. (1987). Research productivity and scholarly accomplishment of college teachers as related to their instructional effectiveness: A review and exploration.Research in Higher Education 26(3): 227–298.

    Google Scholar 

  • Feldman, K. A. (1988). Effective college teaching from the students' and faculty's view: Matched or mismatched priorities?Research in Higher Education 28(4): 291–344.

    Google Scholar 

  • Feldman, K. A. (1989). Instructional effectiveness of college teachers as judged by teachers themselves, current and former students, colleagues, administrators, and external (neutral) observers.Research in Higher Education 30(2): 137–194.

    Google Scholar 

  • Frey, P. W. (1973). Student ratings of teaching: Validity of several rating factors.Science 182(4107): 83–85.

    Google Scholar 

  • Frey, P. W. (1976). Validity of student instructional ratings as a function of their timing.Journal of Higher Education 47(3): 327–336.

    Google Scholar 

  • Frey, P. W., Leonard, D. W., and Beatty, W. W. (1975). Student ratings of instruction: Validity research.American Educational Research Journal 12(4):435–444.

    Google Scholar 

  • Glass, G. V. (1978). Integrating findings: The meta-analysis of research.Review of Research in Education 5: 351–379.

    Google Scholar 

  • Glass, G. V., McGaw, B., and Smith, M. L. (1981).Meta-Analysis in Social Science. Beverly Hills, Calif.: Sage.

    Google Scholar 

  • Glass, G. V., and Stanley, J. C. (1970).Statistical Methods in Education and Psychology. New York: Macmillan.

    Google Scholar 

  • Greenwood, G. E., Hazelton, A., Smith, A. B., and Ware, W. B. (1976). A study of the validity of four types of student ratings of college teaching assessed on a criterion of student achievement gains.Research in Higher Education 5(2): 171–178.

    Google Scholar 

  • Grush, J. E., and Costin, F. (1975). The student as consumer of the teaching process.American Educational Research Journal 12(1): 55–66.

    Google Scholar 

  • Hazelton, A. E. (1980). A study of the validity of student ratings of college teaching assessed on a criterion of student achievement in a first course in calculus. Unpublished doctoral dissertation, University of Florida.

  • Hoffman, R. G. (1976). Student descriptions of instructional characteristics as relevant indices of teaching effectiveness. Unpublished doctoral dissertation, University of Maryland.

  • Hoffman, R. G. (1978). Variables affecting university student ratings of instructor behavior.American Educational Research Journal 15(2): 287–299.

    Google Scholar 

  • Isaacson, R. L., McKeachie, W. J., Milholland, J. E., Lin, Y. G., Hofeller, M., Baerwaldt, J. W., and Zinn, K. L. (1964). Dimensions of student evaluations of teaching.Journal of Educational Psychology 55(6): 344–351.

    Google Scholar 

  • Kendall, M., and Stuart, A. (1979).The Advanced Theory of Statistics (Vol. 1, 4th ed.). New York: Macmillan.

    Google Scholar 

  • Kulik, J. A., and Kulik, C.-L. C. (forthcoming). Meta-analysis in educational research,International Journal of Educational Research.

  • Land, M. L. (1979). Low-inference variables of teacher clarity: Effects on student concept learning.Journal of Educational Psychology 71(6): 795–799.

    Google Scholar 

  • Land, M. L. (1981). Actual and perceived teacher clarity: Relations to student achievement in science.Journal of Research in Science Teaching 18(2): 139–143.

    Google Scholar 

  • Land, M. L., and Combs, A. (1981). Teacher clarity, student instructional ratings, and student performance. Paper read at the annual meeting of the American Educational Research Association.

  • Land, M. L., and Combs, N. (1982). Teacher behavior and student ratings.Educational and Psychological Research 2(1): 63–66.

    Google Scholar 

  • Land, M. L., and Smith, L. R. (1979). The effect of low inference teacher clarity inhibitors on student achievement.Journal of Teacher Education 30(3): 55–57.

    Google Scholar 

  • Land, M. L., and Smith, L. R. (1981). College student ratings and teacher behavior: An experimental study.Journal of Social Studies Research 5(1): 19–22.

    Google Scholar 

  • Leventhal, L. (1975). Teacher rating forms: Critique and reformulation of previous validation designs.Canadian Psychological Review 16(4): 269–276.

    Google Scholar 

  • Linn, R. L., Centra, J. A., and Tucker, L. R. (1975). Between, within, and total group factor analyses of student ratings of instruction.Multivariate Behavioral Research 10(3): 277–288.

    Google Scholar 

  • Marsh, H. W. (1984). Students' evaluations of university teaching: Dimensionality, reliability, validity, potential biases, and utility.Journal of Educational Psychology 76(5): 707–754.

    Google Scholar 

  • Marsh, H. W. (1987). Students' evaluation of university teaching: Research findings, methodological issues, and directions for future research.International Journal of Education Research 11(3): 253–388.

    Google Scholar 

  • Marsh, H. W., Fleiner, H., and Thomas, C. S. (1975). Validity and usefulness of instructional quality.Journal of Educational Psychology 67(6): 833–839.

    Google Scholar 

  • Marsh, H. W., and Overall, J. U. (1980). Validity of students' evaluations of teaching effectiveness: Cognitive and affective criteria.Journal of Educational Psychology 72(4): 468–475.

    Google Scholar 

  • McKeachie, W. J. (1987). Instructional evaluation: Current issues and possible improvements.Journal of Higher Education 58(3): 344–350.

    Google Scholar 

  • McKeachie, W. J., Lin, Y.-G., and Mann, W. (1971). Student ratings of teacher effectiveness: Validity studies.American Educational Research Association 8(3): 435–445.

    Google Scholar 

  • Mintzes, J. J. (1976–77). Field test and validation of a teaching evaluation instrument: The Student Opinion Survey of Teaching. A report submitted to the Senate Committee for Teaching and Learning, Faculty Senate, University of Windsor. Windsor, Ontario: University of Windsor.

    Google Scholar 

  • Morgan, W. D., and Vasché, J. D. (1978).Journal of Economic Education 9(2): 123–126.

    Google Scholar 

  • Morsh, J. E., Burgess, G. G., and Smith, P. N. (1956). Student achievement as a measure of instructor effectiveness.Journal of Educational Psychology 47(2): 79–88.

    Google Scholar 

  • Murray, H. G. (1983). Low-inference classroom teaching behaviors in relation to six measures of college teaching effectiveness.Proceedings of the Conference on the Evaluation and Improvement of University Teaching: The Canadian Experience (pp. 43–73). Montreal: McGill University, Centre for Teaching and Learning Service.

    Google Scholar 

  • Office of Evaluation Services (1972). Student Instructional Rating System responses and student characteristics. SIRS Research Report No. 4. East Lansing, Michigan: Michigan State University.

    Google Scholar 

  • Orpen, C. (1980). Student evaluations of lecturers as an indicator of instructional quality: A validity study.Journal of Educational Research 74(1): 5–7.

    Google Scholar 

  • Overall, J. U., and Marsh, H. W. (1977). The relationship between students' evaluations of faculty and instructional improvement. Los Angeles: University of California (ERIC Document Reproduction Service No. ED 138 165).

    Google Scholar 

  • Pambookian, H. (1974). Initial level of student evaluation of instruction as a source of influence on instructor change after feedback.Journal of Educational Psychology 66(1): 52–56.

    Google Scholar 

  • Rankin, E. F., Jr., Greenmun, R., and Tracy, R. J. (1965). Factors related to student evaluations of a college reading course.Journal of Reading 9(1): 10–15.

    Google Scholar 

  • Remmers, H. H., Martin, F. D., and Elliott, D. N. (1949). Are students' ratings of instructors related to their grades?Purdue University Studies in Higher Education 66: 17–26.

    Google Scholar 

  • Rosenthal, R. (1984).Meta-analytic Procedures for Social Research. Beverly Hills, Calif.: Sage.

    Google Scholar 

  • Rubinstein, J., and Mitchell, H. (1970). Feeling free, student involvement, and appreciation.Proceedings of the 78th Annual Convention of the American Psychological Association 5: 623–624.

    Google Scholar 

  • Silver, N. C., and Dunlap, W. P. (1987). Averaging correlation coefficients: Should Fisher'sz transformation be used?Journal of Applied Psychology 71(1): 146–148.

    Google Scholar 

  • Smith, L. R., and Land, M. L. (1980). Student perception of teacher clarity in mathematics.Journal of Research in Mathematics Education 11(2): 137–146.

    Google Scholar 

  • Solomon, D., Rosenberg, L., and Bezdek, W. E. (1964). Teacher behavior and student learning.Journal of Educational Psychology 55(1): 23–30.

    Google Scholar 

  • Strube, M. J. (1988). Averaging correlation coefficients: Influence of heterogeneity and set size.Journal of Applied Psychology 73(3): 359–368.

    Google Scholar 

  • Turner, R. L., and Thompson, R. P. (1974). Relationships between college student ratings of instructors and residual learning. Paper read at the annual meeting of the American Educational Research Association.

  • Whitely, S. E., Doyle, K. O., Jr., and Hopkinson, K. (1973). Student ratings and criteria for effective teaching. Report 731F. Minneapolis, Minn.: Measurement Services Center, University of Minnesota.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Sociology, State University of New York at Stony Brook, 11794-4356, Stony Brook, NY

    Kenneth A. Feldman

Authors
  1. Kenneth A. Feldman
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Feldman, K.A. The association between student ratings of specific instructional dimensions and student achievement: Refining and extending the synthesis of data from multisection validity studies. Res High Educ 30, 583–645 (1989). https://doi.org/10.1007/BF00992392

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00992392

Keywords

Search

Navigation