Before I launch into a historical overview of student evaluation of teaching (SETs) at Geneseo since 1969, I would like to summarize my overall impressions of this history. Only a few years during the last thirty have been free of some discussion, often heated, about the SETs. While the principle of student evaluation has remained an accepted one by Geneseo professors throughout the years, the actual forms used over the years (and there have been several, virtually all "home-grown") have never gained their wholehearted support. I have copied the appropriate minutes from the Faculty/College Senate deliberations about the SETs in order to write this history, and they make a stack several inches high, representing untold hours of research, deliberation, and creative endeavor by large numbers of faculty members.

Surely, we could put that energy to better use in the future by freeing the college community from the responsibility of creating and administering our own home-grown instrument. My recommendation is that we move to use the services of the IDEA Center at Kansas State University. Their long and short forms are the result of many years of refinement in the light of both the huge (and ever growing) body of research in the field and the changes in teaching styles that demand modifications of the traditional evaluative approach. Adopting a professionally developed and processed instrument like the IDEA would also bring the benefit that only this type of well designed form meets with the approval of researchers in the field of evaluation. We need to be as sure as we can be that our instrument is truly measuring teaching effectiveness, and the IDEA is as close as it gets. Moreover, it provides a 12-question "Learning Outcomes" section that I regard as an extremely valuable addition, particularly in light of our upcoming Middle States assessment. In 1991 the Middle States Accreditation team "praised our institutional assessment and the SOFI process, but urged that we do more student assessment to evaluate specific learning and skills development." Adopting the IDEA form would help us to meet this challenge.

The story of SETs at Geneseo begins in 1969 with a hand-tallied 31 item student-initiated evaluation form despite a motion against the use of this form, which fell to defeat at the Senate meeting of 12. 9. 1969. This was a stormy time in the relationship between faculty and students all over the Western world, and Geneseo clearly had its own tensions. This context of student unrest is made clear in such records as the minutes of a State University Faculty Senate meeting of 9. 24/25. 1969, which refer to "student unrest and related problems," including "the incident at Buffalo (destroying R. O. T. C. office)." Mention of "the current occupation of the Blake A lounge" in Geneseo Senate's Executive Committee (EC) meeting minutes of 3. 3. 1970 and the formation early in 1970 of "The Committee for a University" at S. U. N. Y Binghamton and Buffalo to counteract "the threats and demands of a militant minority (students and faculty)" also attest to the general sense of urgency to build bridges between the two groups that would grant students greater control over their education than they had previously enjoyed.

The student form continued in 1970 in revised form with a 23-item machine scored instrument still administered by the students. In the EC meeting of 9. 20. 1970 Senate Chair C. R. Bailey stated that the students were looking to "make the evaluation mandatory on survey courses." Vice President for Academic Affairs (VPAA) T. Colahan declared himself "totally opposed to anything like this being mandatory." He disclaimed any connection between himself and the new reduced form. However, at the EC meting of 1. 26. 1971 Bailey mentions a memorandum from VPAA Colahan dated 1. 18. 1971 and entitled: "Mandated Evaluation of Teaching." The idea was to mandate them for each Fall Term. At their meeting of 1. 26. 1971, the Student Affairs Committee (SAC) supported mandated evaluations; the VPAA Office would take charge. The Faculty Affairs Committee (FAC) supported this change as well "if implemented properly."

At the meeting of 2. 5/6. 1971 the State University Senate resolved to recommend "the regular use of student evaluations of instructors and courses." They urged faculty senates to work to implement this change. On 2. 23. 1971 the Undergraduate Academic Affairs Committee (UAAC) heard from Professor Deutsch about the results of a survey he conducted of 20-30 faculty members. "No one sampled is in favor of the suggested proposal or accompanying form," he reported, and the "majority [were] against mandatory student evaluation." "Majority for student administered evaluation or questionnaire to be used by students only, i. e. not to be used for any administrative purpose." If such a form were to be required, "it must not be the only criteria of teaching ability." Later in the meeting VPAA Colahan "stated that he was asking the faculty to do something about evaluation, but if they do not the students will continue to do it in their own way. It may also come from another angle since we are the profession most open to public scrutiny." Despite concerns expressed especially in UAAC, at the EC meeting of 3. 23. 1971 Chair Bailey moved mandatory evaluations for the Fall of 1971 with a form to be "developed locally by an ad hoc committee." Results "will be made available to departmental committees concerned with tenure, promotion, and merit increases," as well as administrators. The proposal was modified at the Senate meeting of 5. 4. 1971 to delay mandatory use for a year while the ad hoc committee developed a suitable instrument. The key concern here was that "validity can be determined" before requiring the instrument's use.

In 1971 the ad hoc committee created a form, still to be voluntary, according to the UAAC meeting of 4. 18. 1972. Its Chair, D. Harke, produced the final report dated 9. 19. 1972 about the trials of the academic year 1971-72, arguing that "the Committee is satisfied with the external validity of the questionnaire. It is statistically good and checks out with the 'grapevine.'" At the Senate meeting of 1. 23. 1973 UAAC and FAC moved use of the form "for a trial period of three years" with "all students, faculty and administrators" having ready access to the results. The discussion that followed made clear the fact that the SETs would contribute to personnel evaluations. Concerns about the form evaluating "performance type teaching" surfaced but did not prevail; those about the relationship between grades and SETs met with the argument from Harke that "students who receive higher grades appear to be more critical." A three-year trial carried by 37/18, and the SET system gained Presidential approval on 1. 26. 1973. At the 2. 6. 1973 EC meeting President MacVittie congratulated the Senate for "formaliz[ing] a procedure for the evaluation of instruction: and stressed faculty improvement as an added benefit. The FAC report of 5. 14. 1973 stressed "that this form is but one of many means of evaluating faculty instruction." The instrument became known as the "CIE."

A Spring 1974 survey sent to 305 faculty with 63% response rate found 38.8% for mandatory continuance, 47.6% for voluntary continuance, and 13.5% for abolition.

In the same period the CIE Committee revised the form. In 1975 the question of grade inflation surfaced. An ad hoc Grading Study Committee found "a fairly steady increase in the number of A and B grades, and a corresponding decrease in the number of C, D, and E grades, over a period of at least the last six years." Yet student quality "as measured by Regents Scholarship Examination and Scholastic Aptitude Test scores . . . peaked with the incoming class in 1972 and [has] since fallen rather sharply." This was a national trend [as was student evaluations!] One possible solution suggested was that Geneseo "eliminate course-instructor evaluations by students." A Spring 1975 Faculty Survey found strong support for the fact that "expected grade influences the student's evaluation of a faculty member:" greatly = 42%, somewhat = 42%, little = 13% and not at all = 3%. A question about how the CIE should continue met with the following responses: mandatory for all = 36%, mandatory for non-tenured = 1%, mandatory for all except first semester = 8%, voluntary = 32%, abolish = 17% and other = 7%.

The report of the Committee on "Tenure Review" working during 1974/75 pointed to the use of CIEs in some departmental personnel reviews. Some departments, however, felt them to be "worthless." When the committee met with President MacVittie on 4. 16. 1975, they "agreed that departments should be urged to supplement this instrument with various forms of peer evaluation to provide a broader basis for assessing this factor in making tenure decisions." (A 1976 Personnel Practices Subcommittee Report found that evaluations "are often used by the committees, but the results are interpreted in different ways by different departments.") In 1975/76 the CIE committee made a report, received by a Joint Committee of FAC, SAC, and UAAC, which essentially recommended continuation of the CIE but with continual revisions. The Joint Committee added: "We urge the Evaluation Committee to respond to the significant and growing number of studies that indicate non-correlation with what the evaluation purports to measure--the quality of a person's teaching." After the renewal passed, at the Senate meeting of 4. 13. 1976 a petition surfaced signed by 58 faculty members asking for a referendum to reverse the CIE renewal; the Fall 1976 ballot included this initiative. The author was Dr. Andrew Heitzman who claimed that he had secured 150 faculty signatures. On 9. 22. 1976 the results produced a victory for the referendum supporters: the votes were against CIE: 114 and for CIE: 105. President MacVittie nevertheless confirmed that the CIE policy was still in effect at the EC meeting of 9. 11. 1976. When FAC interviewed J. McNally on 11. 30. 1976, they asked: "Has the instrument in any of its forms been validated as a measurement of the instructor's teaching effectiveness. Answer - No." Concern also surfaced about whether the instrument was "state of the art." At the FAC meeting of 12. 7. 1976 Heitzman spoke, mentioning "the University Commission on Purposes and Goals recommending CIE at SUNY campuses." A student questionnaire sent to 400 with 92 responses showed that 84% of respondents "evaluated instructors on the basis of grade expectations." At the FAC meeting on 2. 1. 1977 members developed a new questionnaire about the CIEs for distribution to faculty. At first it included a statement: "The CIE program should be abolished," which disappeared in the final version. At the FAC meeting of 2. 16. 1977 President MacVittie remarked that the CIE was mandated for 3 years and then discussion could resume. They were "not mandated by Albany." The results of the questionnnaire were reported at the 3. 8. 1977 FAC meeting. 53% either Strongly Agreed or Agreed to mandatory CIEs, 65% Disagreed or Strongly Disagreed that "the present CIE form measures teaching effectiveness well," 69% Disagreed or Strongly Disagreed that the current CIE form should continue in use, and 41% Strongly Disagreed that the current CIE form should be used in personnel decisions. Another 16% Disagreed, while only 27% Agreed or Strongly Agreed to personnel action use.

In 1977 the CIE committee introduced the possibility of using the Purdue Cafeteria System. In that same academic year (1977/78) the possibility of using Northwestern's Endeavor form also surfaced but as it was copyrighted, the use of a local form appeared the "cheaper" option; other forms were reviewed as well. Eventually the College rejected use of the Purdue system as it only included 5 common questions. (FAC committee annual report, 1977-1978) The Senate Agenda for 4. 26. 1977 included the resolution: "In the absence of quantitative documentation showing that the CIE/and or THE CAFETERIA are valid methods of measuring teaching effectiveness, the Faculty Affairs Committee believes that no claims should be made that either evaluation form measures teaching effectiveness." The Senate passed this resolution on 4. 26. 1977, replacing FAC with Faculty Senate.

In the 1977/78 academic year the CIE Committee proposed a revised form for a three-year term that did not include many of the FAC/SAC suggestions for a new form. Senators did not like the fact that general questions about course and instructor were still included despite their opposition. These questions were "excessively simplistic. Some students are no doubt recommending a course and/or instructor because tough important subject matter is covered with clarity and intellectual rigor, while other students may give a good rating because they believe that a good grade can be obtained without too much work." They had wanted to "measure teaching effectiveness rather than just rapport," and the committees had "recommended wordings of questions that had been tested for reliability." Why did the CIE committee alter the questions? Later that year a further revised CIE form did pass the Senate. During the 1978/79 academic year use of CIEs for graduate and summer courses passed. A McNally memorandum of 4. 22. 1980 argued "that "Old" item #16 (Taking everything into account, I consider this instructor . . .) correlated quite significantly with almost all of the items on the "New" form." The "Old" form was "unifactor," while the "New" form encompassed 5 factors in the questions. In 1980/81 a FAC/SAC Grade Inflation Subcommittee met once but hostility, especially from students, closed it down. Dean Joan Schumaker was nonetheless worried; she had had to raise the Dean's List from 3.25 to 3.4.

In the academic year 1981-82 the CIEs came up once again for renewal. At the FAC meeting of 2. 23. 1982 concern reemerged about the "statistical reliability and validity of the CIE form." Were they measuring effective teaching? Did publishing the results in the Lamron skew the results? What was their "precise function?" At the Senate meeting of 3. 9. 1982 McNally stated that "experientially CIE's have been and are useful, but their validity has not been completely tested and remains debatable." The CIE committee wanted time to "conduct further study of the form's content, structure and validity." It was at this time that a crucial dividing point in the history of the SETs at Geneseo took place. B. Godsave asked the Senate on 4. 13. 1982 to rename the CIE the "Student Opinion of Faculty Instruction." This was becasue "an opinion poll does not require agonizing over validation." His motion carried 48/27 with 2 Abstentions. In a prepared statement for the meeting Godsave claimed: "We have yet to define teaching effectiveness here at Geneseo" so how could we "measure what we have not defined. . . . The university is not the place to encourage inappropriate research and measurement techniques. We do an injustice to ourselves and our students when we knowingly do so. We expect our students to be critical of such practices. What poor examples we set." Godsave's SOFI idea passed and in December, 1982, a new "SOFI" form came into being.

At theEC meeting of 2. 28. 1984 the question of the personnel use of the SOFIs then surfaced. Did the Faculty Personnel Committee (FPC) make use of "the SOFI results to evaluate candidates? W. Cook replied that they are often submitted by candidates, but that the committee has not asked for them on its own initiative during this past year. W. Rhodes noted that they are not supposed to be used in evaluations for promotion, granting of tenure, etc, but have been so used in some cases. It is only an understanding, not a written policy that they are not to be used in these evaluations." In the Senate meeting of 12. 11. 1984 the question of validity arose briefly once again. that year the SOFI renewal carried it to Spring 1988. Therefore in the academic year 1987/88 the question of SOFI renewal came up yet again.

At a FAC meeting on 9. 22.1987 McNally claimed that "even with the current 15 questions, statistical analysis shows that students are in effect responding to one issue: how good is the professor?" He added later that "they act as a kind of emotional release for students." On 3. 1. 1988 J. Bearden presented a factor analysis of the SOFI and found a single factor very dominant: "general opinion of the course and the instructor." These FAC members seemed to accept that use of SOFIs for personnel decisions was a fact; in a meeting of 9. 6. 1987 "W. Gohlman reported that Dr. Colahan, Vice President for Academic Affairs, had indicated that it was legal to use the SOFI's for evaluiaton of faculty consistent with the Board of Trustees guidelines." The Draft SOFI Fact Sheet that this committee developed spelled out the fact that "the SOFI's are widely used in making personnel decisions." That year the Senate renewed the same form for 3 years.

On 11. 5. 1991 FAC members met with members of the FPC and learned that "the FPC relies heavily on the SOFI data." At another meeting on 11. 12. 1991 they learned from Provost D. Spencer that "the current SOFI used at Geneseo is fundamentally for evaluation purposes. Its data is more pertinent to evaluation [than development] because it asks the same questions of all students in all courses. It is not geared to apply particularly well to variables such as differences in class size, core courses, studios and labs." Spencer added later that "the Administration views SOFIs as a pretty important component in the overall package." He also suggested that "a new instrument can be exciting for a campus." At their 12. 3. 1991 meeting the FAC explored a wide range of options for the future of SETs, including "separate instruments designed to be appropriate to different academic areas within the College" and a menu approach. In the Senate meeting of 3. 31. 1992 Provost Spencer said that the SOFI comments "are very helpful in making" personnel decisions. This statement pursued a new issue related to the use of the SETs at Geneseo--handing in of student comments for personnel actions. In the FAC meeting of 14. 4. 1992 "it was pointed out that historically the instructor has been the sole reviewer of this written infirmation, even though no official policy on this has ever been drafted. In light of the importance for students to be candid and critical in their remarks, it seems only appropriate that the instructor should have ultimate control over sho else, if any, should see these responses." In the EC of 4. 21. 1992 "Provost Spencer noted that the written comments are used to evaluate faculty for renewal, tenure and promotions. W. Cook noted that faculty members are not required to submit the written comments to evaluation committees. President harter observed that all the written comments should be available to members of evaluation committees, not merely those comments the faculty member under consideration wishes to submit. J. Bulsys noted that current practice is the result of tradition, not policy." A decision about this policy was to come the following year.

In the Senate meeting of 4. 21. 1992 F. Fidura "expressed the opinion that this is not a true evaluation instrument based upon validation research. Rather, it is an opinion instrument. K. Kallio responded that the committee took into account research into the literature with respect to the validity of such instruments as well as factor analyses of the current document." The new form presented 4 sets of complex questions and summaries to elicit careful thought from the students. The intent was since "a wide variety of sound teaching methods are used in classes at the College . . . the working of SOFI items should not favor either explicitly or implicitly one method over another."

During 1992-1993 there was much discussion about the possibility of requiring SOFI comments for personnel decisions. The policy that emerged included: "Systematic prose comments reflecting student opinion of teaching effectiveness, according to an established departmental procedure" and "submission of SOFI written comments in these processes is voluntary on the part of the instructor. Should a candidate choose to submit SOFI written comments for review, it is strongly recommended that all SOFI forms from any given class be forwarded." In the Senate meeting of 4. 27. 1993 mention was made of "the many complaints re: the weight of SOFI data in issues of renewal, tenure and promotion."

The issue of the 3 "conflicting objectives" of the SOFIs came up again in the EC of 9. 17. 1994, as yet another renewal year began. "Addressing any one of these may compromise the other two." In 1994/1995 the FAC worked to create a form that was calibrated to the Form H categories for teaching. At the Senate meeting of 4. 25. 1995 Biology's Senators expressed "strong opposition to structuring the SOFI around form H-1. . . . The SOFI is the only standardized evaluation took, and linking it directly to form H-1 may give the SOFI prevalence over other forms of evaluation. . . . Linking the SOFI to H-1 may limit the SOFI's use to personnel decisions and detract from its important role in providing faculty with feedback for self-improvement." The new form launched in 1995 was to stay in place until 2000.

In 1998 I wrote a position paper surveying what I found to be the present state of the SETs and recommended serious consideration of real changes in the system of evaluating teaching at Geneseo. At the FAC meeting of 4. 21. 1998 I discussed my findings with committee members. D. Watt "argued that the pivotal issue is whether or not there are qualities of excellent teaching that can be identified and evaluated by students. . . . Watt thinks we can identify such qualities, and students can evaluate them, but that we need to start our discussion of SOFIs with a college wide discussion about the qualities that constitute excellent teaching. . . . Committee members agreed that SOFI scores have acquired an importance in personnel decisions that is unfortunate. They think that junior faculty worry about SOFI scores and may adjust their practices to getting good SOFIs, which may not lead to good teaching. Some expressed the opinion that this may lead to grade inflation. . . . In reply to Watt, Latorella argued that students do not always recognize excellence in teaching, but sometimes regard professors who appear well-organized and sympathetic, rather than professors who offer solid content, challenging work, and current knowledge of their discipline. Watt replied that, although it is dangerous to place too much importance on a narrow range of SOFI scores, it is still possible to use them fairly in the long run. He argued that the key is caution, balance, and moderation. Lima responded that in an ideal world, caution would be practiced, but this is not an ideal world. She added that individual negative SOFI comments can also be used against professors in personnel decisions as well as low SOFI numbers. Committee members agreed that negative comments can stand out." Latorella drew attention to the problem that "the numbers resulting from SOFIs scores are not statistically usable," while deHart worried about the variables that might skew SOFI results. "Committee members agreed that it is difficult to interpret what the scores mean. We asume validity where there may be none." The committee decided that Geneseo should start to review the SOFI system in the next academic year in preparation for the mandated renewal in 199-2000. That led to the creation of my SOFI Working Group in the Fall of 1998.

In conclusion, the SET system at Geneseo has caused consistent debate for its entire history. As best I can tell, our various forms have never been properly validated according to the external criterion that counts, student learning of knowledge and skills. Yet the CIE/SOFI forms have found their way into the heart of our personnel system as the overwhelmingly most important factor in judging effective teaching. It is time for us to abandon the "homegrown" approach in favor of a nationally recognized system that will place our student evaluations on a sound and meaningful foundation.