Research Article: 2018 Vol: 22 Issue: 2
Marc D. Miller, Henderson State University
Melissa Furman, Augusta University
Lonnie Jackson, Henderson State University
Student evaluations of teaching (SET) or of an instructor’s performance in the classroom are used for a variety of reasons in higher education. Administrative uses of student evaluations are used for evaluation of teaching performance (Emory, (2003)), faculty effectiveness (Centra, 1983), and course effectiveness (Marsh, 1987). Faculty also use student evaluations for a variety of internal reasons including self-evaluation of teaching effectiveness (Smith, 2008) and course/program evaluation (Marsh, 1987). The student evaluation, therefore, is expected to have a high amount of reliability and validity as they are used to make important decisions both at the program and individual level (Marsh & Roche, 1997).
There have been multiple attempts to determine the efficacy of using SET for both formative and summative evaluation of teaching quality and effectiveness. Issues such as overall instrument efficacy (Chen & Howshower, 2003), inter-rater reliability (Denson, Loveday & Dalton 2010), construct measurement (Marsh 1984), and various levels of instrument and construct validity (Galbrath, Merrill & Kline 2012, Marsh & Roche, 1997) have been called into question. SET research remains an important topic to higher education and as Spooren, Brockx, & Mortelmans (2013) indicate the usefulness and validity of the use of SET for program, course, or instructor evaluation has not been established.
Students expect that their evaluations will be used to make decisions concerning the faculty member and the course (Worthington, 2010). However, faculty have voiced concerns and the literature has investigated the concern that students are not always the best authority of performance (Cohen, 1980 & Subramanya, 2014). One of the most commonly cited reasons for these concerns are whether or not the student has the desire and the ability to provide an accurate and reliable rating of an instructor’s performance (Boysen, Kelly, Raesly & Casner 2014). Several studies have examined the student’s motivation to perform well on the student evaluation (Alauddin & Kifle, 2014.), however further study is warranted to better understand the causal factors for the desired behaviors of doing a fair, but useful evaluation of the course and professor.
Identifying the causal factors that predict behavior is the central theme of the research stream introduced by Ajzen & Fishbein (Ajzen & Fishbein, 1977; Ajzen, 1985; Ajzen & Madden (1986)). The Theory of Planned Behavior (TPB) was introduced by Ajzen in 1985 (Ajzen (1985) and has been widely accepted as a means for predicting the intention to perform a particular behavior and subsequently the behavior itself. The model has been applied in a variety of settings including technology use (Mathieson, 1991; Siragusa & Dixon, 2009; Cheon, Lee, Crooks & Song 2012), education (Lee, Cerreto & Lee, 2010; Murugesan & Jayavelu, 2015; Wang, Wang, & Wen, 2015; Yan & Sin, 2014), healthcare (Godin & Kok, 1986, Bai & Kang, 2008, Hadadgar, Changiz, Masiello, Dehghani, Mirshahzadeh, & Zary 2016) and social sciences (Poulter & McKenna, 2010; Madden, Ellen & Ajzen (1992), Taylor & Todd, 1995). In addition, the model has been evaluated in a meta-study on entrepreneurship education and found that the model is robust when applied correctly (Fretschner, 2014). Hence, the model should also be used to predict student intentions to perform well on an end of course evaluation.
The student evaluation of the course and the faculty are important for both administrative oversight of academic programs and individual improvement of teaching methods (Rehak & McKinney, 2015). It is imperative that the evaluator has the ability and the motivation to perform such actions with a high degree of accuracy. Lei, Bartlett, Gorney & Herschbach (2010) found that low student self-confidence can be a demotivating factor for students to comply with faculty desires (motivation). Further, Sharma, Van Hoof & Pursel (2013) found that student decision to comply with faculty expectations were influenced by both personal and external factors (attitudes). Therefore, it would seem plausible that the TPB which is comprised of both an attitudinal and a motivational component would be appropriate to be used to study the student’s motivation to comply with a well performed student evaluation of teaching. Thus, the purpose of this study is to apply the Theory of Planned Behavior in the domain of student perceptions of course evaluations. The significance of the results is a better understanding of the casual factors that lead to the desired student behaviors of diligence in evaluating faculty performance.
The social science theories that explain the causal factors of behavior include much of the work of Fishbein & Ajzen (Ajzen & Fishbein, 1977). The theory of reasoned action (TRA) was the first such theory in their body of work which indicates that behavior is predicted by the intention to perform such behavior (Fishbein, 1980). Behavioral intent is formed through a combination of attitudes and subjective norm toward the behavior. Attitude is comprised of a person’s beliefs about the behavior in question. That is, what are the beliefs concerning the outcome of performing the behavior? Subjective norm is a set of normative beliefs that are assessments of what other people think about the behavior. The model has been widely applied to a variety of situations in the social sciences (i.e. Bagozzi & Baumgartner, 1992). The model does assume that the subject has complete volitional control over the behavior in question (Shepard, Hartwick & Warshaw, 1988).
Ajzen (1988, 1991) extended the TRA with the addition of a third construct, perceived behavioral control (PBC). This extension was necessary in order to account for the limitation of the TRA relative to behaviors in which people have incomplete volitional control. PBC is similar to the construct of self-efficacy, which describes an individual’s perception of their ability to effectively perform the behavior in question. The TPB is presented in figure 1.
The TPB suggests that behavior is influenced by intention which is in turn influenced by three types of beliefs: behavioral (attitude), normative (subjective norm), and control (perceived behavioral control). As Ajzen (1991) points out, each of these four components of the model provide evidence of the causes of behavior and thus can serve as a means to change or influence the desired behaviors of the individual. Thus, the TPB provides a framework to study the antecedents of the desired behavior of effective course evaluation on the part of the students.
Attitude toward the Behavior
As Al-Rafee & Cronan (2006) point out, attitude could be considered one of the most important constructs in social psychology and has been found to be the most significant predictor of behavioral intention. Attitude toward the behavior in question is comprised of beliefs about the outcomes of a particular behavior. Ajzen (2006) defines this as “the degree to which performance of the behavior is positively or negatively valued.” The current literature of student expectations of performing the evaluation generally supports this aspect of the model. Ahmadi, Helms & Raiszadeh (2003) found that students believed that the student evaluation was important to both salary and advancement of faculty. In addition, they found that students believed that the evaluation should affect future course improvements. Chen and Hoshower (2003) found that students were motivated to give meaningful input if they believed that the results would be used to improve the course. Surratt & Dessell (2007) found that students indicated a willingness to engage in course faculty evaluations, but did express frustration that their feedback may not be taken seriously. In summary, these studies suggest that the attitude toward the behavior of high performance on a course evaluation would predict their intention to take the job seriously.
Further, Spooren & Christiaens (2017) found that students who valued SET procedures tended to provide higher SET scores.
H1: Attitude toward the faculty/course evaluation will have a significant relationship to the student’s intention to effectively evaluate the course/faculty.
Subjective Norm
Ajzen (2006) indicates that subjective norm is the perception of social pressure to participate or not to participate in a behavior. That is, do other students and faculty provide peer pressure to do well on student evaluations? Ahmadi, Helms & Raiszadeh (2001) found that over 50% of students responded that they discuss the professor’s performance in the classroom with other students. In addition, websites such as ratemyprofessor.com have grown in popularity over the last several years (Kindred and Mohammed, 2005). Felton and Stinson (2004) show that while students are somewhat wary of the results, they do use them in making course and faculty selections. Therefore, subjective norm should be related to the intention to effectively rate the course and faculty member.
H2: Subjective Norm will have a significant relationship to the student’s intention to effectively evaluate the course/faculty.
Perceived Behavioral Control
Perceived behavioral control refers to people's perceptions of their ability to perform a given behavior. Ajzen (1985) postulates that perceptions of one’s ability to perform a given action, combined with intention, can act as a proxy measure of actual behavioral control. PBC and the construct of self-efficacy are thought to be similar (Ajzen & Madden, 1986.) Bandura (1997) describes perceived self-efficacy as “beliefs in one’s capabilities to organize and execute the courses of action required to produce given attainments (p. 3).” The stronger the perceived self-efficacy, the more vigorous and persistent are their efforts (Bandura, 1986). That is, the more people believe in themselves, the harder they will try to perform a particular behavior.
The literature supports the relationship between academic self-efficacy and academic performance in the university setting (Chemers, Hu & Garcia, 2001; Bandura, Barbaranelli, Caprara & Pastorelli 1996). Gore (2006) validates these findings with the provision that efficacy beliefs should be measured after the student has had some experience in college level courses. This is consistent with Bandura’s body of work, which suggests that self-efficacy beliefs are stronger after repeated successes and failures. As most colleges ask students to perform course/faculty evaluations on a regular basis, it is plausible to assume that student’s self-efficacy beliefs regarding course evaluations would be high.
H3: Students perceived behavioral control will predict their behavioral intention to perform effectively on course/faculty evaluations.
In summary, the literature supports the Theory of Planned Behavior as a model that should predict the student’s ability to evaluate course/faculty effectiveness. The three predictors of intention are theory based and provide evidence as to the causal determinants of a student’s behaviors.
Scale Development
Ajzen (2006) provides a methodology for constructing a TPB questionnaire. This methodology was used to create the instrument to measure the four constructs of interest: Attitude, Subjective Norm, Perceived Behavioral Control, and Behavioral Intention. An actual measure of behavior was not included in this study. Many studies indicate that the strong relationship between behavioral intention to actual behavior indicate that intention is a strong surrogate measure for behavior (Jones & Kavanagh, 1996; Jang & Namkung, 2009).
Measures of Intention
Intention was measured with the following items on a 7-point Likert scale using Strongly Agree (7) and Strongly Disagree (1) as anchor points. Behavioral Intention (BI) was measured by a summation of these two scales.
BI 1 I intend to carefully evaluate a faculty member’s classroom performance at the end of this semester.
BI 2 I will try to evaluate my professor’s classroom performance with the utmost care and professionalism at the end of this semester.
Measures of Belief: Attitude, Subjective Norm and Perceived Behavioral Control
Ajzen (2006) points out that “beliefs play a central role in the theory of planned behavior. They are assumed to provide the cognitive and affective foundations for attitudes, subjective norms, and perceptions of behavioral control (p. 7).” Further, these beliefs are comprised of the strength of the belief and the outcome evaluation of that belief.
Subjective Norm is measured by accessing the strength of the belief that the student’s referent groups are important multiplied by the outcome evaluation. Two referent groups were identified for the student’s subjective norm: other students and professors. Therefore, the items used for this scale were:
SN1 Overall, other student’s opinion of me is important. (strength of the belief)
SN2 Most of my student peers whose opinions that I value think carefully evaluating a professor’s classroom performance is important. (outcome evaluation)
SN3 Overall, my professor’s opinion of me is important. (strength of the belief)
SN4 My professor believes that I should carefully evaluate their classroom performance. (outcome evaluation)
These questions were measured with a 7 point Likert type scale anchored by Strongly Disagree to Strongly Agree. The measure was then computed by the following equation:
(SN1 * SN2) + (SN3 * SN4)
Perceived behavioral control was measured in a similar fashion as a summative measure of the following two items:
PBC1: I feel confident that I can effectively evaluate my professors’ classroom performance.
PBC2: It is mostly up to me whether or not I do a good evaluation of my professor’s classroom performance.
For attitude measurement Ajzen (2006) recommends the use of a semantic differential scale to measure both instrumental and experiential qualities. The set of paired adjectives are and are measured as a summation of the following (Table 1):
Table 1 To Carefully Evaluate a Faculty Member’s Performance in the Classroom |
||||||||
Pleasant | 1 | 2 | 3 | 4 | 5 | 6 | 7 | unpleasant |
Good | 1 | 2 | 3 | 4 | 5 | 6 | 7 | bad |
Enjoyable | 1 | 2 | 3 | 4 | 5 | 6 | 7 | unenjoyably |
Interesting | 1 | 2 | 3 | 4 | 5 | 6 | 7 | boring |
Useful | 1 | 2 | 3 | 4 | 5 | 6 | 7 | useless |
Important | 1 | 2 | 3 | 4 | 5 | 6 | 7 | unimportant |
The entire scale is noted in Table 2 below:
Table 2 Scale Items |
||
Variable | Item | Scale |
Behavioural Intention | ||
BI 1 | I intend to carefully evaluate a faculty member’s classroom performance at the end of this semester. | 7 point Likert type scale anchored by Strongly Disagree to Strongly Agree |
BI 2 | I will try to evaluate my professor’s classroom performance with the utmost care and professionalism at the end of this semester. | |
Subjective Norm | ||
SN1 | Overall, other student’s opinion of me is important. | 7 point Likert type scale anchored by Strongly Disagree to Strongly Agree |
SN2 | Most of my student peers whose opinions that I value think carefully evaluating a professor’s classroom performance is important. | |
SN3 | Overall, my professor’s opinion of me is important. (strength of the belief) | |
SN4 | My professor believes that I should carefully evaluate their classroom performance. | |
PBC1 | I feel confident that I can effectively evaluate my professors’ classroom performance. | 7 point Likert type scale anchored by Strongly Disagree to Strongly Agree |
PBC2 | It is mostly up to me whether or not I do a good evaluation of my professor’s classroom performance. | |
Attitude (A) | ||
Six item semantic differential scale with paired adjectives as shown above. |
The measurement instrument was given to 248 students enrolled in the College of Business Administration at a midsize university in the southeastern United States. The classes were undergraduate upper division courses in management, marketing, and accounting. The average age of the respondent was 22.3. This group was chosen based on the several different characteristics. First, these students were either juniors or seniors and thus had multiple opportunities to complete evaluations of faculty performance. Therefore, this eliminated freshman who might have never completed a faculty evaluation. This was important as the study is attempting to evaluate their ability to accurately appraise a faculty member’s performance and thus they would have needed to have had multiple opportunities to accomplish this task.
The group sampled was representative of the College as a whole for students who had completed greater than 60 hours and thus had junior or senior standing. The sample consisted of 51% males and 49% females. Based on the number of hours completed, the group had evaluated or had the opportunity to evaluate an average of 31 courses. The group responded to the question “I usually complete the evaluation of faculty performance in the classes that I have taken.” with an average score of 4.00. This would indicate that the group has had experience with student evaluation of teaching (SET).
Cases where students did not completely fill out the survey were removed from the sample for a total of 230 usable responses.
Ajzen (2006) recommends that the attitude scale be examined for internal consistency to determine the subset of semantic pairs to be used in the final analysis. Principle Components Analysis was performed on the attitude scale and found that the six items loaded on one principle factor. The first factor was determined that the scale exhibited adequate reliability and validity properties (Table 3a and 3b). Thus, the scale for Attitude toward Behavior (AB) was generated through summation of the six item scale. The scales of the model were analyzed with Chronbach’s Alpha and all met the required coefficient of greater than 0.70 (Nunnally, 1978). These results are shown in Table 4.
Table 3a Principle Components Analysis |
||||||
Component | Initial Eigenvalues | Extraction Sums of Squared Loadings | ||||
Total | % of Variance | Cumulative % | Total | % of Variance | Cumulative % | |
1 | 3.389 | 56.481 | 56.481 | 3.389 | 56.481 | 56.481 |
2 | 1.113 | 18.542 | 75.023 | 1.113 | 18.542 | 75.023 |
3 | .546 | 9.098 | 84.122 | |||
4 | .407 | 6.780 | 90.901 | |||
5 | .324 | 5.396 | 96.298 | |||
6 | .222 | 3.702 | 100.000 | |||
Extraction Method: Principal Component Analysis. |
Table 3b Principle Components Analysis |
||
Component | ||
1 | 2 | |
Pleasant/unpleasant | 0.716 | 0.293 |
Good/Bad | 0.788 | -0.270 |
Enjoyable/Unenjoyable | 0.705 | 0.566 |
Interesting/Boring | 0.760 | 0.432 |
Useful/Useless | 0.769 | -0.483 |
Important/Unimportant | 0.767 | -0.461 |
Table 4 Scale Reliability |
|
Scale | Alpha |
Attitude Toward Behaviour (6 items) | 0.845 |
Subjective Norm (2 items) | 0.830 |
Perceived Behavioural Control (2 items) | 0.805 |
Behavioural Intention (2 items | 0.880 |
Multiple Regression Analysis
The tests of hypotheses were analyzed with multiple regression analysis. The r-squared value of the model was .285 while the F test was 29.987 (p = 0.000). In other words, 29% of the variation of behavioral intent to successfully rate a professor’s performance can be explained by the model components of Attitude Toward the Behavior (AB), Subjective Norm (SN) and Perceived Behavioral Control (PBC). The results of the regression are presented in Table 5.
Table 5 Regression Results |
|||||
Hypothesis | Scale | Standardized Coefficient | t-value | Significance | Reject/Accept |
H1 | Attitude Toward Behaviour | 0.259 | 4.358 | 0.000 | Fail to Reject |
H2 | Subjective Norm | 0.109 | 1.907 | 0.058 | Reject |
H3 | Perceived Behavioural Control | 0.371 | 6.313 | 0.000 | Fail to Reject |
The Theory of Planned Behavior does predict the student’s intentions to carefully evaluate a professor’s classroom performance. The three hypotheses of the model provide insight into the variables that will predict these intentions.
Attitude toward Behavior (Hypothesis 1)
We fail to reject Hypothesis 1 which indicates that attitude toward the behavior is an important predictor of behavioral intention. This indicates that if the student believes that the process of evaluating a professor is generally pleasant, important, useful, etc., then they will try their best to do a good job on the evaluation.
Subjective Norm (Hypothesis 2)
Subjective Norm was found to be less of a significant factor for the student to carefully evaluate a professor’s performance in the classroom. That is, it does not matter if the student’s peers or other professor’s beliefs toward the individual doing a good job on performance evaluation are high or low. For this study, Hypothesis 2 was rejected. However, it should be noted that this factor was marginal and could have been accepted under a less rigorous interpretation of the data.
Perceived Behavioral Control (Hypothesis 3)
We fail to reject Hypothesis 3 that indicates that the student is fairly confident in their ability to successfully evaluate a faculty member’s classroom performance. In other words, the higher the PBC, the greater the intention to successfully evaluate the performance.
The purpose of this study was to determine the causal effects of a student’s intention to do well on a course evaluation. The theory of planned behavior (TPB) indicates that intention to perform a particular behavior is a strong indicator of the actual behavior in question. If a student evaluation of the performance of a faculty member is to be taken seriously, then we must have some assurances that the student is taking the evaluation seriously. That is, if important decisions are to be made about the faculty member with this data, then we must be assured that the evaluation is being done in a manner that would reflect the seriousness of the subject matter. The TPB model does lend credence to the intention of the student to try their best on the course evaluation. This result is consistent with a similar result found in Taylor (2015) when studying student subject choices of post compulsory education. Thus, the model seems to be robust in this domain.
The model indicates that if a student is generally positive toward the evaluation, then they will try their best (AB). Therefore, to improve student effort on faculty evaluations, interventions could be conducted to show the seriousness of the subject matter that should result in a more positive attitude toward the process of faculty evaluation. This research does not answer the question of what factors affect these attitudes. That is, where do these attitudes come from and how then can they be affected? Future research should be conducted to tease out the factors (such as why they believe that are shown to be antecedents of the attitude toward the behavior of doing well on the faculty/course evaluation.
The area of subjective norm was found to be a weak predictor of behavioral intention. This is inconsistent with the literature in both the TPB and student evaluations. This study examined both the student’s professors and their peers. Perhaps there are other referent groups that should be identified. Does a student’s parents or employers matter in this process? Future research should examine these groups and others to find out if the subjective norm component is important to this type of behavior.
A student’s perception of their ability to perform the behavior was found to be a strong predictor of their behavioral intent. That is, if the student believes that they can, then they will probably try very hard to do perform well on the evaluation. Training interventions could be designed to overcome this area for students who do not believe that they can or are marginal in their self-reported abilities. Future research should be conducted to show the effects of these types of training interventions.
Finally, this study does not show the relationship between behavioral intention and actual behavior. While the majority of studies indicate that behavioral intention is a strong proxy variable for behavior, this area should be further researched. This overall finding is consistent with that found by Zhang et al (2017) who found that students do value the SET, but that the effect of satisfaction degree on students’ academic achievement is limited. Therefore, the need to find the variables that affect this outcome is important.
The primary question would be to identify a measurable construct that indicates that the student was able to seriously and carefully evaluate the professor’s classroom performance. Perhaps, controlled laboratory experiments could be conducted where a student is shown a poor professor and then asked to rate them. This type of study would lend a further understanding of the BI to Behavior relationship.
Overall, student evaluation of teaching effectiveness is important and improving the overall value of the process should be a principle objective of faculty and administration. This study indicates that the process can result in useful evaluations that can be used to improve faculty teaching effectiveness. Further, this study provides specific targets to improve the process by evaluating the attitude/behavior relationship of the students providing the feedback.