Do Online Students Perform Better than Face-to-face Students ? Reflections and a Short Review of some Empirical Findings article

The increases in the number of online courses given by universities have been quite dramatic over the last couple of years. Nowadays, many universities even give complete degree programs online where instructions and lectures in the form of, for example, streaming videos, are available for students to watch 24 hours a day. In a sense, the use of Internet and Interactive Computer Technologies (ICT) in higher education can be compared to any other type of teaching tool, such as the blackboard and overhead projectors. The motivation for using the Internet and ICT in higher education, from an economic point of view, is if they are more effective as teaching tools compared to any relevant alternative. That is, all else being equal, if the Internet is an effective teaching tool in that students who attend online courses or complete degree programs perform better in terms of marks in the final exam compared to face-to-face students. In this paper we reflect on and summarize some of the empirical findings in the literature on the effects of online teaching on student performance compared to face-to-face equivalents.

TIC en la educación superior puede compararse con cualquier otro tipo de herramienta aplicada a la enseñanza, como la pizarra o el proyector de transparencias.El argumento a favor del uso de Internet y las TIC en la educación superior, desde un punto de vista económico, es que son más eficaces como herramientas aplicadas a la enseñanza en comparación con cualquier otra alternativa relevante.Es decir, Internet será una herramienta de enseñanza eficaz si, en igualdad de condiciones, los estudiantes que siguen cursos o programas de titulación completa impartidos por Internet obtienen mejores resultados en términos de calificaciones en los exámenes finales que los estudiantes que siguen cursos presenciales.En este estudio comentamos y resumimos algunos de los resultados empíricos disponibles en los estudios publicados acerca de los efectos de la enseñanza en línea sobre el rendimiento de los estudiantes respecto a los estudiantes que siguen cursos presenciales.

Introduction
The Internet is about to cause a minor revolution within the, in many other respects, conservative world of higher education.Not only has the Internet for some students outperformed university libraries as the main source for information and facts, it is also challenging face-to-face lectures as a teaching tool.Today, stand alone online courses and complete degree programs are offered via the Internet by many universities.Also virtual universities have appeared, giving entire courses over the Internet, with some of them not even having a campus and therefore no on-campus students.The progress in this direction is going fast and it is probably not possible, or even desirable, to stop.However, it is sometimes wise to reflect, just for a moment or two, on the increased supply of online courses in higher education.What are the pros and cons of this development?
One of the advantages of online courses is the fact that it is now possible to reach students who otherwise would not, for different reasons, have undertaken university studies.This could be due to practical reasons such as their family situation not allowing them to move to a university town.Or the individual might prefer to attend an online course or a complete online based degree program as online courses in many cases offer more flexible study hours.For example, a student who has a job could attend the virtual class watching instruction films and streaming videos of lectures after working hours, whenever they are.Hence, online courses make it possible for students to live far from campus.Whatever the reason, if online courses attract students who otherwise would not have attended higher education, this is of importance from a policy perspective as it has a positive effect on the accumulation of human capital.
Other potential pros of online courses relative to campus equivalents relate to the other actors, the providers of higher education.From the universities' point of view, online courses could be more cost-effective compared to campus courses if it means that the university can teach more students using the same or less resources compared with an equivalent campus based course.For instance, if the teacher can re-use materials such as streaming videos of lectures several times it could decrease the teacher's time for preparation and realisation of the course the next time the same course is given, meaning that both teachers and universities save resources.However, in a study based on nearly 4,000 students enrolled on introductory economics at universities in the USA, Sosin et al. (2004) come to the conclusion that there was no significant difference in time spent between teachers who use a large amount of Interactive Computer Technology (ICT) in their teaching and those who use none or only a small amount.
Moreover, the use of Internet and ICT in higher education could be a good pedagogical tool and hence save time and resources for both students and teachers.In other words, other things being equal, it could be that, for every hour spent studying, a student who attends an online course learns more than a student who takes the same course as an ordinary, face-to-face, campus course.This would not only save teachers' time and university resources through the decreased number of re-examinations, but also student's time.
What are the cons?All the pros of online courses could of course be cons: if these courses do not tend to attract more students (including students who would otherwise not have undertaken a university education); if the preparation and realisation of the courses tend to consume more resources than an ordinary campus course, etc.Even if not all the pros are turned into cons, it might be the case that the cons outweigh the pros, or vice-versa.Again, whatever the result of such an evaluation, the final result is of interest to policy makers as it would suggest whether online education should or should not be encouraged.
The objective of this paper is not to validate all pros and cons of online teaching in higher education and to come up with the answer to whether online teaching, is from an economic perspective, good or bad.Instead, we have a narrower perspective in that the main purpose of this paper is to reflect on and summarise some of the existing empirical knowledge regarding the most important determinants of student success in university courses.In particular, we will focus on the question of whether students who attend an online-based course, where the Internet and ICT are used to a high degree as a teaching tool, tend to perform better or worse compared to students who attend ordinary, faceto-face and campus based courses, where the Internet and ICT are used to a low degree, or not at all.The reflections and reviews in this paper focus on studies based on quantitative analysis and regression models, preferably multiple regression analysis.We will also try to distinguish between different subjects.That is, is there any evidence suggesting that online teaching and the use of Internet and ICT is better suited for some courses or subjects than others?
As this literature is quite extensive, we do not claim or suggest that our review covers all existing papers within this field but instead we will summarize what we believe to be the some of the most interesting findings.And even though we focus on university courses, the overall discussions regarding the use of Internet and ICT as teaching tools could to some extent also apply to other situations.
In what follows, online course or online education will be used as a comprehensive term for Internet education and the use of ICT in higher education.In the same manner, face-to-face education will be used for traditional campus courses.The rest of this paper is organised as follows.In section 2 we discuss some methodological issues that are of importance in the analysis of student performance.These include different measures of student performance, other potentially important determinants of student performance and whether the course is given online or on campus.This section also includes a short comment on some econometric issues.Some empirical results are presented and discussed in section 3, and the conclusions are in section 4.

Methodological issues
2.1.How to measure student performance One of the more fundamental questions in analysing the effect of a specific teaching tool on student performance is to define the concept of student performance.How should it be measured?In the economics literature, student performance is often measured as the student's mark in a written test at the end of the course.In many cases, this mark is only reported as pass or fail where fail corresponds to a mark below the previously set level for pass.Even though, in some cases, students are only interested in whether they have failed or passed (or passed with distinction, a grade often reported), which is also the information given in the final certificate, this is a rather crude measure of student performance, as the researcher can not distinguish between students who have high marks and students who only just pass.A more sophisticated analysis is of course possible if the researcher can get access to the precise test score, which will give them more information (see, among others Brown and Liedholm, 2002).However, one important issue when using test scores is that the researcher usually does not have any information regarding the extent to which the test actually reflects the contents of the course.
Independent of which of these measures is used to capture student performance (test scores or fail/pass), one potential problem is that the researcher often does not have information regarding the student's previous knowledge.Therefore, by only using test scores, the researcher does not have any information on whether the student has gained any knowledge during the course but only what level the student has (or does not have) at the end of the course.This problem could be overcome if student performance is measured as the difference between the student's previous knowledge and that at the end of the course, although there are difficulties in collating this information.One relates to the design of the pre-course test in relation to the post-course test (the final exam) as it would be problematic to use exactly the same questions in both tests.
Another difficulty is that students may have objectives other than just high scores in the written test.They might just want to sit in during the lectures to enjoy the atmosphere, to socialise with friends who attend the same course and lectures, or might be happy just to have been accepted on the course.It could also be the case that the student is applying for a job and taking a course rather than being unemployed, as being a student might look better for potential employers.In this situation, the student might actually not be interested in learning anything and hence not pay attention, no matter what teaching tool is used.If there are a number of students who are predetermined to fail the final exam due to the reasons stated above, and this is not controlled or corrected for in the empirical analysis, this could give biased and inconsistent parameter estimates which, in turn, could generate misleading conclusions.
When it comes to comparisons of student performance across subjects and courses it is important to consider that the form of the final exam sometimes differs between subjects, and even across courses in the same subject.For instance, introductory courses on economics are most typically examined through written exams at the end of the course, while a combination of lab-reports and written tests is frequently used in chemistry and electronics.Oral examinations are sometimes used in linguistic courses, etc.However, despite these drawbacks and difficulties, student performance is most often measured in terms of grades or marks in different tests during the course.One reason for this is that this information is relatively easy to get hold of.At the end of the course each teacher is, in most cases, obliged to report whether students have failed or passed.It is important to be aware of the weaknesses of the measures of student performance frequently used when reading the literature.

Other potentially important factors for student success
The literature on the effects of online education on student performance is closely related to the more general literature on important determinants of student success.In a sense, online teaching could be seen as just another teaching tool, and could actually be compared to the use of the more traditional blackboard.For instance, if the use of the blackboard does not contribute to the student's ability to understand and assimilate the information provided by the teacher, then the use of the blackboard should be questioned.The same goes for the use of the Internet and online teaching in higher education.
Hence, before we proceed and discuss the main issue of this paper -the effects of online teaching on student performance -let us discuss some other potentially important determinants of student performance used in the literature.One of the reasons for doing this is to highlight the fact that there are other factors likely to affect student performance besides whether the student attends an online or face-to-face course.We will not cover all these factors, nor will we discuss their measurability -that would be beyond the scope of this paper -but just highlight that there are other factors that matter for student success apart from the course being online or at campus.However, the issue is of importance from many perspectives, and maybe most so from an econometric point of view, as the failure to include relevant variables in the model could lead to biased and inconsistent parameter estimates.In the end, this would cause misinterpretations of the parameter estimates and their corresponding significance.
Firstly, in online courses, the Internet and ICT are used as a substitute for the ordinary face-to-face teaching technique.However, if face-to-face teaching does not contribute to the learning of the students, then online teaching and the use of Internet and ICT in higher education is compared with something that does not contribute to the learning of the students.
When it comes to student attendance in class and their performance, the results are ambiguous.For instance, controlling for student motivation, an issue we will return to later on, Romer (1993) found that attendance did contribute to the academic performance of the students on a macroeconomics course he taught in the autumn of 1990.Similar results had previously been found for courses on macroeconomics by Schmidt (1983) and also by Park and Kerr (1990) for a money and banking course.These results were later verified by, among others, Durden and Ellis (1995).
However, contrary to these results, Brown et al. (1991) did not find any evidence that a student who attended typically structured classes with lectures performed better on the Test of Understanding College Economics (TUCE) compared to students who attended a standard microeconomics principles course.What they did find was that students who attended the lectures performed better on essay questions than those who did not.In an early paper, McConnell and Lamphear (1969) found no significant difference in performance of students with and without classroom attendance.
Motivation is often considered to be one of the most important driving forces for people to reach their goals.Some argue that motivation actually out-competes talent when it comes to sports activities: It does not matter how talented you are, if you do not have the right motivation, you are deemed to be second.Even though this might be the case, motivation is quite difficult to measure.It could be reflected by the number of training hours (or, in our case, the number of hours spent studying and/or in class), but this is not necessarily a good measure.For instance, a student could attend class because of being raised to be a dutiful person, but not pay attention to what the teacher has to say in class.Another possibility is to ask the students how motivated they are.One problem with this method is, of course, that the results will be highly subjective.Romer (1993) used the number of non-compulsory problem sets the student did during the course as a proxy for motivation: the more non-compulsory problems sets, the more moti-vated the student.Even though this is probably a reasonably good measure of motivation, it does still suffer from the same drawbacks as the number of hours in class.The argument has also been made that innovations in the use of technology in higher education, such as the use of ICT, actually could have a positive effect on motivation, attitude and student performance (see Talley, 2005).
Student motivation and attendance relate to the factor, often quoted in the literature, of how much time and concentration the student dedicates to other activities.Durden and Ellis (1995) find a negative yet insignificant effect of jobs on the side.Other studies have also used measures of the number of hours spent on activities such as sports and social activities.The effect of physical training could go either way, depending on how much time the student spends on such activities.It could be argued that physical exercise improves your ability to concentrate and should therefore have a positive effect on student performance.On the other hand, the undertaking of too much physical exercise will have a negative effect on the available time for studies, as a day only has 24 hours, and it might also require spending time on recovery and rest between training sessions.
As mentioned in the discussion of different definitions of student performance, it is reasonable to assume that the student's previous knowledge will affect their ability to pass the final exam.Pre-knowledge is typically measured as previous college grades or previous experience in higher education.For instance, Park and Kerr (1990), Anderson et al. (1994), and Durden and Ellis (1995) find high preuniversity grades to have a positive effect on student performance.Roamer (1993) and Coates et al. (2004) find the same positive effect of previous experience from university studies on student performance.
Two other variables frequently used in this type of studies are age (Anderson et al., 1994, andCoates et al., 2004) and gender (see Durden andEllis, 1995, andCoates at al, 2004, just to name two).It could be argued that age reflects maturity and hence should have a positive effect on performance, but it is also reasonable to assume that the ability to learn new things decreases with age.Anderson et al. (1994) found a negative, decreasing effect of age on student performance, while Coates et al. (2004) found no effect at all.The effect of gender is rather difficult to interpret as it is not reasonable to believe that, for instance, men are more intelligent than women.Instead, gender is likely to capture differences in how we are brought up and other social factors.Such effects are probably also reflected by the different measures of race used in the literature (see Brown andLiedholm, 2002, andCoates et al., 2004).
Like students, teachers differ in several respects such as personality, talent, and pedagogical skills, factors that are likely to affect student performance, motivation, and attitude.In many studies on the effects of online teaching on student performance, the same teacher is teaching both the online and the face-to-face course, which should mean that the teacher's contribution to each individual student's attitude, motivation and performance is kept constant.However, the very same teacher may actually perform and act differently in different environments.For instance, the teacher could be inexperienced in use of the Internet and ICT which could have a significant effect on the ability to contribute to the student's learning.Navarro (2000) addressed the issue of teacher performance in online versus face-to-face situations.Based on interviews, formal discussions and questionnaires to more than 100 teachers and instructors, Navarro comes to the conclusion that a large majority of the teachers believed that they performed similarly or better in the online environment.If one believes that older students are more motivated, a possible explanation for this result is that the average age of the online students in this study was higher than that of the face-to-face students.Navarro also found that more motivated teachers tend to use ICT and new teaching techniques to a larger extent than less motivated teachers.Teachers also differ in their attitudes towards adopting and introducing new teaching methods.For instance, from a survey among academic members of the American Economic Association (AEA) and teachers listed in the College Marketing Guide (CMG), Becker and Watts (2001) conclude that teachers in economics are quite reluctant to adopt new teaching methods.One explanation put forward by the authors is that the introduction of new teaching techniques and ICT is associated with a sunk cost, a cost that economics teachers are not willing to accept.The results presented by Navarro (2000) support the idea that the development of traditional face-to-face lectures is more cost effective (less time consuming) for teachers compared to online lectures.
There are of course also other student characteristics and factors that could have a significant impact on their performance, such as their family situation (for instance the number of children), mental health status, etc.In a study on the influence of different learning style preferences on student success in online versus face-to-face environments, Aragon et al. (2002) found that students can learn equally well in either format, regardless of learning style, provided that the course is developed around adult learning theory using good instructional design guidelines.Unfortunately, they do not make any formal analysis of significant differ-ences between online and face-to-face courses.The results presented in the paper do not indicate that one learning style has a significantly larger impact on, for instance, the performance of online students as compared to face-toface students.However, that issue was beyond the scope of their paper.

A short comment on some econometric issues
The failure to use appropriate econometric tools and estimators in empirical analyses of phenomena such as student performance could result in biased and inconsistent estimates, possibly leading to misleading inferences and, eventually, misinterpretation of the results.There are at least three econometric issues that need special attention.The first is that of the factors that are regarded the most important determinants for student success, one that is considered to be endogenous, namely effort or the time the student spends on studying.If, for instance, a student's goal is only to pass the course, and a new, very effective teaching tool is introduced, the student could actually spend less time studying and still reach their goal.
Consequently, the time the students spend on studying is not determined outside the model and therefore the econometric analysis should include instrumental variables.Even though the use of instrumental variables is now standard in econometric software, the technique is nearly exclusively used by economists.
The second issue relates to self-selection.That is, it could actually be the case that a specific type of student who tends to perform better than other students chooses either campus or online courses.Coates et al. (2004) find that failure to account for self-selection of students either in face-to-face or online courses could lead to misinterpretation of the results.Their solution to the problem of self-selection was the use of an endogenous switching regression model.However, self-selection is only occasionally discussed and corrected for in the literature.Becker et al. (1996) provide evidence that students with higher levels of previous knowledge in the subject tend to be more likely to continue on the course than those with lower levels.In other words, those students who are less likely to pass the final exam tend to drop-out before they do it, which means that there is a group of students who actually would have failed the exam but are not included in the sample.Other reasons are that students realise that they have chosen the "wrong" course and/or that they have other interests.If this is not accounted for in the empirical specification, it could lead to an upward bias of the parameter estimates.
On the other hand, it is reasonable to assume that some students never complete their studies because they get job offers before the final exam.In contrast to the other reasons for dropping out, these students have a higher probability of passing the final exam and a failure to control for this type of drop-outs will result in a downward bias of the estimated parameters.

Some empirical results
So what does the literature tells us?The results presented in a study by Brown and Liedholm (2002) based on 710 macroeconomics students in the USA suggest that campus students tend to perform better compared to online students.Brown and Liedholm used test scores as the dependent variable and also control for gender, the students' preknowledge in mathematics,1 and their high-school grades.However, they do not control for factors such as the degree to which the students attend class, if they work, if they are engaged in other activities in their spare time, motivation, age, or self-selection.Other important characteristics of the students and their backgrounds are also lacking, Brown and Liedholm's results are supported by the findings by Coates et al. (2004), who base their findings on 126 macroeconomics students in the USA.In contrast to Brown and Liedholm, these authors controlled for the students' age which could reflect motivation if we consider that older students are more focused.They also controlled for how much the students work besides their studies and use an endogenous switching regression model to correct for self-selection.
Even though previous results based on economics students suggest that campus students tend to perform better compared with online students, these results are by no means general for all courses and subjects.It is not even a general result when it comes to economics.For instance, Sosin et al. (2004) find the use of technological tools in teaching economics at the introductory level to have a positive effect on student performance.These results are based on a quite extensive data set of nearly 4,000 students taught by 30 instructors across 15 different departments.However, all these students attended campus courses, even though the technological tools differ between the different courses.Therefore, these results are not comparable with other results presented in this review.
In another study on 345 computer science students, Dutton et al. (2002) found online students perform significantly better compared to their peers who take the campus version of the same course.One tempting explanation for this difference between economics and computer science students is that those interested in computers tend to choose computer science instead of economics, and if so, this result is driven by self-selection.Dutton et al. also found that students who take the online version of the course tend to be older, are to a lower extent enrolled in traditional undergraduate programs, are more likely to have jobs and children and tend to live further away from campus as compared to face-to-face students.Hence, the online students have a greater need for flexible studying hours.Among the students included in this study, face-to-face students value the ability to get advice from the lecturer as an important factor for choosing the face-to-face version of the course.Moreover, the proportion of drop-outs is higher within the group of online students.Although this study provides many interesting results, the potential problem of self-selection between the online and face-to-face courses is not addressed, nor is the endogeneity of study time and the fact that the proportion of drop-outs is higher among those who choose the online version of the course.The potential problem of self-selection can probably not be ignored as the authors find prior computer experience to have a positive impact on the performance of online students, and when this is controlled for in the model, the significant difference in performance between online and face-to-face students is reduced.
Hoskins and van Hoff (2005) analysed the effects of the dialogue method via an online environment on student performance among 110 undergraduate psychology students.They find the dialogue method to have a positive effect on student performance.Again, however, the potential problem of self-selection is not properly addressed.As the authors point out, the students who seize the opportunity to take part in this dialogue method tend to be more motivated, and this group also differs from others when it comes to age and gender.These factors, together with the fact that they base their conclusions on differences in mean values between this group of students and the rest, make it difficult to compare these results with other results based on multiple regression analysis where it is possible to control for a number of different student characteristics.
Analysing student performance of 62 management students in the USA, Neuhauser (2002) finds no significant difference in the mean value of test scores for online versus face-to-face students.The potential problem of self-selection is discussed but not controlled for.And even though the author has information on gender, age, effectiveness of tasks and course effectiveness, this information is not controlled for in the empirical analysis.Instead, only mean values of the different characteristics of the separate groups are compared.

Final reflections
Analyses of the dramatic increase in the number of online courses offered at universities are of importance from many perspectives.One relates to the potential of online education to attract new groups of students who would otherwise not have undertaken university studies.If online courses attract these groups of students this is of importance as it will have an effect on the accumulation and distribution of human capital.Another important issue is the effect on student performance.If, all else being equal, students who attend online courses tend to perform better compared to face-to-face students, this is an argument for substituting face-to-face teaching for online teaching techniques for campus students as well.
The main conclusion from this literature review is that there is no general support of the hypothesis that online students should perform better compared to face-to-face students.Some studies, especially those based on economics students report the opposite result.Other studies report results that support the hypothesis.This disparity in the results of the studies can partly be explained by the fact that the methodologies differ, making them non-comparable.Another possible explanation is that in the papers reviewed here, as well as in most papers reviewed elsewhere, online teaching and face-to-face lectures are treated as a homogeneous good, making no distinction regarding how the Internet and ICT are used as teaching tools.Instead, online teaching is assumed to be used in the same way independently of teachers, their characteristics and teaching methods.To return to the comparison with the blackboard made earlier in this paper, it would be the same as saying it does not matter how you use the blackboard or what you write on it during the lectures as long as you use it!
Although the papers presented in this review reveal many interesting and important insights, the challenge for future research is to combine the main contribu-tions from each of these studies.As in all quantitative empirical research within the social sciences, data collection and the use of appropriate statistical methods are crucial ingredients.For future research, we suggest the following procedure: i) Collect data from many courses and subjects.The number of students within each group should be large; ii) Construct a measure which reflects the knowledge gained during the course.This could be based on the difference between the student's pre-knowledge and final test scores; iii) Include all relevant explanatory variables such as motivation, time spent in lectures, time spent on other activities, which could either distract the student from studies or have a positive effect on the ability to concentrate, as well as motivation, family situation, age, and gender.All student characteristics could be collected through the use of a student diary where the student reports all these activities during the course; iv) Include teacher characteristics such as the teacher's background, age, motivation, pre-skills in online and face-toface teaching; v) Include differences in teaching methods used by the individual teachers; vi) Use appropriate estimators in order to minimise potential bias and inconsistency in the estimators; vii) Use instrumental variables to handle the endogeneity of study time; viii) Correct for self-selection; ix) Analyse and correct for drop-outs in order to evaluate to what extent the drop-outs affect the parameter estimates.

Mounir Dahmani
PhD student in Economics at the University of Paris Sud mounir.dahmani@gmail.comMounir Dahmani is a PhD student in Economics at the University of Paris Sud.He has a degree in Industrial Economics from the University of Tunis and Master's in European Markets and Firms strategies.He is currently teaching Microeconomics, Macroeconomics and organisation theory of the firm at the University of Paris Sud.In the early years of his PhD, he participated in various research projects including eLene-TT and eLene-EE European projects.The research topics covered by his PhD are: growth theory, competencies and ICT, e-learning, human capital and digital divides.