RUSC v12i1 | Alemán [en]

MOOCs: Game Changer or Passing Fad?

Indicators of pedagogical quality for the design of a Massive Open Online Course for teacher training

Lorena Yadira Alemán de la Garza

Monterrey Institute of Technology and Higher Education (ITESM), Mexico

Teresa Sancho-Vinuesa

Universitat Oberta de Catalunya (UOC), España

Marcela Georgina Gómez Zermeño

Monterrey Institute of Technology and Higher Education (ITESM), Mexico

Submitted in: June 2014

Accepted in: November 2014

Published in: January 2015

Recommended citation

Alemán, L. Y., Sancho-Vinuesa, T., & Gómez Zermeño, M. G. (2015). Indicators of pedagogical quality for the design of a Massive Open Online Course for teacher training. RUSC. Universities and Knowledge Society Journal, 12(1). pp. 104-118. doi


Massive Open Online Courses (MOOCs) have generated high expectations and revolutionized some educational practices by providing open educational resources for reference, usage and adaptation; therefore, their pedagogical quality is often questioned. The objective of this study is to identify indicators related to pedagogical, functional, technological and time factors in order to assess the quality of the MOOC entitled “Liderazgo en gestión educativa estratégica a través del uso de la tecnología” (Leadership in strategic educational management through the use of technology), offered as a teacher training program through Coursera to 10.161 participants. Via the Delphi method, a group of 55 experts agreed that time is a key factor to be considered in the design of learning activities. It was concluded that without measuring results, the success of a MOOC could not be evaluated; thus, institutions and consortia must establish evaluation indicators to focus their efforts on the enhancement of pedagogical quality. By providing relevant information, the learning potential of educational resources based on connectivism principles can be evaluated, and so can the quality of MOOCs. The goal is to contribute to a vision of a future in which everyone has access to a world-class education.


Massive Open Online Courses, MOOC, evaluation, pedagogical quality, teacher training

Indicadores de calidad pedagógica para el diseño de un curso en línea masivo y abierto de actualización docente


Los cursos en línea, masivos y abiertos (MOOC) han generado importantes expectativas y han revolucionado algunas prácticas educativas, al ofrecer recursos educativos abiertos para su consulta, uso y adaptación; sin embargo, con frecuencia se cuestiona su calidad pedagógica. El objetivo de este estudio es identificar indicadores relacionados con factores pedagógicos, funcionales, tecnológicos y de tiempo, para evaluar la calidad del MOOC Liderazgo en gestión educativa estratégica a través del uso de la tecnología, ofrecido como recurso de actualización docente en Coursera a 10.161 participantes. Mediante el método Delphi, un grupo de 55 expertos acordó que el tiempo es un factor clave a considerarse en el diseño de las actividades de aprendizaje. Se concluye que sin medición de los resultados no se puede valorar el éxito de un MOOC, por ello instituciones y consorcios deben establecer indicadores de evaluación para enfocar sus esfuerzos para la mejora de su calidad pedagógica. Si se proporciona información relevante se podrá evaluar el potencial de aprendizaje que poseen los recursos educativos basados en principios conectivistas y reconocer la calidad pedagógica de los MOOC, con el objetivo de coadyuvar a la visión de un futuro en el que todos tengan acceso a una educación de clase mundial.

Palabras clave

cursos en línea masivos y abiertos, MOOC, evaluación, calidad pedagógica, actualización docente

1. Introduction

Currently, Massive Open Online Courses, or MOOCs, have generated high expectations and revolutionized pedagogical practices by providing open educational resources for reference, usage and adaptation (UNESCO, 2012). c-MOOCs, which gave rise to this phenomenon, adopted a pedagogical strategy with an epistemological basis grounded in connectivism; they also promoted educational change, not only through technological advances, but also through the theoretical developments that emerged from this field of study (Zapata-Ros, 2013). They provided access opportunities to the general public and were facilitated by renowned professionals who took a vital role. Thus, c-MOOCs made knowledge from a wide range of prestigious universities available to the whole world at a speed, scope, scale and price that no traditional course would be in a position to offer (Bell, 2012). Just a decade ago, this type of training would have been impossible without the current advances of technology (Friedman, 2013; Skiba, 2013).

Siemens (2004) posits that connectivism is an educational theory characterized by considering learning as an extension of a personal network through which participants learn, share knowledge and comprehend. He claimed that the most important contribution of MOOCs resided in their potential to change the relationship between students and teachers, academia and the community at large, by offering a broad and diverse virtual space, a meeting place for the exchange of ideas. He emphasized that anyone enrolling on a MOOC would probably find learning in its most exposed form, through platforms that not only invited participants to see and hear, but also to participate and collaborate.

Despite its rapid integration into the educational offering, the Institute for Prospective Technological Studies claims that the criteria for evaluating the educational quality of a MOOC lack transparency and warns that, for the moment, the implemented educational models are evidently not sustainable (Aceto, Borotis, Devine, & Fischer, 2014).

For Sangrà and Wheeler (2013), the massive aspect of the courses, which is promoted as something positive, has never been a feature of successful training. They believe that, in MOOCs, informal learning has found a perfect ally in ICT in general, and in online learning in particular. This emphasizes the need to investigate MOOCs in depth in order to establish whether they represent real opportunities for learning in informal settings, or if they are simply attempts to formalize the informal.

This article presents the results of a study aimed at identifying indicators for the educational quality of a MOOC, which will greatly improve the design of the course entitled “Liderazgo en gestión educativa estratégica a través del uso de la tecnología” (Leadership in strategic educational management through the use of technology), offered as a teacher training update strategy by a higher education institution. In the analysis, the assessment of the indicators by a group of 55 experts is presented, and the profile and expectations of 10,161 participants in the first edition of the course are described.

1.1. Background

Although the history of MOOCs may seem short in absolute terms, Boven (2013) locates its origins in open and distance education. He notes that many emerging movements have adopted the principles advocated by educational reformers, such as those proposed by the “Education for All” ideal. This perhaps explains why MOOCs have so often been described as “recurrent resources in the discourse of educational openness” (McAuley, Stewart, Siemens, & Cormier, 2010, p. 46).

As its name suggests, a MOOC is an online course accessible to virtually anyone who wishes to participate with unlimited attendance (EDUCAUSE, 2011). For Tschofen and Mackness (2012), MOOCs are online courses that appeal to a wide variety of participants around the world; they are massive because literally thousands of people can participate in them, and they are open because participants can openly share resources, ideas and experiences without any requirements. They constitute a collective creation of knowledge, resulting in a composition greater than the sum of its parts. In many ways, they are a microcosm of a nation (Liyanagunawardena, Adams, & Williams, 2013).

While reviewing the educational research undertaken, ever-increasing interest in the study of MOOCs in recent years was noted. This contemporary interest is manifested in the evolution of their conceptual references and the definition of their main features. According to Rodriguez (2012), the term ‘MOOC’ was coined by Dave Cormier when the number of students on the course entitled “Connectivism and Connective Knowledge (CCK08)” offered by George Siemens and Stephen Downes reached a total of 2,300. After analyzing the results of their experience, they could see MOOCs as a new modality of online education and stated that its implementation required conceptual changes in the processes of teaching and learning, from the perspective of teachers and students alike.

1.2. Theoretical perspectives on learning

In pedagogy, learning theories are the basis of educational processes, and they often refer to principles of behaviorism, cognitivism and constructivism to create instructional environments. Adding technology to existing learning theories raises many questions and prompts theorists to review them continuously, in order to adapt them as learning conditions change. These three theories were developed at a time when learning had not been impacted by the use of technology in education. At some point, the underlying conditions have been so significantly altered that additional modifications are no longer feasible. An interconnected world allows us to explore how information is acquired; connectivist environments have facilitated the creation of networks to share resources, ideas and experiences, and they must be evaluated. Faced with this new reality, a completely different approach is necessary (Siemens, 2004).

The SCOPEO (2013) report confirms that the “first generation” of MOOCs was based on connectivism, a pedagogical theory that posits that personal knowledge is created from a network that provides its members with information. In turn, these members provide feedback with additional information generated within the same network. The process ends when this information, which may come from different nodes, transforms and alters the knowledge base, and generates new learning for individuals. For Chiecher and Donolo (2013), MOOCs have broken curricular rigidity, questioning the ability of traditional teaching to meet training needs in changing conditions. Training offerings in which teaching differs from the what is commonly referred to as “traditional classroom education” are being overwhelmingly adopted. This has given rise to critical reflection on the perception of learners – and is something that demands a paradigm shift.

Addressing the future of MOOCs, some experts predict that they will mutate into different forms of learning with a greater emphasis on participants’ support and a decrease on their number to allow adequate attention. Experts emphasize the need to implement evaluation systems to establish pedagogical quality criteria that go beyond limited assessments, based on the “reputation” of the educational institution offering them (Menéndez, 2013).

1.3. Indicators of the educational quality of a MOOC

For the educational community, it is a reality that the use of technology in teaching practices has led to the emergence of modern social structures and organizational forms, in which the traditional space and time referents are no longer valid (Garrido, 2003). Given the lack of space and the growing demand for admission to educational institutions, online education is renowned for offering opportunities to expand the educational range and coverage. For Moore (2013), distance education is a psychological construct that depends on macro factors such as dialogue, structure and autonomy.

In relation to their pedagogical design and to the learning expectations of students, MOOCs imply a change of instructional schemes. At first, their structure was thought of in a minimalist way to allow participants to design their own learning, but research findings have since revealed that, in order to improve their pedagogical quality, multiple factors related to how, where and when participants learn must be considered (The New Media Consortium & Universitat Oberta de Catalunya, 2012).

Although MOOCs imply the design of new schemes that recognize studies outside the classroom, the lack of standards for evaluating their pedagogic quality is often questioned (Bernal, Molina, & Perez, 2013). For Gómez-Zermeño and Alemán (2012), the integration of technology into educational processes requires the establishment of both theoretical foundations and evaluation mechanisms to identify their numerous sources of influence. They point out that the design of technology-based strategies that seek to strengthen access to quality education poses significant challenges to educational researchers yet offers attractive advantages that could encourage their adoption.

In the MOOC Quality Project by the European Foundation for Quality eLearning, it became apparent that the evaluation of MOOCs was more complicated than in other online offerings due to the lack of an agreed definition on what they are, to the involvement of perceived external factors and to the lack of consensus on their purpose (EFQUEL, 2013). Bernal et al. (2013) recommend that MOOCs should apply the same quality standards used in formal open and distance courses. The fact that they are massive, open and online calls for greater rigor in their quality in order to overcome dissimilar profiles and a lack of monitoring of results and objective attainment.

In assessing the educational quality of distance education and open learning resources, Arias (2007), Cabero and Romero (2007), Gómez-Zermeño (2012), Gómez-Zermeño, Rodríguez, and Márquez (2013), Domingo and Marquès (2011), and Roig et al. (2013) used indicators related to pedagogical, functional and technological factors. For Barbera, Gros, and Kirschner (2012), time is a critical factor that has also been used as a quality measure, since it is related to the amount and the sequence in which people learn through the accumulation of experiences. In collaborative learning environments, the implementation of strategies that promote participants’ self-regulation is recommended (Franco-Casamitjana, Barbera, & Romero, 2013).

2. Method

The objectives of this research can be addressed via different methodologies. Given the object of study, a quasi-experimental design was adopted. Cross (2013) claims that educational research into MOOCs generates large methodological and interpretive challenges, as it poses new dynamics in the teaching-learning process. Strengthening its design involves considering the relationship between research and educational innovation. For Schmelkes (2001), research into an educational innovation may have a quasi-experimental design that does not require evaluation of a random sample. Moreover, the results are measurable with one post-test group (Shadish, Cook, & Campbell, 2002).

Regarding quality indicators to strengthen the design of a course of this nature, a set of indicators was selected from studies by Arias (2007), Barbera et al. (2012), Cabero and Romero (2007), Franco-Casamitjana et al. (2013), Gómez-Zermeño (2012), Gómez-Zermeño et al. (2013), Domingo and Marquès (2011), and Roig et al. (2013). A group of experts reviewed and validated these indicators using the Delphi method. This method involves selecting a group of experts, who are asked their opinion on issues relating to the future, implementing consecutive anonymous rounds to ensure the autonomy of participants. The predictive power of this method is based on the systematic use of intuitive judgment by all experts (Astigarraga, 2003).

Once the set of selected indicators had been identified, a questionnaire entitled “MOOC-I-Quality Indicators” was designed, consisting of closed questions that assess 50 indicators on a 4-point Likert scale. For the analysis, the indicators were classified into 15 subcategories related to Pedagogical, Functional, Technological and Time factors (Table 1). This instrument was applied to a group of 55 experts involved in the design, development and delivery of MOOCs, as well as in distance education courses and open learning resources offered by the institution. This group was formed by 14 lecturers responsible for design and content generation, and a total of 41 tutors, instructional designers, graphic designers, programmers and audiovisual producers.

Based on studies by Breslow, Pritchard, DeBoer, Stump, Ho, and Seaton (2013), the “MOOC-I-Participants’ Information” questionnaire, made up of 30 questions gathering demographic, employment, educational update, expectation and opinion data from those on the course, was designed. Taking into consideration the assessment made by the experts for indicators of pedagogical quality, the MOOC entitled “Liderazgo en gestión educativa estratégica a través del uso de la tecnología” (Leadership in strategic educational management through the use of technology) was designed and offered as a teacher training update strategy; this questionnaire was administered to 10,161 participants on the first edition of the MOOC.

3. Results Analysis

Aligned with the objectives of this research, the analysis of two different issues is presented. First, the assessment of quality indicators for the design of MOOCs and, second, after the MOOC in question had been designed and implemented, the students’ profiles and expectations.

3.1. Indicators for assessing the pedagogical quality of a MOOC

The results of the Delphi method show that while experts rated the indicators related to the Time factor higher, they also recognized that the quality of a MOOC must take the other factors into account (Figure 1). It is worth noting that the indicator for the time to Take exams was rated the highest. Studies by Barbera et al. (2012) corroborate that time remains a crucial strategic issue in online teaching, requiring explicit attention from teachers and designers, since it affects students’ learning. For Franco-Casamitjana et al. (2013), time management skills determine self-regulation of students and members of a group alike (Guitert, 2011). In distance education, autonomy refers to the extent to which students decide “what to learn, how to learn, and how much they learn” (Moore 2013, p. 68).

Figure 1. Indicator categories

When analyzing the results by subcategory (Table 1), the experts considered the pedagogical quality of the content of a MOOC to be a relevant factor. They also recommended a review of the pedagogical approach, the provision of tutorials and the specification of the evaluation process of educational activities. For Moore (1983), the content or topic of study determines the dialogue between teachers and learners; it also constitutes one of the main characteristics of open educational resources. Thus, excellent quality should be ensured (UNESCO, 2012).

Table 1. Results of indicator assessment by subcategory






Pedagogical approach

Tutorial and evaluation

Adequacy and adaptation to users

Motivational capacity









Ease of use

Autonomy and user control

Functionality of the documentation





Interaction and dialogues


Visual environment

Design and technology








Take exams

Perform activities

Perform exercises

Study the topics

Calendar / Schedule

Participate in discussion boards







Regarding the subcategories of functional factors, experts believe that Ease of use is an important factor, followed by Autonomy and user control, and Functionality of the documentation. Under the subcategory Ease of use, the indicator “Exercise instructions are clear and easy to understand” was positively valued, while “The activities suggest the use of additional documents (blogs, wikis, etc.)” was considered less important. Among the Technological factors, the subcategories Interaction and dialogues, Navigation and Visual environment were rated highly, while Versatility was not considered relevant. The experts mentioned, in relation to the indicators of Interaction and dialogues, that communication is one of the most important elements, emphasizing the importance of making means of communication available to foster exchanges among and between participants (discussion boards, news, etc.) and tutors.

In the experts’ opinion, MOOCs enable the comprehensive development of competitive citizens who are able to position themselves as agents of change. Experts recognize that to meet the growing demand for education, the adoption of innovative models that develop commitment and linkage between participants is required (Brown, 2013). Such models should also emphasize the need to implement an evaluation system in order to improve the educational quality of resources like MOOCs.

3.2. Participants’ profiles and expectations

Based on the quality indicators assessed by experts, the MOOC entitled “Liderazgo en gestión educativa estratégica a través del uso de la tecnología” (Leadership in strategic educational management through the use of technology) was designed and implemented using the Coursera’® platform. For three weeks, the call for mass participation in the course was disseminated via institutional electronic media and social networks. A total of 10,161 participants enrolled on the MOOC in question. The participants were from 79 countries located in all continents: The Americas 90.0%, Europe 9.0%, Asia 0.6%, Oceania 0.1% and Africa 0.1%. According to Coursera, 85% of the study’s population came from emerging economies. One of the main features of a MOOC is its openness to enrolment; so all the participants who confirmed their interest in voluntarily registering for free by answering the questionnaire “MOOC-I- Participants’ Information” formed the population of this study. Table 2 describes their main features.

Table 2: Participants’ profiles on the MOOC entitled “Liderazgo en gestión educativa estratégica a través del uso de la tecnología” (Leadership in strategic educational management through the use of technology)





Country of residence

Mexico 57%, Colombia 7%, Peru 6%, Argentina 3%, Chile 3%, Ecuador 2%, United States 2 %, Venezuela 2%, Dominican Republic 2%, Guatemala 1%, Brazil 1%, Honduras 0.7 %, El Salvador 0.7%, Costa Rica 0.6%, Bolivia 0.5%, Uruguay 0.5%, Puerto Rico 0.3%, Nicaragua 0.3%, Paraguay 0.3%, Panama 0.2%, Canada 0.2%, and with 0.1% Trinidad and Tobago, Haiti, French Guyana

Spain 7%, United Kingdom 0.3%, Italy 0.2%, Russian Federation 0.2%, Portugal 0.2%, Germany 0.2%, France 0.2%, and with 0.1% Greece, Poland, Switzerland, Belgium, Ireland, Denmark, Ukraine, Turkey, Norway, The Netherlands, Czech Republic, Hungary, Austria, Sweden, Serbia, Romania, Moldavia, Malta, Macedonia, Lithuania, Latvia, Cyprus, Croatia, Aruba, Andorra

China 0.3%, and with 0.1% Hong Kong, Korea, India, Vietnam, Taiwan, Australia, Philippines, Thailand, Singapore, New Zealand, New Caledonia, Mauritius, Malaysia, Japan, Morocco, South Africa, Angola, Algeria, Pakistan, Kazakhstan, Israel, Iran, Islamic Republic


Female 59.4% and Male 40.6%


Average of 37 years 9 months old, highest frequency 34 years old, range from 14 to 76; 75.0% is 45 or younger, 25% is older than 45.



Teacher 60.9%, Principal 11.0%, Pedagogical advisor10.9%, Higher 2.5%, Inspector 0.4%, Student 14.2%

Educational level

Preschool 8.5%, Elementary School 17.6%, Secondary School 18.7%, High School 21.4%, Superior 33.8%

Geographical zone

Urban 76.0%, Rural 11.3% and Urban/Rural 12.7%

Type of school

Public 56.2%, Private 32.8% and Public/private 11.0%

School equipment

Media room 20.1%, Internet in media room 18.8%, Classroom 10.4%, Internet in classroom 12.7%, Principal’s office 17.5%, Internet in Principal’s office 17.8% and none 2.7%


Educational level

Undergraduate 52%, Graduate 37%, High School 7%, PhD 4%

Years of Service

5 years or less 28.6%, 6 to 10 years 20.6%, 11 to 15 years 14.6%, 16 to 20 years 11.7%, 21 to 25 years 7.4%, 26 to 30 years 5.9%, more than 31 years 3.5%, No years of service 7.7%

Frequency of teacher training update courses

1 to 2 courses 53.7%, 3 to 4 courses 22.0%, 5 to 6 courses 4.5%, more than 7 courses 2.0%, does not participate in teacher training update courses 17.7%

Types of teacher training update courses

National Catalogue of Continuing Education 11.3%, Centre for Training and Teacher training Update 9.7%, National Program of Teaching Career 7.4%, Requested courses 21.6%, courses from public institutions 16.5% and courses from private institutions 18.7% and other courses 14.8%


Classroom 35.8%, Online 19.4%, Blended 40.7% and other 4.1%

Use of ICT level

None 0.6%, Basic 20.5%, Intermediate 38.5%, Advanced 32.7%, Expert 7.7%

Development of ICT level

None 3.0%, Basic 30.2%, Intermediate 39.3%, Advanced 22.7%, Expert 4.9%


Reason for participating

Take a MOOC 15.4%, Model of Strategic Educational Management 31.1%, Technological tools 25.4%, Técnologico de Monterrey course 17.7%, Learn what a MOOC is 10.1% and Other 0.3%

Main expectation

Teacher professional development 46.5%, Points for Teaching Career 4.8%, Know a MOOC 14.7%, Evidence of MOOC participation 13.6%, Evidence of Tecnológico de Monterrey 18.2%, Economic stimulus 2.3%


School 26.9%, Home 67.7% and Cybercafé 5.4%

Hours participation

No specific hour 27.1%, 8:00am-10:59am 9.1%, 11:00am-02:59pm 7.3%, 03:00pm-05:59pm 8.3%, 06:00pm-08:59pm 22.0%, 09:00pm-12:00am 26.2%

Intention to complete the course

I intend to finish the course 96.03%, I just want to know the agenda 2.01%, I just want to participate in some activities 1.52% and I do not intend to finish the course 0.45%

In relation to their expectations, 43.3% of the participants believed that through MOOCs they could acquire enriched learning and 23.6% mentioned that they would experience higher learning proficiency compared to a classroom course, while 29.8% thought that they would be getting the same learning experience. It is relevant that 96% of the participants expressed commitment to the successful completion of the course, and 68.99% reaffirmed their assurance to strengthen their professional development through participation in other MOOCs offered as part of a teacher training update strategy.

On completion, the MOOC achieved a terminal efficiency rate exceeding 22% (see MOOC 12 in Figure 2), which is considered “atypical”, as well as a higher commitment rate from participants, which reached 52.15%. It is noted that the average terminal efficiency rate of all MOOCs this institution has offered is 4%, which coincides with the percentage reported in studies by the University of Pennsylvania Graduate School of Education (Penn GSE, 2013).

Figure 2. Terminal efficiency rate with highest number of enrolments

4. Conclusions

In this paper, quality indicators endorsing the design of a MOOC on Leadership in strategic educational management have been identified, as have the profiles of participants in its first edition and the rate of terminal efficiency.

The main conclusion from this research is that the success of a MOOC cannot be evaluated without measuring results. Thus, institutions and consortia should establish indicators to focus efforts on improving their pedagogical quality. Design changes and development cannot be planned either, as these actions require the implementation of appropriate mechanisms to measure the participants’ performance, which should be integrated into the standards of their educational model. When applying quality criteria, it should be considered that the indicators cannot operate in isolation, so it is necessary to define them in order to build a complete picture of the educational system that will receive feedback.

According to 55 experts, time is a key factor that impacts the pedagogical quality of a MOOC; however, they emphasize that the results are a logical consequence of the interrelationship between the Pedagogical, Functional, Technological and Time factors. When designing a MOOC, the time it will take participants to review the content, videos, resources, exercises and tests, and to take an active part in activities and collaborative learning discussion boards should be considered (Gros, Barbera, & Kirschner, 2010). The designed system involves promoting self-regulation skills, and must include the elements and approaches that support the theory of connectivism in order to be able to offer open educational resources that provide a real strategic opportunity to improve the quality of education (UNESCO, 2012).

As the use of technology in education advances, the understanding of the elements and principles of connectivism will be challenged, and educational research will provide evidence on the network capacities and possibilities intertwined with the various learning styles of individuals. By providing pertinent information, the learning potential of MOOCs to improve teacher training may be evaluated. Only then can the pedagogical quality of MOOCs offered by educational institutions be recognized and contribute to the vision of a future in which everyone has access to a world-class education.

4.1. Future research

There are significant areas of opportunity related to the evaluation of MOOCs, as standards regarding their pedagogical quality have yet to be agreed upon. Little is known about their uses and scope, or about the ways in which they are integrated into the educational models of different institutions and consortia. Separate lines of research are therefore proposed to address this issue. On the one hand, it is crucial to carry on studying in detail the results of each of the indicators that make up the different subcategories in order to understand the interplay of all factors and their level of impact on pedagogical quality. On the other hand, the definition of success of a similar course should go further than the rate of terminal efficiency and delve deeper into the analysis of traffic patterns to understand the interests and behavior of its participants.

5. References

  • Aceto, S., Borotis, S., Devine, J., & Fischer T. (2014). Mapping and Analysing Prospective Technologies for Learning. Seville, Spain: Joint Research Centre, Institute for Prospective Technological Studies.
  • Arias, J. (2007). Evaluación de la calidad de Cursos Virtuales: Indicadores de Calidad y construcción de un cuestionario de medida (Doctoral dissertation). University of Extremadura, Extremadura, Spain.
  • Astigarraga, E. (2003). El método delphi. San Sebastián, Spain: Universidad de Deusto.
  • Barbera, E., Gros, B., & Kirschner, P. (2012). Temporal issues in e-learning research: A literature review. British Journal of Educational Technology, 43(2), 53-55. doi: 10.1111/j.1467-8535.2011.01255.x
  • Bell, M. (2012). Massive open online courses moving ahead with MOOCs. Internet@Schools, 19(5). Retrieved from
  • Bernal, Y., Molina, M., & Pérez, M. (2013). La Calidad de la Educación a Distancia: El caso de los MOOC. Revista Iberoamericana para la Investigación y el Desarrollo Educativo, 3(10), 1-13. Retrieved from
  • Boven, D. (2013). The Next Game Changer: The Historical Antecedents of the MOOC Movement in Education. eLearning Papers, 33, 1-7. Retrieved from
  • Breslow, L., Pritchard, D., DeBoer, J., Stump, G., Ho, A., & Seaton, D. (2013). Studying learning in the worldwide classroom: Research into edX’s first MOOC. Research & Practice in Assessment Journal, 8, 13-25. Retrieved from
  • Brown, S. (2013). Back to the future with MOOCs. ICICTE 2013 Proceedings, 237-246. Retrieved from
  • Cabero, J., & Romero, R. (2007). Diseño y producción de TIC para la formación. Barcelona, Spain: UOC.
  • Chiecher, A., & Donolo, D. (2013). Trabajo grupal mediado por foros. Aportes para el análisis de la presencia social, cognitiva y didáctica en la comunicación asincrónica. In A. C. Chiecher, D. S. Donolo, & J. L. Córica (Eds.), Entornos virtuales y aprendizaje. Nuevas perspectivas de estudio e investigaciones (pp. 151-198). Mendoza, Argentina: Editorial Virtual Argentina.
  • Cross, S. (2013). Evaluation of the OLDS MOOC curriculum design course: participant perspectives, expectations and experiences. OLDS MOOC Project, Milton Keynes, United Kingdom.
  • Domingo, M., & Marquès, P. (2011). Classroom 2.0 Experiences and Building on the Use of ICT in Teaching. Comunicar, 18(37), 169-174.
  • EDUCAUSE (2011). 7 things you should know about MOOCs. EDUCAUSE Learning Initiative.
  • European Foundation for Quality in e-Learning [EFQUEL] (2013). The MOOC Quality Project. Retrieved from
  • Franco-Casamitjana, M., Barbera, E., & Romero, M. (2013). A Methodological Definition for Time Regulation Patterns and Learning Efficiency in Collaborative Learning Contexts. eLC Research Paper Series, 6, 52-62. Retrieved from
  • Friedman, T. (2013, January 26). Revolution hits the universities. The New York Times, The opinion pages. Retrieved from
  • Garrido, A. (2003). El aprendizaje como identidad de participación en la práctica de una comunidad virtual (Doctoral dissertation). Universitat Oberta de Catalunya, Barcelona.
  • Gómez-Zermeño, M. G. (2012). Bibliotecas digitales: recursos bibliográficos electrónicos en educación básica. Comunicar, 20(39), 119-128. Retrieved from
  • Gómez-Zermeño, M. G., & Alemán, L. Y. (2012). Administración de proyectos de capacitación basados en tecnología. Monterrey, Mexico: Editorial Digital del Tecnológico de Monterrey.
  • Gómez-Zermeño, M. G., Rodríguez, J. A., & Márquez, S. (2013). Estudio Exploratorio-Descriptivo “Curso Híbrido: Contabilidad V”. Revista de Investigación Educativa de la Escuela de Graduados en Educación, 4(7), 70-79. Retrieved from
  • Gros, B., Barbera, E., & Kirschner, P. (2010). Time factor in e-Learning: impact literature review. eLC Research Paper Series, 0, 16-31. Retrieved from
  • Guitert, M. (2011). Time management in virtual collaborative learning: the case of the Universitat Oberta de Catalunya (UOC). eLC Research Paper Series, 2, 5-16. Retrieved from
  • ITESM (2014). Formación que transforma vidas. Monterrey, Mexico: Tecnológico de Monterrey.
  • Liyanagunawardena, T., Adams, A., & Williams, S. (2013). MOOCs: A systematic study of the published literature 2008-2012. The International Review of Research in Open and Distance Learning, 14(3), 202-227. Retrieved from
  • McAuley, A., Stewart, B., Siemens, G., & Cormier, D. (2010). The MOOC model for digital practice. Digital ways of knowing and learning. Charlottetown, Canada: University of Prince Edward.
  • Menéndez, L. (2013). Estudiar ‘online’ y gratis. Escritura Pública, 80, 18-21. Retrieved from
  • Moore, M. G. (1983). Theory of transactional distance. In D. Keegan (Ed.), Theoretical Principles of Distance Education (pp. 22-38). New York, USA: Routledge.
  • Moore, M. G. (2013). The theory of transactional distance. In M. G. Moore (Ed.), Handbook of distance education (3rd ed., pp. 66-85). New York, USA: Routledge.
  • Observatorio de Innovación Educativa (May, 2014). Reporte Edu Trends. Monterrey, Mexico: Tecnológico de Monterrey. Retrieved from
  • Penn GSE (2013). Study shows MOOCs have relatively few active users, with only a few persisting to course end. University of Pennsylvania: Graduate School of Education. Retrieved from
  • Rodriguez, O. (2012). MOOCs and the AI-Stanford like Courses: two successful and distinct course formats for massive open online courses. European Journal of Open, Distance, and E-Learning, 2012(2), 1-13. Retrieved from
  • Roig, R., Flores, C., Álvarez, J., Blasco, J., Grau, S., Guarinos, I., … Tortosa, M. (2013). Características de los ambientes de aprendizaje on-line para una práctica docente de calidad. Indicadores de evaluación. Alicante, Spain: Universidad de Alicante.
  • Romero, M. (2011). The time factor in an online group course from the point of view of its students. eLC Research Paper Series, 2, 17-28. Retrieved from
  • Sangrà, A., & Wheeler, S. (2013). New Informal Ways of Learning: Or Are We Formalising the Informal? RUSC. Universities and Knowledge Society Journal, 10(1), 286-293.
  • Schmelkes, S. (2001). La investigación en la innovación educativa. Mexico: CINVESTAV.
  • SCOPEO (2013). MOOC: Estado de la situación actual, posibilidades, retos y futuro. SCOPEO Report No 2, June 2013. Retrieved from
  • Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, USA: Houghton Mifflin.
  • Siemens, G. (2004). Connectivism. A learning theory for the digital age. International Journal of Instructional Technology and Distance learning, 2(1), 3-10. Retrieved from
  • Skiba, D. (2013). MOOCs and the Future of Nursing. Nursing Education Perspectives, 34(3), 202-204. doi: 10.5480/1536-5026-34.3.202
  • The New Media Consortium & Universitat Oberta de Catalunya (2012). Perspectivas tecnológicas: educación superior en Iberoamérica 2012-2017: Un Análisis Regional del Informe Horizon del NMC y la UOC. Barcelona, Spain: UOC.
  • Tschofen, C., & Mackness, J. (2012). Connectivism and dimensions of individual experience. The International Review of Research in Open and Distance Learning, 13(1), 124-143. Retrieved from
  • UNESCO (2012). 2012 Paris OER Declaration. 2012 World Open Educational Resources (OER) Congress. Paris, France: UNESCO.
  • Zapata-Ros, M. (2013). MOOCs, una visión crítica y una alternativa complementaria: La individualización del aprendizaje y de la ayuda pedagógica. Universidad de Alcalá de Henares, Spain. Retrieved from

About the authors

Lorena Yadira Alemán de la Garza

Coordinator of Special Projects and Continuing Education, Monterrey Institute of Technology and Higher Education (ITESM), Mexico

Doctoral candidate on the Education and ICT (e-learning) doctoral program at the Open University of Catalonia (UOC), Spain. She holds a master’s degree, with honors, in Educational Institution Administration from ITESM, and a bachelor’s degree, with honors, in Business Administration from TecMilenio University, Mexico. She has worked as a postgraduate lecturer in the Graduate School of Education at ITESM on master’s degree programs in Educational Institution Administration, in Education and in Educational Technology. She is a member of the Research Chair in The School as a Knowledge Organization. She is the technical manager on educational research projects of the Mexican National Council of Science and Technology (CONACYT). She is the coordinator of university extension programs and special projects. Of note among these is the Higher Middle Education Teacher Training program (PROFORDEMS) and the New Teaching Profile diploma.

Instituto Tecnológico y de Estudios Superiores de Monterrey (ITESM)

Avda. Eugenio Garza Sada, 2501 Sur

Edificio CEDES, piso SS1

64849 Monterrey (Nuevo León)


Teresa Sancho-Vinuesa

Lecturer and IN3 Researcher, Open University of Catalonia (UOC), Spain

Vice-rector for Research and Innovation at the UOC. She holds a doctorate in Electronic Engineering and a bachelor’s degree in Mathematics. She is the principal researcher of the CIMANET (Online Science and Mathematics) RD&I group at the UOC. She has lectured in the field of numerical analysis, probability theory and stochastic processes at La Salle School of Engineering and Architecture, Spain. She has been a member of the TEXTO pedagogical and editorial team, and has coordinated Mathematics on Information Technology Studies at the UOC. She has been the academic coordinator and director of the doctoral program in Information and Knowledge Society at the UOC. She is currently responsible for the Mathematics subjects on the bachelor’s degree program in Technical Engineering and Telecommunications. She has been a researcher on the interdisciplinary research program on the Information Society in Catalonia (Project Internet Catalonia), jointly led by Manuel Castells and Imma Tubella; within this framework, she headed a study on changes occurring in Catalan universities by analyzing Internet use. She has now resumed her activities in the field of Mathematics teaching for Network-based Engineering.

Universitat Oberta de Catalunya (UOC)

Rambla del Poblenou, 156

08018 Barcelona


Marcela Georgina Gómez Zermeño

Researcher-Lecturer, Director of the Education Research Center, Monterrey Institute of Technology and Higher Education (ITESM), Mexico

She holds a doctorate in Educational Innovation from the Graduate School of Education at ITESM, and a master’s degree in Information and Communication Technology Engineering Sciences. INT-CITCOM, France Télécoms Higher Education. She also holds a bachelor’s degree in Computer and Administration Systems from ITESM. She is a tenured lecturer at the Graduate School of Education at ITESM on master’s degree programs in Education and in Educational Technology, and on the doctoral program in Educational Innovation. She is a member of the Research Chairs in The School as a Knowledge Organization and in Innovation in Technology and Education, and of the Mexican Education Research Council (COMIE). She is the technical manager on educational research projects of the Mexican National Council of Science and Technology (CONACYT) and on the ALFA program of the European Commission. She forms part of the National System of Researchers (SNI) Level 1.

Instituto Tecnológico y de Estudios Superiores de Monterrey (ITESM)

Avda. Eugenio Garza Sada, 2501 Sur

Edificio CEDES, piso SS1

64849 Monterrey (Nuevo León)



  • There are currently no refbacks.

 Universitat Oberta de Catalunya. eLearn Center

RUSC. Universities and Knowledge Society Journal is an e-journal edited by the Universitat Oberta de Catalunya (Barcelona).

Creative Commons
The texts published in this journal are – unless indicated otherwise – covered by the Creative Commons Spain Attribution 3.0 licence. You may copy, distribute, transmit and adapt the work, provided you attribute it (authorship, journal name, publisher) in the manner specified by the author(s) or licensor(s). The full text of the licence can be consulted here: