Email this article 
Hide
Show all
1. Introduction
In the past few years, there has been increasing interest in Massive Open Online Courses (MOOCs) as an innovative form of Technology-Enhanced Learning (TEL) in higher education. MOOCs are leading the new revolution of TEL, by providing new opportunities to a massive number of learners to attend free online courses from anywhere in the world without any entry requirements (Liyanagunawardena, Adams, & Williams., 2013). The current MOOC-related literature has categorized MOOCs into two main types: “cMOOCs” and “xMOOCs” (Daniel, 2012). cMOOCs provide a space for self-organized learning where learners can define their own objectives, present their own ideas, and collaboratively create and share knowledge. cMOOCs enable learners to build their own networks via blogs, wikis, Google groups, Twitter, Facebook, and other social networking tools outside the learning environment without any monitoring from the teacher (Kruiderink, 2013). On the other hand, for universities and educational institutions, the choice about how to use the MOOC environment to educate thousands of learners is more related to content-based xMOOCs that provide limited communication space between the course participants (Gaebel, 2013). Unlike cMOOCs, communication in xMOOCs happens within the platform itself.
cMOOCs build upon connectivism, proposed by George Siemens as a new learning theory for the digital age. Connectivism aims to build knowledge through interaction in learner networks and views learning as a network forming process (Siemens, 2005; Downes, 2006).
On the other hand, xMOOCs are mainly driven by behaviorism and cognitivism theories with some social constructivism components that focus on learning-by-doing (i.e., experimental, project-based, or task-based) activities. Figure 1 shows the key concepts of cMOOCs and xMOOCs.
Figure 1. Key concepts of cMOOCs and xMOOCs (Yousef, et al., 2014a)
Recently, new forms of MOOCs have emerged. These include smOOCs as small open online courses with a relatively small number of participants (e.g., COER13) and blended MOOCs (bMOOCs) as hybrid MOOCs including in-class and online mediated instruction (e.g., OPCO11). Figure 2 shows the different types of MOOCs and their underlying learning theories. (Coates, 2013; Gaebel, 2013; Yousef, et al., 2014a).
Figure 2. MOOC types (Yousef, et al., 2014a)
Connectivism and Connective Knowledge (CCK08) was the first cMOOC offered in 2008 by George Siemens and Stephen Downes at the University of Manitoba based on connectivism. This course attracted more than 2,200 informal participants from all over the world. The success of CCK08 led elite United States institutions, i.e., Stanford University, Harvard, and MIT to offer free online courses in 2011 called “extension MOOCs” (xMOOCs). These courses are quite different from cMOOCs. They follow behaviorism and cognitivism learning theories, which consider all learning experience as a result of the impact of human action with the environment (Daniel, 2012, Yousef, et al., 2014a). In 2013, E-teaching.org in Germany organized the Collaborative Online Course on Open Educational Resources (COER13), with a relatively small number of participants (i.e., less than 1,000 registered). This model, called smOOCs, entails cMOOC collaborative network interactions but uses the structure of xMOOCs by providing weekly instructional videos, reading materials and relevant web resources for each unit (Yousef, et al., 2014a; Arnold, Kumar, Thillosen, & Ebner, 2014). The Open Course (OPCO11) is an example of a bMOOC which represents a new MOOC model that aims to bring in-class (i.e., face-to-face) interactions and online learning components together. bMOOCs are in their infancy and there are different approaches for designing and embedding bMOOC environments in the higher education landscape.
Regardless of the several debates, both for and against MOOCs, the fact is that MOOCs have succeeded in attracting thousands of participants worldwide per course. Despite their increasing popularity, MOOCs suffer from several limitations. Several studies have reported a high drop-out rate averaging 95% of course participants, as well as other pedagogical problems concerning assessment and feedback (Hill, 2013). One of the potential reasons for that is the complexity and diversity of MOOC participants. This diversity is not only related to cultural and demographic attributes, but also to the diversity of motives and perspectives when enrolled in MOOCs. This raises a serious question about the different patterns of MOOC stakeholders and their perspectives when participating in MOOCs. Thus, there is a need to analyze and cluster the interest patterns of MOOC stakeholders. This paper is an endeavor to:
- Raise the importance of considering the different patterns of MOOC stakeholders.
- Cluster the different patterns of MOOC stakeholders to build a deeper and better understanding of their behaviors.
- Analyze MOOC stakeholder perspectives.
- Highlight some future research opportunities in the area of MOOCs that should be considered in the development of MOOC environments.
In light of these goals, this paper is structured as follows: Section 2 describes the research methodology and how we collected the research data. In Section 3, we present details of the clustering analysis results of different MOOC stakeholder patterns. Then, in Section 4, we discuss the results of MOOC stakeholder motives and perspectives. Finally, Section 5 gives a summary of the main findings of this paper and highlights new opportunities for future work.
2. Methodology
This study follows the action research methodology. Action research is an interactive inquiry process that allows researchers to examine the results of several research phases in a collaborative context with data-driven collaborative analysis to understand the underlying identified problem (Heller, 2004). The study consists of three phases. Firstly, we designed a survey to collect and identify different goals from MOOC stakeholders when they participate in MOOCs. Secondly, we transcribed and analyzed the survey data using different concept mapping analysis methods. Thirdly, we discuss the main characteristics of each MOOC stakeholders cluster.
2.1 Survey Design
The data analyzed here were gathered from an open-ended question at the beginning of a two page Likert-scale questionnaire about the quality of MOOCs, in order to collect feedback from different MOOC stakeholders concerning the objectives behind participating in MOOCs. The first part of the questionnaire consisted of questions related to the participant’s demographic information, experience in TEL, and the main open-ended question was “What are your goals/objectives when participating in MOOCs?” The second part of the questionnaire consisted of closed-ended questions that aimed to identify specific criteria that needed to be considered when designing and implementing MOOCs. The preliminary results of the criteria analysis are discussed in (Yousef, et al., 2014b). In the paper at hand, we focus on the analysis of the responses to the open-ended question above in order to cluster the different MOOC stakeholder perspectives.
We invited a wide sample of MOOC stakeholders to participate in the survey. A total of 205 completed the survey (107 learners who had participated in one or more online courses and 98 professors who had taught at least one MOOC). Only 158 respondents answered the open-ended question from the first part.
2.2 Participants
The demographic profile of this survey was divided into professors (as MOOC providers) and learners. More precisely, the participants were as follows:
- Professors: 76 professors who had taught a MOOC completed this survey: 41% from Europe, 42% from the United States and 17% from Asia.
- Learners: 82 learners participated in the survey. A slight majority of these learners was female (53%). Of the learners, 14% were aged between 18 and 24 years, 23% between 25 and 29, nearly 13% between 30 and 34, 13% between 35 and 39, and 37% over 40. About 36% were Bachelor’s students, 40% Master’s, 12% PhD, and 12% at high school and other levels. All of them had taken one or more online courses, and 92% had prior experience with MOOCs. These learners came from 41 different countries and cultural backgrounds in Europe, United States, Australia, Asia, and Africa.
2.3 Limitations
This survey may not be generalizable due to the limited number of participants who responded to this survey. Despite the low response rate, the heterogeneous profiles and goals of the respondents makes our sample valid in this field. The analysis of the collected dataset provides a major step forward in the understanding of MOOC stakeholder perspectives.
3. Data Analysis
We received 158 responses (N = 158) to the main open-ended question “What are your goals/objectives when participating in MOOCs?”, reflecting different MOOC goals and perspectives. Our initial intention was to split up the analysis of the survey results based on the learners’ and professors’ perspectives and analyze the interest patterns within these two groups. After analyzing the results, we found, however, that there is no significant difference between the two groups. Thus, we decided to merge the two groups and analyze the whole dataset to highlight the main clusters of MOOC stakeholder perspectives. We used the inductive category development method for applying qualitative content analysis (Mayring, 2003). We then applied the Leximancer concept analysis approach (Smith & Humphreys, 2006) and the Nvivo 10 cluster coding similarity approach (Richards, 1999) to perform an automatic analysis of the conceptual content of the survey answers. In the following sections, we give a detailed report of the results from the analysis phase.
3.1 Inductive Category Development Method
Mayring’s qualitative content analysis method was developed in the 1980s to analyze open-ended surveys and interviews transcripts (Mayring, 2003). This inductive category development included six iterative steps as shown in Figure 3.
Figure 3. Inductive category development method (Mayring, 2000)
We started applying the inductive category development method by formulating an initial description of the meaning of a cluster and writing a memo about it. We then created an initial version of the categories around the core terms: hybrid learning, design, flexibility, quality of content, lifelong learning, collaborative learning, openness, and student-centered learning. Within a feedback loop we discussed the definition of each category to ensure that we had a similar understanding of the category meanings. After that, two experts who have experience with MOOCs and who had been working independently from each other started mapping all the survey responses to these categories. The result of this step was two lists of categories marked with the text segments that are very relevant to each category. We confirmed those lists by applying the inter-rater reliability statistical formulas to measure the agreement achieved. Table 1 shows the results of inter-rater reliability between the two experts based on Cohen’s kappa and Krippendorff’s alpha.
|
Coding |
Percent Agreement |
Cohen’s Kappa |
Krippendorff’s Alpha |
N Agreements |
N Disagreements |
N Cases |
|---|---|---|---|---|---|---|
|
Expert 1 & Expert2 |
87.3% |
0.848 |
0.848 |
138 |
20 |
158 |
The Cohen’s kappa and Krippendorff’s alpha coefficients for inter-rater reliability are 0.848, thus indicating a high level of agreement (87.3%) in the mapping of the responses to the categories.
3.2 Leximancer Concept Analysis Approach
In addition to the manual inductive category development method, we leveraged the Leximancer concept analysis tool to perform the clustering analysis of the survey responses. Leximancer is an automated text mining method that extracts the main concepts from the survey responses. In Leximancer, concepts are not merely keywords, but focused clusters of related, defining terms as conceptualized by the text author (Leximancer, 2013). The procedures behind Leximancer are based on Bayesian statistical theory, where fragmented pieces of evidence can be used to predict what is actually happening in a system (Smith & Humphreys, 2006).
Leximancer assisted us in analyzing and clarifying the quantitative findings of the textual content from the survey responses and illustrating them as concept dimensions of MOOC patterns through the processes of (1) conducting semantic concept retrieval of MOOC stakeholder objectives, (2) viewing concept maps of objectives in graph format, and (3) clustering the concepts into piles to show how they are related to each other (Cretchley, Gallois, Chenery, & Smith, 2010; Smith & Humphreys, 2006; Watson, Smith, & Watter, 2005).
In order to upload the survey data into the Leximancer system, we created a CSV file with the 158 survey responses. A concept map was automatically generated by extracting the most important concepts from the MOOC stakeholder objectives. The algorithms used to generate this concept map do not only analyze well-structured text, but also text where the stakeholders used dot points or short answers. This concept map illustrates a deeper look at how objectives are related to each other, as shown in Figure 4. Each concept on the map represents some of the MOOC stakeholder objectives reported in the survey. Each concept has a colored text that indicates the relationship of this concept to other concepts with the same color in the map. The colored lines do not only consider the relationship among the same concepts group (i.e., with the same cluster), but also the intersections between different concepts groups.
Figure 4. MOOC objectives concept map generated by Leximancer
In a next step, Leximancer groups related concepts that co-occur with other concepts in the map. As a result, similar concepts are clustered together, as illustrated in Figure 5.
Figure 5. Clustering of MOOC stakeholder objectives
The final step in the Leximancer analysis is to identify the label that best represents each cluster. In order to attach significant labels to the clusters, we checked the concept labels that the Leximancer system proposed and combined them with the category labels that have been used in Mayring’s inductive category development analysis in Section 3.1. As a result, the following eight clusters were identified: blended learning, instructional design and learning methodology, flexibility, high quality content, lifelong learning, network learning, openness, and student-centered learning.
We validated the clustering results by applying the inter-rater reliability coefficient between the mapping of the responses to the cluster labels provided by the two experts and Leximancer. Table 2 shows the results of pairwise percent agreement, pairwise Cohen’s kappa, and Krippendorff’s alpha. The high Cohen’s kappa and Krippendorff’s alpha coefficients for inter-rater reliability (0.893) reveal an accurate clustering of the responses.
|
Coding |
Avg. Pairwise Percent Agreement |
Avg. Pairwise Cohen’s Kappa |
Krippendorff’s Alpha |
N Cases |
|---|---|---|---|---|
|
Expert 1 & Expert 2 & Leximancer |
91.139% |
0.893 |
0.893 |
158 |
Figure 6 shows the different patterns of MOOC stakeholders (i.e., their goals when participating in MOOCs). As a next step in the analysis, we investigated the relationship among these clusters by applying the Nvivo 10 cluster coding similarity approach (Richards, 1999).
Figure 6. Number of participants in each cluster (N=158)
3.3 Nvivo 10 Cluster Coding Similarity Approach
A similarity metric is a statistical method used to calculate correlation among clusters. The Nvivo 10 cluster coding similarity approach allows the clustered data to be analyzed in terms of similarities in attribute values based on Pearson’s correlation coefficient, Jaccard’s coefficient, and Sørensen’s coefficient (Bazeley & Jackson, 2013; Richards, 1999).
We provided the final eight clusters of MOOC stakeholders and the responses associated with each cluster as input to Nvivo 10. We then applied the coding similarity metric to measure the similarity between these clusters. The result was a horizontal diagram that shows similar items on the same branch and dissimilar items on different branches, as shown in Figure 7.
Figure 7. MOOC stakeholders cluster coding similarity
There is little work that attempts to find the relationship between stakeholder motives when involved in MOOCs and the type of MOOC itself. The result of the cluster coding similarity provides the opportunity to detect potential relationships between stakeholder objectives and MOOC type. As shown in Figure 7, the blended learning, flexibility, high quality content, and instructional design and learning methodologies clusters are tied together in the first branch. This grouping reflects the main features of xMOOCs characterized by a replication of traditional educational practices driven by formal learning institutions. xMOOCs have predefined course structures, focus on the provision of high quality content, and follow teacher-led instructional design methodologies. Moreover, xMOOCs provide flexible access to a wide range of learning materials and offer the opportunity to bring together online and face-to-face learning.
On the other hand, lifelong learning, network learning, openness, and student-centered learning are grouped together in the second branch. This grouping reflects the main characteristics of cMOOCs. Unlike xMOOCs, which focus on formal learning, cMOOCs are often used to support open, networked, self-organized, and lifelong learning. This kind of learning tends to be experimental, spontaneous, and free from rigid curricula; thus offering new opportunities for personal development (Fernández, 2013).
Table 3 summarizes the relationship between stakeholder perspectives when involved in MOOCs and MOOC type. Besides xMOOCs and cMOOCs, we present possible stakeholder perspectives in smOOCs and bMOOCs, driven by Figure 2 and the characteristics of these MOOC types as discussed in the literature (Coates, 2013; Gaebel, 2013; Yousef, et al., 2014a).
|
Clusters |
cMOOCs |
xMOOCs |
smOOCs |
bMOOCs |
|---|---|---|---|---|
|
Blended Learning |
- |
√ |
(√) |
√ |
|
Flexibility |
- |
√ |
- |
√ |
|
High Quality Content |
- |
√ |
√ |
√ |
|
Instructional Design and Learning Methodologies |
- |
√ |
- |
√ |
|
Lifelong Learning |
√ |
- |
(√) |
(√) |
|
Network Learning |
√ |
- |
(√) |
(√) |
|
Openness |
√ |
- |
(√) |
(√) |
|
Student-Centered Learning |
√ |
- |
√ |
(√) |
|
√ Completely (√) Partly – Very limited supported |
||||
4. Discussion
Our study aimed to cluster and analyze the main stakeholder objectives behind participating in MOOCs. In the previous sections, we presented the details of the clustering analysis of MOOC stakeholder perspectives. In short, the main perspectives include blended learning, flexibility, high quality content, instructional design and learning methodologies, lifelong learning, network learning, openness, and student-centered learning. In this section, we focus on the discussion of the clustering results by performing both a quantitative and qualitative analysis.
4.1 Quantitative Analysis
Figure 6 shows the clustering results and the number of participants in each cluster. Nearly one third of MOOC stakeholders (49 out of 158) consider lifelong learning as the main objective behind their participation in MOOCs. Of the participants, 30% were interested in instructional design and learning methodologies, and high quality content. The remaining clusters, i.e., network learning, flexibility, openness, blended learning, and student-centered learning include relatively fewer participants.
The high number of participants assigned to the lifelong learning cluster can be explained by the demographic information in the survey. In fact, the majority of the respondents (82%) were adults aged over 30 years, where 46% were over 40. This finding is in agreement with Liyanagunawardena et al.’s (2013), de Waard et al.’s (2011), and Hill’s (2013) findings, which showed that most of the participants who have participated in MOOCs are adult learners over the age of 30, and are often referred to as lifelong learners.
4.2 Qualitative Analysis
The aim of the qualitative analysis is to build a deeper and better understanding of MOOC stakeholder perspectives. This can help MOOC providers in designing and implementing successful MOOC environments that address the different goals of their participants. In the following sections, we discuss the stakeholder perspectives in each cluster.
4.2.1 Blended Learning
Blended learning has become an important TEL model by integrating online and traditional face-to-face learning (Yousef, et al., 2014c). In our study, 5.7% of MOOC stakeholders reported that their primary goal for participating in MOOCs was to enhance their classroom learning and to improve relationships with teachers and peers. Some representative objectives in the cluster are: “enhancing capabilities”, “acquiring better study habits”, and “getting used to new technologies for learning”, “try to reduce the effort of the teacher with students in his class without losing quality”, “to experiment interactivity at a distance and integrating MOOCs with traditional classes”, and “to support face-to-face learning with Technology-Enhanced Learning” .
4.2.2 Flexibility
One of the successful factors in MOOCs is flexibility (Mackness, Mak, & Williams, 2010). Along that line, 9% of MOOC stakeholders reported that the major reason for their participation in MOOCs was the ability to access information and resources at a time and a place convenient to them. Some objectives included in this cluster are: “learning at my own pace”, “diversity of learning material”, and “communicate with peers synchronously as well as asynchronously across space, time, and pace”.
4.2.3 High Quality Content
This cluster reflects the significance of high quality content to empower and engage people around the world to participate in MOOCs. High quality content was a major goal for 13% of the participants. Some of the objectives in this cluster are: “to learn from the best universities all over the world”, “to gain experience from top universities”, and “get free online courses from the world’s leading universities”.
4.2.4 Instructional Design and Learning Methodologies
The instructional design and learning methodologies cluster represents 17% of MOOC stakeholders. The focus in this cluster is on a pedagogical design that can engage learners to attend courses, and on technological design criteria that can make MOOCs more dynamic. Participants in this cluster mainly aimed to investigate new learning methodologies and to research innovative instructional design approaches. Some representative objectives are: “provides some scaffolding for learners”, “learn complementary techniques”, “to promote a new pedagogical paradigm for personal knowledge management”, and “learning how to develop and organize effective MOOCs or flipped classrooms”, and “how to investigate some new component of assessment methods”.
4.2.5 Lifelong Learning
MOOCs open doors for new lifelong learning opportunities (Kop, Fournier, & Mak, 2011). This cluster stresses the advantage of MOOCs for those who are working full-time or have taken a break from formal education. Of the stakeholders, 31% consider lifelong learning as the main objective behind their participation in MOOCs. This high number reflects the fact that people are tending to learn through MOOCs for their personal or professional interest rather than obtaining an official academic degree. Representative objectives for this cluster are: “self-improvement for career advancement”, “professional development”, and “MOOCs open the mind to expand my horizon and ongoing learning for job requirements”.
4.2.6 Network Learning
This cluster reflects the original concept of early cMOOCs launched by Downes and Siemens (CCK08), which are based on connectivism. In the network learning model, learners are allowed to network together for developing, discussing and exploring alternatives, and for sharing responsibilities for their learning. Of the participants, 12% had network learning as a major goal behind their participation in MOOCs. Some representative objectives are: “working cooperatively in groups”, “share goals, ideas, resources, activities” and “supporting each other”.
4.2.7 Openness
This cluster reflects the 4Rs that characterize openness, i.e., Reuse, Revise, Remix, and Redistribute (Peter & Deimann, 2013). Openness also refers to accessing open educational resources (OER), e.g., course notes, PowerPoint presentations, video lectures and assessment, thus providing a learning experience to a vast number of participants around the globe regardless of their location, age, income, ideology, and level of education, without any entry requirements or course fees. This cluster represents 7.6% of MOOC stakeholders in our study. Some representative objectives are: “provide materials that are easy-to-update”, “the most important one, all of the courses are free”, “how I learn with OER”.
4.2.8 Student-Centered Learning
Student-centered learning puts the learner at the center of the learning activity (Chatti, 2010). Student-centered MOOCs focus on the interests of the learners rather than teachers and providers. They provide a space for learners to be active participants in the learning process and to get mutual support. In our study, only 4.4% of MOOC stakeholders mentioned student-centered learning as a goal. Representative objectives in this cluster are “put myself in the shoes of a student”, “learn in a semi-organized structure as opposed to an organized ‘school’ system”, “self-regulated”, and “self-reflection on the learning process and the impact of different learning designs from a learner perspective”.
5. Conclusion and Future Work
MOOCs are an innovative form of Video-Based Learning (VBL) in the sense that they provide opportunities to a massive number of learners to attend free online courses around the globe. However, the high drop-out rate averaging 95% has been frequently noted in MOOC-related literature. One of the potential reasons for that is the complexity and diversity of MOOC participants. This diversity of MOOC participants is not only related to the cultural and demographic profile, but also to the motives and perspectives when enrolled in MOOCs. This paper aimed to cluster the different patterns of MOOC stakeholders in order to build a deeper and better understanding of their behaviors. To the best of our knowledge, this paper represents the first attempt to cluster MOOC stakeholder perspectives.
We conducted an online survey in order to answer an open-ended question “What are your goals/objectives when participating in MOOCs?”. We received 158 responses from learners and professors. We applied different concept mapping analysis methods in order to analyze the survey responses. The clustering resulted in a set of eight groups. The cluster with the highest number of participants is lifelong learning (49), followed by instructional design and learning methodologies (27), high quality content (21), network learning (19), flexibility (14), openness (12), blended learning (9), and student-centered learning (7). The computation of the similarity between the clusters, which indicates the relationships between the same, resulted in two bigger clusters. One reflects the characteristics of xMOOCs and contains blended learning, flexibility, high quality content, and instructional design and learning methodologies clusters. The other reflects the characteristics of cMOOCs and contains lifelong learning, network learning, openness, and student-centered learning. According to this clustering, the number of participants with goals related to cMOOCs (87) was found to be slightly higher than those interested in xMOOCs (71). However, most MOOC implementations continue to focus on xMOOCs that follow a top-down, controlled, teacher-centered, and centralized learning model. Attempts to implement bottom-up, student-centered, really open, and distributed forms of MOOCs (i.e., cMOOCs) are the exception rather than the rule. Thus, we need to put more emphasis on the implementation of hybrid MOOCs that can combine the advantages of both xMOOCs and cMOOCs to meet the goals of a wide range of participants. This might be a solution for reducing drop-out rates in the current MOOCs. Our future work will investigate a set of specific criteria related to each cluster. These criteria would help us in designing successful hybrid MOOCs reflecting different stakeholder perspectives.
