Methodological proposal for the application of international benchmarking in order to assess the quality of virtual higher education

This research focuses on providing a benchmarking methodology that allows any university offering virtual education (e-learning or b-learning) to compare itself to the best international universities in order to improve the quality of the virtual education offered. In order to verify the adequacy of the proposed methodology, it has been applied to the Polish Virtual University (PUW). On the other hand, the Open University of Catalonia (UOC) was chosen as the leading partner university. The benchmarking performed has allowed the author to obtain solid results in order to draw up and implement a complete action plan for improvement, which will make it possible to increase, in the short and medium terms, the quality of the virtual education offered by PUW. It is therefore concluded that the proposed benchmarking methodology is adequate in order to assess and improve the quality of virtual higher education. El objetivo de este trabajo consiste en proporcionar una metodología de benchmarking que permita a las universidades que ofrezcan educación virtual compararse con las mejores universidades a nivel internacional al objeto de mejorar la calidad de su educación virtual. Para comprobar la idoneidad de la metodología propuesta, esta se aplica a la Universidad Virtual de Polonia (PUW). Como universidad socia (líder) se elige la Universitat Oberta de Catalunya (UOC). El benchmarking realizado ha permitido obtener unos resultados sólidos para la elaboración y posterior implantación de un completo plan de acción de mejora que permitirá mejorar a corto y medio plazo la calidad de la educación virtual de la Universidad Virtual de Polonia. Por ello, se concluye que la metodología de benchmarking propuesta es adecuada para evaluar y mejorar la calidad de la educación superior virtual.


Introduction
The development of virtual learning environments, the emergence of competiveness in those environments and the students' expectations and conditions require universities to focus on a high quality virtual education and to apply methods in order to improve its quality.
One of the methods is benchmarking, that is, a systematic process carried out by a specific organization in order to learn from the best ones on the local (national benchmarking) or global (international benchmarking) market through a comparative assessment of products, services, processes, methods, procedures and strategies (Fazlagić, 2006). Based on that comparison, the university obtains knowledge regarding the quality of the education offered by other universities and identifies the reasons due to which they obtain better results in order to learn from them and improve the quality of a specific action (Cobo, 2009).
Nowadays, benchmarking is being increasingly applied to the quality management of virtual higher education, as confirmed by different projects regarding the application of it to that area. Table 1 shows the projects considered of interest to this research.  Marshall (2009) These and other projects of unquestionable utility have made it possible to carry out comparative analysis of various universities based in different countries using a set of specific criteria. However, no methodologies have yet been designed, based on the results from those projects, that could be used as practical support by universities in order to introduce the method in a correct and satisfactory manner. Given the lack of a proper methodology, the purpose of this research is to draw up a benchmarking methodology, with all the necessary phases, in order to apply it to universities offering virtual education based in different countries so as to improve its quality.

Assessment of the quality of virtual higher education
The quality of virtual education is a complex matter, especially, given its multifactorial nature. Some authors link its quality to learning processes, products and services based on the use of information and communication technologies (ICTs) (Marúm-Espinosa, 2011). Others (Ardila-Rodríguez, 2011) refer to the quality of virtual education as user satisfaction, system excellence, positive academic results and good social impact. According to Silvio (2006), such quality should be understood as the union between academic excellence, equity and social relevance. In his opinion, bringing quality to virtual education means studying it from its singular and specific aspects, taking into consideration all variables, criteria and specific indicators arising from the characteristics of virtual education.
According to that author, the quality of virtual education cannot be compared or reduced to the criteria applied to traditional education, as it requires specific assessment models and methodologies adapted to the different contexts of virtual education. Rubio (2003) highlights two types of models that can be used to assess the quality of virtual education: 1) Partial focus based mainly on one of the following: • Assessment of the educational activity As can be observed, a university can assess the quality of its virtual education using different methods in order to reach the highest quality and the best results possible based on its competitiveness' conditions. One of the methods is benchmarking, the purpose of which is to discover the very best practices.
3. The proposed methodology for performing international benchmarking of the quality of virtual higher education Nowadays, there are several methodologies describing how benchmarking should be carried out. However, most of them have been designed for application to the industrial sector and, thus, it is difficult to apply them in order to assess the quality of virtual education, as they do not take into consideration its specific context. For this reason, the author proposes her own methodology to carry out such benchmarking, setting out all the necessary phases and subphases (see Figure 1).

Subphase 1: Definition of the purposes of the benchmarking
The start point of each benchmarking process is to define its purposes and main tasks to be fulfilled.

Subphase 2: Definition of the quality variables and indicators
In this subphase, all the variables and indicators to be used to measure quality will be defined.

Subphase 3: Selection of the data collection methods and sources
In this subphase, the type of data necessary to carry out the research will be defined. Given the relevance of data collection, various data sources will be defined.

Subphase 4: Design and verification of the tools to be used
The last step of phase I consists in designing data collection tools to be used during the benchmarking process.

Phase II. Search for the best practices (identification of the partner university)
Subphase 5: Development of the criteria to be applied to select the partner university The first subphase of phase II, consists in developing the criteria to be applied to select the universities to be considered for the final selection of the partner university (the university with the best practices).

Subphase 6: Identification of potential partner universities
This subphase requires exhaustive research using secondary data in order to identify the best university according to certain variables. Based on the results from that research, the universities with the best possible practices will be identified in order to select potential partner universities.

Subphase 7: Selection of the partner university
In this subphase, the university that best fulfills all the criteria set out in subphase 5 will be identified.

Subphase 8: Contacting the selected partner university
At this point, the first formal contact with the selected partner university will be made. For this purpose, it is necessary to contact the persons in charge of the that university, informing them of the purposes of the research, as well as of the reasons why the selected university is considered one of the best of its kind, asking them whether they are interested in participating in the research.

Phase III. Data collection and analysis
Subphase 9: Analysis of the university that requires improvements Based on the indicators drawn up in subphase 3, all the internal data necessary to assess the performance of the university will be collected, using the collection sources and tools created in that subphase.

Subphase 10: Analysis of the partner university
Based on the indicators drawn up in subphase 3, all the internal data necessary to assess the performance of the partner university will be collected, using the collection sources and tools created in that subphase.

Subphase 11: Comparison of the data obtained and identification of the gaps
In this subphase, the results from the application of the criteria to the analyzed university will be compared to those obtained from their application to the partner university. The differences between the results will be defined in order to identify the gaps between the two universities.

Phase IV. Action
Subphase 12: Drafting of an action plan for quality improvement In this phase, an improvement plan containing specific actions to be taken will be designed in order for the analyzed university to achieve the very best practices.

Subphase 13: Implementation of the action plan
This subphase consists in implementing the previously drawn up plan, and in following up and monitoring its implementation.

Empirical application of the proposed methodology. Research results
In order to verify the efficiency of the proposed methodology, the author applied it from April to August 2013 to the Polish Virtual University (PUW). 1 The benchmarking was performed according to the methodology described in Figure 1.
Phase I. Planning of the benchmarking project

Subphase 1: Defining of the purposes of the benchmarking
The main purpose of the benchmarking was to assess the quality of the education offered by PUW in order to compare it to a leading university and define the gaps between them so as to identify the steps that should be taken in order to improve the quality of the education offered by PUW.

Subphase 2: Definition of the quality variables and indicators
Based on the analysis described in Table 1, it can be concluded that the most common benchmarking variables applicable to virtual education can be divided into five main groups: • Strategic plan • Institutional context • Educational agents • Teaching-learning processes

• Virtual platforms
The aforementioned variables were applied to this research. Table 2 below defines each of the aforementioned variables, as well as the criteria and indicators to be used for each of them.
1. According to the specialist literature, benchmarking may take anywhere between a few days and several months. The period mainly depends on the data that one wants to collect and the methods used to compile them (Stapenhurst, 2009;Levy & Valcik, 2012

Subphase 3: Selection of the data collection methods and sources
The author applied documentary analysis methods and social research techniques in order to collect all the necessary data. For this purpose, semi structured face-to-face interviews and non-participant observation of the virtual classroom were carried out. Both primary and secondary sources were used for documentary analysis.
Regarding primary sources, the author used, among others, the documentation provided by the researched university for the variables described above, that is, strategic plans, curricula, course documents, student statistics, teaching guides and materials. Regarding secondary sources, the author used, among others, information materials provided by the Education Ministries of Poland and Spain, reports drafted by the Polish E-learning Association, the Polish Accrediting Commission, the Spanish National Agency for Quality Assessment and Accreditation (ANECA) and the Catalan University Quality Assurance Agency (AQU) A total of eight semi-structured interviews were carried out in order to understand, as much as possible, the issues related to the scope of this research. The interviews were carried out with directors of virtual programs (2), human resources managers (2), managers of virtual classrooms (2) and teachers-tutors (2).

Variables Conceptual definition Criteria Indicators
Strategic plan A document showing the current and future position of the university from a strategic point of view.

Vision
It clearly defines the desired future.
It is motivating for all university members.

Mission
It explains why the university exists.
It is achievable.

Values
It can guide the managers in their activities.
It encourages the team spirit of the university.

Strategic purposes
They are ambitious.
They a They are coherent.
They are achievable.

Institutional context
The university's set of resources and characteristics.

Infrastructure
The university has all the necessary facilities.
The university has one or more virtual libraries.
There are standards for the technological infrastructure configuration.

Human resources
Teaching staff (teacher/student ratio).

Educational agents
The persons involved in the teaching-learning process

Students
The students are informed of the admittance and graduation profile.
The students are offered additional services.
There are rules regarding the students' rights and duties.

Teachers-tutors
There are teacher/tutor selection procedures.
There are teacher/tutor assessment procedures.

Teachinglearning process
The cognitive activity carried out by students guided by teacherstutors in order to gain knowledge and skills.
Studies program The program recipients are well defined.
The program is up to date.

Teaching materials and resources
They are varied.
There are resources for students with special needs.
There are procedures for the validation of teaching materials.

Methodology
The teaching strategies applied by teachers-tutors are varied.
The learning process is sufficiently encouraged.
Feedback is given to each student.

Tutoring
There are instructions regarding communication paths with tutors.
The tutors' tasks and functions are well defined.
The tutoring is monitored.

Learning activities
The activities are varied.
A schedule has been defined for each activity.
The way each activity should be delivered is well defined.

Learning assessment
An assessment methodology has been well defined.
The students are involved in the assessment process.

Virtual platform
The tool used to support the virtual education process

Course management by users
There are activity management tools.
There are tools to encourage team work.
There are tools to create a personal learning environment (PLE).

Communication management
Asynchronous communication.

Assessment and follow-up
There are assessment exercises.
There are self-assessment exercises.
Standardization The platform meets the SCORM standard requirements.
The platform meets the A/AA accessibility requirements.

Subphase 4: Design and verification of the tools to be used
In order to collect all the necessary data to carry out the benchmarking, the author drew up five assessment protocols to be used for the comparative assessment.
These were: • A comparative assessment protocol of the strategic plans • A comparative assessment protocol of the institutional context • A comparative assessment protocol of the teaching-learning process • A comparative assessment protocol of the educational agents • A comparative assessment protocol of the virtual platform The results from the application of the aforementioned protocols have been arranged according to the Likert scale, ranging from 1 (non-fulfillment of the indicator) to 4 (high fulfillment of the indicator). Furthermore, comments were made regarding each of the applied indicators (qualitative assessment), as the author considered that subjective values should be included in the assessment. For this reason, the objective data obtained from the application of the protocols was completed with the author's comments on the results obtained (see Table 3).
The drafted protocols were assessed by four experts in virtual higher education.
Phase II. Search for the best practices (identification of the partner university) The author chose a Spanish university as partner university, given that Spanish universities offer highly A description of the selection process for the partner university is given below: -During subphase 5, the author created six selection criteria: online mode, access to ICTs, course variety, other services offered, quality and prestige.
-During subphase 6, a total of 41 Spanish universities offering virtual education were analyzed. As a result, four preselected universities were thoroughly assessed, applying the above-mentioned criteria.
-During subphase 7, UOC was selected as the partner university, as it satisfactorily fulfilled all the applied criteria.
-Subphase 8 consisted of sending a letter to the partner university, requesting its authorization to carry out the benchmarking and explaining the functioning of the entire process.

Subphases 9 and 10: Internal analysis of PUW and UOC
The purpose of these subphases is to collect data about the partner university and the quality of its virtual education. For this reason, prior to collecting that data, the quality of the virtual education offered by PUW was assessed in order to increase the possibility of detecting all areas to be improved. The assessment was carried out using comparative assessment protocols (see Table 3 for an example of those protocols) and the data collection sources described in subphase 3.  Comment: Depends on the subject, but the nature of most of the activities is the same, that is, forum participation.
Comment: Learning activities are divided in two groups: 1) Application activities (individual and non-immersive activities).
2. There is an activities schedule 1 2 3 4 1 2 3 4 Comment: There is no activities schedule. Comment: The guide includes a "Learning planning" chapter that contains a navigation map for each course describing all the activities to be performed and their schedule.
3. The way the activities should be delivered has been described 1 2 3 4 1 2 3 4 Comment: The students have to post the completed activities in the virtual classroom's delivery box.
Comment: The students have to post the completed activities in the virtual classroom's delivery box.

Subphase 11: Comparison of the data obtained and identification of the gaps
Once the quality of the virtual education offered by PUW had been assessed, its situation as compared to the partner university was defined in order to identify possible gaps (see Table 4) between the two universities in five main areas, as well as to determine their causes.
A brief summary of those gaps is given below.
The first variable used by the author was the strategic plan of each university. One of the biggest weaknesses of PUW in this area is that its vision and the values that it should abide by for its future development have not been developed. Furthermore, the strategic mission and objectives of PUW are significantly less well defined than those of UOC.
Another relevant variable that creates a positive and a negative gap in comparison to the partner university is the institutional context. The positive gap is that the teaching load of PUW (student/teacher ratio) is lower than that of the partner university. The negative gap is that PUW lacks not only a virtual library but also strategies for the availability of technological services and human resources management.
Regarding educational agents, there is a clear negative gap between the two universities, as PUW does not provide its students with any additional services, that is, it does not assist them with their access to the job market and it does not encourage them to participate in educational events or scientific research/programs. What is more, not even PUW's teachers participate in such events. Moreover, PUW has not defined any teacher/tutor selection procedure.
Regarding the teaching-learning process variable, there is a very significant negative gap between the two universities. In this case, PUW has not set out any procedures for the updating or drafting of studies programs.
Furthermore, it lacks educational material, learning strategies and technologies, and the activities it offers are not varied. Finally, it should be noted that the qualification criteria and assessment systems applied by PUW teachers are not detailed enough. Phase IV. Action

Subphase 12: Drafting of an action plan for quality improvement
In order to remove or reduce the gaps identified in phase III, an improvement action plan was designed, consisting of specific actions to be taken so that PUW could not only remove the said gaps, but also achieve the very best practices. Table 5 shows an example of the plan drafted for PUW.

Activities timetable
Subphase 13: Implementation of the action plan In this subphase, the drawn-up action plan had to be implemented and monitored. This subphase is extremely important, as any failure could lead to the strategic objectives not being reached and, thus, the quality of the virtual education offered by PUW not being improved. In order to avoid that risk, a set of the most relevant indicators was drawn up in order to ensure that each step of this subphase would be carried out.

Conclusions
The results obtained from the application of the proposed benchmarking methodology enable the following conclusions to be drawn:

Regarding the methodology
The drawn-up methodology sets out all the relevant phases in order to carry out international benchmarking of the quality of virtual higher education. It is not a static methodology, as it allows a dynamic application based on modifications and the evolution of quality assessment models through the adaptation, elimination and implementation of new indicators and variables. Regarding the application of the methodology By applying the methodology, the author has been able to verify its complete usefulness and great potential when it comes to improving the quality of virtual education by discovering the very best practices and collecting all the necessary data in order to assess, in a comparative manner, the quality of the virtual education offered by different universities and, as a result, to draw up an improvement action plan for such education.
Regarding the utility of the methodology The methodology was created to perform international benchmarking of the quality of the virtual education offered by Polish universities. The application of a comparative analysis (with a leading international university) made it possible to establish a thorough diagnosis of the actual quality of the virtual education offered by the selected Polish university, as well as to discover the very best practices in order to increase the quality of it and to initiate a learning process based on the best virtual education universities at an international level. The creation of a clear and objective methodology allowed the author to use the results obtained to draft an action plan for the continuous improvement of the quality of the virtual education offered by the chosen Polish university. It is believed that the use of the methodology described above could extend beyond the purposes initially set and become a tool that brings added value and knowledge to all institutions offering virtual education (both in Poland and abroad).