Home » Evaluation

Evaluation

 

Do you have a best practice? Please upload it.  Complete the registration and let us know more of your project.

Registration and database access: http://esolve.gtk.uni-pannon.hu/db

Current good practices: http://esolve.gtk.uni-pannon.hu/good-practices/

Evaluation system

The evaluation system used by the E-SOLVE project is based on the assessment system called ECBCheck. “ECBCheck is a quality improvement scheme for E-Learning programmes. It supports organisations to measure how successful their e-learning programmes are and allows for continuous improvement through peer collaboration and bench learning. ECBCheck forms a participative quality environment which allows its members to benefit in a variety of ways by having access to tools and guidelines for their own practice on the one hand, and being able to obtain a community based label on the other hand. – See more at: http://www.ecb-check.org/#sthash.0Z9o4jD2.dpuf

The ECBcheck criteria system is very elaborated. It bases on experience of several acknowledged experts, education practitioners. The results are accepted and used by many educational institutes.
However, the partners of the E-SOLVE project found that at some points minor additions may improve the criteria system, particularly with respect of new pedagogical aspects as collaborative learning and support, the learners can expect from the providers and the system. The differences between the two evaluation systems are the following:

– B.7 Learning process is based on problem solving activities and practice based assignments to enable authentic learning experience.
– D.3.5 Progress tracking (teacher, learner)
– D. 3.6 List of used resources.
– D. 3.7 Learning materials meet copyright regulations.
– E.5 Content layout is attractive and motivating.
– H. Support and easy to use.
– H.1. Is there 24/7 support?
– H.2 Is login streamlined?
– H.3 User support team.
– H.4 The virtual learning environment runs on an adequate server, which guarantees its stability.
– H.5 The virtual learning environment is accessible through different browsers and operating systems.

The evaluation and nomination of best practice examples have been done in a two stage procedure. At first EDEN from the A.D.A.M. database, carried out a preselection procedure focusing on a small subset of criteria. Our primary selection criteria was excellence in conjunction with online accountability for the evaluation process. As a result we gained a shortlist of solutions consisting 112 candidates to be nominated as good practices. During the second stage the full criteria system was used to evaluate the solutions and rank them accordingly. During our evaluation we followed the ECBCheck guidelines as close as possible. It is clear that contrary to the self-evaluation, our evaluation was limited to the documentation and online material open for all or registered users.

Evaluation Criterions

A. Information About and Organization of the Programme

A.1 General Description, Objectives and Programme Organization
A.1.1 All information potential learners need to orient their decision about taking the course is available.
A.1.2 Learning objectives state clearly the expected improved competency. All learning objectives are measurable (e.g. stated according to the SMART criteria).
A.1.3 The methodological approach is clearly explained. It presents how the methods/approaches selected lead to the achievement of the learning objectives. Learners are able to understand the added value of technology enhanced learning.
A.1.4 Information on contact persons is available, mentioning name, area of responsibility, and contact details.
A.2 Organizational and Technical Requirements
A.2.1 People responsible for the analysis, design, development, implementation, and evaluation of the programme (internal staff/external experts) are adequately qualified.
A.2.2 Technical requirements necessary to allow for adequate participation in the programme/course are clearly described.

B. Target Group Orientation

B.1 Target group’s learning needs are taken into account and considered in the design of the programme/course (curriculum, methodology).
B.2 The programme workload and schedule are consistent with the target groups requirements.
B.3 Key stakeholders are involved in the programme design through participatory process.
B.4 Learners have access to counseling services and advice both prior to the start of the programme and during its implementation.
B.5 The programme foresees processes to bridge learning deficits of low achievers
B.6 A mechanism or system for complaints and appeals is available to learners.
B.7 Learning process is based on problem solving activities and practice based assignments to enable authentic learning experience.

C. Quality of the Content

C.1 The content of the course/ programme is coherently presented and subdivided in logical sequences of modules and/or lessons/sections, organized in such a way that enables comprehension and retention.
C.2 The contents are provided in a flexible manner, allowing different learning paths.
C.3 Content is gender sensitive. It takes into account cultural diversity.
C.4 Media rich content is utilized exclusively with a fixed and definite purpose.

D. Programme/ Course Design

D.1 Learning Design and Methodology
D.1.1 The blending of learning methods (online, face-to-face, self-learning, tutor-facilitated, asynchronous, synchronous) is adequate and meets the needs of the learners.
D.1.2 A learner centered learning design facilitates the development of the desired skills and expected competencies described in the learning objectives.
D.1.3 Social and collaborative activities are included in the programme methodologies and contribute to the achievement of the learning objectives.
D.1.4 The learning methodologies offer opportunities for learners to determine their learning pace.
D.1.5 Through the learning activities learners are encouraged to consider and use systems thinking and to view issues from different perspectives.
D.1.6 The content of the programme is organized in a logical sequence from the simpler to the most complex concepts. The modules/lessons/units build progressively on each other.
D.2 Motivation/ Participation
D.2.1 The learning methodologies motivate learners to actively participate in the learning process.
D.3 Learning Materials
D.3.1 The programme offers a learning experience which is relevant to professional practice.
D.3.2 Each learning unit is presented with the elements needed to guide learners in achieving the learning objectives.
D.3.3 Modules/lessons/units provide a glossary of terms associated to the learning materials.
D.3.4 A distinction is made between compulsory and recommended study/reading materials. The bibliography is commented.
D.3.5 Progress tracking (teacher, learner).
D. 3.6 List of used resources.
D. 3.7 Learning materials meet copyright regulations.
D.4 eTutoring
D.4.1 A tutor provides guidance and accompanies the learners throughout the learning process.
D.4.2 A tutor provides feedback to learners on tasks/activities learners are required to perform.
D.4.3 Tutors have specific knowledge and competences to facilitate online courses/programmes.
D.4.4 A set of predefined tutoring skills are being used as a standard within the program.
D.5 Collaborative Learning
D.5.1 Collaborative learning is explicitly supported.
D.5.2 Search functions are available for forums, discussion boards or blogs, whenever such tools are utilized.
D.6 Assignments & Learning Progress
D.6.1 Assignments are clearly formulated and adequately explained to learners. Learners have a clear understanding of what they are expected to perform and how their performance will be measured.
D.6.2 Assignments and knowledge assessment tests are aligned with the learning objectives, enabling adequate measurement of their achievement.
D.6.3 Learners progress and achievements are monitored and evaluated.
D.6.4 Assignments and/or knowledge assessment tests and tasks are designed using different approaches, including self-assessment and peer-review.
D.6.5 Individual feedback is provided to learners whenever a task requires problem solving skills. The feedback has an analytical approach and considers the way the solution was provided.
D.7 Assessment & Tests
D.7.1 A specific timeframe is foreseen for tutor’s to provide learners with feedback on assignments and knowledge assessments tests. Learners are informed of the timeframe.

E. Media Design

E.1 Accessibility standards have been applied.
E.2 Usability standards are met.
E.3 The navigation (through the mandatory learning materials) allows learners to know about their progress and position in relation to the overall content.
E.4 Screens, table of content, and learning materials, including additional resources are printable.
E.5 Content layout is attractive and motivating

F. Technology

F.1 The downloadable learning materials have common formats and acceptable size.
F.2 The technology adequately supports the learning strategies utilized. In particular, the technology is in accordance with the IT infrastructure available to the target group.

G. Evaluation & Review

G.1 A comprehensive evaluation process is foreseen at the end of the course/programme to evaluate its quality and overall coherence, and contribute to its further improvement for subsequent deliveries.
G.2 Learners’ feedback on the programme/course delivery are collected through a questionnaire or other means.
G.3 A report, based on the feedback collected from learners, is prepared and includes clear recommendations for further improvement of the programme for subsequent deliveries.
G.4 A process for integrating the recommendations for improvement is foreseen and is part of programme/course design activities, in order to ensure continuous programme/course improvement.

H. Support and easy to use

H.1. Is there 24/7 support?
H.2 Is login streamlined?
H.3 User support team.
H.4 The virtual learning environment runs on an adequate server, which guarantees its stability.
H.5 The virtual learning environment is accessible through different browsers and operating systems.

 

download english version:

download hungarian version: