Page 3 of 5
2. 2. Insufficient exploration of examination information: The statistical analysis of OUC digital examination information is made up of two parts, course results and examination papers. The contents can be seen in Table 1. Since they are limited by the design and plan of the existing examination system structure and functionality, examination progress and results are difficult to analyze and apply. A mere sample analysis does not make the best use of statistics on examination results to aid in teaching and learning and examination design, causing the missing of the examination progress information and incomplete overall information of the examinations.
Table 1 Statistics and analysis of current OUC digital examination information
|
Statistical items on OUC Course Results
|
OUC examination analysis system
|
|
Basic statistical indicators include: number of applicants, enrolment, number of attendees, number of passes, number of passes per person, number of violations, number of violations per person, average score and pass rate of formative assessment, average score of summative examination, pass rate, variance, standard deviation
|
Collection methods for examination paper information: 40 to 60 courses are randomly selected each semester. The person in charge of each provincial university will fill in the score registration card. The score-reading card will be used to collect the original information from the examination papers.
|
|
Overall statistical table of the OUC course assessment scores
|
Statistical indicators: average score, pass rate, standard deviation, difficulty, and reliability
|
|
Statistical table of assessment scores for each course
|
Content analysis: top and bottom 10 units and courses in terms of average scores and pass rate. Course examination paper analysis covers: mapping the distribution of scores, the quality dynamic curve of test item layouts, structural analysis of test difficulty, analysis of test reliability, statistical analysis reports, the teacher in charge can make modification and improvement combining qualitative analysis
|
|
Statistical table of course assessment scores for each provincial university
|
2.3. Lack of close integration of modern information technology: The development of information technology creates both challenges and opportunities for the development of higher education. In The NMC Horizon Report: 2012 Higher Education Edition, New Media Consortium (NMC) identified six emerging technologies and six key trends in higher education. Over the coming five years, a new model for online teaching and learning based on cloud computing will come into being. People expect to be able to work, learn and study whenever and wherever they want to. Mobile apps, tablet computers, game-based study, learning analytics, and new technologies based on action-based computing and the Internet of things will be increasingly applied in the higher education field. However, the OUC examinations have yet to develop in this direction and have a long way to go before they fully integrate modern information technology.
2.4. Lack of scientific examination design: Examinations are used to screen, test and assess learners’ physical and mental development. The active objects of examinations are human beings, who possess objective initiative and creativity. Examination content should be based on the pre-existing mental and physical development qualities of the examinees as well as the physical and mental development qualities required by the examination organizers. Otherwise, the examinations will evaluate non-existing or unnecessary aspects. At present, there is a gap between teaching and examinations in the OUC examination system. Traditional examination methods are used, where 60% to 80% of test items should come from teaching content. This includes both closed and open book examinations, of which closed book examinations occupy a higher proportion. The question types used by each course are often relatively fixed, such as fill in the blank, multiple choice, short questions, and analysis questions. The test papers often lack of direction and encouragement. Most of the questions are memory-based, for example, giving a brief description of a working principle or calculating a formula. The OUC students are mostly adult learners who are looking to improve their knowledge. This group is characterized by extensive life experience, a degree of past work achievement, relatively strong knowledge acquisition and application skills, and weaker memory. As a result, the current examination system does not meet their needs as learners and does not encourage them to grow and develop.
2.5. Inability to meet individual examination requirements: the OUC national unified examinations are held over a four day period once a semester, usually on the Friday, Saturday, Sunday and Monday of the second to last week of the semester. There are examinations for approximately 900 different subjects. This creates a complicated examination plan and makes it difficult to meet the actual examination requirements of different teaching locations. For example, due to regional time difference reasons, when an examination is held in Beijing at 8:30 am, it is still before dawn in Xinjiang. In addition, adult learners face conflicts between work and study. There is a large number of students who have applied to take an examination, but are unable to attend. According to incomplete statistics, over the past three years, there have been six million applicants for “undergraduate and junior college subjects of Open Education”. However, due to work-study conflicts, approximately 500,000 of them could not attend the national unified examinations, accounting for 8% of the total. We can see that there is still a gap between the national unified examinations and the actual needs of learners and teaching locations.
III. Examination principles
The concept of modern examinations places an emphasis on “people first”, and indicates that examination is a special activity of human society. Examinations comprise examination knowledge, examination practice and examination evaluation with their specific nature and rules. There are five basic principles of examination activities, which are people first, differentiation, systems, dynamics and control (Liao Pingsheng, 2003). The principle of “people first” is a reflection of the nature of examinations, which requires that their design and implementation should both meet the inherent needs of human development in an all-around way and strive to realize human value. The principle of differentiation refers to the objective difference and scalability of an individual’s physical and mental qualities. Examinations are a social system that has a specific purpose, structures and function. Examinees, examiners and examination methods are the three sub-systems as part of examinations. At the same time, examination activity process indulging test design, test formulation, test implement, score evaluation, quality analysis, and examination administration including examination policies and regulations, examination institutions, examination teams, examination finance, examination facilities, examination plan, examination design, examination implementation, examination information, examination research, examination cultural exchange all have its own system, which exist in the examination system as sub systems. The dynamic examination principle reflects the internal and external relationship of examination activities, and allows for a high level view of the diverse patterns that occur within the examination process. The control principle reflects the characteristics and operational principle of examinations. Order and controllability are the basic features of examinations. Strict control is necessary to achieve the purpose of the examination.
IV. Research conclusions
1. The design principles of digital examinations
Based on the theory of examination and current situation and problems facing the OUC, this research proposes that the OUC’s digital examinations adhere to six design principles, namely intercommunication between databases; the importance of scientific examinations; integration of information technology; integrated design of teaching, learning and assessment; implementation of a range of assessment methods; and development of different combinations of modules.
1.1 Intercommunication between databases system: The most conspicuous advantage of digital examinations is the high efficiency of data processing, which is the fundamental reason for human beings being replaced by machinery. No matter how many databases have been built for the digital examination system, all the information should be standardized, linked and shared, ensuring higher efficiency and lower costs.
1.2 The importance of scientific examinations: No matter how advanced the technology, digital examinations are still inherently a form of examination. Digital examination designs should strictly adhere to the rules of examinations, focus on the scientific nature of examination design and achieve digital examinations in the true sense of the term, in order to ensure that there is a balance between computer-generated and human-generated examinations. In this way, even while taking digital examinations, students can still experience the care, encouragement and inspiration they receive from their teachers.
1.3 Integration of information technology: Digital examinations are a key component of digital teaching. Digital examinations are closely related to information technology. Digital examinations should be adapted to meet society’s need for learner formation. The digital examinations should timely absorb new information technologies and employ familiar, commonly used information technologies in order to facilite the learning and development for learner from all walks of life.
1.4 Integration of teaching, learning and assessment: Examinations are an important part of teaching and education. During the digital examination design process, the cultivation of positive study behavior and motivation to study should be included in the assessment, since independent learning ability is an important factor in evaluating distance education objectives, and is also a key feature of the OUC. Progressive assessments should be integrated into the teaching process so that students can form good study and learning habits. Instead of one-off examinations, summative assessment takes the form of multiple examinations, for which students can preregister according to their objective learning needs.
1.5 Implementation of a range of evaluation methods: Due to the different characteristics of distance education and distance learners, assessment should not adopt a single method. A range of assessment methods should be implemented, in order to meet the needs of adult learners with different levels of social experience. Examinations should be adapted to meet a range of factors such as knowledge retention, study habits, study attitude, study plan, learning outcomes and and work results.
1.6 Development of different combinations of modules: Digital examinations can be divided into different modules according to the needs of teaching and individual students. Courses choose modules according to their specific characteristics. Basic, core and professional courses can be designed according to the different objectives of each major, and different modules can be applied.