Methodological aspects of internal assessment of knowledge in primary school

Preverjanje in ocenjevanje znanja sta pomembni stopnji učnega procesa, saj z njima ugotavljamo doseganje učnih ciljev in standardov znanja pri učencih. S tem imata ključno vlogo pri zagotavljanju kakovosti vzgojno-izobraževalnega sistema. Učiteljevo preverjanje znanja, ki je namenjeno ocenjevanju, ima v slovenskem šolskem sistemu izrazito selekcijsko funkcijo pri napredovanju posameznika v osnovni šoli in pri prehodu na višjo stopnjo izobraževanja, hkrati pa vpliva na oblikovanje njegovih učnih in poklicnih aspiracij. Z odločujočo vlogo, ki jo ima notranje ocenjevanje pri izobraževalni prihodnosti osnovnošolcev, je predenj postavljena zahteva po doseganju njegove visoke veljavnosti, objektivnosti, zanesljivosti in občutljivosti. V prvem delu empirične raziskave smo metodološke vidike notranjega preverjanja in ocenjevanja proučili z ugotavljanjem mnenj in ravnanj osnovnošolskih učiteljev, povezanih z notranjim preverjanjem in ocenjevanjem ter zunanjim oz. nacionalnim preverjanjem znanja, v drugem delu raziskave pa z analizo njihovega ocenjevanja pisnih preizkusov znanja pri slovenščini in matematiki v četrtem razredu. V prvem delu raziskave, v katerem smo podatke zbirali z anketo, je sodelovalo 882 učiteljev iz osnovnih šol vseh regij v Sloveniji. Vzorec sestavlja 381 učiteljev razrednega pouka, ki so poučevali različne razrede, in 501 učitelj predmetnega pouka, ki so poučevali različne šolske predmete. Podatki drugega dela raziskave temeljijo na vzorcu 24 učiteljev razrednega pouka, ki so v dveh obdobjih dvakrat ocenili iste pisne preizkuse znanja petih učencev pri slovenščini oziroma matematiki. Pisna preizkusa znanja je za namen ocenjevanja znanja svojih učencev pripravila učiteljica četrtega razreda na eni izmed slovenskih osnovnih šol, učitelji ocenjevalci pa so kriterije ocenjevanja postavili individualno. Sodelujoči so pisna preizkusa znanja tudi ovrednotili z vidika zahtevnosti in tipov nalog, jasnosti, natančnosti in nedvoumnosti navodil ter ustreznosti števila možnih točk pri posamezni nalogi ter napisali pisno refleksijo o lastni ocenjevalni praksi. Rezultati raziskave so pokazali, da osnovnošolski učitelji višje vrednotijo številčno kot opisno ocenjevanje tako z vidika informativne in motivacijske vrednosti opisnih in številčnih ocen kot merskih karakteristik ocenjevanja. Podobno višjo veljavnost, zanesljivost in objektivnost pripisujejo notranjemu ocenjevanju v primerjavi z zunanjim. Raziskava je pokazala, da učitelji razrednega pouka in učitelji predmetnega pouka poleg znanja večinoma vključujejo v oceno tudi več drugih komponent, kot so kakovost ustnega izražanja in lepopisje, trud oziroma marljivost učenca ter aktivno sodelovanje pri obravnavi nove učne vsebine. Nezanemarljiv delež učiteljev v oceno vključuje še druge komponente (delovne navade, odnos do učne vsebine idr.), če pa bi imeli možnost odločanja, bi različne komponente poleg znanja ocenjevalo še več učiteljev, kot jih to že počne. Ocenjevanje komponent, ki jih ni mogoče objektivno oceniti, znižuje ne le objektivnost, ampak tudi veljavnost in zanesljivost ocenjevanja, zato med predlogi sprememb predlagamo tudi premislek o posameznih standardih znanja v učnih načrtih, katerih doseganja ni moč ocenjevati objektivno, posledično pa tudi ne veljavno in zanesljivo. Učitelji so izrazili, da za ocenjevanje znanja učencev uporabljajo različne načine, uporabo alternativnih načinov preverjanja in ocenjevanja podpirajo kot dopolnitev tradicionalnim načinom preverjanja in ocenjevanja, čeprav na splošno niso niti prepričani glede merskih karakteristik takšnega ocenjevanja niti se večinoma ne strinjajo, da bi lahko nadomestili tradicionalne načine. Večina učiteljev je izpostavila, da kriterije ocenjevanja učencem predstavijo v različnih okoliščinah, na splošno pa ocenjujejo, da dobro poznajo kriterije ocenjevanja drugih učiteljev na šoli in da so ti med učitelji usklajeni. Večina je tudi izjavila, da pri pripravi pisnih preizkusov znanja pogosto sodelujejo z drugimi učitelji na šoli. Kljub statistično pomembnim razlikam, ki so se v izraženih mnenjih in praksi pokazale med učitelji razrednega in učitelji predmetnega pouka, glede na razred in predmet poučevanja ter njihovo delovno dobo, odgovori učiteljev kažejo, da si učitelji na splošno prizadevajo zagotavljati merske karakteristike notranjega ocenjevanja in močno verjamejo, da jim to tudi uspeva. V drugem delu raziskave je analiza občutljivosti ocenjevanja pokazala večjo občutljivost ponovljenega ocenjevanja, občutljiveje so na splošno ocenjevale strožje ocenjevalke. Povprečna ocena posameznega preizkusa znanja se je na podlagi koeficienta intraklasne korelacije izkazala za zelo objektivno. Pri matematiki je bila ta ocena še bolj objektivna kot pri slovenščinito je znano že iz tujih raziskav objektivnosti ocenjevanja znanja jezika in matematike. Pri drugem ocenjevanju se je objektivnost ocen pri obeh predmetih še povečala. Med predmetoma pa so se pokazale tudi razlike glede objektivnosti ocenjevanja posamezne ocenjevalke. Pri slovenščini se je pri prvem ocenjevanju pokazala slaba objektivnost ocenjevanja tako z vidika absolutnega ujemanja števila točk oziroma številčnih ocen kot z vidika konsistentnosti. Pri drugem ocenjevanju se je konsistentnost določanja števila točk izboljšala in jo lahko obravnavamo kot zadostno, kar pa ne velja za številčno ocenjevanje. Določanje številčnih ocen je bilo tudi pri drugem ocenjevanju nekonsistentno in posledično tudi z vidika absolutnega ujemanja ocene posamezne učiteljice s povprečno oceno celotne skupine ocenjevalk za posamezni preizkus znanja neobjektivno. Ocenjevanje posamezne ocenjevalke je bilo pri matematiki pričakovano objektivnejše kot pri slovenščini. Medtem ko je bilo določanje števila točk posamezne ocenjevalke konsistentno tako pri prvem kot pri drugem ocenjevanju, se je pri drugem ocenjevanju pokazalo tudi visoko absolutno ujemanje števila dodeljenih točk s povprečnim številom točk celotne skupine ocenjevalk za posamezni preizkus znanja. Kot pri slovenščini pa je bilo tudi pri matematiki določanje številčne ocene obakrat nekonsistentno in z vidika absolutnega ujemanja ocene posamezne učiteljice s povprečno oceno celotne skupine ocenjevalk neobjektivno. Zanesljivost ocenjevanja je bila višja za preizkuse znanja iz matematike in nižja za tiste iz slovenščine: pri ocenjevanju znanja iz matematike je bila nezanesljiva le ena od trinajstih ocenjevalk, medtem ko je bilo takšnih pri slovenščini sedem od enajstih. Kljub razlikam v ocenjevanju učiteljic lahko na podlagi podobnih rezultatov drugih raziskav, v katerih so ocenjevalci uporabljali skupne ocenjevalne kriterije, zaključimo, da so ocenjevalke v naši raziskavi na splošno zanesljivo ocenile preizkuse znanja iz matematike, medtem ko je bila zanesljivost ocenjevanja pri slovenščini nižja, a primerljiva z izsledki preteklih raziskav. Pričujoča raziskava je prva v Sloveniji, ki celovito obravnava notranje ocenjevanje znanja v osnovni šoli, tako z vidika pogledov učiteljev na lastno ocenjevanje, kot tudi v odnosu do nacionalnega preverjanja znanja, z vidika načinov izvedbe ocenjevanja ter okoliščin, v katerih poteka, kot z vidika analize značilnosti učiteljevega ocenjevanja pisnih preizkusov znanja. Rezultati raziskave pritrjujejo nekaterim izsledkom predhodnih raziskav o notranjem ocenjevanju znanja ter opozarjajo na vidike notranjega ocenjevanja znanja, ki jih velja premisliti in spremeniti, da bi se izboljšale njegove merske karakteristike. Na podlagi ugotovitev raziskave tako podajamo predloge za nadaljnje raziskovanje ter smernice za izboljšanje merskih karakteristik notranjega ocenjevanja znanja na ravni vzgojno-izobraževalnega sistema, šole in posameznega učitelja.Knowledge assessment and grading represent important instructional stages aimed at determining the achievement of learning objectives and learning standards among pupils. As such, they play a significant role in ensuring the quality of the educational system. In Slovenia, internal grading has a significant selective function in one’s progression in primary school as well as one’s progression towards a higher level of education. At the same time, it influences the formation of their educational and career aspirations. Due to its deciding role in the future of pupil\u27s education, it is crucial for internal grading to be valid, reliable, objective and sensitive. In the first part of the empirical research, we examined methodological aspects of internal assessment and grading by analysing opinions and practices of primary school teachers in terms of internal and external assessment and grading. In the second part of the research, we analysed teachers\u27 grading of written exams in Slovene and Mathematics among fourth-grade primary school pupils. In the first part, data were collected through a survey, using a questionnaire. The sample consisted of 882 primary school teachers from different Slovenian regions, 381 of whom were class teachers and 501 of whom were subject teachers. The data from the second part of the research were collected on a sample of 24 class teachers who each graded written exams either in Slovene or Mathematics from five fourth-graders on two different occasions. Written exams were prepared by a class teacher and were intended for internal grading of the pupils in her class. In the grading process, each rater had to set her own assessment criteria. All the participants also evaluated the written exams in terms of difficulty and types of items, clarity, precision and unambiguity of the instructions, as well as the maximum score assigned to each item. Additionally, they each wrote a reflective essay on their assessment and grading practices. The results showed that primary school teachers rate numerical grades higher than descriptive grades when it comes to their informative and motivational value as well as their measurement characteristics. The same applies for internal grading in relation to external grading where they attribute higher validity, reliability and objectivity to the first. The research showed that class teachers as well as subject teachers take into account other components besides knowledge in the grading process, such as pupils’ verbal and writing skills, their effort, and active participation in the classroom. A considerable share of teachers also includes various other components into the grades (pupils’ working habits, their attitudes towards learning content, etc.), and an even larger share would do so if given the autonomy to decide by themselves. Grading of different components that cannot be graded objectively per se lowers validity and reliability of grading. Based on that, we propose a reflection on particular learning standards in the curricula, the achievement of which cannot be graded objectively, consequently producing grades with low validity and reliability. The teachers stated that they used different assessment strategiesthey support the use of alternative assessment as a supplement to the traditional assessment strategies, but are unsure of the measurement characteristics of alternative assessment strategies and do not agree with them replacing the traditional strategies. The majority of the teachers asserted that they present assessment criteria to their pupils on different occasions. They generally know the criteria used by the other teachers from the same school and believe these criteria agree with their own. The majority of them also stated that they frequently collaborate with the other teachers at their school in preparation of written exams. Despite the significant differences in the opinions and practices between the class teachers and the subject teachers, between teachers of different years and of different subjects, and between those with different work experience, their answers speak of their effort to ensure good measurement characteristics of internal grading, as well as of their high trust of actually achieving that. The analysis of the sensitivity of grading from the second part of the research in general suggests a higher sensitivity of the second grading. The sterner raters proved to perform more sensitive grading compared to the more lenient ones. According to the Intraclass Correlation Coefficient, the average score of an individual exam proved to be highly objective (with a high interrater reliability) for the Slovene exam and even more so for the Mathematics exam. These findings (higher objectivity for Mathematics compared to languages) are confirmed by similar findings from the literature. The second grading proved to be more objective than the first for both subjects. Objectivity of a single rater was found to be different for the Slovene and Mathematics exams for both the first and the second grading. For the Slovene exam, objectivity of the first grading of a single rater was poor in terms of both absolute agreement and consistency. The second grading resulted in sufficient consistency of scores, but not grades. Both the first and the second grading were inconsistent and resulted in biased grades in terms of absolute agreement of an individual rater’s numerical grade compared to the average grade from all of the raters for an individual exam. As expected, the grading of an individual rater was found to be more objective in the case of Mathematics than Slovene. While the scoring of an individual rater proved to be consistent on both occasions, the second grading also resulted in a high absolute agreement of the assigned scores with the average scores from all of the raters for an individual exam. Similar to Slovene, assignment of the numerical grades in Mathematics was inconsistent and poor in terms of absolute agreement of the individual rater’s grade with the average grade from all of the raters on both occasions. Rater reliability was also found to be higher for the Mathematics exams than Slovene exams: in Mathematics, only one out of thirteen raters was unreliable, whereas in Slovene, there were seven unreliable raters in the group of eleven. Despite differences in their grading comparison to the results from existing studies in which raters used uniform assessment criteria leads to a conclusion that for Mathematics, rater reliability of the teachers in our research was good, whereas rater reliability for Slovene was lower, but still comparable to the results of the previous studies. The present research is the first research in Slovenia which deals with internal grading in primary school in a comprehensive manner: it tackles teachers’ views on their own assessment and grading, and their opinions about the national assessmentit deals with assessment strategies and circumstances surrounding the assessment and grading processes, as well as the analysis of classroom grading of written exams. The present results confirm some of the previous findings on internal grading and point out the aspects of internal grading that need to be reconsidered and reconceptualised to improve its measurement characteristics. Based on our findings, we propose extensions of our research methodology and procedures, and present guidelines for the improvement of measurement characteristics of internal grading on the level of educational system, individual school and individual teacher