In the period 29.01.2020 – 29.09.2020 the Media Literacy Coalition is implementing a project for the creation and pilot testing of a unified national index of media literacy skills in high school and a methodology for their measurement. The High School Media Literacy Skills Index builds on the European Digital Competence Framework (DigComp 2.1), but identifies the range of knowledge required specifically for high school. Following two test surveys, the results of which were processed through psychometric analysis, and the subsequent revision and validation of the methodology within the framework of the project “Integral Approach for the Development of Students’ Media Literacy”, it has been made available to the Ministry of Education and Science as a tool to further align the European digital competence framework in schools and to regularly measure the levels of related cognitive skills among Bulgarian students.

In January 2022, tenth graders from 176 classes participated in the first national survey in Bulgaria to assess the digital-media competences of high school students. It was carried out in more than 150 schools across the country, randomly selected according to the updated methodology, and supported by the Ministry of Education and Science. Students complete an online test within 60 minutes, answering two types of questions – (i) multiple-choice questions and (ii) questions that require them to formulate their own reasoning. As can be seen from the table below, students completed two versions of the test:

Number of students who took the test using the methodology in January 2022.

The analytical report, which will be published in full on 21 June, presents the results
of this study, processed by experts of the Centre for Assessment in Pre-School and School Education (CAPSE) – a partner of the Media Literacy Coalition in the implementation of the project “An Integral Approach to the Development of Students’ Media Literacy”. The goal of the project is to provide a comprehensive picture of the digital media literacy (DML) skills of high school students and to support the development of teacher training methodologies that will introduce new teaching methods leading to the development of needed skills in youth. In the framework of this project, the experts of the CMGs and the CPSC are preparing a methodology for teaching media literacy throughout the educational process, i.e. regardless of the subjects taught, on which at least 200 teachers will be trained. In the long term, the project will contribute to a complete change in the way of teaching and will increase important skills of upper grade students, namely young people who are about to become an active part of society.

These objectives are also enshrined in the Media Literacy Policy, which fulfils Bulgaria’s commitment as an EU Member State to introduce and implement measures to develop the media literacy of its citizens, as set out in the Broadcasting Act. However, the existence of the methodology for measuring digital-media literacy cannot sufficiently satisfy the objectives of the media literacy policy and the expected results of its implementation. In this sense, finding a more appropriate tool to measure and assess Bulgaria’s readiness to develop media literacy of its citizens and, if deemed necessary, a tool to identify and directly measure indicators relevant to the presence and extent of media literacy remains on the agenda.

Summary, conclusions and recommendations

By 2022 – the year in which the Minister of Culture will present the policy for the development of media literacy of Bulgarian citizens and the European Commission should receive Bulgaria’s first report on the implementation of the requirements of Article 33a(2) of the EU Directive 2018/1808* – stakeholders in the field of education and media studies do not have a tool to directly measure the set of knowledge, skills, understanding and attitudes related to media literacy. The methodology presented in this paper is a validated tool for exploring certain aspects of digital literacy and, to a lesser extent and indirectly, media and information literacy of students in the upper secondary stage in Bulgaria, which, after further refinement and validation, could play a complementary role in efforts to measure and assess the quality of formal education.

The main hypothesis of the research design presented in this paper is confirmed – students in grade 10 have serious difficulties in extracting and interpreting information from text, including media texts, and data, and these difficulties are particularly evident when assessing their competencies and cognitive skills through open-ended questions, and especially in tasks that require them to articulate reasoned reasoning. These results correspond with the low levels of reading literacy of Bulgarian students identified by the PISA surveys, whose adaptive definition to the changing information and communication environment increasingly covers competences in the intersection between digital and media literacy – information literacy and online communication.

A new PISA survey in 2022 (the results of which will be released in 2023) will provide an opportunity to further test the extent to which the two methodologies measure the same and/or interrelated competences, and the extent to which the results of these measures overlap and corroborate. The close link between the two methodologies is implicit in the choice of theoretical framework on which the methodology for measuring students’ digital-media skills is built, and in the extension of the PISA concept of reading comprehension to integrate the practices and cognitive skills inherent in interacting with information and communicating in digital environments, which require a much broader set of skills (digital and media literacy skills). At this stage, this close relationship should be seen as an argument for using the methodology presented in this report, especially since, with its narrower and more focused scope, it can be used to test students at smaller intervals than the PISA surveys, which are conducted at three-year intervals.

Thus, this instrument can be complementary and supportive to the OECD’s task of not only measuring comparatively the literacy of students in nearly 80 countries, but also informing education experts and policy makers in Bulgaria about the long-term reforms and the immediate tasks ahead on the path to catching up with the best education systems according to their rankings in the PISA surveys.

In order to adequately address the identified deficits in reading literacy and digital-media skills of Bulgarian students, the Bulgarian education system needs a systematic and intensive reform in terms of the ways of teaching and assessment and the types of content that students interact with on a daily basis, acquiring the knowledge set out in the state education standards and, to a lesser or greater extent, in the CSPs themselves, and hence in the general education curricula. The integration of media texts and daily work with various digital information sources and tools is key to developing students’ critical thinking and, more generally, to building metacognitive skills[1] and attitudes. These skills and attitudes are critically needed by modern citizens for their professional and personal fulfillment and responsible participation in the public sphere and democratic processes[2].

The results also show that despite the widespread and autonomous use among most students of Internet-connected devices, which gives them immediate access to information, knowledge and channels of communication, developing the skills associated with unlocking the full potential of this access does not happen to the extent required by itself – it is the job of the school. Underestimating or limiting this role of the school, the realisation of which goes through a rethinking of the tasks facing teachers, the curriculum and the learning process, must be seen as a risk for civil society in Bulgaria, which has to deal with a number of current and possible future crises. Meeting and overcoming such crises, strengthening the resilience and progress of society in response to digital transformation and the effects of globalisation, goes through prioritising efforts to shape and foster a culture of democratic citizen participation and setting educational goals that begin but do not end with formal education, oriented towards achieving and maintaining the highest possible levels of digital, media and information literacy throughout life.

———–

• By 19 December 2022 and every 3 years thereafter, Member States shall report to the Commission on the implementation of paragraph 1, which established the requirement (1) Member States shall promote and take measures to develop media literacy skills

[1] In addition to critical thinking and problem-solving skills, students increasingly need to develop learning skills and the ability to recognize their own knowledge, skills, attitudes and values, or so-called metacognitive skills (OECD 2030), and acquire the knowledge to effectively apply reading strategies while navigating the Internet, notes the 2021 OECD report “21st Century Readers. Developing Literacy Skills in a Digital World” in response to the question “Why is reading key for 21st century citizens and societies?” The OECD Learning Compass 2030 differentiates three different types of skills:

Cognitive and metacognitive skills, which include critical thinking, creative thinking, learning skill building and self-regulation
social-emotional skills, which include empathy, self-efficacy, responsibility and cooperation
and practical and physical skills, which include the use of new information and communication technology devices
These skills are seen as part of a holistic concept of competence involving the mobilisation of knowledge, skills, attitudes and values in response to complex demands.

OECD (2018), Education 2030: The Future of Education and Skills. Position paper, http://www.oecd.org/education/2030/E2030%20Position%20Paper%20(05.04.2018).pdf.

[2] https://www.oecd.org/education/2030-project/teaching-and-learning/learning/skills/Skills_for_2030.pdf

Important: The originally circulated version of this publication has been edited in light of expert comments made at the press conference launching the report. It also reflects a correction regarding the conduct and publication of the PISA 2022 results – the test has already been conducted, the results should be released in 2023.

[translated automatically from the original in Bulgarian]