School-level predictors for the use of ICT in schools and students’ CIL in international comparison

Education systems around the world face new challenges from the rapid developments in technology and society’s transition towards an information or knowledge society (Anderson 2008; Eickelmann 2011; Voogt and Knezek 2008). Besides discussing new ways of learning and the potentials of ICT from a pedagogical point of view, schools and school systems have acknowledged that new skills and competences are needed to prepare students for life and work in the information age. Thus, the need for students to develop such new kinds of skills, i.e. digital literacy or computer and information literacy (CIL), to enable them to participate effectively in the digital age is constantly gaining in importance (European Commission 2014; Fraillon et al. 2013; Voogt et al. 2013). In this context, it seems to be increasingly important to look at the contexts in which students develop such skills and examine the factors which support or hinder their acquisition. In this regard, the school itself is particularly relevant (e.g. Davis et al. 2013; Eickelmann et al. 2016; Hatlevik et al. 2014; Petko et al. 2015, 2016; Tondeur et al. 2008).

With regard to the factors that contribute to the development of CIL in schools, the ICILS 2013 study (International Computer and Information Literacy Study, 2010–2014; Fraillon et al. 2014) conducted by the International Association for the Evaluation of Educational Achievement (IEA) provides first-time data both on students’ CIL as well as on school-level factors in different education systems. The study investigates the CIL of secondary school students (Grade 8) in 21 education systems using computer-based tests. In addition, it gathers representative student, teacher and school data related to the contexts in which students develop these competencies in all participating countries. By means of an in-depth analysis of ICILS 2013 data, this paper investigates the factors that support or hinder the development of students’ CIL at school level by comparing four education systems around the world (including the top-performing country Czech Republic) using student achievement data as well as data obtained from school and teacher questionnaires.

To understand the relevance of school factors for the acquisition of CIL, the contextual framework of ICILS 2013 (Fraillon et al. 2013) serves as theoretical background in our research. The ICILS framework provides a model which categorizes relevant factors that are in agreement with the multilevel structure inherent in the student CIL acquisition process. It differentiates between antecedents and processes, following the assumptions that antecedents influence processes and that processes are closely linked to the outcome, i.e. the level of CIL competence. It is assumed that both—antecedents and processes—need to be taken into account to explain variation in students’ CIL (see Fig. 1).

As a secondary analysis of the ICILS 2013 data, this paper focuses on four school-level factors as part of both the antecedents and the processes to identify supporting and hindering factors: (1) the school’s ICT equipment, (2) the teaching staff’s professional development, (3) school goals, and (4) the teaching staff’s views/self-efficacy. All of these factors are relevant in the ICILS 2013 contextual framework (Fraillon et al. 2013) and have also been identified as relevant for ICT implementation in schools in other research (e.g. Eickelmann 2011; Kozma 2003; Law et al. 2008). Figure 2 shows the underlying research model behind this paper and the analyses it contains.

The objective of this paper was to examine four education systems (Australia, Germany, Norway and the Czech Republic) with regard to the relevance of school-level factors for the use of ICT by teachers in teaching and learning as well as the effect of the latter on students’ CIL, as measured in IEA ICILS 2013. Based on the ICILS 2013 theoretical framework and results from previous research¸ four aspects appeared to be crucial and were thus taken into account for the analysis carried out in this paper: (1) ICT equipment, (2) professional development of teaching staff, (3) school goals, and (4) views/self-efficacy of teaching staff.

The results for our first research question reveal both similarities in the four education systems yet also some country-specific results. As far as the similarities are concerned, the self-efficacy of teaching staff with regard to ICT was identified as a very important supporting factor in all four education systems. This confirms earlier research which identifies teachers as a keystone species (Davis et al. 2013) when it comes to the integration of ICT in schools as well as the apparent importance of teachers’ own perceptions of their competences (Drossel et al. 2016) on both the individual and the collective level (i.e. among the entire teaching staff in a school). As a consequence, a common developmental strategy within a school might be to attribute importance to providing support for teachers, thereby raising their own assessments of their competencies. This support might help to make (secondary school) teachers feel more capable in using ICT in their teaching activities and in adapting their competences in the subject-specific use of ICT. Professional development activities are one way of strengthening this support factor and could be given stronger emphasis in the transition of education systems to 21st century needs.

Moving to the country-specific results, the participation of a large proportion of teaching staff in courses on the use of ICT was identified as a supporting factor for the use of ICT in teaching in Australia. Keeping in mind the comparably high participation rates of Australian teachers in personal development activities (Fraillon et al. 2014), this result underlines the need for ongoing development of technological and pedagogical applications of ICT in schools along with the need for accompanying teacher education and could serve as an example for other countries.

In the Czech Republic, the importance of school goals, or more precisely goals referring to the development of students’ competences in ICT, might possibly be explained by the fact that successful schools use these to establish a bottom-up counterpart to the nationwide ‘framework educational programme for basic education’ established by the Ministry of Education (MŠMT 2007), which includes a detailed account of how ICT should be integrated in each subject and which attributes the relevance of ICT for teaching and learning in a top–down way. Under this national framework, each school has to adapt and integrate more holistic strategies into their own program, adopting the national plans within the scope of the single schools’ conditions. In addition, the Czech School Inspectorate evaluates whether the individual school programs contain everything that needs to be included under the nationwide framework. It also conducts inspection visits to schools to ensure that the program has been enforced.

As far as Germany is concerned, the most relevant school-level predictor for the use of ICT by teaching staff seems to be the availability of corresponding pedagogical support in the classroom. This perhaps offers another indication that teachers in Germany are not sufficiently trained in using ICT (Eickelmann et al. 2016b). Indeed, at school level, this represents the greatest challenge to the development of the German education system. This result is complemented by the finding that the focus of the development of support systems for schools in Germany is still more technical than pedagogical. Furthermore, the responsibility for implementing technical ICT support lies at regional or local authority level, which leads to great variation in support systems across the country and, in some cases, can cause problems when teachers need immediate technical support in the classroom. Future developments in the German education system could therefore focus on providing better pedagogical support for schools and on better preparing teachers for the demands of the pedagogical integration of new technologies in teaching and learning (Eickelmann 2016).

Positive views on ICT on the part of teaching staff are one of the supporting factors for the use of ICT which could be identified at school level for Norway. This result could be interpreted as an indication that the Norwegian top-down strategy used to implement ICT in schools nationwide following the 2006 National Plan (Erstad and Quale 2009) might not have taken into consideration the perspective of all teachers and the need to convince teachers of the usefulness of the use of ICT in teaching and learning.

Regarding our second research question, a significant positive correlation between the use of ICT by teaching staff and the students’ level of CIL could only be identified for Germany. In comparison to previous results dealing with the students’ use of ICT in teaching and learning on an individual level, which showed a negative correlation with students’ CIL in Germany (Eickelmann et al. 2014b) and could be explained by the aforementioned compensatory approach, this result underlines the potential of the use of ICT by teaching staff in teaching and learning for the competencies of their students. In Germany’s case, it could be interesting for further research to conduct the analyses separately by school type as there is a large difference between the competence levels of students at Gymnasia (grammar schools) and other secondary schools.

A limitation of the research presented in this paper is the above-mentioned cross-sectional data base. Even though the research approach takes into account both a theoretical framework as well as previous empirical results and applies a hypothesis-testing method, no causality can be postulated. Therefore, alternative hypotheses and interpretations might apply. The results for Germany, for example, where a less favorable student–computer-ratio has an effect on students’ average CIL at one school could also be interpreted the other way round: schools with a higher average level of students’ CIL might not necessarily have great ICT equipment. This interpretation might be supported by acknowledging that computers in German schools are still often used as an instrument to compensate learning needs and, thus, follow the objective of promoting students with lower competencies rather than all students (for more on this topic in the case of primary schools see, for example, Drossel and Eickelmann 2014). Furthermore, the result that cooperation concerning ICT in communities of practice in teaching has a positive effect on students’ CIL might also be interpreted differently: in schools where students have high levels of CIL, there would seem to be a higher recognition of the need for teachers to cooperate in a more institutionalized and formal way. In the case of our second research question in particular, alternative hypotheses should be taken into consideration. The significant, positive correlation between the use of ICT by the teaching staff and the students’ level of CIL could also be interpreted as an indication that teachers in schools where students show high levels of CIL use ICT more often for teaching, possibly because they assume that their students use ICT in a more competency-oriented and thoughtful way.

Based on these findings, further research besides longitudinal studies should also draw on a qualitative approach to generate more detailed explanations regarding the relevance of the various factors. Comparative case studies might, for example, gain more detailed insights into supporting and hindering factors and thus serve to generate knowledge which provides valuable hints for the development of educational systems from an international and intercultural perspective. This seems to be crucial in appreciating the transition towards an information and knowledge society and the related challenges for education systems. Furthermore, more of the factors that have been identified in previous research as relevant for the use of ICT (e.g. school leadership) should also be included in further analyses.

Concerning the implications for school development, it can be concluded that in all four countries studied, the characteristics of the teaching staff are important for the frequency of computer use but not necessarily for CIL acquisition. This points to a need for increased skills-based learning with ICT and competence-orientated learning supporting the acquisition of CIL. For this, acknowledgement of the relevance of school-level factors is required.