EUR-Lex Access to European Union law
This document is an excerpt from the EUR-Lex website
Document 52016DC0364
REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS Final evaluation of the multi-annual EU programme on protecting children using the Internet and other communication technologies (Safer Internet)
REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS Final evaluation of the multi-annual EU programme on protecting children using the Internet and other communication technologies (Safer Internet)
REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS Final evaluation of the multi-annual EU programme on protecting children using the Internet and other communication technologies (Safer Internet)
COM/2016/0364 final
EUROPEAN COMMISSION
Brussels, 6.6.2016
COM(2016) 364 final
REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS
Final evaluation of the multi-annual EU programme on protecting children using the Internet and other communication technologies (Safer Internet)
1.overview
This report presents the findings of the evaluation of the Safer Internet programme 2009-2013 1 , hereafter 'the Programme'. The Commission concludes, on the basis on this evaluation that the Programme has been successful in achieving its main goals. The Programme was highly relevant in terms of the problems addressed and the stakeholders involved, and responded well to changing technologies and problems. It demonstrated that EU intervention had been both necessary and effective in the delivery of results. The pan-European dimension of the activities, the establishment of Safer Internet Centres (SICs) in all Member States, and the close cooperation between different stakeholders are some of the key success factors.
The evaluation is based on an external study using data collected through surveys and interviews with project participants and a wider range of stakeholders. The conclusions of the external survey are sound and corroborated by the conclusions of the project reviews and by findings from the parallel work to benchmark Safer Internet policies in the Member States 2 . The findings indicate that the Commission should consider:
continuing the most effective activities, namely the SICs, focused research and European coordination of actions;
improving measures for monitoring activities undertaken in the projects;
continuing communication and cooperation at all levels, and;
ensuring greater involvement of Member States and industry.
In addition the evaluation found that:
SICs play a key role in the coordination of actions and initiatives. In the majority of Member States, the SIC is recognised as the key platform for the implementation and coordination of actions and initiatives.
issues related to online safety for children is high on the policy agenda in all Member States but there is room for improvement in the design of the policies, in particular in relation to the coordination, collection of evidence and evaluation of the policies.
2.background
This report concerns the final evaluation of the Safer Internet Programme 2009-2013, with the objective to assess the Programme for its relevance, effectiveness and efficiency, as well as its broader impacts and sustainability. Article 5 of the Programme Decision 3 , stipulates that the Commission evaluates the manner in which the projects have been carried out and their impact in order to assess whether the objectives have been achieved, and that the Commission will submit a final evaluation report at the end of the Programme. The evaluation will also feed into recommendations as input for future Safer Internet policy and on the follow-up of the Programme through the digital services infrastructure for Safer Internet services implemented under the Connecting Europe Facility 4 .
With a budget of MEUR 55 the Programme co-funded projects in the Member States, Iceland, Norway, Serbia (through a MoU) and provided limited funding to Russia for participation in European networking activities Its main objectives, set out in the following Action Lines, were to:
1.Increase public awareness in particular among children, parents and teachers about opportunities and risks related to the use of online technologies, and means of staying safe online.
2.Fight against illegal content and harmful conduct online by reducing the amount of illegal content circulated online; and by dealing adequately with harmful contact and conduct, with particular focus on online distribution of child sexual abuse material (CSAM), grooming and cyber-bullying.
3.Promote a safer environment by bringing together stakeholders, including children, and by fostering self-regulatory initiatives.
4.Establish a knowledge base for dealing adequately with existing and emerging uses of the online environment and relevant risks and consequences, with a view to designing adequate actions aimed at ensuring online safety for all users.
92% of the budget co-funded projects under actions 1 and 2 (the European network of SICs and its coordination), 5% funded projects for establishing a knowledge base and 2 % was allocated to the third action. The average co-funding rate was 73% and the average duration was 26,5 months.
Compared to the earlier Safer Internet Programmes 5 the general scope of the Programme was extended to include emerging online technologies, coverage of harmful contact and conduct and the establishment of a knowledge base within this field. The main mechanism for implementing the Programme was annual Work Programmes outlining priorities, actions and funding instruments for each year. The co-financing of projects selected on the basis of calls for proposals resulted in a wide range of projects aimed at creating a safer online environment for children. The projects were complemented by non-funded activities which ranged from supporting Safer Internet Day, organizing the Safer Internet Forum, supporting industry self-regulation, and international cooperation.
The recommendations of the interim evaluation of the Programme in 2011 were incorporated in the continued implementation of the Programme such as engaging with youth through regular activities, extending the coverage of helplines and initiatives supporting self-regulation, such as the CEO Coalition to make the internet a better place for kids, were maintained.
More specifically, the Programme co-funded Safer Internet Centres in all EU Member States, Iceland, Norway, Serbia and Russia. SICs carried out activities to raise public awareness; they operated helplines from which children and parents could get support and advice on online related issues. The awareness centres and helplines belong to the INSAFE network. 6 In most countries SICs also include hotlines, whose main task is to receive reports from the public about CSAM, and if assessed as illegal, notify law enforcement and the hosting ISP for the content to be removed. The hotlines are coordinated by the INHOPE Association. 7 Under the action for fighting against illegal content the Programme also co-funded initiatives to support law enforcement in the fight against CSAM. These activities have in turn supported the compliance by Member States with their obligations on prevention of child sexual abuse and removal of CSAM under the child sexual abuse directive 8 , as well as with their commitments under the Global Alliance against child sexual abuse online. 9
In order to prepare for the follow-up of the Programme, the Programme funded a pilot for developing a platform to share resources, services and good practices for SICs; and establishing a database that enhances hotlines' capacity to identify and analyse reports of CSAM. The review of this project concluded that the INSAFE/INHOPE networks are increasingly consolidated across Europe and that their impact is undeniable. The project was said to play a key role in Safer Internet work in Europe and beyond partly due to the overall strength of the dissemination which has positioned the project as a reference in the field.
Under its third action 'promoting a safer environment', the Programme has supported self-regulatory agreements and through this pushed for increased use of safety practices on websites and devices used by children. Online technologies develop at a fast pace and industry self-regulation is seen as the most effective approach to be able to react quickly concerning new developments of children's use and risks. The online industry has responded positively to Commission initiatives and other joint activities. The most recent example is the CEO Coalition to make the Internet a better place for kids 10 where 31 global companies across the value chain were invited to sign up to VP Kroes' initiative to work together. This builds on earlier support from the Programme to self-regulation initiatives 11 taken by the industry at the European level.
Civil society was included through actions such as the national youth panels, as well as through the funding given for the eNACSO network of children's welfare NGOs. 12 The aim of this network was to develop a concerted approach by sharing experiences, and by developing joint strategies, in order to ensure that the needs of children are better taken into account in national, European and international discussions around online safety.
Knowledge enhancement is crucial in order to develop robust policy approaches, especially in this rapidly changing area, and increased knowledge on young people's use of new media and technology was a central aim for the work in the Programme. The Programme supported the EUKidsOnline network 13 , whose activities have proven instrumental as a source of accurate trend analyses and for getting input on future research and policy priorities; and a broad range of research projects covering a range of topics 14 . In addition to raising the research profile of Safer Internet related issues, the knowledge developed fed into the knowledge base for all activities in the Programme.
The Programme has also funded a number of studies to gather evidence, specifically the SIP-Bench study 15 and a study on Safer Internet policies and indicators in the Member States 16 . The overall aim of the latter was to improve the understanding of current national policies and actions and to design a sustainable tool enabling identification and sharing of good practices.
The European Strategy for a Better Internet for Children (BIK strategy) 17 has been since its launch in 2012 a central reference document for the activities undertaken in the Programme. The BIK strategy sets out a comprehensive approach through four pillars: stimulating quality content online for young people; stepping up awareness and empowerment; creating a safe environment for children online and fighting against child sexual abuse and exploitation. The Programme contributed to actions under each of these through: the promotion of positive content through national and European competitions; developing and sharing awareness resources and promoting best practices; hotlines and training for dealing with CSAM.
3.EVALUATION OBJECTIVES AND METHODOLOGY
The final evaluation examined the implementation of the Programme, with respect to its relevance, efficiency and effectiveness and assessed its broader impacts and sustainability as well as its EU added value and coherence, which were assessed as parts of these criteria. The evaluation is qualitative in nature and includes limited quantitative evidence but additional data is available in the external evaluation report. The Commission will feed the results of this evaluation into further policy and action development.
To consider the relevance of the Programme the evaluation centred its questions around the evolution of the problems addressed by the Programme and how it responded to these changes, the relevance of the actions and their added value to the key target groups.
The efficiency of the Programme was evaluated through questions about overall satisfaction with the implementation, the efficiency of the activities, the adequacy of funding and its leveraging effect.
Effectiveness considered the contribution the Programme made to protecting children online and the broader digital environment, whether the same results or activities might have been achieved by other more effective means or without the Programme, and what improvements might be made in any follow-up actions.
Impact and sustainability looked at evidence of the sustainability of impacts, whether changes would have happened without EU funding and what would have been the alternatives as sources of funding, and the complementarity/overlap with other national or EU Programmes.
The external evaluation employed a number of data collection tools in order to gather evidence to inform this report. The data collection phase included a survey targeting project participants in the Programme as well as a survey open to a wider range of stakeholders representing all relevant stakeholder groups – industry, public authorities, education, and NGOs.
These were complemented by an interview programme of semi-structured interviews with stakeholders ranging from Commission officials and Programme beneficiaries to external actors such as industry, researchers and Member State and law enforcement representatives. Finally, case studies were carried out illustrating different forms of Safer Internet cooperation in four Member States.
In addition an analysis was carried out of the final review reports of projects, focusing on the extent to which objectives were achieved and their impacts.
3.1.Limitations – robustness of findings
The data collection and analysis carried out has a number of intrinsic limitations, whose impact was mitigated to a maximum possible extent:
Measuring the impacts of the projects was not possible because of insufficient monitoring tools. The assessment of impact and sustainability was therefore mainly based on the perceptions of project participants, which is often not objective.
The evaluation takes into account the inherent limitations of the findings of surveys: the answers received reflect the views of a sample of relevant stakeholders and not those of the entire population who has a stake in this domain. Secondly, stakeholders' views convey an individual rather than a holistic perspective.
Based on the elements above, this evaluation has been carried out on the basis of the best available data. Whenever reliable quantitative data is lacking, this is indicated as appropriate and counter balanced with qualitative data and considerations.
4.FINAL EVALUATION FINDINGS
The findings of the evaluation show that the Programme has been successful in achieving its main goals. The success of the Programme was particularly based on its pan-European character and coherence, and the close cooperation between the different stakeholders.
The initiatives funded have had a positive impact on the rights of children. Article 24 of the Charter of Fundamental Rights of the European Union, recognising that children are independent and autonomous rights-holders and prescribes that children have the right to protection and care necessary for their well-being, have been taken into account in the promotion of Safer Internet policies.
The findings presented below by each evaluation criterion represent the consolidated results of the work carried out by the Commission and the surveys and the interviews conducted for the external study.
4.1.Relevance
The results of the evaluation confirm that the implementation of the Programme took account of new technologies and new online behaviours such as children going online at an ever younger age on a variety of connected devices, and that consistent revision in strategy were needed due to the fast changing nature of the field. This was made possible through the flexibility offered by annual work programmes and calls for proposals. Stakeholders acknowledged that the Programme has become a significant forerunner internationally, and has provided leadership as well as financial support to Member States.
This evaluation has also confirmed that the Programme's actions were comprehensive and have both involved and targeted a wide spectrum of different stakeholder groups. 'Increasing public awareness' and 'Fighting against illegal content' were regarded as the most relevant objectives, while stakeholder opinions are more split on 'Promoting a safer online environment' and 'Establishing a knowledge base'. Nevertheless, all four actions were viewed as responding to the needs of children engaged in the Internet. There has been an added value for the Programme's target groups: most stakeholders consulted were not aware of activities similar to their project undertaken by other organisations at national level; and there was a strong indication that without EU funding, the majority of projects would not have been carried out, or at least not to the same extent, due to the lack of financial resources specifically allocated for promoting safer use of online technologies by children.
The evaluation results emphasised the importance of education and positive content. Children need to access quality education that includes a digital component to help protect against various risks when confronted with such rapidly changing technologies. The evaluation underlined the benefits of children's participation and the need both for children to be aware of possibilities offered by digital technologies and for parents to be sufficiently engaged in their children’s online activities. To be active and effective participants online and to be empowered in recognising harmful content and navigating their own safety online, children need digital competences, ideally provided through school curricula.
Programme participants and other policy and industry actors valued highly the research conducted and the importance of disseminating this research throughout the rest of the Programme. Specific input to the survey underlines the need to conduct further in-depth international research on children’s online behaviour 18 , and on efficient methods of education.
Finally, Programme beneficiaries and industry wished to continue their collaboration, including through more specific actions for vulnerable children, disabled children and children with difficult situations at home.
4.2.Efficiency
Project participants reported a high level of satisfaction with the management of the Programme carried out by the Directorate-General Communications Networks, Content and Technology (DG CNECT), though there may be some bias due to vested interest in supporting the Programme. Still, some beneficiaries identified areas for improving the process of applying for future EU grants in the field, such as: simplified procedures and reporting requirements; more flexible requirements to accommodate country specificities; longer-term planning of work and funding; and more information on other funding Programmes in related areas.
Stakeholders generally recognised that the Commission ensured an efficient management of the Programme. The average time to contract was 292 days for the 77 projects that were negotiated during the programming period. Actions of the Programme, and especially the focus on network activities, were largely considered as being ‘good value for money’ but there is scope for more consolidation and collaboration at EU level, across services, and between programme beneficiaries. The findings on the adequacy of funding were split evenly between those who found it sufficient and those who did not, and of those who found it insufficient, nearly all had access to funding from other sources. However, these results could be biased depending on whether the beneficiary had access to additional funding or not.
Funded projects often benefitted from synergies with other initiatives. The fact that SICs exist in 27 Member States is perceived by stakeholders as the main added value of the Programme as this allows the sharing of knowledge and good practice. This is confirmed in the mapping of Safer Internet policies, which concludes that in the majority of the EU countries, the SIC is recognised as the key platform for the implementation and coordination of actions and initiatives. 19
The evaluation confirmed that the projects funded under the Programme have contributed to achieving the Programme’s objectives. This has been done by raising public awareness on safer use of online technologies, by enhancing cooperation between stakeholders, by tackling harmful conduct online, and by involving and informing children in and on online safety. Concerns were however raised that the effectiveness of the Programme and its activities in the future may be endangered by a lack of sustained EU funding and reluctance by industry and national governments to step in.
Regarding improvements for the future, activities which could have received more consideration include social work, and, further research and projects designed to improve children's digital skills. Finally, stakeholders saw the need for industry to play a larger role in enhancing the effectiveness of activities launched by the Programme.
4.3.Impact and Sustainability
The findings from the surveys and interviews with project participants and stakeholders, with the caveat that they could be biased, show that the programme has had a medium to high impact, mainly in raising awareness, fighting illegal content, increasing digital skills and by enhancing cooperation and creating synergies.
Awareness raising, combined with providing education, training and related tools and materials to be used by children, was the most cited key The coordination provided through the Insafe network is seen to maximise synergies and impact, and for example, in 2014 the joint efforts of the network reached out to more than 22,5 million people across (and beyond) Europe. 20
The second strong area of impact was achieved in the fight against illegal content, and specifically the work of the hotlines. The establishment of a system of notice and take-down of CSAM and the improved cooperation between hotlines and law enforcement have led to faster removal. In 2011, 60% of the reported content was removed within 1 to 3 days compared to over 90% in 2014. 21 Also, over the programme period, hotlines received an increasing number of reports 22 leading to an increase in sites deleted by the INHOPE network.
The findings further show that the Programme has been influential in relation to other national and international activities. This is evidenced by references made to research and results from the Programme. European Safer Internet activities are often considered good practice internationally, and have been taken up in Latin America, the US and Asia-Pacific. As an illustration, the Safer Internet Day, which stems from the Programme, is an international yearly event to raise awareness of child online safety. It is organised annually in more than 100 countries and recognised across Europe, North America, South America and the Asia-Pacific region.
The majority of stakeholders considered that project activities would not have been carried out without EU funding, and they voiced concerns about obtaining funding from national governments or the industry and many believed that in this regard, continued funding by the Commission is necessary. Project beneficiaries said that the impact of their projects would continue in the medium to longer term with a crucial role for public authorities and industry in funding future activities.
5.EVALUATION CONCLUSIONS AND RECOMMENDATIONS
On the basis of the external evaluation and its parallel work, the Commission arrives at the following conclusions as input for deliberations on future Safer Internet policy.
–The evaluation shows that the Programme has been successful in achieving its main objectives. The Programme was highly relevant in terms of the problems addressed and the stakeholders involved, and responded well to evolving technologies and problems.
–It is demonstrated that EU intervention had been both necessary and effective in the delivery of results. The pan-European dimension and coherence of the activities, the establishment of SICs in all Member States, and the close cooperation between different stakeholders have contributed to the successful implementation consolidation and collaboration at EU level, across services, and between programme beneficiaries
–The programme was recognised to have been delivered efficiently, although certain procedures were highlighted as presenting scope for simplification.
–The programme has had a medium to high impact, mainly in raising awareness due to the focus on networking and creation of synergies, and fighting illegal content; the establishment of a well-functioning system of notice and take-down of CSAM, complemented by voluntary efforts from industry to counter distribution of CSAM through their infrastructure, has significantly reduced the time to removal. 23
–The Commission has exercised a recognised leadership role both with respect to Member States and internationally where European Safer Internet activities are often considered good practice and have been taken up worldwide such as the celebration of Safer Internet Day.
5.1.Continuation of most effective activities
In order to keep up and build on the results achieved, the following types of activities that have to date proven to be the most effective, could be further sustained:
-the work of the SIC awareness centres, which should be continued by further reinforcing awareness-raising activities, including the dissemination of information and tools for children on safer use of online technologies and continuation of the annual Safer Internet Day; to further expand the work of the helplines and their cooperation with national child protection systems should be encouraged; also the work of the hotlines which has been a major achievement of the Programme should be continued.
-to ensure that the SICs reach out to children in disadvantaged or vulnerable situations (such as children with disabilities, children at risk of poverty, children in migration) allowing them to embrace the benefits of digital technologies by their full inclusion.
-continued research at EU and national level on children’s behaviour online and the risks they face to keep the knowledge base up to date and to ensure any initiatives remain relevant and focused on those areas that need the most attention. Given the lack of focused research in this area, there is a need to think about how to encourage researchers to contribute to the knowledge base on children’s use of the Internet and provide the material for the continuing development of actions, and to ensure that the rights-based approach evident in EUKidsOnline is replicated in future research, with a view to further realising the rights of the child in the online environment. In this context, helplines could be a valuable source of information, as they are the first point of contact, consulted directly by children and parents.
-to continue the mapping of Safer Internet policies on a regular basis to get an overview of how the challenges concerning online safety are met and addressed in policies and initiatives across Europe, and to identify emerging patterns and approaches taken by the Member States.
Overall, the coordinating and supervisory role of the Commission in the implementation of the Programme, and in particular through the SICs has been very valuable for Member States and all stakeholders involved. 24 Thus, a continued coordination role for the Commission in future Programmes would be of added value.
5.2.Facilitate the monitoring of impact of projects
Determining and measuring the impacts of projects co-funded by the EU, is currently not possible because of insufficient monitoring tools. For the future, it is therefore advisable that a robust monitoring system is being set up and that regular independent evaluations are carried out. Therefore, measurable implementation and performance indicators need to continue being further developed, allowing for a comparison and better assessment of projects of the same kind and to highlight good practices.
5.3.Continued need for effective communication
Communication should be pursued between all relevant entities and stakeholders in order to share good practices and to ensure that there is no overlap between future initiatives. In this context, the Commission should further enhance existing cooperation across the various relevant Commission services on issues related to internet safety and full range of illegal content and monitor consistency of their work. In addition, the cooperation and exchange of good practices between different types of stakeholders should be further enhanced.
5.4.Encourage industry and Member States’ involvement
The evaluation has demonstrated that the activities funded under the Programme are valuable, but at the moment there is limited financial involvement from the industry. Given existing concerns of project participants over future funding of initiatives, stronger industry involvement should be sought as alternative sources of funding of activities currently funded by the Commission and, to some extent, by national governments. This could be a long-term objective which would enable the Commission to conduct research on how to best improve the (financial) engagement of industry more generally.
Member States’ involvement in future initiatives should also be further encouraged. Thus, while determining the level of resources for future activities, the availability of national and other sources to complement the EU funding should be assessed at an early stage. Moving forward, the need for national co-financing should be clearly and continuously communicated by the Commission so that no Member States solely rely on EU-funded projects in this policy area. Highlighting good practices of Member States by the Commission and communicating this to all other Member States could incentivise them to increase national funding.
Published as: Mapping Safer Internet policies in the Member States: The Better Internet for Kids (BIK) Map ISBN: 978-92-79-47784-3
European Framework for Safer Mobile Use by Younger Teenagers and Children; Social Networking Principles for the EU
