EUROPEAN COMMISSION
Brussels, 1.12.2016
SWD(2016) 286 final
COMMISSION STAFF WORKING DOCUMENT
On the implementation, results and overall assessment
of the European Year for Development 2015
Accompanying the document
REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS
On the implementation, results and overall assessment
of the European Year for Development 2015
{COM(2016) 525 final}
Contents
Section 1: Executive summary
Section 2: Introduction
Section 3: Background to the EYD initiative
Section 4: Evaluation questions
Section 5: Method
Section 6: Results
Section 7: Answers to the evaluation questions
Section 8: Conclusions
Annex 1: Procedural information
Annex 2: Stakeholder consultation
Section 1: Executive summary
In 2015 the Commission conducted a year-long communication campaign to inform citizens about EU development cooperation and its benefits, and to encourage their active interest, critical engagement and direct involvement. This Staff Working Document presents the findings of an independent evaluation (by Coffey and Deloitte) of the implementation, results and overall performance of the European Year of Development (EYD2015) campaign. The aim is to help EU institutions learn how to run better communication campaigns (including other "European Years") by identifying areas in which EYD2015 fell short, while taking advantage of innovations that worked effectively, building on the networks created or developed, and integrating successful practices and experiences used elsewhere in Europe.
Findings
Relevance and complementarity: EYD2015 met the needs of its participating partners, while complementing other European, national and local initiatives to promote development cooperation. The objectives set by the European Parliament and the Council in the EYD2015 Decision were designed to dovetail with existing national and local development communication approaches in Europe. The branding of the exercise was strong, although the evaluator found that stronger political messages on some issues would have further increased the efficacy of the campaign. The campaign's approach to communicating about development, particularly through the co-curation of EYD2015 social media accounts and the use of story-telling to illustrate the results of European development policies, was appropriately pitched and successful.
Effectiveness and impact: Political and public support was good, notably in terms of outreach to young people. The general effectiveness of campaign management at central level is illustrated by the strong relationships forged with partners, the active involvement of MEPs and the visibility achieved through non-funded organisations. The focus on targeting young people was clearly acknowledged in stakeholder surveys and project reporting, although the reliance on campaign partners as multipliers tended to limit opportunities for messages to reach beyond the development community and civil society. The provision of a comprehensive campaign toolkit (motto, visual identity, materials) was beneficial, and the device of assigning a specific theme to each month of the year was effective in developing a more focused approach to communicating with young people. Although the website might have been made more accessible to the general public, the extensive use of online and social media (particularly including co-curation and partnering) was highlighted as very positive. As regards press coverage, activities at national level were effective in addressing the campaign's three objectives. Overall, while the sheer volume of activity can reasonably be assumed to have had a significant impact, with the attention of political leadership very much focused on migration issues, that impact is liable to have been most significant at the level of public opinion.
Efficiency, coherence and EU added value: The focus on working through partnerships (mainly with members of the development community) to implement the EYD2015 campaign was an efficient way of empowering stakeholders and reaching a wider European public, especially young people. Although the evaluator acknowledges that it was difficult to gauge efficiency with the tools available, partners generally concurred that campaign activities provided good value for money. While some of the contractor's media outreach work in the Member States had mixed results owing to coordination issues, coordination between the Commission and the national administrations implementing EYD work programmes (NWPs) worked very well. With the exception of some doubts expressed about complementarity in certain cases, campaign activities were largely deemed coherent, and the evaluator emphasised the EU's added value in this process thanks particularly to sound management and implementation practices at central level.
Sustainability: In the policy recommendations of the Civil Society Alliance and the Joint Declaration by the European Parliament, the Council, and the Commission on the legacy of EYD 2015, NGOs and Member States alike have expressed a strong commitment to maintaining the partnerships created by the campaign for communicating on development issues. EYD 2015 is widely perceived by stakeholders to have opened doors, and to have laid the foundations for the development of dynamic and sustained networks promoting exchanges on best practice, joint campaigns and greater professionalism, notably in working with traditional and social media on development issues; there is in particular a strong motivation to take this "EYD legacy" forward in the implementation of the 2030 Agenda for Sustainable Development.
Conclusions
The overall conclusion of the evaluation is that EYD2015 fulfilled its objectives and successfully reached its intended audiences. The campaign brought practical benefits to all parties involved – Member States as well as NGO partners - in terms of development communication practices and networks, and activities were generally effective and complementary across the EU. Branding was strong, with a corporate, pan-European identity created by the logo, motto and other visuals, supported by a dedicated campaign toolkit, flagship events, and online and social media coverage. Based on these findings, EYD 2015 has enabled the EU significantly to improve its capacity to communicate with European citizens on development issues, and establishes a benchmark and a legacy from which future campaigns in other domains can benefit.
Section 2: Introduction
The evaluator was asked to provide an overview of the results achieved by EYD2015, a qualitative and quantitative assessment of the campaign's implementation, and recommendations as to how the management of future European Years might be improved. The findings of the evaluation will be made available to all Commission departments and EYD2015 stakeholders and will also be published online in order to allow future communication actions to build on the EYD2015 experience.
The task of the evaluator was to show what a pan-European campaign of this kind can achieve in bringing development messages to the general public, to indicate the extent to which it is possible to attract a large and diverse number of interested parties, and to demonstrate how partnerships and networks can add value to informing, engaging and raising awareness on development in the years to come. The findings are intended to be used to improve the communication practices of the development community and EU institutions, especially in the planning and management of campaigns.
In terms of scope, the evaluation covers the activities funded by the Commission's Directorate-General for International Cooperation and Development (DG DEVCO) and carried out by DG DEVCO and its strategic partners under the umbrella of the EYD2015:
EU-level activities with a dedicated emphasis on the EU-wide information and promotional campaign and related activities;
National activities, in particular those carried out by the national coordinators (a field-study of six Member States was undertaken by the evaluator);
Actions led by Civil Society Organisations;
Coordination with strategic partners inside the European Commission;
Coordination with partners outside the European Commission.
Section 3: Background to the EYD initiative
EYD2015 came at a crucial juncture, when major developments affecting the future of every citizen were in the offing. The EYD2015 Decision established three key objectives for the Year as a contribution to the long-term goal of ensuring continued support for development cooperation. The objectives were: (i) to inform citizens about the work of the EU and the Member States on development cooperation; (ii) to foster direct involvement and critical thinking about development policy; and (iii) to raise awareness about the benefits of development cooperation and its global dimension among citizens in partner countries and in the EU.
EYD2015 was designed to have an impact on the way in which the European Commission and its stakeholders communicate on development cooperation, and to contribute to building expertise on how to improve communication in this area. In terms of target audiences, the key goal was to reach the broadest number of people, but with a particular focus on young people (primarily between 15-24 years old). Other key target audiences included development experts/academia, civil society, policy makers and media/information multipliers.
DG DEVCO was responsible for the overall coordination of the initiative, while additional staff was brought in to cope with the workload. The implementation strategy adopted was to work in partnership with a wide range of stakeholders – including civil society representatives - at EU, national and local level, in order to maximise impact and visibility. This approach engendered a number of innovations, such as creating networks involving a wide range of development partners, inviting –for the first time ever – partners to co-curate EYD2015 social media accounts (with a different partner taking over every week), and getting development messages across by telling original stories from partner countries in 23 languages.
At centralised level, this entailed establishing and coordinating partnerships and providing them with comprehensive campaign materials. Thanks notably to the work of the Inter-institutional Working Group, all EU institutions contributed significantly to these tasks, which included the organisation of flagship events, and a series of press, web and social media communication activities all designed to provide opportunities for partners – as well as EU services and staff (including in the Delegations) - to become more involved in global discussions on development. The global EYD2015 motto - "Our world, our dignity, our future" - and the concept of assigning a specific theme to each month of the year , established a strong conceptual framework for communication on the Year.
At decentralised level, the aim was to sign national EYD work programmes (NWPs) with as many Member States as possible, and at the very least to organise EYD events and activities in all 28 Member States, and to agree on a pan-European activity in celebration of a European Development Day. The Commission was also determined to involve the European Civil Society Alliance (CONCORD) as closely as possible to add the voices of civil society and local authorities to the campaign. International organisations such as the United Nations agencies and the World Bank were welcomed as substantial contributors to the EYD2015 campaign, and the participation of the private sector was also encouraged.
The intervention logic diagram below illustrates the links between the campaign's objectives and activities carried out, and the contribution of the different types of activity organised during the year. The campaign budget had a total budget of over EUR 10.5 million, 42% of which was accounted for by the centralised component, with 58% being allocated to decentralised activities. The Commission covered 80% of the cost of the NWPs implemented by Member States and of grants awarded to CONCORD projects.
Establishing a baseline
Development was at the heart of a global policymaking in 2015, when the international community was making decisions on three major new initiatives: the replacement of the Millennium Development Goals with a new set of global goals (the Sustainable Development Goals), as part of the 2030 Agenda on Sustainable Development; the Addis Ababa Agenda for Action on financing for Development; and the Paris Climate Deal at the COP21 conference.
At the start of the EYD campaign, a special Eurobarometer survey assessed European attitudes to development cooperation: with some divergences among the different Member States, altogether some 85% of respondents considered development important in tackling poverty (the number was even higher among the 16-25 age group). However, only 12% of respondents were aware that 2015 had been designated a "year of development" by the EU, and only 4% felt they had good understanding of where development money goes. This data was essential in helping us to understand citizens' awareness of, and attitude to, development issues, and to design better coordinated and targeted activities and events to address their perceptions and concerns.
Section 4: Evaluation questions
The implementation and results of the campaign were assessed in line with five evaluation criteria, as required by the Better Regulation guidelines: effectiveness, efficiency, coherence, relevance and EU added value. Complementarity, impact and sustainability were also assessed. The evaluation questions were categorised in four groups, in each case with a set of sub-questions, as follows:
1. Relevance and complementarity
oTo what extent were the activities of EYD2015 relevant to the needs of the interested parties?
oTo what extent were the activities complementary with other existing national and EU initiatives?
2. Effectiveness and impact
oTo what extent did the activities undertaken in the framework of EYD2015 improve popular and political support?
oTo what extent were the activities undertaken in the framework of EYD2015 effective in achieving its three specific objectives?
oTo what extent were the activities undertaken in the framework of the EYD2015 effective in creating a synergistic pan-EU campaign, with EU citizens as the target audience and special attention to youth?
oTo what extent were EU Member States, European Commission DGs and services and EU Institutions and bodies coordinated and provided with support to organise their own EYD 2015 activities and to participate in the centralised campaign activities?
oTo what extent were other interested parties such as CSOs, International organisations, Private Sector representatives, Academia and Think Tanks engaged? Was their participation in EYD 2015 facilitated? Were they encouraged to initiate and label own activities as EYD2015?
oTo what extent were the activities conducted carried out and outputs produced as planned? To which extent did the centralised activities achieve their expected results?
3. Efficiency, coherence and EU added-value
oHow efficient were the activities undertaken in the framework of EYD2015 in conveying the appropriate messages and achieving the expected results?
oWere the actions undertaken properly coordinated and coherent with other actions?
4. Sustainability
oTo what extent has EYD2015 been successful in delivering sustainable outcomes in relation to its three specific objectives?
Section 5: Method
The methodology was based on an analytical framework consisting of: desk research; a stakeholder survey (asking partner organisations about their views on their own activities and those of the Commission); ex-ante and ex-post strategic interviews of people working for the Commission and EYD2015 partners (other EU institutions, the private sector, international and civil society organisations) regarding their perceptions of the campaign; fieldwork research in six countries implementing a NWP; and attending workshops organised by the European Commission with National Coordinators (NCO), the European Development Days, the 2015 Development Education Awareness Raising (DEAR) Fair, the UK distribution of the World's Best News, the closing events in Luxembourg and a Kapuscinski lecture.
Constraints
The data gathering process had to contend with a number of constraints. Challenges arose due to (i) Member States in some cases not submitting reports at all, and in other cases providing only limited reporting; (ii) a clash of deadlines between the evaluation and reporting from Member States and CONCORD, which further complicated results analysis; (iii) the absence of a requirement on Member States – even those receiving significant sums - to carry out external evaluations of their national activities, which handicapped the overall evaluation process; (iv) staff rotations within DG DEVCO; (v) additional methodological developments and a more extensive reporting process than initially envisaged; (vi) an unwillingness of interviewees outside DG DEVCO to make themselves available within the desired timeframe; and (vii) the fact that respondents were not invited to give an opinion on other organisations' activities (except those of the Commission), for fear that their overview would not be sufficiently broad, and that they might be deterred from answering the survey at all by the additional time required to complete it.
In certain cases, these challenges were ironed out by mid-year, for example by rearranging interviews and standardising the data collection process. However, the evaluator acknowledges that certain findings, conclusions and recommendations need to be considered in the light of these data collection constraints. In particular, the report draws extensively on a survey of stakeholders who had themselves implemented activities under the campaign, which implies a potential risk of positive bias.
Section 6: Results
When the special Eurobarometer survey on development was published in February 2016, it documented an increase in European public support for helping people in developing countries, from 85% in January 2015 to 89% in 2016. The survey further demonstrated that tackling poverty was considered to be in the EU's interest, while the visibility of EYD2015 itself had also improved,recognition of the campaign having increased by 50% by the end of the year. These outcomes in terms of European public opinion trends were particularly encouraging given the significance of the global policy decisions made in 2015 on development cooperation and climate change.
Overall results
In order to maximise impact and visibility, EYD2015 was implemented in partnership with a wide range of stakeholders. Twenty-five Member States formally participated in the campaign by implementing a national work programme (NWP), while EYD2015-related activities took place across all 28 Member States. A major highlight was the European Development Day celebration organised in 19 Member States between September and October 2015. Key outputs and outcomes of these initiatives included:
1,923,240 participants in 3,828 events in all EU Member States
1,105,372 followers on social media received EYD2015 updates from campaign partners
803,873 unique visitors accessed dedicated websites and sub-sites created by partners
The Kapuscinski Development Lectures in 2015 attracted about 7,000 participants plus 35,000 online followers
The campaign toolkit was accessed 14,684 times via the website
1,864,027 copies of EYD material were disseminated throughout Europe
EYD hashtag: 328,900 mentions over the year
3,914 media articles on development cooperation mentioned EYD2015 in the news
250,000 print versions of 2 newspaper editions put together with the World's Best News
EYD2015 Facebook had 32,000 likes and EYD2015 Twitter had 3,550 followers
Implementation at centralised level
Implementation at centralised level was recognised as critical to the success of a pan-European campaign during the preparatory phase. The global motto of EYD2015 - "Our world, our dignity, our future"
- and the device of assigning a specific theme to each month of the year
provided clear guidance to all concerned in terms of messaging. Key achievements and activities included:
282,900 visits to the official EYD2015 website
Campaign materials were made available for download via the EYD2015 website, wherever possible in all EU languages. These included: two EYD2015 videos; a radio spot; a book publication of all 52 "Stories of the Week"; a monthly newsletter; monthly thematic factsheets, infographics and posters; an EYD2015 (thematic) poster; PowerPoint presentations; two editions of the EYD-World Best News newspaper; an educational toolkit on the EYD2015; a full visual identity toolkit; a photo-bank; roll-up templates; and a limited number of giveaways (pens, notepads, lanyards);
Actions specifically targeting young people, including:
oA photo contest ("Our future is on the table") with a photo exhibition at the European Development Days (EDDs
).
oA selection of 12 young people from across the globe who spoke alongside political and civil society leaders in high-level panels at the EDDs.
oA video in collaboration with a young blogger, Sabina Trojanova, who visited five EU projects in 3 continents in 10 days. It was widely disseminated on social media and in cinemas across Europe in the course of November and December 2015 with the hashtag #changeperspective.
The outcome was that the video had become DEVCO's most viewed audiovisual material on its YouTube channel by January 2016.
oDEAR beneficiaries in all Member States took EYD2015 into account in their 2015 programmes, which included: photo exhibitions/competitions, film and music festivals, writing contests, development summer camps, cultural events, people and teachers' fora.
Flagship activities:
oOpening and closing events (under the auspices of the Council Presidencies).
o28 Kapuscinski Lectures
(as opposed to the usual 10 each year).
oThe Ban Ki-moon interactive event for young people at Bozar, which was web streamed and open to young people to ask questions via #AskBanKimoon. It showed up in the Twitter feeds of over 10 million people in the course of the two-hour event.
oEYD2015 was the main theme of the Open Doors Day by the EU institutions on Europe Day (9 May 2015).
oProminence for EYD 2015 at the European Development Days (EDDs).
oThe EYD 2015 Presidency Gender Event in Riga.
oThe UN70 celebrations in the Grand Place Brussels.
oA European Development Day organised in 19 EU Member States with the support of the Danish NGO World's Best News.
European Commission Representations, EU Delegations and European Direct Information Centres also organised numerous activities.
A Joint Declaration on EYD
including lessons learned for the future was signed at the closing ceremony in Luxembourg in December 2015 by the European Parliament, the Council and the Commission. The Delcaration articulates the commitment of the EU and its Member States, in their communication on development policy, to continuing to:
Build on partnerships and engage new stakeholders beyond the development community
Address young people
Put people at the centre
Be interactive
Be fact-based
Keep the spirit of the EYD
Implementation at decentralised level
CONCORD
led a Civil Society Alliance (CSA) of 60 organisations. The partnership resulted in 417 contributions by CSA members to the EYD2015 website, with 20 posts, information on five events and one press release being uploaded. Fifteen videos of various events and a video by sub-grantees 'You can make a Change'
were posted on YouTube. These attracted up to 200 views per video and were also shown at events. The 17 sub-granted projects, with budgets of EUR 10,000-85,000, adopted a variety of approaches: some were local/regional, some were national, some covered a number of EU countries (11-18), and some were international (including countries outside the EU). CONCORD organised or supported 50 events, including two civil society conferences on sustainable consumption with 60 participants and one on sustainable development at the Milan Expo with 160 participants.
Twenty-five Member States implemented national EYD work programmes (NWPs). A great diversity of activities was organised, with stakeholders reporting on them through the monitoring system. In total, data was gathered on 814 events, 194 press activities, 161 materials, 74 websites, and 70 social media accounts. While - given the decentralised nature of the activities organised at national level and the voluntary nature of the reporting requirements - it was not possible to collect information on all the activities that took place, the Commission identified 3,828 events as having taken place in the context of EYD2015. This figure certainly understates the true scale of the national components of the campaign, as not all partners had been able to report on all activities and events by the time the evaluation was completed.
Section 7: Answers to the evaluation questions
On relevance and complementarity:
To what extent were the activities of EYD 2015 relevant to the needs of the interested parties?
To what extent were the activities complementary with other existing national and EU initiatives?
According to the evidence gathered by the evaluator, EYD2015 was both relevant and complimentary to the work of interested parties and to other initiatives put in place by partners. The activities carried out during EYD2015 at both centralised and decentralised level largely corresponded to the needs of the target audiences, particularly the development community (as important multipliers of our messages to a wider audience), but also the general public and young people in particular (see Figure 1 and 2). However, stakeholders expressed some doubts about the relevance of centralised activities.
Figure 1 - Relevance of the messages disseminated by the European Commission by target audience
Source: The evaluation stakeholder survey
Figure 2 - Relevance of the messages disseminated by partner organisations according to their own assessment
Source: The evaluation stakeholder survey
The issue of policy coherence for development was deemed too difficult to communicate on effectively by partners, although this did not detract from the overall appreciation of relevance of the topic. The fact that EYD objectives were broadly framed allowed Member States to adapt messages to local needs, although some national coordinators noted a certain lack of clarity regarding the balance between communicating on EU and Member State development cooperation. Stakeholders also observed that benefits could have been drawn from linking specific political events in 2015 (such as the migration crisis) to the campaign, for example by explaining the link between development cooperation and migration from a broader perspective.
The creation of a brand and a logo, and the storytelling approach adopted to illustrate development cooperation results were all considered to be relevant and to have enhanced the visibility of development action. Levels of complementarity were high across all types of activity and organisation, including at national level. However, some stakeholders did also note that that the motto and the image were not always a perfect fit with the degree of maturity of their own messaging and the level of awareness of public opinion.
On effectiveness and impact:
To what extent did the activities undertaken in the framework of EYD 2015 improve popular and political support?
To what extent were the activities undertaken in the framework of EYD 2015 effective in achieving its three specific objectives?
To what extent were the activities undertaken in the framework of the EYD 2015 effective in creating a synergistic pan-EU campaign, with EU citizens as the target audience and special attention to youth?
To what extent were EU Member States, European Commission DGs and services and EU Institutions and bodies coordinated and provided with support to organise their own EYD 2015 activities and to participate in the centralised campaign activities?
To what extent were other interested parties such as CSOs, international organisations, Private Sector representatives, Academia and Think Tanks engaged? Was their participation in EYD 2015 facilitated? Were they encouraged to initiate and label own activities as EYD 2015?
To what extent were the activities conducted carried out and outputs produced as planned? To which extent did the centralised activities achieve their expected results?
The stakeholder survey and field interviews conducted by the evaluator suggest that many EYD2015 activities helped increase public support for development cooperation and reached new audiences, including the target group of young people. Interviewees reacted positively to the high level of activity in the Member States and to the absence of negative media coverage. While the campaign was successful in informing citizens and in raising awareness about the mutual benefits of development cooperation, interviews indicate that it was more difficult to foster critical engagement. Looking into survey results in particular, tTwo-thirds of survey respondents felt that the Commission's activities did in fact contribute to improving public and political support in equal measure (see Figure 3). The European Parliament stood out as the institution with the strongest impact on the promotion of the campaign, thanks to the very active involvement of MEPs. This is in line with statements by stakeholders highlighting that the presence of policymakers and politicians increased the visibility and impact of EYD2015 activities in comparison with other European Years. Overall, the Commission had a bigger impact on political support, while the activities of the Member States and civil society had a greater effect on public support.
Figure 3 – Impact on public and political support for EYD activities undertaken by the Commission
Source: Evaluation stakeholder survey
The evidence of successful outreach to the general public as a whole is less clear cut, but there was an organised effort to target young people, and there are a number of examples of good practice across all audiences. The World's Best News initiative by the Danish NGO of that name is the main example of cross-border synergies involving young volunteers. Centralised events are felt primarily to have reached policymakers and the development community, but there were also good results from targeting young people. However, several Member States conceded that many events failed to attract audiences beyond the circles of people already interested in cooperation. This phenomenon is to some extent explained by monitoring system data which shows that 38% of the events on which stakeholders reported targeted civil society professionals and 28% of them targeted development cooperation professionals, while 24% of these events did not have a specific target audience. The figures must also be treated with a degree of caution, as it is not possible to distinguish between numbers for dedicated EYD events and numbers for larger events, such as music festivals or exhibitions, where it is not known how many participants were aware of the EYD angle. That said, effective cooperation among various players benefiting from DEVCO support was widely acknowledged in the field research, which also confirmed that cooperation between DEVCO and with a large number of Commission DGs and other EU institutions was consistently good.
The great number of events and activities organised was effective in providing general information and raising awareness. Online and social media activity contributed most to providing information and to projecting the EU as a global player (see also section 6 of this paper). Website visits peaked in May, before dropping over the summer period, and then peaking again towards the end of the year, when the number of visits reached 43,973 in December (see Figure 4).
Figure 4 – Number of visits to EYD 2015 website
Source: Evaluation stakeholder monitoring
Opening the production of website content up to 208 partners and the co-curation of EU social media accounts (47 partners) were good ways of building new relationships and reaching new audiences. However, there was scope for the website to have greater reach and the commitment of partners to co-curation and alignment with EYD2015 messaging varied. The results of the survey presented in Figure 5 show that respondents believed that the website was most effective in informing citizens (78% of positive opinions), highlighting the results of the EU as a global player in development policy (77% of positive opinions) and making citizens more aware of the benefits of development aid (71% of positive opinions). Survey results from website users further support the thesis that the original content (notably Stories of the Week and themes of the month) was the most popular attraction for visitors.
Figure 5 – Effectiveness of the EYD 2015 website in meeting the objectives
Source: Evaluation stakeholder survey
New relationships were also forged with international organisations, academia and the private sector, all of which were effective ways of reaching a wider audience than in the past. As an example, a special preparatory workshop organised for the private sector in 2014 acknowledged that the EU needed to work more closely with private sector partners to promote development policies. The 20 new partnerships created by the end of the campaign was a positive outcome in this respect. Private sector partners saw benefits in raising their visibility with the Commission and some believe this paved the way for a closer relationship in the longer term. Member States also made successful efforts to build new alliances, including leveraging visibility from organisations that did not receive EYD2015 funding.
As a result of the efforts of the Commission and of national coordinators, levels of press coverage were good. However, the evaluation found that as a result of a lack of clarity about roles and responsibilities, there had been difficulties in using the contractor's network to complement this coverage. Media monitoring nevertheless registered some 3,914 articles mentioning EYD2015, of which 70% were published online and 30% appeared in print. These media outlets had an estimated total readership of one billion. Eighteen percent of the articles monitored covered one or other of the 12 monthly themes, with an average of 82 articles per topic, with particular attention dedicated to education (126 articles), women and girls (115 articles), sustainable development and climate change (99 articles), health (96 articles) and food security (91 articles). The countries where coverage was greatest were Spain, Italy, Portugal, Germany and Romania.
The common motto/visual identity, the website and the campaign toolkit provided by the European Commission were considered highly valuable contributions, scoring approval ratings of 89%, 80% and 80% respectively in the stakeholder survey (see Figure 6).
Figure 6 – Contribution of different elements to a synergistic pan-EU campaign
Source: Evaluation stakeholder survey
On efficiency, coherence and EU added value
How efficient were the activities undertaken in the framework of EYD2015 in conveying the appropriate messages and achieving the expected results?
Were the actions undertaken properly coordinated and coherent with other actions?
As no comprehensive data was available for the evaluator to analyse due to the constraints identified in section 5 of this paper, it is possible to draw only cautious conclusions with regard to the findings on efficiency, coherence and EU added-value.
The partnership approach was an efficient way of empowering national and local stakeholders to use funds to generate interest through specific activities and to reach out to the general public and to young people in particular. Field interviews suggest that partners believe that they achieved a great deal with little money and that the Commission had consequently made good use of financial resources available (see Figure 7). Relying partly on partners to produce content for EYD2015, most notably through social media co-curation, proved to be the right approach; the number of co-curation applications received exceeded the available slots, which also resulted in an increase in the number of organisations that partnered with the website. However, some stakeholders did suggest that there were limits to the efficiency of the partnership approach, with some NGOs feeling that it would have been more efficient to carry out an EU communication campaign with an external/independent public relations agency if the intention was to promote EU messages.
Figure 7 – Extent to which Commission activities made good use of financial resources
Source: Evaluation stakeholder survey
The partnership model adopted for EYD2015 required good coordination at EU level, and stakeholder survey respondents felt that human and financial resources were managed efficiently, both centrally and through the delegation of responsibility for national campaigns to the Member States (see Figure 8). Coordination worked well, even if "cruising speed" was only reached during the second half of the Year. There were some initial difficulties in the run-up of the implementation of EYD2015 partly due to understaffing, but in general coordination by the European Commission (EU added value) was rated highly. Some 75% of stakeholder survey respondents thought that Member States, Commission DGs and services, and other EU institutions and bodies received good or excellent support from the DEVCO EYD2015 team in organising their own EYD 2015 activities, while 67% indicated that this had also been the case for their own participation in centralised campaign activities.
Figure 8 – Value added of EU coordination
Source: Evaluation stakeholder survey
Finally, the monitoring requirements were considered burdensome by some funding recipients, who mentioned that they would have preferred more advance warning in order to incorporate these into their work programmes and budgets.
On sustainability
To what extent has EYD 2015 been successful in delivering sustainable outcomes in relation to its three specific objectives?
The building blocks for sustainability have largely been put in place by EYD2015 partners. There are already examples of lessons learned for the future, of networks which wish to build on the work done, and in some cases funding has already been made available. The Commission organised a first post-EYD2015 meeting of development communicators from the Member States in June 2016 to preserve the legacy of the campaign and to carry the momentum and the partnerships created forward into concerted communication work on the the new challenge of the 2030 Agenda for Sustainable Development. The policy foundations are in place in the form of the recommendations of CONCORD/CSA and the Joint Declaration by the European Parliament, Council and Commission on the Legacy of the EYD2015, which paves the way for sustained communication outcomes with all stakeholders at local, national, European and global level.
From a partner's perspective, one of the key sustainable benefits of the EYD campaign identified by survey respondents at Member State level was learning to work beter with the media. Some stakeholders (UK, Poland) also highlighted their role in reaching out to decentralised or rural areas in ways or to an extent that had not happened before, e.g. by having a more citizen-focused website (Estonia), learning to communicate better with young people, the diaspora, activists and other new target groups (Finland, UK), organising competitions and educational initiatives (Slovakia) and new forging cross-border partnerships (between Germany and neighbouring countries and between Sweden and Estonia). Civil society stakeholders particularly highlighted that EYD2015 had "opened a lot of doors", enabling NGOs to work with institutional partners to whom they had previously had limited access. Stakeholders assessed the dissemination of EYD results positively, with a 68% approval rating for the Commission's efforts in this area and 85% for their own.
The evaluator notes the consistent willingness of stakeholders to preserve the content generated during the campaign. One area that partners feel to be in need of more sustainable results is education, where there is an as yet unmet necessity for educational platforms based on thorough information, which could help teachers enrich their lessons and raise awareness about development cooperation among their pupils.
Section 8: Conclusions
The evaluation concludes that the EYD2015 campaign largely met its objectives. On the basis of the evaluation criteria (and bearing in mind the constraints identified), we can assert that the campaign established a solid foundation for sustained improvement in development communication by the EU and its partners.
As regards relevance and complementarity, EYD2015 was successful in helping to raise awareness, and to foster knowledge and understanding among the general public about development cooperation. It proved beneficial to adopt a particular focus on young people and to have timed the Year to help develop a broader sense of engagement with, and ownership of, the Sustainable Development Goals. It was appropriate to create an overarching framework for communication on development cooperation with broad objectives and target groups, allowing Member States and others the freedom to tailor content and activities to the needs of their own particular environments.
The evaluation report shows that all EYD2015 activities were relevant to at least one of the objectives and at least one of the target groups of the campaign, and were complementary across the European Commission, the EU institutions, international organisations and at national level. Various types of coordination mechanisms, generally set up well in advance, helped considerably in identifying and complementing existing initiatives.
On effectiveness and impact, the objectives of providing general information and raising awareness about the benefits of development cooperation and of the EU as a global player were effectively met. The more ambitious objective of stimulating critical thinking and explaining policy coherence for development to the public proved more of a challenge.
EYD2015 led to the creation of new partnerships and exchanges of good practice at European and national level. The creation of a single, unifying logo/label, motto and toolkit was effective in creating a corporate and pan-European identity. The brand was strong and even unfunded partners were motivated to use it and commit to the objectives of the Year. The device of assigning a specific theme to each month of the year undoubtedly enabled partners, particularly NGOs, to communicate more effectively. Opening the website and social media accounts up to partners was an innovative and effective way of consolidating existing partnerships and communities and of building new ones. Thanks to the combined efforts of the Commission, national coordinators, NGOs and the external contractor of the EYD2015 campaign, press coverage was often greater than might have been expected. Coordination across major players, and the work done by the Commission to create partnerships with other EU institutions, international organisations, and civil society was effective, with many new relationships established.
On efficiency, coherence and the EU's added value, the partnership approach is perceived to have been efficient in reaching the development community and civil society as multipliers; while new audiences and partnerships were created among the general public, and young people in particular, as well as among international organisations and the private sector. The European Development Day and the World's Best News initiative that took place in 19 countries in September and October 2015 was a good illustration of how pan-European synergies were achieved. After 2014 a changeover year for both the Parliament and the Commission, coordination between the European Commission and its partners, especially Member States, improved significantly as more staff were deployed. However, staffing levels remained lower than expected, which limited the scope of the Commission to increase its visibility at Member State level. The monitoring requirements were useful, but suffered from being viewed as burdensome, partly owing to a lack of initial clarification, which would have allowed partners to plan and budget better.
In terms of sustainability, new working methods were developed and the Commission programme DEAR took root for the first time in some Member States, although there was scope for more and better dissemination of results. Engagement with the media was one of the key challenges of the year, and the new contacts established and the new structures and networks set up will bring benefits in future. The foundations were laid for sustainable outcomes, notably in the form of a Declaration from the European Parliament, the Council and the Commission, but there is a need to ensure that the impetus is sustained and that the requisite energy and resources allocated to taking the EYD legacy forward into the new challenges of the 2030 Agenda for Sustainable Development. Partners are committed to preserving and consolidating the gains and the media partnerships made in the course of EYD2015, and to continuing to work closely with the Commission on development-related communication.
Annex 1: Procedural information
The independent evaluation report of the EYD2015 communication campaign was carried out for the European Commission by the external contractor Coffey & Deloitte under SPECIFIC CONTRACT n° DEVCO-05/17/2014 implementing the Multiple Framework Contract with reopening of competition n° PO/2012-3/A3/TEP - Lot 1. The contract was signed on November 2014 for 20 months.
Organisation and timing
The evaluation consisted of three parts:
1.Evaluability study
2.Interim evaluation
3.Ex-post evaluation
A European Commission Steering Group was established, made up of members from DG DEVCO and DG COMM.
1 December 2014: First meeting (Kick-off/Inception). This meeting enabled the evaluation team and the European Commission to:
oClarify the objectives and the scope of the assignment;
oDiscuss the enlarged EYD 2015 stakeholder workshop;
oExchange information and documents relevant to the evaluation process;
oAgree on the organisation of individual meetings with the members of the Steering Committee.
Individual meetings were then organised with DG DEVCO and DG COMM officials (Steering Group), as well as with Media Consulta and CONCORD, in order to deepen the understanding of the evaluation team of their functions, activities and objectives.
15 December 2014: First ad-hoc meetings with Steering Group members and evaluation team.
19 January 2015: Second ad-hoc meeting to discuss updates on the progress of the implementation of the EYD2015 and feedback on the draft notes on the website, events and press relation, and to validate the outline of the "evaluability" report and the outline of the intervention logic.
Preliminary desk research
Following the initial interviews and based on the information received, a first set of documents was analysed by the evaluation team, particularly relating to the activities to be undertaken by CONCORD, Media Consulta, and the national work programmes of the EU Member States participating in the EYD2015 campaign.
Refined methodological approach:
The proposed methodology was refined in close collaboration with the European Commission and was based on the kick-off/inception meeting, the preliminary interviews, and the initial desk research.
Monitoring and reporting mechanisms including guidelines and templates:
Following the preliminary desk research and interviews, the evaluation team submitted two draft notes concerning the key monitoring and evaluation aspects to be taken into account when working with the website, organising events and dealing with the press, as agreed during the Kick-off Meeting.
The monitoring and reporting mechanism in the National Work-Programmes were analysed after having asked additional questions, notably to the national coordinators.
Evaluability report:
Consensus was reached on the final version of the "evaluability" report on 6 February 2015, after a meeting with the Steering Group and the evaluation team.
Interim evaluation:
A mid-term report was submitted in July 2015, after a meeting with the Steering Group in which both parties agreed on the templates for the stakeholders.
Ex-post Evaluation Report:
The work was conducted between January and April 2016, and the first draft report was submitted in May 2016. The final version was adopted in June 2016 after two meetings with the Steering Group, one held on 23 May 2016 and the second one on 14 June 2016.
Chronology of the evaluation procedure between September 2015 and July 2016:
Evidence used in the evaluation:
Coffey and Deloitte, the independent evaluators of the EYD2015 campaign, consulted all partners. The work was conducted between 1 January 2016 and 5 April 2016.
Sources used were:
oInterviews with EYD2015 partners:
DG DEVCO project officers and contractors in charge of the EU-wide campaign;
Other DGs and EU institutions;
EU delegations;
International organisations;
European Civil Society Alliance of NGOs, "CONCORD";
DEAR (Development Education and Awareness Raising) Beneficiaries;
Private sector
MediaConsulta
oFieldwork in 6 EU Member States that included the financial and narrative reports of each Member State on their work during the EYD2015 campaign as well as interviews with national coordinators and/or national beneficiaries:
Germany;
Spain;
Hungary;
Luxembourg;
Poland;
United Kingdom
oSurveys
1.Stakeholder survey: A total of 216 replies were collected from the following stakeholder categories:
European Commission
EU institutions other than European Commission
International Organisations
Member States and their sub-grantees
NGOs funded by the EC
Other (mostly EDICs)
EYD stakeholder survey respondent profile
Source: The evaluation stakeholder survey
2.EYD2015 website survey: EYD2015 website users were surveyed about their profiles and their views on the website. The survey was carried out between 12 October 2015 and 15 January 2016 under the title 'We want to hear from you!', and a total of 183 responses were collected.
Annex 2: Stakeholder consultation
The European Commission extensively consulted all partners and stakeholders both during the campaign in 2015 and afterwards, in 2016. The systematic and comprehensive work undertaken by Coffey and Deloitte (C&D) through reporting templates, interviews and fieldwork required the extensive commitment of all partners and stakeholders to reporting and evaluating the EYD2015.
The entire endeavour of the EYD2015 campaign was built on a cooperative approach which entailed constant consultation with the partners and stakeholders before, during and after the Year. The highly participatory approach was one of the main underlying principles in the planning, implementation and evaluation phase. The co-curation of the EYD2015 social media by a different partner every week, the co-production of materials and the fact that 80% of the content of the website was filled by EYD2015 partners was a clear indication of their full involvement.
1.Interinstitutional and interservice consultations:
Throughout 2015, a formal EYD2015 Interinstitutional Working Group and an EYD2015 Interservice Working Group provided a platform for constant communication and consultation on the progress of the implementation of the campaign.
Specifically on the EYD2015 Interinstitutional Working Group:
oParticipants were representatives of the European Commission services, the European External Action Service, the Council, the European Parliament, the Committee of the Regions, the European Economic and Social Committee and the European Civil Society Alliance (CONCORD)
oEight interinstitutional meetings were held in 2015: (6 January; 28 January; 19 March; 5 May; 29 June; 3 September; 26 October; 9 December)
Specifically on the EYD2015 Interservice Working Group:
oMembers were all from European Commission services and the meetings were organised by DG COMM
o2 Interservice meetings were held: (5 February; 25 June)
oIn addition, 16 bilateral meetings were held in 2015 with various EC services with the purpose of exchanging information and agreeing on synergies.
2.Ex-post evaluation report:
Coffey and Deloitte, the independent evaluators of the EYD2015 campaign, consulted all partners. The work as conducted between 1st January 2016 and 5 April 2016.
Sources used were
:
oInterviews with EYD2015 partners:
DG DEVCO project officers and contractors in charge of the EU-wide campaign;
Other DGs and EU institutions;
EU delegations;
International organisations;
European Civil Society Alliance of NGOs (CONCORD);
DEAR (Development Education and Awareness Raising) beneficiaries;
Private sector;
MediaConsulta
oFieldwork in 6 EU Member States that included the financial and narrative reports by each Member State on their work during the EYD2015 campaign and interviews with national coordinators and/or national beneficiaries:
Germany;
Spain;
Hungary;
Luxembourg;
Poland;
United Kingdom
oWeb-based surveys:
1.EYD Website
2.National Coordinators
3.Stakeholders involved in the national work programmes
4.CONCORD members/grantees
oMonitoring reports from DG DEVCO, from DG COMM and from MediaConsulta.
3.EU Member States and national work programme reports:
Participating Member States (25 in total) reported on their activities and work using the detailed templates established for this purpose.
Regular workshops held in the Member States in 2014, 2015 and 2016 contributed to constructive consultation on the progress of the campaign.
Specifically, the dates of the Workshops were: 21 May 2014; 9-10 October 2014; 5 March 2015; 2 September 2015; 9 December 2015; 14 June 2016
4.Public and stakeholder survey:
Two surveys were carried out: a comprehensive survey of stakeholders, which aimed at obtaining their views on the Commission’s and on their own activities, as well as a survey of the public on the europa.eu/eyd2015 website, in order to obtain a snapshot of users and their views.
Stakeholders survey:
Responses were received from (i) Member States, (ii) EU institutions, (iii) EC Representations, (iv) DEAR beneficiaries, (v) international organisations, (vi) EC DGs and (vii) CONCORD
Public survey at the EYD2015 website:
o183 respondents.
oCarried out between 12 October 2015 and 15 January 2016
oTitle: WE WANT TO HEAR FROM YOU!
oOpen to the public and available in all language versions of EYD2015 website
5.DG DEVCO preliminary report:
A preliminary (flash) report with an accompanying infographic was produced in mid-January 2016 and was adopted by the management of DG DEVCO. It was widely distributed with the purpose of informing all involved partners about the achievements of the European Year. This flash report included input from a wide variety of sources, such as DEVCO Units, European Commission services and other EU institutions, CONCORD, Media Consulta and Member States.
6.DEVCO bilateral meetings with partners:
DEVCO met numerous EYD2015 partners in 2016, both to present its preliminary report and also to exchange views and receive feedback on how to move forward with a strategy on communication for development:
Commission:
oparticipation at the External Communication Network (ECN) of the EC and presentation/discussion with communication officers (2 February 2016)
opresentation of the audiovisual materials produced during the campaign to the Audiovisual Network of the EC run by DG COMM (25 February 2016)
EU institutions:
oPresentation/discussion of the preliminary report at the CODEV group of the Council (11 February 2016)
oPresentation of the EYD2015 preliminary report results to the European Parliament and discussion with the DEVE committee on the outcome of the campaign (18 February 2016)
oPreliminary report of the EYD2015 campaign distributed to the Information Group of the Council (23 February 2016)
Member States: bilateral meeting in Brussels with the UK national beneficiary, BOND (12 February 2016)
NGO: meeting with CONCORD, the European Civil Society alliance, in their Brussels office to discuss the campaign, talk about the initial results and agree on next steps (8 March 2016)
Young people/academia: presentation of the results of the EYD2015 campaign at the College of Europe in Bruges (16 March 2016)
International organisations: discussions with the UN Regional Information Office in Brussels, a key partner of the EYD2015 campaign, were held on 23 March 2016.
Throughout 2015, the UN and its Brussels-based agencies were key partners of the EYD2015 and regular meetings were held to coordinate actions, such as the UN Cinema Festival and the UN70 celebrations in the Grand Place.
In the spirit of consultation with all interested parties, consultation rounds on the setup of the EYD2015 campaign were organised twice in Brussels in 2014.Specifically:
the 1st stakeholder event took place in June 2014.
the 2nd stakeholder event took place at The Egg venue in Brussels (9 December 2014) with a large number of participants (over 150).
7.Eurobarometer 441 (2016) on Development:
The Commission published the Eurobarometer survey on Development and the European Year for Development on 29 February 2016. The results of the Eurobarometer survey regarding the views and attitudes of the public towards development and the EYD2015 were very positive and encouraging. All key indicators showed an improvement.
Key findings included the following:
o1 in 5 European Union citizens was aware of the EYD2015
o1 in 3 European Union citizens was aware of the Sustainable Development Goals
o9 in 10 European Union citizens say it is important to help people in developing countries
o8 out of 10 European Union citizens agree tackling poverty in developing countries is also in the EU's own interest
o7 out of 10 European Union citizens view development aid as a way to tackle irregular migration
8.EC Representations and European Information and Documentation Centres:
A report to showcase the involvement of the EC Representations in all EU MS was produced by DG COMM in March 2016.
9.Other reports by partners:
Several other reports were produced by partners and were either sent directly to the Commission or made public:
CONCORD:
oPlease find more information on the project, its achievements and the materials produced, especially its 17 successful sub-granted projects here:
http://concordeurope.org/project/european-year-development-2015/
oYou can find more information on the final report by the independent external evaluator Alecos Kelemenis here:
http://concordeurope.org/2016/05/03/eyd2015-learnings-legacy/
World's Best News:
The Commission also received the report of Danish NGO World's Best News on the joint action to hold a European Development Day in 19 EU MS between September and October 2015.
MediaConsulta (MCgroup):
The external contractor of the European Year sent mid-term and flash report on progress during the campaign.
