EUROPEAN COMMISSION
Brussels, 25.4.2018
COM(2018) 235 final
REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS
FMT:BoldEx Post evaluation of the 2016 European Capitals of Culture (Donostia San Sebastian and Wroclaw)/FMT
{SWD(2018) 140 final}
REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS
Ex Post evaluation of the 2016 European Capitals of Culture (Donostia San Sebastian and Wroclaw)
1.Introduction
This report is presented in accordance with Article 12 of Decision No 1622/2006/EC of the European Parliament and of the Council of 24 October 2006 establishing a Community Action for the European Capital of Culture event for the years 2007 to 2019 1 , which provides that the Commission ensures each year an external and independent evaluation of the results of the European Capital of Culture event of the previous year and reports on that evaluation to the European Parliament, the Council and the Committee of the Regions 2 .
The findings and methodology of the ex post evaluation are presented more comprehensively in the accompanying Commission Staff Working Document.
2.Background to the Action
2.1.The EU Action for the European Capital of Culture (ECOC)
Since the launch – at intergovernmental level – of the European City of Culture in 1985 3 , the scheme developed until reaching the format of a fully-fledged EU Action in 1999 4 . It is currently governed by Decision No 445/2014/EU 5 , but cities which were designated as ECOC for the years up to 2019 were regulated by Decision No 1622/2006/EC.
The ECOC Action is designed to highlight the richness and diversity of cultures in Europe and the features they share, thereby encouraging a greater mutual understanding among European citizens. It is also meant to stimulate a long-term culture-based development of cities in the broader perception of the term, which entails socio-economic impacts, strengthening of cooperation between cultural operators, artists and cities in Europe, as well as local (and foreign) citizens' involvement and participation in culture.
2.2.The selection and monitoring of the ECOC 2016
In accordance with Decision No 1622/2006/EC, Spain and Poland were the two Member States entitled to host an ECOC in 2016.
The two-phased selection process (shortlisting and final recommendation) was carried out in parallel by the relevant authorities of these two Member States (i.e. their respective Ministries of Culture). A panel of thirteen members – six of whom nominated by the Member State concerned and the other seven by European Union institutions and bodies – examined the bids submitted by candidate cities on the basis of the objectives and criteria laid down in Decision No 1622/2006/EC. Fifteen cities in Spain and eleven cities in Poland entered the competition in 2010 and in 2011 the panel recommended that the title of ECOC 2016 be awarded to Donostia San Sebastian and Wroclaw 6 . The two cities were formally designated by the Council of the European Union in May 2012 7 .
Subsequently, both cities were subjected to monitoring arrangements: the progress in the cities' preparations was monitored and guided by a panel consisting of the seven independent experts appointed by the EU institutions and bodies, which had the additional task of ensuring compliance with the programme and commitments on the basis of which the cities had been selected. The representatives of Donostia San Sebastian and Wroclaw attended two formal monitoring meetings convened by the Commission, in autumn 2013 and spring 2015. Upon completion of the monitoring process, the panel made a positive recommendation to the Commission to award a €1.5 million prize in honour of Melina Mercouri to each of the two cities. The pecuniary prize – funded under the Creative Europe programme 8 – was paid to the two ECOC in the autumn of 2015.
2.3.The themes and focus of the two ECOC 2016
Donostia San Sebastian's application was entitled "Cultura para la convivencia" (Culture for coexistence) and had a strong locally-oriented connotation: it was elaborated stemming from the recent past of the city, which was scarred for decades by terrorist activities.
As the city's cultural offer and the capacity of its cultural sector were already very developed at the time of the application, the focus was more on endorsing qualitatively different projects addressing sensitive topics such as violence or terrorism with the aim of helping citizens to find a sense of reconciliation with their past.
The programme's foundations were laid on the two concepts of "Co-existence" and "The City". The former was articulated around the three axes of "Lighthouse of Peace", "Lighthouse of Life" and "Lighthouse of Voices", dealing respectively with integration and coexistence; health, work and environment; and communication. The latter concept worked transversally by providing methodologies – in the fields of sustainability of linguistic diversity, critical thought through contemporary art, citizens’ engagement, connection between culture and technology, and the development of interactive situations in cultural spaces – which were meant to help all individual projects to achieve their own objectives.
Wroclaw's application emerged from the wider strategy of the City of Wroclaw, which emphasised the potential for urban development to be supported by prioritizing investments in cultural and sporting events and infrastructures.
"Metamorphoses of Culture" was the overall concept adopted, which served as a metaphor for both the historical transformation of the city and contemporary processes of cultural and social change (including globalisation, immigration, EU enlargement and the growing role of digital communications). "Spaces for Beauty" was used as the slogan for the ECOC programme as the intention was "to create spaces within which to restore the presence of beauty in public life and in daily habits".
Wroclaw came up with a very clear vision, especially concerning long-term goals, that ranged from increasing awareness and recognition of the Wroclaw/Lower Silesia's cultural identity, to creating public spaces for social activities and shaping of civic attitudes, to increasing the number of visiting tourists. Cultural events were organized in disciplinary sub-programmes (e.g. architecture, film, opera, etc.) and articulated on four "stages" depending on the target, varying from dialogue with local inhabitants to showcasing the city's role in the European and world culture.
3.The external study
3.1.The terms of the evaluation
The evaluation explores the implementation and delivery of the two ECOC 2016 programmes throughout their lifecycle, from their early inception through to sustainability and legacy considerations.
Specifically, it assesses the relevance, efficiency, effectiveness and sustainability of the two ECOC 2016, and examines the EU added value and the coherence and complementarity of the Action to other EU initiatives. Finally, it draws individual and general conclusions emerging from the two experiences and considers implications for future ECOC title-holders, applicants and EU institutions.
3.2.Methodology and limitations of the approach chosen
The evaluation and its methodology were designed to satisfy the requirements of Decision No 1622/2006/EC, and contribute to develop a more detailed understanding of the performance and achievements of the Action. In particular, it constitutes a valuable opportunity to critically reconsider the past year in order to highlight lessons and recommendations for reshaping current wisdom and insights in the light of the new experiences of the host cities.
As for ECOC 2007-2015 previous evaluations, the intervention logic used by the evaluator is based on a hierarchy of objectives corresponding to Decision No 1622/2006/EC.
In order for results to be comparable, the methodology for this evaluation followed the approach for evidence gathering and analysis adopted in previous assessments of the Action 9 .
The evaluation was grounded in two types of data and respective sources:
-Primary data included either data collected during fieldwork or provided by each ECOC such as interviews, online questionnaires and surveys; interviews in particular sought to gain a variety of perspectives on each ECOC, including those of the management teams, decision-makers at local and national levels, key cultural operators, a range of partners involved in the delivery of the programme and a sample of organisations either leading or participating in the actual projects;
-Secondary data sources encompassed EU documents, original bids and applications, internal reports linked to the application processes, monitoring and evaluation reports, studies and reports produced or commissioned by the ECOC, events programmes, promotional materials and websites, statistical data on culture and tourism and quantitative data supplied by the ECOC on finance, activities, outputs and results;
-The evaluation does not include a wider public consultation. As explained in the roadmap 10 , the Action is considered to be local. International participation is scattered within and outside Europe and is difficult to reach. On the other hand opinions expressed would be based on attendance to specific events and would not give useful insights for the evaluation of the ECOC Action as a whole.
As was the case with all previous ex-post ECOC evaluations, the Commission maintains that the adopted methodology is appropriate to produce a report providing a reasonably solid basis on which sensible conclusions may be inferred regarding the ECOC performance.
However, the lack of baseline data to be integrated in a comparative study of the city prior to the win of the title, start of title year and after the implementation of the ECOC year is a limitation. These data are considered of paramount importance in order to achieve a balanced perspective, supported by a cogent and ample data basis, of the actual impact of the Action on a city. However the budget allocated to the evaluation work (approximately 70 000 € each year) is proportionate to the low level of EU funding directly provided to the ECOC (€1.5m Melina Mercouri Prize) and doesn't make it possible to have a before ('baseline') study and an after-picture ('ex-post') study. An ulterior consequence of the modest budget is that the primary evidence data gathering tends to be more of qualitative than quantitative nature; while qualitative data still holds a great importance in the evaluation, lack of quantitative data translates into a lesser dependability, for instance, in the process of proving the objective outcomes and impacts of ECOC on widening participation in culture.
Therefore, the report's conclusions are substantiated by an ample basis of qualitative and "soft" data (e.g. the views and opinions of various types of stakeholders) more than by a comprehensive quantitative set of data.
The Commission is fully aware of – and accepts – those limitations, which had been already identified and communicated in a staff working document accompanying the proposal for a Decision establishing a Union Action for the ECOC for the years 2020 to 2033 11 . With regard to this difficulty, a subsequent Commission's proposal and the Decision ultimately adopted by the European Parliament and the Council 12 foresee that the designated cities themselves – better positioned to have baseline data and gather primary data on the impact of the title – become the main implementers of the evaluation process.
In this respect it is worth underlining that Donostia San Sebastian and Wroclaw commissioned local research activities 13 – respectively by hand of the San Sebastian Foundation and Wroclaw University’s Institute of Sociology – to ameliorate their understanding of the impact of the ECOC on key actors such as cultural institutions, cultural and creative industries and citizens, as well as on indicators like international dimensions and economics.
In conclusion, despite the deficiency of quantitative data and other independent evidence, the Commission finds a sufficient solidity in the evidence gathered to support the evaluation and shares its overall assessment and deductions, which are considered to provide a generally true and complete picture of the ECOC 2016 Action.
4.Main findings from the Evaluation Report
4.1.Relevance of the ECOC Action and the ECOC 2016
According to the findings of the evaluation, the two host cities have elaborated and implemented cultural programmes which were consistent with and relevant to Article 167 of the Treaty on the Functioning of the European Union as regards Union's contribution to the "flowering of the cultures of the Member States". Hosting the ECOC also contributed to economic and social developments in the two cities, particularly within the urban context, which is also consistent with the aims of Decision No 1622/2006/EC.
The evaluation concludes that Donostia San Sebastian's programme reflected local preoccupations and context, with a strong social and political focus. As the pre-existing cultural offer of the city was already very solid, with a high number of international visitors and large European-wide festivals, it was decided to use the ECOC to cultivate stronger connections with new European cultural players – particularly for smaller cultural organisations – but, most prominently, to deliver with dedication a new type of sensitive content relevant to the complex identity of the city and the social and individual well-being of its inhabitants, which also have a clear European dimension.
Wroclaw articulated a programme with sharp long-term objectives, a strong European dimension and coherently interacting with the wider city development strategy; it focused on further strengthening the European character of the city, attracting domestic and international tourists, improving its cultural infrastructures, increasing audiences and pro-active citizens' participation in cultural activities. The programme also explored some of the most difficult elements of its recent history, notably the population transfer between the Soviet Union, Poland and Germany that took place at the end of World War Two.
4.2.Efficiency
Overall, the available evidence (mainly qualitative) suggests that ECOC remains an efficient EU Action providing good levels of returns at EU level for a relatively modest EU investment: the awarding of the title itself has a substantial leverage effect on the amount of funding that host cities dedicate to designing and delivering the ECOC cultural programme, and it is a significant generator of interest and financing from a broad array of stakeholders, including regional and national authorities and private contributors. Moreover, the absolute value of the Melina Mercouri Prize, which is the only monetary contribution that host cities receive from the European Union, is minimal (€1,5m per ECOC) in comparison to the overall costs of an ECOC: the operating expenditure of the ECOC 2016 were of approximately €86,4m for Wroclaw and €49,6m for Donostia San Sebastian.
At city level, both Donostia San Sebastian and Wroclaw established very strong and successful delivery mechanisms and governance arrangements, and both used national and EU funds (e.g. European Regional Development Fund) to implement cultural programmes of high artistic quality and of considerably greater size than usual.
Nevertheless, the evaluation also shows that hosting an ECOC remains a resource-intensive choice that may turn out be challenging: for instance, Donostia San Sebastian saw a significant reduction in its actual budget compared to the original bid stage (a fall of over 40%), driven by an overly ambitious original proposal, but also by very tight pressure on public budget sectors at all territorial levels, political pressure to prioritise budgets on other city issues and a lack of capacity to seek out private sector funding coupled with poor economic conditions.
4.3.Effectiveness
Both ECOC title-holders have proved successful in fulfilling the short-term objectives set out in their applications, most notably the implementation of extensive and innovative cultural programmes with a European dimension and with a pronounced citizens' involvement. Despite having entirely dissimilar visions, programmes and implementation styles, both used the ECOC effectively to explore and articulate themes of local interest with a European resonance.
Donostia San Sebastian supported activities that were clearly different from its pre-existing cultural offer; ECOC was used to tackle a complex and sensitive issue – namely past violence and the differences that the city's communities have with one another – as opposed to simply being about putting on a cultural programme to entertain local and international audiences or promote the city. While recognising this was a "brave choice", the evaluation also admits that it is difficult to quantitatively measure and understand the future societal impacts of such a choice. While Donostia San Sebastian was not preoccupied with raising its already significant international profile through international communication and tourist attraction, figures nevertheless show that a total of 1.08 million people attended ECOC events, with 5-10% being international visitors.
On the other hand, local cultural operators experienced a remarkable network expansion: almost 80% of the organizations involved in the year cooperated with other organizations, in the majority of the cases building new professional relationships that were unlikely to have taken place without the ECOC year.
Wroclaw implemented an extensive cultural programme as an integral part of its wider development strategy for the city, but also proved particularly successful at attracting international visitors: according to research undertaken on behalf of the City of Wroclaw, around 5 million tourists visited Wroclaw in 2016, of which 1.6m came from a foreign country. Data from the Central Statistical Office of Poland also shows that an extra 50,000 international tourists stayed in Wroclaw's hotels in 2016 compared to 2015. The ECOC has also been effective in strengthening cultural co-operation and networks within the city and with international operators: two thirds of the implemented projects had an international connection and reported collaborations in a variety of forms with organisations or artists in 42 other countries – mostly European, but including also countries such as Brazil, Japan and the USA. The communication strategy in general was also effective, so with about 5,500 press articles published about the ECOC year in Poland and 38 other countries in 2016.
These benefits would have been unlikely to arise to the same extent in the absence of ECOC designation; in that sense, the ECOC action has generated clear European added value.
4.4.Sustainability
The evaluation comes to the conclusion that the two 2016 ECOC offer very different potentials for the sustainability of their activities and of improved cultural governance.
For what concerns Donostia San Sebastian, legacy planning of the ECOC has been generally limited, with a lack of a clear formal strategy from the outset and a deficiency of joint understanding by the political parties on what an ECOC legacy should look like.
Although the cultural operators are generally still in place and still delivering strong cultural content beyond 2016, this is more due to the strength of the existing cultural footprint of the city, rather than any specific legacy from the ECOC programme.
On the other hand, legacy planning was much developed and coordinated in Wroclaw, and offered greater potential for the long-term sustainability of the impacts of the ECOC – also thanks to a strong and linear political leadership. This strategy is also being informed by the various research and evaluation activities undertaken by the University of Wroclaw during the ECOC year. The ECOC delivery structure and the City's Departments for Social Affairs and for Culture have put forward a proposal for a strategy for 2017-2020 and beyond: "Culture – The Current!" (Kultura – Obecna!). It will moreover maintain in action the project "mikroGRANTY", a grant support initiative with the objective of building local cultural capacity.
4.5.Coherence
The ECOC Action is relevant and complementary to a variety of EU policies and programmes, impacting not just cultural stakeholders but also those related to employment (with its impact on capacity building for example), enterprise (as an instance, 14% of firms from the cultural and creatives sectors in Wroclaw were involved in the ECOC, i.e. about 450 out of +3,000 firms, while 52% of them felt that they had derived commercial benefits from the ECOC and 40.7% reported an increase in turnover during 2016) or tourism (an extra 50,000 international tourists stayed in Wroclaw's hotels in 2016 compared to 2015). It can also give impetus to investments in cultural infrastructure co-funded by European Regional Development Fund, as exemplified by the considerable success met by the newly opened ERDF co-funded Music Forum concert hall in Wroclaw (with more than 500,000 visitors in 2016).
4.6.EU added value
As already mentioned and illustrated above, the ECOC Action has achieved an impact that would not have arisen through the actions of Member States alone.
The 'label' itself is a key aspect of the EU added value of this Action as it acts as a significant generator of interest from stakeholders not only from the city and but also from far beyond and offers great scope for European cooperation in terms of partnership and transfer of good practices, for example in terms of building a solid governance for the delivery of the ECOC, increasing the capacity of local cultural organizations or attracting projects' ideas from local residents.
5.Main recommendations as well as conclusions and actions from the Commission
The Commission concludes from this report that the ECOC Action remains relevant at EU level as well as greatly valuable for host cities, and generates extensive cultural programmes with positive outputs and impacts which cannot, however, be fully assessed at the current evaluation stage as it is too early after the implementation of the ECOC year. As previously mentioned only evaluations carried out by the host cities themselves could provide a more reliable picture in this regard.
Furthermore, the Commission reckons that the programmes implemented by the two 2016 title-holders:
-were innovative and consistent with the objectives of the ECOC Action,
-reflected its European dimension (in particular Wroclaw, to a lesser extent Donostia San Sebastian as the ECOC in the latter was much more focused on local residents),
-involved local residents and stakeholders (in San Sebastian 60% of projects involved local people in some ways and there were 10,493 hours of volunteer time while Wroclaw designed activities aimed at specific groups),
-brought culture to new audiences through specific strategies (such as "Waves of Energy in Donostia San Sebastian and "MikroGRANTY" in Wroclaw) and
-may lead to legacies both physical (in the form of new or refurbished cultural and logistic infrastructures, as it happened in Wroclaw) or intangible (by creating a space and vehicle that helped communities talk about, understand and come to terms with past differences and helping the city and its residents coexist with one another in the future, as it happened in Donostia San Sebastian) although a proper legacy planning is lacking in Donostia San Sebastian.
These main findings confirm those that had emerged from the 2015 and previous ECOC evaluations, i.e. ECOC title-holders carry out cultural programmes that are more extensive and innovative than the cities' usual annual cultural offer, with a strong European dimension and involving local citizens as well as international visitors, in line with the objectives of the Treaty and the ECOC Action.
Another issue arising from the external study – partly in line with the findings of previous ECOC evaluations – is the lack of baseline data. An ideal procedural method of evaluation of the ECOC Action would entail the undertaking of an assessment study both ex-ante and ex-post. However, budgetary 14 and time 15 constraints only allow for the latter to take place, thus leading to a lack of hard evidence concerning the benefits and impacts of the Action.
The co-legislators recognized this difficulty. In adopting Decision No 445/2014/EU, which will apply to the ECOC titles 2020 to 2033, the European Parliament and the Council decided to displace the burden of the evaluation from the Commission to the title-holders as the latter are the main funders and beneficiaries of the ECOC Action, and are better placed to collect all the data necessary. Such requirement will encourage data collection activities by candidate cities and future title-holders from the early stages of the programme. In this way, it will also contribute to help ECOC improve delivery against the objectives set for the title-year. To assist the cities in this endeavour, the Commission published guidelines on its website 16 .
After eight similar exercises of yearly evaluations – each time on a different pair of ECOC cities – the external study includes only a limited number of recommendations and most of them are very specific and related to anecdotic dimensions of the overall ECOC Action. They usefully complement the many recommendations made in previous years and endorsed by the Commission, in particular the need to establish institutional arrangements in good time, build a stable and effective delivery team benefitting from a strong political support, ensure national buy-in and involvement, ensure the right balance between control and artistic independence, keep the commitment of cultural stakeholders, embed European co-operation into the cultural programme while also actively pursuing widening participation in culture, and plan legacy at an early stage 17 . The Commission agrees with the evaluator's overall recommendation that the ECOC Action should continue, and Decision No 445/2014/EU already foresees such a continuation until 2033. At a more specific level, the evaluator recommends that cities gather and analyse big data and that the ECOC application form should be revised to see how bidding cities plan to do that. The Commission, while recognizing the interest for host-cities to make a better use of big data, cannot include in the application form criteria that are not explicitly mentioned in the Decision governing the ECOC Action. It will however explore how big data can be better addressed in its guidelines for evaluation as also recommended by the evaluator.
Moreover, the Commission shares the contractors' recommendation that the formal and informal support provided by the monitoring panel during the development phase of an ECOC is of crucial importance and should be continued, including the visits to the designated cities.
Finally, regarding the recommendations specifically addressed to EU institutions, the Commission points out that, as recommended by the evaluator, the progress and monitoring reports already give explicit consideration to the issues covered by the selection criteria set out in Decision No 445/2014/EU and that the cities' proposals for monitoring and evaluation are assessed at various times during the selection and monitoring procedures.
OJ L 304 of 3.11.2006, p. 1.
Full text at: https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/files/ecoc-2016-evaluation-en_0.pdf .
Resolution of the Ministers responsible for Cultural Affairs, meeting within the Council, of 13 June 1985 concerning the annual event 'European City of Culture' (85/C 153/02).
Decision No 1419/1999/EC of the European Parliament and of the Council of 25 May 1999 establishing a Community Action for the European Capital of Culture event for the years 2005 to 2019 (OJ L 166, 1.7.1999, p.1). That Decision was amended by Decision 649/2005/EC of the European Parliament and of the Council (OJ L 117, 4.5.2005, p.20).
Decision No 445/2014/EU of the European Parliament and of the Council of 16 April 2014 establishing a Union Action for the European Capitals of Culture for the years 2020 to 2033 (OJ L 132, 3.5.2014, p. 1).
All pre-selection, selection and monitoring reports of the panel are available at the following web-page: http://ec.europa.eu/programmes/creative-europe/actions/capitals-culture_en.htm.
Council Decision No 2012/309/EU of 10 May 2012.
Regulation (EU) No 1295/2013 of the European Parliament and of the Council of 11 December 2013 establishing the Creative Europe Programme (2014 to 2020) and repealing Decisions No 1718/2006/EC, No 1855/2006/EC and No 1041/2009/EC (OJ L 347, 20.12.2013, p.221).
See previous evaluation reports at: https://ec.europa.eu/culture/evaluations_en .
http://ec.europa.eu/smart-regulation/roadmaps/docs/2016_eac_015_evaluation_ecoc_2016.pdf .
See SWD (2012) 226 final, point 2.4.4.
Decision 445/2014/EU of the European Parliament and of the Council, see footnote n° 5.
Some of the results of this research were still in draft form at the time when the European evaluation of the ECOC Action was finalised. The European evaluation used as much of this secondary information as possible, but could not benefit from its final results.
The budget allocated to the evaluation work is proportionate to the level of EU funding directly provided to the ECOC, which is limited to €1.5m Melina Mercouri Prize per ECOC.
Decision No 1622/2006/EC requires that the Commission conducts the evaluation immediately after the title year.
Available at: https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/library/capitals-culture-city-own-guide_en.pdf
See compendium of previous recommendations at: https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/library/ecoc-compendium-recommendations_en.pdf .