EUROPEAN COMMISSION

Brussels, 26.11.2015

COM(2015) 580 final

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS

Ex Post evaluation of the 2014 European Capitals of Culture (Umeå and Rīga)

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS

Ex Post evaluation of the 2014 European Capitals of Culture (Umeå and Rīga)

1.Introduction

This report is presented under article 12 of Decision No 1622/2006/EC of the European Parliament and of the Council of 24 October 2006 establishing a Community action for the European Capital of Culture event for the years 2007 to 2019 1 , which requires that each year the Commission shall ensure the external and independent evaluation of the results of the European Capital of Culture event of the previous year and report on that evaluation to the other EU institutions.

This report puts forward the Commission's position on the main conclusions and recommendations of the external evaluation of the two 2014 European Capitals of Culture 2 .

2.Background to the action

2.1.The EU action for the European Capital of Culture (ECOC) event

The initial scheme of "the European City of Culture" was launched at intergovernmental level in 1985 3 . On the basis of this experience, Decision No 1419/1999/EC 4 established a Community action for the ECOC event for the years 2005 to 2019. Member States were ranked in a chronological order of entitlement to host the event each year.

Decision No 1419/1999/EC was replaced by Decision No 1622/2006/EC which kept the principle of a chronological order of Member States but further refined the objectives of the action and introduced new selection and monitoring arrangements.

Under this Decision, the ECOC overall aims are to highlight the richness and diversity of European cultures and the features they share, as well as to promote greater mutual understanding between European citizens. ECOC shall strive to foster cooperation between cultural operators, artists and cities in Europe, foster the participation of the citizens living in the city and surroundings while raising the interest of citizens from abroad, and be sustainable and an integral part of the long-term cultural and social development of the city 5 .

2.2.The selection and monitoring of the 2014 European Capitals of Culture

In accordance with the Decision No 1622/2006/EC, Sweden and Latvia were entitled to host the ECOC in 2014.

Parallel competitions were managed by the relevant authorities of the two Member States. The selection is in two phases: a pre-selection phase (candidate cities are reduced to a short-list) followed by a selection phase (the short-list is reduced to one single candidate). A panel of thirteen members – six of whom nominated by the Member State concerned and the other seven by European institutions – examine the bids from candidate cities on the basis of the objectives and criteria laid down in the Decision.

In each of these two Member States, four cities applied, from which two were short-listed in Sweden and three in Latvia 6 . The panel finally recommended that Umeå and Rīga be awarded the ECOC title in September 2009.

In May 2010, the two cities were formally designated as ECOC for 2014 by the Council of Ministers of the European Union.

After their nomination, Umeå and Rīga were submitted to monitoring arrangements: the progress in the cities' preparations was monitored and guided by a panel composed of the seven independent experts appointed by the European institutions, which also checked compliance with the programme and commitments on the basis of which the cities had been selected. Umeå and Rīga attended two formal monitoring meetings convened by the Commission, in November 2011 and April 2013. The monitoring process ended up with the panel making a positive recommendation to the Commission on awarding a €1.5m prize in honour of Melina Mercouri to both Umeå and Rīga.

3.The external evaluation

3.1.The terms of the evaluation

The evaluation explores the implementation of the two 2014 ECOC throughout their lifecycle, from their early inception through to their sustainability and legacy, and considers the impact of hosting the title in the two cities. In particular, it assesses their relevance, efficiency, effectiveness and sustainability, draws individual and general conclusions emerging from these two ECOC and considers implications for future ECOC title-holders.

The external evaluation was carried out before the Better Regulation Guidelines 7 entered into force.

3.2.Methodology and limitations of the approach chosen

The evaluation and its methodology were designed to satisfy the standard requirement of the Decision, and contribute to develop a more detailed understanding of the performance and achievements of the action. In particular, it provides an opportunity to look back at the previous year in order to highlight lessons and recommendations going forward based on the experiences of the two host cities.

As for ECOC 2007-2012 previous evaluations, the intervention logic used by the evaluator was based on a hierarchy of objectives corresponding to the Decision.

In order for results to be comparable with previous evaluations, the methodology also follows a consistent approach for evidence gathering and analysis. The two cities were evaluated individually, based on primary data either collected during the fieldwork or provided by each ECOC, as well as the analysis of a range of secondary data sources.

Primary data sources included interviews conducted during two visits to each city or by telephone, as well as through an online survey in Rīga. These interviews sought to gain a variety of perspectives on each ECOC, including those of the management teams, decision-makers at local and national levels, plus key cultural operators, a range of partners involved in the delivery of the ECOC and a sample of organisations either leading or participating in ECOC projects.

The secondary data sources included information in the original ECOC applications, studies and reports produced or commissioned by the ECOC, events programmes, promotional materials and websites, statistical data on culture and tourism and quantitative data supplied by the ECOC on finance, activities, outputs and results.

As was the case with all previous ex post ECOC evaluations, the Commission considers that this methodolody is appropriate to produce a report providing a reasonably solid basis on which sound conclusions can be drawn as to ECOC performance.

There is however a lack of hard evidence on the benefits and impact of the ECOC on the two host cities, partly due to the time and budget constraints of the evaluation. As a result, the report's conclusions are more substantiated by the views and opinions of various types of stakaholders than on the (limited) quantitative data available.

On the other hand, most of the qualitative evidence, in the form of an on-line survey and interview results, is limited to those directly involved in and somehow benefiting from the programme.

Furthermore, some of the objectives set are also broad and difficult to measure and it is too early for this evaluation to have been able to assess any longer term impacts. As a consequence, much of the evidence focuses on ouputs and results or emerging higher level benefits rather than on harder on the ground impacts, which will take more time to materialize. In order to better capture such impacts, it would be useful for the two cities to have longitudinal evaluations, notably to confirm the efficiency of the public spending in the ECOC from a cultural, social and an economic points of view, also using a broader range of evaluative data to support the conclusions.

The Commission is fully aware – and accepts – such limitations, which were alreadly identified in its staff working document accompanying its proposal for a Decision establishing a Union action for the ECOC for the years 2020 to 2033 8 . To remedy the situation, the Commission's proposal and the Decision ultimately adopted on this basis 9 foresee that the cities themselves – better positioned to get primary data on the impact of the title – become the key players in the evaluation process.

Against this background, it is good to see that Umeå (and to a lesser degree Rīga) have been carrying research to better understand the impact of the ECOC on cultural institutions and local residents, as well as in terms of international dimension and economics.

The Commission finds however a sufficient basis in the data and other evidence supporting the evaluation to allow it to share the overall assessments and conclusions of the evaluation, which are considered to provide a broadly true and complete picture of the two 2014 ECOC, although lacking strong data and other independent evidence to support very solid conclusions on efficiency and impacts.

4.The evaluator's findings

The evaluation confirms that many of the findings from previous reports, especially those pertaining to the overall relevance, efficiency and effectiveness of the ECOC action, are still valid. These findings have been partially updated and refreshed with the information gathered during the 2014 evaluation wherever possible. This report concentrates on these new elements.

4.1.Relevance of the ECOC action and the two 2014 ECOC

According to the evaluator, the two host cities saw the ECOC more as a cultural event strengthening and internationalising their cultural offering as well as promoting the cultural diversity and common cultural features of Europe, rather than an economic or social programme. The report concludes that this made the ECOC a very relevant action in relation with Article 167 TFEU looking at the Union's contribution to the "flowering of the cultures of the Member States".

However, the report highlights that Umeå and Rīga also used their ECOC status to tackle other aspects of their respective city's development strategy. As a consequence, many of the activities delivered in 2014 were highly relevant to the two cities' overall political agendas as well as to a range of EU level priorities and objectives in policy areas other than culture, such as urban and regional development, employment, enterprise, tourism as well as general social cohesion policies.

4.2.Efficiency of governance and management and capacity to deliver

The evaluator looked at the cities' capacity to deliver and the efficiency of the governance and management of the ECOC, and it concludes that the delivery mechanisms set up to implement the ECOC generally worked well in both cities.

Umeå 2014 was implemented by a team working directly within the Municipality of Umeå. The report highlights that the team benefited from a strong political support from the very outset as the title-year was intended as a tool to achieve local policy objectives, cultural and otherwise, which allowed for the setting of clear objectives. A key feature of the management structure was the use of the "Open Source" approach, based on co-creation and participation. In essence, this approach involved setting broad themes and criteria for the programme as a whole and then inviting cultural operators and other stakeholders to "fill" the programme themselves. Overall, the report considers that the governance of Umeå 2014 worked well, even if the final team was formed relatively late in the development phase and could have been scaled up at an earlier stage to allow earlier development of much of the content of the cultural programme.

In Rīga, an arm's length organisation, the Rīga 2014 Foundation, was set up by the City Council to coordinate the application and development stages of the ECOC action. It was led by the former Head of the City Council's Department of Culture. According to the report, this model – not unique in terms of other previous ECOC – proved to be the most appropriate for implementation, as it reduced the administrative burden, avoided some of the bureaucratic processes which apply to State or municipal authorities in the country and ensured some form of independence so that the ECOC programme could be developed "freely" without any significant external influence. The report also positively highlights the stability in the administrative team and in the support given by all political structures along the lifecycle of the project.

Both ECOC were designed and delivered in difficult economic times at both city and national levels and therefore financial efficiency was a key aspect of their planning and implementation. Although Umeå and Rīga (respectively 45,6 and 27,3 million euros) had different levels of resources available for their cultural programmes, they both showed that a successful ECOC can be delivered on a relatively modest budget and in times when resources are limited.

The report shows that both ECOC were driven by public rather than private sector funding. It stresses that neither city had been able to generate the private sector income they originally set in their applications, with the private sector funding representing less than 10% of the two ECOC's total budget. Although an ECOC is likely to always be highly dependent on public budgets, the lack of progress that both 2014 host cities made in attracting private sector funding through either direct support, sponsorship or advertising is an aspect worth noting. According to the evaluators, this is to some extent due to the current economic situation and it is hoped that as the economic context improves the reliance on the public purse to drive forward ECOC activity will change for the better.

4.3.Effectiveness in developing cultural activities and cultural and artistic content

Despite a shortage of comprehensive data on results and impacts, the evaluation concludes that both cities delivered an extremely high volume of cultural activities in 2014 (in total, just under 500 in Rīga under the motto "Force Majeure" and just over 1,000 in Umeå under the slogan "Curiosity and Passion") 10 . Although it is difficult to evidence how much "extra" culture was present in 2014 in the two cities compared to preceding years, many of those taking part in the evaluation estimated that it was "at least double".

Both ECOC also managed to design a wide ranging cultural programme which included a strong mix of cultural genres delivered across the whole year and often across various parts of the city and even beyond. They maximised the opportunity available during the year in developing cultural programmes that were neither conservative nor cautious, with cultural events that were bigger and more innovative than before. Although both cities laid on a variety of high profile and "glitzy" events, they also included a large number of smaller cultural activities which were equally well received.

According to the evaluator, the artistic control and management in the design of the cultural programmes of both ECOC came from experienced and trusted players who had a real and deep understanding of the cultural agenda of the cities. This artistic knowledge was very much found in the coordination bodies themselves (as well as in the projects) which did more than simply manage and monitor outputs and budgets.

4.4.Effectiveness in promoting the European dimension

The report estimates that the cultural programme presented by Umeå in 2014 was more European in nature than the city's cultural offering in previous years. It was based on the overarching concept of Umeå as the "Northern room" in the "European house" and Umeå as a place for border-crossing and long-term collaborations within the Sápmi area covering northern parts of Finland, Norway, Sweden and Russia. Within this overarching concept, a number of specific themes were explored, including some relating to the culture, lands and identity of the Sami people, the indigenous people of northern Scandinavia. The cultural programme – structured around the eight seasons of the Sami calendar – explored and highlighted Sami culture in all its forms, e.g. rock-art, music, crafts, dress.

Furthermore, the ECOC enabled Umeå to increase the European – and international – dimension of its cultural offer by collaborations with some international artists, who were specifically invited or commissioned to produce or co-produce events, works and exhibitions. Overall, 195 projects featured some sort of collaboration with artists or other cultural bodies from other (mainly European) countries.

Finally, in the evaluator's view, Umeå 2014 – despite a geographical challenge of being relatively isolated up North – did make the most of the ECOC status in its efforts to attract European audiences to the city and put itself "on the map" thanks to its marketing and communication activities. In this regard, the report mentions the positive achievements of the "Caught by [Umeå]" tour, which brought representatives of the city and its cultural sector into direct contact with audiences in several different European cities prior to the opening of the title year as a way to attract them to Umeå. Furthermore the number of nights booked in all tourist accomodations increased by 24% from 2013 to 2014.

Regarding Rīga, it appears from the report that the European dimension was reflected in a range of different events and activities. This included projects highlighting shared European themes – such as those linked to World War I or the former KGB building – or the support to exchanges and other mobility programmes from across different EU Member States. In this respect, findings show that 40% of all ECOC projects had an international dimension, i.e. involved foreign individuals (a majority of them European) in project implementation such as performers, artists and event organisers.

The ECOC was also seen as a "meeting place" and the opportunity for international artists to come together and work collaboratively on different activities.

With regard to raising the international profile of the city (a key motivation of the ECOC project), Rīga 2014 had mixed results. The number of foreign tourists visiting Latvia increased by 18.9% in the first half of 2014 and 2,800 publications mentioned the ECOC in Rīga (of which 98% were positive), but it appears that less than 2% of the audiences came from outside the country.

4.5.Effectiveness in engaging the citizens and in outreach

The evaluation considers that one key aspect of effectiveness in both cities was around how the ECOC status was instrumental in widening the appeal and reach of culture to community groups and neighbourhoods that simply would not have experienced culture without the existence of the ECOC.

Both programmes were very pro-active in this respect and worked hard to provide cultural content that appealed to a wide variety of different audiences, putting on performances and activities that can be classed as being popular, alternative or innovative in nature. It appears that respectively 71% and 76% of residents in Umeå and Rīga attended an ECOC event while overall 1,6 million people attended ECOC activities in the Latvian capital. Project promoters were encouraged to target underrepresented groups, many activities took place in non-traditional settings outside of the city centre and clear guidance (including during workshops or in the form of guidelines) was given on how best to widen participation.

Furthermore the development arrangements in both cities were inclusive and transparent in terms of having open calls for project ideas that came from the "bottom up", so that projects originated from cultural rather than strategic players. The above-mentioned Open Source approach in Umeå proved a very effective tool in granting the local cultural sector and a wide section of the population considerable opportunity to shape and implement the cultural programme in a way that was both authentic and credible. In the view of the evaluator, this approach is however only appropriate for cities with openness to co-creation and for international collaboration, and where there is also a context of trust and a tradition of consensus.

The report considers that Rīga 2014 had a large amount of success in spreading the activities and benefits of its cultural programme throughout the city and to a set of residents and neighbourhoods traditionally having little in the way of a cultural offer. A full line of the programme (called "Road Map") was dedicated to engaging with organisations and individuals living in these neighbourhoods. The way such groups were empowered and consulted to ensure that they shaped, delivered and enjoyed the cultural programme is a key success in the view of the evaluator. The report sees as a very positive achievement the fact that Rīga 2014 helped to encourage a "bottom up" approach to the design and delivery of the ECOC programme in a country that has a relatively embryonic approach to community development.

4.6.Effectiveness in achieving economic, urban development and tourism impacts

According to the evaluation, there is weaker evidence available on the impact of the ECOC overall in the city.

However, as far as Umeå is concerned, the report concludes that the succeful delivery of the ECOC can be credited to the ambitions of local stakeholders to creatively redesign and remake their city as a centre of growth and a cultural destination. It appears from the evidence collected by the evaluator that the ECOC has made an important contribution to the fulfilment of this ambition, notably in providing impetus to complete some important investments in cultural infrastructure and generating international interest in the city.

Although massively important for the city and its development, the success of Umeå 2014 was very much dependent on it being part of a wider strategy for urban development. Umeå 2014 was not the catalyst for the development of the city, nor the end-point. The city's cultural offering, as it stands, reflects several decades of growth and, in particular, the expansion of the university and the industries that depend on it.

In Rīga, the main focus of the ECOC was clearly on the "cultural" content of the programme rather than on improving infrastructure projects or other physical development either linked to culture or the wider public realm. This meant that the programme was sometimes seen as being less of a driver for physical improvements to the city. Some also felt that there was no obvious physical symbol of Rīga 2014 which people could associate with the year of culture either during 2014 or after it. This lack of a central physical "emblem" of the year was felt by some to mean that an obvious "impact" of the ECOC was missing. However, it appears that the ECOC has allowed those involved in culture in the city to firmly put their policy area on the map.

4.7.Sustainability

The evaluation notes that there were aspects of both cities' approach that showed they had considered sustainability. Although thinking around legacy was established early on in the ECOC lifecycle of each city, this did not however manifest in a strong longer term legacy or sustainability plan to sustain the cultural offer beyond the title year itself. Unsurprisingly, a lack of finance to sustain some of the key aspects of the cultural programme for 2014 was highlighted as the main barrier to sustainability.

Despite the above, there is evidence that culture has risen up the policy agenda in both cities as a consequence of ECOC. Indeed, the ECOC has allowed both cities to showcase the importance and benefits of culture to a level that would not have been possible without this status.

Umeå 2014 was seen as a part of a wider strategy for urban development, in which culture is a key element. The city will continue to pursue its ten-year strategy for culture covering the period 2010-2020, which will be updated to take account of lessons learnt through the ECOC year, and per capita expenditure on culture is expected to continue to grow and remain around twice the average of all Swedish municipalities.

In Rīga, culture is now mentioned in the recently produced municipal Development Plan with the inclusion of priorities such as stimulation of a "creative and European city with a high-quality cultural life" or promotion of the "access to culture and active promotion of both residents' creativity and public participation".

Furthermore, Umeå has seen improvements to its cultural infrastructure which will have a key legacy beyond 2014 as it provides organisations, residents and visitors with new and improved buildings for the years to come. Although Rīga has seen and will see improvements to its cultural infrastructure, these are less associated with the ECOC programme as – for most of them – they were not finished in time to be used during the year itself.

There is also no doubt that both Umeå and Rīga, like other previous ECOC cities, have built up an array of new relationships and networks at city, national and international levels that will be sustained in the future.

Finally, perhaps the biggest aspect of sustainability is around the increased capacity that cultural operators in both cities reported through their involvement in the delivery of various ECOC projects. A key benefit of ECOC overall and one which maximises its sustainability is around the new skills, experiences, track record and knowledge which is found in the city as a consequence of planning and implementing a major year of culture.

5.Main recommendations of the external evaluation and conclusions from the Commission

The Commission concludes from this report that the ECOC action remains extremely relevant at EU level as well as highly valued by those who host it, and generates extensive cultural programmes with positive outputs and impacts which cannot, however, be fully assessed yet. Only longitudinal evaluations carried out by the host cities themselves – as mentioned under 3.2 – could provide a clearer picture in this regard.

The Commission also concludes that the programmes implemented by the two 2014 title-holders were innovative and consistent with the objectives of the ECOC action; they reflected its European dimension, involved many local residents and stakeholders, brought culture to new audiences (through specific tools and strategies), raised the interest of citizens from abroad (albeit to a lesser degree than expected in Rīga) and are likely to lead to some legacy (even in the absence of formal sustainability plans). However, deeper assessments of the extent of the benefits produced against the costs incurred will be useful to confirm the impact of the programme. This is now the responsibility of both cities to invest in research so as to better understand how they have optimised cultural, social and economic benefits and be able to demonstrate the impact of the title-year for the development of the city, thereby justifying the value of public spending.

The report highlights a number of recommendations that emerge out of the evaluators' considerations of the two 2014 ECOC. These recommendations are mainly useful for and therefore addressed to future ECOC when developing and implementing their programmes: the need for political support at both city and national levels to gain stability; the need to build a stable and effective team at an early stage in the development period; the need to go beyond culture and connect the ECOC action with other aspects of the city's strategy including enterprise, social inclusion and physical development; the need for the title-holders to generate "impact" data such as attendance figures, spend of visitors, satisfaction levels as well as wider economic impacts; the need to actively pursue widening participation in culture through dedicated strategies as well as the need to develop longer term plans to sustain their cultural offer beyond the title-year itself. Finally the report recommends that the Commission should develop a compendium of all recommendations from current and future ECOC evaluations and make these available to candidate cities to help them learn from past lessons and good practice.

The Commission recognises the relevance of all these recommendations and will follow the evaluators' suggestion to produce a compendium.

(1)

   OJ L 304 of 3.11.2006, p.1.

(2)

   http://ec.europa.eu/programmes/creative-europe/actions/documents/ecoc-2014-report_en.pdf

(3)

   Resolution of the Ministers responsible for Cultural Affairs, meeting within the Council, of 13 June 1985 concerning the annual event 'European City of Culture' (85/C 153/02); http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:C:1985:153:0002:0003:EN:PDF .

(4)

   Decision 1419/1999/EC of the European Parliament and of the Council of 25 May 1999 establishing a Community action for the European Capital of Culture event for the years 2005 to 2019 (OJ L 166, 1.7.1999, p.1). That Decision was amended by Decision 649/2005/EC of the European Parliament and of the Council (OJ L 117, 4.5.2005, p.20).

(5)

     For more information: http://ec.europa.eu/programmes/creative-europe/actions/capitals-culture_en.htm .

(6)

   All pre-selection, selection and monitoring reports of the panel are available at the following web-page: http://ec.europa.eu/programmes/creative-europe/actions/capitals-culture_en.htm.

(7)

     http://ec.europa.eu/smart-regulation/guidelines/toc_guide_en.htm.

(8)

See SWD(2012) 226 final, point2.4.4.

(9)

 Decision 445/2014/EU of the European Parliament and of the Council of 16 April 2014 establishing a Union action for the European Capitals of Culture for the years 2020 to 2033 and repealing Decision No 1622/2006/EC (OJ L 132, 3.5.2014, p. 1).

(10)

   Counting the number of activities of an ECOC is not easy as one ECOC may consider a carnival as one activity whereas another may count individual activities within the carnival separately.