EUROPEAN COMMISSION

Brussels, 27.4.2017

SWD(2017) 135 final

COMMISSION STAFF WORKING DOCUMENT

EVALUATION

Accompanying the document

Report from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions

Ex Post evaluation of the 2015 European Capitals of Culture (Mons and Pilsen)

{COM(2017) 193 final}
{SWD(2017) 137 final}

1.Introduction

This document describes the methodology and findings of the ex-post evaluation of the European Capital of Culture event (ECoC) for the year 2015. Decision No 1622/2006/EC of the European Parliament and of the Council of 24 October 2006 establishing a Community action for the European Capital of Culture event for the years 2007 to 2019 1 requires, in its Article 12, that the Commission ensures the external and independent evaluation of the results of the European Capitals of Culture from the previous year. The purpose of the yearly evaluation exercise is to establish a comprehensive understanding of the performance and achievements of the action in the relevant year, as well as to draw lessons and recommendations for the future based on the experiences of the two host cities for that year.

The scope of this evaluation study covers the implementation of the ECoC action for 2015, including the selection and monitoring procedures and the implementation of the action by the two 2015 ECoC hosts: Mons in Belgium and Pilsen in the Czech Republic. The evaluation investigated how these two cities developed their respective applications and cultural programmes, how they delivered their year, the benefits they achieved and any legacy issues they experienced.

The findings of the evaluation lead to lessons for the future implementation of the ECoC action.

2.Background

2.1.The European Capital of Culture action

The initial scheme of "the European City of Culture" was launched at intergovernmental level in 1985 2 . On the basis of this experience, Decision No 1419/1999/EC 3 established a Community action for the ECoC event for the years 2005 to 2019. Member States were ranked in a chronological order of entitlement to host the event each year. Decision No 1419/1999/EC was replaced by Decision No 1622/2006/EC which kept the principle of a chronological order of Member States but further refined the objectives of the action and introduced new selection and monitoring arrangements. A new Decision was adopted by the European Parliament and the Council in April 2014 4 .

2.2.Objectives of the ECoC action

The ECoC overall aims are to highlight the richness and diversity of European cultures and the features they share thereby promoting greater mutual understanding among European citizens, and to foster the contribution of culture to the long-term development of the cities. ECoC shall strive to foster cooperation between cultural operators, artists and cities in Europe, foster the participation of the citizens living in the city and surroundings while raising the interest of citizens from abroad, and be sustainable and an integral part of the long-term cultural and social development of the city.

The hierarchy of objectives presented in the Table below is based on the objectives as stated in Decision No 1622/2006/EC, but has been updated to reflect the content of the new legal basis for ECoC post 2019. The general and strategic objectives are taken directly from Article 2 of Decision No 445/2014/EU, with the operational objectives flowing logically from these. They are also informed by the selection criteria detailed in Article 5 of the new 2014 Decision.

Table on ECoC hierarchy of objectives

3.Evaluation questions

The evaluation assessed the relevance, efficiency, effectiveness and sustainability of the European Capitals of Culture 2015. It also examined the EU added value as well as the coherence and complementarity of the action to other EU initiatives.

To measure the relevance, questions were asked about the extent to which the objectives of each ECoC action as well as the extent to which the ECoC's cultural programmes and associated activities were relevant to EU objectives.

The efficiency section included questions about how the management arrangements of each ECoC contributed to the achievement of outputs, results and impacts; the extent to which the selection, monitoring and EU co-financing procedures, introduced by Decision 2006/1622/2006/EC were efficient, the extent to which the ECoC managed to raise the necessary resources, the extent to which the financial and human resources secured by each ECoC were appropriate and proportionate.

Effectiveness had questions on the extent to which the EU-level objectives were achieved and the extent to which the ECoC's own objectives were achieved.

Sustainability was about the extent to which the positive effects of the ECoC action can be considered to be sustainable, the EU added value of the ECoC action and the extent to which the ECoC action is complementary to other EU initiatives.

4.Method

The basis for the evaluation was set up in the evaluation roadmap 5 . The evaluation was overseen by a Steering Group and supported by an external study contract.

The methodology for the evaluation of the 2015 ECoC largely followed the approach adopted in previous studies of the action 6 . The focus of the evaluation methodology has been on research at the city level and in particular the gathering of data and stakeholders' views from Mons and Pilsen. Key evaluation sources were as follows:

-EU level literature: this included higher level EU policy and legislative briefings, papers, decisions and other documents relating to ECoC. This mainly focussed on reports of the selection panels and the original bidding guidance to understand how the two ECoC established themselves in the early days;

-ECoC level literature from Mons and Pilsen: this included the original bids and applications, internal reports linked to the application processes and numerous pieces of literature collected on the cultural programme itself. Key monitoring and in particular evaluation reports were also collected and analysed;

-Quantitative data: where available, evidence linked to each ECoC was collected in relation to budgets and spend details, project numbers and types, participation levels and audience figures as well as other pieces of quantitative data to show and describe the work and benefits of the ECoC in each city;

-Interviews with managing teams 7 : those responsible for the day-to-day design and delivery of the ECoC were interviewed in each city both during 2015 and again in 2016. Almost all of the key individuals linked to the delivery agencies were interviewed including those linked to strategic development, marketing and communication, project implementation and financial management;

-Interviews with key stakeholders 8 : mainly face to face interviews were undertaken with stakeholders both directly and indirectly involved in either the planning or delivery of the ECoC along with those more widely linked to the cultural, social, economic or political agenda of the host cities. Stakeholders included those working in cultural organisations, city/ regional/ national level administrations, tourism and visitor agencies, media organisations as well as voluntary and community organisations. Managers of individual projects and activities supported through the ECoC action that made up the cultural programme of each city were also interviewed;

-Survey of ECoC projects: a specific survey of projects was undertaken by the evaluators in Mons, whilst in Pilsen a survey already undertaken by the Pilsen 2015 Foundation (the body responsible for the ECoC) was used to gain further insight of project managers' views on a variety of different issues linked to the design, delivery, benefits and legacy of the ECoC 9 ;

-The evaluation does not include a wider public consultation. As explained in the roadmap which was published 10 , the action is considered as a local action. International participation is scattered within and outside Europe, and it is difficult to reach. On the other hand the opinions would be based on attendance to specific events and would not give useful insights for the evaluation of the ECoC action as a whole.

The final report of the contract supporting the evaluation provides a detailed understanding of the 2015 ECoC action and within this an assessment of the work and progress of Mons and Pilsen. There are however a number of issues to consider when assessing the strengths of the evidence base used for this study:

-The time and budget available to undertake the evaluation were limited. A study which provides a before ("baseline") and an after picture would be ideal for assessing the full benefits and impact of the ECoC action. However, budget 11 and timing 12 constraints only allow an ex-post evaluation to take place and therefore only an after picture has been studied;

-Although both cities have undertaken some form of evaluation work themselves, some of the results of these studies were still in draft form at the time of the evaluation. The European evaluation of the ECoC action has used as much of this secondary information as possible, but could not benefit from its final results;

-The impact of an action such as ECoC will often only manifest itself fully after the ECoC year itself. Therefore an evaluation undertaken close to the end of the ECoC year, as required by the legal base, is not likely to identify long-term benefits and impacts. Therefore the evaluation methodology must rely more on the views and opinions of stakeholders rather than empirical evidence of impact.

5.Implementation state of play

5.1.The selection and monitoring of the 2015 European Capitals of Culture

In accordance with Decision No 1622/2006/EC, Belgium and the Czech Republic were entitled to host the ECoC in 2015. The selection is conducted in two phases: a pre-selection phase (candidate cities are reduced to a short-list) followed by a selection phase (the short-list is reduced to one single candidate). A panel of thirteen members – six of whom nominated by the Member State concerned and the other seven by European Union institutions – examine the bids from candidate cities on the basis of the objectives and criteria laid down in the Decision.

The two Member States launched their competitions in parallel. In Belgium, only Mons applied for the title, being pre-selected in June 2009 and then recommended in February 2010 to host the title in 2015. In the Czech Republic, there were three candidates, Hradec Králové, Ostrava and Pilsen. In December 2009, the latter two were shortlisted at the pre-selection meeting. In September 2010, Pilsen was recommended to host the title in 2015. The two cities were nominated by the Council of the European Union, upon a recommendation from the Commission, in November 2010 and May 2011 respectively.

After their nomination, Mons and Pilsen were subjected to monitoring arrangements: the progress in the cities' preparations was monitored and guided by a panel composed of the seven independent experts appointed by the European Union institutions, which also checked compliance with the programme and commitments on the basis of which the cities had been selected. Mons and Pilsen attended two formal monitoring meetings convened by the Commission, in November 2012 and April 2014.

During the monitoring phase, the two cities introduced modifications into the programme described in their original applications, in response to a changing environment and to the recommendations of the monitoring panel. The panel also visited the cities where it found appropriate. The monitoring process ended up with the panel making a positive recommendation to the Commission on awarding a €1.5m prize in honour of Melina Mercouri to both Mons and Pilsen.

The section below describes the main features of the ECoC projects for Mons and Pilsen, in particular the cultural programme and the implementation of the two criteria in the Decision: "European Dimension" and "City and Citizens". The report from the supporting study contains further information including the development of the applications and the governance and funding structures.

5.2.Mons 2015

Mons is a city of about 93,000 people and the capital of Hainaut province in Belgium, close to the French border. The city is in the eastern end of an area known as the Borinage, which comprises around thirty municipalities. Mons and the Borinage suffered industrial decline during the post-war period, but in recent years, Mons has benefitted from the arrival of hi-tech companies and as a centre for higher education.

The idea for Mons to host the ECoC emerged from the wider strategy of the municipality to regenerate the city based on culture, tourism and new technologies. Mons's application stated its overall aim as putting itself on the European map as a symbol of economic restructuring based on culture and of successful alliance between the economic and cultural spheres. In line with that aim, the overall theme was "where technology meets culture".

Its cultural programme was divided into four seasons: "Dazzle" (to bring light and warmth to the winter months); "Metamorphosis" (to emphasize the arrival of spring and changes taking place in Mons with new infrastructure developments and possibilities offered by new technology); "Escale" (to encourage visiting or staying in Mons during the holiday period); and "Renaissance" (to emphasize the rebirth of Mons after the decline of key industries with a focus both on the historical characters of the city's "golden age" and on future developments).

Mons 2015 was implemented by a dedicated public utility foundation ("Fondation Mons 2015"), that was founded in March 2006. The Fondation Mons 2015 was entrusted with the task of developing and implementing the cultural programme and the associated communications activities. The four main public authorities that oversaw the Foundation (Fédération Wallonie Bruxelles, Région wallone, Province de Hainaut, Ville de Mons) offered strong political support and guaranteed the Foundation's artistic independence. The team largely remained intact throughout the application, development and implementation phases of the ECoC.

Mons 2015 was one of the better-funded ECoC to date with a total budget of €72.8m (which includes in-kind support from corporate sponsors valued at more than €2m). The four main authorities committed 68 % of the proposed budget at an early stage, in line with their initial commitment. EU funding in the form of the Melina Mercouri Prize was used to increase the overall budget.

One of the main objectives of Mons's application was to "involve citizens in a process of cultural democracy". To this end, local citizens were involved as creators, performers and audiences. Furthermore, specific events were held in the towns and communes neighbouring Mons and in the rest of the Borinage.

The ECoC has gone hand-in-hand with a very substantial development of the cultural infrastructure of the city, in terms of new venues and increased exhibition and performance space; public and private investments of more than €143m were made during the development phase.

Finally, Mons has put in place very concrete plans for continuation activity including a biennial, the first edition of which will be "Mons 2018". This is intended to be "a major cultural date on an international scale that will revive the spirit of the festive European Capital of Culture" and be "based on the values that brought the success of Mons 2015". The biennial will be preceded by events in 2016 and 2017. A new body, the "Fondation Mons 2025", will retain some of the staff of the "Fondation Mons 2015" and operate from the same premises and under the same governance structure. It will continue to operate in partnership with local businesses in the context of "Mons 2025 Business Club", which will serve as a successor to the Club 2015, a not-for-profit association set up in the context of the ECoC project.

5.3.Pilsen 2015

Pilsen is the fourth largest city in the Czech Republic and is situated approximately 90 km southwest of Prague. With a population of 165,000 people, Pilsen is the largest city and the administrative and industrial hub of the Pilsen region, which, with 550,000 inhabitants, accounts for about 5 % of the total Czech Republic's population.

Pilsen is a relatively prosperous city within the Czech Republic with a generally thriving industrial hub and comparatively high levels of employment. Although some social issues do exist within the city, it does not perceive itself nor do statistics suggest that it is a city suffering common urban problems linked to issues such as unemployment, social unrest, crime or pollution.

The motivations to apply for ECoC status were therefore generally not focused on "tackling urban problems such as unemployment or industrial decline" like many ECoC cities have stated in their bids in the past. Instead, the main driver for ECoC was around more simple messages linked to strengthening and diversifying the cultural offer, making the city more outward looking and fundamentally using ECoC as a "vehicle for positive change" throughout the city.

Against this background, the general aim of the Pilsen ECoC application was to explain how the European Capital of Culture would contribute to the opening up of Pilsen towards Europe and other external influences. Hence, the development of the slogan "Pilsen, Open Up!". The cultural events were delivered across four main streams: "Arts and Technologies" (to celebrate and strengthen the link between Pilsen's industrial background, crafts, skills and business); "Relationships and Emotions" (to open up the public space of the city and engage the public in a discussion about their personal and national identity); "Transit and Minorities" (to highlight the diversity of the city and its population); and "Stories and Sources" (to promote tourism based on some of Pilsen's personalities and to reminisce about past events and experiences).

At the heart of the governance arrangements for the ECoC was the Pilsen2015 Foundation. The Foundation was a non-profit organisation established by the city authority in 2010 with the mandate to prepare and implement the ECoC programme overall. The Pilsen2015 Foundation provided strategic direction for the entire ECoC programme, managed a number of key projects and supported the design and delivery of ECoC activities run by other organisations within the city and beyond.

The total budget for Pilsen2015 was €18.2m, which was largely in line with what was projected at the bid stage. The majority of the funds (86 %) were from the public sector (City: 45 %, Region: 10 %, State: 21 % and EU: 10 %), whilst the remainder came from sponsorship, ticketing or merchandising.

There were a large number of individual and practical activities put in place to ensure the involvement and empowerment of residents. In particular, a volunteering programme also engaged active volunteering from Pilsen residents across a range of ECoC projects. A total of 515 volunteers were identified and trained through the ECoC to help undertake a range of roles including crowd control and signposting at larger events, undertaking local marketing as well as helping to set up various events and activities.

Pilsen 2015 included no large (and expensive) capital projects, outside of the Depot2015 13 and the New Theatre.

6.Answers to evaluation questions

The evaluation confirms that many of the findings from previous reports 14 , especially those pertaining to the overall relevance, efficiency and effectiveness of the ECoC action, are still valid. These findings have been partially updated and refreshed with the information gathered during the 2015 evaluation wherever possible. This report concentrates on these new elements.

6.1.Relevance

The experience of 2015 reinforces the finding from previous evaluations that ECoC remains highly relevant to the EU Treaty, particularly Article 167, through contributing to the flowering of Member States' cultures, highlighting common cultural heritage as well as cultural diversity and increasing cultural co-operation within Member States and internationally.

The cultural programmes in Mons and Pilsen demonstrated a high degree of relevance to the policy objectives set at EU level for the ECoC action. Indeed, the cultural programme reflected the cities' own culture and history and expressed the diversity of European cultures in different and innovative ways. In particular, it featured themes and personalities with a connection to Mons (such as Van Gogh and Verlaine) or Pilsen (such as Trnka) but with a European resonance. The experience of both cities shows that the ECoC is relevant to a range of different European policy areas going beyond culture, such as urban and regional development, employment, tourism as well as social and territorial cohesion etc., as illustrated by the many examples given in this section of the document.

Both Mons and Pilsen have seen their cultural offer greatly strengthened because of ECoC which has not only helped them to diversify their cultural scene but also diversify the types of audiences enjoying culture.

For example, 77 % of respondents in Mons share the opinion that Mons 2015 has attracted audiences that did not usually visit cultural venues or events 15 . In both cities, the cultural programme during the ECoC year was more extensive, more innovative and more European in nature compared to the cities' cultural offering in previous years. In Mons, it included 219 projects featuring 2,390 events of different sizes, cultural disciplines and art forms, most of which were new for 2015. Mons 2015 found new and creative ways to use public spaces for artistic purposes. It also featured a significant number of new works that were performed or exhibited for the first time in 2015.

In Pilsen, 600 cultural events and experiences were delivered throughout 2015. Local people themselves were often the subject of various ECoC exhibitions, shows and performances, meaning that local citizens became a key aspect of the overall ECoC programme, which was a relatively new experience in the city and a way to reach out to new types of audiences. Examples include the organisation of "neighbourhood walks" run by local people who delivered "professional" guided walks or the Family Photo Album consisting of around 200 photographs taken in Pilsen and borrowed from local residents and visited by 5,150 people.

Both cities have used ECoC to help them internationalise their cultural offer, thus making it relevant to the European dimension. This is particularly true in Pilsen which used ECoC as a vehicle to help open up the city to Europe: 53 % of cultural players taking part in the ECoC project survey 16 now say that they had good international links with partners because of their participation in the ECoC while this participation was low or non-existent prior to 2015. The ECoC has also strengthened the international dimension of cultural activity within Mons and the Borinage. International collaborations have increased the number of connections with new partners performing in other countries. In Mons, approximately 40 % of the respondents report having collaborated with some of their partners in other countries for the first time.

The ECoC label brings a significant amount of profile to the host city: this is particularly true for smaller host cities like Pilsen or Mons who could not hope to generate the amount of press coverage, visitor numbers and overall interest in its cultural offer without it hosting an ECoC. In Mons, data confirms that there were visits from 450 accredited international journalists and 3,717 articles in the international press or items on international radio and television. The city tourist office experienced a five-fold increase in tourist visits during 2015, reaching a total of 250,000. In Pilsen, the 1.4 million visitors who attended ECoC projects in 2015 represent a 28 % increase from the two years prior to 2015.

The ECoC "label" also acts as a significant generator of interest from stakeholders in the city around culture itself, with both Mons and Pilsen stating that ECoC helped raise the profile of culture among a wide range of policy makers in the city. As an example, Mons 2015 has helped cement the link between culture and tourism in the city and beyond. A key factor here has been the close co-operation between the Ville de Mons, the Foundation and Wallonie Bruxelles Tourisme (including through the local tourist office "Visit Mons").It has also created new capacity for corporate sponsorship of culture. Whilst corporate sponsorship of culture was not absent before 2015, Mons lacked a comprehensive and co-ordinated approach to the attraction of corporate sponsors. A key achievement of the ECoC has thus been the new partnership(s) created with the local corporate sector, encompassing large companies and multinationals, as well as local SMEs. The Club Mons 2015 gathered 841 members and raised a total contribution of 841.000 €.

6.2.Efficiency

Overall, the ECoC action seems to have been implemented with a relatively high level of efficiency at EU level. Indeed, the selection process enabled the selection of cities with the capacity, resources and vision to implement ECoC responding to the objectives of the action. Both cities have also benefited from the monitoring at EU level and from the informal support given by the monitoring panel and the European Commission. In particular, the concerns expressed by the monitoring panel regarding the worryingly slow and stuttering start of the development of Pilsen 2015 prompted the city to react and put forward a clear action plan to help negate the main barriers and problems identified.

At the same time, the modest funding provided by the EU (EUR1.5m in the form of the Melina Mercouri Prize for each ECoC) can be said to have had a considerable leverage effect as getting the ECoC title and Melina Mercouri Prize prompted the two cities to invest considerable sums in their ECoC programmes (approximately EUR72,8m in the case of Mons and EUR18,2m in the case of Pilsen) and in associated infrastructure developments (EUR143,5m for Mons and EUR48,6m for Pilsen). Both cities also report that the Melina Mercouri Prize offers important symbolic value, as it represents an endorsement by the EU of their activities and offers opportunities to positive publicity. However, the impact of the Melina Mercouri Prize could be enhanced by greater publicity at EU level.

At the city level, both Mons and Pilsen have delivered their ECoC in an efficient way. Mons had a budget that was around four times larger than Pilsen but both used national and EU funds to implement cultural programmes of high artistic quality and of considerably greater size than the cities' "usual" cultural offering. Both ECoC cities designed and delivered a large amount of cultural activity with many hundreds of performances, exhibitions and other activity taking place as a direct consequence of ECoC. Both cities also continued ECoC traditions by implementing a programme that was wide, varied and innovative. The programme used a range of cultural genres to entertain its audiences ranging from street art through to world class dance.

To some extent, the 2015 evaluation shows that "money matters" when it comes to ECoC, with the larger Mons budget helping the city achieve more in terms of reach, the content of its cultural programme and the amount of legacy and sustainability being achieved. However, the 2015 evaluation also shows that a small ECoC budget in a comparatively small city can still produce a very strong outcome across a range of different issues. Pilsen should be congratulated in putting on an ECoC which made a large difference across the city and which used its smaller resources in a very efficient way. Although the cultural programme perhaps had fewer benefits in helping showcase Pilsen to Europe, it had a great benefit in helping promote Europe to Pilsen. Indeed, two thirds of respondents to the survey carried out by Pilsen 2015 said that their work and activities were much more diverse internationally as a consequence of taking part in the ECoC year and that this diversity would continue beyond the year itself.

The delivery mechanisms established in both cities were strong and there were very few negative views placed on this aspect coming from the interviews and the surveys carried out. Although both Mons and Pilsen had a different scale of cultural programme and activity, both had similar delivery mechanisms, similar partnership arrangements and similar development processes. Compared to other ECoC, the operation of the governance and management arrangements of Mons 2015 have been relatively smooth and stable, despite the complexity of the Belgian governance context. Mons 2015 benefited from the strong, high-level political support offered by its mayor as well as from the continuity within the key members of the operational team throughout the process. Pilsen had a very difficult start to its development process with it lacking progress in terms of its cultural programme, key ECoC projects and its funding. However, Pilsen also shows that ECoC that have difficult starts can also change direction and make the year positive with the delivery of thousands of events attracting 1.4 million visitors through 2015 and within the limited budget allocated 17 , as long as the right people are in charge who have the power to influence a turn-around in proceedings.

Mons had much stronger involvement of the relevant regional Ministries and related agencies compared to Pilsen. Pilsen was relatively unusual to other ECoC in its relative isolation from national Government and national support. As a consequence, Pilsen struggled to secure budgets and reach international visitors.

6.3.Effectiveness

The ECoC action in 2015 has proved effective against the objectives set for it at EU level, as well as the objectives set by the cities holding the title. The action has achieved an impact that would not have arisen through the actions of Member States alone. In the absence of ECoC, both the 2015 title-holders would have been free to invest their own resources in implementing cultural programmes and developing their cultural infrastructure. However, their designation as ECoC has attracted additional resources, including from private sponsors 18 , as well as greater media coverage 19 , an increase in international tourist visits 20 and enhanced local pride in the city. Regarding the latter, research 21 shows for example that 86 % of the Mons's residents felt that the ECoC had been a positive thing and the project survey suggests that more than half of respondents feel that the ECoC has greatly improved the image of Mons with its own residents. At the same time, the precise magnitude of impact is hard to determine, given the limited baseline data submitted with the ECoC applications.

The effectiveness of the ECoC in 2015 is particularly strong in terms of the two cities maximising the opportunity to strengthen the cultural organisations in Mons and Pilsen. Both cities recognised that ECoC is a powerful tool in helping capacity build and develop local organisations, whether in terms of them developing stronger business plans, helping with marketing or helping their staff put on bigger and better productions. ECoC has also helped these local cultural organisations employ more staff and also buy new equipment that will again make their cultural offer much better quality. The cultural sector in the two cities has acquired skills which it will benefit from in its future activities beyond 2015, according to the answers to the surveys and interviews carried out for the two cities 22 .

The 2015 ECoC evaluation has also found that the two cities put on "new" and "better" cultural content than was previously the case, such as new uses of public spaces with open air events, festivals and urban art installations. In Pilsen, an independent survey (as yet unpublished) on the perceptions of local Pilsen residents showed that 76-80 % of them felt that the city now has a much higher quality offer for them to enjoy. ECoC did not replace or substitute existing cultural content that would have happened in the absence of the year and the review of the cultural programmes shows that Mons and Pilsen made the most of the title-year and were ambitious and innovative. None of the stakeholders taking part in the evaluation felt that 2015 was a lost opportunity and that more could have been done to maximise its content and benefit 23 .

The ECoC in both cities also used the year to encourage cultural organisations in the city to work with one another more than they did previously. Both Mons and Pilsen made it a condition of grant (or involvement in the ECoC programme) to work in partnership with other local cultural players. Joint ticketing, the sharing of equipment and joint marketing were just some of the examples of where the ECoC has helped stimulate better partnership working. Even in cities the size of Mons and Pilsen, the cultural sector can often work in isolation and ECoC is a good vehicle to strengthen this aspect of the cultural infrastructure of future ECoC cities.

The two 2015 cities have been effective in showcasing local culture to the large number of audiences attending ECoC events. In Mons, total audiences were nearly 2.2m people (a record number of 180,000 visitors or the Van Gogh exhibition), most of which must be considered as additional to the audiences of previous years, as most events were new in 2015 and there is no evidence that events and venues outside the Mons 2015 cultural programme suffered any significant loss of audiences. The ECoC also attracted audiences from further afield in 2015 than in previous years. For example, data from the tourist office provides evidence of a marked increase in tourist visits to Mons during 2015, of which many were specifically for cultural reasons.

In Pilsen, with over 600 cultural events delivered throughout 2015, 1.4 million visitors attended activities organised through ECoC projects throughout the year, who spent around €20m directly with ECoC projects. This number of visitors was a 28 % increase from the two years prior to 2015. Main events included the Liberation Festival with 219,000 visitors, the Giant puppets in Pilsen (Skupa's Pilsen festival) with 73,000 visitors or the exhibitions "Jiri Trnka Studio" and "Trnka's Garden" with 44,000 visitors. Both Mons and Pilsen ensured that local cultural talent enjoyed as much attention as possible with many local cultural venues enjoying audience numbers much higher than they had seen before. This has given them vital experience and confidence to use in the future and overall will help the cultural operators raise their profile beyond their normal reach. There was some tension in Pilsen around the need to involve the European dimension (with European cultural organisations) at the same time as involving local organisations. Pilsen dropped some local cultural projects in favour of European ones in order to reduce the programme which caused a certain amount of "anti-ECoC" activity. Linked to this, although both ECoC had some star performers, neither of them included high profile cultural celebrities in their programme and instead focussed on nurturing local talent.

In terms of being effective around targeting specific groups in the city, then Mons has been more successful than Pilsen in this respect. Mons had a number of projects specifically targeted at different groups – whether that is the young, old or disadvantaged. Pilsen had less opportunity to be effective in this area – again mainly due to its budget which only allowed it to focus on the wider audience. This is not to say that more "minority groups" did not benefit but rather that they were not a focus of specific cultural activity.

Although the evidence of the number of foreign visitors was less available for Mons, the qualitative evidence from both cities backed up by data from Pilsen (where 5 % of the project audiences were from outside of the country) shows that ECoC have been less effective in attracting international visitors. Although evidence from previous ECoC evaluations is mixed, the more recent evaluations (2014/2015) show that ECoC programmes need to be relatively big budget in order to lay on and market a cultural programme that is big enough to specifically attract foreign visitors to make a special journey to the city. Although foreign visitors do attend and enjoy ECoC projects, it may be that they are already in the city because of its wider attractions rather than specifically there just because of ECoC.

Pilsen did not have any key physical developments as part of its ECoC year outside of the Depot2015. Interestingly, instead of local stakeholders complaining that this lessened the impact and effectiveness of the overall ECoC programme they were adamant that this did not reduce any of the benefits. Although it reduced a more obvious legacy for the programme (see sustainability) it did not seem to dampen the enthusiasm or passion that local cultural operators had of the ECoC year. Most were quick to state that new buildings do not necessarily lead to successful ECoC and all were keen to put on a rich and varied cultural programme rather than spend their limited budget on new capital projects/ buildings.

6.4.Sustainability

The timing of this evaluation makes it difficult to draw conclusions about sustainability. The research has identified some potential for sustainability of activities and impetus, particularly in Mons where there are concrete plans for a legacy event. However, further research would be needed to identify the extent of sustainability in practice.

Both the 2015 ECoC genuinely thought and planned for sustainability and legacy. They were both keen to ensure that ECoC lasted more than one year in terms of its benefits and impact. Because of its bigger budget, Mons has more obvious sustainability and legacy than Pilsen and has more physical infrastructure in place in terms of new cultural buildings and facilities. Mons also had more concrete legacy plans in place including the Mons 2018 which involves a high profile festival in line with Mons 2015.

Pilsen did not have a clear legacy strategy in place. Although they were aware of the importance of prolonging the benefits of ECoC there was less in the way of a specific plan to help this become a reality. Having said this, the core team from the Pilsen 2015 Foundation is still in place and they personally drive forward many of the legacies as well as the learning from the year. As an example, the Director of Pilsen 2015, is currently leading the DEPO2015 cultural venue, one of the most important legacies of the year. There was a certain amount of fragility around this though and a danger this could be lost if key staff move on. Properly planning for sustainability for the ECoC rather than "hoping" that a longer term legacy appears is a key learning point here.

Both Mons and Pilsen still have their Foundations in place post 2015. This is now becoming more common practice among ECoC who are recognising that having an independent body driving forward policy and practice within the cultural agenda of the city is highly beneficial. The Foundations set up by ECoC are therefore increasingly doing more than simply overseeing the ECoC year and are becoming a much more established part of the cultural infrastructure of the host cities. The Foundation staff in both cities is still very much involved in the development and delivery of culture and are still using their vast amount of knowledge, skills and experience to drive forward positive change across the city's cultural offer. Again, based on the last few ECoC evaluations, ECoC staff seems to be remaining in post after the year rather than moving to another city which has benefits all round for the sustainability theme.

The main legacies of the ECoC are often less tangible to see but the evaluations carried out over the years suggest that there are nevertheless very important longer term impacts of the action. As with other recent ECoC evaluations, stakeholders in Mons and Pilsen articulated the legacies of the year in terms of stronger skills, stronger relationships and a higher profile for culture in the city more widely. These less tangible legacies will equip the cultural operators to deliver better quality cultural offers and will strengthen the organisations delivering cultural projects well beyond the year itself. For example the main "softer" legacies identified by stakeholders in Pilsen were threefold 24 :

-A stronger international dimension to the cultural offer, with links to mainly European partners formed in 2015 generally staying in place beyond the ECoC year. 12 % of ECoC projects were continuing to strengthen their links with international partners beyond 2015;

-A stronger set of skills and capacity held by cultural operators developed as a consequence of designing and delivering ECoC projects throughout 2015. 42 % of ECoC projects reported that the technical skills linked to performance production, marketing, ticketing, lighting and social media were all being used beyond the year itself; and

-A stronger set of links and networks within the city among cultural operators reported during the year itself was again continuing beyond 2015. Again, these joint activities were still occurring due to the legacy of strong relationships built up over the ECoC year.

Another longer term legacy of the ECoC in Mons and Pilsen has been around how the programmes have attracted a new type of audience to experience and enjoy culture. As with other recent ECoC, widening participation away from the "converted" and laying on cultural projects that appeal to those who "usually watch TV" will have an important legacy for both cities.

6.5.Coherence

The ECoC Action has also proved to be complementary to other EU policies and programmes. In particular, it has been reinforced by and added value to investments made the ERDF; the ECoC has stimulated cities to use ERDF for investments in cultural infrastructure and has given greater impetus to the completion of those investments in time for the title-year. In Mons, there was a wider strategy for the development of culture and tourism in Mons from the early 2000s, which foresaw investments in physical infrastructure and facilities. However, a potential bid was discussed as early as 2002 and the decision to bid was made in 2004 in the knowledge that Mons would have a good chance of winning. In that context, it is clear that the many investments that were initiated from 2007 onwards were clearly intended to support the ECoC application and, in the event of a successful application, the title-year 25 . In Pilsen, a total of €48.5 million was invested in infrastructure relating to Pilsen 2015, including two new venues, the New Theatre and DEPO2015. The Cultural Factory, that received the approval for financing from the ERDF, had to be abandoned after asbestos was found in the roof, with a considerable increase in the related costs that resulted in the project to be abandoned and replaced by DEPO2015.

ECoC often helps galvanise a city and its stakeholders to get behind culture in a way that was never possible before, not just in terms of cultural stakeholders but also those related to employment, enterprise, tourism and city investment. This means that the overall relevance of the ECoC action to a variety of European policy areas is assured.

6.6.EU added value

As already mentioned above, the ECoC action has achieved an impact that would not have arisen through the actions of Member States alone. In the absence of ECoC, both the 2015 title-holders would have been free to invest their own resources in implementing cultural programmes and developing their cultural infrastructure. However, their designation as ECoC has attracted additional resources, including from private sponsors, as well as greater media coverage, increased international tourist visits and enhanced local pride in the city. These benefits would have been unlikely to arise to the same extent in the absence of ECoC designation; in that sense, the ECoC action has generated clear "European added value".

The "label" which ECoC gives to host cities is one of the key aspects of the European added value of the Action. This label brings a significant amount of profile to the host city at a level that would simply not be possible without ECoC status. The ECoC label also acts as a significant generator of interest from stakeholders in the city around culture itself; both Mons and Pilsen stated that ECoC helped raise the profile of culture among a wide range of policy makers in the city. Interestingly, in Pilsen, stakeholders felt that the added value of ECoC around the European dimension was particularly high because many of the cultural operators in the city did not previously have links with European organisations.

7.Conclusions

The ECoC action is highly valued by the hosting cities that can obtain positive impacts during the year, as well as during the preparation phase. The action also remains relevant at EU level. However, its long-term impacts cannot be assessed at this stage.

The lack of baseline data for comparison of the situation in the city before winning the title, after nomination and after the ECoC year makes the evaluation of impacts difficult. In this sense, the provisions of Decision 445/2014/EU, which will cover the ECoC titles from 2020 to 2033, foresee that the cities shall be responsible for the evaluation of the results. This requirement is part of the criteria used to assess the applications and should allow for the collection of data at early stages to establish a baseline.

The programmes implemented by the two 2015 title-holders were innovative and consistent with the objectives of the ECoC action; they reflected its European dimension, involved many local residents and stakeholders, brought culture to new audiences through specific strategies (in particular in Mons, to a lesser extent in Pilsen) and have a planned legacy both physical (new cultural venues) and intangible (in the form of a biennale in Mons, in Pilsen with the continuation of the activities in Depo2015 and in the form of increased capacity and increased cultural offer in the city). However, deeper assessments of the extent of the benefits produced against the costs incurred will be useful to confirm the impact of the programme. This is now the responsibility of both cities to invest in research so as to better understand how they have optimised cultural, social and economic benefits and be able to demonstrate the impact of the title-year for the development of the city, thereby justifying the value of public spending.

Annex 1 – Procedural information

The evaluation was led by Directorate General for Education, Culture, Youth and Sports (DG EAC). It is included in the Work Programme of Creative Europe for 2015 26 and in the Agenda Planning with the reference 2015/EAC/012.

The evaluation was supported by an external and independent evaluator, under a service contract. The service contract has been implemented via a Framework Contract with reopening of competition and in accordance to the Financial Rules Applicable to the General Budget of the Union 27 and its Rules of Application 28 .

The evaluation Roadmap was adopted in December 2015 29 .

According to the Roadmap, a Steering Committee including staff from DG EAC and from the Directorate General from Regional Policy was established in May 2015. The Steering Committee met in four occasions: to prepare the Terms of Reference (approved on 23 June 2015), to approve the Inception Report in November 2015, to discuss the draft final report in October 2016 and to approve the final report in November 2016. Extensive correspondence between the Steering Committee members was held in between the meetings to follow-up on the evaluation.

The evaluation initial schedule foresaw a final report in the third quarter of 2016. Due to the late availability of data used for the evaluation (and gathered by the cities) it was agreed to delay the submission of the final report to the fourth quarter of 2016.

The evaluation did not need to be submitted to the Regulatory Scrutiny Board, as it is not considered a major evaluation, does not contain an impact assessment and does not constitute a fitness check for legislation.

The 2015 evaluation of the ECoC used a series of data sets to inform its findings. The main ones being:

-Interviews with over 50 stakeholders directly and indirectly involved in the planning, operation and delivery of the two ECoC programmes;

-A survey of 163 projects that were part of the two ECoC programmes who delivered the cultural programme attached to the intervention;

-A literature review of ECoC and European level information on the two ECoC including application/ bid information, EC Committee reports, cultural programme brochures, web sites and news articles. The ECoC's own external evaluations have also been used to inform the European evaluation process.

Together, the above evidence base provides the evaluation with a valid and rounded set of data to inform the views on the main aspects of the ECoC evaluation including efficiency, effectiveness, sustainability and relevance. This view has been informed by:

-The scale of the consultation exercise. Over 200 individuals have fed their views and opinions into the evaluation process either through face to face interviews, telephone interviews or through the project level survey. The majority was done through face to face interviews over a series of visits to each city by the evaluation team;

-The nature of the consultation exercise. The evaluators were keen to consult with those who had a more indirect and external view of the two ECoC. These stakeholders including journalists, those not directly benefitting from the ECoC (e.g. rejected projects) as well as those working in the wider cultural policy agenda at regional and city level. This ensures the evaluation is not simply based on those who benefitted the most from the ECoC.

Annex 2 – Stakeholders' consultation

The stakeholders were consulted via targeted consultations (interviews or phone interviews) and via an on-line survey, mainly on projects that had participated in the ECoC year or had submitted a proposal that was rejected. For Mons, the on-line survey of projects (survey questions below) was implemented by the contractor supporting the evaluation, while for Pilsen, the contractor relied on the survey of projects that was implemented by the Foundation itself and which questions aligned with the information sought.

The consultations included the team responsible for the implementation of the ECoC, the political statements involved in the project, the projects participating in the programme or having submitted proposals to participate that were rejected, and personalities attached to the cultural tissue in both cities. The objective of the consultations was to have evidence supporting the findings and conclusions of the evaluation. The questions used for the survey of projects in Mons can be found in the report produced by the experts assisting the Commission:

https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/ecoc-2015-evaluation_en.pdf .

Similar questions were included, where necessary, in the survey of projects carried out by the Pilsen 2015 Foundation.

The Commission's minimum standards have not been met for this stakeholders' consultation. The consultation was restricted to relevant stakeholders in the two cities hosting the title. The characteristics of the action and the scope of the evaluation do not make it necessary to extend the consultation to a wider public, as indicated in the roadmap published. 30

Annex 3 details the evaluation questions for which the stakeholders' consultation was used. These regard mainly the questions which answers are not based on factual data. The stakeholders' consultation was particularly useful to find information about the impact of the ECoC in the cultural offer of the city, the participation of citizens and local cultural operators, the building of capacity for local cultural operators and legacy prospects.

The questionnaire used for the on-line survey of projects can be found in the Report supporting the evaluation 31 .

Annex 3 – Methodology applied

The figure below presents the overview of the methodology. A more detailed overview of the methodology and sources used for each of the evaluation questions is presented in the subsequent tables and text.

1. Overview of methodology and tasks of the evaluation:

Table 1.1 Evaluation questions: Relevance, EU added value and coherence

Table 1.2 Evaluation questions: Efficiency

Table 1.3 Evaluation Questions: Effectiveness

Table 1.4 Evaluation Questions: Sustainability

2.Details of the relevant evaluation tasks and methodologies:

Inception phase

The aim of the inception phase was to finalise the evaluation framework and research tools, address the project steering group's comments regarding the proposed method and collect background information on the Mons and Pilsen.

Initial consultations

The task consisted in identifying stakeholders to consult with, targeting people that are knowledgeable about the two cities, their cultural programmes and the achievements, lessons learnt and impact of the ECoC, including national, regional and local stakeholders. Careful consideration was given to identify stakeholders to talk to who were likely to have a more impartial or ‘external’ perspectives of the ECoC rather than simply speaking to people who are directly associated with the ECoC cultural programmes in Mons and Pilsen. Discussion at the kick off meeting identified a need for strong triangulation and to gain opinions from a broader set of stakeholders.

During the initial first visit to the cities described below, the contractors supporting the evaluation specifically sourced stakeholders for consultation who did not directly benefit or take part in the ECoC programme and who they felt would have an honest, impartial and unbiased view of how the programme was developed or implemented. While the aim was not to look for purely negative views, the contractors wanted to speak with stakeholder who, for example:

did not deliver an ECoC project or take part in the cultural programme attached to the ECoC;

were a member of the local and national press who had written either an honest or negative piece on the ECoC (as opposed to simply writing a press release which, for example, communicated the content and dates of the cultural programme);

had applied for ECoC funding (to be part of the ECoC cultural programme) and were rejected;

were involved in ECoC activity but were often vocal about a particular aspect (e.g. the quality of culture on offer, the lack of audiences, the lack of diversity of cultural content);

were not directly involved in the cultural policy agenda in the cities but instead had an economic, social or environmental view on the ECoC.

The list of stakeholders consulted is provided in the annex to the Report 32 . This list was partly developed by the ECoC delivery teams but also through an internet search for stakeholders who work in the above fields. The list was further expanded during the second visits themselves (through a snowballing technique - i.e. asking stakeholders for other stakeholders who could be useful "external" individuals to consult). With this in mind, in Pilsen, the list of stakeholders consulted includes:

a former Programme Director who left after a difficult early development period for the ECoC;

a Director of a local theatre in the city (who is a key part of Pilsen's cultural offer) who did not receive any ECoC funding but ran a range of cultural activities during the ECoC year;

two local journalists who were generally critical of the ECoC during its development stage and who produced press content that pointed to weaknesses to the proposed cultural programme;

three Board members of the Pilsen ECoC who were generally vocal about the shortcoming of the programme during its design.

Within Mons, the list of confirmed stakeholders interviewed includes:

a journalist from a national newspaper, which provided coverage of positive and negative stories relating to the ECoC during the preparation phase and the title year;

an academic researcher from the Université catholique de Louvain, who has undertaken independent research into the ECoC and its effects;

a representative of the City of Bruges, who had been involved in Bruges2002 and managed the Bruges's participation as a "partner city", although Bruges did not receive funding from Mons2015;

a current member of the regional parliament of Wallonia, who was previously deputy Mayor of Mons between 2012-14 but not during the title year.

The following organisations were also invited to participate in an interview with respect to Mons:

ERDF Managing Authority for the Wallonia region;

Regional government of Wallonia (Direction générale opérationnelle de l'Économie, de l'Emploi et de la Recherche);

Cultural operators that responded to the call for projects and whose projects were not selected for co-financing; and

Large foreign companies that have located in Mons in recent years, e.g. Google, Microsoft.

First visits to the cities

Initial face to face visits were made, to Mons on 10 November and Pilsen at the end of November 2015. Those consulted with during the initial visit to Pilsen were:

the overall Director for the ECoC responsible for its design and implementation;

the Programme Director for the ECoC responsible for the content of the cultural programme and the delivery of some of its larger projects;

a Programme Manager for the ECoC programme responsible for partner liaison and the delivery of ECoC projects run centrally by the delivery team;

the Manager of the DEPO 2015 project, often seen as the most significant and important project of the programme (see previous chapter for a description).

More ad-hoc and shorter discussions were also undertaken with five staff working in local cultural facilities which were toured during the first visit.

Those consulted during the initial visit to Mons were:

Yves Vasseur (Commissaire général Mons2015);

Marie Noble (Commissaire adjoint artistique Mons 2015);

Pascal Keiser (Chef de Projets Technologies Mons2015); and

Yoann Waroquier (Welcome Team Mons2015).

As in Pilsen, short ad-hoc discussions were also undertaken with staff working in local cultural facilities which were toured during the first visit.

These visits have also allowed us to establish contacts with the managing teams, introduce ourselves and outline the evaluation approach, discussing our information and data requirements. We have also been able to agree the approach to future research tasks including the stakeholder interviews and initial information bulletins, as well as witness some cultural events and visit key venues, institutions and projects.

Desk research including big data feasibility

This involved collection and review of literature, primary data from the delivery agency and available secondary data, as well as web and social media statistics. The list of the literature that has been collected at the European level which has been reviewed and assessed for usefulness for the evaluation can be found in the Report 33 . Much of the European level literature is helpful to either understand the policy drivers at EU level which support the two 2015 ECoC or provide background on the bidding and application stages for both 2015 cities (i.e. little of it provides information useful for the content, delivery and impact of the two programmes). More academic literature at the EU level linked to the ECoC has also been searched for. Instead of looking at academic literature generally on ‘European culture’ and also literature before 2015, we specifically looked at up to date (i.e. produced in 2015) pieces which linked to the ECoC Action. This search did not prove fruitful, although some academic literature at the city level was found.

To complement the above desk research task we contractor supporting the evaluation assessed the extent to which each ECoC has used big data as well as analyse web and social media to increase visibility and interest in the ECoC among country residents and internationally.

Online survey of projects

The survey of projects has proved to be an important source of complementary quantitative data for previous evaluations, and was done for 2015, supported by additional qualitative interviews of projects .The survey was performed by the contractor supporting the evaluation for Mons, while for Pilsen they relied on the survey performed by the implementing body itself.

Second visits to the cities

With a clear view emerging from desk research and project survey as to the overall relevance, efficiency and effectiveness of each ECoC, the contractor undertook a three day visit to each city. The visit focused on conducting in-depth interviews with the managing teams, their key stakeholders and cultural operators identified during the initial visits- please see the annex for more information.

Building on the interviews undertaken in the first visits, the interviews with the delivery teams served four purposes: i) gathering further factual data and information about the cultural programme and its achievements; ii) identifying the "story" of the ECoC throughout its lifecycle, i.e. conception, application, development, delivery, legacy; iii) gaining a critical (albeit "insider") perspective on the relevance, efficiency, effectiveness and sustainability of the ECoC; iv) triangulating emerging findings, for example, those emerging from the desk research and project survey.

Interviews with other key stakeholders in the city (e.g. municipality, chamber of commerce, commercial sponsors, media representatives, and cultural operators) were also essential to identifying the overall "story" of the ECoC from conception, through application, to development and delivery. These individuals offered an external, alternative and broader perspective on the ECoC.

Additional project and stakeholder interviews

These interviews seek to gather an in-depth understanding of the effect that ECoC had for the individuals involved, their organisations and the local culture sector as a whole. It also allowed to ensure that findings are based on consensus across a range of target groups as well as to identify key differences in the experience of different stakeholders. In the case of projects, the criteria for inclusion focused on whether it represents good practice in relation to at least one of the key dimensions of the evaluation, including (but not limited to) increased European cooperation, the effective targeting of key groups of citizens or neighbourhoods, audience development strategies, or legacy effects.

Analysis and reporting phase

Once all the information gathered, the experts fulfilled an analysis of the available information, drawing conclusions and triangulating data to ensure consistency and accuracy. The limitations of the data gathered have been explained in the report and in the Staff Working Document. After the first drafting exercise, the results have been shared with the ECoC for a factual check. Both the Mons and Pilsen ECoC were asked to undertake a check on the completeness of data being used to address each evaluation topic. Their comments, where relevant, were included in the Final Report submitted by the contractor supporting the evaluation.

(1)

OJ L 304, 3.11.2006, p. 1.

(2)

Resolution of the Ministers responsible for Cultural Affairs, meeting within the Council, of 13 June 1985 concerning the annual event 'European City of Culture' (85/C 153/02);

  http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:C:1985:153:0002:0003:EN:PDF .

(3)

Decision 1419/1999/EC of the European Parliament and of the Council of 25 May 1999 establishing a Community action for the European Capital of Culture event for the years 2005 to 2019; OJ L 166, 1.7.1999, p. 1. Decision as amended by Decision No 649/2005/EC (OJ L 117, 4.5.2005, p. 20).

(4)

Decision No 445/2014/EU of the European Parliament and of the Council of 16 April 2014 establishing a Union action for the European Capitals of Culture for the years 2020 to 2033 and repealing Decision No 1622/2006/EC (OJ L 132, 3.5. 2014, p. 1). This Decision, which covers the ECoC titles 2020 to 2033, retains the general structure and main elements of the previous Decision while introducing improvements to maximise the benefits of holding the title as well as taking part in the competitive process for all bidding cities and their citizens. Improvements include among others the introduction of more explicit and measureable criteria, the reinforcement of conditionality for the payment of the Melina Mercouri Prize and the obligation for the cities – instead of the Commission – to carry out the ex post evaluation of the ECoC year

(5)

  http://ec.europa.eu/smart-regulation/roadmaps/index_en.htm .

(6)

See evaluation reports at:

http://ec.europa.eu/dgs/education_culture/more_info/evaluations/index_en.htm?page=1&mxi=12 .

(7)

See list of interviewees in Annex 1 of the Final Report of the contract supporting the evaluation. A total of 17 people in the Mons 2015 Foundation and 7 people in the Pilsen 2015 Foundation were interviewed. The Final Report is available at: https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/ecoc-2015-evaluation_en.pdf .

(8)

A total of 14 other stakeholders in Mons and 20 other stakeholders in Pilsen were interviewed.

(9)

43 projects responded to the survey in Mons while 120 projects responded to the project survey carried out by the Pilsen 2015 Foundation.

(10)

http://ec.europa.eu/smart-regulation/roadmaps/docs/2015_eac_012_capitals_of_culture_en.pdf

(11)

The budget allocated to the evaluation work is proportionate to the level of EU funding directly provided to the ECoC (€1.5m Melina Mercouri Prize).

(12)

Decision No 1622/2006/EC requires that the Commission conducts the evaluation immediately after the title year.

(13)

DEPO2015 was originally a derelict bus depot located about half a mile from the centre of Pilsen. As part of the ECoC project, the bus depot was converted into a living space where businesses and culture are combined in innovative solutions. It hosted many activities of Pilsen 2015 and is now hosting co-working offices as well as a workshop for creative industries to make and build various products and art installations.

(14)

See footnote 6.

(15)

Evaluation conducted by KEA European Affairs for the Mons 2015 Foundation (p. 15); http://www.keanet.eu/wp-content/uploads/Mons-2015-ECoC-demystifying-the-risk-of-cultural-investment-ENG.pdf

(16)

Pilsen2015 project survey carried out by the West Bohemian University.

(17)

65 % of projects taking part in the Pilsen2015 project survey carried out by the West Bohemian University reported that they thought the overall programme was well run, which is relatively high considering the large difficulties and negative press which it encountered in its early days. The survey carried out by Ondrej Jirkovsky (as yet unpublished) on the perceptions of local Pilsen residents on the quality of culture in Pilsen showed that 76-80 % of residents felt that the city now has a much higher quality offer for them to enjoy.

(18)

See example of Mons given under point 6.1 "Relevance".

(19)

See data given under point 6.1 "Relevance".

(20)

See data already provided under point 6.1 "Relevance".

(21)

Evaluation conducted by KEA European Affairs for the Mons 2015 Foundation (p. 16); http://www.keanet.eu/wp-content/uploads/Mons-2015-ECoC-demystifying-the-risk-of-cultural-investment-ENG.pdf .

(22)

For Mons, see http://www.keanet.eu/wp-content/uploads/Mons2015-Rapport-IV-final-19072016.pdf?4f4eb7 , p. 47. For Pilsen, 42 % of ECoC projects reported that the technical skills linked to performance production, marketing, ticketing, lighting and social media were all being used beyond the year itself to provide stronger, bigger and higher quality activities post-2015;

(23)

https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/ecoc-2015-evaluation_en.pdf

(24)

Pilsen2015 project survey carried out by the West Bohemian University.

(25)

See in particular KEA evaluation, op. cit., p. 11.

(26)

  http://ec.europa.eu/dgs/education_culture/more_info/awp/docs/c-2014-5313_en.pdf

(27)

  http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:02012R0966-20160101&from=EN

(28)

  http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:02012R1268-20160101&from=EN

(29)

  http://ec.europa.eu/smart-regulation/roadmaps/docs/2015_eac_012_capitals_of_culture_en.pdf

(30)

http://ec.europa.eu/smart-regulation/roadmaps/docs/2015_eac_012_capitals_of_culture_en.pdf

(31)

  https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/ecoc-2015-evaluation_en.pdf  

(32)

  https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/ecoc-2015-evaluation-report_en.pdf

(33)

See previous footnote.