EUROPEAN COMMISSION

Brussels, 30.4.2018

SWD(2018) 159 final

COMMISSION STAFF WORKING DOCUMENT

Mid-term evaluation of Creative Europe

Accompanying the document

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL

Mid-term evaluation of the Creative Europe programme (2014-2020)

{COM(2018) 248 final}

Glossary

1.INTRODUCTION

As required by the legal base of the Creative Europe programme the Commission has undertaken a mid-term evaluation of the programme, covering also the ex-post evaluation of its predecessor programmes (Media 2007, Culture 2007-2013 and Media Mundus) 1 .

The evaluation covers the implementation of the Creative Europe programme, its predecessor programmes and all their sub-programmes, in the territory of all countries which are eligible to be programme beneficiaries.

The purpose of the evaluation is to:

- Assess the effectiveness of the measures taken to achieve the Programme's objectives, the efficiency of the programme and its European added value.

- Address the Programme’s internal and external coherence, the continued relevance of all of its objectives, and the scope for simplification.

- Address the contribution of Creative Europe to the policy priorities of the EU.

- Assess the long-term impact of the predecessor programmes.

This SWD is based on an external and independent evaluation as well as thematic studies, a wide range of data sources and stakeholder dialogue, including an online public consultation.

The methodological approach to the independent evaluation combined a variety of data collection and assessment methods, as explained in chapter 4 (Method of the Evaluation): (1) collection and analysis of programme data and (2) contextual data; (3) Six Focused Evaluations including counterfactual analysis; (4) Surveys among the beneficiaries of the programme; (5) scoping and key informant interviews; (6) an Open Public Consultation on Creative Europe.

Only a limited number of Creative Europe projects were finalised at the time of the evaluation (1.5 % for Culture) making difficult to measure the real impact of the programme. The evaluation is therefore mainly based on applications' data and stakeholder consultations and for indicators like job creation the data was extrapolated from the MEDIA and Culture surveys (see chapter 5.3). The surveys targeted all the successful project lead applicants and responses received were equivalent to approximately 20% of target group.

Furthermore the existent indicators, as established by the legal base, were not sufficiently related to the outcomes of the schemes and projects of the Programme. In this regard the Commission services prepared a concrete proposal to strengthen the monitoring framework system (see table of the revised set of "Creative Europe" indicators in annex 4 ), that should result in the adoption of supplementary performance indicators by a "Delegated Act", in line with article 20 of the Creative Europe Regulation.

The implementation of the new set of indicators will allow a proper monitoring of the programme. This monitoring will be done regularly, at 3 stages, based on project data: (1) data from the applications of selected projects; (2) data related with the implementation of the projects; (3) once finalised and validated by EACEA, output data of the projects. These data will feed the Qlikview dashboard in real time and the regular monitoring process by EACEA.

Finally, these data will be gathered at programme level to feed the final evaluation report, assessing the impacts and the sustainability of the programme to be submitted to the European Parliament and the Council by 30 June 2022, as established in the legal base 2 .

This Staff Working Document summarises the main findings of the evaluation and accompanies the mid-term Evaluation Report submitted to the European Parliament and the Council.

2.BACKGROUND TO THE CREATIVE EUROPE PROGRAMME

Creative Europe (2014-2020) is the European Union's framework programme for support the European cultural and creative sectors 3 .

The value of the cultural and creative sectors (CCSs) in Europe was EUR 560 billion in 2014, representing 4.5% of EU GDP, whilst it employed 8.5 million workers, equivalent to 3.8% of Europe's workforce. European cultural and creative sectors are inherently diversified along national and linguistic lines, thus contributing to a rich cultural landscape.

Cultural and creative sectors encompass all activities based on cultural values and/or artistic and other creative expressions. They are a major asset for Europe because they produce works rooted in our unique cultural heritage, talent and capacity for innovation. They are a source of innovative ideas that can be turned into products and services that create growth and jobs across the economy. They also shape our values and identities and therefore have an intrinsic value for European citizens.

At the same time, cultural and linguistic diversity gives rise to obstacles to the circulation of works and the mobility of cultural operators which hampers the development of a shared cultural area and limits the choices available for citizens and consumers.

Whilst fully respecting the principle of subsidiarity, the CE Programme seeks to complement and build on the support policies of the Member States. The Programme provides combined financial support mainly through grants and a loan guarantee facility. It responds to the main challenges affecting the sectors, namely fragmentation along national and linguistic lines, the transformation of the value chain by the digital revolution and the globalisation of the economy leading to the tough competition from global players.

Recognising the intrinsic value of culture as a public good and the dual societal and economic value of the cultural and creative sectors, the Programme pursues the cultural, industrial and training goals of the European Union. The twin general objectives of the Programme are to safeguard cultural and linguistic diversity and to strengthen the competitiveness of the European cultural and creative sectors.

The specific objectives are to support the sectors' capacity to operate transnationally and internationally; to promote the transnational circulation of works and the transnational mobility of operators; to strengthen the financial capacity of SMEs; to foster policy development.

Furthermore it pursues cross-cutting themes of supporting economies of scale and critical mass, as well as ensuring a more level playing field, by taking account of low production capacity countries, in particular in the audiovisual sector.

The programme structure, aligned with the objectives, comprises three parts: (i) a MEDIA Sub-programme, supporting quality content, cross-border access and distribution, promotion of works and skills in the audiovisual sector; (ii) a Culture Sub-programme, supporting cross-border cooperation, platforms, networking in all cultural and creative sectors (except audiovisual), as well as literary translations; (iii) a Cross-sectoral Strand, including a Guarantee Facility and transnational policy cooperation and support to policy development.

The programme took into account the specific nature of the different sectors, their different target groups and their particular needs through tailor-made approaches within two independent sub-programmes and a cross-sectoral strand.

Creative Europe is open to cultural and creative organisations from EU Member States, as well as non-EU Member States. It provides support mainly to SMEs, non-profit organisations as well as public bodies

The Programme allocates funds through annual Work Programmes and promote simplification of implementation through e-applications, lump sums and rationalisation of procedures.

A network of Creative Europe Desks, covering all participating countries, has the mission of reaching out to potential beneficiaries and raising the visibility of the Programme in the sector and more widely.

Creative Europe programme intervention logic:

NEEDS:

-Consolidated market for cultural works with optimal circulation and large choice of consumers

-Capacity of the cultural sector to adjust to the impact of globalization and the digital shift

3.IMPLEMENTATION / STATE OF PLAY OF CREATIVE EUROPE

During the period 2014-2016, the volume of support provided by Creative Europe to the cultural and creative sectors (CCS) has amounted to €544 million, subdivided between €327 million and €179 million for the MEDIA and Culture Sub-programmes respectively as well as €38 million through the Cross-sectoral Strand (including the €14.8 million committed for the implementation of the CCS GF).

Since the start of the Programme, around 2,580 CCS entities have benefited from the Creative Europe support, including 1,280 audiovisual companies and 1,300 cultural organisations. Around 150 entities and operators received support under the Cross-sectoral Strand. The CCS GF has been implemented only recently. As of June 2017, the first SMEs have started to receive loans under the scheme, through the first participating bank in Spain.

1Creative Europe MEDIA Sub-programme: Supported schemes and spend per scheme (million)

2Creative Europe CULTURE Sub-programme: Supported schemes and spend per scheme (million)

3Creative Europe Cross-Sectoral Strand: Supported schemes and spend per scheme

4.METHOD OF THE EVALUATION

This evaluation was supported by an external and independent study, together with thematic studies and stakeholder dialogues. The methodological approach to the independent study combined a variety of data collection and assessment methods:

1.Collection and analysis of programme data included collection, structuring, review and analysis of data from programme databases and documentation, , including data on project applications, final project reports and programme management.

2.Collection and analysis of contextual data have been undertaken continuously and up to the interim stages of the evaluation. The key external data sources reviewed and used in the analysis include the contextual data collected by the European Audiovisual Observatory (EAO), Eurostat and Federation of European Publishers’ Annual Statistics as well as other surveys (e.g. Eurobarometer), reports and studies.

3.Six Focused Evaluations (FEs) covered a total of 10 key schemes under MEDIA and Culture sub-programmes field, assessing the performance of these schemes across the six evaluation criteria and six sectors. The methodologies for the FEs included counterfactual analyses of data on 70 supported projects and 30 rejected applicants’ case studies.

4.Surveys among the beneficiaries of the programme (i.e. projects supported by the current and predecessor programmes). The survey's results are analysed in annex 2 together with the responses to the OPC. The purpose of the surveys was fourfold:

- to complement the existing programme and project monitoring data feeding especially ‘softer’ indicators and evaluation issues for which monitoring data is not available; - to inform the evaluative assessment especially as regards to effectiveness and added value, collecting across the main project types, under the Creative Europe Programme as well as from completed projects under the MEDIA, MEDIA Mundus and Culture Programme; - to inform the efficiency and implementation assessment of these programmes; - and to inform the relevance and coherence assessment. The surveys targeted all the successful project lead applicants under the Creative Europe Programme and predecessor programmes selected schemes. Over 1,400 responses were received, equivalent to approximately 20% of target group.

5.Around 20 scoping interviews and 103 key informant interviews were carried out with key informants/stakeholders of the current and predecessor programmes, including interviews with national policy makers, sector organisations, funding bodies and Creative Europe Desks (CEDs), programme managers and EU-level sub-sector representing organisations as well as international institutions .

6.An Open Public Consultation on Creative Europe was launched on 23 January 2017 and it was closed on the 23 April 2017. A total of 547 contributions were received (see more in annex 2 – stakeholder consultation).

7.The evaluation of the MEDIA sub-programme also draws on other studies on distribution, online aggregators, training, film heritage, cinema networks and market access. Furthermore, as foreseen in the 2014 Communication on European Films in the Digital Era 4 , the European Commission launched the European Film Forum, a structured dialogue with stakeholders, to develop a common strategic agenda for the audiovisual sector 5 .

The CCSs are particularly difficult to quantify, because of the need to consolidate statistical methodologies at European level and the existence of data gaps 6 . Also, there is an intrinsic problem in measuring the contribution of small European programmes to policy fields governed by subsidiarity where national policies and programmes play a leading role. Furthermore, there are some data gaps on the impacts of the Programme, stemming primarily from weaknesses in the indicators established in the legal base 7 which, although they have been regularly monitored, did not always sufficiently correspond to the outcomes of the schemes and projects financed by the Programme. Taking account of the above limitations the evaluation has been conducted using the best available data.

More information on the methodological approach, data collection and key challenges is presented in annex 3.

5.ANSWERS TO THE EVALUATION QUESTIONS

In accordance with the Commission's Better Regulation Guidelines 8 , the Creative Europe interim evaluation addresses evaluation questions under each of the sections, which are structured around six evaluation criteria: relevance, coherence, effectiveness, efficiency, sustainability, and EU added value. 

5.1.RELEVANCE

Overall, the programme’s rationale for intervention and objectives remain relevant to the key challenges of the cultural and creative sectors and EU policy priorities. The Impact Assessment of CE found that cultural and creative markets are fragmented along national and linguistic lines, whilst the digital disruption of the landscape is ongoing and global competition from major players is intensifying. Greater access to finance is more pressing than ever because of the need to invest in such a competitive environment. The Programme's support to cross-border circulation and cooperation as well as to innovative market instruments therefore continues to be relevant to help the sector face these challenges. At the same time the low budget limits the scope of the action and its impact and dampens the ambition of the cultural organisations for cross-border activity which is badly needed given the high fragmentation of the cultural landscape.

Creative Europe also responds to both the ongoing and emerging needs of the AV and Culture sectors. As emerged from the review of stakeholder consultation/interview results, Creative Europe continues to address the ongoing needs of the European CCS, and even more so than the previous programmes thanks to its ‘business-oriented’ dimension. The main ongoing needs of CCS are development of skills and talent, developing quality and innovative content, cross border circulation and promotion and access to finance.

Concerning emerging needs, in the audiovisual field, European films have lower audiences than US films and are on average available in fewer European countries. The MEDIA Sub-programme responds to this challenge by strengthening the capacity of European works to appeal to cross-border audiences and to be distributed. Also, the digital revolution is changing the audiovisual landscape i.e. the arrival of global online players in Europe, the drop in DVD sales, the difficulty for many films to be screened in theatres, pressure on broadcaster revenues.

Therefore MEDIA schemes have been adapted. The development schemes support quality content with the main focus being on quality story telling irrespective of its format or distribution model, which is in line with the increasingly blurred distinction between cinema and TV. It also potentially covers film heritage while new opportunities are being offered by digital technology for this category of films. Distribution support, which incentivizes distributors to promote non-national films, also covers online distribution costs. Compared to national support schemes, MEDIA has also the characteristic of supporting activities such as promotion of European works online, audience development or market access or training with a certain focus on the latest technological and market development. Stakeholders including several national film bodies are globally in favour of this orientation.

However, the fragmentation of the MEDIA Sub-programme into 14 separate schemes addressing specific projects and categories of beneficiaries potentially limits the flexibility of the programme and weakens its capacity to foster collaboration across the industry . In particular, the structure itself of the selective distribution scheme has the effect of dividing the same film by as many territories of distribution, which isolates the work of each national distributor from others and from production companies. This is at odds with current practices where a film starts to be promoted already at the production stage. Also, this structure gives insufficient margin of manoeuvre to adapt the schemes during the 7 year period of implementation of the programme.

Also, the MEDIA Sub-programme plays only a limited role in promoting European works in global markets. This could be reconsidered if we consider that the market share of European films outside of Europe is just as high as within Europe and that there are significant opportunities for growth. In fact, a number of producers, distributors and government bodies have called for increased support for international distribution and promotion in third countries.

Regarding the emerging needs in the Culture Sub-programme, stakeholders see the introduction of the European networks scheme as more relevant than the generic operational grants of the previous programme in terms of offering cultural sub-sectors a framework to develop the necessary skills and business models with a view to remaining competitive in the digital economy. They also consider Creative Europe as relevant to respond to the necessity of increasing the transnational and international circulation of European cultural and creative works 9 .

Creative Europe is relevant to the strategies pursued by the Member States. Most national strategies aim to ensure the creative sectors are skilled, resourceful and innovative in the digital era, while recognising the contribution they potentially make to the economy 10 . As such, the strategies pursued at the national level are also relevant to EU policy objectives emphasising innovation and inclusion to foster jobs and growth in the CCS (e.g. Europe 2020). At the same time, national funding schemes have been operating with reduced resources as a result of budget cuts in a number of Member States. Only two EU Member States have increased their CCS funding as a share of GDP post-2008, while 12 have reduced funding and 14 have remained stable 11 . This has led in some cases to a reorientation of priorities in certain countries with a stronger focus on enhancing the financial independence of operators in the CCS, notably through the concept of cultural entrepreneurship. This reorientation mirrors the rationale and objectives of Creative Europe’s Culture schemes, which are more geared towards capacity building and the development of new business models compared to those of Culture 2007-2013.

Furthermore the tax incentives offered in some Member States to attract foreign investments in an effort to enhance the viability of national AV operators in an increasingly competitive global environment are both relevant and complementary to Creative Europe’s financial support towards capacity-building and internationalisation prospects of the EU AV industry 12 .

Creative Europe continues to be fully relevant to EU policy priorities. Creative Europe is part of the 2014-2020 Multi-Annual Financial Framework designed to contribute to the delivery of the Europe 2020 strategy for smart, sustainable and inclusive growth. As such, a wide range of policy and industry stakeholders have noted that the priorities of Creative Europe are highly relevant to those of Europe 2020 in terms of realising the potential of the CCS as drivers of innovation-driven growth and sustainable employment in the digital era. This also credits Creative Europe’s explicit commitment to investing in skills and capacity building to support the consolidation of the AV and culture sectors in a fast-changing and globalised market environment.

Moreover Creative Europe MEDIA facilitates the adaptation of the European audiovisual industry to the Digital Single Market, in particular by underpinning the cross-border access objective of the copyright reform, the promotion and prominence requirements of the AVMSD (Audiovisual Media Services Directive) and helping build the capacity of industry to operate at a European level. For example, several measures adopted by MEDIA have aimed to strengthen capacity regarding online distribution of and access to European works e.g. through support to catalogues for VOD platforms.

On another level, EU policy also underlines the importance of investing in culture to stimulate social cohesion, job creation and economic growth in regions and cities. This priority is, under the programming period 2014-2020, clearly captured under the Creative Europe Culture -funded European Capitals of Culture (ECoC) special action. Indeed, the 2014 ECoC Decision now requires cities to present a long-term cultural strategy when bidding for the title. On that basis, the Commission expects that cultural players in ECoC cities will be able to benefit from local authorities’ strategic support over the longer term, both in terms of capacity-building and inter-city or cross-border networking.

Greater access to finance is more pressing than ever because of the need to invest in such a competitive environment at a time of cuts in public support in many Member States. The Cross-sectoral strand is responding through an innovative market instrument, the Cultural and Creative Sectors Guarantee Facility. The Guarantee Facility, with an initial allocation of € 120 million from 2014-2020, aims to encourage banks to invest in the sector thus boosting the potential for growth. The strong market response to its launch in 2016, with three Guarantee Agreements signed in the first year, shows the relevance of this innovative instrument to the needs of the market. It has been decided that the European Fund for Strategic Investments shall top-up the initial allocation by 50% in 2017.

Lastly, the schemes under the 2007-2013 MEDIA and Culture Programmes and Creative Europe continue to be relevant to the objective of preserving cultural heritage and diversity in Europe as well as promoting artistic creativity, in accordance with Article 167 TFEU and the 2005 UNESCO Convention respectively.

5.2.COHERENCE

The Creative Europe programme is broadly internally (among the different programme strands and the actions within these) and externally coherent (with other EU interventions and policies) although some aspects can be improved.

Creative Europe provides a set of actions to support the AV and cultural sectors and is internally coherent with both the MEDIA and the Culture sub-programmes. No overlaps or duplications in the actions funded have been identified. The sub-programmes mainly function independently of each other. Strategy and planning of the MEDIA sub-programme lie with DG CNECT and equivalent activities for the Culture sub-programme lie with DG EAC. Most stakeholders consulted noted that the coherence of EU support to the CCS improved under Creative Europe 13 as compared to the predecessor programmes. However stakeholders consulted noted that communication between the DGs and EACEA could be expanded and further inputs from EACEA in the strategy and planning process could help identify how to increase the integration of these strands in practice.

The MEDIA Sub-programme schemes cover the traditional value chain of the audiovisual industry in a comprehensive manner, supporting skills and capacity building, content development and production, distribution of works, promotion and access to markets. A specific scheme has been created to cover online distribution. No overlaps or gaps between the schemes have been found whereas they fit coherently together, for example with Development Slate and Development Single schemes supporting portfolios of projects and individual projects respectively.

The MEDIA Sub-programme has helped the European audiovisual industry to adapt to the Digital Single Market. It provides for measures accompanying the copyright reforms, for example through support to catalogues of European films, support to licensing hubs, support to a more efficient use of subtitling and an Animation Action Plan. Also, the new scheme "Promotion of European Works Online” supports the industry’s adaptation to the proposed provisions of the future AVMS Directive on a quota of European works for VOD services and obligation to make such works prominent.

Coherence between EU and national film policies is important given the central role of national film funds. The national film funds represent the bulk of the support to the audiovisual sector, with approximately € 2.5 billion per year compared to € 120 million from MEDIA. The European Film Forum and the dialogue with the European Film Agency Directors (EFADs) were launched to develop a shared strategic vision for the sector, following the Commission Communication on European film in the digital era. Also, the Open Method of Coordination group of Member States was launched in July 2017, further to the request by the Education, Youth, Culture and Sport Council. Its mandate is to identify complementarities between film policies at regional, national and EU level with a view to improving the circulation of films. The dialogue so far has shown that the impact of MEDIA would be strengthened if its objectives were integrated more fully into the Member State film policies and the activities of the film funds.

The Culture sub-programme’s schemes are coherent with each other 14 , although more could be done to clarify the relationship between individual schemes among stakeholders and beneficiaries. While there is strong coherence between Cooperation Projects and respectively Networks and Platforms, the relationship between Networks and Platforms is less clear 15 . Meanwhile, Literary Translation occupies a niche area within the Culture sub-programme. None of the stakeholders or beneficiaries from Cooperation Projects, Networks, and Platforms referred to this scheme, suggesting low awareness and a lack of integration with the rest of the Culture sub-programme. This is likely to be because Literary Translation has a much more targeted beneficiary group than the others.

The Cross-sectoral strand contains a mixture of schemes which have a cross-cutting character together with some activities which benefit mainly specific sectors. As a result the coherence of the cross-sectoral strand is somewhat uneven. The largest activity, the CCS Guarantee Facility, has established terms and conditions which are open to all SMEs in the cultural and creative sectors and with the objective of strengthening the financial capacity across the cultural and creative sectors. The Facility is a major innovation of Creative Europe which diversifies the range of instruments deployed by the Programme compared to the exclusive grant approach of the predecessor programmes. 80% of respondents to the public consultation believed that the Guarantee Facility has the potential to strengthen the coherence of Creative Europe

The special call for Refugee Integration Projects, which provided support to AV, cultural, and cross-sectoral projects that aimed to facilitate the integration of refugees into European communities, could serve as a model for future more cross-sectoral work.

The Creative Europe programme is broadly externally coherent.

Creative Europe’s objectives and priorities are overall coherent and complementary with those of national programmes, thus respecting the principle of subsidiarity. What distinguishes Creative Europe most from national funding schemes is its focus on international and transnational activities, which are rarely funded at national level. Interviewees confirm that Creative Europe fills an important gap, as not only does it provide support for the additional activities of European networking organisations, but it also enables national organisations to reach out across the borders.

Creative Europe is largely coherent with other EU funding 16 and support mechanisms relevant to the AV and cultural sectors. Although other EU funding can benefit the AV and culture sectors, they do not specifically focus on AV and culture and their main objectives are different than those of Creative Europe. Nevertheless, the surveys show that more could be done to increase beneficiaries’ awareness of the existence of other EU funding streams 17 .

Creative Europe is largely coherent with Europe 2020’s objectives of smart, sustainable, and inclusive growth and its flagship, the Digital Single Market strategy launched in 2015. In particular, MEDIA facilitates the adaptation of the European audiovisual industry to the Digital Single Market, by underpinning the cross-border access objective of the copyright reform, the promotion and prominence requirements of the AVMS Directive and helping build the capacity of industry to operate at a European level. For example, several measures adopted by MEDIA have aimed to strengthen capacity regarding online distribution of and access to European works e.g. through support to catalogues for VOD platforms.

On the other hand there is a strong coherence between Creative Europe and the European Agenda for Culture 18 ’s objective of promoting culture as a catalyst for creativity is evidenced through Creative Europe’s support for capacity-building and skills development for CCS operators. In particular capacity building and skills development is embedded across Culture sub-programme schemes, most notably Cooperation Projects and Networks. The MEDIA sub-programme has a specific scheme devoted to training. There is less evidence of coherence with the European Agenda for Culture’s objective of promoting culture as a vital element in the EU’s international relations, as the support is focused mostly on facilitating and supporting transnational relationships within the EU. However, the eligibility criteria for some schemes 19 allow participation from non-EU member states as member organisations.

There is coherence between the objectives of EU membership to the European Audiovisual Observatory (EAO) and the objectives of the EAO, as well as opportunities to make better use of available data.

Lastly, Creative Europe sits alongside a range of sometimes overlapping programmes at the international level but avoids duplication in practice because of the overall shortage of international funding 20 .

5.3.EFFECTIVENESS

Globally, Creative Europe and the predecessor programmes have been or are successful in achieving or progressing towards their objectives, delivering their intended outputs (i.e. from programmed activities), results (outcomes of those activities) and having the anticipated impact.

The achievements of the predecessor programmes have to be examined in the light of the Lisbon Strategy 21 , making significant contributions against the EU strategic priorities:

-The predecessor programmes’ support was essentially oriented towards supporting the CCS and professionals and more could have been done to ‘put the citizen first’ and tap into ‘new’ technologies.

Predecessor programmes delivered on many of their general and specific objectives but could have been more effective in some areas:

- The predecessor MEDIA programme delivers strongly against its key development and circulation objectives, but could have been more effective in following market developments;

- The predecessor Culture programme was effective in increasing the transnational mobility and capacity, but less so in strengthening the operators financially and promoting their work with underrepresented groups;

- MEDIA Mundus was effective in improving European and third countries professionals’ knowledge of audiovisual markets, but less so in facilitating access to films from third countries.

Concerning key outputs, results and impacts, the MEDIA, MEDIA Mundus and Culture predecessor programmes were overall effective in delivering, while MEDIA Mundus impact was fragmented:

Creative Europe is making an important contribution towards the EU2020 employment target 22 and the priorities of the current Commission: stimulating investment for the purpose of job creation; and a deeper and fairer internal market with a strengthened industrial base 23 .

The increase in Creative Europe spending during the first three years, however, almost ‘matched’ the growth of the sectors (20% increase in employment), with the current programme annual spending on average 18% higher than that of predecessor programmes.

With regard to job creation, the Creative Europe Programmes generated an estimated 24 3,000 employment positions in the period 2014-2016 directly in the organisations benefiting from the programmes . One fourth (or around 750) of these positions were estimated to be permanent employment positions while the rest were estimated to be temporary positions (around 2,250).

The support of the Creative Europe programme has led to the creation of an estimated 0.9 full-time employment places per programme beneficiary on average (1.6 FTEs among the beneficiaries of the MEDIA Sub-programme and 0.3 among the beneficiaries of the Culture Sub-programme). The Culture survey and the focused evaluations confirmed the lower propensity of the Culture Sub-programme to create employment positions as part of the Creative Europe Programme as well as the lower share of permanent positions created. This lower propensity could be partially explained by grant amounts that have been on average 46% lower for Culture operators than they have been for companies supported by MEDIA under the current Programme. Furthermore, this lower capacity could be in large part due to the four times lower Programme support to operators in culture than audiovisual sub-sectors.

As for the priority of the current Commission on ‘strengthening the industrial base’, the programmes have strengthened organisations that benefited from funding support, especially in the MEDIA programmes 25 . The majority of respondents reported ‘major’ or ‘moderate’ impact of the programmes’ funding on increased turnover or market position of their organisations.

Creative Europe delivers on its general objectives of safeguarding cultural diversity and strengthening competitiveness 26 .

The MEDIA Sub-programme has focused in particular on support to transnational circulation of works to enable audiences to have increased access to films from other European countries. Over 40% of the MEDIA Sub-programme budget supports distribution and exhibition, extensively contributing to the cross-border distribution of works. For example, in 2014 the Distribution schemes supported over 400 films, equivalent to over 25% of Europe’s annual film production. For the period 2014-16 it is estimated that films receiving MEDIA distribution support reached an average of over 65 million total admissions per year. The external evaluation shows that support has increased the availability of legal offers in a greater number of territories than would otherwise have been the case.

MEDIA co-finances distribution costs including subtitling, which is key to increasing access to works, particularly as it enables reaching cross-border niche audiences online. However, the costs of subtitling can be an obstacle for small productions. Also, in practice the use of subtitles is restricted to the territories for which they were originally intended, with no economies of scale through sharing with distributors from other territories. Therefore, more efficiency needs to be achieved regarding the costs as well as the use of subtitling, especially when they publicly funded.

Support to exhibition is given through the Europa Cinemas network, the MEDIA Sub-programme’s biggest beneficiary. This network spans 33 countries and boosts the theatrical exhibition of non-national European films. It is estimated that 1 euro invested in the network generates €13 for the audiovisual sector through additional audiences.

The Online Distribution scheme allocated on average €5.6 million to 19 companies per year, for the period 2014-16. The evaluation finds that overall the supported projects have performed below the market. According to project data the revenue growth of the seven supported SVOD services, at 7%, has been significantly lower than the growth shown by the wider sector overall, at 113%. This shows that the platforms have attracted very limited audiences. The scheme, and notably the action encouraging the development of new models of promotion and distribution, lacks input from tech companies which could offer relevant know-how, including on issues such as data analytics. During this period, no European wide service or strong network of national VOD services has been established. Due to these mixed results the scheme was revised in 2017 to focus on the promotion of European works online.

The MEDIA Sub-programme has been effective in building the capacity of audiovisual professionals operating at transnational level, as regards creative, business and digital skills. Approximately 60% of training participants reported improved skills and competences. The Development schemes strengthen the ability of operators to develop works with transnational circulation potential, through seed funding. These schemes provide extensive support, funding over 400 projects per year, representing approximately 25 % of Europe’s annual film production (although not all development projects will lead to production). Nonetheless, the external evaluation shows there is a lack of awareness of stakeholders that the support to development covers works irrespective of their format and platform of distribution or transmedia content. Under the Development schemes only 2% of support has been attributed to projects primarily intended for online exploitation 27 .

The MEDIA Sub-programme supported films have achieved international recognition by winning prizes in the world's major film festivals. Success stories include winning the Oscar for Best Foreign Language Film from 2013-2016 and winning the Palme d'Or from the Cannes Film Festival 8 times since 2007. In annex more detailed information is given about success stories. Through these results the MEDIA Sub-programme has become recognized as a quality brand in the industry.

Culture programmes supported most sub-sectors in balanced way, but concerns are raised about the prioritisation of economic objectives. Overall the cultural operators are more reluctant to embrace the new ‘competitiveness’ general objective of Creative Europe Programme. A minority of stakeholders consulted considered that the greater programme focus on competitiveness is strength of the new Programme 28 . Furthermore the current programme supports 45% less projects, compared to the predecessor Culture Programme and the average size of projects supported by the Culture Sub-programme has more than doubled compared to the predecessor period 29 , but the size of operators benefiting from the Culture programmes has changed very little 30 . The increasing size of the projects can be seen as an aspect of Creative Europe’s increased orientation towards market rationale and is also likely to shift the implementation towards organisations and countries that have more experience in managing larger projects. While many projects supported by the Culture programmes are transdisciplinary, the dominants sectors are: Visual arts, Cultural heritage, Performing arts: music and opera, Performing arts: theatre, street arts, circus arts and puppetry (see table).

31 Estimation of the Culture programmes support by culture sub-sector

Source: Culture survey (% of respondents by Culture sub-sector), Programme data (yearly funding)

Activities supported by the Culture programmes contribute to the creation of a ‘European cultural space’ and promotion of diversity, as most of the Culture Sub-programme schemes include a specific focus on audience development, aiming to bring European cultural works to non-national audiences and the schemes (especially the Cooperation Projects) contribute strongly to transnational mobility of creative and cultural players.

The Creative Europe Sub-programmes have been fairly effective in achieving most of their specific objectives and priorities, particularly in promoting the transnational circulation of cultural and creative / audiovisual professionals and works, as well as and increasing their capacity to operate internationally 32 . However, both sub-programmes have been relatively less effective in pursuing the specific objective of increasing access to European cultural and creative works by underrepresented groups.

The contribution of the Creative Culture Sector Guarantee Facility (CCS GF) to priorities of the Cross-sectoral Strand is limited to date, as the CCS GF started to operate two years after the start of the Creative Europe programme. Nevertheless, in 2016, the implementation of the CCS Guarantee Facility has been very effective as three delegations agreements were signed with financial intermediaries and calls were launched to welcome applications from CCS SMEs. At the end of the first semester 2017, all budgetary allocations were committed for the year. The CCS GF is expected to cover 15% to 20% of the estimated financing gap of the sector.

Both MEDIA and Culture currently make significant contributions to the aims of both the Digital Single Market Strategy (DSMS) and the European Agenda for Culture. Because of the separation between the sub-programmes, and the two DGs managing the programmes, the contribution to each strategy may not be effectively integrated. There may therefore be an opportunity to maximise the contribution each sub-programme makes to these strategies.

The Culture Sub-programme contributes to all objectives of the European Agenda for Culture, the promotion of:

(i) cultural diversity and intercultural dialogue - achieved through encouraging and promoting cross-border mobility, the promotion of mobility and digitisation in the field of cultural heritage, and intercultural dialogue. These elements are effectively supported by all schemes under Culture 2007 and Creative Europe;

(ii) culture as catalyst for creativity - achieved through promoting synergies between culture and education, capacity building and skills training, and fostering a favourable environment for the sector’s development. This objective is clearly supported through the Culture Sub-programme schemes’ support to skills training and capacity building;

(iii) culture in the EUs international relations - achieved through enhancing the role of culture in the EU’s external relations, promoting diversity of cultural expression, and fostering cooperation with third countries - is supported by the Networks and Cooperation Projects, which enable cooperation between EU and third-country cultural organisations and in a few cases, contribute to activities in the field of cultural diplomacy.

The MEDIA Sub-programme also makes a significant contribution to the Agenda for Culture but it requires further recognition.

The Digital Single Market Strategy (DSMS) adopted in 2015 is designed to bring the EC’s single market principles to the digital realm with a focus on three pillars: (i) ‘improving access to digital goods and service across the EU’; (ii) ‘creating an environment where digital networks and services can prosper’; and (iii) ‘supporting the use of digital as a driver for growth’.

DSMS is increasingly supported by MEDIA as actions under the Sub-programme with a digital focus are expanding in scope and scale. Under Creative Europe, the scope of MEDIA’s digital focus has widened through the introduction of the Online Distribution scheme, which includes support to European VOD services as well as to digital packages of European AV work and to the innovative distribution and promotion of European AV work.

In addition, some of the MEDIA schemes covering more ‘traditional’ AV activities – such as Training, Cinema Automatic and film development – also have a significant digital component. These contributed to the DSMS in particular through providing ICT/digital capacity building, and through supporting the online visibility of projects.

The DSMS’ new copyright framework includes measures to widen access to online content across the EU, partly funded by MEDIA. In line with this, and to relate Online Distribution further to the DSMS, in 2017 the scheme has been adapted to cover two key objectives: (i) supporting innovation and new distribution/ promotion models to seize the opportunities of online technology; and (ii) increasing European films in catalogues. Support will increase from €5.6 million (2016) to around €9.5 million (2017). These changes are likely to make the MEDIA Sub-programme increasingly efficient in its contribution to the DSMS’s objectives.

There is some concern in the wider AV sector that the increasing alignment of MEDIA with the DSMS – and a corresponding increase in the proportion of funding for ‘online’ actions – will result in a decrease in funds for traditional mechanisms for production and exhibition, and that while a clear strategy exists for the expansion of digital activity, there is a lack of strategic consideration about the implications of digital expansion on traditional AV activities.

In addition the Culture Sub-programme also contributes to the objectives of the DSMS. This is particularly pertinent given that the new copyright agenda includes cultural heritage as a key focus and includes all ‘online goods’ in its remit, suggesting that the culture sector will become increasingly relevant to the objectives of the DSMS.

Creative Europe overall delivers it planned outputs, results and impacts:

56% of the Creative Europe Programme is to be spent on MEDIA Sub-programme, 31% on Culture Sub-programme and further 13% on the measures supported under the Cross-sectoral Strand of the Programme 33 .

During the first three years of operations, 39% of the total Programme budget has been appropriated. The total budget of the Creative Europe Programme represents a 28% increase compared to the combined budgets of the predecessor MEDIA, MEDIA Mundus and Culture programmes.

It also needs to be noted that only around one quarter of the Creative Europe projects were completed by the time of the evaluation. This needs to be considered when examining the extent the Programme has delivered its anticipated results, and especially in relation to impacts that can be expected at this mid-term review stage.

In terms of outputs Creative Europe has delivered or delivers a total of 6,286 projects during its first three years of implementation or on average 2,095 projects per year. This is exactly 15% more projects than yearly averages of the predecessor programmes combined, hence the performance of Creative Europe in terms of outputs is aligned with the relative increase in annual budgets.

The MEDIA Sub-programme is overall effective in the way its training, development, distribution and promotion schemes deliver its outputs and impact.

In terms of outputs, the current MEDIA Sub-programme shows a 26% increase in terms of number of projects supported, with an average annual budget that is almost the same as for the predecessor Programme 34 . The current MEDIA also supports the same number of titles each year (580) across various schemes as the Predecessor (571).

Concerning key impacts, it is too early to assess the impact of the current MEDIA Sub-programme but the comparative assessment of the effectiveness in generating impacts tends to favour the current MEDIA Sub-Programme 35

The Culture Sub-programme delivers against most of its planned outputs and impacts.

In terms of outputs, the current Culture Sub-programme shows a 45% decrease in terms of number of projects supported (from 264 to 146 on average per year) and 33% less operators.

About key impacts, even if it is too early to assess the impact of the current Culture Sub-programme, a comparative assessment of the effectiveness in generating impacts seems to indicate that the current effectiveness of the Culture Sub-Programme is on par with its predecessor 36 .

- The Cross-sectoral Strand overall delivers on its expected outputs and results under transnational policy cooperation and support provided by CEDs 37 .

The Creative Europe Desks (CEDs) are active in each Member State and in all other countries participating in the programme. Since 2014, CEDs were established as fully merged desks covering the whole programme or organised in offices according to the sub-programmes. Their number and structure is decided at national level on the basis of different factors such as sectors dimension and policy priorities. While this flexibility allows countries to adapt to the specific needs of national contexts, it could also cause an unbalance between resources allocated to MEDIA and Culture desks.

Creative Europe spent around €15 million (or 40% of the total budget of the Cross-sectoral Strand) on the services of Desks between 2014 and 2016. The annual spending on CEDs has increased by 6%, comparable to the predecessor Programmes spending on MEDIA desks and antennae and the Cultural Contact Points (from €4.75 million to €5.05 million per year on average).

The CEDs provide a wide range of services that span from promoting the programme (mostly through their website, social media, paper based advertisement and promotional merchandise) and the schemes to assisting organisations with their applications. Overall stakeholders are strongly satisfied with CEDs’ support and the way they reached out to potential applicants, although some smaller stakeholders (e.g. small culture organisations or micro companies in the audiovisual sector) feel that not enough efforts are done in order to reach them.

The unification of the predecessor programmes under Creative Europe encouraged the development of cross-sectoral initiatives, bringing together audiovisual players and culture organisation. However, it seems that less than a fifth of the total events organised in 2015 by the CEDs were joint initiatives, showing that more could be done in this area.

Nevertheless CEDs should do more on transnational cooperation for instance supporting co-productions and collaborative distribution projects, sharing success stories and best practices between different countries.

Creative Europe is open to a diversity of participants, including smaller operators and companies. Overall, the organisations participating to the analysed schemes are well spread across Europe. The Culture pool of beneficiaries is composed mostly of Public organisation (24%) and Non-profit association (61%), while the MEDIA one is mostly composed of private companies (78%).

5.4.EFFICIENCY

Considering the huge size and range of the audiovisual and cultural sectors and the geographic areas covered, the limited budgets allocated to the programme were not sufficient to create a major impact on a European scale and/or at sectoral level. However, the actions implemented contributed to such impacts for specific schemes and in specific sectors.

EUR 1.1 billion were channelled to the sectors by the three predecessor programmes from 2007 to 2014 whilst the Creative Europe Programme has a budget of €1.46 billion for 2014-20. Creative Europe funds for 2014-16 were € 544 million, equivalent to an annual support of 0.03 % of the value of the sector, estimated at € 560 billion.

Creative Europe was implemented efficiently with the absorption rate for the programme reaching 86.7% of the planned budget from 2014 to 2016 38 . Annual amounts awarded represent less than 0.1% of the estimated annual contribution of the sectors to the EU GDP. Given the programme budget, Creative Europe and predecessor programme could only have been expected to somewhat contribute to generate impacts at sector and/or EU level 39 .

Amounts awarded (in EUR million) under Creative Europe at mid-term (2016 data incl.) and under the predecessor programmes

ICF analysis of EC provided databases with financial data (Maximum grant awarded)

Shortcomings at different stages of the implementation of the Programme were identified, and although improvements materialised over time, there is room for further improvement.

The application process and reporting requirements are perceived as burdens by the beneficiaries, but have been improved from predecessor programmes to Creative Europe. The application processes were clear to the applicants but slightly burdensome 41 and management processes are more bureaucratic than for some of the Member States funds.

Simplification of the application process took place from one programming period to another concerning, inter alia, the digitisation of the application process (and kick-off for the reporting process), clearer templates and guidelines; higher selection threshold; and more focused selection criteria.

Simplified grants in the form of lump sums were introduced for MEDIA in 2014 to simplify the programme management. This has sped up payment procedures and has simplified application and reporting requirements for beneficiaries. However, the amounts of the lump sums need to be updated to realign them with the real costs incurred by beneficiary projects, as these have decreased, notably because of digitisation. The nature of the grants under the Culture sub-programme did not allow for an efficient introduction of lump sums.

MEDIA accounts for 40% of total number of payment transactions processed by the EAC Executive Agency and only contributes to about 25% of the overall spent budget, on a yearly basis. Furthermore, the Sub-programme entails over 1,500 commitments for about 2,000 projects per year. This leads to quite high administrative costs.

The scope of the MEDIA Sub-programme was widened without a corresponding increase in the budget. For example, the Video Games scheme has an annual budget of € 2 million and can only play a modest role vis à vis the video games industry which had a value of USD 110 billion in 2015.

The level of competitiveness and the quality of the proposals increased 42 from one programming period to another reflecting that the schemes are insufficiently funded compared to the potential interest they generate; a large number of good applications are rejected and at the same time a relevant number continue to be of low quality / ineligible. While an indication of its highly competitive nature and high level of demand, the very low rate of projects selected undermines the reputation of the programme and leads to frustration among many rejected applicants. The MEDIA sub-programme has higher selection rates in closely targeted schemes such as Cinema Automatic. However, the Development schemes have a lower selection rate of 15% because of high demand. While an indication of the highly competitive nature of the schemes, low selection rates represent a waste of resources for rejected applicants and leads to frustration.

Creative Europe confronts important challenges on the monitoring of the programme, mainly related with the implementation of the indicators as established by "Creative Europe" legal basis 43 :

- The hierarchy of general and specific objectives listed in the Regulation applies uniformly to the three strands of the Programme; however there are numerous (15) and repetitive (for MEDIA and Culture) priorities that could be easily generalised to apply to the whole CCS;

- Several of the objectives and priorities of the programme and sub-programme don't have the correspondent indicators;

- Some indicators specified in the Regulation are not appropriate for the purpose of monitoring of the Programme’s progress and performance against intended objectives;

These limitations were the main reason for the restricted monitoring practices at programme level, namely the lack of systematic reporting from the Commission on the indicators provided in Article 18 of the Regulation and to the fact that data has not been aggregated to assess the Programme’s progress in reaching planned results. Furthermore due to the duration of the projects just a limited number of projects were closed at the time of the evaluation (2016 data).

In order to deal with the complexity of indicators implementation, during the external evaluation a seminar was organised, to reflect on "performance monitoring framework", involving Commission services (EAC, CONNECT) and EACEA. As a result, the external evaluators prepared a note with a proposal of a new performance indicators framework, including a revision of the indicators existing in the Regulation 1295/2013.

Based on the proposition and its own assessment the Commission services already prepared a a concrete proposal to strengthen the monitoring framework system, processes and indicators for the Creative Europe Programme. It will result in the adoption of supplementary performance indicators by a "Delegated Act", in line with Article 20 of the Creative Europe Regulation. At the same time the EACEA is adapting the reporting templates and processes, accordingly.

Reporting requirements varied across schemes and sub-Programmes. The majority survey respondents agreed that reporting requirements are reasonable given the grant provided. Qualitative evidence collected from the interviews shows that on average organisations devoted more than one month of a full time employee to comply with reporting requirements. Beneficiaries also noted the heaviness of audit and verification processes (and their duplication when co-funding applied). This being said reporting arrangements and associated burden have also improved from predecessor programmes to Creative Europe .

5.5.SUSTAINABILITY

Largely the actions of the programme are continued in some form after the end of EU financing support and its effects remain. The sustainability of the results lies primarily in the continuation of partnerships that were started under Creative Europe and the predecessor programmes and that largely is continued in some form beyond the projects’ lifecycle 44 .

Under MEDIA, the sustainability of audiovisual companies has been strengthened as the support helps them to compete in tough international markets. Beneficiaries state that there are activities which would have been implemented only to a limited extent without the Creative Europe MEDIA Sub-programme support. In Distribution Automatic half of the funded companies would have invested less in European non-national films without the funding. Therefore, MEDIA support is addressing market failures. This is underpinned by the public consultation, where 61 % of MEDIA Sub-programme respondents indicated that they improved their market position due to Programme support. This translates into the continued intention of organisations to continue European filmmaking i.e. a sustained interest to develop, co-produce or distribute works with a European dimension.

Collaboration on transnational projects can additionally lead to sharing of best practice and development of new approaches. Moreover, the European Film Forum (EFF) has established a structured dialogue with stakeholders to share knowledge and understanding about the challenges facing the industry and possible solutions. These exchanges of best practice, innovations and understanding have sustainable effects which last beyond the duration of the activities themselves.

Concerning Culture sub-programme schemes, the sustained impact is found in partnerships and in the exchange of peer learning that is facilitated by the projects. Across all Culture schemes, beneficiaries developed stronger relationships with project partners and learnt from their respective experiences 45 . Participants also gained greater awareness of their sector at an international level. Reported sustainable benefits relates to professionalization and internationalisation of careers, skills development, and building capacity which were often developed thanks to the international dimension of the projects given by Creative Europe support 46 .

For some schemes (notably Cooperation Projects, Networks and Platforms), dissemination is an important mechanism for achieving the desired wider effects on the sectors, in helping to share what is learnt at an international level. Beneficiaries of these schemes are overwhelmingly positive about the impact of their dissemination activities, although these generally appear to be relatively passive forms of communication, such as websites or social media. Given the importance of dissemination, however, partners could have placed more focus on engaging with on their national audiences, and also have made more use of the CEDs as a mechanism for further sharing of results.

Through the Cross-sectoral Strand, Creative Europe Desks play a valuable role in sharing knowledge and disseminating best practices across the sectors. However, the evaluation shows that partners and beneficiaries could also have promoted results more and made more use of the Desks as a mechanism for further sharing of results. Only around a third of survey respondents consider their outputs ‘fully exploited’.

Programme-level dissemination events have to date focused on either Culture or MEDIA, rather than integrating the two strands. For Culture, the project showcase conference which used to be held annually under Culture has now been restarted under Creative Europe. The conference mainly showcases beneficiaries of the Culture sub-programme, and there may be scope to enhance the synergies between the programmes by having shared conferences or networking events which address cross-sectoral issues.

5.6.EU ADDED VALUE

In general, the value resulting from Creative Europe is additional to the value that could result from interventions which would have been carried out at national or regional levels alone.

The main EU added value comes from the strong international dimension of the programmes. There are few international funds that support transnational cooperation and those that do it, are of significantly smaller scale. Projects and activities undertaken without Creative Europe support would therefore have been of a largely national/ regional/local focus and of a reduced size and scope, if they were delivered at all. Some partnerships, created at application stage, were even maintained despite the project not having been awarded funding 47 . Established networks and partnerships furthermore facilitate subsequent transnational operation, as they reduce the costs of identifying, contacting and vetting partners.

Concerning the MEDIA Sub-programme, several activities are unique and fill gaps in the national support programmes. For example, the Development scheme with its European dimension is not replicated at national level and plays a significant role in facilitating productions which appeal to cross-border audiences. This is also true for the Training scheme, whose transnational dimension is not covered by the Member States. More than 1800 professionals are trained annually through transnational and international projects.

Transnational networks and cross-border partnerships, such as the European Film Promotion network of European promotion bodies, provide durable EU added value by facilitating transnational operations as they reduce the costs of identifying partners and defining joint activities. 82% of the beneficiaries say they have continued collaboration with their project partners.

As shown in the external evaluation, the MEDIA Sub-programme helps to scale up size and scope from a largely national focus to an international one. Thus MEDIA has supported a genre called 'European cinema', but also by creating an appetite for diverse 'European' content and supporting the distribution of films that would otherwise have found it more difficult to travel and be screened in cinemas.

The support from Cinema Automatic and Cinema Selective schemes provides an important source of revenue for cross-border film distribution. Without this funding, many films would not have been released outside their home countries, as companies would not have taken the risk to distribute them. Distribution of films made in one EU Member State to another is thus enabled because of the funding. The TV Programming scheme has allowed companies to increase their production capacity and take part in more ambitious projects capable of appealing to cross-border audiences. 64% of respondents to the public consultation believe that the MEDIA Sub-programme has complemented existing actions at national level by supporting audiovisual subsectors or operators that would not have received support.

For MEDIA, there is evidence of this structuring effect through knowledge-sharing across those schemes which have a focus on transnational partnerships, for example a peer learning effect around how other countries have used the scheme the TV Programming scheme, or on Distribution schemes, beneficiaries saw one of the main benefits of the funding as bringing professionals together and collectively supporting their capacity to collaborate and operate transnationally.

The MEDIA Sub-programme has applied positive discrimination measures to increase the participation of companies from low and medium capacity countries in the programme. Several Member States raised concerns about unintended consequences of these measures which led to some distortion of the results of the schemes. Therefore a rebalanced set of measures was adopted within the 2018 Creative Europe Work Programme. The external evaluation indicates that it remains nonetheless necessary to define more clearly the needs of lower capacity countries and the role of the Creative Europe MEDIA Sub-programme in addressing them, in full respect of subsidiarity.

For Culture, even if activities funded are less closely embedded in existing processes, estimates suggest that the largest share of organisations would have not implemented or have implemented a significantly reduced number of activities without EU support 48 .  Only a minority of Cooperation Projects are likely to have happened without the funding received through the scheme, and if they were delivered it would be with significantly reduced scope or different focus 49 . Similarly for Network project activities 50 and Platform projects 51 would not have taken place without support from the scheme or alternatively, activities would be of a much smaller scale or depth. For Literary Translation, the difference between supported and unsupported projects is less significant. Without the funding, publishers are likely to translate around two-thirds of the books that they would have translated with EU funding but these works would have been translated to a less diverse range of languages; the scheme therefore supports the objective of cultural diversity.

There is evidence of the consolidation or strengthening of cultural operators for the Culture sub-programme in a certain sub-sector (structuring effects) over time across Networks, Platforms, and Cooperation Projects. For instance 52 Network projects were able to take best practice, for Platforms, this consisted primarily of consolidation within a sector and Cooperation Project beneficiaries were supported to develop an international profile and to operate in non-national contexts. Nevertheless when projects are smaller it appears less likely to result in a structuring effect.

Under the Cross-sectoral strand, the added value is difficult to assess at this stage as most activities have only recently been launched. However, the Guarantee Facility has high potential EU added value because it fills a gap in Member State support policies which do not have a dedicated instrument to address the financing gap of the cultural and creative sectors. Moreover, the Guarantee Facility will allow for sharing of knowledge and data across territories as well as the diversification of risk.

6. CONCLUSIONS

The results of this evaluation will help improve the implementation of the Creative Europe from 2018-2020 and will inform the design of the successor programme. This section of the SWD outlines the key conclusions.

The evidence presented in the evaluation shows that, overall, the Creative Europe programme has performed well, with a high level of demand and participation from stakeholders.

During the first 3 years of implementation (2014-2016) Creative Europe supported 2,580 beneficiaries, including 1,280 audiovisual companies and 1,300 cultural organisations. Beneficiaries are small and medium sized companies, non-profit organisations or public bodies. Stakeholders are very supportive of the programme.

As the EU programme for the cultural and creative industries, Creative Europe remains highly relevant. The programme addresses ongoing challenges in the sector, such as fragmentation of markets and the digital shift. It has contributed to the EU 2020 goals on growth and jobs. By supporting the circulation of diverse content it has also contributed to spreading and defending European values.

Creative Europe was also found relevant to the changing priorities of the sectors, to the strategies pursued by the Member States and to EU policy priorities. However, despite positive achievements, more needs to be done to fully reap the opportunities that the digital shift presents, taking into account new audience and consumption patterns and how cultural and creative works are made, produced, accessed and monetised in the digital economy.

MEDIA schemes have adapted to changes induced by the digital revolution, which adds to their relevance. Training increasingly focuses on digital. Distribution schemes have also integrated this component. A specific scheme was created to address the increasingly important issue of the presence of European content on VOD services.

However, the current fragmentation of MEDIA into 14 separate schemes limits flexibility and weakens the capacity to achieve collaboration across the industry. In particular, collaboration between distributors across territories and with producers to promote films, which is a key challenge, is not addressed.

The flagship Cultural and Creative Sectors Guarantee Facility has had a strong start. Three Guarantee Agreements were signed in the first year, showing the relevance of this innovative market instrument.

The Creative Europe programme is broadly internally (among the different programme strands and the actions within these) and externally coherent (with other EU interventions and policies)

The schemes are internally coherent as they are designed to cover the traditional value chain of the audiovisual industry and avoid overlaps and gaps. MEDIA has a strong link to audiovisual policy. Today, its objectives coincide with the objectives of wider access across borders of the Digital Single Market strategy. It provides for measures accompanying the copyright reforms and supporting industry’s adaptation to the future AVMS Directive on the promotion of European works in VOD services.

MEDIA impact would be further strengthened through greater coherence with national film support programmes, which provide the bulk of the support to the audiovisual sector (approximately € 2 billion per annum compared to approximately 115 million for MEDIA). To this end, a dialogue was launched in 2015 with the European Film Agency Directors (EFADs) in order to develop a common, strategic vision for the industry. Furthermore, the Open Method of Coordination Group of Member States’ Experts on Improving the Circulation of European Films was established in July 2017.

There is a strong coherence between Creative Europe and the European Agenda for Culture’s objective of promoting culture as a catalyst for creativity is evidenced through Creative Europe’s support for capacity-building and skills development for CCS operators.

The Guarantee Facility in the Cross-sectoral strand has strengthened the coherence of the Programme. It has established terms and conditions which ensure the eligibility of all relevant stakeholders. A number of other cross-cutting projects have been undertaken, such as on Refugee Integration and the Module on Masters in Arts and Sciences.

However, the design and the limited budget of the Cross-sectoral strand have not allowed for significant projects other than the Guarantee Facility.

Concerning effectiveness, Creative Europe and the predecessor programmes have been or are successful in achieving or progressing towards their objectives, delivering their intended outputs (i.e. from programmed activities), results (outcomes of those activities) and having the anticipated impact. The Programme made an important contribution towards the EU2020 employment objective by generating an estimated 3,000 jobs in the period 2014-2016, directly in the beneficiaries.

The MEDIA Sub-programme has been effective in facilitating the cross-border circulation of works. It has provided extensive support to the creation of works which could have cross-border appeal and to the distribution of films and TV series across Europe, including online. Distribution schemes supported over 400 films per year, equivalent to 25% of Europe's annual film production, and these films went on to reach a total of over 65 million admissions per year.

However, the share of non-national European works in cinema admissions remains weak, at 10 %, compared to the share of US films of 65%. Also, US films are more present and are more visible on VOD than European films. In this context there are a number of weaknesses as regards MEDIA support to the distribution and promotion of works. The Distribution schemes are still highly focused on theatrical release and, for example, do not provide for "straight to VOD" release strategies; the beneficiaries of the online distribution scheme have a limited audience; VOD platforms with a cross-border appeal or a strong European network of VOD services have not been established. Moreover, MEDIA has grown in scope over the years without an equivalent increase in budget and as a result, the funding is spread too thinly among many beneficiaries.

The thin spread of funding from the Culture sub-programme has created some frustration, as many good projects are rejected every year. At the same time the structuring and leverage effect of the funded (and in some cases also the non-funded) projects is way above their real financial value.

The Guarantee Facility has a budget of €121 million from Creative Europe and is expected to leverage €692 million in loans, which is a very effective way of increasing access to finance. Given the strong market take up, the Commission has topped up the budget of the Guarantee Facility by 50% with funds from the European Fund for Strategic Investment.

Overall efficiency of Creative Europe is satisfactory, with support being delivered smoothly. But, considering the size and range of the audiovisual and cultural sectors at European level and the geographical areas covered by the Programmes evaluated, the size of the budgets of Creative Europe and its predecessor programmes were not sufficient to create major impact of a European scale and/or at sectoral level. Creative Europe funds for 2014-16 were € 544 million, equivalent to an annual support of 0.03 % of the value of the sector, estimated at € 560 billion.

The cost effectiveness of the programmes was satisfactory and overall improved from one programming period to another. The efficiency of most of the schemes reviewed has improved or remained stable. The increase in the cost per projects was usually compensated either by a higher level of outputs and or contributions to higher levels of outcomes and or impacts. Beneficiaries have nonetheless noted the heaviness of audit and verification processes.

The indicators in the legal basis are not sufficiently related to the outcomes of the schemes and projects of the Programme. A better monitoring framework would help to assess the efficiency of the programme. The Commission services already presented a concrete proposal to strengthen the monitoring framework system. It will result in the adoption of supplementary performance indicators by a "Delegated Act", in line with Article 20 of the Creative Europe Regulation

The Programme budget is modest compared to the size of the sectors and national programmes.

For MEDIA, efficiency has been pursued, for example through digitisation of the application process. The introduction lump sum payments has sped up the payment and application procedures, although the average real costs of the projects need to be monitored. The administrative cost of implementation is quite high due to the processing of a high number of small contracts with single beneficiaries.

The Cultural and Creative Sectors Guarantee Facility has been rapidly implemented since its launch in 2016.

Whilst the Commission and the Agency have closely monitored the delivery of outputs, there has not been sufficient monitoring of progress in reaching planned results.

The sustainability of Creative Europe support has been ensured in particular through the development of durable partnerships and networks that go beyond the life of individual projects.

The MEDIA Sub-programme has had sustainable effects by enabling the sustained activities of companies competing in tough international markets. Half of the funded companies reported that they would have invested less in distribution of non-national films without MEDIA funding. Also, the sharing of knowledge and best practices through collaborative projects and through the European film Forum have had a sustained effect by developing a common vision.

Some of the beneficiaries under Culture have become major players in European cultural landscape.

Creative Europe has had clear EU added value in line with the intentions of the Programme: the creation of international networks and cross-border partnerships, which constitute the main and the most durable structuring elements of Creative Europe; transnational networks that increase the capability of CCS operators overall and especially for those working in niche fields and in the Member States where such capabilities are less developed; the support of organisations in taking (calculated) risks, which allows them to focus on operating with a European dimension; the reputational effect for programmes’ beneficiaries that increases the potential interest of partners and financial intermediaries and their trust in supported projects. This reputational aspect was also reported to facilitate access to additional funding.

The EU added value of the MEDIA Sub-programme has been demonstrated in the establishment of networks for European cooperation, such as Europa Distribution or Europa International, which have allowed distributors and sales agents to join up their efforts. MEDIA has helped independent companies to scale up and has shared the risks inherent with cross-border circulation. It has had a structuring effect by creating an ecosystem at European level where companies from across Europe can come together and collaborate.

Furthermore, measures have been adopted for a "level-playing field" to facilitate the participation of companies from lower capacity countries, thus strengthening diversity. It remains nonetheless necessary to define more clearly the needs of lower capacity countries and the role of the Creative Europe MEDIA Sub-programme in addressing them, in full respect of subsidiarity.

Concerning Culture, the activities funded are less closely embedded in existing processes, although they can generally be closely aligned with the general activities of the organisation.

Annex 1 – Procedural information concerning the process to prepare the evaluation

Lead DG: Directorate General for Education and Culture (EAC)

Agenda Planning number: 2016/EAC/006 Interim evaluation of Creative Europe.

The requirement for the interim evaluation of the Creative Europe programme derives from Article 18 §3 of Regulation (EU) 1295/2013 establishing Creative Europe, stipulating that "… the Commission shall establish a mid-term evaluation report, based on an external and independent evaluation …".

The interim evaluation of the Creative Europe programme started in 2015 and has been guided by Terms of Reference adopted by the Commission. An evaluation roadmap, summarising the design, purpose and scope of the Creative Europe interim evaluation, was published in December 2015 53 .

An Inter-Service Group (ISG) gathering representatives of different Directorates-General of the Commission (EAC CNECT, SG) and the EACEA was set up in October 2015 and held several meetings, steering the evaluation and preparing the Staff Working Document and the mid-term evaluation report to the European Parliament and the Council.

The interim evaluation was coordinated by Directorate D (Culture and Creativity) of the Commission's Directorate-General for Education, Youth, Sport and Culture (DG EAC) with inputs from Commission DGs (CNECT, EAC and SG) and EACEA.

An open public consultation on the interim evaluation of Creative Europe programme was conducted from 23 January to 24 April 2017.

Annex 2 – Synopsis report of the stakeholder consultations

1. Introduction

Creative Europe brings together the predecessor programmes, Culture Programme, MEDIA Programme and elements of MEDIA Mundus. It is now composed of three strands: the MEDIA Sub-programme, the Culture Sub-programme and the Cross-sectoral Strand.

This report summarises the outcomes of the open public consultation (OPC) on the Creative Europe Programme, which took place from 23 January to 24 April 2017. It also reflects on the findings of the targeted surveys conducted among the beneficiaries of MEDIA and Culture sub-programmes of Creative Europe.

The objective of the OPC and the two targeted surveys was to gather comments and perspectives to help the assessment of the relevance, coherence, effectiveness, efficiency, sustainability and added value of the Programme. The OPC also included forward looking questions regarding a possible successor programme to Creative Europe after 2020. 54

The OPC questionnaire was published on ‘Your Voice in Europe’ online platform, in English. A total of 557 responses were received during the consultation period 55 . The MEDIA and Culture surveys collected a total of 839 responses (400 and 439 responses respectively 56 ) from the beneficiaries of the current Programme 57 .

2. Overview of respondents

2.1. Overview of OPC respondents

The majority (63%) of the OPC respondents participated in the consultation as representatives of an organisation, company or institution. The rest (37%) replied to the OPC in their individual capacity. 0 overleaf provides the breakdown of the respondents to the OPC by category.

A total of 327 respondents (232 representing their organisation and 95 individuals) indicated that they operate or have an interest in the audiovisual sector; and the majority of them reported being involved in audiovisual production 58 , distribution of audiovisual works, organisation of audiovisual events and film festivals. Furthermore, 216 respondents (152 representing their organisation and 64 individuals) noted that they are active or have an interest in the culture sector covering a wide range of fields such as visual arts, cultural heritage, design and applied arts and architecture, literature and performing arts.

Figure 1.1 Breakdown of the OPC survey respondents who answered in their of private capacity (blue) and on behalf of their organisation (red)

54% of the respondents who replied on behalf of an organisation (189 out of 353) represented private companies, academic, training and research organisations, international organisations or Creative Europe Desk. The rest of respondents (46%) represented sector(s) organisation/ association, public authorities, social partners in the creative sectors and other organisations (such as NGOs, non-profit organisations, arts organisations, film festivals). Around one third (34%) of these respondents indicated that the scope of their organisation’s mandate was international, followed by national (31%), EU level (17%), regional (12%) and local (7%). The geographical distribution of responses is presented in 0 below 59 .

Almost half of the respondents to the OPC (47%) mentioned that they or their organisation have received financial support from the Creative Europe Programme (MEDIA or Culture Sub-programmes, or the Cross-sectoral Strand) while a small majority (53%) indicated that they or their organisation have not received financial support of the Programme. Furthermore, 48% of the respondents indicated that they would like to respond to the consultation in relation to the MEDIA Sub-programme, 23% in connection with the Culture Sub-programme and further 29% concerning the Creative Europe Programme as a whole.

Figure 1.2 The geographical distribution of the responses to the Creative Europe OPC

2.2. Overview of surveys’ respondents

The majority of respondents to the MEDIA survey who provided responses in relation to the current Sub-programme were private companies (73%), while further one fifth (21%) represented non-profit associations. Most respondents (99%) indicated that they represent micro (70%), small (25%) or medium-sized (5%) company, organisation or association. These were located across different countries with France (11%) and Germany (10%) having the highest shares of MEDIA survey participants.

The majority of the beneficiaries (57%) who replied to Culture survey in relation to the current Programme represented a non-profit organisation 60 . Further 23% represented a public organisation, 7% for profit private companies, 5% education and training institutions and 8% other types of organisations. The main Culture sub-sectors represented by the Culture survey respondents were: visual arts (18%), performing arts – theatre (18%) and cultural heritage (15%), while fewer respondents represented design (8%) and literature (5%) sub-sectors.

While the respondents to Culture survey represented organisations that are overall slightly larger than the companies represented in MEDIA survey, an overwhelming majority (90%) of Culture survey respondents were also micro (54%), small (24%) and medium-sized (13%) organisations. No geographical pattern was identified in the distribution of Culture survey respondents – the responses were scattered among the countries with respondents from Italy (9.3%) and France (7.5%) providing the highest shares of replies.

3. Analysis of the results by consultation topic

This section of the report summarises the key findings from the OPC 61 , supported by the analysis of the MEDIA and Culture surveys’ results. The results of these consultations are organised in sections examining the relevance, effectiveness, efficiency, coherence and added value of the Programme.

3.1. Relevance

The respondents were asked a number of questions regarding the relevance of the Creative Europe programme in the light of the needs of the audiovisual and cultural sectors 62 .

MEDIA Sub-programme

Overall, over 70% of the respondents confirmed that the MEDIA priorities are still relevant or extremely relevant to the challenges and needs within the sector they operated in.

According to the OPC responses received, the MEDIA priorities with the highest relevance are:

- supporting operators in developing European audiovisual works, including co-productions, with international circulation potential

- stimulating interest and improving access to European audiovisual works by supporting audience development 63 .

Enhancing skills of audiovisual professionals for the use of new technologies and business models to develop their audiences, and supporting theatrical distribution of European audiovisual works (transnational marketing, branding, distribution and exhibition activities) are still/extremely relevant priorities 64 . Similarly, respondents also agreed that facilitating European audiovisual operators’ access to principal markets and business tools 65 and promoting transnational marketing, branding and distribution of European audiovisual works on all other non-theatrical platforms are still/extremely relevant priorities 66 . Supporting the development of new business models for distribution of European audiovisual works was also rated as still/extremely relevant. 67

Some respondents to the OPC identified additional priorities that MEDIA Sub-programme should address, namely: (i) opening up to other sub-sectors/type of projects/stakeholders (62 respondents) 68 ; (ii) supporting digitalisation (37 69 ); (iii) supporting professionalisation (24), for example, by investing in professional training; (iv) supporting innovation (16) such as developing comprehensive innovation strategies for Europe's film sector, developing new types of projects/content or investing in new technologies; (v) facilitating access to global markets (15); (vi) and development of different areas of the audiovisual sector (13 70 ).

The respondents to this question also suggested new types of projects that could be funded by the MEDIA sub-programme, these are: (i) covering interdisciplinary works (12 respondents), for example combining film with other art forms or cross-cultural collaboration between audiovisual producers and cultural organisations; (ii) opening up to other sub-sectors/type of projects or stakeholders such as starting writers, directors and producers or self-employed individuals(5); (iii) supporting transnational distribution (2); and (v) developing journalism projects (2).

Findings of the OPC regarding the relevance of the Creative Europe’s MEDIA Sub-programme were confirmed in the results of MEDIA survey. A total of 96% of respondents indicated that the MEDIA priorities remain relevant – 53% of these respondents were of the opinion that priorities does not need to be adjusted, 43% reported that the priorities remain relevant, but some adjustment would be needed. The Sub-programme’s relevance in contributing to the sector’s digitalisation and modernisation needs so as to compete in an increasingly globalised audiovisual market was highlighted by 90% of the respondents.

Culture Sub-programme

Overall, around 90% of the respondents believe that most priorities of the Culture Sub-programme are still relevant or extremely relevant.

According to the OPC respondents, stimulating interest and improving access to European cultural and creative works appears to be the priority with the highest relevance 71 . This is closely followed by:

- facilitating professional opportunities of cultural and creative players by supporting organisations and international networking 72 ,

- supporting international touring, events, exhibitions and festivals 73 ,

- enabling international careers of cultural and creative players, where possible on long-term basis 74 , and

- enhancing skills of cultural and creative players for the use of new technologies and business models to develop their audiences 75 .

Supporting the circulation of European literature seems to be the priority with the relatively lowest relevance in respondents’ views – around three fourths 76 of respondents think that it is still/extremely relevant.

Some respondents suggested further possible priorities for the Culture Sub-programme such as: (i) opening up to other sub-sectors/type of projects/stakeholders (suggested by 27 respondents); (ii) access to finance (15), (iii) professionalisation (13), including education and/or training in cultural and creative marketing, PR, data handling, management and leadership; (iv) internationalisation (9), such as support for cultural exchange with countries in Africa or Asia and supporting the mobility of artists.

A number of respondents also suggested additional cultural sectors that need to be supported by the Programme: digital arts (7); gastronomy (3) and education (3). Some respondents also believe that there are other types of projects that the Sub-programme should support, the most popular ones being small scale (16 respondents) and interdisciplinary projects (8). Respondents also indicated that they would like to have support for educational projects (7), European touring (5), cross-sectoral projects (4), and local projects of European significance (4).

Relevance of the Creative Europe Culture Sub-programme priorities was also confirmed by 93% of the programme beneficiaries surveyed. Of these 47% were of the opinion that Culture priorities remain extremely relevant and do not require adjustments, while 46% of respondents indicated priorities remain relevant, but need some adjustments. The relevance of the new European Platforms scheme was confirmed by 67% of the participants arguing that this scheme had considerable relevance and added value under the Creative Europe Programme.

Creative Europe Programme

78% of the OPC respondents who provided their feedback on the Programme as a whole indicated that its objectives are still relevant to the challenges and needs of their sectors.

According to the OPC respondents the Programme’s objectives with the highest relevance are:

- increasing the capacity of European cultural and creative sectors to operate transnationally 77 ,

- increasing the transnational circulation of European cultural and creative works 78 ,

- enhancing innovation and creativity in the European cultural and creative sectors 79 , and

- developing audiences for European cultural and creative works 80 .

The objectives that remain overall relevant, but to a smaller extent as indicated by the OPC respondents, are:

- increasing access to European cultural and creative works by children, young people, people with disabilities and other under-represented groups 81 ,

- increasing the transnational circulation of European artists and other professionals 82 ,

- strengthening the financial capacity of companies and operators in cultural and creative sectors 83 , and

- preserving and promoting cultural and linguistic diversity 84

According to the respondents, the objective of enhancing the competitiveness of the European cultural and creative sectors 85 .

A number of respondents have identified further objectives that could be tackled by the Programme: (i) digitalisation (30 respondents 86 ); supporting SMEs (11 87 ) and (ii) ; (iii) strengthening European cooperation (8); (iv) promoting cultural dialogue (8) and (v) distribution to foreign markets (8).

3.2. Coherence

Coherence looks at the extent the different actions supported by the Creative Europe Programme complement each other and are coherent with other initiatives at national, EU and international levels.

MEDIA Sub-programme

Overall, the majority of the OPC respondents agree that the Sub-programme support is fully or partially complementary to national 88 , EU 89 and international 90 funding for the audiovisual sector 91 . The results of MEDIA survey suggest that the Sub-programme funding is mostly aligned with national policy support in the participating countries. 

Two fifths of the respondents indicated that the national funding opportunities/instruments were partially complementary with the Sub-programme support and the same share (40%) agreed that they were fully complementary. Further 4% of respondents indicated that they were mainly overlapping and 1% that they were fully overlapping.

Regarding the EU funding, 63% of the respondents who answered this question agreed that the funding from the MEDIA Sub-programme was either fully complementary (32%) or partially complementary (31%) to the support provided by other EU instruments. Only 3% of respondents thought that they are mainly overlapping and less than 1% indicated that they were fully overlapping.

Similar views were received from the respondents regarding the coherence of the MEDIA Sub-programme with other international funding opportunities: 30% of the respondents reported that they are fully complementary, 29% that they are partially complementary, 2% believed that they are mainly overlapping and 1% of respondents thought that they are fully overlapping 92 .

Findings of MEDIA survey support the results of the OPC: 60% of survey respondents reported full alignment between the Sub-programme and national policies, further 36% indicated the policies and support instruments are partially complementary. The results of MEDIA survey further suggest that the Sub-programme provides coherent coverage of the traditional value chain of audiovisual works. The survey respondents reported that funding priorities were comprehensive 93 , allowing for internal synergies and avoiding possible gaps 94 .

Culture Sub-programme

More than a half of the OPC respondents agreed that the support provided by the Culture Sub-programme is either fully or partially complementary to national 95 , EU 96 and international 97 funding 98 . The results of Culture survey show that the Sub-programme funding is largely complementary with national policy support. 

A relatively small number of respondents (approximately 2%) indicated that the funding opportunities are mainly overlapping and approximately 1% that they are fully overlapping. Some of the respondents who indicated that the support of the Culture Sub-Programme is overlapping with other funding opportunities explained that there were similar national programmes (6 respondents).

The results of the Culture survey show that the support of Culture Sub-programme is seen to be largely aligned with the national policies for supporting culture by Programme’s beneficiaries: 48% of the respondents indicated that the support is partially complementary and 43% - fully aligned with national policies. Furthermore, most respondents 99 to the survey indicated that Sub-programme support is comprehensive and a large majority 100 were of the opinion that there are complementarities between the types of projects funded.

Creative Europe Programme

Respondents’ views on the coherence of the Creative Europe Programme slightly vary in relation to its different elements.

Majority of the OPC respondents were of the opinion that the new schemes ‘Audience Development’ 101 and ‘International co-production funds’ 102 strengthen the coherence and impact of the MEDIA Sub-programme. Similarly, a large majority of the MEDIA survey respondents indicated that the new schemes add significant value to the Programme 103 .

A bit less than a half 104 of the OPC respondents believe that the integration of MEDIA, Culture and the Cross-sectoral Strand under the Creative Europe Programme has improved the coherence and impact of the European Union’s support to the cultural and creative sectors 105 . Most MEDIA and Culture surveys’ respondents agreed that the integrated the audiovisual programme MEDIA with the Culture sub-programme and the Cross-sectoral Strand under ‘Creative Europe’ has improved the coherence and impact of the European Union’s support to the cultural and creative sectors (67% and 53% respectively) 106 .

The respondents to the OPC were overall less convinced that the new Cultural and Creative Sector Guarantee Facility has the potential to strengthen the coherence and impact of the Creative Europe Programme 107 and that integrating MEDIA Mundus within the MEDIA Sub-programme of Creative Europe has improved its coherence and impact 108 . Around 40%-50% of the OPC respondents did not have an opinion on these topics. A majority (61%) of MEDIA survey respondents fully or partially agreed with the first part of the statement. This was the case for 54% of Culture survey respondents respectively. Regarding the integration of MEDIA Mundus, the programme beneficiaries were also more positive about the impact of this change than the OPC respondents: 46% of fully agreed and 28% partially agreed that this improved the coherence and impact of the current Programme.

49% 109 of the OPC respondents believed that the greater focus on entrepreneurship and competitiveness of the creative and cultural sectors is a strength of the new Programme, however 26% 110 disagreed with this statement. Only 29% of the MEDIA Sub-programme beneficiaries and 26% of Culture survey respondents fully agreed with this statement, 34% and 35% agreed partially, 10% and 14% fully or partially disagreed.

A bit more than one third 111 of OPC respondents agreed that the focus of the current MEDIA Sub-programme has been strengthened by the discontinuation of the following schemes that were supported under the predecessor MEDIA Programme (2007-2013): Support for interactive audiovisual works, Initial training and Digitalisation of cinemas. However, 27% of respondents disagreed or strongly disagreed with the statement. MEDIA survey results show that 30% of beneficiaries who replied to this question fully agreed with this statement, 26% partially agreed and 6% fully or partially disagreed.

Similarly, 28% 112 respondents believe that the new scheme ‘Support to the Development of European Video Games’ strengthens the coherence and impact of the MEDIA Sub-programme whilst 24% disagree or strongly disagree. Almost a half (48%) of the respondents who answered this question did not express an opinion 113 . From the MEDIA survey, 26% of respondents fully agreed, 16% partially agreed and 9% opposed the statement. A very high share (49%) had no opinion on this topic.

3.4. Effectiveness

The OPC and surveys’ respondents were invited to answer a series of questions assessing the extent Creative Europe has delivered its expected outputs, results and impact as well as the extent the Programme as a whole has contributed to the relevant European policy agenda and strategies.

MEDIA Sub-programme

Respondents’ opinions regarding the extent to which the MEDIA Sub-programme has achieved its priorities slightly vary.

The respondents indicated that the following MEDIA priorities are largely achieved:

- supporting operators in developing European audiovisual works, including co-productions, with international circulation potential 114 ;

- supporting the theatrical distribution of European audiovisual works: transnational marketing, branding, distribution and exhibition activities 115 ;

- facilitating European audiovisual operators’ access to principal markets and business tools 116 ; and

enhancing the skills of audiovisual professionals for the use of new technologies and business models to develop their audiences 117 .

Respondents also pointed out that the MEDIA Sub-programme is still effective, but to a smaller extent in relation to the following priorities:

- stimulating interest in and improve access to European audiovisual works by supporting audience development: promotion, events, film education and festivals 118 .

- promoting the transnational marketing, branding and distribution of European audiovisual works on all other non-theatrical platforms 119 ; and

- supporting the development of new business models for distribution European audiovisual works 120 .

Some respondents identified barriers or factors that are hindering/ have hindered the achievement of these priorities. The main barriers, in respondents’ view are budgetary limitations (13 respondents), bureaucratic burden (12), MEDIA funding not being accessible by all stakeholders (9) lack of a common EU/ National culture strategy (6); and lack of innovation (5). The unbalanced funding between schemes (3) and unsuccessful international distribution (3) are also considered barriers to the effectiveness of the Programme.

Culture Sub-programme

Respondent’s opinions regarding the extent to which the Culture Sub-programme achieved its priorities slightly vary.

Respondents agree that the Culture Sub-programme is largely effective in delivering on the following priorities:

- facilitating the professional opportunities of cultural and creative players by supporting organisations and international networking 121 ;

- supporting the circulation of European literature 122 .

The respondents also indicated that the Culture Sub-programme is effective in promoting the following priorities, but to a smaller extent:

- supporting international touring, events, exhibitions and festivals 123 ;

- stimulating the interest in and improve access to European cultural and creative works 124 ;

- enabling international careers of cultural and creative players, where possible on a long-term basis 125 ; and

- enhancing the skills of cultural and creative players for the use of new technologies and business models to develop their audiences 126 .

As in the case of the MEDIA Sub-programme, the respondents have identified: (i) bureaucracy (13); the fact that funding is not accessible by all stakeholders (12); and insufficient funding (10) as main barriers obstructing the achievement of the Culture Sub-programme’s priorities.

Creative Europe Programme

Overall, the OPC and surveys’ respondents had similar views on the extent to which the different objectives of the Creative Europe programme were achieved.

Overall the respondents agree that the Programme has been effective in achieving the following objectives:

- increasing the transnational circulation of European cultural and creative works 127 ;

- increasing the transnational circulation of European artists and other professionals 128 ;

- increasing the capacity of European culture and creative sectors to operate transnationally 129 ;

- preserving and promoting cultural and linguistic diversity 130 ; and

- enhancing the competitiveness of the European cultural and creative sectors 131 .

Respondents also agreed that Creative Europe is effective in delivering on the following priorities, but to a smaller extent:

- developing audiences for European cultural and creative works 132 ;

- enhancing innovation and creativity in the European cultural and creative sectors 133 ;

- strengthening the financial capacity of companies and operators in cultural and creative sectors 134 ; and

- increasing access to European cultural and creative works by children, young people, people with disabilities and other under-represented groups 135 .

In respondents’ view, the main barriers hindering the achievement of these objectives are similar to those listed in relation to the MEDIA and Culture priorities: (i) Programme not accessible by all stakeholders (17 respondents); (ii) insufficient funding (15) and (iii) bureaucracy (7). A smaller number of respondents also indicated that the lack of innovation (3); quality control over project funded (2); the unsuccessful international distribution (2); and the fragmented market (2) could be possible barriers.

3.5. Efficiency

The respondents were asked to answer questions that examined the extent the results of the Creative Europe Programme have been delivered at a reasonable cost.

MEDIA Sub-programme

Opinions on the efficiency of the MEDIA programme varied. The OPC respondents generally agreed 136 that the results of the MEDIA Sub-programme could not have been achieved using less funding.

A bit more than a half 137 of the OPC respondents also think that the budget of the Creative Europe MEDIA Sub-programme is not sufficient to address the key European challenges faced by the audiovisual sector. Some of these respondents also stressed that: (i) resources should be redistributed between schemes (9 respondents); (ii) promising projects do not receive funds; (iii) current resources are not proportionate to the increased number of members (6); (iv) administrative duties take away a part of the budget (6); (v) the required investment is too big (5); (vi) MEDIA funding is not accessible or fully accessible by all stakeholders (5); and (vii) programme expectations exceed available funds (5).

A bit more than two fifths 138 of respondents think that the results of the Creative Europe MEDIA Sub-programme could not have been achieved in a shorter period of time. Further 13% believe that this could have been done, including through the optimisation of the application procedures (16 respondents). A smaller number of respondents (3) also believe that increasing funding would have shortened the period of time necessary to deliver results.

A similar share 139 of OPC respondents do not agree that other policy instruments or mechanisms could have been more cost-effective than the Creative Europe MEDIA Sub-programme in addressing the audiovisual sector’s needs. However some respondents believe that elements such as higher quality communication (2) and more flexible procedures (2) could be more cost effective.

There was some agreement that the Creative Europe desks effectively reach out to potential applicants and support them in the application process: 64% 140 of the MEDIA survey respondents agreed with this statement. In addition, a number of the respondents suggested ways in which the desks could support them in the application process more effectively by improving the quality of the communication (7 respondents) and promoting the initiatives more actively (6). Some respondents also pointed out that their desk is not reachable for small stakeholders (6).

Exactly one third 141 respondents were of the opinion that the current project selection procedures under the Creative Europe’s MEDIA Sub-programme ensure there is timely information on project award decisions, while the same share disagreed with this statement. Some of the respondents who disagreed indicated that the timeliness of the current selection procedures could be improved mainly by having faster application outcome notification (16 respondents) and by reducing the red tape in the process (11 respondents). A smaller number of respondents suggested a two-step application process (4), changing when the award decisions are announced (3), more frequent communication during the application process (2) and redesign of application forms (2).

The results from the MEDIA survey show a similar pattern to the OPC responses: one fourth of the respondents indicated that their company would not have implemented the project or any of the project activities without the Sub-programme support. A bit less than two thirds of the beneficiaries stated that their project would have been implemented on a smaller scale. The results of the MEDIA survey confirm that the EACEA and the desks are overall efficient in providing support to MEDIA applicants. A majority of beneficiaries stated that the application process was clear (94%), yet one third found it hard to prepare a project application.

Culture Sub-programme

Opinions on the efficiency of the Culture programme were also diverse. Two thirds of the OPC respondents 142 believed that the results of the Creative Europe Culture Sub-programme could not have been achieved using less funding.

A bit more than a half 143 respondents were of the opinion that the budget of the Creative Europe Culture Sub-programme is insufficient to address the key challenges of the European cultural and creative sector. Further 11 respondents stated that the resources are insufficient to match current needs and 10 respondents stressed that many projects are not achieved due to lack of funding. Eight respondents also mentioned that funding does not match the schemes’ expectations, while 7 respondents said that there is a lack of national support (especially for small projects) and 6 of them mentioned that small organisations are incapable of competing.

A bit more than two fifths 144 of the OPC respondents do not agree that the results of the Creative Europe Culture Sub-programme could have been achieved in a shorter period of time. Close to half (48%) had no opinion on this topic. Three respondents suggested that the period of time could be shortened mainly by optimising the application procedures.

The respondents’ opinions were also divided regarding the timeliness of the award decisions: 27% agreed that there is timely information on project award decisions under the Culture Sub-programme, while a bit less than one fourth (23%) disagreed with this statement 145 . Several respondents suggested that predictable timing (7), a two-step application process (5), more calls (4), simplification of the evaluation (4) and application processes (3) as well as rescheduling the eligibility period (2) could be ways to improve the timeliness of the current selection procedures.

One fourth 146 of the OPC respondents disagreed that other policy instruments or mechanisms could have been more cost-effective than the Creative Europe Culture Sub-programme in addressing the cultural and creative sector’s needs, while the majority of 65% did not express an opinion about this topic. One in ten respondents did agree that other instrument could have been more cost-effective. Some of the examples received for more cost effective responses include simplifying procedures (3 respondents) or introducing small scale grants (2 respondents).

Around a half 147 Culture survey respondents agreed that the Creative Europe desks effectively reach out to potential applicants and support them in the application process, while 15% disagreed or fully disagreed with the statement. Some of the respondents who disagreed with this statement indicated that promoting the initiatives more actively (5 respondents), engaging with small organisations more actively (3) and increasing the competence of the staff (3) could improve the desks’ services.

The results of Culture survey show similar findings in relation to the efficiency of the Sub-programme: 39% of the respondents stated that they would have not implemented the project without the Culture’s support. A small majority of the Culture survey’s respondents (56%) indicated that some of their projects’ activities would have been implemented without the EU grant. A large majority of respondents agreed that the application process was clear (90%) but more than one third (39%) of respondents stressed that it was hard to prepare their projects’ applications.

3.6. EU added value

The respondents were asked to assess the added value of the Creative Europe Programme in the context of other national, European or international level interventions.

MEDIA Sub-programme

For a majority of the OPC respondents the MEDIA Sub-programme has contributed to improving the national, European or international support measures for the audiovisual sector 148 ; to complementing existing actions on national, European or international level by enabling industry collaboration across borders and across the value chain 149 and to supporting audiovisual subsectors or operators that would not have received support otherwise 150 .

Around one third 151 of the OPC respondents were of the opinion that the lessons learnt from the implementation of the Creative Europe’s MEDIA Sub-programme have been applied elsewhere. Furthermore 15% of the respondents indicated that the MEDIA Sub-programme has integrated lessons from other cultural and economic sectors, while a considerable share of respondents (82%) had no opinion regarding this issue.

The MEDIA survey results further show that the received Sub-programme funding positively impacted on the beneficiaries’ opportunities to obtain funding from other sources such as grants from public sector and commercial funding (63% and 45% of the MEDIA respondents respectively).

Culture Sub-programme

For a small majority of the OPC respondents the Creative Europe’s Culture Sub-programme has complemented existing actions on national, European or international level by supporting cultural subsectors or operators that would not have received support otherwise 152 and contributed to improving the national, European or international support measures for the cultural and creative sectors 153 .

Around one fourth 154 of the OPC respondents agreed that the Creative Europe’s Culture Sub-programme has integrated lessons from other cultural and economic sectors and a bit more than one fifth 155 the lessons learnt from the implementation of the Creative Europe Culture Sub-programme have been applied elsewhere. Note that a large share of respondents (over 70%) had no opinion regarding these statements.

The Culture survey results indicated that the received Sub-programme funding similarly to the MEDIA Sub-programme had a positive impact on obtaining funding from other sources such as grants from public sector (57%) and a slightly less significant impact on commercial funding (30%).

Creative Europe Programme

The OPC respondents were also asked to provide examples of how Creative Europe has improved upon actions taken in other national, European and international programmes. Several respondents mentioned that the Programme contributed to development of new funds (8 respondents), provided funds to otherwise unsupported projects (8), increased international cultural exchange (7), helped to establish European networks (7), influenced the distribution of local funds (7) and increased the mobility of artists (6). According to the respondents the Programme also influenced the development of national policies (5), widened the distribution of European productions (5) and brought recognition to additional cultural and creative sectors (4).

3.7. Sustainability

This section explores the extent to which the activities currently supported by the Creative Europe Programme would continue if the EU support were to be discontinued. It also examines the extent the results of the Creative Europe Programme have been properly disseminated.

The strong majority 156 of the OPC respondents were of the opinion that most of the current activities or elements of the programme would be significantly affected without EU support, while 12% indicated that none of the current activities or elements would be likely to continue without the EU support provided by the Programme. The respondents identified international collaboration (62), international projects (25), international distribution (22), festivals (8), project development support (8), workshops (6), European networks (5), training programmes (4) as elements that would be discontinued without the EU support. Further, 11 respondents also indicated that projects with insufficient local support would be affected. A relatively small number of respondents (7%) indicated that most of the current activities or elements of the Programme would be likely to continue unaffected without EU support while 6% thought that most of the current activities or elements of the Programme would be moderately affected without EU support.

More than half of the Culture survey respondents stressed that information and/or dissemination activities were a main part of their project. In 88% of the cases sustainability was ensured through continued collaboration with the partners after the project closure. In case of the MEDIA Sub- programme dissemination activities were less significant and only one third of the beneficiaries identified it as an important part of their project.

Future outlook 157  

The OPC respondents were asked to express their views on several items regarding a possible successor programme to Creative Europe.

Respondents suggested several trends that will shape the future of the audiovisual sector. The most popular trend identified by 68 respondents, out of 291 who answered this question, is towards the use of video on demand (VOD) services, followed by innovation in technology (suggested by 43 respondents), shifting towards digitalisation (32), finding ways to increase audiences for cinemas (25), developing virtual reality technology (23) and funding new platforms for distribution of content (22). A lower number of respondents also mentioned internationalization of productions, development of copyright legislation, multidisciplinary work and development of TV series as possible trends for the future of the audiovisual sector.

Respondents also expressed their views on trends that could shape the future of the culture sector. Out of 211 respondents, 40 believed that digitalisation is an important element of the sector in the future, followed by internationalisation (30), multidisciplinary work (26), and technological innovation (22). Fewer respondents identified education, cultural diversity and funding as relevant items that could shape the future of the sector.

Respondents were also asked to indicate the most important issues, problems, opportunities or priorities for the cultural and creative sectors to be addressed by the future Programme.

In relation to the audiovisual sector, the main issues mentioned were competition with other major markets (14 respondents), copyright/piracy (13) and lack of funding (12). The main opportunities and priorities identified by the respondents were: supporting internationalization (15), supporting traditional cinema (13), innovation in technology (9), support for cultural education (7), new distribution models (7) and strengthening European identity (7).

In culture sector, the most noticeable problems highlighted related to limited funding (14 respondents), followed by the preservation of cultural identity (6) opposition to business oriented culture (4), lack of national support (4) and intellectual property rights (4). According to the respondents the most important opportunities of the culture sector include shared cultural values (14), supporting cultural education (12), promoting cultural diversity (11), supporting internationalisation (9) and promoting the significance of culture in society (7).

Respondents also expressed their views on what priorities should be maintained and what new priorities should be introduced in a possible successor programme to Creative Europe.

For the audiovisual sector, 40 out of 80 respondents who provided additional comments suggested that the current priorities should be maintained. Respondents proposed professional training (11), accessibility to small/new companies (11) and international distribution (10) as new priorities that should be considered.

The majority of the respondents who provided their suggestions in relation to the culture sector indicated that the current priorities should be maintained (16 respondents), in particular the focus on international exchange (15). The respondents identified increasing accessibility to small/new companies (15) and development of professional skills (9) as new priorities that should be taken into account when considering future funding support.

A large majority of respondents agree that an (i) increased budget would maximise the impact of the successor programme 158 . Respondents also indicated that (ii) increased focus on capacity-building activities 159 ; (iii) better access for newcomers 160 ; (iv) increased focus on market instruments 161 ; (v) better defined priorities 162 ; (vi) increased coordination/synergy with other EU programmes 163 ; (vi) and (vii) increased use of cross-cutting calls 164 would maximize the impact of the successor programme. 165

Several respondents identified additional elements that could be addressed by the successor programme, including: simplified administrative procedures, better information about the programme, better tools for dissemination, audience development strategies and focus on cooperation with non-EU countries. Several respondents also suggested increasing the focus on artistic work, creativity and development, digitalisation in distribution and production, new technologies, social innovation, inclusion and gender diversity.

Annex 3 – Methods used in preparing the evaluation (more details on the methodology described in section 4)

1. Methodological approach and data collection

The methodological approach to this evaluation combined a variety of data collection and assessment methods. The approach is summarised in Figure 1.2

4Method of approach for the evaluation 

The data collection exercises are briefly described below:

Financial and detailed programme data collection and analyses included the collection, structuring, review and analysis of data from programme databases and documentation 166 . Basic data was collected on the projects supported by Creative Europe from 2014 to 2016. Basic data in relation to applications received 167 was also collected for the entire evaluation period (2007-2016) 168 . Collection of detailed project data was also completed with 440 projects’ reports and analysed. They have mostly informed the assessment of programmes and sub-programmes’ outputs and outcomes.

Desk research and retrieval of contextual data have been undertaken continuously and up to the final stages of the evaluation. Contextual data were used, where appropriate, to compare programme performance with market performance. The key sources reviewed and used in the analysis included data collected by the European Audiovisual Observatory (EAO), Eurostat and Federation of European Publishers’ Annual Statistics as well as other surveys (e.g. Eurobarometer), reports and studies. They have mostly informed the population of contextual indicators and hence the assessment of programmes and sub-programmes’ outcomes and impacts.

20 scoping interviews and 103 key informant interviews were carried out with key programme stakeholders of the current and predecessor programmes, including interviews with national policy makers, sector organisations, funding bodies and Creative Europe Desks (CEDs), programme managers and EU-level sub-sector representing organisations as well as international institutions. They have mostly informed the assessment of the outcomes and impacts generated by the programmes via the perceptions of stakeholders 169 .

90 interviews were carried out with projects beneficiaries, unsuccessful applicants and scheme lead contacts at the EC and EACEA in the context of the Focused Evaluations. They have mostly informed the focused evaluation and the assessment of the performance of the schemes in generating outcomes and impacts.

MEDIA and Culture surveys were conducted among the beneficiaries of MEDIA and Culture programme fields (i.e. projects supported by the current and predecessor programmes). A total of 1,411 complete responses to the surveys were received across the MEDIA and Culture programme fields (605 complete responses 170 and 806 complete responses respectively 171 ). They have mostly informed the assessment of the outcomes and impacts generated at programme, sub-programme and to a lesser extent scheme level.

Six focused evaluations (FEs) covered a total of 10 key schemes under MEDIA and Culture programme fields, assessing the performance of these schemes across the six evaluation criteria and six sectors (i.e. distribution, development and TV production for MEDIA and cooperation, networks and platforms, as well as literary translation for Culture). The methodologies for the FEs included a review of the data collected through all the above listed data collection exercises, 13 additional interviews with EACEA and EC officers managing the schemes as well as counterfactual analyses of data on 70 supported projects and 30 rejected applicants’ case studies 172 . They have provided in-depth insights into the performance of the scheme by enabling comparative and counterfactual analyses.

The Open public consultation (OPC) on Creative Europe was launched on 23 January 2017 and remained open for a period of 13 weeks. A total of 547 contributions were received. The OPC has mostly informed the assessment of the outcomes and impacts generated at the Programme and sub-programme levels.

A proposal for a future Programme Monitoring Framework (PMF) has been developed for the Creative Europe Programme.

A review of the level playing field (LPF) provisions of the MEDIA Sub-programme led to recommendations on the positive discrimination aspects of Creative Europe.

2. Key challenges and how these were addressed

The following key challenges were encountered and addressed in the context of this study:

·Access to programme data. Financial data was readily available for the predecessor programmes and the Creative Europe programme but limited in terms of the description of outputs (e.g. projects only). Information related to projects outcomes and impacts were not readily available. Detailed programme data was extracted on a sample of 440 project reports. Such data have informed the scheme level as well as programme level performance assessment.

·Access to contextual data. Contextual data 173 was only available for specific macro-level indicators and or specific sectors. When no EU-wide comparative sources was available the evaluators relied on specific studies at EU and or national level as well as on proxy indicators so as to put the performance of Creative Europe and predecessor programme into perspective.

·Access to stakeholders. A few knowledgeable stakeholders in small Member States could not been identified due to the relatively distant object of the evaluation and lack of like for like replacement. Efforts were made to contact and interview ‘unresponsive’ organisations, with limited success and more interviews were conducted in similar Member States and sector organisations were possible.

·Extent of the coverage of the object of the evaluation. For instance, the terms of reference included 71 evaluation questions that covered the different levels and period of the programmes and the schemes to varying degrees. For primary research exercises, this led to long survey and interview questionnaires being drafted some of the questions receiving a limited number of answers. When faced by limited evidence, other sources of information were used and triangulate to arrive to robust findings. For advanced analytical exercises (e.g. six focused evaluation), this meant relying of sampling techniques to generate the required findings.

3. Strengths and limitations of the method

The evaluation method as implemented has the following strengths:

oThe two surveys of beneficiaries have been very well received and achieved 174 a reasonable response rate (above 20% when invalid email addresses were removed). However, a low response rate was received in relation to MEDIA Mundus programme beneficiaries;

oThe stakeholder interviews delivered in-depth insights into Programme and Sub-programme performance. The wealth of qualitative information gathered enabled to substantiate survey findings and provide findings on specific sub-sector and/or scheme performance;

oProgramme data was analysed and additional interviews undertaken 175 to inform the Focused Evaluations. This enabled the evaluator to further triangulate Programme and surveys’ data gathered 176 with additional evidence.

The evaluation method as implemented has the following weaknesses (measures for addressing these weaknesses are presented after the description of each methodological limitation):

oOnly a limited number of projects were finalised at the time of the evaluation (1.5 % for Culture) making difficult to measure the real impact of the programme. The evaluation is therefore mainly based on applications' data and stakeholder consultations and for indicators like job creation the data was extrapolated from the MEDIA and Culture surveys (see chapter 5.3).

oThe survey of beneficiaries may suffer from a positive bias across all stakeholder types. This is typical and inherent to all surveys. Some MEDIA and Culture surveys’ questions were similar to the survey undertaken by the CEDs. ICF survey of beneficiaries led to similar level of positive responses (albeit a bit lower). The slight positive bias has been acknowledged in the analysis of survey results. Triangulation with other sources of data helped in correcting the stakeholder bias. Even if the two surveys (MEDIA and Culture) achieved a reasonable response rate (globally above 20% when invalid email addresses were removed) this may introduce some bias as the respondents may be not fully representative of the programmes beneficiaries' universe.

oThe respondents to the MEDIA survey who benefited from the Development schemes are overrepresented as a share of all respondents if compared to the share of all funding allocated for development schemes in relation to total MEDIA funding. The extent to which this influences the findings was assessed by running a number of simulations on the survey data. The overrepresentation of beneficiaries of the ‘Development Schemes’ in the MEDIA survey is more pronounced in relation to the predecessor Programme than in relation to the current programme. Such bias was not evidenced for particular MEDIA schemes other than the development schemes.

oPrimary research exercises mostly targeted the programme stakeholders – i.e. the supply side of the creative and cultural sectors. The perceptions and opinions of stakeholder on the “demand side” were mainly captured by the Open Public Consultation 177 . Additional interviews also scheduled and partly run with two representative organisations 178 , one of which declined to partake in the interview.

oRelying on a sample of projects to extract programme data exposes the research design to potential sampling biases. This was an issue with regard to very small samples used as part of some of the Focused Evaluations. Analyses relying on small samples have been reported as “illustrative examples” supporting more robust findings. They are never presented as robust findings in themselves in the report.

Comparative analyses were undertaken on the basis of like-for like comparisons, to the extent this is possible. The context within which the programmes were implemented as been taken into account but only to some extent. For instance, comparing the first three years of the predecessor programmes to the first three years of Creative Europe would have led to different results than comparing “average” performance across the two programming periods. Nevertheless comparative analyses on the basis of close to like-for-like comparisons provide useful insights into the performance of the programmes especially when accompanied with a narrative on the context within which these were implemented and or account of the main differences.

SWD ANNEX 4 - Creative Europe Programme    

ANNEX

Creative Europe MEDIA SUBPROGRAMME

Contents

A.    Conclusions of the European Film Forum 2015 – 2016    

1.    Feedback from Stakeholders    

2.    European film Forum: continuing dialogue in the months to come    

B.    Stakeholder Opinions on the future of Creative Europe MEDIA    

1.    Are MEDIA priorities still relevant?    

2.    Should other priorities be considered?    

3.    Should other types of project be supported?    

4.    Which trends and drivers as well as problems and opportunities are seen?    

5.    Which priorities shall a successor programme include?    

6.    What can be done to maximise the impact of a successor programme?    

C.    Creative Europe MEDIA subprogramme success stories    

A.Conclusions of the European Film Forum 2015 – 2016

Working together to ensure a sustainable Future for European works

The Commission launched the European Film Forum (EFF) in February 2015 to help the European audiovisual sector adapt to the digital shift through a structured dialogue with industry professionals and public authorities.

Since then EFF events have been organised in various places from Berlin to Cannes, Venice to Annecy, Tallinn to Amsterdam, helping to shape the debate and bring players from across the value chain to listen and discuss together.

At a time when the Commission is pursuing the implementation of its Digital Single Market (DSM) strategy, the 25th anniversary of MEDIA provides an opportunity to take stock and reflect on the lessons learnt so far from these enriching discussions.

1.Feedback from Stakeholders

The need to invest in talent and skills

Investment in people is the pillar for a successful European audiovisual sector. Europe would not have its rich and diverse film and audiovisual production without the talent of its creators and authors, the skills of highly specialized technicians required for production and post-production and the ingenuity and knowhow of producers, sales agents and distributors who organise, finance and distribute the productions.

In this context of global competition for talent, training and support becomes more essential not just to nurture new talents and avoid the brain drain but also to give existing professionals the possibility to adapt to the transformation that is galvanizing the sector.

Tomorrow’s talent needs to be equipped with a mix of creative, technological and entrepreneurial skills, allowing them to make the most of existing data, technologies and social media, for content production, distribution and promotion. Incubators and mentoring schemes are essential to unleash the potential of the next generation of audiovisual professionals. Young creators must be equipped with the competences required to tell a single story or story experience across multiple platforms and formats (transmedia) as well as with promotion and digital marketing skills.

For the animation industry, the main challenge is to equip creators and tech specialists with the managerial skills needed to build capacity and scale up at the European level. The challenge is also to create an ecosystem that can enable professionals to stay in Europe.

Audiovisual storytelling must embrace innovation

Good stories not only stand the test of time. They also have no borders. Storytelling is the main ingredient required for the success of any given work. While there is agreement on the importance of storytelling, the format or medium could be different. And innovation is not just coming from the content but also from experimentation with different formats.

The distinction between cinema and TV has become increasingly blurred with the main focus being quality story telling. We have seen TV series converted to film and vice-versa and many good stories successfully broken down into different formats. We have seen examples of franchises, sequels or adaptations being made into video games, and of video games being made into films (e.g. Assassins’s Creed, originally an Ubisoft game or Angry Birds). We have also witnessed strategies of transmedia storytelling allowing for interaction with users.

The importance of the development phase has been highlighted not only for script-writing, which is a key aspect for increasing the commercial potential of the films, but also for the identification of audiences and preparation of financing and business plans.

Fostering access to content across borders

On average, while results are better for co-productions in terms of their ability to travel beyond borders (particularly European coproduced TV series), European films travel less than US films and are available in fewer countries. Wider access to content for audiovisual works and films is thus one of the main objectives of EU policy. But it is also a cornerstone of the business of producers, distributors and cinemas, to mention just a few players in the value chain.

Overall, the film industry has accepted the need to look for ways for a wider exploitation of rights and cross-border circulation. However, stakeholders have expressed concerns on reforms that could undermine the current film financing model. Most stakeholders see territoriality and presales in return for exclusive rights granted territory by territory to distributors and broadcasters as the driver for the financing and distribution of audiovisual productions, notably premium content with the biggest potential to travel, and consider that this commercial freedom should be preserved.

One issue that often emerged is the difficulty for an increasingly growing number of films to find their audiences, as content in cinemas, TV, VOD and online proliferates at a rapid pace. Cinema remains the preferred channel of distribution, because it creates value and helps build the brand of the films, which is important for their success in other windows. However, as cinemas do not have the capacity to adequately screen all films produced in Europe, exhibitors believe that some films would better benefit from other distribution models. Over the past few years, we have thus seen some innovation with varying results in order to distribute films differently: from day and date releases of non-national European films in some countries to the multiplication of online film festivals or strategies of direct to VOD distribution. We have also seen heritage films being released in an innovative way, freshly restored and digitized then re-released in cinemas and in parallel available on DVDs and on VOD.

Some stakeholders have noted that in certain cases where the film might not generate high box office figures in cinema, it might make sense to have near real-time release both for cinema and VOD. Views on sequencing of release windows are, however, mixed among stakeholders.

Stakeholders have unanimously identified subtitling and dubbing (particularly for animation) as key elements to reach European non-national as well as global markets. Meanwhile, some niche players within the industry have also called for flexibility in the use of language versions beyond the territories for which they were originally intended. Technology allows users online to use the language version of their choice irrespective of the country where they are established; with digital technology, cinemas could easily organise specific screenings of films in the language of expat communities.

The importance of promotion

One of the key themes throughout the discussions has been the importance of promotion, visibility and prominence of EU content. Consumers today have a much larger choice and as they become their own gatekeepers, not relying for example on linear television, promotion becomes even more important. Promotion alone however will not work if the content they are looking for is not available or very difficult to find. However, to date, public funding has not directly tackled this issue of promotion.

There is growing consensus from all audiovisual players that promotion is key to a sustainable and competitive European audiovisual industry. Moreover, this is an area which offers huge potential for collaboration across the value chain. Common or synchronized strategies for making films available online in different territories enable costs savings and reaching out to larger audiences. Increasingly promotion campaigns start at the production stage in order to create a fan base upfront through social media or to launch crowdfunding or crowdsourcing campaigns.

Findability and prominence have been addressed at length. While the first challenge is that European content is made available online to consumers, once that challenge is surmounted, that content needs to be placed in a prominent position in order to be easily findable by the end-user. It was stressed that it does not help the consumer if the content is accessible but cannot be found. The issue of prominence is also addressed in the Audiovisual Media Services Directive, which is currently being discussed in Council and Parliament.

Reinvent financing models

While cinema reached its highest level of admissions in 2015, traditional TV channels are facing increasing competition directly from other generalist and thematic TV channels and indirectly from VOD services. This means that they are investing less in film and cinema.

In spite of the previously mentioned importance of the principle of territoriality, we have also heard that minimum guarantees in return for exclusive distribution agreements are decreasing in their importance, which means that new avenues for the financing of European works need to be explored. Alternative models such as crowdfunding or crowdsourcing are, for the time being, primarily useful for creation of a fan base or for attracting potential distributors, but not yet as a substantial source of revenues.

In the area of animation, we have heard how difficult it is to raise funding for high-scale productions in Europe. For this reason, even European brands have had to go international to finance their productions. These players argue that public funding needs to be targeted towards rewarding success because this will in turn trigger more investment and growth within the sector.

A more sustained exploitation of European works including in unsold territories, which takes account of the principle of territoriality, would certainly contribute to increase cultural diversity at the level of distribution and be in line with the DSM objective of wider access. There was no consensus however regarding the value and potential additional revenues this exploitation would generate, which is why the stakeholders welcome public support to explore this area.

While there is no consensus on this, we have heard certain players say that objectives of public funding may need to be reconsidered so as to incentivise audiovisual companies to use private investment and to be less dependent on subsidies.

2.European film Forum: continuing dialogue in the months to come

The discussions during the European Film Forum events have been enriching and also extremely informative and led to some positive first results. There is a far better understanding across the value chain on the issues to be addressed and on the work that should be done in the coming year. The accompanying measures aiming to ensure wider access to content across the EU announced in the Communication on promoting a fair, efficient and competitive European copyright-based economy in the Digital Single Market are the first result of this dialogue.

The debate needs to continue in the coming months and years particularly as we approach the mid-term evaluation of the MEDIA programme. Emphasis will be placed on the following issues in future EFF editions:

Sector is diverse: a constant theme is that there is no onesize- fits-all model. No one film is the same; what works for a production might not work for another, and distribution can change from one country to another. The way in which the animation sector finances and distributes its production is not the same as the one prevailing for other genres. And that also varies between TV and feature film.

Foster collaborations: Collaborations will become more important in future, not only across the value chain but also across national borders. Different players in the value chain are joining forces, sharing competences and knowledge, optimizing existing resources and implementing larger scale projects. The Commission wants to continue to discuss with all stakeholders how to develop new and better collaborative models, to be competitive not only in a European context but also in the global markets. Talent and Skills: Skills will remain a key area of interest. Knowledge-sharing and skills development are of utmost importance not just in the area of content creation but also for merging technological know-how in various areas and building a much wider knowledge base.

Promotion: The dialogue will be pursued on promotion and visibility, which has been identified as a key strategic factor and one which will grow in importance. Data is becoming more and more valuable and has the potential to reshape the industry. While some players are already using data analytics to help make content that is specifically targeted to a particular audience, big data can help companies make more suitable choices in terms of marketing, promotion and distribution. Even here, collaboration is essential particularly between producers, exhibitors and distribution channels.

Customised personalised distribution strategies: The gap between production and audiences keeps growing and more needs to be done to ensure that the European works produced, especially when public funding is involved, find their intended audiences within the entire European Union. New strategies tailored made to each film, using new media and data in smart ways will be essential for European works to reach new audiences. This is especially true for the new generations who are digital natives and who represent the future of audiovisual markets.

Licensing: The structured stakeholder dialogue on licensing will be launched in the first quarter of 2017 for the purpose of streamlining licensing practices, eventually leading to a more sustained exploitation and wider availability of European works. Financing: New forms of financing are needed to enable the investment that the sector requires to be competitive at the scale of the digital single market. For too long, the cultural and creative sector has had difficulty in accessing finance. Building on the experience of the Guarantee Facility we will discuss ways of how to develop market instruments which could complement the grant schemes of the MEDIA programme

B.Stakeholder Opinions on the future of Creative Europe MEDIA

This paper summarises the main responses of stakeholders to the Open Public Consultation to the "forward looking" questions on the Creative Europe MEDIA Sub-programme. This analysis complements the overall Synopsis Report on the Open Public Consultation in Annex 2 of the Staff Working Document Mid-Term Evaluation of Creative Europe.

1.Are MEDIA priorities still relevant?

Film schools, academies and training providers generally perceive the existing priorities as still relevant for all organisations.

Producers, including TV and video games stakeholders, consider the priorities as extremely relevant or still relevant. Nonetheless, some said that certain priorities would not be relevant e.g. promotion of marketing and distribution on all platforms would not be relevant. Video games stakeholders consider the current MEDIA priorities as relevant, however they feel their sector is not sufficiently addressed.

In general, distributors, sales agents, cinemas and VoD (Video on Demand) stakeholders think the current priorities are relevant. Some feel that actions focusing on new models of distribution or audience development are not relevant while others consider that such actions are valid but should be streamlined in other schemes.

A strong majority of festivals, audiovisual events and film literacy stakeholders as well as public authorities and international organisations agree that MEDIA priorities are extremely relevant.

Main conclusions: Overall, 93% of 360 respondents who answered the questions of the Open public consultation agree that the MEDIA Sub-programme priorities are still relevant or extremely relevant to the challenges and needs within the sector they operated in.

2.Should other priorities be considered?

Half of respondents from film schools, academies and training providers recommend that new priorities, e.g. creative skills and gender equality, shall be considered.

Around half of producers, including TV and video games stakeholders, say that there is no need to cover other priorities. Others suggest priorities like the development of innovative content to fully exploit artistic possibilities and new ways of production as well as a focus on talent and skills. Reaching out for new audiences was also highlighted, for example through supporting the promotion of European productions across borders and more subtitling. The need to overcome imbalances between Member States was mentioned.

The views of distributors, sales agents, cinemas and VoD stakeholders are mixed concerning the need for other priorities. Most of them believe the existing priorities are sufficient. The Europa Cinemas network was highlighted as particularly successful. A minority of stakeholders were in favour of new priorities. These include: adapting to the digital shift by addressing costs of digitization, value creation relying on valorisation of IP, virtual and augmented reality and big data. Other suggestions included a more audience-focused approach at all stages, better access to content, adapting to consumers' needs and strengthened support for media literacy through the education systems.

The majority of the responses from festivals, audiovisual events and film literacy stakeholders see no need for other priorities. Two respondents stress the importance of media literacy including the support to children's films and film education.

Public authorities and international organisations ask to adapt MEDIA to the new realities of production and distribution, create better synergies and increase cooperation in programme management as well as strengthen film education activities and skills development. They also ask to improve access to European audiovisual heritage, to enhance international cooperation and the external dimension of the programme as well as to strengthen the Cultural and Creative Sectors Guarantee Facility.

Main conclusions: Stakeholders have mixed views on whether other priorities not currently covered by the MEDIA Sub-programme should be considered. 49% of the respondents agree and 51% disagree.

3.Should other types of project be supported?

Film schools, academies and training providers have mixed views on supporting other types of projects. Half of the respondents recommend supporting projects improving networking and mobility of students/ emerging talents. Half are fine with the current status.

Producers, including TV and video games stakeholders, suggest several projects such as scouting talent and creative networks for individual creators. The video games stakeholders ask to be supported in the 'Access to Market' scheme and to allow online games and apps linked to TV programming to be supported.

The majority of distributors, sales agents, cinemas and VoD stakeholders see no need in supporting other types of projects. A minority suggests backing the promotion of European audiovisual works on VoD and educational programmes on arthouse films, documentaries and series.

Festivals, audiovisual events and film literacy stakeholders are overall satisfied with the existing projects. A minority suggest supporting best practices in media literacy and building networks beyond cinema.

In contrast, most of the respondents from public authorities and international organisations favour other types of projects. They suggest having the flexibility and agility to support new types of content, story-telling and cross media (including virtual reality), promotion beyond Europe and digitisation of film heritage. Also it is stated that new distribution models for TV and online need to achieve better impact.

Main conclusions: Respondents have mixed views on whether other types of projects should be supported by MEDIA, with 49 % in favour and 51% satisfied with the existing focus.

4.Which trends and drivers as well as problems and opportunities are seen?

For a majority of film schools, academies and training providers the development and co-production of TV content are trends to be embraced.

Producers, including TV and video games stakeholders, identify digital as a driver for synergies with new disciplines, although the main issue is content and the access to content in cinemas. A concentration of commercial operators in distribution coupled with the audience's focus on a limited number of over-marketed works is seen as endangering diversity. There are calls for fair regulation, to help compete on the international market, hybrid distribution and a cross-genre approach. Also talent retention in Europe is a key factor given the global market.

The majority of the distributors, sales agents, cinemas and VoD stakeholders point out the growth of VoD while highlighting it remains a tiny source of revenues. Some state that cinema is thriving and the modernization of cinema should be at the heart of the programme. For both cinema and VoD there is a risk of polarisation on the most commercial US and EU films. There are dramatic changes in the audio-visual landscape due to the arrival of global digital players, mainly from the US, and changing consumer behaviour. Some respondents say that Europe needs distribution companies operating in several territories or working together through commercial agreements. Furthermore, Brexit and the unknown consequences for the European audiovisual industry are mentioned.

No clear tendency is seen in the answer of the stakeholders from festivals, audiovisual events and film literacy. The main new opportunities mentioned are the growth of VoD platforms, the importance of film festival and theatrical distribution as well as the important role of film festivals and European cultural diversity.

Public authorities and international organisations address various issues. Problems are seen due to the fragmentation and the uneven development of the European audiovisual industry, the disruptive change that affects creation (huge amount of content, different types of content) and distribution (variety of distribution platforms) and the erosion of traditional territory-based business and exploitation models. There is a need for audience education as well as accessibility and findability of European works. Drivers are seen as digitisation and new technology, new business models as well as the development of innovative content/ projects relying on several formats, cross-media, virtual and augmented reality, video games.

Main conclusions: Respondents suggested several trends that will shape the future of the audiovisual sector. The most popular trend identified by 68 respondents, out of 291 who answered this question, is VoD, followed by innovation in technology, digitalization, traditional cinema, virtual reality technology and innovation in distribution.

5.Which priorities shall a successor programme include?

Most of the stakeholders have responded to the question on the priorities of a successor programme as for the current programme. Therefore only additional comments are mentioned in this section.

The existing priorities are seen as still relevant for a successor programme by the film schools, academies and training providers.

Producers as well as TV and video games stakeholders want to make European works travel more, also internationally, and see development as key. Private financial investment should be incentivised. Diversity and gender balance should be included in a future programme. New hybrid approaches which reinforce the transversality between cultural and audiovisual projects should be supported.

Distributors, sales agents, cinemas and VoD suggest, for example, strengthening cinema and VoD collaboration. The European Commission should support an open dialogue on the increasing volume of film production when this is not warranted by audience demand or screening capacity in cinemas. MEDIA should support new vertical business models integrating production and distribution.

Diversity and support to small countries, film literacy for the creation of future audiences as well as support for children's and kids content is requested by festivals, audiovisual events and film literacy stakeholders.

Respondents from public authorities and international organisations want to enhance the focus on the demand side of the value chain, quality production, content curation and distribution in Europe and the world. The strengthening of the audiovisual industries of low capacity countries is also raised by several stakeholders. Furthermore, increased synergies with other EU programmes are encouraged, as well as a deepening of stakeholder dialogue through the European Film Forum.

Main conclusions: 50% of respondents suggest that a successor programme should be like the current MEDIA. There are diverse suggestions for new priorities: creativity and development, skills, digitalisation in distribution and production, new technologies, social innovation and inclusion as well as gender equality and diversity.

6.What can be done to maximise the impact of a successor programme?

Half of respondents from film schools, academies and training providers recommend introducing cross-cutting approaches such as gender equality.

Amongst producers, including TV and video games stakeholders, there were mixed views on the use of cross-cutting calls and increasing access for newcomers. There was a strong consensus on the need for an increased budget.

The main three topics mentioned by distributors, sales agents, cinemas and VoD are also an increased budget, greater access for newcomers and support for market instruments. Some ask MEDIA to become better acquainted with individual countries and individual markets. Furthermore the development of new models of film financing and distribution as well as using technology without harming existing models based on territoriality should be supported.

Also the respondents from festivals, audiovisual events and film literacy ask for an increased budget. A focus on capacity building activities is considered a very important issue in order to maximize the impact of a future programme.

Public authorities and international organisations ask for a very agile and flexible new programme in order to respond quickly to rapidly-evolving needs of the sector. Similar to other stakeholder groups three very important topics are an increased budget, the focus on capacity building as well as on market instruments. A better coordination between European and national levels is also suggested.

Main conclusions: The great majority of the audiovisual stakeholders (91%) agree that an increased budget would maximize the impact of the successor programme. An increased focus on capacity-building activities (60%) is seen as important as well. Other topics are the increased focus on market instruments (52%) and better defined priorities (51%). A fair share of respondents called for better access for newcomers and increased coordination and synergy with other EU programmes. The use of cross-cutting calls is suggested by a minority.

C.Creative Europe MEDIA subprogramme success stories

MEDIA has co-financed some of the best European cinematographic works and TV series, helping promising films to scale-up and achieve international recognition. But beyond individual support for European audiovisual works, MEDIA proves its strength through a structuring effect, bringing together different knowledge and expertise and creating value for the audiovisual sector as a whole. MEDIA funding schemes are covering the whole value chain of the audiovisual industry by supporting skills, content development and production, distribution of works, promotion and access to markets.

This Annex provides examples of success stories supported by MEDIA. The films selected represent a variety of stories, genres and artistic approaches from various European countries.

From Development to Distribution

Distribution in cinemas

Europa Cinemas is a MEDIA co-funded network of cinema theatres. With over 1000 cinemas in 33 countries, Europa Cinemas has helped audiences for European films to grow and to discover a large diversity of films from all over Europe. 6 out of 10 screenings are films from Europe and 35 % of screenings are dedicated to non-national European films. I, Daniel Blake (Palme d'Or 179 2016), Toni Erdman and Julieta, were the top 3 most successful films in the Network in 2016. About 60% of the total admissions of these films were generated in the Europa Cinemas network.

Online distribution

The Spanish VOD platform Filmin is co-funded by MEDIA since its beginning in 2008. Over time, Filmin has grown and became one of the most dynamic VOD platforms in the Spanish market. Currently, with a catalogue of +/- 10.000 films of which +/-65% European, the platform has reached 20.000 subscribers for their Spanish-VOD service and is registering more than 250.000 paid transactions every year. Filmin's catalogue features films from 26 EU countries allowing audiovisual works from different parts of Europe to reach wider audiences.

Support for professionals

Training

MEDIA helps more than 1800 professionals to be trained annually through transnational and international projects. TorinoFilmLab (TFL) is a strong talent incubator supported by MEDIA. Seven TFL-supported films were represented in the Cannes selection in 2016. The Happiest day in the life of Olli Mäki by Juho Kuosmanen won Un Certain Regard Award, and Wolf and Sheep by Shahrbanoo Sadat received the Art Cinema Award of the Directors’ Fortnight.

Industry events 

(1)

Regulation (EU) N° 1295/2013 - article 18 §3

(2)

Regulation (EU) N° 1295/2013 - article 18 §5

(3)

Regulation (EU) N° 1295/2013 - article 2 §1 'cultural and creative sectors' means all sectors whose activities are based on cultural values and/or artistic and other creative expressions, whether those activities are market- or non-market-oriented, whatever the type of structure that carries them out, and irrespective of how that structure is financed. Those activities include the development, the creation, the production, the dissemination and the preservation of goods and services which embody cultural, artistic or other creative expressions, as well as related functions such as education or management. The cultural and creative sectors include inter alia architecture, archives, libraries and museums, artistic crafts, audiovisual (including film, television, video games and multimedia), tangible and intangible cultural heritage, design, festivals, music, literature, performing arts, publishing, radio and visual arts.

(4)

COM (2014) 272

(5)

https://ec.europa.eu/programmes/creative-europe/events/20161201-25-years-media_en

(6)

Creating growth, Ernst and Young, December 2014

(7)

Creative Europe Regulation 1295/2013 Article 18

(8)

 More information here: http://ec.europa.eu/smart-regulation/guidelines/toc_guide_en.htm .

(9)

 Out of all the CCS stakeholders taking part in the OPC, 87% agree that Creative Europe is relevant in terms of making the most of globalisation to increase the transnational circulation of European cultural and creative works.

(10)

As based on the review of the current national strategies of DE, FR, IE, LT, NL, PT, SI

(11)

 Eurostat data (2009-2014): General government expenditure by function (COFOG – GF0802, 0803, 0806)

(12)

 Based on interviews with CEDs and national policy stakeholders, include NL, NO, PT, UK

(13)

 This finding is based on the surveys and case study interviews. Most MEDIA and Culture Survey respondents reported that the integration of these sub-programmes and strand under Creative Europe has improved the coherence and impact of EU support to the CCS (61 per cent and 50 per cent, respectively). However, a significant proportion had no opinion or were not sure (33 per cent and 39 per cent, respectively), indicating uncertainty among a large proportion of respondents whether this change produced any real benefits. Interview responses were also divided, corroborating the survey findings. It is unclear whether the high degree of uncertainty was also due in part to a lack of understanding of coherence as a concept.

(14)

 91 per cent of respondents agreed that the different priorities under the Culture sub-programme were comprehensive, and 74 per cent of respondents agreed that the types of projects funded under the Culture sub-programme were complementary with each other. A review of the scheme guidelines also shows that the individual aims and priorities of Culture sub-programme schemes complement and extend each other.

(15)

 This was evidenced in the stakeholder and beneficiary interviews. Interviewees were asked open-ended questions about complementarity with other funding schemes, and no stakeholder or beneficiary mentioned the complementarity between Networks and Platforms. This lack of awareness is unsurprising, as the two schemes have different beneficiaries: Networks supports sector-facing organisations, whereas Platforms supports audience-facing organisations.

(16)

 Like Erasmus +, COSME, Horizon 2020 and the European Regional Development Fund.

(17)

 27 per cent of MEDIA survey respondents and 25 per cent of Culture Survey respondents do not know whether Creative Europe is coherent with other EU funding for their sector.

(18)

  https://ec.europa.eu/culture/policy/strategic-framework_en  

(19)

 Such as Networks, Platforms, and Cooperation Projects.

(20)

 There is a small set of other international funding and support mechanisms available to the AV and cultural sectors, including the Council of Europe (e.g. Eurimages for film co-productions), cooperation with EFTA countries and through EU ACP, as well as other inter-regional initiatives (e.g. Filmkontakt Nord for the AV sector, Nordic Cultural Foundation for the cultural sector). Some Member States also have access to international development funds that can support the AV and cultural sectors, such as those of UNESCO or USAID

(21)

 Presidency conclusions, Lisbon European Council, 23-24 March 2000 http://www.consilium.europa.eu/en/uedocs/cms_data/docs/pressdata/en/ec/00100-r1.en0.htm

(22)

 Europe 2020 targets http://ec.europa.eu/europe2020/targets/eu-targets/index_en.htm

(23)

  https://ec.europa.eu/priorities/sites/beta-political/files/juncker-political-guidelines-speech_en_0.pdf

(24)

Data on temporary and permanent employment generated by the MEDIA and Culture surveys, extrapolated based on the number of beneficiaries in respective periods. Sole beneficiaries and lead applicants under the Culture programmes have been placed together for this calculation. Calculation accounts for the different rates the lead applicant and partner organisations generated temporary and permanent employment based on the Culture survey data. For MEDIA all categories of respondents (lead applicant, partner, and coordinator) were treated as one category – as the number of respondents that placed their organisation in the categories ‘partner’ and ‘coordinator’ was relatively low.

(25)

 The majority of respondents to the surveys (67% respondents to the MEDIA survey and 53% of the respondents to the Culture survey) reported ‘major’ or ‘moderate’ impact of the programmes’ funding on increased turnover or market position of their organisations.

(26)

 According to more than 700 Programme beneficiaries Creative Europe equally contributed to its two general objectives: around 70% of respondents indicated that the programme has been ‘very effective’ or ‘effective’ in achieving each of these two general objectives. This balance in the extent Programme is addressing its general objectives is also reflected in the OPC results. 80% of the OPC respondents (n=510) indicated that the Programme has achieved the ‘Preserving and promoting cultural and linguistic diversity’ general objective ‘to a moderate’, ‘to a large’ or ‘to a very large' extent, the same is true for 78% of respondents (n=506) in relation to the ‘Enhancing the competitiveness of the European cultural and creative sectors’ general objective

(27)

Project data from EACEA database

(28)

 Only around one fifth of all respondents to the Culture survey agree with the statement as opposed to a bit less than one third of MEDIA survey respondents. This is also confirmed by the OPC.

(29)

 An average award to a project supported by predecessor Culture Programme was €196 thousand compared to €409 thousand under the current Culture Sub-programme.

(30)

 Under the predecessor Programme 88% of the operators were micro, small or medium-sized. Under the current this share has increased by 3 percentage points to 91%.

(31)

Table shows the total Culture programmes, predecessor Culture programme and Culture Sub-programme budget allocation per culture sub-sector based on the shares of respondents to the Culture survey (representing projects supported by the predecessor and current Culture programmes).

(32)

As demonstrated by OPC results and the replies to the MEDIA and Culture surveys.

(33)

 Regulation No 1295/2013, Article 24 ‘Financial provisions’.

(34)

 The average annual spending during the first three years of MEDIA Sub-programme was €109 million, compared to 109.8 million during the predecessor Programme

(35)

 The quality of the productions achieved by the TV Programming scheme is on par with the one achieved by its predecessor, Cinema Automatic and Cinema selective supported works achieved more national admissions, increased the average numbers of territories in which the supported works are distributed and achieved an equal level of quality of works distributed

(36)

 For instance, Cooperation projects reached out to 26% less people under Creative Europe than under its predecessor while the achievements in terms of impact on increasing the diversity of activities undertaken is broadly similar. The literary translation scheme achieved a significantly higher number of books sold per year than under the Culture predecessor programme.

(37)

 Note that the delay of the start-up of the CCS GF is due to financial allocations made available 2 years after the start of the budget

(38)

 Does not include the CCF GF budget implementation rate, which incurred some delays but which are expected to catch-up in the remaining years of the programme i.e. from 2016 and onwards.

(39)

 This was corroborated by the findings of the OPC where 47% of MEDIA sub-programme stakeholders and 52% of the Culture Sub-Programme stakeholders considered that the budget of the Creative Europe Programme is insufficient to address the key challenges of the European cultural and creative sector.

(40)

Including 2016 commitments for the CCS GF (€14.8 million).

(41)

 Under Creative Europe, MEDIA Sub-programme respondents considered, overwhelmingly, that application processes had been very clear: 95% (377) of survey respondents (Q89) indicated that the application process for their Creative Europe project had been ‘very clear’ or ‘clear’. 38% (153) indicated that the preparation of their application to Creative Europe co-funding had been either ‘hard’ or ‘very hard’. As for respondents to the Culture Programme surveyed: the majority of respondents (62% (268)) had considered that the application process for their Creative Europe project had been ‘quite clear’. 29% (124) indicated it had been ‘very clear’

(42)

 The percentage of projects selected for the schemes under Culture sub-programme - Creative Europe is 16% as compared to 26% for the schemes under the Culture predecessor programme. For the schemes under the MEDIA sub-programme - Creative Europe is 39% as compared to 47% for the schemes under the MEDIA predecessor programme.

(43)

 Article 18 of Regulation 1295/2013.

(44)

 Beneficiaries indicate that their project and/or collaboration with their partners usually continued after project completion: three quarters (75 per cent) of survey respondents. Beneficiaries implementing ongoing projects and literary translations projects were not asked this question.

(45)

 For example, of Cooperation Project beneficiaries under the predecessor Culture programme, 82 per cent reported that they had some follow up or continued relationship with the project partners after their project completion. 87 per cent under Creative Europe.

(46)

 As evidenced by survey findings and beneficiary interviews. For Cooperation Projects, 90 per cent of Culture programme respondents and 75 per cent of Creative Europe respondents found it effective or very effective in ‘developing skills and competences’. Beneficiary interviewees stated that the transnational component was crucial in achieving this.

(47)

 As evidenced by counterfactual interviews for Cooperation Projects and Platforms.

(48)

 For instance, on Cooperation Projects, estimates of the change in organisations’ activities without the EU funding suggest that the largest share of organisations would have implemented a significantly reduced number of activities, on average 35 per cent under Culture and 38 per cent under Creative Europe (based on survey findings and extrapolated using project database).

(49)

 Counterfactual interviewees outlined in Focused Evaluation 4 were unable to secure alternative funding for their projects. Under Creative Europe: over 30 per cent of respondents indicated that they would not have implemented any of the project activities if they had not been successful.

(50)

 Around one-third of survey respondents under the predecessor Culture Programme, and 40 per cent of those under Creative Europe, would not have implemented any of the project activities without the funding. Of those that indicated some activities would take place, the average proportion that they anticipate would be undertaken is 50 per cent (under the predecessor Culture Programme) and 34 per cent (under Creative Europe).

(51)

 Based on counterfactual analysis set out in Focused Evaluation 5. Of the projects that indicated some of the activities would have happened without the Creative Europe funding, the average amount of activity was 37 per cent.

(52)

 See Focused Evaluation 5: Networks & Platforms; Focused Evaluation 4: Cooperation Projects

(53)

  http://ec.europa.eu/smart-regulation/roadmaps/docs/2016_eac_006_evaluation_creative_europe_media_mundus_prog_en.pdf  

(54)

  https://ec.europa.eu/programmes/creative-europe/creative-europe-2017-consultation_en

(55)

The OPC was open to submissions for a period of 12 weeks from mid-October 2016 to mid-January 2017.

(56)

The surveys also targeted the beneficiaries of the MEDIA and Culture predecessor programmes. A total of 1,411 complete responses to the surveys were received: 605 across MEDIA and 806 across Culture programmes fields.

(57)

MEDIA survey had a response rate of 22.2%, while Culture survey achieved 18.4% response rate.

(58)

In order to determine if the high number of respondents involved in the audiovisual production has influenced the OPC results, testing was undertaken, using three key questions of the OPC survey with high number of responses. The results from respondents representing the production sub-sector were compared with the OPC survey results covering all respondents’ categories. This comparison demonstrated that there are very small differences (five percentage points on less) in the types of responses provided.

(59)

In order to determine if the high number of respondents from Germany has influenced the OPC results, testing was undertaken, using two key questions of the OPC survey with high number of responses. The test results showed only small differences of less than five percentage points.

(60)

44% represented non-for-profit association and 13% non-for-profit company.

(61)

A detailed overview of the OPC responses is provided in Annex 1.

(62)

A large majority of OPC respondents (71%) indicated that they have detailed knowledge of the Creative Europe Programme‘s objectives and priorities, 24% noted that they were aware of the existence of the Programme, but have no detailed knowledge of its objectives and priorities and 5% replied that they were not really familiar with the Programme, its objectives and priorities. Similarly, two thirds of the MEDIA survey respondents (66%) indicated that they are well informed about the priorities of this Sub-programme. The same was true for a bit more than two thirds (69%) of Culture survey respondents.

(63)

93% (or 366) out of 396 respondents agreed that that these priorities are either relevant or extremely relevant.

(64)

89% (or 354) of 397 respondents indicated that that these priorities are either relevant or extremely relevant.

(65)

77% (or 303) of 394 respondents indicated that that these priorities are either relevant or extremely relevant, while 17% (or 38) indicated that they were not really relevant.

(66)

75% (or 298) of 397 respondents indicated that that these priorities are either relevant or extremely relevant, while 17% (or 67) indicated that they were not really relevant.

(67)

72% (or 285) of respondents indicated that that these priorities are either relevant or extremely relevant, while 20% (or 78) of respondents did not agree that this priority was relevant.

(68)

Several respondents suggested that interdisciplinary works and developing virtual reality technology could be additional sub-sectors/type of projects that the MEDIA Sub-programme could open up to. Other examples include content for children or web media content (podcast, YouTube, etc.).

(69)

Some of the key points suggested by respondents in relation digitalisation include developing new production and distribution models and developing trainings related to upcoming technologies such as transmedia storytelling, digital storytelling or the effects of digital marketing on consumption of films.

(70)

Respondents suggested that the Programme should support joint productions of European programmes, focus on international audiences and facilitate collaboration across different cultural and creative sectors.

(71)

95% of respondents or 252 out of 265 believe it is either still relevant or extremely relevant

(72)

94% or 247 out of 264 believe it is either still relevant or extremely relevant

(73)

92% or 246 out of 268 believe it is either still relevant or extremely relevant

(74)

90% or 242 out of 268 believe it is either still relevant or extremely relevant

(75)

90% or 241 out of 269 believe it is either still relevant or extremely relevant

(76)

73% or 194 of the 266 believe this priority is either still relevant or extremely relevant

(77)

88% or 455 out of 516 respondents indicated that they agree to a large or very large extent

(78)

86% or 442 out of 515 respondents indicated that they agree to a large or very large extent

(79)

81% or 419 out of 515 respondents indicated that they agree to a large or very large extent

(80)

80% or 410 out of 513 respondents indicated that they agree to a large or very large extent

(81)

77% or 393 out of 514 respondents indicated that they agree to a large or very large extent

(82)

74% or 383 out of 515 respondents indicated that they agree to a large or very large extent

(83)

73% or 372 out of 511 respondents indicated that they agree to a large or very large extent

(84)

71% or 367 of the 514 respondents who answered this question agree to a large or very large extent

(85)

70% or 361 out of 513 respondents indicated that they agree to a large or very large extent

(86)

The main objective suggested by respondents was the digitalisation of cinemas. Respondents also mentioned that another objective of the Creative Europe programme should be the digitisation and preservation/archival of audiovisual content as a way of preserving the cultural heritage.

(87)

Respondents stressed that the Creative Europe Programme should help to strengthen the position of individual artists and freelancers for example by providing funding for technical equipment; provide support for professionals at the beginning of their careers, including students; as well as help promoting SMEs competing with the big conglomerates.

(88)

81% or 321 out of 396 respondents who answered this question

(89)

66% or 258 out of 393 respondents who answered this question

(90)

62% or 242 out of 392 respondents who answered this question

(91)

Across the questions related to the coherence of the MEDIA Sub-programme with national, EU and international support, a relatively small number of OPC respondents (approximately 3% or 10) thought that these funding opportunities are mainly overlapping and less than 1% (or 2) believed that they are fully overlapping.

(92)

A considerable number of respondents (39% or 140 regarding international funding, 35% or 124 regarding EU funding and 59 regarding national funding) indicated that they didn’t have any knowledge about this topic.

(93)

95% of respondents agreed that the MEDIA Sub-programme covered all the main audiovisual sub-sectors in need of EU support

(94)

77% of respondents agreed that the different types of projects funded under the MEDIA Sub-programme were complementary to each other

(95)

65% or 170 out of 260 respondents who answered this question

(96)

55% or 142 out of 257 respondents who answered this question

(97)

51% or 132 out of 157 respondents who answered this question

(98)

A considerable number of respondents (46% or 117 regarding international funding, 41% or 106 regarding EU funding and 31% or 81 regarding national funding) indicated that they didn’t have any knowledge about this topic.

(99)

91% of respondents agreed that the different priorities under the Culture sub-programme were comprehensive

(100)

74% indicated that the types of projects funded under the Culture sub-programme were complementary with each other

(101)

61% or 241 out of 392 respondents who answered this question agreed or strongly agreed to the statement

(102)

51% or 198 out of 390 respondents who answered this question agreed or strongly agreed to the statement

(103)

41% of the respondents who provided a reply to this question agreed that the new ‘Audience Development’ scheme adds significant value to the Creative Europe Programme; further 26% of the respondents partially agreed to this statement. The same was true for 48% and 22% of respondents in relation to the new ‘International co-production funds’ scheme.

(104)

46% or 239 out of 517 respondents who answered this question agreed or strongly agreed to the statement

(105)

While 40% had no opinion or were uncertain and 14% disagreed or strongly disagreed.

(106)

However, a significant proportion also had no opinion or were not sure (27% and 40%, respectively), suggesting uncertainty among a large proportion of respondents whether this change produced any real benefits.

(107)

43% or 224 out of 515 respondents who answered this question agreed or strongly agreed to the statement

(108)

41% or 159 out of 392 respondents who answered this question agreed or strongly agreed to the statement

(109)

Or 130 out of the 268 OPC respondents who answered this question agreed or strongly agreed to the statement

(110)

Or 71 out of 268 respondents who answered this question disagreed (17%) or strongly disagreed (9%) with the statement

(111)

34% or 134 of the 394 OPC respondents who answered this question agreed or strongly agreed to the statement

(112)

Or 111 out of 393 OPC respondents who answered this question agreed or strongly agreed to the statement

(113)

The high share of respondents who have indicated that they have no opinion on this question is likely due to its sub-sector specific nature.

(114)

OPC: 76% of respondents agree to a large or very large extent; MEDIA survey (‘I don’t know’ answer category was removed to improve comparability): 80% of the respondents reported the Sub-programme ‘effective’ or ‘very effective’ in relation to the particular priority;

(115)

OPC: 74% agree to a large or very large extent; MEDIA survey: 94%;

(116)

OPC: 69% agree to a large or very large extent. Area not covered by the MEDIA survey, due to length limitations

(117)

OPC: 48% agree to a large or very large extent; MEDIA survey: 64%

(118)

OPC: 63% agree to a large or very large extent; MEDIA survey: 59%

(119)

OPC: 58% agree to a large or very large extent. Area not covered by the MEDIA survey, due to length limitations.

(120)

OPC: 53% agree to a large or very large extent; MEDIA survey: 63%.

(121)

OPC: 60% agree to a large or very large extent (‘I don’t know’ answer category was removed to improve comparability); Culture survey: 82% of the respondents reported the Sub-programme ‘effective’ or ‘very effective’ in relation to the particular priority.

(122)

OPC: 44% of the 148 respondents who answered this question agreed to a large or very large extent. A lot of OPC respondents (43%) selected ‘I don’t know’ answer option in relation to this priority. (To avoid these answers influencing the analysis of the extent various Culture Sub-programme priorities have been effectively achieved, this answer category was removed from the calculation.); Culture survey: 83%

(123)

OPC: 60% agree to a large or very large extent; Culture survey: 76%

(124)

OPC: 53% agree to a large or very large extent; Culture survey: 62%

(125)

OPC: 48% agree to a large or very large extent; Culture survey: 76%;

(126)

OPC: 42% agree to a large or very large extent; Culture survey: 76%.

(127)

OPC: 63% agree to a large or very large extent (‘I don’t know’ answer category was removed to improve comparability); Culture survey: 80% of the respondents reported the Sub-programme ‘effective’ or ‘very effective’ in relation to the particular priority

(128)

OPC: 58% agree to a large or very large extent; Culture survey: 80%

(129)

OPC: 56% agree to a large or very large extent; Culture survey: 80%

(130)

OPC: 54% or agree to a large or very large extent; Culture survey: 70%

(131)

OPC: 46% agree to a large or very large extent; Culture survey: 70%

(132)

OPC: 52% agree to a large or very large extent; Culture survey: 64%

(133)

OPC: 51% agree to a large or very large extent; Culture survey: 64%

(134)

OPC: 41% agree to a large or very large extent; Culture survey: 62%

(135)

OPC: 44% agree to a large or very large extent; Culture survey: 48%

(136)

74% or 283 out of 384 respondents disagreed or strongly disagreed with the statement that the results of the MEDIA Sub-programme could have been achieved using less funding

(137)

52% or 200 out of 387 disagreed or strongly disagreed with the statement

(138)

43% or 164 out of 384 disagreed or strongly disagreed with the statement

(139)

43% or 163 out of 377 disagreed or strongly disagreed with the statement

(140)

Or 245 of the 383 respondents who provided their answers to this question agreed or fully agreed with this statement

(141)

33% or 126 out of the 386 respondents to this question agreed or fully agreed with this statement

(142)

66% or 170 of 258 respondents disagreed or fully disagreed with the statement

(143)

54% or 140 of 260 respondents disagreed or fully disagreed with the statement

(144)

44% or 114 of 259 respondents disagreed or fully disagreed with the statement

(145)

A total of 257 respondents provided their responses to this question. Almost a half (49%) of respondents to this question selected the ‘No opinion or uncertain’ answer option.

(146)

25% or 64 out of 253 disagreed or fully disagreed with the statement

(147)

52% or 134 out of 259 agreed or fully agreed with the statement

(148)

68% of 2013 of the 377 respondents who answered this question agreed or fully agreed with this statement. 29% of respondents had no opinion on this topic.

(149)

64% or 243 out of 378 respondents who answered this question agreed or fully agreed with this statement

(150)

67% or 256 out of 380 respondents who answered this question agreed or fully agreed with this statement

(151)

32% or 113 out of 376 respondents who answered this question agreed or fully agreed with this statement, while a large majority (65%) had no opinion on the topic

(152)

57% or 141 out of the 248 respondents who expressed an opinion agreed of fully agreed with this statement, while 37% had no opinion on this topic.

(153)

58% or 145 out of the 249 respondents who expressed an opinion agreed of fully agreed with this statement. 35% had no opinion on this topic.

(154)

25% or 61 out of the 246 respondents who expressed an opinion agreed of fully agreed with this statement. 71% had no opinion on this topic.

(155)

21% or 52 out of the 247 respondents who expressed an opinion agreed of fully agreed with this statement. 74% had no opinion on this topic.

(156)

75% or 371 out of the 493 respondents who answered this question

(157)

No survey results are provided in this section as the surveys conducted did not include forward looking questions.

(158)

83% out of 509 respondents agree with this statement to a large or very large extent (‘I don’t know’ answer category removed for the data presented in this paragraph to facilitate comparison)

(159)

62% out of 452 respondents agree with this statement to a large or very large extent

(160)

59% out of 450 respondents agree with this statement to a large or very large extent

(161)

57% out of 485 respondents agree with this statement to a large or very large extent

(162)

51% out of 491 respondents agree with this statement to a large or very large extent

(163)

48% out of 356 respondents agree with this statement to a large or very large extent

(164)

38% out of 364 respondents agree with this statement to a large or very large extent

(165)

To facilitate comparison and prioritisation, the answer option ‘I don’t know’ was removed from the calculation.

(166)

For instance from the Regulation and Decisions establishing the programmes, annual work programmes, MEDIA implementation reports, interim evaluations of predecessor programmes, impact assessment for establishing the current programme, and various other data and documents on outputs of the various programmes and schemes).

(167)

This primarily focused on the 10 key schemes supported by the programmes, included in the scope of the MEDIA and Culture focused evaluations.

(168)

Basic programme data was also received on MEDIA Mundus activities supported in 2011, completing this data collection request.

(169)

The confidentiality of the responses of the interviewees

(170)

In addition, 139 partial responses were recorded to MEDIA survey. Partial response is recorded when a respondent started filling in the survey but did not submit it. The MEDIA Survey achieved a 22.2% response rate. The first invitation to participate in MEDIA survey was sent to 2727 valid email addresses (478 were not valid). Two consecutive reminders were sent to those beneficiaries that had not provided their responses by mid-way and week before the closing date of the survey.

(171)

The first invitation to take part in Culture survey was sent to 4378 valid email addresses (427 were not valid). Two consecutive reminders were sent to those beneficiaries that had not provided their responses by mid-way and week before the closing date of the survey. In addition, 222 partial responses were recorded to Culture survey. Partial response is recorded when a respondent started filling in the survey but did not submit it. The Culture survey achieved a 18.4% response rate.

(172)

The applicants that remained just under the selection threshold were to be used as a control group for the counterfactual analysis.

(173)

For instance, macro-level indicators such as employment in or the amounts of public subsidies targeting the creative and cultural sectors.

(174)

Based on the positive comments to the closing questions provided by both surveys and the lack of critical comments to the open ended questions of the survey (usually if respondents are frustrated with the survey design they do not hesitate to comment on shortcomings of the survey design in relation to particular questions) and also through technical advice provided through the dedicated email account for any survey-related enquires.

(175)

Five interviews were scheduled and undertaken in the process of developing the LPF note and further four in relation to the PMF note. In addition the relevant data from the key informant interviews was used to inform these notes.

(176)

It also helped to assess gaps in the Programme data (in relation to these notes these mostly relate to 2016 data that is currently in the process of being made available by EACEA) as well as with the identification of the relevant external data sets that will be of particular relevance for the FEs and the rest of the reporting under this assignment.

(177)

The evaluation team promoted the OPC to the relevant consumer and citizen organisations and associations to fill this gap. If they chose to use this opportunity to provide their positions in the framework of the OPC, these will be given proper attention in the analysis included in the Final report.

(178)

Europa Nostra’ citizens’ movement for the safeguarding of Europe’s cultural and natural heritage (supported under Networks scheme) and the European Association for Viewer's Interests (EAVI).

(179)

 List of MEDIA supported films awarded at Cannes Film Festival: https://ec.europa.eu/digital-single-market/en/news/25-years-media-25-years-critical-acclaim