EUR-Lex Access to European Union law

Back to EUR-Lex homepage

This document is an excerpt from the EUR-Lex website

Document 52019SC0374

COMMISSION STAFF WORKING DOCUMENT EVALUATION Accompanying the document Report from the Commission to the Council and to the European Parliament for the interim evaluation of the Programme for the Competitiveness of Enterprises and Small and Medium-sized enterprises

SWD/2019/374 final

Brussels, 14.10.2019

SWD(2019) 374 final

COMMISSION STAFF WORKING DOCUMENT

EVALUATION

Accompanying the document

Report from the Commission to the Council and to the European Parliament

for the interim evaluation of the Programme for the Competitiveness of Enterprises and Small and Medium-sized enterprises

{COM(2019) 468 final}


COMMISSION STAFF WORKING DOCUMENT

EVALUATION

Accompanying the document

Report from the Commission to the Council and to the European Parliament

for the interim evaluation of the Programme for the Competitiveness of Enterprises and Small and Medium-sized enterprises

1.Introduction

Purpose and scope    

2.Background to the intervention

Description of the intervention and its objectives    

3.Implementation / state of Play

Description of the current situation    

4.Method

4.1.Short description of methodology

4.2.Limitations and robustness of findings

5.Analysis and answers to the evaluation questions

5.1.Relevance

5.2.Effectiveness

5.3.Efficiency

5.4.Coherence

5.5.EU added value

6.Conclusions AND LESSONS LEARNED

Annex 1: Procedural information

Annex 2: Stakeholder consultation

Annex 3: Methods and analytical models

Annex 4: Interim Evaluation of the International Intellectual Property Rights (IPR) SME Helpdesk, a specific COSME measure



Glossary

Term or acronym

Meaning or definition

COSME

The Programme for the Competitiveness of Enterprises and small and medium-sized enterprises (COSME) (2014 - 2020)

SME

Small and medium-sized enterprises

CIP

The Competitiveness and Innovation Programme (CIP) (2007-2013)

EIP

The Entrepreneurship and Innovation Programme (EIP) (2007-2013)

MFF

Multi-annual Financial Framework

EIF

The European Investment Fund

EASME

The Executive Agency for SMEs

LGF

Loan Guarantee Facility

EFG

Equity Facility for Growth

EEN

The Enterprise Europe Network

EYE

The Erasmus for Young Entrepreneurs

IPR SME Helpdesk

Intellectual Property rights for SMEs Helpdesks

IPR

Intellectual Property rights

EFSI

The   European Fund for Strategic Investments

SO

Specific Objective

MS

Member States

WP

Work Programme

EU

European Union

R&I

Research and Innovation

KETs

Key Enabling Technologies

FC

Framework Conditions in the Single Market

GA

Grant agreements

IOs

Intermediary Organisations

TA

Thematic Area

OPC

Open Public Consultation

SBA

Small Business Act

1.Introduction

Purpose and scope

This evaluation assesses the relevance, effectiveness, coherence, efficiency and EU added-value of the Programme for the Competitiveness of Enterprises and small and medium-sized enterprises (COSME) (2014 - 2020) from 2014 to 2016, in conformity with article 15.3 of its basic act 1 :

“By 2018 at the latest, the Commission shall establish an interim evaluation report on the achievement of the objectives of all the actions supported under the COSME programme at the level of results and impacts, the efficiency of the use of resources and its European added value, with a view to a decision on the renewal, modification or suspension of the measures. The interim evaluation report shall also address the scope for simplification, its internal and external coherence, the continued relevance of all objectives, as well as the contribution of the measures to the Union priorities of smart, sustainable and inclusive growth. It shall take into account evaluation results on the long-term impact of the predecessor measures and shall feed into a decision on a possible renewal, modification or suspension of a subsequent measure.”

The interim evaluation contributed to the preparation of the future programme in the next Multi-annual Financial Framework for the period 2021-2027.

The performance of the delegated entities (The European Investment Fund (EIF) and the Executive Agency for SMEs (EASME) is an important aspect of the assessment of the efficiency and effectiveness criteria and for addressing the scope for simplification. This evaluation will feed into the mid-term evaluation of EASME that will analyse in more depth the performance of the Agency in implementing the programmes delegated to it.

The evaluation covers the 28 Member States of the European Union and the COSME participating countries. The article 6 of the COSME regulation allows third countries to participate in the programme. In 2016, the following countries participate in COSME: Albania, Montenegro, the Republic of North Macedonia, Iceland, Turkey, Serbia, Moldova and Armenia 2 . Ukraine 3 and Bosnia signed the international agreement for participation to COSME in respectively May and June 2016, pending ratification by the national parliaments 4 , but these agreements were not in force during the period contemplated by this evaluation.

2.Background to the intervention

Description of the intervention and its objectives

The Programme for the Competitiveness of Enterprises and small and medium-sized enterprises (COSME) (2014 - 2020) 5 is the successor of the Competitiveness and Innovation Programme (CIP) (2007-2013) 6 . With a total budget of EUR 2.3 billion, COSME aims to contribute to the Commission’s top political priority of creating more growth and jobs. It also aims to support the creation and growth of small and medium-sized enterprises, and promote the competitiveness of EU enterprises and entrepreneurship.

SMEs are the main source of economic growth and job creation in the EU. They constitute 99% of European businesses, provide two out of three private sector jobs and contribute more than half of the total value-added created by businesses in the EU. The impact assessment accompanying the proposal for the 2014-2020 COSME programme identified the following main problems 7 SMEs are facing across the EU:

·Cross-border regulatory fragmentation;

·Excessive administrative burden for starting a business;

·Lack of entrepreneurship spirit in the EU;

·Difficulties to access to finance;

·Significant barriers when doing business across borders inside and outside Europe.

To respond to these market failures and address the problem, the COSME Regulation sets two general objectives and four specific objectives to address these problems.

General objectives:

1.To strengthen the competitiveness and sustainability of the EU's enterprises, particularly SMEs and

2.To encourage entrepreneurial culture and promoting the creation and growth of SMEs.

Specific objectives (SOs):

1.(SO 1) Easing access to finance for SMEs, both through guarantees and equity (at least 60% of the total budget);

2.(SO 2) Helping SMEs access to markets and internationalise (indicative 21.5% of the total budget);

3.(SO 3) Creating a favourable environment for enterprises and support their competitiveness (indicative 11% of the total budget);

4.(SO 4) Promoting entrepreneurship (indicative 2.5% of the total budget).

The remaining 5% of the total budget are available for administrative expenditures in particular for the EASME subsidy, horizontal IT tools, corporate communication.

 

The Commission delegates the implementation of COSME to the Executive Agency for SMEs (EASME) 8 for all objectives 9 with the exception of the implementation of the financial instruments under specific objective 1 entrusted to the European Investment Fund (EIF).

The diagram in the next page shows the intervention logic of the COSME programme:

3.Implementation / state of Play

Description of the current situation

COSME is the successor of the EIP pillar of the Competitiveness and Innovation Framework Programme (CIP). CIP ran between 2007 and 2013 with a budget envelope of EUR 3.6 billion 10 . Its objectives were to contribute to strengthening productivity, innovation capacity, and sustainable growth. It comprised three ‘pillars’: the Entrepreneurship and Innovation Programme (EIP), the Information Communication Technologies Policy Support Programme (ICT-PSP), and the Intelligent Energy Europe Programme (IEE). The EIP accounted for slightly less than 60% of the CIP budget. In the 2014-2020 programming period, the innovation activities (including ICT-PSP and IEE) were transferred to Horizon 2020, while the activities under the EIP programme were largely transferred to COSME. The EIP supported access to finance for SMEs, the Enterprise Europe Network (established in 2008 by integrating two former networks: the Euro Info Centres and the Innovation Relay Centres, Erasmus for Young Entrepreneurs, the IPR Helpdesks for SMEs, and activities aimed at fostering entrepreneurship culture and creating better framework conditions for SMEs operating in the EU. It also provided actions to support innovation in enterprises (including eco-innovation) and innovation governance and culture.

While EIP under CIP strived ‘to support entrepreneurship and innovation and to promote the development and growth of SMEs across the EU’, COSME takes a more targeted approach and aims at improving the business environment to enhance competitiveness by focusing on framework conditions, entrepreneurship, access to finance and/or markets. All references to innovation are left out in COSME, while special attention to tourism has been introduced. 11 .

In most cases, the specific actions that were implemented or piloted in the EIP programme are continued also in COSME, with only minor changes. More substantial changes have been introduced in the financial instruments and the EEN services, listed in Table 1 , below.

Table 1: The financial instruments and EEN services in CIP versus COSME

Focus area

Under EIP

Under COSME

Financial instruments

Three financial instruments:

·the High Growth and Innovative SME Facility (GIF)

·the SME Guarantee Facility (SMEG), and

·the Capacity Building Scheme (CBS)

Two financial instruments:

·EFG was created as a translation of GIF2 with adjusted terms and conditions of investment

·The LGF is a direct translation of the SMEG Windows on loans, microcredit and securitization

·The CBS was discontinued because of a low take up

EEN

Key focus is on facilitating cooperation among SMEs

·Key focus is on facilitating access to markets and internationalization (within the EU)

·More customized services to SMEs

·Expansion of the online services

·Advisory services as an additional service

Source: Technopolis Group, based on CIP/EIP and COSME programme documents

The final CIP evaluation 12 concluded that the programme objectives related directly to the needs, problems and issues they were intended to address. In addition, the concentration of actions on areas where EU action could make a difference had been identified as a positive shift. CIP also proved to be flexible and able to respond to emerging issues. However, stakeholders raised the issue of fragmentation due to the financing of many small activities and the evaluation recommended a continued need to justify each small-scale measure separately, a need to strengthen mechanisms for the overall coordination of actions (to avoid unnecessary proliferation and achieve greater synergies), and a need to improve the monitoring and communication of the aims and achievements of small scale measures. It pointed out that in general, the indicators chosen to monitor the implementation and results of the actions were not all responding to the SMART criteria. Questions were also raised concerning synergies across the three components of CIP, and issues were spotted in relation to the coherence of the programme, amongst which the coherence of the Financial Instruments with the support offered by the Structural Funds.

The need to further develop and improve the monitoring system was a major finding in the final EIP evaluation, which recommended the development of “a clear overall monitoring system that regularly provides data in an easily accessible and standard format”. 13  The evaluation team noted that there was a very limited synergy or overlap with financial instruments on a national level and that the various financial instruments of the Commission were still seen to be fragmented and overlapping. Both the interim and final evaluation of the EIP 14  highlighted that there was some sort of competition between venture capital and loan guarantee schemes under the ERDF and under the EIP programme. The EIP final evaluation considered that the demand-driven approach to the financial instruments could give way to one that is more pro-active and especially one that concentrates on promoting facilities in countries not yet covered. Finally, the EIP evaluation raised the danger of having too many small activities funded by the EIP programme and noted the absence of “a mechanism that could ensure greater coordination and synergies among the different activities”. It proposed to establish an enhanced co-ordination function in the management of the programme that would have the SBA as a major reference point and focus on “an active exploitation of the potential synergies, both within the Programme and with external actions”. 

The COSME programme is implemented through annual work programmes, which are set up according to the Commission's policy priorities and approved through the examination procedure by the Member States' Committee.

The budget execution rates of COSME are good. . The operational budget 15 available in 2014 (EUR 247 million), was executed at 99.87%; in 2015 the operational budget available (EUR 329 million) was executed at 99.38%; in 2016 the operational budget available (EUR 336 million) was executed at 96.69%. The implementation structures for the financial instruments and the Enterprise Europe Network were in place and delivering. The total authorized appropriations for 2016 amount to EUR 298 million. (There was a further reinforcement of EUR 550 million of the financial instruments through EFSI (see section below). From 2016, the programme has integrated priorities of the Single market strategy, particularly the priority to support SMEs, start-ups and scaling-ups.

Access to Finance - specific objective 1

60% of the Programme’s resources are allocated to the COSME Financial Instruments, the Loan Guarantee Facility (LGF) and the Equity Facility for Growth (EFG). They build on the success of the CIP Financial Instruments (2007-2013) which helped to mobilise more than EUR 20 billion of loans and EUR 3 billion of venture capital to over 380,000 SMEs in Europe. The COSME financial instruments operate in conjunction with the financial instruments set up under Horizon 2020 as part of a single debt financial instrument and a single equity financial instrument respectively, supporting the growth of enterprises and their R&I activities. They can also complement financial instruments set up under the European Structural and Investment Funds.

The calls for expression of interest for the financial instruments published in 2014 have generated a strong interest from financial intermediaries across the EU and non-EU Member State countries participating in COSME, especially for the LGF. The strong market demand for the LGF continued in 2015, with the consequence that the whole 2014-2015 budget available was exhausted by mid-2015. Thanks to the guarantee provided by the European Fund for Strategic Investment (EFSI) 16 the EIF could continue to sign COSME LGF guarantee agreements in the second half of 2015. At the end of 2016, almost 143 000 SMEs in 21 participating countries received financing for almost EUR 5.5 billion 17 under the enhanced LGF. The COSME LGF enhancement thanks to the EFSI guarantee continued in 2016 and 2017, with a positive effect on financing made available to riskier SMEs at a much earlier moment in time as would have been the case with the COSME budget only, thereby speeding up the financing of the real economy.

Given the specific features inherent to an equity facility, with more complex due diligence and fund raising processes, signature of fund agreements took more time. The first five transactions under EFG, out of which three multi-stage funds in combination with the InnovFin Equity Facility for Early Stage set up under Horizon 2020, have been signed end of 2015. First investments into SMEs took place in 2016 under these agreements.

 

The total budget foreseen for the financial instruments under the 2016 work programme (EUR 295.3 million) was fully implemented.

Under specific objective 1, COSME also finances surveys to measure the access to finance situation in Europe 18 , awareness raising means about the EU financial instruments 19 , as well as other support actions on access to finance for about EUR 1 million each year.

Access to markets – specific objective 2

COSME helps European enterprises and, in particular SMEs, to do business in the Single market as well as to export outside Europe.

More than two thirds of the COSME budget for access to markets is devoted to the Enterprise Europe Network (EEN), which helps SMEs to find business, technology and research partners in the EU and third countries. It also promotes the participation of SMEs in EU funding programmes such as Horizon 2020 and the European Investment and Structural Funds.

The EEN started its activities as planned on 1/1/2015 (the activities in 2014 were still covered by the CIP programme). 92 consortia were selected as a result of two calls for proposals. The Network currently includes 525 organisations in the EU and 80 organisations in the eight COSME participating countries; they signed framework partnership agreements covering a seven year period and specific grant agreements for the operational periods 2015-2016 and 2017-2018.

The Network (about 3.000 staff members) involved, on yearly basis, more than 200.000 SMEs in local events, provided specialised advisory services to 70.000 SMEs on access to finance, IPR business and technology, resource efficiency services etc. About 25.000 SMEs participate in matchmaking events every year resulting in about 2.500 international partnerships signed between SME.

Starting from July 2017, the EEN is implementing an action focused on start-ups/scaling up, a priority of the Single Market Strategy, for a budget of EUR 2.6 million. The EEN scale-up advisors will help young European SMEs scaling up their activities in the Single Market by training, mentoring, coaching and innovation and internationalisation services.

Under specific objective 2 COSME also funds several other smaller projects; the most important are:

·The Your Europe Business portal for about EUR 0,5 million per year;

·The intellectual property rights (IPR) helpdesks in China, Latin America and south east Asia for EUR 7,2 million;

·The EU Japan Centre for Industrial Cooperation EUR 2,7 million per year;

·Improved points of single contacts in the Member States for EUR 1 million;

·Improving access to SMEs in public procurement, EUR 1 million.

Improve framework conditions for businesses – specific objective 3

COSME funds actions aiming at facilitating SMEs' access to IPR, public procurement, clusters and digital technologies, cutting red tape and promoting the "Think Small First" principle and improving the competitiveness of strategic sectors which have a strategic importance for the European economy including, tourism.

The Commission implements this objective through a wide variety of smaller actions, through calls for proposals and calls for tenders such as:

·The E-Skills for competitiveness and Innovation, about EUR 3 million per year;

·The Cluster internationalization program for SMEs, ranging from EUR 3,5 to 5 million per year;

·The Cluster Excellence programme for EUR 1,5 million per year;

·Modernization of Industry activities (Key Enabling Technologies, Advanced Manufacturing, etc.) for EUR 5 million per year on average;

·Several activities to support SMEs in industrial sectors (Construction, Chemistry, Aeronautics, Defense, the food supply chain, Resource Efficiency, Design, Bio Based Products, …); ranging from EUR 5 to 8 million per year;

·The monitoring and follow-up of the SME policy, for EUR 5 million per year on average:

oThe SME Performance Review;

oThe SME assembly;

oThe SME Envoys Network;

oThe outreach tools like Business Planet on Euronews and the SME portal;

·EU REFIT platform for regulatory burden reduction, EUR 0,5 million on average per year

·The support to Tourism sector for a budget ranging from EUR 5 to 9 million per year;

·The promotion of the social economy and social entrepreneurship, for a budget ranging from EUR 0.5 to EUR 2 million per year.

Between 2014 and 2016, EUR 109.3 million were allocated to this objective, representing 12% of the financial envelope. The first calls under this objective were published at the end of 2014 and considering the period of execution, most of the actions only started producing results after the time scope for this evaluation.

The High-level Group on administrative burden contributed to the exchange of good practices and the follow-up up and paved the way for the EU REFIT platform. The SME Performance Review provided an overview of the European SMEs and detailed information on the implementation of the Small Business Act. The Member States-specific information was fed into the European Semester country specific recommendations. The network of SME Envoys, the annual SME Assembly and the European SME week provided fora for awareness raising and exchange of good practices in all areas of the Small Business Act. Last but not least, the 4th season of business planet on Euronews TV started on 12 June 2015.

The e-skills for jobs campaign covered 30 countries, involving 384 organisations and gathering more than 300.000 participants with some 115 million people exposed to the campaign.

The call for proposals "Design based consumer goods" (with an available budget of EUR 4 million) resulted in 10 SME projects awarded to develop innovative, digital solutions to improve design processes and communication in the textile and clothing value chain, innovative products embedding digital technologies to products and new business models addressing environmental and societal challenges.

The tourism initiatives contributed to strengthening the competitiveness of the European tourism sector. The initiatives dedicated to the increase of senior and young tourists' flows contributed to generate an extension of the tourism season. Actions to enhance tourism accessibility contributed to improving facilities and services for tourists with special access needs. The visibility of European Destinations of Excellence has been improved, which will trigger higher numbers of potential tourist arrivals and local spill-over revenues for the destinations and tourism operators. Awareness-raising activities and events generated a better visibility and up-take of tourism initiatives.

In 2014-2016, the Clusters Go International calls involved 25 European Strategic Cluster Partnerships (ESCPs), representing 145 cluster organisations across 23 European countries and more than 17,000 European SMEs. 15 out of the 25 selected ESCPs were funded. The other 10 partnerships were awarded the ESCP label and encouraged to continue their partnerships on a voluntary basis, without COSME funding. The 15 co-funded partnerships account for approximately 95 clusters reaching out to over 10,300 European SMEs.

WATIFY is an awareness raising campaign launched by the European Commission in 2014 seeking to promote digital entrepreneurship and modernisation of the European industry. Implemented in two distinct phases, the campaign sought to reach SMEs, public authorities and other intermediaries across Europe and inform them about opportunities brought by digitalisation.

Entrepreneurship and Entrepreneurial culture – specific objective 4

COSME supports exchanges among European educators and trainers to develop best practices in entrepreneurship education in the EU. The programme focuses on groups such as young people, women or senior entrepreneurs in order to benefit from mentoring or other tailored schemes.

The main tool is the Erasmus for Young Entrepreneurs mentoring programme, which helps new entrepreneurs acquire and build entrepreneurial skills and knowledge, and to further develop their business activity by learning from experienced entrepreneurs. It started in 2009 as a pilot and is now a key action in the COSME programme.

It is implemented through yearly call for proposals resulting in the signature by EASME of grant agreements (GA) for 24 month periods. In general there are always a high number of good projects that cannot be financed because of a lack of funding.

For the Call 2014 (cycle 7): 111 proposals submitted, 15 projects selected involving 111 organizations from 28 COSME countries; 40 projects for a volume of approx. 14 million EUR could not be funded due to budget constraints For the 2015 call, 105 proposals were submitted, with 17 proposals selected involving 128 organisations. In 2014 the original budget of EUR 5,5 million was reinforced by nearly one million to award more proposals. In 2015, the budget was increased up to EUR 7,8 million and in 2016 it is intended to increase it up to at least EUR 8,7 million. The implementation of the activities started on 1/2/2015 with network of 175 Intermediary Organisations (IOs) in 32 countries (26 Member States and 6 COSME countries).

By the end of 2016, 11.000 entrepreneurs’ profiles were accepted and 4.200 matches involving over 8.400 entrepreneurs were established.

Under this specific objective COSME also funds several other smaller projects; the most important are:

·Entrepreneurship education from EUR 0,4 to EUR 1 million on average per year;

·Women’s entrepreneurship, for EUR 0,5 to EUR 1 million on average per year;

·Senior’s entrepreneurship for EUR 0,5 to EUR 1 million on average per year;

·Digital Entrepreneurship, EUR 2,5 million in 2015;

·Support to Migrant Entrepreneurship, EUR 1,5 million per year.

4.Method

4.1.Short description of methodology

This Staff Working Document is supported by three external studies, one for the whole programme 20 and two separate evaluations of the IPR SME helpdesks 21 and WATIFY. 22 . The methodological framework for this evaluation builds upon the evaluation criteria of relevance, efficiency, effectiveness, coherence and EU added-value. These evaluation criteria assess the relationships between the various components of the intervention logic presented on p.9. Taking account that this is an interim evaluation, most of the expected mid-term results are not yet measurable. Therefore, the primary focus was on the extent to which the programme sets the building blocks for reaching the intended mid- and long-term results and impacts. This implies an emphasis on the adequacy and quality of the programme design, i.e. the formulated objectives, the invested resources, and the activities implemented – in other words, the relevance and coherence criteria.

In terms of effectiveness, the assessment focused on the attainment of the expected outputs and short-term outcomes, and the enabling factors or barriers for the attainment of the outcomes and expected results, such as the capacity of the programme to reach and involve the stakeholders needed.

The evaluation covers all actions of the programme with a sense of proportionality for the four specific objectives of COSME.

The evaluation focusses on the main actions of the COSME Programme:

·The financial instruments: Loan Guarantee Facility and Equity Facility for Growth;

·The Enterprise Europe network, for which an evaluation was already performed for the 2008-2014 period, adding the two years of activity (2015-2016) under the COSME programme;

·The Erasmus for Young Entrepreneurs, with the support of an existing study.

Together, these instruments represent more than 80% of the budget of the programme and constitute the principal actions for three of the four specific objectives.

The remaining 20% of COSME budget is implemented through a variety of smaller actions across the four specific objectives. In order to evaluate the four specific objectives of the programme 23 , the supporting study contractor was asked to select a sample of representative activities on the following basis:

·The assessment of the relevance, coherence, and efficiency in implementation criteria covers all actions.

·The Assessment against the effectiveness criterion focuses on the Key Actions, complemented by four case studies.

·The European added-value and the efficiency / cost-effectiveness criteria remain covered through the analysis of the Key Actions only.

The Key Actions include: the financial instruments (LGF and EFG), the Enterprise Europe Network services, the SME Policy actions, the Cluster Internationalisation programme, and the Erasmus for Young Entrepreneurs programme.

Actions covered through case studies are: the Vulcanus scheme in the EU-Japan Centre action; the e-skills/digitisation actions, the Tourism actions (grants), and the Women Entrepreneurship action.

A mix of quantitative and qualitative methods for the evidence collection and analysis was used to allow for a proper triangulation of the evidence collected, ensuring the robustness of the findings.

Tools and methods used in the evaluation

Methodological approach

Tools & methods

Number

Qualitative methods

 

 

 

Desk research

260+

Interviews

120

Case studies

4

Workshop

1

Qualitative targeted consultation

7

Quantitative methods

Targeted consultation

7

Public consultation

1

Statistical analyses of secondary data

4

Technopolis, December 2017, https://ec.europa.eu/docsroom/documents/28084

The desk research encompassed a broad range of sources, ranging from programme and policy documents to market studies, impact assessments and preceding and/or other evaluations with the analysis of more than 260 documents.

120 interviews were organised to collect background and context information on facilitators and barriers for the attainment of the expected effects as well as the eventual unintended effects.

Four case studies were conducted on SO2 – Access to market (the trainee programme “Vulcanus in Japan”, run by the EU-Japan Centre), SO3 – Favourable environment (the Tourism programme and the E-skills for competitiveness and innovation actions) and SO4 – Entrepreneurship (the Women Entrepreneurship action).

Seven ‘Qualitative’ targeted consultations were implemented where the population is lower than 100. However, the small numbers did not allow for meaningful statistical analysis. The questionnaires mostly include open-ended questions, geared at collecting qualitative information to complete the information gathered by interviews.

Seven ‘classical’ targeted consultations were implemented where the population was higher than 100, allowing for meaningful and robust statistical data analyses. The questionnaires mostly include close-ended questions and only a few open-ended questions. The responses were analysed through descriptive statistics, in several cases also included cross-analyses.

The public consultation 24  was a 12-week internet-based questionnaire. More than 190 inputs and 15 position papers have been received from Business and trade associations, Citizens of the EU, non-profit organisations, Financial intermediaries (private equity, business angels, venture capital funds, banks, promotional banks, guarantee institutions), Intermediary structures such as incubators/clusters/innovation & growth agencies and Public authorities (member states, European agencies).

Four types of analyses were conducted on secondary data:

·The portfolio analysis entailed the analysis of the COSME budget distribution over the different structural components of the programme, ie the Strategic Objectives and their operational (sub)objectives, the policy mix used in the programme (i.e. the policy instruments) and the modes of implementation.

·The composition analysis focused on the stakeholders that benefit of the COSME programme funding. The outcomes of this analysis informed especially the analysis of effectiveness, in terms of the capacity of the programme to attract the type of stakeholders needed for the attainment of the desired effects and the analysis of cost-effectiveness

·The time-to-grant analysis, based on EASME management data, and the success rate analysis of the open calls for grants, based on the EASME proposal database, were conducted to feed into the assessment of the COSME efficiency in the implementation of the programme.

·The cost-effectiveness analysis (CEA) entailed quantifying the not-monetised benefits that would be generated by one Euro of total costs.

The methodology for the WATIFY study used desk research, media and social media analysis, surveys (for a total of 1673 responses) and interviews (45) to collect triangulated data.

4.2.Limitations and robustness of findings 

The supporting study 25 , despite the limitations presented below, provided robust evidence in relation to the five evaluation criteria. It allows to take stock of progress made since the beginning of the programme , it evidences what works well and what doesn't and has implemented sufficient mitigating measures to contrast the below identified limitations.

Data availability and reliability of the data analyses

The COSME regulation foresees the publication of a yearly monitoring report. At the time of the evaluation, in 2017, only the 2014 monitoring report was published, the 2015 report was only available in a draft version and the 2016 data was partially available. Besides, the variety of implementation modes (financial instruments, contracts, grants) and the diversity of actions implied different data sources and there is not a consolidated source of data besides the monitoring report.

On top of that, except for access to finance, the enterprise Europe network or the Erasmus for young entrepreneurs that are long-standing activities inherited from the CIP programmes, most COSME actions were ongoing and not finalised. This represent about 20% of the programme and a substantial part of the objective 3, framework conditions and competitiveness and a less substantial part of access to markets and of entrepreneurship.

Consequently, a major challenge for this evaluation was the difficulty to obtain the needed data and the quality of the data provided. While the contractor succeeded in collecting all data needed for the portfolio analysis, data that could be collected in relation to the profile of the programme beneficiaries was limited to some action lines. Even though these action lines accounted for about 80% of the budget committed, it implies that the data related to the geographical and sectoral profile of other activities of the programme beneficiaries had to be estimated.

Overall, the data collected through the seven targeted consultations can be considered as reliable; the survey population resulted representative for the overall population (or in the case of the LGF SME beneficiaries, sufficiently representative for the target sample). There is one exception: in the case of the consultation targeting EEN SME clients, data on the profile of these beneficiaries was not available, so no stratified random sample could be created (as was done for the LGF beneficiaries, instead). The distribution of the questionnaire to the SMEs by the intermediaries implied also that the evaluation had no control over the process; the lack in data on the profile of the EEN SME clients hindered a quality control of the representativeness of the survey population.

Taking this uncertainty on the reliability of the data into account, the contractor complemented and triangulated the information collected through this survey with the (preliminary) data from the EEN Performance Enhancement System (PES) database and with the other monitoring data provided by EASME.

The requirement to contact the SME beneficiaries of the larger actions (in the case of the EFG even for the ‘qualitative’ consultation) only by using a cascade approach, i.e. through the intermediaries, deserves a specific note. The fact that the contractors' study team did not have direct control over the number of SME clients that were invited to participate in the survey resulted problematic in some cases. In the case of the survey to the LFG beneficiaries, it implied that the contractor did not receive any response from the beneficiaries located in France because the financial intermediary did not circulate the survey before the deadline, despite the contractors' best efforts. In the case of the EFG qualitative consultation, only three out of five intermediaries were available for collaboration and the team received only four completed questionnaires (out of 12) from the EFG SMEs. In the case of the EEN clients' SMEs, the number of respondents from COSME-participating third countries was too low to make a meaningful analysis.

The contractor therefore endorsed the recommendation made by the CIP and EIP final evaluation teams to ensure in the future the possibility for evaluation teams directly to contact beneficiary SMEs.

As regards WATIFY, the main weakness of the methodology lied with the limited availability of data for the first phase of the campaign (2014-2015). Yet the monitoring data collected was sufficient to formulate answers to the evaluation questions and in some cases, present comparisons between the two phases of the campaign. Therefore, adequate triangulation of qualitative and quantitative data collected both throughout the campaign’s implementation and during this external evaluation, allows for the formulation of evidence-based answers to the evaluation questions.

5.Analysis and answers to the evaluation questions

5.1.Relevance

·How relevant is the COSME programme to the stakeholders (public and private) and the EU citizen?

Both intermediaries and beneficiary SMEs consider the financial instruments in COSME to be relevant to their needs, both in terms of the conditions set and the size of the financing offered - even though there are some issues remaining.

The objective of the LGF is to help the financial intermediaries provide more loans and leases to riskier SMEs, thus expanding the range of SMEs they can finance. The ‘on demand’ principle and flexibility in the design of the programme aim at facilitating the adaptation to the market needs and to the needs of financial intermediaries. 26  The programme supports all kinds of SMEs that face difficulties in accessing debt finance, without considering any specific sector or any specific country base, but rather the SMEs’ level of risk for the lending sector. COSME takes a flexible approach and allows any institution implementing the LGF to increase its activity with ‘high risk’ or ‘higher risk’ SMEs as per its own definition of risk. 27 This greatly facilitates the take up of the LGF.

Nevertheless, financial intermediaries mentioned that some needs are not sufficiently supported by the LGF, in their responses to the targeted consultation and/or interviews 28 . The subordinated loans for more than EUR 150k to an innovative SME cannot be supported under InnovFin nor under COSME and the loans over EUR 1.5 million (assuming a 50% guarantee rate). The surveyed LGF intermediaries also considered the time limitations to the guarantees (a minimum of 12 months and maximum of five years for guarantee amounts above EUR 1.5 million and 10 years for those below) to be inadequate for the needs of financial intermediaries and SMEs.

The objective of the EFG is to crowd-in more capital for pan-European operations at the growth and expansion stage. Interviewed stakeholders regarded this focus of the EFG instrument as appropriate. The flexibility to crowd-in the pan European Fund(s)-of-Funds is also regarded as relevant. However, the clause that the EFG can invest in a VC (only) up to 25% of the total amount is viewed by funds and the EIF as suboptimal for creating an attractiveness effect for other private investors. In comparison, EFSI can invest up to 50%.

The single web portal on EU finance (to be accessed through Your Business Europe or www.accesstofinance.eu ) addresses the information asymmetry failure. It provides contact details of around 1,500 intermediaries and information on all financial instruments under the new EU programmes 2014-2020 (COSME, EaSI, InnovFin under Horizon 2020, Life and the EU Investment Plan under EFSI, and the new financial instruments supported by European Structural and Investment Funds). It also provides information on how to access financial instruments from 12 different EU programmes in 39 countries, and on EU financial instruments with a clean-tech investment focus for climate-change objectives. The portal reached 150,000 monthly page views in 2015.

According to the surveyed SMEs that benefited from the the EEN services, the network addressed three major failures 29 :

·The broad range of information services, covering a.o. H2020 funding opportunities, information on financing opportunities and opportunities to increase competitiveness and innovativeness in the Single Market and beyond, address the clear (and ongoing) information asymmetry failure SMEs face to enhance their competitiveness

·Through its advisory services, the EEN also addresses the capability failure, i.e. the SMEs’ need for know-how on management processes, customer needs and how to enter foreign markets

·The Network also addresses networking failures by helping SMEs to find (international) cooperation partners and by providing an avenue for intermediary organisations to collaborate.

Confirmation of the relevance of the EEN services to SMEs can be found in the importance attributed to these actions by the SME clients ( Figure 1 ). In general, the SMEs’ view on the extent to which Network services address their needs is quite positive.

Figure 1: SME clients’ views on the importance of the EEN services

Source: Technopolis, December 2017, https://ec.europa.eu/docsroom/documents/28084 ; p.36

The tourism actions addressed the information asymmetry and networking failures. More than 60% of survey respondents thought the programme largely covered these failures. While the COSME actions therefore show a particularly high level of relevance for the SMEs in the Tourism sector, the responses also show two major gap areas: the exploitation of synergies between the tourism offer and other industries, and the enhancement of skills among tourism providers and their employees.

In the field of Entrepreneurship, the COSME programme includes information and awareness-raising measures, several portals and support services, networks to share knowledge and best practices, strategic intelligence platforms and measures to develop entrepreneurial human capital. Besides the Erasmus for Young Entrepreneurs (EYE) programme, COSME also funded actions on digital entrepreneurship, women’s entrepreneurship, migrant entrepreneurship, senior entrepreneurship and entrepreneurship education, and early warning/ second starters.

EYE has a high level of relevance to the needs of the respective target groups, with no major gaps identified. A large numbers of Host and New Entrepreneurs who had participated in the EYE programme during the 2014-2016 period were surveyed. These (potential) entrepreneurs assessed the extent to which a series of barriers to entrepreneurship were present in their country today. Their responses provide widespread confirmation of the existence of market and systemic failures in this area. The large majority (jointly, 92%+) agreed that entrepreneurs face obstacles to setting up businesses, that new businesses struggle to survive and grow, and that there is a lack of effective policy support related to entrepreneurship. There was also a widespread (though less strongly held) belief that there is a lack of entrepreneurial culture, weak entrepreneurial spirit, and low levels of entrepreneurial firm creation in Member States.

Figure below shows that more than 90% of these survey respondents agreed that COSME – through its largest entrepreneurship action, the EYE programme - helped to address all these areas to some extent, and particularly the lack of entrepreneurial culture and spirit.

Source: Technopolis, December 2017, https://ec.europa.eu/docsroom/documents/28084 ; p.38In the field of Internationalisation, the emerging markets are covered in COSME through the clusters internationalisation and IPR Helpdesks. A survey undertaken by the ECCP platform with the ESCPs’ project coordinators shows that, by June 2016, in the early phases of the implementation of the partnerships, 47.4% of them had selected USA as a target for international cooperation. Further popular markets targeted included Brazil, Canada, China, Japan and Mexico. 30  These are in line with the EU largest trade partners. Other actions such as the EU’s Low-Carbon Business Actions (LCBAs) in Brazil and Mexico, supported through the EU Foreign Partnership Instrument are examples of other approaches targeting the matching and direct funding of businesses from emerging European industries with partners from new emerging geographical markets 31 .

The 2015 Flash European Barometer on Internationalisation of SMEs 32 showed that SMEs considered the complicated administrative procedures, high delivery costs, and difficulties in identifying business partners abroad to be key bottlenecks for internationalisation. 33  A fourth area of concern is the perceived high investment cost.

COSME actions such as the EU-Japan Centre 34 and the IPR Helpdesks 35 are very relevant in reducing the administrative burden. Actions providing support to identifying business partners, such as the European Strategic Cluster Partnership (ESCP) – Clusters Go International programme (further: ‘Cluster Internationalisation’) and the business missions organised in the context of the European Cluster Collaboration Platform (ECCP) 36 and the EU-Japan centre are very relevant to the overall need of SMEs in building trust in finding suitable partners. The COSME actions do not address the fourth barrier, which is the high costs that come with building trust in finding suitable partners. A potential upgrade of the programme could therefore include other types of support, such as demonstration projects, feasibility studies and knowledge-acquisition activities.

For the EU citizen, the relevance of the programme is mainly related to economic growth. The major drivers for economic growth constitute the key focus of COSME. COSME shows limited relevance for topics beyond the economic sphere, though, when considering numbers of actions funded and budget allocated to them. The high-level objectives related to inclusive and sustainable growth are addressed mainly in an indirect manner.

The programme relevance related to sustainable development is therefore mainly an indirect one. For example, among the Venture Capital (VC) funds that received EFG funding, about three quarters are focusing on the ICT sector and clean technologies. Societal goals are also the funding purpose of many national promotional banks that signed an agreement for the LGF. Sustainable growth was explicitly pursued also through the Network focus on ‘Measures to increase SME access to energy efficiency, climate and environmental expertise’.

·To what extent are the initial objectives of the COSME programme (still) pertinent to the needs and problems it was designed to address?

According to the supporting study, the four specific objectives are still relevant.

First, despite the economic recovery in the EU, access to finance remains an issue for SMEs; the situation remains uneven among countries. In addition, the negative correlation between finance and firm size, age, and degree of innovation that emerged from the 2014 Survey on the Access to Finance of Enterprises (SAFE) seems to persist. The 2017 ECB survey, for example, shows that in the euro area large enterprises continue to experience easier access to finance than SMEs, except in the case of equity financing. 37  

The explicit objective of the LGF is to help the financial intermediaries in providing more loans and leases to riskier SMEs that have difficulties in obtaining funding through the traditional banking system. LGF directly addresses the financing gap; the limits set to the tickets also ensure that the facility is of specific interest to SMEs. The objective of the EFG, instead, is to cover the needs of start-ups and SMEs in general in their growth and expansion stage.

Second, a 2015 EC Staff Working Document 38 indicated that in the field of internationalisation, performance among EU SMEs shows limited progress. While the percentage of EU-based SMEs with extra-EU imports and exports of goods improved since 2008, the European Commission reported in 2014 that only 25% of EU-based SMEs were engaged in export activities and an even smaller share of SMEs (7%) exported beyond the EU.

Third, reflecting the considerations made in the EC Staff Working Document, both the 2015 39  and the 2016 Annual Growth Surveys (AGS) 40 show that pursuing structural reforms in the MS is key to modernising EU economies and ensuring better framework conditions for business. Further removing regulatory barriers to the Single Market, establishing the Digital Single Market and continuing the REFIT process for simplifying existing legislation have been highlighted as priorities for 2015; according to the 2016 AGS, increasing transparency, efficiency and accountability in public procurement, ensuring open and competitive product service markets, and modernising the public administration remain further challenges in the EU.

Fourth, fostering an entrepreneurial spirit among the European citizen has been a major policy objective during the last two decades. The SBA factsheet 2016 informs that at the EU28 level, there has been continuous (yet moderate) progress since 2008. Early stage entrepreneurial activity and entrepreneurship intentions have steadily increased since 2008; however, the factsheet emphasises, “more policy efforts are needed to turn entrepreneurship into a desirable career choice and to promote education as a means to develop an entrepreneurial attitude”.

·How has the programme adapted to the political priorities, for instance to the new single market strategy and its particular objective to support start-ups and scaling-ups?

The policy context of the COSME programme starts with the Europe 2020. The policy basis of the COSME programme is the SBA Review of 2011. The Entrepreneurship Action Plan that was published in the same year (2011), as well as the Single Market Strategy and the Start-up and Scale-up Initiative, both launched in 2016, constitute the other building blocks for the strategic decision-making on the COSME Programme.

The Single Market Strategy sets the need to support SMEs in the context of the many obstacles that SMEs, start-ups and young entrepreneurs are facing, such as access to finance, the complexity of VAT regulations, aspects of company law and how to comply with various regulatory requirements in different markets. The Start-up and Scale-up Initiative was set up in November 2016 to “reduce barriers for young businesses in the Single Market”. It followed shortly upon the launch of the Single Market Strategy and aligns with the COSME objectives, namely providing SMEs with a better access to finance and a better business environment. It echoes the continuous work of the Commission to foster better regulation 41  (through actions such as REFIT and the High-Level Group on Administrative Burden). The European Commission Investment Plan for Europe approved in 2014, also contributes to the COSME policy context.

The portfolio of actions in the COSME programme has achieved a high level of policy relevance. COSME responds particularly well to the priorities set out in the recent Single Market Strategy and Start-up and Scale-up Initiative (2016), setting a pronounced focus on support to start-ups, scale-ups, and second starters, but includes also several of the priorities defined by the Juncker Commission. The Entrepreneurship Action Plan 2020 guides the design and implementation of the COSME entrepreneurship actions.

The flexibility in the COSME programme allows it to respond in a swift manner to emerging high-priority topics by launching new actions addressing new or more emphasised priorities, and by steering more budget towards areas that constitute a priority area of rising importance.

The WATIFY campaign is underpinned by several key initiatives. The holistic Europe 2020 Strategy includes A Digital Agenda for Europe as one of its pillars. The objective of the Digital Agenda for Europe is to accelerate innovation, economic growth and quality of life by maximising the social and economic potential of ICT usage. Therefore, the Digital Single Market (DSM) strategy, launched in 2015, provides businesses and individuals with access to online activities without restrictions. The DSM specifically aims to: 1) provide better access for consumers and businesses in reaching online goods across Europe; 2) create favourable conditions for digital networks and services; and 3) maximise the growth potential of the European Digital Economy. The DSM strategy encompasses the Digitising European Industry initiative, which consists of five pillars and provides a set of measures to help industry, SMEs, researchers and public authorities to maximise their gains from digitalisation as well as to support and link up national level actions to apply a coherent integrated approach to digitisation across the EU.

Overall, the WATIFY campaign helps to put into action EU policy objectives in the field of industry digitisation and thus has strong relevance against EU policy context.

5.2.Effectiveness

·To what extent is the COSME Programme progressing towards its specific objectives?

The financial instruments have been effective in reaching their outputs, even though progress is modest for the EFG. The LGF reached a considerable number of SMEs, proposed overall more favourable financing terms, demonstrated leverage in SME financing, and increased the supply of debt finance in most Member States.

According the mid to long term objectives set in the annex of the COSME regulation for the LGF were to get by 2020 a Value of financing mobilised ranging from EUR 14,3 billion to EUR 21,5 billion and a number of firms receiving financing which benefit from guarantees from the COSME programme ranging from 220 000 to 330 000.

The European Investment Fund (EIF) has – in the time-period evaluated – managed to assign around EUR 611.7m (including the EFSI enhancement) allowing for a volume of EUR 5.5bn in debt financing provided to 140,000 SMEs for the LGF. It signed EFG agreements with nine financial intermediaries contributing a total of EUR 101m to the selected funds, which in turn provided 12 SMEs with risk capital to support their expansion and growth. The EFG is hereby falling behind on its targeted commitments. Geographically, the beneficiary SMEs are concentrated in the more mature financial markets.

The Enterprise Europe Network (EEN) has shown a strong capacity to reach SMEs throughout Europe, including 150,000 SMEs that participated in the EEN co-operation and advisory services. Thanks to the EEN services, over 8,000 SMEs set up co-operation agreements 42 with SMEs ‘across the border. The EEN services mainly supported SMEs in improving the quality of their goods and services, leading to (expected) growth in turnover and especially, entry to new (geographical) markets, both in Europe and beyond.

The Impact Evaluation of the Network (2008-2014) 43 concluded that the average net effect for SMEs participating in the Network was 3.1% increased annual growth. This result is confirmed in the COSME interim evaluation where SMEs present an overall growth rate that is nearly 6.5 percentage points higher than a comparison group. Surveyed SMEs argued that the core role of the Network is to help companies grow and find new partners, and that the Network services are adequate in doing so, which is further reflected in the high rates of satisfaction where 86% of SME clients 44 surveyed are satisfied with the Network’s services and 93% would recommend its services to others.

The evaluation of the IPR SME helpdesks 45 reveals that the action is very relevant and that results indicate it could reach its objective on the medium and long term. Evidence shows that it has been well designed to make EU SMEs aware about their Intellectual Property Rights (IPR) in third countries' markets and contributes to the overall objective of the COSME programme. In April 2017 the IPR SME helpdesks had registered 3154 helpline users. With 1347 registered helpline users against 1000 targeted, China performs well. The Latin-America also with 1217 registered against 800 targeted. Due to the belated implementation of its helpline, the South-East HD reached only 590 users against 1000 targeted.

The EU-Japan Centre for industrial cooperation was effective in fostering industrial and trade cooperation between the EU and Japan. Its objective is to facilitate and foster the cooperation between European and Japanese companies and clusters by providing policy analysis, services to businesses, SME support (e.g. on tax and public procurement), organising information events and knowledge-sharing opportunities as well as networking activities.

Activities organised by the EU-Japan Centre in 2014-2016

Activity

Nr of activities / stakeholders reached

Policy seminars and analysis events

In 2014, eleven events; 2015 ten events were held; in total, close to 3,000 participants

Company visits to address the practical adaptation of Japanese management and production systems to Europe

Around five company visits per year, lasting one day each

Webinars to inform EU SMEs of possible partnering opportunities and support for improving EU manufacturers’ competitiveness

Average nr of participants per webinar: 107

Cluster matchmaking events to inform EU SMEs of Japanese clusters and the opportunities for cluster-to-cluster partnerships

Since 2014, three actions per year, attracting between fifteen to twenty SME

Source: EU-Japan Centre for Industrial Cooperation. Activity Report for Fiscal Year 2015

Under the framework conditions and competitiveness objective, most of the actions were still under implementation at the time of the evaluation. Nevertheless several actions such as the SME envoy Network, the Clusters Go international or the Tourism actions, have reached the milestones in terms of outputs set in the COSME yearly work programmes.

Overall, stakeholders are satisfied with the results of the SME policy actions. The Envoys perceive the network as a highly valuable EU-level forum where ideas and practices are exchanged. The SME Performance Review as well as the factsheets are used by the national representatives and are considered to be of high quality. Many events are organised throughout Europe during the SME Week, and the SME Assembly gathers stakeholders from all MS. The main outreach tool, Business Planet, is broadcasted on Euronews. There is a general perception that the activities organised at the EU-level contribute to a convergence of SME policies across COSME countries.

For the Tourism actions, the consulted stakeholders and beneficiaries indicated as direct results the creation of new strategic partnerships across borders and more visibility in international markets of touristic destinations in the EU. The Tourism actions are geared towards the involvement of actors along the value chain. Based on the data available, they directly involved about 100 companies, active in the tourism sector, 50 private enterprises active in other sectors and 200 organisations ( mainly business associations active in tourism), public agencies, user organisations, and professional service providers (e.g. marketing specialists).

The "Clusters Go International" actions show good potential to lead to an improved strategic position in global value chains and an enhanced access to potential inward investors as survey respondents and interviewees alike highlighted that the activities are based on a strategic analysis. The limited budget is flagged as a hurdle for their internationalisation activities and was reported by 90% of the clusters as an important bottleneck.

In October 2017, the Clusters Go International action involved 25 European Strategic Cluster Partnerships (ESCPs), launched under the "Clusters Go International action representing 145 cluster organisations across 23 European countries and more than 17,000 European SMEs. Only 15 out of the 25 selected ESCPs were funded due to budget constraints. They account for approximately 95 clusters reaching out to over 10,300 European SMEs. The Clusters Go International action succeeded in involving a high number of clusters, some even at a voluntary level, testifying the high interest in this action in the EU cluster community. Moreover, projects show overall good progress in reaching the outputs expected, such as partnership agreements.

The overall achievements of the 1st generation of European Strategic Cluster Partnerships (ESCPs) for going international 2016-2017 prove their success in supporting SME internationalisation. Over a 24 months period, they involved 2000 SMEs in activities targeting international third-markets generating 85 concrete business cooperation cases with international partners. 370 Cluster-to-Cluster events and 3010 Business-to-Business events have been conducted, whilst 39 Memoranda of Understanding and 45 collaboration projects implemented between EU clusters and international peer organisations. The interest in the action remained high for 2018-2019 with 23 new partnerships established since 46 .

The e-skills action reached the outputs expected. Stakeholders interviewed were positive. In 2016, it covered 21 different countries where events were organised and many stakeholders and high-level policy-makers were involved, including members of the European Parliament. Stakeholders and survey respondents unanimously indicated the effective engagement of various stakeholders such as education and training organisations, employees and entrepreneurs as the most important factor in driving progress towards an enhanced awareness and especially an enhanced availability of digital skills.

The two WATIFY campaigns managed to achieve 4 of the 5 specific objectives: to promote the transformation of industry through advanced technologies by sharing success stories ; to stimulate digital entrepreneurial culture by identifying and promoting digital transformation of SMEs and digital start-ups and their success (or failure) stories at pan-European, national, regional and local levels; to spark and support new business ideas in the digital economy and boost their successful commercialisation; and to strengthen business confidence in the application and added value of digital and key enabling technologies, business processes and new business models. However, WATIFY 2 did not succeed in supporting Europe’s regions to translate their smart specialisation strategies in the area of digital and key enabling technologies. Neither desk research, nor interviews were fruitful in collecting information on the support provided by WATIFY to regions for the translation of their smart specialisation strategies into actual projects.

Under the Entrepreneurship objective, the Erasmus for Young Entrepreneurs (EYE) was successful in reaching its expected outputs and outcomes. It involved in total around 2,000 New Entrepreneurs and we estimate that after approximately a year and a half, it led to the creation of 250 new companies, EUR 5 million of additional turnover, and about 1,000 new jobs. The most prominent weakness of this action line lies in the small scale of its activities.

·What are the factors driving or hindering progress and how do they link with different actions of the programme?

The key factor of success for COSME is to rely on an extended network of intermediaries that ensures proximity to SMEs and to stakeholders. This is particularly the case for the EEN, the clusters or EYE but also for the financial instruments.

The LGF has reached a high level of take-up thanks to the flexibility of its design. However, financial intermediaries report high administrative costs for entering into an agreement with the EIF and complying with its requirements are problematic. The threshold of EUR 150 000 above which the financial intermediaries must check if the SME does not meet any of the innovation criteria for funding under the H2020 InnovFin instrument creates the negative effect of inducing the intermediaries to limit their financial offer to this threshold. Most important, the threshold is deemed inadequate to the needs of many SMEs. The EFG is falling behind on its targeted commitments. The growth projects of these SMEs mainly focus on expansion in the existing (local) markets and the development of new products and services. The key barrier identified in the supporting study is the maturity of Venture Capital markets in the EU. Geographically, the beneficiary SMEs are so far concentrated in the more mature financial markets.

The effectiveness of the EEN services relate to the profile of the EEN member organisations and their embeddedness in their national and regional environments. There is a close relationship between the services provided in the context of COSME and these organisations’ ‘usual’ services. The credibility of the organisation in delivering quality services is a primary condition for the use of the EEN services by SMEs. Another enabling factor is the proximity of the members to the local SMEs, as well as the potential for knowledge sharing and information. However, SMEs are not always aware that the EU funds EEN services and SMEs awareness of the existence of the Network could be improved.

Despite the success of the Erasmus for Young Entrepreneurs, the entrepreneurship objective does not seem to have a strategic focus in COSME work programmes. Actions originate from different places and are then managed across multiple Commission services, with only informal and unstructured interaction across them.

Except for the financial instruments and the EEN, the fragmentation of the programme into many smaller activities for 20% of the programme reduces the potential impact of actions. This is particularly the case for EYE and for the Clusters.

·To what extent is the Programme contributing to the five union priorities for a sustainable and inclusive growth?

The COSME intervention logic indicates that contributions by the COSME programme to the higher-level EU policy priorities are expected in four areas: an increase in ‘Jobs and growth’, an enhanced ‘Business creation and growth’, a strengthened ‘Global competitiveness of the SMEs’, and an increase in ‘Inclusive and Sustainable Growth’.

In the graphs below we first set these data in the context of statistical data related to the SMEs’ employment shares at the sectoral level ( Figure 2 ) and the developments from this perspective in recent years ( Figure 3 ). Sectors with the highest shares of COSME beneficiaries are highlighted in dark red; those with high shares are highlighted in pink.

This analysis suggests that COSME has a high potential for contributing to the attainment of the Jobs and Growth objective in sectors that are the most important for the EU economy.

However, as explained in the relevance section, its potential to reach impact in relation to global and societal challenges is limited. The main COSME Instruments, such as the Enterprise Europe Network or the Loan Guarantee Facility target all SMEs without distinction of sector or activity. This universal offer means that the support to policy objectives such as Climate change, Gender Mainstreaming or Youth Unemployment can only be indirect and due to undesired effects.

Figure 2: SME employment shares at sector level

Source: Annual report on European SMEs 2015/2016; Eurostat, Statistical Offices & DIW Econ

Figure 3: Change in employment in SMEs

Source: Annual report on European SMEs 2015/2016; Eurostat, Statistical Offices & DIW Econ

5.3.Efficiency

·Are the costs generated by the programme proportionate to the benefits generated?

COSME is operating at a reasonable level of efficiency overall, with its principal components all being implemented in line with the specific objectives and timetables set out in the annual work programmes. The feedback from beneficiaries is generally positive with some exceptions. A substantial majority is reporting a good cost-benefit ratio for their individual participations. For example, more than two thirds of EEN respondents (68%, 158 respondents) reported that the benefits outweigh the time investment.

Surveyed beneficiaries report some issues of administrative burden. For example, half of the Cluster Go International (CGI) Participants responding to the survey thought the duration of the CGI project funding was not appropriate; about 40% of them thought the funding needed for the internationalisation activities was not correctly estimated. The financial intermediaries indicated that the reporting requirements are burdensome and risk reducing the attractiveness of the LGF. They also argued that the EFG’s administrative burden is substantially higher than is normal in the Venture Capital (VC) sector.

After three years of experience under the COSME programme, the IPR SME helpdesks has been showing effectiveness and should now focus the resources to gain in efficiency. To that effect, improved service quality is needed, as well as tackling the problems of insufficient promotion of activities and improving partnering services. There is no need for additional means in order to achieve more impact, however the use of the current ones could be optimised. The fact that intermediaries act as multipliers, can be seen as an efficiency lever, provided the visibility increases concurrently thanks to more promotion efforts. The strategy to reach the critical mass of SMEs needs to be improved. The evaluation concludes on the need for continued funding for the three regions and considers extending this type of service to India.

The two WATIFY campaigns cost EUR 1,6 million and allowed to reach some 200,000 participants through an initial and closing event, matchmaking events, awareness raising campaign, video clips and case studies. Participants to the matchmaking events insisted on their usefulness and even asked for more of these events. On the awareness raising events there could have been less for the same effect and usefulness. For the launching and closing events, the co-organisers mentioned that it generated some administrative burden. Surveyed SMEs agreed that their participation has resulted or will result in a number of benefits for their company: their increased competitiveness in the market, higher profit margin, higher quality of their products, more clients, new business partnerships, support in introducing technological changes and funding opportunities for technological changes. Overall beneficiaries found the campaigns useful and cost-effective. Several potential improvements were identified: The delayed launch of the WATIFY website prior to the launch of the second campaign created obstacles for the promotional activities; The timing in organizing the events caused some inefficiencies and unexploited potential; The significant Facebook presence in countries outside the EU Member States or COSME programme countries for the second phase of WATIFY shows that this communication channel could have been used in a more efficient way.

·What are the regulatory/administrative costs and the benefits for the different stakeholders?

COSME’s contribution to EASME for the period 2014-2016 was EUR 21,7m, which covered the operational (administrative) aspects of the programme. This is approximately EUR 7,2 million per year and represents 6.6% of the budget committed to COSME in the same period, excluding the budget for ‘Access to Finance’ managed by EIF. 47  

For the EEN, approximately 600 Network member organisations are grouped in consortia organised at the national or regional level that have been selected through open calls. On average, approximately 5.5% of the COSME budget made available to the Network member organisations is dedicated to management/coordination activities.

The EYE European Partnerships (EPs) are consortia of Intermediary Organisations officially awarded by EASME, following an open call for proposals. Their staff costs are included in the grant provided by COSME

The European Investment Fund (EIF): The EIF acts as entrusted entity for the implementation of the COSME financial instruments through a mandate given by the Commission. For the period 2014-2016, the costs for the COSME implementation by the EIF amounted in a total of EUR 4.5 million for EFG and EUR 17 million for the LGF. These figures corresponded for EFG to 2.6% of the total EU budget committed and for LGF to 4.5%.

The supporting study 48 presented several Cost Effectiveness Analysis in terms of number of SMEs reached, the increase in turnover or the employment generated.

For example:

Each EUR 1 million invested in the EEN has led to an increase in employment of circa 377 people among client SMEs. This is based on an increase of two employees per company in the period 2014-2016 (calculated from our SME client survey), the total number of companies supported (210,000), and the additionality factor.

Each EUR 1 million invested in LGF has led to an increase in employment of circa 235 people among client SMEs. The evaluators further estimated – after accounting for the additionality – that for each EUR 1 million invested into the LGF (effects fully attributable to the Loan Guarantee Facility):

·additional employment of 491 will be created;

·additional EUR 22 million in turnover will occur in treated SMEs;

·470 SMEs will be supported.

Each EUR 1 million invested in EYE has led to an increase in employment of 33 people among SMEs (Hosts) and New Entrepreneurs. Again, this a very positive result; however, it implies that EEN and LGF are more cost-effective than EYE in terms of supporting job creation.

The Cluster Go International action reached 3,800 SMEs per EUR 1 million invested.

·How efficient is the implementation structure and governance of the programme?

The main strength of COSME is the clarity of its work programme descriptions, leading to a relatively small share of ineligible proposals responding to the calls for grants. No major administrative bottlenecks were identified and the simplification measures that were introduced were overall considered as positive, even though the new IT system clearly had some teething problems.

A major weakness of the programme design is the unbalanced distribution and strong fragmentation of the budget available. As stipulated by the COSME regulation, the large majority of the COSME budget (80%) is used for the two key actions. 49 The remaining 20% of the budget is spread over a large number of small actions. This fragmentation negatively influences the potential for cost-efficiency in the programme implementation and accentuates the limits in strategic steering and coordination of the programme. Competition in some COSME calls can be quite high, especially for the smaller actions, which can be considered a direct consequence of the funding fragmentation. This places an additional burden on applicants, which can be particularly problematic for SMEs and may lead to a narrowing of the applicant base, both geographically and sectorally.

Regarding the contribution of EASME to an efficient management of the COSME programme, two issues can be noted. First, the time-to-grant for the open calls was shorter than the target set by EASME. Second, the number of small actions delegated to the agencies requires the involvement of a proportionally higher number of staff than for bigger actions such as the EEN. In addition, surveyed beneficiaries indicated also the difficulties in using the EASME IT systems in the first years of the programme and the user-friendliness of the IT tools.

Another major weakness of the programme is the quality of data management. The lack in quality and especially completeness of the data on the beneficiaries of the programme constitutes a major hurdle for an efficient management of the programme implementation. The spread of the implementation responsibilities over the many actors involved (the EIF, EASME and in 2014-16, about 15 DG GROW units) creates a considerable challenge to efficient programme management and coordination, with a risk of weakened programme oversight. It also makes it very much harder to evaluate the programme. The various reporting systems’ focus on activities and outputs is equally problematic. A stronger attention for the collection of data on the profile of the beneficiaries is needed to allow for a proper monitoring of the programme outcomes and progress towards reaching its objectives.

5.4.Coherence

·Are the different actions of COSME implemented coherently between them?

Overall, the COSME programme shows a good level of internal coherence: efforts are increasingly being made to create synergies and no substantial overlaps could be identified. The EEN services are the most-often mentioned targets for synergies. The risk is to overcharge the EEN with responsibilities to which the Network cannot respond in an efficient way, and with the risk of losing the focus on its core activities. More synergies could be created by taking a more integrated, cross-thematic approach.

The communication activities related to the financial instruments created a certain abundance of information; those related to the entrepreneurship actions appeared to be run in isolation from each other. The assessment is positive in relation to the synergies and complementarities created between the Your Europe Business portal and the EEN services, and between other COSME activities geared towards internationalisation such as the EEN and Clusters Go International programme.

There is a positive coherence in relation to the complementarities and synergies created between Clusters Go International with other COSME activities geared towards internationalisation and the emerging markets of the main EU trade partners are covered through the action 50 .

A better cooperation between the IPR SME helpdesks and the Cluster Internationalisation actions should be a priority for the future. The interaction between the three helpdesks should be maintained, as well as with other COSME initiatives (mainly the EEN) and other EU and national initiatives.

·How are the different COSME actions overlapping/synergizing with the activities of ESI funds, H2020, EFSI, EASI and other EU programmes?

The division of tasks between COSME and H2020 is clear and we noted no significant overlaps between COSME and H2020 or other EU activities. Nevertheless, there is room for the creation of more synergy and complementarities, especially for the SME internationalisation activities. Overlaps between the COSME LGF and financial instruments established under ESIF may occur when addressing similar SMEs in the regions. The latter was also highlighted in the independent Evaluation 51 of the EFSI Regulation, which was concluded on 4 June 2018, namely that there is a risk of competition between ESIF financial instruments and LGF, and even indirectly EFSI.

The more than 625 members of the EEN help about 250.000 SMEs per year to increase their competitiveness and innovativeness in the Single Market and beyond by offering integrated services. There is a clear market failure for SMEs regarding information, funding, accessing new markets and networking to internationalize their business outside the home country. Network organisations provide information on funding opportunities made available by the various EU programmes such as COSME financial instruments, Horizon2020 SME Instrument and Structural Funds.

The Cluster Go international can also play a major role to incentivise European SMEs across regional industrial ecosystems to enter into joint business projects with partners in third countries 52 for other related actions, such as the EU’s Low-Carbon Business Actions in Brazil and Mexico, supported through the EU Foreign Partnership Instrument 53 .

·To what extent has the coherence been maximised in view of national programmes?

The interplay between the EU, national and regional levels of support for SMEs is not always synergistic and complementary, even though no substantial overlaps were identified. Especially for actions where proximity to the SME communities is a strong enabling factor, better coordination with national and regional actions would further improve coherence. This is especially the case for the EEN services and the COSME actions supporting cluster organisations.

Both EFG and LGF have adopted a bottom up approach, have the flexibility to combine with national programmes and appear to be rather complementary with other schemes. However, Surveyed LGF intermediaries see both complementarities (9 out of 16 intermediaries) and overlaps (10 out of 16) of the LGF instrument with national level actions.

Coherence with initiatives at the national/regional levels is a topic of relevance especially for the EEN services. The Network builds on regional players that are embedded in the local environment. The survey with SME clients of the Network shows that most clients were unsure whether the Network services are complementary to, or overlap with, existing national or regional support in their own country; the findings from the survey with Network members showed similar results. During the interviews, the Network intermediaries explained that they actively seek this complementarity. For example, when prioritising their activities, they aim to focus only on those services that are not yet available in their region. Moreover, many intermediaries explained that they have a good collaboration with other stakeholders that provide services to SMEs in the region; they organise joint events or activities, and actively seek to coordinate their actions. These national and regional stakeholders include ministries, municipalities, chambers of commerce, EU info centres, specialists in IPR, lawyers, etc. By means of signposting, the Network intermediaries refer SMEs to other stakeholders in the field and vice versa when relevant.

In conclusion, while just over half of the Network clients and members feel that the Network services are complementary to services provided at national or regional level, there is scope for improvement.

In the context of the Cluster internationalisation programme, interviewees indicated that there is no clear coordination between EU-level and national level cluster policies, which is perceived as problematic especially in the context of the limited funding available for the cluster internationalisation actions in COSME. The interviewees considered that the limited COSME funding for CGI might be considered as an incentive for clusters to co-invest also their own resources to achieve the desired results. They emphasised that this approach does not sufficiently take account of the diverging levels of funding for clusters in the EU Member States. According to these interviewees, this approach hinders a level playing field for cluster organisations in countries without direct cluster policies. They therefore emphasised the need for an improved coherence and creation of complementarity in the EU funding for cluster internationalisation with the availability of national level funding. In this context, it is to be noted that only 3% of the CGI participating clusters are located in EU13 countries.

The complementarity of WATIFY with similar national initiatives emerges from its European character. Besides these initiatives of a more cross-cutting nature, there are ones focused on specific topics / sectors. Some countries that used WATIFY materials agreed that while there are similar initiatives to WATIFY, it is important to have a European initiative encouraging enterprises to digitalise. Internally, the WATIFY campaign was overall coherent (in terms of there not being conflicting events or activities organised). There is a complementary aspect between awareness raising and matchmaking events. The latter were organised under WATIFY umbrella, and often included the cooperation with local or regional stakeholders. The former, in contrast, were organised by various actors in different European countries, and were used by WATIFY as platforms to introduce and promote the campaign, its objectives and messages.

5.5.EU added value

·What is the added-value of the programme and actions being conducted at the EU level?

COSME has a good level of European added-value. The European dimension constitutes the very essence of the design of its actions and is crucial for their implementation and effectiveness. In most cases, the ‘subsidiarity’ of these actions, i.e. the benefit of their implementation at the European rather than national or regional levels, is therefore very high. While many national and regional initiatives also seek to strengthen the competitiveness of SMEs, the scale of support through COSME and its availability to financial and business support intermediaries in every EU Member State are quite distinct and highly additional. In several cases, the EU level actions have not only been additional to, but have also helped in enhancing national, regional and local level measures.

Indeed, COSME provides important financial leverage that allows national and regional intermediaries to provide lending products, which have a higher risk profile compared to their normal product offering, and it allows offering support to a substantially bigger number of businesses than they might otherwise be able to service. Furthermore, national support programmes, especially in smaller markets, have higher leverage when combined with an EU-level programme. However, the added-value of the LGF suffers from the non-targeted approach, which so far has implied that the support did not sufficiently reach the less mature financial markets. In terms of EFG financing, intermediaries very clearly stated that EU action is additional, despite that as recognised by the independent Evaluation 54  of the EFSI Regulation, which was concluded on 4 June 2018, there were initial overlaps with the Expansion and Growth Window under the EFSI Equity instrument. Despite that the two instruments largely have the same investment focus, operations have been allocated to EFSI since it has greater resources and can offer more favourable conditions.

Moreover, the analysis from the supporting study reveals that the financing – supported by the LGF for example – also has triggered an additional investment from 21 % to 29 % as shown in the figure below.

Source: COSME Interim Evaluation (Technopolis, based on survey data)

Further, by taking into account the actual value of the principal financing received by those SMEs, the evaluators have estimated that LGF has de facto triggered approximately EUR 1,25 billion as additional investments as shown in the figure below.

Source: COSME Interim Evaluation (Technopolis, based on survey data)

The distribution in the two figures above is based on the SME definition. From the graphical presentation in both figures, it is evident that whereas the medium or large enterprises trigger higher additional investment percentage wise, i.e. 29 %, the smaller, but more numerous, the enterprises are, the greater value of additional investments will flow in the EU economy, i.e. EUR  959 million.

The supporting study also has identified via a dedicated survey that 65 % of the respondent SMEs are of the opinion that they would have borne negative consequences for their business plans in absence of LGF support.

Source: COSME Interim Evaluation (Technopolis, based on survey data)

As shown in the figure above, these negative consequences for 65 % of the SMEs mean de facto that 44 % would have continued their business project on a smaller scale and 21 % would not have been able to implement their business plans at all.

The European dimension is at the core of the Enterprise Europe Network. It allows for the attainment of a scale and quality of the services, and in some cases, even the implementation of the service as such, that would otherwise not be possible. The EEN’s role as an intermediary, involving Chambers of Commerce and national/regional innovation agencies, provides the advantage of a gateway for reaching SMEs; its EU added-value could be strengthened by focusing on those services that build upon the transnational characteristics of the Network itself, i.e. the internationalisation and single market penetration. Options to ensure an appropriate level of co-financing and/or funding from EU funds could be examined.

The IPR SME helpdesks add value at EU level by developing and broadening their expertise to all EU SMEs. In order that any SME in any EU MS have access to an equivalent quality of IPR service, it is of specific value to be able to offer them an IPR one-stop-shop to accompany them in their globalisation projects.

The Cluster Go International program (with its fostering of European partnerships among clusters in order jointly to develop and implement internationalisation strategies) shows very high EU added value, where the scale and depth of support goes well beyond the support that can be offered to European SMEs through national or regional initiatives 55 .

A separate evaluation of cluster initiatives from 2014 56 concluded for instance that the European cluster initiatives "generated added value through structuring cluster cooperation contributing to the emergence of European meta-clusters" and have reached out to a wide range of clusters, while flagging out that they lacked the financial means to involve SME sufficiently.

·To what extent and why do the issues addressed by the programme continue to require action at EU level?

The enhanced added-value for the Union of the proposed financial instruments lies in addressing market failures that cannot be addressed by Member States. 57 Funding at the supranational level in Europe allows the continent to match levels of funding seen in the United States. A discontinuation of the instruments would set at risk the current positive developments.

We have seen in the analysis of relevance that the market gap on access to finance for SMEs is still an issue despite the progress made. According to LGF intermediaries, if LGF support were no longer available, SMEs would face more difficulties in getting access to finance. It could be expected that fewer SMEs would get a loan or that interest rates and collateral requirements would rise. The effect of a hypothetical termination of the LGF support on financial intermediaries would be a reduction of activities, both in terms of a lower volume of (counter) guarantees provided and in terms of a reduced number of products offered to higher risk SMEs.

The EFG intermediaries considered action at the European level to be crucial for the European equity market to gain maturity. Equity markets in Europe are still relatively weak and only a few larger Member States can sustain their own equity markets.

The EEN member organisations stated in the survey that the quality, the coverage and the variety of services in their own country or region would decrease or stop if the Commission would lower funding. Full regional coverage and the principle of reciprocity are essential for the effectiveness of the Network. Therefore, if even only a small number of regions would not able to continue their support offer, it would have negative impacts on the service delivery in other parts of the Network as well.

On Entrepreneurship, there is still substantial room for further improvement, with data persistently showing relatively low levels of entrepreneurial interest and activity, as well as high rates of failure and slow rates of growth amongst newly established firms.

Also in the case of the other COSME action lines, the ‘benefit of their implementation at the European rather than national or regional levels, is very high. This is especially the case for those actions where the European scale is of critical importance for the attainment of positive effects also at the national and regional levels. Examples of such actions are those related to eSkills, the Tourism actions bringing together actors in European value chains, the Clusters Go International programme fostering European partnerships among clusters in order jointly to develop and implement internationalisation strategies. Multiple barriers still exist preventing SMEs from undergoing digital transformation. Considerable differences between the MS are still present resulting regions lagging behind in terms of digital transformation. The interviewed stakeholders agreed that a campaign such as WATIFY is needed and met market needs. Last, but not least, the SME policy actions providing strategic information as well as network opportunities for national policy-makers, fostering improved SME policy-making throughout Europe as well as an enhanced convergence. The scale and depth of support that COSME offers goes well beyond the support that can be offered to European SMEs through national or regional initiatives.

6.Conclusions and Lessons Learned

·What is working/what is not working, how it links to the intervention?

COSME actions are highly relevant in addressing the challenges related to fostering economic growth and creating employment opportunities and is strongly aligned to the evolving needs of SMEs. COSME supports competitiveness and growth of all SMEs. The strength of the programme lies in its focus on enhancing the SMEs’ capacities for growth and in its attention to the SMEs’ needs for concrete information and knowledge that are of immediate and practical use in their business practices. The programme also addresses in a direct and concrete manner the need of young entrepreneurs to strengthen their entrepreneurship skills.

However, this approach limits the possibility of specifically addressing other issues. Its two major actions, LGF and EEN, accounting for about 80% of the total budget committed, target all SMEs without making distinctions to reflect the programme’s ‘inclusive and sustainable growth’ and ‘global competitiveness’ policy objectives. The two major actions can therefore create only ‘unintended’ effects in those areas.  As a result, the relevance of COSME for policy objectives such as gender mainstreaming or climate change is limited.

The strength of the programme lies in particular in the use of intermediaries who have a direct and longstanding contact with SMEs for the implementation of the programme. COSME exploits the proximity of these intermediaries to SMEs and facilitates the integration of services provided by COSME with services provided by these intermediaries in their national and regional contexts. This allows for the provision of customised SME support tailored to the needs and challenges of specific sectors, such as tourism, textiles, creative industries etc. and to reach a high multiplier effect of actions.  COSME has thus a good level of European added-value. The European dimension constitutes the very essence of the design of COSME actions. Moreover, COSME actions have helped to enhance national, regional and local level measures for SME support and the teaming up of actors across EU value chains to boost internationalisation and industrial modernisation. The feedback from beneficiaries is generally positive with a substantial majority reporting a good cost-benefit ratio for their participation.

COSME is complementary to other EU policy instruments and programmes supporting business, without significant overlaps. It is the only EU programme designed to support SMEs' competitiveness and growth. Its ‘non-targeted’ offering addresses the basic needs of enterprises, facilitating access to finance, fostering cooperation and providing access to advice. It responds to existing needs, because a financing gap continues to exist in Europe, depending on the national context, and information asymmetries, specifically those related to market opportunities in the European Single Market and the global market, continue to require policy intervention for facilitating knowledge transfer and exchange as well as cross-border co-operation;

The supporting study highlights several aspects to be improved:

First, as stipulated by the COSME regulation, some 80% of the budget is used for promoting SMEs' access to finance and access to markets. The remaining 20% is allocated between a number of smaller actions with a limited budget and with a low cost-benefit ratio regarding the implementation. EASME (Executive Agency for SMEs) implements 35% of the overall programme budget, with two flagship COSME actions (Enterprise Europe Network and Erasmus for Young Entrepreneurs) as well as a host of smaller projects. The fragmentation of actions and limitations in terms of budget (as flagged out for the EYE and cluster measures) hampers the effective implementation 58 .

Second, while COSME is successful in fostering economic growth and the creation of employment opportunities, there is scope for strengthening its responsiveness to EU objectives related to EU global competitiveness and sustainable and inclusive growth. COSME's main instruments are designed to address all SMEs, independently of their sector and industrial specialisations. They only indirectly address challenges such as climate change, gender mainstreaming or youth unemployment.

Third, while there are no major overlaps with other policy initiatives, there is scope for creating further synergies with other EU, national and regional actions.

Fourth, in relation to monitoring the evaluation points out to a lack of centralised data about implementation and indicators are mostly based on outputs rather than on long-term effects. Some efforts are still needed to centralise the data that at present is dispersed between the coordinating of the programme, the units managing individual actions and the delegated entities.

·Lessons learned:

While acknowledging – based on the available evidence – the strength of the COSME financial instruments to deliver a financial support to the SMEs that most needed it, several areas for improvement are identified:

·Better links between financial instruments and other parts of the future SME programme;

·Better co-ordination between financial instruments and national EU schemes;

·Discontinuation of the EUR 150 000 threshold which has a negative impact on the efficient delivery of the guarantee facility and created significant administrative burden.

A future EU loan guarantee facility should help to ensure a more level playing field for SMEs in those countries where, according to current studies, the needs among SMEs are highest. The existing Equity Facility for Growth is considered to be effective. Nevertheless, it appears necessary to reduce the number of financial instruments and to align the Equity Facility for Growth with the equity facilities established under EFSI.

The limitation of budget beyond the financial instruments and the EEN creates a high competition between the remaining actions. It represents 20% of the budget for a high number of actions. A fragmented budget reduces the potential impact of these actions and it creates a high competition in the published calls.

There is still scope for simplification. Stakeholders report a good cost-benefit ratio for their participation in the programme but report that IT systems could be more user-friendly and there is administrative burden for participating in the programme actions.

At the level of coherence, while actions are rather synergetic with other EU programmes or complement well national and regional instruments for SME support, there are still some overlaps.

Finally, the lack of a centralised database for the implementation of the programme reduces the effectivity of monitoring and evaluation as well as the visibility of the programme.

·Does actual performance meet expectations?

COSME in the period 2014-2016 produced positive results. The programme implementation is on track to reach the objectives set out in the legal basis by the end of 2020.

The COSME LGF has been successfully set-up and is fully on track to delivering on the targets set in terms of SMEs to be financed and volume of financing to be made available. It confirms that the "COSME actions fully respond to the SMEs’ current needs to access finance".

The EYE programme has been established as pilot project in 2008 under CIP and since 2014 incorporated under COSME. It has been particularly effective in reaching its knowledge- and skills-related objectives and has laid the ground for important outcomes in terms of business relationships and internationalization for the participating entrepreneurs. However, the scale of the activities (in total about 2,000 new entrepreneurs) poses limits to the importance of this action line within the overall programme portfolio.

The limited evidence available suggests that COSME will have a positive impact on its beneficiaries’ growth and employment. The EIF has an essential role in the implementation of the financial instruments and EASME plays an important and much appreciated role in creating learning opportunities for the EEN members to enhance their capacities and better serve the SMEs in their local environments.

The EEN is very relevant to the overall need of SMEs in building trust in finding suitable partners. However, it does not address the barrier of high costs that come with building trust in finding suitable partners. As a result, a potential upgrade of the programme could therefore include other types of support, such as demonstration projects, feasibility studies and knowledge-acquisition activities.

COSME meets the expectations in terms of relevance, efficiency, effectiveness and coherence. Its actions show a high level of EU added-value. However, some efforts are still needed to improve efficiency, effectiveness and coherence. Some issues already pointed out by the CIP and EIP evaluations have not been solved during the 2014-2016 period: the fragmentation of the programme into smaller actions and the data management.

·What issues need to be addressed or will it resolve over time?

COSME covers a broad range of actions to implement the objectives set out in the COSME regulation with a limited budget. A clearer definition of the COSME priority areas should enable the programme to reach more efficiency and effectiveness. Several possibilities could be envisaged:

1) Deploying its existing resources in a more focused / strategic manner;

2) Entering new strategic partnerships with other EU programmes by strategically sharing the responsibility for interventions in specific areas; and

3) Increasing its overall budget in order to allow it to run both the non-targeted approach and strategic programmes in parallel.

Before 2020, an increase in the overall budget is not a valid option. The MFF review did not retain this possibility, except for the financial instruments.

A more strategic prioritisation of objectives in the design of the work-programmes could be implemented for the remaining years until the end of the programme in 2020. This could help reduce the fragmentation of the budget. A more centralised approach to data management would ensure higher efficiency in the programme management and a strengthened capacity for a quality implementation of the monitoring and evaluation function. It would also pave the way for the creation of the monitoring system for the post-2020 period.

For the design of the post-2020 period successor to COSME, an ambitious scenario could be proposed in the impact assessment of the successor programme to strengthen the overall effectiveness of the programme. However, considering the potential reduction of the overall EU budget due to the BREXIT and the new challenges the EU has to tackle, (such as Defence and Security), the most realistic option seems to be to better exploit the available budget, and focus on the most successful actions and reserve a budget for new ideas and pilots. Synergies with other programmes could be also exploited in order to really focus on the actions where COSME is the most effective, and to support enterprises in all stages of their life-cycle, from their creation to their growth.

At this stage, substantial changes in the financial instruments or the Enterprise Europe network will be counter-productive to implement. Considering the sophisticated implementation chain and the respective concomitant amendments procedures needed, it is reasonable to expect that any tangible impact would only produce effect after 2020. The potential burden imposed on intermediaries and SMEs following any changes should not be underestimated either. It is more realistic to accept that the conclusions of the evaluation will rather feed the design of the post-2020 instruments.

However, in the impact assessment for the next multi-annual financial framework, the Commission proposes:

·To improve the internal coherence of different parts of the programme, better linkages will be established by further strengthening the existing cooperation with the Enterprise Europe Network 59  (use the Enterprise Europe Network as a stronger signposting element);

·A better upstream co-ordination between financial instruments for SMEs established by Member

· States and those established under the SME window of the InvestEU Fund will be sought by using the existing SME Envoys Network 60  as an information exchange forum; 

·To reduce administrative burden for SMEs and financial intermediaries and to improve the impact of the SME guarantee facility, the EUR 150 000 threshold will be eliminated.

This interim evaluation will pave the way for the final evaluation of COSME that will also feed the detailed design of the first work programmes of the period post 2020. The final evaluation should analyse more in detail the wide variety of COSME actions to provide the Commission with data that will support the strategy to select the actions that will compose the future COSME in the next multi-annual financial framework. Preliminary findings of the interim evaluation have informed the impact assessments on Single Market and Invest EU programmes for the next Multi-annual Financial Framework 2021-2027.



Annex 1: Procedural information

1.Lead DG, Decide Planning/CWP references

lead DG: GROW

Decide Planning: PLAN_2016_223

CWP:

2.Organisation and timing

An inter-service steering group was created in Q4 2016. The steering group was constituted of the following DGs:

·BUDG

·SG (A.2 (ex C.1))

·ENV

·EMPL

·REGIO

·RTD

·TRADE

·ECFIN

·ENER

·EAC

In total 7 meetings of the Interservice steering group (ISSG) have been organised between Q4 2016 and December 2017.

After having been consulted with the ISSG and approved by the secretariat general the roadmap was published the 16/01/2017. No feedback was received from stakeholders.

The consultation strategy and the Terms of reference for the supporting study were submitted to the ISSG during Q4 2016. The supporting study was concluded by using a framework contract with Technopolis Group. The Specific contract was signed on 19/12/2016 and the contract started in January 2017.

The inception meeting took place on 27/02/17 and subsequently the inception report was submitted to the ISSG. The inception report was approved on 01/06/2017.

The interim report was submitted on 24/05/2017 and served to prepare the draft final report that was received on 04/10/2017. It was presented to the ISSG (including presentation by the contractor) at the 7th meeting of 13/10/17. The comments from DG GROW and other members of the group have been transmitted to the contractor to prepare the final report. From the ISSG, GROW, ECFIN, REGIO and SG requested changes.

The Final invoice was received the 23/10/17. Contract report approval was suspended 20/10/17 in order to consider ISSG comments. The Final Report was received 20/11/2017. It was sent to the members of the ISSG with deadline of 27/11/17 for comments or approval.

Although comments and corrections had been taken into account by the contractor, there were still corrections and modifications made and requested by the ISSG prior to finalising the document. The revised final report was received the 08/12/17. On 12-13/12/17 the approval of SG, ECFIN, TRADE and REGIO was received, no other member of the ISSG has made any objection so the final report was considered as approved by the ISSG and the final report was sent to all members of the group.

The final versions of all deliverables received from the contractor including IPR annexes from FWC are registered in ARES (ARES(2017)6154624 of 14/12/17).

3.Exceptions to the better regulation guidelines

none

4.Consultation of the RSB (if applicable)

not applicable

5.Evidence, sources and quality

The major difficulty for this evaluation has been the lack in data to measure results and impact. By the time of evaluation, only major and long-standing actions produced sufficient data to allow for measuring results. The evaluation measured on basis of outcomes and outputs produced the probability for the programme's actions to reach the desired effects. The main evidence for this evaluation is the supporting study performed by Technopolis Group. The quality of the supporting study has been positively assessed by the ISSG and other limitations are indicated in the synopsis and methodological report.

The supporting study took into account the below evidence, together with evidence from the evaluation of the IPR SME helpdesks. The complete list of sources is indicated in the methodological report.

·Regulation (EU) 1287/2013 of the European Parliament and of the Council of 11 December 2013 establishing a Programme for the Competitiveness of Enterprises and small and medium-sized enterprises (COSME) (2014 - 2020) and repealing Decision No 1639/2006/EC. 61

·Decision 1639/2006/EC of the European Parliament and of the Council of 24 October 2006 establishing a Competitiveness and Innovation Framework Programme (2007-2013) 62

·Impact Assessment for the proposal for a COSME regulation COM (2011)834 63

·MFF review 64

·Ex-ante Preparatory documents for COSME, including public consultation 65

·CIP Impacts and Results, including evaluations 66

·EACI evaluations 67

·Final evaluation of the Enterprise Europe Network 2008 – 2014 68  

·SME Performance Review and Small Business Act Factsheets 69  

·The Small Business Act for Europe, including the Report on the 2014 public consultation 70  

·Study commissioned by the Committee of the Regions: Implementation of the Small Business Act for Europe (SBA) and Entrepreneurship Policies at Local and Regional Level 71

·The Single Market Strategy 72

·Public consultation under the Start-up Initiative 73

·The Single Market Forum 74

·SAFE (Survey on Access to Finance) 75  

·Flash Eurobarometer 354, ENTREPRENEURSHIP IN THE EU AND

BEYOND, 08/2012 76  

·Flash Eurobarometer on the internationalization of SMEs 77 : 

COSME web page 78  

·EASME web page 79  

·EEN web page 80

·EYE (Erasmus for Young Entrepreneurs) web page 81  

·EIF, Loan Guarantee Facility (including implementation status) 82  

·EIF, Equity Facility for Growth 83 :

·Statistics, data and studies on Tourism 84  

·Data on regional funds:

-The ex post evaluations of the 2007-2013 ERDF and Cohesion Fund programmes: http://ec.europa.eu/regional_policy/en/policy/evaluations/ec/2007-2013/#1  

-ESF and its support to entrepreneurship and micro-finance ( http://ec.europa.eu/esf/main.jsp?catId=3 )

-For the EAFRD: http://ec.europa.eu/agriculture/evaluation/rural-development-reports/index_en.htm  

-For the EMFF: http://ec.europa.eu/fisheries/cfp/eff/index_en.htm and http://ec.europa.eu/fisheries/cfp/emff/index_en.htm   

-as regards the 20014-2020 programmes:

oThe expected outcomes, planned funding, etc. for all 5 ESI Funds: https://cohesiondata.ec.europa.eu/  (see for instance the search by SME competitiveness: https://cohesiondata.ec.europa.eu/themes/3 ).
http://ec.europa.eu/regional_policy/en/policy/evaluations/data-for-research/  

oERDF financial instruments: http://ec.europa.eu/regional_policy/en/funding/financial-instruments/  

Annex 2: Stakeholder consultation

Three types of consultations were conducted: the open public consultation, the ‘quantitative’ targeted consultations, and the ‘qualitative’ targeted consultations. In addition, 121 interviews were conducted and carried out seven quantitative consultations, seven qualitative consultations and one public consultation.

The European Commission implemented the questionnaire for the public consultation on the “Your Voice in Europe” website ( https://ec.europa.eu/info/consultations/interim-evaluation-programme-competitiveness-enterprises-and-small-and-medium-sized-enterprises-cosme-2014-2020_en ) on 10 May 2017 until the 31 of August 2017.

Both intermediaries and beneficiary SMEs were approached by targeted consultations. In total, 4,100 responses were received. The response rates among intermediaries was about 45%; it was between 10% and 20% among SMEs. The public consultation received 195 responses and 14 position papers.

For WATIFY, The stakeholder consultation strategy included three main activities:

1.An on-line target survey to the main beneficiaries of the awareness raising campaign

2.A general on-line survey targeting SMEs covering 16 Member States conducted through online panels

3.45 interviews covering all types of direct stakeholders



Synopsis Report

1    Overview of the consultation strategy    

2    Stakeholder groups involved in the consultations    

2.1    Stakeholders interviewed    

2.2    Stakeholders involved in the qualitative targeted consultations    

2.2.1    Results for SO1 – Access to finance    

2.2.2    Results for SO3 – Favourable environment    

2.3    Stakeholders involved in the quantitative targeted consultations    

2.4    The public consultation    

2.4.1    Overview    

2.4.2    Number and profile of the participants    

3    Robustness of the consultation results    

3.1    The quantitative targeted consultations    

3.2    The qualitative targeted consultations    

3.3    The public consultation    

4    Summary of the results    

4.1    Relevance    

4.2    Effectiveness    

4.3    Efficiency    

4.4    Coherence    

4.5    European added value    

Overview of the consultation strategy

Three types of consultations were conducted: the open public consultation, the ‘quantitative’ targeted consultations, and the ‘qualitative’ targeted consultations. In addition, 121 interviews were conducted and carried out seven quantitative consultations, seven qualitative consultations and one public consultation ( Table 2 ).

Both intermediaries and beneficiary SMEs were approached by targeted consultations. In total, 4,100 responses were received. The response rates among intermediaries was about 45%; it was between 10% and 20% among SMEs. The public consultation received 195 responses and 14 position papers.

Table 2: Tools and methods used in the evaluation

1.Consultation approach

2.Tools

3.Number

4.Qualitative consultations

5.Interviews

6.121

7.Qualitative targeted consultation

8.8

9.Quantitative consultations

10.Targeted consultation

11.7

12.Public consultation

13.1

1.Source: Technopolis

A major purpose of the high-level interviews was to collect background and context information on facilitators and barriers for the attainment of the expected effects as well as the eventual unintended effects. Wherever possible, the interviews were conducted face-to-face. Interviews with EU officials took place in the beginning of the study to reach an improved understanding of the overall context to the programme and the processes for the programme design.

The ‘classical’ targeted consultations were implemented in cases where the population was higher than 100 and would therefore allow for meaningful and robust statistical data analyses. The questionnaires mostly included close-ended questions and only a few open-ended ones. The responses were analysed by means of descriptive statistics, in several cases also including cross-analyses. We foresaw seven quantitative surveys in the Inception Report; they were all implemented.

The public consultation was a 12-week internet-based questionnaire. The objective was to cover all potential relevant stakeholders, in particular those that were not targeted through the targeted consultations and interviews, and to collect feedback on the COSME programme and its objectives. The questionnaire for the public consultation aimed at reaching all types of stakeholders. Given that the targeted consultations focused exclusively on the specific objectives, it was agreed to use the public consultation to collect data that would provide information on the programme as a whole.

Table 3 shows the evaluation questions covered by the different consultation methods. The consultation questionnaires allowed for a proper triangulation of the evidence collected and helped mitigating the risk of lack in data availability. For each evaluation question, the analysis was based on multiple sources and methods

Table 3: Use of the consultations per evaluation question

14.Evaluation question

15.Interviews & qualitative targeted consultations

16.Quantitative targeted consultation

17.Public consultation

18.Q1.1) Programme implementation and evolution since 2014

19.X

20.Q2.1) Ongoing relevance of the objectives

21.X

22.X

23.X

24.Q2.2) Relevance to the stakeholders and the EU citizen

25.X

26.X

27.X

28.Q2.3) Adaptation of the objectives to the political priorities

29.X

30.Q3.1) Progress towards achieving the specific objectives

31.X

32.X

33.Q3.2) Factors driving or hindering progress

34.X

35.X

36.Q3.3) Progress towards contributing to the 5 Union priorities

37.X

38.X

39.Q4.1) Regulatory/administrative costs and benefits for the different stakeholders

40.X

41.X

42.Q4.2) Costs (direct and indirect) proportionate to the benefits

43.X

44.X

45.Q4.3) Efficiency of the programme implementation structure and governance

46.X

47.X

48.Q5.1) Internal coherence

49.X

50.X

51.

52.Q5.2) Coherence with ESI funds, H2020, EFSI, EASI and other EU programmes

53.X

54.X

55.X

56.Q5.3) Coherence with national programmes

57.X

58.X

59.X

60.Q6.1) Added value of implementation at the EU level

61.X

62.X

63.X

64.Q6.2) Ongoing need for action at EU level and why

65.X

66.X

67.X

2.Source: Technopolis

The time plan of the consultation strategy was that the targeted consultations would take place in the month of April, ending in the beginning of May. The statistical analyses of the survey data were scheduled for the month of May, in time for the evaluation team to start drafting the internal reports at the thematic area level in the month of June. This strict time schedule was dictated by the need of the evaluation team to run the triangulation process, eventually conduct additional interviews, and subsequently, various runs of the aggregation of the evaluation findings (at thematic area and programme level), in time for the delivery of the draft final report in the beginning of September.

Table 4 presents an overview of the public consultation process, from start to finish.

Table 4 Overview the open public consultation process

68.Activity

69.Timing

70.Phone call with the EC

71.10 February 2017

72.Delivery of first draft questionnaire

73.24 March 2017

74.Review from the EC

75.29 March 2017

76.Validation of final questionnaire

77.26 April 2017

78.Editing in EN by EC

79.26 April – 5 May 2017

80.Translation in other EU languages

81.6 May – 12 June

82.Deadline of the consultation

83.Launch date: 10 May

84.Closing date: 31 August

85.EC to send inputs to Technopolis Group

86.Latest results received mid-September (13/09/2017)

3.Source: Technopolis

Stakeholder groups involved in the consultations

Stakeholders interviewed

Overall, the contractors' team conducted 121 interviews. These were distributed over the SOs as shown in Table 5 , below.

Table 5: Stakeholders interviewed

87. SO

88. Interviewed stakeholders

89. Nr

90.SO1 – Access to finance

91.Representatives from EU institutions (DG GROW, DG REGIO, DG RTD, DG ECFIN, EIF and EIB)

92.15

93.intermediary and industry organisations

94.6

95.SO2 – Access to market

96.EU officials who were responsible for the Network and for other programmes aimed at fostering coherence within the Network.

97.9

98.Network members

99.19

100.Umbrella organisations were also interviewed so as gain the views of SMEs and intermediaries at a higher aggregated level.

101.5

102.SO3 – Favourable environment

103.EU officials in DG GROW, EASME and other DGs

104.16

105.Beneficiaries (cluster managers)

106.6

107.National government representatives

108.4

109.Umbrella organisations and national experts

110.10

111.SO4 – Entrepreneurship

112.Individuals responsible for the design, management and oversight of COSME entrepreneurship actions – including European Commission officials, EASME staff and individuals within the EYE support office

113.20

114.Host Entrepreneurs – participants in and beneficiaries of the EYE programme

115.5

116.New Entrepreneurs - participants in and beneficiaries of the EYE programme

117.6

4.Source: Technopolis

Stakeholders involved in the qualitative targeted consultations

Table 6 Overview of the participation in the qualitative surveys

118.SO

119.Actors & stakeholders targeted

120.Target sample

121.Nr of responses

122.Response rate

123.SO1 – Access to finance

124.LGF intermediaries

125.21

126.16

127.76%

128.EFG intermediaries

129.5

130.3

131.60%

132.EFG SMEs

133.12

134.4*

135.33%

136.Members of European Intermediary Organisations & LGF sub-intermediaries

137.6

138.5

139.83%

140.SO3 – Favourable environment

141.SME Policy: SME Envoys

142.28

143.15

144.54%

145.SME Policy: SPR Working Group members

146.127 (according to the email list received from EC)

147.23

148.18%

149.Sectoral competitiveness: European stakeholder organisations, national stakeholder organisations (associations), national and regional public authorities

150.72

151.7

152.10%

153.E-skills: European stakeholder organisations, national stakeholder organisations dealing with e-skills

154.31**

155.8

156.26%

5.Source: Technopolis - * conducted through a cascade approach **out of the 53 e-mail addresses received, only 37 were valid. Out of them 6 contacts were contacted for interviews and 31 for the qualitative survey.

Results for SO1 – Access to finance

Among the 61 financial intermediaries receiving LGF funds, 20 were approached. Four qualitative surveys were conducted:

1A written questionnaire was sent via e-mail to 21 LGF intermediaries, asking the recipients to complete the questionnaire within two weeks. Five of them were the financial intermediaries (cat 1 promotional banks and cat 2 guarantee societies) and others were selected according to their type (Promotional Institution, Guarantee Institution, Commercial Bank, Leasing Company), country of establishment, direct/counter Guarantees and risk category in order to have a wider and representative possible coverage. Some of them where involved also in the CIP SMEG and overall 50% were currently using also InnoFin SMEG; they gave us more insights on the process and leverage effect.

Two reminders were sent to LGF intermediaries that did not respond by the fixed deadline. Finally, the team received 16 completed questionnaires, corresponding to a response rate of 76%. One LGF intermediary explicitly refused to participate, while four intermediaries did not answer. Eight responding intermediaries provided direct guarantees for loans and another eight intermediaries provided counter guarantees.

2A written survey of EFG intermediaries was conducted. The team originally planned to collect data from five EFG intermediaries. The responding EFG intermediaries make equity and equity-related investments in growth-oriented start-ups and SMEs all over Europe. Only three out of five contacted intermediaries completed the written questionnaire.

3The EFG intermediaries were asked to cascade the survey to selected SMEs 85 that received equity funding from the EFG instrument. It was planned that 12 SMEs were to be contacted by their respective intermediaries. Due to the limited response from the intermediaries, only a limited number of SMEs that were beneficiaries of the EFG (EFG SMEs) could be reached. As a result, the team received only four completed questionnaires (out of 12) from EFG SMEs.

4Five of the six contacted European Intermediary Organisations were asked by the team to cascade the survey to their members. Except for one intermediary, the European intermediary organisations contacted their members prior to the interviews and collected information on their members’ opinions and views. In two cases, the interviews were even conducted in the form of phone conferences in which representatives of both the umbrella organisation and individual members of the intermediary participated. In this way, the opinions and views expressed during the interviews directly reflect those of individual members.

Results for SO3 – Favourable environment

We launched a total of four ‘qualitative’ surveys related to the actions under SO3. A first step in the process was the drafting of the survey questionnaires. Table 7 below, gives an overview of this process, which was rather time consuming due to the number of surveys to be covered.

Table 7: Overview of status of qualitative surveys

157.Survey

158.Date of draft

159.Approval date

160.Launch of survey

161.SME Policy – SME Envoys

162.21.03.2017 - draft 1; 27.03.2017 – draft 2

163.27.03.2017

164.28.03.2017

165.SME Policy – SPR Working Group

166.27.03.2017

167.11.04.2017

168.11.04.2017

169.E-skills

170.10.04.2017

171.10.04.2017

172.28.04.2017

173.Sectoral competitiveness

174.10.04.2017

175.18.04.2017

176.02.05.2017

6.Source: Technopolis

The results of these surveys vary substantially. There was a good level of response to the survey in the field of SME Policy (perhaps due to the fact that it was sent out by the European Commission), while response rates were low for e-skills and sectoral competitiveness. Even though the contacted actors had participated in events or activities of the COSME actions in the field of KETs, e-skills, or sectoral competitiveness, some of them mentioned that they were not aware of the COSME programme nor had they had any interaction with it. In addition, despite being contacted several times, the members of the Digital Coalition for Skills and Jobs (to whom the e-skills survey was targeted) were not responsive – only the Coalition members contacted for interviews were willing to contribute.

Stakeholders involved in the quantitative targeted consultations

Table 8 presents an overview of the quantitative surveys, including the target group, population, target sample, responses, expected response rates and response rates. Out of the seven consultations, five reached or exceeded the expectations in terms of response rate. One did not reach its targets: the survey to the EEN member organisations.

Table 8: Overview – Quantitative surveys

177.#

178.Action

179.Type of stakeholder

180.Pop.

181.Targeted sample

182.Number of responses

183.Expected response rate

184.Response rate reached

185.Closing date in 2017

1

Loan Guarantee Facility

Beneficiary SMEs

140,000

3,875

356*

6.3%

9%

18 July

2

Enterprise Europe Network (EEN)

EEN member organisations

523**

530

270

~60%

48%

17 May

3

Enterprise Europe Network (EEN)

Beneficiary SMEs

(EEN clients)

100,000

50,000

2,334

~5%

5%

17 May

4

Clusters Go International

Cluster managers

136

136

48

25-30%

35%

5 May

5

Tourism (Calls)

Beneficiary SMEs and Intermediary agencies / organisations

377

389

162

20%

41%

10 May

6

Erasmus for Young Entrepreneurs

New Entrepreneurs

2,343

2,117

595

25%

28%

5 May

7

Erasmus for Young Entrepreneurs

Host Entrepreneurs

1,800

1,620

245

10%

15%

5 May

7.Source: Technopolis - *Reached via 20 financial intermediaries. **Reached via 92 coordinators.

The public consultation

Overview

The approach of the public consultation followed the consultation process defined by the European Commission (Better Regulation Toolbox/Tool #50). The questionnaire was available in all EU languages. The European Commission implemented the questionnaire for the public consultation on the “Your Voice in Europe” website and the questionnaire was available in English on 10 May 2017. It remained accessible for 12 weeks, starting from the date on which the last translation was made available online. The dissemination of the public consultation fell under the responsibility of the European Commission. EASME and the EC’s communication units advertised the public consultation on COSME, EASME and EIF websites; through EEN networks and relevant national contact points; through other agencies and networks under COSME; and via other EC social media.

Number and profile of the participants

The public consultation was closed on the 31 August 2017. A total of 195 responses were received, as well as 14 position papers. The survey responses were analysed using descriptive statistics.

The questionnaire also included questions that allowed to profile the respondents to the public consultations. Half of the respondents were active in industry/business; the second largest group comprised of financial intermediaries (financial institutions or finance providers). The respondents under ‘others’ included law firms, research centres, innovation centres, NGOs and associations; business organisations; public authorities; and citizens (see Figure 4 ). Five countries account for two thirds of the answers (France, the UK, Italy, Spain and Germany).

Figure 4 Public consultation - profile of the participants

Source: COSME public consultation, 2017, n=195

Two-third of the respondents, i.e. 88 respondents, indicated that their organisation had benefitted from the COSME programme. Five respondents were involved in more than one action (e.g. EEN as a member organisation and beneficiary, etc.). The Loan Guarantee Facility (LGF) is, by far, the most recurrent action.

Table 9 Number of respondents per type of financial support

186.Type

187.Action

188.Number of beneficiaries

189.Access to finance

190.Loan Guarantee Facility

191.55 in total:

192.29 companies

193.16 financial institutions

194.10 others

195.Equity Facility for Growth

196.5

197.Access to markets

198.Enterprise Europe Network (EEN) – EEN member organisations

199.15

200.Enterprise Europe Network (EEN) – beneficiaries

201.13

202.Entrepreneurship

203.Erasmus for Young Entrepreneurs – New Entrepreneurs

204.4

205.Erasmus for Young Entrepreneurs – Host Entrepreneurs

206.4

207.Favourable environment

208.Cluster internationalisation

209.2

210.Tourism (calls)

211.2

212.

213.Other:

214.Selected as an expert

215.NCP for COSME including financial instruments and publishing calls for proposals in all areas / Coordinator EEN including extra support for EYE-participation

216.COS-DESIGN

217.3

218.

8.Source: COSME public consultation (n=88)

Robustness of the consultation results

A key method to ensure robustness of the analysis in an evaluation is the triangulation of the findings, i.e. the validation of information collected through one method by comparing it with the information collected through another method. In this section, instead, we expand on the robustness of the findings collected through the single consultation methods.

The quantitative targeted consultations

Apart of the more than satisfactory response rates reached for the quantitative targeted consultations, the profile of the responding organisations is also in close to all surveys very similar to the target sample, thus guaranteeing the statistical representativeness. Exceptions to the rule are the survey of the Network client SMEs, managed through a cascade approach, for which less responses than expected were reached; in this case, also the representativeness of the responses cannot be checked. Another exception is the survey related to the Clusters Go International programme where more funded ESCPs than non-funded ones responded compared to the target sample. In this case, however, this does not imply forcefully a lack of representativeness for the funded ESCPs.

Table 10: Criteria and results regarding the statistical representativeness analysis

219.Survey

220.Response rate

221.Criteria for representativeness

222.Result

223.SO1 - LGF beneficiaries

224.9%

Country of residence

Sector of activity

A higher response rate than recorded in previous exercises The distribution of the survey responses is well balanced in terms of country and sector of activity in comparison with the target sample. One notable exception is France (no responses received)

225.SO2 - Network member organisations

226.48%

Country grouping

Network member role

Response rates from both coordinator (47%) and partner groups (48%) are roughly equal

Distribution of the survey population is very similar to the target sample

227.SO2 - Network client SMEs

228.5% of the estimated target population (cascade approach)

n.a.

Over two thirds of respondents located in EU15 Members States, but sufficient respondents located in EU13 MS to analyse questions based on country groupings

Number of respondents from COSME-participating third countries too low to make meaningful analysis.

229.SO3 – Clusters Go International

230.35%

Funded versus voluntary ESCPs

The response rate for funded organisations was higher than that for voluntary organisations

231.SO3 –Tourism

232.41%

Country grouping

High level of alignment between the country groups split between the target sample and respondent groups, for those for which this information was available.

233.SO4 - New entrepreneurs

234.

235.28%

Country grouping

Distribution of responses (among those respondents for which the geographical location is known) is very similar to that of the target sample

236.SO4 - Host entrepreneurs

237.

238.15%

Country grouping

Distribution of responses (among those respondents for which the geographical location is known) is very similar to that of the target sample

The qualitative targeted consultations

As shown in Section 0, the qualitative consultations gave different results. While three out of the four surveys related to SO1 reached a reasonable level of response rate, the same can be said only for one out of four surveys related to SO3.

The qualitative surveys for which the number of returned questionnaires was below expectation are the ones targeting the EFG beneficiary SMEs, and the surveys related to SME policy actions (one out of two), the sectoral competitiveness actions, and the e-skills actions. A factor playing a role for the EFG beneficiaries was the cascade approach used; for the sectoral competitiveness one, the wide variety of sectors to be covered was a hindering factor, as well as the related difficulty to identify the relevant stakeholders to include, even with the support from the EC.

Overall, the quality of the responses was also often below the expected level; the full exploitation of this evaluation tool was hindered by the impossibility for the evaluation team to set up a direct follow-up interaction with the respondents. These consultations did not have the intention to reach ‘statistical representativeness’.

The public consultation

Public consultations present limits for use in the context of an evaluation due to the impossibility of controlling and checking the ‘statistical representativeness’ of the information provided. For example, the profile of the respondents shown in Section 0 clearly indicates a strong representation of stakeholders involved in the financial instruments and respondents involved in a mix of other COSME actions. A use of these data in a statistical manner would therefore have been inappropriate. Instead, we considered the responses as well as the opinion papers submitted as illustrations of a viewpoint, in contrast to our original intentions, used the data exclusively to corroborate findings deriving from various other data collection tools used.

Summary of the results

Relevance

Quantitative consultations

Looking at the financial instruments, it appears that they matched the beneficiaries’ needs. More than half of the respondents indicated that they preferred the option that included the EU-COSME guarantee, though other sources of finance were available to them that would cover all or part of their required amount. The survey shows that the EEN member’s view on the extent to which Network services help SMEs in addressing their needs is in general quite positive. There is a good match between the type of services financed by the Network and the demand of companies. When it comes to tourism, respondents were positive, with each of 12 needs rated as being addressed ‘to a great extent’ or ‘to some extent’ by at least two thirds of them. All cluster organisations responded positively to the benefits resulting from collaboration, calling for the creation of stronger linkages between locations brought about by COSME actions. When it comes to the EYE, the respondents’ answers provide widespread confirmation of the existence of market and systemic failures in the area of barriers to entrepreneurship and entrepreneurial culture were present in their country today.

Interviews

Financial intermediaries interviewed mentioned that some financial intermediary needs – and therefore, in cascade, SME needs - are not sufficiently supported by the LGF. Interviewed stakeholders regarded the focus of the EFG instrument as appropriate. Interviewed intermediaries indicated that their EEN services matched the needs of companies and SMEs in the field very well. Interviewees indicated the creation of knowledge-exchange opportunities is perceived as critical in the field of KETs. They considered there was a gap in the support from COSME from that perspective; more support for awareness raising activities was seen as beneficial. Stakeholders interviewed agreed that support by the SME policy actions is highly relevant for the dissemination of the ‘Think Small First’ principle.

Public consultation

Networking actions rank as the most relevant actions of the programme amongst the respondents. Two thirds of them considered network-building and cooperation as relevant. Financial instruments come second, with on average, a little less than two thirds of the respondents thinking grants and loan portfolio guarantees are relevant.

Effectiveness

Quantitative consultations

The EU-COSME guarantee appears to enable focused investments in purchases or improvements/developments among our survey respondents in areas of capital expenditure, regardless of business size. Respondents were rather positive about the potential for the financing to further enable expansions in existing markets and the development of products or services that are new to the market. Results from the survey show that the services related to knowledge enhancement of the EEN are rated highly by the SMEs. Moreover, Network members stated that their participation in the Network allowed them to build stronger networks for their organisation. When it comes to tourism, respondents’ answers suggest that COSME-funded tourism projects are achieving their expected objectives in relation to visibility and capacity. Looking at cluster organisations, over half of the responding organisations reported that they had, at least to some extent, achieved (or were expected to in the near future) the objectives of the actions. A majority of EYE beneficiaries, i.e. the New Entrepreneurs, reported improvements in all of the suggested skill areas, and in particular various personal and language skills. Host Entrepreneurs reported widespread benefits for their business activities as a result of hosting a New Entrepreneur.

Interviews

According to interviewees, the EEN services had a positive indirect effect on business cooperation at the regional level thanks to participation in the information events. The interviewees emphasised on the importance of the knowledge- and network-sharing activities among the Network members for the overall functioning and quality of the services provided to SMEs. The interviewees considered the e-skills actions effective especially in raising awareness on the needed skills and in engaging the relevant stakeholders and policy-maker communities. Most national representatives interviewed stated that the SME performance review provides reliable statistical data and analysis on EU SMEs and SME policies. Some interviewees mentioned that the SME Envoys network could be more exploited. The interviewees find that the clusters and cluster managers are well placed to facilitate the internationalisation of their member SMEs as cluster managers are acquainted with the technologies of the companies in their network and can promote the “ecosystem” to outside markets.

Public consultation

COSME fully responded to its key objective of supporting jobs, growth and investment, with half of the respondents who believe that the programme largely contributed to it. Additionally, almost half of the respondents believe that the COSME programme supports the creation of start-ups, the growth of SMEs and business transfer. As much respondents believe that the programme eases access to finance for SMEs through loans. Improved international visibility and more cooperation with partners from other countries are the two main expectations respondents have regarding COSME

Efficiency

Quantitative consultations

Survey responses paint a positive picture of the EU-COSME guarantee. The majority of respondents reported that they had not experienced additional costs, administrative burden or complexity as a result of their EU-COSME guarantee. When it comes to the EEN members, the survey results show that only 9% of the Network members state that benefits of the Network do not outweigh the time investment. Regarding the tourism actions, respondents were generally positive about both (financial and non-financial) aspects, with 85% (122 respondents) either strongly agreeing or agreeing that the Programme provides funding in a time-efficient manner. Looking at cluster organisations, over a third of them thought the reporting and monitoring requirements set by the programme were reasonable. However, many of the organisations felt the funding requirements were not correctly estimated by the ‘Clusters Go International’ programme. Though New Entrepreneurs do receive programme funding to cover certain travel, accommodation and other costs during their placement, our survey suggests that this is often not sufficient to cover costs during their stay. Most respondents were broadly satisfied across all of the aspects covered by the survey, including the timing and administration, as well as the support and assistance that was provided by intermediary organisations at different stages.

Interviews

The interviews indicated that EASME is quite efficient in the tasks they run (EEN) thanks to the experience built up during the CIP programming. The transition to the COSME programme did not have a significant impact on their tasks, in which efficiency was preserved. The availability of EASME to respond to questions from the Network members was highly praised by the interviewees. CGI participants interviewed felt unrealistic expectations were set by the EC especially for Strand 2 actions.

Public consultation

According to the respondents, the COSME procedures and administrative documents are fair, transparent and clear. In addition, the overall programme is considered as well structured. On the other hand, the programme’s biggest weakness is in its capacity to communicate and reach the potential beneficiaries (less than one in five respondents thinks it has the capacity). A couple of questions were targeted at respondents from non-EU participating countries. Overall, it appears that the COSME programme has not been very convincing to those respondents.

Coherence

Quantitative consultations

When it comes to the LGF, it is difficult to make robust conclusions about the complementarity of the EU-COSME guarantee and other initiatives, as there is little uptake (and thus opinion on) other sources of funding (both European and national or regional) for the same or similar activity among beneficiaries. Looking at the EEN and other EU programmes, survey results show a high level of complementarity between the Network and H2020. At national level, the maturity of the role of the Network members needs to grow, while the recognition of the Network by stakeholders in the region needs to be further developed. Though at least 50% of COSME tourism actions beneficiaries said they were aware of the campaigns, the respondents indicated a generally low uptake of the tools. Survey respondents were rather more positive when asked to what extent they use the tourism-related networking opportunities offered by the European Commission through the COSME programme. Regarding the cluster organisations, the proportion of respondents reporting complementarity between Clusters Go International and other activities greatly outweighed those reporting any overlap with any of these other schemes. When it comes to the Entrepreneurship TA, survey respondents did not point at specific overlaps between individual COSME entrepreneurship actions.

Interviews

Desk research and interviews showed a high level of synergy with the EEN by allowing SMEs to send out questions from the portal to the Network members. Interviews show that there is a high level of complementarity between the Network and H2020. Interviewees considered that there is room for creating more synergies between the cluster internationalisation actions and the actions supporting cluster value chains for S3. A more ‘holistic’ integrated approach to supporting Cluster internationalisation as part of the COSME Programme would be beneficial. Interviewees mentioned the need to coordinate the approach to cluster internationalisation with the new COSME cluster support actions targeting the strengthening of value chains for smart specialisation. In the context of the Cluster internationalisation programme, interviewees indicated that there is no clear coordination between EU-level and national level cluster policies.

Public consultation

On average, two thirds of the respondents did not know what to answer regarding the coherence of COSME with other EU programmes. Nonetheless, Horizon 2020, the European Fund for Strategic Investment (EFSI) and the European Structural and Investment Funds (ESIF) are the programmes with which COSME works the most in synergy/is the most complementary.

European added value

Quantitative consultations

Survey responses show that the EU-COSME guarantee creates significant value for beneficiaries. The EU-COSME guarantee both provides funding for those that had struggled to receive funding for their projects from other sources, as well as supporting projects to be conducted at a larger scale than would be possible without the funding. The survey results show that responding SME clients held quite positive views about the importance of the EU level Network services as opposed to only national support. Survey respondents were very positive about the value of EU level support in the field of tourism. Survey respondents were also very positive when asked to what extent they agree that the tourism-related actions funded by COSME have resulted in specific developments that would not have happened otherwise. Four out of five cluster organisations, part of the Clusters Go International programme, strongly agree that cluster cooperation is essential to create a more competitive European Union, while none of the organisations disagreed with this statement. Host and New Entrepreneurs who have participated in the EYE programme were nearly unanimous in their belief that the EYE scheme should continue in the future. Indeed, the vast majority of both groups stated that this was ‘very important’.

Interviews

Interviewees generally agree that there are no private institutions nor national governments that can achieve the results that COSME (financial instruments) has managed to achieve in terms of leverage effects and scope and risk profile of the SMEs involved. In the interviews, the intermediaries clearly indicated that for all the ‘structural’ elements of the Network, the European dimension constitutes a pre-condition. Many interviewees indicated a shortage of funding in their countries or regions for this type of SME support; in this case, the additionality is therefore of a financial kind.

Public consultation

Overall, a third of the respondents considers that the overall value of COSME compared with national and/or regional SMEs and entrepreneurship support programmes is higher. More than half of the respondents believe the programme helped financing projects which otherwise could not be supported at national and/or regional level.

Annex 3: Methods and analytical models

Table of Contents

1    Introduction    

2    The methodological framework    

2.1    Methodological approach    

2.2    Overview of the evaluation tools and methods    

2.2.1    Brief description of tools and methods    

2.2.2    Matrix for the triangulation    

2.3    Robustness of the results    

2.3.1    The quantitative targeted consultations    

2.3.2    The qualitative targeted consultations    

2.3.3    The public consultation    

2.4    Flow of the analysis    

3    Desk research    

3.1    Overview    

3.2    Details at the Thematic Areas level    

3.2.1    SO1 – Access to finance    

3.2.2    SO2 – Access to market    

3.2.3    SO3 – Favourable environment    

3.2.4    SO4 – Entrepreneurship    

4    Statistical data analysis    

4.1    Sources    

4.2    Portfolio analysis    

4.3    Composition analysis    

4.4    Analysis of COSME programme management and proposal data    

4.5    Cost-Effectiveness analysis    

5    The consultations    

5.1    The targeted consultations – quantitative    

5.1.1    Overview    

5.1.2    Sampling strategy and results    

5.1.3    Robustness of the results    

5.2    The targeted consultations - qualitative    

5.2.1    Overview    

5.2.2    Sampling strategy    

5.2.3    Results for SO1 – Access to finance    

5.2.4    Results for SO3 – Favourable environment    

5.3    The public consultation    

5.3.1    The structure of the public consultation questionnaire    

5.3.2    The implementation of the public consultation    

5.3.3    Outcomes of the public consultation    

5.3.4    Statistical analysis    

6    Interviews    

6.1    Overview    

6.2    Results    

6.2.1    SO1 – Access to finance    

6.2.2    SO2 – Access to market    

6.2.3    SO3 – Favourable environment    

6.2.4    SO4 – Entrepreneurship    

Appendix A - COSME programme KPIs, and the extent to which these KPIs have been realized    

Appendix B - Statistical data analyses results    

Appendix C - List of documents analysed    

Appendix D – Lists of interviewees    

List of Tables

Table 1 Evaluation questions    

Table 2 Tools and methods used in the evaluation    

Table 3 Use of the methodological mix per evaluation question    

Table 4 Literature list at the programme level    

Table 5 Evidence table for the desk research and literature review    

Table 6 Preliminary list of the available relevant databases    

Table 7 Portfolio analysis levels    

Table 8 Overview of available data on beneficiaries    

Table 9 Overview of available data on intermediaries    

Table 10 Calls on which time to grant analysis is based    

Table 11 Time to grant by thematic area and action line    

Table 12 Cost-effectiveness analysis    

Table 13 Overview – Quantitative surveys    

Table 14 Overview of the participation in the qualitative surveys    

Table 15 Overview of status of qualitative surveys    

Table 16 Overview of responses to SO3 short survey consultations    

Table 17 Overview of the open public consultation process    

Table 18 Number of respondents per type of financial support    

Table 19 Overview of the interviews conducted, by SO    

Table 20 SO3 – Performed interviews    

Table 21 Legend to Table 22 below    

Table 22 Main activities, KPIs, and the extent to which these KPIs have been realised    

Table 23 Country penetration rates of LGF guarantee 2014-2016    

Table 24 Geographical distribution - shares by country aggregation of number of beneficiaries for 2014-2016    

Table 25 Geographical distribution – estimated number of final recipients by country in EU for 2014-2016    

Table 26 Geographical distribution – weighted shares of participants by country in EU for 2014-2016    

Table 27 Geographical distribution - shares by country aggregation of cumulative budget for 2014-2016    

Table 28 Geographical distribution (money allocated to final beneficiaries in EUR k for 2014-2016)    

Table 29 Sectoral involvement (estimated shares of cumulative budget 2014-2016)    

Table 30 Sectoral involvement (shares of final beneficiaries 2014-2016)    

Table 31 Sectoral involvement (budgets in EUR m)    

Table 32 Sectoral involvement (number of final beneficiaries)    

Table 33 List of documents analysed SO1    

Table 34 List of background documents SO2    

Table 35 List of background documents SO3    

Table 36 List of documents analysed SO4    

Table 37 Interviews performed SO1    

Table 38 Interviews performed SO2    

Table 39 Interviews performed SO3    

Table 40 Interviews not performed SO3    

Table 41 Interviews performed SO4    

Table of Figures

Figure 1 Evaluation criteria and intervention logic in an interim/mid-term evaluation    

Figure 2 Intervention logic of the COSME programme    

Figure 3 Flow of the analysis    

Figure 4 LGF beneficiaries: comparison of the country distribution of the whole population, target sample and survey population    

Figure 5 LGF beneficiaries: comparison of the sector distribution of the whole population, target sample and survey population    

Figure 6 Network member survey target sample and respondents, by county    

Figure 7 Network member survey target sample and respondents, by role    

Figure 8 Network client SME company survey respondents, by country group    

Figure 9 Distribution of population in sample and responses between type of cluster organisation    

Figure 10 Profile of target sample and SO3 tourism survey respondents, by country group    

Figure 11 Distribution of population in sample and responses by types of countries    

Figure 12 Distribution of population in sample and responses by types of countries    

Figure 13 Public consultation - profile of the participants    

List of abbreviations

Abbreviation

Full name

CEA

Cost-Effectiveness Analysis

CIP

Competitiveness and Innovation Framework Programme

COSME

Programme for the Competitiveness of Enterprises and Small- and Medium-sized Enterprises

DG CONNECT

Directorate-General for Communications Networks, Content and Technology

DG GROW

Directorate-General for Internal Market, Industry, Entrepreneurship and SMEs

DG REGIO

Directorate-General for Regional and Urban Policy

DG RTD

Directorate-General for Research and Innovation

DG TRADE

Directorate-General for Trade

DGT

Directorate-General for Translation

DG EAC

Directorate-General for Education, Youth, Sport and Culture

EASME

Executive Agency for SMEs

EC

European Commission

EEN

European Enterprise Network

EFG

Equity Facility for Growth

EIB

European Investment Bank

EIF

European Investment Fund

EIP

European Innovation Partnerships

ESCPs

European Strategic Cluster Partnerships

EU

European Union

EYE

Erasmus for Young Entrepreneurs

FWC

Framework Contract

HE

Host Entrepreneur

ICT

Information and Communication Technology

IPR

Intellectual Property Rights

KETS

Key-Enabling Technologies

LGF

Loan Guarantee Facility

NACE

Statistical Classification of Economic Activities in the European Community

NCP

National Contact Point

PES

Performance Enhancement System

RPAS

Remotely Piloted Aircraft Systems

SBA

Small Business Act

SMEs

Small- and Medium-sized Enterprises

SO

Specific Objective

SPR

SME Performance Review

UEAPME

European Association of Craft, Small- and Medium-sized Enterprises

Introduction

This annex is structured as follows:

·The methodological framework used for the evaluation is presented in Section 1

·We then describe the approach taken and provide information on the results at SO level for the methods used for the evaluation (including the case studies), i.e. the desk research (Section 2), the statistical data analysis (Section 3), the targeted and public consultations (Section 4) and the interviews (Section 5).

There are four appendices to the report, providing additional data (Appendix A and Appendix B), lists of documents analysed (Appendix C) and interviewees (Appendix D), and the targeted (quantitative) and public consultation questionnaires).



1.The methodological framework

In this chapter, we give an overview of our approach taken to the evaluation. Section 1.1 presents the key concepts that guided our evaluation, including the programme’s intervention logic and the evaluation questions.

Section 1.2 summarises the evaluation tools and methods we used, the triangulation matrix we designed and the flow of analysis we followed.

The objective of the study was to evaluate the relevance, effectiveness, coherence, efficiency and EU added value of the programme activities funded in the 2014-2016 period. The study findings should contribute to the decision-making on the renewal, modification or suspension of the programme actions and measures, to set the basis for the next evaluation of the COSME programme until 2020 and contribute to the preparation of the future programme in the next Multiannual Financial Framework (MFF), and to feed into the mid-term evaluation of EASME.

The scope of this evaluation was the COSME activities conducted in the period 2014-2016.

1.1.Methodological approach

The Terms of Reference (ToR) indicated the evaluation criteria of relevance, efficiency, effectiveness, coherence and EU added value as overall framework for the study. These evaluation criteria assess the relationships between the various components of the intervention logic ( Figure 5 ).

Figure 5 Evaluation criteria and intervention logic in an interim/mid-term evaluation

Source: Technopolis

During the inception phase, the scope for the analyses against these evaluation criteria was defined as follows:

·Assessment of the relevance, coherence, and efficiency in implementation criteria covering all actions

·Assessment against the effectiveness criterion focused on the Key Actions, complemented by four case studies

·The European added value and the efficiency / cost-effectiveness criteria remain covered through the analysis of the Key Actions only

The Key Actions included: the financial instruments (LGF and EFG), the Enterprise Europe Network services, the SME Policy actions, the Cluster Internationalisation programme, and the Erasmus for Young Entrepreneurs programme.

Actions covered through case studies are: the Vulcanus scheme in the EU-Japan Centre action; the e-skills/digitisation actions, the Tourism actions (grants), and the Women Entrepreneurship action.

The supporting study was performed on the basis of six leading evaluation questions and fifteen sub-questions for this evaluation.

Table 11 Evaluation questions

Key question

Evaluation question

1) Implementation - State of play

Q1.1) How has the programme been implemented and how has it evolved since the start in 2014?

2) RELEVANCE: Do the objectives correspond to the current needs?

Q2.1) To what extent are COSME’s initial objectives still pertinent to the needs, problems and issues it was designed to address?

Q2.2) How relevant is the programme to the stakeholders (public and private) and the EU citizen?

Q2.3) How has the programme adapted to the political priorities, for instance to the new Single Market Strategy and its particular objective to support Start-ups and Scaling-ups?

3) EFFECTIVENESS: How effective is the programme in achieving its objectives?

Q3.1) To what extent is the COSME programme progressing towards achieving its specific objectives?

Q3.2) What are the factors driving or hindering progress and how do they link with different actions of the programme?

Q3.3) To what extent is the Programme progressing towards contributing to the 5 Union priorities for smart, sustainable and inclusive growth (incl. climate objectives and gender mainstreaming)?

4) EFFICIENCY: To what extent were the effects (benefits) achieved at a reasonable cost?

Q4.1) What are the regulatory/administrative costs and the benefits for the different stakeholders?

Q4.2) Are the costs (direct and indirect) generated by the programme proportionate to the benefits generated?

Q4.3) How efficient is the implementation structure and governance of the programme?

5) COHERENCE: To what extent are the individual actions financed by COSME internally coherent and to what extent is COSME externally coherent with other EU and MS programmes/policies with similar objectives?

Q5.1) Are the different actions of COSME implemented coherently between them?

Q5.2) How are the different COSME actions overlapping/synergizing with the activities of ESI funds, H2020, EFSI, EASI and other EU programmes?

Q5.3) To what extent has the coherence been maximised in view of National programmes?

6) EU ADDED VALUE: What is the EU value added?

Q6.1) What is the added value of the programme and actions being conducted at the EU level?

Q6.2) To what extent and why do the issues addressed by the programme continue to require action at EU level?

Source: Technopolis

Time is needed for effects such as mid-term results and long-term impacts to occur. In the case of interim evaluations such as the current one focusing on the COSME programme, most of the expected mid-term results are not yet be measurable. The focus is therefore on the assessment whether the programme succeeded in setting the conditions that can be expected to lead to these expected results. This implies an emphasis in this evaluation on the adequacy and quality of the programme design, i.e. the formulated objectives, the invested resources, and the activities implemented – in other words, the relevance and coherence criteria.

In terms of effectiveness, the assessment focused on the attainment of the expected outputs and short-term outcomes, and the enabling factors or barriers for the attainment of the outcomes and expected results, such as the capacity of the programme to reach and involve the stakeholders needed.

The intervention logic was a critical component of our methodological framework. The consistent use throughout our analyses of the categories and types of effects that were defined in this intervention logic allowed us to structure our findings and draw conclusions at the programme level, going beyond a mere aggregation of the findings at the SO level. Figure 6 , below, sets out the intervention logic of the COSME programme and its expected effects.

Figure 6 Intervention logic of the COSME programme

Source: Technopolis

1.2.Overview of the evaluation tools and methods

The consultant used a mix of qualitative and quantitative methods for the evidence collection and analysis. We briefly describe our approach for these evaluation tools and methods in Section 1.2.1, below. A full description of each tool and method is provided in the next chapters. In Section 1.2.2 we present the matrix that constituted the framework for the triangulation of the evaluation findings. Section 1.3 sets out the flow of analysis in the evaluation and its workplan.

1.2.1.Brief description of tools and methods

Table 12 , below, presents the tools and methods used in this evaluation.

Table 12 Tools and methods used in the evaluation

Methodological approach

Tools & methods

Number

Qualitative methods

 

 

 

Desk research

260+

Interviews

121

Case studies

4

Workshop

1

Qualitative targeted consultation

7

Quantitative methods

Targeted consultation

7

Public consultation

1

Statistical analyses of secondary data

5

Qualitative methods

The desk research encompassed a broad range of sources, ranging from programme and policy documents to market studies, impact assessments and preceding and/or other evaluations. The list has been in constant expansion throughout the evaluation. The review of the literature and desk research was an activity that took place throughout the entire lifespan of the evaluation. Nevertheless, it was concentrated especially in the first stages of the evaluation, enabling the study team to conduct the interviews with the stakeholders in an informed manner.

A major purpose of the high-level interviews was to collect background and context information on facilitators and barriers for the attainment of the expected effects as well as the eventual unintended effects. We also conducted some interviews at the overall programme level. These interviews with EU officials took place in the beginning of the study to reach an improved understanding of the overall context to the programme and the processes for the programme design.

In this evaluation, we conducted four case studies covering three SOs: SO2 – Access to market (the trainee programme “Vulcanus in Japan”, run by the EU-Japan Centre), SO3 – Favourable environment (the Tourism actions under the programme and the E-skills for competitiveness and innovation actions), and SO4 – Entrepreneurship (the Women Entrepreneurship action). Each case study covered the overview of the programme’s progress in attaining the objectives set out in the intervention logic, and to the extent possible, an appraisal of their progress towards attaining their expected results. Another important focus of the case study was the analysis of the relevance and coherence of the COSME actions. The results fed into the wider analysis of SO2, SO3, and SO4, as well as of the COSME programme overall.

A day-long workshop was organised on May 31, 2017, involving representatives of the Member States and the COSME Third Countries as well as Key Stakeholders. The objective of the workshop was two-fold: presenting the then-current results to the stakeholders as well as receiving feedback from them, especially regarding the relevance, coherence and European added value of the COSME programme.

The first half of the workshop was dedicated to the presentation of the COSME interim evaluation (the objectives and the methodology) and the first findings of the team for each SO, followed by a Q&A session. During the second half, participants split into four groups. Positive and negative aspects of the COSME programme were discussed as well as the ideal design of an SME support programme. A plenary session concluded the workshop where the outcomes of the breakout sessions were discussed.

On request of the EC, this evaluation included the conduct of ‘qualitative’ targeted consultations, replacing interviews. This type of surveys will be implemented where the population is lower than 100; the small numbers therefore did not allow for meaningful statistical analysis. The questionnaires mostly include open-ended questions, geared at collecting qualitative information.

Quantitative methods

The ‘classical’ targeted consultations were implemented in those cases where the population was higher than 100 and would therefore allow for meaningful and robust statistical data analyses. The questionnaires mostly include close-ended questions and only a few open-ended questions. The responses were analysed through descriptive statistics, in several cases also included cross-analyses.

The public consultation was a 12-week internet-based questionnaire. The objective of the public consultation was to cover all potential relevant stakeholders, in particular those that were not targeted through the targeted consultations and interviews, and to collect feedbacks from them on the COSME programme and its objectives. More specifically, we received input from:

·Business and trade associations

·Citizens of the EU

·Non-profit organisations

·Financial intermediaries (private equity, business angels, venture capital funds, banks, promotional banks, guarantee institutions)

·Intermediary structures such as incubators/clusters/innovation & growth agencies

·Public authorities (member states, European agencies)

The questionnaire for the public consultation was quite straightforward as it aimed at reaching all types of stakeholders. Given that the targeted consultations focused exclusively on the specific objectives, it was agreed to use the public consultation to collect data that would provide information on the programme as a whole.

In total four types of analyses were conducted on secondary data:

·The portfolio analysis entailed the analysis of the COSME budget distribution over the different structural components of the programme, ie the Strategic Objectives and their operational (sub)objectives, the policy mix used in the programme (ie the policy instruments) and the modes of implementation. We covered both the planned budget distribution (based on the Work Programmes) and the actual spending.

·The composition analysis focused on the stakeholders that benefit of the COSME programme funding. The outcomes of this analysis informed especially the analysis of effectiveness, in terms of the capacity of the programme to attract the type of stakeholders needed for the attainment of the desired effects and the analysis of cost-effectiveness. Dimensions for analysis were the geographical location of the beneficiaries, the type of stakeholders involved in the actions funded, such as SMEs, professional associations, intermediaries, national/regional administration officials, and consultancies providing services to the EC, and the sector of activity of the SMEs and other private sector stakeholders involved.

·The time-to-grant analysis, based on EASME management data, and the success rate analysis of the open calls for grants, based on the EASME proposal database, were conducted to feed into the assessment of the COSME efficiency in the implementation of the programme.

·The cost-effectiveness analysis (CEA) entailed quantifying the not-monetised benefits that would be generated by one Euro of total costs. Typically, this entails defining the benefit-cost ratio, i.e. dividing the benefits by costs. CEA is normally used to identify the “value for money” of programmes. The CEA focused on SO actions that have similar objectives, rather than on the SOs in general. In addition, for the CEA to provide a meaningful result, a measure to identify ‘good performance’ was needed; typically, this is obtained through comparison. Our approach was to assess the cost-effectiveness of the SO Actions in such a way that the outcomes for the different SOs can be compared to each other. This implied a focus of the ‘effectiveness’ factor on the only comparable outcome indicator possible, the number of SMEs and individuals supported.

1.2.2.Matrix for the triangulation

For each evaluation question, the analysis was based on multiple sources and methods ( Table 13 ). This allowed for a proper triangulation of the evidence collected, ensuring the robustness of the findings. It also helped mitigating the risk of lack in data availability.

Table 13 Use of the methodological mix per evaluation question

Qualitative

Quantitative 

Desk research

Interviews (including the “qualitative” surveys)

Case studies

Workshops

Secondary data analysis / Composition & portfolio analysis

Other secondary data analysis

Cost-effectiveness analysis

Statistical analysis targeted consultation data (“quantitative” survey)

Public consultation

1) Implementation - State of play 

Q1.1) How has the programme been implemented and how has it evolved since the start in 2014?

X

X

X

X

2) RELEVANCE: Do the objectives correspond to the current needs? 

Q2.1) To what extent are COSME’s initial objectives still pertinent …

X

X

X

X

X

X

Q2.2) How relevant is the programme to the stakeholders (public and private) and the EU citizen?

X

X

X

X

X

Q2.3) How has the programme adapted to the political priorities …

X

X

X

3) EFFECTIVENESS: How effective is the programme in achieving its objectives? 

Q3.1) To what extent is the COSME programme progressing towards achieving its specific objectives?

X

X

X

X

X

Q3.2) What are the factors driving or hindering progress and how do they link with different actions of the programme?

X

X

X

X

X

X

Q3.3) To what extent is the Programme progressing towards contributing to the 5 Union priorities ….

X

X

X

X

X

4) EFFICIENCY: To what extent were the effects (benefits) achieved at a reasonable cost?

Q4.1) What are the regulatory/administrative costs and the benefits for the different stakeholders?

X

X

X

X

Q4.2) Are the costs (direct and indirect) generated by the programme proportionate to the benefits generated?

X

X

X

X

X

Q4.3) How efficient is the implementation structure and governance of the programme?

X

X

X

X

X

X

5) COHERENCE: To what extent are the individual actions financed by COSME internally coherent and to what extent is COSME externally coherent with other EU and MS programmes/policies with similar objectives?

Q5.1) Are the different actions of COSME implemented coherently between them?

X

X

X

X

X

Q5.2) How are the different COSME actions overlapping/synergizing with the activities of ESI funds, H2020, EFSI, EASI and other EU programmes?

X

X

X

X

X

X

Q5.3) To what extent has the coherence been maximised in view of National programmes?

X

X

X

X

X

X

6) EU ADDED VALUE: What is the EU value added?

Q6.1) What is the added value of the programme and actions being conducted at the EU level?

X

X

X

X

X

Q6.2) To what extent and why do the issues addressed by the programme continue to require action at EU level?

X

X

X

X

X

Source: Technopolis

1.3.Robustness of the results

A key method to ensure robustness of the analysis in an evaluation is the triangulation of the findings, i.e. the validation of information collected through one method by comparing it with the information collected through another method. 

In this section, we discuss the measures of triangulation and the findings of the robustness checks related to the consultations that were implemented in this study.

·Triangulation is especially important for the validation of findings that derive from the statistical analysis of survey responses, due to the potential for statistical bias (see Section 2.3.1, below).

·However, robustness checks and quality assurance measures should be applied also for the other consultation methods (see Section 2.3.2 and Section 2.3.3, below).

More detailed information is provided also in the sections for the individual actions in the sections below.

1.3.1.The quantitative targeted consultations

It is well known that surveys may suffer from a series of statistical biases that make the treatment of responses quite delicate for statistical purposes. These include:

·Sampling bias – the distortion arising from the non-randomness of the sample extraction, or the patent under-representation of relevant geographic areas, sectors, etc.

·Non-response bias – the distortion arising from the possibility that the non-respondents share a characteristic that is important but will be missed in the results

·Response bias – the answer distortion arising from the way the questionnaire is structured, or the perceived expectations of the sponsor/surveyor, or laziness (primacy, extreme responding, fatigue, etc.)

Some of these standard statistical problems (sampling bias, data errors) were present also in the surveys conducted in this study.

However, a part of the more than satisfactory response rates reached, the profile of the responding organisations was in close to all surveys very similar to the target sample, in general guaranteeing statistical representativeness

A first exception to the rule was the survey of the Network client SMEs. This survey was managed through a cascade approach and reached less responses than expected. In addition, because of the cascade approach and therefore the unknown size of the population, the study team could not check the representativeness of the responses. In the main report for this study (section 2.3.4), we emphasise the negative consequences of this approach for the statistical value of survey data in the context of evaluations and endorse the recommendation made by the CIP and EIP final evaluation teams to ensure the possibility for evaluation teams directly to contact beneficiary SMEs.

A second exception is the survey related to the Clusters Go International programme where more funded than non-funded ESCPs responded to the survey, compared to the target sample. In this case, however, this does not imply forcefully a lack of representativeness for the funded ESCPs.

Surveys to stakeholders involved in the programme are often considered to be unreliable sources for information due to the expected bias in the responses by beneficiaries in the programme. We took this into account by analysing the data in terms of distribution of the responses in the different groupings along a 5-point Likert scale and by considering especially the ‘extremes’ in these distributions.

Most important, input provided through the surveys was always corroborated through the other methods used

1.3.2.The qualitative targeted consultations

The qualitative consultations gave different results. While three out of the four surveys related to SO1 reached a reasonable level of response rate, the same can be said only for one out of four surveys related to SO3.

The qualitative surveys for which the number of returned questionnaires was below expectation are the ones targeting the EFG beneficiary SMEs, and the surveys related to SME policy actions (one out of two), the sectoral competitiveness actions, and the e-skills actions. A factor playing a role for the EFG beneficiaries was the cascade approach used; for the sectoral competitiveness one, the wide variety of sectors to be covered was a hindering factor, as well as the related difficulty to identify the relevant stakeholders to include, even with the support from the EC.

Overall, the quality of the responses was also often below the expected level; the full exploitation of this evaluation tool was hindered by the impossibility for the evaluation team to set up a direct follow-up interaction with the respondents.

These consultations did not have the intention to reach ‘statistical representativeness’, so there is no reason to doubt the value of the input provided. However, the efficiency of this type of exercise is highly questionable.

1.3.3.The public consultation

Public consultations present limits for use in the context of an evaluation due to the impossibility of controlling and checking the ‘statistical representativeness’ of the information provided. In the case of this evaluation, for example, the profile of the respondents clearly indicates a strong representation of stakeholders involved in the financial instruments and respondents involved in a mix of other COSME actions. A use of these data in a statistical manner would therefore have been inappropriate. Instead, we considered the responses as well as the opinion papers submitted as illustrations of a viewpoint, and used the data exclusively to corroborate findings deriving from various other data collection tools used.

1.4.Flow of the analysis 

The evaluation was implemented in three phases: the inception, data collection and analysis, and reporting phases, whereby the latter two phases took place in two steps each ( Figure 7 ).

In the data collection and analysis phase we focused our activities on setting out the ‘state-of-play’ of the COSME implementation and the conduct of the analyses (desk research and interviews) focusing on the relevance and coherence criteria. This allowed us to report on our preliminary findings for these criteria in the Interim report (D3). In the interim report we also reported on the progress in the targeted and public consultations.

These preliminary findings were further validated in the next step, in particular during the workshop with the COSME committee members. The collection of evidence through the other methods and its triangulation led to the formulation of findings at the thematic area level. These findings were reported internally, allowing for an aggregated analysis and the drawing of conclusions and drafting of recommendations at the COSME programme level, which were then reported in the draft and final version of the Final report.

Figure 7 Flow of the analysis

2.Desk research

In this chapter we present the desk research strategy followed for the evaluation. We present an overview of this strategy in Section 2.1 while we detail it at thematic level in Section 2.2.

2.1.Overview

In Appendix B we provide the full list of all documents consulted per SO.

Table 14 , below, lists the documents and studies that were of relevance at the overall programme level.

Table 14 Literature list at the programme level

Type of document

Examples of relevant documents

COSME programme documentation

EU (2013), Regulation (EU) No 1287/2013 of the European Parliament and of the Council establishing a Programme for the Competitiveness of Enterprises and small and medium-sized enterprises (COSME) (2014 - 2020) and repealing Decision No 1639/2006/EC.

Ex-ante Preparatory documents for COSME, including public consultation: Final report, annex 1, 1a and 2;

Impact Assessment for the proposal for a COSME regulation:

Impact Assessment for the proposal for a COSME regulation COM (2011)834;

Impact Assessment for the proposal for a COSME regulation SEC(2011) 1452;

Impact Assessment for the proposal for a COSME regulation SEC(2011) 1453;

COSME work programmes: 2014, 2015 and 2016;

M&E available at the programme level

COSME monitoring reports 2014 & 2015 (2016 under development during the study)

Documents and sources for programmes prior to COSME – CIP:

Impact assessment (2005); Interim evaluation (2010); Final evaluation (2011);

CIP performance review 2007-2012;

Implementation, evaluation, performance and beneficiaries reports for each year of 2007 – 2012: http://ec.europa.eu/cip/documents/implementation-reports/index_en.htm

Decision 1639/2006/EC of the European Parliament and of the Council of 24 October 2006 establishing a Competitiveness and Innovation Framework Programme (2007-2013);

EU Policy documents

EC (2014), For a European Industrial Renaissance, COM/2014/014 final 

EC (2012), A Stronger European Industry for Growth and Economic Recovery Industrial Policy Communication Update, COM/2012/0582 final

EC (2010), An Integrated Industrial Policy for the Globalisation Era Putting Competitiveness and Sustainability at Centre Stage, COM/2010/0614 final

EC (2008, 2011), Small Business Act

EC (2015), Single Market Strategy

EC (2016), Europe's next leaders: the Start-up and Scale-up Initiative,  COM/2016/0733 final

EC (2012), Entrepreneurship 2020 Action Plan

EC (2011)  Industrial Policy: Reinforcing competitiveness

EU Policy Studies

Pellegrin, J. et al (2015), EU Industrial Policy: Assessment of Recent Developments and Recommendations for Future Policies, CSIL, CASE, University of Milan, Economisti Associati, a study for: European Parliament, Directorate General for Internal Policies, Policy Department A: Economic and Scientific Policy

Dhéret, C., Morosi, M. (2014)  Towards a New Industrial Policy for Europe, EPC Issue Paper No.78 

Sylvest, J. et al. (2014) How can European industry contribute to growth and foster European competitiveness?, Danish Technological Institute, A study for: European Parliament, Directorate General for Internal Policies, Policy Department A: Economic and Scientific Policy

Source: Technopolis

A concrete output of our desk research was the overview of the achievements of the KPIs that are listed in Appendix A. This analysis was performed in the context of our assessment of the efficiency of the COSME programme, specifically related to the monitoring activities.

The table below gives an overview of the types of documents used for the assessment of COSME against the evaluation questions and topics.

Table 15 Evidence table for the desk research and literature review

 

 

SO1

SO2

SO3

SO4

Nr. question

Evaluation topics

Programme documents

Studies / secondary data

Other studies

Programme documents

Studies / secondary data

Other studies

Programme documents

Studies / secondary data

Other studies

Programme documents

Studies / secondary data

Other studies

 IMPLEMENTATION STATE OF PLAY

 

Rationale of the programme and its actions

X

X

X

X

X

X

X

X

X

X

Objective hierarchy

X

X

X

X

Link with actions funded under CIP

X

X

X

X

X

X

X

X

Budget allocation over objectives

X

X

X

X

Budget allocation over types of actions

X

X

X

X

Budget allocation over types of calls and instruments

X

X

X

X

Budget allocation over types of stakeholders

X

X

X

X

RELEVANCE

Q2.1)

Ongoing validity of the rationale

X

X

X

X

X

X

Link developments/changes in budget allocations

X

X

X

X

Q2.2)

Alignment needs/actions/stakeholders

X

X

X

X

X

Deficiencies in alignment needs/actions

X

X

X

X

The decision-making process for the actions undertaken

X

X

X

X

The societal relevance of the programme

X

X

X

X

X

X

Q2.3)

The policy relevance of the programme

X

X

X

X

EFFECTIVENESS

Q3.1)

Outputs so far

X

X

X

X

X

X

Results so far

X

X

X

x

X

Alignment with expectations

X

X

X

X

X

Q3.2)

Appropriate stakeholders targeted

X

X

X

X

Appropriate stakeholders reached

X

X

X

Appropriate policy mix

X

X

X

X

Appropriate budget allocation

X

X

Q3.3)

Progress towards policy objectives

X

X

X

EFFICIENCY

Q4.1)

Administrative costs / stakeholders

X

X

X

Affordability / burden

Simplification measures

X

X

X

X

Q4.2)

Administrative costs / implementation structure

X

X

X

Cost-effectiveness

X

Comparison across actions

X

Q4.3)

Time-to-grant / / use of funding instruments / distribution of funding

X

X

X

X

X

X

Implementation / delegation

X

X

X

X

Monitoring system

X

X

X

X

X

X

X

COHERENCE

Q5.1)

Internal coherence - SO

X

X

X

X

Internal coherence - communication instruments

X

X

X

X

X

Internal coherence - programme

X

X

X

X

Internal coherence - coordination process

X

X

X

Q5.2)

External coherence - EU

X

X

X

X

X

X

External coherence - EU process

Q5.3)

External coherence - MS

X

X

X

X

X

X

External coherence - MS process

X

X

EU ADDED VALUE

Q6.1)

Importance of action at EU level

X

X

X

X

X

X

Additionality

X

X

X

Q6.2)

Ongoing need for action at EU level

X

X

Source: Technopolis

2.2.Details at the Thematic Areas level

2.2.1.SO1 – Access to finance

Desk research and secondary data analysis were conducted based on 60 documents identified during the inception phase and data collection phase.

Overall the exercise of gathering the documentation was challenging as EU officials did not have at end a knowledge database concerning the “Access to Finance” actions. This resulted in collecting documents throughout the overall study and up to September 2017. All available documentation was carefully processed and related to other findings.

The collection of secondary data is primarily important for the portfolio analysis as well for assessment of the effectiveness and the efficiency of the actions. Secondary data on the LGF and EFG were obtained from EIF notably Quarterly reports from 2014 to end of 2016, Annual reports 2014 to 2016, and Growth & Employment report 2015.

DG GROW shared a selection of other monitoring data of both instruments:

·Extraction of DG GROW commitments to both EFG and LGF

·Extraction of remuneration and fees paid to EIF for both instruments

The list of documents analysed is presented in Appendix A.1

2.2.2.SO2 – Access to market

One of first steps in the evaluation was the collection of the relevant documentation for the desk research on the actions under SO2. We contacted the relevant EC officers with a documentation request related to the following actions:

·Enterprise Europe Network

·Your Europe Business portal

·Improved Points of Single Contact

·Facilitating Access to Light Remotely Piloted Aircraft Systems (RPAS) Regulation

·Exchanges of Good Practices in the Area of Compliance Assistance and Schemes

·Improving SMEs’ Access to Public Procurement

·SME Internationalisation Portal

·EU-Japan Centre

·China, Latin America and South-East Asia IPR SME Helpdesks

·Support to SME Internationalisation

·Industrial Policy Cooperation

Most secondary data were identified in a meeting with Unit H2 on 9 February 2017 and have reached us via DG GROW. Additional secondary data were sent by EASME.

Overall the exercise of gathering the documentation was challenging, as many EU officials were very busy, especially within the holiday period that overlapped with the short time period between the inception report and the interim report. Furthermore, although many actions come from the Work Packages of 2014 and 2015, not all actions have been implemented to the extent that was initially expected. By the end of the evaluation, while information was made available on all the actions, for some actions the information remained limited. All available documentation was carefully processed and related to other findings.

The collection of secondary data is primarily important for the portfolio analysis as well for assessment of the effectiveness and the efficiency of the actions. Secondary data on the Enterprise Europe Network were obtained from EASME. for;

The largest source of collected information is the Performance Enhancement System (PES) database, which relies on regular reporting done by the Network members. The aim of the PES is to measure the achievements of the Network members, and to monitor the overall Network performance. It includes an Inception Report (6 months after the start of the Consortium), quantitative reporting on the 16 key variables of the PES (12 months, as well as 24 months after the start of the Consortium), self-assessments by the Consortium, as well as some other reflective elements. 86

It often takes quite some time to finalise all the data for the PES, therefore the time available to prepare the data within the evaluation was short. A selection of the preliminary PES data that were available in time to be included in the final report was provided to the evaluation team. EASME also shared a selection of other monitoring data of the Network, namely:

·Extraction of Achievements, providing insight into the number of Partnership Agreements and Advisory Service Outcomes realised per consortium

·Extraction of the Partnership Opportunity Database, providing insight into the number of partnering opportunities that were published per consortium, as well as the number of expressions of interest that those profiles generated

·Extraction of the new Impact Survey results. This is a new survey and therefore the results are still incomplete. It does provide insight into the way impact data are currently collected and what kinds of data are produced.

The list of documents analysed is presented in Appendix C.2

2.2.3. SO3 – Favourable environment

Desk research has been completed for all actions, and has aimed to provide an overview of the economic developments and policy issues within the different thematic areas under SO3: SME Policy, regulatory and administrative burden, innovation performance in SMEs, digitisation and KETs, as well as sectoral competitiveness.

The secondary data we used came from the COSME Work Programmes 2014-2015 and the COSME Activity Reports 2014 and 2015 for all actions in focus. Where available, Calls for Proposals / Projects for the various streams of actions (especially for the cluster internationalisation actions, tourism, and e-skills actions), and technical progress reports or final implementation reports were consulted. Moreover, for SME Policy, the SME Performance Review publications, documents published by SME Envoys and the description of the mission of the SME Envoys were used.

Beyond the COSME work programme and activity reports, the level of access, utility and comprehensiveness of further secondary data varies per field. In some cases, it has been difficult to obtain a comprehensive overview of the status of the implementation and interim results achieved, especially for the measures where the effectiveness dimension was in focus. In particular, synthesis reports of the projects funded under the Cluster Internationalisation call were only partially available, as the data collection for the first-year performance review had not been completed by the time of the evaluation, as several cluster partnerships had yet to submit their second progress reports. In the case of the tourism actions, no synthesis reports of the individual project monitoring and activity reports were available and, due to their non-standardised format, individual progress reports could not be used to provide a more comprehensive overview of the progress of actions and outputs in the Tourism sector. To counterbalance the lack of data in these cases, the results of the survey with beneficiaries of these two actions were used to illustrate the level of progress achieved in a more aggregated manner.

The list of documents analysed is presented in Appendix C.3

2.2.4.SO4 – Entrepreneurship

Desk research activities have been ongoing throughout the evaluation, but were a particular focus of the first stages of the study, to inform consultation activities and early analysis. The evaluation team collected and analysed secondary sources in relation to all the entrepreneurship actions in scope. Information was sought from the European Commission and EASME to ensure that both technical and policy-relevant information was considered. In addition, online searches and interviews were used to complete the review.

Additional information was obtained in relation to the key entrepreneurship action (EYE). This included information from the EYE programme database (containing information on all intermediaries and participants involved in EYE placements), past evaluations (the interim evaluation of the EYE pilot and the EYE statistical analysis), a selection of recent six-monthly monitoring reports produced by the EYE support office (Eurochambres) and the EYE programme guide and quality manual.

Information on the individual actions was captured in a structured format to facilitate analysis along the main evaluation questions. These secondary sources are drawn upon and referenced throughout the report, but particularly when analysing the context, relevance and coherence of the entrepreneurship actions.

The list of documents analysed is presented in Appendix C.4


3.Statistical data analysis 

In this chapter we begin with an overview of our sources of information (Section 3.1). Then we present the portfolio analysis (Section 3.2), the composition analysis (Section 3.3), the programme management data (Section 3.4) and the cost-effectiveness analysis (Section 3.5).

3.1.Sources

Ten structured databases have been identified for the programme and the different SOs, based on input provided by the officials in charge of the databases at DG GROW, EASME and EIF ( Table 16 , below). Contacts with these officers and the EC officers acting as the SO contact persons continued throughout the study for the retrieval of some more detailed information and the completion of this list.

These datasets provided input for the composition and portfolio analysis depicting the programme’s ‘state-of-play’ as well as for the other analyses, especially those related to the relevance, effectiveness and efficiency questions. Some may serve also for the targeted consultations.

Table 16 Preliminary list of the available relevant databases

Database

Name

SO

Level

Contact

1

Data hub for COSME

SO2, 3, 4

Projects

EASME

2

Data from Annual operational reports

SO1

Aggregated + financial intermediaries

DG GROW

3

Data from the Growth and employment report 2015 (LGF)

SO1

Aggregated

DG GROW

4

Data from Quarterly reports

SO1

Aggregated + financial intermediaries

DG GROW

5

Data from EIF

SO1

Intermediaries and recipients

EIF

6

Achievement database

SO2

Partnership agreements and Advisory service outcomes

EASME

7

Performance Enhancement System

SO2

Consortia and network partners

EASME

8

Various data for EEN

SO2

/

EASME

9

EYE database

SO4

HE, NE, EP, IO

DG GROW

10

Statistics

SO1, 2, 3, 4

Countries

Publicly available

Source: Technopolis

3.2.Portfolio analysis

For the portfolio analysis, both the data on planned and committed budgets were collected. The budgetary distribution was calculated across policy instruments, implementation modes, and thematic areas (and corresponding action lines) and combinations thereof.

The outcomes of this analysis especially informed:

·the analysis of the relevance of the programme, in terms of the budget allocated for the attainment of specific objectives and sub-objectives as an illustration of importance attributed

·the analysis of effectiveness, in terms of the level of the specific effects that can be expected

·the analysis of implementation efficiency, in terms of the extent to which the COSME programme allocated sufficient levels of budget to specific actions and the projects within these actions, and the eventual fragmentation of the budget over multiple actions with similar objectives

Table 17 Portfolio analysis levels

Policy instruments

Implementation modes

Thematic areas

Financial instrument

Information instruments

Knowledge & networking instruments

Ad hoc grant Art. 190 RAP

Administrative arrangement

Calls for tender (service contracts)

Direct contracts

FWC (specific contracts)

Grants

Indirect Management

Membership fees

Not defined

Reimbursement experts / members

Remuneration experts / evaluators

Specific grant agreement under Framework

Partnership Agreement

Business mgt capacity

Digitisation & KETs

Entrepreneurship

Information campaign

Internationalisation

Regulations & administrative burden

Responsible innovation

Sectoral competitiveness

SME policy

Support to access finance

Source: for financial instruments: EIF quarterly report Q4/2016; for EEN and EYE: databases provided by COSME

3.3.Composition analysis

The composition analysis, i.e. the profiling of the stakeholders involved in COSME, has been conducted; however, the predominant use of agreements for the allocation of the COSME budget, in particular their delegation agreements (indirect management) or Framework Partnership Agreements, constituted a challenge for the identification and analysis of the final beneficiaries’ profile.

This analysis was therefore conditioned by the integration of the COSME data hub with the databases related to the specific actions (financial instruments, EEN, EYE, etc.) as well as, most importantly, by the availability of detailed information on the stakeholders involved, covering the intended dimensions for analysis. Particularly in the case of the type of stakeholders involved the original databases were insufficient for composition analysis due to the lack of a predetermined typology of stakeholders. To amend this, database entries were manually checked in order to attribute stakeholders to the typology designed for this evaluation (i.e. industry, industry association, association, public administration/agency, research/RTO).

Similarly, the sectoral representation in the original databases was insufficient for the composition analysis due to the absence of a systematic sectoral identifier, such as the NACE rev.2 classification, which was only available for SO1. To amend this, a concordance with NACE rev.2 codes was developed. It should be noted, however, that the concordance provides only for a rough approximation, given that some sectors do not correspond to a NACE rev.2 classification (e.g. Organic/Bio/Eco products and services, etc.) or some of the entries were too broad and required a matching with multiple NACE rev.2 levels (e.g. Mining, machinery for mining, basic metals and related products: B + C28.92 + C24).

This analysis was therefore conditioned by the integration of the COSME data hub with the databases related to the specific actions (financial instruments, EEN, EYE, etc.) as well as, most importantly, by the availability of detailed information on the stakeholders involved, covering the intended dimensions for analysis. Dimensions for analysis were the geographical locations of the beneficiaries, as well as the types of stakeholders involved in the actions funded and their sectoral representation.

This analysis encountered difficulties in terms of data availability related to the types of stakeholders and sectors of activity. No data was available in relation to gender, except for the EYE participants.

The information available by Action Line is described in Table 18 and Table 19 , below.

Table 18 Overview of available data on beneficiaries

SO

Location

Type of stakeholder

Sector

Budget

Gender

Financial instruments

Yes

Yes

Yes

Yes

No

EEN services

Yes

No

Yes

Not applicable: EEN network member organisations receive funding to provide services to final beneficiaries

No

Cluster Internationalisation/ Tourism (grants)

Yes for type of call=calls for proposals

Yes

Yes for type of call=calls for proposals

Yes for type of call=calls for proposals

No

EYE

Yes

Yes

Yes for Host entrepreneurs. It is not applicable for New entrepreneurs.

Yes for New Entrepreneurs. This is a fixed monthly amount adjusted by country. Host Entrepreneurs do not receive funding.

Yes

Source: for financial instruments: EIF quarterly report Q4/2016; for Cluster Internationalisation: European Cluster Collaboration Platform; for EEN and EYE: databases provided by COSME; for Tourism: COSME data-hub.

Table 19 Overview of available data on intermediaries

SO

Location (country)

Type of stakeholder

Budget

Gender

Financial instruments

Yes

Yes

Yes

No

EEN services

Yes

Yes

Yes

No

EYE

Yes

Yes

No

No

Source: for financial instruments: EIF quarterly report Q4/2016; for EEN and EYE: databases provided by COSME.

Additional analyses on geographical distribution and sectoral involvement are available in Appendix B.

3.4.Analysis of COSME programme management and proposal data

The time to grant is measured as the number of months between the deadline for the submission of proposals and the signature of grants for successful proposals. The empirical scope of the time-to-grant analyses covers the calls listed in Table 20 .

Table 20 Calls on which time to grant analysis is based

Business mgt capacity

EEN Administration & services

COS-Adhoc-2014-2020

COS-EEN-2014-2-01 FPA

COS-EEN-2014-2-04

COS-EEN-SGA-16-B-07-2016-1 SCALE UP CALL

COS-EEN-SGA2-2016-2-01

EEN-SGA-COSME-2015-01-1

Entrepreneurship

Entrepreneurship education

COS-ENTEDU-2014-4-06

EYE

COS-EYE-2014-4-05

COS-EYE-2015-4-01

COS-EYE-FPA-2016-4-01

COS-EYE-SGA-2016-4-01

Migrants

COS-MigrantsENT-2016-4-02

Second starters

COS-EarlyWarningEU-2016-4-01

Internationalisation

Cluster internationalisation

COS-CLUSTER-2014-3-03

IPR Helpdesks

COS-IPR-2014-2-05

SME Internationalisation

COS-ETPO-2015-2-01

Regulations & administrative burden

Access to public procurement

COS-APP-2016-2-05

Access to regulations

COS-RPAS-2014-2-03

Privacy

COS-DRONES-2016-03-02

Single points of contact

COS-SPOC-2016-02-03

Sectoral competitiveness

Clusters

COS-CLUSTER-2014-3-04-02-1

COS-CLUSTER-2015-3-02

Design-based consumer goods

COS-DESIGN-2015-3-03-1

COS-DESIGN-2015-3-06

Retail

COS-IPPDS-2015-2-02-1

Tourism

COS-TEDEN-2014-3-15

COS-TEDEN-2015-3-05

COS-TFLOWS-2014-3-15

COS-TOUR-2015-3-04-1

COS-TSUST-2014-3-15

COS-TSYNER-2014-3-15

SME policy

SBA implementation

COS-DCFB-2015-3-01-1

Source: Technopolis elaboration based on raw data provided by COSME

The table below presents a complete and detailed time-to-grant analysis for each action line (compared to the tables in the core body of the report, where only the main action lines are listed). For transparency purposes, the table includes also the action lines for which data are not available, in order to provide a complete overview of the data provided.



Table 21 Time to grant by thematic area and action line

Themes

Action lines

Average Time to grant in days

Average Time to grant in months (assumption month=30 days)

Min Time to grant in months (assumption month=30 days)

Max Time to grant in months (assumption month=30 days)

Business mgt capacity

EEN Administration & services

234.8

7.8

3.9

26.8

Business mgt capacity overall average

234.8

7.8

3.9

26.8

Entrepreneurship

Entrepreneurship education

209

7.0

7.0

7.0

EYE

218.6

7.3

3.8

10.0

Migrants

Second starters

178

5.9

5.9

5.9

Entrepreneurship overall average

217.6

7.3

3.8

10.0

Internationalisation

Cluster internationalisation

222.6

7.4

6.7

8.0

IPR Helpdesks

190.7

6.4

2.7

10.0

SME Internationalisation

114.5

3.8

3.7

4.0

Internationalisation overall average

205.3

6.8

2.7

10.0

Regulations & administrative burden

Access to public procurement

277.5

9.3

8.3

10.5

Access to regulations

226

7.5

7.5

7.5

Privacy

Single points of contact

147

4.9

4.9

4.9

Regulations & administrative burden overall average

247.2

8.2

4.9

10.5

Sectoral competitiveness

Clusters

181.5

6.0

5.4

7.0

Design-based consumer goods

235.4

7.8

7.6

8.3

Retail

174

5.8

5.8

5.8

Tourism

250

8.3

5.6

14.1

Sectoral competitiveness overall average

239.9

8.0

5.4

14.1

SME policy

SBA implementation

215.4

7.2

6.1

9.1

SME policy overall average

215.4

7.2

6.1

9.1

Grand Total (overall average for all thematic areas)

232.8

7.8

2.7

26.8

Source: Technopolis elaboration based on raw data provided by COSME

3.5.Cost-Effectiveness analysis

Our indicators reflect the number of outputs and outcomes produced by the (million) Euros invested (e.g. “how many SMEs have been effectively supported by Euro spent?”). This approach provides metrics on efficiency that can be compared across actions.

To conduct the analysis, we have brought together the costs and benefits reported for each action. The analysis focus on four main actions: LGF (Access to finance), EEN (Business Management Capacity), Cluster Go International (Internationalisation), and EYE (Entrepreneurship). Together these four actions account for 62% of the budget committed to COSME in 2014-2016 (EUR 547 million out of EUR 879 million).

In terms of costs, we have considered the (i) direct cost to the programme (i.e. budget committed to COSME in 2014-2016 and administrative costs paid to EASME and EIF), and (ii) the indirect cost to beneficiaries to account for the cost to the ‘system’. In the case of EEN and Cluster Go International this includes the co-funding from partner / member organisations; in the case of EYE this includes additional costs incurred by the host and new entrepreneurs; and in the case of LGF this includes the additional guarantee provided under EFSI. Two measures of CEA indicators have been calculated based on those costs (with and without indirect costs). Only the indicators that include indirect costs are presented in the Main Report.

In terms of benefits, we have arrived at grossed-up estimates of benefits considering information from existing monitoring data (such as the population of beneficiaries) and the results obtained from our surveys.

As such, the analysis does not pretend to be exhaustive. In terms of cost, the analysis focuses mainly on the costs and investments sustained by the EC and participant companies / organisations. The analysis does not consider the potential costs to wider society. Similarly, the analysis of benefits focuses on the impact on participant organisations and does not include the analysis of externalities (i.e. the potential benefits on organisations and on communities not directly involved in any of the specific programmes).

We have mainly used four sources of information in our analysis:

·Information on budget committed (and additional EFSI guarantee used in the case of LGF)

·Information on administrative costs paid to EASME (proportionally distributed across actions to equate to 6.6% of budget committed) and EIF

·Monitoring data from the different actions, which have provided figures on total participations and have been used to arrive at grossed-up estimates.

·Our beneficiary surveys (including, the LGF SME survey, EEN SME client survey, Cluster organisation survey, HE surveys and NE surveys), which have provided key parameters for our estimations, including indirect costs

We have reached estimates for a total of 22 indicators. The section below provides full overview of the costs and benefits mapped for each of those actions. In the case of LGF we have estimated 8 additional indicators to account for the fact that the current 2014-2016 budget envelope (from COSME and EFSI guarantee) will continue to deliver results as the financial intermediaries have 2 to 3 years to build up loan portfolios under the guarantee agreements signed under 2014-2016 budget envelope. So far, a total of 143,000 SMEs has been supported, and it is expected that a total of 300,000 SMEs will be supported under guarantee agreements signed with financial intermediaries within the same budget envelope.

Table 22 Cost-effectiveness analysis

Note that any discrepancies between the parameters and the final figures are due to rounding.

CEA Indicator

Costs

Benefits

CEA

Indicator I

CEA

Indicator II

Description

Direct (Budget committed)

Value (mEUR ) [+ administrative fees]

Indirect

Value (mEUR ) (Cost to the system)

Value / unit

Value /unit per EUR million invested

Value /unit per EUR million invested

[A]

[B]

[C]

[C]/ [A]

[C] / ([A] + [B])

Support to access finance – LGF (achieved so far)

[1] Number of SMEs supported through COSME financing, per EUR million invested

COSME Budget committed:

349.2 mEUR

[+26.3 mEUR ]

COSME Budget committed:

349.2 mEUR

[+26.3 mEUR ]

EFSI guarantee:

262.4 mEUR

No data on the value of costs for SMEs has been collected however evidence suggests that additional costs to SMEs are small

EFSI guarantee:

262.4 mEUR

No data on the value of costs for SMEs has been collected however evidence suggests that additional costs to SMEs are small

143,344 SMEs

SMEs supported so far as reported in monitoring data

381.7

224.7

[17] Number of SMEs supported through COSME financing (that would not have had access to other sources of funding), per EUR million invested

90,307 SMEs

143,344 SMEs multiplied by the percentage of SMEs that stated that the financing supported by a EU-COSME guarantee was the only option available or that other options were available but did not cover the full amount (63% according to the LGF SME survey)

240.5

141.6

[18] Value of credits leverage, per EUR million invested

5,547.2 mEUR

Value of Principal Amount as reported in monitoring data

14.8

8.7

[19] Value of private funds leverage, per EUR million invested

1,247.6 mEUR

Grossed-up figures based value of private funds leveraged, as a percentage of Principal Amount (as reported by survey respondents) multiplied by the value of the Principal Amount, considering size of the company.

3.3

2.0

[5] Increased turnover by companies that had access to LGF (in EUR ), per million EUR invested

10,653 mEUR

Average value of turnover at the time of first inclusion (362 kEUR ), multiplied by the average growth on turnover (20. 5%) as reported in the “COSME Loan Guarantee Facility – Employment and Growth report, 2015”. This figure is then grossed-up by multiplying the average value of the increased employment with the total number of SMEs (143,344)

28.4

16.7

[11] Increased employment by companies that had access to LGF, per million EUR invested

237,431 people

Average number of employees at the time of first inclusion (3.9), multiplied by the average growth on employment (42.7%) as reported in the “COSME Loan Guarantee Facility – Employment and Growth report, 2015”. This figure is then grossed-up by multiplying the average value of the increased employment with the total number of SMEs (143,344)

632.3

372.2

[8] (Additional) Increased turnover by companies that had access to LGF (in EUR ), per million EUR invested

6,711.3 mEUR

Grossed-up value of increased turnover (as shown in indicator [5]) multiplied by % of SMEs that stated that ‘the financing supported by a EU-COSME guarantee was the only option available or that other options were available but did not cover the full amount’ (63% according to the LGF SME survey)

17.9

10.5

[14] (Additional) Increased employment by companies that had access to LGF, per million EUR invested

149,581 people

Grossed-up value of increased employment (as shown in indicator [11]) multiplied by % of SMEs that stated that the financing supported by a EU-COSME guarantee was the only option available or that other options were available but did not cover the full amount (63% according to the LGF SME survey)

398.3

234.5

Support to access finance – LGF (expected)

[1b] Number of SMEs supported through COSME financing, per EUR million invested

COSME Budget committed:

349.2 mEUR

[+26.3 mEUR ]

EFSI guarantee:

262.4 mEUR

No data on the value of costs for SMEs has been collected however evidence suggests that additional costs to SMEs are small

300,000 SMEs expected to be supported

As estimated by the European Commission. All the estimates below use a rule of three to recalculate the estimates based on 143,000 SMEs. In practice, all the above indicators are multiplied by a factor of 2.09.

798.9

470.3

[17b] Number of SMEs supported through COSME financing (that would not have had access to other sources of funding), per EUR million invested

189,000 SMEs

300,000 SMEs multiplied by % of SMEs that stated that the financing supported by a EU-COSME guarantee was the only option available or that other options were available but did not cover the full amount (63% according to the LGF SME survey)

503.3

296.3

[18b] Value of credits leverage, per EUR million invested

18,904.7 mEUR

Based on EIF Q4 2016 operational report. A simple ‘rule of three’ gives a value of 11bnEUR , but we use the 18bn figure as it considers the real implementation conditions under each Guarantee Agreement with the respective Financial Intermediaries and (ii) it is based also on a previous experience gained in the implementation of similar debt financial instruments.

50.3

29.6

[19b] Value of private funds leverage, per EUR million invested

2,611.0 mEUR

Grossed-up figures based on the average value of private funds leveraged by survey respondents and number of SMEs expected to be supported (300,000), considering size of the company

7.0

4.1

[5b] Increased turnover by companies that had access to LGF (in EUR ), per million EUR invested

22,295.1 mEUR

Average value of turnover at the time of first inclusion (362 kEUR ), multiplied by the average growth on turnover (20. 5%) as reported in the “COSME Loan Guarantee Facility – Employment and Growth report, 2015”. This figure is then grossed-up by multiplying the average value of the increased employment with the total number of SMEs expected to be supported (300,000)

59.4

34.9

[11b] Increased employment by companies that had access to LGF, per million EUR invested

496,911 people

Average number of employees at the time of first inclusion (3.9), multiplied by the average growth on employment (42.7%) as reported in the “COSME Loan Guarantee Facility – Employment and Growth report, 2015”. This figure is then grossed-up by multiplying the average value of the increased employment with the total number of SMEs expected to be supported (300,000)

1,323.2

778.9

[8b] (Additional) Increased turnover by companies that had access to LGF (in EUR ), per million EUR invested

14,045.9 mEUR

Grossed-up value of increased turnover (as shown in indicator [5]) multiplied by % of SMEs that stated that ‘the financing supported by a EU-COSME guarantee was the only option available or that other options were available but did not cover the full amount’ (63% according to the LGF SME survey)

37.4

22.0

[14b] (Additional) Increased employment by companies that had access to LGF, per million EUR invested

313,054 people

Grossed-up value of increased employment (as shown in indicator [11]) multiplied by % of SMEs that stated that the financing supported by a EU-COSME guarantee was the only option available or that other options were available but did not cover the full amount (63% according to the LGF SME survey)

833.6

490.7

Business management capacity - EEN

[2] Number of SMEs reached and supported, per EUR million invested

COSME Budget committed:

144.5 mEUR

[+ 9.6 mEUR ]

57.8 mEUR

[Based on co-funding from EEN members, equivalent to 40% of the EC contribution]

210,000 SMEs

Estimations prepared in the context of this evaluation

1363

991

[6] Increase in turnover linked to services provided by EEN (in EUR ), per million EUR invested

34,785 mEUR

Grossed-up figure of turnover based on average turnover growth per SMEs in the period 2014-2016, as reported in the EEN SME client survey (696 kEUR ), multiplied by the total number of SMEs supported (50,000 SMEs).

949

690

[12] (Additional) Increase in turnover linked to services provided by EEN (in EUR ), per million EUR invested

1,739.2 mEUR

Grossed-up figure of turnover (as shown in indicator [6]) multiplied by an additionality factor of 6.5%.

The additionality factor is based on our calculations on growth presented in EEN Thematic Report, where we show that turnover among EEN SME clients has grown 6.5 percentage points more than a benchmark group. This estimate excludes outliers and considers death rates.

2727

1983

[9] Increase in employment linked to services provided by EEN, per million EUR invested

200,000 employees

Grossed-up figure of employment based on average growth on number of employees per SMEs in the period 2014-2016, as reported in the EEN SME client survey (4 multiplied by the total number of SMEs supported (50,000 SMEs).

62

45

[15] (Additional) Increase in employment linked to services provided by EEN, per million EUR invested

19,000 employees

Grossed-up figure of turnover (as described above in indicator [9]) multiplied by an additionality factor of 9.5%.

The additionality factor is based on our calculations on growth presented in EEN Thematic Report, where we show that employment among EEN SME clients has grown 9.5 percentage points more than a benchmark group. This estimate excludes outliers and considers death rates.

518

377

Internationalisation - Cluster Go International

[20] Number of cluster supported, per EUR million invested

2.3 mEUR

[+ 0.1 mEUR ]

0.6 mEUR

[Based on co-funding from partners, as funded ESCP received 75% co-funding from the EC]

93

Based on number of partners as reported in ESCP-4i website

38.8

31.4

[3] Number of SMEs supported by Cluster, per EUR million invested

11,246

Based on estimated number of SMEs supported in Strand 1 and 2, according to ESCP-4i 2nd synthesis report

4,998.2

3,798.0

Entrepreneurship - EYE

[4] Number of SMEs supported, per million EUR invested

24.3 mEUR

[+ 1.6 mEUR ]

12.2 mEUR

[Based Calculated based on the information provided by HE and NE via the survey, and the number of HE and NE involved in the EYE in the period of analysis]

3,737

2,117 New entrepreneurs and 1,620 Host Entrepreneurs, as reported in monitoring data

144.4

98.0

[21] Number of new entrepreneurs that have created their own business (thanks in part to their participation in EYE), per million EUR invested

241 New entrepreneurs

Based on percentage of NEs creating a business according to our survey (11.4%), multiplied by the number of new entrepreneurs (2,117). Our calculation (11.4%) includes those NE that have reported turnover in 2016 (ie had a business generating turnover in 2016).

9.3

6.3

[22] Number of entrepreneurs engaging in at least one business cooperation agreement, per million EUR invested

993 Entrepreneurs

Based on percentage of NE engaging in at least one business cooperation agreement according to our survey (46.9%) multiplied by the number of new entrepreneurs (2,117)

38.4

26.0

[7] Increase in turnover linked to EYE (in EUR ), per million EUR invested

209.0 mEUR

Grossed-up turnover figures reported in the HE and NE surveys, and number of HE and NE. The figure includes the increase in total turnover for both HE (203.8 mEUR ) and NE (5.2 mEUR ). The estimated average increase in turnover for HE is 125.7 kEUR and for NE is 2.5 kEUR ]

8.1

5.5

[13] Increase in employment linked to EYE, per million EUR invested

122.2 mEUR

Based on grossed-up figure of turnover and an additionality factor of 58% and 73% for HE and NE respectively. These factors are calculated based on survey responses. 58%of HE that agree that NE contributed to growth in companies' turnover, while 73% % of NE that agree that EYE has helped them to establish a new business. We use these parameters and assume that in 58 per cent of the cases attribution for growth is 100% while in 47 per cent attribution is 0%. A similar assumption is applied to NE

85.6

58.1

[10] (Additional) Increase in turnover linked to EYE (in EUR ), per million EUR invested

2,216 employees

Grossed-up turnover figures reported in the HE and NE surveys

4.7

3.2

[16] (Additional) Increase in employment linked to EYE, per million EUR invested

1,247 employees

Based on grossed-up figure of turnover and an additionality factor of 41% and 73% for HE and NE respectively. These factors are calculated based on survey responses. 41%of HE that agree that NE contributed to growth in companies' employment. In the case of NE we use the same measure for turnover. We use these parameters and assume that in 41 per cent of the cases attribution for growth is 100% while in 59 per cent attribution is 0%. A similar assumption is applied to NE.

48.2

32.7

Source: Technopolis (2017)

4.The consultations 

This chapter presents the different types of consultation we used. We begin with the quantitative targeted consultations (4.1): first an overview, then the sampling strategy, then the results for each SO. In section 4.2 we present the qualitative targeted consultations. The last section deals with the public question (0): its structure, its implementation, the outcomes and its limits.

4.1.The targeted consultations – quantitative

4.1.1.Overview 

This section presents an overview of the targeted consultation, specifically the seven ‘quantitative’ surveys that have informed this evaluation.

Table 23 presents an overview of the surveys, including the target group, population, target sample, responses and response rates, while a more detailed reporting on the sampling methods, data collection process and response rates is provided in the sub-sections below.

All draft questionnaires were included in the Inception Report and presented in further iterations; final approvals of the questionnaires were obtained from the corresponding contact points at the EC. The final questionnaires are provided in Error! Reference source not found. to this report.

Table 23 Overview – Quantitative surveys

#

Action

Type of stakeholder

Population

Target sample

Number of responses

Response rate

Closing date in 2017

1

Loan Guarantee Facility

Beneficiary SMEs

70,000

3,870

356*

9%

18 July

2

Enterprise Europe Network (EEN)

EEN member organisations

523**

530

270

48%

17 May

3

Enterprise Europe Network (EEN)

Beneficiary SMEs

(EEN clients)

210,000

50,000

2,334

5%

17 May

4

Cluster internationalisation

Intermediary agencies and

cluster organisations

136

136

48

35%

5 May

5

Tourism (Calls)

Beneficiary SMEs and Intermediary agencies / organisations

377

389

162

41%

10 May

6

Erasmus for Young Entrepreneurs

New Entrepreneurs

2,343

2,117

595

28%

5 May

7

Erasmus for Young Entrepreneurs

Host Entrepreneurs

1,800

1,620

245

15%

5 May

Source: Technopolis - *Reached via 20 financial intermediaries. **Reached via 92 coordinators.

4.1.2.Sampling strategy and results

The following paragraphs explain our sampling approach for the quantitative surveys. In the case of the short surveys no sampling is needed as all stakeholders will be approached.

We have taken careful consideration of potential issues related to ‘survey fatigue’ when designing our sampling strategy.

SO1 - Sampling of the beneficiary SMEs of Loan Guarantee Facility

The ToR provides a series of good recommendations regarding the sampling strategy for the financial instruments. There are also lessons to learn (in terms of expected response rates) from the “Final Evaluation of the Entrepreneurship and Innovation Programme” (2011) 87 . Taking those elements into account, we estimated that we would need to address 70,000 companies to obtain a response of 3,150.

We requested EIF/ DG Grow for access to the full list of LGF recipient companies, including all contact details necessary for implementing the survey (name, email address, telephone number).

We followed the following steps:

·We drew a stratified random sample of circa 70,000 companies from the amended list and invite them to participate in our (targeted) consultation. Stratification ensured the sample has a good spread across countries and sectors (provided this information is recorded alongside contact details).

·To arrive to a sample of 3,870 companies, we needed an (implicit) response rate of 5.6%, which was virtually the same as the response rate from the EIP. This sample size guaranteed a good representation across sectors and countries.

·A response of 3,150 represented a representative sample, with 95% confidence interval and a margin of error of +/- 6%.

We monitored responses in real time to make sure we are achieving a good spread of responses.

The Commission signed off on the survey of stakeholders of the SME Loan Guarantee Facility on 5 May 2017.

The survey was translated into 16 languages and released to an estimated 3,870 beneficiaries via 20 financial intermediaries in 19 countries. The study team constructed a random sample of 250 beneficiary companies for those intermediaries that had served a greater number of clients. Those intermediaries that had served fewer than 250 clients were asked to distribute the survey to all beneficiary clients.

Via this method, the companies that had received financing supported by an EU-COSME guarantee were originally invited to contribute to the study via an online survey between 19 May and 23 June 2017. Subsequent surveys were launched over the following weeks; to boost the number of responses and facilitate participation, the survey deadline was first extended to 30 June and then to 7 July 2017.

The survey received 356 responses, equivalent to a response rate of 9%.

SO2 - Sampling of the EEN member organisations

We approached EEN members through a broad call that was distributed by the EEN coordinators. We received access to the full list of EEN members, including all contact details necessary for implementing the survey (name, email address).

Enterprise Europe Network member organisations were originally invited to contribute to the study via an online survey between 12 April and 3 May 2017. The survey was initially released to 92 coordinators, and, to boost the number of responses and ease of participation, the survey was subsequently released to 431 partner organisations on 19 April. A further reminder e-mail was circulated to both groups on 26 April.

Over the course of implementation of the survey, the study team worked directly with both coordinators and partners to ensure that all relevant parties received the survey invitation. This resulted in additional partners receiving the invitation that were not initially included in the data, and subsequent greater access to client groups for the second survey. It also ensured that the survey has reached individuals that had replaced colleagues that were no longer in relevant positions.

Survey progress was briefly interrupted when the EEN members communicated through the SAG bureau that the timing of the survey was a bit inconvenient given the holiday period, and that they had received word from EEN members that were not in the position to send out the SME survey in time. An additional reminder was set out with May 17 as a new end date of the survey, providing the EEN members with an additional opportunity to participate in the survey and to send out the SME survey to their clients.

Overall, we received 270 responses, accounting for 48% of the target sample (the highest response rate of the targeted quantitative consultations).

SO2 - Sampling of the EEN client SMEs

All 92 coordinators were approached in the first instance, with a request to distribute details of our survey of client SMEs through the network.

Working closely with these coordinators we reached out to most of EEN’s 523 organisations, both within and outside of the EU. Through these organisations, we also issued further requests to around 50,000 client SMEs. Based on our experience of the previous EEN evaluation we expected the response rate from EEN member agencies to be high. At the same time, we expected SME response rates to be much lower, and we aimed for a ~5% response rate from those contacted.

Prior to the dissemination of both surveys, a letter from the Head of Unit H2 was posted on the Enterprise Europe Network intranet, advising Network members of the upcoming consultation, and seeking their support to disseminate the SME survey to their client base. The time available to answer the survey was extended once to accommodate for the busy schedules of the stakeholders.

Enterprise Europe Network client SMEs were also invited to participate in the study via an online survey questionnaire. The survey of client SMEs was disseminated as a cascade survey, i.e. Network members invited to participate in the first survey (above) were asked to support the study in distributing an open web link to their client base.

As with the survey of Network members, the survey of client SMEs was released in two phases, the first on 12 April 2017, when the study team wrote to all individual Network coordinators following their own survey invitation, and subsequently, on 19 April, when the study team wrote to the group of Network partners, following their own invitation to participate in the survey of Network members. We provided a text that Network members could forward to their own clients to participate in the study.

The study team worked closely with Network members to ensure as high as possible participation of SMEs in the survey. To accommodate national holidays in many countries, the study team further wrote to all 530 coordinators and partners on 26 April 2017. The survey closed at the end of 17th May.

Of the 50,000 SMEs of the target sample, we received 2,334 responses or 5%. Though rather low, this result met our expectations.

SO3 - Cluster organisations

We addressed the 136 cluster organisations that are funded through the Cluster Internationalisation calls. We requested access to the full list of cluster organisations, including all contact details necessary for implementing the survey (name, email address).

In our survey of cluster organisations in 2014, in the context of the “Evaluation of cluster initiatives managed by DG Enterprise and Industry”, we had obtained a response rate of circa 8%. This time, we expected a higher response rate as the 136 clusters had only recently received support from COSME. We therefore aimed at reaching a response rate of at least 25-30%, or 34-40 responses.

We received 48 answers, or a 35% rate, thus higher than our expectations.

SO3 – Tourism actions

We addressed the 377 organisations that were successful in their applications to the open calls for proposals in the field of Tourism. The list of these beneficiaries is available in the COSME Data Hub.

Participants and beneficiaries of actions funded through COSME tourism calls were invited to contribute to the study via an online survey that took place between 26 April and 10 May 2017. To boost the response rate, the study team sent a reminder on 3 May 2017. The survey was closed on the 10 May 2017. In total, 389 participants and beneficiaries – both intermediary organisations and SMEs – across 33 countries were invited to contribute to the online survey consultation for this interim evaluation.

We received 162 answers, or a 41% rate.

SO4 - EYE Host entrepreneurs

As the key action funded within the entrepreneurship area, the EYE programme was the focus of targeted consultation surveys. Specifically, we invited all Host and New Entrepreneurs in the programme during the 2014-2016 period to provide information and feedback on their participation in this action. As was reported above, these surveys received a good response rate from both groups of beneficiaries (with 841 Host and New Entrepreneurs responding in total). Analysis of the responses to these questionnaires has fed into various parts of the evaluation, particularly around the effectiveness and efficiency of the programme. Follow-up interviews were also conducted with a number of the beneficiaries (as reported below).

We asked EASME/ DG Grow for access to the full list of Host entrepreneurs, including all contact details necessary for implementing the survey (name, email address).

To avoid survey fatigue, we selected (randomly) a proportion of the total number of Host entrepreneurs involved in EYE in the period 2013-2016. We reached a sample of 1,620 Host entrepreneurs. Based on the response rates obtained in the prior exercise, we expected to attain a response from 250 HE as a result. In the end, we received – in line with our expectations - 245 answers which make up for 15% of the target sample.

Host Entrepreneurs were invited to contribute to the study via an online survey that was open between 12 April 2017 – 5 May 2017. To boost the response rate, the study team sent out a single reminder on 26 April 2017.

SO4 - EYE New entrepreneurs

We asked EASME/ DG Grow for access to the full list of Host entrepreneurs, including all contact details necessary for implementing the survey (name, email address).

To avoid survey fatigue, we selected (randomly) a proportion of the total number of New entrepreneurs involved in EYE in the period 2013-2016. We reached a sample of 2,117 New entrepreneurs. Based on the response rates obtained in the prior exercise (10%), we expected to attain a response from 100 NE as a result. Of the 2,117 entrepreneurs contacted, we received 595 answers (much higher than our expectations) or a response rate of 28%.

New entrepreneurs were invited to contribute to the study via an online-survey between 12 April 2017 – 5 May 2017. To boost the response rate, the study team released a single reminder on 26th April 2017.

4.1.3.Robustness of the results

SO1 – Access to finance

Overall, the survey received 356 responses, equivalent to a response rate of 9%. This represents a higher response rate than recorded in previous exercises such as the 2011 Final Evaluation of the Entrepreneurship and Innovation Programme (where the response rate was approximately 6.3%). 88

Figure 8 below shows a comparison by country of the distribution of the LGF beneficiaries as a proportion of the population, of the target sample and the survey population, while Figure 9 , displays the distribution of LGF beneficiaries within sectors of activity. These figures illustrate the degree of the representativeness of the survey population, and we emphasise that the survey population should not be used to profile all beneficiaries.

The survey did not target all countries in which there are beneficiaries of EU-COSME guarantees. 89 For the most part, the shares of companies in each country are higher within the survey population than in the overall beneficiary population, except in Spain and Italy.

Overall, the distribution of the survey responses is well balanced in terms of country and sector of activity in comparison with the target sample. One notable exception is France, where despite our best efforts we did not receive any response as the financial intermediary did not circulate the survey before the deadline.

The distribution of sectors is largely consistent between the population and the survey respondents. This is particularly evident in the most prominent sectors, such as ‘Wholesale and retail trade; repair of motor vehicles and motorcycles’, and ‘Manufacturing’. 90

Figure 8 LGF beneficiaries: comparison of the country distribution of the whole population, target sample and survey population

Source: Technopolis, based on provided beneficiary details and survey data

Figure 9 LGF beneficiaries: comparison of the sector distribution of the whole population, target sample and survey population

Source: Technopolis, based on provided beneficiary details and survey data. NB[1]: there is a slight difference between the sectors in the provided information (where ‘Wholesale and retail trade’ is combined with ‘Repair of motor vehicles and motorcycles’ as per the NACE classification) and the list of sectors within the survey questionnaire, where these sectors are presented separately. The survey data responses have been combined for the purposes of this comparison. NB[2]: The reclassification of survey respondents resulted in the addition of one sector not originally offered in the survey questionnaire: ‘Water supply; sewerage; waste management and remediation activities’. This is not shown in this chart.

SO2 – Access to market

Survey of Network Members

In total, the survey targeted 531 Network members: 89 coordinators and 442 partners across 37 countries. At the time of reporting (May 17), the survey has received 270 responses from across 35 countries. This represents an overall response rate of 48%. The response rates from both coordinator (47%) and partner groups (48%) are roughly equal.

The distribution of the survey population is very similar to the target sample in its two main characteristics: country grouping and Network member role. Figure 10 and Figure 11 , below, summarise this.

Figure 10 Network member survey target sample and respondents, by country

Source: Technopolis, based on survey data. Base = 270

Figure 11 Network member survey target sample and respondents, by role

Source: Technopolis, based on survey data. Base = 270

Survey of Network client SMEs

The cascade approach meant that the study team did not have direct control over the number of SME clients that are invited to participate in the survey. We estimated the target sample for the client SME survey to be 50,000, and set out a target to achieve a response rate of 5% (2,500 client companies). The survey of SME clients received 2,334 SME responses across 34 countries, 91 representing a response rate of 5% from the estimated target population.

Over two thirds of respondents (1,639) are located within EU15 Members States, though there are sufficient respondents located in EU13 Member States (25%, 568 respondents) to analyse questions based on country groupings. The number of respondents from COSME-participating third countries (4%, 98 respondents) is too low to make meaningful analysis. The distribution of the survey respondent base is shown in Figure 12 , below.

Figure 12 Network client SME company survey respondents, by country group

Source: Technopolis, based on survey data. Base = 2,309

SO3 – Favourable environment / Cluster internationalisation

A total of 48 responses were received, equivalent to a 35% response rate, which is higher than the desired response rate of 25%.

The response rate varied between the two main groupings: ‘funded’ and ‘voluntary’ ( Figure 13 ). The response rate for funded organisations (51%) was higher than that for voluntary organisations (15%), which is to be expected, given the closer involvement of funded organisations with the programme.

Figure 13 Distribution of population in sample and responses between type of cluster organisation

Source: Technopolis

Respondents were asked to characterise the main sector of activity of their company. Respondents were spread fairly evenly across the sectors, with a relatively high representation within the information and communication (ICT) sector (16.7%). Nearly half of the respondents placed the activities of their company in the ‘Other (please specify)’ category, which refers to transport (aerospace and rail), sports, and high-tech industry.

SO3 – Favourable environment / Tourism

The survey received 162 responses from 32 countries 92 , equivalent to a 41% response rate. Respondents have been split into three main country groupings: EU15 countries (those that were Member States prior to 2004), EU13 countries (those that joined as Member States in 2004 and subsequently), and COSME-participating third countries. 93   Figure 14 , below, illustrates the high level of alignment between the country groups split between the target sample and respondent groups, for those for which this information was available.

Figure 14 Profile of target sample and SO3 tourism survey respondents, by country group

Source: Contact data provided by the Commission, and Technopolis, based on survey data

SO4 - Entrepreneurship

New entrepreneurs

In total, 2,117 New Entrepreneurs from 37 countries were invited to contribute to the survey consultation for this evaluation. In the three weeks that the survey was open, 595 responses were received, equivalent to a 28% response rate (and higher than the expected response rate of 25%, based on results obtained in prior studies).

Most countries in the sample population were represented by at least one respondent, apart from two EU countries, Malta and Luxembourg. However, it is worth noting that these countries only had two contacts each, or 0.1% of the sample population. The numbers for some countries are rather small for the purposes of analysis, so we divided them into three groups as explained above.

Figure 15 shows that the distribution of responses (among those respondents for which the geographical location is known) is very similar to that of the target sample when considering the three country groupings: pre-existing EU Member States (as of 2004), expansion countries (those joining the EU post-2004), and COSME-participating third countries.

Figure 15 Distribution of population in sample and responses by types of countries

Source: Contact data provided by the Commission, and Technopolis, based on survey data

Host entrepreneurs

In total, 1,620 Host Entrepreneurs from 36 countries were invited to contribute to the survey consultation for this evaluation. In the three weeks it was open, the survey received 245 responses, equivalent to a 15% response rate, which is the expected response rate (based on the results obtained in prior studies).

Respondents were asked to characterise the main sector of activity of their company. All but three of the categories provided were represented by at least one respondent. These three were: ‘Repair of motor vehicles and motorcycles’, ‘Electricity, gas, steam and air conditioning supply’, and ‘Mining and quarrying’. The highest share of respondents reported their main sector of activity as ‘Other’, followed by ‘Arts, entertainment, and recreation’ and ‘Information and communication (ICT)’.

Figure 16 shows that also in this case, the distribution of responses (among those respondents for which the geographical location is known) is very similar to that of the target sample when considering the three country groupings: EU13 MS, EU15 MS and COSME-participating third countries.

Figure 16 Distribution of population in sample and responses by types of countries

Source: Contact data provided by the Commission, and Technopolis, based on survey data

4.2.The targeted consultations - qualitative

4.2.1.Overview 

The qualitative targeted consultations were used only for two SOs: SO1 on Access to finance and SO3 on Favourable environment. As mentioned above, this type of survey was implemented where the population was lower than 100.

Table 24 below presents an overview of the participation in the qualitative surveys.

Table 24 Overview of the participation in the qualitative surveys

Target sample

Number of responses

SO1 – Access to finance

LGF intermediaries

21

16

EFG intermediaries

12

4

SO3 – Favourable environment

SME Policy

28

15

SME Policy

Sectoral competitiveness

127 (according to the email list received from EC)

23

72

7

E-skills

31*

8

Source: Technopolis

4.2.2.Sampling strategy

SO1 - Sampling financial intermediaries and sub-intermediaries of Loan Guarantee Facility

Among the 61 financial intermediaries receiving LGF funds, we decided to approach 21 of them. Five of them (the financial intermediaries) had been already identified (cat 1 promotional banks and cat 2 guarantee societies) and other were selected according to their type (Promotional Institution, Guarantee Institution, Commercial Bank, Leasing Company), country of establishment, direct/counter Guarantees and risk category in order to have a wider and representative possible coverage; but also some of them who implemented as well CIP SMEG and overall 50% currently implementing also InnoFin SMEG. These gave us insights of the process and leverage effect.

4.2.3.Results for SO1 – Access to finance

Four qualitative surveys were conducted:

·qualitative surveys of the LGF and EFG intermediaries

·qualitative survey of the EFG SMES

·qualitative survey of the members of European Intermediary Organisations

A written questionnaire was sent via e-mail to 21 LGF intermediaries, asking the recipients to complete the questionnaire within two weeks. Two reminders were sent to LGF intermediaries that did not respond by the fixed deadline. Finally, the team received 16 completed questionnaires, corresponding to a response rate of 76%. One LGF intermediary explicitly refused to participate, while four intermediaries did not answer. Eight responding intermediaries provide direct guarantees for loans and another eight intermediaries provide counter guarantees.

A written survey of EFG intermediaries was conducted. The team originally planned to collect data from five EFG intermediaries. The responding EFG intermediaries make equity and equity-related investments in growth-oriented start-ups and SMEs all over Europe. The latter were asked to cascade the survey to selected SMEs 94 that received equity funding from the EFG instrument via the intermediary. It was planned that 12 SMEs were to be contacted by their respective intermediaries. However, only three out of five intermediaries completed the written questionnaire for EFG intermediaries. This also implied that only a limited number of SMEs that were beneficiaries of the EFG (EFG SMEs) could be reached. As a result, the team received only four completed questionnaires (out of 12) from EFG SMEs.

Five of the six contacted intermediary organisations were asked by the team to cascade the survey to their members. Except for one intermediary, the European intermediary organisations contacted their members prior to the interviews and collected information on their members’ opinions and views. In two cases, the interviews were even conducted in the form of phone conferences in which representatives of both the umbrella organisation and individual members of the intermediary organisations participated. In this way, the opinions and views expressed during the interviews directly reflect those of individual members.

4.2.4.Results for SO3 – Favourable environment

We launched a total of four ‘qualitative’ surveys related to the actions under SO3. A first step in the process was the drafting of the survey questionnaires. Table 7 below, gives an overview of this process, which was rather time consuming due to the number of surveys to be covered.

Table 25 Overview of status of qualitative surveys

Survey

Date of draft

Approval date

Launch of survey

SME Policy – SME Envoys

21.03.2017 - draft 1; 27.03.2017 – draft 2

27.03.2017

28.03.2017

SME Policy – SPR Working Group

27.03.2017

11.04.2017

11.04.2017

E-skills

10.04.2017

10.04.2017

28.04.2017

Sectoral competitiveness & KETS

10.04.2017

18.04.2017

02.05.2017

The results of these surveys vary substantially. There was a good level of response to the survey in the field of SME Policy (perhaps due to the fact that it was sent out by the European Commission), while response rates were low for e-skills and sectoral competitiveness. Even though the contacted actors had participated in events or activities of the COSME actions in the field of KETs, e-skills, or sectoral competitiveness, some of them mentioned that they were not aware of the COSME programme nor had they had any interaction with it. In addition, despite being contacted several times, the members of the Digital Coalition for Skills and Jobs (to whom the e-skills survey was targeted) were not responsive – only the Coalition members contacted for interviews were willing to contribute.

Table 26 Overview of responses to SO3 short survey consultations

Survey

Actors targeted

No of targeted actors

No of responses received

SME Policy

SME Envoys

28

15

SPR Work group members

127 (according to the email list received from EC)

23

Sectoral competitiveness

European stakeholder organisations, national stakeholder organisations (associations), national and regional public authorities

72

7

E-skills

European stakeholder organisations, national stakeholder organisations dealing with e-skills

31*

8

Source: Technopolis - *out of the 53 e-mail addresses received, only 37 were valid. Out of them 6 contacts were contacted for interviews and 31 for the qualitative survey.

4.3.The public consultation

The objective of the public consultation was to cover all potential relevant stakeholders, including those who were not covered by the targeted consultations and interviews, and to collect feedback from them on the COSME programme and its objectives. More specifically, we expected inputs from:

·Citizens

·Companies

·European business organisations

·National business organisations

·Trade unions

·Non-governmental organisations

·National public authorities

·Regional/local public authorities

·Financial institutions/Finance providers

·Academia/Research centres

·Professional consultancies/law firms

·Incubators/clusters/innovation support centres

4.3.1.The structure of the public consultation questionnaire 

The questionnaire for the public consultation was quite straightforward as it aimed at reaching all types of stakeholders. Given that the targeted consultations focused exclusively on the specific objectives, it was agreed to use the public consultation to collect data that would allow a general assessment of the evaluation criteria for the overall programme. The questionnaire included 10 questions focusing on all evaluation criteria except for efficiency.

Because public consultations are available in all EU languages, translation of open responses can become very time-consuming, can critically delay the treatment of responses, and undermines comparability between Member States. Therefore, we collected all questions by means of closed questions (multiple choices). As it is still important to provide stakeholders with the opportunity to express themselves in an open structure, we included an open question at the end of the questionnaire to collect additional information and all types of comments from the respondents.

4.3.2.The implementation of the public consultation

The approach of the public consultation followed the consultation process defined by the European Commission (Better Regulation Toolbox/Tool #50). Following an initial phone conversation with the client specifically regarding the public consultation in early February 2017, Technopolis Group submitted a draft questionnaire on the 24 March 2017 and feedback was shared by the European Commission on the 29 March 2017. A final questionnaire, resulting from an internal EC drafting and validation process, was validated by the end of April 2017.

As required in the Terms of References, the questionnaire was available in all EU languages. The Commission translation service (DGT) took care of the English version of the consultation and further translations in all other EU languages followed. The European Commission implemented the questionnaire for the public consultation on the “Your Voice in Europe” website and the questionnaire was available in English on 10 May 2017. It remained accessible for 12 weeks, starting from the date on which the last translation was made available online.

The dissemination of the public consultation fell under the responsibility of the European Commission. EASME and the EC’s communication units advertised the public consultation on COSME and EASME websites; through EEN networks and relevant national contact points; through other agencies and networks under COSME; and via other EC social media.

Responses have been monitored regularly (brief update reported by the EC every two weeks) to identify the types of stakeholders (for example, stakeholders in specific countries) from which fewer responses were received and to dedicate efforts to advertise the public consultation to them. After the end of the consultation period, the Commission forwarded to our team the raw data collected in the original language, in both Word and Excel formats.

Table 27 Overview of the open public consultation process

Activity

Timing

Phone call with the EC

10 February 2017

Delivery of first draft questionnaire

24 March 2017

Review by the EC

29 March 2017

Validation of final questionnaire

26 April 2017

Editing in EN by EC

26 April – 5 May 2017

Translation in other EU languages

6 May – 12 June

Deadline of the consultation

Launch date: 10 May

Closing date: 31 August

EC to send inputs to Technopolis Group

Latest results received mid-September (13/09/2017)

Source: Technopolis

4.3.3.Outcomes of the public consultation 

The public consultation was closed on the 31 August 2017. We received the final results by mid-September (13/09) 2017. A total of 195 responses were received, as well as 14 position papers.

The questionnaire also included questions that allowed us to profile the respondents to the public consultations.

Half of the respondents are active in industry/business; the second largest group is comprised of financial intermediaries (financial institutions or finance providers). The respondents under ‘others’ included law firms, research centres, innovation centres, NGOs and associations; business organisations; public authorities; and citizens (see Figure 4 ). Five countries account for two thirds of the answers (France, the UK, Italy, Spain and Germany).

Figure 17 Public consultation - profile of the participants

Source: COSME public consultation, 2017, n=195

Two-third of the respondents, i.e. 88 respondents, indicated that their organisation had benefitted from the COSME programme. Five respondents were involved in more than one action (e.g. EEN as a member organisation and beneficiary, etc.). The Loan Guarantee Facility (LGF) is, by far, the most recurrent action.

Table 28 Number of respondents per type of financial support

Type

Action

Number of beneficiaries

Access to finance

Loan Guarantee Facility

55

29 companies

16 financial institutions

10 others

Equity Facility for Growth

5

Access to markets

Enterprise Europe Network (EEN) – EEN member organisations

15

Enterprise Europe Network (EEN) – beneficiaries

13

Entrepreneurship

Erasmus for Young Entrepreneurs – New Entrepreneurs

4

Erasmus for Young Entrepreneurs – Host Entrepreneurs

4

Favourable environment

Cluster internationalisation

2

Tourism (calls)

2

Other

Selected as an expert

NCP for COSME including financial instruments and publishing calls for proposals in all areas / Coordinator EEN including extra support for EYE-participation

COS-DESIGN

3

Source: COSME public consultation (n=88)

4.3.4.Statistical analysis 

The survey responses were analysed using descriptive statistics.

The large number of respondents enabled us to undertake cross-analyses. We compared their views on the relevance of COSME actions to their views on the results (and the other way around) and to the benefits they expect from the programme (and the other way around). We also compared the expectations of the respondents with the areas of improvement they identified (and the other way around).

5.Interviews

In this chapter, after a brief overview (5.1), we present the results from our interview strategy (5.2), here again for each SO.

5.1.Overview

Overall, our team conducted 121 interviews. Table 29 provides an overview of the interviews conducted for each SO.

Table 29 Overview of the interviews conducted, by SO

 SO

 Interviewed stakeholders

 Nr

SO1 – Access to finance

representatives from EU institutions (DG GROW, DG REGIO, DG RTD, DG ECFIN, EIF and EIB)

15

intermediary and industry organisations

6

SO2 – Access to market

EU officials who were responsible for the Network and for other programmes aimed at fostering coherence within the Network.

9

Network members

19

Umbrella organisations were also interviewed so as gain the views of SMEs and intermediaries at a higher aggregated level.

5

SO3 – Favourable environment

EU officials in DG GROW, EASME and other DGs

16

Beneficiaries (cluster managers)

6

National government representatives

4

Umbrella organisations and national experts

10

SO4 – Entrepreneurship

Individuals responsible for the design, management and oversight of COSME entrepreneurship actions – including European Commission officials, EASME staff and individuals within the EYE support office

20

Host Entrepreneurs – participants in and beneficiaries of the EYE programme

5

New Entrepreneurs - participants in and beneficiaries of the EYE programme

6

Source: Technopolis

5.2.Results

5.2.1.SO1 – Access to finance

The team conducted 21 phone interviews with high level stakeholders. These comprised 15 interviews with representatives from EU institutions (DG GROW, DG REGIO, DG RTD, DG ECFIN, EIF and EIB) and six interviews with intermediary and industry organisations. The rate of response was equivalent to 100%, i.e. all relevant stakeholders could be reached and interviewed as originally planned.

See Table 47 , Appendix D for the list of interviewees.

5.2.2.SO2 – Access to market

For the evaluation of the Access to Markets actions, the interviews conducted mainly targeted the Enterprise Europe Network and other programmes/actions related to the Network. We focused on interviewing EU officials who were responsible for the Network and for other programmes aimed at fostering coherence within the Network. We spoke to a large number of Network members in order to gain deep knowledge of how the Network works in practice. Finally umbrella organisations were also interviewed so as gain the views of SMEs and intermediaries at a higher aggregated level.

We have been successful in conducting interviews during the evaluation despite the coinciding holiday period. Almost all interviews that were foreseen were conducted during the time set out in the project planning, and two additional interviews were conducted to provide further insight. The list of the 34 interviews is shown in Table 48 , Appendix D.

5.2.3.SO3 – Favourable environment

For SO3, we have performed 36 interviews to cover the SME Policy and cluster internationalisation key actions, the case studies on tourism and e-skills, as well as the thematic areas such as KETs and sectoral competitiveness. Some interviews have been performed, due to various difficulties in reaching the interviewees. Bottlenecks encountered included the limited availability of some of the interviewees (e.g. in the field of cluster internationalisation and e-skills), as well as potential lack of interest or suitability to contribute. The latter was especially the case in the field of sectoral competitiveness and KETs, where stakeholders are not aware of COSME actions in this field and are reluctant to answer questions. Where possible, replacements were found, however, in most cases, it was not possible to find a replacement in due time.

An overview of the number of interviews performed is shown in Table 30 .

Table 30 SO3 – Performed interviews

No. interviews conducted

EU officials

DG GROW

8

EASME

2

Other DGs

6

Beneficiaries (intermediaries)

6

National governments

4

Umbrella organisation & national experts

10

Total

36

Source: Technopolis

Table 49 , Appendix D lists the people with whom interviews were conducted for all the actions foreseen for the evaluation under the COSME SO3 work stream.

Table 50 (same appendix) lists the persons with whom interviews were planned initially, but not performed by the end of the evaluation due to lack of availability, refusal, or other reasons.

5.2.4.SO4 – Entrepreneurship

The team undertook interviews with three main groups of stakeholders, focusing mainly on the key entrepreneurship action (EYE), but also covering many of the other actions in the portfolio. A small number of interviews were also undertaken with other relevant stakeholders. The main groups of interviewees were:

·Individuals responsible for the design, management and oversight of COSME entrepreneurship actions – including European Commission officials, EASME staff and individuals within the EYE support office

·Intermediary organisations - which are responsible for matching, managing and monitoring EYE relationships between New and Host Entrepreneurs.

·Host Entrepreneurs – participants in and beneficiaries of the EYE programme

·New Entrepreneurs - participants in and beneficiaries of the EYE programme

Table 51 , Appendix D lists the 30 individuals interviewed in relation to the EYE programme and other entrepreneurship actions.


Appendix A- COSME programme KPIs, and the extent to which these KPIs have been realized 

Table 31 Legend to Table 32 below

COSME's Medium term KPIs' targets for 2017 met.

KPIs have changed. Even though goal attainment cannot be measured, new indicators show promising developments.

Targets not met or no info at all available

Table 32 Main activities, KPIs, and the extent to which these KPIs have been realised

Result indicator

Situation in 2011

COSME's Medium term KPIs for 2017

Did COSME meet the KPI targets?

 Source

Activities to improve Competitiveness

Number of simplification measures adopted

The Commission's simplification programme was updated in 2010 and is on track to cut red tape by 25% in 2012. There were 5 simplification measures per year done until 2010.

About 7 simplification measures per year.

2014: 27% reduction of the burden

monitoring report

Number of "fitness" checks on quality and value-added of activities

Four "fitness" checks including stakeholders were launched in 2010 for environment, transport, employment and industrial policies. Feedback included
comments on legislation and value-added of activities.

The feedback approach with "fitness" checks will be extended to other policies and lead to simplifications impacting positively on industry. Up to twelve "fitness" checks are foreseen, with the objective of better regulation.

2014-2016: 15 regulation impact assessments
2014: 8

2015: 4

2016: 3

Data provided by DG GROW

Level of adoption by companies of European sustainable production and product tools, including EMAS, eco-label, and eco-design

Approximately 35,000 ISO 14001 EMS certifications and 4,500 EMAS registration, 18,000 licences for the EU Ecolabel

Significant number of companies monitor their performance, apply environmental management systems and achieve improvement in resource productivity and environmental performance. Significant part of production are resource efficient and environmentally friendly products

(2005-)2015: 39,340 EMAS/ISO 14001 certifications

http://ec.europa.eu/environment/emas/emas_registrations/statistics_graphs_en.htm

Developing SME policy

Number of Member States using SME test

Number of Member States using SME test: 15 MS

Number of Member States using SME test: 21 MS

2015: "Of the 24 responding countries, 20 countries reported that they already had an SME-test in place"

EU MSs reporting about their SME test
http://ec.europa.eu/DocsRoom/documents/19261

Increased EU-wide publicity of the European Enterprise Awards with media publications/clippings in all Member States

 Number of media publications/clippings in all Member States: 60 in 2010

Number of media publications/clippings in all Member States: 80

2014: 6
2015: 15

Meltwater (maybe they have the data internally)

Reduction of start-up time and complexity for new enterprises

Reduction of start-up time: 7 working days

Reduction of start-up time: 5 working days

In 2015, the average time to start a private limited company in the EU was 3.4 days and the cost was EUR 315. The reduction in average time is mostly due to simplifications implemented in Austria, Bulgaria and Sweden. The slight augmentation of average cost is mostly due to increasing costs in Hungary

http://ec.europa.eu/growth/smes/promoting-entrepreneurship/advice-opportunities/start-up-procedures_en

New business concepts

Number of new products/services in the market

So far, this activity was restricted to analytical work of limited scale.

Target for the cumulative number of new products/services to be 5 in 2017 (increasing to 15 in 2018 and 25 in 2019).

Design-based consumer goods?
2015: EUR 11.2 Mio budget, 27 proposals, 4 awarded projects

2016: EUR 4 Mio, 26 proposals, 6 awarded projects

Call COSME Design Based Consumer Goods - Main outcomes & 10 success stories (EASME presentation)

Level of additional exports and corresponding monetary amounts

 

As for exports, no expected impact in 2017 yet. The share of exports of the first generation of participating SMEs will appear in 2018 with a target increase of 20%.

N/A

N/A

Feedback from stakeholders on quality and value-added of activities

 

At least 70% of SMEs participating in 2014 shall express a positive impact on their turn over in a survey done end 2017

Design-based consumer goods?
2015: "The stage of implementation does not allow measuring other relevant indicators"

monitoring report

Tourism

Number of applications to funding

Number of applications to funding (to all calls for proposals) in total: around 75 per year (average for 2011)

Number of applications to funding (to all calls for proposals) in total: more than 100 per year

2014: n/a
2015: 216

monitoring report

Percentage of SMEs (and trend) in applications for tourism-related funding opportunities

Up to date, no calls for proposals were directly addressed to SMEs

30% of calls for proposals directly addressed to/involving SMEs in partnerships

Most of calls for proposals (except ad-hoc grants) required the involvement of SMEs in the projects consortia.

SMEs account for around 150 out of 377 organisations which were beneficiaries of the open calls for proposals in the field of tourism, which stand for roughly 40% of all beneficiaries.

(Out of the SMEs beneficiaries around 100 companies were active in the tourism sector and additional 50 private enterprises were active in other sectors)

EASME/ COSME data hub

Number of destinations adopting the sustainable tourism development models promoted by the European Destinations of Excellence

Number of European Destinations of Excellence awarded in total 98 (on average 20 per year – in 2007- 10, in 2008-20, in 2009-22, in 2010-25, in 2011-21, 2013 - 19)

158 winners and up to 500 runners-up destinations adopting the sustainable tourism development models promoted by the European Destinations of Excellence initiative.

2014: new 18 destinations selected
2015: 13 projects on promotion of the EDEN destinations awarded

2016: new 17 destinations selected

2017: up to 17 projects on promotion of the EDEN destinations to be awarded

EASME

Support for entrepreneurship

Feedback on the public perception of entrepreneurship (% of EU citizens that would like to be self-employed as measured by Eurobarometer)

Figures from 2007 and 2009 are stable at 45%

Increase of EU citizens that would like to be self-employed to 50%

Most recent Eurobarometer on entrepreneurship is 2012
2012: 37% of EU27 would prefer to be self-employed

32% of EU27 find it desirable (fairly + very) to become self-employed within the next five years

53% of EU27 have a broadly favourable opinion of entrepreneurs

Flash Eurobarometer 354 (2012) (volume AA)
http://data.europa.eu/euodp/en/data/dataset/S1024_354

Number of states implementing entrepreneurship solutions developed at EU level: 22 (2010)

Number of nationally run programmes available to SMEs from other MS: 5

Number of nationally run programmes available to SMEs from other MS: 10

Entrepreneurship Education?
2015: "to early to assess how many countries participated in the activities of the network"

monitoring report

Number of simplification measures adopted for SMEs

5 simplification measures per year (2010).

About 7 simplification measures per year

No information available

N/A

Financial Instruments for growth

Number of firms receiving loan (credit) guarantees and value of lending

Proposed instruments are not yet launched and different from current instruments, so data from current instruments may not be comparable

Number of firms receiving loan (credit) guarantees (+/-95 000) and value of lending (+/- EUR 10.7 billon)

2014: 3 operations signed, EUR 851 Mio of financing to around 13,000 SMEs (leverage: 1 to 10)
2015 (aggregate): EUR 1,292 Mio of financing to more than 51,000 SMEs

2015 (expected): 24 operations, EUR 6,200 Mio of financing to around 95,000 SMEs (leverage: 1 to 30)

monitoring report

Number of VC-backed firms and value of investments (of which cross border deals)

 

Number of VC-backed firms: (+/- 180) and value of investments (+/- EUR 220m)

2014 (expected for the lifetime of the programme): EUR 2,6 to 4 billion, risk capital to 362 to 544 firms
2015: 5 operations, EUR 395 Mio, risk capital to 53 SMEs

monitoring report

Enterprise Europe Network

Number of partnership agreements signed

Partnership agreements signed: 1.950 (2010)

Partnership agreements signed: 3.000/year

2013-2014: 5,273 SMEs
2015-2016: 6.223 SMEs

(2 years periods)

monitoring report

Increased recognized Network brand and brand Culture (e.g. brand awareness among SME population)

Not measured yet

30% of SMES reached

2014: 411,000 SMEs involved in promotion and information local events
2015: 8% recognition among SME population (Eurobarometer survey n°421)

2015: 10,5 mio SMEs reached via (digital) information services

monitoring report

Clients satisfaction rate (% SMEs stating satisfaction, added-value of specific service)

Clients satisfaction rate (% SMEs stating satisfaction, added-value of specific service): 78%

Clients satisfaction rate (% SMEs stating satisfaction, added-value of specific service): >80%

2014: 85%
2016 : 86% of respondents declared to be satisfied or very satisfied with EEN services

93% would recommend it to others

monitoring report

Number of SMEs receiving support services

Number of SMEs receiving support services: 435.000 (2010)

Number of SMEs receiving support services 250.000/year (revised indicator)

2014: 132,668 SMEs
2015-2016: 254.057 SMEs/Y

monitoring report

Number of SMEs participating in brokerage events and company missions

Number of SMEs participating in brokerage events and company missions: 45.000 (2010)

Number of SMEs participating in brokerage events and company missions: 60.000/year

2013-2014: 50,719 SMEs
2015
-2016: 45.953 SMEs with 156.408 meetings

monitoring report

SME business support in markets outside the EU

Share (%) of SMEs involved in international activities (exports, imports, FDI and other activities) outside the EU)

13 % (2009)

17 % (2017)

2015:
- Exported: 20%

- Imported: 19%

- Subcontractor based abroad: 7%

- Worked as a subcontractor: 5%

- Worked for R&D with a partner abroad: 4%

- Invested in a company abroad: 2%

Flash Eurobarometer 421 (2015)

International Industrial Cooperation

Number of cases of improved alignment between EU and third countries’ regulations for industrial products

It is estimated that in regulatory co-operation with main trading partners (US, Japan, China, Brazil, Russia, Canada, India) there is an average of 2 relevant areas of significant alignment of technical regulations

3 relevant areas of significant alignment of technical regulations with main trading partners (US, Japan, China, Brazil, Russia, Canada, India) (2017)

No information available

N/A

Number of areas and good practices of the EU Small Business Act which have been introduced in neighbourhood and candidate countries

It is estimated that on average in the three policy region (candidate countries region, neighbourhood East and neighbourhood MED) of the 10 policy areas of the SBA at least 3 of those have been regulated in these countries.

5 policy areas of the SBA in the three policy region
(candidate countries region, neighbourhood East and

neighbourhood MED) (2017)

No information available

N/A

Source: Techonopolis

Appendix B- Statistical data analyses results

2.1 Analysis of the geographical distribution

The action lines considered in the analysis, for which data on final beneficiaries are available, are:

·LGF

·EFG

·EEN

·EYE

·Cluster Internationalisation

·Tourism

Table 33 below presents the penetration rate by country for the action line LGF. The penetration rate measures the percentage of SMEs, employees and SME turnover reached by LGF compared to total number of SMEs, employees and SME turnover in the country.

Table 33 Country penetration rates of LGF guarantee 2014-2016

LGF SMEs (all)

Penetration rates… based on:

Country

Total number of SMEs

Total number of employees (First Inclusion)

Total value of SME Turnover (First inclusion) (in EUR)

Average age

number of SMEs

number of employees

SME Turnover

Austria

1,176

8,110

907,520,155

7.6

0.40%

0.40%

0.00%

Belgium

471

4,062

955,471,071

5.1

0.10%

0.20%

0.00%

Bulgaria

660

12,797

949,484,069

9.1

0.20%

0.90%

0.01%

Czech Republic

1,812

16,371

2,459,727,894

11.3

0.20%

0.70%

0.01%

Denmark

209

840

6,404,467

6.9

0.10%

0.10%

0.00%

Estonia

255

4,370

576,987,811

10.4

0.40%

1.30%

0.01%

France

51,047

155,539

8,577,346,400

9.5

1.60%

1.60%

0.00%

Germany

8,840

25,475

3,568,813,371

2.3

0.40%

0.10%

0.00%

Greece

322

3,972

523,598,067

16.1

0.00%

0.20%

0.00%

Hungary

1,475

10,290

741,963,920

11.5

0.30%

0.60%

0.00%

Italy

28,518

142,168

19,111,421,957

13.3

0.80%

1.30%

0.00%

Latvia

67

1,453

111,387,715

12.8

0.10%

0.30%

0.00%

Lithuania

71

3,770

253,300,460

13.2

0.00%

0.50%

0.00%

Montenegro

521

3,563

191,344,923

10

1.60%

3.80%

0.02%

Netherlands

440

916

50,546,318

5.3

0.00%

0.00%

0.00%

Poland

3,351

24,381

2,325,042,247

7.4

0.20%

0.40%

0.00%

Romania

642

6,665

408,234,103

10.8

0.10%

0.30%

0.00%

Slovakia

146

521

31,315,673

8.7

0.00%

0.00%

0.00%

Slovenia

392

8,767

723,330,826

15.6

0.30%

2.10%

0.01%

Spain

40,166

102,973

7,287,511,069

6.8

1.70%

1.30%

0.00%

United Kingdom

2,763

19,041

2,204,655,663

8.6

0.20%

0.20%

0.00%

Total

143,344

556,044

51,965,408,179

9

0.60%

0.70%

0.00%

Source: Technopolis, based on EIF monitoring data at end of 2016 and National statistics 2016 (2017)

Table 34 presents, for each action line analysed, the shares of beneficiaries belonging to a country aggregation (EU15, EU13, COSME 3rd countries and Other countries) over the total number of beneficiaries reached by that thematic area.

Table 34 Geographical distribution - shares by country aggregation of number of beneficiaries for 2014-2016

Action line

EU15

EU13

3rd countries

Other

Access to finance / LGF

80.9%

49.3%

18.6%

 

Access to finance / EFG

0.01%

 

 

 

EEN

11.1%

40.7%

73.5%

100.0%

EYE

2.4%

7.6%

7.7%

 

Cluster internationalisation

5.4%

1.8%

 

 

Tourism grants

0.2%

0.5%

0.2%

 

Grand total

100%

100%

100%

100%

Source: for LGF and EFG: EIF quarterly report Q4/2016; for EEN and EYE: databases from COSME; for Cluster Internationalisation: European Cluster Collaboration Platform. For all the other action lines: datahub. Note: the datahub only contains information on calls for proposal.

Table 35 reports the estimated number of beneficiaries reached in each EU country by the different action lines.

Table 35 Geographical distribution – estimated number of final recipients by country in EU for 2014-2016

Country

Access to finance / LGF

Access to finance / EFG

EEN

EYE

Cluster Internationalisation

Tourism grants

AT

1,176

351

125

250

12

BE

471

590

149

586

19

BG

660

553

60

114

8

CY

0

84

84

0

2

CZ

1,812

863

82

52

3

DE

8,840

2

2,630

345

1,050

9

DK

209

198

56

914

4

EE

255

177

30

0

3

EL

322

2

815

227

112

13

ES

40,166

3,174

951

1,375

57

FI

0

231

36

140

2

FR

51,047

4

2,449

174

2,171

21

HR

0

293

76

0

9

HU

1,475

692

84

43

9

IE

0

1

219

66

0

7

IT

28,518

1

2,011

1,025

879

80

LT

71

406

135

25

5

LU

0

19

9

0

1

LV

67

351

74

0

8

MT

0

67

10

0

6

NL

440

1

1,275

199

440

10

PL

3,351

2,013

246

88

2

PT

0

450

142

254

11

RO

642

1,025

253

0

5

SE

0

558

75

480

1

SI

392

413

126

0

25

SK

146

388

114

0

2

UK

2,763

1

3,392

393

301

18

Total

142,823

12

25,687

5,346

9,274

352

Source: for LGF and EFG : EIF quarterly report Q4/2016; for EEN and EYE: databases from COSME; for Cluster Internationalisation: European Cluster Collaboration Platform. For all the other action lines: datahub. Note: the datahub only contains information on calls for proposal.

Table 36 provides the percentages of beneficiaries reached by each action line, weighted by the number of SMEs in each country.

Table 36 Geographical distribution – weighted shares of participants by country in EU for 2014-2016

Country

Access to finance / LGF

Access to finance / EFG

EEN

EYE

Cluster Internationalisation

Tourism grants

AT

0.4%

0.0%

0.1%

0.0%

0.1%

0.0%

BE

0.1%

0.0%

0.1%

0.0%

0.1%

0.0%

BG

0.2%

0.0%

0.2%

0.0%

0.0%

0.0%

CY

0.0%

0.0%

0.2%

0.2%

0.0%

0.0%

CZ

0.2%

0.0%

0.1%

0.0%

0.0%

0.0%

DE

0.4%

0.0%

0.1%

0.0%

0.0%

0.0%

DK

0.1%

0.0%

0.1%

0.0%

0.4%

0.0%

EE

0.4%

0.0%

0.3%

0.0%

0.0%

0.0%

EL

0.0%

0.0%

0.1%

0.0%

0.0%

0.0%

ES

1.7%

0.0%

0.1%

0.0%

0.1%

0.0%

FI

0.0%

0.0%

0.1%

0.0%

0.1%

0.0%

FR

1.6%

0.0%

0.1%

0.0%

0.1%

0.0%

HR

0.0%

0.0%

0.2%

0.1%

0.0%

0.0%

HU

0.3%

0.0%

0.1%

0.0%

0.0%

0.0%

IE

0.0%

0.0%

0.1%

0.0%

0.0%

0.0%

IT

0.8%

0.0%

0.1%

0.0%

0.0%

0.0%

LT

0.0%

0.0%

0.2%

0.1%

0.0%

0.0%

LU

0.0%

0.0%

0.1%

0.0%

0.0%

0.0%

LV

0.1%

0.0%

0.4%

0.1%

0.0%

0.0%

MT

0.0%

0.0%

0.3%

0.0%

0.0%

0.0%

NL

0.0%

0.0%

0.1%

0.0%

0.0%

0.0%

PL

0.2%

0.0%

0.1%

0.0%

0.0%

0.0%

PT

0.0%

0.0%

0.1%

0.0%

0.0%

0.0%

RO

0.1%

0.0%

0.2%

0.1%

0.0%

0.0%

SE

0.0%

0.0%

0.1%

0.0%

0.1%

0.0%

SI

0.3%

0.0%

0.3%

0.1%

0.0%

0.0%

SK

0.0%

0.0%

0.1%

0.0%

0.0%

0.0%

UK

0.2%

0.0%

0.2%

0.0%

0.0%

0.0%

Total

0.6%

0.0%

0.1%

0.0%

0.0%

0.0%

Source: for LGF and EFG : EIF quarterly report Q4/2016; for EEN and EYE: databases from COSME; for Cluster Internationalisation: European Cluster Collaboration Platform. For all the other action lines: datahub. Note: the datahub only contains information on calls for proposal.

Table 37 shows the estimated shares of cumulative budget allocated for the action lines analysed, over the cumulative budget allocated in each group of countries. The cumulative budget is the budget allocated for each action line from January 2014 to December 2016.

The action line EYE is not analysed because the amount of money received by the final beneficiaries (the young entrepreneurs) depends on the country in which they work.

Table 37 Geographical distribution - shares by country aggregation of cumulative budget for 2014-2016

Action line

EU15

EU13

3rd countries

Access to finance / LGF

96.5%

96.3%

58.5%

Access to finance / EFG

1.5%

 

 

Business mgt capacity (EEN)

1.7%

3.1%

38.3%

Entrepreneurship

0.2%

0.4%

1.6%

Tourism

0.1%

0.3%

1.1%

Grand total

100.0%

100.0%

100.0%

Source: for LGF and EFG : EIF quarterly report Q4/2016; for EEN and EYE: databases from COSME; For all the other action lines: datahub. Note: the datahub only contains information on calls for proposal.

Table 38 presents the total overview of budget allocated to final beneficiaries through the four action lines LGF, EFG, Tourism and EEN, between 2014 and 2016.

Table 38 Geographical distribution (money allocated to final beneficiaries in EUR k for 2014-2016)

Country

LGF*

EFG*

Tourism

EEN

Grand total

AL

-

-

-

233

233

AM

-

-

-

144

144

AT

216,065

-

490

3,960

220,515

BE

81,172

-

583

3,719

85,474

BG

91605

-

163

2,112

93,880

CY

-

-

103

649

752

CZ

199,655

-

102

2,736

202,493

DE

552,956

36,011

322

22,676

611,965

DK

17,968

-

113

2,074

20,155

EE

73,580

-

130

510

74,220

EL

34,189

11,237

425

0

45,851

ES

1,011,408

-

1 421

14,125

10,255,33

FI

-

-

55

1,784

1,839

FR

1,274,179

81,52

668

15,821

1,298,820

HR

-

-

303

1,157

1,460

HU

49,878

-

345

2,711

52,934

IE

-

1,000

225

0

1,225

IS

-

-

53

527

580

IT

1,532,541

1,250

2 197

17,373

1,551,164

LT

12,226

-

201

898

1,099

LU

-

-

16

705

721

LV

6,177

-

169

649

6,995

ME

11,012

-

-

179

11,191

MK

-

-

43

273

316

MT

-

-

205

0

205

NL

17,318

3,000

370

0

20,688

PL

138,928

-

99

8,285

147,312

PT

-

-

204

2,844

3,048

RO

42,545

-

86

2,177

44,808

RS

-

-

-

482

482

SE

-

-

38

2,959

2,997

SI

106,561

-

590

1,161

108,312

SK

3,318

-

28

1,170

4,516

TR

-

-

85

5,578

5,663

UK

74,050

3,364

439

0

77,853

Grand Total

5,547,173

64,014

6,653

119,671

5,725,443

Source: for LGF and EFG - EIF quarterly report Q4/2016; for Entrepreneurship education - EYE database; for all the other action lines - datahub. Note: the datahub only contains information on calls for proposal. * The figures presented in the table for LGF and EFG refers to investments into eligible final beneficiaries (financing transactions).

2.2 - Analysis of the sectoral involvement

This section provides the distribution by sector of the number of beneficiaries reached and budget allocated to final beneficiaries, for the action lines LGF, EFG, EEN, EYE, Cluster Internationalisation and Tourism grants.

Table 39 and Table 40 present the breakdown of, respectively, the estimated number and the shares of final beneficiaries reached, by the principal NACE sectors.

Table 39 Sectoral involvement (estimated shares of cumulative budget 2014-2016)

Sector

Description

Access to finance / LGF

Access to finance / EFG

EEN

EYE

Cluster internationalisation

Tourism grants

Grand total

A

Agriculture, forestry and fishing

5,874

1

508

172

1,291

2

7,848

C

Manufacturing

16,564

2

8,276

625

3,076

-

28,543

D

Electricity, gas, steam and air conditioning supply

128

1

2,132

69

1,444

-

3,774

E

Water supply; sewerage; waste management and remediation activities

487

-

-

26

349

-

862

F

Construction

18,057

 -

 -

647

190

-

18,894

G

Wholesale and retail trade; repair of motor vehicles and motorcycles

36,830

3

 -

133

 -

-

36,966

H

Transporting and storage

8,907

 -

 -

42

373

5

9,327

I

Accommodation and food service activities

14,666

 -

 -

381