EUROPEAN COMMISSION

Brussels, 10.8.2016

SWD(2016) 273 final

COMMISSION STAFF WORKING DOCUMENT

Evaluation

Accompanying the document

REPORT FROM THE COMMISSION TO THE COUNCIL AND THE EUROPEAN PARLIAMENT

On the implementation of Directive 2007/2/EC of March 2007 establishing an Infrastructure for Spatial Information in the European Community (INSPIRE) pursuant to article 23

{COM(2016) 478 final/2}
{SWD(2016) 243 final/2}

Evaluation of DIRECTIVE 2007/2/EC establishing an Infrastructure for Spatial Information in the European Community (INSPIRE) and underpinning the report on the implementation

Contents

1. INTRODUCTION

1.1. Purpose of the evaluation

1.2. Scope of the evaluation

2. BACKGROUND TO THE INITIATIVE

2.1. Description of the initiative

2.2. Baseline

3. PROCESS, METHODOLOGY AND LIMITATIONS

4. IMPLEMENTATION RESULTS

5. EVALUATION

5.1. Effectiveness

5.1.1. Effectiveness of actions to establish a legal framework for sharing spatial data across the EU (Specific Objective 1)

5.1.2. Effectiveness of actions to establish Member States internal and EU level coordination (Specific Objective 2)

5.1.3. Effectiveness of actions to identify the spatial data sets (Specific Objective 3)

5.1.4. Effectiveness of actions to document the identified spatial data sets (Specific Objective 4)

5.1.5.Effectiveness of actions to make the documented spatial data accessible online (Specific Objective 5)

5.1.6. Effectiveness of actions to bring spatial data sets organised in interoperable data models online (Specific Objective 6)

5.1.7. Summary of the evaluation of effectiveness

5.2. Efficiency

5.2.1. Costs and benefits

5.2.2. Comparison to the ex-ante evaluation

5.2.3. Cost-effectiveness of specific provisions

5.2.4. Administrative burden

5.2.5. Available resources

5.3. Relevance

5.3.1. Need for EU action

5.3.2. Relevance of objectives and actions

5.4. Coherence

5.4.1. Coherence within the environmental policy domain

5.4.2. Coherence within the other policy domains

5.5. EU Added Value

6. CONCLUSIONS

1.INTRODUCTION

1.1.Purpose of the evaluation

The Directive establishing an Infrastructure for Spatial Information in the European Community, INSPIRE Directive 1 came into force 15 May 2007. It is implemented in various steps with full implementation expected by 21 October 2020. The actions envisaged by this instrument aim at removing obstacles hampering the sharing of data between and across all levels of government According to Article 23 of the Directive, the Commission had to present a report on the implementation of INSPIRE to the European Parliament and to the Council by 15 May 2014 2 . In addition, this Commission Staff Working Document (SWD) provides the evaluation identified in the Regulatory Fitness and Performance (REFIT) 3 programme in 2013 4 . It aims at assessing whether the INSPIRE Directive is still fit-for-purpose at the half way mark of its implementation.

1.2.Scope of the evaluation

The evaluation is looks at the status of implementation and performance of the INSPIRE Directive broadly in line with the Better Regulation Guidelines 5 .

To this end, the SWD presents objectives, framework and methodology of the policy evaluation. It recalls the policy context and objectives of the Directive, including its intervention logic. Based on an analysis of the evidence collected, it answers the key evaluation questions on assessing the effectiveness, efficiency, relevance and coherence as well as the EU added value of the objectives and actions pursued.

2.BACKGROUND TO THE INITIATIVE

2.1.Description of the initiative

Developing and implementing EU environment policy is dependent on a solid knowledge and evidence base from a large number of areas, not only in air, water or nature but also in all economic sectors which influence the state of the environment.

Since 2002, the EU Sustainable Development Strategies 6 , 7 and the 6th Environmental Action Programmes 8 addressed these issues and identified the need for better information to support the integrated 9 knowledge-base for environmental policies. As a result, the Commission came forward with a proposal for the INSPIRE Directive in 2004 10 focussing mainly on 'spatial data', i.e. information related to a location or area on Earth 11 .

A wide range of environmental, geographical, social and economic spatial data exists or is being collected. Such location-based information is relevant for environmental policies, covering many thematic areas (water, air, biodiversity, waste, emissions, impact assessments, natural and technological hazards, public access to environmental information, etc.). Also policies having an impact on the environment (such as transport, agriculture, energy, land-use planning, regional development, etc.) systematically collect spatial data which are valuable in the context of environment policy development and implementation.

The INSPIRE Directive was designed to address specifically the thematic environmental policies needs for spatial data by removing the four major obstacles to the availability of such data:

1.A wide variety of organisational, cultural, institutional, financial and legal obstacles hampered the sharing and re-use of spatial data by public authorities and public stakeholders.

2.Spatial data was difficult to find online on the Internet and it was poorly or not documented.

3.Many public authorities did not have online services in place allowing others, including the public to discover, to access and use their spatial data.

4.Spatial data was often organised in incompatible formats making it difficult to combine different spatial data sets in the absence of a common vocabulary.

The intervention logic for the INSPIRE Directive (Figure 1) sets out the different measures of the Directive and how they were expected to interact.

The general objective 14  of INSPIRE Directive is the establishment of an EU infrastructure for spatial information based on compatible Member States infrastructures and useable in an EU and trans-boundary context 'for the purposes of EU environmental policies and policies or activities which may have an impact on the environment'.

In order to achieve this, the INSPIRE Directive pursues the following six specific objectives:

1.To establish a coherent legal framework for sharing spatial data across the EU.

2.To put in place coordination structures at Member States and EU level.

3.To identify the spatial data needed 15 .

4.To document the identified spatial data through metadata 16 .

5.To ensure that the documented spatial data is accessible online through information technology, , IT services allowing its discovery, view and download and, where needed transformation.

6.To organise the documented spatial data in interoperable data models with a common vocabulary and online accessible through the IT services.

These objectives are translated into 14 'actions', which are the specific provisions of the INSPIRE Directive with obligations mainly on Member States but also on EU institutions. Those actions should be implemented according to an agreed timetable, in several ‘steps’ as laid down in the INSPIRE Directive and its implementing rules 17 . Overall, the actions can be combined to five distinctive steps. The first one is related to actions of legal and administrative nature such as transposition, the establishment of coordination structures and the establishment of a data policy. Steps 2-5 relate to a sequential set of actions all related to ‘spatial data’. They have to be identified (step 2) and documented (step 3). Thereafter, online services (step 4) have to be established and finally the spatial data should be transformed (step 5) in accordance with agreed data models in order to facilitate the re-use of the data. Since these actions have to be taken for different data themes 18 (see Annex I, II and III of the Directive) at different times, the illustration of the specific steps, actions and outputs can be somewhat complex. Hence, a simplified overview of main implementation steps and outputs is presented in figure 1 and a more detailed overview is provided in the box below and in Annex 1, Part A. At the time of this evaluation, only four out of the five steps had to be completed and the fifth was still in the future (2020) covering a large volume of spatial data.

Steps and actions required by the INSPIRE Directive (see details in Annex 1):

The generic intervention logic described in Figure 1 is the basis for the evaluation where the following elements have been assessed: effectiveness, efficiency, relevance, coherence and EU added value. A detailed intervention logic is provided in Annex 1, Part B. Before assessing these elements, the status of implementation in relation to the actions which were already required before 2014 is described mainly on the basis of the Member State reports from 2013 or later where additional data existed.

Figure 1: INSPIRE Implementation major milestones and outputs

2.2.Baseline

During the preparation and transposition phases of the INSPIRE Directive (between 2004 and 2010), the Commission conducted yearly state-of-play studies 30 , covering 27 EU, four EFTA and one Candidate countries. The studies assessed the state and the rate of progress made by those countries in relation to the identified obstacles.

The state-of-play summary report 31 of December 2007 is used for establishing a baseline for the national spatial data infrastructures as it reflects the situation at the time of entry into force of the Directive.

Some key findings of the report, for which 19 EU and two EFTA countries provided information, are summarised below:

With regard to specific objective 1 - A coherent legal framework for sharing spatial data

At the time, spatial data policies were highly heterogeneous or missing across the EU. They were not only different between countries but in some cases also between regions in the same country. Different data policies were applied for different types of spatial data. There were no or few activities to better align data policies across Member State borders.

The main differences existed for the restrictions that can be made on data sharing and the charges that are required for the provision of data sets and services by the Member States.

For example, seven Member States 32 had different terms and conditions for the public sector including different pricing policies or data that was only available for other public authorities. Five Member States 33 did not have special terms for the public sector but applied the same conditions as for data requests from the private sector. Only in three Member States 34 a coordination body addressing issues related to data policies was in place.

Three Member States 35 saw a possibility to incorporate data sharing arrangements in new legislation and four countries 36 considered harmonising the terms and conditions for spatial data sharing using a single model licence.

With regard to specific objective 2 - Coordination structures at Member States and EU level

The report showed that coordination structures similar to those required by the INSPIRE Directive started to emerge in some Member States. They were, however, operational to varying degrees and with different levels of user involvement. In nine Member States 37 cross-departmental structures were being set-up but not all of them had a formal mandate yet.

With regard to specific objective 3 - The spatial data needed for policies is identified

The definition of spatial data is broad ("data with a direct or indirect reference to a specific location or geographical area", Article 3.2) and the scope of the Directive is wide (cf. Article 4.1). Hence, it is difficult to identify a baseline and benchmark success in relation to the identification of spatial data sets given the difficulties in identifying an upper limit or total number of datasets falling within scope of the Directive. Having said this, it was possible to establish a baseline which allows, at least, analysing relative progress.

In 2007, 21 countries 38 had identified 1635 data sets covering all 34 different themes of the INSPIRE Directive's annexes I, II and III (INSPIRE data themes). Out of these, 1384 were unique data sets from 311 stakeholders and 738 network services from 231 organisations. There were considerable differences between countries regarding the coverage of the 34 INSPIRE data themes.

For example, nine out of the 21 countries did not identify data sets for more than ten of the 34 data themes as illustrated in Figure 2. In particular the identification of data sets in the themes listed in Annex III (which includes the data most relevant for environment policies) was very limited as illustrated in Figure 3.

Figure 2: Number of data themes covered in the annexes of the INSPIRE Directive without data sets reported - Source: Spatial Data Infrastructures in Europe: State of Play 2007

Figure 3: Number of countries for which no data set is reported for the theme listed in Annex III of the INSPIRE Directive - Source: Spatial Data Infrastructures in Europe: State of Play 2007

With regard to specific objective 4 - The identified spatial data is documented

The documentation of data sets through ‘metadata’ was in the early stages. Only a small part of the spatial data was documented. For the some 1635 identified data sets and services, 66.7% were reported to have some documentation. If metadata were available, they were highly heterogeneous and mostly not following international standards or were not harmonised in content. Of the identified datasets, only 35.5% met international standards (i.e. EN ISO 19115).

There were important differences between the countries as illustrated in Figure 4.

Figure 4: % of the data sets and services for which ISO (in accordance to standard EN ISO 19115) metadata exist - Source: Spatial Data Infrastructures in Europe: State of Play 2007

With regard to specific objective 5 - The documented spatial data is accessible online

The documentation is a pre-requisite for developing online services. Therefore, it is not surprising that IT services to discover spatial data, view and, in particular, to download them were only partially available. For the 21 countries for which data were available, only 55.7% identified data sets could be discovered online, 49.2% could be viewed and only 27.2% could be downloaded (or parts thereof).

With regard to specific objective 6 - Interoperable spatial data

The 2007 report showed increased activities in data standardization in general. Some organisations already used harmonised data exchange formats within their domains (such as meteorological services, marine hydrographic offices, statistical offices, space agencies, some mapping agencies). However, there was no agreed initiative within the EU or beyond which systematically addressed interoperability across spatial data sets and services. A few countries, organisations and projects reported initial strategies and actions in this area 39 . However, a cross-thematic semantic interoperability framework for services and spatial data, with unique identifiers, typologies, agreed data structures and content encoding did not exist.

3.PROCESS, METHODOLOGY AND LIMITATIONS

All Member States reported on the implementation of the INSPIRE Directive in accordance with Article 21.3 (deadline: May 2013) 40 . At the same time, the Commission decided to combine the preparation of the implementation report (in accordance with Article 23) with an evaluation conforming to the approaches laid down by European Commission on policy evaluations at the time. Later that year, the INSPIRE evaluation was formally included in the Regulatory Fitness and Performance Programme (REFIT) programme 41 .

Following the latest Commission guidelines 42 , the evaluation addresses the effectiveness, efficiency, coherence, relevance and EU added value of the INSPIRE Directive at its half way implementation mark. The evaluation questions are listed in Annex 2.

An overview on the process to prepare this REFIT evaluation is provided in Annex 3. A Steering Group composed of the Commission’s DG Environment and DG Joint Research Centre (JRC) as well as the European Environment Agency (EEA) was set up and other Commission services were regularly informed, e.g. through the established Commission Inter-Service Group COGI 43 (Commission Inter-Service Group on Geographical Information), chaired by Eurostat.

Moreover, consultation with the Member States took place through the INSPIRE National Contact Points (NCP) 44  and the established Expert Groups supporting the INSPIRE implementation.

As first steps, a review of the INSPIRE relevant documents 45 and the INSPIRE library 46 (e.g. the annual State-of-Play studies and Member State monitoring and implementation reports) and the annual INSPIRE conferences archive 47 took place as desk research. Moreover, an open public internet consultation 48 took place (from 2/12/13 - 24/2/14, Annex 4) and a study contracted 49 with a focus on analysing in more detail some INSPIRE services reported by the Member States.

The results of all this work was published as an official EEA/JRC Technical report 50 in November 2014 and therefore largely responding to the substantial requirements set out in Article 23. This report is a key information source for this policy evaluation as its findings are based on the INSPIRE relevant documentation sources. Where relevant and necessary, this evaluation SWD is complemented with evidence from other documented sources. Every effort was made to follow the new Guidelines to the widest extent possible in the finalisation of the evaluation. However, since the substantial work had already been completed by then and no new data had been collected since, this evaluation could not fully anticipate all the methodological and data needs now enshrined in the Guidelines (see more details in Annex 5).

Moreover, the following limitations should also be taken into account when reviewing this evaluation:

The 3-yearly country reports improved in quality between 2010 and 2013. Despite differences in the level of detail, the majority of the reports can be considered as a good basis for comparison. The most important limitations regard the findings on cost-efficiency, where despite the availability of a methodological guidelines 51 and preparatory workshop 52 , Member States reported that cost figures are difficult to obtain and compare. On benefits, most countries report in qualitative terms, with the general observation that INSPIRE is not yet sufficiently implemented to assess benefits in quantitative terms.

The yearly monitoring indicator reports 53 , available since 2010, also improved considerably in quality. However, issues on completeness and interpretation (for example on what data set should be reported under which INSPIRE data theme) remain an issue. In addition, the quantitative indicators 54 on availability and conformity of data sets and services were not collected online yet because the infrastructure and IT tools were not in place in time. The online service is now available and will facilitate information gathering, processing and comparison of data.

Information on the use of the data and services is provided in the 3-yearly reports. Despite the detailed guidance in the template51 for country reports, the information on the use of infrastructure for policy purposes is highly variable from one country to another. This depends largely on the state-of-play of the implementation of the IT services and actual use in applications.

The quantitative indicators extracted from the EU Geo-portal 55 , provide another comparable quantitative basis. However, as shown in the assessment study 56 , there are discrepancies between the yearly monitoring reported data sets and services and those made available through the EU Geo-portal. Also here, different interpretations by those entering the metadata on what needs to be catalogued under which INSPIRE data theme, make it sometimes difficult to compare between countries.

The findings from the public consultation 57 in 2013 suffered to certain extent from a lack of geographical and thematic comparability (with a relatively low participation 58 of some countries and/or on certain data themes). 

Despite these issues, it was possible to present a substantial analysis of the implementation of the INSPIRE Directive based on data until 2014, i.e. the national reports of 2013 and the annual reports of 2014, and to identify some strengths, weaknesses, opportunities and challenges.

4.IMPLEMENTATION RESULTS

The INSPIRE Directive is implemented through the steps and actions related to the six specific objectives presented in section 2.1.

This section summarises the current situation and progress made towards reaching these specific objectives mainly based on the national reports provided by the Member States.

To this end, the Commission adopted on 5/6/2009 the implementing rule 59 for reporting and monitoring by 15/5/2010 which specified detail and content. On this basis, Member States sent their first national reports in 2010 60 . Since then, the Member States also provided annual monitoring reports. This evaluation considered the first (2010) and second (2013) national reports as well as the annual monitoring reports until 2014. The large majority of the Member States was able to provide all these pieces of information in accordance with the Commission Decision85 and in a timely manner (Figure 5).

Figure 5: Monitoring and reporting by countries (EU + EEA/EFTA) (White fields indicate that no report was provided).

Action (step 1): Member States had to transpose INSPIRE in national law by 15/5/2009.

Only one Member State (Denmark) completed this task on time. 16 Member States finalised their transposition within a year, and 3 Member States of the remaining 10 needed almost two years before communicating their laws (see table 1). Croatia became the 28th Member State of the EU on 1 July 2013 and notified transposition of the INSPIRE Directive on time (i.e. in May 2013). As main reasons for the delays, Member States informed of political (e.g. change of government following elections), legal (e.g. constitutional requirements to transpose in parliamentary processes at national and regional level) and administrative (e.g. extended consultation procedures or delays in the administrative processes) delays.

Table 1: Overview of delays in notification of transposition by Member States

Note: *HR only had to notify transposition at their date of accession and did so on time (notwithstanding actual conformity with the Directive).

The Commission supported the transposition process through two dedicated workshops and assessed conformity of transposition after the formal notification of each Member State. 26 cases of non-conform transposition 61 were addressed by means of a structured dialogue (EU Pilot 62 ) to resolve the underlying issues.

By the end of 2015, eight EU Pilots (CZ, DE, FI, FR, LT, PL, PT, UK) were still open where specific provisions in the INSPIRE Directive seem to be missing in the domestic laws, and/or where they seem to have been incorrectly, or incompletely transposed. There is currently one Member State (Finland (Aaland)) addressed by a Letter of Formal Notice focusing on the legal framework for data accessibility in an INSPIRE compliant manner.

The non-conformity resulted from different problems with diverging levels of complexity including availability of services and data for the purposes of public access and data-sharing between authorities as well as the incomplete or incorrect transposition of annexes. In the majority of cases the EU Pilot process gave the opportunity to have a constructive dialogue where the outcome in most of the cases involved amending legislation to ensure the necessary conformity. The outstanding issues regarding conformity concern these legal or technical issues.

Action (step 1): Member States should have adopted by 15/5/2009 measures for the sharing of spatial data sets and services precluding any restrictions likely to create practical obstacles occurring at the point of use, to the sharing of spatial data sets and services between public authorities.

This action requires the correct transposition of Article 17 of the Directive and these issues were identified in some of the non-conformity cases (see above). Moreover, the review, and where necessary revision (in case of practical obstacles), of the data policies of those organisations holding the spatial data sets is essential to apply this provision correctly.

In addition to the Directive, the Commission adopted a Commission Regulation 63 regarding the access to spatial data sets and services of the Member States by EU institutions and bodies under harmonized conditions that entered into force on 19/10/2011. This Regulation sets out that metadata must include the conditions applying to access and use for EU institutions and bodies and, in particular:

Member States are requested to provide access to spatial data sets and services without delay and at the latest within 20 days after receipt of a written request;

Mutual agreements may allow an extension of this standard deadline.

If data or services can be accessed under payment, Community institutions and bodies have the possibility to request Member States to provide information on how charges have been calculated.

While fully safe-guarding the right of Member States to limit sharing when this would compromise the course of justice, public security, national defence or international relations Member States are encouraged to find the means to still give access to sensitive data under restricted conditions, (e.g. providing generalized datasets)

Upon request, Member States should give reasons for these limitations to sharing.

However, these implementing rules only regulate access and use by EU institutions and bodies. It is mainly on national level where public authorities experience outstanding data policy obstacles which also affect the access and use at EU level. The implementing rule was complemented by guidelines for implementation and one on good practices .

According to the 2011 state-of-play report, fifteen 64 Member States out of twenty-seven declared to have a framework or policy for sharing spatial data between public authorities in place in 2010. The majority of the remaining Member States reported 'partial' frameworks. The report further noted that several countries started to take measures to improve sharing between public authorities through simplified licensing mechanisms 65 .

The EEA/JRC Technical evaluation report based on the 2013 Member States reports, confirms a positive trend, yet notices that Member States adopted heterogeneous policies between public authorities depending on the types of spatial data and levels of government involved, for example:

-Nine Member States 66 adopted legislative frameworks that include data and service policies complemented in the UK 67 and NL with a general licencing framework 68 . Ten Member States 69 have adopted or are preparing the basis for open data policies. Fifteen Member States 70 adopted for the public sector overall sharing arrangements. Ten Member States 71 have – or are integrating their data policies in broader eGovernment initiatives. 11 Member States 72 extensively use individual arrangements between the public authorities.

-However, the policies sometimes apply to a limited set of INSPIRE data themes such as for example: topographic maps, geographical names, addresses, ortho-imagery. In addition, it remains in several cases unclear if all spatial data falling under the scope of INSPIRE are covered, which arrangements apply for which data sets and if they are coherently applied by all organisations and/or regions. In a number of countries, there are different arrangements for sharing agreed between central and regional governments, different ones from region to region and within the regions again differences depending on the types of data and authorities involved.

Eight Member States 73 reported in 2013 the removal of data policy obstacles while 17 Member States 74 reported outstanding issues with regard to removing data policy obstacles (see table 2).

Table 2: Grouping of Member States according to main obstacles identified in their data policy (based on 2013 reports)

PT and MT did not report particular issues related to data policies. BG reported that, at this stage, it cannot provide a list of barriers to the sharing of spatial datasets and services between the public authorities or from the public authorities to the EU institutions and bodies.

In summary, this screening of the national data policies for sharing data demonstrated that they are highly variable and heterogeneous. However, only a more in-depth analysis of the current situation can identify the most relevant obstacles and ascertain whether the flexibility provided by the Directive (cf. Article 17.2 - 'do not create practical obstacles at the point of use') is applied correctly.

Action (step 1): The Commission had to establish an INSPIRE Regulatory Committee (Article 22) and, together with the EEA, set up coordination structures to support the implementation process through the national contact points (Article 19).

The Commission established the INSPIRE Regulatory Committee in 2008 75 . Moreover, the Commission set-up a participatory and transparent process for the involvement of stakeholders 76 in the development of the INSPIRE implementing rules adopted 77 between 2008 and 2014.

Member States identified national contacts points by 15 May 2009 as required.

The EU level coordination of stakeholder participation to the implementing rules development has been a major collaborative effort between the Commission and the Member States. Between 2007 and 2014, over 240 Member State and stakeholders experts involved in this Commission's coordinated effort produced forty detailed guidance documents, which were submitted for public reviews 78 and tests on feasibility and impacts. Over 10.000 comments have been resolved in an open and transparent participatory process, which has resulted in the step-wise adoption of ten 79 legal acts, including amendments (Commission Regulations and Decisions) by the Commission following for most, also a scrutiny review by Council and European Parliament.

The Directive sets three target dates by which implementing rules should have been adopted by the Commission. The other rules were adopted by the Commission according to the operational logic for implementing INSPIRE. For example, network services for discovery and view were adopted ten months after the meta-data regulation. Download and transformation followed twelve months later. The monitoring and reporting implementing rule was adopted one year before Member States had to produce their first implementation report.

As presented in the table below, three implementing regulations were adopted with approximately 0.5 to 1.5 years of delay in relation to the target dates in the Directive.

Table 3: Overview on the adoption process for the different Implementing rules

All implementing rules were endorsed without negative opinions or abstentions by the Member States represented in the INSPIRE Regulatory Committee and passed where required with success the scrutiny of the European Parliament and Council. The review and endorsement process included as well extensive implementation guidelines 81 , which did not go through the legislative procedure and are therefore not binding.

The delayed adoption of the implementing rules shifted the final date for the implementation of the Directive by approximately 1.5 years. This did not create additional burden on Member States as the Directive set the deadlines 82 for implementation in function of the dates of adoption of the implementing rules by the Commission.

To further support the coordinated implementation, the Commission and the European Environment Agency (EEA) set up the INSPIRE Maintenance and Implementation Framework (MIF) 83 in 2013. The MIF provides a platform for collaboration at EU level to support and facilitate the implementation process and share good practices.

The two main pillars of the MIF are a Commission expert group called INSPIRE Maintenance and Implementation Group (MIG) with representatives nominated by the INSPIRE national contact points for a technical (MIG-T) and a policy related sub-group (MIG-P). This is complemented by a pool of experts drawn from the stakeholder community through an open call. The European Parliament is kept informed and experts invited to participate.

The basis of the work of the MIG and its sub-groups is a Maintenance and Implementation Work Programme (MIWP) 84 that is based on implementation support issues submitted by INSPIRE stakeholders.

Action (step 1): Member States had to monitor through their coordination structures the implementation and use of their INSPIRE infrastructure and report to the Commission and the public in accordance with Article 21 of the Directive.

To this end, the Commission adopted on 5/6/2009 the implementing rule 85 for reporting and monitoring by 15/5/2010 which specified detail and content. On this basis, Member States sent their first national reports in 2010 86 and informed the Commission that the functioning national coordination structures, expected by 15/5/2009, became operational between 2009 and 2011 (with the exception of BE and HU). Member States implemented different structures and mechanisms for coordination depending on their institutional context (e.g. federal, non-federal, more or less regionalised).  87 , 88 , 89 In 18 Member States 90 one department was given the lead.

To coordinate the contributions of the public authority users and producers, the majority 91 of the Member States set-up a national coordination body with the participation of different ministries politically responsible at the national level for those organisations or represented directly or indirectly through their organisations. In federal/regionalised Member States 92 , also the regions and/or local authority organisations were included in the coordination structure, with the responsibility for cross-departmental 93 coordination at their level of government. In a number of Member States (e.g. UK, SK, FR, IE, DK, NL, CZ) environmental ministries are in the lead. In other countries ministries responsible for agriculture and forestry (FI), mapping and land surveying (e.g. ES, SE, ES, LT, SL, EL), public administration (PL) or information technology and communication (MT, BG) are leading the coordination effort.

A survey 94 carried out for the 2011 state-of-play report, showed that most of the 26 participating Member States put mechanisms in place to involve the user communities (Figure 6). In 23 Member States the users were represented in the coordination structures, in ten Member States specific user groups were set-up while fourteen Member States involved existing user associations. Eleven Member States set up user forums and feedback is collected through surveys and procedures for handling complaints (for example in SE 95 ). Eighteen Member States focussed mainly on the public sector with some including the academic sector, while eighteen out of twenty-five Member States also involved the private sector.

Figure 6: Mechanisms to involve users

With regard to the provision of information to the national stakeholders, most 96 Member States have established online national information websites 97 on INSPIRE with varying degrees of functionality and content. The majority 98 of the websites provide up-to-date information on the national status of implementation, action plans, events, links to EU level, national and regional spatial data portals and documentation sources.

The Member States provided also feedback on the implementation and use of the INSPIRE services by the stakeholders 99 .

For example, FR reported in 2014 5.3 billion 'hits' for INSPIRE network services compared to 2.4 billion in 2013. The Swedish Environmental Protection Agency reported in 2013 a more than 100-fold increase in request for services since their launch at the end 2011. The Flanders region (BE) reported an increase in the number of downloads from 6683 in 2010 to 30083 in 2011 by public authority users in the Flemish administration. CZ reported an increase from 589 million service requests in 2011 to 1.08 billion in 2012. FI reported 1.7 million searches through the discovery services in 2012, 475 million view service request (a 70% growth over 2011), and although not yet fully implemented, there were five times more download service requests (4 to 20 million) between 2011 and 2012. A number of countries (DE, ES, UK, EL) however experienced difficulties in coordinating reliable quantitative feedback on the use of the network services through off line procedures.

With regard to feedback on the actual use of the spatial data sets by the stakeholders for the purpose of environmental policies and/or policies which may have an impact on the environment the majority 100 of the Member States reported a wide range of examples of applications and links to portals (e.g. portals providing information on the environment 101 , INSPIRE data portals, or portals for specific thematic applications). However, only six 102 provided direct references to environmental directives or related reporting activities.

Action (step 2): According to the INSPIRE implementation roadmap (Figure 1) all digital spatial data sets falling under the 34 spatial data themes and scope of INSPIRE should have been identified, documented and made available online by the Member States through network services conform with INSPIRE by 3 December 2013.

Spatial data sets are the core of INSPIRE. Without data there is no content, and without content the infrastructure has no meaning. Spatial data sets from the 34 themes defined by INSPIRE are to be used and combined to answers particular questions in support of local, national and European environmental policies and policies or activities which may have an impact on the environment. Steps 2 to 5 (or specific objectives 3, 4, 5 and 6) include all the actions that are required in relation spatial data. These actions need to be implemented step-by-step over time to fully deliver on the objectives set out by the Directive.

As a first action (or step 2), Member States had to identify the relevant spatial data sets falling under the remit of the INSPIRE Directive (cf. Article 3.3 and 18). As such, there is no pre-defined list or upper limit of spatial data sets. Hence, spatial data are identified step-by-step and may also reduce in number over time if they are combined or re-organised (e.g. when many local data sets are combined into a new regional one).

All Member States reported the identification of some datasets at the first deadline but numbers were up from the original 1635 but still generally very low (below 10000 in 2009). They were certainly incomplete given the broad definition of spatial data and the wide scope of the Directive (cf. Article 4.1). However, it is difficult to identify a benchmark success or a target for the deadline in relation to the identification of spatial data sets given the difficulties in identifying an upper limit or total number of datasets falling within scope of the Directive.

Gradually, the Member States identified an increasing number in datasets (see table 4). The total volumes of identified and documented spatial data sets remained relatively stable and rather low (below 1000) between 2010 and 2013 (for 20 Member States less than 120 datasets per country per year - see Figure 7). Only two Member States (Germany and Spain) reported more than 1000 spatial data sets from the outset.

Figure 7: Evolution 2010-2013 – Volume of spatial data sets with metadata (logarithmic scale)

Since 2013, eight Member States (AT, DE, FR, UK, FI, PT, PL, IT) have brought a significant amount of new data sets online while the others still seem to lag behind as shown in Figure 8.

Figure 8: Total volume of spatial datasets identified by EU Member States

At EU level, this resulted in an important increase in the number of data sets identified and reported from 13.914 data sets in 2010 to 56.220 reported in 2014. However, there were important differences between countries as shown in Figure 7 and in more detail in Table 4. As mentioned above, it is underlined that an increase in datasets alone is not necessarily a measure of success. Some Member States have reduced the number of datasets simply by combining fragmented or dispersed datasets which can produce less datasets but each (new) dataset having a better quality and helping to achieve the objectives of the Directive with regards to interoperability. A more detailed analysis would be necessary to identify such trends in a quantitative manner.

Table 4: Numbers of reported documented spatial data sets 2010 to 2014

Member States have been giving initially priority to identifying the spatial data covered by the 13 spatial data themes of Annex I and II 103 of the INSPIRE Directive. This is illustrated in Figure 9 where in 2012 the proportion of Annex I,II data was still relatively high compared to data sets covered by the 21 Annex III spatial data themes.

Figure 9: % of Annex I,II versus Annex III spatial data sets in 2012

The monitoring results of 2014 for the year 2013 show for several Member States an important increase in the proportion of Annex III data sets as shown in Figure 10.

Figure 10: % of Annex I,II versus Annex III spatial data sets in 2014

This is positive evolution as several of the Annex III spatial data themes which had to be available in 2013 cover environmental data such as the location and data on sources of emissions, on the quality of air, water, soils, on the status of the biodiversity, on waste and on data for the assessments of natural risks such as floods etc. However, given the larger number of data themes and the expected higher number of spatial data sets that fall under Annex III, absolute number are still very low and fall behind expectations.

Action (step 3): By 3 December 2013, all the identified spatial data sets need to be documented conform to the implementing rule on metadata (Reg. No 1205/2008). Spatial data sets falling under annexes I and II had to be compliant already in December 2010.

Documenting identified spatial data allows it to be found online more easily. To find spatial data sets they need to be documented conform to documentation standards and content ('metadata'). The documentation should, for example, allow the users to assess the quality of the data and to be informed about the use conditions. To this end, the Commission adopted an implementing rule 104 as regards metadata on 3/12/2008.

As mentioned earlier, there has been a steady growth in the number of spatial data sets since 2010 (Table 4) and most of them were documented with some kind of metadata.

In the end of 2010, overall nine countries (on 24 countries that reported these figures) had metadata for more than 70% of the reported spatial data sets and services, whereas eleven countries were below the 50% mark (see figure 11). The situation was better for the datasets of Annex I and II, where 19 countries score more than 70%. Also it could be shown that the situation had improved drastically between 2009 and 2010 for nine countries, however, all Member States were trailing behind the deadline to a larger or lesser extent.

Since then, the situation improved step-by-step. Based on the yearly monitoring reports 105 of the Member States for the year 2012 , on average 77% of INSPIRE Annex I, 66% of INSPIRE Annex II, and 39% INSPIRE Annex III data were documented with INSPIRE metadata in 2012. In 2012 only 9 out of 27 countries had more than 90% of their Annexes I and Annex II data sets, which were placed in the infrastructure, documented with metadata.

Ultimately, all the identified data sets had to have metadata which were in conformity with the above-mentioned Implementing Regulation. In 2013, only 12 Member States had over 80% of their documentation conform. Moreover, the conformity with metadata standards for the data sets falling under data themes in Annex I and II of the Directive are generally higher than those falling under Annex III (see table 4.1 of JRC/EEA report) 106 . Hence, the full implementation of the metadata standards remains an issue in many Member States as shown in Figure 11. 

Figure 11: Evolution 2010-2013 - % of spatial data sets with conform metadata

Action (step 4): By December 2013, Member States needed to have online IT network services for discovery, view and download in place for all of their documented spatial data sets in line with the relevant implementing rules.

For the spatial data for which metadata was created, interoperable online network services needed to be implemented. Moreover, it is a prerequisite that the identification (step 2) and the documentation (step 3) have been completed for a given dataset. Hence, all the results (percentages) given below need to take account that there are already significant gaps in identifying and documenting the datasets. The implementation gaps are therefore cumulative.

The Commission adopted on 19/10/2009 107 (discovery, view) and on 08/12/2010 108 (download, transformation) the implementing rule as regards the network services complemented with technical Guidelines to support their implementation.

Depending on the type of services and spatial data sets, the Member States had to meet 4 different deadlines by which services should have been implemented:

1.Discovery and view services had to be implemented by 9 November 2011 for INSPIRE Annex I-II data and by 3 December 2013 for INSPIRE Annex III data.

2.Download and (where applicable) transformation services had to be implemented by 28 December 2012 for INSPIRE Annex I-II data and by 3 December 2013 for INSPIRE Annex III data.

In accordance with Article 15(1) of the Directive, these services had to be made accessible through the EU Geo-portal.

Based on the monitoring reports provided by the Member States for the year 2012, on average 63 % of the metadata for the spatial data sets and services were available through the discovery services. There has been a positive evolution in a number of Member States as in 2013, 80 to 100% of the documented spatial data sets had discovery services allowing them be found online as shown in Figure 12. Nevertheless, only 12 out of 28 countries had more than 90 % of the metadata for datasets compliant with the earlier steps available through the discovery services.

Figure 12: Evolution 2010-2013 - % of spatial data sets with discovery services

In 2010 on average, only 27 % of the spatial data sets were available through the view services and only eight Member States had more than 50 % of their spatial data sets available through view services. Since then, however, there has been a positive evolution noticeable as shown in Figure 13.

Figure 13: Evolution 2010-2013 - % of spatial data sets with view services

A similar trend is noticeable for download services. However, only about a quarter of the Member States have 60 to a 100% of their reported spatial data sets accessible for download with a majority below 50% as shown in Figure 14.

Figure 14: Evolution 2010-2013 - % of spatial data sets with download services

On average 41 % of the services were in conformity with the implementing rules, with 10 countries having less than 3 % of their services in conformity. In some Member States, a decrease is observed because of potentially lower number of data sets or delays in providing download services for newly identified data sets. There is no detailed analysis as regards the different progress for the data sets falling under different data themes set out in annex I, II and III.

Action (step 5): Member States needed to harmonise their data sets by transforming new data sets (newly collected or extensively restructured) into data models in line with the implementing rules on interoperability (Reg. No 1089/2010 as last amended by Reg. (EU) No 1312/2014). For existing data sets, a transitional period of seven years was foreseen, i.e. for Annex I+II data sets by 2017 and for Annex III data sets by 2020. 

Important efficiency gains are expected once spatial data is harmonised or, in other words, organised in common data models 109 . This leads to interoperability where data sets can be shared more easily across borders or disciplines which makes (end-user) applications less costly to implement. It also helps reducing duplication in data collection and promotes the broad dissemination and use of data.

To this end, the Commission adopted on 08.12.2010 110 for INSPIRE Annex I, II and on 10/12/2013 111 for Annex III the implementing rules as regards interoperability of spatial data sets and services. The implementing rule was complemented with technical guidelines to support its implementation. In addition, in the context of the Maintenance and Implementation Framework, the Commission implemented the INSPIRE Interactive Data specification site 112 which offers the INSPIRE data providers with IT tools and documentation to implement the INSPIRE data specifications. Several training packages 113 and IT tools 114 to facilitate the transformation of spatial data sets to INSPIRE have been developed and put to operational use by the public and private sector (in many cases with EU co-funding).

Depending on the type of spatial data sets, the Member States have to meet four different deadlines for transforming their existing spatial data sets in the interoperable data specifications.

1.By 23 November 2012, newly created 115 or extensively restructured INSPIRE Annex I data had to be organised according to INSPIRE data specifications.

2.By 21 October 2015, newly created or extensively restructured INSPIRE Annex II and III data had to be organised according to INSPIRE data specifications.

3.By 23 November 2017, ALL INSPIRE Annex I data should be organised according to INSPIRE data specifications.

4.By 21 October 2020, ALL the INSPIRE Annex II and III data should be organised according to INSPIRE data specifications

In this evaluation, only the first deadline can be assessed. Based on the monitoring tables provided by the Member States for the year 2012, only 14 Member States reported on their state of compliance with the INSPIRE data specifications for Annex I. The degrees of full conformity are generally low which however at this stage of the implementation does not necessarily constitute a compliance issue as such obligations only applies to spatial data sets newly created or extensively restructured since the adoption of the Directive. The percentage shown in the table below consequently illustrates the situation for all Annex I spatial data reported.

Table 5: Percentage of conformity with implementing rules for all Annex I spatial data reported in 2012

Source: EEA/JRC Technical evaluation report

Figure 15 presents the overall state-of-play of implementation for all annexes by 2013. It is not surprising that the overall percentage of documented data sets which have been harmonised is low. Most Member States also reported that they do not have any or many "new" data sets. Hence, the transformation of existing data sets was still a future action that most Member States had not started. However, despite all efforts that have been taken to address the issues of feasibility and proportionality when preparing the implementing rules in relation to this step, several Member States reported that the agreed rules are still challenging and they expect the implementation to be more time and resource consuming than they had anticipated when deciding on the rules.

Figure 15:    Evolution 2010-2013 - % of spatial data sets conform to common data models (all annexes).

In summary, progress has been made in implementation, particularly in some Member States. However, the implementation gaps in most Member States are significant. They are the result of accumulated delays in the implementation process and underline the differences in speed and quality of implementation. The findings in relation to the different implementation steps can be summed up as follows:

All Member States now have coordination structures in place, but their effectiveness is variable. In particular, coordination between national mapping agencies and environment data managers can be improved in many Member States.

The national data policies for sharing data are highly variable and heterogeneous. Many Member States reported important, outstanding data policy issues that hamper the efficient sharing of spatial data between public authorities and with the public.

Significant progress has been made as regards the spatial datasets identified and reported yearly, with close on 60 000 spatial datasets being reported in 2014 against less than 1 400 in 2007. However, for many Member States, the numbers remain low (less than 150), and with limited thematic coverage compared to some of the better performing countries.

Implementation of the documentation obligations is at an advanced stage; yet only 12 Member States had documented 80 to 100 % of their reported spatial data in line with the INSPIRE Directive by 2014.

Member States made progress on the online discovery services available for their identified and documented datasets. These services give users access to the documentation. The overall availability of digital services for viewing and downloading spatial data for further use was less advanced. This is mainly the result of the cumulative delays and shortcomings in the previous implementation steps. Also here there are noticeable differences between the Member States and the type of services concerned.

The interoperability of the spatial datasets has not advanced much mainly because the main implementation deadlines are still in the future (2017, 2020).

Based on the trends observed already in 2013 and 2014 and newer information that the is now available to the Commission services (and which are reported by the Member States since May 2016), the dynamics of the implementation has increased and good progress has been made, in particular as regards the identification of data sets (step 2) and the metadata compliance (step 3). The main challenges remain with the interoperability provisions and with the data policies.

5.EVALUATION

The progress made over time towards achieving the objectives and targets set out in INSPIRE Directive has been presented already in section 4 (implementation results). This section builds on these results and evaluates the findings in relation to the five evaluation criteria taking into account additional evidence, such as the results of the public consultation.

5.1.Effectiveness

The evaluation on effectiveness looked at mainly at comparing this progress to the initial expectations and aims at identifying the main factors have contributed to or stood in the way of achieving the specific objectives. The specific evaluation questions were:

-What progress has been made over time towards achieving the objectives and targets set out in INSPIRE in various Member States? Is the progress made in line with initial expectations and is the geographical coverage of implementation consistent?

-Which main factors have contributed to – respectively stood in the way of achieving these objectives? (for example, gaps, complexity or inconsistency in the measures or working methods of INSPIRE, the timely and coherent transposition in national legislation)

They are assessed together for each of the specific objectives.

Overall, the effectiveness was often reported in relation to the parallel national policies on open data and eGovernment. For administrative reasons (different competent authorities) and the low relevance of environmental issues in a wider policy context, many Member States kept both processes separate. Some Member States, however, including some of the more advanced, demonstrated that there was ample scope for creating ‘win-win’ approaches where the implementation of the INSPIRE Directive becomes a building block for eGovernment activities. This is also confirmed by looking at the more detailed implementation steps.

5.1.1.Effectiveness of actions to establish a legal framework for sharing spatial data across the EU (Specific Objective 1)

There are two main actions evaluated under this specific objective, the transposition and the data sharing policies.

Overall, the transposition process has been delayed for most Member States. Some countries 116 explained explicitly that the delays in the 2011 State-of-play report 117 and the 2010 Member States reports were mainly due to constitutional situation and related administrative challenges, as the INSPIRE Directive needed to be transposed at several levels of government (regional and national). No evidence has been found that the delays have been specific to the transposition of the INSPIRE Directive and delays in transposition occur also in other areas for a variety of administrative, technical, legal or political reasons. However, the delays in transposition had an impact on the setting up of the coordination structures and vice versa.

The state-of-play reports 118 showed that although the Member States already had some coordination in place in 2007, the structure, role and mandates of those coordination bodies were still under discussion at the time of the entry into force of the Directive. The most difficult issues during transposition were reaching agreement between the public administrations at all relevant levels of government (within and across borders) on the coordination structures and the measures for sharing spatial data sets and services including whether and how to grant public access.

Once the Directive was transposed, the Commission identified issues of non-conformity in most Member States (all except Greece and Croatia, see section 4). However, most of these issues could be resolved easily because they were mostly of legal and technical nature (e.g. wording). None of these issues prevented Member States from starting to implement the Directive. However, the delays in transposition and the adaptation necessary after the Commission identified non-conformity did not help the implementation process either and distracted attention away from other required action. Having said this, several Member States with late transposition and issues of non-conformity were able to implement the Directive much better than the rest and are amongst the frontrunners now (namely AT, DE, FR, UK, FI, PT, PL).

The adoption of measures for the sharing of spatial data sets and services was also affected by the transposition process.

The implementation progress showed that the national data policies for sharing data are highly variable and heterogeneous. In some cases there is evidence that it does not comply with the obligations set out in Article 17 of the Directive (i.e. Member States have to ensure that any restrictions likely to create practical obstacles occurring at the point of use, to the sharing of spatial data sets and services between public authorities are precluded) because obstacles are still reported, whether it be fees that are required or restrictions of use are in place even between public authorities. This is confirmed by the feedback from the public consultation.

The users participating to the public consultation responded as follows to the question: 'data policy is no obstacle for use'.

Table 6: Responses to the statement 'data policy is no obstacle for use'.

The variations between annexes are not very strong, clearly illustrating that a majority of users still experience data policy obstacles when wanting to use INSPIRE data

The responses of the data producers are generally consistent with user experience, albeit slightly more positive:

-Only 53% of the data producing organizations declared to have a policy allowing access with no restrictions at the point of use in accordance with Article 17(2) of the Directive.

-The arrangements for the sharing of spatial data sets and services have to be open to public authorities of other Member States (Article 17(4).) However, according to the data producers responding to the public consultation only 67% of the data policies of the organizations allow the access, exchange and use of spatial data sets and services by the public authorities of other EU Member States.

-The arrangements for the sharing of spatial data sets and services have to be open on a reciprocal and equivalent basis (Article 17(5)). However, according to the data producers responding to the public consultation only 54% of the data producers grant the EU institutions and bodies and bodies established by international agreements the same terms as the public authorities within their country.

Where there are still practical obstacles identified they range from charging disproportionate fees to technical obstacles or restricting the re-use of the data. Such diverse and heterogeneous data policies also created an additional administrative burden, since each individual data policy requires specific administration (e.g. system of collecting the fees) which a more harmonised, open data policy usually does not need.

The 2011 state-of-play report already raised the concern that the Member States were mainly focused on fulfilling the technical obligations and requirements of the INSPIRE implementing rules and not on the general purpose of facilitating data sharing through more effective arrangements and data policies.

The heterogeneity of the measures taken by the Member States and the outstanding issues seem to be the result of the significant degree of flexibility laid down in Article 17(3) of the Directive. For example, the Directive allows Member States to licence and/or to require payment from, the public authorities or EU institutions and bodies. The Directive does not require Member States explicitly to simplify their licencing frameworks. It allows Member States to charge for spatial data sets and services to ensure their quality and supply, including eventually a 'reasonable return on investment' and respecting the self-financing requirements of public authorities. Only spatial data sets and services provided by Member States to EU institutions and bodies in order to fulfil their reporting obligations under EU legislation relating to the environment are explicitly exempt of charging.

The Commission proposal included a more harmonised way for promoting "open data" and a proposal for an implementing rule to establish common licensing conditions. It did not contain the various options and flexibility on data sharing which are now present in Article 17(3). During the negotiations, additional flexibility was introduced by the co-legislator to take account of the diversity of situations and data policy cultures across the EU. The implementation process has now shown that the complexity of licensing conditions and the wide range of data policies and charging policies can still be an obstacle for effective data sharing in a large number of Member States. At the same time, several Member States have used the opportunity to review and simplify their data policy often following the "open data" approach.

In terms of external factors, a number of Member States are implementing open data policies for public sector data in general 119 and/or in the context of wider eGovernment initiatives. This seems to have influenced progress positively over time. However, such policies are not necessarily covering all spatial data covered by INSPIRE and derogations similar to those 120 referred to in Article 17(3) of the INSPIRE Directive can be applied according to the Directive on the Re-use of Public Sector Information (Directive 2003/98/EC, known as the 'PSI Directive' revised by Directive 2013/37/EU ). In other countries, complex and heterogeneous national data policies and the absence of a pan-European data policy hinder the free flow of data — a concern also recognised in the wider Digital Single Market.

Due to the variety of situations and the "burden of proof" being with the Commission that the conditions of Article 17(3) are not complied with, it is not possible to determine the lack of compliance without further in-depth analysis. However, it is evident that the current legal provisions did not ensure that data sharing has become significantly easier across the EU. To the contrary, the situation seems to be more complex than before and the consistency with other legal frameworks is also not entirely ensured.

Moreover, the lack of enforcement was reported as additional factor by two countries (EL, CY) which established a comprehensive license framework by law.

In summary, these complex and diverse data policies linked to the INSPIRE Directive still creates practical obstacles at the point of data use and needs further attention.

5.1.2.Effectiveness of actions to establish Member States internal and EU level coordination (Specific Objective 2)

As set out in section 4, the progress on the main actions under this objective was presented in relation to setting up national and EU coordination structures.

The effectiveness of the national coordination structures in place and coordinating the contributions of the users and producers varies from country to country. Some good examples exist as well as many areas for improvement. This is confirmed by the 2013 public consultation 121 which inquired about the effectiveness of the coordination efforts at national level. Figure 16 shows that there is some variation in the perception of the effectiveness of the coordination among national-level public sector organizations, local public sector ones, and the private sector on the extent to which the implementation of INSPIRE is well coordinated. As shown, the level of agreement and strong agreement to the question declines from 70% for public sector national organisations, to 44% for both local public sector, and private sector. This suggests that there is room for improvement in engaging better both local level 122 and private sector in the INSPIRE implementation. This observation is further supported as national coordination has been identified by the stakeholders in the public consultation as the second biggest issue for the implementation of the INSPIRE Directive. Another issue of concern is the coordination between environmental authorities and other authorities (such as mapping agencies) in particular in those countries where the environment authorities have not been identified as competent authority for implementing the Directive. This often led to a lack of clarification and conflicting priority setting between relevant authorities at national level, in particular those responsible for environmental data (e.g. identifying minimum required environmental datasets to be covered by INSPIRE). In Denmark, e.g., the focus was initially on Annex I and II datasets (see figure 9) and only lately extended to focus on the more environmentally-relevant datasets in Annex III (see figure 10). This was a result of an increased coordination effort in Denmark between the Agency for Data Supply and Efficiency (the Danish competent authority working on INSPIRE) and the Danish environmental authorities since 2014.

Figure 16: Responses to question on national coordination in INSPIRE public consultation by type of organisation.

The delays accrued during the transposition of the Directive may have had an influence on the effectiveness of the coordination efforts. Coordination requires a strong mandate for those organisations in charge as well as the necessary financial resources. A number of Member States (e.g. ES, PT, CY, IE, EL) referred to a lack of resources for coordination actions as a result of the economic crisis. Moreover, re-organisations driven by political changes affected coordination effectiveness during a certain period in several Member States (e.g. PT, IT, EL, BG, HU, CY).

The coordination is also not always comprehensive or it is difficult to collect relevant data to demonstrate its effectiveness. Some Member States (DE, SE) reported having focussed their coordination efforts during the reporting period mainly on the production of metadata and implementation of services. Other stated that the uptake of the services is still in an initial low phase (e.g. UK) and/or that systematic monitoring on the use of spatial data through INSPIRE services for particular applications is either not done (ES), limited to internal work processes (NL), or where they are implemented in the context of open data initiatives feedback on the actual use of data is generally not collected (e.g. AT, UK, DE).

The EU coordination required a number of actions leading to the establishment of a Committee which had to adopt of a number of implementing rules as well as the establishment of an implementation support mechanism at EU level. The adoption of the implementing rules was somewhat delayed (between 0.5 and 1.5 years) in comparison to the target dates set by the Directive but this did not have a significant negative consequence for the implementation because the Directive already foresaw an fixed period for implementation from the moment of adoption of the respective implementing rule.

The delays were generally not caused by lack of resources or circumstances external to the project. However, some of the use-cases proved to be more complex than initially anticipated. Certain technical domain standards turned out to be not sufficiently mature and needed more testing. The time to test the technical specifications, to consult the stakeholders and conduct ex-ante feasibility and benefit assessments took longer than initially estimated.

The additional coordination structures to support the implementation (Maintenance and Implementation Framework) remain in place in 2016. They work effectively and are widely appreciated across the Member States. At the same time, many Member States highlighted the need for improving EU-level coordination (the European Commission and EEA) in guiding Member States towards priorities in identifying the spatial datasets, in particular for environmental and related policies (e.g. in relation to reporting).

5.1.3.Effectiveness of actions to identify the spatial data sets (Specific Objective 3)

The main action under this objective is set out in Article 18 of the Directive which requires Member States to identify their relevant spatial data sets falling within the scope of the 34 themes defined in the INSPIRE Annexes. This action is an essential first step regarding those actions dealing with data management because the identified spatial data sets are the basis for taking the subsequence actions in a stepwise approach (see specific objectives 4, 5 and 6).

Member States followed different 123 strategies to identify their spatial data sets. Some Member States (e.g. ES, DE, FR, BE) reported initially data sets resulting from an inventory of spatial data across all 34 INSPIRE themes. Other Member States (e.g. NL, LT) had initially a more restricted, step-wise approach, focussing on a more limited thematic coverage of data sets mainly related to the INSPIRE Annex I data themes or on those which they intended to reorganise as national authentic data sets first (NL 124 ). In 2010 most of the spatial data sets (50%) were reported by only two countries 125 (Table 4). From 2012 onwards a number of Member States (DE, FR, IT, UK, PT, AT, PL) reported increasing numbers of documented spatial data sets while others had only small increases or in two cases (ES, CZ) a reduction of the number of spatial data sets due to their efforts in reorganising their data sets. At EU level this resulted in an important increase in the number of data sets identified and reported from 13.914 data sets in 2010 to 56.220 reported in 2014 with however important differences between countries (as shown in Table 4).

The volume of spatial data sets as an indicator for effective implementation should be interpreted with careful considerations on how the spatial data sets are managed or available in a Member State. A low numbers of spatial datasets identified and documented in some countries may indicate a coordination and implementation problem. However, a small increase or even a reduction in data sets can, in some cases, also indicate an improvement in the availability and usability of the data sets since they have been consolidated by bringing together many diverse data sets and sources into one or several national ones which are more harmonised. In fact, such consolidation efforts are still necessary in some Member States since the current decentralised hosting of important data sets reduces the effective use and interoperability of such data.

There is a lack of analytical evidence to explain the choices made by the Member States in identifying their spatial data sets falling under the scope of INSPIRE. However, some factors influencing this action can be derived from the evaluation:

Strategic choices and timing considerations – Several Member States focussed initially on a limited number of spatial data sets, mostly those covered by INSPIRE Annex I. A number of Member States only started to involve the environmental data user and producing organisations at a much later stage given the difference in timing between the implementation of the three INSPIRE Annexes. A number of Member States reported early (as from 2010) identified spatial data from all INSPIRE annexes whether or not metadata was already created. Others only reported spatial data sets for which metadata was created, first Annex I and only later Annex II and III.

Lack of environmental policy focus - Not sufficient focus has possibly been given to the spatial data requirements of environmental policies 126 even in countries where driven by eGovernement and/or 'open data' initiatives a larger number of INSPIRE data sets became available. These external factors – such as initiatives on open data – may have received a higher political priority than INSPIRE with its focus on mostly environment-related data.

Coordination factors - In most Member States environmental and other stakeholder ministries and agencies are reported to be part of the coordination structure. In practice however, the identification of spatial data sets for environmental policies seem to have suffered from the lack of coordination between those responsible for INSPIRE and the various thematic environmental communities, expert groups and committees which are interacting with the Commission and the EEA on reporting and implementation issues.

Timing issues - Member States possibly did not sufficiently take into account the deadline of December 2013 by which all identified and documented spatial data sets should have been fully on-line 'as–is'. They remained focussed on the 2020 deadline for full implementation whereby all identified data sets have been interoperable.

Reluctance to share – Some Member States reported a reluctance to report/identify spatial data sets for sharing by spatial data producers (thematic, different administrative levels) for various reasons (business models, lack of transparency, impact on resources, resistance to change - current data management practices).

Spatial data sets are produced and used by a large number and variety of organisations which in most cases are both producers and users 127 of their own spatial data (Figure 17) and in many cases, also use those produced by others. This complex relationship of demand and supply emphasises the importance of coordination in order to identify and bring online those spatial data sets most relevant for the user communities.

As shown in Figure 17, which illustrates user demand versus the producer supply, the user demand is generally higher for INSPIRE Annex I and II spatial data given their wide range of application going beyond environmental policies. However, also for the more diversified 21 INSPIRE Annex III spatial data themes there is an important demand for spatial data sets more directly related to environmental policies.

As to the relevancy of the spatial data sets for environmental policies and policies or activities which may have an impact on the environment, Figures 9 and 10 show that since 2013 proportionally more INSPIRE Annex III are being reported. As a significant part of the spatial data relevant for environmental policies is covered by the Annex III data themes this suggests a positive evolution. However, there are still important differences between the Member States also with regard to the thematic coverage of Annex III data and the direct relevance of the identified data for reporting and implementation of environmental legislation. Often such environmental data could not be (easily) found despite that fact that they should have been made available "as is" already. This means that even without implementing the next steps (specific objectives 4, 5 and 6), these data should at least be accessible for use.

Figure 17: User demand versus producer supply for INSPIRE spatial data (PC: public consultation of 2013, SDIC: spatial data interest communities, LMOs: legally mandated organisations)  128

5.1.4.Effectiveness of actions to document the identified spatial data sets (Specific Objective 4)

Once the spatial data sets were identified, the next action was to ensure that they are documented in accordance with the Implementing Regulation 129 as regards metadata. To assist this implementation step, technical guidelines . a metadata editor 130 and metadata validator were developed and agreed at EU level. Nevertheless, the implementation progress was mixed across all Member States (see section 4). These outcomes were confirmed by the 2013 public consultation which yields comparable results as users considered 68% - 67% - 49% of the data documented (but not necessarily conform 131 ) for INSPIRE Annex I-II-III data respectively.

Specific issues were also evident on the metadata for services where in 2012 on average only 56% of the services were documented with INSPIRE metadata, with 8 countries facing significant delays (deadline was 2009 for Annex I and II). However, as shown in Figure 18, the situation has been improving significantly during the years 2013 and 2014 with more and more Member States making discovery services available providing access to 80 – 100% of the documented metadata for spatial data sets and services.

Figure 18: Conformity of metadata for spatial data services evolution 2010 to 2014.

With regard to completing their metadata, Member States seem to have followed different approaches. Some considered it a priority to bringing data and services into the infrastructure, even when not fully documented. Others applied more stringent quality checks before doing this. Other issues related to an initial lack of training and skills to understand the documentation specifications and guidelines, the willingness, time and resources needed to create this meta-data and an initial lack of software tools for these purposes as well as for verifying the quality of the documentation. Some of these issues have been overcome in the meantime.

5.1.5.Effectiveness of actions to make the documented spatial data accessible online (Specific Objective 5)

For the spatial data for which metadata was created, interoperable online network services needed to be implemented. Since 2010, there has been significant progress in accessing different services online but there are still significant shortcomings for the different services (discovery, view and download) as described in section 4. Those trends, such as the lower level of availability of download services in comparison to discovery services, are consistent with the response of the data providers to the public consultation:

-51 % responded that their organisation’s spatial data sets and services were discoverable through web services with a significant portion indicating an incomplete discoverability (not all INSPIRE data they manage can be discovered).

-53 % indicated that their organisation’s spatial data sets were viewable through some web services with a significant portion indicating an incomplete access.

-32 % reported that their organisation’s spatial data sets were downloadable with a significant percentage indicating the non-availability for download.

-48 % responded that their discovery services were also compliant with the implementing rules, 43 % declared that their view services were compliant with the implementing rules and 37 % indicated that the download services were compliant with the implementing rules.

With regard to the access through the EU Geo-portal only 41 % of the services reported in 2012 can be found back in the portal with a very wide disparity across countries (from 0% to 92%). In addition, of those only 62% could be accessed and only 5% were in-line with the (non-binding) technical guidance provided.

This affects the effectiveness of the implementation overall and the EU Geo-portal as a single entry point to the EU spatial data infrastructure. This also affects the usability and usefulness of the EU Geo-portal. As a result, only 31% of the respondents to the public consultation used the EU Geo-portal at some time and individual feedback suggests that they were often not finding what they were looking for.

The main factors for the insufficient implementation of this step in the Directive were similar to those under other steps, namely the lack of sufficient financial and human resources. On one hand, the economic crisis played a role, on the other hand where INSPIRE relevant data is managed by a large number of smaller organisations, resources are an issue when online web services need to be set up and kept online. Other factors mentioned in the public consultation regard the lack of skills, training and software tools for setting up services and for checking the validity of the services against the INSPIRE implementing rules.

5.1.6.Effectiveness of actions to bring spatial data sets organised in interoperable data models online (Specific Objective 6)

The main deadlines for reorganising spatial data sets according to interoperable spatial data models conform to the implementing rule are set in the future 132 . The past deadlines (23/11/2012, 21/10/2015) to transform datasets only applied to spatial data sets which are 'newly created or extensively restructured'. Hence, it is not surprising that implementation levels in 2013 were still low (see section 4).

Data producers participating to the public consultation seem to confirm the current status. When responding (33 % of all producers) to the question on the % of data already brought in-line with the data specifications, less than 4% indicated that more than 75% of their data sets were already in conformity, 33% reported less than 25% conformity while 40% of them did not know if such actions were going on in their organisation.

A main factor with regard to the 2012 and 2015 deadlines is the fact that Member States decide whether or not a spatial data set is newly created or heavily restructured. For example LT reported that no new official data sets were created or restructured.

The (future) implementation steps are, without a doubt, the most challenging and resource intensive step which is why the Directive foresaw that the implementing rules should be "feasible and proportionate in terms of their likely costs and benefits" (cf. Article 7.2). Hence, a number 133 of Member States as well as the public consultation results emphasise the need for capacity building, including training for raising skills and resources in general.

Moreover, feedback from Member States also confirmed that there are still significant efforts and costs involved in transforming existing datasets to meet the future requirements (deadlines in 2017 and 2020). They also expressed concern regarding the higher complexity and the feasibility of adapting all spatial data to common data models by 2017 (Annex I) or 2020 (Annex II and III) as previously anticipated. Hence, the deadlines will be difficult to achieve for most Member States, e.g. due to the technical complexity of the interoperability implementing rules and guidelines that require the application of IT tools and skills that are often absent. The different starting points of Member States in terms of preparedness of meeting the deadlines set by the Directive were also identified as another reason for the upcoming challenges. Hence, many Member States and users call for a flexible, pragmatic and user-driven application of the existing data specifications.

Positive factors influencing progress are the increasing availability of INSPIRE-specific IT tools developed by the public and private 134 sector with and without EU level financial support. Such tools make it easier to transform data or, once available, can be used by other data owners provided they are "open source". A large number of training courses developed by the public 135 and private 136 sector are becoming available as online e-learning platforms while the transformation of existing spatial data sets conform to INSPIRE has been undertaken in several projects at local, regional 137 , national 138 and European 139 scales.

As regards the overall implementation gaps and delays related to the steps 2 to 5 (specific objectives 3 to 6), several Member States reported that the economic crisis and pressure on national budgets had a major impact on the allocation of required, up-front investment (e.g. in IT infrastructure and technical skills development in administrations). Where such investments have taken place, improvements in the implementation effectiveness have been noted. This aspect will also be discussed in the efficiency section.

5.1.7.Summary of the evaluation of effectiveness

At this half-way point in the implementation process, the evaluation of effectiveness of the Directive varies significantly. Overall, there is good progress since there are clear improvements as compared to the situation in 2007 and the availability and management of spatial data is now more advanced. Some Member States have also demonstrated that effective implementation is possible. However, the overall level of progress (as also presented in section 4) is very heterogeneous, generally below expectation and implementation gaps in most Member States are significant. They are the result of accumulated delays in the implementation process and underline the differences in speed and quality of implementation. Significant challenges remain in particular with the interoperability provisions and with the data policies.

There are many factors influencing this low level of effectiveness which can often be related to implementation efforts (and investment) by Member States. Most have not done enough in this respect, and can close their implementation gaps through, for example, more investment in skills and infrastructure, better coordination, improving the free flow of data by updating their data policies, better streamlining with national policies on eGovernment and open data. Moreover, there are indications that some implementation efforts which are not due until 2020 may be costly and difficult to achieve, in particular in relation to data harmonisation. Hence, collaboration between the Commission services, the Member States and the user communities may be useful to set priorities and ensure that the existing data specifications can be implemented in a more flexible, pragmatic and user-driven way so as to increase the effectiveness of this upcoming implementation step.

5.2.Efficiency

5.2.1.Costs and benefits

What are the costs and benefits associated with the implementation of the INSPIRE Directive in various Member States?

The costs and benefits associated with the implementation of the INSPIRE Directive were determined on the basis of reported Member State figures for the period 2010-2012. Overall, the information was patchy and diverse. Hence it was not possible to calculate EU-wide figures for costs and benefits, nor was it not always possible to compare the information available.

On costs, most of the Member States reported some information except for HU and DK. However, in several cases the costs are not systematically measured and/or different cost breakdowns have been applied. In other cases, the figures are provided only for part of the infrastructure or examples for a number of involved organisations (RO, DE). Also one-off costs and maintenance costs can often not be distinguished. It was also noted that it is difficult to separate cost due to INSPIRE from investments in data infrastructure which was anyhow planned and made.

On the basis of the available information, reported costs on average annual basis 140 (SE: 4.7, AT: 2.5. DE: 2.3 (national DE components only) FR: 13.5 LT: 0.497, LU: 0.9 FI: 1.63 CZK: 2.8, SI: 2.5 U.K: 2.8 141 in € million/year for the period 2010-2012) were below the initial impact assessment estimates 142 of €4 to 8 million per year at the national level depending on the size of the Member State and of €60000-94000 per region of 250000– 350000 inhabitants.

On benefits, Member States reported mostly in qualitative terms. They generally consider that benefits are yet to be fully realised but that they are starting to emerge in terms of improved data access, better cooperation across the public sector, skills and capacity building, less duplication of work, improved information for supporting environmental policy, better e-government services to citizens and business.

A few Member States 143 reported quantitative impact assessments, presenting estimates of future benefits:

-With respect to improved environmental management the UK Environment Agency for England and Wales estimated that the benefits of implementing INSPIRE in reducing environmental risk are equivalent to £5.1 million/year 144 . The 2012 UK Benefits Realisation Strategy 145 estimates annual quantifiable benefits across UK government departments at £m 470-510. Evidence of cost-saving and increased efficiencies at the local levels are also emerging, for example the Barrow Borough Council attributed £26.000 (or £0.38 per inhabitant) annual cost savings from implementing INSPIRE services 146 .

-NL reported a consolidated government view to the public consultation process 147 noting benefits through the collaboration and harmonisation between different layers of government and competent authorities. Harmonisation has led to more efficient generation of management information and creates benefits for the private sector, saving them the effort of data integration; new (mobile) applications are being developed on the basis of the information now available. The 2009 NL Cost-benefits analysis 148 for INSPIRE predicted that until 2013, the annual costs would exceed the benefits and increase the negative cumulative balance of both. From 2014 onwards, the benefits would be higher and the negative cumulative balance should decrease as shown in Figure 19. An update of this cost-benefit analysis 149 was recently published following the finalisation of this evaluation. It concludes that the costs are higher than originally anticipated. The total annual costs range from €3 to 5 million euro in the period from 2010 to 2024. Between 2010 and 2015, these were actually realised costs whereas the remainder are predictions which are higher compared to the predictions in the previous study. On the benefits side, INSPIRE does indeed ensure that data are more interchangeable, that the comparison and the quality of data are better and the availability is higher. Overall, the total value of benefits is lower than previously estimated. However, the new study is also highlighting that many benefits still cannot be quantified. They conclude that better management of benefits is needed to direct investments into those areas most beneficial for the different user groups.

Figure 19: Phasing of the costs and benefits over time in the basic model (amounts in EUR million) in The Netherlands – Source NL 2013 INSPIRE report , page 22.

-SE conducted an ex-ante/ex-post benefit survey with public authorities. SE considers that the envisaged benefits of the infrastructure have been realised. The spatial data cooperation has paid dividends both internally and externally. There are also examples of increased benefits for third parties. Access to a greater volume of spatial data via the spatial data cooperation has opened the eyes of certain organisations to new possibilities and areas of application and is also contributing to better and more reliable decisions. The biggest efficiency benefits resulted from changes in the data policy having to sign only one agreement 150  at fixed licence fee to obtain access to the whole pool of spatial information offered by all contributing organisations. Other benefits, in terms of results, regard more and better quality data available, improved dialogue and networking, better cooperation on data collection. INSPIRE has led to lower costs for the private sector in case they needed public data. Moreover, improved crisis management and shortened response times in serving information to the public were also observed by the authorities as a result of the INSPIRE implementation.

-LT reported that according to their cost-benefit analysis of INSPIRE at least 10000 working days are saved nationally per year with an expected annual increase of such savings at a rate of 5-10%.

-As part of its eGovernment strategy , DK reported to the UN 151 the release of a variety of public sector data for free and open reuse. The release was supported by a business plan that indicated an annual potential for economic growth due to open data in excess of €100 million from 2020 onwards. Among the data that are now accessible to citizens, private sector companies and public authorities are, e.g., spatial data covered by INSPIRE such as topographic data and the national digital elevation model. DK also changed the data policy of the Danish Geodata Agency from revenue-based to open and free funded by public appropriations also in order to implement the respective INSPIRE provisions.

Other examples of Member States which reported in more qualitative terms on the benefits observed and anticipated:

-Seven Member States 152 reported efficiency gains between Open Data initiatives and INSPIRE in cases where the same standards are used. In such cases, some countries reported an increase in skills and capability, a real benefit with more public bodies collaborating and using these skills and capabilities in providing other data and information related services. Overall, this led to more efficient access to information 153 as well as better and cheaper eGovernment services for citizens and businesses, thus improving transparency and creating business opportunities using environmental data. This boosted research and innovation potential.

-Organisations involved in reporting on air quality to the EU levels in the UK, BE, NL, DE 154 and IT reported efficiency gains and improved sharing across borders when applying INSPIRE solutions. BG observed improved efficiency in the reporting under the European environmental legislation in several organisations.

-PL reported on reduced costs for obtaining reference data and cost savings within municipalities as well as better policy outcomes through improved access and feedback from citizens on planning documents and positive effects on the availability of data for environmental impact assessments. PL and RO report on benefits from the integration of INSPIRE services with the agricultural subsidy Land Parcel Information System (LPIS). The efficiency of the systems improved and environmental aspects were taken better into account. Similar benefits were reported from administrations involved in forest fires hazard management.

-LU reported INSPIRE and its resulting national geoportal as an important factor in the national administrative simplification programme while the use of geospatial data by administrations, private sector and citizens continues to rise.

-FR reported benefits from faster discovery of the data, easier use of the data, limitation of the restrictions and reduction of the cost barriers thanks to mastering new tools and rising competence on environmental themes. The optimisation of the business exchanges between partners, the reduction in duplications of data lead to an increase in dissemination, better reuse and time savings. An increase in the quality of the data and their accuracy and an increase in the interest of users are also benefits found.

-In AT, the costs-benefit ratio of INSPIRE is at present considered to be unfavourable, noting that significant benefits can arise only when a larger number of harmonised data files are available.

-FI reported an increasing awareness of spatial data and its possibilities.

-LT noted a positive impact on the labour market.

-DK considered benefits of an effective infrastructure for spatial information, including the contribution of INSPIRE, extending beyond public digitisation and into growth and innovation for undertakings and citizens.

These examples demonstrate that some of the expected benefits are gradually emerging. Some countries have also demonstrated that their investment into an improved spatial data infrastructure has reduced the time and investment needed to prepare reports to the EU level. E.g. in Ireland, investments in connecting the digital infrastructure between authorities reduced the time to prepare a report on industrial installations for the European Union from months to days 155 .

Although the feedback to the public consultation does not allow for a country-to-country comparison, the responses with regard to costs and benefits seem to align with the overall trends reported by the Member States. For example, both user and producers rank better data discovery and access as the largest benefit realised so far. This is followed by benefits derived from greater interoperability through the use of international standards and improvements in internal data management processes. Other benefits identified range from business research opportunities, better cross-border governmental, knowledge transfer and the combined impact of Open Data and INSPIRE.

However, when asked if 'the benefits of INSPIRE will be greater than the costs', 16% of the respondents to the public consultation tend to disagree, 38% have no opinion while 38% agree that this would be the case.

This result does not come as a surprise given the variable progress made in the Member States and the fact that according to the ex-ante cost-benefit assessment and those conducted by NL (2009) and UK, INSPIRE benefits will only start to out weight costs approximately later in the implementation process. Such benefits would also require that investments are made and that obstacles with regard to access to and use are sufficiently removed. Whilst the outlook is positive as 49% of the respondents to the public consultation agreed that 'INSPIRE has helped me/my organisation in becoming more efficient and effective' while 24% tend to still disagree and 27% expressing no opinion, the recently published update of the cost-benefit analysis from the Netherlands 156 sheds some questions onto this outlook. Since they have now found higher costs than anticipated which are not outweighed by the benefits, the management of investments towards those areas with most benefits becomes even more important.

5.2.2.Comparison to the ex-ante evaluation

Are the results achieved so far commensurate with the resources put forward and in line with the ones expected from the ex-ante evaluation of INSPIRE?

The costs and benefits in the ex-ante evaluation of INSPIRE were based on the extended impact assessment 157 for the Commission proposal for a Directive presented in 2004. The proposal had some important difference in comparison to the Directive which was finally adopted. For example, the proposal included a more harmonised way for promoting "open data" and a proposal for an implementing rule to establish common licensing conditions, It did not contain the various options and flexibility on data sharing which are now present in Article 17(3). As there was no impact assessment on the adopted Common position 158 in 2006 or the Directive in 2007, figures on costs and benefits estimated in the INSPIRE ex-ante impact assessment and those reported by the Member States in 2013 do not have exactly the same baseline.

As mentioned above, the reported implementation costs varied from 0.5 to 13.5 million €/year with most Member States reporting between 2 to 3 million €/year. In most cases this is below the original estimates which ranged from 4 to 8 million €/year in the initial impact assessment 159 . The wide variety of investment costs reported by the Member States in 2013 suggests also that there has been a lack of investment in many Member States which may partially explain the gaps in implementation documented in the sections on state-of-play and effectiveness. As also demonstrated before, the benefits from the implementation of the INSPIRE Directive can only materialise after initial investments in the spatial data infrastructure and the necessary skills of the staff running this infrastructure have taken place. It was not possible to do a more detailed analysis of what influenced the investments or the lack thereof. It seems, however, plausible that investment costs can be reduced where synergies are created with related initiatives, such as open data and promotion of eGovernment.

5.2.3.Cost-effectiveness of specific provisions

Can any specific provisions in INSPIRE be identified that make cost-effective implementation more difficult?

The public consultation and Member States reports point to several main factors influencing the cost-effective implementation of the INSPIRE Directive:

1)Effective data policy on sharing issues continue to score highest on the list of obstacles.

2)Addressing the skills gap for dealing with the technical complexity of implementation, in particular in smaller organisations.

3)Avilability of sufficient resources also for capacity building.

4)Effective coordination and communication mechanisms.

5)Flexible spatial data harmonisation provisions and development of tools which would allow more efficient application of the alignment to data model provisions ("simplification of use").

Underlying these issues are either the INSPIRE specific provisions, the objectives and actions on data sharing (i.e. Article 17), the existing coordination structures in the Member States (which influences resource allocation and capacity building), as well as the agreed implementing rules as well as guidelines (see earlier sections). Whilst all the implementing rules were unanimously endorsement by the INSPIRE Regulatory Committee based on feasibility tests conducted by organisations in the Member States at the time, there are now some indications that efforts and costs involved in applying these rules may be higher than expected. In particular the data "interoperability" specifications (or better, the specification transforming spatial data to conform to common data models), which was mentioned under step 5 above, is proving to be the most costly and challenging step. Member States have to comply with these provisions only by 2017 or 2020 (depending on the data concerned). In recent expert group discussions 160 they indicated that their national experiences over the past three years in simultaneously harmonising such data would involve high costs and practical difficulties which was not always balanced by the benefits that the higher degree of interoperability created (e.g. the efficiency gains). The experts suggested that there was a lack of clear priority setting at national and EU level, i.e. it was not possible to identify the most important datasets for end-user applications amongst the data themes, in particular those of Annex III, where the highest user needs and benefits can be expected. The experts called on closer collaboration between the EU level and the national level implementation and suggested to assess together the feasibility and benefits of implementing related provisions at reasonable costs. Furthermore, it was discussed that in addition to such a "simplification of rules", the "simplification of use" (see above) can be another avenue to be explored by working together towards common tools, guidance and trainings.

The perceived technical complexity of the implementing rules is also strongly related to a lack of capacity and skills (see point 2 in list above). Moreover, investments (see point 4) to build up the capacities skills are needed to deal with the technical implementation issues related to the management of metadata, network services and data interoperability. Without such efforts INSPIRE would not have the desired benefits.

With regard to (see point 1) the current provision for data sharing, their cost-effectiveness depends on the choices made by the Member States. The provisions of in particular Article 17 could be assessed in more detail and compared with the most cost-effective implemented practices implemented in Member States. Also the original proposals from the Commission, i.e. the development of some generic, harmonised, "open data" national licensing conditions may be worth considering again at national level.

Finally, improving coordination and communication (see point 3) can positively affect the cost-effectiveness. Sharing of experiences and aligning efforts and investments across administrations and borders can have further positive impacts on the cost-effective implementation, for example the RO LPISweb 161 is a result of a twinning project with DE. Actions to this end are on-going and are planned in the INSPIRE Maintenance and Implementation Framework. This could be complemented by strengthening cross-border collaboration on priority issues and on applications.

5.2.4.Administrative burden

What kind of administrative burden and costs for public authorities and other public users have been identified?

The main administrative burden for the implementation of the INSPIRE Directive falls on public authorities. The main administrative costs would relate to the monitoring or reporting obligations under INSPIRE. Moreover, the perception of burden varies but is generally related to the costs of coordination, IT infrastructure, service implementation and harmonisation. Precise cost figures, which would allow applying the Standard Cost Model 162 , were not reported or available. At the time, the ex-ante impact assessment did also not include a separate cost item for reporting or administrative burden.

Four countries(FI, LT, SE, SK) provided estimates of the financial costs of monitoring and reporting combined. SE reported 0.75% (mio€ 0,033 of 4,7, LT 0,9% (mio€ 0,045 of 0.4975) , FI 4% (mio€ 0,067 of 1.63) of the implementation cost. This indicates that the administrative burden appears to be low. Overall, these administrative costs identified for the implementation of the INSPIRE are far lower than the benefits and administrative cost savings that can be achieved through a modern and shared spatial data infrastructure (see above).

Nevertheless, Member State experts 163   call on the Commission to review the existing monitoring and reporting obligations based on Commission Decision 2009/442/EC. In particular the three-annual national report is considered too burdensome and duplicating information also gathered under the monitoring framework with the help of the EU Geoportal 164 and the EEA's dashboard. This also related to wider discussions on the streamlining of environmental reporting which are currently under evaluation in a Fitness Check on environmental monitoring and reporting 165 .

5.2.5.Available resources

Have the resources needed to implement INSPIRE been available?

It has already been discussed above that the investments into implementation appear to be (too) low in many Member States (cf. section 5.2.2). This is confirmed by the ex-ante cost-benefit assessments which have been conducted by some Member States (FI, FR, NL, SE, UK). However, most Member States did not report on the actual investments made.

On the basis of the concrete outputs so far, it is clear that the situation varies from country to country. A number of Member States 166 seem to have been more successful in mobilising resources, in particular those where INSPIRE was recognised as a major component for eGovernment and Open data strategies. By combining investments between the INSPIRE implementation and the eGovernment action planning, economies of scale can be created 167 . In many countries (such as AT, BE, DE, ES, FR, IT and UK) resource allocation is also a regional issue and differences can be noted from region to region. A number of Member States commented on the negative impact of the economic crisis as regards the investments that have been undertaken (e.g. BG, CY, ES, GR, IE, IT, PT, RO).

EU funding has been able to mitigate some of these issues and several projects have been supported by EU programmes. A few Member States (such as BG, LT, RO) and some regions in DE reported on the use of Cohesion funding for parts of their INSPIRE implementation without however specifying the amounts directly allocated to INSPIRE. Several Member States reported the participation of a number of organisations to projects with EU level funding from other instruments (eContent, CIP 168 , eInfrastructures, the research framework programmes 169 , LIFE+, ISA 170 , Space, ISPA 171 ) without further budgetary information. Such projects can help to develop solutions and tools that can be used by all Member States (reusable components), which can improve efficiency across multiple policy sectors. However, these opportunities are not used systematically by Member States.

5.3.Relevance

Relevance looks at the relationship between the needs and problems and how they evolved over time, taking into account external factors. To recall, the INSPIRE Directive has the general objective to establish and EU infrastructure for spatial information to underpin the knowledge base for environment policy. It addresses this goal through its specific objectives and actions.

5.3.1.Need for EU action

To what extent does INSPIRE still match current needs and do they continue to require action at EU level?

From a policy perspective, the INSPIRE Directive remains or is even increasingly relevant. This is most clearly shown through the Commission priorities relating to the 2015 EU Digital Single Market 172 strategy. It identified the need to increase cross-sector interoperability in the public sector (with the revision of the European Interoperability Framework) where INSPIRE is of major relevance. Promoting eGovernment services and the need to apply the ‘digital by default’ and ‘use once’ principles are all enshrined in the INSPIRE Directive.

Moreover, the 2011 assessment 173 of the 6th Environment Action Programme 2002-2012 emphasised again the need for a more extensive knowledge base for policies. It refers to the further implementation of INSPIRE as an instrument for improving environment information systems as environmental information is often incomplete and not always available on time. The Seventh Environment Action programme 2013-2020 174   follows up by further stressing the need to have credible, comparable and quality-assured data and indicators more readily available and accessible to those involved in defining and implementing policy. It recognises progress made, yet emphases the fact that data collection and quality remain variable and that the multiplicity of sources can make access to data difficult. It refers directly to INSPIRE as part of an enabling framework to further develop the EU-wide electronic data exchange with enough flexibility to encompass new areas.

The implementation of the INSPIRE Directive is important for the ability of Member States to compile greenhouse gas (GHG) inventories, in particular the data needing to be reported under LULUCF Decision No 529/2013/EU (greenhouse gas emissions and removals resulting from activities relating to land use, land-use change and forestry) and the GHG Monitoring Mechanism Regulation (EU) No 525/2013.

The 2012 Commission Communication on 'Improving the delivery of benefits from EU environment measures: building confidence through better knowledge and responsiveness' 175 , emphasises the relevance of INSPIRE for implementation, compliance promotion and the more systematic information and active dissemination of information to ensure up-to-date and comparable information across Europe. 

Moreover, the application of the INSPIRE Directive is potentially important for achieving the aims of the on-going Fitness Check 176 for environmental monitoring and reporting being carried out as part of the Better Regulation agenda. 177  

From the perspective of policies or actions having an impact on the environment, the continued relevance of the INSPIRE Directive is reported in many policy documents (e.g. marine/maritime216, transport 178 , health, climate action, development, research and innovation, enlargement, development, telecommunications, humanitarian aid, agriculture, security, and space policies).

For example, the EU Space policy includes the Copernicus programme 179 and is implemented with the European Space Agency and other non-EU agencies active in the area. It represents a considerable EU investment to build up an earth observation capacity in space to deliver data and information services for policies based on satellite observations. According to the Copernicus Regulation, the data and service policy as well as the implementation of the services have to conform with INSPIRE rules. Such information delivery requires an unimpeded access to the non-space spatial data which is not part of the data collection of the programme. Implementing INSPIRE in a way that it serves Copernicus is therefore highly important for the EU Member States to fully reap the benefits from the Copernicus programme.

INSPIRE also has international relevance. It contributes, from an EU perspective, to the Group on Earth Observation (GEO) 180 initiative and United Nations Committee of Experts on Global Geospatial Information Management (UN-GGIM) 181 initiatives and drew the attention of the World Bank, which supports its application as a 'good governance' practice in its relevant projects 182 world-wide.

There are also external drivers which constitute a source of new requirements or 'needs' at EU level which did not exist to this extent at the time of the development of the Directive. For example, the EU  'Better Regulation' 183   agenda is increasingly relevant driver for delivering the INSPIRE objectives 184 as it potentially improves the effectiveness and efficiency through better access to spatial data for policy implementation, assessment and development. Evidence on benefits reported by the Member States shows that already at this stage, where implementation is more advanced, it allows data be brought more efficiently together for reporting 185 under various environmental acts with a potential to reduce administrative regulatory burden. Among the main benefits reported (for the administrations, business and citizens) are gains in time both in searching and accessing the right and necessary metadata and data; increased flexibility because many types of users can make use of the same infrastructure; improved openness and transparency because such data are open to the public; and improved integrity and trust because the infrastructure is continuously improved. 

The relevance is also confirmed by the public consultation 186 which demonstrated substantial support for the objectives of the Directive, considering them still relevant for removing the obstacles to data sharing that hamper effective and efficient implementation of environmental policy.

5.3.2.Relevance of objectives and actions

To what extent are the general and specific objectives of INSPIRE still relevant to the issues (obstacles) they address?

Building on the previous section, the relevance of the general and specific objectives remains true and is widely recognised since spatial infrastructure is not yet in place and the obstacles originally identified are still persistent at this half way point of implementation. This is supported by the public consultation where 92% of the respondents consider that the specific objectives of INSPIRE Directives for making spatial data and services more easily accessible are still pertinent (2% disagreed). The Member State reports and the national coordination structures (18 out of 19 participating countries 187 ) which participated to the public consultation strongly confirm this.

Are the actions of INSPIRE still appropriate?

Similarly, it has been demonstrated above that also the actions taken under the five steps and linked to the specific objectives are still relevant at this half way point in the implementation. Also the results of the public consultation seem to indicate that the 'actions are still appropriate to meet the objectives' with 49% agreeing, 13% disagreeing and 21% having no opinion.

The percentage of disagreement/no opinion may indicate that some actions have not yet lead to the expected outcomes and impacts. For example, on the positive side, 79% of respondents considered that INSPIRE improved availability and accessibility (9% disagree) to spatial data and services and that there is a growing uptake in other policy areas and concrete applications in the Member States such as transport, utility management, telecommunications, spatial planning and eGovernment. The main policy area for which, to date, the INSPIRE actions have not resulted in as much success, is environment. Also on the negative side, users still encounter obstacles when they need to use spatial data covered by the INSPIRE Annexes I to III, as illustrated in the table below.

Table 7:    User Responses to INSPIRE public consultation by perceived obstacles to data use 188

This result is directly related to the action on data and services policies for sharing. The current actions are laid down for government-to-government sharing which continue to be relevant but seem to lack effectiveness, as discussed above. From a general perspective, this is confirmed by several Member States reports and the public consultation in which 50 % of users still find policy obstacles, in total or in part, to data sharing. Similarly, only little more than half of the respondents from data producing organisations indicate that their organisation has a policy in place addressing the INSPIRE requirements. It will be necessary to assess in more detail whether this issue relate to poor or outstanding implementation or to more fundamental aspects related to the way the provisions are laid down in the Directive (see also in sections 0, 5.1.1 and 5.1.7).

Also coordination remains crucial for realising the outputs of the other INSPIRE actions as well as for taking into account external factors (e.g. eGovernment policy developments). There is, however, the need to strengthen cross-border coordination as indicated through the public consultation where 55 % recognise the INSPIRE potential to improve access and use of spatial data across borders and only 20 % consider INSPIRE well-coordinated between neighbouring countries.

Overall, the actions related to spatial data sets are also still appropriate but lack effectiveness and efficiency. This also applies to the specific action on reporting and monitoring on the implementation and use of INSPIRE, remain relevant. However, the process could be assessed in view of further reducing administrative burden and increasing effectiveness in terms of delivering information required under the Better Regulation initiative.

5.4.Coherence

To recall, Article 1(1) of the Directive specifies that INSPIRE needs to serve 'the purposes of Community environmental policies and policies or activities which may have an impact on the environment'. The ‘coherence’ analysis examines the extent to which INSPIRE is coherent internally, e.g. how the INSPIRE actions operate together to achieve their objectives. The external dimension relates to the interaction with other EU environmental legislation and other policies or interventions which have similar objectives.

The internal coherence of the Directive has proven to be sound because Member States largely follow its steps (see above) and has not been analysed further.

5.4.1.Coherence within the environmental policy domain

To what extent is INSPIRE coherent with other environmental policies and initiatives?

From a legislative perspective, following the entry into force of the INSPIRE Directive, there is an increasing number of more recent or revised existing EU environmental legislative acts and implementation guidelines 189 explicitly referring to INSPIRE. In particular, these references introduce requirements for spatial data needed for the implementation of measures and/or reporting. This is the case for:

the Marine Strategy Framework Directive, MSFD 2008/56/EC,

the guidelines of the Water Framework Directive, WFD 2000/60/EC and Floods Directive 2007/60/EC,

the 2011/850/EU Commission Implementing Decision 190 as regards the reciprocal exchange of information and reporting on ambient air quality,

the site information format for NATURA 2000 sites 191 ,

the reporting system 192 for Industrial Emission Directive, IED 2010/75/EU, and

the Directive 2012/18/EU (SEVESO III) on the control of major-accident hazards involving dangerous substances on 'the way information is managed'.

Although such legal references are not strictly necessary, as all EU legislation applies even without being mentioned in other pieces of legislation, the references to the INSPIRE Directive in the strategies, guidance documents, preambles or the bodies of legal acts adopted after INSPIRE are making legal obligations more visible and stimulating the coordination and collaboration between concerned organisations in the Member States. Overall, this has a positive effect on the coherent application of the laws.

However, from an operational implementation perspective, this coherence does not necessarily appear in practice without further efforts. Therefore, the Commission initiated a number of INSPIRE-based reporting pilot projects, in collaboration with the EEA and Member States starting in 2009 on air quality. Other actions are undertaken in the context of the Maintenance and Implementation Work Programme, in particular improving coherence for reporting under the Water Framework Directive and Urban Waste Water Directive (as part of the evolution of the Water Information System for Europe), the Marine Strategy Framework Directive and the Industrial Emission Directive in coherence with reporting under SEVESO III and the E-PRTR Regulation. On NATURA 2000 and other protected sites, a number of national site management information systems incorporated INSPIRE services and data specifications. References to INSPIRE unique identifiers for protected sites are already part of the reporting obligation.

Despite these efforts, the current reporting systems are only partially making use of the INSPIRE rules and specifications and more work is needed. This is no surprise and has several reasons. First, the INSPIRE Regulation 193 regarding interoperable data specifications only entered into force between 2010 and 2014 (depending on which data themes in the annexes of the INSPIRE Directive were covered). As a consequence, the implementation deadline for most of the environmental data themes which are covered in the reporting obligations for the above-mentioned pieces of legislation are in Annex III and need to be transformed only by 2020. Second, the INSPIRE services through which such harmonised spatial data could be harvested by reporting applications are outstanding partially because of the above-mentioned timelines. Third, as pointed out before, not all relevant spatial sets for reporting have as yet been identified by Member States. They have often not been made a priority since the reporting process was (and is) largely carried out without using the national spatial data infrastructures. Fourth, the reporting cycles of the various pieces of legislation are not aligned with the implementation of INSPIRE. Hence, several reporting deadlines apply every year until 2020 and no transitional arrangements have yet been agreed on how to move from a reporting process before INSPIRE to one that makes best use of the INSPIRE tools and services. Unless the implementation of INSPIRE is accelerated, the reporting systems will have to be based pragmatically on other traditional data and information collection procedures and IT solutions, at least for a transitional period.

Nevertheless, as documented in the section on efficiency (cf. 5.2), several Member States 194 already reported initial benefits from INSPIRE related to reporting, including reporting to the public as part of the obligations under the Public Access to Environmental Information Directive 2003/4/EC. This is particularly important as the Commission 195 launched a Fitness Check 196 of reporting obligations in the field of environment policy. The INSPIRE Directive has been identified as a key tool to assist in making the monitoring and reporting process more efficient.

In addition to the above, the consistent application of INSPIRE rules in conjunction with the Public Access to Environmental Information Directive 197 , in particular the active dissemination provisions (see Article 7), provide the potential to facilitate data sharing and better inform and empower citizens.

5.4.2.Coherence within the other policy domains

To what extent is INSPIRE coherent with wider EU policies and other interventions which have similar objectives?

Several other EU policies and strategies not falling under the environmental acquis, as well as guideline documents refer to the INSPIRE Directive. The degree to which this happens varies from references in the recitals of the acts to the INSPIRE Directive as a relevant framework to 'without prejudice' or 'to be taken into account' clauses 198 in the legal act.

Through this, overall coherence with the most relevant policies is ensured, at least from a legislative point of view. Improvements in the practical applications are still possible across the board. Areas of particular relevance for coherence are:

-Coherence with the 2015 EU Digital Single Market (DSM) strategy: The Communication 199 and accompanying Staff Working Document (SWD) 200 emphasise the importance of essential cross-sector interoperability and standards in areas such as, inter alia, environment, transport, health, energy, and the need for integrating European and national portals to work towards a 'Single Digital Gateway' to create a user friendly information system for citizens and business. The SWD refers to the INSPIRE Directive and the need for public authorities to improve the sharing and re-use of their data. INSPIRE and its interoperability framework between public sector entities, once fully implemented, may contribute to translating the presented ingredients of public sector modernisation such as the 'One Stop Shop' and 'Once Only' concepts into reality. Therefore, the full implementation of the INSPIRE Directive has been included as one of the actions in the most recent eGovernment Action Plan 201 . Moreover, links have been established to the initiatives of the European Open Science Cloud and the European Data Infrastructure, under the ‘European Cloud initiative’ 202 .

-Coherence from a Digital Economy perspective: INSPIRE is globally coherent with and relevant for the EU " Open data " 203  initiative and Directive 2013/37/EU amending Directive 2003/98/EC on the re-use of public sector information 204 , 205 . INSPIRE services are effectively implemented in support of the implementation as reported by the majority of the Member States. INSPIRE services combined with Open data policy start to find their application even at local scales 206 , 207 . The Open Data initiative, launched by the Commission in December 2011 as one of the pillars of EUs data economy, is to an enable better governance and financial growth. Geospatial data account for an estimated 80% of public sector information 208 and are considered the most significant category of open public data due to their high production, procurement and update costs, as well as their relevance in multiple thematic areas and domains. The availability of such high value data, for which INSPIRE provides a service and interoperability infrastructure, has the potential to create and sustain a multi-billion market of applications and services 209 . Some inconsistencies as regards the data policies (and the related derogations) exist in relation to the Directive on the re-use of public sector information which may need further attention. This relates to broader issues on the free flow of data, identified as a priority issue for the Digital Single Market.

-Coherence with the European Interoperability Framework, EIF 210 : The EIF defines an agreed overall approach to interoperability. It sets the principles of a conceptual model for European public services and describes interoperability on different levels: legal, organizational, semantic and technical. It considers that at all these levels barriers for interoperability exist and solutions have to be developed. For example, legal interoperability concerns how to deal with differences in legislation. INSPIRE is an important element in cross-sector interoperability between public administrations. Consequently, the EU Interoperability Solutions for public Administrations (ISA) and ISA2 programme has contributed substantially to the implementation of a number of activities in the Maintenance and Implementation Work Programme of INSPIRE. In particular, the European Union Location Framework (EULF) and Reusable INSPIRE Reference Platform (ARE3NA) projects relate to the implementation of the INSPIRE Directive, EULF from a policy and usage perspective and ARE3NA in promoting the reuse of technical components. This contribution continues in the context of its successor, the ISA2 programme which includes the ELISE project. The ISA action dedicated to the definition of the DCAT Application profile for data portals in Europe (DCAT-AP) 211 (a specification based on the Data Catalogue vocabulary (DCAT) for describing public sector datasets in Europe) enables cross-data portal search for data sets including those made available under INSPIRE, making them better searchable across borders and sectors.

-Coherence with other policies and initiatives: INSPIRE is mentioned in the context of numerous EU initiatives, policies and legislation such as the Global Monitoring for Environment and Security (GMES) in 2011 212 and in 2014 as Copernicus 213 , the Group on Earth Observation (GEO) 214 initiative, the Common Information Sharing Environment for the EU maritime domain (CISE) 215 , the European Marine Observation and Data Network (EMODnet) 216  , 217 , the Common Agricultural Policy - integrated administration and control system 218 , Directive 2014/89/EU establishing a framework for maritime spatial planning and integrated coastal management 219 , Regulation 1255/2011 220 establishing a Programme to support the further development of an Integrated Maritime Policy , Directive 2010/40/EU 221 and Commission Delegated Regulation (EU) 2015/962 222 on Intelligent Transport Systems, the European Observation Network for Territorial Development and Cohesion (ESPON Programme) 223 which supports policy development related to EU Cohesion Policy initiatives, Humanitarian Aid 224 , in the policy domain of energy (e.g. geothermal database, building specifications with regard to energy efficiency) and public health (e.g. cancer registry, registry of Genetically Modified Organisms, the Connecting Europe Facility programme with its Access to re-usable public sector information Digital Service Infrastructure making available searchable references to European public datasets including INSPIRE ones through a multilingual harmonised pan-European data portal 225 ).

An important spin-off of this growing coherence are references to INSPIRE in EU level funding programmes (environment, cohesion and regional development, humanitarian aid and disaster management 226 , agriculture, maritime, research and innovation, information society, health, transport, enlargement 227 and neighborhood policy). This increases the number of projects of which a subset is listed on the INSPIRE Forum 228 , supporting the development, implementation and application of INSPIRE for a wide range of policies. It also helps overcome the investment deficit which was discussed earlier.

5.5.EU Added Value

The evaluation of the EU added value examines again the justification for EU level intervention also in the light of changes since the Directive was adopted. As such, EU-added value looks for effects which can reasonably be argued to have occurred due to INSPIRE rather than any other factors. In particular, cross-border and EU level use cases can demonstrate where the application of the INSPIRE Directive has an added value which would have not been possible without EU level action.

The EU added value of the INSPIRE Directive will fully emerge only when the EU infrastructure for spatial information based on compatible Member State infrastructures is fully established. As demonstrated earlier, this is not yet the case as the envisaged implementation process is only partially completed. Moreover, many Member States are falling behind the interim implementation milestones. Unsurprisingly, there are only limited EU level benefits which have materialised to date. For example with the partial exception of air quality, EU level reporting, has not yet benefited significantly from the INSPIRE Directive. Thus, the full potential of EU added value has not materialised yet.

On the positive side, Member States, in particular those where implementation has progressed most, reported positive effects in breaking down their internal obstacles preventing the more effective sharing of their spatial data between public administrations and across borders (including in some cases across their regional borders). Simplification and harmonisation of data policies and licenses combined with a technical infrastructure allowing easier discovery, access and use of spatial data are attributed to a large extent to INSPIRE. This has also generated a number of cross-border collaborations and improvements when it comes to environmental data sharing (e.g. BE, DE, IT, NL and UK reported efficiency gains and improved sharing across-borders when applying INSPIRE solutions to air quality data sharing).

The INSPIRE Directive implementation also offered added value as it pooled the expertise of Member States together through the EU level coordinated development of the implementing provisions and its follow-up Maintenance and Implementation support work programme. This has produced a number of solutions and tools which could be re-used by Member States. Moreover, the many platforms of collaboration (e.g. through EU funded projects) have enabled the promotion of best practices, the development of common (re-useable) tools, a better common understanding, the sharing of guidance and increased possibilities for learning from each other. This has generated an EU level capacity and knowledge pool and created an wide network of specialists which is illustrated by the high interest and the growing number of participants in the annual INSPIRE Conferences 229 which are co-hosted between the European Commission and a host country.

International standardisation bodies in this field, whose standards are combined in the implementing provisions and guidelines, consider INSPIRE as their largest interoperability 230 cross-border test-bed globally. INSPIRE raises the wide-spread use of those standards, stimulates the portability of technical solutions and thus reduces costs and redundancies.

At the same time, the current EU added value has not yet translated into a situation where the benefits at EU level clearly outweigh the costs of implementation. The disparities in the past and current level of investments by the Member States and their regions risk further delay. Only if all (or at least most) Member States have a fully functioning spatial data infrastructure in place can the EU level benefits be reaped, e.g. through dedicated EU level applications. This is already happening in some selected fields, such as Intelligent Transport Systems (which is using INSPIRE), where dedicated investments were made. Also the Information Platform for Chemical Monitoring 231 is using basic INSPIRE features to access a multitude of data sources which facilitates risk assessment of chemicals in the context of the REACH Regulation 232 . Step-by-step, a number of other uses for data covered by the INSPIRE Directive can be further exploited at EU level e.g. environmental reporting supporting impact assessments or evaluations, research and innovation. In particular the chance to exploit the potential of data received through the Copernicus programme is important since these remote sensing data often need to be combined with spatial data to add value and context.

EU level funding has contributed to support implementing the infrastructure as well as to building capacity and training. However, this was not sufficient to avoid the geographically uneven build-up of the infrastructure.

As explained earlier, the delays can partially be explained with the fact that crucial INSPIRE implementing provisions were only adopted recently (the last one in the beginning of 2014). Since then, the rate of progress shown by a number of Member States is promising.

In addition to EU-wide application and uses, the INSPIRE Directive was also designed to create EU added value through improved cross-border cooperation spatial data management, not just in the environmental field. Whether it is sharing data on air quality, marine pollution or flood risk management, environmental solutions often need cross-border collaboration. However, the lack of a wider-spread implementation of INSPIRE based information services for these purposes still limits its cross-border added value.

To overcome these issues, national priority setting which differs greatly in terms of identifying those spatial datasets most needed for cross-border applications or for reporting activities at EU level (i.e. some focus on air quality 233 , others on marine data 234 ) can be coordinated better across the EU or between Member States. Finally, collaboration between the Commission and Member States has generally been seen as positive but can be strengthened further by, for example, developing implementing tools and components together rather than each Member State ‘reinventing the wheel’.

6.CONCLUSIONS

The full implementation of the INSPIRE Directive is designed to deliver an EU infrastructure for spatial information by the end of 2020 based on compatible infrastructures in the Member States and useable in an EU and trans-boundary context. The evaluation of the INSPIRE Directive confirms that the overall relevance of the Directive to meeting policy needs in an efficient manner remains high, and is expected to increase with time, given the drive towards a digital economy as set out by the Digital Single Market strategy which includes important elements of the Directive.

Overall, the application of the INSPIRE Directive did lead to an increased availability and better access to spatial data and services compared to the 2007 baseline situation. However, good progress in implementation has been made in only the few Member States where the necessary investments were made and implementation of the Directive and where it was aligned with wider national action on open data policies and better eGovernment services. The implementation gaps identified are significant and result from accumulated delays in the process, underlining the differences in speed and quality of implementation.

Looking at the specific objectives and the related actions, some specific conclusions can be drawn.

A coherent and effective legal framework for sharing spatial data and services across the EU is fairly advanced but not yet fully established. In particular the legal arrangements for data sharing, such as data policies, are still too complex and/or too heterogeneous in many Member States to be effective.

Coordination at the national and EU level is well established but leaves room for improvements, in particular in relation to national coordination with open data and eGovernment initiatives.

The identification of relevant spatial data sets advanced in most Member States with the increasing volume of spatial data sets brought into the infrastructure, in particular since 2013. However, there are significant differences between the Member States and there remain issues regarding the relevancy and completeness of the identified data sets for environmental policy purposes.

The documentation of the identified spatial data sets and services (metadata) has increased steadily and is one of the areas most advanced in the implementation. However, there are therefore still efforts needed to close this implementation gap and to generally improve the availability and quality of the metadata.

The availability of online network services is incomplete and varied. Only a part are currently accessible through INSPIRE network services for discovery, view and download. Also here there important differences noticeable between the Member States and the type of services concerned. The access through these services via the EU Geo-portal also varies significantly.

The interoperability of the spatial data sets is not far advanced mainly because the main implementation deadlines are still in the future (2017, 2020). The evidence suggests that important efforts will be required to meet these future deadlines and targets. Whilst there is general agreement that important efficiency gains can be realised, once spatial data is organised in common data models, Member States and stakeholders expressed concerns regarding the (perceived) complexity and the feasibility of implementation of the agreed INSPIRE data models and guidelines.

As a result of all these shortcomings, overall effectiveness has suffered. In particular, the significant remaining obstacles created by the data policies in many countries impede effective progress and perpetuate the administrative burden because data cannot be easily shared between administrations. Nonetheless, some Member States have shown that the process is possible and report positively on the resultant benefits, if only in qualitative terms.

This is confirmed by the evaluation of efficiency from front-runner Member States that invested in implementation early on, developed more open data policies and aligned the INSPIRE Directive with their national priorities on open data and the drive for eGovernment. Upfront costs however are higher than benefits since data will have to be made available in the required ways first before being used for end-user applications. Many Member States made insufficient investments, probably because of the economic crisis. EU level funding for the implementation and maintenance of INSPIRE have helped the Member States to close some implementation gaps and increase the synergies between related policy instruments (e.g. those implemented in the Digital Single Market).

Overall, the actions required by the implementation of the INSPIRE Directive remain relevant. The evaluation of coherence has uncovered areas needing attention, in particular the development of the data policies creating obstacles in the internal (digital) market which is also of relevance to the ‘free flow of data’ initiative.

Finally, future EU added value can be significant. Addressing the above-mentioned issues and focusing on end-user needs and applications in a cross-border and EU context can assist implementation and help prioritise resources and investments.

In conclusion, this evaluation has demonstrated that the INSPIRE Directive is still largely fit for purpose but that further efforts have to be made at EU and Member State level and some fine-tuning may need to be considered to overcome the areas where effectiveness, to date, is suboptimal such as with regard to data policies.

ANNEX 1

Part A: Detailed INSPIRE Implemenation Roadmap

(Source: EEA/JRC Technical report, 10/2014)

Part B: Detailed INSPIRE Intervention logic
(MS= Member States, EC= European Commission)

Annex 2

Evaluation questions

The following evaluation questions were agreed:

Effectiveness (to which extent have objectives been achieved?)

-What progress has been made over time towards achieving the objectives and targets set out in INSPIRE in various Member States? Is the progress made in line with initial expectations and is the geographical coverage of implementation consistent?

-Which main factors have contributed to – respectively stood in the way of achieving these objectives? (for example, gaps, complexity or inconsistency in the measures or working methods of INSPIRE, the timely and coherent transposition in national legislation)

Efficiency (have the objectives been achieved at reasonable costs?)

-What are the costs and benefits associated with the implementation of the INSPIRE Directive in various Member States? (For example, did INSPIRE have a positive, quantifiable impact on the efficiency of public authorities and other public users?)

-Can any specific provisions in INSPIRE be identified that make cost-effective implementation more difficult?

-Are results achieved so far commensurate with the resources put forward and in line with the ones expected from the ex-ante evaluation of INSPIRE?

-What kind of administrative burden and costs for public authorities and other public users (enterprises including SMEs, private citizens etc.) have been identified?

-Have the resources needed to implement INSPIRE been available?

-Can the INSPIRE Directive and implementing rules be made more cost-efficient?

Relevance (INSPIRE objectives and actions still pertinent to the needs?)

-To what extent does INSPIRE still match current needs and do they continue to require action at EU level?

-To what extent are the general and specific objectives of INSPIRE still relevant to the issues (obstacles) they address?

-Are the actions of INSPIRE still appropriate?

Coherence

-To what extent is INSPIRE coherent internally?

-To what extent is INSPIRE coherent with other environmental policies and initiatives?

-To what extent is INSPIRE coherent with wider EU policies and other interventions which have similar objectives?

EU added value

-What is the EU-added value of INSPIRE in comparison to what could be achieved at Member States national and/or regional level activities?

-To what extent do the issues addressed by INSPIRE continue to require action at EU level?

Annex 3

Procedural information concerning the process to prepare the evaluation or Fitness Check

Lead DG: European Commission Directorate-General Environment, DG ENV

Organisation

A policy evaluation project, conform to the standing operation procedures for European Commission policy evaluations was launched in DG ENV in March 2013.

According to these procedures, Terms of Reference 235 (TOR) were prepared.

These TOR presented the purpose and use of the evaluation, the activities which require evaluation, the scope of the evaluation, the documents and data sources used, the evaluation questions, the methods and phases of the evaluation and the organisation of the evaluation.

A Steering Group was set up by the Commission on 08/08/2013, with the mandate to validate the TOR and project documents, to provide access to information as required by the evaluator, to support and monitor the work of the evaluator and to assists with the assessment of the quality of the draft and final Policy Evaluation Report.

The Steering Group was composed of DG ENV, JRC and EEA representatives in liaison with the Commission Inter-Service Group COGI 236 (Commission Inter-Service Group on Geographical Information), chaired by Eurostat. COGI has been since the conception of INSPIRE the Commission internal consultation and advice group on INSPIRE. In addition, the policy evaluation points of contact of all Commission services were invited.

In the spirit of the collaborative and transparent process in which INSPIRE has been developed, the Commission associated the INSPIRE National Contact Points (NCP) 237  to the Steering Group.

An 'Evaluator' is required for all policy evaluations. Evaluations may be conducted by an external Evaluator, under contract to the Commission or may be conducted internally. The INSPIRE policy evaluation applied a mixed model.

As Evaluator, the TOR presented the INSPIRE Coordination Team, INSPIRE CT (DG ENV, JRC and since 2013 the EEA – previously Eurostat), which has been providing the EU level coordination of INSPIRE since conception.

The EEA as an independent organisation has the general task to assist its member states and Commission with the assessment of the environmental policies and policies having an impact on the environment (Article 3 of the EEA Regulation). As such, the European Environment Agency, EEA was leading the policy evaluation study.

Following the delivery, steering group consultation and publication of the policy evaluation study report, this Commission Staff Working document (SWD) was prepared to accompany the INSPIRE Article 23 Report to Council and European Parliament.

Agenda Planning – Timing

Milestone 1: Stakeholder information – TOR of Policy Evaluation Project endorsed by the INSPIRE Article 22 Committee and Article 19.2 National Contact Points at 10th Meeting of the Committee.

Milestone 2: Establishment of the Policy Evaluation Steering Group by the Commission. The TOR foresees a Steering Group composed of DG ENV, JRC and EEA representatives, including the Commission Inter-Service Group COGI (Commission Inter-Service Group on Geographical Information), chaired by Eurostat. COGI has been since the conception of INSPIRE the Commission internal consultation and advice group on INSPIRE. In addition, the Directorate General Evaluation Correspondents, were invited to propose additional members for the policy evaluation Steering Group and/or provide comments to the project documents.

-The Terms of Reference (TOR) of the mid-term INSPIRE Policy Evaluation Project .

-The Background document to a public consultation as foreseen in the TOR.

-The draft ‘questionnaire’ prepared in view of the public consultation.

-Draft TOR for an external contract “direct observations of a sample of INSPIRE services and data sharing measures”

-The draft “INSPIRE Implementation Public Consultation” document contains a section as it appeared on the YOURVOICE website and a section with a draft ‘Questionnaire’ as it became available YOURVOICE.

Milestone 4 and 5: Public consultation 241 - All official languages.

Milestone 8: On behalf of DG ENV, the EEA conducted, with the technical support of the JRC a technical policy evaluation study. Following a review by Steering Group and processing of the comments received, the study report was presented and discussed with the stakeholders at the INSPIRE conference in May 2014. It was published in November 2014 as an official EEA/JRC Technical report 242 .

Evidence used in the evaluation/Fitness Check

Given the abundant availability of INSPIRE relevant documents and data sources preference was given to desk research, complemented with a survey organised as a public consultation (see SDW Annex 2) and an independent contract study with a focus on a sample of direct observations of INSPIRE services reported by the Member States.

The evaluation disposed of an extensive source of data and information acquired from the 2004-2010 State-of-Play studies, pre-INSPIRE and INSPIRE international conferences (1999-2013), national and cross-border conferences, official country reports (2010,2013), yearly country monitoring reports including indicators (since 2010), data sets and services provided through the EU Geo-portal, reports from EU and national related projects and activities, EU-national-international policy documents, public consultations and an independent assessment on the technical implementation of INSPIRE (2013).

External expertise

The independent contract study 243 provided external expertise with a focus on a sample of direct observations of INSPIRE services reported by the Member States.

ANNEX 4

Stakeholder consultation

The public consultation met all standing Commission standards (procedures applicable in 2013). The questionnaire and detailed statistics, graphs and findings and conclusions are available in: Public Consultation on INSPIRE Implementation http://inspire.ec.europa.eu/index.cfm/pageid/201/consultation/59835

Who was consulted

A questionnaire designed and translated in all EU official languages and open to all, addressing producers, users and coordinating bodies of spatial data and services – both from the public and private sectors, including the general public.

There were 698 completed replies by the end of the consultation from more than 30 countries (27 within the EU, 3 in the European Economic Area, 4 other European countries, and 2 from US/Canada). Thirty percent of replies came from only two countries (Germany and Spain) with over 100 replies each. 14 countries provided between 10 and 40 replies, and 13 countries provided fewer than 10. This skewed distribution does not allow a country by- country analysis of the results. It should also be noted that some countries had a process of internal consultation leading to a few consolidated replies reflecting a wider body of opinion than the simple number of replies would suggest.

Most respondents came from the public sector (68%) but it is noticeable that 13% also came from private citizens. The table below provides the absolute number of respondents by type (Numbers are rounded to nearest whole number so percentages do not add to 100.)

How the consultation was done

The questionnaire was published in all official EU languages on the European Commission web site for public consultations “Your Voice” and promoted internally in the Commission through the Steering Group which endorsed the questionnaire, through the INSPIRE website, INSPIRE Forum, and with direct mails to the INSPIRE national contact points, mailing lists of experts and participants to the INSPIRE conferences and registered Spatial data Interest Communities and Legally Mandates organisations, to the EEA EIONET and to all expert groups and committees involved on the environmental acquis. The Member States national contact points were requested as part of the Steering Group to further promote the participation of the stakeholders in their countries.

What did we consult on

The questionnaire firstly collected general information on the respondent, their status as well their interest (as user and/or producer, coordinating body) for each of the data themes covered by INSPIRE. They were asked to comment on their previous eventual involvement in the development and implementation of INSPIRE.

Secondly, the respondents were asked to express their views on a 5-point scale (agree strongly, agree, no opinion, disagree, disagree strongly) with regard to the progress made on the specific INSPIRE objectives – documentation, network services, data specifications – the presence and effectiveness of data sharing policies.

For examples: User Perspectives on documentation and services for INSPIRE Annex I Themes.

As ‘users’ they were asked to provide this for each of the data themes of interest.

As ‘producers’ they were asked to comment on the INSPIRE ‘readiness’ of their documentation network services, data specifications and data policies.

For example: Data Producer Responses on Existence of Data Sharing Policy in their Organisation towards Other Public Administrations

In addition general opinions related to the REFIT criteria - relevance, effectiveness, efficiency and coherence issues were asked.

Three open-ended questions were also provided to allow views on the key challenges encountered in implementing/using INSPIRE, key benefits, and key suggestions for changes for the future. For further detail see the full report of the 'Public Consultation on INSPIRE Implementation':

http://inspire.ec.europa.eu/index.cfm/pageid/201/consultation/59835

What are the results?

Almost 700 responses were received to the consultation from public and private sector, academia, and private citizens. The key messages from the public consultation are:

Relevance: - There was almost unanimous view across all participants in the public consultation that the objectives of INSPIRE of making spatial data and services more easily shared and used are still pertinent (92% agree, 2% disagree)

Effectiveness: - INSPIRE is starting to work and addresses the key barriers identified at the outset of this initiative that prevented the sharing and use of the spatial information needed to support environmental policies and policies affecting the environment.

Most progress has been done in documenting data, and making such data discoverable and viewable through web services. There are however delays, particularly for Annex I and II data that should all have become available by the time of the survey. Delays are also present for Annex III, both for completing the metadata and for making data available via download services.

The area of greater concern is the delay by the Member States in putting in place measures necessary to remove obstacles to the sharing of data at the point of use among public administrations. Only about half of the data producers indicated that such policy measure had been put in place in their organisation, and this was felt by users still finding data policy as a major barrier. Taking into consideration that such measures should have been in place since 2009, this delay is clearly significant.

Improving communication, and sharing of best practice, reducing as far as possible complexity of technical specifications, and improving coordination are key suggested changes.

Efficiency: - INSPIRE is delivering benefits to public administrations through improved data management processes and increased skills/competences in managing and publishing spatial data and related services. 46% of respondents agree that the benefits will be greater than the costs while 16% disagree and 38% did not express an opinion.

Coherence: - With regard to the EU Digital Single Market initiative: - 70% of the respondents agree that INSPIRE contributes to more general eGovernment initiatives with 7% disagreeing. 83% agree that INSPIRE contributes to more open data policies for the public sector with 5% disagreeing. With regard to other policies: - 70% agree that INSPIRE improves access to information for policies other than environment with 7% disagreeing. With regard to innovation 73% agree on the positive impact of INSPIRE with 8% disagreeing.

ANNEX 5

Methods and Analytical models used in preparing the evaluation

Applied Method:

The Commission evaluation practices allow for a wide range of evaluation methods. Such methods could include desk research, Delphi panels, SWOT analysis, statistical analysis, surveys, interviews, direct observations, field studies etc. Several 244 analytical methods can be applied to assess performance: Cost-benefit analysis (CBA), Multi-criteria analysis (MCA), Least cost analysis (LCA), Cost-effectiveness analysis (CEA), Counterfactual analysis (CA) and SWOT analysis (SWOT).

The purpose of the retrospective INSPIRE policy evaluation at its current stage of implementation is the assessment of the performance of the policy intervention against the objectives and targets laid down in the implementation roadmap.

Given the interim stage of implementation of INSPIRE, the nature of the policy instrument (a Directive complemented with implementing rules adopted as Commission Regulations and Decisions), and taking into account the available information, 245 preference has been given to apply a Multi-criteria analysis, MCA.

MCA is a technique to reach a judgement based on an explicit set of objectives and associated criteria. MCA allows the simultaneous assessment of effectiveness, efficiency and coherence of the policy and to capture and evidence distributional impacts (e.g. in terms of stakeholder types, EU regions/countries or time). MCA can also be used to assess the effectiveness of the implementation process where different implementation approaches have been pursued.

MCA can be applied as the INSPIRE implementation entails a number of specific objectives and quantitative targets/criteria which have to be met by a given deadline as documented in Annex 4: The detailed intervention logic of INSPIRE of this SWD. A number of these objectives are more qualitative, for example the Coordination objective and the Data and Services Policy objective. Others, such as the Documentation objective, the Interoperable services objective and Interoperable spatial data objective are monitored through quantitative indicators available at a Member State level and in the EU Geo-portal.

The use of CBA, Cost-benefit analysis had to be excluded at this stage of the implementation of INSPIRE given the lack of comparable Member State data reported for monetizing direct benefits and direct costs from an economic, social and environmental impact perspective. A methodology for conducting a CBA was developed and prepared 246 with the Member States in view of the 2013 report and made available  247 with other monitoring and reporting guidance documents and reporting templates. The Member State Report Temple section 12 Cost/Benefit Aspects (Article 16) provided guidance on cost and benefits categorisations and breakdowns. As an alternative, Member States could apply a REFIT focussed approach by reporting on a number of Efficiency, Effectiveness and Broader benefits categories more in line with a Cost-effectiveness analysis (CEA) than with a CBA. Where individual Member States conducted CBA and/or CEA, their findings and references to their studies are included in the SWD section on efficiency.

With regard to administrative burden resulting from INSPIRE, the 'Standard Cost Model for assessing administrative costs imposed by EU legislation' 248 could be applied to a certain extent to the Monitoring and Reporting Objective of INSPIRE. Some Member States provided more detail on the administrative costs incurred by public authorities for meeting and monitoring and legal reporting to the Commission.

Analytical Methods:

The interim INSPIRE policy evaluation did not use any analytical tool specified in the Better Regulation Toolbox 249 .

(1)   Directive 2007/2/EC, OJ L 108, 25.4.2007, p.1

(2)  The finalisation was delayed due to extensive analysis that was carried out also as a result of inclusion of the INSPIRE Directive in the REFIT programme.

(3)   COM(2012)746 final - EU Regulatory Fitness  

(4)  Communication - Regulatory Fitness and Performance Programme (REFIT) and Staff Working Document (SWD) (COM(2013)685 of 2 October 2013)

(5)  COM(2015)111. Note: The decision to carry out an evaluation under its REFIT programme in 2013 was well before the adoption of the Commission Guidelines for Better Regulation in May 2015 (COM(2015)111). As a consequence, this evaluation could not fully anticipate all the methodological and data needs now enshrined in the guidelines. Despite these issues, the evaluation guidelines have been used to the widest extent possible in the finalisation of the evaluation.

(6)  Commission Communication " A sustainable Europe for a better world: A European strategy for Sustainable Development " (COM(2001) 264)

(7)   2009 Review of the EU Sustainable Development Strategy COM (2009) 400

(8) Decision 1600/2002/EC laying down the Sixth Community Action Programme
(OJ L 242, 10.9.2002, p.  1)

(9)  Integrated across policy and economic sectors

(10)  COM(2004)516

(11) INSPIRE Directive Article 3(2), definition of 'spatial data': 'any data with a direct or indirect reference to a specific location or geographical area'. For example: spatial data can be a polygon with coordinates defining the borders of a protected site, the exact location of a point of emission into the environment, the borders of an industrial site, facility, building or an administrative unit. But it can also be data collected on species occurring within and referenced to such a protected site or the measurements taken by a sensor at the point of emission, it can be administrative data related to an industrial facility or statistical data related to the geographical area of the administrative unit such as for example, population density or a value describing the air quality at such a location during a certain period.

(12)  For further illustration, see https://www.youtube.com/watch?v=xew6qI-6wNk

(13)   http://www.eea.europa.eu/about-us/what/seis-initiatives/inspire-directive

(14)  INSPIRE Directive preamble (5) and Article 1

(15)  According to the thematic scope defined in the Annexes I, II and III of the Directive and detailed in INSPIRE - "Data Specifications" –D2.3: Definition of Annex Themes and Scope' , 2008

(16)  Metadata provides structured information allowing the spatial data to be discovered online, to know its origins, the conditions for use and to evaluate its fitness for purpose.

(17)  The Directive requires that common Implementing Rules (IR) are adopted in a number of specific areas (Metadata, Data Specifications, Network Services, Data and Service Sharing and Monitoring and Reporting). These IRs are adopted as Commission Decisions or Regulations. The Commission is assisted in the process of adopting such rules by a regulatory committee composed of representatives of the Member States and chaired by a representative of the Commission (this is known as the Comitology procedure).

(18)  The three annexes of the Directive cover 34 spatial data themes. Annex I contains basic data themes e.g. coordinate reference systems, addresses, cadastral parcels or transport networks; Annex II has geographic data themes e.g. elevation or land cover; Annex III covers environmental, health and energy data themes e.g. monitoring facilities, industrial, agricultural or aquaculture facilities, natural risk zones, habitats or energy resources.

(19)  The three annexes of the Directive cover 34 spatial data themes. Annex I contains basic data themes e.g. coordinate reference systems, addresses, cadastral parcels or transport networks; Annex II has geographic data themes e.g. elevation or land cover; Annex III covers environmental, health and energy data themes e.g. monitoring facilities, industrial, agricultural or aquaculture facilities, natural risk zones, habitats or energy resources.

(20)   Commission Regulation (EC) No 1205/2008  

(21)  Commission Regulation 1089/2010

(22)   http://inspire.ec.europa.eu

(23)   http://ec.europa.eu/isa/index_en.htm

(24)   https://ec.europa.eu/digital-single-market

(25)   https://ec.europa.eu/digital-single-market/en/news/infographic-egovernment-action-plan-2016-2020-glance

(26)  SWD(2016) 166, p. 10.

(27)  E.g. https://www.plumelabs.com/  or https://www.simacan.com or http://tn-its.eu/the-jrc-led-transportation-pilot-in-video/

(28)  E.g. http://www.directionsmag.com/entry/dutch-kadaster-tackles-european-inspire-initiatives-for-spatial-data-infras/122509 ;

(29)  E.g. https://joinup.ec.europa.eu/node/150019/

(30)   INSPIRE State-of-play summary and country reports 2004-2011

(31)   Spatial Data Infrastructures in Europe: State of Play 2007

(32)  BE, CY, LT, SL, IT, FR, SI

(33)  CZ, DK, NL, PL, PT

(34)  CY, PT, ES

(35)  DK, FI, CZ

(36)  DK, FI, NL, ES

(37)  ES, DE, NL, CZ, IT, IE, UK, BE, FR

(38)  The survey also included EEA/EFTA and Candidate Accession countries.

(39)  2007 1st INSPIRE Conference, ESDI for the Environment  

(40)  See all national reports and annual monitoring results at: http://inspire.ec.europa.eu/index.cfm/pageid/182/list/indicators

(41)  Communication - Regulatory Fitness and Performance Programme (REFIT) and Staff Working Document (SWD) (COM(2013)685 of 2 October 2013)

(42)  New guidelines covering all aspects of Better Regulation including evaluation were adopted in 2015. They can be found at: http://ec.europa.eu/smart-regulation/guidelines/toc_guide_en.htm .

(43)   Eurostat - Statistical requirements compendium, 2015, Page 146  

(44)   INSPIRE National Contact Points

(45)  The evaluation disposed of an extensive source of data and information acquired from the 2004-2010 State-of-Play studies, pre-INSPIRE and INSPIRE international conferences (1999-2013), national and cross-border conferences, official country reports (2010, 2013), yearly country monitoring reports including indicators (since 2010), data sets and services provided through the EU Geo-portal, reports from EU and national related projects and activities, EU-national-international policy documents, public consultations and an independent assessment on the technical implementation of INSPIRE (2013).

(46)   INSPIRE library

(47)   INSPIRE Conference archive

(48)   Summary Report INSPIRE Public Consultation 2013

(49)   INSPIRE Evaluation: Summary of findings for EU Member States - Assessing data and services metadata resources through direct observations, 17/09/2014 .

(50)   Mid-term evaluation report on INSPIRE implementation. EEA/JRC Technical report, 10/2014 .

(51)   INSPIRE: Template for country reports  25.01.2013

(52)   Cost and Benefits of implementing the INSPIRE Directive Workshop, 15-16/10/2012

(53)   INSPIRE Monitoring and reporting

(54)   Commission Decision regarding INSPIRE monitoring and reporting  05.06.2009

(55)   EU Geo-portal

(56)   INSPIRE Evaluation: Summary of findings for EU Member States - Assessing data and services metadata resources through direct observations, 17/09/2014 .

(57)   Report INSPIRE public consultation, 2013  

(58)  There were 698 completed replies by the end of the consultation from more than 30 countries (27 within the EU, 3 in the European Economic Area, 4 other European countries, and 2 from US/Canada). Thirty percent of replies came from only two countries (Germany and Spain) with over 100 replies each. 14 countries provided between 10 and 40 replies, and 13 countries provided fewer than 10. This skewed distribution does not allow a country by- country analysis of the results. It should also be noted that some countries had a process of internal consultation leading to a few consolidated replies reflecting a wider body of opinion than the simple number of replies would suggest. Source: Report INSPIRE public consultation, 2013

(59)   Commission Decision 2009/442/EC

(60)   2010 Member States reports

(61)  Except EL and HR

(62)   EU pilot : If a possible infringement of EU law is identified by the Commission or reported in a complaint, the Commission attempts to quickly resolve the underlying problem with the Member State concerned by means of a structured dialogue (EU Pilot).

(63)   Commission Regulation (EU) No 268/2010 - as regards the access to spatial data sets and services of the Member States by Community institutions and bodies under harmonized conditions

(64)  DE, DK, ES,FR, IT, NL, SE, CY, CZ, EE, HU, IT, PL, SI, SK

(65)  For example: The 2011 UK Location Strategy provides guidance to support the public authorities in licensing their data based on an UK Government Licensing Framework .

(66)  SE, FR, FI, IE, LT, LU, LV, DE, ES

(67)  UK Public Sector End User Licence INSPIRE

(68)  For example: The UK Location Council endorsed the UKGLF as the licensing framework for the use of spatial information covered by the INSPIRE Regulations

(69)  DK, FR, FI, IE, UK, NL, DE, DE, ES, AT

(70)  AT, BE, CY, CZ, DK, EE, FR, DE, LV, LT, PL, PT, SK, ES, UK

(71)  SE, DK, FI, IE, DE, UK, CY, EL, AT, CZ

(72)  AT, EE, BG, CZ, SL, HU, SK, LT, MT, RO , BE

(73)  DK, EE, ES, FI, NL, LU, PL, SE

(74)  AT BE BG CZ CY DE FR EL FR IT HU LT LV RO SI SK UK

(75)   European Commission – Comitolgy Register

(76)   INSPIRE Work Programme Transposition Phase 2007-2009 and Update to the INSPIRE Work programme 2007-2009: status of the Implementing Rules Development.

(77)   INSPIRE Roadmap for adoption implementing rules

(78)  In the period 2007-2014 19 public consultation s were held

(79)   INSPIRE implementing rules adoption roadmap

(80)   INSPIRE Implementing Rules

(81)   INSPIRE Technical Guidelines

(82)  For example; Article 6 - Member States shall create the metadata referred to in Article 5 in accordance with the following timetable: (a) not later than two years after the date of adoption of implementing rules…

(83)   Maintenance and Implementation Framework, MIF

(84)   Maintenance and Implementation Work Programme, MIWP

(85)   Commission Decision 2009/442/EC

(86)   2010 Member States reports

(87)  According to the 2011 survey on coordination, funding and sharing measures, the surveying and Mapping Agencies are leading in 14 countries, Environmental Agencies in 10, and ICT Agencies in Bulgaria and Malta.

(88)   2010 and 2011 State-of-play reports

(89)   Mid-term evaluation report on INSPIRE implementation. EEA/JRC Technical report, 10/2014 .

(90)  DK, FI, SE, PT, HU, AT, EL, LU, PL, RO, SI, SK, LT, EE, LV, CY, MT and BG

(91)  Except BE, HU

(92)  Such as AT, DE, FR, GR, UK, ES, UK, IT

(93)  For example for France, status 2012: Consider the regional level as the key level for the successful implementation of INSPIRE. For the 27 regions, 17 established a regional coordination structure, 3 have two extending to the local authorities, 6 were in the course of establishing a coordination structure, one still outstanding.

(94)   INSPIRE & NSDI State of Play: D3.2 - Detailed survey on use and usability of INSPIRE & NSDI

(95)  SE carries out annual user surveys and integrates that information together with information from the regulatory yearly monitoring in a dashboard using the Balanced Score Card methodology. Three perspectives are taken into account: the data and services aspects, user satisfaction and cooperation. In the survey, nine questions relate to the level to which the availability and usability of data and services corresponds to user demand and expectations. The user survey includes also possibilities to comment on the implementation of INSPIRE and NSDI. Besides the involvement of user representatives in the coordinating structure, there is a user forum and complaint mechanism. Source: http://ijsdir.jrc.ec.europa.eu/index.php/ijsdir/article/view/192/292  

(96)  Except BE (websites at regional WL,VL,BXL level), IE

(97)   INSPIRE in your country

(98)  Except HU, RO, BG, IE

(99)  Public authorities, the public and for cross-border purposes (as requested in the  Commission Decision 2009/442/EC on reporting and monitoring)

(100)  25/28 AT, BE, BG, CY, CZ, DK, EE, EL, ES, FI, FR, HU, HR, , LT, LU, LV, MT, NL, PT, RO, SE, SI, SK, UK

(101)  For example: Scotland's Environment Web

(102)  BG (WFD, Floods), EE (Floods), PL (EIA), UK (Floods, Noise, WFD, Air Quality), FR (WFD, MSFD, NL (Air quality reporting).

(103)  Annex I and II covers 13 spatial data themes such as transport networks, buildings, addresses, hydrography, cadastral parcels, administrative units, protected sites, digital elevation, and bathymetry, land cover, geology and images taken by satellite or airborne sensors.

(104)   Commission Regulation (EC) No 1205/2008 of 3 December 2008 implementing Directive 2007/2/EC of the European Parliament and of the Council as regards metadata, OJ L 326, 04/12/2008, p. 12–30

(105)  Monitoring: According to Commission Decision 2009/442/EC of 5 June 2009 implementing the INSPIRE Directive, EU Member States have to report annually a number of indicators for monitoring the implementation and use of their infrastructures for spatial information. The information provided includes a list of spatial data sets and services belonging to those infrastructures.

(106)  Footnote 50

(107)   Commission Regulation (EC) No 976/2009 of 19 October 2009 implementing Directive 2007/2/EC of the European Parliament and of the Council as regards the Network Services  19.10.2009

(108)   Commission Regulation amending Regulation (EC) No 976/2009 as regards download services and transformation service  08.12.2010

(109)   The Basic Data Programme – A Danish Infrastructure Model for Public Data , page 42, 2014

(110)   COMMISSION REGULATION (EU) No 1089/2010 of 23 November 2010 implementing Directive 2007/2/EC of the European Parliament and of the Council as regards interoperability of spatial data sets and services  08.12.2010

(111)   COMMISSION REGULATION (EU) No 1253/2013 of 21 October 2013 amending Regulation (EU) No 1089/2010 implementing Directive 2007/2/EC as regards interoperability of spatial data sets and services  10.12.2013

(112)   INSPIRE Interactive Data specification site

(113)   LINKVIT , SMEspire , e ENV plus,

(114)   HUMBOLDT Alignment Editor (HALE) , FME INSPIRE solutions pack , GO Publisher

(115)  'Newly created' applies to ALL spatial data collected AFTER the adoption of INSPIRE on 15 May 2007.

(116)  BE, AT DE, FI, UK and ES

(117)  See Footnote 30 

(118)   Spatial Data Infrastructures in Europe: State of Play 2007

(119)  Driven by the EU Digital Agenda/Digital Single Market: 'The Commission's work in the area of open data is focussing on generating value through re-use of a specific type of data – public sector information, sometimes also referred to as government data. That is all the information that public bodies produce, collect or pay for. Examples are: geographical information, statistics, weather data,.." which includes "Achieving efficiency gains through sharing data inside and between public administrations" and " •Fostering participation of citizens in political and social life and increasing transparency of government".

(120)   European legislation on reuse of public sector information – Summary of Directive 2013/37/EU

(121)   Summary Report INSPIRE Public Consultation 2013

(122)  Example of local level good practice coordination: UK Local Governments Association – A guide to INSPIRE compliance in Local Government

(123)  INSPIRE 2011 State-of-play report

(124)  NL however already identified Annex III data sets – for example: data related to meteorology and atmosphere - see: KNMI Impact Analysis INSPIRE – Internal report, 2008

(125)  ES (38%) followed by DE (12%)

(126)  As initially documented in INSPIRE D.2.3 "Data Specifications" – Definition of Annex Themes and Scope" 2008 and refined in the INSPIRE Technical Guidelines 2013.

(127)  Based on the analysis of the EU level INSPIRE register of spatial data interest communities (SDIC: 513) and organisations with a legal mandate (LMO: 280) for spatial data falling under one or more of the INSPIRE data themes. The 2013 public consultation confirms this assessment with similar results as shown Figure 17 (User demand versus producer supply for INSPIRE spatial data).

(128)  The INSPIRE register of organisations (SDIC/LMO) was established falling a continuous open call for interest since 2006 to allow wide variety of organisations to participate in the implementation of the INSPIRE Directive and, in particular, the technical preparation of implementing rules.

(129)   Commission Regulation (EC) No 1205/2008 of 3 December 2008 implementing Directive 2007/2/EC of the European Parliament and of the Council as regards metadata, OJ L 326, 04/12/2008, p. 12–30

(130)  The INSPIRE Metadata editor makes it possible to create INSPIRE compliant metadata

(131)  Respondent to the public consultation cannot assess if metadata is conform to the technical specifications.

(132)  23/11/2017 for Annex I spatial data sets, 21/10/2020 for Annex II and III spatial data sets.

(133)  AT, CY, EE, EL, ES, FR, HU, HR, LT, LV, PT, SI, UK

(134)   ArcGIS for INSPIRE

(135)   INSPIRE and Data Management” Training ,

(136)   INSPIRE Training with GO Publisher , GeoSolultions INSPIRE Support , LINKVIT , GeoSmartCity INSPIRE Training , GISIG E-learning platform , GGP INSPIRE Training and support , Compass INSPIRE Solutions

(137)   Providing INSPIRE-compliant access to utility services: the case of sewage networks in Flanders, Belgium

(138)   Dutch cadastre INSPIRE conform , The German Marine Data Infrastructure , UK National Biodiversity Network to deliver INSPIRE compliance for species data , Geodateninfrastruktur Deutschland: INSPIRE success story – Implementing e-reporting of air quality based on INSPIRE at national level

(139)   One-Geology Europe , European Location Framework, ELF , Reporting and exchanging air quality information using e-Reporting

(140)  Note: member States who did report, mostly reported costs for a 3 year period 2010-2012.

(141)  20% of UK Location Programme 2009-2014 budget 11.0 million/5 years http://inspire.ec.europa.eu/reports/stateofplay2011/rcr11UKv122.pdf  

(142)   Contribution to the extended impact assessment of INSPIRE , 24/09/2003

(143)  UK, NL, SE, LT

(144)   http://www.poweredbyinspire.eu//documents/0403-sustainability-carlyle.pdf .

(145)   UKLP - Benefits Realisation Strategy, 2012

(146) See http://inspire.ec.europa.eu/events/conferences/inspire_2012/presentations/66.pdf

(147) INSPIRE, The Dutch way   “” - Observations on implementing INSPIRE in the Netherlands, The Dutch response to the Public consultation on the implementation of INSPIRE

(148)   Costs-benefits analysis INSPIRE in The Netherland, 2009 .

(149)   Rapport: Kosten-Baten Analyse INSPIRE Nederland 2016

(150)   Country Report of Sweden on the Swedish Spatial Data Infrastructure to United Nations Committee of Experts on Global Geospatial Information Management (UN-GGIM ), 2015

(151)   Country Report of Denmark to the United Nations Committee of Experts on Global Geospatial Information Management (UN-GGIM) , 2014

(152)  DE, DK, ES, FI, NL, SE and UK.

(153)   The Basic Data Programme — A Danish Infrastructure Model for Public Data , 2014.

(154)   Geodateninfrastruktur Deutschland: INSPIRE success story – Implementing e-reporting of air quality based on INSPIRE at national level

(155)  See footnote Error! Bookmark not defined.

(156)  See footnote 149

(157)   SEC(2004) 980

(158)   Inter-institutional File: 2004/0175(COD ) - Communication from the Commission to the European Parliament pursuant to the second subparagraph of Article 251 (2) of the EC Treaty concerning the common position of the Council on the adoption of a Directive of the European Parliament and of the Council establishing an infrastructure for spatial information in the Community (INSPIRE), 14 February 2006.

(159)   Contribution to the extended impact assessment of INSPIRE .

(160)  As discussed at the INSPIRE Maintenance and Implementation Expert Group in December 2015.

(161)  LPIS - Land Parcel Information System in Romania  

(162)  EC Reducing Administrative Burdens -– Standard Cost Model http://ec.europa.eu/smart-regulation/refit/admin_burden/scm_en.htm  

(163)  As discussed at the INSPIRE Maintenance and Implementation Expert Group in December 2015.

(164)  http://inspire-geoportal.ec.europa.eu/

(165)  http://ec.europa.eu/environment/legal/reporting/fc_overview_en.htm

(166)  UK, SE, DK, FI, DE, NL, AT, LV, EE, LT, LU, BE (VL) and SI

(167)  Which is one of the reasons why the INSPIRE implementation now features in the EU's eGovernment Action Plan (COM(2016)179).

(168) European NEtwork for Redistributing Geospatial Information to user Communities - Open Data For example: Competiveness and Innovation Programme: ENERGIC OD - , Start date: 2014-10-01, End date: 2017-09-30,

(169)

For example: SANY - Sensors Anywhere , FP6-IST , Start date: 2006-09-01, End date: 2009-08-31, SMARTOPENDATA - Linked Open Data for environment protection in Smart Regions , FP7-ENVIRONMENT, Start date: 2013-11-01, End date: 2015-10-31

(170)   Interoperability Solutions for European Public Administrations

(171)   Instrument for Structural Policies for Pre-Accession (ISPA)

(172)   COM(2015) 192

(173)   The Sixth Community Environment Action Programme FINAL ASSESSMENT , COMM (2011) 531 Final  

(174)   Seventh Environment Action programme 2013-2020 , DECISION No 1386/2013/EU

(175)   Commission Communication on Improving the delivery of benefits from EU environment measures: building confidence through better knowledge and responsiveness

(176)  See Roadmap

(177)  See Commission Work Programme 2016 (COM(2015) 610, Annex 2)

(178)   DIRECTIVE 2010/40/EU on the framework for the deployment of Intelligent Transport Systems in the field of road transport and for interfaces with other modes of transport

(179)   Regulation (EU) No 377/2014 establishing the Copernicus Programme  

(180)  See for example: http://www.eurogeoss.eu/default.aspx

(181)   UN-GGIM: EUROPE 2014 – 2017 Work Plan

(182)   World Bank Group – Croatia Partnership Country Program Snapshot , April 2015

(183)  Commission Communication on Regulatory Fitness and Performance Programme (REFIT) -  COM(2014)368

(184) Reducing environmental risk through INSPIRE (quantified benefits) UK Environment Agency, 2013.

(185)   Geodateninfrastruktur Deutschland: INSPIRE success story – Implementing e-reporting of air quality based on INSPIRE at national level

(186)  94 % agree — 1 % disagree — 5 % no opinion: page 28 Summary Report INSPIRE Public Consultation 2013 .

(187) 16 Member States, AT, BE – no opinion, CZ, DK, FI, DE+ 4 regional, PL, PT, RO, SL, SI, ES, UK, LV, LU, NL) +NO +Serbia

(188)   Summary Report INSPIRE Public Consultation 2013 , page 22

(189)   Common strategy for implementing Directive 2000/60/EC - Updated Guidance on Implementing the Geographical Information System (GIS) Elements of the EU Water policy

(190)   2011/850/EU: Commission Implementing Decision of 12 December 2011 laying down rules for Directives 2004/107/EC and 2008/50/EC of the European Parliament and of the Council as regards the reciprocal exchange of information and reporting on ambient air quality (notified under document C(2011) 9068)  

(191)   COMMISSION IMPLEMENTING DECISION concerning a site information format for NATURA 2000 sites (2011/484/EU)  

(192)   ENV.C.3/SER/2014/0004 Call for tender- SERVICE CONTRACT FOR THE DEVELOPMENT OF ELECTRONIC REPORTING TOOLS FOR THE INDUSTRIAL EMISSIONS DIRECTIVE 2010/75/EU (reporting period 2013-2016)

(193) Commission Regulation (EU) No 1089/2010 as last amended by Regulation (EU) No 1312/2014.

(194)  AT, BG, DE, UK, LT,PT

(195)  COM(2015)215 and Commission Work Programme 2016 (COM(2015) 610, in particular Annexes 2 and 5 thereof.

(196)   http://ec.europa.eu/environment/legal/reporting/fc_overview_en.htm

(197)   Directive 2003/4/EC

(198)   Directive 2014/89/EU establishing a framework for maritime spatial planning

(199)  Communication on A Digital Single Market Strategy for Europe , COM(2015) 192 final

(200)  Commission Staff Working Document, A Digital Single Market Strategy for Europe - Analysis and Evidence, COM(2015) 192 final  

(201)  See action 19 in COM(2016) 179

(202)  COM(2016) 178

(203)   Communication on Open Data , COM(2011) 882 final

(204)   Directive 2013/37/EU amending Directive 2003/98/EC on the re-use of public sector information  

(205)   INSPIRE Empowers Re-Use of Public Sector Information, Bastiaan van loenen, Michel Grothe, International Journal of Spatial Data Infrastructures Research, 2014, Vol.9, 86-106

(206)   Local government needs structure to exploit data , New Local Government Network ( NLGN ), Report , April, 2015.

(207)   INSPIRE and Open data – Local action, Ashfield District Council publishes data under the European INSPIRE Directive and as Open Data, Updated March 2015.

(208)   EU Project Publica Mundi , Publishable Summary , 2014

(209) 2012 - RE-USE OF PUBLIC SECTOR INFORMATION – Catalogue and highlights of studies, cases and key figures on economic effects of changing policies , 2012

(210)  COM(2010) 744, Towards interoperability for European public services

(211)   https://joinup.ec.europa.eu/asset/dcat_application_profile/description

(212)  Regulation (EU) No 911/2010 of the European Parliament and of the Council of 22 September 2010 on the European Earth monitoring programme (GMES) and its initial operations (2011 to 2013).

(213)   http://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1407245998429&uri=CELEX:32014R0377

(214)  See for example: http://www.eurogeoss.eu/default.aspx

(215)   Common Information Sharing Environment for the EU maritime domain, CISE  

(216)   COM(2014) 254 final: Innovation in the Blue Economy: realising the potential of our seas and oceans for jobs and growt h. "Three further EU initiatives, the Copernicus Marine Service, the Data Collection Framework for fisheries16 and WISE-Marine for environmental data will be integrated with EMODnet using common standards such as INSPIRE and comply with the principles of the Shared Environmental Information System "

(217) Directive 2014/61/EU   on measures to reduce the cost of deploying high-speed electronic communications networks

(218)   Commission Regulation (EC) No 1122/2009  

(219)   Directive 2014/89/EU establishing a framework for maritime spatial planning and integrated coastal management

(220)   REGULATION (EU) No 1255/201 , Article 3(c) 'a comprehensive and publicly accessible high quality marine data and knowledge base which facilitates sharing, reuse and dissemination of these data and knowledge among various user groups using existing data, thus avoiding duplication of the databases; for this purpose, the best use shall be made of existing Union and Member State programmes, including INSPIRE and GMES'.

(221) Action Plan and Directive 2010/40/EU  on the framework for the deployment of Intelligent Transport Systems in the field of road transport and for interfaces with other modes of transport

(222)   Commission Delegated Regulation (EU) 2015/962 with regard to the provision of EU-wide real-time traffic information services.

(223)   INSPIRE SDIC – ESPON Programme  

(224)   Union civil protection legislation (Decision on a Union Civil Protection Mechanism) and Guidance for Recording and Sharing Disaster Damage and Loss Data

(225)   https://data.europa.eu/europeandataportal

(226)   COMMISSION STAFF WORKING PAPER SEC(2010) 1626 final - Risk Assessment and Mapping Guidelines for Disaster Management

(227)   INSPIRATION project  

(228)   INSPIRE Forum – projects  

(229)   http://inspire.ec.europa.eu/index.cfm/pageid/501

(230) EUROPEAN INTEROPERABILITY FRAMEWORK FOR PAN-EUROPEAN eGOVERNMENT SERVICES, 2004 "An interoperability framework can be defined as a set of standards and guidelines that describes the way in which organisations have agreed, or should agree, to interact with each other." Source: .

(231)   https://ipchem.jrc.ec.europa.eu/

(232)   http://ec.europa.eu/environment/chemicals/reach/reach_en.htm

(233)   Geodateninfrastruktur Deutschland: INSPIRE success story — Implementing e-reporting of air quality based on INSPIRE at national level .

(234)   The German Marine Data Infrastructure and the Marine Strategy Framework Directive , 2015

(235)   Terms of Reference : Policy Evaluation for the INSPIRE Article 23 - 2014 Commission Report to Council and European Parliament

(236)  COGI: services represented: SG, DIGIT, COMP, DEVCO, EAC, REGIO, ENER, ELARG, AGRI, EEAS, CNECT, SANCO, ECHO, EMPL, R&I, HR, OIB, COMM, MARE, JRC, DEVCO, MARKT, REGIO, COMP, ESTAT, MOVE, OP, EASME, EAC, GROW, OIL, EMPL, TAXUD , CLIMA, TAXUD

(237)   INSPIRE National Contact Points

(238)  REFIT/policy evaluation INSPIRE - public consultation press release http://europa.eu/rapid/press-release_IP-13-1216_en.htm

(239)  Report public consultation: http://inspire.ec.europa.eu/reports/consultations/INSPIRE_Public_Consultation_Report_final.pdf

(240)  Note: The study report was initially planned for May 2014. However, due to the inclusion of INSPIRE in the REFIT process in October 2013 the TOR and roadmap for its delivery were adjusted to take account of the specific REFIT requirements.

(241)  Public consultation: http://inspire.ec.europa.eu/index.cfm/pageid/201/consultation/59835  

(242) Mid-term evaluation report on INSPIRE implementation. EEA/JRC Technical report, 10/2014 .

(243)   INSPIRE Evaluation: Summary of findings for EU Member States - Assessing data and services metadata resources through direct observations, 17/09/2014 .

(244) European Commission Better Regulation Toolbox: Useful analytical methods to compare options or assess performance  

(245) 3 yearly Member State reports, yearly reported monitoring indicators, state-of-play studies, public consultation and a technical study evaluating the existing components of the respective INSPIRE infrastructures with a focus on a sample of direct observations of INSPIRE services reported by the Member States.

(246)   Workshop on 'Cost and Benefits of implementing the INSPIRE Directive', JRC, 2012 .

(247)   INSPIRE Monitoring and Reporting – legislation, guidelines and supporting documents .

(248)   European Commission Better Regulation Toolbox: Standard Cost Model for assessing administrative costs imposed by EU legislation

(249)   European Commission Better Regulation Toolbox: The use of analytical models in impact assessment or evaluation .