52013DC0883

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL

ON THE FINAL EVALUATION OF THE IMPLEMENTATION OF THE COMMUNITY STATISTICAL PROGRAMME 2008-2012

1.           Introduction

Decision No 1578/2007/EC[1] of the European Parliament and of the Council on the Community Statistical Programme 2008 to 2012 (CSP) was adopted on 11 December 2007.

Article 6 of the Decision states that ‘At the end of the period covered by the programme, the Commission, after consulting the Statistical Programme Committee, shall present an evaluation report on the implementation of the programme, taking into account the views of independent experts. That report shall be completed by the end of 2013 and subsequently submitted to the European Parliament and the Council.’.

Article 6 adds that ‘Reporting on the outcome of reprioritisation, including estimations of costs and burdens for statistical projects and fields covered by this statistical programme, as well as an assessment of emerging statistical needs, in particular for new Community policies, shall form part of the intermediate progress report and the final evaluation report.’.

This document constitutes the required final evaluation report and is based on a report by an external evaluation company and on the mid-term evaluation report. Chapter 2 summarises the CSP’s main achievements, chapter 3 presents the outcome of the reprioritisation and chapter 4 the implementation of Eurostat’s Vision communication[2], a major new initiative that, although issued after the CSP was adopted, reshaped the activities in 2010-12 and is also relevant for going forward. Chapter 5 presents considerations on the recommendations made by the external consultant.

The executive summary of the final report of the external consultant can be found at http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/Report_expert.pdf.

2.           Main Achievements of the Programme

This chapter focuses on the achievements gained over the course of the programme and covers all its objectives.

Summary

The CSP contains over 130 objectives, the majority of which achieved a high degree of implementation. Institutional objectives and those aiming to improve the legislative framework had the highest degree of achievement, — 100 % and 95 % respectively of their objectives could be considered achieved — while some 10 % of the production-related objectives were not adequately achieved. More detailed information can be found in the executive summary of the report of the external consultant. As regards these objectives, account must be taken of the fact that European statistics are based on those produced by Member States (MSs), with the result that the realization of some objectives suffer from the constraints in terms of production processes and resources available in the MSs. Resources also had to be redeployed to meet new statistical needs that emerged during the CSP period, particularly in the areas of macroeconomic and social statistics due to the financial crisis.

The financial implementation was generally good, showing a trend towards increased efficiency.

Cross-cutting issues

The role of statistics in European integration

The process of preparing the new European system of accounts (ESA 2010) culminated in the adoption of the ESA 2010 Regulation by the European Parliament (EP) and the Council in May 2013[3], and the implementation of the NACE Rev. 2 classification advanced considerably. A Union register of multinational enterprise groups is expected to be completed by end of 2013 and is already being used by some MSs. Statistical and IT standards for the exchange of data and metadata became increasingly available within the European Statistical System (ESS), e.g. the Statistical Data and Metadata eXchange (SDMX) standards, and the ESS Metadata Handler was developed for the production and dissemination of harmonised ESS metadata. Internet usage steadily increased, also for purposes other than data dissemination. Policies and tools for harmonised confidentiality management in the ESS were developed. The portal for Cooperation among Researchers and Official Statisticians is being used to store the results of research in official statistics and open source software tools such as ‘Demetra’ and ‘R’ were promoted and made available on the Open Source Observatory and Repository website.

Relations with stakeholders

The ESS

The ESS Committee (ESSC) endorsed a revised version of the European Statistics Code of Practice (CoP) in 2011 and updated the ESS Quality Assurance Framework (v1.1) at the end of 2012. The European Statistical Governance Advisory Board reports annually to the European Parliament and the Council on the implementation of the CoP. The European Statistical Training Programme was developed annually as part of a long-term strategy.

Cooperation with users

The creation of the European Statistical Advisory Committee (ESAC) helped to improve communication with different types of users. A general Eurostat user satisfaction survey was conducted in 2009 and has been carried out annually since 2011. User surveys were also carried out for specific domains subject to a rolling review. User satisfaction was also assessed through personal contacts with users, such as in working groups and the hearings inside the Commission.

Technical cooperation with third countries and cooperation with international organisations

Eurostat collected and published a wide range of data for the enlargement process and, to a lesser extent, for the European Neighbourhood countries. They were produced with the help of regional cooperation programmes implemented by Eurostat.

Instruments

Better regulation and the balance of costs and benefits

There has been an increase in the number of domains now governed by legislation (energy, environmental accounts) in order to ensure timely and complete data transmission by MSs. Moreover, the basic legal act – Regulation on European Statistics[4] – was adopted. By the end of CSP, further amendments to this act were proposed and the legislative procedure was initiated. Proposals for simplifying and recasting legislation were submitted in the domains of energy, transport, Intrastat, structural business statistics (SBS), agriculture, tourism and statistical returns on carriage of goods by road. Some changes are already in place, for example for Intrastat, agriculture, transport and tourism. More details are given in the section of statistical domains per title. A priority setting mechanism in the ESS was developed, based on a strategy-driven approach, but measuring burden and costs proved to be difficult as they are not calculated by all MSs in a systematic or homogenous way for all statistical fields. More information can be found in the chapter on reprioritisation.

Compliance monitoring

Regular compliance monitoring for enhancing the quality of European statistics was implemented.

Increased ability to respond to users’ needs

Exploratory work to combine different statistical sources was completed by merging data from different surveys (e.g. Information and Communications Technologies (ICT) and SBS to estimate ICT impact, statistical matching of social surveys, trade and business data, and greenhouse gas emissions from energy and aviation). Such combination is routinely carried out by several MSs with administrative and survey data. The use of ad-hoc modules was extended in the areas of social and business statistics in order to increase responsiveness to new user needs.

Financial support for the programme period

A financing decision was adopted annually for the operational implementation of the work programme. The financial execution of the available credits was satisfactory and monitored at central level. The grant policy was oriented towards projects with a European dimension, in particular multi-beneficiary grant agreements. Since 2011 interested beneficiaries have had the option of using standard scales of unit costs by staff category and a flat rate for indirect costs.

Utilising partners’ expertise for Union purposes

New types of collaborative networks (ESSnets) were developed to share expertise and channel knowledge and resources in the ESS. They encompass project work performed by some MSs, with the results being made available for the entire ESS and being used in many domains. A new policy was adopted by the ESSC in 2012 that emphasised embedding ESSnets projects in the ESS.VIP (Vision infrastructure projects) programme and establishing Centres of Competence.

Dissemination

Following the launch of the revamped website in 2009 with new sections such as ‘Statistics Explained’ and improved functionalities, in 2010 a new dissemination chain was put in place. It was based on a common reference database used by all statistical domains at Eurostat, thereby improving efficiency and facilitating the integration of statistics across different domains. The number of downloads has been steadily increasing. The Census Hub is a valuable experience of collaboration across the ESS for dissemination based on common standards.

Cross-sectional activities

The Europe 2020 strategy replaced the Lisbon strategy and Eurostat published new indicators to replace the structural indicators. The list of sustainable development indicators (SDIs) was updated. The GDP and beyond initiative has led to improvements and the production of new indicators in a number of statistical areas, in particular economic, social and environmental statistics. Among others, the initiative was a stimulus to elaborate the set of quality of life indicators. Quality profiles of European indicators were developed and made available. Dissemination was improved with the creation of Statistics explained pages on SDIs and of a Europe 2020 indicators web page.

Support was provided for the enlargement and to the European Neighbourhood countries to strengthen their statistical systems for building sustainable capacity to produce good quality statistics, in line with European standards serving the needs of evidence-based policy decisions.

Statistical domains per title

Free movement of goods: New legislation was adopted in 2009 for Intrastat[5], introducing new exemption thresholds which enable MSs to significantly reduce the response burden on enterprises. Methods and tools for better integration of the different types of statistical information required from businesses were developed in the Modernisation of European Enterprise and Trade Statistics (MEETS) programme[6]. In September 2010, the task force on the reconciliation between Balance of Payments (BoP) and International Trade Statistics (ITS) issued its final report. In 2012 the SIMSTAT (Single Market Statistics) project was launched containing several simplification measures, in particular the proposal to enable the exchange of micro-data (enterprise level data) on intra-EU exports in goods between MSs.

Agriculture: The legislative bases for agricultural statistics (census, farm structure survey (FSS)[7], agricultural production[8], livestock and meat statistics[9]) were adopted and/or updated. Data from the 2007 FSS sample survey and from the agricultural census 2009/10 were published on Eurostat’s website. Data on production (crops, livestock) and rural and agro-environmental indicators were released. With a view to simplification, it was decided not to collect information on the income of the agricultural households sector at a harmonised EU level, due to the difficulty for MSs to produce comparable statistics and to high costs and the response burden. Efforts were made to meet the statistical information needs for the Commission’s maritime policy. A Regulation on permanent crops[10], simplifying previous legislation, was adopted.

Free movement of persons, services and capital: The NACE Rev. 2 was implemented. Key legislations on BoP[11], ITS, foreign direct investment and foreign affiliates (FATS)[12] was implemented and updated. Data on the internationalisation of R&D were the subject of feasibility studies. The methodology for statistics on residence permits was developed and implemented.

Justice, freedom and security: Implementing measures on migration and asylum statistics were completed in 2010[13]. An ad-hoc module on the labour market integration of migrants was conducted. The 2011 Action Plan for measuring crime in the EU established the need to complement the available data with statistics on victims and an EU victimisation survey was proposed.

Transport: The legal basis for transport statistics was consolidated and a modal split indicator (road, rail, inland waterways) was made available. Some initial data on urban transport and additional performance indicators were published but not intermodal transport indicators. The Regulation on Maritime Transport statistics was simplified[14].

Economic and monetary policy: Eurostat contributed to the review of the UN System of National Accounts that was finalised in 2009 and to the preparation of new BoP Manual (BPM6). The process of preparing the new European system of accounts (ESA 2010) was carried out and concluded with the adoption of the ESA 2010 Regulation by the European Parliament and the Council in May 2013. The methodological update also included the implementation of NACE Rev. 2 in most areas of economic statistics. Compliance monitoring of national accounts and prices was strengthened, which led to noticeably improved data availability in terms of quantity and quality. Evidence on identifying differences in labour market statistics and national accounts data on employment is available, and the coherence between ESA 95 and employment growth (by branches) in the EU labour force survey (LFS) appears in the published metadata. The Greek crisis was at the origin of a renewal of the capacity of Eurostat to provide support to economic and monetary policies. A new Directorate focusing on public finance statistics was created, and Eurostat was given enhanced supervisory powers.

Employment: The annual ad-hoc modules to the LFS were successfully implemented. All planned outputs were achieved regarding the implementation of the LFS, its legal basis and the dissemination of results. Regional and urban data on employment and unemployment were produced. The use of European sampling schemes for the production of monthly unemployment data was not pursued as the growing number of countries made it possible to provide monthly estimates. The coverage of the Structure of Earning Surveys and the Labour Cost Survey was improved, the classification adapted to NACE Rev. 2 and the data disseminated, including microdata sets. A legal basis was adopted for job vacancy statistics[15].

Common commercial policy: The new Extrastat Regulation[16] was implemented and the methodology for trade statistics was adapted to the Manual on Statistics in International Trade in Services 2010. The MEETS programme was launched. The legal bases for BoP and FATS were updated.

Social policy, education, vocational training and youth: There was a greater need for social statistics data during the crisis, increasing their relevance, especially the EU survey on income and living conditions. It was consolidated and its dissemination improved (longitudinal microdata). A programme of modernisation of social statistics was launched. As part of its social dimension, the Beyond GDP initiative pushed for the measurement of well-being and multi-dimensional poverty in the EU. This is more ambitious than the measurement of the Human Development Index which was targeted in the CSP. Legal bases for social protection statistics[17], for the 2011 population and housing census[18], and for statistics on education and lifelong learning[19] were implemented, providing better harmonisation of surveys on Adult Education (AES) and Continuous Vocational Training. A proposal for a demography Regulation was also adopted by the Commission[20] and population data required for the purpose of qualified majority voting were made available on an annual basis. The Census Hub portal was developed. Microdata for AES were made available. A dedicated section on youth was opened on Eurostat’s website. An overview of all gender-related statistics was prepared, specific data on gender were produced, and Eurostat signed a Memorandum of understanding with the European Institute for Gender Equality.

Public health: A framework Regulation on public health and health and safety at work[21] was adopted and completed with two implementing regulations for statistics on causes of death and accidents at work[22]. A European Health Interview Survey was conducted and a European Health and Social Integration Survey was launched. A manual for the System of Health Accounts was released in 2011.

Consumer protection: Consumer price statistics are well developed in the ESS, as major economic indicators with high frequency and detail. Data on consumer empowerment were available through a Eurobarometer. Statistics on organic farming are under production, but data on control and monitoring activities and the food consumption chain were discontinued as food safety statistics were on the list of negative priorities.

Industry: A major milestone in improving business statistics is the MEETS programme, covering the period 2009-13 and implemented under a different financial envelope. In addition, a roadmap for the improvement and integration of business statistics legislation (FRIBS) was launched in 2012. The community innovation surveys (CIS) for 2008 and 2010 were implemented and the 2012 survey prepared. Globalisation and entrepreneurship indicators were developed. NACE Rev. 2 was implemented in the area of business statistics. The information society Regulation was amended[23] and new indicators were developed and made available for Europe’s Digital Competitiveness report. A Regulation on tourism statistics was adopted[24] and Tourism Satellite Accounts were disseminated in 2011. The Regulation on energy statistics was also adopted[25] and data collected. A methodology and indicators on energy efficiency/energy savings were developed. A legal obligation to collect statistics on electricity and gas prices paid by households was not introduced as the voluntary data collection met needs adequately.

Economic and social cohesion: Regional data were converted following two revisions of the NUTS territorial classifications. The annual collection of data at Local Administrative Unit 2 (commune level) was implemented. The Urban Audit data collection was renewed in 2012. The implementation of INSPIRE was ensured by Eurostat for the statistical areas and by other Directorates General (DGs) for other implementation aspects.

Research and technological development: Several surveys (CIS, R&D, career path of doctorate holders) were largely harmonised and implemented. Research in official statistics, included in the 7th framework programme, was given a strong boost with the launch of ESSnets. Various projects were launched on harmonising statistical disclosure control in the ESS and setting up the integrated infrastructure to access confidential data. Anonymised microdata for scientific purposes were provided for a number of surveys.

Environment: The data centres on waste and on natural resources and products are operational. The Waste Statistics Regulation[26] was simplified and aligned with other reporting obligations. The 2009 Regulation concerning statistics on pesticides[27] lays down that sales data must be transmitted by the MSs from 2012 onwards. A project on streamlining environmental indicators produced an inventory and around 120 indicator fact sheets with proposals on streamlining. A consistent set of European environmental indicators was published but there is a lack of metadata. The methodological development of environmental-economic accounts continued and a legal basis was adopted in 2011. Methodological guidelines and manuals were produced for particular areas of the accounts.

Development cooperation: Cooperation focused on assisting donors and regional statistical organisations to increase the visibility and importance of statistics for policy design and monitoring, and on providing technical assistance in this respect. Several tools for statistical cooperation were developed to support the production of statistical data (national accounts and external trade) or the EU delegations in the design of statistics-related projects in developing countries.

Economic, financial and technical cooperation with third countries: Technical assistance focused on regional statistical programmes in which technical, training and capacity-building tools were provided to developing countries. Data resulting from cooperation projects were published.

3.           Outcome of reprioritisation

A methodology for the reprioritisation process was developed by a task force specifically set up for the purpose. The methodology was based on an administrative evidence-based priority setting mechanism, which relies on the measurement of production costs and the burden of statistics and on the use of Summary Information Sheets (SISs) for both new and existing statistical requirements. However, this methodology was difficult to implement consistently at national level as the measurement of burden and costs were not calculated by all MSs in a systematic or homogenous way. As a result, the majority of the SISs did not contain the final assessment on cost-effectiveness and, in turn, this did not prove to be an effective tool for setting priorities. Therefore, Eurostat proposed a new strategy-driven approach to the ESS, based on an annual review of existing statistical requirements, on identification of negative priorities and on sunset clauses.

Priority setting was performed during the preparation of the annual work programmes and involved a rolling process of consultation between Eurostat and the other Commission DGs as well as among the various ESS members, including ESAC. The annual priority-setting mechanism actually ensured that annual programmes could cope with emerging statistical priorities, even if not laid down in the CSP. Since 2010, the priority-setting mechanism also involved the identification of negative priorities, i.e. activities to be stopped. A list of new activities and of negative priorities is included in the executive summary of the external consultant’s final report.

Furthermore, it was not possible to base the priority-setting mechanism on objective criteria, since there was no systematic information available either on costs associated with new statistical requirements or on the potential savings associated with the negative priorities. It should be noted however that Commission Decision 2012/504/EU of 17 September 2012 on Eurostat, which makes the distinction between European statistics and other statistics (Article 2) allows Eurostat to focus on European statistics and services to other DGs, therefore improving the consistency and transparency of the priority setting mechanism.

Even if there is no evidence that the reprioritisation process was partly guided by systematic considerations on efficiency gains, Eurostat made a big effort to increase the efficiency of the production system. A rather approximate and aggregated way to assess the results is to look at the ratio between the number of published tables and the number of Eurostat staff. While the former increased sharply, the number of staff producing statistics remained almost constant over the period and even decreased in 2010 and 2011.

4.           Implementation of Eurostat’s Vision

Eurostat’s Vision involves most — if not all — phases of the statistical business process and could not have been completed in only three years. Major developments are provided for in the European Statistical Programme 2013-2017 (ESP)[28] but the basis for its implementation was established for organisational, legislative and concrete aspects. We can cite the establishment of Sponsorship Groups within the ESS to promote the sharing of knowledge and good practice among ESS members at a strategic level, the launch of the modernisation of social statistics and the introduction of the European Master’s in Official Statistics. Eurostat has begun an internal and external reflection process for redrafting legislation in the area of business statistics in order to put in place a framework regulation (FRIBS) but the implementation of legislative acts is a long process. The concrete initiatives mainly consisted of VIP and ESSnet projects. The former addresses cross-cutting domains of development. The latter is an initiative formalising a long-standing tradition of cooperation among ESS members with the final aim of developing tools and common solutions to be widely adopted by all the National Statistical Institutes (NSIs).

The reengineering of statistical operations is a long and complex process, and the results of VIP and ESSnet projects have therefore, in most cases, not yet been included in the production chain of NSIs. NSIs will need resources to invest in the modification of processes, even though they might bring cost-efficiencies in the long-run.

Eurostat has intensified its communication efforts and provided technical expertise and support to MSs on the implementation of its Vision. However, the degree to which NSIs have adopted the ESS Strategy and the Vision is still mixed, with NSI attitudes ranging from enthusiastic to sceptical — if not reluctant. The risk of lacking momentum in a scenario of resources shortage in the ESS can be mitigated by positioning the Vision at the core of the ESP, but this strategy needs to be supported by adequate means, in particular, adequate management should be put in place. The launch of the ESS.VIP programme, which builds on the lessons learnt from the ESSnet and VIP programmes, provides a new framework for implementing the Vision.

5.           Considerations on the recommendations made by the external consultant

The external consultant who assessed the programme submitted his final evaluation report on the CSP only after the new ESP 2013-2017 had been adopted. In this report, he made recommendations based on his analysis of the CSP. Given the timing of the final assessment, only those recommendations, on a better designing of the ESP and its explicit links with the AWP, which had already been presented in the interim evaluation report[29] on the CSP and recommendations by the European Court of Auditors[30] could be taken into account when designing the ESP and so not those on the reprioritisation and on the Vision. This chapter provides a brief overview of actions being taken by Eurostat in response to the recommendations of the external consultant[31].

One set of recommendations concerns the effectiveness of the CSP and has or is being already implemented for the ESP. The new programme has been designed with measurable objectives and with the possibility of a mid-term review. An explicit link is being made between the objectives and actions of the AWP and the objectives of the ESP, and indicators to measure the achievement of objectives are expected.

As regards reprioritisation, Eurostat intends to continue to review the strategic priorities including negative priorities after consultation of the users, while making a constant effort to reduce costs, in line with the general Commission policy. Eurostat will also consider developing in collaboration with MSs an analysis of costs and merits. Information should include costs (for Eurostat and MSs, at least qualitatively), respondent burden and alternative related available information. In addition it is explicitly foreseen in Article 6 of the ESP that the Commission shall, in the preparation of the annual work programmes, ensure effective priority-setting and an annual review of, and report on, statistical priorities. It also stipulates that prioritisation shall contribute to the reduction of costs and burdens for new statistical requirements by reducing statistical requirements in existing domains of European statistics and shall be pursued in close cooperation with the MSs. Implementing the Vision is also an important instrument to improve efficiency in the production systems and reduce costs.

Eurostat provides financial and technical support to MSs for the implementation of the Vision. The recently discussed ESS.VIP programme already spans five years, and in September 2013 the 18th ESSC established a High Level Task Force to reflect on the original Vision and to update it taking into account the experience with its implementation so far, the present situation of NSIs and Eurostat as well as the evolving user needs and IT developments.

[1]               OJ L 344, 28.12.2007

[2]               Commission Communication (2009) 404 on the production method of EU statistics : a vision for the next decade

[3]               Regulation (EU) No 549/2013 of the European Parliament and of the Council - OJ L 174, 26.6.2013, p. 1

[4]               Regulation (EC) No 223/2009 of the European Parliament and of the Council - OJ L 87. 31.3.2009, p. 164

[5]               Regulation (EC) No 222/2009 of the European Parliament and of the Council - OJ L 87, 31.3.2009, p. 160

[6]               Decision No 1297/2008/EC of the European Parliament and of the Council - OJ L 340, 19.12.2008, p. 76

[7]               Regulation (EC) No 1166/2008 of the European Parliament and of the Council - OJ L 321, 1.12.2008, p. 14

[8]               Regulation (EC) No 543/2009 of the European Parliament and of the Council - OJ L 167, 29.6.2009, p. 1

[9]               Regulation (EC) No 1165/2008 of the European Parliament and of the Council - OJ L 321, 1.12.2008, p. 1

[10]             Regulation (EU) No 1337/2011 of the European Parliament and of the Council - OJ L 347, 30.12.2011, p. 7

[11]             Commission Regulation (EC) No 1055/2008 - OJ L 283, 28.10.2008, p. 3 and Commission Regulation (EU) No 555/2012 - OJ L 166, 27.6.2012, p. 22

[12]             Commission Regulation (EC) No 834/2009 - OJ L 241, 12.09.2009, p 3

[13]             Commission Regulation (EU) No 351/2010 of 23 April 2010 - OJ L 104, 24.4.2010, p. 37

[14]             Directive 2009/42/EC of the European Parliament and of the Council of 6 May 2009 - OJ L141 of 6.6.2009, p. 29

[15]             Regulation (EC) No 453/2008 of the European Parliament and of the Council - OJ L 145, 4.6.2008, p. 234

[16]             Regulation (EC) No 471/2009 of the European Parliament and of the Council - OJ L 152, 16.6.2009, p. 23

[17]             Commission Regulation (EU) No 263/2011- OJ L 71, 18.3.2011, p. 4

[18]             Commission Regulation (EC) No 1201/2009 - OJ L 329, 15.12.2009, p 29

[19]             Commission Regulation (EU) No 823/2010 of 17 September 2010 - OJ L 246, 18.9.2010, p. 33

[20]             COM(2011) 903

[21]             Regulation (EC) No 1338/2008 of the European Parliament and of the Council - OJ L 354, 31.12.2008, p. 70

[22]             Commission Regulation (EU) No 328/2011- OJ L 90, 6.4.2011, p. 22 and Commission Regulation (EU) No 349/2011 - OJ L 97, 12.4.2011, p. 3

[23]             Regulation (EC) No 1006/2009 - OJ L 286, 31.10.2009, p 31

[24]             Regulation (EU) No 692/2011 of the European Parliament and of the Council - OJ L 192, 22.7.2011, p. 17

[25]             Regulation (EC) No 1099/2008 of the European Parliament and of the Council - OJ L 304, 14.11.2008, p. 1

[26]             Commission Regulation (EU) No 849/2010 of 27 September 2010 - OJ L 253, 28.9.2010, p. 2

[27]             Regulation (EC) No 1185/2009 of the European Parliament and of the Council - OJ L 324, 10.10.2009, p 1

[28]             Regulation (EU) No 99/2013 of the European Parliament and of the Council - OJ L 39, 9.2.2013, p. 12

[29]             http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2010:0346:FIN:EN:PDF

[30]             Special Report No 12 // 2012

[31]             http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/Report_expert.pdf