EUR-Lex Access to European Union law
This document is an excerpt from the EUR-Lex website
Document 52012SA0012
Special Report No 12/2012 ‘Did the Commission and Eurostat improve the process for producing reliable and credible European statistics?’
Special Report No 12/2012 ‘Did the Commission and Eurostat improve the process for producing reliable and credible European statistics?’
Special Report No 12/2012 ‘Did the Commission and Eurostat improve the process for producing reliable and credible European statistics?’
Special Report No 12/2012 ‘Did the Commission and Eurostat improve the process for producing reliable and credible European statistics?’
LIST OF ABBREVIATIONS AAR : annual activity report code of practice : European Statistics Code of Practice EDP : excessive deficit procedure EFTA : European Free Trade Association ESAC : European Statistical Advisory Committee ESGAB : European Statistical Governance Advisory Board ESS : European statistical system ESSC : European Statistical System Committee (ESS Committee) Eurostat : Statistical Office of the European Union GNI : gross national income IAC : internal audit capability MEETS : modernisation of European enterprise and trade statistics NSI : national statistical institute OLAF : European Anti-Fraud Office QAF : quality assurance framework SAI : supreme audit institution SWP : annual statistical work programme EXECUTIVE SUMMARY I. Public confidence in European statistics is essential in a Union where political decisions need to be evidence-based and where an increasing number of decisions are directly triggered by statistical data or by indicators derived from them. II. The Court assessed whether the Commission and Eurostat have improved the process for producing reliable and credible European statistics. This depends on adherence to the European Statistics Code of Practice which provides the standards for developing, producing and disseminating statistics, and on the implementation of the European statistical programme. III. The Court’s audit found that the European Statistics Code of Practice has only been partly implemented and that full implementation remains a challenge for all those involved both at the European level and within Member States. The code sets demanding standards but lacks strong verification and enforcement tools. IV. Sufficiently reliable information on the current state of implementation of the code throughout the European statistical system (ESS) is not available. The Commission’s recent initiatives to give new momentum to achieving full compliance with the code go in the right direction but are not sufficient to address all concerns. Ambiguity about the nature of the obligation to adhere to the code persists. For cases where misrepresentations of data may occur, no appropriate inspection mechanism has yet been proposed. No proposal has been made to develop an independent supervisory function. V. Flaws in the design of the statistical programme 2008 to 2012 hamper its use as an effective planning, monitoring and accountability tool. Re-prioritisation of statistical activities towards new challenges has been slower than expected. The draft programme 2013 to 2017 provides an opportunity to reengineer the ESS in order to make it more efficient and flexible provided that it is supplemented by precise targets and milestones laid down in the annual planning and verified in a process of systematic annual reporting. VI. The Court recommends moving towards a system of European statistics which guarantees professional independence, sufficient resources and strong supervision including sanctions for cases where quality standards are not respected. INTRODUCTION PUBLIC TRUST IN EUROPEAN STATISTICS 1. Reliable and credible statistics are vital for the performance of the tasks entrusted to the European Union. Statistics are needed, in almost all areas, for the formulation, application, monitoring and assessment of the policies laid down in the treaties, and for the collection and allocation of EU funds. More and more European policies are directly based on statistics and statistical indicators are increasingly used as a trigger for policy decisions or sanctions. 2. Public trust in European statistics cannot be taken for granted. Already before the recent turmoil in the global economy a report published by the Commission in April 2008 [1] showed that the proportion of citizens who stated that they did not trust economic statistics (45 %) was almost identical to those claiming that they had trust in such statistics (46 %). 3. In 2005 and again in 2010, the European Council endorsed calls for improving quality of European statistics with a view to strengthening the economic governance of the EU [2]. The Council concluded in 2005 inter alia that the focus should be on developing the operational capacity, monitoring power, independence and accountability of Eurostat. THE REGULATORY FRAMEWORK FOR PRODUCING EUROPEAN STATISTICS 4. Article 338 TFEU stipulates that the production of statistics has to conform to impartiality, reliability, objectivity, scientific independence, cost-effectiveness and statistical confidentiality. Furthermore, it must not entail excessive burdens on economic operators. 5. The regulation on European statistics [3] provides definitions of the principles laid down in the treaty and addresses issues of statistical governance as well as the production and dissemination of European statistics. 6. The former Statistical Programme Committee [4] adopted the European Statistics Code of Practice [5] (hereafter "the code of practice" or "the code") on 24 February 2005. The code further elaborates how European statistics are to be developed, produced and disseminated in conformity with the statistical principles set out in the regulation on European statistics. 7. The 5-year European statistical programme sets priorities and defines the main fields and the objectives for the development, production and dissemination of European statistics. The current 2008 to 2012 programme [6] has planned expenditure of 274 million euro. THE EUROPEAN STATISTICAL SYSTEM 8. European statistics are developed, produced and disseminated in the framework of the European statistical system (ESS). The regulation on European statistics defines the ESS as a partnership between the Union statistical authority and the national statistical institutes (NSIs). 9. The NSIs have responsibility for coordinating all activities at national level for the development, production and dissemination of European statistics. NSIs and other national authorities with responsibility for European statistics may receive grants from the EU budget without prior call for proposals. 10. The European Statistical System Committee (hereafter "the ESS Committee") provides professional guidance to the ESS. Composed of the representatives of the NSIs, it is amongst other things in charge of examining statistical legislation proposed by Eurostat. EUROSTAT: THE STATISTICAL OFFICE OF THE EUROPEAN UNION 11. The Statistical Office of the European Union (hereafter "Eurostat") operates as the Union statistical authority. Eurostat is based in Luxembourg. It is a directorate-general of the Commission that enjoys "technical autonomy" [7] as regards the choice of scientific techniques, definitions and methodologies. In legal documents, the Union statistical authority is referred to as "the Commission (Eurostat)" [8]. This terminology is meant to clarify that Eurostat has a distinct role to play despite the fact that it is part of the Commission. 12. Eurostat’s main role is to process and publish comparable statistical information at European level. Normally Eurostat does not collect data itself. This is done in Member States by their statistical authorities. They verify and analyse national data and send them to Eurostat which consolidates these figures with a view to ensuring that they are comparable. 13. Eurostat’s staff numbers have been stable over the last years. As at 1 January 2012, the total number of staff members was 794. Eurostat estimates that, at country level, throughout the ESS as a whole at least 50000 staff are involved in the production of European statistics. 14. Commitment appropriations available to Eurostat [9] amounted to 92,9 million euro in 2011. The annual average amount over the period 2003 to 2010 was 89,5 million euro with a peak reached in 2005 when 107,2 million euro was available. AUDIT SCOPE AND APPROACH 15. The overall objective was to assess whether the Commission and Eurostat have improved the process for producing reliable and credible European statistics. This depends on the implementation of the code of practice and on the management of the European statistical programme. The audit addressed the following two questions: (a) Have the Commission and Eurostat taken all steps necessary to fulfil their role in achieving the implementation of the code of practice throughout the European statistical system? (b) Does Eurostat manage well the multiannual statistical programme as a tool for improving the production of European statistics? 16. The audit also covered the contributions of the European Statistical Governance Advisory Board (ESGAB) [10] and of the European Statistical Advisory Committee (ESAC) [11] to the process of producing reliable and credible European statistics. 17. The audit focused on the implementation of the code of practice and the management of the European statistical programme in relation to the process of producing European statistics. It did not assess the reliability of specific statistical outputs. Most audit work was carried out between May and December 2011, with some update work performed until June 2012. 18. Audit work included interviews with Eurostat staff, an examination of documentation available at Eurostat and of its replies to a detailed questionnaire, information visits to Eurostat stakeholders within and outside the Commission, observation of meetings of the ESS Committee, and an examination of samples of ex ante and ex post controls carried out by Eurostat’s unit in charge of financial management in connection with grant payments as well as of a sample of procurement procedures. OBSERVATIONS IMPLEMENTING THE EUROPEAN STATISTICS CODE OF PRACTICE 19. The audit found that the code of practice has only been partly implemented and that full implementation remains a challenge for all those involved both at the European level and within Member States. The Court examined (a) the steps taken since 2005 and the reasons for delays and setbacks; (b) whether the current tools for assessing the implementation of the code are adequate; (c) whether recent initiatives of the Commission are sufficient and satisfactory. THE ACTIONS TAKEN BY THE COMMISSION AND EUROSTAT SINCE 2005 WERE NOT SUFFICIENT TO ACHIEVE FULL IMPLEMENTATION OF THE CODE THE CODE OF PRACTICE SETS DEMANDING STANDARDS 20. The code of practice [12] sets the standards for developing, producing and disseminating European statistics. The code was developed in response to a request from the Council in June 2004 [13] following problems with the Greek government deficit and debt figures [14]. 21. The code of practice is meant to ensure that European statistics meet user needs through an efficient production process taking place in a credible institutional environment which guarantees professional independence. It consists of 15 principles covering not only the institutional environment, but also statistical processes and statistical outputs (see Box 1). Each principle has between three and nine good-practice indicators to identify to what extent the principle is being complied with. BOX 1 SUMMARY OF THE PRINCIPLES OF THE CODE OF PRACTICE Source: European Court of Auditors. Code of practice — 15 principles | Institutional environment | 1 Professional independence Professional independence ensures credibility | 2 Mandate for data collection Clear mandate laid down by law allowing to collect information | 3 Adequacy of resources Sufficient resources for meeting statistical requirements | 4 Quality commitment Continuously improve process and product quality | 5 Statistical confidentiality Guaranteeing the privacy of data providers | 6 Impartiality and objectivity Respect of scientific independence and equal treatment of users | Statistical processes | 7 Sound methodology Adequate tools and expertise | 8 Appropriate statistical procedures Appropriate procedures applied from data collection to data validation | 9 Non-excessive burden on respondents Monitoring and reducing the response burden | 10 Cost-effectiveness Monitoring the use of resources and introducing improvements | Statistical output | 11 Relevance Meet the users’ needs and monitoring user satisfaction | 12 Accuracy and reliability Accurately and reliably portray reality | 13 Timeliness and punctuality Standards are met concerning timely publication of statistics | 14 Coherence and comparability Statistics are consistent internally, over time and comparable between regions and countries | 15 Accessibility and clarity Clear presentation, suitable dissemination and impartial access | 22. The European Statistical Governance Advisory Board (ESGAB) is a body of high-level experts tasked to provide an independent overview of Eurostat and of the ESS as a whole as regards the implementation of the code of practice (see Box 2). To date, ESGAB focused on three [15] of the 15 principles of the code, and reported [16] that whilst progress has been made in implementing the code, the pace of progress has not met its expectations. BOX 2 EUROPEAN STATISTICAL GOVERNANCE ADVISORY BOARD (ESGAB) Assessing the implementation of the code of practice Established in 2009, the main task of ESGAB is to report to the European Parliament and the Council on the implementation of the code of practice "insofar as it relates to the Commission (Eurostat)" including an "assessment of the implementation of the code in the European statistical system as a whole" (see Article 2 of the decision establishing ESGAB). ESGAB’s role is advisory and it has no supervisory powers. The seven members of the board are selected from among experts possessing outstanding competence in the field of statistics. They perform their duties in their personal capacity and shall act independently. ESGAB members receive no remuneration, most of them still being professionally active. Eurostat makes available one full-time secretary who must act on the instructions of the board and independently of the Commission. Eurostat has the status of an observer and is usually represented in meetings by its director-general and its deputy director-general. ESGAB met for the first time in March 2009, held six to seven 1-day meetings per year and published its most recent annual report in December 2011. Source: European Court of Auditors. 23. In September 2011 the ESS Committee [17] approved limited changes to the code of practice. Figure 1 summarises the history of the code of practice. FIGURE 1 HISTORY OF THE CODE OF PRACTICE +++++ TIFF +++++ Source: European Court of Auditors. DELAYED ACTION BY THE COMMISSION TO PERSUADE GOVERNMENTS TO ADHERE TO THE CODE 24. In May 2005, the Commission recommended [18] Member States to ensure that the principles of the code are respected by their statistical authorities and to ensure that their statistical services are professionally organised and resourced to produce European statistics in a manner that guarantees independence, integrity and accountability. Furthermore, Member States were invited to provide the information necessary to enable the Commission to monitor adherence to the code [19]. 25. Not until 2011 did the Commission announce [20] that it would propose that Member States should formally commit themselves, based on the code of practice, to taking all necessary measures to maintain confidence in their statistics and to monitoring the implementation of the code through so-called "commitments on confidence in statistics" (see paragraphs 60 to 64). THE COMMISSION’S 2008 REPORT DID NOT IDENTIFY THAT PROFESSIONAL INDEPENDENCE WAS NOT SECURED IN THE ESS AS A WHOLE 26. Once the code of practice was promulgated, the Commission and Eurostat focused on promoting its implementation through cooperation with and support to national statistical offices (NSIs). To that end Eurostat organised peer reviews which were carried out in the 31 NSIs of the EU Member States and EFTA [21] countries and in Eurostat over the period 2006 to early 2008. 27. These peer reviews were limited to NSIs and their coordinating role within the national statistical systems and focused on the code’s indicators related to the institutional environment. Table 1 shows the results of these assessments. The peer reviews did not assess whether NSIs complied with the indicators of the code related to statistical processes. Most of the indicators related to statistical outputs were also not covered. Nevertheless the list of improvement actions in the peer review reports addressed all principles of the code. Those actions relating to statistical processes and outputs were solely based on the results of the self-assessments carried out by the NSIs. TABLE 1 RESULTS OF PEER REVIEW EXERCISE 2006 – EARLY 2008 [221] Source: European Court of Auditors on the basis of peer review reports. NSI | Indicators "largely met" or "fully met" | Indicators "partly met" | Indicators "not met" | NSIs with all indicators met Austria | 35 | 0 | 0 | Finland | 35 | 0 | 0 | Norway | 35 | 0 | 0 | Sweden | 35 | 0 | 0 | United Kingdom | 35 | 0 | 0 | NSIs with some indicators partly met Czech Republic | 33 | 2 | 0 | Denmark | 33 | 2 | 0 | France | 33 | 2 | 0 | Italy | 33 | 2 | 0 | Lithuania | 33 | 2 | 0 | Portugal | 33 | 2 | 0 | Slovakia | 33 | 2 | 0 | Spain | 33 | 2 | 0 | Netherlands | 32 | 3 | 0 | Poland | 32 | 3 | 0 | Slovenia | 32 | 3 | 0 | Switzerland | 32 | 3 | 0 | Germany | 31 | 4 | 0 | Ireland | 31 | 4 | 0 | Hungary | 29 | 6 | 0 | Latvia | 29 | 6 | 0 | Romania | 29 | 6 | 0 | Estonia | 26 | 9 | 0 | Greece | 26 | 9 | 0 | Luxembourg | 24 | 11 | 0 | NSIs with some indicators not met Bulgaria | 27 | 7 | 1 | Liechtenstein | 27 | 7 | 1 | Malta | 23 | 11 | 1 | Iceland | 28 | 5 | 2 | Cyprus | 25 | 8 | 2 | Belgium | 24 | 9 | 2 | 28. Based on the results of the peer reviews amongst ESS members, the Commission reported in October 2008 on the implementation of the code [22], and concluded that the "self-regulatory approach works very well". The report highlighted the need for further action but, with regard to professional independence and objectivity, it stated the following: "Independence from political and other external interference with production and dissemination of European statistics and an objective choice of methods, sources and techniques seem to be ensured in practice across the ESS." However, subsequent events reported by the Commission in January 2010 [23] showed that this was not the case. 29. In 2011, ESGAB noted in its annual report with regard to the principle of professional independence "that statistical laws have been modernised and transparency has increased since 2009 in many countries, but that professional independence is not secured in the ESS as a whole." Details from the report underpinning this assessment are in Box 3. BOX 3 ESGAB FINDINGS RELATED TO PROFESSIONAL INDEPENDENCE "Half of the members of the European statistical system consider their institutional setting to be free of constraints on the principle of professional independence." "In four countries — Germany, Greece, Latvia, Switzerland — difficulties were observed in modernising the statistical law or in implementing it." "ESGAB notes that three countries — Denmark, Poland and Romania — have no concrete plans for modernising their statistical law, even though the current legislation cannot be considered to be fully in compliance with the code." "In eight countries the decision-making process related to resource allocation is seen as a risk to professional independence." "ESGAB notes that there have been several recent episodes where the head of the NSI has been changed after an election." "In 11 countries the rules for appointing and dismissing top management were perceived to be adequate. However, in eight countries ambiguity was observed or the practical implementation is deemed to be unsatisfactory." Source: ESGAB annual report 2011, p. 7 and 8. EUROSTAT CURRENTLY MONITORS NSIS’ REPORTING ON IMPROVEMENT ACTIONS WITHOUT VERIFYING THEIR IMPLEMENTATION 30. After the 2006 to 2008 round of peer reviews, Eurostat monitored the implementation of the improvement actions agreed by NSIs as a follow-up to the peer reviews [24]. Eurostat bases its monitoring on replies of NSIs to annual questionnaires. 191 actions had already been completed when monitoring began in 2008, with 677 actions outstanding. By March 2011 [25], 273 actions [26] were not yet completed. Figure 2 shows the progress towards completion of the improvement actions by NSIs. The detailed state of completion is shown in the Annex. 31. For some of the actions reported as "completed" by NSIs, the actual final outcome is not known to Eurostat, as no further details or explanations were provided. Moreover, NSIs report 116 actions as "ongoing with no specific deadline", which makes it difficult to assess progress. 32. For many NSIs, 5 years or more have passed since the peer review took place. Conclusions on the current status of implementation of the code of practice, based only on reporting the progress of the improvement actions agreed during the peer reviews, are unreliable. FIGURE 2 COMPLETION OF IMPROVEMENT ACTIONS AS REPORTED SINCE 2008 +++++ TIFF +++++ Source: Eurostat; the total of remaining actions does not include two new actions defined in 2011. THE COMMISSION ITSELF DOES NOT YET FULLY COMPLY WITH THE CODE OF PRACTICE 33. In its recommendation of 25 May 2005, the Commission undertook to ensure that Eurostat would respect the principles of the code and that Eurostat would be professionally organised and resourced to produce European statistics in a manner that guarantees independence, integrity and accountability. THE COMMISSION DECISION OF 1997 ON EUROSTAT’S ROLE IS NOT IN LINE WITH THE CODE NOR WITH THE REGULATION ON EUROPEAN STATISTICS 34. In April 2011 the Commission announced its intention to amend its decision of 1997 on Eurostat’s role [27]. The decision of 1997 is not fully compliant with the principles introduced by the code of practice in 2005. In particular, no reference at all is made to the principle of professional independence (principle 1 of the code). The Commission decision grants Eurostat only "technical autonomy". It is silent about the role of the director-general of Eurostat and does not define the meaning of the title "Chief Statistician of the European Union" attributed to him in recent years. 35. Moreover, the decision of 1997 has not been brought in line with the 2009 regulation on European statistics. Article 6 of the 2009 regulation entrusts Eurostat with sole responsibility for developing, producing and disseminating European statistics. In contrast, the Commission decision stipulates in its Article 6 that the Commission may decide that services other than Eurostat are to participate in the production process for statistics, and in which activities and to what extent. The consequence is that although Eurostat is entrusted with the sole responsibility for producing statistics, other directorates-general are also currently producing them. EUROSTAT’S INDEPENDENCE IS LESS CLEARLY DEFINED THAN OLAF’S 36. The audit compared the status given by the Commission to Eurostat with the status given to the European Anti-Fraud Office (OLAF). Like Eurostat, OLAF is a directorate-general of the Commission but under an expectation to act in full independence when performing its tasks. 37. In contrast to the Commission decision on Eurostat of 1997, the Commission decision on OLAF of 1999 [28] contains detailed provisions to protect the operational independence of the Office and of its director-general, and in particular: (a) an explicit obligation for the director-general not to seek or take instructions from the Commission, any government or any other institution or body; (b) a supervisory committee to protect the Office against undue interference; (c) a renewable fixed-term mandate for the director-general; (d) a selection process requiring a favourable opinion from the supervisory committee as to the qualification of the short-listed candidates for the post of director-general; (e) an endorsement by the European Parliament and the Council, before the Commission appoints the director-general. PRACTICAL STEPS TO ENSURE EUROSTAT’S PROFESSIONAL INDEPENDENCE WERE TAKEN BUT THE UPCOMING NEW COMMISSION DECISION ON EUROSTAT’S ROLE SHOULD CONSOLIDATE PROGRESS 38. In practice, the Commission has taken steps to better ensure the professional independence of Eurostat. In particular, the current director-general was recruited in 2008 through a procedure open to external candidates. Likewise, the Commission has opened recent recruitment procedures for Eurostat’s directors to external candidates. 39. However, such progress remains fragile as long as it is not clearly laid down in the relevant legal provisions. Unless provided otherwise in its upcoming decision on Eurostat’s role, the Commission retains the option to fill top management positions (director-general and directors) at Eurostat by transfer of senior officials from other posts within the Commission without prior publication of the vacancy. 40. In addition, the fact that a significant proportion of Eurostat’s operational credits comes through sub-delegations from other directorates-general of the Commission [29] runs counter to principles 1 (professional independence) and 3 (adequacy of resources) of the code of practice, as it makes Eurostat, in part, financially dependent on other Commission services. 41. In June 2012, a decision was still pending whether Eurostat and/or other directorates-general are tasked with carrying out investigations to establish the existence of misrepresentations of deficit and debt data with a view to imposing fines [30]. EUROSTAT HAS NOT YET OVERCOME PROBLEMS WITH ITS OWN IMPLEMENTATION OF THE CODE 42. From Eurostat’s internal reporting to ESGAB, it appears that, as of March 2011, 27 of the 69 improvement actions agreed in the peer review report of 2007 [31] were still pending. Details are shown in Table 2 which provides also an overview on the implementation of the recommendations made by ESGAB since 2009. TABLE 2 EUROSTAT’S IMPLEMENTATION OF IMPROVEMENT ACTIONS (AS OF MARCH 2011) Source: Information provided by Eurostat to ESGAB. State of implementation according to Eurostat’s reporting | Number of actions agreed under the peer review in 2007 | Number of ESGAB’s recommendations issued in its 2009 and 2010 reports | Implementation until March 2009 (1)Actions completed | 35 | | Implementation since April 2009 Actions terminated This action has been completed | 3 | 5 | No further work on this issue is planned | 3 | 0 | No longer relevant | 1 | 0 | (2)Total actions terminated | 7 | 5 | Actions pending This work is considered to be ongoing with no specific deadline | 10 | 15 | The work is progressing as planned and is not yet due | 10 | 4 | There have been delays within Eurostat | 3 | 0 | Further progress now depends on authorities outside of Eurostat | 2 | 0 | The work has been included in a new action | 1 | 0 | Absence of clear statement on the implementation status | 1 | 0 | (3)Total actions pending | 27 | 19 | GRAND TOTAL (1+2+3) | 69 | 24 | 43. The audit found that Eurostat has difficulties in fully implementing principle 6 of the code (impartiality and objectivity) as regards impartial access to data for users. Exceptions to the application of the code and the principle of impartiality exist, notably in connection with the excessive deficit procedure (EDP) [32] where the timing of sharing information within the Commission is not in line with Eurostat’s "Protocol on impartial access to Eurostat data for users" [33]. 44. With regard to other Commission directorates-general, Eurostat has not yet overcome the problem already described in the peer review report of 2007 [34] that these services collect data on their own account when they feel that Eurostat might be unresponsive to their requests. "Lack of coordination between policy DGs and Eurostat on statistical work" was notified by Eurostat and accepted by the Commission central services as a cross-cutting critical risk in 2010 and a number of measures have been launched in 2011 to mitigate it. ASSESSING THE STATUS OF IMPLEMENTATION OF THE CODE OF PRACTICE ACROSS THE ESS REQUIRES BETTER INFORMATION THE REGULATION ON EUROPEAN STATISTICS DOES NOT PROVIDE FOR PROCEDURES TO VERIFY ADHERENCE TO THE CODE OF PRACTICE 45. The partners in the ESS are expected to cooperate in good faith. The regulation on European statistics does not provide for procedures to verify adherence to the principles of the code of practice. The excessive deficit procedure (EDP) is the only statistical domain where the legislator empowered and obliged Eurostat to perform verification work on-the-spot in Member States [35]. A COMPREHENSIVE APPROACH IS NEEDED FOR PEER REVIEWS 46. Peer reviews remain the most important tool for independently assessing the status of implementation of the code by Eurostat and NSIs. Another round of peer reviews is envisaged for 2013 but in June 2012 decisions were still pending on where and how to launch them. These reviews will need to address those principles so far covered only to a limited extent (see Table 3). 47. The 2006–08 peer reviews followed a common methodology focusing on the "Institutional environment" part of the code. Other principles were only covered through self-assessments. In 2011, the task force "Sponsorship on quality" finalised a quality assurance framework (QAF) related to statistical processes and statistical outputs. It identifies possible activities/methods/tools that can provide guidance and evidence for the implementation of the indicators of the code. The ESS Committee did not yet decide on how to use the QAF. TABLE 3 COVERAGE OF THE 15 PRINCIPLES OF THE CODE Source: European Court of Auditors. (√ = Covered) Code of practice principles adopted in 2005 | Self Assessments 2005-06 | Peer Reviews 2006–08 | Action plans 2007-08 | Monitoring of Action plans since 2008 | Annual ESGAB assessments since 2009 | Institutional environment | 1.Professional independence | √ | √ | √ | √ | √ | 2.Mandate for data collection | √ | √ | √ | √ | X | 3.Adequacy of resources | √ | √ | √ | √ | √ | 4.Quality commitment | √ | √ | √ | √ | √ | 5.Statistical confidentiality | √ | √ | √ | √ | X | 6.Impartiality and objectivity | √ | √ | √ | √ | X | Statistical processes | 7.Sound methodology | √ | X | √ | √ | X | 8.Appropriate statistical procedures | √ | X | √ | √ | X | 9.Non-excessive burden on respondents | √ | X | √ | √ | X | 10.Cost-effectiveness | √ | X | √ | √ | X | Statistical output | 11.Relevance | √ | X | √ | √ | X | 12.Accuracy and reliability | √ | X | √ | √ | X | 13.Timeliness and punctuality | √ | X | √ | √ | X | 14.Coherence and comparability | √ | X | √ | √ | X | 15.Accessibility and clarity | √ | √ | √ | √ | X | 48. Other statistical producers (e.g. regional statistical offices) or providers of administrative data used for statistical reporting (e.g. ministries of finance) were not covered by the first round of peer reviews (see Figure 3) and remain to be covered by future reviews. 49. Conducting the reviews on a horizontal organisation-wide basis alone does not make it possible to ascertain that systems operate in the same manner in all statistical domains of an NSI. As there are over 100 statistical domains, it will not be possible to cover them all at once. A solution could be rolling peer reviews which each year look into a limited number of specific statistical domains. Although not directly comparable in scope and approach, Eurostat has already some experience with organising rolling reviews in the framework of its evaluation activities. FIGURE 3 COVERAGE OF THE 2006–08 PEER REVIEWS +++++ TIFF +++++ Source: European Court of Auditors. 50. In the first round the peer review teams had three members, normally two from NSIs and one from Eurostat. ESGAB recommended [36] that peer reviews should be carried out "by an autonomous peer-review team" and "that the new set of peer reviews must be more streamlined and standardised than the previous ones". Including peer reviewers not belonging to the ESS would strengthen the independence and credibility of the reviews. ESGAB OBTAINS INFORMATION FROM MEMBER STATES ON A VOLUNTARY BASIS 51. The decision establishing ESGAB does not set out how it is to obtain the factual basis for its annual assessments of the state of implementation of the code. Taking into account Eurostat’s annual monitoring reports, ESGAB has developed its own tools to collect further information. It sent questionnaires to NSIs and stakeholders, invited representatives from the top management of NSIs for brief exchanges of views ("country dialogues") and wrote letters to national governments to request clarifications regarding their adherence to the code. 52. ESGAB stressed in its 2011 report that there is a "difficulty in obtaining factual information about how the code is implemented in practice." Under the present arrangements and given the limited resources (see Box 2) the scope of ESGAB’s monitoring will remain clearly limited. THE COMMISSION’S RECENT INITIATIVES GO IN THE RIGHT DIRECTION BUT DO NOT ADDRESS ALL MATTERS OF CONCERN 53. In April 2011, the Commission issued the communication entitled "Towards robust quality management for European statistics" [37]. The communication was meant to draw the lessons from the financial crisis. The Commission announced its intention to implement "a preventive approach to verifying government finance statistics" and to further strengthen the governance of the ESS through a number of proposals. THE PREVENTIVE APPROACH TO VERIFYING GOVERNMENT FINANCE (EDP) STATISTICS WAS LAUNCHED IN 2011 54. The regulation on the application of the protocol on the excessive deficit procedure requires Eurostat to carry out in all Member States regular "dialogue visits", as well as possible "methodological visits". Following the weaknesses identified in Greek government deficit and debt statistics [38], Eurostat’s powers were extended in July 2010 [39]. 55. For methodological visits, Eurostat has been given audit-like powers to verify the accounts which underlie the data reported by Member States. However, these powers may only be used in exceptional cases where significant risks or problems with respect to the quality of the data notified by a Member State have been clearly identified. So far, these powers have only been applied for assessing the quality of Greek government deficit and debt statistics. 56. In order to overcome the limitations imposed by the exceptional character of methodological visits, Eurostat launched in 2011 a new type of visits, so-called "upstream dialogue visits" in addition to its "standard dialogue visits". Upstream visits are specifically directed to entities supplying the accounts underlying the data reported by Member States. 57. Eurostat faces the double challenge of increasing both the number and the quality of its dialogue visits. Improving quality will necessarily imply longer visits. Table 4 shows that the number of dialogue visits increased significantly in 2011. However, at the time of the audit, it was not possible to assess the quality of upstream visits and the cooperation of relevant Member State bodies as the reports on the visits were not yet available. 58. Following an internal reorganisation, since 1 January 2012 a Eurostat directorate with about 50 staff, internally redeployed, has been exclusively dedicated to government finance statistics. Member States are expected to provide the assistance of experts in national accounting on a voluntary basis [40]. In October 2011, Eurostat had established a list of 30 experts from 12 Member States. TABLE 4 EUROSTAT’S EDP VISITS TO MEMBER STATES Source: European Court of Auditors based on information provided by Eurostat. Number of "dialogue visits" | | 2006 | 2007 | 2008 | 2009 | 2010 | 2011 | Belgium | 1 (1 day) | | 1 (1 day) | | 1 (1 day) | | Bulgaria | | | 1 (2 days) | | 1 (2 days) | 1 (4 days) [7777] | Czech republic | | 1 (2 days) | | 1 (3 days) | | 1 (2 days) | Denmark | 1 (1 day) | | 1 (2 days) | | | 1 (2 days) | Germany | | 1 (1 day) | | 1 (1 day) | | 1 (2 days) [6666] | Estonia | | 1 (2 days) | | 1 (2 days) | | 1 (2 days) | Ireland | 1 (2 days) | | 1 (2 days) | | 1 (2 days) | | Greece | 1 (1 day) [9999] | | | | | 1 (3 days) [7777] and 2 (3 and 2 days) | Spain | | 1 (2 days) | | 1 (2 days) | | 1 (2 days) | France | 1 (1 day) | | 1 (1 day) | | 1 (1 day) | | Italy | | 1 (2 days) | | 1 (2 days) | | 1 (2 days) | Cyprus | 1 (1 day) | | 1 (2 days) | | 1 (1 day) | | Latvia | 1 (2 days) | | | 1 (2 days) | | 1 (3 days) | Lithuania | 1 (2 days) | | 1 (2 days) | | 1 (2 days) | | Luxembourg | 1 (1 day) | | | 1 (1 day) | | 1 (1 day) | Hungary | 1 (2 days) | | 1 (2 days) | | 1 (2 days) | | Malta | 1 (1 day) | | 1 (1 day) | | 1 (2 days) | | Netherlands | | 1 (2 days) | | | 1 (1 day) | 1 (1 day) | Austria | | 1 (2 days) | | 1 (2 days) | | | Poland | | 1 (2 days) | | 1 (2 days) | | 1 (2 days) | Portugal | 1 (2 days) | | 1 (2 days) | | | 3 (2, 2 and 2 days) | Romania | | 1 (2 days) | 1 (2 days) | | 1 (2 days) | 1 (3 days) [7777] and 1 (3 days) | Slovenia | 1 (2 days) | | 1 (2 days) | | | 1 (2 days) | Slovakia | | 1 (2 days) | 1 (2 days) | | | 1 (2 days) | Finland | | 1 (2 days) | | | 1 (2 days) | 1 (2 days) | Sweden | | 1 (1 day) | | 1 (2 days) | | 1 (2 days) | United Kingdom | | 1 (2 days) [8888] | | 1 (2 days) | | 1 (3 days) | Total | 13 (19 days) | 13 (24 days) | 13 (23 days) | 11 (21 days) | 11 (18 days) | 21 (44 days) and 3 (10 days) [7777] | Number of "methodological visits" | Greece | 2 (3 and 3 days) | | 2 (2 and 5 days) | 1 (3 days) | 4 (3, 2, 3 and 22 days) | | Grand Total | 15 (25 days) | 13 (24 days) | 15 (30 days) | 12 (24 days) | 15 (48 days) | 24 (54 days) | 59. Eurostat’s current approach is to enhance cooperation not only with NSIs but also with supreme audit institutions (SAIs) in Member States in order to obtain assurance that upstream public finance data (including financial statements) are properly scrutinised before they are used for EDP reporting. The SAIs’ Contact Committee has set up a task force to explore the possibilities for cooperation between SAIs, Eurostat and NSIs. THE COMMISSION ANNOUNCED A STRENGTHENING OF THE GOVERNANCE OF THE EUROPEAN STATISTICAL SYSTEM 60. The Commission announced in April 2011 [41] two major initiatives to strengthen the governance framework of the ESS: a proposal on "targeted amendments" to the regulation on European statistics and the so-called "commitments on confidence in statistics", a new tool for effective implementation of the code of practice. In April 2012, the Commission submitted a legislative proposal to the European Parliament and the Council [42]. 61. In its communication the Commission stated that the proposal on targeted amendments to the regulation on European statistics "will make clear that the principle of professional independence of national statistical institutes applies unconditionally." Furthermore, the mandate of statistical authorities for data collection will be enhanced when data are extractable from available administrative records. 62. The announced revision of the regulation on European statistics will also set up a legal framework for the new "commitments on confidence in statistics" whereby the Member States should "formally commit themselves, based on the code of practice, to taking all necessary measures to maintain confidence in their statistics and to monitoring the implementation of the code." To that end, the Commission planned to draft "an agreed core of basic principles" with a view to incorporating it in the regulation on European statistics. 63. According to the Commission, the detailed contents of each commitment will be elaborated with the respective Member States. Each Member State would define its own quality assurance and improvement programme, reflecting its progress in implementing the code of practice and identifying priority actions needed to ensure proper implementation of the minimum standards. 64. So far, the "commitments on confidence in statistics" have received a cautious endorsement by the Council, welcoming the proposal "in its principle" only, "with the launch of pilot exercises as a first stage" [43]. FURTHER ENHANCEMENT OF THE LEGAL FRAMEWORK WOULD HELP TO SUPPORT FULL IMPLEMENTATION OF THE CODE 65. In 2005 the code of practice was conceived as a self-regulatory instrument. Since then numerous references to the code have been introduced in statistical legislation. The regulation on European statistics of 2009 stipulates that European statistics are to be developed, produced and disseminated "in conformity with" the statistical principles set out in the treaty "and further elaborated" in the code of practice [44]. Nevertheless, there is ambiguity about the nature of obligations arising from the existing regulation with respect to adherence to the code. This is for example reflected in the recommendation of the Task Force on Strengthening Economic Governance in the EU that the "binding nature of the "European statistics code of practice" should be reinforced" [45]. The amendments to the regulation on European statistics proposed by the Commission in April 2012 will not resolve the existing ambiguity about the nature of the obligation to adhere to the code. Whilst the Commission maintains that the code is a self-regulatory instrument, it proposes to introduce a provision which requires Member States to "take all necessary measures to implement the code of practice" [46]. 66. Furthermore, the regulation on European statistics does not provide for a procedure allowing for inspections of statistical production processes from the initial data source to the final outcome, for cases where misrepresentations of data might have occurred. Without such a mechanism [47] it is not possible to act swiftly in such cases. To be credible, an inspection mechanism would also require independent supervision. 67. In its April 2011 communication, the Commission did not address the major issue raised by ESGAB concerning its mandate and the resources at its disposal. In its 2010 report [48] ESGAB had underlined that its legal base should be strengthened in order to provide a possibility to act appropriately if the credibility of the ESS as a whole is at risk. In the 2011 report this point was reiterated [49] and in addition ESGAB stated that its legal basis would benefit from clarification of the expectations set by its founders and the means to achieve them [50]. In future, ESGAB’s role should include independently overseeing peer reviews and inspections of statistical production processes. MANAGING THE MULTIANNUAL EUROPEAN STATISTICAL PROGRAMME 68. The reliability and credibility of statistics depend as much on the adherence to and implementation of the code of practice as they depend on the management of the European statistical programme [51]. 69. The European statistical programme provides the framework for the development, production and dissemination of European statistics. The programme decision for the years 2008 to 2012 was adopted in December 2007 [52]. It lays down 131 "objectives" and "main initiatives", many of them requiring steady implementation over the entire programme period through annual statistical work programmes (SWPs). Figure 4 shows the set-up for programming the production of European statistics. FIGURE 4 SET-UP FOR PROGRAMMING THE PRODUCTION OF EUROPEAN STATISTICS +++++ TIFF +++++ Source: European Court of Auditors. 70. The programme 2008 to 2012 has a financial envelope of 274,2 million euro for the 5-year period. Most of the financing supports investment and capacity building for new initiatives in Member States. Other legal instruments provide additional funding of actions, for example the regulation on farm structure surveys [53] with a financial envelope of 58,85 million euro for the period 2008 to 2013 and the programme for the modernisation of European enterprise and trade statistics (MEETS) [54] with an envelope of 42,5 million euro for the period 2009 to 2013. Nevertheless, the bulk of the cost for producing statistical information has to be borne by Member States [55]. THE REQUIREMENTS OF THE FINANCIAL REGULATION FOR QUALITY EVALUATIONS WERE NOT MET FOR THE 2008 TO 2012 PROGRAMME THE LACK OF AN EX ANTE EVALUATION OF THE 2008–12 PROGRAMME HAD A NEGATIVE IMPACT ON ITS DESIGN AND MAKES ITS MONITORING DIFFICULT 71. Prospective ("ex ante") evaluations are essential for the preparation and efficient management of Union programmes. Ex ante evaluations help to ensure that the delivery of policy objectives will be successful, that the instruments used will be efficient and that reliable evaluation of results based on appropriate indicators will be possible later on [56]. 72. Eurostat did not perform an ex ante evaluation prior to the adoption of the 2008–12 programme. This had a negative impact on the design of the programme and on monitoring its implementation, as no indicators were defined. 73. In June 2010, the Commission reported to the European Parliament and the Council on the mid-term evaluation of the 2008–12 programme [57]. The report stated that "many of the objectives (around 90 %) are on track and likely to be achieved by the end of 2012 although resource constraints in the ESS may render this more difficult than expected". The report did not give a detailed account of progress in achieving the 131 objectives of the programme. 74. Based on Eurostat’s replies to a questionnaire, the Court sought to assess the state of implementation of the programme as of July 2011. For 68 of the 131 objectives, the assessment was difficult in the absence of appropriate indicators. Moreover, in most of these cases, objectives are vaguely worded (see Box 4) so that limited activity is sufficient to claim that these objectives have been or are being achieved. The audit found that, even under these favourable assumptions, at least 20 % of the 131 objectives were not likely to be achieved on time. BOX 4 EXAMPLES OF VAGUELY WORDED OBJECTIVES Extracts from Annex I ("Cross cutting issues") and Annex II ("Objectives and actions") of the 2008–12 programme "reduce the distance between users and producers by improving communication with different groups and networks of users" (objective 13), "extend the use of ad hoc modules in Community surveys in specific cases, increasing the responsiveness to new needs" (objective 24), "the existing lifelong learning statistics framework in terms of quality will be improved" (objective 85), "the development of the relevant statistics for monitoring food safety will be pursued" (objective 98), "the visibility of statistics in national and regional development plans will be increased" (objective 127). Source: Decision on the 2008–12 programme. THE MID-TERM EVALUATION DID NOT IDENTIFY THE NEED FOR REVISION OF THE 2008–12 PROGRAMME 75. The Commission’s mid-term report did not consider whether a revision of the programme was necessary in the light of developments after the adoption of the programme in December 2007, namely: (a) the financial crisis and the need to shift priority to statistics related to economic and monetary policy, taking into account the impact on NSIs’ resources; (b) the challenge of improving the Greek statistical system and the need to implement a preventive approach to avoid similar cases elsewhere in the ESS; (c) new political initiatives launched by the Commission, notably the EU 2020 strategy for smart, sustainable and inclusive growth [58]; (d) the new legal framework for cooperation within the ESS provided by the 2009 regulation on European statistics; (e) the Commission communication on a new approach for producing European statistics ("a vision for the next decade") [59]. 76. As the programme was not revised to take into account these new developments, parts of it have become outdated. There is a gap between actual activities and what has been provided for by the legislator. REPORTING ON PROGRAMME IMPLEMENTATION AND ACHIEVEMENTS IS UNSATISFACTORY NO FURTHER REPORTS ON THE IMPLEMENTATION OF THE 2008–12 PROGRAMME BEFORE THE END OF 2013 77. Eurostat confirmed that, apart from the 2010 report on the mid-term evaluation, no other reports exist or will be produced on the implementation of the programme, neither for internal nor for external purposes, before the final evaluation report which is due by the end of 2013. NO CONSOLIDATED REPORTING ON THE IMPLEMENTATION OF THE ANNUAL STATISTICAL WORK PROGRAMMES 78. Given the characteristics of the multiannual programme, Eurostat’s main tool to steer statistical activities of the ESS is the annual statistical work programme (SWP) of the Commission. This programme includes a detailed list of policy- and/or expenditure-related points for action ("outputs") to be achieved during the year [60]. No systematic information is available as to whether the outputs are delivered. 79. Weaknesses in reporting also hamper ESAC in playing its advisory role in full. ESAC serves as a communication channel for users, respondents and producers of statistics on the objectives and priorities of the Union’s statistical information policy (see Box 5). The audit found that ESAC is not yet playing its advisory role in full, in part due to its broad brief covering all statistical areas, its heterogeneous composition and the limited availability of some of its members. Eurostat could improve its support to ESAC’s functioning through more and better-tailored information on the budgetary and financial implications of statistical programming choices and on the implementation of statistical programmes. BOX 5 EUROPEAN STATISTICAL ADVISORY COMMITTEE (ESAC) Bringing together users, respondents and producers of statistics The European Statistical Advisory Committee (ESAC) was set up in June 2009 and has 24 members: 12 members in their personal capacity representing users, respondents or other stakeholders in Union statistics (including the scientific community, the social partners and civil society), 11 members appointed directly by the institutions and bodies to which they belong and the director-general of Eurostat as an ex officio member without a voting right. ESAC meets three times a year for 1-day sessions in Brussels. Eurostat provides the secretariat of ESAC. During its first 3 years, ESAC issued a contribution to the discussion about the modernisation of structural business statistics, a one-page statement entitled "Good decisions need good data, good data require sufficient statistical resources", and, at the request of Eurostat, four opinions related to draft statistical programmes. So far, neither the European Parliament nor the Council made use of the option to request an opinion from ESAC. In June 2011, ESAC agreed that each member should establish contacts with users in one or two Member States to set up a network of national user representatives. Source: European Court of Auditors. EUROSTAT HAS IMPROVED MANAGEMENT OF GRANTS FROM THE PROGRAMME’S FINANCIAL ENVELOPE AND HAS STARTED TO TACKLE WEAKNESSES IN PROCUREMENT 80. The decision on the 2008–12 programme [61] requires national authorities and Eurostat to strive for a reputation of good management and efficiency in order to strengthen the credibility of statistics. 81. Eurostat uses two main tools to implement the financial envelopes made available under the 2008–12 programme and other programmes: grant agreements and public procurement contracts. Figure 5 shows the respective shares of the two instruments in the total amounts committed over the years 2007 to 2011 [62]. FIGURE 5 SHARE OF GRANTS AND CONTRACTS IN TOTAL AMOUNTS COMMITTED Grants and contracts distribution +++++ TIFF +++++ Source: European Court of Auditors based on Eurostat data. GRANT MANAGEMENT 82. Eurostat gives more than 90 % of its grants to NSIs and other national authorities designated by the Member States. 83. Article 109(2) FR stipulates that grants may not have the purpose or effect of producing a profit for the beneficiary. Therefore, in order to determine the appropriate amount of the final grant payment, Eurostat cannot limit itself to verifying whether the objective of a grant agreement was achieved, but must also verify the resources actually used by the beneficiaries to achieve the objective. This obligation makes the management of grants more cumbersome than the management of procurement contracts. Since 2011, interested beneficiaries are given the possibility to use standard scales of unit costs by staff category to simplify grant management. An alternative approach to consider would be a performance-based system, which relies on agreed indicators and objectives (outputs and outcomes) [63] without the need to link payments to costs incurred. A SIGNIFICANT RISK OF OVERPAYMENTS HAD TO BE ADDRESSED AND MANAGED 84. Reservations had to be made concerning Eurostat’s grant management in the annual activity reports (AAR) of the director-general of Eurostat for the years 2001 to 2004 and 2006. They concerned the inadequate definition and calculation of eligible costs and the inadequate number of controls carried out, resulting in a significant risk of overpayments. Often, no supporting documents were available for ex post controls regarding staff costs, equipment costs, consumables and supplies and other direct costs [64]. EFFORTS TO IMPROVE FINANCIAL MANAGEMENT OF GRANTS PRODUCE RESULTS 85. In April 2009, Eurostat redefined its control strategy. The approach was to shift emphasis from ex post controls (after final payments) to ex ante controls (prior to final payments). The objective was to identify and prevent errors and irregularities before final payments, allowing for immediate correction and avoiding time-consuming recovery actions. 86. Under reinforced ex ante controls beneficiaries are asked to provide Eurostat with documents supporting a sample of the eligible costs declared. 87. Eurostat’s efforts to improve the financial management of grants, including through significantly clearer guidelines for beneficiaries, produce results. Nevertheless, some beneficiaries [65] experienced consistent difficulties in submitting to Eurostat the documentation required to determine whether costs where eligible. 88. The audit examined a sample of reinforced ex ante controls carried out by Eurostat and found most of them adequate. However, the following shortcomings had not yet been addressed by Eurostat. (a) The samples selected for verifying the staff cost declared were often not complying with the minimum requirements laid down in Eurostat’s own guidelines. (b) Only in one case verifications were carried out on the spot with a beneficiary to assess its time registration system and supporting documents for staff cost. (c) The controls were not documented by a check-list which made it difficult to assess their nature and adequacy. (d) The very low number of payment requests randomly selected by Eurostat for controls [66] makes it difficult to assess the overall risk of error. MANAGEMENT OF PROCUREMENT PROCEDURES 89. Eurostat uses calls for tender mainly for IT procurement and for buying statistical services [67]. Procurement procedures within Eurostat are to a large extent decentralised to operational directorates. 90. The Court’s audit confirmed major risks reported by Eurostat’s internal audit capability (IAC) in February 2011 to management. In its risk analysis, the IAC had identified potentially excessive reliance on third parties and quasi-monopoly situations and weaknesses in the evaluation process, including potential favouritism. MARKET CONCENTRATION AND DIFFICULTIES WITH SELECTION AND AWARD CRITERIA 91. With regard to market concentration, the Court’s analysis of Eurostat’s follow-up tables on contract awards over the period 2009 to 2011 showed that the top three contractors had obtained 30 % of the total amount awarded through procurement. Market concentration was aggravated by the fact that certain Eurostat units depended to a large extent on the top three contractors. With regard to the procurement process, the Court’s audit found inappropriate use of the negotiated procedure and recurrent difficulties in defining and applying selection and award criteria. 92. It was too early for the Court’s audit to assess the impact of Eurostat’s action plan of March 2011 designed to address the recommendations made in the IAC report. Implementation of some key measures of this action plan only started in January 2012. FOSTERING COMPETITION UNDER THE BEST-VALUE-FOR-MONEY PROCEDURE 93. To foster competition, Eurostat decided that it will no longer systematically announce a maximum budget in tender specifications. From 2012, this has only been allowed as an exception to be duly justified. However, Eurostat has maintained the use of minimal thresholds for the quality component of award criteria under the best-value-for-money procedure. Such minimal thresholds weaken price competition, in particular when the weighting of quality and price criteria is set at 70:30 which is most frequently the case in Eurostat’s procurement procedures. RE-PRIORITISATION WITHIN THE ESS IS BEHIND SCHEDULE AND HAMPERED BY THE ABSENCE OF RELIABLE INFORMATION ON COST THE CONSEQUENCES OF THE FINANCIAL AND DEBT CRISIS MAKE RE-PRIORITISATION EVEN MORE IMPORTANT 94. NSIs and the ESS as a whole have to tackle the consequences of the financial crisis, which sometimes involve severe budget cuts. This often requires a change of priorities. There are now difficult trade-offs to be made between maintaining or increasing the quality of existing statistical products, responding to requests for statistics in new domains and the need to invest in measures to increase productivity in the longer term. The choices made can have an impact on statistical innovation which may suffer as a result. THE FULL COST OF PRODUCING EUROPEAN STATISTICS IS NOT KNOWN 95. Article 6(1) of the decision on the 2008–12 programme required the Commission to provide a preliminary analysis of the financial burden-sharing between Union and Member State budgets. Due to a lack of appropriate data, this analysis was limited to the general conclusion that grants from the EU budget were relatively small compared to the total annual expenditure of Member States caused by the production of European statistics [68]. REPRIORITISATION WAS BEHIND SCHEDULE 96. The audit found that Eurostat was behind schedule with the re-prioritisation exercise [69]. Although required under Article 6(3) of the decision, the Commission did not report on the outcome of reprioritisation in its mid-term evaluation. Such reporting should have included estimations of costs and burdens for statistical projects and fields covered by the 2008–12 programme, as well as an assessment of emerging statistical needs, in particular for new Union policies. THE 2010 STRATEGY-DRIVEN APPROACH PRODUCES LIMITED RESULTS SO FAR 97. Because of the lack of progress with establishing a systematic measurement of cost and the difficult situation of the ESS members in the context of budget cuts, Eurostat proposed a new strategy-driven approach for re-prioritisation to the ESS Committee in November 2010. Its implementation (essentially through identifying "negative priorities" [70] on an annual basis) has produced limited results so far. 98. The list of negative priorities for 2011 consisted almost exclusively of new actions to be postponed. For the 2012 work programme, practical consequences of the new approach were assessed as "limited" according to a report submitted to the ESS Committee in May 2011. This points to a need for additional efforts to encourage statistical innovation, for example by proposing to users to phase out statistics which have become less relevant in exchange for new statistics. 99. When invited by Eurostat to issue an opinion on the draft programme for 2012, ESAC stated [71] that it encounters some difficulty in determining priorities because of the fact that the budgetary consequences of different decisions are hard to assess due to the different methodologies used to calculate costs in the Member States, the lack of financial information and, in some cases, the problems of differentiating between the European and national shares in the costs of producing statistics. These are all important elements required for Eurostat’s strategy. THE DESIGN OF THE DRAFT 2013 TO 2017 PROGRAMME HAS IMPROVED BUT SUCCESSFUL IMPLEMENTATION DEPENDS ON BETTER ANNUAL PLANNING AND REPORTING 100. In late 2011 the Commission adopted the draft statistical programme 2013–17 as a proposal for a regulation of the European Parliament and of the Council [72]. 101. In comparison to the programme currently in force, the proposal provides a clearer hierarchy of priorities and a more flexible structure. The draft programme sets out an ambitious infrastructure of statistical information and has a strong focus on implementing new methods for producing European statistics. 102. The annex to the programme ("Statistical infrastructure and objectives of the European statistical programme 2013–17") provides objectives together with implementation actions which can serve as indicators. Most of these 104 indicators are useful for achieving measurable progress provided that they are supplemented by precise targets and milestones laid down in the annual planning and verified in a process of systematic annual reporting. 103. From 2014, the programme will include objectives in the area of enterprise and trade statistics and a separate programme for the modernisation of European enterprise and trade statistics will no longer be necessary [73]. 104. The implementation of the programme is to be covered from two different multiannual financial frameworks (for the years 2007 to 2013 and 2014 to 2020) for a total of 299,4 million euro. Options to better synchronise the multiannual statistical programme with the multiannual financial planning were not considered during the ex ante evaluation and the impact assessment of the programme. 105. 146,6 million euro are allocated to the objective "Implement the new method of production of European statistics". No further details are provided on the breakdown of this amount nor with regard to the role of "non-profit-making organisations" eligible for grants [74] which have as their primary objectives and activities the promotion and support of the implementation of the code of practice and of new methods of production of European statistics aiming at efficiency gains and quality improvements at European level. CONCLUSIONS AND RECOMMENDATIONS 106. Statistics are essential for Europe’s democratic process and good governance. Public confidence in European statistics is crucial in a Union where political decisions need to be evidence-based and where an increasing number of decisions are directly triggered by statistical data or by indicators derived from them. 107. The European Union, its Member States, and their statistical authorities must conform with the obligations set out in Article 338 TFEU on impartiality, reliability, objectivity, scientific independence, cost-effectiveness and statistical confidentiality in the production of statistics. Since 2005 considerable efforts have been made to enhance the European statistical system. However, the move towards a better quality framework for European statistics is slow, is not yet completed and remains a challenge for all those involved. RECOMMENDATION 1 The European Union, its Member States and their statistical authorities share a common responsibility for maintaining trust in Europe’s democratic process. They should strengthen the system of European statistics to ensure professional independence, sufficient resources, effective supervision, with sanctions and swift improvement measures for cases where quality standards are not respected. 108. The audit found that the European Statistics Code of Practice has only been partly implemented and that full implementation remains a challenge for all those involved both at the European level and within Member States. The code sets demanding standards but lacks strong verification and enforcement tools. The Commission did not involve national governments in the responsibility of respecting and applying the code. It has not yet taken all steps required to guarantee its own full adherence to the code. Better information on the current state of implementation of the code throughout the European statistical system (ESS) is required. The Commission’s recent initiatives to give new momentum to achieving full adherence to the code go in the right direction but not far enough to address all concerns. Ambiguity about the nature of the obligation to adhere to the code persists, no inspection mechanism is proposed for cases where misrepresentations of data might have occurred and no proposal is made to develop an independent supervisory function. RECOMMENDATION 2 In order to achieve full implementation of the European Statistics Code of Practice the Commission should: (a) propose amendments to the regulatory framework for the production of European statistics that provide a sound basis for review, enforcement and, in appropriate cases, verification and inspection covering the institutional environment of statistical production, the statistical processes and the statistical output both at EU and national level; (b) take the necessary steps to ensure legal certainty about the nature of the obligation to adhere to the code of practice; (c) propose, in relation to (a) and (b), to develop a supervisory function to oversee reviews, verifications and inspections, for example by extending the current remit of the European Statistical Governance Advisory Board (ESGAB); (d) enhance the professional independence of the Chief Statistician of the European Union by appointing her/him for a fixed-term mandate after having received a favourable opinion from ESGAB and an endorsement by the European Parliament and the Council; (e) bring its internal decision on Eurostat’s role in line with the requirements of the code of practice, enable Eurostat to apply its protocol on impartial access to data without restriction and phase out the mechanism of sub-delegated operational credits for statistical production which makes Eurostat, in part, financially dependent on other Commission services; (f) launch the new round of peer reviews in the European statistical system, envisaged by the Commission for 2013, covering compliance with all principles of the code of practice and including a strong external element to allow for independent assessments and comparable results; (g) consider introducing rolling peer reviews for the most important statistical domains covering the entire production chain including providers of administrative data. 109. Flaws in the design of the statistical programme 2008 to 2012 hamper its use as an effective planning, monitoring and accountability tool for improving the production of European statistics. Eurostat’s grant management improved but weaknesses in procurement persist. Reprioritisation of statistical activities towards new challenges was slower than expected. The draft programme 2013–17 provides an opportunity to reengineer the ESS in order to make it more efficient and flexible. RECOMMENDATION 3 In order to fully exploit the potential of the upcoming European statistical programme for the years 2013 to 2017 Eurostat should: (a) supplement the implementation actions included in the European statistical programme by precise targets and milestones to be (re)defined each year in the annual statistical programmes and followed up through a process of systematic and consolidated annual reporting to the European Parliament and the Council; (b) use the option of revising the programme once its implementation is under way in case of major developments, also considering whether the programme should be prolonged to synchronise it with the multiannual financial framework 2014 to 2020; (c) ensure a systematic review of statistical priorities, notably through regular assessments of the relevance of statistical outputs and of costs and burdens for the ESS and its members and for respondents; (d) encourage, in the context of reprioritisation, where appropriate, statistical innovation, for example by phasing out existing statistical products in exchange for new ones; (e) improve its support to ESAC’s functioning through more and better-tailored information on the budgetary and financial implications of statistical programming choices and on the implementation of statistical programmes; (f) simplify and improve the efficiency of the financial management of grants by resorting to standard scales of unit costs for staff and to lump sums for data sets provided through surveys. In this context, it should explore the option of a performance-based system of grant management, which relies on agreed indicators and objectives; (g) enhance competition in procurement procedures, notably by giving more weight to the price criterion in best-value-for-money procedures and avoiding minimum thresholds that weaken price competition. This report was adopted by Chamber IV, headed by Mr Louis GALEA, Member of the Court of Auditors, in Luxembourg at its meeting of 26 June 2012. For the Court of Auditors +++++ TIFF +++++ Vítor Manuel da Silva Caldeira President [1] Special Eurobarometer "Europeans’ knowledge of economic indicators", p. 37. [2] See Presidency Conclusion of the European Council of 22 and 23 March 2005 and Conclusion of the European Council of 28 and 29 October 2010. [3] Regulation (EC) No 223/2009 of the European Parliament and of the Council of 11 March 2009 on European statistics and repealing Regulation (EC, Euratom) No 1101/2008 of the European Parliament and of the Council on the transmission of data subject to statistical confidentiality to the Statistical Office of the European Communities, Council Regulation (EC) No 322/97 on Community Statistics, and Council Decision 89/382/EEC, Euratom establishing a Committee on the Statistical Programmes of the European Communities (text with relevance for the EEA and for Switzerland) (OJ L 87, 31.3.2009, p. 164). [4] The Statistical Programme Committee assisted the Commission in the general coordination of statistical programmes at EU and national level. It ceased to exist in 2009 when the regulation on European statistics was adopted. [5] The code of practice was never published in the Official Journal. Its most recent version can be found on Eurostat’s website http://epp.eurostat.ec.europa.eu/cache/ITY_OFFPUB/KS-32-11-955/EN/KS-32-11-955-EN.PDF [6] Decision No 1578/2007/EC of the European Parliament and of the Council of 11 December 2007 on the Community statistical programme 2008 to 2012 (text with EEA relevance) (OJ L 344, 28.12.2007, p. 15). [7] Article 5 of the Commission Decision 97/281/EC of 21 April 1997 on the role of Eurostat as regards the production of Community statistics (OJ L 112, 29.4.1997, p. 56). [8] For the definition of "the Commission (Eurostat)" see Article 6 of the regulation on European statistics. [9] Hereinafter referred to as "operational credits". Figures do not include administrative expenditure. [10] Decision No 235/2008/EC of the European Parliament and of the Council of 11 March 2008 establishing the European Statistical Governance Advisory Board (text with EEA relevance) (OJ L 73, 15.3.2008, p. 17). [11] Decision No 234/2008/EC of the European Parliament and the Council of 11 March 2008 establishing the European Statistical Advisory Committee and repealing Council Decision 91/116/EEC (text with EEA relevance) (OJ L 73, 15.3.2008, p. 13). [12] The code was promulgated as a self-regulatory instrument through the Commission recommendation of 25 May 2005 on "the independence, integrity and accountability of the national and Community statistical authorities", COM(2005) 217 final of 25 May 2005. In September 2011, the code was revised by the ESS Committee. The revised version is published on Eurostat’s website but the Commission has not updated its recommendation of 25 May 2005. [13] "The Council notes that on several occasions the fiscal statistics had been revised after a new government took office. The Council considers that the compilation and reporting of statistics for the European deficit procedure must not be vulnerable to political and electoral cycles. (...) The Council considers that integrity, independence and accountability of data compilers, and the transparency of the compilation methods, underpinned by the appropriate institutional arrangements, are crucial to ensure (...) high-quality statistics." (Conclusions of the Economic and Financial Affairs Council of 2 June 2004, Doc. 9779/04 (Presse 172), p. 11). [14] In April 2004, when a new Greek government took office, it ordered a "fiscal audit" that resulted in significant revisions of Greek government deficit and debt figures going back to 1997. Under this revision, Greece was shown to have been above the 3 % deficit ceiling in 1999. [15] Principles 1, 3 and 4. [16] Third annual report to the European Parliament and the Council on the implementation of the European Statistics Code of Practice by Eurostat and the European statistical system as a whole by the European Statistical Governance Advisory Board, published on 1 December 2011. [17] Based on the report of its task force "Sponsorship on quality" co-chaired by Eurostat and Statistics Norway. [18] COM(2005) 217 final. [19] Recommendations B, C, and G. [20] COM(2011) 211 final of 15 April 2011 "Towards robust quality management for European Statistics". [21] The European Free Trade Association (EFTA) is an intergovernmental organisation set up for the promotion of free trade and economic integration to the benefit of its four Member States: Iceland, Liechtenstein, Norway and Switzerland. [221] There was no further round of peer reviews since 2008. All 35 indicators examined by peer reviews related to principles 1 to 6 and 15 of the code (compliance with principles 7 to 14 of the code was not covered). [22] COM(2008) 621 final of 7 October 2008 "2008 report from the Commission to the European Parliament and the Council on implementation of the code of practice". [23] COM(2010) 1 final of 8 January 2010 "Report on Greek government deficit and debt statistics". [24] Areas where weaknesses were identified during the peer reviews but without improvement actions are not covered by the Eurostat’s monitoring. [25] See the 2011 Eurostat monitoring report on NSI compliance with the code of practice. [26] Not including two new actions defined in 2011. [27] OJ L 112, 29.4.1997, p. 56. [28] Commission Decision 1999/352/EC, ECSC, Euratom of 28 April 1999 establishing the European Anti-fraud Office (OLAF) (OJ L 136, 31.5.1999, p. 20). [29] The share of credits subdelegated by other directorates-general of the Commission in the total of commitments made was 24 % in 2011 (27 % in 2010 and 33 % in 2009). More than two thirds of these subdelegated credits came from DG Agriculture and Rural Development. [30] Article 8 of Regulation (EU) No 1173/2011 of the European Parliament and the Council of 16 November 2011 on the effective enforcement of budgetary surveillance in the euro area (OJ L 306, 23.11.2011, p. 1). Fines may amount to up to 0,2 % of the GDP of the Member State concerned. [31] Report on peer review on the implementation of the European Statistics Code of Practice in Eurostat, 24– 26 October 2007. [32] Council Regulation (EC) No 479/2009 of 25 May 2009 on the application of the protocol on the excessive deficit procedure annexed to the Treaty establishing the European Community (OJ L 145, 10.6.2009, p. 1). [33] Published on Eurostat’s website (http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/Impartiality_protocol_REV2_FINAL_EN.pdf). [34] See page 20 of the report on eer review on the implementation of the European Statistics Code of Practice in Eurostat, 24- 26 October 2007. [35] Related to the quality of gross national income (GNI) figures, Article 6 of Council Regulation (EC, Euratom) No 1287/2003 of 15 July 2003 on the harmonisation of gross national income at market prices (GNI regulation) (OJ L 181, 19.7.2003, p. 1) provides an option for the Commission to carry out "information visits" to Member States as part of its verification work; see paragraphs 2.28. and 2.30 of the Court’s 2010 annual report. [36] See recommendation 4 of the 2011 annual report. [37] COM(2011) 211 final. [38] Report on Greek government deficit and debt statistics, COM(2010) 1 final. [39] Council Regulation (EU) No 679/2010 of 26 July 2010 amending Regulation (EC) No 479/2009 as regards the quality of statistical data in the context of the excessive deficit procedure (OJ L 198, 30.7.2010, p. 1). [40] Article 12(1) of Regulation (EC) No 479/2009. [9999] The dialogue visit took place on the same day as the methodological visit. [8888] During this visit a joint meeting with the Irish NSI took place. [7777] Upstream dialogue visit. [6666] Standard dialogue visit with an upstream element. [41] COM(2011) 211 final. [42] COM(2012) 167 final of 17 April 2012 "Proposal for a regulation of the European Parliament and of the Council amending Regulation (EC) No 223/2009 on European statistics". [43] See the Council conclusions of 20 June 2011 on the Commission communication "Towards robust quality management for European Statistics" (http://www.consilium.europa.eu/uedocs/cms_data/docs/pressdata/en/ecofin/122932.pdf). [44] Article 1 and Article 11(1). [45] Section 2.1.4 of the report of 21 October 2010 which was endorsed by the European Council of 28 and 29 October 2010. [46] See proposed new Article 11(3). [47] The excessive deficit procedure (EDP) is currently the only domain with such a mechanism in place. [48] Recommendation No 10 of ESGAB’s annual report 2010. [49] Recommendation No 8 of ESGAB’s annual report 2011. [50] In its resolution of 17 June 2010 (P7_TA(2010)0230) on the quality of statistical data in the Union and enhanced auditing powers by the Commission (Eurostat), the European Parliament called on the Commission and the Council "to involve the European Statistical Governance Advisory Board more closely as an independent adviser; the advisory board may assist the Commission (Eurostat) during its visits to the Member States" (OJ C 236E, 12.8.2011, p. 76). [51] Union legal acts also refer to the programme as "the Community statistical programme" or "the Union statistical programme". [52] Decision No 1578/2007/EC (hereinafter referred to as "the decision on the 2008–12 programme"). [53] Regulation (EC) No 1166/2008 of the European Parliament and of the Council of 19 November 2008 on farm structure surveys and the survey on agricultural production methods and repealing Council Regulation (EEC) No 571/88 (text with EEA relevance) (OJ L 321, 1.12.2008, p. 14). [54] Decision No 1297/2008/EC of the European Parliament and of the Council of 16 December 2008 on a programme for the modernisation of European enterprise and trade statistics (MEETS) (text with EEA relevance) (OJ L 340, 19.12.2008, p. 76). [55] The full cost of producing European statistics is not known (see paragraph 95). [56] Article 21(1) of the implementing rules of the financial regulation (FR). [57] COM(2010) 346 final of 30 June 2010. [58] COM(2010) 2020 final of 3 March 2010. [59] COM(2009) 404 final of 10 August 2009. [60] A reference to the multiannual programme was given for each output for the first time in the 2011 SWP. However, as the 2008–12 programme has not been amended to take into account recent developments, such links are difficult to make. As a consequence, in many places, references are made to the "general objectives" of the multiannual programme or to an annex of the multiannual programme which refers to "better regulation" in very general terms. [61] Article 3(b). [62] In 2011, 172 grant agreements and 312 contracts were signed (283 grant agreements and 320 contracts in 2010). [63] See section 4.2 of COM(2012) 42 final of 8 February 2012 "A simplification agenda for the MFF 2014–20". [64] In 2008, the Commissioner in charge of Eurostat informed the Commission on the outcome of fourteen ex post controls carried out by Eurostat in 2005 and 2006 on cost declarations made by eleven NSIs and three other grant beneficiaries. Out of a total amount of 14 million euro covered by these controls, the eligibility of 6,6 million euro (47,14 %) was questioned by Eurostat. Finally, to settle these cases taking account of outputs and results achieved, flat-rate corrections for 1,3 million euro were made. [65] Since 2009, eight beneficiaries came temporarily under risk-based reinforced ex ante controls covering most of their payment requests. [66] Over the period 2009–11 eleven requests under reinforced examination, all dating from 2009. [67] With regard to statistical services, the Commission decision of 23 July 2003 to internalise tasks to the maximum extent was not fully implemented. Some Eurostat units continue to rely on external service providers for data management and collection tasks. [68] As a rule, co-funding from the Union budget is not possible for contributions of NSIs to the production of European statistics when these contributions are required by European statistical legislation. [69] Set out in detail in section 3.7. ("The balance of costs and benefits") of Annex I of the 2008–12 programme. [70] "Negative priorities" include repealing existing legal requirements for producing statistics, stopping a voluntary data collection based on a gentlemen’s agreement amongst Eurostat and NSIs and reviewing areas for simplification. [71] Opinion on the draft Commission annual statistical programme 2012 of 28 March 2011. [72] COM(2011) 928 final of 21 December 2011 "Proposal for a regulation of the European Parliament and of the Council on the European statistical programme 2013–17". No explanation is provided why a regulation is proposed instead of a decision. [73] The current MEETS programme will end on 31 December 2013. [74] Article 11(3) of the proposal for a regulation. -------------------------------------------------- ANNEX STATUS OF IMPROVEMENT ACTIONS RECOMMENDED IN THE PEER REVIEWS (SITUATION AS AT MARCH 2011) Source: ESGAB annual report 2011. Principle | Number of Peer Reviews Improvement Actions | | Number of outstanding peer reviews improvement actions (March 2011) | | | | Total | Completed | | Further progress outside of the NSI | No further work is planned | Progressing and not yet due | Delays within the NSI | Ongoing — no specific deadline | Included in a new action | Total | | New actions (defined in 2011) | Total remaining actions | | February-08 | May-09 | March-10 | March-11 | | | | | | | (a) | (b) | | (c) | (b–a+c) | 1 | Professional Independence | 34 | 1 | 9 | 4 | 7 | | 6 | 1 | 1 | 2 | 3 | 0 | 13 | | 0 | 13 | 2 | Mandate for data collection | 25 | 1 | 8 | 2 | 3 | | 3 | 0 | 2 | 0 | 6 | 0 | 11 | | 0 | 11 | 3 | Adequacy of resources | 48 | 1 | 18 | 12 | 2 | | 4 | 0 | 4 | 1 | 6 | 0 | 15 | | 0 | 15 | 4 | Quality commitment | 103 | 1 | 28 | 8 | 13 | | 3 | 1 | 22 | 12 | 12 | 3 | 53 | | 4 | 54 | 5 | Statistical confidentiality | 43 | 2 | 19 | 4 | 6 | | 0 | 1 | 2 | 1 | 8 | 0 | 12 | | 0 | 12 | 6 | Impartiality and objectivity | 46 | 2 | 16 | 9 | 6 | | 1 | 0 | 2 | 3 | 7 | 0 | 13 | | 0 | 13 | 7 | Sound methodology | 46 | 3 | 15 | 6 | 5 | | 1 | 0 | 4 | 2 | 9 | 1 | 17 | | 1 | 17 | 8 | Appropriate statistical procedures | 40 | 2 | 10 | 6 | 4 | | 0 | 0 | 11 | 1 | 6 | 0 | 18 | | 0 | 18 | 9 | Non-excessive burden on respondents | 54 | 0 | 16 | 6 | 3 | | 3 | 0 | 7 | 1 | 17 | 1 | 29 | | 1 | 29 | 10 | Cost effectiveness | 48 | 0 | 16 | 7 | 6 | | 3 | 0 | 7 | 3 | 6 | 0 | 19 | | 1 | 20 | 11 | Relevance | 27 | 1 | 11 | 7 | 4 | | 0 | 0 | 2 | 0 | 2 | 0 | 4 | | 0 | 4 | 12 | Accuracy and reliability | 35 | 1 | 7 | 7 | 4 | | 0 | 0 | 4 | 3 | 7 | 2 | 16 | | 1 | 15 | 13 | Timeliness and punctuality | 14 | 0 | 5 | 2 | 3 | | 1 | 0 | 0 | 0 | 3 | 0 | 4 | | 1 | 5 | 14 | Coherence and compatibility | 33 | 1 | 9 | 4 | 4 | | 1 | 1 | 3 | 2 | 7 | 1 | 15 | | 2 | 16 | 15 | Accessibility and clarity | 81 | 2 | 29 | 10 | 6 | | 1 | 0 | 9 | 4 | 17 | 3 | 34 | | 2 | 33 | | | | | | | | | | | | | | | | | | | | Total | 677 | 18 | 216 | 94 | 76 | | 27 | 4 | 80 | 35 | 116 | 11 | 273 | | 13 | 275 | | % | 100 % | 3 % | 32 % | 14 % | 11 % | | 4 % | 1 % | 12 % | 5 % | 17 % | 2 % | 40 % | | | | | | | | | | | | | | | | | | | | | | --------------------------------------------------