I.

We have audited:

II.

In our opinion, the consolidated accounts of the European Union (EU) for the year ended 31 December 2016 present fairly, in all material respects, the financial position of the Union as at 31 December 2016, the results of its operations, its cash flows, and the changes in net assets for the year then ended, in accordance with the Financial Regulation and with accounting rules based on internationally accepted accounting standards for the public sector.

III.

In our opinion, the revenue underlying the accounts for the year ended 31 December 2016 is legal and regular in all material respects

IV.

In our opinion, except for the effects of the matter described in the basis for qualified opinion on the legality and regularity of payments underlying the accounts paragraph, the payments underlying the accounts for the year ended 31 December 2016 are legal and regular in all material respects.

V.

We conducted our audit in accordance with the IFAC International Standards on Auditing (ISAs) and Codes of Ethics and the INTOSAI International Standards of Supreme Audit Institutions (ISSAIs). Our responsibilities under those standards are further described in the Auditor’s responsibilities section of our report. We are independent in accordance with the International Ethics Standards Board for Accountants’ Code of Ethics for Professional Accountants (IESBA Code) together with the ethical requirements that are relevant to our audit, and we have fulfilled our other ethical responsibilities in accordance with these requirements and the IESBA Code. We believe that the audit evidence we have obtained is sufficient and appropriate to provide a basis for our opinion.

VI.

Expenditure recorded in 2016 covering spending on a reimbursement basis (3) is materially affected by error. Our estimated level of error for payments made on a reimbursement basis is 4,8  %. Our overall estimated level of error (3,1  %) is still above our materiality threshold, but it is not pervasive. Payments made on an entitlement basis are not affected by a material level of error (4). Our overall conclusion is corroborated by the Commission’s analysis of amounts at risk presented in the annual management and performance report for the EU budget.

VII.

Key audit matters are those matters that, in our professional judgment, were of most significance in our audit of the consolidated accounts of the current period. These matters were addressed in the context of our audit of the consolidated accounts as a whole, and in forming our opinion thereon, and we do not provide a separate opinion on these matters.

VIII.

The EU balance sheet includes pension and other employee benefits amounting to 67,2 billion euro at the end of 2016. This is one of the most significant liabilities in the balance sheet, accounting for almost a third of the total 2016 liabilities of 234,8 billion euro.

IX.

The majority of this provision for pension and other employee benefits (58,7 billion euro) relates to the Pension Scheme of Officials and Other Servants of the European Union (the ‘PSEO’). This pension liability covers the ‘defined benefit’ guaranteed by Article 83 of the Staff Regulations of Officials of the European Communities (the ‘Staff Regulations’) and Article 4(3) of the Treaty on the European Union (the ‘TEU’). The liability recorded in the accounts reflects the amount which would have been included in a pension fund had one been set up to pay existing retirement pension obligations (5). In addition to retirement pensions, it covers invalidity pensions and pensions paid to widows/orphans of EU officials. Under Article 83 of the Staff Regulations, the benefits paid under the pension scheme are charged to the EU budget, Member States jointly guarantee the payment of the benefits and officials contribute one third of the cost of financing the scheme.

X.

The PSEO is a mandatory occupational pension scheme for EU civil servants, under which contributions from staff and from the institutions and bodies that employ them are used to finance future pensions. It is designed, through adjustments to the rate of contribution to the scheme and to pensionable age, to be in actuarial balance by default. The number and variety of parameters used to calculate a long-term projection of pension costs underlines the actuarial nature of this calculation, which is ultimately performed by Eurostat on an annual basis.

XI.

As part of our audit, we evaluated the actuarial assumptions and resulting valuation for the pension provision. We checked the numerical data, the actuarial parameters, the calculation of the provision as well as the presentation in the consolidated balance sheet and the notes to the consolidated financial statements. Our audit of the fair value of the provision detected some incompleteness and inaccuracies in the underlying primary database which do not have a material impact on the EU consolidated accounts. The Commission, as disclosed in note 2.9 to the consolidated financial statements, will take further steps to strengthen its processes used for calculating the employee benefits liability, which we will keep under review.

XII.

At year-end 2016, the Commission estimated that incurred eligible expenses due to the beneficiaries but not yet reported amounted to 102 billion euro (year-end 2015: 106 billion euro). It recorded these as accrued expenses.

XIII.

We examined the methodologies and control systems for year-end estimates applied in the main directorates-general. We drew samples of invoices and pre-financing payments and carried out work on these elements to address the risk of the accrual having been misstated. We sought additional explanation from the Commission’s accounting services for the general methods applied, and in particular for the new method applied for the 2014-2020 programming period in Cohesion.

XIV.

The work we performed enables us to conclude that the estimate of the overall amount of accrued charges stated in the consolidated balance sheet is fair for the main directorates-general. However, in some smaller directorates-general we found systemic weaknesses in relation to the year-end entries. The Commission developed an action plan in this respect.

XV.

Authorities in the Member States transfer a part of the funding advanced by the Commission to financial instruments in the form of loans, equity instruments or guarantees.

XVI.

For the 2007-2013 multiannual financial framework (MFF), EU law did not require these authorities to produce periodic reports on sums held in these instruments for the preparation of the accounts. The Commission therefore estimated the use made of advances, on the basis of the latest available report (in this case, from year-end 2015), assuming that funds would be used in full and evenly over the period of operation (initially up to 31 December 2015, but later extended to 31 March 2017). We note that even though we find the use of financial instruments in 2016 to be outside the eligibility period (see chapter 6, paragraphs 6.20 and 6.21), the Commission does not seek to recover these amounts. Therefore the presentation in the balance sheet and note 2.5 to the consolidated financial statements reflects this Commission position.

XVII.

For the 2014-2020 MFF, authorities need to provide information in every cost claim on advances paid to financial instruments and disbursements made from them to final beneficiaries. On the basis of this information an estimated amount is calculated and recognised in the accounts for the period between the date of the last cost claim received and year-end.

XVIII.

We examined the procedure put in place for recognising the related pre-financing, and we consider that the amount stated in the balance sheet is fair.

XIX.

Management is responsible for the other information. The other information comprises the Financial Statement Discussion and Analysis, but does not include the consolidated accounts and our auditor’s report thereon. Our opinion on the consolidated accounts does not cover the other information and we do not express any form of assurance conclusion thereon. In connection with our audit of the consolidated accounts, our responsibility is to read the other information and, in doing so, consider whether the other information is materially inconsistent with the consolidated accounts or our knowledge obtained in the audit or otherwise appears to be materially misstated. If, based on the work we have performed, we conclude that there is a material misstatement of this other information, we are required to report that fact. We have nothing to report in this regard.

XX.

In accordance with Articles 310 to 325 of the TFEU and the Financial Regulation, management is responsible for the preparation and presentation of the consolidated accounts of the European Union on the basis of internationally accepted accounting standards for the public sector and for the legality and regularity of the transactions underlying them. This responsibility includes the design, implementation and maintenance of internal control relevant to the preparation and presentation of financial statements that are free from material misstatement, whether due to fraud or error. Management is also responsible for ensuring that the activities, financial transactions and information reflected in the financial statements are in compliance with the authorities which govern them. The Commission bears the ultimate responsibility for the legality and regularity of the transactions underlying the accounts of the European Union (Article 317 of the TFEU).

XXI.

In preparing the consolidated accounts, management is responsible for assessing the EU’s ability to continue as a going concern, disclosing, as applicable, matters related to going concern and using the going concern basis of accounting unless management either intends to liquidate the entity or to cease operations, or has no realistic alternative but to do so.

XXII.

The Commission is responsible for overseeing the EU’s financial reporting process.

XXIII.

Our objectives are to obtain reasonable assurance about whether the consolidated accounts of the European Union are free from material misstatement and the transactions underlying them are legal and regular and to provide, on the basis of our audit, the European Parliament and the Council with a statement of assurance as to the reliability of the accounts and the legality and regularity of the transactions underlying them. Reasonable assurance is a high level of assurance, but is not a guarantee that an audit will always detect a material misstatement or non-compliance when it exists. These can arise from fraud or error and are considered material if, individually or in the aggregate, they could reasonably be expected to influence the economic decisions of users taken on the basis of these consolidated accounts.

XXIV.

For revenue, our examination of value added tax and gross national income-based own resources takes as its starting point the relevant macroeconomic aggregates on which these are calculated, and assesses the Commission's systems for processing these until the contributions of the Member States have been received and recorded in the consolidated accounts. For traditional own resources, we examine the accounts of the customs authorities and analyse the flow of duties until the amounts are received by the Commission and recorded in the accounts.

XXV.

For expenditure, we examine payment transactions when expenditure has been incurred, recorded and accepted. This examination covers all categories of payments (including those made for the purchase of assets) other than advances at the point they are made. Advance payments are examined when the recipient of funds provides justification for their proper use and the Institution or body accepts the justification by clearing the advance payment, whether in the same year or later.

XXVI.

As part of an audit in accordance with ISAs and ISSAIs, we exercise professional judgment and maintain professional scepticism throughout the audit. We also:

XXVII.

We communicate with the management regarding, among other matters, the planned scope and timing of the audit and significant audit findings, including any significant deficiencies in internal control that we identify during our audit.

XXVIII.

From the matters communicated with the Commission and other audited entities, we determine those matters that were of most significance in the audit of the consolidated accounts of the current period and are therefore the key audit matters. We describe these matters in our auditor’s report unless law or regulation precludes public disclosure about the matter or when, in extremely rare circumstances, we determine that a matter should not be communicated in our report because the adverse consequences of doing so would reasonably be expected to outweigh the public interest benefits of such communication.

1.1.

We are the EU’s independent auditor. In accordance with the Treaty on the Functioning of the European Union (TFEU), we:

1.2.

The work we do for the statement of assurance (explained in Annex 1.1 ) directly fulfils the first and second of these objectives. In chapter 3 and some policy areas (7), the work we do for the annual report also addresses the economy, efficiency or effectiveness of spending. Taken together, our audit work also provides a key input into our opinions on proposed financial legislation.

1.3.

European Union (EU) spending is an important — but not the only — means of achieving policy objectives. Other important means include the use of legislation and the freedom for goods, services, capital and people to move throughout the EU. In 2016, EU spending amounted to 136,4 billion euro (8), representing 2,0  % of EU Member States’ total general government spending and 0,9  % of EU gross national income ( Box 1.1 ).

1.4.

Our observations concern the European Union’s consolidated accounts (9) (the ‘accounts’) for the 2016 financial year. We received them, together with the accounting officer’s letter of representation, on 26 June 2017. The accounts are accompanied by a ‘Financial Statement Discussion and Analysis’ (FSDA) (10). This analysis is not covered by our audit opinion. In accordance with the auditing standards, we have, however, assessed its consistency with information in the accounts.

1.5.

The accounts published by the Commission show that, at 31 December 2016, total liabilities amounted to 234,8  billion euro compared to 162,7  billion euro of total assets. The economic result for 2016 amounted to 1,7  billion euro (11).

1.6.

Our audit found that the accounts were free from material misstatements. We present our observations on the financial and budgetary management of EU funds in chapter 2.

1.7.

Key audit matters (12) are those matters that, in our professional judgment, were of most significance in our audit of the financial statements for the current period. These matters were addressed in the context of our audit of the financial statements as a whole and in forming our opinion thereon, but we do not provide a separate opinion on these matters. In accordance with International Standard of Supreme Audit Institutions (ISSAI) 1701, we report on key audit matters as part of our opinion (paragraphs VII to XVIII of our statement of assurance).

1.8.

We examine EU revenue and expenditure (13) to assess whether it is in compliance with applicable laws and regulations. We present our audit results for revenue in chapter 4 and for expenditure in chapters 5 to 10 ( Box 1.2 ). Our key findings were:

1.8 and 1.9.

Regarding financial instruments for Cohesion policy (referred to in Box 1.2), the Commission considers the disbursements made to final recipients until end March 2017 in line with its closure guidelines to be within the set eligibility period (see Commission reply to paragraphs 6.20 and 6.21).

1.9.

Our overall estimated level of error improved compared to that of recent years ( Box 1.3 ), but it continues to exceed our benchmark for materiality of 2 %. Nevertheless, around half of 2016 expenditure was free from material error ( Box 1.4 ).

1.10.

We estimate the level of error in reimbursement-based expenditure at 4,8  % (2015: 5,2  %), which significantly exceeds the materiality threshold of 2 % ( Box 1.4 ). In this type of expenditure, beneficiaries claim EU funds for eligible costs they have incurred. Such payments include research projects (chapter 5) and training schemes (chapter 6), as well as projects in regional and rural development (chapters 6 and 7) and development projects (chapter 9). In these schemes, recipients must provide information demonstrating that they are engaged in an activity eligible for support and showing incurred costs for which they may claim reimbursement. In doing so, they have to follow complicated rules regarding what can be claimed (eligibility) and how costs can be properly incurred (e.g. public procurement or state aid rules). This leads to errors which affect our conclusion for MFF headings 1a ‘Competitiveness’ (paragraphs 5.28-5.29), 1b ‘Cohesion’ (paragraphs 6.38-6.39), 2 ‘Natural resources’ (overall — paragraphs 7.32-7.33), and 4 ‘Global Europe’ (paragraphs 9.34-9.35).

1.10.

The ECA has confirmed that projects using Simplified Cost Options (SCOs) are less error-prone than the reimbursement of actual costs. For this reason, the Commission has been actively promoting their utilisation in Cohesion policy and rural development over the last few years. Furthermore, using SCOs reduces the administrative burden and improves the focus on results by Managing Authorities, Paying Agencies and beneficiaries, in particular when SCOs are based on the achievement of project outputs or results.

Public procurement and state aid rules are not specific to EU expenditure. To address the difficulties encountered in these areas, the Commission is implementing action plans with the Member States (see Commission reply to paragraphs 6.15 and 6.18).

In agriculture, rural development remains an area which merits close scrutiny, in particular for investment type measures. The Commission systematically requests Member States to design remedial action plans when control deficiencies are identified and supports their implementation. However, taking into account the need to balance legality and regularity with the achievement of policy objectives while bearing in mind the delivery (management and control) costs, it cannot be expected with any real certainty that an error rate for payments to beneficiaries below 2 % would be attainable with reasonable efforts. Nevertheless, the corrective capacity of Member States' recoveries and the Commission's financial corrections in the years following the year of expenditure enables the Commission to get assurance on the CAP expenditure.

See also Commission reply to paragraphs 1.12 and 6.7.

1.11.

We estimate the level of error for entitlement-based expenditure (including administrative expenditure) at 1,3  % (2015: 1,9  %, Box 1.4 ). In this type of expenditure, beneficiaries receive payment if they meet certain conditions. The payments concerned include student and research fellowships (chapter 5), direct aid for farmers and agri-environment measures (chapter 7) and salaries and pensions (chapter 10). We saw improvements in all areas of entitlement-based expenditure, and we conclude that the estimated level of error for direct aid to farmers was below materiality.

1.11.

1.12.

As shown in Box 1.5 , ‘Cohesion’ was the biggest contributor to our estimated level of error for 2016, followed by ‘Natural resources’, ‘Competitiveness’ and ‘Global Europe’. This distribution is in line with our findings for 2015.

1.12.

1.13.

‘Competitiveness’ (chapter 5): the estimated level of error is 4,1  %, a decrease on 2015 (4,4  %). Much of the expenditure is made on a cost-reimbursement basis, and the errors here essentially reflect different categories of ineligible cost (in particular personnel costs, other direct costs, and indirect costs).

1.13.

1.14.

‘Cohesion’ (chapter 6): the estimated level of error is 4,8  %, lower than the 2015 result (5,2  %). Expenditure in this area is dominated by cost reimbursements. Ineligible costs in expenditure declarations and ineligible projects account for 70 % of the error.

1.14.

1.15.

‘Natural resources’ (chapter 7): the estimated level of error is 2,5  %, a decrease on 2015 (2,9  %). The European Agriculture Guarantee Fund (EAGF) accounts for more than three quarters of expenditure in this area and is free from material error (1,7  %), while in rural development we continue to find a high level of error (4,9  %), particularly for reimbursement expenditure. We did not find material error in entitlement-based expenditure on direct support to farmers, which has simplified land eligibility rules and an effective ex ante control system (IACS) that allows automated cross checks between different databases.

1.15.

See also Commission reply to paragraphs 1.10 and 1.12.

1.16.

‘Global Europe’ (chapter 9): the estimated level of error is 2,1  %, a decrease on the results in 2015 (2,8  %). Missing essential supporting documentation and overstated Commission interim (14) clearings account for two thirds of the total.

1.17.

‘Administration’ (chapter 10): this area is not affected by a material level of error. The estimated level of error is 0,2  %, a decrease compared to the 2015 results (0,6  %). Most expenditure in this area is on the salaries, pensions and allowances paid by EU institutions and bodies.

1.18.

We do not estimate levels of error for other areas of expenditure, including MFF heading 3 (chapter 8). In total, expenditure covered by our statement of assurance in these areas amounted to 2,8  billion euro (2,2  % of the expenditure covered by our audit). Work performed in these areas continues to contribute to our overall conclusions on 2016.

1.19.

Eligibility errors in cost reimbursement schemes continue to be the main contributor to the estimated level of error:

1.19.

Simplification represents the most effective way of reducing both the risk of errors as well as the cost and burden of control. For those programmes with persistently high levels of error, the Commission continuously takes actions, both preventive and corrective, to address their root causes and their impact. More details can be found in the Commission Communication ‘Root causes of errors and actions taken (Article 32(5) of the Financial Regulation)’ — COM(2017) 124 of 28.2.2017.

1.20.

Our estimated level of error in various areas of expenditure shows a much stronger correlation with the basis for payment (i.e. reimbursement or entitlement) than it does with the management mode (15). We found the highest levels of error in ‘Cohesion’ (under shared management) and ‘Competitiveness’ (managed directly by the Commission and indirectly through entrusted entities). Reimbursement schemes dominate expenditure in both of these areas ( Box 1.7 ).

1.21.

For 2016, the estimated level of error for shared management expenditure as a whole amounts to 3,3  % (2015: 4,3  %), and 3,3  % (2015: 4,2  %) for all other forms of operational expenditure (16). The estimated level of error for administrative expenditure is 0,2  % (2015: 0,6  %).

1.22.

Each Commission directorate-general (DG) produces an annual activity report (AAR). This includes a declaration in which the Director-General provides assurance that the report properly presents financial information and that transactions under his/her responsibility are legal and regular. The report also provides an account of the achievement of the key policy objectives (discussed in paragraphs 3.5 and 3.18 to 3.32) and a management report by the Director-General to the Commissioners.

1.22.

1.23.

This year, for the first time, all DGs estimated a level of error in ‘relevant expenditure’. The definition of relevant expenditure (17) is in line with our definition of underlying transactions ( Annex 1.1 , paragraph 10).

1.24.

Following the introduction of the concept ‘amount at risk at closure’ in the 2015 annual management and performance report (AMPR), DGs included this information in their AARs. The Commission focused on the impact of multiannual corrective mechanisms, which will reduce its own estimated level of error.

1.24.

1.25.

As regards legality and regularity indicators, the figures disclosed in the AARs for amounts at risk at payment are, in most cases, broadly in line with our estimates of the level of error ( Box 1.8 ) (18).

1.25.

1.26.

We found that DGs within the same MFF heading took different approaches to presenting error rates and amounts at risk (19). We also found that there were methodological risks in the work done by audit authorities for 2014-2020 expenditure (20). We also found that significant adjustments had to be made by the Commission to the control statistics presented by paying agencies  (21).

1.26.

Please see also Commission reply to paragraphs 6.33 and 6.34.

1.27.

The overall error rate at payment calculated by the Commission for 2016 in the AMPR was material, being between 2,1  % and 2,6  % (compared with 2,3  % and 3,1  % for 2015). Our estimated level of error is 3,1  % (2015: 3,8  %).

1.27.

1.28.

The 2016 FSDA reports total implemented financial corrections and recoveries of 3,4 billion euro. Box 1.9 shows the distribution of the reported amounts by spending area and the stage in the spending cycle at which the Commission applied the corrective actions (declaration/acceptance of expenditure/payment from the budget).

1.28.

1.29.

In 2016, the Commission recorded 0,6 billion euro of corrections and recoveries at source before accepting expenditure. These take the form of deductions of ineligible amounts from claims before the corresponding payments are made from the EU budget (i.e. amounts deducted are not subsequently included in our audited population, since the transactions we select for testing are taken from payments accepted by the Commission). Such recoveries (0,4 billion euro) represented more than four fifths of all corrective activity in direct and indirect management areas in 2016.

1.29.

1.30.

The remaining 2,8 billion euro related to claims that the Commission had already accepted (and so were included in the population of payments we audited). Of this amount:

(a)

For ‘Natural resources’, around 2 billion euro concerned agriculture conformity decisions that the Commission recorded as assigned revenue available to fund agricultural spending.

(b)

For ‘Cohesion’, around 0,6 billion euro represented withdrawals by Member States of previously accepted claims for reimbursement of projects/expenditure and their replacement with new projects/expenditure. Such withdrawals do not result in the return of funds to the EU budget (23).

(b)

The legislation foresees the possibility for Member States to withdraw irregular expenditure and replace it by new, regular one. This protects the EU budget, in conformity with the principle that in shared management, Member States are responsible in the first instance for detecting irregularities and making financial corrections where needed.

(c)

For direct and indirect management areas (mainly ‘Competitiveness’ and ‘Global Europe’), around 0,1 billion euro related to amounts recovered after payment.

1.31.

The Commission and the Member States may apply corrections and recoveries in a variety of ways, following the order set out in the Financial Regulation (24):

(a)

Money unduly spent: basing corrections on the value of the erroneous transactions (individual corrections).

(b)

Systematic errors within a representative sample of transactions: by extrapolating the results to the whole population (extrapolated corrections).

(c)

System weaknesses where it is not possible to precisely calculate the financial implications for the budget: by applying the rates defined in sector-specific guidelines for each situation (‘flat rate corrections’).

(c)

Flat-rate corrections are used when there is a substantiated risk that the system weaknesses established have had a financial impact on the EU budget but it is not possible, with proportionate effort, to identify more precisely the financial damage. They are applied to address system weaknesses and the associated risks they generate for the EU budget, and, as a consequence, they also redress individual irregularities.

1.32.

In our estimate of the level of error, we do not quantify many situations in which the Commission and the Member States apply and report corrections and recoveries:

(a)

Flat-rate corrections are applied in order to redress system weaknesses rather than to eliminate individual irregularities. System weaknesses indicate a risk of irregular expenditure, but it is not always possible to identify a link between these weaknesses and any irregular payment we find and quantify in our estimate of the level of error.

(a)

The Commission protects the EU budget from expenditure incurred in breach of law. Where the exact amounts cannot be identified precisely with proportionate effort, the Commission may apply extrapolated or flat-rate corrections in order to fulfil this obligation. Flat rate corrections are a strong incentive for the Member States to improve their management and control systems.

(b)

Some cases which the Commission assesses as errors leading to corrections are not (quantifiable) errors for us. For example, missed deadlines, lack of publicity and cross-compliance infringements are not taken into account in our estimate of the level of error.

(b)

The Commission notes that corrections from cross-compliance infringements are not included in its estimate of corrective capacity.

1.33.

We seek to take account of corrective measures taken by the Member States and the Commission where these precede our examination. We check whether these corrections (including recoveries of money unduly paid to beneficiaries and corrections at project level) have been applied and adjust the level of error estimate whenever appropriate. However, the impact of corrective measures varies significantly between different areas of spending and different corrective measures.

1.34.

For 41 transactions sampled in 2016, the Commission and Member State authorities had applied corrective measures that directly affected the transaction and were relevant for our calculations. These measures reduced our estimated level of error by 1,2 percentage points (2015: 0,5 percentage points). Changes in the number of transactions affected and in the measures’ impact on our estimated level of error do not indicate that corrective action has become any more or less effective, as these measures concern a relatively small share of our sample and some fluctuation is to be expected from one year to the next.

1.34.

1.35.

We report cases of suspected fraud, whether identified during our audit (including our work on performance) or on the basis of information provided directly to us to the European Anti-Fraud Office (OLAF) for preliminary analysis and possible investigation. We cannot comment on individual cases or on OLAF’s response to these. In 2016:

—

we assessed the legality and regularity of some 1 000 transactions for our audit work on the annual report and, in addition, produced 36 special reports;

—

we reported to OLAF eleven instances of suspected fraud found during our audits (2015: 27), as well as five cases based on information provided by the public (2015:15);

—

the suspected fraud most frequently concerned the artificial creation of conditions to meet eligibility criteria, the non-delivery of goods/services, and the declaration of costs not meeting the eligibility criteria, followed by conflicts of interest and other procurement irregularities.

1.36.

As at 31 December 2016, the cases of suspected fraud we had reported to OLAF between 2010 and 2016 had resulted in 67 investigations, of which 16 were still ongoing. Of the 51 investigations it had completed and closed, OLAF informed us that 28 had either concluded with recommendations to recover money and consider judicial proceedings or were closed because action had already been taken by the Commission or the national authorities. As at 31 December 2016, OLAF reported recommendations for recoveries from these cases totalling 247 million euro. Where OLAF closed a case without making recommendations for further action, OLAF’s most common conclusion was that there was no evidence of fraud affecting the EU’s financial or other interests.

1.37.

The key function of this chapter is to support the audit opinion presented in the statement of assurance.

1.38.

Our audit results show improvements over the last few years. We conclude that entitlement-based expenditure is free from material error and that reimbursement-based expenditure is affected by material error. Based on this, we conclude that the error is not pervasive.

2.1.

This chapter presents our observations on EU budgetary and financial management in 2016 and identifies a number of key risks and challenges for future budgets.

2.2.

The budget for a given year must adhere to the ceilings for commitment and payment appropriations set out in the 2014-2020 multiannual financial framework (MFF)  (1), which states that the EU can, in current prices, commit up to 1 087,2  billion euro and make payments of up to 1 025,4  billion euro between 2014 and 2020 (2). Commitment and payment appropriations (3) for the year must fall within the limits established by the Own Resources Decision (4) (ORD).

2.3.

The Commission presented a mid-term review of the implementation of the current MFF, which was accompanied by legislative proposals. Part of this chapter builds on views we have previously expressed on those legislative proposals that are relevant to the proposal for the next MFF (5).

2.4.

Payments made against budget appropriations were 13,7  billion euro lower than the appropriations available in the initial adopted budget (130,2  billion euro out of a possible 143,9  billion euro (6), or 90,5  % of the available total) (see Box 2.1 ).

2.5.

The low level of payments mainly resulted from authorities in the Member States submitting lower than anticipated claims for European Structural and Investment (ESI) funds. This was due to delays in implementing programmes in the first years of the current MFF. The Commission has given the following explanations (7) for these delays:

(a)

the consequence of significant delays in the implementation of the 2007-2013 programmes;

(b)

the EU’s late adoption of the legal bases for the 2014-2020 programmes (8);

(c)

delays in Member States setting up and notifying the EU of the national authorities responsible for managing EU funds;

(d)

the time needed to adjust to changes in the rules for programmes and projects for new period;

(e)

the granting of an extra year in the 2014-2020 period for Member States to make payments against commitments (the ‘n+3 rule’) (9).

2.6.

As the Commission noted (10), delays in implementing ESI Fund programmes during the first three years of the 2014-2020 MFF exceeded those experienced at the start of the 2007-2013 period on structural and cohesion fund programmes.

2.7.

In light of the low level of payment requests received during the year, the budgetary authorities amended the 2016 budget to decrease payment appropriations by 7,3  billion euro (11). Even with this reduction, the low number of payment requests led to the highest surplus since 2002: 6,4  billion euro (12) (2015: 1,3  billion euro).

2.8.

The 2014-2020 MFF has a number of instruments to facilitate the operation of the budget and hold funds in reserve. The most important of these are:

(a)

The Global Margin for Payments (GMP), which, as set out in Article 5 of the MFF Regulation, makes it possible to transfer unused payment appropriations to future years, provided the MFF ceiling is not exceeded. Thus, the unused 13,7  billion euro from 2016 referred to in paragraph 2.4 can be carried forward for use in future years. The limit, stated in 2011 prices, for the transfer of unused payment appropriations provided in the MFF for years 2018-2020, is 31 billion euro (2018: 7 billion euro; 2019: 11 billion euro; 2020: 13 billion euro).

(b)

Four ‘special instruments’ (13) to react to unforeseen events and changing priorities. According to the Commission’s mid-term review in 2016 (14), the total amount still available up to 2020 was 7,4  billion euro. According to a Council statement (15) issued at the time of the mid-term revision of the MFF, it remains open to the budgetary authority to decide on a case-by-case basis whether payments relating to special instruments can be counted above the payment ceiling.

(c)

contingency margin (16) of up to 0,03  % of GNI, (around 4,5 billion euro) making it possible, as a last resort, to transfer appropriations between years and headings in response to unforeseen circumstances, provided the MFF ceilings are not exceeded. This margin was used for payment appropriations in 2014, to be offset against 2017-2020. For commitment appropriations, the contingency margin was used to transfer 2,1  billion euro to MFF headings 3 and 4 from other headings in 2016 and 2017 in response to the refugee crisis (17).

(d)

The Global Margin for Commitments (GMC) (18), which makes it possible to transfer unused commitment appropriations to future years, but only to be used for growth and employment, in particular youth unemployment. Almost all of the GMC for 2014 and 2015 (1,8  billion euro out of 2,0  billion euro (19)) has been allocated to the EFSI.

2.9.

These instruments have been put in place to deal with emergency situations and provide some flexibility in the context of an EU budget system designed to provide predictable funding to long-term EU expenditure programmes. However as EU expenditure programmes were delayed in the first years of MFF 2014-2020 and are expected to pick up in the years up to 2020, there may not be sufficient flexibility remaining within the MFF ceilings to fund the EU’s response to any unexpected events that may still occur before the end of the current MFF.

2.9.

2.10.

In addition to making use of the margins described above, the Council may by unanimous decision amend the MFF regulation itself to change the ceilings for appropriations (20).

2.11.

In 2016, the EU committed 155,2  billion euro out of a possible 156,1  billion euro, 99,4  % of the total available (21) excluding assigned revenue.

2.12.

As a result of the almost full use of commitment appropriations and the lower than planned payments, outstanding commitments firstly fell in 2014 and then rose again in 2015 and 2016, reaching an all-time high of 238,8  billion euro (see Box 2.9 ). They are also set to rise further in 2017, as the 2017 budget provides for commitment appropriations of 157,9  billion euro and payment appropriations of 134,4  billion euro, implying an increase in outstanding commitments of up to 23,5  billion euro.

2.13.

This stock of outstanding commitments represents the cumulative difference between commitments and the payments made on them to date, less de-commitments. De-commitments represented a relatively small proportion of commitments made in the period 2007-2016 (27 billion euro out of a total of 1 446  billion euro or 1,9  %).

2.14.

The vast majority of outstanding commitments are for multiannual spending programmes, where the time lag between a commitment and its corresponding payment may be more than a year, and often up to three years. Based on 2016 figures, outstanding commitments at year-end were 72 % higher than in 2007 and were equivalent to 2,9  years (22) of payments, compared to 2,2  years in 2007. This rise in outstanding commitments risks creating funding problems in future (see paragraphs 2.32 to 2.39).

2.14.

2.15.

In addition to outstanding commitments, the EU had a number of significant long-term liabilities, guarantees and legal obligations still outstanding at the end of 2016.

2.16.

The EU’s single biggest long-term liability is the 67,2  billion euro for staff pensions (2015: 63,8  billion euro). Although EU employees contribute to their pensions, these contributions are treated as general revenue of the EU budget and the liability is not offset by any specific asset.

2.17.

In recent years, the EU has made increasing use of the budget to back guarantees for loans and financial instruments (see Box 2.2 ). These are contingent liabilities disclosed in the EU accounts (23). A margin must be maintained between the level of payment appropriations in the budget and the maximum level allowable under the ORD in order to cover these guarantees should the need arise.

(*)

The exposure at 31 December 2016 only includes disbursed amounts to final beneficiaries. A further 17,5 billion euro had been contracted but not yet disbursed at the end of 2016 (9,0 billion euro under the external lending mandate guarantee, 6,9 billion euro under the EFSI guarantee and 1,6 billion euro under financial assistance on MFA and Euratom loans).

(1)

See note 4.1.1 of the ‘Consolidated annual accounts of the EU — Financial year 2016’ (‘2016 EU accounts’).

(2)

See note 4.1.2 of the 2016 EU accounts.

(3)

See note 4.1.3 of the 2016 EU accounts.

2.18.

The EU budget has two funds to back the budgetary guarantees: the Guarantee Fund for external actions, with an asset value of 2,3  billion euro, and the EFSI Guarantee Fund, with an asset value of 1,0  billion euro at the end of 2016 (24) that will rise to 8 billion euro by 2022. It also maintains a provision against expected losses on guarantees, which stood at 0,9  billion euro at the end of 2016 (25) after provisions of 0,5  billion euro were added during the year, including for loans to Syria. For the guarantees relating to financial assistance to Member States, any liability arising can be covered by appropriations over and above the MFF ceiling (26) but must be paid through the EU budget.

2.19.

The Commission has proposed establishing a common provisioning fund to cover the financial liabilities arising from financial instruments, budgetary guarantees and financial assistance (27).

2.20.

There were also significant long-term legal obligations not covered by outstanding budgetary commitments, totalling 22 billion euro (28) at year-end (see Box 2.3 ), which will be paid out from future annual budgets.

2.21.

Financial instruments benefiting from EU budgetary support have been increasing (see Box 2.4 ) and are projected to continue to do so up to 2020. Careful management is necessary to ensure the effective, efficient and economical use of available funds, and that any changes are made only after a careful impact assessment.

2.21.

Whenever changes are foreseen, the ex ante assessment should include an analysis of lessons learnt in previous similar instruments.

For centrally managed instruments the relevant legal bases foresee mid-term reviews or evaluations.

For all such future mid-term reviews of financial instruments, the Commission will ensure that they cover the lessons learnt and the effect of any major socio-economic changes on the rationale of the instrument and the corresponding contribution from the EU budget.

2.22.

The EFSI will continue to grow and is becoming a major source of funding (29). By the end of 2016, the EIB Group had signed contracts worth 21,3  billion euro for the EFSI (30). The investment period of EFSI is not yet over, Box 2.5 shows the distribution of EFSI funding by Member State to the end of 2016.

2.22.

2.23.

The EFSI is a new centralised innovative instrument for Member States to support investment within the EU. Under the EFSI’s governance arrangements, implementation powers are delegated to the EIB, with more limited public scrutiny than for other instruments supported by the EU budget, as illustrated in Box 2.8 .

2.23.

2.24.

Given that ESI funds represent a significant proportion of some Member States’ general government expenditure, it is important to note that, in nine Member States, outstanding commitments on ESI funds represent more than 15 % of general government spending in 2016 (see Box 2.6 ).

2.25.

Given the sizeable commitment appropriations still available under the 2014-2020 MFF, Member States where ESI funds represent significant percentage of general government expenditure may find it challenging to identify sufficient high-quality projects on which to spend the available EU funds or to provide co-financing.

2.25.

2.26.

Receipts from the EU budget make up a significant proportion of some Member States’ capital investment in any given year. This means that the size and timing of such receipts can have significant macro-economic effects such as on investment, growth and jobs, which need to be sufficiently taken into account when planning future expenditure in the EU budget (see Box 2.7 ) (31).

(***)

Croatia joined the EU in 2013.

2.27.

A key financial management challenge in 2016 was mobilising resources to deal with the refugee crisis and the integration of migrants, including emergency support within the EU. The EU’s financial response has involved increasing funding in some existing areas, setting up new funding instruments and encouraging additional contributions from third parties (32).

2.28.

The overall amount of funds mobilised for the refugee and migration crisis was not reported by the Commission in 2016 and is difficult to estimate. As in many cases, these funds make up only part of the allocated budgetary appropriations in the policy areas concerned. Although the Commission reports on each individual instrument, in order to be able to report the overall figure it would need to establish a specific reporting structure.

2.28.

The Commission will analyse possible consolidation of its existing reporting in order to produce comprehensive information on refugee and migration expenditure.

2.29.

The EU budget has evolved greatly over the years, and a number of other mechanisms for funding EU policies have been added. This development has been driven by the need to find additional funds to respond to new challenges, such as the financial crisis, climate action, migration and refugees, and to use available funds more efficiently and innovatively, such as for loans, guarantees and equity investment.

2.29-2.31.

The current EU financial architecture has made it possible to mobilise funds for new priorities and to do more with less. The EU budget is not the only tool to finance the EU policies. However, the EU accounts include all instruments and consolidate all agencies with an impact on the EU budget. The EU accounts are audited by the ECA. Some mechanisms were created to respond to the Euro area crisis and are of an inter-governmental nature. For this reason, they are outside the framework of the EU budget. Other instruments such as EFSI and the European Investment Fund complement the more traditional delivery mechanisms of the EU budget, enhancing its outreach and leverage effect.

Concerning the new financial mechanisms, all bodies managing them provide yearly accountability reports and an opinion by an external auditor. Moreover, for reasons of accountability and transparency, specific reports such as on financial instruments, trust funds and the Facility for Refugees in Turkey are also provided.

The Commission adopted a reflection paper on the future of the EU finances, in which the current EU financial architecture is explained. This subject will be a key element of the next MFF.

2.30.

As a result, the number of entities and instruments involved in financing the implementation of EU policies has increased considerably, creating a complex web of arrangements around the EU budget (see Box 2.8 ).

2.31.

The EU budget is a single mechanism subject to specific rules. This helps to provide accountability and transparency to our stakeholders, including the European Parliament and the Council. The increasing use of other financial mechanisms to deliver EU policies alongside the EU budget risks undermining this level of accountability and transparency, as reporting, audit and public scrutiny arrangements are not aligned. For example we audit and report on only some of the financial mechanisms, which limits public scrutiny. In our view, the current arrangements make it difficult to manage, audit, scrutinise and report effectively and to provide an adequate overview of the cost and benefits of the EU for citizens.

(1)

Including legal commitments not yet covered by budgetary commitments appropriations: Connecting Europe Facility (CEF), Copernicus, ITER, Galileo, fishery agreements and others.

(2)

SME Initiative managed by the EIB on behalf of the Member States by shared management with the Commission; other financial instruments managed by the EIB on behalf of the Commission as an entrusted entity.

(3)

Governing/management board with members from each Member State as decision-making body.

(4)

Managed by the EIB on behalf of the Commission as an entrusted entity.

(5)

Funds and operations are implemented by the Member States.

(6)

Governing/Management board with members from Member States and other stakeholders as decision-making body. Not all Member States are represented in every Board.

(7)

Only the part of the Blending Instruments funded by the General Budget of the EU is discharged by the budgetary authorities.

(8)

Development Cooperation Instruments (DCI): Neighbourhood Investment Facility (NIF); Investment Facility for Central Asia (IFCA) and Asian Investment Facility (AIF); Latin America Investment Facility (LAIF).

(9)

Financed by commercial banks. Management Board consists of members from each Member State.

(10)

Only operations guaranteed by the general budget of the EU are audited by the ECA.

(11)

Budget implementation report sent to Council — Discharge by the agencies’ management boards. The Commission may participate on behalf of the EU in the agencies’ projects. The Member States of each agency provides funding. The High Representative of the Union for Foreign Affairs and Security Policies is responsible for the agencies’ overall organisation and functioning.

(12)

Intergovernmental entity.

(13)

The ECA’s audit powers are limited to examining the operational efficiency of the ECB’s management.

(14)

Only the EU mandate of the EIB is audited by the European Court of Auditors. EIF is a part of the EIB Group.

(15)

Shareholders are EU Member States, other countries, the EU and the EIB.

2.32.

There is a risk that a backlog of unpaid claims similar to the one in 2013-2015 may be created in the final years of the current MFF and the early years of the next. This is because the payment appropriations approved under the current MFF may not be adequate to pay all claims submitted for payment until 2020 (33).

2.32.

2.33.

The Commission concluded in its forecast to the end of 2020 in the mid-term review (34) that ‘the overall payments ceilings of the 2014-2020 MFF should be just sufficient, avoiding any new abnormal backlog of unpaid claims at the end of the period.’ However, this was conditional upon special instruments being counted over and above the MFF ceilings and the removal of the maximum amounts for the use of the global margin for payments in 2018-2020 set out in the MFF regulation. Following the revision of the MFF regulation (35), it is not certain whether special instruments will in fact be counted over and above the ceilings (see paragraph 2.8(b)), and maximum amounts on the global margin for payments remain (see paragraph 2.8(a)). Moreover, a higher than anticipated amount of payment claims may be submitted by Member States.

2.33.

2.34.

If existing trends continue (36), we project that outstanding commitments (see Box 2.9 ) will reach a record 262 billion euro by 2020, which is close to the Commission’s projection in the mid-term review (37).

2.35.

These outstanding commitments will need to be paid off using payment appropriations from the next MFF, mainly during the first three years of the next MFF.

2.36.

Clearing this increase in outstanding commitments, and thus addressing the additional exposure and the risk of backlogs being created, could be achieved by including sufficient additional payment appropriations over and above those required to cover new programmes of the next MFF and/or reducing commitment appropriations for new programmes under the next MFF to leave enough payment appropriations to cover these outstanding commitments.

2.37.

This would be preferable to using any delays in the implementation of programmes under the new MFF to generate enough payment appropriations to cover the outstanding commitments, as happened unintentionally in the early years of the current MFF, since this would simply postpone the problem again.

2.38.

Article 323 of the TFEU provides that the ‘… European Parliament, the Council and the Commission shall ensure that the financial means are made available to allow the Union to fulfil its legal obligations in respect of third parties’. The Own Resources Decision of the Council (38) requires that ‘an orderly ratio between appropriations for commitments and appropriations for payments shall be maintained to guarantee their compatibility …’.

2.39.

The Commission will have the opportunity to address this issue when presenting its proposals for the new MFF (39) on the basis of a forecast for spending needs in the period after 2020 (40), as recommended in our annual reports in recent years (41).

2.40.

The delays in the implementation of programmes in the first three years of the current MFF led to the transfer of commitment appropriations from 2014, mainly to 2015 and 2016, and low payments in 2016 (paragraphs 2.4 to 2.10).

2.41.

Special instruments and margins were used extensively, but the amounts left may not be sufficient to fund unexpected events that may still occur before 2020 (paragraphs 2.8 to 2.10).

2.41.

2.42.

A record level of outstanding commitments has been created, 72 % higher than in 2007. This has increased the amounts owed by the EU and thus the financial exposure of the EU budget (paragraphs 2.11 to 2.14). The overall financial exposure of the EU budget has grown, with significant long-term liabilities, guarantees and legal obligations (paragraphs 2.15 to 2.20) implying that careful management needs to be applied in the future.

2.42.

2.43.

The EU is making increasing use of financial instruments. The establishment of EFSI creates new governance arrangements with more limited public scrutiny than for other instruments supported by the EU budget (paragraphs 2.21 to 2.23).

2.43.

2.44.

EU funds form a significant share of some Member States’ expenditure, and the volume of funds and timing of their receipt can have a considerable macro-economic impact such as on investment, growth and jobs (paragraphs 2.24 to 2.26).

2.45.

The Commission mobilised various resources to deal with the refugee and migration crisis, but it did not establish a reporting structure to enable it to report comprehensively on the use of the funds involved (paragraphs 2.27 and 2.28).

2.45.

The Commission will analyse possible consolidation of its existing reporting in order to produce comprehensive information on refugee and migration expenditure.

2.46.

The mechanisms for funding EU policies have become increasingly complex, which risks weakening accountability and transparency (paragraphs 2.29 to 2.31).

2.46.

2.47.

A backlog of payments may develop towards the end of the current MFF and in the first few years of the next MFF, and financing the new MFF will require realistic budgetary appropriations to cover projected outstanding commitments (paragraphs 2.32 to 2.39).

2.47.

2.48.

We recommend that the Commission:

—

Recommendation 1: take into account the growth in outstanding commitments in its forecast of payment appropriations for the next MFF in order to help ensure an orderly balance between commitment and payment appropriations.

—

Recommendation 2: together with the European Parliament and the Council, ensure a consistent approach to the issue of whether or not special instruments are counted within the ceilings for payment appropriations in the MFF.

—

Recommendation 3: for management and reporting purposes, establish a way of recording EU budgetary expenditure that will make it possible to report on all funding related to the refugee and migration crisis.

—

Recommendation 4: invite the European Parliament and the Council, in the context of the debate on the future of Europe, to consider how the EU budgetary system could be reformed to provide a better balance between predictability and the responsiveness as well as how best to ensure overall funding arrangements are no more complex than necessary to meet EU policy objectives and guarantee accountability, transparency and auditability.

3.1.

Each year, in this chapter, we analyse a number of aspects relating to performance: the results achieved by the EU budget, which is implemented by the Commission in cooperation with the Member States (1). This year we have looked specifically at:

3.2.

Most large and complex organisations have procedures for performance measurement and reporting to provide an indication of the success of operations and highlight where improvements should be made. In paragraphs 3.3-3.12, we provide an overview of the performance reporting framework. In paragraphs 3.13-3.51, we compare it with standards and good practices applied elsewhere.

3.3.

The Commission has introduced several changes in this area in recent years — notably by heading up the ‘budget focused on results’ (BFOR) initiative (2). Performance reporting is at the heart of the initiative, which aims to find ways of improving the openness, timeliness and efficiency of communication about the results of the EU budget.

3.3.

3.4.

Box 3.1 gives an overview of the performance reporting framework. It reflects the distinction made by the Commission between its own performance and that of the EU budget. In this section we describe the main reports shown in the box.

3.4.

3.5.

The Financial Regulation requires the Commission to prepare ‘programme statements’ (3) to justify the funding requested for EU programmes in the draft budget. For each programme, the statements include an update on implementation, together with key financial and performance information (4), using indicators that will apply throughout the seven-year multiannual financial framework (see Box 3.2 ). This approach enables the budgetary authority to take performance into account during the budgetary procedure.

3.5.

3.6.

In some areas of spending, the TFEU, the Financial Regulation or sector-specific rules require the Commission to prepare sectoral reports. Since the aim, structure and content of these reports vary according to the different requirements, their focus on performance aspects also varies. Some sectoral reports provide wide-ranging, quantified performance information. The Commission monitors some sector-specific performance data online and frequently updates information.

3.7.

Commission directorates-general (DGs) have prepared multiannual (2016-2020) ‘strategic plans’, which set goals giving medium-term direction to operational activities. In ‘management plans’, DGs break down their multiannual strategy and objectives into annual outputs, actions and interventions. Each DG’s annual activity report (AAR) contains results by reference to the objectives set and an assessment of the extent to which operational expenditure has contributed to policy achievements (6). The AARs also include financial and management information, such as the results of checks on legality and regularity.

3.8.

In its Better Regulation Guidelines (BRGs) (7), the Commission defines an evaluation as an evidence-based judgment of the extent to which an intervention is:

3.9.

The BRGs commit the Commission to evaluating, in a proportionate way, all EU spending and non-spending activities intended to have a social or economic impact. Each DG is responsible for evaluating its own activities. Evaluations may be fully centralised in a horizontal unit, fully delegated to operational units or, in a hybrid model, decentralised for operational management purposes but supported by a central evaluation unit.

3.10.

Until the end of 2015, the Commission maintained an evaluations database (8) with records of nearly 1 600  evaluations carried out between 1999 and 2015, an average of 100 each year (see Box 3.3 ). The total cost of evaluations in 2016 was more than 60 million euro (210 million euro if studies (9) are included). The Commission has since discontinued the public evaluations database and introduced a ‘studies database’. According to the Commission, the main purpose of the latter is to allow evaluation findings to be shared between the different EU institutions, whereas the public platform for sharing all evaluations and study information is the EU Bookshop.

3.11.

The AMPR constitutes a response to the requirements in both Article 318 TFEU (for a results-based evaluation of the EU’s finances) and Article 66(9) of the Financial Regulation (for a summary of annual activity reports). By combining two previously separate reports, the Commission aimed to produce a single document for the discharge authority covering both the management of the EU budget and its evaluation of the results obtained.

3.11.

3.12.

There are other Commission reports, both regular and occasional, that include information on objectives and performance indicators, as well as project benefits or achievements.

(i)

The annual general report on the activities of the European Union (10), published in March, gives details of the previous year’s major events, initiatives and decisions grouped by the Commission’s ten priorities.

(ii)

The annual EU budget financial report (11), published in July as part of the Commission’s integrated financial reporting package, deals with budgetary management in the previous year.

(iii)

‘The Juncker Commission two years on’ (12) was a set of presentations published in November 2016 to provide an overview of the progress made on the Commission’s ten priorities in the two years since it took office.

(iv)

The President’s annual ‘State of the Union’ (13) speech to the European Parliament in September, for which supporting documents are published.

(v)

The ‘EU Results’ web tool (14), an online collection of EU-funded projects, each with a statement of results and, sometimes, numerical data.

3.13.

To identify good practices in performance reporting we:

(i)

consulted relevant international standards, such as IPSASB Recommended Practice Guideline 3 — ‘Reporting Service Performance Information’ (15);

(ii)

carried out a survey on performance reporting; in addition to a comprehensive reply from the Commission, we considered 17 replies from EU Member States, other national governments and other international organisations;

(iii)

reviewed publicly available performance information for a selection of 14 governments and international organisations (16);

(iv)

examined the assessment reports issued by the Multilateral Organisation Performance Assessment Network (MOPAN) (17) between 2011 and 2016; these always included elements of organisations’ performance reporting.

3.14.

While there are important differences between the EU and other international organisations and governments, the latter also have to comply with legal requirements and are subject to specific operating constraints. Other international organisations are no less exposed to complications in establishing a link between their actions and ultimate impact/performance. We identified six areas (B to G below) in which the good practices we observed could help the Commission to further develop its performance reporting.

3.14.

3.15.

In Australia, government departments produce three core documents as part of the performance framework: portfolio budget statements quantifying programme budgets, corporate plans setting out the purposes and activities of public entities, and annual reports with feedback on results. Entities must ensure coherence (a ‘clear read’) between these documents (18). The performance criteria in each document must be reported collectively in a single annual performance statement (placed in annual reports at the end of the financial year).

3.16.

In France, the state budget is subdivided into ‘missions’. Each of the 31 missions in the budget comprises several programmes (118 in 2017). Each programme has a strategy, objectives and quantified performance indicators, and the most important indicators are also identified as mission indicators. The use of identical indicators ensures coherence between missions and programmes. The indicators cover both programme performance and the performance of government departments.

3.17.

In Canada, all federal organisations receiving budget appropriations must plan and manage their operations and report their performance against a set of 16 government-wide outcome areas. This framework establishes a clear, straightforward relationship between financial allocations, programme activities and national outcomes.

3.18.

Before the start of the 2014-2020 MFF, the Commission instructed its services to ‘deepen the performance framework’ (19). To achieve this, it set as an internal rule (20) that the new programme objectives and indicators must feature clearly in programme statements, management plans and AARs. This was done for the first time in 2014, enabling the Commission to present a coherent performance narrative: targets for which money was allocated in the budget became part of the planning process in the relevant DG’s management plan, and the results were reported in its AAR.

3.18.

3.19.

Since 2016, however, the Commission has no longer applied this approach. Instead, it has created a new planning document, the five-year strategic plan, in which DGs are expected to set new organisational objectives and indicators that are aligned with the priorities of the Commission but are distinct from the objectives and indicators already existing for programmes. AARs now report on the performance framework set in the strategic and management plans and should refer to the programme statements for the results of programmes. However, out of the six 2016 AARs we reviewed, only one used cross-references to the performance reported in programme statements. This hampers comparability between the different types of performance documents and does not reflect the fact that the Commission is ultimately responsible for implementation of the budget with regard to the principles of sound financial management.

3.19.

For this reason, aligning objectives and indicators in the performance framework of the Commission services with those defined in the legal bases of programmes is no longer required. However, the Commission services are invited to make reference to programme statements when describing their activities in relation to the programmes. This helps to limit duplication of information and ensure consistency of data and will be reinforced in the future.

The Commission considers that its responsibility for implementing the budget is clear. Each Director-General signs in his/her Annual Activity Report a declaration of assurance stating that the resources have been used for the intended purpose and in accordance with the principle of sound financial management and that the underlying transactions are legal and regular. At corporate level, the College takes overall political responsibility for the management of the EU budget by adopting each year the Annual Management and Performance Report for the EU budget.

3.20.

The Commission uses a different reporting breakdown for spending and performance. AARs report on DGs’ annual payments by type of activity/spending programme, but on performance by the achievement of general and specific objectives with no indication of the corresponding expenditure. As a consequence, it is not possible to assess how much was spent on pursuing the set objectives. The AMPR similarly does not include information on expenditure relating to the performance of programmes broken down by general and specific objectives.

3.20.

3.21.

In 2016 the OECD carried out a performance budgeting survey in OECD countries and at the Commission. The chart in Box 3.4 compares the existence, use and consequences (21) of performance frameworks in the various jurisdictions. The OECD considered the Commission’s performance framework to be the most extensive, which may partly be explained by the volume of legal requirements in the EU. However, the OECD chart indicates that the use and consequences of the framework for decision-making do not reflect this higher level of specification.

3.21.

Performance information on the spending programmes is used by the Commission for (a) monitoring programme progress, (b) accountability and transparency, (c) informing decision-making about the management of a programme; (d) justifying or proposing adjustment of the allocation of funds to the budgetary authorities; (e) developing proposals for changes to the legal basis or for future programmes.

The Commission notes however that while performance information is taken into account during the annual budgetary procedure, it is not the only factor driving budgetary decisions.

3.22.

Some countries have progressively reduced the complexity of their performance framework and the number of objectives and indicators.

(i)

In August 2001 the French Parliament passed legislation (22) establishing new rules for preparing and implementing the state budget. Between 2014 and 2017, this involved the Ministry of Economy and Finance reducing the number of objectives and indicators by 20 % and 24 % respectively.

(ii)

In 2012 and 2013, the Netherlands introduced an ‘accountable budgeting’ reform which set stricter conditions for the government’s use of policy information (performance indicators and texts explaining policy objectives). The number of performance indicators in budget documents was halved.

3.23.

The programme statements for the EU’s 2017 draft general budget contain 294 objectives and 709 indicators (see Box 3.2 ). They are particularly highly concentrated under MFF headings 1a, 3 and 4. Through the BFOR initiative, the Commission is currently undertaking a review of its indicators to provide input for the next generation of spending programmes. Box 3.5 illustrates the progress the Commission could be aiming for, though account should also be taken of certain factors specific to the institution.

3.23.

The comparison made by the ECA does not take account of key contextual factors such as the number and nature of interventions, policy areas covered or the mode of implementation, nor does it measure the quality or relevance of the indicators.

The Commission also notes that the number of objectives and indicators used in the strategic plans for the Commission services has been reduced significantly with the recent reform. The number of general objectives has been reduced from 84 to 11, of impact indicators from 187 to 37, of specific objectives from 426 to 386, and result indicators from 969 to 825.

3.24.

Other governments and international organisations have taken steps to ensure balanced performance reporting as recommended by international standards (23).

(i)

In the USA, annual performance reports should provide a clear picture of performance results, show trends over time, link to the source of the performance information and discuss progress and shortfalls.

(ii)

The WHO ‘end of biennium assessment’ for 2014-2015 included an ‘Overview of major achievements and challenges’ for each programme area (24). In the foreword to the WHO’s main performance report for 2014-2015 (25), the Director-General remarked that ‘Failures as well as successes are frankly presented’. In line with this, the text systematically described the challenges together with the results.

(iii)

The World Bank assessed (26) the adequacy of its monitoring and evaluation to provide input into the ongoing efforts to enhance its effectiveness and promote a culture for developing solutions and evidence-based decision-making. It identified significant shortcomings in the design and use of its systems.

(iv)

MOPAN’s 2015-2016 assessment of the Inter-American Development Bank (27) noted a new section in the bank’s Development Effectiveness Overview called ‘learning from failure’.

3.25.

While the 2016 AAR instructions (28) do not explicitly mention the principle of balanced reporting, they do state that ‘the executive summary brings together, in a realistic perspective, the main messages as far as policy achievements are concerned and the conclusions on financial management and internal control’.

3.25.

3.26.

Notwithstanding the above, the AARs we reviewed contained limited information on the performance shortfalls and challenges relating to the DGs’ objectives. It is also difficult to identify these shortfalls and challenges because no space is specifically set aside for them in the AARs.

3.26.

3.27.

With the 2015 AMPR, the Commission set out to provide a comprehensive overview of performance based on an EU budget focused on results. However, the AMPR did not provide comprehensive coverage of performance and was overly positive, the only shortfalls to which it refers being implementation delays. The report also:

3.27.

The 2015 Annual Management and Performance Report refers to a number of shortfalls and problems to be addressed. Section 1 of the report refers not only to challenges affecting the timely implementation of programmes but also to limitations in the assessment of programme performance (e.g. references to the evaluations on the 2007-2013 Health programme and programmes in budget heading 4 pointing to constraints in assessing the overall performance of these programmes).

—

provided limited insight into the results of the Europe 2020 strategy, whereas this was requested by the European Parliament in its 2014 discharge decision,

—

The Annual Management and Performance Report is not a specific report on Europe 2020 (30). Nevertheless, the report includes a summary account of progress on Europe 2020 headline targets and relevant indicators and covers both the state of implementation of the 2014-2020 programmes and the results from the 2007-2013 programmes structured around the Europe 2020 priorities of smart, sustainable and inclusive growth.

The Commission monitors and reports on the Europe 2020 strategy through the European Semester process. Detailed and comprehensive reporting on Member State performance towards their national Europe 2020 strategy targets is included in the Country Reports as part of the European Semester, as well as in the yearly updates on the Europe 2020 indicators published by Eurostat. The programme statements also provide a link between expenditure and the Europe 2020 strategy. They show the estimated contribution of programmes to the headline targets (theoretical link) and to the Europe 2020 priorities.

—

did not always clearly explain the influence of external factors on results,

—

The 2015 Annual Management and Performance Report describes the operating context by referring in the introduction to the major political and economic challenges of the year. It clearly states that the EU budget is only one instrument, used in conjunction with others (i.e. national budgets and other policy and regulatory tools at EU and Member State levels) to deliver policy results and that 80 % of the EU budget is spent with the Member States. The report concludes under its section 1 on performance and results that ‘…There are limits to the degree to which indicators can capture a precise picture of performance…In addition, contextual factors have an important influence on final results, and indicators cannot eliminate or adjust for these factors’ and that ‘performance is a shared responsibility: responsibility for delivering on the Europe 2020 strategy as well as on the objectives of the financial frameworks is shared to a large extent with Member States’.

—

was published too late to be reviewed in our annual report (31).

—

The deadline for adopting the report under the Financial Regulation is 15 June. The Commission relies on financial data from Member States in relation to shared management programmes which only becomes available in the course of March. It is therefore not possible to adopt the report significantly earlier.

3.28.

In the Netherlands, the measurement and reporting of indicators is subject to audit. Annual ministerial reports contain a mandatory feedback section on the reliability of performance information.

3.29.

In the US, all government agencies publish a ‘Fiscal Year Annual Performance Report and Fiscal Year n+2 Annual Performance Plan’. This document describes how the agencies will ensure the accuracy and reliability of the data used to measure progress towards their performance goals.

3.30.

In Australia, the 2017 Report on Government Services supports the use of imperfect data for the sake of more comprehensive reporting (32). To make up for this deficiency, every indicator in the report contains an assessment of the comparability and completeness of the underlying figures.

3.31.

Another aspect of data quality is timeliness. Recent MOPAN assessment reports have described interesting trends in real-time reporting at the Vaccine Alliance, UNAIDS and the World Bank. Our survey (33) also showed that close to half of the respondents report on performance more frequently than once a year (see Box 3.6 ).

3.32.

Currently, the Commission’s performance documents do not systematically include an assessment on the quality of the data used. Deficiencies in data quality may be revealed, but there is no obligation to do so. Performance reports identify data sources, but they do not have a specific section disclosing matters of data accuracy and reliability. According to the Better Regulation toolbox (34), external evaluators can be asked to comment on the quality of data and provide recommendations on how to obtain better data, but this is not a rule.

3.32.

As to evaluations, the Better Regulation Toolbox stipulates that evaluation Staff Working Documents prepared by the Commission services at the end of an evaluation must contain a clear summary of any insufficiencies in the data used to support the conclusions and the robustness of the results. Moreover, the Better Regulation Guidelines stipulate that any limitations to the evidence used and the methodology applied, particularly in terms of their ability to support the conclusions, must be clearly explained in the evaluation reports. This is further strengthened in the Better Regulation Guidelines/Toolbox revision (SWD(2017) 350 final of 7 July 2017).

3.33.

Good practices in Australia, in France and at the World Bank are illustrated below.

Box 3.7 —

3.34.

The Commission uses quantitative and qualitative data to present performance information through a combination of graphics and narrative descriptions. Box 3.10 shows that — for a selection of reports — although charts, graphs and tables are common, textual information predominates. The Better Regulation toolbox has a section on the use of visual aids, but this is limited to only three types (problem trees, objective trees and intervention logic diagrams). Other, more general tools for enhancing visual presentation — such as colour-coding or infographics — are not discussed.

3.34.

The Better Regulation Toolbox includes a dedicated tool on visual aids. This tool aims at presenting some examples of instruments that are considered particularly relevant for policy interventions as a way to raise awareness among Commission services of the added value of such aids. It is by no means meant to bring together all existing visual aids.

3.35.

Navigation aids — for example glossaries, lists of abbreviations and alphabetical indexes — are almost never used. The Commission has not adopted the practice of producing interactive online versions of key performance reports.

3.36.

The UNESCO evaluations we reviewed put forward recommendations together with ‘strategic options for consideration’. All recommendations follow the same pattern: they define a goal and the means for achieving it, followed by a list of actionable suggestions for implementation.

3.37.

A recent WHO evaluation (37) assesses the reach, usefulness and use of a sample of approximately 15 000  publications over 10 years. The evaluation makes six strategic recommendations and presents specific actions side-by-side with the text of each recommendation to explain what they entail and promote follow-up.

3.38.

The BRGs do not require evaluators to make recommendations for consideration by the Commission; we consider that this should be the basis for a well-functioning follow-up system including action plans addressing weaknesses. The practice on making recommendations varied among the twelve evaluations we reviewed:

3.38.

3.39.

The Commission has not carried out a study on its use of evaluation results, or had one made, since 2005.

3.39.

3.40.

We found several good practices in the follow-up of evaluations and how it is reported: