This document is an excerpt from the EUR-Lex website
Document 52020DC0810
REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL evaluating the main findings of the pilot studies established under Regulation (EU) 2016/792 on harmonised indices of consumer prices and the house price index
REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL evaluating the main findings of the pilot studies established under Regulation (EU) 2016/792 on harmonised indices of consumer prices and the house price index
REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL evaluating the main findings of the pilot studies established under Regulation (EU) 2016/792 on harmonised indices of consumer prices and the house price index
COM/2020/810 final
EUROPEAN COMMISSION
Brussels, 14.12.2020
COM(2020) 810 final
REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL
evaluating the main findings of the pilot studies established under Regulation (EU) 2016/792 on harmonised indices of consumer prices and the house price index
1.Background
Regulation (EU) 2016/792 of the European Parliament and of the Council on harmonised indices of consumer prices and the house price index was adopted in May 2016 1 . This Regulation lays down a common framework for the development, production and dissemination of the Harmonised Index of Consumer Prices (HICP), the Harmonised Index of Consumer Prices at Constant Tax rates (HICP-CT), the Owner-Occupied Housing Price Index (OOHPI) and the House Price Index (HPI). These four indices together are referred to as the ‘harmonised indices’.
Article 8 of Regulation (EU) 2016/792 makes it possible for the Commission to launch pilot studies, to be carried out on a voluntary basis by Member States, whenever improved basic information is required for the compilation of the harmonised indices, or when the need for improved comparability of the harmonised indices is identified.
The Union’s general budget shall contribute to the financing of such pilot studies, where appropriate.
These studies shall assess the feasibility of obtaining improved basic information or adopting new methodological approaches.
The results of the pilot studies shall be evaluated by the Commission (Eurostat) in close cooperation with Member States and the main users of the harmonised indices, taking into account the benefits of having improved basic information or new methodological approaches relative to the additional costs of production of harmonised indices.
The Commission shall submit a report to the European Parliament and the Council evaluating, if applicable, the main findings of the pilot studies by 31 December 2020 and every 5 years thereafter. This report fulfils this obligation.
2.Pilot studies conducted under Regulation (EU) 2016/792
Since the adoption of Regulation (EU) 2016/792, two rounds of pilot studies have been completed, supported by grants from the Union’s general budget for 2016 and 2017. Further rounds have been launched in subsequent years, but the studies funded under these rounds are ongoing and cannot yet be evaluated.
2.1. Pilot studies based on the 2016 budget
A call for proposals 2 was launched in 2016, covering the following objectives:
Objective A: Automatic linking of GTIN 3 codes/shop-specific codes to ECOICOP 4
Objective B: Implementing scanner data from supermarkets or other retailers
Objective C: Development of new methodologies for processing scanner data or other transaction data
Objective D: Web scraping as a source for the HICP
Objective E: Improvement of the compilation of expenditure weights for the HICP, including weights for internet purchases
The first three objectives aimed to study various technical aspects related to the use of improved basic information derived from scanner data from supermarkets or other retailers. Another new source for improved basic information is data collected automatically from the internet (‘web scraping’), which is covered by the fourth objective. These objectives also support methodological studies that improve the comparability of the harmonised indices regarding the processing of these new data sources. The fifth objective aims for better comparability between countries and access to improved basic information on the compilation of expenditure weights, specifically given the rising share of purchases on the internet in many Member States.
Grants were awarded to 15 pilot studies in 10 Member States and one EFTA country. Of the 15 studies, eight related to scanner data (objectives A, B and C), six to web scraping (objective D) and one to expenditure weights (objective E). The total budget committed for these projects was EUR 1 180 835.04.
The 2016 call for proposals also included objectives related to real estate statistics. However, these studies covered subjects that were outside the scope of Regulation (EU) 2016/792 (quarterly house sales indicators, regional house price indices and commercial real estate indicators). The projects funded under these objectives are therefore not regarded as pilot studies in the sense of article 8 of Regulation (EU) 2016/792.
2.2. Pilot studies based on the 2017 budget
A new call for proposals 5 was launched in 2017, covering the following objectives:
Objective 1: Implementing transactions data in HICP compilation processes
Objective 2: Adapting the sampling and compilation methodologies of the HICP to the challenges of the digital economy
Objective 3: Improvement of quality adjustment methods for the HICP
The first objective invited further pilot studies on the use of scanner data from supermarkets or other outlets, to improve the basic information and comparability of the methods underlying the HICP 6 . Objectives 2 and 3 aimed to improve comparability of various aspects of the HICP methodology, particularly sampling and quality adjustment. The increasing digitalisation of society and of the economy brings challenges for price measurement, which could be studied under these objectives.
Grants were awarded to 15 pilot studies in nine Member States and one EFTA country. Of the 15 studies, five related to scanner data (objective 1), five to the digital economy (objective 2) and five to quality adjustment (objective 3). The total budget committed for these projects was EUR 1 081 642.46.
Like the 2016 call for proposals, the 2017 call also included objectives related to real estate statistics covering subjects outside the scope of Regulation (EU) 2016/792 (quarterly house sales indicators, regional house price indices and commercial real estate indicators). The projects funded under these objectives are therefore also not regarded as pilot studies in the sense of article 8 of Regulation (EU) 2016/792.
3.Evaluation of the pilot studies
While the Commission is solely responsible for the evaluation and approval of the final reports of the pilot studies, the results are shared with the national statistical institutes of other Member States and EFTA countries and the European Central Bank, which is the main user of the HICP. The pilot studies are presented at workshops, both while in progress and when completed. The final reports are shared among the above institutions through an on-line application (CIRCABC, Communication and Information Resource Centre for Administrations, Businesses and Citizens).
This sharing of information allows all national statistical institutes to take note of and learn from the experiences of others. The findings of the pilot studies are also discussed among peers, which helps them to identify best practices. This fosters common understanding and common approaches, which is vital to improving the comparability of the harmonised indices on a continuous basis.
The pilot studies launched in 2016 and 2017 have led to a larger number of countries using scanner data and/or employing web scraping as data sources for the HICP. The number of product groups that are covered by these new data sources is also steadily increasing.
Currently, 10 Member States and all three EFTA countries use scanner data to some extent as a data source for their HICP. In addition, a survey carried out in early 2020 among national statistical institutes revealed that 15 Member States and two EFTA countries currently apply web scraping, to varying degrees, in the production of their HICP.
The major advantages of the new data sources are the increases in coverage and representativeness of samples that can be achieved, compared to traditional ‘manual’ price collection in shops, leading to more reliable measurement of price changes. At the same time, the new data sources are generally less costly than traditional methods of price collection.
The increase in the use of new data sources is not only due to the EU-funded pilot studies, as many countries have undertaken the necessary testing and development work from their own budgets. Nevertheless, the pilot studies made a significant contribution to progress in the use of new data sources for the HICP.
It is less straightforward to assess the contribution of the projects on improved comparability (objective E in 2016 and objectives 2 and 3 in 2017). However, the results of most projects (e.g. improved sampling and quality adjustment procedures) have been, or will be, implemented in the HICP and will thereby improve the quality of the HICP in the respective countries. The projects have also contributed to the common pool of knowledge and experience on sampling and quality adjustment procedures, so they generally promote harmonisation. For example, the findings will be used in developing improved guidance and recommendations on these procedures, including updates to the HICP Methodological Manual 7 and, ultimately, could lead to implementing acts in line with Article 4(4) of Regulation (EU) 2016/792.
4.Conclusions
The evaluation of the findings of the pilot studies funded by the Union’s budget in 2016 and 2017, as presented in this report, shows that they have contributed significantly to the development and use of new data sources in the HICP. In addition, several Member States and EFTA countries have been able to improve their methodological approaches, thereby contributing to the store of common knowledge on these issues. In particular, the use of new data sources will improve the quality of the HICP while reducing their production costs.
The Commission considers the pilot studies to be an important instrument in the continuous improvement of the harmonised indices and therefore intends to continue their funding.
OJ L 135, 24.5.2016, p. 11.
Restricted to national statistical authorities of the Member States and European Free Trade Association (EFTA) countries.
Global Trade Item Number.
European Classification of Individual Consumption according to Purpose.
Also restricted to national statistical authorities of the Member States and EFTA countries.
As the HICP at constant tax rates (HICP-CT) is derived from the HICP, any improvement in the HICP will apply equally to the HICP-CT. In the remainder of this report, ‘HICP’ also encompasses HICP-CT.