4.1.Objectives

The challenge for addressing the problems outlined in section 2 of the report is to create a legislative environment for the European Social Surveys that will guarantee the quality of the data in the long run. The system of European social surveys must be sufficiently robust and maintain a high quality in an environment characterised by rapid innovation of methodology and IT, the availability of new data sources, changing needs and expectations of data users and continued pressure on available resources.

Objectives link the analysis of the problem to the options for the policy response. General objectives have been defined to target the identified problems as presented in the figure below.

.

In an impact assessment context, it is crucial that specific objectives are broad enough to allow consideration of all relevant policy alternatives without prejudging a particular solution.

The figure above lists the specific objectives, based on the identified problems in chapter 2. The specific objectives aim to address the current stovepipe nature of social statistics (first problem driver) by enabling the integration of the different domains so as to improve responsiveness to new multidimensional data requirements, while reducing and limiting costs. They also aim to tackle the legal inflexibility of data collections (second problem driver) by considering a common framework for the different surveys that would allow improving data quality, facilitating the use of new methods and sources, as well as flexibility concerning the adoption of the technical aspects of the surveys.

The selection of policy options that will be defined in the next chapters determines the shape of the operational objectives. These are analysed in chapters 5-7 and are therefore not yet included. All these policy options will be assessed on their contribution to these specific objectives

4.2.Coherence of objectives with other EU policies

On a general level, the initiative is in line with EU policy:

•The political priorities of the Commission, in particular the priorities dealing with job and growth, the digital market, the fairer dimension of the EMU, the migration, for which good analytical and monitoring tools are necessary for policy formulation and evaluation, encompassing at the same time several domains of social and economic statistics, creating the need for more coherence between data sources.

•The Europe 2020 Strategy calls for indicators in order to monitor headline targets, such as promoting employment, improving education levels and promoting social inclusion through the reduction of poverty. The calculation of these indicators requires timely statistical input that should be produced as efficiently as possible in the context of modern statistical data collection and production. The current lack of cross-domain integration hampers the common analysis of data from existing data collections.

•The Commission’s initiative on better regulation emphasises that the evaluation of EU policies will lead to an increasing need for targeted data (Commission Communication ‘Strengthening the foundations of Smart Regulation – improving evaluation’ COM(2013) 686).

•The initiative is part of the REFIT programme 12  which aims to ensure that EU legislation remains fit for purpose and delivers the results intended by EU law makers. In the context of REFIT, the initiative aims to minimise the burden of data collection and data production while ensuring that current and future needs for European statistics are met.

•

•Current EU legislation on statistics relating to persons and households has been regularly reviewed over the past years, the relevant evaluation reports are available from: http://ec.europa.eu/eurostat/web/quality/domain-specific-results (under "population and social conditions").

The policy objectives described above provide the framework for solving the problems defined in this impact assessment, and are also coherently shared with a number of policy documents in the field of social statistics. Below, there are a number of reference documents, each of which defines a coherent set of principles or priority areas to specifically guide Eurostat and the Member States in the modernisation of European social statistics:

•The Communication from the Commission to the European Parliament and the Council on the Production Method of EU Statistics: a Vision for the Next Decade (COM (2009) 404) 13 states that a new generation of statistical legal acts would deal with broader areas of statistics and emphasises the increasing use of multiple data sources and innovative data collection methods, as well as the growing importance of cross-domain integration and concepts.

•The Joint ESS Strategy for the implementation of the Commission Communication on the production method for EU statistics adopted by the ESSC on 20th May 2010 14 ;

•The Wiesbaden Memorandum 15 adopted by the European Statistical System Committee (ESSC) on the 28th of September 2011. A new conceptual design for household and social statistics demands that the core social surveys providing data (including microdata) on persons and households should be streamlined, that additional and less frequent micro-data collections should complement the core social surveys and that better access to administrative data, the re-use of existing and access to new data sources, should be developed at national and EU level.

•The paper on The Modernisation of Social statistics 16 is intended as a comprehensive strategy document and roadmap for the modernisation of social statistics during the period 2012-2017.

•The European Statistical Programme 2013-2017 (Regulation (EU) No 99/2013) 17 identifies statistics on People’s Europe’ (i.e. social statistics) is one of the three pillars of the statistical information system. Each of the pillars covers a set of primary statistics that will provide the input for policy indicators and accounting systems. In order to reach the objectives of the European Statistical Programme 2013-2017 in the most efficient and consistent manner, Eurostat has launched the project aimed to establish a cross-cutting legal framework for the production of European statistics relating to persons and households and based on data at each individual level.

5.Policy options: what are the various options to achieve the objectives

5.1.Background

In response to the problems defined and objectives set, two strands of policy options are proposed. Each of these strands introduces a variety of policy options that develop solutions for the problem drivers defined in Chapter 2 in different ways. The following strands for policy options are defined:

•    Addressing the domain fragmentation of social statistics

•    Addressing the flexibility of data collection on persons and households

These strands will be described in the next paragraphs. Subsequently, for each of the policy options identified it will be explicitly described how the option corresponds to the core problems, and how these impact the key stakeholders in chapter 6. Chapter 7 will recombine the most appropriate options so as to shortlist the most appropriate solutions.

5.2.Addressing the domain fragmentation of social statistics

Under this strand, four policy options have been considered, each of which addresses domain fragmentation to a different extent. The two key elements targeted by these policy options are (1) the legal context in the EU, which currently consists of various domain-specific regulations and agreements, and (2) the ‘stovepipe’ design of social statistics production. While these two elements are related, these are not the same, as also confirmed by a majority of NSIs. For this reason, these cannot be addressed by unidimensional policy options. Integrated EU legislative instruments for the various domains do not necessarily have to remove the ‘stovepipe’ model of data production to address the main problems identified in chapter 3. To save the considerable costs of changing national data production processes, an integrated legislative instrument could allow MS to produce data ‘in parallel’ and still improve the responsiveness, quality, possibility for innovation and costs of social data collection. At the same time, reducing the fragmentation of production processes could also take place without integrating legislation. In fact, progress in standardising statistical concepts across data collection has been made even under the current domain-specific legal set-up (e.g. definition of few “core social variables” to be included in different social surveys), however with mixed success so far. Data users also indicated in the consultation that “the final outcome of revising the ESS should not become the legal act per se, but it should create a path and ease the way towards improved European social statistics”. Given the importance of this distinction for key stakeholders, specific policy options have been defined that take this view into account. The table below presents the main policy options.

Comparing the options in more detail:

5.3.Addressing rigidity of data collection

The second strand of policy options included in the impact assessment focuses on the flexibility of data collection. A relevant distinction under this heading is the legal provisions on the programmatic elements (“what type of main data should be collected, at which frequency, about whom and with which level of accuracy”) and on various technical elements (“how should data be specified and transmitted"). The policy options considered in the impact assessment each add a certain element of flexibility to the existing legal provisions. A distinction between flexibility on substantive and technical elements is particularly relevant, as this determines which actors makes the changes. As such, these contribute to different types of impacts. The table below presents the main policy options.

Comparing the options in more detail:

6.Analysis of impacts: What are the policy impacts of the different policy options and who will be affected

6.1.Introduction

A detailed impact assessment has been conducted for each of the policy options presented in the previous chapter. First, the impact of each of the policy options will be assessed towards achieving the overall objectives. The following key impacts have been defined based on the objectives of the proposal.

•Better responsiveness to societal needs

•Ability to ensure quality of social statistics

•Promoting the use of innovative methods and procedures

In addition, the impact of each policy option on the costs of the production of social statistics has been assessed. This assessment allows conclusions to be drawn regarding the extent of how the policy options cost-efficiently contribute to the objectives. Finally, the impacts of the policy on specific stakeholders have been assessed, for which we have identified the following types of stakeholders:

•Users of social statistics

•Producers of social statistics

•Respondents involved in social statistics data collection (individuals, households).

In addition to the direct impacts of the policy options on stakeholders in this chapter, the next chapter will analyse the more indirect impacts associated with the shortlisted policy options.

6.2.Assessing impacts: addressing domain fragmentation

This section explores the impacts of the policy options that address domain fragmentation.

6.2.1.Baseline (option 1.0): no additional action

Under the baseline scenario, domain-specific legal instruments are in place at the EU level, while Member States and Eurostat follow different guidelines, procedures and methods for the production of the separate data collections. As a result, Member States continue to collect and process statistical data for each of the domains independently and in parallel (stovepipe design). As being a status quo, and having limited potential for addressing the key drivers, this option is rapidly examined.

Effectiveness

The continuation of the status quo limits the responsiveness of data collection to societal needs. As a result, the system of social statistics would not be able to respond to a growing demand of such issues.

Eurostat also faces limitations in ensuring the quality of statistical data. Many changes to improve the quality require the full legislative procedures for each domain-specific Regulation. Most NSIs also consider that a continued fragmentation of data collection / handling processes contribute to the high risk of divergence in methods, and consequently the increase of incoherence across data collections and between countries.

The same limitations are also relevant to the development of innovative tools. The domain-specific Regulations, further strengthened by fragmented production processes, do not facilitate the development of combined data collection methods, or the systematic use of alternative sources such as administrative registers.

Efficiency / impacts on stakeholders

The combination of domain-specific Regulations and fragmented production processes contributes to increasing inefficiencies in the data collection of social statistics.

The status quo is "counter-innovative", as changes are lengthy and would keep the various inefficiencies in data collection in place.

There would be no additional costs for NSIs in the status quo, but the possibilities to affront inefficiencies would be limited

6.2.2.Policy option 1.1: No change EU legislation, ‘defragment’ production processes across data collections

The first policy alternative addresses the fragmented data collection on persons and households by Member States. In addition to the already existing efforts (see baseline scenario), this policy option would introduce additional (possibly legal) measures to reduce the fragmentation (“defragment”) of the national production processes of social statistics. It would do so by pursuing increased standardisation of the various production processes, definitions, and variables. This would help in reducing the overlaps in data. All of this is done however, without changing the current legal structure at EU level, with domain-specific Regulations.

Whilst the analysis of this policy option does not actually address the policy objectives, and even could worsen the situation, it is an option that some stakeholders could consider valid, from a user perspective. Data producers may also see it as an acceptable solution. However, it is likely to maintain the status quo. It is therefore useful to analyse rapidly this option and not discard it upfront.

Effectiveness

Data users in the consultation phase indicated that possible changes due to defragmentation of national data production processes would in theory allow Member States to better respond to user demands for cross-domain issues, such as well-being (a complex combination of living conditions, income, labour status, health, etc.).

However, due to the need to amend the domain specific regulations and the voluntary agreements, this option has considerable limits in responsiveness to societal needs at the EU level. This policy option therefore, negatively impacts the responsiveness of social statistics to societal needs.

Because of the persisting legal fragmentation, any introduction of uniform standards for social statistics would require Eurostat to amend each of the domain-specific Regulations at the same time. It would involve costs and an important risk of not achieving the objectives, as divergences might remain in the course of the technical and legal developments. This policy option does not solve the problem of quality control on cross-domain data collection, which is likely to become even more relevant in the future (see problem definition). This risk is also underlined by data users, who fear that the lack of integrated EU legislation on social data collection could lead to lower quality and reduced interest for some specific sub-domains (e.g. health).

Policy option 1.1 opens up possibilities for Member States to use and combine new / innovative methods for data collection or production.

Efficiency / impacts for stakeholders    

The measures proposed under policy option 1.1 have a positive impact on the long-term development of production costs of social statistics. Over the short-term however, NSIs indicated that the actions to reduce fragmentation, such as increasing standardisation of processes, variables, or definitions do not reduce costs, but instead create significantly higher costs for their institutions. In particular, the implementation of new processes and methods, or the possible changes in definitions, variables or guidelines are considered costly. For instance, the assessment of the quality of administrative registers before their use for the production of statistics may represent a high initial cost. NSIs reported that such implementation costs would be considerably higher in the short-term than the potential gains.

At the same time, the lack of integration of the domain-specific regulations at EU level poses some other efficiency issues. This Member States' led integration will require substantial monitoring activity at Eurostat level, as risks of divergences between Member States will by high. This creates considerable costs for the European political and legislative system and monitoring and compliance assessment costs.

Finally, the risk of divergences between separated EU legal acts is high. This option could end up in a situation that is not very different from the status quo. In addition, its acceptability by Member States will be difficult.

6.2.3.Policy option 1.2: Integrate EU legislation, no change in the fragmentation of production processes

The second policy alternative is the symmetric of policy option 1.1. It includes the legislative package to create an integrated legislative framework for all data collections on persons and households in the ESS. As such, it would replace the separate domain-specific Regulations currently in place and streamline decision-making at EU level to respond to new data challenges. At the same time, this policy alternative does not include additional efforts to defragment the national production environments for social statistics through common principles. Consequently, the production processes at Member State level may continue to be organised by different domains.

Effectiveness

An integrated legal framework for social statistics would facilitate a better responsiveness of social statistics to societal needs. Changing needs for social statistics or additional cross-domain modules in existing data collections (consider for instance including elements on disability statistics, migration statistics, vulnerable groups) can be added relatively easily by amending one overarching legislative instrument, thus increasing responsiveness to new data needs. When doing so, it is crucial that possible legal reforms ensure the continuity of time series.

Legal integration would improve the position of Eurostat in the management and governance of the social statistical system. It allows Eurostat to respond to new challenges with uniform (legal) measures, which would implement or repeal legal provisions for the entire social domain at once, instead of for each Regulation separately. This puts Eurostat in a better position to ensure the quality of collected data in terms of comparability and coherence. While changes to the legal structure can be more efficiently implemented at the European level, a sufficient number of checks-and-balances would need to be in place to sufficiently address NSI’s and users’ concerns, in relation to continuity of data provision. For stakeholders a continued focus on outputs remains of the highest importance. Due to the persistence of domain-specific production processes, these checks and balances also need to follow this structure, thus undoing some of the efficiency benefits of legal integration.

The integration of the various domain-specific Regulations impacts the possibilities to introduce innovative methods into law, and as such, also has indirect impacts on the efficiency of data collection. Again, the persistence of fragmented processes and data collections at the level of some Member States could undo much of the possible positive impacts.

Efficiency/impact on stakeholders

This policy option streamlines the legislative structure of European social statistics and thereby, contributes to a reduction of costs to change existing programming (necessary changes can be implemented more cost-efficiently at the EU level; instead of changing five domain-specific Regulations and opening up two informal agreements, one amendment procedure would suffice).

Despite this increasing potential for efficiency, the continued fragmentation at the national level also means a continuation of the existing stovepipe model in the Member States. This could explain why NSIs reported that an integrated legislation would decrease the cost of data collection for 11/28 countries, while 8 Member States expect increases in costs. As a result, this policy option would bring efficiency gains for the European legislative process, but not considerably improve the efficiency of data production processes by NSIs.

The resulting impacts for data users are twofold. This policy option limits the possibilities to meet the increasing user demand for cross-domain data collection on social issues at the EU level, due to persisting domain-specific data production. At the same time, those demands that fall within specific domains can be met more efficiently, as these can be introduced into integrated legislative packages that extend to the entire social statistics domain at once.

In theory, this policy option leaves overlaps and unnecessary redundancies in data collection in place, which would keep higher costs for data producers. However, in the consultation, most NSIs indicated a preference for limited implementation costs of this policy option, even in light of continued inefficiencies in the long run. An integration of the legislative system, without changes in the requirements on national data collections, various production processes or quality criteria is the preferred option for a majority of NSIs.

6.2.4.Policy option 1.3a: Integrate existing Regulations, ‘defragment’ production processes

The third policy alternative combines the elements of the first and the second options. It integrates the existing statistical EU legislation for social statistics into a common coherent framework, by repealing the existing domain-specific Regulations and replacing these with an integrated legislative package. In addition, it introduces the measures to reduce the fragmentation of the production processes at the Member State level, for data collections that are currently regulated by law. For data collections that are currently not regulated by law, Eurostat would encourage Member States to integrate their data collections. Legal and soft measures facilitating the removal of the ‘stovepipes’ in the business processes of national statistical institutes in combination with coherent cross-domain statistical legislation for social statistics offers the most comprehensive consolidation and integration of the production of social data in the European statistical system.

Effectiveness

The combination of integration of EU legislation on social statistics with measures to allow a reduction of the fragmentation of data collection and production processes contributes to a higher responsiveness to societal needs. This responds well to the flexibility needs defined by the policy users in the consultation.

The effectiveness to ensure the quality of data collection also increases under policy option 1.3a. An integrated legislative instrument helps Eurostat in promoting the consistency of statistical output across different domains and data collections, like the quality reporting framework or the need for good sample frames. It also helps in dealing effectively with possible newly arising quality concerns that could come from innovative approaches, like the use of new data sources. At the same time, the reduction of fragmentation of data production processes offers more options to ensure the quality of data as well.

A reduction of fragmentation in data production processes creates incentives for Member States to use and combine alternative data sources and methods across various domains. This is further helped by integrated legislation at the EU level. This could for instance, contribute to an increasing use of administrative registers as input for social statistics, that would be supported at EU level. Such a focus would not only reduce the cost of data collection but also contribute to a reduction of response burden of respondents.

Efficiency / Impacts on stakeholders

The efficiency gains to be made by integrating social statistical legislation into an overarching legislative instrument in place of the currently existing domain-specific Regulations, are procedural in nature, but are also substantial. Streamlining the process of changing requirements under one legislative instrument would lead to considerable efficiency gains on the side of individual NSIs and Eurostat. Due to the possibilities to reduce overlaps in data collections and combine survey data with other sources (such as administrative registers), efficiency gains can also be expected on the response burden on persons and households that contribute to the data collections.

The combination of legal integration and reduction of fragmentation of production processes contributes to potential efficiency gains at the European level. This could allow for a better integration and a more flexible design of survey instruments, as social survey variables can be grouped into thematic fields (“modules”). This has been successfully introduced already in LFS and tested in a “laboratory experiment”, 18 where lower costs and increased possibilities for cross-tabulation of variables were achieved.

To assess the efficiency of the proposed policy option in more detail, NSIs were asked to indicate their expectations for the development of costs for data collection in an environment with integrated legislation and defragmented production processes. It is noted that the question of expected cost developments for NSIs is complicated, because it cannot be seen separate from the (expected) needs of NSI’s national users (such as national policymakers). The main concerns are related to the expected costs for the collection of data, as these constitute roughly 65% of the total costs made by NSIs. As such, these easily outweigh other costs, such as design (roughly 13%) or processing, dissemination and transmission of data (roughly 22% of the costs). A majority of NSIs expect lower or at least similar costs for data collection in the medium-term. However, it is not possible to evaluate the cumulated impact on costs of the different modernisation initiatives, such as combined methodological work, inter-institutional collaboration and IT developments.

Despite these efficiency gains at the system level for the collection of data, NSIs point out the risk of significant one-off cost increases in the short-term. This is mainly due to various requirements on standardising quality criteria, technical definitions and statistical qualifications, related to the defragmentation of production processes and legal integration. The costs for these changes are a potential burden for NSIs, who have no choice but to revise their procedures and data collection methods to reflect new requirements, if codified in law. As a result, NSIs are hesitant towards legislative instruments involving integration, if accompanied with the need for far-reaching overhaul of their own procedures. Related to this, various NSIs and data users also point to the importance of continuity of data series, and possible risk of disruption. As continuity of data series is as important for Eurostat, the NSIs and data users, it is assumed that this policy option remains within the parameters to ensure continuity of data series.

Over the medium-term (5 to 10 years), 18 out of the 28 NSIs are of the opinion that an integrated legislation would increase the one-off costs in the re-design phase, and 13 out of 28 NSIs fear increased costs for the treatment and transmission of data. However, it is noted here that the types of costs expected by the NSIs are generally in the form of investments (design of IT tools, questionnaires, training materials, etc.) which could be recovered with repetitions of data collection operations. In addition and as already mentioned above, the costs for design or transmission are relatively minor in comparison to the costs of collection of data. Only a minor cost-saving in data collection has the potential to outweigh considerable one-time cost increases in the design or transmission. The box below provides a model of those costs to underline this issue.

Data users point primarily towards the possibilities related to a combined integration and defragmentation. Users strongly adhere to national actions to defragment existing production processes (‘stovepipes’) and various types of data users assigned a high priority to such actions.

6.2.5.Policy option 1.3b: Integrate social statistics governance in combination with defragmentation of national processes across data collections

When considering the integration of domain-specific Regulations at the EU level it is possible to include the current data collections that are currently not governed by law as well. HBS / HETUS data collections which are currently run on a voluntary basis, i.e. governed by gentlemen’s agreements, would therefore be turned into legislation when integrating this with other data collections in social statistics. Due to the considerable differences in place in the various Member States, legal integration at the EU level alone is not sufficient, and should be accompanied by substantial harmonisation at the national level as well, as proposed in this policy option. For this reason, this policy option also assesses the impacts of expanding the scope to HBS / HETUS.

Effectiveness

In terms of effectiveness, relatively similar impacts are relevant as for policy option 1.3a, except that these would now also be applicable for the HBS / HETUS data collections. However, this provides a number of additional impacts that are explored here.

Policy option 1.3a showed a higher responsiveness to societal needs. This is even more so when including HBS / HETUS. Such fixed procedures would come in the place of the current less formal system that is also subject of detailed discussions. The codification into law increases the formal procedures, which replace the comparable informal coordination mechanism. Effectively, the responsiveness to societal needs would increase, as more domains would be covered, compared to option 1.3a.

The quality of social statistics in the EU is ensured by the common implementation by 28 Member States of technical and substantial requirements on the collection of data. With legal provisions that cover on all surveys, these are all governed by a similar instrument to ensure the quality of social statistics. These potential gains in quality are found on top of the already defined quality improvements established for option 1.3a. Overall, the highest gains in ensuring quality are found for policy option 1.3b.

The use of innovative methods and procedures is not limited by could be further supported when the legislation would also extend to HBS / HETUS. These two surveys in particular could benefit from innovation, as it can be foreseen that in the near future new data sources for consumption and time use wold become available, in particular from the use of new technologies.

Efficiency / Impacts on stakeholders

As already shown under policy option 1.3a, considerable efficiency gains can be made by integration, both for NSIs as for Eurostat. However, as the gains increase, so do the implementation costs, and this generates quite different costs for national data producers on the short term. Due to the informal governance character of HBS / HETUS, Member States have developed relatively different approaches in their data collection that require to be redesigned or at least adapted once these are brought under the broader legal framework.

In fact, in the latest round of HETUS only 15 countries contributed with data. In the survey conducted for this impact assessment, 10 countries indicated that they currently do not collect any data for HETUS, and are not planning to do so in the near future. For a number of Member States, no answer has been collected on this point. For this reason, the impact of moving HETUS into European legislation would not only bring some implementation costs as calculated under policy option 1.3a, but would require the introduction of a new data collection in some Member States 19 . The situation of HBS is rather different. All Member States are conducting this survey, as it is the basis for other statistics, in particular the price index. However, here, the current level of harmonisation would imply some methodological alignments for some Member States to EU standards in the design phase. The data collection phase would however remain untouched and would benefit from the defragmentation of the data collections as explained earlier.

Considering the two surveys, it is worth underlining their particular nature: they are both based on detailed collection (of expenditures or time used), realised with the use of diaries, in which respondent take notes of the required information, making the survey burdensome and lowering the response rate. Technology and innovation can greatly help in that problem. New data collection mode and tools like smart devices, scanning of invoice, tracing of movement are already used or tested by many Member States. In the long run, it could even be expected that a substantial part of the required information would be accessed without the intervention of the respondents. Therefore, methodological and technological developments should be realised prior to the entry into force of a legislative instrument covering these domains.

6.2.6.Summary of impacts

The increasingly negative impacts of the continuation of the status quo on the core problems, underline the need for addressing domain fragmentation in European social statistics. The continuation of the separate domain-specific Regulations (under policy option 1.0 and 1.1) proves a limiting factor in responding to societal needs.

The main purpose of the policy options discussed in this section is to reduce the negative impacts on the core problems of the domain fragmentation in European social statistics. To do so, legal integration is one important pathway, as well as addressing the fragmented production processes of social statistics (‘stovepipes’). The policy options that seek to more actively reduce fragmentation of production processes (1.1 and 1.3) show positive impacts in reducing overlaps in data collection, regardless of any changes in the legal structure. These two policy options also show positive impacts on the possibilities for NSIs to start using innovative methods of data collection. One negative impact that affects these policy options together, concerns the possible higher initial implementation costs for NSIs associated with changes in the design phase and the production process. All NSIs indicated their concern for these costs.

Policy option 1.3a / 1.3b combine the positive elements of the legal integration with the positive impacts of defragmentation of production processes. While option 1.3b is associated with the largest increases in quality of data collection, it also comes with relatively high implementation costs. As policy option 1.3a does not have a similar drawback, this option seems at first sight to be the preferred alternative. However, provided that methodological work is developed and better use of the possibilities offered by the new technologies is exploited, option 1.3b also remains a better option in the longer run that would have the advantage of bringing the entire field of social statistics under one roof.

6.3.Assessing impacts: addressing rigidity of data collection

6.3.1.Introducing policy options

This section summarises rapidly the impacts related to the policy options that modifies the level of flexibility in the data collection of European statistics. As these options are of a more procedural nature but important for the stakeholders (in particular the NSIs) the details of their analysis are described in Annex 5. Three main policy options have been defined:

•    Baseline (option 2.0): Changes to programme or technical elements require changes in legal instruments.

•    Policy option 2.1: Statistical programming set by Parliament / Council, technical elements more flexible to change.

•    Policy option 2.2: Statistical programming and technical elements relatively flexible to change.

The options presented in this section can vary independently from the policy options presented in the previous section. Possible impacts of a combination of the shortlisted policy options under this heading and on domain fragmentation are explored in more detail in Chapter 7.

6.3.2.Summary of impacts

The assessment of impacts for each of the policy options shows the paradox of flexibility for European social statistics. Increased flexibility contributes substantially to the responsiveness of the data collected. Secondly, additional flexibility due to delegating the technical matter of social statistics to the experts would contribute positively to the quality of data, as it would allow Eurostat to respond rapidly to new issues through the ESS. However, flexibility also runs the risk of considerably higher costs for NSIs. The key is therefore, to balance the flexibility in responding to user needs with the stability requested by NSIs.

From the policy options defined, option 2.1 performs best in balancing these needs in view of effectiveness and efficiency of the proposals. It keeps stability in social statistics by fixing programmatic elements by law, while introducing more flexibility in the technical matter. Despite its positive impacts, it is necessary to pay attention to possible costs that may arise as a result of the flexibility. It could be necessary to fix in the basic law the main parameters determining the costs of the statistical operations for the NSIs.

7.Combining the different options: Impacts of shortlisted options

7.1.Short-listing policy options

Based on the impact assessment of the previous chapter, this section further elaborates the impacts for the short-listed options. The previous chapter short-listed policy option 1.3a (Integrate existing Regulations, ‘defragment’ production processes) and option 1.3b (Integrate social statistics governance in combination with defragmentation of production processes and across data collections) in the longer run to address the fragmentation of statistical domains in the EU. These policy options propose an integration of the domain-specific Regulations at the EU level, in combination with efforts at the level of data production and handling. These policy options increase the possibilities for responsiveness, quality, innovative methods, and the ability to reduce and limit costs, created by the fragmented design of social statistics in the EU most effectively.

To address the rigidity of the current system of data collection in the social domain, policy option 2.1 (statistical programming set by Parliament / Council, technical elements more flexible to change) has been short-listed. This policy option underlines the need to uncouple programmatic requirements from technical requirements and further implements the spirit of the Statistical Regulation (223/2009). This should contribute to the quality of decision-making and as such, improve the responsiveness, quality and capacity towards use of innovative methods.

7.2.Intervention logic and legal basis

Based on the selection of short-listed policy options, all the information has been collected to complete the “objectives tree”, which guides the intervention logic, in the light of the problems drivers and the related identified problems.

The legal basis for this approach is provided by TFEU Article 338, which grants the European Parliament and Council the competence to adopt measures for the production of statistics where necessary for the performance of the activities of the Union. The legal basis for social statistics is further established by the “Statistical law” (Regulation 223/2009). TFEU 338(2) further requires that “the production of Union statistics shall conform to impartiality, reliability, objectivity, scientific independence, cost-effectiveness and statistical confidentiality; it shall not entail excessive burdens on economic operators”. The policy options short-listed by this impact assessment are in line with these legal requirements, and seek to improve the impartiality, reliability, objectivity, scientific independence, cost-effectiveness and statistical confidentiality of social statistics, in an effort to provide the necessary and quality input for national and European policymakers.

The short-listed policy options in combination would require the adoption of a framework legal act, repealing the currently existing legal acts. The formulation of this framework Regulation should already contribute (or at least allow) towards reducing the fragmentation of social data collections at the national level, while respecting the subsidiarity principle. In order to implement policy option 2.1, its design calls for a multi-tier regulatory framework. This framework would enable the possibility to govern the various statistical business processes by different regulatory instruments (combining the use of implementing and delegating acts, and non-legal instruments such as ESS agreements, guidelines, manuals or model questionnaires). In order to ensure coherence with the Statistical Regulation, this framework would also include the various safeguards against unforeseen cost increases that currently exist in the domain-specific Regulations.

The paragraphs in the remainder of this chapter address the overall impact analysis for the retained policy options. In addition, where no impacts are found, this is explicitly mentioned.

7.3.Impacts on effectiveness

Under option(s) 1.3, the integrated existing Regulations and the defragmented production processes at the national level combine to increase the responsiveness to societal needs. In addition, the increased possibilities to promote consistency across different domains make for a better position for Eurostat to ensure the quality of social statistics. This policy option also has an enabling effect on the use of innovative tools and methods. It allows better cooperation and integration of NSIs and ONA at national level, in line with the amended regulation 223/2009. As under policy option 1.3b more domains are covered, the positive impacts increase in comparison to those under 1.3a. Sections 6.2.5 and 6.2.6 discussed these impacts in more detail.

Policy option 2.1 also has positive impacts on the three aspects of effectiveness. Without the distortions caused by the currently also regulated technical details it will be easier to ensure that political priorities are clearly laid down in the Regulation and to adapt these if required. At the same time, it will be easier for Eurostat and the NSIs to implement technical changes with the aim of improving the quality of social statistics and to incorporate innovative tools and methods. Section 6.3.3 discusses these impacts in more detail.

Furthermore, these two options, as well as their combination, meet the simplification objectives of the REFIT programme, in particular by streamlining five regulations into one single regulatory framework.

7.4.Impacts on efficiency

The implementation of policy options 1.3a and b cause one-off design cost for the NSIs. However, structural costs are estimated to be reduced, mainly because of a reduction of redundancies / overlaps in data collection and the re-use of production systems.

Depending on the concrete implementation of the production methods at the national level (adoption of technological and methodological innovations, improved access to administrative registers), the increased costs in the initial design phase of social surveys could be largely compensated by decreased costs in the data collection phase, which represents two-thirds of the total statistical production costs.

In figures, based on the scenarios of the model and the respective time spans indicated in the box under the paragraph 6.2.5:

Under policy option 1.3a, where HBS and HETUS are not considered in the integrated regulation (scenario 4), the respective variation in costs would amount to a €54 million cost increase in the design phase and a €7.5 million cost decrease in the collection phase, leading to a net present value of €-2 million

Policy option 1.3b (scenario 1) leads to an estimated increase in costs of €10.3 million in the design phase (at EU level), while leading to a decrease of €20.8 million in the data collection.(net present value of €-10.4 million) A similar calculation with more contained design costs and more modest decreases in data collection costs (scenario 2) would bring costs of €5.2 million in the design phase, compared to €8.3 million of economies in data collection (net present value of €-3.1).

Under policy option 1.3b, when the period for estimating the data collection costs reduction is less restrictive (scenario 3), the reduction in data collection costs could increase to 44.4 million, compared to a 10.3 million cost increase in the design phase (net present value of -34 million).

In addition, future costs related to changing legislation are expected to be lower as a result of integration and defragmentation. Section 6.2.5 discussed these impacts in more detail.

For policy option 2.1, an estimate of the impact on costs could not be made; this is too much depending on the actual implementation (technical details). It is clear that flexibility can have both positive (implement changes in an efficient tempo, flexibility in production) and negative (more often changes are made) impacts. Section 6.3.3 discusses these impacts in more detail.

7.5.Impacts on stakeholder

The impacts on stakeholders largely follow from the previous sections. Users will profit from the increased flexibility in terms of adaptation to users’ needs and the possibility to combine data on cross-domain issues.

From the users’ perspective it is essential that, in this process of change, the continuity of output is guaranteed. In particular, the stability of important time series should be ensured and the volume of the principal statistical outputs should not be reduced. This requires also addressing and minimising the impact that standardisation and introduction of innovative techniques may have an output.

Under policy option 1.3, NSIs will face substantial one-off cost related to design. Costs will be lower at structural level. The impact of option 2.1 on costs could be kept under control provided that the essential elements determining the costs for NSIs are fixed in the framework legislation.

For respondents, the response burden is expected to be reduced under all preferred options, as a result of the possibility to combine data and the increased possibilities to use administrative data.

Sections 6.2.5 and 6.3.3 discuss these impacts in more detail.

7.6.Impacts on SMEs

The short-listed policy options do not have direct or indirect impacts for SMEs. The proposed legislative instrument only concerns statistics on persons and households.

7.7.Economic impacts

The impact assessment shows that there are indirect economic impacts of the short-listed policy options. These contribute towards an improved timeliness and consistency of social indicators and in particular, in the areas of labour markets, and income. As such, the short-listed policy options would develop a more efficient statistical evidence base for policies that contribute towards stimulating and consolidating economic growth and job creation. Through the improved statistical data available, the short-listed policy options further support the development of policies aiming at boosting competitiveness and innovation.

Moreover, social statistics are a crucial element in the allocation of public expenditure to social policies. The annual cost of producing social statistics is estimated at €176 million, which is only marginal given the extent of spending on social expenditure by the EU and Member States: Eurostat estimates that Member States spend on average 29.5% of the GDP on social expenditure. Therefore, if improved, social statistics would contribute to a gain in the efficiency of social policies that would easily offsets the costs of producing such information.

7.8.Social impacts

The collected evidence in the impact assessment points to indirect social impacts. The short-listed policy options will provide improved relevance and timeliness of social indicators on issues, such as employment, unemployment, transitions in the labour market and the evolution of poverty. Combined with existing macroeconomic indicators, this will enable more integrated and efficient analyses of data that can be used to calculate the social impacts of future policy decisions.

Coherent statistics that can be better combined can also contribute towards identifying emerging multi-dimensional topics (e.g. well-being, combining living conditions, health, income, education, skills, labour status, etc.) and thereby help policymakers to design relevant public policies.

7.9.Environmental impacts

The collected evidence does not suggest that the short-listed policy options have direct or indirect impacts on environmental issues.

7.10.Impacts on fundamental rights

The short-listed policy options do not create additional impacts on fundamental rights. The most relevant for this impact assessment are possible impacts in personal data protection (laid down by Article 8 of the EU Charter of Fundamental Rights). However, none of the selected policy options foresees a change in provisions on personal data protection. The national and EU Statistical Laws provide for guarantee of protection of data by the NSIs, which have strong policies for protecting respondent confidentiality, anonymising records, preserving questionnaires, etc. in place.

8.Monitoring and evaluation: How would actual impacts be monitored and evaluated

The European Commission REFIT agenda makes evaluation of all new legislative measures a priority for the Commission. The proposed legislative instrument will also be subject to a complete evaluation in order to assess, amongst other things, how effective and efficient it has been in terms of achieving the objectives presented in this report and to decide on whether new measures or amendments are needed.

This section summarises existing and future monitoring mechanisms.

It is important to refer first to the existing monitoring and evaluation tools which are valid for the whole of the statistical production of Eurostat, that already allow for good analysis of the evolution of the effectiveness and the efficiency of the new statistical initiative and the quality of the data produced. To summarise:

The European Statistical programmes (currently regulation 99/2013) foresee systematically mid-term and final evaluation of the programme. Social statistics are an integral part of these reporting mechanisms 20 .

The Eurostat management plan foresees the follow up of key performance indicators that apply as well to social statistics 21 .

Users' satisfaction surveys are carried out on a regular basis 22 .

Eurostat also carried out systematic Rolling Reviews in accordance with the binding standards approved by the Commission. Rolling Reviews involved not only the assessment of the statistical data produced but also the process to produce them, the interactions with data providers and interactions with users of the data. They formed part of the Quality Assurance Framework developed by Eurostat in 2007 23 .

Each statistical domain is also monitored through quality reports, regularly produced by Member States and analysed and compiled by Eurostat, as part of the statistical quality insurance framework described under the box "Quality of statistics" in paragraph 2.1 and covering Relevance, Accuracy and reliability; Timeliness and punctuality; Accessibility and clarity; Coherence and comparability. Furthermore, these existing frameworks, in particular the domain specific information, have been used in the initial analysis steps, as explained under 4.2 and will be used as the regular quality reporting mechanism that will be an integral part of the legal initiative to be proposed.

Concerning burden and costs, several elements are already routinely collected as part of the data collection, like the interview duration that is a good indicator of the burden on the respondent;

In addition, the costs elements identified under the paragraph 2.4.3 will be monitored as part of the routine data collection. In particular, the evolution in the use of administrative data, the data collection form (web interview, face to face interview, etc.), the size of the sample and of the questionnaires, and the frequency of data collection will be monitored at very detailed level (at both individual and variable level). This will allow measuring the progresses in the use of these techniques and the burden impact.

Furthermore, a central monitoring indicator for the implementation of the proposed legislation would be the variation of cost of social data collection at an aggregated EU level, in which we distinguish the costs for NSIs in the design, collection and transmission of data.

Whilst these indicators on costs are not directly comparable across Member States, due to obvious differences in size, approach towards statistics and other developments (as explained under 2.4), they are however useful over time as they allow the tracking of the development of costs of social statistics within the EU and for each Member State separately. Such costs may be expressed in terms of staff involved (e.g. full-time equivalent number of mobilised interviewers) or financial resources (e.g. budgetary appropriations for a given data collection). As such, this indicator provides crucial monitoring input on one of the central issues of the new legislation. It will require improved and harmonised costs reporting framework for the whole ESS that identifies the statistical production phases.

This set of improved monitoring instruments that would cover all quality dimensions, burden costs, and new techniques would also allow for identifying and sharing good practices across data collections and between Member States as well as allow for analysing the trade-offs between quality and costs.

The table below presents a sample of monitoring indicators that have been identified for assessing the effectiveness of the legislative instrument. Each one allows for the monitoring of the development in the implementation of the legislation against the baseline measurement presented in this impact assessment. These indicators will be used in the evaluation framework developed above. In addition, qualitative information about modernisation plans (e.g. pilot studies) should be also included.

Annex 1: Procedural information

Lead DG: DG ESTAT

Agenda planning reference: 2014/ESTAT/002

This impact assessment is prepared by DG ESTAT to support preparation of a Framework Regulation for the Production of European Statistics on Persons and Households, in the context of the European Commission's Regulatory Fitness and Performance programme (REFIT). Eurostat enlisted the help of a contractor to gather and analyse relevant information during the second half of 2014. The associated inception report was published in July 2015 25 .

Preparation of this impact assessment (IA) started in 2014 with the aim of comparing the impacts of different simplification options. As this was prior to the adoption of the Better Regulation guidelines, there has been no full evaluation done to support this initiative. However, the regular evaluation mechanism in place at Eurostat (quality reports, rolling reviews, evaluation of the European statistical programme, management plans) as well as an ex-post analysis of the current situation (through discussions with NSIs, interviews with main users, public consultation targeting general users and thorough desk research) formed a central element of the IA process. An inter-service steering group was established in July 2015 and involves the following DGs: Secretariat-General (SG), Budget (BUGET), Justice (JUST) Education and Culture (EAC), Neighbourhood and Enlargement Negotiations (NEAR), Employment, Social Affairs and Inclusion (EMPL), Migration and Home Affairs (HOME), Regional and Urban Policy (REGIO), Health and Food Safety (SANTE), Economic and Financial Affairs (ECFIN) and Internal Market, Industry, Entrepreneurship and SMEs (GROW), as well as the Legal Service (SJ). The inter-service steering group met twice, the first meeting was held on 9 July 2015, and the second took place on 27 January 2016.

Consultation of the RSB

(The recommendations of the RSB and the subsequent changes in the IA report will be summarised hereby, "this should presented in tabular format – the first column identifying the Board's recommendation and the second column how the IA Report has been modified in response")

Evidence used in the IA

In reference to the evidence that constitutes the basis for the Impact assessment, it is first pertinent to mention the evaluation tools currently in place, that already allow for good analysis of the evolution of the effectiveness and the efficiency of the new statistical initiative and the quality of the data produced. These tools include: the European Statistical programmes (currently regulation 99/2013), which foresees systematically mid-term and final evaluation of the programme, of which social statistics represent an integral part 26 ; the follow-up of a series of performance indicators, as planned by the Eurostat management, which also applies to social statistics 27 ; and the Users satisfaction surveys carried out on a regular basis 28 . In addition, the ongoing and continuous assessment of the quality of the statistics undertaken by Eurostat, presented in the form of Quality reports 29 , not only represented a source of extensive and reliable evidence for the preparation of the impact assessment, but also constitutes an already-in-place system for monitoring and evaluating the impacts a new legislation on social statistics.

Systematic Rolling Reviews, in accordance with the binding standards approved by the Commission, were also carried out. They formed part of the Quality Assurance Framework developed by Eurostat in 2007 30 , and involved not only the assessment of the statistical data produced but also the process to produce them, the interactions with data providers and interactions with users of the data, representing an significant source of additional evidence. Rolling reviews and quality reports are domain specific, and had to be complemented by facts covering the whole of social statistics on individuals and households.

Further evidence was therefore gathered ad hoc in the context of the preparation of this impact assessment, from the consultation to the stakeholders –Directorate Generals of the European Commission, National Statistics Institutes (NSIs), and the open public (i.e. academics/researches, business and business associations, interest groups, media and non-governmental organisation, and individuals/freelances)– that took place in the context of the preparation of this impact assessment, which is summarized in Annex 2. The combination of the information provided from the perspective of the data producers (National Statistics Institutes) and that of the data users –from within (DGs and EU agencies) and outside of the EU administration (e.g. external micro-data users)–, is a guarantee of the robustness and the relevance of the information collected. An additional aspect that contributes to the robustness and comprehensiveness of the evidence gathered thorough the stakeholders' consultation is the use of several modes of data collections (i.e. the use two online questionnaires addressed to the NSIs and to the open public, respectively, in combination with in-depth interviews to data users and data producers), which broadened the scope of the exercise, and allowed to undergo a more detailed analysis of the problems at hand, and a thorough assessment of the possible solutions from the perspectives of the different stakeholders.

External expertise used in the IA process

In addition to the stakeholders' consultation, additional sources of external expertise have been consulted. First, the contractor entitled to assist has been able to provide relevant input for the preparation of the assessment in their function as statistical consultant. Together with their support in the execution of the stakeholders' consultation and the analysis of the evidence gathered, they played a relevant role in the development of the problem tree –which allowed to identify and determine the relationships among the main problems, problem drivers and contextual variables–, that constituted the basis from which the subsequent work for the impact assessment was built upon.

Second, the inter-service group (ISG) created in the context of this impact assessment, in which 11 DGs were represented, constituted –besides its already established function of steering the IA process and contributing to the collective preparation of the IA report, and in relation to the fact that the DGs are main data users of the social statistics provided by Eurostat–, a most relevant source of external expertise for the preparation the Impact assessment of new legislation. DGs represented in the ISG were consulted at a preliminary stage, showing their needs and support to the initiative.

In addition, the opinion of the ESAC (European Statistical Advisory Committee) was also sought, as an external source of expertise. In their feedback, the (ESAC) shows support for the proposal for a framework regulation for European statistics on persons and households, while referring to the development of a framework regulation as essential in order to support the European Commission's current, more extensive, agenda on social policy, and to increase the integration and consistency of social data from across Europe 31 .

Finally, the expertise in the NSIs has been heavily used, in particular in the context of formal occasions, with the regular various working groups, task-forces, groups of directors and directors general as well as informally through various bilateral consultations.

Annex 2: Stakeholder consultation

Introduction

This section presents the results of the stakeholder consultation that was carried out amongst the main stakeholders. The approach towards and methodology of the consultation have been agreed upon by the Steering Committee. The consultation was carried out during July-December 2015. It meets the standards set by the European Commission. This section starts with an overview of the process, of each of the activities conducted. Subsequently, the outcomes of the consultation process are summarised in a factual manner.

The full results of the consultations are available at:

Eurostat public consultation web page: http://ec.europa.eu/eurostat/about/opportunities/consultations/iess

Open public consultation report: http://ec.europa.eu/eurostat/documents/10186/7237349/Open-public-consultation-report.pdf  

Consultation of the data users: http://ec.europa.eu/eurostat/documents/10186/7237349/Data-users-consultation-report.pdf  

Consultation of the data producers: http://ec.europa.eu/eurostat/documents/10186/7237349/Data-producers-consultation-report.pdf  

The following stakeholders were targeted for consultation:

Data producers: this category comprises the authorities responsible for the collection and compilation of social statistics. It mainly includes the NSIs and Eurostat. However, in this context, the data producers are at the same time, the main representative of the primary data providers, i.e. the households.

Data providers: this category includes not only the respondents but also national institutions that are keepers of administrative data files, such as for example, the Social Security or Tax Administration. The NSIs in the broader sense, are seen as a proxy of the primary data providers (i.e. households), given the difficulty of conducting stakeholder interviews within this category.

Data users: within this group a distinction can be made between institutional users – Commission DGs users, national administrations, other international organisations, or professional staff working within other EU institutions, and other external users, such as mass media and academics.

Amongst data users, a special mention of a particular body that has been consulted should be made: the European Statistical Advisory Committee (ESAC), established in 2008, by the Decision No 234/2008/EC of the European Parliament and the Council. It gathers 24 members representing users, respondents and other stakeholders of European Statistics (including the scientific community, social partners and civil society) as well as institutional users (like e.g., the Council and the European Parliament) According to article 8 of the Statistical Regulation (223/2009), the ESAC could be consulted in the process of possible new legislation.

The consultation process has been implemented as follows:

Producers of European social statistics: National Statistical Institutes (NSIs) of the EU (opinions gathered through a questionnaire to all EU NSIs and in-depth interviews to a selected subset based on an interview script). The consultation with producers of social statistics was limited to that of National Statistical Institutes, which play the role of coordinators for the National Statistical System in the Member States and EFTA countries. It was requested in the invitation for the consultation, that NSIs take into account the opinion of other producers, such as ministries for social affairs, labour, health, etc. NSIs have already expressed their opinions through high-level documents (ESS 2020 Strategy 32 , Wiesbaden Memorandum 33 , etc.) on the importance of modernising the production of social statistics. They have also discussed, within the framework of the DSS Group (Directors of Social Statistics), the initiative of integrating the different European legal acts on social statistics. 25 NSIs in the Member States and 2 additional EFTA NSIs responded to the survey. Eight NSIs were interviewed on a selective basis with a wide coverage of different statistical systems and resources, to express their opinion on the basis of additional in-depth interviews.

Users of European social statistics: selected Directorates-General (DGs) of the European Commission. Opinions were gathered through in-depth interviews with selected DGs: DG ECFIN, DG EMPL, DG SANTE, DG EAC, DG HOME, DG REGIO and DG JUST in order to complement information collected, during the annual bilateral hearings, by Eurostat.

General users (opinions gathered through an Open Public Consultation – OPC- based on an on-line questionnaire). Particular care has been taken to alert potential stakeholders and advertise about the consultation (see annex 2). 103 responses were received in the period 23rd July to 15th November 2015 (16 weeks). 35% of these responses were academic/researchers, 26% public authorities in Member States and the rest included staff of EU institutions (10%), consultants/individuals (9%), businesses/business associations (8%) and other.

These three consultations, their process, consultation instruments and results are detailed in three specific reports 34 .

The consultation included the perception of the stakeholders on the problems identified and confirmed by the steering committee and the opinions on the policy options which were adequately described. To establish this initial identification of problems, the relevant documents were reviewed such as:

•Europe 2020 Strategy

•Opinions of users: Opinions of the ESAC (European Statistics Advisory Committee), User Satisfaction Survey carried by Eurostat

•Documents of the European Statistical System: European Statistical Programme 2013-2017, Vision for the next decade, COM(2009)404, Wiesbaden Memorandum on Social Statistics, ESS Modernisation Strategy for Social Statistics (ESSC 2014/23/6/EN), DSS and DSS Board meetings and discussions, Rolling reviews of social statistics by Eurostat, regular quality reports, etc.

Open public consultation

This section provides a summary of the consultation process. This section lists each of the relevant steps taken in the consultation, starting with Open Public Consultation (OPC). All steps were agreed upon with Eurostat and the Steering Committee. In preparation for the open public consultation, profiles of general data users were established, including academic/research institutes, Business, EU institutions, individuals, Interest groups, media, NGOs, international organisations, public authorities, other. In order to collect responses, a promotion strategy was put in place, consisting in identifying collectives of users and sending mails with links to the questionnaire. This included emails sent to:

•Representatives at the NSIs in several working groups (140 recipients);

•Members of Business Europe (former UNICE) and national representatives (78), to UEAPME and national representatives / full members (46), to ETUC and European Trade Union Federations (48), Think Tanks (72);

•Professional statistical associations: Federation of European National Statistical Societies and member associations (49), International Statistics Associations (16), National Statistics Associations (26);

•Archive resources and research institutes (68), social research institutes and Universities from countries with fewer responses received in the first weeks (266), Equinet Members of the equality bodies, ENSR (European Network for Social and Economic Research) (31 members), ESF Expert Evaluation Network (30 members);

•Other experts, NGOs, including the Social Platform that advertised the consultation in amongst their members, consultants, researchers, and other uses of social statistics (around 400 recipients).

Following the EC Stakeholder Consultation Guidelines 2014, and in order to ensure consistency and user-friendly access to information, the standard consultation page template was used. The standard page provides details about:

-title and policy field of consultation;

-target groups;

-period of consultation (12 weeks as general rule, 16 weeks for this consultation);

-objective of consultation;

-instructions on how to submit contributions;

-information about the results of the consultation and next steps;

-privacy statement.

The OPC questionnaire prepared and uploaded (EU Survey Tool), as well as a consultation document that summarised the initial research identifying potential problems of social statistics and a description of policy options.

103 responses were received in the period 23th July to 15th November 2015 (16 weeks). 35% are academic/researchers, 26% public authorities in Member States and the rest includes staff of EU institutions (10%), consultants/individuals (9%), businesses/business associations (8%) and other.

Consultation with producers of European social statistics

A list of contacts was established based on the members of the Directors of Social Statistics (DSS) Group (representatives of Member States and EFTA countries). Eurostat officially announced an invitation to participate to the interviews at the DSS meeting on the 25th of February 2015. In addition, a series of in-depth interviews was carried, with representatives from a selection of Member States that would include large and small territorial size (to take into account the needs for higher geographical breakdown of social statistics), “older” and “newer”, as well as NSIs with more or less intense use of administrative registers for social statistics. Representatives were selected from: Denmark, Germany, France, the Netherlands, Poland, Portugal, Romania and United Kingdom.

To receive feedback from all NSI, an online questionnaire to all relevant NSIs was prepared and put on-line (EU Survey Tool). The online questionnaire was accompanied by a document with the initial research identifying potential problems of social statistics and a description of policy options. The questionnaire was piloted with the help of the NSIs of Lithuania and Spain. In total, 25 responses from EU and 3 from EFTA NSIs were received. Particular attention was paid to collecting feedback from Member States for which social impact assessment is especially relevant, like programme countries.

Consultation with institutional social statistics users

A list of specialised statistics users was proposed to target with in-depth qualitative interviews. In the selection various European Commission Directorates-General and agencies were included; it was agreed with Steering Committee to carry out in-depth interviews with DG ECFIN, DG EMPL, DG SANTE, DG EAC, DG HOME, DG REGIO and DG JUST. This includes a mix of the various affected policy domains requiring social statistics. Contacts were made to establish face to face meetings in September-November 2015. In preparation for the interviews, respondents received an outline of the relevant questions to prepare. In the end, all 7 selected DGs were interviewed.

Findings / Outcomes of the consultation

Perception of the problems

Preliminary desk research allowed to propose a list of problems in social statistics and possible relations of causality between drivers and problems. This list was described in the documents accompanying the stakeholder consultation for discussion and opinion. The consulted stakeholders – general users, specialized users in EC DGs and NSIs - acknowledged such problems.

The perceived problems can be summarised as follows:

•    Data users are concerned with the risk of European official statistics lacking relevance (usefulness), due to a combination of quality issues (coverage of emerging social issues, constrained timeliness and limited comparability and coherence);

•    Data producers (NSIs) are concerned with the high production costs and the pressure to cover the informational needs in a context of emerging social crises within a short timeframe, non-official ubiquitous and almost real-time information. Response burden is also an issue (risk of decreasing response rates and thus data quality). NSIs would like to have more support for their modernisation processes (innovation in technology, methodology, use of new data sources, management, etc.) that intend to address the need of decreasing production costs.

Need for increased responsiveness of official European social statistics to emerging information needs

Relevance is recognised as one of the quality dimensions of official statistics and describes the degree in which statistics are responsive to the needs of users.

Without neglecting what has been done so far, European policymakers (DGs) require more responsiveness from the ESS. Better responsiveness implies the ability to quickly accommodate emerging informational needs within the European statistical programmes (multi-annual and annual), but also within the national statistical plans. The rigidity of the current system in planning European social statistics, along with calendars specific to each statistical data collection set and specific legal instruments, does not help to create a high level of responsiveness, even if the system of modules in surveys provides some flexibility for the coverage of emerging topics. When social topics are not covered by European official statistics, the European Commission and other EU institutions collect and disseminate other statistical operations (e.g. the Eurobarometer, the European Social Survey, the European Quality of Life Survey and other surveys setup by for instance EU agencies) which can quickly respond to an emerging need for information, but do not benefit from the technical support of EU NSIs (and the related statistical quality).

NSIs are also concerned with the usefulness of their production, which strongly depends on their capacity of disseminating timely data on “hot” issues (social crises, new policy issues, multi-dimensional social aspects, etc.). NSIs have to be sure that the planning process is flexible enough to accommodate new needs. However, at the same time it has to provide a clear forecast of resources needed. As mentioned above, the multi-annual planning at ESS level, which occurs over a 5-year process, in addition to the relatively closed set of research domains in the current regulations, are made flexible by the possibility of including modules, but most NSIs term require no less than 2 years to be prepared for such modules. According to most NSIs’ opinion, any reform of the legal environment should carefully tackle the issue of programming, ensuring mid-term predictability and responsiveness. Linking data across data collections can provide a technical solution to new information needs not covered by current data collections (for instance, the combination of data on income and well-being), but this may be hampered by the current fragmented legal setting and the associated lack of common definitions and variables across surveys.

The information needs of data users in general (e.g. researchers, media, interest groups, businesses, NGOs, etc.) were, to some extent, not met over the last two years (e.g. topics related to social impact of the crisis). Furthermore, those data users that need more timely data (e.g. business, freelancers, media and NGOs) are not satisfied with the capacity of adaptation of European social statistics towards new user needs.

The creation of coherent data pools (e.g. data warehouses) and increasing the capacity to link data from different data collections are important actions, according to data users, that could improve the relevance of European social statistics.

From the point of view of data users, greater attention to social developments and simplification of legal procedures would improve the relevance and timeliness of European social statistics.

Coherence and comparability to be increased for higher quality of social statistics

The lack of complete coherence across European social statistics is a major problem for all stakeholders and in particular, for the users. The results of the open public consultation showed (with a certain frequency) that the respondents perceived that social surveys were not fully coherent or aligned: approximately one third of the respondents have made attempts on occasion to align data from different surveys, which were perceived as incoherent. It is important to highlight that data other than official statistics from Eurostat have been used by the respondents with a certain frequency, due to perceived incoherence and comparability problems. The fact that users sometimes used other data instead of official statistics from Eurostat (because of incoherence and comparability problems of these) can be seen as a warning sign for prompt reaction from the producers of statistics.

Coherence of social data collections is fundamental to address multi-dimensional social issues. Coherence is another dimension of data quality recognised at the ESS level. It not only can improve the relevance of statistics to users, but can also simplify the processes of production of statistics by sharing tools. The lack of coherence and comparability of social statistical operations is not perceived as a consequence of the legal framework but of the methodological aspects and of the historical development of European social statistics as separate “stovepipes” with distinct methodologies and production processes. While work is ongoing amongst NSIs and Eurostat with the exchange of experiences (working groups, collaborative networks) and there is room for improved coherence in the individual regulations, an integrated regulation is deemed necessary by all stakeholders to enforce the methodological coherence by providing standards for variables, statistical units, classifications, reference periods, etc. applicable to all data collections.

More innovation needed for decreasing the high production costs and response burden

NSIs, together with Eurostat, have embarked on a modernisation process of official statistics, which spans technological innovations, methodological progress and institutional/legislative actions. Users are generally less concerned with the production process for statistics, but it is a key issue for NSIs. Modernisation processes related to the integration of the production processes for increased efficiency within NSIs are taking place without a legislative framework. For instance, IT improvements are present outside the legal environment.

The possibility of increasing the efficiency of the statistics production system has been acknowledged as an important issue.

On one hand, the vast majority of respondents of the open consultation consider it to be 'extremely important' in having access to free-of-charge data. A policy of free access to (main) social statistics requires increased efficiency in terms of costs within the production cycle. The ‘pressure’ is even higher in a period when resources for producing official statistics are scarce and the possibility of large data, as well as data available from other sources, exists. At the same time, efficiency of social statistics –in terms of tax-payers money and burden on respondents- is also important for the users.

While the perception of statistics producers with regards to cost is outstanding, the total cost for the production of the EU social statistics in the scope of the proposed integrated regulation is estimated around 300 Million Euro (176.2 Million on an annual basis), which is a small percentage of the budget for social policies within the EU. It is remarkable that EU DGs are sensitive to NSI cost issues. In all cases where applicable, DGs expressed their willingness to delegate budgets to Eurostat to satisfy data needs. Within these same lines, it can be concluded that DGs require official statistics for sound policymaking, but do not neglect the use of other European (non-official) data for short-term or ad-hoc needs. In fact, non-ESS surveys on social issues are financed with EU budget (e.g. EQLS, EWCS, Eurobarometer).

However, at the level of NSIs, the cost of additional surveys or of increased samples is still a major concern. A legislative framework that would provide for standardisation of processes across data collections and countries would facilitate the re-use of “tools” for data treatment and dissemination (accounting for about 22% of the total costs). The increase in data collection costs (which account for about 65% of total costs) that would be due to higher precision requirements or higher regional disaggregation would offset gains in efficiency of processes. However, this may only be significant in the case of large countries with a regional structure. This increase in requirements may take place independently of a legislative modification at the EU level, since NSIs could consider it satisfactory to the needs of national users (policymakers). It is difficult to evaluate in detail the cumulated impact on costs of the different modernisation initiatives which combine methodological work, inter-institutional collaboration and IT developments. As mentioned above, the innovation in data collection technologies is taking place without a necessary supportive legislative framework.

However, the modernisation process based on increased use of administrative data can be strengthened by a sound legislative basis. This not only increases the efficiency of the production system by decreasing the cost of data collection (which accounts for about 65% of total costs) but also the response burden of respondents. While in most countries the current legislation allows NSIs to access these sources, an explicit provision of this in an EU integrated regulation is considered by NSIs as a strong leverage. This has to be accompanied by more methodological work to assess the quality of administrative registers and harmonise the concepts used with them for the production of social statistics. The use of other innovative data sources (e.g. Big Data) is still very exploratory.

Generally, the modernisation process is strongly supported by users. Users strongly adhere to the actions proposed for overcoming these problems. All groups of respondents assigned high or relatively high importance to all actions. This might be interpreted as a call for attention from the users with regards to the need for improvement of European social statistics at different levels and dimensions. It may also suggest that there is room for higher NSI engagement with users within the process of modernisation.

The recent trends in user needs highlight the importance of engaging with users. Emphasising the importance of the user focus, it is of extreme relevance to conclude that, based on the OPC results, users support the actions proposed towards the modernisation of the European social statistics. There is a trade-off between timeliness and quality –coherence and comparability- (which are important dimensions for the statistics users), but users should be informed (and should comprehend and accept) that there can be limitations on what producers can achieve in a given time, although their needs have been considered in the planning and prioritisation phase of the production process.

Annex 3: Who is affected by the initiative and how

The production and use of European social statistics involves 4 main categories of stakeholders:

•Data providers: this category includes not only the respondents to surveys (households, individuals) but also national institutions that are keepers of administrative data files, such as for example, the Social Security or Tax Administration.

•Data producers: this category comprises the authorities responsible for the collection and compilation of social statistics. It mainly includes the NSIs and Eurostat.

•Data users: this includes institutional users –national administrations, other international organisations, or professional staff working within the Commission services or DGs and other EU institutions, and other external users, such as mass media and academics.

Amongst data users, a special mention of a particular should be made: the European Statistical Advisory Committee (ESAC), established in 2008, by the Decision No 234/2008/EC of the European Parliament and the Council. It gathers 24 members representing users, respondents and other stakeholders of European Statistics (including the scientific community, social partners and civil society) as well as institutional users (like e.g., the Council and the European Parliament) According to article 8 of the Statistical Regulation (223/2009), the ESAC could be consulted in the process of possible new legislation.

The preferred policy option has a significant impact on data producers and data users.

Data users

Data users benefit of an increased coherence of the different existing sets of social data. The use of common concept definitions, classifications and breakdowns, statistical units, and the coordination of survey calendars improve the quality perceived by statistics users.

The preferred policy option allows for greater flexibility in the statistical work programme by providing for mechanisms to accommodate emerging information needs. The current scenario is characterized by the existence of flexible data collection instruments which can provide for ad hoc information needs (e.g. Eurobarometer or other surveys run for instance by EU agencies) but these are not coordinated with European social surveys. Under the preferred policy option, the system will increase flexibility as well as integrate social surveys; this has the double advantage of covering new information needs by producing new data and by combining them with those already collected.

In order to fully benefit from an increased coherence, users may explore innovative ways of combining data from different data collections. Methodologies such as data linking and data matching can be applied by advanced users (e.g. academic researchers). They should also contribute to the identification of potential inconsistencies across data collections. Feedback from users can be institutionally channelled through the European Statistical Advisory Board and several other less formal channels, like domain specific networks and conferences.

To benefit of greater flexibility, users have to closely work with statistical authorities to achieve proportionality in the satisfaction of information needs and production efforts. This implies ensuring an effective mechanism for the identification of negative priorities (which can also be undertaken by the European Statistics Advisory Committee). At the European Commission level, the institutional users (DGs, agencies) will continue working closely with Eurostat in the definition of needs and develop the mechanisms for budget delegations, formalised with Memorandum of Understanding between each user DG and Eurostat and the annual formal "hearing" with each DG.

Data producers

The production of European social data is mostly done at national level by NSIs and other public authorities, which are public institutions and therefore with public budgets. Additional requirements of information mean higher budgets, unless measures to significantly increase efficiency are taken.

The cost of accommodating new information needs includes new methodological work, new data collection, new processing or dissemination methods. The bulk of costs go to data collection. Adding new variables in an existing data collection implies adaptation of statistical software (data entry, validation, processing, etc.) which can be offset with the modernisation of production (e.g. using more administrative data, better technology or sharing methodological solutions among NSIs).

Carrying out new surveys, increasing the frequency of data collections or significantly increasing sample sizes (for better precision or more disaggregation of data) has a significant impact on cost. Unless constraints on the increase in cost are given, policy options increasing the flexibility may highly impact the budgetary needs of NSIs.

It has to be mentioned that the most expensive surveys are those carried out with large samples and high frequency (e.g. LFS) or those based on diaries (e.g. HBS, HETUS). The current system of modules in LFS provides for the accommodation of new information needs. The HBS is already carried out by all NSIs even if not legally regulated. The HETUS has a low frequency (every 10 years). Thus, important cost increases are expected if higher precision is required for existing surveys. Policy options that consider mechanisms for (1) evaluating the relevance of additional information requirements and (2) simplifying the work programmes by excluding the data collections which are not deemed relevant anymore (negative priorities) would provide safeguards for un-controlled increases in cost.

NSIs would however benefit of a simpler, integrated legislation that would allow for reducing the legal work to update the statistical regulations, that would consider the whole system of social surveys when planning and designing each data collection and that would promote the modernisation of processes (e.g. by facilitating access to administrative data and other sources) and increase the methodological coherence across surveys.

Before adopting by legal acts significant modifications of the current social surveys, feasibility and other technical studies will be carried out. In particular, additional precision requirements involving higher costs, as well as the introduction of new variables in data collections, should be duly justified by their relevance to users. A minimum time of 2 years for the implementation of significant modifications should be respected. These initial costs of methodological development may be supported by the Commission, and the long term costs limited.

In practice, the joint work of Eurostat and the NSIs to integrate social surveys will continue and will have to be consolidated in implementing measures. The role of the DSS will thus be reinforced as the main technical forum of discussion on social statistics, advising the ESSC and the Commission. The balance of costs and benefits, in financial terms, is difficult to assess since it depends on the actual implementation of changes in existing surveys. Progress in the harmonised and comparable measurement of production costs in NSIs would not only improve the management at the national level, but also provide a sound basis for monitoring and evaluating the impact of any modernisation process – be it or not related to legal obligations.

Annex 4: Analytical models used in preparing the impact assessment.

This IA analysis has pushed at the maximum possible the use of quantitative data, having in mind a principle of proportionality. Due to a variety of reasons, the possibilities to use quantitative data couldn't cover all the aspects of the policy options envisaged. As a result, the use of modelling has been limited from the start and decreased even further in the course of the IA, due to the quality of the collected data. This is discussed in detail in the consultation reports.

Worth mentioning in this Annex is the approach to the baseline measurement and the impact assessment of the cost related to data production. Two aspects should be understood prior to explaining the approach as such:

1.There is a lack of existing harmonised and comprehensive data in the Member States. Targeted data collection was needed.

2.Difficulties mentioned by stakeholders to assess precisely the future costs of the various scenarios. The details of the future regulation(s) are unknown but will probably determine a major share of the impact on cost. A qualitative approach based on changes in costs was proposed.

Ad 1. Lack of existing harmonised and comprehensive data

There is very limited information available on the ´problems´ identified in the IA. There are Eurostat studies that collection such information, for example on key performance indicators , users satisfaction surveys and systematic Rolling Reviews in accordance with the binding standards approved by the Commission.

Rolling Reviews involved not only the assessment of the statistical data produced but also the process to produce them, the interactions with data providers and interactions with users of the data. They formed part of the Quality Assurance Framework developed by Eurostat in 2007. Each Rolling Review implies a thorough assessment of: (1) the process of producing data, (2) the quality of the data produced (3) the users' satisfaction and assessment of data quality, (4) the data provider's satisfaction, (5) the resources used by Eurostat and Member States.

In addition, each statistical domain is monitored through quality reports, regularly produced by Member States and analysed and compiled by Eurostat, as part of the statistical quality insurance framework. These reports encompass the following dimensions: Relevance, assessment of user needs and perceptions; Accuracy and reliability; Timeliness and punctuality; Accessibility and clarity; Coherence and comparability; Cost and burden; Confidentiality.

However, due to the limited resources, these data are not compiled often. Besides, most of the data is not observed at a level of detail (variables, sources) that would be necessary and are either valid for the whole statistical production or domain specific. Furthermore, it is recognized that “the methodologies for measuring production costs and response burden imposed by statistics are still rather heterogeneous both across various Member States and across different statistical areas as confirmed by data on total resources collected through Rolling Reviews”.

Estimating the cost of surveys is a complex task, and has been in the debate in NSIs for decades. Attempts to standardise the methods for cost accounting have been made, but they are in general focused on household surveys carried out on a project basis. For recurrent surveys carried out by NSIs, some costs are spread over the whole statistical production (e.g. management costs of NSIs, IT infrastructure, training of specialists). In the case of social statistics using administrative registers, the accounting for costs of the process of collecting such administrative records is not borne by NSIs but is still covered by government budgets. Differences in staff salaries, country size in terms of population and territory add to difficulties with obtaining comparable data. Differences in frequency of surveys require considering an annualised cost dividing the total cost of a survey by the number of years between rounds (for instance, the cost of a survey carried out every 5 years is simply annualised by dividing this total cost by 5).

Cost accounting by types of expenditure (IT, salaries, printing and sending questionnaires, etc.) are relatively easy to collect but an analytical accounting per phases of the process is not immediate. NSIs have, in general, put in place internal systems for obtaining estimates according to their own needs.

There are in general two sources of information on the costs of statistical production: budget appropriations and analytical accounting. The first source does not allow completely to discriminate costs for social statistics from those of other operations (for example, the institutional IT systems, the maintenance of a cartographic base, the general training of interviewers, etc.). The second is in general institution-specific, less comparable, and most importantly, is not accessible externally to the institutions. It was therefore necessary to collect the information directly from the NSIs.

For all these reasons, a limited collection of the costs at Member State level was necessary. This included the provision of estimates of costs by broad phases of the statistical process (design, data collection, data processing and dissemination) - described below. Some Member States were not able to provide the complete information and thus some modelling (imputation) of missing data was necessary.

Ad 2. Difficulties mentioned by stakeholders to assess precisely the future costs of the various scenarios

The details of the future regulation(s) are unknown but will probably determine a major share of the impact on cost

In the policy options defined in the main text, elements such as integration and harmonisation are defined in a general manner. However, actual details of the regulations will determine the differences in cost. As such, opening up the possibility to integrate, harmonise, use innovative methods etc. will probably not make a big difference. Without knowledge of these details, quantification is impossible. For instance, there is no clear view on how these details (definition of variables, reliability requirements, etc.) relate to available administrative data or alternative data collection methods.

In other words, the approach usually applied, using the Standard Cost Model, is not feasible. Defining exact scenario’s in which all the relevant parameters were prescribed at implementation level would be very uncertain, due to for instance the multidimensional nature of the assumptions. Moreover, cost calculations for such scenarios would have required an unproportioned effort from the NSIs.

Using the NSI survey, an effort was made to come to an adapted version, refraining from using actual ‘costs’ in absolute monetary terms and using relative terms. The NSIs were asked to assess how the cost of different steps of statistical production would evolve under each policy option. However, the findings made clear this was not feasible either. Answers were spread wide, with no logical explanation for the differences found. Apparently, the need to make assumptions on many options not yet known in details was too high to come to decent indicators.

Cost measurement approach

In order to assess the cost of the policy options, NSIs were asked to provide a baseline (quantitative) and to assess how the cost of different steps of statistical production would evolve under several scenarios (defined in general terms). For the latter, alternative approaches have been part of the NSI survey, but ultimately, a qualitative approach was used.

Costs of existing data collections have therefore been collected, as well as some breakdowns of these costs. With some caveat, these data are however giving adequate orders of magnitude of the baseline scenario. Qualitative impact on costs of policy options compared to the baseline scenario has also been sought, with mixed results.

For the breakdowns of the current costs, in order to refer to a common language for describing the statistical process, the Generic Statistical Business Process Model (GSBPM) has been used. This model is used by the Commission and NSIs to improve standardisation of statistical activities.

The costs related to the statistical business process could in theory be broken down following the Generic Statistical Business Process Model (GSBPM) and its eight phases as indicated below.

•Specify needs: Identification and confirmation of statistics’ needs, new information request, statistical outputs required to meet the user needs and data sources availability checks.

•Design activities: detailed design of statistical outputs, variables, collection methods and instruments, design frame and sample, design the statistical processing methodology to be applied (including specification of routines for coding, editing, imputing, estimating, …)

•Build: design collection instruments and its preparation and testing; new and enhance existing components and services needed for the “process” and “analyse” phases (dashboard functions and features, information services, workflow frameworks,…); components and services needed for the dissemination; technical testing of programmes and routines; test of the statistical business process (small-scale data collection, processing and analysis of the collected data); and activities to put the configured processes and services, into production ready for use (documentation about the process components – manuals, and training).

•Collect: creation of frame and sample selection (including coordination of samples between instances of the same statistical business process; quality assurance and approval of the frame,…); set up collection (including the collection strategy, planning and training activities, agreeing terms with any intermediate collection bodies,…); run and finalize the collection.

•Process: activities to clean of data and their preparation for analysis (i.e. data integration; data classification and code; data revision and validation; edit and imputation; derivation of new variables and units; calculation of weights; calculation of aggregates; and finalization of data files).

•Analyse: production and examination of statistical products and prepare them for dissemination (i.e. data transformation into statistical outputs, (e.g. production of indices, trends, recording of quality characteristics); output validation and explanation; application of disclosure controls; and finalization of outputs “fit for purpose” and reaching the required quality level.

•Disseminate: update output systems (formatting and loading data and metadata ready to be put into output databases); production of dissemination products; manage release; promote dissemination and manage users support.

•Evaluate: evaluation of the success of a specific instance of the statistical business process by: gather evaluation inputs (e.g. feedback from users, process metadata; system metrics; staff suggestions); conduct evaluation (i.e. evaluation report including recommendations for changes if appropriate); and agreement of an action plan based on the evaluation report (including also consideration of a mechanism for monitoring the impact of those actions).

In order to reduce the burden for NSIs as much as possible, these eight phases were consolidated further to broad statistical phases:

•    Design (specify needs, design activities, build);

•    Data collection (Collect);

•    Data treatment (process, analyse);

•    Dissemination (disseminate, evaluate).

Not all NSIs were able to provide cost estimates with this breakdown by broad statistical phases. Missing information was imputed by using the data of the closest country in terms of population size. No information has been collected on the disaggregation of cost according to investments for each phase of the statistical process (e.g. development of new tools for data collection) and current expenditure (e.g. salaries of interviewers carrying out the data collection). Thus, the figures presented have to be taken with care. They do not represent a detailed accountancy of the cost of social surveys.

* *

*

Overview of the costs of data production under different policy options:

Under the paragraph 6.2.3, a box has presented several model based scenarios. The yearly details of the proposed approach are presented here.

It is important to note that the model is based on several assumptions described in the paragraph 6.2.3, that includes the acceptation of the limits of the data collected at the occasion of the consultation of the stakeholders. The other assumptions are summarised in the following table, together with the results aggregated for all the data collections.

The policy options 1.1. (No change in EU legislation, defragment production processes across data collection) and 1.2 (integrate EU legislation and no change in the fragmentation of the production process) have not been subject to this model based approach, as they do not entail significant changes of costs (for the option 1.2) and because policy option 1.1 has a high risk of no efficiency gain, or at least a high risk of divergences between surveys and between Member States as described in paragraph 6.2.2.

Table 3: Assumptions and yearly details of the model based scenarios.

Annex 5: Details on the impact assessment of the options addressing the rigidity of the data collection.

1.1Introducing policy options

This section details the impacts related to the policy options that modifies the level of flexibility in the data collection of European statistics that are summarised in section 6.3.

Three main policy options have been defined:

Baseline (option 2.0): Changes to programme or technical elements require changes in legal instruments.

Policy option 2.1: Statistical programming set by Parliament / Council, technical elements more flexible to change.

Policy option 2.2: Statistical programming and technical elements relatively flexible to change.

The options presented in this section can vary independently from the policy options presented in the previous section. Possible impacts of a combination of the shortlisted policy options under this heading and on domain fragmentation are explored in more detail in Chapter 7.

1.2. Baseline (option 2.0): Programmatic and technical elements mainly fixed by basic law

The current legislative system for social statistics consists of a fixed programme for data collection, in which both the requirements on programming and technicalities are most of the time referred to the Parliament and Council. The programme is broadly set at the EU level on a multi-annual basis (European Statistical Programme) with annual work programmes, and at the national level in a similar way. Calendars of the different surveys are not integrated, but recent efforts have been made at the DSS level to map the entire system of social statistics. As a result, each element that has to be changed, either technical or programmatic in nature, needs to undergo a lengthy legislative process that fully involves the European Parliament and Council (in addition to “internal” technical discussions in working groups and the European Statistical Committee).

Effectiveness

The continuation of the status quo has a limiting impact on the effectiveness of the collection of social statistics. An important risk that affects the effectiveness of the current system revolves around the quality of decision-making. Due to the indiscriminate mix of programming (policy) and very technical implementation matter (technicalities) in legislative packages, political attention is diverted from the essential task to balance European information requirements against cost and response burden. As a result, decision-making on programmatic elements of social statistics is relatively rigid and prevents an adequate response of social data collection to societal needs. It currently takes a considerable amount of time to start the production of new data in response to newly formulated policy challenges. New data needs may require changes in the definition of variables collected in the social surveys, a new relevant breakdown, cross-tabulation from different sources or other methodological innovations, in addition to changes in IT-systems for processing and disseminating the results at the level of NSIs. Any of these technical changes has to be reflected in an amendment of a legal act, with all the relevant formal processes associated to it. As a result, many data users report an increasing use of other non-official sources, such as the Eurobarometer or other data sources developed for instance by EU agencies. The methodology of these data collections are not based on a Parliament / Council Regulation and can be adapted to the needs of specific users with relative ease.

The status quo limits the possibility for Eurostat to act on immediate quality issues, as allowed under the ‘statistical Regulation’ (Regulation 223/2009). Even though currently no major concerns were identified with regard to the quality of social statistics due to the flexibility, stakeholders expect that with the increasing use of new and innovative methods, it will also become more important for Eurostat to be able to act swiftly and effectively on emerging quality issues.

The current legislative environment does not facilitate the development of innovative methods and procedures for social statistics. Due to the role of the Council and Parliament in both setting legislation of the programmatic and many technical elements, the legal system is relatively rigid and limits the possibilities to introduce innovative elements into legislation.

Efficiency

Due to the legal structure of the social statistics system, it is not likely to adapt to newly emerging possibilities and possible cost-savings. The availability of innovative methods and increasing pressures on the budgets of NSIs across the EU contribute to an increasing use of innovative and cost-saving methods and processes at the national level (e.g. increased use of administrative registers). Capacities for reacting to and framing these developments will either prevent them or lead to divergence in the data coming from Member States.

Despite these expected increases in inefficiencies, NSIs also pointed out the need for stability in data collection, both in programmatic and in technical terms. Frequent changes in data collection needs or technical requirements may contribute to disruptions in existing time series, add to survey burden and increase the work required by NSIs to implement new requirements. As the baseline scenario highly favours the status quo, the short-term costs for NSIs remain relatively low.

1.3. Policy option 2.1: Programmatic elements fixed by law / technicalities more flexible

Policy alternative 2.1 suggests splitting the flexibility for the programmatic requirements and for the technical requirements. This policy option proposes to focus the political attention of the Parliament and Council on the programmatic content of data collection, while leaving the technical specification outside EU legislation, to be discussed and agreed upon within the ESS. In a way, this policy option implements the statistical governance based on the principle of division of responsibilities as envisioned in the EU's statistical law (223/2009). This Regulation formulates the role for Parliament and Council in statistical programming (see article 13), while mainly defining a role for the Commission and implementing partners in more technical and quality aspects, most of which would be dealt with by governance supported by technical discussions in the relevant forums (DSS, ESSC and working groups). When talking about technical quality aspects, this would concern elements, such as sample size and precision assessment, statistical definitions, geographical detail, quality reports, sampling frames characteristics, transmission aspects, etc.

Effectiveness

This policy option contributes positively to the responsiveness of social statistics to societal needs. The European Parliament and Council are left to define the programmatic priorities of data collection in regular multi-annual time intervals. An important difference with the baseline scenario is that the political actors no longer have to consider technical details of statistical methodology. As a result, these political actors will be better enabled to balance European information requirements (data needs) against cost and response burden. In line with this, the consulted EU policymakers reported the need for better accommodation of emerging informational needs in the European statistical programmes (multi-annual and annual), but also in the national statistical plans. The removal of the technical content from the political level contributes positively towards focusing political attention on such needs.

The policy option also improves the possibilities for Eurostat to ensure the quality of statistics collected by all Member States. In cooperation with its implementing partners (NSIs), Eurostat will be able to respond more rapidly and efficiently to certain quality issues, as its technical changes to the regulatory system would not have to involve the Parliament and Council.

Thirdly, this policy option is more effective towards facilitating the introduction and use of innovative methods, as developed by some NSIs into EU legislation. Based on its frequent interactions in forums and workshops, Eurostat can effectively and authoritatively work together with its implementing partners and respond immediately to their needs.

Efficiency / impacts on stakeholders

This policy option creates the possibilities for the system of social statistics to benefit from differences in cost structure of various data collection methods. The additional flexibility of setting technical elements of statistical data collection allows the gradual introduction of innovative methods to replace methodologies that are no longer cost-efficient. As such, the flexibility of technical requirements has a positive impact on the future efficiency of social data collection. In the short-term however, a majority of NSIs expressed their concern for such flexible arrangements with regards to technical arrangements in the production of social statistics, even if they were included in the decision-making process. According to most NSI’s opinions, any reform of the legal environment should carefully tackle the issue of programming, ensuring mid-term predictability and responsiveness.

For NSIs, the term flexibility often equals “programmatic flexibility”, i.e. additional or changed data demands in terms of coverage or calendar of data collections, which are accompanied by considerable implementation costs that are usually unforeseen in their budgets. For this reason, NSIs warned during the consultation for flexibility and indicated the need for NSIs to have the possibility to explore optimal solutions for data production in their national contexts. Technical flexibility is seen as a risk by some NSIs if costly requirements are made, such as an increased precision or greater disaggregation of the data, which lead to increased data collection costs.

No specific impacts that result from increased “technical flexibility” have been identified for data users. For respondents and participants in social statistics, this policy option does not have substantial impacts. Efficiency gains can be obtained if flexible arrangements are used in integrating various modules. However, these efficiency gains for respondents are modest.

1.4. Policy option 2.2: Programmatic / technical elements flexible

This policy option introduces additional flexibility for the technical specifications of data collection, but goes beyond policy option 2.1 by also increasing the flexibility on the programmatic requirements of social statistics. It would introduce a legal framework, to be approved by the Parliament and Council, which broadly defines the types of issues that belong to the umbrella of social statistics. The more specific programmatic contents would be set by Eurostat, in collaboration with its implementing partners. This would provide additional flexibility in responding to programmatic needs of the users.

Effectiveness

This policy option would increase the possibilities to respond rapidly to newly rising phenomena with social statistics. As such, this policy option contributes to a higher potential for responsiveness of societal needs. In order to collect new data fit for policy purposes, it is necessary to have a flexible system that allows rapid responses. The European policymakers interviewed for this IA further underlined this need.

However, at the same time, this policy option limits the possibility of the Parliament or Council introducing new programmatic elements, as these would be set in detail by Eurostat, in cooperation with the NSIs. Therefore, the impact on responsiveness would theoretically be increased, but at the same time limited due to the lack of political involvement and mandate given to statisticians.

The impacts of the flexibility of the technical aspects are not different between this policy option and option 2.1. This policy option has therefore, no difference in impact for Eurostat’s ability to ensure the quality of data or the introduction and use of innovative methods by NSIs. It enables Eurostat to respond rapidly and efficiently to quality issues and also generates a similar effect on the introduction and use of innovative methods by NSIs, which is overall favourable.

Efficiency / impacts on stakeholders

The major impact on efficiency of production of social statistics depends on the flexibility of the technical elements of legislation and are therefore, not substantially different than policy option 2.1.

However, the added flexibility in terms of programmatic contents introduces a considerable uncertainty for NSIs, who associated this policy option during the consultation with risks of very high implementation costs. The risks perceived are so high that they reject this option. Such implementation costs are for instance, those with regards to the design of new data collections, the data collection and the transmission of data. For this reason, NSIs underlined the importance that additional flexibility of programmatic provisions is also accompanied by the development of an operational method for negative priority identification, in which users and producers of national and European data would discuss which elements could be discontinued. As an indication of the costs related to flexible programming, NSIs indicated in the stakeholder consultation that the development of additional survey modules generally requires two years of preparatory work.

However, the risks perceived by the NSIs are stakeholders are so high that they reject this option.

For data users, the additional flexibility provided by this policy option would theoretically expand their possibilities to request additional data collection, and therefore, have a positive impact. In a similar logic, additional flexibility may also be used to decrease the number of data requests. In the ICT-HH survey for instance, indicators that have become the standard (use of mobile phones, or internet connection at home) may be removed when no longer considered informational. However, as noted above, negative priority setting is not likely. The increased flexibility for programmatic specification does not necessarily provide a substantial impact for respondents; this only becomes the case if the scope of social statistics is considerably broadened.

1.5. Summary of impacts

In addition to the summary of impacts presented under the item 6.3.2, under all policy options 2, users underlined the importance of securing outputs by addressing for instance the risks of breaks of time series and discontinuation of information.

(1) The ESS is the partnership between the Community statistical authority, which is the Commission (Eurostat), and the national statistical institutes (NSIs) and other national authorities (ONAs) responsible in each Member State for the development, production and dissemination of European statistics.

(2)  See the box on statistical quality under section 2.1

(3) http://www.eesc.europa.eu/resources/docs/jean-claude-juncker---political-guidelines.pdf http://europa.eu/rapid/press-release_SPEECH-14-546_en.htm https://ec.europa.eu/commission/2014-2019/thyssen/announcements/speech-general-assembly-social-platform_en http://ec.europa.eu/economy_finance/assistance_eu_ms/greek_loan_facility/pdf/assessment_social_impact_en.pdf http://ec.europa.eu/commission/2014-2019/thyssen/announcements/keynote-speech-annual-conference-director-generals-national-statistical-institutes-eu-member-states_en http://data.consilium.europa.eu/doc/document/ST-14102-2015-INIT/en/pdf  See for instance  ,  ,  , , ,  

(4) http://ec.europa.eu/eurostat/web/quality/evaluation See  

(5) http://ec.europa.eu/smart-regulation/roadmaps/docs/2014_estat_002_social_statistics_en.pdf See:

(6)

Eurostat public consultation webpage: http://ec.europa.eu/eurostat/about/opportunities/consultations/iess  

Open public consultation report: http://ec.europa.eu/eurostat/documents/10186/7237349/Open-public-consultation-report.pdf  

Consultation of the data users:   http://ec.europa.eu/eurostat/documents/10186/7237349/Data-users-consultation-report.pdf

Consultation of the data producers:  http://ec.europa.eu/eurostat/documents/10186/7237349/Data-producers-consultation-report.pdf

(7)  Consultations and analysis were carried on at the occasion of the various institutionalised events and procedures within the ESS (ESSC meetings, directors group meetings, working groups, etc.) as well as with the users (bilateral meetings, annual systematic hearings, ISC, etc.).

(8) http://ec.europa.eu/eurostat/documents/64157/4392716/ESS-QAF-V1-2final.pdf/bbf5970c-1adf-46c8-afc3-58ce177a0646   

(9) Other problems, beyond the direct Eurostat remit are also to mention: budget cuts for statistics in Member States, increasing disinterest of citizens decreasing the response rates, practical impossibilities to access some specific sub-populations, like those affected by extreme poverty and small minorities, or to measure cross border phenomena.

(10)  From here, this notation refers to the relevance on a 0-10 scale as valued by the NSIs in the consultation (survey); in this case it refers to the magnitude of this problem, the degree to which slow capacity of the ESS to adapt to new information needs will cause loss of relevance of European statistics.

(11)  http://ec.europa.eu/eurostat/product?code=KS-RA-13-020

(12) http://ec.europa.eu/smart-regulation/refit/index_en.htm  See:  

(13) http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2009:0404:FIN:En:PDF  See:

(14) http://ec.europa.eu/eurostat/web/ess/eurostat/management-issues See:   

(15) https://www.destatis.de/EN/AboutUs/Events/DGINS/Document_Memorandum.pdf;jsessionid=B466B23B69A315B330112553404E2011.cae2?__blob=publicationFile  See: 

(16) https://www.destatis.de/EN/AboutUs/Events/DGINS/Document_PaperEUROSTAT.pdf?__blob=publicationFile  See:  

(17) http://ec.europa.eu/eurostat/web/european-statistical-system/statistical_programmes See  

(18) http://ec.europa.eu/eurostat/cros/sites/crosportal/files/Karlberg_etal%20no892unblinded.pdf  “A European toolbox for a modular design and pooled analysis of social survey programmes” by Martin Karlberg (Martin.Karlberg@ec.europa.eu), Fernando Reis, Cristina Calizzani, Fabrice Gras, in Proceedings of NTTS 2015. . 

(19)  In order to estimate the additional costs of introducing HETUS to the remaining Member States, the current costs (€24.9 million see section 2.4) for the current 14 Member States are taken as starting point. It is assumed that extrapolating these costs to the remaining 14 Member States would increase the costs proportionally, in this case by another €24.9 million, which on an annual basis is estimated at €2.5 million. Even though relatively small at the EU level (1.4%) of the annually estimated costs, these are borne unequally among a smaller number of Member States.

(20) http://ec.europa.eu/eurostat/web/quality/general-evaluation-results See

(21) The five key performance indicators are the following: Number of data extractions made by external users from Eurostat reference databases (EuroBase and Comext) via the Eurostat website, Percentage of users that rate as "Very good" or "Good" the overall quality of European statistics, Percentage of users that rate as "Very good" or "Good" the timeliness of European statistics for their purposes, Percentage of users that rate as "Very good" or "Good" the comparability of European statistics among regions and countries, Residual error rate (RER)

(22) http://ec.europa.eu/eurostat/web/quality/general-evaluation-results See

(23) http://ec.europa.eu/eurostat/web/quality/evaluation Each Rolling Review is domain specific and implies a thorough assessment of: (1) the process of producing data, (2) the quality of the data produced (3) the users' satisfaction and assessment of data quality, (4) the data provider's satisfaction, (5) the resources used by Eurostat and Member States More detail could be found under .

(24) http://ec.europa.eu/eurostat/ramon/nomenclatures/index.cfm?TargetUrl=LST_NOM&StrGroupCode=LEX_MANUAL&StrLanguageCode=EN  See 

(25) http://ec.europa.eu/smart-regulation/roadmaps/docs/2014_estat_002_social_statistics_en.pdf See:

(26) http://ec.europa.eu/eurostat/web/quality/general-evaluation-results See  

(27) The five key performance indicators are the following: Number of data extractions made by external users from Eurostat reference databases (EuroBase and Comext) via the Eurostat website, Percentage of users that rate as "Very good" or "Good" the overall quality of European statistics, Percentage of users that rate as "Very good" or "Good" the timeliness of European statistics for their purposes, Percentage of users that rate as "Very good" or "Good" the comparability of European statistics among regions and countries, Residual error rate (RER)

(28) http://ec.europa.eu/eurostat/web/quality/general-evaluation-results See

(29) http://ec.europa.eu/eurostat/web/quality/quality-reporting See:

(30) http://ec.europa.eu/eurostat/web/quality/evaluation  More detail could be found under

(31) http://ec.europa.eu/eurostat/documents/42577/4167614/ESAC+opinion+3+Dec+2015/421abf9c-4300-445f-a840-f13cadeee306 See:

(32) http://ec.europa.eu/eurostat/documents/42577/6906243/ESS+vision+2020_V2/35911206-3968-4548-adcc-882c797d9ca4  See:  

(33) https://www.destatis.de/EN/AboutUs/Events/DGINS/Document_Memorandum.pdf?__blob=publicationFile See 

(34)

Eurostat public consultation webpage: http://ec.europa.eu/eurostat/about/opportunities/consultations/iess  Open public consultation report: http://ec.europa.eu/eurostat/documents/10186/7237349/Open-public-consultation-report.pdf

Consultation of the data users: http://ec.europa.eu/eurostat/documents/10186/7237349/Data-users-consultation-report.pdf

Consultation of the data producers:  http://ec.europa.eu/eurostat/documents/10186/7237349/Data-producers-consultation-report.pdf