ANNEX

Content and evaluation criteria of the report on the quality of structural statistics on labour costs and earnings

The reports on quality shall include information on all six dimensions of the European Statistical System (ESS) quality definition, namely relevance, accuracy, timeliness and punctuality, accessibility and clarity, comparability, and coherence.

1.   Relevance

‘Relevance’ denotes the degree to which statistics meet current and potential users' needs. It refers to whether all statistics that are needed are produced and the extent to which concepts used (definitions, classifications, etc.) reflect user needs.

The quality reports on structural statistics on labour costs and earnings respectively shall cover:

—	a summary of national core users,

—	a description of their main needs including an assessment of their level of satisfaction with the data offered,

—	a description of any gap (completeness) or part which is not considered relevant any longer by the users (redundancy). With respect to completeness, missing variables and missing breakdowns of the variables shall be clearly described.

The coverage of the items referring to national core users and their main needs, including the provision of information on user satisfaction, is optional.

By way of derogation from Article 2, the optional information on the level of satisfaction with the data offered, if provided, shall be transmitted to Eurostat at the latest 36 months after the end of the reference period.

2.   Accuracy

‘Accuracy’ in the general statistical sense denotes the closeness of computations or estimates to the exact or true values of the variables under consideration.

2.1.   Sampling errors

As an indication of accuracy, the coefficient of variation (1) shall be calculated and transmitted for key variables of both surveys.

(1)

In the case of labour cost statistics the coefficient of variation shall be calculated and transmitted for the variables   ‘Annual labour costs’  (2) and   ‘Hourly labour costs’  (3). Apart from the coefficients of variation for the population as a whole, separate coefficients of variation should also be made available for both variables for the following individual breakdowns: — NACE sections, — NUTS level 1 (if appropriate), — size band (1-9 (if appropriate), 10-49, 50-249, 250-499, 500-999, 1 000+).

(2)

In the case of structural statistics on earnings the coefficient of variation shall be calculated and transmitted for the variables   ‘Gross earnings in the reference month’  (4) and   ‘Average gross hourly earnings in the reference month’  (5). Apart from the coefficients of variation for the population as a whole, separate coefficients of variation should also be made available for both variables for the following individual breakdowns: — full-time (separately for men and women) and part-time employees, — NACE section, — occupation (ISCO-88 at the 1-digit level), — age band (under 20, 20-29, 30-39, 40-49, 50-59, 60 and over), — NUTS level 1 (if appropriate), — level of education (ISCED 0 to 6), — size band of the enterprise (1-9 (if appropriate), 10-49, 50-249, 250-499, 500-999, 1 000+).

The breakdown by level of education is optional.

In addition, the Member States shall transmit a list of those cells of the multidimensional tables of the publication programme at European level as agreed with the Member States which are deemed not to be sufficiently reliable.

If non-probability sampling is used, a description of the possible sources of imprecision due to the sampling technique used and their effect on the estimations, if available, shall be provided instead.

2.2.   Non-sampling errors

2.2.1.   Coverage errors

The quality reports on structural statistics on labour costs and earnings respectively shall include the following information on coverage:

—	description of any difference between the reference and the study population,

—	estimated rates of under- and over-coverage (6) for the reference population.

2.2.2.   Measurement and processing errors

The quality reports on structural statistics on labour costs and earnings respectively shall include the following information on measurement and processing errors (7):

—	information on variables with non-negligible measurement and processing errors,

—	information on main sources of (non-negligible) measurement and processing errors, their impact on the accuracy of the estimations and, if available, on methods applied for correction.

2.2.3.   Non-response errors

The quality reports on structural statistics on labour costs and earnings respectively shall include the following information on non-response errors:

—	unit response rate (8),

—	item imputation rate (9) and the effect of the imputation on the accuracy of the estimations for the variables ‘Annual labour costs’ and ‘Gross earnings in the reference month’ respectively as defined in point 2.1,

—	overall imputation rate (10). If the information is not available for all mandatory variables, the calculation shall be based on those mandatory variables for which the necessary information is available.

2.2.4.   Model assumption errors

If modelling is used, the quality reports on structural statistics on labour costs and earnings respectively shall include a description of the models used and an indication of their effects on the estimates (e.g. estimated share of the variable's total), at least for the variables ‘Annual labour costs’ and ‘Gross earnings in the reference month’ respectively as defined in point 2.1.

Particular emphasis shall be given to models for the correction of non-sampling errors such as coverage of units of all requested size classes or NACE sections, imputation or grossing-up to correct for unit non-response.

3.   Punctuality and timeliness

‘Punctuality’ refers to the time lag between the release date of data and the target date when it should have been delivered, for instance with reference to dates announced in official release calendars, laid down by Regulations or previously agreed among partners.

‘Timeliness’ of information reflects the length of time between its availability and the event or phenomenon it describes.

3.1.   Punctuality

In order to understand and to remove problems related to punctuality, information shall be delivered on the process of the survey implementation at national level with a special emphasis on the correspondence between scheduled and actual dates:

—	deadlines for the respondents to reply, also covering recalls and follow-ups,

—	period of the fieldwork,

—	period of data processing,

—	dates of publication of first results.

3.2.   Timeliness

The quality reports on structural statistics on labour costs and earnings respectively shall contain information on the length of time between the release of data and the reference period of the data at national level.

4.   Accessibility and clarity

‘Accessibility’ refers to the physical conditions in which users can obtain data: where to go, how to get access, delivery time, convenient marketing conditions (copyright, etc.), availability of micro or macro data, various formats and data carriers (paper, files, CD-ROM/DVD, Internet), etc.

‘Clarity’ refers to the degree of comprehensibility, including data information environment, i.e. whether data are accompanied by appropriate metadata, illustrations such as graphs and maps, whether information on their quality is available (including limitations on use) and the extent to which additional assistance is provided.

By way of derogation from Article 2, information on accessibility and clarity shall be transmitted to Eurostat at the latest 36 months after the end of the reference period.

4.1.   Accessibility

The quality reports on structural statistics on labour costs and earnings respectively shall contain the following information on the dissemination modes of the results:

—	references for core results publications, including those with commentary in the form of text, graphs, maps, etc,

—	information on what results, if any, are sent to reporting units included in the sample.

4.2.   Clarity

The quality reports on structural statistics on labour costs and earnings respectively shall contain the following information on the comprehensibility of the results and the availability of metadata:

—	description of and references for metadata provided,

—	references for core methodological documents relating to the statistics provided,

—	description of main actions carried out by the national statistical services to inform users about links to the data.

5.   Comparability

‘Comparability’ aims at measuring the impact of differences in applied statistical concepts and measurement tools/procedures when statistics are compared between geographical areas, non-geographical domains, or over time.

5.1.   Geographical comparability

The quality reports on structural statistics on labour costs and earnings respectively shall contain information on differences between national and European concepts, especially on the definition of statistical units, populations, reference times, classifications and definitions of variables, and their effects on the estimates.

5.2.   Comparability over time

The quality reports on structural statistics on labour costs and earnings respectively shall contain information on changes in definitions, coverage and methods compared with previous surveys, and their effects on the estimates. Information on changes in definitions, coverage and methods, however, is not requested when these changes are the result of amendments to Community legislation.

6.   Coherence

‘Coherence’ of statistics is the extent to which they can be reliably combined in different ways and for various uses. It is, however, generally easier to identify cases of incoherence than to prove coherence.

Sources with similar or even identical variables to structural statistics on earnings and labour costs are the Labour Force Survey (LFS), Structure of Business Statistics (SBS), Labour Cost Index (LCI) and National Accounts (NA). The quality reports on structural statistics on labour costs and earnings respectively shall contain comparisons of data on the following variables from these sources, in total and broken down by NACE sections, and give indications of the reasons if the values differ considerably.

(1)

In the case of structural statistics on labour costs, comparisons shall be made between: — the number of hours actually worked in the reference year, expressed per employee (11), and the average actual hours worked in the main job per year of the LFS, again expressed per employee, — the variable ‘wages and salaries’, expressed per employee (12), and the same variable, per employee of the SBS, — the average annual growth rates of the variable ‘hourly labour costs’  (13) and the average annual growth rate of the unadjusted LCI. The growth rates should refer to the reference year of the survey and that of the previous survey, — the variable ‘compensation of employees’, expressed per employee (14), and the same variable of the NA.

(2)	In the case of structural statistics on earnings, comparisons shall be made between: — the variable ‘gross annual earnings in the reference year’, expressed per employee (15), and the variable ‘wages and salaries’, per employee, of the NA.

---

(1)  The coefficient of variation is the ratio of the square root of the variance of the estimator to its expected value. It is estimated by the ratio of the square root of the estimate of the sampling variance to its estimated mean. Both numerator and denominator of the ratio defining the coefficient of variation should be provided, together with the resulting coefficient of variation. The estimation of the sampling variance must take the sampling design into account.

(2)  Code D (total labour costs), the sum of the values of codes D1, D2, D3 and D4 minus D5 in Annex I to Regulation (EC) No 1726/1999.

(3)  Code D (total labour costs), the sum of the values of codes D1, D2, D3 and D4 minus D5, divided by the value of code B1, in Annex I to Regulation (EC) No 1726/1999.

(4)  Code 4.2 (total gross earnings in the reference month) in Annex I to Regulation (EC) No 1916/2000.

(5)  Code 4.3 in Annex I to Regulation (EC) No 1916/2000.

(6)   ‘Under-coverage’ refers to units erroneously not included in the frame. ‘Over-coverage’ relates either to units that are in fact out of scope or to units that do not exist in practice.

(7)  Measurement errors are errors that occur at the time of data collection. There are a number of sources of measurement errors, including the survey instrument, the respondent, the information system, the mode of data collection and the interviewer. Processing errors are errors in post-data-collection processes such as data entry, coding, keying, editing, weighting and tabulating.

(8)  The unit response rate is the ratio, expressed as a percentage, of the number of responses to the total number of in-scope respondents.

(9)  The item imputation rate gives the imputed values for a specific variable as a percentage of the total number of values for this variable.

(10)  The overall imputation rate gives the imputed values for all variables as a percentage of the total number of values for these variables.

(11)  Code B1, divided by the value of code A1, in Annex I to Regulation (EC) No 1726/1999.

(12)  Code D11, divided by the value of code A1, in Annex I to Regulation (EC) No 1726/1999.

(13)  As defined in point 2.1.

(14)  Code D1, divided by the value of code A1, in Annex I to Regulation (EC) No 1726/1999.

(15)  Code 4.1, divided by the number of employees, in Annex I to Regulation (EC) No 1916/2000.