52024SC0036

Result and impact indicators consist of more qualitative types of indicators, which are a short- and long-term consequence of the application of the Regulation. These indicators should be collected by means of an online survey and/or interviews with national competent authorities.

6.Benchmark and targets

The benchmark against which progress will be measured is the baseline situation following the entry into application of the Regulation. For existing data sources, the baseline will be defined based on the most recently collected data from the entry into application of the Regulation; whereas, for new data sources, the first collected data will serve as baseline.

With regard to targets (a proxy for success criteria), given the transnational nature of the dissemination of terrorist content online and the different data collection methods and sources, it is considered more effective to measure aggregated progress across Member States against an average baseline at Union level. When possible, progress will be measured for each Member State against its own baseline.

7.Table: monitoring framework for the Regulation

7.1Data to be collected from Member States’ competent authorities

ID KPI	KPIs 2	Type of indicator	Monitoring areas	Sub-monitoring area	Level of reporting obligation	Type of data	Data sources	Units of measurement	Units of measurement - description
KPI 1	Establishment of clear rules by each Member State on penalties applicable to infringements of this TCO Regulation by HSPs and implementation of measures necessary to ensure that they are implemented (Art. 18.3)	Output	Safeguard mechanisms and fundamental rights (freedom to conduct a business)	Penalties and administrative or judicial review proceedings	Level 1	Primary	NCA consultation / survey	0-1 dichotomous scale (Yes or No)	- 0 means that the Member State did not establish clear rules on penalties applicable to infringements of this TCO Regulation for HSPs and did not implement measures necessary to ensure that such rules are implemented by HSPs - 1 means that the Member State established clear rules on penalties applicable to infringements of this TCO Regulation for HSPs and implemented measures necessary to ensure that such rules are implemented by HSPs
KPI 2	Member States shall ensure that the NCA(s), when deciding whether to impose a penalty and when determining the type and level of penalty, take into account all relevant circumstances (list the elements) (Art. 18.2)	Output	Safeguard mechanisms and fundamental rights; freedom to conduct a business	Penalties and administrative or judicial review proceedings	Level 1	Primary	NCA consultation / survey	Yes/no and 0-7 discrete scale	Yes/No (have/have not taken into account all circumstances) - 0 means that the NCA, when deciding whether to impose a penalty and when determining the type and level of penalty, does not consider the relevant circumstances (none of the elements listed in Art. 18.2) - 7 means that the NCA, when deciding whether to impose a penalty and when determining the type and level of penalty, takes into account all relevant circumstances (all the elements listed in Art. 18.2)
KPI 3	Member States to ensure that a systematic or persistent failure to comply with obligations pursuant to Article 3(3) (HSP to remove TCO or disable access as soon as possible, within 1 hour) is subject to financial penalties of up to 4 % of the HSP’s global turnover of the preceding business year.	Output	Safeguard mechanisms	Penalties and administrative or judicial review proceedings	Level 1	Primary	NCA consultation / survey	0-1 dichotomous scale (Yes or No)	- 0 means that the Member State does not ensure that a systematic or persistent failure to comply with obligations pursuant to Article 3(3) (HSP to remove TCO or disable access as soon as possible, within 1 hour) is subject to financial penalties of up to 4 % of the HSP’s global turnover of the preceding business year. - 1 means that the Member State ensures that a systematic or persistent failure to comply with obligations pursuant to Article 3(3) (HSP to remove TCO or disable access as soon as possible, within 1 hour) is subject to financial penalties of up to 4 % of the HSP’s global turnover of the preceding business year.
KPI 4	HSP reporting to the NCA within 3 months of receipt of the decision and on an annual basis thereafter on the specific measures that it has taken and that it intends to take in order to comply with paragraphs 2 and 3 of article 5 (Art. 5.5)	Output	Public security	Procedures for removal orders including cross-border	Level 1	Secondary	NCA Transparency Report	0-1 dichotomous scale (Yes or No)	- 0 means that the HSP does not always report to the NCA within 3 months of receipt of the decision and on an annual basis thereafter on the specific measures that it has taken and that it intends to take in order to comply with paragraphs 2 and 3 of Article 5 - 1 means that the HSP always reports to the NCA within 3 months of receipt of the decision and on an annual basis thereafter on the specific measures that it has taken and that it intends to take in order to comply with paragraphs 2 and 3 of Article 5
KPI 5	NCAs' activity(ies) or programme(s) to raise awareness among HSPs of the importance and risk of TCO (exchange of good practices and challenges)	Output	Public security	Transparency and awareness from EC, NCAs and HSPs	Level 3	Primary	NCA consultation / survey	0-1 dichotomous scale (Yes or No)	- 0 means that the no awareness-raising activity or programme was put in place by the NCA regarding the importance and risks of terrorist content online; - 1 means that at least one awareness-raising activity or programme was put in place by the NCA regarding the importance and risks of terrorist content online.
KPI 6	Consideration by NCAs and MS of the proportionality of the risk and resources of the HSP in the assessment of proactive measures taken by HSPs to moderate TCO (by performing risk assessment for instance)	Output	Fundamental rights (freedom to conduct a business)	Competitiveness and effect on SMEs	Level 3	Primary	NCA consultation / survey	1-5 Likert scale	- 1 means strongly disagree; - 5 means strongly agree.
KPI 7	The number of effective procedures implemented by the Member States to enable the HSP and content providers to have the right to an effective remedy (Art. 9.3)	Result	Safeguard mechanisms and fundamental rights (freedom of expression and information, the respect for private life, the protection of personal data)	Remedies mechanism	Level 2	Primary	NCA consultation / survey	(2) 0-N discrete scale (absolute number of procedures for an effective remedy)	- 0 means that the Member States did not implement any effective procedures to enable HSPs and content providers to have the right to an effective remedy. - N means that the Member States implemented N effective procedures to enable HSPs and content providers to have the right to an effective remedy Quartile computation of the indicator. The quartiles will have to be computed once the data will be collected.
KPI 8	The number of cases in which removal orders and decisions taken were subject to administrative or judicial review proceedings and information on the outcome of the relevant proceedings (Art. 8.1 (c)) (by NCA)	Result	Safeguard mechanisms	Penalties and administrative or judicial review proceedings	Level 1	Secondary	NCA Transparency Report	(7) 0-N discrete scale (absolute number of times the removal orders were subject to admin or judicial proceedings)	- 0 means that none of the removal orders and decisions taken were subject to administrative or judicial review proceedings - N means that N times, the removal orders and decisions taken were subject to administrative or judicial review proceedings
KPI 9	The number of decisions imposing penalties by NCA and a description of the type of penalty imposed (Art. 8.1 (d))	Result	Safeguard mechanisms and fundamental rights (freedom to conduct a business)	Penalties and administrative or judicial review proceedings	Level 1	Secondary	NCA Transparency Report	(8) 0-N discrete scale (absolute number of decisions taken to impose a penalty by NCA)	- 0 means that no decision was taken to impose a penalty by NCA and thus no description of the type of penalty imposed was provided. - N means that N decisions were taken to impose a penalty by NCA and thus a description of the type of penalty imposed was provided.
KPI 10	Number of procedures for removal orders and referrals sent to/ received by the HSP as well as cross-border ones (Art. 7.3(c))	Result	Public security	Procedures for removal orders including cross-border	Level 1	Secondary	HSP & NCA Transparency Report	(9) 0-N discrete scale (absolute number of removal orders / referrals received by HSPs)	- 0 means that no procedure for removal orders and referrals was received by the HSP including cross-border ones - N means that N procedure(s) for removal orders and referrals were received by the HSP including cross-border ones
KPI 11	Number of cases in which the content which had been subject to a removal order was reinstated by the HSP following a reasoned decision finding that the concerned content was not considered as terrorist (Art. 4.2)	Result	Public security	Procedures for removal orders including cross-border	Level 1	Secondary	NCA Transparency Report	(14) 0-N discrete scale (absolute number of times content was reinstated on platform following a decision)	- 0 means that a HSP did not reinstate a content on the platform which had been subject to a removal order (following a reasoned decision finding that the concerned content was not considered as terrorist) - N means that a HSP reinstated N times a content on the platform which had been subject to a removal order (following a reasoned decision finding that the concerned content was not considered as terrorist)
KPI12	Frequency at which the HSP informed the NCA of the removal of terrorist content or the disabling of access to the terrorist content (indicating the specific time of the removal) (Art. 3.6)	Result	Public security	Procedures for removal orders including cross-border	Level 1	Secondary	NCA Transparency Report	0-1 dichotomous scale (Yes or No)	- 0 means that the HSP does not always inform the NCA of the removal of terrorist content or the disabling of access to the terrorist content) - 1 means that the HSP always informs the NCA of the removal of terrorist content or the disabling of access to the terrorist content)
KPI 13	Frequency at which the HSP informed the NCA of its incapacity (due to force majeure or de facto impossibility not attributable to the HSP) to comply with the removal orders (Art 3.7)	Result	Public security	Procedures for removal orders including cross-border	Level 1	Secondary	NCA Transparency Report	0-1 dichotomous scale (Yes or No)	- 0 means that the HSP does not always inform the NCA of its incapacity (due to force majeure or de facto impossibility not attributable to the HSP) to comply with the removal orders - 1 means that the HSP always inform the NCA of its incapacity (due to force majeure or de facto impossibility not attributable to the HSP) to comply with the removal orders
KPI 14	Number of Member States connected to Europol for channelling removal orders and referrals	Result	Public security	Procedures for removal orders including cross-border	Level 3	Primary	NCA consultation / survey	0-1 dichotomous scale (Yes or No)	- 0 means that the MS is not connected to Europol for channelling removal orders and referrals; - 1 means that the MS is connected to Europol for channelling removal orders and referrals.
KPI 15	Number of HSPs that the NCA (established in the same country as the HSP) considers to be exposed to TCO (based on number of removal orders issued by the HSP, at least 2 removal orders should have been issued by the HSP to be considered as exposed to terrorist content) (Art. 5.4 (a))	Result	Public security	HSPs policy for moderating terrorist content and its functioning, i.e., specific measures	Level 2	Primary	NCA consultation / survey	(16) 0-N discrete scale (Absolute number of HSP(s) considered as exposed to TCO)	-0 means that no HSP is considered to be exposed to TCO; - N means that the NCA considers there are N HSPs exposed to TCO.
KPI 16	Number of NCAs publishing annual transparency reports on TCO	Result	Public security	Transparency and awareness from EC, NCAs and HSPs	Level 2	Secondary	NCA Transparency Report available online	(18) 0-N discrete scale (absolute number of annual transparency reports published by NCAs)	-0 means that no annual transparency reports were published; - N annual transparency reports were published by NCAs
KPI 17	Additional costs due to additional resources needed related to the introduction of removal orders for NCAs and HSPs	Impact	HSPs	Administrative burden and costs on public authorities related to the TCO Regulation	Level 3	Primary	NCA consultation / survey	Yes/No 0-3 discrete scale	Yes/No - 0 means that no additional costs were incurred by the NCA due to additional resources needed related to the introduction of removal orders; - 3 means that several costs were incurred by the NCA due to additional resources needed related to the introduction of removal orders.
KPI 18	Additional costs due to the reporting obligations of the TCO Regulation for NCAs	Impact	NCAs	Administrative burden and costs on public authorities related to the TCO Regulation	Level 3	Primary	NCA consultation / survey	Yes/No 0-3 discrete scale	Yes/No - 0 means that no additional costs were incurred by the NCA related to the reporting obligations of the Regulation; - 3 means that several costs were incurred by the NCA related to the reporting obligations of the Regulation.
KPI 19	Costs related to the risk assessment done by NCA	Impact	NCAs	Competitiveness and effect on SMEs	Level 3	Primary	NCA consultation / survey	(20) 0-N discrete scale	- 0 means that no costs were incurred by the NCA related to the risk assessment; - N relates to the amount of money incurred by the NCA for the risk assessment.

7.2Data to be collected from HSPs

ID KPI	KPIs 3	Type of indicator	Monitoring areas	Sub-monitoring area	Level of reporting obligation	Type of data	Data sources	Units of measurement	Units of measurement - description
KPI 20	Preservation of TCO by the HSP for a period of six months after the removal or disabling (duration, criteria for when the data should be preserved, implementation of appropriate technical and organisational safeguards) (Art. 6)	Output	Safeguard mechanisms and fundamental rights (freedom of expression and information)	Preservation of content and related data	Level 2	Primary	HSP consultation / Survey	Yes/no and 1-5 Likert scale	Yes/No - 1 means that the HSP never preserved TCO for a period of six months from the removal or disabling of content (duration, criteria for when the data should be preserved, implementation of appropriate technical and organisational safeguards) - 5 means that the HSP always preserved TCO for a period of six months from the removal or disabling of content (duration, criteria for when the data should be preserved, implementation of appropriate technical and organisational safeguards)
KPI 21	TCO and related data that is preserved by the HSP is subject to appropriate technical and organisational safeguards, that such data is only accessed and processed for the purpose of the removal order by the HSP to ensure a high level of security of the personal data concerned (Art. 6.3)	Output	Safeguard mechanisms and fundamental rights (freedom of expression and information, the respect for private life, the protection of personal data)	Preservation of content and related data	Level 2	Primary	HSP consultation / Survey	Yes/no and 1-5 Likert scale	Yes/No - 1 means that when TCO and related data was preserved by the HSP, the HSP never implemented appropriate technical and organisational safeguards, in particular to ensure that such data is only accessed and processed for the purpose of the removal order - 5 means that when TCO and related data was preserved by the HSP, the HSP always implemented appropriate technical and organisational safeguards, in particular to ensure that such data is only accessed and processed for the purpose of the removal order
KPI 22	Information about the removal or disabling is accessible to the content provider by the HSP when TCO is removed (Art. 11.1)	Output	Safeguard mechanisms and fundamental rights (freedom of expression and information, the respect for private life, the protection of personal data)	Remedies mechanism	Level 2	Primary	HSP consultation / Survey	Yes/No and 1-5 Likert scale	Yes/No - 1 means that the HSP never makes information about the removal or disabling of content accessible to the content provider when it removes or disables access to terrorist content - 5 means that the HSP always makes information about the removal or disabling of content accessible to the content provider when it removes or disables access to terrorist content
KPI 23	HSP informs complainant(s) within 2 weeks about the actions taken following a complaint received (Art. 10.2)	Output	Safeguard mechanisms and fundamental rights (freedom of expression and information, the respect for private life, the protection of personal data)	Complaints mechanism	Level 2	Primary	HSP consultation / Survey	Yes/No and 1-5 Likert scale	Yes/No - 1 means that the HSP never informs complainant(s) within 2 weeks about the actions taken following a complaint received - 5 means that the HSP always informs complainant(s) within 2 weeks about the actions taken following a complaint received
KPI 24	Consideration of the rights and legitimate interest of the users in the design and implementation of specific measures by HSPs to counter terrorist content online, in particular users’ fundamental rights concerning freedom of expression and information, respect for private life and protection of personal data (Art 5.3 (c))	Output	Safeguard mechanisms and fundamental rights (freedom of expression and information, respect for private life and protection of personal data)	Consideration of users’ rights in the development of such safeguard’s mechanisms	Level 1	Primary	HSP consultation / Survey	0-1 dichotomous scale (Yes or No)	- 0 means that the HSP does not consider the rights and legitimate interest of the users in the design and implementation of specific measures to counter terrorist content online, in particular users’ fundamental rights concerning freedom of expression and information, respect for private life and protection of personal data - 1 means that the HSP considers the rights and legitimate interest of the users in the design and implementation of specific measures to counter terrorist content online, in particular users’ fundamental rights concerning freedom of expression and information, respect for private life and protection of personal data
KPI 25	Application of diligence and non-discrimination in the design and implementation of specific measures (safeguard mechanisms) by the HSP (Art 5.3 (d))	Output	Safeguard mechanisms	Consideration of users’ rights in the development of such safeguard’s mechanisms	Level 1	Primary	HSP consultation / Survey	0-1 dichotomous scale (Yes or No)	- 0 means that the HSP does not apply diligence and non-discrimination in the design and implementation of specific measures (safeguard mechanisms) - 1 means that the HSP applies diligence and non-discrimination in the design and implementation of specific measures (safeguard mechanisms)
KPI 26	Existence of specific measures implemented by the HSP in relation to the identification and removal of or disabling of access to terrorist content (Art 7.3(a))	Output	Public security	HSPs policy for moderating terrorist content and its functioning, i.e., specific measures	Level 1	Secondary	HSP Transparency Report	0-1 dichotomous scale (Yes or No)	- 0 means that the HSP did not implement specific measures in relation to the identification and removal of or disabling of access to terrorist content - 1 means that the HSP implemented specific measures in relation to the identification and removal of or disabling of access to terrorist content
KPI 27	Existence of specific measures implemented by the HSP to mitigate the level of exposure of its services to terrorist content (Art 5.3 (a))	Output	Public security	HSPs policy for moderating terrorist content and its functioning, i.e., specific measures	Level 2	Primary	HSP consultation / Survey	0-1 dichotomous scale	- 0 means that the HSP has not put in place nor planned by 2024 any specific measures in relation to the identification and removal of or disabling of access to TCO; - 1 means that the HSP has put in place specific measures in relation to the identification and removal of or disabling of access to TCO.
KPI 28	Existence of specific measures implemented by the HSP to address the reappearance online of material which has previously been removed or to which access has been disabled because it was considered to be terrorist content. (Art. 7.3 (b))	Output	Public security	HSPs policy for moderating terrorist content and its functioning, i.e., specific measures	Level 1	Secondary	HSP Transparency Report	0-1 dichotomous scale (Yes or No)	- 0 means that the HSP did not implement specific measures to address the reappearance online of material which has previously been removed or to which access has been disabled because it was considered to be terrorist content - 1 means that the HSP implemented specific measures to address the reappearance online of material which has previously been removed or to which access has been disabled because it was considered to be terrorist content
KPI 29	Use of open-source technologies (specialised hash matching technology) to improve the detection of TCO across the HSPs community in the EU	Output (as optional)	Public security	Collaboration between HSPs, Member States & other relevant stakeholders for TCO	Level 3	Primary	HSP consultation / survey	0-1 discrete scale (Yes or No)	- 0 means the HSP never makes use of open-source technologies to improve the detection of TCO; - 1 means the HSP makes use of open-source technologies to improve the detection of TCO.
KPI 30	The extent to which the HSP makes use of the tools established by Europol or other temporary/ad interim ones for the effective implementation of the Regulation (Art. 14.4)	Output (as optional)	Public security	Collaboration between HSPs, Member States & other relevant stakeholders for TCO	Level 3	Primary	HSP consultation / Survey	0-1 discrete scale (Yes or No)	- 0 means that the HSP never uses the tools established by Europol or other temporary/ad interim ones; - 1 means that the HSP uses the tools established by Europol or other temporary/ad interim ones.
KPI 31	The number of cases in which the HSP, after receiving a removal order or a decision pursuant to Article 4.4 or Article 5.4, 6 or 7, challenged such a decision (Art. 9.1)	Result	Safeguard mechanisms and fundamental rights	Remedies mechanism	Level 2	Primary	HSP consultation / Survey	(1) 0-N discrete scale (absolute number of challenged decisions following a removal order)	- 0 means that the HSP did not challenge the decision after receiving a removal orders or decisions pursuant to Article 4.4 or 5.4, 6 or 7 - N is the number of challenged decision(s) by HSPs after receiving a removal orders or decisions pursuant to Article 4.4 or 5.4, 6 or 7 Quartile computation of the indicator. The quartiles will have to be computed once the data will be collected.
KPI 32	The number of complaints received by the HSP (Art 21.1 (d) and Art 7.3 (d)) (from content providers)	Result	Safeguard mechanisms	Complaints mechanism	Level 1	Secondary	HSP Transparency Report	(3) 0-N discrete scale (absolute number of complaints received by a country)	- 0 means that a HSP did not receive complaints from content providers - N means that a country received N complaints from content providers Quartile computation of the indicator. The quartiles will have to be computed once the data will be collected.
KPI 33	Establishment of an effective and accessible mechanism by the HSP allowing for content providers to submit a complaint concerning removal or disabling of terrorist content (Art. 10.1)	Result	Safeguard mechanisms and fundamental rights (freedom of expression and information, the respect for private life, the protection of personal data)	Complaints mechanism	Level 2	Primary	HSP consultation / Survey	Yes/no and 0-3 discrete scale	- Yes/no - 0 means that the HSP has not established nor planned by 2024 an effective and accessible mechanism for content providers to submit a complaint concerning removal or disabling terrorist content online - 3 means that the HSP established an effective and accessible mechanism for content providers to submit a complaint concerning removal or disabling terrorist content online
KPI 34	The number of times content was reinstated by the HSP following a complaint received from a content provider (ratio: complaint received / valid complaint = content restored to the platform) (Art 21.1 (d) and Art 7.3 (d))	Result	Safeguard mechanisms	Complaints mechanism	Level 1	Secondary	HSP Transparency Report	(4) 0-N discrete scale (absolute number of times content has been reinstated on platform following a complaint)	- 0 means that a HSP did not reinstate a content on the platform following a complaint received from a content provider - N means that a HSP reinstated N times a content on the platform following a complaint received from a content provider Quartile computation of the indicator. The quartiles will have to be computed once the data will be collected.
KPI 35	The number of administrative or judicial review proceedings brought by the HSP (Art. 7.3 (e))	Result	Safeguard mechanisms	Penalties and administrative or judicial review proceedings	Level 1	Secondary	HSP Transparency Report	(5) 0-N discrete scale (absolute number of administrative or judicial appeal proceedings by HSPs)	-0 means that no administrative or judicial review proceedings was brought by the HSP -N means that N administrative or judicial appeals proceedings were brought by the HSP Quartile computation of the indicator. The quartiles will have to be computed once the data will be collected.
KPI 36	The outcome of administrative or judicial review proceedings brought by the HSP (Art. 7.3 (e))	Result	Safeguard mechanisms	Penalties and administrative or judicial review proceedings	Level 1	Secondary	HSP Transparency Report	0-1 dichotomous scale (Yes or No)	- 0 means that the HSP did not report on the outcome of administrative or judicial review proceedings - 1 means that the HSP report on the outcome of administrative or judicial review proceedings.
KPI 37	The number of cases in which the HSP was required to reinstate content or access thereto as a result of administrative or judicial review proceedings (Art. 7.3 (f))	Result	Safeguard mechanisms and fundamental right (freedom of expression and information, the respect for private life, the protection of personal data)	Penalties and administrative or judicial review proceedings	Level 1	Secondary	HSP Transparency Report	(6) 0-N discrete scale (absolute number of times a content was reinstated on platform following admin or judicial review proceedings)	-0 means that a HSP did not reinstate a content on the platform as a result of administrative or judicial review proceedings -N means that a HSP reinstated N times a content on the platform as a result of administrative or judicial review proceedings Quartile computation of the indicator. The quartiles will have to be computed once the data will be collected.
KPI 38	Number of procedures for removal orders and referrals sent to/ received by the HSP as well as cross-border ones (Art. 7.3(c))	Result	Public security	Procedures for removal orders including cross-border	Level 1	Secondary	HSP & NCA Transparency Report	(9) 0-N discrete scale (absolute number of removal orders / referrals received by HSPs)	- 0 means that no procedure for removal orders and referrals was received by the HSP including cross-border ones - N means that N procedure(s) for removal orders and referrals were received by the HSP including cross-border ones
KPI 39	Number of removal orders scrutinised (Art. 7.3(c))	Result	Public security	Procedures for removal orders including cross-border	Level 1	Secondary	HSP Transparency Report	(10) 0-N discrete scale (absolute number of scrutinised removal orders)	- 0 means that no removal order was scrutinised - N means that N removal orders were scrutinised
KPI 40	Number of removal orders that were rejected (Art. 7.3(c))	Result	Public security	Procedures for removal orders including cross-border	Level 1	Secondary	HSP Transparency Report	(11) 0-N discrete scale (absolute number of rejected removal orders)	- 0 means that no removal orders were rejected - N means that N removal orders were rejected
KPI 41	Number of decisions to effectively remove terrorist content online (following removal order) (Art. 7.3(c))	Result	Public security	Procedures for removal orders including cross-border	Level 1	Secondary	HSP Transparency Report	(12) 0-N discrete scale (absolute number of decisions taken to effectively remove terrorist content online)	- 0 means that no decision was taken to effectively for removal orders and referrals was received by the HSP including cross-border ones - N means the number of decisions taken to effectively remove terrorist content online (following removal order)
KPI 42	Number of removal orders that were effectively proceeded within one hour of receipt of the removal order by the HSP (Art. 3.3)	Result	Public security	Procedures for removal orders including cross-border	Level 2	Primary	HSP consultation / Survey	(13) 0-N discrete scale (absolute number of removal orders that were effectively processed within 1 hour)	- 0 means that no removal orders were effectively proceeded within one hour of receipt of the removal order by the HSP - N means that N removal orders were effectively proceeded within one hour of receipt of the removal order by the HSP
KPI 43	Number of HSPs putting proactive measures in place to identify, remove or disable the access to TCO	Result	Public security	HSPs policy for moderating terrorist content and its functioning, i.e., specific measures	Level 2	Secondary	HSP Transparency Report	(15) 0-N discrete scale (absolute number of HSP(s) putting in place proactive measures to identify, remove or disable the access to TCO)	- 0 means that no HSP has put proactive measures in place to identify, remove or disable the access to TCO - N is the number of HSP(s) that have put proactive measures in place to identify, remove or disable the access to TCO
KPI 44	Number of HSPs publishing transparency reports on TCO	Result	Public security	Transparency and awareness from EC, NCAs and HSPs	Level 2	Secondary	HSP Transparency Reports available online	(17) 0-N discrete scale (absolute number of annual transparency reports published by HSPs)	-0 means that no annual transparency reports were published; - N annual transparency reports were published by HSPs.
KPI 45	The specific measures implemented by the HSP are targeted and proportionate, taking into account, in particular, the seriousness of the level of exposure of the services of the hosting service provider to terrorist content as well as the technical and operational capabilities, financial strength, the number of users of the services of the hosting service provider and the amount of content they provide. (Art 5.3)	Result	Fundamental rights (freedom to conduct a business)	Competitiveness and effect on SMEs	Level 2	Primary	HSP consultation / survey	1-5 Likert scale	-1 means strongly disagree; - 5 means strongly agree.
KPI 46	Number of HSPs using digital tools (automated tools) for the moderation of TCO (to detect/identify TCO)	Impact	HSPs	Administrative burden and costs on HSPs related to TCO moderation	Level 3	Primary	HSP consultation / Survey	0-1 dichotomous scale (Yes or No)	- No means that the HSP does not use digital tools for the moderation of TCO; - Yes means that the HSP uses digital tools for the moderation of TCO.
KPI 47	Additional costs due to the reporting obligations of the TCO Regulation for HSPs	Impact	HSPs	Administrative burden and costs on HSPs related to TCO moderation	Level 3	Primary	HSP consultation / Survey	Yes/No 0-3 discrete scale	Yes/No - 0 means that no additional costs were incurred by the HSP related to the reporting obligations of the Regulation; - 3 means that several costs were incurred by the HSP related to the reporting obligations of the Regulation ("to a great extent")
KPI 48	Number of HSPs contracting third parties (vendors) for TCO moderation	Impact	HSPs	Administrative burden and costs on HSPs related to TCO moderation	Level 3	Primary	HSP consultation / Survey	0-1 dichotomous scale	- 0 means that the HSP does not contract third parties for TCO moderation - 1 means that the HSP contracts third parties for TCO moderation.
KPI 49	Costs related to the application of the 1-hour deadline for removal orders	Impact	HSPs	Administrative burden and costs on HSPs related to TCO moderation	Level 3	Primary	HSP consultation / Survey	Yes/No 0-3 discrete scale	Yes/No - 0 means that no costs were incurred by the HSP related to the application of the 1-hour deadline for removal orders; - 3 means that several costs were incurred by the HSP related to the application of the 1-hour deadline for removal orders.
KPI 50	Costs related to the content moderation or filtering technologies for HSPs (for TCO only)	Impact	HSPs	Administrative burden and costs on HSPs related to TCO moderation	Level 3	Primary	HSP consultation / Survey	Yes/No 0-3 discrete scale	Yes/No - 0 means that no costs were incurred by the HSP related to the content moderation or filtering technologies; - 3 means that several costs were incurred by the HSP related to the content moderation or filtering technologies.
KPI 51	Number of new staff to be hired for this activity of content moderation/filtering technologies (for TCO only)	Impact	HSPs	Administrative burden and costs on HSPs related to TCO moderation	Level 3	Primary	HSP consultation / Survey	0-N discrete scale	- N is the discrete number of new staff hired for the activity of content moderation/filtering technologies
KPI 52	Costs related to the trainings of staff for this activity	Impact	HSPs	Administrative burden and costs on HSPs related to TCO moderation	Level 3	Primary	HSP consultation / Survey	Yes/No 0-3 discrete scale	Yes/No - 0 means that no costs were incurred by the HSP related to the training of new staff; - 3 means that several costs were incurred by the HSP related to the training of new staff.

ID KPI	KPIs	Type of indicator	Monitoring areas	Sub-monitoring area	Level of reporting obligation	Type of data	Data sources	Units of measurement	Units of measurement - description
KPI 53	European Commission awareness-raising activity(ies) or awareness programme among Member States and HSPs of the importance and risk of TCO and to support related activities	Output	Public security	Transparency and awareness from EC, NCAs and HSPs	Level 3	Secondary	European Commission and desk research	0-1 dichotomous scale (Yes or No)	- 0 means that no awareness-raising activity or programme among MS and HSP was put in place by the EC regarding the importance and risks of terrorist content online; - 1 means that at least one awareness-raising activity or programme among MS and HSP was put in place by the EC regarding the importance and risks of terrorist content online.
KPI 54	Number of EU institutions/organisations publishing annual transparency reports on TCO	Result	Public security	Transparency and awareness from EC, NCAs and HSPs	Level 3	Secondary	EU institutions / orga. Transparency report available online	(19) 0-N discrete scale (absolute number of annual transparency reports published by EU institutions/organisations)	-0 means that no annual transparency reports were published; - N means that the EU institutions/organisation has published N annual transparency reports.

Wählen Sie die experimentellen Funktionen, die Sie testen möchten.