0.2.2

For multi-stage approved vehicles, type-approval information of the base/previous stage vehicle, list the information for each stage. (This can be done with a matrix)

0.5.1

For multi-stage approved vehicles, company name and address of the manufacturer of the base/previous stage(s) vehicle: …

0.6

Location and method of attachment of statutory plates and location of vehicle identification number: …

0.6.1.

On the chassis: …

0.6.2.

On the bodywork: …

17.3.4.

Maps and positioning

17.3.5.

Other hardware

17.5.3.

Acceptance criteria

17.5.4

Demonstration of compliance

17.7.1.

Description of the adopted approach

17.7.2.

Selection of nominal, critical and failure scenarios

17.7.3.

Description of the used methods and tools (software, laboratory, others) and summary of the credibility assessment

17.7.4.

Description of the results

17.7.5.

Uncertainty of the results

17.7.6.

Interpretation of the results

17.8.4.

Means to ensure data security and data protection

List of Figures / Tables

Acronyms

Annex I - Simulation Handbook

Annex II – Operating Manual

This information document comprises the information relevant for the automated driving system and shall be completed in accordance with the template laid down in Annex I to Commission Implementing Regulation (EU) 2020/683.

The ADS shall be capable of performing the entire DDT.

The capability of the ADS to perform the entire DDT shall be determined in the context of the ODD of the ADS.

1.1.3.

The system shall demonstrate anticipatory behaviour in interaction with other road user(s), in order to ensure stable, low-dynamic, longitudinal behaviour and risk minimising behaviour when critical situations could become imminent, e.g. with unobstructed and obstructed pedestrians or cyclist or with other vehicles crossing or cutting-in in front of the ADS.

1.1.4.

The requirements related to the entire DDT shall be fulfilled in the reverse direction as well, if the reverse gear is required by or declared in the ODD

The ADS shall detect and respond appropriately to objects and events relevant for the DDT within the ODD.

(g) environmental conditions (e.g. lower speed due to rain, snow).

The ADS shall comply with traffic rules of the country of operation

(b) using the audible warning device where appropriate.

(c) interacting safely with road safety officers / law enforcement agents, road maintenance workers, emergency service personnel, road inspectors, etc.

(d) for dual mode vehicles, the ADS status (manual driving mode or fully automated driving mode) shall be recognizable for Road safety officers / law enforcement agents

1.3.2.

In the absence of specific traffic rules, vehicles with ADS intended to carry standing or unrestrained vehicle occupants shall not exceed a combined horizontal acceleration of 2.4 m/s2 (in absolute value and calculated as the combination of lateral and longitudinal acceleration), and shall not exceed an acceleration rate of change of 5 m/s3.

Depending on the factors influencing the risk to occupants and other road users, it might be appropriate to exceed these limits, such as emergency operations.

The ADS shall be able to detect the risk of collision with other road users, or a suddenly appearing obstacle (debris, lost load) and shall be able to automatically perform appropriate emergency operation (braking, evasive steering) to avoid collisions and minimise risks to safety of the vehicle occupants and other road users.

If a collision can be safely avoided without causing another one, it shall be avoided by the ADS.

In the event of an unavoidable alternative risk to human life, the ADS shall not provide for any weighting on the basis of personal characteristics of humans.

The protection of other human life outside the fully automated vehicle shall not be subordinated to the protection of human life inside the fully automated vehicle.

After the evasive manoeuvre the vehicle shall aim to resume a stable motion as soon as technically possible.

The ADS shall detect and respond to ADS or/and vehicle malfunctioning behaviour

The ADS shall self-diagnose faults and failures.

The ADS shall evaluate its ability to fulfil the entire DDT.

The ADS shall respond safely to a fault/failure in the ADS that does not significantly compromise ADS performance.

The ADS shall execute a safe fall-back response to achieve a MRC in the event of a failure of the ADS and/or other vehicle system that prevents the ADS from performing the DDT.

If failures affect the braking or steering performance of the vehicle, the MRM shall be carried out with consideration for the remaining performance.

The manufacturer shall define the acceptance criteria from which the validation targets of the ADS are derived to evaluate the residual risk taking into account existing accident data 1 , data on performances from competent and carefully driven manual vehicles and technology state-of-the-art. 

For vehicles of categories M2, M3, N2 and N3, the data elements listed in point 9.2. shall be retrievable even after an impact. To demonstrate that capability, the following applies:

Either:

(a)(a) After a mechanical shock applicable to on-board data storage devices, if any, at a severity level as specified in the component test of Annex 9C of the 03 series of amendment to UN Regulation No. 100 7 , and

(b)(b) On-board data storage device(s) shall be mounted in the vehicle cab/passenger compartment or in a position of sufficient structural integrity to protect against physical damage that would prevent the retrieval of data. This shall be demonstrated to the technical service together with appropriate documentation (e.g. calculations or simulations);

Or,

(c) (c) The manufacturer demonstrates fulfilling the requirements of point 9.5.3. (e.g. for M2 / N2 vehicles derived from M1 / N1).

If the ADS allows manual driving for the purpose of maintenance or to take over after a minimum risk manoeuvre is provided in the fully automated vehicle, the vehicle shall be provided with means to enable the person driving the vehicle to perform the driving task safely in accordance with the safety concept of the manufacturer. Except in case of failure, the ADS shall continue to perform OEDR, detect an obstacle (e.g. vehicles, pedestrian) in the manoeuvring area and shall support the driver in bringing the vehicle immediately to a stop to avoid a collision.

If manual driving control is limited to 6 km/h, it is not necessary for the driver to stay within the fully automated vehicle. The control can be performed via a remote control located in the vicinity of the vehicle provided that the vehicles stays in the direct line of sight of the operator. The maximum distance over which control is possible by a remote control shall not exceed 10 metres.

If, in manual driving, the vehicle is intended to be controlled at speeds higher than 6 km/h, the vehicle shall be considered as dual mode vehicle and shall comply with Annex II, Part I, Appendix 1 of Regulation (EU) 2018/858.

The manufacturer shall draw up an operating manual. The purpose of the operating manual is to ensure the safe operation of the fully automated vehicle by means of detailed instructions to the owner, vehicle occupants, transport service operator, on-board operator, remote intervention operator and any relevant national authorities.

When the fully automated vehicle includes the possibility of manual driving for the purpose of maintenance or to take over after a minimum risk manoeuvre, it shall also covered by the operating manual.

1.3.

Parameters to be used for the lane change scenarios by the fully automated vehicle

1.3.1.

The scenarios and parameters, with regard to lane change, shall be applied as specified in UN Regulation No.157 11 .

Parameters to be used for the turning and crossing scenario by the fully automated vehicle.

In the absence of more specific traffic rules, the following requirements shall be taken into account with regard to interaction with other road users involved in the movement when turning and crossing (see Figure 1) in dry and proper road pavement conditions.

In the case of merging with privileged traffic during turning with and without crossing the opposite traffic direction, privileged traffic in the target lane should not have to decelerate. However, it must be ensured that the TTC of the approaching privileged traffic in the target road (case (a) in Figure 1) never falls below the threshold defined as:

With:

equal to the speed of the fully automated vehicle

equal to the speed of the privileged approaching traffic

equal to 3 m/s2 being the maximum admissible deceleration for the privileged approaching traffic.

In the case of a turning manoeuvre crossing the opposite traffic direction, when considering oncoming traffic, privileged traffic in the target lane should not have to decelerate. However, if justified by the traffic density, it must be ensured — in addition to the distance from the approaching privileged traffic in the target road — that the TTC of the privileged crossing traffic to the fictitious collision point (point of intersection of the trajectories, case (b) in Figure 1) never falls below the threshold defined as:

With:

equal to the speed of the privileged conflicting traffic

equal to 3 m/s2 being the maximum admissible deceleration for the privileged crossing traffic.

equal to 1.5s being the reaction time of the privileged crossing traffic

The same applies to crossing with privileged traffic (case (c) in Figure 1): The TTC of privileged traffic to the imaginary collision point (point of intersection of the trajectories) shall never fall below the threshold defined in the present point.

Parameters to be used for the emergency manoeuvre scenarios by the fully automated vehicle (DDT under critical scenarios)

The ADS shall avoid a collision with a leading vehicle which decelerates up to its full braking performance provided that there was no cut-in by another vehicle.

Collisions with cutting in vehicles, pedestrians and cyclists travelling in the same direction, as well as with pedestrians who can start jaywalking shall be avoided at least within the conditions determined by the following equation.

With:

being the time to-collision at the moment of the cut-in of the vehicle or cyclist by more than 30 cm in the lane of the fully automated vehicle.

being the relative speed in metres per second [m/s] between the fully automated vehicle and the cutting-in vehicle (positive if the ADS is faster than the cutting-in vehicle).

being the maximum deceleration of the fully automated vehicle and assumed to be equal to

2.4 m/s2 for fully automated vehicles transporting standing or not fastened vehicle occupants;

6 m/s2 for other fully automated vehicles.

being the time required by the fully automated vehicle to initiate an emergency braking and assumed to be equal to 0.1 s

being the time to reach the maximum deceleration and assumed to be equal to

0.12 s for fully automated vehicles transporting standing or not fastened vehicle occupants;

0.3 s for other fully automated vehicles

The compliance with this equation is required only for road users cutting in, and only if the inserting road users were visible at least 0,72 seconds before cut-in:

This results in a required collision avoidance when another road user enters ego lane above the following TTC values (for example shown for speeds in 10 km/h steps). These requirements shall be met independently of environmental conditions.

Urban and rural driving conditions

The ADS shall avoid a collision, up to a speed of 60 km/h, with an unobstructed pedestrian crossing with a lateral speed component of not more than 5 km/h or an unobstructed cyclist crossing with a lateral speed component of not more than 15 km/h in front of the vehicle. This shall be ensured independently from the specific manoeuvre the ADS is undertaking.

In the case the pedestrian or the cyclist proceed with higher speed than the aforementioned values and the ADS can no longer avoid collision, the control strategy of the ADS may change between collision avoidance and mitigation only if the manufacturer can demonstrate that this increases the safety of the vehicle occupants and the other road users (e.g. by prioritizing braking over an alternative manoeuvre).

The ADS shall mitigate a collision with an obstructed pedestrian or cyclist crossing in front of the vehicle by reducing its speed at impact by at least 20 km/h. This shall be ensured independently from the specific manoeuvre the ADS is undertaking.

For the purpose of demonstrating fulfilment of the previous requirements related to crossing of pedestrians and cyclists in front of the vehicle, test and assessment scenarios developed under the European New Car Assessment Programme (Euro NCAP) may be taken as guidance.

Motorway driving conditions

The relevant scenarios, with regard to pedestrian crossing, shall be applied as specified in UN Regulation No.157.

In the case the pedestrian crosses with parameter values outside the boundaries specified in UN Regulation 157 and the ADS can no longer avoid collision, the control strategy of the ADS may change between collision avoidance and mitigation only if the manufacturer can demonstrate that this increases the safety of the vehicle occupants and the other road users (e.g. by prioritizing braking over an alternative manoeuvre).

Parameters to be used for Hub to Hub fully automated vehicles

Operation on motorway

The relevant scenarios, with regard to operation speed and automatic lane change, shall be applied as specified in UN Regulation No.157.

Motorway entry

The fully automated vehicle shall be able to safely enter the motorway by adapting the speed to the traffic flow, and activate the relevant direction indicator according to the traffic rules.

The direction indicator shall be deactivated once the vehicle has performed the lane change manoeuvre (LCM). The parameters used in the lane change scenario shall be applied.

Motorway exit

The fully automated vehicle shall be able to anticipate the targeted motorway exit by driving on the adjacent lane to the exit lane and shall not unnecessarily decelerate before the LCM into the exit lane starts.

The fully automated vehicle shall apply the direction indicator in accordance with the traffic rules, and perform the LCM into the exit lane without undue delay.

The direction indicator shall be deactivated once the LCM has been completed in compliance with the traffic rules in the country of operation.

Passing a toll station

Depending on the ODD, the fully automated vehicle shall be able to select the proper passing gate, and adapt its speed to that permitted limits within the toll area while considering the traffic flow.

Operation on other road types than motorways

Depending on the ODD, the relevant scenario defined in points 1.3. to 1.5 above shall be applied.

Parameters to be used for Automated valet parking

Depending on the ODD, the relevant scenarios defined in points 1.3. to 1.5 above shall be applied. The parameters to be used for these scenarios may need to be adapted to take into account of the limited driving speed and the general lack of visibility that may occur in a parking facility. Special attention shall be given to avoiding of collision with pedestrians and in particular with children and prams.

Scenarios not covered by point 1.

Scenarios that are not listed in point 1 shall be generated to cover reasonably foreseeable critical situations, including failures and traffic hazards within the operational design domain.

When ADS capabilities depend on remote capabilities, scenarios shall include failures and traffic hazards stemming from the corresponding remote capabilities.

The method to generate scenarios that are not listed in section 1, shall follow the principles set in Appendix 1 to Part 1 of this Annex.

The method used by the manufacturer to generate scenarios that are not listed in point 1 shall be documented in the documentation package to be provided for the ADS assessment.

Appendix 1 to Part 1 of Annex 3

Principles to be followed to derive scenarios relevant for the ODD of the ADS

Scenarios shall be derived from the ODD of the fully automated vehicle.

Generation and classification of scenarios

From a qualitative perspective, scenarios can be classified into Nominal / Critical / Failure and correspond to normal or emergency operation. For each of these categories, a data-based approach and a knowledge-based approach can be used to generate corresponding traffic scenarios. A knowledge-based approach utilizes expert knowledge to identify hazardous events systematically and create scenarios. A data-based approach utilizes the available data to identify and classify occurring scenarios.

Nominal scenarios

A series of analytical frameworks can help the manufacturer to derive additional nominal scenarios to ensure coverage for the specific application. These frameworks are divided into:

ODD analysis

An ODD may consist of scenery elements (e.g., physical infrastructure), environmental conditions, dynamic elements (e.g., traffic, vulnerable road users) and operational constraints to the specific ADS application. The aim of this analysis is to identify the characteristics of the ODD, allocate properties and define interactions between the objects. Here the effect of ODD on the behaviour competencies is explored. An example of the analysis is provided in Table 1.

Requirements

The manufacturer shall provide a documentation package which gives access to the basic design of the ADS and the means by which it is linked to other vehicle systems or by which it directly controls output variables as well as off-board hardware/software and remote capabilities.

The function(s) of the ADS, including the control strategies, and the safety concept, as laid down by the manufacturer, shall be explained.

Documentation shall be brief, yet provide evidence that the design and development has had the benefit of expertise from all the ADS fields which are involved.

For periodic technical inspections, the documentation shall describe how the current operational status of the ADS can be checked.

The type-approval authority shall assess the documentation package which shall show that the ADS:

(a)    is designed and was developed to operate in such a way that it is free from unreasonable risks for a vehicle occupants and other road users within the declared ODD and boundaries;

(b)    fulfils the performance requirements of Annex 2 of this Regulation;

(c)    was developed according to the development process/method declared by the manufacturer.

Documentation shall be made available in three parts:

(a)    Application for type-approval: the information document submitted to the type-approval authority at the time of the type-approval application shall contain brief information on the items listed in Annex I. It will become part of the type-approval.

(b)    The formal documentation package for the type-approval, containing the material listed in this section 3. (with the exception of that of point 3.5.5.) which shall be supplied to the type-approval authority for the purpose of conducting the ADS type-approval. This documentation package shall be used by the type-approval authority as the basic reference for the verification process set out in point 4. of this annex. The type-approval authority shall ensure that this documentation package remains available for a period of at least 10 years counted from the time when production of the vehicle type is definitely discontinued.

(c)    Additional confidential material and analysis data (intellectual property) of point 3.5.5. which shall be retained by the manufacturer, but made open for inspection (e.g. on-site in the engineering facilities of the manufacturer) at the time of the ADS type-approval. The manufacturer shall ensure that this material and analysis data remains available for a period of 10 years counted from the time when production of the vehicle type is definitely discontinued.

General description of the ADS

A description shall be provided giving a simple explanation of the operational characteristics of the ADS and ADS features.

The description shall include :

the operational design domain such as maximum speed of operation, road type (e.g. dedicated lane), country(ies)/areas of operation, road conditions and environmental conditions required (e.g. no snow), etc.)/ Boundary conditions

basic performance (e.g. object and event detection and response, off board infrastructure needed during operation)

Interaction with other road users

main conditions for minimum risk manoeuvres.

interaction concept with vehicle occupants, the on board operator (if applicable) and the remote intervention operator (if applicable).

the means to activate or deactivate the ADS by the on-board operator (if relevant) or the remote intervention operator (if relevant), vehicle occupants (if relevant) or other road users (if relevant).

operational measures (e.g. on-board operator or remote intervention operator needed) to be met to ensure safety during the fully automated vehicle operation.

Description of the functions of the ADS

A description shall be provided giving an explanation of all the functions including control strategies to ensure the robust and safe operation of the ADS and the methods used to perform the dynamic driving tasks within the ODD, and the boundaries under which the automated driving system is designed to operate, including a description on how this is ensured.

Any enabled or disabled automated driving functions for which the hardware and software are present in the vehicle at the time of production, shall be declared and are subject to the requirements of this annex as well as Annex 2 of this Regulation, prior to their use in the vehicle. The manufacturer shall also document the data processing if continuous learning algorithms are implemented.

A list of all input and sensed variables shall be provided and the working range of these defined, along with a description of how each variable affects the ADS behaviour.

A list of all output variables that are controlled by the ADS shall be provided and an explanation given, in each case, of whether the control is direct or via another vehicle system. The range over which the ADS is likely to exercise control on each such variable shall be defined.

Limits defining the boundaries of functional operation including ODD-limits shall be stated where appropriate to ADS performance.

The human machine interface (HMI) concept with the vehicle occupants/on-board operator/remote intervention operator (if any) when ODD limits are approached and then reached shall be explained. The explanation shall include the list of types of situations in which the ADS will generate a support request to the on board operator/remote intervention operator (if applicable), the way the request is performed, the procedure that handles a failed request and the minimum risk manoeuvre. Signals and information given to the on-board operator/remote intervention operator, vehicle occupants and other road users in each of the above aspects shall also be described.

ADS layout and schematics

Inventory of components.

A list shall be provided, collating all the units of the ADS and mentioning the other vehicle systems as well as off-board hardware/software and remote capabilities that are needed to achieve specified performance of the ADS to be approved according to its ODD.

An outline schematic showing these units in combination, shall be provided with both the equipment distribution and the interconnections made clear.

This outline shall include:

(a) perception and objects detection including mapping and positioning

(b) Characterisation of Decision-making

(d) The ADS data elements.

(e) links and interface with other vehicle systems, off-board hardware/software and remote capabilities

Functions of the units

The function of each unit of the ADS shall be outlined and the signals linking it with other units or with other vehicle systems shall be shown. It shall include off board systems supporting the ADS and other vehicle systems. This may be provided by a labelled block diagram or other schematic, or by a description aided by such a diagram.

Interconnections within the ADS shall be shown by a circuit diagram for the electric transmission links, by a piping diagram for pneumatic or hydraulic transmission equipment and by a simplified diagrammatic layout for mechanical linkages. The transmission links both to and from other systems shall also be shown.

There shall be a clear correspondence between transmission links and the signals carried between units. Priorities of signals on multiplexed data paths shall be stated wherever priority may be an issue affecting performance or safety.

Identification of units

Each unit shall be clearly and unambiguously identifiable (e.g. by marking for hardware, and by marking or software output for software content) to provide corresponding hardware and documentation association. Where a software version can be changed without requiring the replacement of the marking or component, the software identification must be by software output only.

Where functions are combined within a single unit or indeed within a single computer, but shown in multiple blocks in the block diagram for clarity and ease of explanation, only a single hardware identification marking shall be used. The manufacturer shall, by the use of this identification, affirm that the equipment supplied conforms to the corresponding document.

The identification defines the hardware and software version and, where the latter changes such as to alter the function of the unit as far as this Regulation is concerned, this identification shall also be changed.

Installation of sensing system components

The manufacturer shall provide information on the installation options for the individual components that comprise the sensing system. These options shall include, but are not limited to, the location of the component in/on the vehicle, the material(s) surrounding the component, the dimensioning and geometry of the material surrounding the component, and the surface finish of the materials surrounding the component, once installed in the vehicle. The information shall also include installation specifications that are critical to the ADS’s performance, e.g. tolerances on installation angle.

Changes to the individual components of the sensing system, or the installation options, shall be notified to the type-approval authority and be subject to further assessment.

Safety concept of the manufacturer and validation of the safety concept by the manufacturer

The manufacturer shall provide a statement which affirms that the ADS is free from unreasonable risks for the vehicle occupants and other road users.

In respect of software employed in the ADS, the outline architecture shall be explained and the design methods and tools used shall be identified (see 3.5.1). The manufacturer shall show evidence of the means by which they determined the realisation of the ADS logic, during the design and development process.

The manufacturer shall provide the type-approval authority with an explanation of the design provisions built into the ADS so as to ensure functional and operational safety. Possible design provisions in the ADS are for example:

(a)    fall-back to operation using a partial system.

(b)    redundancy with a separate system.

(c)    diversity of systems performing the same function.

(d) removal or limitation of the automated driving function(s).

If the chosen provision selects a partial performance mode of operation under certain fault conditions (e.g. in case of severe failures), then these conditions shall be stated (e.g. type of failure) and the resulting limits of effectiveness defined (e.g. immediate initiation of a minimum risk manoeuvre) as well as the warning strategy to the operator/remote operator, occupants and other road users (when applicable).

If the chosen design provision selects a second (back-up) or diverse means to realise the performance affected by the fault , the principles of the change-over mechanism, the logic and level of redundancy and any built-in checking features shall be explained and the resulting limits of effectiveness defined.

If the chosen design provision selects the removal of the automated driving function(s), this shall be done in compliance with the relevant provisions of this regulation. All the corresponding output control signals associated with this function shall be inhibited.

The manufacturer shall also provide the type-approval authority with an explanation of the operational safety measures to be put in place for the safe operation of the ADS such as an on-board operator or a remote intervention operator, supporting off-board infrastructure, transport and physical infrastructure requirements, maintenance measures, etc.

The documentation shall be supported, by an analysis that shows, how the ADS will behave to mitigate or avoid hazards that can have a bearing on the safety of vehicle occupants and other road users.

The chosen analytical approach(es) shall be established and maintained by the manufacturer and shall be made open for inspection by the type-approval authority at the time of the type-approval and afterwards.

The type-approval authority shall assess the application of the analytical approach(es):

(b)    Inspection of the safety approach at the ADS level including a top down (from possible hazard to design) and bottom up approach (from design to possible hazards). The safety approach may be based on a failure mode and effect analysis (FMEA), a fault tree analysis (FTA) and a System-theoretic process analysis (STPA) or any similar process appropriate to system functional and operational safety.

(c)    Inspection of the validation/verification plans and results including appropriate acceptance criteria. This shall include testing appropriate for validation, for example, hardware in the Loop (HIL) testing, vehicle on-road operational testing, testing with real end users, or any other testing appropriate for validation/verification. Results of validation and verification may be assessed by analysing coverage of the different tests and setting minimum coverage thresholds for various metrics.

The documentation shall confirm that at least each of the following items is covered where applicable under (a)-(c):

(i)    Issues linked to interactions with other vehicle systems (e.g. braking, steering);

(ii)    Failures of the automated driving system and system risk mitigation reactions;

(iii)    Situations within the ODD where a system may create unreasonable safety risks for the vehicle occupants and other road users due to operational disturbances (e.g. lack of or wrong comprehension of the vehicle environment, lack of understanding of the reaction from the operator/remote operator, vehicle occupants or other road users, inadequate control, challenging scenarios)

(iv)    Identification of the relevant scenarios within the boundary conditions and management method used to select scenarios and validation tool chosen.

(v)    Decision making process resulting in the performance of the dynamic driving tasks (e.g. emergency manoeuvres), for the interaction with other road users and in compliance with national traffic rules

(vi)    Reasonably foreseeable misuse by the vehicle occupants/other road users, mistakes or misunderstanding by the operator/remote operator/occupants/other road users (e.g. unintentional override) and intentional tampering of the ADS.

(viii)    Cybersecurity threats on the safety of the ADS (to be covered by the analysis done in accordance with UN Regulation No 155 on Cyber Security and Cyber Security Management System).

(viii)    Operational safety issues: problems with the supporting off-board infrastructure, problem with the remote intervention operator, loss of connectivity, lack of maintenance, etc.

The assessment by the type-approval authority shall consist of spot checks to establish that argumentation supporting the safety concept is understandable and logical and implemented in the different functions of the ADS. The assessment shall also check that validation plans are robust enough to demonstrate safety (e.g. reasonable coverage of chosen scenarios testing by the validation tool chosen) and have been properly completed.

It shall demonstrate that the operation of fully automated vehicle is free from unreasonable risks for the vehicle occupants and other road users in the operational design domain, i.e. through:

(a)    an overall validation target (i.e. overall validation acceptance criteria) supported by validation results, demonstrating that the entry into service of the ADS will overall not increase the level of risk for the vehicle occupants and other road users compared to a manually driven vehicles ; and

(b)    a scenario specific approach (i.e. scenario based validation acceptance criteria) showing that the ADS will overall not increase the level of risk for the vehicle occupants and other road users compared to a manually driven vehicles for each of the safety relevant scenarios;

The type-approval authority shall perform or shall require to perform tests as specified in point 4. of this Annex to verify the safety concept.

This documentation shall itemize the parameters being monitored and shall set out, for each failure condition of the type defined in point 3.5.4. of this annex, the warning signal to be given to the operator/remote operator/vehicle occupants/other road users and/or to service/technical inspection personnel.

This documentation shall also describe the measures in place to ensure the ADS is free from unreasonable risks to vehicle occupants, and other road users when the performance of the ADS is affected by environmental conditions e.g. climatic, temperature, dust ingress, water ingress, ice packing, inclement weather.

Verification and tests

Taking into account the results of the analysis of the manufacturer’s documentation package, the type-approval authority shall request the tests to be performed or witnessed by the Technical Service to check specific points arising from the assessment.

The functional operation of the ADS, as laid out in the documents required in point 3., shall be tested as follows:

Verification of the function of the ADS

The type-approval authority shall verify the ADS under non-failure conditions by testing on a track a number of selected functions, as deemed necessary by the type-approval authority, from those described by the manufacturer, and by checking the overall behaviour of the ADS in real driving conditions including compliance with traffic rules.

These tests shall include scenarios whereby the ADS is overridden by the remote intervention operator (if applicable).

These tests can be based on test scenarios listed in Part 3 of this Annex and/or on additional scenarios not covered by part 3.

The test results shall correspond with the description, including the control strategies, provided by the manufacturer in point 3.2. and shall comply with the performance requirements of this regulation.

Verification of the ADS safety concept

The reaction of the ADS shall be checked under the influence of a faults in any individual unit by applying corresponding output signals to electrical units or mechanical elements in order to simulate the effects of internal failure within the unit. The type-approval authority shall conduct or witness this check for at least one individual unit, but shall not check the reaction of the ADS to multiple simultaneous failures of individual units.

The type-approval authority shall verify that these tests include aspects that may have an impact on vehicle controllability and user information (HMI aspects e.g. interaction with the operator/remote operator).

The type-approval authorities shall also check a number of scenarios that are critical for the Object and Event Detection and Response (OEDR) and Characterisation of the decision-making and HMI functions of the ADS (e.g. object difficult to detect, when the ADS reaches the ODD boundaries, traffic disturbance scenarios, connectivity issue, problem with off board systems, remote capabilities issues e.g. the absence of the remote intervention operator) as defined in this regulation.

The verification results shall correspond with the documented summary of the hazard analysis, to a level of overall effect such that the safety concept and execution are confirmed as being adequate and in compliance with the requirements of this regulation.

Simulation tool and mathematical models to verify of the safety concept may be used in accordance with Annex VIII to Regulation (EU) 2018/858, in particular for scenarios that are difficult on a test track or in real driving conditions. Manufacturers shall demonstrate the scope of the simulation tool, its validity for the scenario concerned as well as the validation performed for the simulation tool chain (correlation of the outcome with physical tests).To demonstrate the validity of the simulation toolchain, the principles of Part 4 of this Annex shall apply. Simulation shall not be a substitute for physical tests in Part 3 of this Annex.

The manufacturer shall have a valid certificate of compliance for the safety management system (SMS) relevant to the vehicle type being approved.

Safety management system (SMS)

In respect of the ADS, the manufacturer shall demonstrate to the type-approval authority in terms of a safety management system (SMS) that effective processes, methodologies, training and tools are in place, up to date and being followed within the organization to manage the safety and continued compliance throughout the ADS lifecycle.

The design and development process shall be established and documented including safety management system, requirements management, requirements’ implementation, testing, failure tracking, remedy and release

The manufacturer shall ensure effective communication channels between manufacturer departments responsible for functional/operational safety, cybersecurity and any other relevant disciplines related to the achievement of vehicle safety.

The manufacturer shall have processes aimed at collecting vehicle data, and data from other sources to monitor to analyse safety-relevant incidents/accidents caused by the engaged automated driving system. The manufacturer shall report to type-approval authorities, market surveillance authorities and the Commission the relevant occurrences in accordance with part 5 of this Annex.

The manufacturer shall have processes to manage potential safety-relevant gaps post-registration and to update the vehicles if necessary.

The manufacturer shall demonstrate that periodic independent internal process audits (e.g. every 2 years) are carried out to ensure that the processes established in accordance with points 5.1 to 5.5. are implemented consistently.

Manufacturers shall put in place suitable arrangements (e.g. contractual arrangements, clear interfaces, quality management system) with suppliers to ensure that the supplier safety management system comply with the requirements of points 5.1. (except for vehicle related aspects like ‘operation’ and ‘decommissioning’), 5.2, 5.3 and 5.6.

5.8.

Certificate of compliance for safety management system

5.8.1.

An application for a Certificate of Compliance for Safety Management System shall be submitted by the manufacturer or by their duly accredited representative to the type-approval authority.

5.8.2.

It shall be accompanied by the undermentioned documents in triplicate, and by the following particular:

(a) Documents describing the Safety Management System.

(b) A signed Declaration of Compliance of the SMS with all the requirements for safety management according to this Regulation, using the model as defined in Appendix 3 to this Annex.

5.8.3.

When this audit of the SMS has been satisfactorily completed and in receipt of a signed declaration from the manufacturer according to the model as defined in Appendix 3, a certificate named Certificate of Compliance for SMS as described in Appendix 4 (hereinafter the Certificate of Compliance for SMS) shall be granted to the manufacturer.

5.8.4.

The Certificate of Compliance for SMS shall remain valid for a maximum of three years from the date of deliverance of the certificate unless it is withdrawn.

5.8.5.

The type-approval authority may at any time verify that the requirements for the Certificate of Compliance for SMS continue to be met. The type-approval authority shall withdraw the Certificate of Compliance for SMS if major non-conformities in the compliance with the requirements laid down in this Regulation are discovered and not immediately addressed.

5.8.6.

The manufacturer shall inform the type-approval authority or its technical service of any change that will affect the relevance of the certificate of compliance for SMS. After consultation with the manufacturer, the type-approval authority or its Technical Service shall decide whether new checks are necessary.

5.8.7.

In due time, the manufacturer shall apply for a new or for the extension of the existing Certificate of Compliance for SMS. The type-approval authority shall, subject to a positive audit, issue a new Certificate of Compliance for SMS or extend its validity for a further period of three years. The type-approval authority shall verify that the SMS continue to comply with the requirements of this Regulation. The type-approval authority shall issue a new certificate in cases where changes have been brought to the attention of the type-approval authority or its Technical Service and the changes have been positively re-assessed.

5.8.8.

The expiry or withdrawal of the manufacturer’s Certificate of Compliance for SMS shall be considered, with regard to the vehicle types to which the SMS concerned was relevant, as modification of approval, which may include the withdrawal of the approval if the conditions for granting the approval are not met anymore.

Reporting provision

The reporting of the safety assessment of the ADS safety concept as well as the audit of the safety management system of the manufacturer shall be performed in such a manner that allows traceability, e.g. versions of documents inspected are coded and listed in the records of the Technical Service.

An example of layout for the report on the assessment of the ADS safety concept from the Technical Service to the type-approval authority is provided in Appendix 1 to this part. The listed items in this Appendix are outlined as minimum set of items that need to be covered.

The granting type-approval authority shall issue the safety assessment results to be annexed to the type-approval certificate based on the documentation provided by the manufacturer, the report of the assessment of the ADS safety concept by the technical service and on the outcomes of the verification and test campaigns performed in accordance with Part 3 of this Annex. An example of a possible layout for the safety assessment results is given in Appendix 4.

Competence of the auditors/assessors

The assessment of the ADS safety concept and the audit of the safety management system under this part shall only be conducted by assessors/auditors with the technical and administrative knowledge necessary for such purposes. They shall in particular be competent as auditor/assessor for ISO 26262-2018 (Functional Safety - Road Vehicles), and ISO/PAS 21448 (Safety of the Intended Functionality of road vehicles); and shall be able to make the necessary link with cybersecurity aspects in accordance with UN Regulation No 155 and ISO/SAE 21434). This competence shall be demonstrated by appropriate qualifications or other equivalent training records.

Safety assessment report No:

Identification.

Vehicle make:

Vehicle type

Means of identification of vehicle type if marked on the vehicle:

Location of that marking:

Manufacturer’s name and address:

If applicable, name and address of manufacturer’s representative:

Manufacturer’s formal documentation package:

Documentation reference No:

Date of original issue:

Date of latest update:

Assessment method

Description of the assessment processes and methodologies

Acceptability criteria

Results of the review of the documentation package

Review of the ADS description.

Review of Manufacturer’s safety concept and the manufacturer analysis.

Review of the Verification and Validation performed by the manufacturer

Review of the methods and tools (software, laboratory, others) and the credibility assessment

Review of ADS data requirements and specific data elements for event data recorder for fully automated vehicles

Checks of the Cyber Security and Software Updates certificates are covering he ADS.

Review of the information provided in the Operating Manual

Review of the provisions for the periodic roadworthiness tests of the ADS:

Review of additional information not included in the Information Document

Verification of ADS functions under non-failure conditions (referred to in point 4.1.1. of Annex 3 Part 2 to Regulation (EU) …/…(1) [PO: this Regulation])

Rationale for the selection of test scenarios

Selected test scenarios

Test reports

Test No. (add as many as the tests performed)

Objectives of the test

Test conditions

Measured quantities and measuring devices

Acceptability criteria

Test results

Comparison with the manufacturer’s supplied documentation

Verification of ADS safety concept under single failure (referred to in point 4.1.2. of Annex 3 Part 2 to Regulation (EU) …/…(1) [PO: this Regulation])

Rationale for the selection of test scenarios

Selected test scenarios

Test reports

Test No. (add as many as the tests performed)

Objectives of the test

Test conditions

Measured quantities and measuring devices

Acceptability criteria

Test results

Comparison with the manufacturer’s supplied documentation

Safety management system certificate (shall be appended to this test report)

Date of the assessment

Final judgement on the safety assessment outcome

This assessment has been carried out and the results reported in accordance with Commission Implementing Regulation (EU) …/…(1) [PO: this Regulation]

Technical Service carrying out the assessment
Signed: .......................................        Date: ........................................

Comments:

Identification.

Vehicle make

Vehicle Type

Means of identification of vehicle type if marked on the vehicle:

Location of that marking:

Manufacturer’s name and address:

If applicable, name and address of manufacturer’s representative:

Manufacturer’s formal documentation package:

Documentation reference No:

Date of original issue:

Date of latest update:

Assessment method

Description of the assessment processes and methodologies

Acceptability criteria

Verification of ADS functions under non-failure conditions (referred to in point 4.1.1. of Annex 3 Part 2 to Regulation (EU) …/…(1) [PO: this Regulation])

Rationale for the selection of test scenarios

Selected test scenarios

Verification of ADS safety concept under single failure (referred to in point 4.1.2. of Annex 3 Part 2 to Regulation (EU) …/…(1) [PO: this Regulation])

Rationale for the selection of test scenarios

Selected test scenarios

Assessment results

Results of the review of the Information Document

Results of the verification of ADS functions under non-failure conditions

Results of the verification of ADS safety concept under single failure

Results of the assessment of the Safety Management System

Results of the verification of provisions for the periodic roadworthiness tests

Final judgement on the safety assessment outcome

General provisions

Pass- and fail-criteria to assess ADS safety shall be based on the requirements set out in Annex 2 and the scenario described in Part 1 of this annex. The requirements are defined in such a way that the pass/fail criteria can be derived not only for a specific set of test parameters, but also for all safety-relevant combinations of parameters that may occur in the operating conditions covered by the type approval and the specified operating range (e.g., speed range, longitudinal and transverse acceleration range, radii of curvature, brightness, number of lanes). For conditions not tested but that may occur within the defined ODD of the system, the manufacturer shall demonstrate as part of the assessment described in part 2 to the satisfaction of the type-approval authority, that the vehicle is safely controlled.

These tests shall confirm the minimum performance requirements described in Annex 2 and the functionality of the ADS and the safety concept of the manufacturer as described in Part 2 of this Annex. Test results shall be documented and reported in accordance with point 6 of part 2 of this annex.

The test site shall comprise characteristics (example: friction value) that correspond to the specified ODD of the ADS. The ADS shall also be tested on-road in accordance with the applicable law of the Member State granting the type-approval and provided that tests can be carried out safely and without any risk to other road users.

Vehicle conditions

The subject vehicle shall be tested with any permissible vehicle load. No load alteration shall be made once the test procedure has begun. The manufacturer shall demonstrate, through the use of documentation, that the ADS works at all load conditions.

5.3.

It shall be verified, that the condition of the system is according to the intended testing purpose (e.g. in a fault-free condition or with the specific faults to be tested).

6.

Test tools

In addition to real vehicles, state-of-the-art test tools may be used to carry out the tests, replacing real vehicles and other road users (e.g., soft targets, mobile platforms, etc.). The replacement test tools shall comply with the characteristics relevant for sensory performance assessment, real vehicles and other traffic participants. Tests shall not be carried out in a way that would endanger the personnel involved, and significant damage of the vehicle being tested must be avoided where other means of validation are available.

The manufacturer shall declare the system boundaries to the type-approval authority. The type-approval authority shall define different combinations of test parameters (e.g. present speed of the vehicle, type and offset of target, curvature of lane, etc.) in order to test the ADS. The selected test cases shall provide sufficient test coverage for all scenarios, test parameters and environmental influences. Adequate robustness of the perceptions systems for the ADS against input/sensor data malfunction and adverse environmental conditions shall be demonstrated.

Test parameters selected by the type-approval authority shall be recorded in a test report in a manner that allows traceability and repeatability of the test setup.

Tests scenarios to assess the performance of the ADS on a test track (points 8.1., 8.2., 8.5, 8.6, 8.7, 8.8, 8.9.) and on-road (8.3., 8.4., 8.10.)

The scenarios included in the following points have to be considered a minimum set of tests. At the request of the type-approval authority, additional scenarios that are part of the ODD can be executed. If a scenario described in point 8 of this annex does not belong to the ODD of the vehicle, it shall not be taken into consideration.

Depending on the ODD, test scenarios shall be selected as part of the type-approval test. The test scenarios shall be selected in accordance with Part 1 of this annex. Type-approval testing may be carried out on the basis of simulations, manoeuvres on the test track and driving tests on real road traffic. However, it may not be based solely on computer simulations and at the time of type-approval, the type-approval authority shall conduct or shall witness at least the following tests to assess the behaviour of the ADS.

Lane keeping

The test shall demonstrate that the fully automated vehicle does not leave its lane and maintains a stable motion inside its lane across the speed range and different curvatures within its system boundaries.

The test shall be based on the ODD of the ADS and shall be executed at least :

a)with a minimum test duration of 5 minutes;

b)with a passenger car target as well as a power-two-wheeler (PTW) target as the other vehicle;

c)with a lead vehicle swerving in the lane; and

d)with another vehicle driving close beside in the adjacent lane.

Lane changing manoeuvre (LCM)

The tests shall demonstrate that the fully automated vehicle does not cause an unreasonable risk to safety of the vehicle occupants and other road users during a lane change procedure, and that the ADS is able to assess the criticality of the situation before starting the lane change manoeuvre (LCM) throughout the entire operational speed range. These tests are only required if the fully automated vehicle is capable of performing lane changes either during a Minimal Risk Manoeuvre or during regular operation.

a)with the fully automated vehicle performing lane change to the adjacent (target) lane;

b)merging at lane end;

c)merging into an occupied lane.

a)with different vehicles, including a PTW approaching from the rear;

b)in a scenario where it is possible to execute a lane changing manoeuvre in regular operation;

c)in a scenario where a lane changing manoeuvre in regular operation is not possible due to a vehicle approaching from the rear;

d)with an equally fast vehicle following behind in the adjacent lane, preventing a lane change;

e)with a vehicle driving beside in the adjacent lane preventing a lane change;

f)in a scenario where a LCM during a minimal risk manoeuvre is possible and executed;

g)in a scenario where the fully automated vehicle reacts to another vehicle that starts changing into the same space within the target lane, to avoid a potential risk of collision.

These tests shall ensure, that the fully automated vehicle detects and adapts to a variation of different road geometries which can occur within the intended ODD across its whole speed range.

The test shall be executed with at least the list of scenarios below based on the ODD of the ADS:

a)T-junctions (3-way intersections) with and without traffic lights, with different rights of way;

b)crossroads (4 or more way intersections) with and without traffic lights, with different rights of way;

c)roundabouts.

Each test shall be executed at least:

a)without a lead vehicle;

b)with a passenger car target as well as a PTW target as the lead vehicle / other vehicle;

d)with and without approaching or passing vehicles.

Response to national traffic rules and road infrastructure

These tests shall ensure that the fully automated vehicle complies with national traffic rules and that it adapts to a various permanent and temporary changes of the road infrastructure (e.g. road construction sites) in the entire speed range.

The tests shall be executed with at least with the list of scenarios below that are relevant for the ODD of the ADS:

a)different speed limit signs, so that the ADS has to change its speed according to the indicated values;

b)signal lights and/or stop instructed by a road safety officer / enforcement agents with situations of going straight, turning left and right;

c)pedestrian and cyclist crossings with and without pedestrians/cyclist approaching / on the road.

d)temporary modifications: e.g., road maintenance operations indicated by traffic signs, cones and other signalisation, access restrictions.

e)motorway entry, exit and toll stations.

Each test shall be executed at least:

a)without a lead vehicle;

f)with a passenger car target as well as a PTW target as the lead vehicle / other vehicle.

Collision avoidance: Avoid a collision with road users or objects blocking the lane

The test shall demonstrate that the fully automated vehicle avoids a collision with a stationary vehicle, road user or fully or partially blocked lane up to the maximum specified speed of the ADS.

This test shall be executed at least with the following scenarios, where relevant in the ODD:

a)with a stationary passenger car target;

b)with a stationary PTW target;

c)with a stationary pedestrian target;

d)with a pedestrian target crossing the lane with a speed of 5 km/h, also in the presence of other objects relevant in the ODD (e.g. a ball, a shopping bag, etc.);

e)with a pedestrian target moving within and partially occupying the lane of the ADS and following the same or the opposite direction of the ADS with a speed of up to 5 km/h;

f)with a pedestrian target swerving in the same lane of the ADS;

g)with a cyclist target crossing the lane with a speed of 15 km/h;

h)with a cyclist target which travelling in the same direction with a speed of 15 km/h

i)with a target representing a blocked lane;

j)with a target partially within the lane;

k)with one or more different types of unpassable objects relevant in the ODD (e.g., a dustbin, a fallen bicycle or scooter, a fallen traffic sign, a stationary or moving ball, etc.);

l)with multiple consecutive obstacles blocking the lane relevant in the ODD (e.g., in the following order: ego-vehicle -motorcycle - car);

m)on a curved section of road.

Avoid emergency braking before a passable object in the lane. A ‘passable object’ is such an object, that may be rolled over without causing an unreasonable risk to the vehicle occupants or other road users.

The test shall demonstrate that the fully automated vehicle is not is not initiating an Emergency Braking with a deceleration demand greater than 5 m/s2 due to a passable object in the lane relevant for the ODD (e.g., a manhole lid or a small branch) up to the maximum specified speed of the ADS.

This test shall be executed at least with the following scenarios, where relevant in the ODD:

a)without a lead vehicle;

b)with a passenger car target as well as a PTW target as the lead vehicle / other vehicle.

Following a lead vehicle

The test shall demonstrate that the fully automated vehicle is able to maintain and restore a stable motion and a safety distance to a vehicle in front and is able to avoid a collision with a lead vehicle which decelerates up to its maximum deceleration.

This test shall be executed at least with the following scenarios, where relevant in the ODD:

a)across the entire speed range of the fully automated vehicle;

b)using a passenger car target as well as a PTW target as lead vehicle, provided standardised PTW targets suitable to safely perform the test are available;

c)for constant and varying lead vehicle velocities (realistic speed profile);

d)for straight and curved sections of road;

e)for different lateral positions of lead vehicle in the lane;

f)with a deceleration of the lead vehicle of at least 6 m/s2 mean fully developed deceleration until standstill.

Lane change of another vehicle into lane (cut-in)

The test shall demonstrate that the fully automated vehicle is capable of avoiding a collision with a vehicle or other road user cutting into the lane of the fully automated vehicle up to a certain criticality of the cut-in manoeuvre.

The criticality of the cut-in manoeuvre shall be determined according to the provisions introduced Part 1 of this annex and depending on the distance between the rear-most point of the cutting-in vehicle and front-most point of the fully automated vehicle.

The test shall be executed at least with the following scenarios, where relevant for the ODD:

a)with different TTC, distance and relative velocity values of the cut-in manoeuvre, covering types of cut-in scenarios in which a collision can be avoided and those in which a collision cannot be avoided;

b)with cutting-in vehicles travelling at constant longitudinal speed, accelerating and decelerating;

c)with different lateral velocities, lateral accelerations of the cut-in vehicle;

d)with a passenger car as well as PTW targets as the cutting-in vehicle, provided standardised PTW targets suitable to safely perform the test are available.

Stationary obstacle after lane change of the lead vehicle (cut-out)

The test shall demonstrate that the fully automated vehicle is capable of avoiding a collision with a stationary vehicle, road user or blocked lane that becomes visible after a preceding vehicle avoided a collision by an evasive manoeuvre. The test shall be based on the requirements set out in Annex 2 and scenario parameters in part 1 of this annex. For conditions not tested that may occur within the defined operating range of the vehicle, the manufacturer shall demonstrate as part of the assessment described in Annex 3, Part 2 to the satisfaction of the relevant authorities that the vehicle is safely controlled.

The test shall be executed at least with the following scenarios, where relevant for the ODD:

a)with a stationary passenger car target centred in lane;

b)with a PTW target centred in lane;

c)with a stationary pedestrian target centred in lane;

d)with a target representing a blocked lane centred in lane;

e)with multiple consecutive obstacles blocking the lane (e.g. in the following order: ego-vehicle – lane change vehicle – motorcycle – car).

Parking

The test shall demonstrate that the fully automated vehicle is able to park in different parking spaces and parking layouts under different conditions; and that during the parking manoeuvre it is not causing damage to the surrounding objects, road users and itself.

The test shall be executed at least with the following scenarios, where relevant for the ODD:

(a) with parking spaces parallel and perpendicular to the road;

(b) on even and slant surfaces;

(c) with other vehicles in the surrounding parking spaces, including PTWs and bicycles;

(d) parking to parking spaces with different geometrical dimensions;

(e) on different road slope angles;

(f) with another vehicle cutting in to the parking space during the parking manoeuvre.

Navigating in a parking facility

The test shall demonstrate that the ADS is able to handle the low driving speed and the general lack of visibility that may occur in a parking.

The test shall be executed at least with the following scenarios, where relevant for the ODD:

a)    with an initially obstructed pedestrian target crossing the path of the ADS with a speed of 5 km/h.

b)    with a vehicle driving out of a parking place in front of the ADS.

c)    with a stationary obstacle in the path of the ADS.

d)    with different paths, where the infrastructure is obstructing the field of view.

e)    with a small obstacle on the floor after a ramp obstructed by other objects in the path of the ADS.

Specific scenarios for motorway

Motorway entry

The test shall demonstrate that the ADS is able to safely enter the motorway.

The test shall be executed at least with the following scenarios, where relevant for the ODD:

a)    with different vehicles, including a PTW approaching from the rear;

b)    with vehicles approaching with different speed from the rear;

c)    with a platoon of vehicles driving beside in the adjacent lane.

Motorway exit

The test shall demonstrate that the ADS is able to safely exit the motorway.

The test shall be executed at least with the following scenarios, where relevant for the ODD:

a)    without a lead vehicle;

b)    with a passenger car target as well as a PTW target as the lead vehicle / other vehicle.

c)    with other vehicle(s) or obstacle(s) blocking the motorway exit.

Toll station

The test shall demonstrate that the ADS is able to select the proper passing gate, and adapt its speed to that permitted within the toll area.

The test shall be executed at least with the following scenarios, where relevant for the ODD:

a)    with and without a lead vehicle;

b)    with other vehicles blocking the passing gate(s);

c)    with closed and opened passing gates.

d)    with different permitted speeds in the toll area.

For dual mode vehicles, transition between the manual driving mode and the fully automated mode.

The test shall demonstrate that the ADS takes over the DDT in a safe manner and only when the vehicle is standstill.

The test shall be executed at least with the following scenarios, where relevant for the ODD:

a)with and without human driver present in the vehicle;

b)with open and closed vehicle doors;

c)with and without obstacles around the vehicle,

d)inside and outside of the dedicated transition area, if applicable

The test shall be executed at least with the following scenarios, where relevant for the ODD:

a)in a situation where transition is possible and executed;

b)in a situation where transition is not possible to be executed.

General

The credibility can be achieved by investigating and assessing five properties of Modelling and Simulation (M&S):

(a)    capability – what can the M&S do, and what the risks are associated with it;

(b)    accuracy – how well does M&S reproduce the target data;

(c)    correctness – how sound & robust are M&S data and algorithms;

(d)    usability – what training and experience is needed.

(e)    fit for purpose – how suitable is the M&S for the ODD and ADS assessment.

At the same time, the credibility assessment framework shall be general enough to be used for different M&S types and applications. However, the goal is complicated by the broad differences between ADS features and the variety of M&S types and applications. These considerations require a (risk-based/informed) credibility assessment framework relevant and appropriate to all M&S applications.

The credibility assessment framework provides a general description of the main aspects considered for assessing the credibility of an M&S solution together with principles on the role of third parties assessors in the validation process with respect to credibility. Concerning the latter point, the type-approval authority shall investigate the produced documentation supporting credibility at the assessment phase, whereas the actual validation tests occur once the manufacturer has developed the integrated simulation systems.

Ultimately, the outcome of the current credibility assessment shall define the envelope in which the virtual tool can be used to support the ADS assessment.

The requirements of this part are therefore intended to demonstrate the credibility of any simulation model or virtual toolchain for its use in ADS validation.

Definitions
For the purpose of this annex

‘abstraction’ means the process of selecting the essential aspects of a source system or referent system to be represented in a model or simulation, while ignoring non-relevant aspects. Any modelling abstraction carries with it the assumption that shall not significantly affect the intended uses of the simulation tool.

‘closed loop testing’ means a virtual environment that takes the actions of the element-in-the loop into account. Simulated objects respond to the actions of the system (e.g. system interacting with a traffic model).

‘deterministic’ means a term describing a system whose evolution over time can be predicted exactly and a given set of input stimuli will always produce the same output.

‘driver-in-the-loop (DIL)’ is typically conducted in a driving simulator used for testing the human–automation interaction design. DIL has components for the driver to operate and communicate with the virtual environment.

‘Hardware-In-the-Loop (HIL)’ involves the final hardware of a specific vehicle sub-system running the final software with input and output connected to a simulation environment to perform virtual testing. HIL testing provides a way of replicating sensors, actuators and mechanical components in a way that connects all the I/O of the electronic control units (ECU) being tested, long before the final system is integrated.

‘model’ is a description or representation of a system, entity, phenomenon, or process.

‘model calibration’ is the process of adjusting numerical or modelling parameters in the model to improve agreement with a referent.

‘model Parameter’ are numerical values used to support characterizing a system functionality. A model parameter has a value that cannot be observed directly in the real world but that must be inferred from data collected in the real world (in the model calibration phase).

‘model-in-the-loop (MIL)’ is an approach which allows quick algorithmic development without involving dedicated hardware. This level of development usually involves high-level abstraction software frameworks running on general-purpose computing systems.

‘open loop testing’ means a virtual environment that does not take the actions of the element-in-the loop into account (e.g. system interacting with a recorded traffic situation).

‘probabilistic’ is a term pertaining to non-deterministic events, the outcomes of which are described by a measure of likelihood.

‘proving ground or test-track’ is a physical testing facility closed to the traffic where the performance of an ADS can be investigated on the real vehicle. Traffic agents can be introduced via sensor stimulation or via dummy devices positioned on the track.

‘sensor Stimulation’ is a technique whereby artificially generated signals are provided to the element under testing in order to trigger it to produce the result required for verification of the real world, training, maintenance, or for research and development.

‘simulation’ is the imitation of the operation of a real-world process or system over time.

‘simulation model’ is a model whose input variables vary over time.

‘simulation toolchain’ is a combination of simulation tools that are used to support the validation of an ADS.

‘software-in-the-loop (SIL)’ is where the implementation of the developed model will be evaluated on general-purpose computing systems. This step can use a complete software implementation very close to the final one. SIL testing is used to describe a test methodology, where executable code such as algorithms (or even an entire controller strategy), is tested within a modelling environment that can help prove or test the software.

‘stochastic’ means a process involving or containing a random variable or variables. Pertaining to chance or probability.

‘validation of the simulation model’ is the process of determining the degree to which a simulation model is an accurate representation of the real world from the perspective of the intended uses of the tool.

‘vehicle-in-the-loop (VIL)’ is a fusion environment of a real testing vehicle in the real-world and a virtual environment. It can reflect vehicle dynamics at the same level as the real-world and it can be operated on a vehicle test bed or on a test track.

‘verification of the simulation model’ is the process of determining the extent to which a simulation model or a virtual testing tool is compliant with its requirements and specifications as detailed in its conceptual models, mathematical models, or other constructs.

‘virtual testing’ is the process of testing a system using one or more simulation models.

Components of the credibility assessment framework and related documentation requirements

The credibility assessment framework introduces a way to assess and report the credibility of M&S based on quality assurance criteria where the levels of confidence in the results can be indicated. In other words, the credibility is established by evaluating the following M&S influencing factors that are considered as main contributors for M&S properties and therefore for the overall M&S credibility: (a) M&S management; (b) team’s experience and expertise; (c) M&S analysis and description; (d) data/input pedigree and (e) verification; validation, uncertainty Characterisation . Each of these factors indicates the level of quality achieved by M&S, and the comparison between the obtained levels and the required levels shall determine whether the M&S is credible and fit to use for virtual testing. A graphical representation of the relationship between the components of the credibility assessment framework is shown below.

Models and simulation management.

The M&S lifecycle is a dynamic process with frequent releases that shall be monitored and documented. Management activities shall be established to support the M&S in a work product management fashion. Relevant information on the following aspects shall be provided.

The M&S management process shall:

(a)    Describe the modifications within the releases;

(b)    Designate the corresponding software (e.g., specific software product and version) and hardware arrangement (e.g., XiL configuration);

(c)    Record the internal review processes that accepted the new releases;

(d)    Be supported throughout the full duration of the virtual model utilization.

Release management.

Any M&S toolchain’s version used to release data for certification purposes shall be stored. The virtual models constituting the testing toolchain shall be documented in terms of the corresponding validation methods and acceptance thresholds to support the overall credibility of the toolchain. The developer shall enforce a method to trace generated data to the corresponding M&S version.

Quality check of virtual data. Data completeness, accuracy, and consistency shall be ensured throughout the releases and lifetime of an M&S toolchain to support the verification and validation procedures.

Team's experience and expertise.

Even though experience and expertise (E&E) are already covered in a general sense within the organization, it is important to establish the basis for confidence in the specific E&E for M&S activities.

The credibility of M&S depends not only on the quality of the simulation models but also on the E&E of the personnel involved in the validation and usage of the M&S. For instance, a proper understanding of the limitations and validation domain will prevent the possible misuse of M&S or misinterpretation of its results.

Therefore, it is important to establish the basis for the manufacturer’s confidence on the E&E of:

(a)    The Teams that will validate the simulation toolchain and,

(b)    The Teams that will use the validated simulation for the execution of virtual testing with the purpose of validating the ADS.

A proper management of the team’s E&E increases the level of confidence on the credibility of M&S and its outcomes by ensuring that the human factors behind the M&S are taken into consideration and any possible human component risk is controlled, as is expected in any suitable Management System

If the manufacturer's tool chain incorporates or relies upon inputs from organisations or products outside of the manufacturer's own team, the manufacturer will provide an explanation of measures it has taken to support its confidence in the quality and integrity of those inputs.

Team’s E&E consists of two levels.

Organizational level

The credibility is established by setting up processes and procedures to identify and maintain skills, knowledge, and experience to perform M&S activities. The following processes shall be established, maintained and documented:

(i)    Process to identify and evaluate the individual’s competence and skills;

(ii)    Process for training competent personnel to perform M&S-related duties

Team level

Once a M&S has been finalised, its credibility is mainly dictated by the skills and knowledge of the individual/team that will validate the M&S toolchain and use the M&S for the validation of the ADS. Credibility is established by documenting that these teams have received adequate training to fulfil their duties.

The manufacturer shall then:

(i)    provide the basis for the manufacturer’s confidence in the E&E of the individual/team that validates the M&S Toolchain.

(ii)    provide the basis for the manufacturer’s confidence in the E&E of the individual/team that uses the simulation to carry out virtual testing in order to validate the ADS.

The manufacturer’s demonstration of how it applies the principles of ISO 9001 or a similar best practice or standard to ensure the competence of its M&S organization and the individuals in that organisation will be the basis for this determination. The type-approval authority may not substitute its judgment on the E&E of the organisation or its members with that of the manufacturer.

Data/input pedigree

The data/input pedigree contains a record of traceability from the manufacturer’s data used in the validation of the M&S.

Description of the data used for the M&S

(a)    The manufacturer shall document the data used to validate the model and note important quality characteristics;

(b)    The manufacturer shall provide documentation showing that the data used to validate the models covers the intended functionalities that the toolchain to virtualise;

(c)    The manufacturer shall document the calibration procedures used to fit the virtual models’ parameters on the collected input data.

Effect of the data quality (e.g. data coverage, signal to noise ratio, and sensors’ uncertainty/bias/sampling rate) on model parameters uncertainty.

The quality of the data used to develop the model will affect the estimation and calibration of the model parameters. Uncertainty in model parameters will be another important aspect in the final uncertainty analysis.

Data/output pedigree

The data/output pedigree contains a record of the M&S outputs used for the ADS validation.

Description of the data generated by the M&S

(a)    The manufacturer shall provide information on any data and scenarios used for virtual testing toolchain validation.

(b)    The manufacturer shall document the exported data and note important quality characteristics.

(c)    The manufacturer shall trace a M&S output to the corresponding simulation setup

Effect of data quality on M&S credibility

(a)    The M&S output data shall be sufficiently wide to ensure the correct execution of the validation computation. The data shall sufficiently reflect the ODD relevant to the virtual assessment of the ADS.

(b)    The output data shall allow for a consistency/sanity check of the virtual models via possibly exploiting redundant information

Managing stochastic models

(a)    Stochastic models shall be characterised in terms of their variance

(b)    Stochastic models shall be ensured the possibility of deterministic re-execution

M&S analysis and description

The M&S analysis and description aim to define the whole M&S and identify the parameter space that can be assessed via virtual testing. It defines the scope and limitations of the models and toolchain and the uncertainty sources that can affect its results.

General description

The manufacturer shall provide a description of the complete toolchain along with how the simulation data will be used to support the ADS validation strategy

The manufacturer shall provide a clear description of the test objective

Assumptions, known limitations and uncertainty sources

The manufacturer shall motivate the modelling assumptions that guided the design of the M&S toolchain

The manufacturer shall provide evidence on:

(i)    How the manufacturer-defined assumptions play a role in defining the limitations of the toolchain;

(ii)    The level of fidelity required for the simulation models

The manufacturer shall provide justification that the tolerance for sim-real correlation is acceptable for the test objective

Finally, this section shall include information on the sources of uncertainty in the model. This will represent an important input to final uncertainty analysis, which will define how the model outputs can be affected by the different sources of uncertainty of the model used.

Scope (how the M&S is used in the ADS validation)

The credibility of the virtual tool shall be enforced by a clearly-defined scope of utilization the developed models.

The matured M&S shall allow a virtualisation of the physical phenomena to a degree of accuracy which matches the fidelity level required for certification. Thus, the M&S will act as a ‘virtual proving ground’ for ADS testing.

Simulation models need dedicated scenarios and metrics for validation. The scenario selection used for validation shall be sufficient to that the toolchain will perform in the same manner in scenarios outside of the validation scope.

The Manufacturer shall provide a list of validation scenarios together with the limitations of the corresponding parameters.

The ODD analysis is a crucial input to derive requirements, scope and effects that the M&S must consider in order to support ADS validation.

Parameters generated for the scenarios will define extrinsic and intrinsic data for the toolchain and the simulation models.

Criticality assessment

The simulation models and the simulation tools used in the overall tool-chain shall be investigated in terms of their responsibility in case of a safety error in the final product. The proposed approach for criticality analysis is derived from ISO 26262, which requires qualification for some of the tools used in the development process.

In order to derive how critical the simulated data are, the criticality assessment shall consider the following parameters:

(a)    The consequences for human safety e.g. severity classes in ISO 26262.

(b)    The degree to which the simulated results influence’s the ADS

From the perspective of the criticality assessment, the three possible cases for assessment are:

(a)    Those models or tools that fall within the red boxes are clear candidates for fully following the credibility assessment;

(b)    Those models or tools that fall within the yellow boxes may or may not be candidates for fully following the credibility assessment at the discretion of the type-approval authority;

(c)    Those models or tools that fall within the green boxes are not required to follow the credibility assessment.

Verification

The verification of an M&S involves analysing the correct implementation of the conceptual/mathematical models building up the M&S toolchain. The verification contributes to the M&S’s credibility by providing assurance that the M&S will not exhibit unrealistic behaviour for a set of inputs that cannot be tested. The procedure is based on a multi-step approach including code verification, calculation verification and sensitivity analysis.

Code verification

Code verification involves tests demonstrating that no numerical/logical flaws affect the virtual models

The manufacturer shall document the execution of proper code verification techniques, e.g. static/dynamic code verification, convergence analysis and comparison with exact solutions if applicable

The manufacturer shall provide documentation showing that the exploration in the domain of the input parameters was sufficiently wide to identify parameters’ combination for which the M&S shows unstable or unrealistic behaviour. Coverage metrics of combinations of parameters may be used to demonstrate the required exploration of the models behaviours.

The manufacturer shall adopt sanity/consistency checking procedures whenever data allows.

Calculation verification

Calculation verification deals with the estimation of numerical errors affecting the M&S

The manufacturer shall document numerical error estimates (e.g. discretization error, rounding error, iterative procedures convergence);

The numerical errors shall be kept sufficiently bounded to not affect validation.

Sensitivity analysis

Sensitivity analysis aims to quantify how model output values are affected by changes in the model input values and thus to identify the parameters having the greatest impact on the simulation model results. The sensitivity study also helps to determine the extent to which the simulation model satisfies the validation thresholds when it is subjected to small variations of the parameters. It is therefore fundamental to support the credibility of the simulation results.

The manufacturer shall provide supporting documentation demonstrating that the most critical parameters influencing the simulation output have been identified by means of sensitivity analysis techniques such as by applying a perturbation of the model’s parameters;

The manufacturer shall demonstrate that robust calibration procedures have been adopted when identifying and calibrating the most critical parameters in order to increase the credibility of the developed toolchain.

Ultimately, the sensitivity analysis results will also help to define the inputs and parameters whose uncertainty Characterisation needs particular attention in order to properly define the uncertainty of the simulation results.

Validation

The quantitative process of determining the degree to which a model or a simulation is an accurate representation of the real world from the perspective of the intended uses of the M&S. It requires the selection and definition of several elements.

Measures of performance (metrics)

The measures of performance are the metrics used to compare the simulation model with the real world. Measures of performance are defined during the M&S analysis.

Metrics for validation may include:

(i)    discrete value analysis e.g. detection rate, firing rate;

(ii)    time evolution e.g. positions, speeds, acceleration;

(iii)    flow of actions based analysis e.g. distance/speed calculations, TTC calculation, brake initiation.

Goodness of fit measures

The analytical frameworks are used to compare real world and simulation metrics. They are generally key performance indicators (KPIs) indicating the statistical comparability between two sets of data.

The validation shall show that these KPIs are met.

Validation methodology

The manufacturer shall define the logical scenarios used for virtual testing toolchain validation. They shall be able to cover to the maximum possible extent the ODD of virtual testing for ADS validation

The exact methodology depends on the structure and purpose of the toolchain. The validation may consist of one or more of the following:

(i)    validate subsystem models e.g. environment model (road network, weather conditions, road user interaction), sensor models (radio detection and ranging (RADAR), light detection and ranging (LiDARs), camera), vehicle model (steering, braking, powertrain);

(ii)    validate vehicle system (vehicle dynamics model together with the environment model);

(iii)    validate sensor system (sensor model together with the environment model);

(iv)    validate integrated system (sensor model + environment model with influences form vehicle model).

Accuracy requirement

The requirement for the correlation threshold is defined during the M&S analysis. The validation shall show that these KPIs are met. e.g. using the correlation methodologies as defined in Annex 2.

Validation scope (the part of the toolchain to be validated)

A toolchain consists of multiple tools, and each tool will use a number of models. The validation scope includes all tools and the relevant models subject to validation.

Internal validation results

The documentation shall not only provide evidence of the simulation model validation but shall also be used to obtain sufficient information on the processes and products that provide overall credibility of the toolchain used.

Documentation/results may be carried over from previous credibility assessments.

Independent validation of results

The type-approval authority shall assess the documentation provided by the manufacturer and may carry out physical tests of the complete integrated tool

Uncertainty characterisation

This section is concerned with characterising the expected variability of the virtual toolchain results. The assessment shall be consist of two phases. In a first phase the information collected in the M&S analysis and description and the data/input pedigree sections are used to characterise the uncertainty in the input data, in the model parameters and in the modelling structure. Then, by propagating all the uncertainties through the virtual toolchain, the uncertainty in the model results is quantified. Depending on the uncertainty in the model results, proper safety margins will need to be introduced by the manufacturer in the use of virtual testing for ADS validation.

Characterisation of the uncertainty in the input data

The manufacturer shall demonstrate to have appropriately estimated the critical model’s inputs by means of robust techniques such as multiple repetitions for the assessment of the quantity;

Characterisation of the uncertainty in the model parameters (following calibration)

The manufacturer shall demonstrate that the critical model’s parameters that cannot be estimated identically are characterised by means of a distribution and/or confidence intervals;

Characterisation of the uncertainty in the M&S structure

The manufacturer shall provide evidence that the modelling assumptions are given a quantitative characterisation of the generated uncertainty (e.g. comparing the output of different modelling approaches whenever possible).

Characterisation of aleatory vs. epistemic uncertainty:

The manufacturer shall aim to distinguish between the aleatory component of the uncertainty (which can only be estimated but not reduced) and the epistemic uncertainty deriving from the lack of knowledge in the virtualisation of the process (that can instead be reduced).

Documentation structure

This section sets out how the above information will be collected and organised in the documentation provided by the manufacturer to the relevant authority.

The manufacturer shall produce a document (a ‘simulation handbook’) structured in line with the present outline to provide evidence for the topics presented.

The documentation shall be delivered together with the corresponding release of the M&S and related produced data.

The manufacturer shall provide clear references that allow the documentation to be traced back to the corresponding M&S/data;

The documentation shall be maintained throughout the whole lifecycle of the M&S utilization. The type-approval authority may audit the manufacturer by assessing their documentation and/or by conducting physical tests.

Definitions

For the purpose of this annex,

‘Occurrence’ refers to safety-related situation involving a vehicle equipped with an automated driving system.

‘Non-critical Occurrence’ means an occurrence involving an operational interruption, defect, fault or other circumstance that has or may have influenced ADS safety and that has not resulted in an accident or serious incident. This category includes for example minor incidents, safety degradation not preventing normal operation, emergency/complex manoeuvres to prevent a collision, and more generally all occurrences relevant to the safety performance of the ADS on-road (like interaction with remote operator, etc.).

‘Critical Occurrence’ means each occurrence in which the ADS is engaged at the time of a collision event and because of which:

(a) at least one person suffers an injury that requires medical assistance as a result of being in the vehicle or being involved in the event;

(b) the fully automated vehicle, other vehicles or stationary objects sustain a physical damage that exceeds a certain threshold or any vehicle involved in the event experiences an airbag deployment.

Notifications and reporting by the manufacturer

The manufacturer immediately notify critical occurrences to the type-approval authorities, market surveillance authorities and the Commission. 

The manufacturer shall report within one month any short-term occurrences, as described in Appendix 1, which needs to be remedied by the manufacturer to the type-approval authorities, market surveillance authorities and the Commission.

The manufacturer shall report every year to the type-approval authority that granted the approval on the occurrences listed in Appendix I. The report shall provide evidence of the ADS performance on safety relevant occurrences in the field. In particular, it shall demonstrate that:

(a)no inconsistencies are detected compared to the ADS safety performance assessed prior to market introduction;

(b)the ADS respects the performance requirements set by this Regulation;

(c) any newly discovered significant ADS safety performance issues have been adequately addressed and how.

The granting type-approval authority shall share this information with type-approval authorities, market surveillance authorities and the Commission.

Type-approval authorities, market surveillance authorities and the Commission may request the manufacturer supporting data used to elaborate the information provided into the in-service reporting and notifications. These data shall be exchanged by means of an agreed data exchange file.

Any pre-processing of data should be notified to the granting type-approval authority in the in-service Data Report.

OCCURRENCE

SHORT-TERM

REPORTING

(1 month

PERIODIC REPORTING

(1 year)

1. Occurrences related to the ADS performance of the DDT, such as

1.a. Safety critical occurrences known to the ADS manufacturer or OEM

X

X

1.b. Occurrences related to ADS operation outside its ODD

X

X

1.c. Occurrences related to ADS failure to achieve a minimal risk condition when necessary

X

X

1.d. Communication-related occurrences (where connectivity is relevant to the ADS safety concept)

X

1.e. Cybersecurity-related occurrences

X

1.f. Interaction with remote operator (if applicable) related to major ADS or vehicle failures

X

2. Occurrences related to ADS interaction with fully automated vehicle users, such as:

2.a. User-related occurrences (e.g. user errors, misuse, misuse prevention)

X

3. Occurrences related to ADS technical conditions, including maintenance and repair:

3.a. Occurrences related ADS failure resulting in a request to intervene to the operator or the remote intervention operator

X

3.b. Maintenance and repair problems

X

3.c. Occurrences related to unauthorised modifications (i.e. tampering)

X

4. Occurrences related to the identification of new safety-relevant scenarios

X

(if modifications made by manufacturer to address a newly identified and significant ADS safety issue involving an unreasonable risk, including description of any previously unanticipated scenarios.)

X

0.1.

Make (trade name of manufacturer):

0.2.

Type:

0.2.1.

Commercial name(s) (if available):

0.3.

Means of identification of type, if marked on the vehicle:

0.3.1.

Location of that marking:

0.4.

Category of vehicle:

0.5.

Name and address of manufacturer:

0.8.

Name(s) and address(es) of assembly plant(s):

0.9.

Name and address of the manufacturer's representative (if any):

1.

Additional information (where applicable): see Addendum.

2.

Technical service responsible for carrying out the tests:

3.

Date of test report:

4.

Number of test report:

5.

Remarks (if any): see Addendum.

6.

Place:

7.

Date:

8.

Signature:

6.

Description and/or drawing of the ADS including:

6.1.

ODD, system boundaries and specified maximum speed of the ADS declared by the manufacturer:

6.2.

Description of the main functions of the ADS

6.2.1.

Vehicle-internal functions

6.2.2

Vehicle-external functions (e.g. backend, off-board infrastructure needed, operational measures needed)

6.3

Sensing system (incl. components):

6.4.

Installation of the ADS sensing system:

6.5.

Software identification of the ADS:

7.

Written description and/or drawing of the ADS human supervision

7.1.

Remote operator and remote intervention on the ADS

7.2.

Means to activate, deactivate of the ADS

7.3.

Monitoring in the inside of the vehicle

7.4.

Any system limitations due to environmental or road conditions

8. 

Written description and/or drawing of the information given to vehicle occupants and other road users

8.1.

System status:

8.2.

Request to the on-board operator/remote intervention operator:

8.3.

Minimum risk manoeuvre:

8.4.

Emergency manoeuvre:

9.

ADS data elements

9.1.

ADS data elements verified after the tests performed in accordance with Annex 3 part 3:

9.2.

Documentation concerning data retrievability, data integrity self-check and protection against manipulation of stored data verified: yes/no

10.

Cyber security and software updates

10.1.

Cyber security type-approval number:

10.2.

Software update type-approval number:

11.

Assessment on functional and operational safety aspects of the automated driving system

11.1.

Manufacturers document reference for the assessment (including version number):

11.2.

Information document

12.

Technical service responsible for conducting approval tests

12.2.

Date of the test report issued by that service

12.3

(Reference) Number of the report issued by that service

12.

Annexes

Addendum 1: Information document for automated driving systems (refer to Annex I- to [PO: This Regulation ]).

Addendum 2:    Member States and specific areas where the vehicle manufacturer has declared that the ADS had been assessed to comply with local traffic rules.

List of documents in the approval file deposited at the administration services having delivered the approval and which can be obtained upon request.

Addendum 3: ADS Assessment report/test results by the granting type-approval authority.

Addendum 4 : Certificate of Compliance for SMS