Within the German Federal Government, energy policy is the responsibility of the Federal Ministry for Economic Affairs and Climate Action (Bundesministerium fur Wirtschaft und Klimaschutz (BMWK)). The Federal Ministry for the Environment, Nature Conservation, Nuclear Safety and Consumer Protection (Bundesministerium fur Umwelt, Naturschutz, nukleare Sicherheit und Verbraucherschutz (BMUV)) is responsible for nuclear energy, including nuclear safety, radiation protection and research in nuclear safety and waste management.

The major aim of the German energy policy is an affordable, secure and environmentally friendly energy supply. This aim shall be reached through the ongoing energy transition, which plans to produce energy on a sustainable basis and to maintain one of the most energy efficient, safe and environmentally compatible economies in the world. The energy transition includes the following steps:

• The last three German nuclear power plants (NPPs) will be taken off-grid by the end of 2022.

• A greater share of renewable energy shall be used according to the Renewable Energy Sources Act adopted by the Federal Government in January 2021 65 % of the energy supply should be generated by renewables by 2030.

• Germany shall become less dependent on oil and gas imports.

• The emissions of greenhouse gases shall be reduced by at least 65% by 2030 relative to 1990 levels, and greenhouse gas neutrality shall be reached by 2045.

• Power grids are undergoing major expansion and modernization in order to meet demand.

• Energy needs shall be reduced by more economical and efficient use.

• The restructuring of the energy supply shall be a driver of innovation for Germany as an industrial base in order to generate growth and create sustainable and secure jobs [1].

The Energy Concept document, adopted by the Federal Government in 2010, remains Germany's guiding document for realizing the energy transition to date. In 2011, it was supplemented by the decision to completely phase out nuclear from electricity generation by the end of 2022. Several further acts were adopted to support the energy transition; for example, the Coal Phase-out Act provides for shutting down coal-fired electricity generation by 2038.

More information and the structured plan of the energy transition can be found on the BMWK website.

Germany is one of the largest energy consumers in the world and is currently expanding generation capacities for primary energy from renewable sources as part of the implementation of its energy transition, and to comply with the obligations inherent in the Paris Climate Agreement signed in 2015 and from EU regulations. However, around 80% of its primary energy consumption [2] still has to be provided by fossil fuels. Germany must import the majority of the energy resources it requires. The most significant source countries for fossil fuel imports to Germany are the Russian Federation, Norway and the Netherlands.

Around 2% of crude oil production and 5% of natural gas production were derived from domestic production in 2020 [2]. Mining of hard coal was phased out in 2018. Of all the energy resources in Germany, lignite is the only non-renewable energy resource, which is available in large, economically extractable amounts.

The demand for natural uranium is covered entirely by imports. Since the closure of the Wismut facility in East Germany in 1990, there has been no mined production of natural uranium in Germany.

An overview of the estimated available energy sources in Germany is given in Table 1. The remaining potential of fossil and nuclear fuels is given as reserves, being defined as proven volumes of energy resources economically exploitable at today's prices and using today's technology. Renewables are given as installed capacity in 2020, hydropower as run-of-river, storage, and pumped storage power station with natural inflow. Exajoule (EJ) equivalent is calculated for a period of 10 years.

TABLE 1. ESTIMATED AVAILABLE ENERGY SOURCES 2020

¹ Solid includes lignite. Reserves of hard coal are 0.

Source: [2, 3].

Table 2 gives an overview of the energy consumption in Germany.

TABLE 2. ENERGY CONSUMPTION

*Latest available data, please note that compound annual growth rate may not be representative of actual average growth.

**Total energy derived from primary and secondary generation sources. Figures do not reflect potential heat output that may result from electricity co-generation.

Source(s): United Nations Statistical Division, OECD/IEA and IAEA RDS-1

German electricity policy is based on three fundamental objectives: sustainability, security of supply and economic efficiency. The main challenge will be integrating an increasing number of plants generating electricity from renewable sources, including a large number of offshore wind farms in the Baltic and North Seas. Consequently, the German electricity grid and the electricity market are facing new challenges.

The Energy Industry Act (Energiewirtschaftsgesetz), together with secondary legislation enacted under it, specifies the regulatory framework governing grid access and transmission fees for electricity and gas. The objective is to provide the public with a secure, affordable, consumer-friendly, efficient and environmentally sound supply of grid electricity and gas. Enforcement lies with the Federal Network Agency (Bundesnetzagentur), which regulates electricity, gas, telecommunications, postal and railway networks spanning two or more federal states and network operators with more than 100 000 customers. Network operators with fewer than 100 000 customers are regulated by regulatory agencies in the individual German federal states.

Generation

Germany's electricity supply is undergoing radical change. At present, conventional energy sources (including nuclear) generate approximately 60% of Germany's electricity. However, the ongoing expansion of renewable energy and the phase out of nuclear energy for power generation will change the composition of the electricity mix. For the first time in 2020, the renewable energy sources, specifically biomass, wind and photovoltaic (44%) produced more electricity than the fossil sources combined (oil, natural gas and coal 40%) [2].

There are more than 1000 commercial electricity producers in Germany. The five largest electricity producers are RWE AG, LEAG, EnBW AG, E.ON SE, and Vattenfall GmbH, which contribute about 65% to the German electricity market.

Though demand for electricity is forecast to remain relatively flat, construction projects for power plants using conventional fuels and for those using renewables are currently in the planning, preparation or building phase in order to replace existing plants, particularly nuclear power plants and coal fired power plants, slated for closure.

Transmission

Germany has a comparatively well developed and intricately meshed electricity grid. The four transmission system operators are TenneT TSO GmbH, Amprion GmbH, TransnetBW GmbH and 50 Hertz Transmission GmbH. They provide non-discriminatory third party access to their networks for all generators. All decisions on grid access and access fees can be appealed to the Federal Network Agency or to the respective regional regulator (Landerregulierungsbeh rde). Grid fees, which cover transmission operations and investments, are charged to distributor companies, which are then passed on to the end users through retail rates. Transmission system operators charge distribution companies via a postage stamp rate, at a single flat rate per kW of maximum demand.

Under the Renewable Energy Sources Act (Erneuerbare-Energien-Gesetz (EEG)), grid system operators are required to connect plants generating electricity from renewable sources to their system at standard rates and to guarantee priority feed-in and transmission of electricity to such plants. The extension of renewables, intensified transboundary power trading and new conventional power plants are the main reasons for the plans to modernize the existing transmission grid and to build new extra high voltage transmission lines.

The network development plan 2019 2030 (Netzentwicklungsplan (NEP)) was confirmed by the Federal Network Agency in December 2019. Information about the development of grid expansion can be found on the dedicated information platform of the Federal Network Agency.

The total length of the German ultra-high voltage transmission grids is about 37 000 km. For the most part, electricity is transmitted as alternating current (AC) with a maximum voltage of 220 kV or 380 kV. Yet under the Federal Requirements Planning Act, new high voltage direct current (DC) transmission lines are planned, with a voltage of up to 525 kV. Around 46% of the projects under the Power Grid Expansion Act (EnLAG) had already been completed and had gone into operation by the end of the fourth quarter of 2019. This corresponds to around 800 km of transmission lines. At the end of the fourth quarter of 2019, a total of 183 km had gone into operation on the basis of projects approved under the Federal Requirement Plan Act (BBPlG) [1].

Germany's grid is linked to its neighbours power grids via cross-border connections. The interconnection capacity is equivalent to about 15% of total capacity. Electricity is physically exchanged with eleven neighbouring countries: Austria, the Czech Republic, Denmark, France, Luxembourg, the Netherlands, Poland, Sweden, Switzerland, Belgium and Norway. Some of these transmission lines are already high voltage direct current systems (i.e. with Denmark) or are under construction (i.e. with Belgium and Norway). In 2019, Germany exported 72.8 TWh and imported 40.1 TWh (physical electricity exchange), amounting to net exports of 32.7TWh. The largest share of Germany's net exports went to the Netherlands, Austria and Poland, while imports came mostly from France [1].

Distribution

At the level of the distribution grids, electricity is transmitted at high, medium and low voltages. The high voltage grid is linked to the ultra-high voltage grid and distributes electricity to urban areas (60 kV to 220 kV; grid length approximately 94 000 km). The medium voltage grid distributes the electricity to regional transformer substations or directly to large facilities (6 kV to 60 kV; grid length approximately 520 000 km). The low voltage grid distributes the power to end users (230 V or 400 V; grid length approximately 1 190 000 km). There are about 900 distribution system operators in Germany, mainly regional and municipal grid operators.

Baseload power is provided mainly by hard coal, lignite and nuclear power plants, which typically run with a maximum number of operating hours but can also be operated in load following operation. These plants provide about 40% of the total electricity production. At present, the proportion of renewable energy in the electricity supply is about 40%. This value is an average for the whole year, though it can reach higher levels at peak times. Phases with low renewable availability therefore need to be covered by facilities such as flexible conventional power stations to ensure the reliability of the power supply.

Table 3 gives an overview of the electricity production in in Germany.

TABLE 3. ELECTRICITY PRODUCTION

*Latest available data, please note that compound annual growth rate may not be representative of actual average growth.

**Electricity transmission losses are not deducted.

: data not available.

Source: United Nations Statistical Division, OECD/IEA and IAEA RDS-1

Table 4 displays several energy related ratios.

TABLE 4. ENERGY RELATED RATIOS

*Latest available data.

Source: RDS-1 and RDS-2

: data not available.

The current organizational structure in licensing and supervision of nuclear power plants in Germany is shown in Fig. 1.

FIG. 1. Current organizational structure licensing and supervision of nuclear power plants

Updated from Source: [4].

The regulatory body is composed of the nuclear licensing and supervisory authorities of the Federal Government and federal state governments. The federal ministry that oversees nuclear safety and radiation protection is the BMUV. The licensing procedure for operation and decommissioning and the continuous regulatory supervision of the facilities lie within the responsibility of the individual federal states (Land Ministry). Subordinate authorities to the BMUV are the Federal Office for the Safety of Nuclear Waste Management (Bundesamt fur die Sicherheit der nuklearen Entsorgung (BASE)) which is the central licensing and supervisory authority in the field of nuclear waste management, in addition to duties under the Federal Office for Radiation Protection (Bundesamt fur Strahlenschutz (BfS)). A detailed description of the regulatory authorities is given in Section 3.1.1.

The BMUV is regularly advised by the Reactor Safety Commission (RSK), the Commission on Radiological Protection (SSK) and the Nuclear Waste Management Commission (ESK). The RSK provides advice in matters of nuclear safety including matters with respect to the physical protection of nuclear installations. The SSK provides advice in matters of protection against ionizing and non-ionizing radiation, and the ESK in matters of nuclear waste management. The members of the commissions are obliged by statutes to express their opinion in a neutral and scientifically sound manner and are appointed by the BMUV. The results of the commission's consultations are formulated in the form of general recommendations and statements on individual cases and published.

The authorities in charge may consult authorized experts in the licensing and supervisory procedures. These can include both independent experts and independent technical expert organizations. The BMUV draws on the external expertise of several such organizations. In particular, these are the Gesellschaft fur Anlagen- und Reaktorsicherheit (GRS) gGmbH, Brenk Systemplanung GmbH, and ko-Institut e.V. The nuclear supervisory authorities of the Lander usually seek advice from the major technical expert organizations of the T Vs (T V represents Technical Inspection Association, i.e. T V Nord, T V S d and T V Rheinland). As a rule, framework agreements exist between the nuclear licensing and supervisory authorities of the Lander and the T Vs, which oblige T Vs to perform certain tasks in the long term and to provide the necessary know-how including appropriately qualified personnel.

Some of the responsibilities of the authorities involved in supervision of disposal, storage and transportation of nuclear waste are different from the ones involved in the supervision of nuclear power plants. Figure 2 bellow gives an overview of the responsibilities of the Federation and the federal states in regard to the supervision of a disposal facility.

Updated from Source: [5].

In Germany, three nuclear power plants (PWRs), with a total gross capacity of 4.291 GWe, are in operation. Table 5 shows the status of German NNPs. Figure 3 shows their geographical location and their status as of the end of March 2022.

According to the current legal situation, the licence for power operation will expire at fixed shutdown dates or even before if the electricity volume for that installation, as listed in the Atomic Energy Act or as derived from an allowable transfer of electricity volume, has been produced. Table 5B shows the residual electricity volumes of the German NPPs as of 31 December 2021.

In 2021, NPPs contributed approximately 11.8% to the gross electricity production, which corresponds to 69.0 TWh. The average availability of German NPPs is shown in Table 5C.

In total, 46 research and training reactors were built and operated in Germany. At present, most research reactors are shut down or being decommissioned, though six research facilities two with a thermal power of more than 50 kW(th) and four small training reactors are still in operation.

TABLE 5. STATUS AND PERFORMANCE OF NUCLEAR POWER PLANTS

FIG. 3. Nuclear power plants in Germany (including prototype and experimental reactors) as of 31 M rz 2022.

Updated from Source: [4].

TABLE 5B. RESIDUAL ELECTRICITY VOLUMES OF GERMAN NPPs AS OF 31 DECEMBER 2021 [TWh]

¹ The NPP M lheim-K rlich was shut down in September 1988. According to the Atomic Energy Act the electricity volume of the NPP M lheim-K rlich can be transferred to KKE, GKN 2, KKI 2, KBR, KRB B, KRB C.

Source: [6].

TABLE 5C. AVERAGE AVAILABILITY OF GERMAN NUCLEAR POWER PLANTS

Note: Preliminary data.

Time availability: available operating time/calendar time.Energy availability: available energy/nominal energy.Capacity availability: energy generated/nominal energy.

Updated from Source: [7].

According to the Atomic Energy Act, a licence for operating an NPP is only granted if the applicant proves that the necessary technical and organizational precautions for safe operation have been taken. During operation, the plant operator must fulfil his or her responsibilities continuously, which the licensing and supervisory authority verifies and ensures.

Planned modifications of an NPP are to be assessed systematically with regard to the impacts on the safety level of the NPP. Modifications with insignificant impacts on safety levels do not require a licensing procedure, but they do require accompanying inspections by the safety authorities within the framework of the supervisory procedure. Significant modifications of an NPP or its operations require a licence from the appropriate authority (see Section 3.1.2).

According to the Atomic Energy Act, safety reviews must be carried out every ten years and follow the standardized national criteria. Safety reviews consist of a deterministic safety status analysis, a probabilistic safety analysis and a deterministic analysis on physical protection of the plant. The results are then submitted to the supervisory authority and usually assessed by independent experts who act by order of the supervisory authority.

Germany participated actively in the European Union's Stress Test and in its follow-up process, initiated by the European Nuclear Safety Regulators Group (ENSREG). During ENSREG workshops in April 2013 and 2015, it was stated that Germany's NPPs had already completed significant enhancements to robustness ahead of the events at the Fukushima Daiichi NPP (e.g. filtered containment venting and mobile pumps). The German National Action Plan identified that further work was required in some technical areas which are relevant to the stress test. The National Report and the National Action Plan of Germany can be found at www.ensreg.eu/EU-Stress-Tests/Country-Specific-Reports/EU-Member-States/Germany.

Furthermore, Germany participated in the first European Union Topical Peer Review with the topic Ageing Management of Nuclear Power Plants, coordinated by ENSREG. The BMUV submitted a National Assessment Report in 2018 and in September 2019 the National Action Plan for the implementation of the results in due time. For further information, see www.ensreg.eu/eu-topical-peer-review.

As of 16 March 2022, 33 nuclear power plants, including prototype and experimental reactors, had been permanently shut down. Of these, 26 facilities are being dismantled and are intended to be released from nuclear regulatory control, including one facility in safe enclosure. Three facilities have already been completely dismantled and the sites released from nuclear regulatory control. Four facilities are in post-operation. Further information (shutdown date and reason, etc.) is shown in Table 6. For decommissioning, a licence is required from the relevant licensing authority of the federal state in which the nuclear installation is sited. The licensing and supervisory process is described in Sections 3.1.1 and 3.1.2.

Each decommissioning project runs individually. The course of the project, the financing, the choice of the decommissioning strategy and many other boundary conditions depend strongly on the type of plant and the owner of the plant. NPPs and uranium enrichment and fuel assembly plants are usually operated by energy supply companies. The licensees are obliged to continuously build up financial reserves for the decommissioning of their installations. On the other hand, research reactors, prototype and experimental reactors as well as prototype plants for fuel supply are mostly located in research centres or at universities. Their decommissioning is financed predominantly by the federal budget.

The decommissioning of the NPPs Greifswald and Rheinsberg of the former East Germany is financed from the federal budget as well, as is the decommissioning of facilities for uranium ore mining and processing in the former East Germany.

Under the Atomic Energy Act and in line with the polluter-pays principle, the operators must pay for the shut-down and dismantling of the NPPs, as well as for the management of the nuclear waste produced by them, including the cost of waste disposal. On 16 June 2017, the Act on the Reorganization of Responsibility in Nuclear Waste Management entered into force. Under the Act, the NPP operators remain responsible for the financing and management of decommissioning of the nuclear power plants and the proper packaging of the nuclear waste, while the Federal Government is responsible for the implementation and financing of interim storage and disposal of radioactive waste. On 3 July 2017, the operators transferred financial means amounting to approximately 24 billion into a public law fund to cover the costs of interim storage and disposal of radioactive waste. The spent fuel and radioactive waste, as well as the interim storage facilities defined in the Act, are in the process of being transferred to the state owned BGZ Company for Interim Storage (BGZ Gesellschaft fur Zwischenlagerung mbH (BGZ)).

Since 2017, safe enclosure is not allowed for NPPs by an amendment to the Atomic Energy Act. In individual cases the competent authority can approve temporary exceptions for facility parts only when required for reasons of radiation protection.

TABLE 6. STATUS OF DECOMMISSIONING PROCESS OF NUCLEAR POWER PLANTS

Source : IAEA - PRIS

Safe operation of nuclear power plants is a top priority for the Federal Government, and consequently research in this field is continued and extended. For a more complete list of organizations, please refer to Appendix 2.

The Federal Government published its 7th Energy Research Programme in 2018. Research on nuclear safety and waste management is an integral part of the Energy Research Programme.

The Federal Ministry of Education and Research (Bundesministerium fur Bildung und Forschung (BMBF)) promotes projects and institutions with funds of around 44 million, focusing on basic science issues regarding waste disposal, reactor safety research and radiation research. The BMBF currently supports the development of fusion reactors through institutional funding (around 126 million in total). For further details and information see Reference [9].

The BMUV promotes research projects and institutions, focusing on the security of nuclear facilities and nuclear safety, radiation protection, as well as the security of nuclear supply and disposal. BASE is promoting research in particular on the topics of site selection procedure, public participation, interim storage and transport of radioactive waste, as well as nuclear safety and security. BfS is performing and promoting research in the fields of biological radiation effects, medical radiation protection, epidemiology, dosimetry, emergency preparedness, and environmental radioactivity, thereby supporting the BMUV in technical and scientific questions of fundamental importance for the protection of people from ionizing and non-ionizing radiation. Furthermore, with the formation of the new German government in December 2021, the responsibility and budget of the project funding programme for nuclear safety and waste disposal research was transferred from the BMWK to the BMUV. Within this programme, BMUV currently provides approximately 22 million annually for reactor safety research, funding experimental or analytical studies of the performance of nuclear reactors under accident conditions, studies concerning the safety of pressure retaining components and the development of probabilistic safety analysis. Another approximately 16 million is provided annually for waste management research including extended interim storage, waste treatment and geological disposal research.

Two prototypes of advanced reactor design were developed in Germany: the pebble bed high temperature reactor (Thorium-Hochtemperaturreaktor, THTR 300) at HRB/BBC and a fast breeder reactor (Schneller Natriumgek hlter Reaktor, SNR 300) at Interatom/Siemens. The THTR 300 was connected to the grid in 1985 and shut down in 1989. The SNR 300 was completed in 1985 but never commissioned.

As a Member State of the European Union, the Nuclear Energy Agency of the Organisation for Economic Co-operation and Development (OECD/NEA), and the IAEA, Germany supports various international programmes in nuclear safety and nuclear waste management. In direct international cooperation, Germany also supports projects and organizations such as licensing and supervisory authorities, technical support organizations and research institutes.

Appendix 1 enumerates the international, multinational and bilateral agreements in which Germany is involved, including Germany is Contracting Party to the Convention on Nuclear Safety (CNS) and to the Joint Convention on the Safety of Spent Fuel Management (JC). The CNS Eighth Review Meeting National Report and the JC Sixth Review Meeting National Report are available on the BMUV website.

According to EU regulations, Germany is obliged to participate in IAEA safety reviews at least every ten years. An Integrated Regulatory Review Service (IRRS) took place from 31 March to 12 April 2019. The publicly available report confirms that Germany's regulatory framework for nuclear safety meets the internationally applicable standards. The review team found that Germany's nuclear licensing and supervisory authorities are mature and competent and highlighted the effective cooperation with other organizations and interested parties. The Review Team identified a good practice for its Integrated Measuring and Information System for the Monitoring of Environmental Radioactivity. An ARTEMIS mission (Integrated Review Service for Radioactive Waste and Spent Fuel Management, Decommissioning and Remediation) took place from 23 September to 04 October 2019. The ARTEMIS report concluded that Germany has a mature legal and regulatory framework for the safety of radioactive waste and spent fuel management. The recently restructured organizational framework for the National Programme contains the necessary elements for safety and programme implementation. An IRRS Follow-up Mission is planned for 2023, while the ARTEMIS Follow-up Mission will take place in November 2022. The progress made on implementing reviewers suggestions in the 2019 Missions will be evaluated. The BMUV is currently preparing self-assessment reports for Germany.

Germany currently participates in most of the OECD/NEA joint projects and hosts the OECD/NEA Primary Coolant Loop Test Facility and the Thermal-hydraulics, Hydrogen, Aerosols and Iodine projects. For project descriptions see www.oecd-nea.org/jointproj/.

Germany is a republic with a federal structure and is composed of 16 federal states, in German called Lander. The Federal Chancellor determines the competence of the supreme federal authorities by organizational decree. Accordingly, the responsibility for the nuclear safety of nuclear installations and radiation protection was transferred to the BMUV. As part of the Federal Government, the BMUV is involved in legislation (legislative power), while the Lander implement the Atomic Energy Act on behalf of the Federation (federal executive administration).

The licensing procedure for operation and decommissioning and the continuous regulatory supervision of the facilities lie within the responsibility of the individual federal states (see Table 7).

TABLE 7. THE LAND'S LICENSING AND SUPERVISORY AUTHORITIES FOR NPPs (INCLUDING PROTOTYPE AND EXPERIMENTAL REACTORS)

In the case of facilities overseeing safekeeping and disposal of radioactive waste, state supervision is regulated differently from the division of tasks between the Federation and the Lander. The nuclear waste management sector was reorganised in order to efficiently select a site for a disposal facility for high level radioactive waste. Accordingly, the BASE was established in 2014 as the central licensing and supervisory authority in the field of waste management.

Belonging to the BMUV's portfolio are two subordinate authorities responsible for nuclear safety, radiation protection and nuclear waste management issues.

The subordinate authority of the BMUV in the area of radiation protection is the BfS. The BfS pools competencies in the area of effects and risks of ionising and non-ionising radiation (e.g. in mobile communication and UV protection), radiological emergency preparedness, monitoring of environmental radioactivity as well as medical and occupational radiation protection. With four technical divisions it provides support for the supervisory body, the BMUV, with regard to statutory tasks in the areas of radiation protection and performs research in its areas of responsibility.

As a subordinate authority of the BMUV, the BASE performs regulatory, licensing and supervisory tasks regarding disposal, storing, handling and transporting of high-level radioactive waste; regulates the search for and selection of a final disposal facility for high level radioactive waste (site selection procedure), organizing participation of the public in the process; provides support for the supervisory body, the BMUV, with regard to nuclear safety and performs research in its areas of responsibility.

According to the Atomic Energy Act, the erection, operation or holding of a stationary installation for the production, treatment, processing or fission of nuclear fuel, an essential modification of the installation or its operation and also the decommissioning and dismantling require a licence. A licence may only be granted if the licensing requirements specified in the Atomic Energy Act are met. New licences for the erection and the operation of installations for the fission of nuclear fuel for the commercial generation of electricity and for facilities for the reprocessing of irradiated nuclear fuel will not be issued anymore in Germany. The licensing authorities are the supreme Lander authorities stipulated by the Lander governments.

The licensing procedures have to be carried out in accordance with the Nuclear Licensing Procedure Ordinance. This stipulates the application process with the submission of documents, public participation, the opportunity to divide the process into several licensing steps (partial licences) and the observation of other licensing requirements (e.g. for non-radioactive emissions and for discharges into water bodies). Furthermore, in certain cases an environmental impact assessment has to be carried out. In some cases, the Nuclear Licensing Procedure Ordinance is not applicable. Then, a nonformal licensing procedure is carried out.

For further information see Reference [4].