Contents ii iii iv v vi
Editorial Board On the Cover Publishing Schedule and Advertiser’s Index Editor’s Note Soundings ... Kjetil Nordby, Etienne Gernez, Steven Mallam
Essays 1
Not Just an Inconvenience: A Defence of the Regulatory Value of Liability Rules for Remotecontrolled and Autonomous Ships Hannah Stones, University of Southampton 9
Artificial Intelligence and Machine Learning in the Marine Industry: Application in Equipment Health Monitoring Siavash Nejadi, Jennifer Busler, Seaspan 14
Development of a Marine Cybersecurity Demonstration Platform Grace Pearcey, Memorial University of Newfoundland
26
Achieving Seafarer Success in Maritime Digitalization Anders Bergh, Yara Marine Technologies 32
Enhancing Data Management Through SmartVessel.io Peter Walsh, Seashore Maritime Services
44 Autonomous Ship Activities in Norway Ørnulf Jan Rødseth
Norwegian Forum for Autonomous Ships
53
Towards the Seafarer of Tomorrow: Maritime Worker Competencies in the Autonomous Age Steven Mallam, John Cross
Fisheries and Marine Institute
1
32
132
Peer-Reviewed Papers 60
An Autonomous Obstacle Avoidance Methodology for Uncrewed Surface Vehicles Facilitated by Deep-learning Based Object Detection Tianye Wang, Fangda Cui, Qi Li, Youssof Abaza,
Kevin Wang, Shiwei Liu, Wenwen Pei Marine Thinking Inc.
80
Lodestar … Sadie Lange and Hannah Button
82
To What Extent is a Collision with an Autonomous Vessel Considered a Marine Collision in Light of UAE Law? Ramzi Madi, Al Ain University 98
River Ice Monitoring Using Unsupervised ISODATA Algorithm and Different Optical and SAR Satellite Datasets: A Case Study from the Churchill River in Labrador, Canada Meisam Amani, Sahel Mahdavi
WSP Environment and Infrastructure Canada Shuanggen Jin, Henan Polytechnic University
Spindrift 108 Q&A with Tor Erik Jensen 110 Trade Winds 120 Inside Out … Remotely Controlling USVs Towards a Low-carbon Future Henry Robinson, Dynautics 123 Perspective 124 Turnings 126 Reverberations … DVL-INS Systems for Dynamic Positioning Derek Lynch, Sonardyne International Ltd 130 Homeward Bound … Smart Ships Unleashed: Mixed Reality as the Transformative Core Iain Whyte, Kognitiv Spark 132 Parting Notes … Fast Friends Grayson Shallow The Journal of Ocean Technology, Vol. 18, No. 4, 2023 i
PUBLISHER (ACTING) Kelley Santos info@thejot.net
MANAGING EDITOR Dawn Roche Tel. +001 (709) 778-0763 info@thejot.net
ASSISTANT EDITOR Bethany Randell Tel. +001 (709) 778-0769 bethany.randell@mi.mun.ca
GRAPHIC DESIGN/SOCIAL MEDIA Danielle Percy Tel. +001 (709) 778-0561 danielle.percy@mi.mun.ca
TECHNICAL CO-EDITORS Dr. David Molyneux Dr. Katleen Robert Director, Ocean Engineering Research Centre Canada Research Chair, Ocean Mapping Faculty of Engineering and Applied Science School of Ocean Technology Memorial University of Newfoundland Fisheries and Marine Institute WEBSITE AND DATABASE Scott Bruce
FINANCIAL ADMINISTRATION Michelle Whelan
EDITORIAL ASSISTANCE Paula Keener, Randy Gillespie
EDITORIAL BOARD Dr. Keith Alverson University of Massachusetts USA
S.M. Asif Hossain National Parliament Secretariat Bangladesh
Kelly Moret Hampidjan Canada Ltd. Canada
Dr. Randy Billard Virtual Marine Canada
Dr. John Jamieson Dept. Earth Sciences Memorial University Canada
Dr. Glenn Nolan Marine Institute Ireland
Dr. Safak Nur Ertürk Bozkurtoglu Ocean Engineering Department Istanbul Technical University Turkey Dr. Daniel F. Carlson Institute of Coastal Research Helmholtz-Zentrum Geesthacht Germany Dr. Dimitrios Dalaklis World Maritime University Sweden Randy Gillespie Windover Group Canada Dr. Sebnem Helvacioglu Dept. Naval Architecture and Marine Engineering Istanbul Technical University Turkey
Paula Keener Global Ocean Visions USA Richard Kelly Centre for Applied Ocean Technology Marine Institute Canada Peter King University of Tasmania Australia Dr. Sue Molloy Glas Ocean Engineering Canada Dr. Kate Moran Ocean Networks Canada Canada
A publication of
ii The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Dr. Emilio Notti Institute of Marine Sciences Italian National Research Council Italy Nicolai von OppelnBronikowski Memorial University Canada Dr. Malte Pedersen Aalborg University Denmark Bethany Randell Centre for Applied Ocean Technology Marine Institute Canada Prof. Fiona Regan School of Chemical Sciences Dublin City University Ireland
Dr. Mike Smit School of Information Management Dalhousie University Canada Dr. Timothy Sullivan School of Biological, Earth, and Environmental Studies University College Cork Ireland Dr. Jim Wyse Maridia Research Associates Canada Jill Zande MATE, Marine Technology Society USA
Academic and Scientific Credentials The Journal of Ocean Technology is a scholarly periodical with an extensive international editorial board comprising experts representing a broad range of scientific and technical disciplines. Editorial decisions for all reviews and papers are managed by Dr. David Molyneux, Memorial University of Newfoundland, and Dr. Katleen Robert, Fisheries and Marine Institute.
On the
Cover
The Journal of Ocean Technology is indexed with Scopus, EBSCO, Elsevier, and Google Scholar. Such indexing allows us to further disseminate scholarly content to a larger market; helps authenticate the myriad of research activities taking place around the globe; and provides increased exposure to our authors and guest editors. All content in the JOT is available online in open access format. www.thejot.net
A Note on Copyright The Journal of Ocean Technology, ISSN 1718-3200, is protected under Canadian Copyright Laws. Reproduction of any essay, article, paper or part thereof by any mechanical or electronic means without the express written permission of the JOT is strictly prohibited. Expressions of interest to reproduce any part of the JOT should be addressed in writing. Peer-reviewed papers appearing in the JOT and being referenced in another periodical or conference proceedings must be properly cited, including JOT volume, number and page(s). info@thejot.net
SAM HANKINSON
Mythos AI is a marine autonomy company founded by a team that features experience from self-driving teams from Silicon Valley. The US based company is developing autonomous vessel technologies to address severe vulnerabilities in the supply chain and marine transportation. The company’s entry to market product is state-ofthe-art self-driving vessel technology that delivers real-time navigation data. This can eliminate delays and inefficiencies in port and shipping operations, while training self-driving algorithms to address the staggering shortage of skilled mariners that is crippling the industry. Mythos’ cutting-edge technology will automate marine transportation, unlocking new revenue models for ports, cost savings for shipping companies, and unrivalled operational efficiency across the supply chain. https://mythos-ai.com
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 iii
Publishing Schedule at a Glance The JOT production team invites the submission of technical papers, essays, and short articles based on upcoming themes. Technical papers describe cutting edge research and present the results of new research in ocean technology or engineering, and are no more than 7,500 words in length. Student papers are welcome. All papers are subjected to a rigorous peer-review process. Essays present well-informed observations and conclusions, and identify key issues for the ocean community in a concise manner. They are written at a level that would be understandable by a non-specialist. As essays are less formal than a technical paper, they do not include abstracts, listing of references, etc. Typical essay lengths are up to 3,000 words. Short articles are between 400 and 800 words and focus on how a technology works, evolution or advancement of a technology as well as viewpoint/commentary pieces. All content in the JOT is published in open access format, making each issue accessible to anyone, anywhere in the world. Submissions and inquiries should be forwarded to info@thejot.net.
Upcoming Themes All themes are approached from a Blue Economy perspective.
Spring 2024
Remote operation centres: accessing the ocean
Summer 2024
Deep (ocean) learning
Fall 2024
Sensing the ocean: lights, camera, sensors
Winter 2024
Safety first: humans at sea
Advertiser’s Index
Stay informed
CIOOS Educational Passages Marine Institute OceansAdvance SBG Systems
Each issue of the JOT provides a window into important issues and corresponding innovation taking place in a range of ocean sectors – all in an easy-to-read format with full colour, high-resolution graphics and photography.
24-25 31 IFC,13,43,115 IBC 22
CONTACT US The Journal of Ocean Technology c/o Marine Institute P.O. Box 4920 155 Ridge Road St. John's, NL A1C 5R3 Canada +001 (709) 778-0763 info@thejot.net www.thejot.net
iv The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Don't forget to follow us!
Editor's Note When we decided to publish an issue on smart ships, our call for content listed topics such as innovations in digitalization on board ships – autonomous ships; alarm monitoring systems; power management; dynamic positioning; integrated navigation systems; control systems; simulators; human factors/crew safety; electronic chart tables; ship design and operations; artificial intelligence, machine learning, and data management; and other related topics. This jam-packed issue is a combination of essays, technical papers, and short articles from researchers, developers, and technology users from around the globe. Inside you will find a series of essays that focus on user interface standards for safe maritime operations; laws, rules, and regulations; artificial intelligence and machine learning for equipment health monitoring; cybersecurity platforms; data management; and maritime worker competencies. The technical papers look at obstacle avoidance methodologies and laws surrounding marine collisions. RANDY GILLESPIE
In the Spindrift section, there are short articles on propulsion control, shallow water dynamic positioning reference systems, mixed reality, plus an interview with legal professional Mr. Tor Erik Jensen of the University of South-Eastern Norway. We also look at some groups working in this sector, including the SmartShip Project, Maritime and Research Innovation UK, and One Sea Association. Our team believes there is something for everyone in this issue; we hope you agree. Happy reading!
Dawn Roche is Managing Editor of the Journal of Ocean Technology.
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 v
Soundings Developing User Interface Standards for Energy-Conscious and Safe Maritime Operations The ongoing efforts towards decarbonizing the maritime industry are creating unprecedented opportunities for innovation in ship technology and operations. New technologies introduced into already complex and safety-critical maritime workplaces can create both foreseen and unforeseen consequences on work systems and its operators (see Further Reading #1). Any new technology added to an existing operational context impacts safety because mariners must simultaneously manage and operate new green shipping systems and protocols, while continuing to manage and execute traditional navigation and operational ship tasks. Due to the diversity of the systems in use on board ships, mariners need to invest considerable effort into developing operational literacy across differing systems and contexts (see Further Reading #2). The introduction of new technologies on board inevitably requires the need for differing skill sets and competencies, updated education and training programs, differing work task considerations and prioritization, and potentially increased cognitive workloads. One critical area of research in this field is the focus on how new solutions are integrated into current work systems and work practices, both on board individual ships, as well as on a more macro level, such as fleet management and shoreside monitoring of one or multiple ships and their operations. There is currently no guidance or regulation to support the design of user interfaces for energy-conscious decision-making within the larger scope of safe and efficient maritime operations. If new systems are difficult to use, people will not be able to take advantage of their full potential and impact, leading to a gap between theoretical performance of new innovations and the actual performance achieved in the realities of complex sociotechnical systems under real-world operations. Furthermore, there is a threat that maritime decarbonization efforts led by introducing new technologies may contribute to new safety challenges. This double threat is aggravated by a current lack in design precedence, design guidance, and design regulation supporting how to design for safe and energy conscious operations in the maritime domain. To address this challenge, the OpenZero project (full project name: OpenZero – Digital User Interface Design for Energy-Conscious and Safe Maritime Operations) is looking to an alternative approach based on user-centred design and open innovation. OpenZero will focus on the end user’s perspective and experiences of operating green ships, with the goal to enhance mariners’ abilities to learn and safely operate new green technologies to their full theoretical potential. OpenZero will apply state-of-the-art research in user-centred human-computer interaction, user experience behavioural design, and human factors approaches to analyze the end users and context of use requirements to develop an open user interface design system for energy-conscious and safe maritime operations. As ocean industries are global, heavily regulated, and traditionally slow to adopt new initiatives and changes, there is typically a gap between technological development, research production, and results adopted in the real world. An open innovation approach can help reduce and remove barriers that can impede application and adoption. OpenZero currently has 30 project partners consisting of system developers and integrators, regulatory bodies and authorities, academic and research actors, design consultants, and ship owners. Among these partners are industry
vi The Journal of Ocean Technology, Vol. 18, No. 4, 2023
leaders, and together these companies have delivered systems to over 50,000 ships worldwide, accounting for approximately one half of the current global fleet. OpenZero will build upon a comprehensive and streamlined framework for maritime design developed progressively over the past ten years, a state-of-the-art design guideline crossing all maritime workplaces: the OpenBridge Design System (see Further Reading #3). OpenBridge consists of a large library of design components specifically made for maritime workplaces. The entire library is open source and free. Its components can be used to create many current and future maritime digital user interfaces (e.g., ECDIS, conning, radar, alerts and alarms, fire systems, thruster, and button design, etc.) that adhere to current maritime regulations. The OpenBridge system goes beyond screen-based user interface design and includes support for multi-screen systems, physical interaction devices (e.g., thruster and button design and layout), and entirely new interface technologies, such as augmented reality and remote monitoring and control. OpenBridge is currently in use globally with over one thousand maritime companies having registered to access it. There are regularly new products reaching the market developed with the OpenBridge system, both by our direct industry partner collaborators, as well as maritime companies not officially affiliated with the OpenBridge consortium. By leveraging the OpenBridge Design System, OpenZero research and development results will benefit from a large body of design case-based precedence as well as established tools and methods for collaborative user centred design (see Further Reading #4). Most importantly, all outputs from OpenZero will be directly integrated into a design system that promotes design consistency across differing systems and context of use. All current and future OpenBridge developers will get direct access to OpenZero results for free, while our open innovation
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 vii
approach ensures design guidance from the project is available to all interested stakeholders. Furthermore, the relevance of OpenZero outputs may extend beyond the shipping and maritime industries and add value to other transport and safety critical domains or beyond. The OpenZero project runs from 2024-2028 and is funded partially by the Research Council of Norway through its Collaborative and Knowledge-Building Project Program. More information on OpenZero and related research and development maritime projects can be found online at the Ocean Industries Concept Lab.
1. Mallam, S.C.; Nordby, K.; Johnsen, S.O.; and Bjørneseth, F.B. [2020]. The digitalization of navigation: examining the accident and aftermath of US Navy Destroyer John S. McCain. In: Proceedings of The Royal Institution of Naval Architects Damaged Ship V, 55-63. ISBN No: 978-1-911649-05-2. 2. Nordby, K.; Mallam, S.C.; and Lützhöft, M. [2019]. Open user interface architecture for digital multivendor ship bridge systems. WMU Journal of Maritime Affairs, 18(2), 297-318. DOI: 10.1007/s13437-019-00168-w. 3. Nordby, K.; Gernez, E.; and Mallam, S. [2019]. OpenBridge: designing for consistency across user interfaces in multi-vendor ship bridges. In: Proceedings of Ergoship 2019, pp. 60-68. Western Norway University of Applied Sciences, Haugesund, Norway. ISBN: 978-82-93677-04-8. 4. Mallam, S.C.; Nordby, K.; Haavardtun, P.; Nordland, H.; and Westerberg, T.V. [2021]. Shifting participatory design approaches for increased resilience. IISE Transactions on Occupational Ergonomics and Human Factors, 9(2), 78-85. DOI:10.1080/24725838.20 21.1966131.
Dr. Kjetil Nordby and Dr. Etienne Gernez are both with the Insitute of Design, Oslo School of Architecture and Design in Oslo, Norway. Dr. Steven Mallam is cross-appointed with Fisheries and Marine Institute of Memorial University of Newfoundland, St. John’s, Canada, and the Department of Maritime Operations, University of South-Eastern Norway, Borre, Norway.
viii The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Not Just an
Inconvenience A Defence of the Regulatory Value of Liability Rules for Remote-controlled and Autonomous Ships by Hannah Stones
PIXABAY.COM/CONGERDESIGN
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 1
Liability rules are not just an inconvenience to the status quo to be excluded or replaced. Liability rules have a regulatory value in prevention, which cannot be easily replicated through other mechanisms. Introduction As technology advances there is a need to support that innovation, to enable testing and development, to facilitate deployment and the integration of the next era of shipping technology. This contrasts with the burden of law, which can be ominous for innovators. These somewhat opposing positions create a perception of tension and conflict between law and technology, but this essay explores how to address this perception and in doing so ease the sense of uncertainty created by these systems. This perceived tension can create significant challenges for the stability and consistency of law. Uncertainty results in a desire to provide clarity, predictability, reassurance, and reduce costs/losses. Within the context of the risks and opportunities presented by remote-controlled and autonomous systems (e.g., navigation), there can be a suggestion that liability rules (meaning civil liability rules) should be excluded from such incidents and replaced by an alternative compensation system (see Further Reading #2, 3 for an exploration of this issue for cars). The complexity and burden of liability rules make this appear to be an easy and convenient solution, which supports the introduction of remote-controlled and autonomous systems. However, this essay contends that this would result in a regulatory shortfall due to liability rules’ deterrence effect. Using law and economics (a type of legal theory that uses the method of economic theory to analyze law), liability rules will be shown to be more than a secondary mechanism of law, and as comprising a fundamental part of obligations from the outset. In order to achieve the aim of parity of safety for remote-controlled and autonomous ships with existing ships, liability rules cannot be considered a mere
2 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
inconvenience. This essay addresses this misconception by explaining the role of liability rules, and how that role complements that of other laws, as well as addressing the conceptual conflict which has arisen between liability rules and technology. Efficient Liability Rules Assuming a good or effective law, or legal rule, is defined as an efficient law, then the rule must produce an efficient allocation of resources (see Further Reading #4). An efficient allocation of resources is achieved by a reduction in accident costs in liability rules (see Further Reading #1). This can involve a redistribution of costs from their natural distribution. The loss will naturally fall on the injured party, for example, a person who is injured will have the cost of physical injury, loss of work, etc. and society will potentially bear the cost of treating the injury and providing social welfare. This means for a liability rule to be considered efficient, it must reduce the cost of accidents, which is more than just redistribution of the loss. Although redistribution can contribute to a reduction in the cost of accidents, it is limited by the nature of money. Thus, more is needed to reduce the cost of accidents through minimizing the number and severity of accidents. A reduction in accident costs contains subgoals, according to Calabresi: reduction in the number and severity of accident costs, reduction in societal costs of accidents, and a reduction in administrative treatment of accidents. Liability rules achieve this through a combination of prevention and compensation of accidents. This reveals the significance of prevention within liability rules (Figure 1). Liability rules will then need to take the most appropriate form to achieve the necessary compensation and deterrence to reduce the cost of accidents. Liability Rules: Fault-based Liability and Strict Liability The most common liability rules are faultbased liability and strict liability, although
HANNAH STONES HANNAH STONES
Figure 1: Reduction of accident costs.
Figure 2: Reasonable care (prevention of loss) and damages (compensation of loss) to reduce the costs of accidents in fault-based liability.
there are alternatives such as fault-pools and guaranteed compensation schemes. Fault-based liability, which is most commonly associated with negligence liability in tort law, involves a requirement that the responsible party take reasonable care. If they fail to do so, then they will be liable if they are at fault and need to pay compensation or damages. Reasonable care and the imposition of damages together allow for all the sub-goals of the reduction in accident costs to be achieved (through a combination of prevention and compensation in one rule), and thus achieve a reduction in
accident costs while allowing for the beneficial activity to continue within the market (Figure 1). This can be characterized from the injured party’s perspective as the receiving of a compensatory payment in the reduction of risk through reasonable care or damages (Figure 2). Strict liability, which is most commonly associated with product liability and enterprise liability, will impose liability regardless of whether reasonable care has been taken. Strict liability is considered within law and economics to have less of a deterrent effect
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 3
than fault-based liability, as reasonable care is not motivated through the avoidance of compensatory payments. Although, strict liability does create a cost in the strict imposition of liability for compensation, which can be avoided through accident prevention. Fault-based liability can be considered disadvantageous in its limited compensation provision and failure to redistribute as many losses as possible to the party with deeper pockets. Depending on the area and its risks, law will determine which of the two will achieve the greatest reduction in accident costs: the legal solution depends on the nature of the risk of loss. The appropriate liability rule needs to create behavioural and systematic change to address the risk. Why Liability Rules can be Easily Mischaracterized as an Unnecessary Inconvenience for Remote-controlled and Autonomous Ships Liability rules have traditionally assumed the necessity of direct and active human involvement in shipping operations. Humans taking safety precautions is considered the primary mechanism of preventing and reducing losses; that is why it is crucial to minimize human error. The assumption that these systems remove the human to eliminate human error is challenged by human involvement still being present and fundamental in remotecontrolled and autonomous systems. Instead, this change can be characterized in how these systems will change the point of human error; for example, instead of navigational error on the bridge, it may occur in the autonomous route calculation or from shore on a cyberenabled bridge, or in the construction of the ship – its sensors and its systems being an ongoing responsibility to develop and improve. The role of law has been to require higher standards of safety through regulatory and liability rules (as well as criminal liability): standards must be met or there will be legal consequences. Within this, there is an acceptance and acknowledgment that some
4 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
losses are unavoidable, but law provides a strong motivation for a minimum standard. That standard has implicitly assumed a direct role for humans. However, this does not mean that it cannot evolve. Corporate relationships, ship building, and ship management are already complex and involve distant persons (both human and legal) and law has developed to provide strong prevention and compensation through flexible liability rules. Ships may use remote-controlled or autonomous systems, or both, and they may use them in addition to existing operational systems, and the existing duties and benefits of law can still apply. Therefore, law can embrace and apply to new mechanisms of operation. By taking the position that this technology can be fostered while maintaining the legal regime, the challenges of law for remote-controlled and autonomous systems can be reconceptualized. Remote-controlled and autonomous systems can be perceived as equal alternatives to direct human control, meaning that the human assumption of law is not a barrier to innovation. Remote-controlled and autonomous systems can be another mechanism by which the carrier fulfils these obligations. By reconceptualizing traditional crewing models, reduced crewing models, zero crew on board, remote-control, autonomy, and combinations thereof as different mechanisms of reducing the cost of accidents, the relationship between liability rules and technology can also be reconceptualized. Technology can thus be considered by law not with skepticism as to its disruptive power, but as furthering and sharing the aims of law. Technology can approach law not as a barrier but as reflecting the aim of the society and market in which it is situated. Deterrence in Law: Regulations and Liability The argument against the necessity of liability rules for the prevention of losses is that regulations are all that is needed for prevention (Figure 3). Therefore, the erroneous conclusion is that liability rules can be excluded, and a compensation scheme can be implemented to complement the prevention
HANNAH STONES
provided by regulations. However, this assumes that regulations alone are sufficient and ignores the added benefit of prevention within liability rules. The favouring of regulations alone is supported by the greater degree of certainty of required behaviours to be fulfilled. Yet, this creates, arguably, less flexibility for the adoption of remote-controlled and autonomous systems. This misconception, which favours regulations alone, risks the surrender of claimants’ rights and the potential for inequality between injured parties by dismissing the preventative value of liability rules. Additionally, this creates a separation between ships based on the operational system. This is problematic in two regards for the integration of remotecontrolled and autonomous ships: distinction and uncertainty. In the creation of a distinction between ships, the aim of parity in improving safety is undermined. There is uncertainty in the variety of levels of smart technology, which is exacerbated by retrofitting, ships which alternate between systems, and incremental updates. To dismiss liability rules is, therefore, to dismiss their significant role within shipping law.
Figure 3: Preventative and regulatory value of laws. Regulations, at the top, have the highest regulatory value, but liability rules also have substantial regulatory value.
Distinction + Uncertainty = Inequality To further consider the issues of creating inequality, there must be an appreciation of the extent of loss when an incident occurs. Some of the most significant losses are losses to the person (e.g., personal injury and death) and all the costs of those should be appreciated. Consider the example of the passenger who does not know or understand the different systems that could be used on any ship. This passenger happens to be carried on a passenger ship utilizing autonomous navigation and hazard avoidance systems. The navigation system malfunctions, so that the course is altered into more hazardous waters and the sensors do not detect a large rock. This results in an allision, jolting the ship, causing the passenger to fall and damage their spine, resulting in permanent lower-body paralysis (Figure 4). The liability rules in the Athens Convention relating to the Carriage of Passengers and their Luggage by Sea 2002: The Consolidated Text of the Athens Convention 1974 and the 2002 Protocol (Athens 2002) would treat this type of incident as a shipping incident, and the carrier would be liable up to 250,000 SDR
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 5
HANNAH STONES
Figure 4: Illustrating the scenario of a passenger injured on a ship using autonomous systems.
(special drawing rights) under strict liability, and 400,000 SDR under fault-based liability with a presumption of fault in shipping incidents (Articles 3 and 7). However, remotecontrolled and autonomous systems create uncertainty as to whether it is a ship, and who should be liable. Therefore, one solution to address the uncertainty is the simple exclusion and replacement of the liability rules (Figure 5 shows alternative options to Athens 2002). However, it creates an additional administrative cost for the passenger-claimant in pursuing damages: they must determine under which system to make their claim or risk being unsuccessful in their claim. This means the solution only displaces the uncertainty and puts it on the weakest party. However, it is important to recognize that the alternatives are not without merit, especially regarding compensatory benefits and reduced court administrative costs. Yet, they are not more effective at reducing the overall cost of accidents further. The most concerning aspect of making operational distinctions between claimants, beyond administrative burden, is that it affirms
6 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
the perception of increased risk. The problem is that by creating a separate system, it implies that the risk to the passenger is different. This is due to the perception of an unfamiliar risk as being higher. However, this perception should not be the focus of law. Law needs to focus on the shared risks and the similarities. A remote-controlled or autonomous ship is still a ship and exposed to the same maritime perils. If it carries passengers, then it is still a passenger ship exposed to the same perils and types of risk as any passenger ship. Focusing on the operating system to draw a legal distinction is to focus on too specific a detail and ignore the bigger picture. Trading a tort-based right, even if it results in a higher amount of financial compensation, ignores the preventative value of the right to potential injured parties. Therefore, an alternative compensation scheme means the rights of the injured party are lesser because they lack the preventative value that tortbased rights have. To just compensate is not as effective at reducing accident costs, as it primarily re-distributes the loss. Therefore, despite the compensatory value, it would
HANNAH STONES
Figure 5: Options to resolve the uncertainty on the assumption that the law of a State incorporates Athens 2002 for passenger ships but desires to exclude it from remote-controlled and autonomous passenger ships.
Compensation is of great importance when a loss does occur, but there needs to be an acknowledgment of the contribution to prevention in liability. Relying on regulations alone will, of course, still have strong prevention, but it will be lessened and any approach to maritime safety needs to prioritize prevention and respect the value of liability rules in contributing to a reduction of accident costs (Figure 6). Suggestions of exclusion and replacement ignore the necessity of including prevention for any replacement, due to the erroneous assumption that regulations, and in the most extreme incidents, criminal law, provide enough prevention.
uncertainty caused by the perceived dichotomy between law and technology. However, liability rules should not be considered as an impediment. Utilizing law and economics reveal the purpose of liability rules as in harmony with technology and the market: to reduce the cost of accidents and thus make shipping safer. This is supported by humans, their laws, and the systems they design and use on ships. The cost-benefit analysis acknowledges that there will still be losses with these systems on ships, as there are on existing ships. Liability rules, especially fault-based liability rules, create a motivation to take care with the acknowledgment that there will be additional costs when reasonable care is not taken. Liability rules maintain an emphasis on minimizing these losses, even if they cannot be eliminated.
Conclusion Law is often considered to be an impediment to innovation, and thus the temptation to exclude some more onerous aspects of law is understandable. It is a product of the
Through rejecting the fallacy of conflicting purposes and revealing the flexibility of liability rules in fulfilling the purpose of safer shipping, the complementary nature of laws and technology can be found. Therefore,
ultimately leave claimants worse off by removing an incentive to reduce the number and severity of accidents.
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 7
HANNAH STONES
Figure 6: Preventative shortfall in the exclusion of liability rules, regardless of changes in compensation. Although regulations provide the greater preventative effect, this is the consequence of ignoring the regulatory value of liability rules.
liability rules are not an inconvenience nor are remote-controlled and autonomous systems. Flawed reasoning cannot be allowed to dismiss the value of liability rules in shipping law, just as fear of the unknown cannot be allowed to prevent life-saving technology. Acknowledgment Thank you to Dr. Simone Schroff, associate professor at the University of Plymouth, for her valuable comments on this essay. u
1. Calabresi, G. [1970]. The costs of accidents: a legal and economic analysis. Yale University Press. 2. Geistfeld, M.A. [2017]. A roadmap for autonomous vehicles: state tort liability, automobile insurance, and federal safety law. 105 CLR 1611. 3. Geistfeld, M.A. [2018]. The regulatory sweet spot for autonomous vehicles. 53 Wake Forest L.Rev. 337. 4. Posner, R.A. [2014]. Economic analysis of law, 9th edition. Wolters Kluwer.
8 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Dr. Hannah Stones is a lecturer in law at Southampton Law School, University of Southampton. She is a member of the Institute of Maritime Law at the University of Southampton. She specializes in researching and teaching maritime law, and tort law and theory. This essay is based on work that Dr. Stones completed as part of her doctoral work at the University of Southampton, which was funded by the Leverhulme Trust Doctoral Scholarship, via Southampton Marine and Maritime Institute, and the Vice Chancellor’s Scholarship at the University of Southampton.
ISTOCKPHOTO.COM/SUPHANAT KHUMSAP
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 9
SEASPAN
Figure 1: Diagram representing various components of the digital twin. The artificial intelligence (AI)/machine learning (ML) platform interacts with the user, the digital twin, and the physical object. It includes physics-based or ML-based simulation models, the learning algorithm, as well as human-in-the-loop (HITL) components that assist in making correct decisions.
Introduction The application of artificial intelligence (AI) and machine learning (ML) in the marine industry is rapidly advancing, promising enhancements in efficiency, safety, and cost-effectiveness. The integration of AI/ML is shaping both shipbuilding and maintenance through improved design optimization, automated manufacturing, enhanced supply chain management, advancements towards predictive maintenance, and optimizing fuel consumption. AI/ML-driven condition monitoring, predictive maintenance, and safety analysis are pivotal for vessel life cycle management. Many modern vessels are equipped with sensors that can be used to monitor equipment health and prevent failures, reducing downtime, and optimizing spare parts management. The ongoing fusion of AI/ML in these sectors stands to transform maritime efficiency and safety. In this essay, we showcase an application of AI/ML technology in maintenance and remote diagnosis within fleet operations. Our progress includes anomaly detection and equipment health monitoring to swiftly identify irregularities and preempt failures. Supported by digital twins and AI algorithms, this proactive strategy transitions fleet
10 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
management from preventive and reactive approaches to predictive maintenance. To overcome challenges with data ingestion, centralization, and analysis, we are developing digital twins to simulate and optimize maintenance schedules, bridging the gap between complex data and actionable insights. Digital Twins A digital twin (Figure 1) is a virtual representation of a physical object or process. It is designed to provide accurate and dynamically updated representation of the form and functional response of a specific physical twin. Using a digital twin, we monitor performance, test different scenarios, predict issues, and find optimization opportunities. The digital twin mimics its real-world counterpart, receives live data from it, and changes accordingly to mimic the physical object through its life cycle. The twining process, in our application, involves numerous pieces working as a uniform system. The breakdown of the digital twin includes:
• • •
Digital visualization Data synchronization Modelling and predictive analytics
SEASPAN
Figure 2: Machine learning model development and inference workflow. The workflow begins with data ingestion and preparation pipelines as the initial steps. Subsequently, the training process involves partitioning the data into training, validation, and test sets for building a baseline model. Once the model is trained, it is deployed and integrated into the inference pipeline for real-time modelling or simulation.
• •
Simulation and testing Intelligent digital twin services
Data Ingestion Real-time data from sensors installed on the actual equipment is continuously fed into the digital twin. This data includes highfrequency, low-latency time-series data that contain 1) operational parameters, such as fuel consumption, emissions, load, and speed; 2) sensor data, such as temperature, pressure, and vibration; and 3) environmental measurements, including engine room temperature, weather, metocean, etc. Additionally, the digital twin incorporates the 3D representation and CAD model of the equipment’s physical structure, part details, maintenance records, and data collected during vessel tests, making data an invaluable component of our digital twin. Our digital twin continuously collects and integrates real-time sensor data from various sources on the vessel. Prior to implementation and modelling tasks, the data undergoes preprocessing, which includes tasks such as data cleaning, normalization, and feature extraction, to validate and prepare the data for data analytics.
Model Development The digital twin incorporates models and algorithms that simulate the behaviour and performance of the given equipment. The models consider operational parameters and environmental variables and provide both expected sensor values and the associated uncertainties for the given state. Developing the model is a critical step in building an effective digital twin. In our workflow (Figure 2), the baseline model serves as the reference point for evaluating system performance and enables the establishment of a benchmark for comparison. When creating models for digital twins, essentially, we consider a set of key criteria to ensure the model effectively represents its physical counterpart and supports the applications:
• • • • • •
Accuracy and fidelity Computational efficiency Scalability Dynamic behaviour Integration with sensor data Adaptability
In addition to building a baseline or model of what constitutes normal behaviour for the
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 11
SEASPAN
Figure 3: Vibration monitoring and anomaly detection using multivariate model-based approach. Drift values less than 5 are considered as normal behaviour.
equipment, using historical and real-time data, the digital twin employs predictive analytics to anticipate and forecast the equipment’s future performance. This allows for proactive maintenance and troubleshooting and enables the digital twin to assist in scheduling maintenance and repairs based on the equipment’s actual condition and usage patterns, optimizing maintenance activities to reduce downtime and costs. Equipment Health Monitoring Condition-based monitoring (CBM) involves continuously monitoring the condition of equipment or assets in real time or at regular intervals. The primary objective of CBM is to detect early signs of potential issues, faults, or deterioration in machinery, systems, or infrastructure. By identifying these problems before they escalate into major failures, CBM helps reduce downtime, extend the lifespan of equipment, and optimize maintenance efforts. This enables us shifting away from the prevailing preventive maintenance strategies that rely on performing maintenance operations on a predetermined schedule, which can be equipment usage, time, or a combination of both. In our equipment health monitoring application, first we create a model that estimates the expected values of the sensors (dependent variables) based on multiple
12 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
operating parameters (independent variables) using machine learning algorithms (Boosted Trees and Bayesian inference). Once we have established the baseline model, we compare the actual sensor values with the values predicted by the model. If the drift or the difference between the measurements and the predicted values exceeds a predefined threshold, we classify it as an anomaly. Similarly, we compare equipment performance against the model and monitor drift from the baseline. This approach is valuable since we have a good understanding of the system’s behaviour, and we can model it accurately. Figure 3 shows the results of vibration drift monitoring for six months of data collected from a vibration sensor installed on the shaft bearing of a tugboat. Our AI/ML anomaly detection and drift monitoring algorithms are able to both detect bearing defect and any gradual divergence from normal behaviour. The highlighted sections in Figure 3 show examples of shaft bearing greasing and grease seal sleeve inspection detected by the algorithm. Later, the updated model considers the multivariate patterns in greasing and inspection periods and demotes it as the expected behaviour. More data is expected over the next 12 months of operation to look at long-term bearing wear and degradation.
Conclusion In this essay, we introduced the workflow for developing the baseline model and using it for anomaly detection and equipment health monitoring. This methodology integrates various operating parameters and multivariate data obtained from the sensors, which is superior to the traditional threshold-based data monitoring. Our approach enables predictive maintenance, shifting away from the prevailing practice of preventive maintenance. The primary goal is to lower maintenance expenses, minimize downtime, and bolster safety across the entire fleet. u
Dr. Siavash Nejadi is a data scientist at Seaspan. His current interests include machine learning, deep learning, data-driven modelling, and operations research. With extensive experience in both industry and academia, he has contributed to various projects focusing on statistical analysis, artificial intelligence, simulation, and production optimization. Dr. Nejadi holds a PhD in engineering from the University of Alberta. Jennifer Busler leads a multidisciplinary team at Seaspan to develop, explore, and advance technical solutions in digital twins, data analytics, 3D and immersive visualization, autonomy, and green technologies for new ship construction and in-service support. As a passionate advocate for innovation integrated into strategic business growth, she has spent her career working across diverse technology areas within Canada’s innovation ecosystem, adopting differentiating technologies that address industry and customer needs, and delivering projects and programs with partnership in academia, industry, and government organizations.
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 13
Development of a Marine Cybersecurity Demonstration Platform
by Grace Pearcey
14 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
GRACE PEARCEY
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 15
Introduction The maritime industry plays an essential role in the lives of everyone. Over 90% of the world’s cargo is transported by ships, in addition to the thousands of passenger vessels, fishing boats, recreational boats, and military vessels on the ocean. This makes ships an enticing target for cyberattacks. From 2017 to 2020, cyberattacks on the maritime industry increased by 900%. Companies have lost hundreds of millions of dollars in a single attack. Despite the significant increase in attacks, most ships at sea today are still at high risk for cyberattacks. The potential impacts of these attacks are increasing as more automation is adopted on board. A significant contributor to the increasing number of cyberattacks is vulnerabilities within marine communications and control systems. Over a four-month research project, we have designed a platform to showcase cyberattacks on these vulnerabilities and their impact. The platform will be used to learn more about possible attacks, how to detect and defend against them, and how to adapt existing systems to become more resilient to potential attacks. Background The main communication and control systems on board vessels are summarized below. All of these systems are vulnerable to cyberattacks. We researched these systems to learn more about possible attacks and which attacks could be replicated on the demonstration platform. Automatic Identification System The Automatic Identification System (AIS) provides situational awareness to ships. Ships must periodically transmit AIS messages containing the ship’s name, position, speed, voyage information, and physical dimensions. AIS allows ships to view marine traffic and make safe decisions. AIS messages are broadcast over Very High Frequency (VHF) radio and are, therefore, susceptible to jamming. Jamming attacks
16 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
transmit noise over the radio frequencies used for AIS, so nearby vessels cannot send or receive any AIS data. A jamming attack could be hazardous in poor weather with low visibility when other ships are nearby. AIS data is neither encrypted nor authenticated, so spoofing attacks where fake AIS data is broadcast is also possible. False AIS information could lead a captain to change the route and steer toward an obstacle. VHF Data Exchange System VHF Data Exchange System (VDES) is a newer system that builds on the capabilities of AIS, provides higher data rates, and uses a more sophisticated modulation scheme. AIS only has a horizontal reach of 74 km, and VDES helps close these gaps with satellite communication, creating a global data exchange system, rather than AIS’s ship-to-ship and ship-to-shore communication. VDES has more bandwidth than AIS, so it can transmit other types and larger quantities of data. Without encryption, VDES is susceptible to the same attacks as AIS. However, VDES standards are yet to be finalized, and the greater capabilities of VDES mean that it could be designed to support encryption. Encryption of messages would mean that only the intended recipient can decrypt and read the message. The adoption of encryption has been met with hesitation due to safety concerns when information is not accessible to everyone on the water. Global Navigational Satellite System Global Navigational Satellite System (GNSS) refers to groups of satellites that provide positional and timing information to GNSS receivers. GPS is the most common GNSS system used in North America. GPS provides latitude, longitude, altitude, and time to receivers. GPS signals are relatively weak, and a hacker can spoof a GPS signal by transmitting a stronger false signal to the target receiver or jam a GPS receiver by sending noise.
Bridge Systems Two systems commonly found on ships’ bridges are an Electronic Chart Display and Information System (ECDIS) and a Voyage Data Recorder (VDR). The ECDIS is a digital chart display used for route planning and monitoring. The ECDIS is the most essential navigational device on a ship. Some ships are even required to have a backup ECDIS. Many ECDISs have out-of-date operating systems that no longer receive security updates. They could be susceptible to ransomware attacks, which would cause significant delays. The VDR records all data during a voyage. It reads data from onboard sensors, information entered into the ECDIS, and even voice recordings from the bridge. The VDR is essential to investigate incidents that occurred while at sea. Modifying information stored in a VDR could be a target for attackers trying to hide a cybersecurity breach. Inter-device Communication Protocols Most vessels use National Marine Electronics Association (NMEA) protocols for communication between onboard devices. These protocols are used in networks on thousands of ships that connect nearly all onboard sensors, monitoring systems, and devices transmitting and receiving tracking information. Autopilots, ECDISs, and instrument display panels all use NMEA data to guide human decision-making and autonomous control. Sensors could include GPS, engine monitors, weather monitors, current sensors, wind sensors, and rudder feedback sensors. AIS and VDES data are also sent through the network. NMEA 0183 is a serial protocol that uses characters to represent data in a humanreadable format. The network is centred around the ECDIS, and double cabling is required for bi-directional communication. NMEA 2000 sends messages over the Controller Area Network (CAN) bus. NMEA 2000 requires less cabling than NMEA 0813, as all devices are connected to a single backbone with T
connectors, as shown in Figures 1 and 2. NMEA 2000 data is transmitted through CAN frames and deciphered by Parameter Group Numbers (PGNs). A PGN for each type of NMEA 2000 message dictates the number and size of fields in each message. The NMEA protocols were not designed with cybersecurity in mind. There is no way of verifying that messages sent between devices are genuine. In the NMEA 2000 network, an attacker can access the entire network from a single compromised device due to the bus topology. Critical monitoring systems for the engine and power systems are fed through the network, and their alarms could be disabled, putting the ship at risk if a problem goes undetected due to filtered NMEA messages. Sensor readings and AIS information used by an autopilot or ECDIS could be modified, ultimately leading to collisions, delays, damages to the ship, and risks to onboard personnel. Platform Design The purpose of the demonstration platform is to have a physical boat that replicates systems on board real ships and demonstrates their vulnerabilities through planned cyberattacks. This project focused on the NMEA 2000 network, the critical communication system that links all devices together. The NMEA 2000 bus topology is highly vulnerable. No security protocols surround NMEA messages; attackers only have to compromise one device to access the entire network. The NMEA attack demonstrations consist of replacing the T connectors between devices with malicious connectors capable of reading and modifying messages. The malicious connectors create many possibilities for attacks. A connector, for example, can modify positional data before it reaches the autopilot, causing the boat to steer off course, demonstrating the significant impacts of a simple and undetectable swap. Similar to the O.MG Elite cable, which looks like a regular phone charger cable but is actually capable of
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 17
GRACE PEARCEY
GRACE PEARCEY
18 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Figure 1: Example of a National Marine Electronics Association (NMEA) 2000 network with two drop cables for NMEA devices and one drop cable for a power source.
Figure 2: A typical National Marine Electronics Association (NMEA) 2000 T connector.
GRACE PEARCEY
snooping on data sent through the cable, the hardware could be scaled down to fit within a regular T connector footprint. The malicious connectors are scalable, so the hardware could be placed on larger boats and ships to demonstrate the effects on a full-scale vessel. Malicious NMEA T connectors could modify latitude, longitude, and heading messages from the GPS. Modified positional data would cause the autopilot to steer the boat away from the planned route. The connectors could modify or erase messages before they are stored in the VDR. The connector could also modify or simulate AIS messages. The connector could use AIS messages to simulate another ship’s coordinates and place the simulated ship in the boat’s path, or the connector could erase AIS messages containing real information from nearby vessels, effectively hiding obstacles in the boat’s path. If the autopilot used AIS data for
Figure 3: The main hardware components of the boat. The National Marine Electronics Association (NMEA) network connections are shown in blue. The malicious NMEA T connector consists of the components inside the dotted box. A Raspberry Pi computer serves as an autopilot and controller for the boat and can be accessed over Wi-Fi. A 12 V battery powers the system.
obstacle avoidance, a strategically simulated boat could force the autopilot to steer around the simulated boat and into a real obstacle. The connector would also be capable of blocking or filtering messages. The results of the attack would vary depending on which connector was blocking messages and how the network is configured. Finally, a connector for one device could pass the device’s messages through without modification while also simulating an additional device. For example, the GPS connector could send the GPS data through but also send fake AIS data. The initial prototype of the platform demonstrates GPS spoofing attacks through the NMEA 2000 network. Figure 3 shows the system’s components and connections. To conduct the GPS attacks, the boat required an autopilot capable of steering the boat along a planned route using a GPS and compass,
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 19
GRACE PEARCEY
GRACE PEARCEY
Figure 4: Chartplotter view showing the boat’s real location at 47.33751°N, 52.442165°W, and heading of 15.6°.
Figure 5: Chartplotter view with no boat.
a NMEA 2000 network, and a malicious T connector to connect the GPS to the network. This will allow the connector to read and modify all GPS messages and simulate any type of NMEA device.
There are four attack modes in the initial design:
The boat uses an open-source marine autopilot, running on a Raspberry Pi computer. The Raspberry Pi is connected to the NMEA 2000 network and provides the autopilot with GPS readings from the network. A second controller program developed for this project runs on the Raspberry Pi to manually control the thruster and the rudder. The propulsion and steering can be controlled from shore over Wi-Fi. The boat’s routes are created in OpenCPN, an open-source chart plotting software, running on an onshore computer, and are sent to the Raspberry Pi via Wi-Fi. The malicious T connector consists of an ESP32 microcontroller and three CAN controllers, one for each network connection. NMEA messages are sent as CAN frames, so the CAN controllers and drivers in the microcontroller convert CAN messages into a readable format. The microcontroller receives NMEA messages from the three connections and deciphers the data based on the messages’ PGNs. The connector alters messages based on their PGNs and the desired attack before passing messages out to the other two sides of the network.
20 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
1. A passive mode, which passes all messages through unaltered. 2. A blocking mode, which stops the transmission of every message at the connector. 3. An opposite direction attack, which, for the autopilot, causes the boat to appear as if it is travelling opposite to the direction it really is. 4. A translation attack, which causes the boat’s position to appear translated by some amount. Cyberattack Demonstrations The four attack modes were tested in the lab. The results are summarized below. Passive Mode Figure 4 shows the boat located in St. John’s, NL, Canada, at 47.33751°N, 52.442165°W, with a heading of 15.6°. The yellow target represents the boat’s position, and the dashed red line represents the heading. This is the true position and orientation of the boat. The T connector successfully passed NMEA messages between connection points. Blocking Mode For the blocking attack, in Figure 5, there is no boat displayed on the map. The T connector blocked all GPS messages.
GRACE PEARCEY
GRACE PEARCEY
Figure 6: Chartplotter view with the boat’s apparent heading at 195.6°.
Figure 7: Chartplotter view with the boat’s apparent location at 48.64136°N, 51.78898°W.
Opposite Direction Attack The results of the opposite direction attack are shown in Figure 6. The heading has been changed by 180°, and the boat appears to be pointing in the opposite direction, while the boat’s actual orientation has not changed from Figure 4. For example, if the boat were to move north, the yellow target would appear to be moving south.
to enter a series of waypoints and force the boat to follow them. Additional expansions include adding equipment for AIS and VDES demonstrations and research.
Translation Attack The results of the translation attack are shown in Figure 7. The boat’s apparent position has been changed to 48.64136°N, 51.78898°W, while the boat has not moved from its position in Figure 4.
The T connector prototype can be scaled down to fit within a regular T connector footprint similar to the malicious phone charging cable. This could be placed on real ships to demonstrate attacks at full scale. This demonstration could be highly impactful in raising awareness of poor network security.
These four attack modes provide a simple illustration of what a malicious T connector can do. This demonstration merely scratches the surface of what is possible.
The final recommendation is to use the platform to investigate the addition of encryption to NMEA messages. This would prevent attackers from intercepting and reading messages. The encryption would have to be possible with existing NMEA 2000 hardware, as replacing this hardware on thousands of ships would be costly.
Conclusion and Future Work Recommendations for future work on the boat include expanding upon the NMEA T connector attacks. This could consist of adding more devices to the network and using a malicious connector for each one, allowing for research on how network arrangements affect the control a hacker could have over the network. The GPS attacks could also be more sophisticated, for example, allowing a hacker
This project has identified significant vulnerabilities within marine systems and developed a platform to demonstrate attacks targeting these weaknesses. The initial prototype focuses on the NMEA 2000 network, which connects nearly all onboard sensors and informational displays. Breaches to this network pose a high risk to the ship and its passengers’ security. Tampering with the network could lead to delays, force a ship to
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 21
take alternate routes, cause damages to the ship if it loses important data for obstacle avoidance, cause issues with the engine, fuel system, or power systems to go undetected, and ultimately lead to loss of life. The impact of cyberattacks on the network is even greater on vessels with increased autonomy, where there is less human interaction to detect a problem. The work enabled by this platform will further cybersecurity awareness within the marine industry and allow for a scalable platform for demonstrations, training, and research. Acknowledgment This project was funded by the Natural Sciences and Engineering Research Council of Canada, and supervised by Dr. Jonathan Anderson at Memorial University. u
22 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Grace Pearcey is an undergraduate student at Memorial University of Newfoundland studying computer engineering. She has completed internships at Avalon Holographics, Ciena, and Memorial University. She cofounded Iceberg ASV, a university design team competing in the international RoboBoat competition. The team designs an autonomous surface vehicle (ASV) capable of autonomous navigation, object detection, and object delivery. She co-leads the group’s software team. Her interests include autonomous systems, marine robotics, and cybersecurity.
Informative Cutting Edge Provocative Challenging Thought Provoking International
thejot.net Follow us @jotnfld
cioos.ca
Helping You Navigate an Ocean of Data Real-time and forecast marine conditions for mariners
CLEARANCE CURRENTS WATER LEVEL WIND
Marine Conditions Navigation R Gray
26 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Seafarer Maritime
ISTOCKPHOTO.COM/METAMORWORKS
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 27
YARA MARINE
Figure 1: A digital twin model created by Yara Marine.
Adoption of digital automation technologies has had varying degrees of success across the diverse landscape of maritime operations. How do we ensure success in digital adoption? Introduction Although automation and digitalization have gained a strong foothold in the maritime sector, particularly at larger companies, many companies remain uncertain about how best to harness these technologies. Insights from the Via Kaizen project suggest that the viability of automation and artificial intelligence (AI) in the maritime industry is contingent on the active engagement and endorsement of seafarers, and a collaborative approach to technological integration in shipping. The Via Kaizen project studied the impact on vessel operations by an AI-based voyage optimization tool (Figure 1) developed by Yara Marine Technologies and Molflow. It involved a comparative study of a chemical
28 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
tanker and a car carrier and revealed a stark difference in user acceptance and technology utilization depending on onboard culture. While the car carrier’s crew readily embraced the system, the chemical tanker’s crew largely abandoned the technology. Project Set-up A web tool was designed as part of the project, which generates optimized voyage plans based on user-defined routes and time constraints. This is done by integrating Molflow’s voyage performance data models with Yara’s Fleet Analytics and FuelOpt systems (Figure 2). The tool optimized voyage plans based on time of arrival and was introduced on board both vessels together with crew training, and then developed further based on user feedback. User stories were instrumental in shaping the tool’s development and ensured that it complemented existing workflows. The ability to import route files from a planning system as
YARA MARINE
Figure 2: Yara Marine’s FuelOp is a user-friendly system that delivers direct, real-time propulsion optimization, resulting in greater energy efficiency and lower fuel costs.
well as the feature to use an archive to quickly access recurring voyages were both developed based on user input. Refinements, such as improved alerts and the ability to re-simulate voyages in response to changing conditions without manual editing, were further direct outcomes of user engagement. The feedback loop established through training, manuals, and continuous dialogue allowed for iterative enhancements, improving the capabilities of the software and the value to its users. Technology Trust Mindset Social science researchers on the project from Halmstad University and the University of Gothenburg noted that the tanker crew’s reluctance to adopt the technology largely stemmed from a lack of trust in the system’s recommendations. Maritime operations are complex, and there were many challenges faced by the tanker and its crew, including structural issues such as unpredictable shortroute spot trading patterns interfering with fuel optimization objectives. For a technology to be adopted, researchers observed that it must not only demonstrate
clear advantages but also resonate with the crew’s operational context, which can only be achieved through intuitive design and a user-oriented approach. In the case of the chemical tanker, the crew did not trust that the system’s recommendations would still get the vessel to port on time. This meant that despite having access to an onboard tool to optimize navigation, the crew’s mindset – which was shaped by its working culture – prevented it from reaping the benefits. By contrast, the car carrier’s crew recognized the value of the system and integrated it seamlessly into its routines. This finding reinforces the significance of truly understanding user needs when designing technologies. Without considering user needs during development and implementation, not only will the acceptance and effectiveness of new technologies be significantly compromised, but so will the ability to harness user expertise and experience in improving technologies over time. Implications from the Study The implications of these learnings extend
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 29
beyond individual cases to the wider maritime industry. The project takeaways emphasize that the human element remains at the heart of maritime operations. Seafarers are not just operators but critical stakeholders in the implementation of digital solutions. Technologies disregarding the practicalities of seafaring and the expertise of the crew are destined to be neglected and abandoned. Such an approach detracts from the potential of digital technology to augment human capabilities and improve the industry. Automation should be seen as a means to support crew decision-making, leveraging large datasets and digital support for, in this case, optimized voyage planning. We are being propelled by technological advances towards a more efficient, digitalized maritime industry, but we must put this power in the hands of our trusted and skilled crew members. The journey towards fully integrated, intelligent automation systems must include fundamental appreciation for the seafarers’ role, crafting technologies that empower rather than exclude. u
30 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Anders Bergh has worked with product development and sales in the marine engineering field for more than 15 years, always with a focus on efficiency gains for seagoing vessels. In his current role, he is managing the Fleet Performance/Customer Success team within Yara Marine Technologies (Vessel Optimization). This has the sole focus on enabling ship owners and operators to employ the right technologies and strategies to get from A to B with as little energy spent as possible.
Enhancing Data Management
32 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
through SmartVessel.io by Peter Walsh
ISTOCKPHOTO.COM/JAMES
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 33
Introduction This essay highlights how Seashore Maritime Services Ltd., through the evolution of SmartVessel.io, has redefined how data management in the maritime industry can be conducted. From a vessel and management perspective, there is an intense amount of data that is involved at the operational level on board a vessel, which relies upon a large reserve of resources to manage effectively. Operational information such as voyage planning, voyage certificates, crewing requirements, crewing certificates, class requirements, machinery parameters, unplanned/planned maintenance, safety management systems, health-safetyenvironmental systems, and operational procedures encompass part of the data that is possible in maritime operations. SmartVessel.io was structured and designed to manage all types of maritime operational data, and it stands out for numerous reasons. It is the only software program that captures all aspects of a vessel’s operational information, it has the capability to communicate with outside sources using Internet of Things capabilities, and has a cloud-based system that allows shore-based managers and Designated Person Ashore (DPA) to monitor, view, and present vessel data in real time directly from the vessel. SmartVessel.io demonstrates how utilizing a single software system to manage and organize all of a vessel’s data, particularly from onshore real-time vessel access, is beneficial in increasing operational efficiencies, reducing reactive measures, and ensuring the effective use of resources. SmartVessel.io Overview Early development attempts for digital technology management systems for vessel applications resulted in the focus of singular entities, therefore resulting in multiple software systems that do not integrate or communicate with each other being used to manage individual vessels or fleets of vessels.
34 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
SmartVessel.io takes a complete approach to vessel data management, which incorporates a cloud-based system with stand-alone capability which allows both shore-based and vessel-based personnel access to the vessel’s data network even when the vessel does not have constant internet connectivity. It does so by using a vessel-based server which saves all SmartVessel.io data in real time, and then syncs to the cloud at a scheduled arrangement or manually when internet connectivity is an issue on board. Once synced, the shoreside personnel can view all data submitted, and can even sign off on documentation, such as toolbox talks and safety checklists, to ensure compliance is met in regard to any regulatory requirements. This connectivity allows for constant and complete real-time monitoring from shore personnel of a vessel’s operational data, ensuring that a high level of conformance is performed on board in regard to regulatory requirements from crew members on board. This allows for management to track non-conformities that are the task of crew members and allows for the continuous follow up of tasks remaining. The successful integration of SmartVessel. io is contingent on the accurate and complete profiling of the operational requirements of the vessel. Requirements that a vessel is bound to can consist of articles from classification societies, insurers, clients, operators, or any other third-party entity. These requirements are imbedded into SmartVessel.io during the system implementation, therefore enabling the system to notify individuals through a role-based structure of their specific tasks associated with the designated requirements or notifications. A key design feature of SmartVessel.io that connects third-party entities to a vessel’s data is through the use of application program interfaces (APIs) and Internet of Things (IoT) processes. APIs and IoT describe the process of connecting and communicating data between different computer programs and different devices which could include sensors, processing
abilities, and software. Through this integration of these concepts, SmartVessel. io is designed to connect to third-party programs or software in order to streamline the transfer of data efficiently between the vessel and relevant stakeholders in numerous applications. For example, two features developed for use in SmartVessel.io relate to government and classification society regulations and requirements. In Canada, the government requires large fishing vessels to record and submit fishing data daily to Fisheries and Oceans Canada (DFO) in order to ensure the vessel is complying with regulations and provide relevant fishing data. In order to streamline this vessel process, SmartVessel.io created a feature that allows inputted fishing data from the bridge to be sent directly to DFO’s data program, therefore removing several data transfer steps and optimizing this common fishing process. Another example with regards to vessel certification processes involves classification societies, which regulate certain standards for vessels and ensure compliance is met on behalf of Flag States. There are numerous classification societies which ship owners and operators can use; however, the most commonly used classification society globally is Det Norske Veritas (DNV). DNV has developed its own accessible program, Veracity, which allows ship owners and operators to monitor vessel certificates, vessel comments, and comment history. In order to streamline the process of updating vessel certificates in SmartVessel.io manually, the developers of SmartVessel.io created an application with DNV to directly link Veracity certificate data to update, track, and monitor vessel certificates for its client in real time within SmartVessel.io. Through the design implementation of cloud syncing and APIs/IoT integration, SmartVessel.io has been developed to optimize the transfer of vessel operational data in real time to a customizable swath of industry stakeholders and invested interests. With the continuous advancements
of technology, SmartVessel.io has the capability to adapt and optimize how that data will be managed. SmartVessel.io Data With SmartVessel.io having the capability to transfer vessel operational data between the vessel, shoreside management, and third-party entities, the features and data categories that are built into SmartVessel.io are essential to capture all relevant operational data on board a vessel. When setting up a new profile into SmartVessel.io, there are two sections per profile: Company Profile and Vessel Profile, in which multiple vessels can be attached under one company profile and are linked to that specific company. Data categorized into these profiles are used as either a historical database to store information, or used as inputs to features within SmartVessel.io that assist with operational aspects of the vessel, or both. Data categories from both profiles are listed in Figures 1 and 2. Each of the data categories for both company and vessel profiles have been implemented into SmartVessel.io because they were determined to be essential components of a vessel’s operations. The format of these data categories is customizable upon implementation, therefore allowing for a company to adapt preestablished internal documentation formats, including checklists and forms, into SmartVessel.io in order to streamline the integration phase. From these data categories, several features standout as essential entities for a vessel’s operations in which data is monitored and notifies relevant roles of required actions to be taken. These consist of Certification Management, Vessel Logs, Planned Maintenance, and Voyage Assignment and Passage Planning. These are critical components to a vessel’s operation and are monitored continuously regardless of vessel statues (i.e., active voyage, dry docking/refit, etc.). The following sections will describe the design of these listed functions as well as how the data is utilized and shown in a real-world application.
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 35
36 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
SEASHORE MARITIME SERVICES
Figure 1: Company data sections within SmartVessel.io.
SEASHORE MARITIME SERVICES
Figure 2: Vessel data categories within SmartVessel.io.
SEASHORE MARITIME SERVICES
Figure 3: Other vessel certificates apart from DNV certificates. DNV certificates issued for this vessel are updated through syncing with DNV’s Veracity program. This includes DNV surveys and DNV comments.
Certificate Management Certification in the maritime industry encompasses a large range, including vessel certifications, crew certifications, machinery certificates, fire equipment certifications, and life-saving appliance certifications. Each of these certification categories has sub-categories for certification for different requirements; for instance, vessel certifications can include classification society certificates, Flag State certificates, and classification society scheduled surveys. Maintaining certification status for any item is essential for a vessel to operate, as failure to renew can prevent a vessel from sailing. As an example, a vessel under Canadian Flag without bunker and wreck removal insurance certificates will not be permitted to sail under Canadian Law. The same applies to a vessel that does not have the required crewing certificates to meet necessary safe-manning requirements such as a required number of first aid providers, or necessary engineering certification. This is why it is essential for the proper management of vessel and crew certificates to be organized efficiently, in a way that allows management to easily identify certificates and surveys that are soon expiring, as well as crewing certificates that are required for the intended
voyage. Once certificate information such as name, issuer, issued date, expiry date, and attached pdf or jpg file is entered into SmartVessel.io (Figure 3), the system tracks and notifies when items are due to relevant stakeholders, who can then effectively plan for their renewal. As well, when a voyage is planned, a crewing list is submitted into SmartVessel.io and is then checked to ensure all crew members have necessary certificates within date, and the system checks to make sure enough relevant crewing certificates are present in order to meet safe manning requirements. Utilizing the one system to monitor all types of vessel certificates instead of individual departments overseeing them not only reduces workloads for managerial position stakeholders, but as well allows for efficient planning of vessel operations. Vessel Logs Vessel logs are an important aspect of a ship’s operation and can consist of different types of logs including deck logs, engine logs, official logs, and housekeeping logs. Logs are official vessel documents that are used to record relevant information of a vessel’s or department’s activities. For instance, an engine logbook (Figure 4) would
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 37
SEASHORE MARITIME SERVICES
Figure 4: Engine log with recordings shown. The top bar can show the different types of engine room logs created between shifts, daily checks, weekly checks, and triply checks.
SEASHORE MARITIME SERVICES
Figure 5: A comparison between two engine log parameters over a given period of time. In this example, the RPM of the main engine is compared to the speed of the vessel. This tool can be used to compare any parameter entered in the engine log.
be used by the vessel’s engineers and would consist of vessel machinery and parameter recordings as well as checks based on a time or shift interval system. These logs act as a vessel and department history and can confirm compliance of crew members’ duties to their individual departments. If a piece of machinery breaks down, the engine logbook would be consulted during an investigation to determine whether any parameter records were altered outside of its normal range recently, or if any issues were noted in general. SmartVessel.io has
38 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
the capacity to create custom log types for different department logs and are tailored to individual vessels and their individual needs, as historic paper logbooks are generic and often have sections that do not comply to all vessel types. SmartVessel.io logs add additional value to this common process by allowing shore personnel to view logs in real time through syncing with the cloud. Therefore, an onshore DPA, who is responsible for the vessel, can actively monitor the synced logs to ensure
SEASHORE MARITIME SERVICES
Figure 6: The planned maintenance schedule for a vessel. The image shows active items that need to be attended to, as well as the date by which they are required to be completed. This section also displays upcoming tasks, completed items, and non-conformity items.
crew compliance of completing the logs and note any items that need attention once the vessel arrives in the next port of call. An additional component to the digital SmartVessel.io log design is that it can trend numerical data over extended periods of time and can be compared to other datasets. For example, an engine log (Figure 5) that records both exhaust temperature and oil temperature of an auxiliary generator on board a vessel would be able to compare both sets of data to note the relation between both parameter sets. This data comparison can also be used during a potential investigation to pinpoint how a piece of machinery was operating before a given breakdown. Other noteworthy entities of SmartVessel.io log designs pertaining to numerical values, such as machinery parameters, are customizable alarm settings that would alert relevant stakeholders if an entered value was outside a necessary range, and as well how the data is entered into SmartVessel.io. There are two options for numerical data to be entered into SmartVessel.io: manually from onboard crew members at given intervals, and automatically through a network of installed sensors. Data from sensors can be periodically recorded into relevant logs, as well as submitted directly to the cloud. With an arrangement using machinery sensors, shoreside management has access to real-time machinery data
even when the vessel is sailing. Logs are an essential component of a vessel’s recorded data structure, and SmartVessel.io creates a system which allows for the confirmation of conformance, and for greater access and analysis of current vessel parameter data. Planned Maintenance Planned maintenance is a critical part of any vessel, as unexpected breakdowns can impose both time loss and excessive costs. Planned maintenance is a set plan to perform designated maintenance tasks on machinery items with the intent of reducing the risk of machinery breakdown or failure. A plan would encompass all critical machinery items on board a vessel, such as main engines, auxiliary generators, freshwater systems, purifier systems, and any other items deemed critical. A planned maintenance system is required for most large ocean-going vessels, with regulators requiring more types of vessels in the future to incorporate them as well. Regulators require these vessels to demonstrate the functionality of their individual systems in order to accept them. It is also in the vessel owners/operators’ best interest to incorporate one as it could interfere with vessel operations if insurance agencies require a system prior to authorizing coverage in certain circumstances.
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 39
SEASHORE MARITIME SERVICES
Figure 7: The list of required tasks necessary for each vessel status. For example, in order for the vessel status to be changed to Departure Ready, all of the triple safety inspection checklists from the previous voyage must be reviewed and signed off by the appropriate officers of the vessel.
SmartVessel.io has established a customizable planned maintenance system (Figure 6) which alerts relevant stakeholders in advance of their required tasks, and tracks the task until it is marked as complete, therefore allowing shorebased management to monitor non-conformities of crew members. Planned maintenance tasks are designated at time-based intervals for machinery items, most often at the recommendation of the manufacturer, such as changing the oil and filter on an engine after one hundred operating hours as a broad example. SmartVessel.io can take operating time parameter data from any machinery item, either inputted manually or through sensor data, and associate that time towards the planned maintenance schedule per machinery item, thus ensuring that accurate time intervals are recorded and notifications issued accordingly. Seashore Maritime Services Ltd. is as well undergoing the process of getting the planned maintenance system under SmartVessel. io ‘Type Approved’ by DNV, which would signify it as a pre-approved system that ship owners/operators would implement without the need for DNV inspection. Voyage Status and Passage Planning A vessel’s scope of operation and status can be defined under several different classifications – Dry Dock/Refit; Cold Layup; Warm Layup;
40 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Standby; Departure-Ready; Active-Voyage; and Pre-Arrival. Distinction between these vessel statuses is important because they require different services and checks, with operators having particular roles per status. SmartVessel.io prioritizes this aspect, and has it designed as the main hub area of the program. The shipowner/operator designates the vessel status in the program and is then prompted with the appropriate required actions. Warm layup, cold layup, and dry dock/refit are all non-operational statuses of the vessel, where the vessel is not operating at sea for extended periods of time (Figure 7). Standby, DepartureReady, Active-Voyage, and Pre-Arrival statuses are all associated within the operational aspects of a vessel and are all grouped into Passage Plans. Passage Plans are the documented arrangement that a vessel will undertake for a given voyage, and SmartVessel.io records data to each associated Passage Plan. When planning a passage plan in SmartVessel.io, the program follows the steps per each operational status prior to their completion, therefore mimicking the actual operation of the vessel. The following are descriptions of the flow of operational statuses sequences.
•
Departure-ready status denotes the circumstances when a vessel is ready to
SEASHORE MARITIME SERVICES
Figure 8: Overview of an already completed passage plan. The coloured bar showcases the changing status of the vessel and how long of a duration of each status: Orange=Standby, Red=Departure-Ready, Green=Active-Voyage, Blue=Pre-Arrival (not shown in screenshot). Using this layout, management can view when items were completed on board the vessel such as drills and checklists.
be carried out, Smartvessel.io provides access to a variety of supplementary documents, such as manufacturers’ manuals, diagrams, and other instructions to assist in carrying out the work. If the maintenance and/or repairs are not carried out during the voyage, they are scheduled through the “Ship Operations” function to be addressed either at a port call, refit, or dry docking. They remain as “outstanding items” until resolved.
undertake a specific voyage. It could be the end point of a long readiness process from cold layup, or immediately following completion of another voyage. When a vessel achieves departure-ready status, the preparations for the actual voyage occurs. Smartvessel.io sends notifications to the assigned officers, based on the planned sailing date from the vessel assignments. Acceptance of these notifications initiates readiness for the upcoming voyage.
•
It is the active-voyage status which moves the Smartvessel.io focus from planning to implementation. Throughout the voyage, SmartVessel.io notifies the responsible ship’s personnel of the actions needed to safely and effectively operate and maintain the vessel. The program will not only provide notifications of the necessary steps but will also make available the standard operation procedures and other documents required. As well, it provides the checklists to confirm that all steps are carried out and maintains a record of those actions. Where maintenance or repairs are to
•
Pre-arrival status is the status following the active-voyage status and is the operational point in the passage plan prior to arriving at the port of destination. Specific checklists are issued with prearrival activities and checks, and ship personnel are notified of their roles and responsibilities for this stage. Once the vessel has arrived in port, the status of the vessel can be changed to any of the three non-operational statuses. Once completed, each passage plan is stored in SmartVessel.io as a history of the vessel (Figure 8) and can be reviewed at any time to view checklists and maintenance tasks performed throughout that specific period of operation.
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 41
Conclusion SmartVessel.io was designed to meet the needs of a future maritime industry and has done so by becoming an integrated platform for a complete vessel data system. With core features such as cloud connectivity and API/ IoT capability allowing for seamless data access and transferring, it allows for the data categories within SmartVessel.io to be an effective tool for both storing and managing vessel operational data in real time, as well act as a historical tool for past operations. With each aspect of SmartVessel.io customizable, the program can be tailored to the needs of each user and their vessel, therefore allowing for a smooth transition between existing paper-based systems to the digital format of SmartVessel.io. The described main vessel operational features of SmartVessel.io are integral components of a vessel’s operation, and the inclusion of these processes into the program allow SmartVessel.io to become a singular management software to oversee a complete vessel dynamic. With many other features of SmartVessel.io not described above, the system has an enormous amount of potential for users to manage their data more effectively. With technology in the maritime industry ever advancing, the integration of effective and efficient data management systems will only benefit vessel owners/ operators with reduced costs and unexpected vessel downtime. SmartVessel.io has the design and the capability of meeting these growing demands, ensuring the effective management of a large fleet of cargo vessels, down to a singular fishing vessel. SmartVessel.io can meet the needs of any client or vessel type for the new advanced age in vessel management. u
42 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Peter Walsh has always been involved in the maritime industry, even from a young age. Starting work in his family-owned marine industrial repair shop while still in grade school, he has been familiar with the maintenance and industrial repair aspect of the industry with a broad range of hands-on experience. A graduate of the Fisheries and Marine Institute of Memorial University, Mr. Walsh first completed his ROV Technician Diploma before continuing on to complete his Bachelor of Technology through the Autonomous Underwater Vehicles Advanced Diploma program. Upon brief stints in the aquaculture and ROV industry, he returned to Newfoundland in 2020 to work with Seashore Maritime Services Ltd., a local marine consultancy company. His duties within the company consist of overseeing safety management system programs for local vessel groups, third-party surveys for marine insurance claims, overseeing vessel refit maintenance work, and as well assisting SmartVessel. io clients with the program. Having recently completed his master of maritime management through Memorial University of Newfoundland, Mr. Walsh is excited to continue his presence in the maritime industry and to explore different aspects of it. https://smartvessel.io
The Yara Birkeland under the Brevik bridge. This vessel transports containerized fertilizer between the factory in Porsgrunn and export ports in Brevik.
44 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Autonomous Ship Activities
in Norway by Ørnulf Jan Rødseth
NFAS
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 45
Introduction The initiative to establish the Norwegian Forum for Autonomous Ships (NFAS) was taken by the Norwegian Maritime Authority, the Norwegian Coastal Administration, the Federation of Norwegian Industries, and MARINTEK (now SINTEF Ocean) in the early part of 2016. The interest in this initiative turned out to be large, so the forum was established October 4, 2016, in Oslo in conjunction with a conference on autonomous ships. More than 110 people attended the initial conference and 30 organizations and individuals signed up to NFAS immediately after the conference. This was a few days after the Trondheim test area for autonomous ships was opened. The forum has continued as an interest organization since then and has arranged a number of national and international meetings. The International Conference of Maritime Autonomous Surface Ships (ICMASS) is co-arranged by NFAS together with various national organizations. The International Ship Autonomy and Sustainability Summit is co-arranged between NFAS, EU’s DirectorateGeneral for Mobility and Transport, and Nor-shipping. NFAS also publishes some whitepapers, such as our definitions document related to autonomous ships. Active participation in the international events as well as in international publications and conferences make NFAS an important and international public front for our members. The main activity, however, is facilitating networking among the members and keeping them informed about developments within the area of autonomous ships. This essay extends this central activity by providing the international community with updated information about maritime autonomous surface ship (MASS) activities in Norway. Why Norway? The maritime industry is one of Norway’s most important industries and plays an important role in value creation and employment in large parts of the country.
46 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Here, the maritime industry includes all businesses that own, operate, design, build, or supply equipment and specialized services to ships and other floating devices. Along the Norwegian coast there is a complete maritime cluster that includes world-leading companies and organizations. The maritime industry is Norway’s second largest in terms of export value, after oil and gas. Total value of exports in 2022 was approximately USD 20 billion, which is 13% of total exports from Norway. The survival of the Norwegian maritime industry is also dependent on advanced technology. Norway is a high-cost country and competitiveness must be based on high efficiency and innovative technical solutions. This can be seen, for example, in being first adopters of LNG as ship fuel and in applying batteries to ship propulsion. It can also be seen in being the first country to launch a merchant ship for autonomous operation, the Yara Birkeland. The Norwegian maritime cluster also makes development of new ship concepts much easier. Advanced equipment industry and yards, experienced and innovative ship owners and operators, internationally leading insurance and juridical expertise as well as solution-oriented authorities work together to develop new and sustainable ship concepts. Finally, ship transport plays a vital role in Norway, which has the world’s second longest coastline after Canada. Many deep fiords and inhabited islands make ships indispensable both for cargo and passenger transport. In Norway, ships are responsible for 41% of inland transport measured in tonkm and for 90% of imports and exports when measured in tons. What Do We Mean by “Autonomous Ship”? There are many opinions on how autonomy in transport should be defined, both with regards to ships and also in more general discussions; see Rødseth and Vagia [2020] under “Further Reading” for an overview of some of the
Table 1: Comparison between cars and ships.
different opinions. To see some of the specifics of ship autonomy, it may be useful to compare a ship to a car as is done in Table 1. The figures are approximate and correspond to a medium sized ship in international trade, e.g., a bulk carrier of around 30,000 deadweight tonnes. There are two important observations one can make from this table:
1. The size and cost of ships make the cost
of maintaining a remote monitoring and support centre relatively small. For cars, remote support or monitoring is, in general, not cost-effective. This means that cars need to aim for fully autonomous operation. 2. The lower speed and longer detection range for ships also make it viable to have a remote operator that oversees several ships. It will be technically feasible to warn the operator sufficiently early for the operator to get full situational awareness before action is required. This is a significant problem for cars, and one will again have to aim for full autonomy rather than human assisted operation. It is also an ongoing discussion if computers will be able to automatically handle any and all situations that may occur during a voyage. Most Norwegian experts agree that this is very unlikely and that getting approval from the Flag State to operate fully autonomously will be very challenging. Thus, in Norway all commercial developers of autonomous ships aim for remote support through a remote operations centre (ROC). The sensor and automation systems on the
ship should be able to operate the ship without human assistance in less demanding operations, e.g., transit in relatively open and uncongested waters, but also to warn the remote operator in time to take over control in a safe manner. At time of writing, there are two companies in Norway that offer ROC services: Massterly and REMOTA. This position is also supported by recent development in the International Maritime Organization (IMO) where the current opinion is that an autonomous ship (MASS) needs to have a master, although the master does not need to be on board. Such a master should also have the means to intervene when necessary. Thus, some form of ROC will be necessary. Levels of Autonomy As was pointed out in the previous section, the prevailing opinion in Norway is that autonomous ships will be assisted from a ROC in complex situations that the automation system cannot be trusted to handle. Furthermore, the automation system should be able to continuously assess the situation and reliably warn the ROC operator sufficiently in advance before takeover for the operator to gain the necessary situational awareness before actions are required. This means that autonomy can be looked at as a binary concept. When automation is in control and if it has not warned the operator about a near handover of control, the automation can be trusted to safely control the ship without any human supervision or intervention: The system is autonomous. In other situations, the operator may be aided by the automation, but the operator is still responsible for safe operation: The system is not autonomous. This also implies that the
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 47
commonly occurring discussions on levels of autonomy are less relevant. However, it is still relevant to talk about, e.g., levels or degrees of automation. This would be dependent on how many of the onboard processes the automation can operate autonomously or under what conditions the system can be autonomous. Notable Commercial Projects There is very high activity on research and development of autonomous ship technology in Norway. NFAS is the national interest organization and currently has 41 members of which six are universities or research institutes. Fourteen industry members also do extensive developments, and several commercial projects are currently under way. The best known is probably Yara Birkeland, which transports containerized fertilizer between the factory in Porsgrunn and export ports in Brevik. The ship is 80 metres long and has a capacity of 120 twenty-foot containers or equivalent units. It is currently running with crew on board but is scheduled for uncrewed operation from 2025. The second project that has received significant international interest is the two automated truck ferries that have been commissioned by ASKO – Norway’s largest grocery wholesaler. These are named Therese and Marit. Each ship has a length of 67 metres and capacity for 16 fully loaded, 29-tonne standard EU trailers. The ships go into a logistics system connecting two warehouses on each side of the Oslo fiord with a fully electric transport system where the ferries are used to significantly reduce the distance travelled by road. These ships are also sailing, but currently with crew on board. Uncrewed operation is planned for 2025. In 2023, ASKO signed a contract to also carry dangerous waste from Moss to a storage facility at Langøya. Massterly, a joint venture between Kongsberg and Wilhelmsen, will supervise and operate
48 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
the three above ships from its ROC in Horten, Norway. The ships are sailing in and near Oslofjorden in the south of Norway as shown in Figure 1. The above two projects are examples of what can be called “logistics extensions,” i.e., relatively small ships that are parts of a more complex transport chain. There are other projects of this type under way in Norway that at the time of writing have not been published. Another group of projects are small passenger ferries for urban waterways. Zeabuz in Trondheim is a spin-off from the local university that has cooperated with the ferry company Torghatten to develop a small city ferry that is operating in Stockholm. This ferry, called Estelle, is currently operated by one combined navigator and safety officer, but can in the future be operated by a single non-navigator safety supervisor. Another Norwegian company, Hyke, will deliver a similar ferry solution for use on the Seine in Paris during the next summer Olympic games. It already has a first design that has been demonstrated in several cities in Norway. For the time being, both companies aim for autonomous ferries that have a single safety person on board to assist passengers during the voyage and in any incident that may occur. A variant of the small passenger ferry is larger highway car ferries. Kongsberg is supplying Bastø-Fosen with auto-crossing and autodocking systems that are also intended for automated operation. However, due to current regulations on minimum safety crewing, the ferries are still operating with normal crews. This is likely to change as time goes by as the wish to provide better services to the customers without a prohibitive penalty in increased crew cost is strong. This includes using more and smaller ferries to reduce waiting times and to provide, e.g., on-demand services on crossings with relatively few passengers. The Norwegian Public Roads Administration recently awarded a contract to Fjord1 for the Lavik-Oppedal route that also includes a commitment for the
Figure 1: Operational area of Yara Birkeland and ASKO ferries (map from https://kystinfo.no/). Table 2: Some already identified market segments.
operator to prepare for and perform tests for autonomous operation of one of the ferries. Some Known Markets for Autonomous Ships The previous section briefly described some commercial companies that pursue autonomous ships as a business idea, and it is obvious that these different areas represent actual markets. Table 2 gives an overview of some market segments we already have identified as relevant for Norwegian industry. The relative size is a guess at how big the market may be. This is a relatively unqualified classification, and for information only. The inland column
specifies if the solution is attractive for inland waterways. This may be accurate for more cases than indicated but shows where projects and analysis have taken place. The passenger column indicates if the ship is carrying passengers and needs to take special safety precautions for these. The three first segments have already been discussed above. The island and small settlement ferries is a special segment for small communities that are dependent on waterway crossing for transport to the mainland, e.g., over fiords or from islands.
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 49
There are several such cases in Norway with settlement sizes varying from less than 50 to around 700. The challenge here is to provide around the clock service without excessive crew cost. One may build a bridge or tunnel, but this is often prohibitively expensive due to water depths or other topographical features. Autonomous ferries with a minimum safety crew, or even using residents themselves with special safety training, could implement more regular services or on-demand services if the traffic patterns warrant it. These concepts are examined in the FlexFerry and Autosafe projects. A very interesting concept of a hub-and-spoke transport system is investigated in the AEGIS project. One of the cases here is a short sea feeder from the continent up along the west coast of Norway that only calls on a few larger and highly automated ports located directly in the main fairway. Here the cargo is transhipped to small electric shuttles that take the cargo to smaller quays in the sheltered waters covered by the main hubs. There are several benefits with this principle:
• • •
Reduced time and energy use for the feeders as they do not have to sail far into the fiords. No emissions or noise in densely populated areas as shuttles are battery operated. Reduced infrastructure area and cost for urban and rural destinations as shuttles can use roll-on, roll-off solutions that can take cargo directly into city centres or to rural destinations with very little requirements to port and terminal infrastructure.
This concept can also be used when ports are moved out of city centres. This is an increasing problem in Norway and many other places as city real estate becomes more and more expensive and requirements of noise and traffic from industrial activities in cities becomes stricter. Normally, moving ports out of the city centre will lead to more truck transport, but this can to a large degree be alleviated by the shuttles in the hub-and-spoke system.
50 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
The AEGIS project is also investigating the possibility of operating the short sea feeders autonomously. This is feasible and can give significant benefits but is more demanding in regulatory terms as it will require acceptance for autonomous sailing by several national states. The relative benefits are also likely smaller than for the hub-and-spoke system with conventionally manned feeders. Short sea bulk transport has been investigated by the AUTOSHIP project. In this project, the case was a fish feed vessel and the project demonstrated autonomous sailing over one of its round trips. This is somewhat similar to the short sea feeder, but this ship is sailing in national waters, which makes approval easier. The case for fully uncrewed operation is not clear, but there are obvious benefits in being able to sail with uncrewed bridge, e.g., during night-time. The AUTOSHIP project also demonstrated autonomous sailing on small inland waterways. This is also an interesting concept that is being developed further in the Netherlands and Belgium. There are also interesting business possibilities in the deep-sea markets, but these will generally require higher investments and will depend on international acceptance of uncrewed and autonomous ships. This makes the risk inherent in starting such projects much higher than for the smaller scale project types described in this section. USV, MASS, and the Different Approval Regimes As in most other coastal countries, we also have significant activities in the uncrewed surface vessel (USV) segment. USVs are smaller vessels that do not fall into the MASS category, i.e., vessels that do not require approval by the Flag State for operations in international or general national trade. There is no clear definition of a USV, but one will often refer to vessels below 25 m length as a USV, while larger vessels are looked at as MASS.
ØRNULF JAN RØDSETH
Figure 2: Gartner hype curve for autonomous ships (author’s opinion).
There is an unclear border between MASS and USV and some of the smaller cargo shuttles from the previous section can sometimes be classified as a USV.
the IMO guidelines on approval of alternative designs which many consider to be the main pathway towards approval of MASS operating in national waters.
In Norway, the USV and MASS markets are mostly separate. Some companies may provide products in both segments, but then the systems normally are quite different and with different types of sensor and control components. The main reason for this is probably the regulatory regime, which is very different between the MASS and USV types of vessels. A MASS will need to have exemptions from minimum number of crew requirements, which in most cases do not apply to USVs. Granting an exemption is legally possible in Norway, and the Norwegian Flag State Authorities have issued a circular that describes the process that must be followed to get this exemption. However, the process requires a risk-based approach and proof that the technical solutions cannot result in an intolerable risk. As neither hazards nor probabilities for undesired outcomes are very well understood, this has proved to be challenging. Neither Yara Birkeland nor the ASKO ferries have, at time of writing, received the final approval to sail uncrewed. This is expected to be resolved by 2025, when both ship types are expected to start uncrewed operation. The Norwegian circular is based on
Currently, there is limited transfer of technology between the USV and the MASS segments. As mentioned, this is probably caused by differences in regulatory regimes. However, it is also clear that experiences from USV development and deployment can provide valuable insight into problems that MASS need to solve. Also, as the USV market matures and larger vessels are put into operation, the differences in technology are likely to become much smaller. Conclusion and Outlook NFAS has been one of the organizers of ICMASS (2019 to 2023) and the International Ship Autonomy and Sustainability Summit (2019 to 2023). Through these events we have observed some of the later day’s changes in opinion on autonomous shipping as, according to the author’s opinion, can be depicted as a Gartner hype curve as illustrated in Figure 2. One can argue that the era of autonomous ships started with the MUNIN project in 2012. For the first year most reactions were negative, but this started to change around 2013 when many of the big actors in the
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 51
maritime business came out with their different autonomous concept ships. After that, there were many articles and public presentations about autonomous ships and the benefits. This did, however, stagnate around 2017, and one could say that we reached the trough of disillusionment as proposed by the Gartner hype curve around the start of our conference series in 2019. This did not last that long as relatively soon new concrete projects were published; in particular, the Yara Birkeland and the ASKO ferries. This was followed up with other projects around the world. In 2023 the impression from presentations both at ICMASS and the Summit is that we are rapidly approaching the plateau of productivity, although practical project developments still are somewhat slow. Thus, the impression is that there are very solid business cases for autonomous ships, but that we need time to develop these and apply results in new business areas. Regulation and approval processes remain an issue, but there are also developments that point towards both an international agreed on regulatory scheme and methods to make the approval process simpler. This must be the subject of future articles, but the conclusion from Norway is that autonomous ships will certainly be a part of our near future. u
IMO [2023]. MSC 107/5/1, Report of the MSC-LEG-FAL Joint Working Group on Maritime Autonomous Surface Ships (MASS) on its second session. Murray, B. et al. [2022]. Approvable AI for autonomous ships: challenges and possible solutions. Proceedings, 32nd European Safety and Reliability Conference. Rødseth, Ø.J. and Vagia, M. [2020]. A taxonomy for autonomy in industrial autonomous mobile robots including autonomous merchant ships. https://iopscience.iop.org/article/10.1088/ 1757-899X/929/1/012003. Rødseth, Ø.J. and Wennersberg, L.A.L. [2023]. A criticism of proposed levels of autonomy for MASS. Proceedings, 33rd European Safety and Reliability Conference.
52 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Ørnulf Jan Rødseth has an M.Sc. in cybernetics and electronic engineering from the Norwegian Institute of Technology (now NTNU) in 1983. He is a wellknown researcher in maritime information and communication technology, and has worked in the area for more than 30 years. He is director for Maritime ITS in ITS Norway and is the general manager of the Norwegian Forum for Autonomous Ships. He is a member of ISO TC8 and IEC TC80 and regularly meets at the International Maritime Organization as an observer for ISO.
ANGIE BISHOP
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 53
Ongoing digitalization and automation developments in the maritime domain are increasingly enabling maritime operations to be monitored, controlled, and supported in distributed and remote locations at sea and on land. As maritime operations evolve, new and emerging technologies reorganize how work systems are designed, including the work tasks and demands of the people operating these systems. This requires a revaluation and update of what knowledge and skills current and future maritime operators will require, with the potential that operators of future ships and marine structures may never actually work or have experience at sea. Literature and research regarding autonomous ships has become much more prevalent in recent years and it is interesting to observe how the topics have evolved over time. While the initial writings were mostly on the technology (for example, Convention on the International Regulations for Preventing Collisions at Sea (COLREG) compliant software or digital twin engines), recently attention is being paid to the just-as-necessary non-technical skills. This is of interest as the shift away from “nuts and bolts” technology places more emphasis on the socio-technical aspects of highly automated and autonomous systems. One of the first observations to make about autonomous shipping is that it is relatively lagging behind other modes of transportation in regard to automation, including road, rail, and air. The first autonomous vehicle was developed in 1995 and was able to navigate on its own but required a driver to accelerate and brake. In 2006 the first vehicle with “active lane keeping assist” entered the commercial market and over ten years ago the technology for fully autonomous vehicles existed. These vehicles have undergone millions of kilometres of road testing and have shown the potential for safe operation, yet they have not found largescale adoption. As Cugurullo and Acheampong point out in their 2023 paper “Fear of AI,” there exists a “… plethora of fears and concerns that our participants feel in relation to AI-driven cars” which are preventing
54 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
general adoption of autonomous vehicles. In rail transport there are various examples of driverless trains in current operation, predominately found in urban transportation systems. The first was the Port Island Line in Kobe, Japan, consisting of a six-km-long system, which began autonomous operation in 1981. Since then, driverless trains have been adopted in many cities, although it should be noted that some still have human attendants on board. The mode of transportation that has the most prevalent level of automation is air. Airplanes have used commercial autopilot for decades. The need to land airplanes in poor visibility encouraged the development of auto-landing systems. Currently pilots are only necessary to taxi and take off and nearly all other parts of the journey can be done autonomously. However, this autonomous ability does not mean that aircraft operators are attempting to reduce the pilot complement. While highly automated and autonomous technologies exist, being deployed across differing modes of transportation, the adoption of autonomous systems has been generally slow to materialize. Consequently, this will have repercussions for the shipping industry with the onboard mariner remaining integral to ship operation into the future. However, this does not mean that ships will continue to be crewed in the current manner and it is expected that crew sizes will reduce. The trend of reducing the number of seafarers on board ships, even while ship sizes and gross tonnage have increased, has been occurring since the 1960s. Technological advancements in shipbuilding and ship technologies have created an inverse relationship where gross tonnage and ship sizes have increased, while the number of seafarers required on board to successfully operate these ships has decreased. The more recent vision and work towards remote and autonomous surface ship operations can be viewed as another step in the natural progression of the decades-long trend of decrewing ships and offshore structures, mainly in an effort to increase operational productivity and economic competitiveness.
Figure 1: A training program for the autonomous age needs to be effective. It should include foundational knowledge, practical skills development, and simulation. ISTOCKPHOTO.COM/IPUWADOL
Although the overall goal and purpose of a system may remain the same (e.g., safe and efficient passage of a ship from point A to B), any new technological advancement or change introduced into an already established system can reorganize functions, tasks, and the overall organization of the people and processes to achieve the stated goal(s). In particular, from a human element perspective, automation has moved worker tasks from active, handson operations (e.g., actively navigating a ship, etc.) to increasingly more passive monitoring tasks of automated equipment and functioning (e.g., monitoring autopilot, responding to alarms, etc.). Humans are poor passive monitors of automated systems and increasingly complicated automation and operational systems can lack transparency, and thus operators do not have a full understanding of the underlying logic or decision-making of an automated system. Furthermore, as more tasks become automated, the tasks “left over” may increasingly be shifted from onboard ship personnel to shoreside personnel, a growing physical and perceptual disconnect between what is occurring at the sharp end of operations on board a structure at sea and the perception, comprehension, and projection of shoreside personnel and their decision-making may
occur. This combination creates an interesting set of challenges for both future onboard and shoreside operator skill sets and training, as well as what and how remote operating centres (ROC) are designed and operated for maritime surface ships in order to best support overall system goals and the people involved in achieving those goals. The human component of a ROC will require new and modified skills; however, it also introduces an opportunity for more specialized personnel and operations. For example, cargo operations and monitoring were the responsibility of the crew during transit; however, as operations move to shoreside it would make sense to have a specialist monitoring the cargo. This specialist would not be distracted by ship operations and could concentrate on only the cargo. While the shore-based operator may not need to be concerned as much about some aspects of ship operations, there will be other areas which will become more critical. What specifically this information is will evolve as autonomous ships enter service, but we can create a broad framework for a training program based on what has been effective (Figure 1). To begin, the foundation knowledge must be provided
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 55
Table 1: Training program for mariners. SMS=ship management system. BRM=bridge resource management. ROC=remote operating centre.
for the skills beyond what was learned in the normal course of being a mariner. The next step is that this knowledge is applied to allow practical skills to develop. The final step is to bring these skills together during simulated operations. In concrete terms, a training program might look like Table 1. To begin with, a mariner would need to know about new and different sensors and information sources. In addition, they would need to be aware of emergency procedures and legal requirements. Finally, cybersecurity will have important implications for remote ship operators. From this basic knowledge, we can start to build up their skills. The knowledge of sensors helps them operate and build situation awareness in the ROC. The ship management system, legal, and cybersecurity aspects will allow them to adapt to emergencies and, of course, simulation with ship handling will be critical. All of this can be brought together in ship simulation exercises administered in ROC training. ROCs will have to support the personnel and their work tasks, and thus is dependent upon an array of variables related to the capabilities of the system – how it is organized and its goals, including operational environment (e.g., location, traffic, weather, etc.), type of ship, type of cargo, uncrewed or reduced crewing on board. Over the past several years, differing concepts have been proposed for how remote operations and ROCs are to be implemented, with the necessity that uncrewed ships are at least as vigilant and safe as a crewed ship. However, there remains no formal guidelines or recommendations on the design or composition of ROCs for maritime surface vessels. There are several examples, spanning from conceptual plans to real-world implemented ROCs and
56 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
equipment, that have been proposed or are in early implementation ranging from single ship control and monitoring to multi-ship fleet management. However, questions remain on the design requirements to both establish and maintain ROC personnel situation awareness across differing control and monitoring paradigms, including level of automation/ autonomy in which a ship is operating or number of vessels managed per operator. Furthermore, the system configuration and human machine interface perspective, including supporting automation transparency for highly automated systems, require further consideration and development. Current ROC examples typically resemble a combination of current bridge equipment (e.g., radar, conning, ECDIS, CCTV view of ship surroundings) and an instructor station for navigation simulator training (e.g., provides an overview of the simulator exercise, ship(s), and environmental parameters of the scenario). However, this may not be reflective of all operational paradigms, particularly when it comes to scaling remote operations from single to multiple vessels. Furthermore, differing inputs and communication channels between personnel on board and shoreside, perceptual and ship-sense support for shoreside operators, or digital twinning may be necessary to support situation awareness of sharp-end operations and optimal decision-making in planning, executing, and monitoring operations. In moving forward with the further implementation of uncrewed and autonomous surface shipping, the development of both the ROCs (as a built environment and the equipment therewithin) and operator skill sets (and by proxy, education and training programs to meet the defined learning outcomes) will require co-development in concert with one
another. There will likely not be a one-size-fitsall model for autonomous shipping as a “single” operational paradigm (and by extension ROCs and their operators) but rather highly differentiated approaches for how de-crewed, uncrewed, highly automated, and autonomous surface vessels are monitored and controlled. A multidisciplinary approach is required for the successful implementation and operations of differing forms of autonomous shipping models wherever evolving technological development is driving change within these complex sociotechnical systems. u
Bainbridge, L. [1983]. Ironies of automation. Cugurullo, F. and Acheampong, R.A. [2023]. Fear of AI: an inquiry into the adoption of autonomous cars in spite of fear, and a theoretical framework for the study of artificial intelligence technology acceptance. Mallam, S.C.; Nordby, K.; van de Merwe, K.; Veitch, E.; Nazir, S.; and Veitch, B. [2022]. Empathy from afar? Towards empathy for future maritime designers and remote operators. Porathe, T.; Prison, J.; and Man, Y. [2014]. Situation awareness in remote control centres for unmanned ships. Strauch, B. [2017]. Ironies of automation: still unresolved after all these years. Tenold, S. [2019]. Bigger and bigger: shipping during the golden age, 1950-73. van de Merwe, K.; Mallam, S.; and Nazir, S. [2022]. Agent transparency, situation awareness, mental workload, and operator performance: a systematic literature review. van de Merwe, K.; Mallam, S.; Nazir, S.; and Engelhardtsen, Ø. [2024]. Supporting human supervision in autonomous collision avoidance through agent transparency.
Dr. Steven Mallam is a researcher and educator specializing in human factors focusing on human and organizational performance in the maritime domain. He is a senior researcher at the Fisheries and Marine Institute, as well as adjunctassociate professor in the Department of Maritime Operations at the University of South-Eastern Norway. John Cross has a diploma in marine navigation and holds an ON II certificate. He spent his early career working as a navigator on bulk carriers and oil tankers. He then became interested in engineering and received a B.Sc. in mechanical engineering and a M.Eng. in ocean engineering. He is a registered P.Eng. and spent time in both the consulting and regulatory industry before joining the Fisheries and Marine Institute. His current research is in two areas: the first is the development of standards and teaching methods for the training of mariners through distance education; and the second is autonomous ships and their impact on shipping in Canada and, in particular, education.
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 57
Informative Cutting Edge Provocative Challenging Thought Provoking International
thejot.net Follow us @jotnfld
An Autonomous Obstacle Avoidance Methodology for Uncrewed Surface Vehicles Facilitated by Deep-learning Based Object Detection Tianye Wang, Fangda Cui, Qi Li, Youssof Abaza, Kevin Wang, Shiwei Liu, and Wenwen Pei To What Extent is a Collision with an Autonomous Vessel Considered a Marine Collision in Light of UAE Law? Ramzi Madi River Ice Monitoring Using Unsupervised ISODATA Algorithm and Different Optical and SAR Satellite Datasets: A Case Study from the Churchill River in Labrador, Canada Meisam Amani, Sahel Mahdavi, and Shuanggen Jin
Avoiding Obstacles Researchers from Marine Thinking investigate and benchmark deep learning-based real-time object detection algorithms for collision avoidance in uncrewed surface vehicles (USVs). Tianye Wang
Dr. Fangda Cui
Who should read this paper?
This paper is essential reading for researchers in the fields of robotics and artificial intelligence (AI) who seek innovative approaches to real-time object avoidance. It is also particularly relevant to engineers and developers working on autonomous systems, as the incorporation of deep learning techniques introduces a data-driven paradigm, improving the precision and adaptability of object avoidance methodologies. AI enthusiasts will find value in exploring the cross-disciplinary insights at the intersection of deep learning and robotics, showcasing practical applications in realworld scenarios. Educators in computer science can leverage this paper to introduce students to cutting-edge technologies and methodologies in the realm of autonomous navigation. Lastly, professionals engaged in the deployment of autonomous vehicles or drones will benefit from understanding how deep learning can enhance safety and efficiency in navigating complex environments.
Why is it important?
Youssof Abaza
The nature of the work involves leveraging deep learning-based object detection to facilitate a real-time obstacle avoidance methodology. This innovation lies in departing from traditional rule-based logical control systems, introducing an artificial intelligence approach that enables systems to learn and adapt to intricate patterns in real-time environments. The application of deep learning signifies a shift toward more efficient and precise object avoidance, with the potential to significantly improve the safety and functionality of autonomous systems. Overall, the work represents a pioneering effort at the intersection of artificial intelligence and robotics, advancing the capabilities of object avoidance through state-of-theart technology.
Shiwei Liu
The development of the obstacle avoidance methodology can significantly enhance the safety and efficiency of USVs during exploration and research missions. This innovation reduces the risk of collisions with both static and moving obstacles, contributing to the longevity of USVs and minimizing potential damage. Ultimately, the improved navigation capabilities can advance ocean surveys, environmental monitoring, and infrastructure maintenance, promoting sustainable practices and minimizing the impact on marine ecosystems.
Qi Li
The technical team at Marine Thinking has started sophisticated field tests for the designed obstacle avoidance system on different types of uncrewed surface vehicles. The technology is expected to be available for commercial applications next summer.
Wenwen Pei 60 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
About the authors
Tianye Wang is an AI-software lead at Marine Thinking. He received his bachelor’s degree and master’s degree in computer science from Dalhousie University. In his role, he leads a team of experts and collaborates with research institutions such as the National Research Council and MITACS responsible for developing AI autopilot systems for USVs and machine learning-based solutions on edge devices for the marine sector. His research interests are computer vision, deep learning, edge computing, sensor fusion, autonomous navigation and mapping, swarm robotics, and machine-learning security. Dr. Fangda Cui obtained his PhD in mechanical engineering in 2016. Before moving to Canada, he was a postdoctoral research associate at the Center for Natural Resources, New Jersey Institute of Technology, US. He also received his MS degree in computer science from Dalhousie University in May 2023. He joined Marine Thinking Inc. as a research associate in March 2023. At Marine Thinking, Dr. Cui is responsible for composing proposals for government projects and conducting scientific research on USVs, leveraging USV techniques on various applications such as oil spills, survey and surveillance, and search and rescue. With a robust blend of academic acumen and practical proficiency, Qi Li is a dynamic software engineer actively contributing her expertise at Marine Thinking Inc. With over a year and four months of on-site experience as a co-op software engineer, she has been immersed in diverse projects, complemented by her concurrent role in a part-time contract position. Ms. Li’s academic trajectory includes the attainment of a bachelor’s degree in statistics at Dalhousie University, where she is currently advancing her studies in pursuit of a bachelor’s degree in computer science. Equipped with a dual background in computer science and statistics, she is well-positioned to confront challenges and make substantive contributions to the realm of software development. Youssof Abaza is a robotics software engineer at Marine Thinking working on marine perception systems. He utilizes the comprehensive knowledge gained from his B.Sc. in computer science and professional experience as a catalyst for teamwork to push the limits of what is achievable. Mr. Abaza’s commitment to the development of cutting-edge solutions is not only professional but stems from a profound personal interest in shaping the future of robotic technologies. Shiwei Liu serves as CTO for Marine Thinking and is extremely familiar with AI technologies and developing products for the marine sector. He holds a master’s degree in computer science from the University of Utah and has over nine years of expertise in machine learning, artificial intelligence, and software engineering fields. Before joining Marine Thinking in 2019, he was a senior data scientist at Vivint Smart Home and led a team of data scientists and engineers to deliver innovative world-class AI solutions to clients. He has been granted over ten US patents in his career, and many are also granted in Canada, Europe, and internationally. Wenwen Pei serves as the CEO of Marine Thinking. With a M.A.Sc. in civil engineering and an M.Eng. in environmental engineering, her academic background lays a solid foundation for her leadership role. She is recognized for her proficiency in optimization for decision-making and project management, skills that have been honed through years of hands-on experience in steering successful projects within the marine industry.
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 61
AN AUTONOMOUS OBSTACLE AVOIDANCE METHODOLOGY FOR UNCREWED SURFACE VEHICLES FACILITATED BY DEEP-LEARNING BASED OBJECT DETECTION Tianye Wang, Fangda Cui, Qi Li, Youssof Abaza, Kevin Wang, Shiwei Liu, and Wenwen Pei Marine Thinking Inc., Halifax, Nova Scotia, Canada shiwei.liu@marinethinking.com | wenwen.pei@marinethinking.com ABSTRACT Uncrewed surface vehicles (USVs) have gained significant attention in the past decade due to their potential commercial applications and a vast global market. An essential capability of USVs is autonomous obstacle avoidance in oceanic environments to prevent collisions. This study focuses on investigating and benchmarking deep learning-based real-time object detection algorithms for collision avoidance in USVs. The candidate algorithms were trained and evaluated using a maritime dataset provided by Marine Thinking Inc. Among the algorithms tested, You Only Look Once (YOLO) v5m demonstrated outstanding performance (mAP50 > 0.84 and mAP50:95 > 0.69) and high frame per second (FPS > 60) in realtime scenarios. Subsequently, the pre-trained YOLO v5m algorithm was converted into a TensorRT engine and deployed within a DeepStream pipeline on an onboard Nvidia Jetson for real-time collision avoidance tests. The results indicate that the YOLO v5m algorithm successfully detected obstacles and provided feedback to the ArduPilot autonomous suite. The ArduPilot, in turn, adjusted the control signal of the USV motor based on the Bendy Ruler obstacle avoidance algorithm, resulting in the successful avoidance of detected obstacles, and preventing collisions. This study presents an efficient, production-ready, and cost-effective autonomous collision avoidance methodology for USVs.
KEYWORDS USV; Autonomous obstacle avoidance; Deep learning; Object detection; DeepStream
62 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
1. INTRODUCTION An uncrewed surface vehicle is a boat or ship that “operates on the surface of the water without a crew” [Yan et al., 2010]. Due to the boom of electronic control and artificial intelligence techniques, USVs with sophisticated capabilities have been built up in the last decade. USVs can be used to increase the efficiency of multiple different tasks, such as underwater object detection, marine infrastructure surveillance [Johnston and Poole, 2017], and oil spill response [Wang et al., 2016], meanwhile massively reducing the cost. It has been forecast that the global USV market could reach billions of dollars by 2027 [GMI, 2022]. One of the essential capabilities of USVs is autonomous obstacle avoidance during navigation as there is no crew on the USVs to steer them away from hazards. In the earlier designs of USVs, the autonomous obstacle avoidance capability is usually overlooked [Liu et al., 2016A], probably due to the sparse traffic conditions at sea. However, published statistical data indicates that 60% of casualties at sea are caused by collisions [Naeem et al., 2012]. USVs have potential risks of colliding with obstacles such as buoys, lobster traps, and marine traffic. Hence, autonomous obstacle avoidance is an essential capability for USVs to increase their safety level [Liu et al., 2016A]. To achieve the capability of autonomous obstacle avoidance, USVs need a module that can accurately detect the potential obstacles on their navigation paths. A detection module consists of at least a sensor (i.e., camera) associated with an object detection algorithm for situational awareness. Traditional object detection algorithms make use of handcrafted
features such as edges and corners, and hence, such algorithms require a lot of manual work to design and implement and may fail to capture objects on a complex background [Mahony et al., 2019]. In the last decade, deep learning (DL) has experienced rapid growth, and many DL algorithms have been developed for realtime object detection tasks, such as Single-shot Detector (SSD) [Liu et al., 2016B], You Only Look Once (YOLO) [Redmon et al., 2015], and Retina Net [Lin et al., 2017], among many others. DL algorithms utilize convolutional neural networks (CNN) to automatically extract and learn characteristic features of objects on various scales [Lecun et al., 1998], leading to higher accuracies and efficiencies compared to traditional algorithms [Zhao et al., 2019]. Whenever obstacles are detected, the information will be sent to the central control module, and the module will adjust the control signals of propulsion of USVs accordingly to avoid collision based on the feedback information. Also, a specific mechanism/ algorithm is required to regulate how the control module will adjust the control signals. Many different methodologies have been adopted to facilitate obstacle avoidance, such as fuzzy logic, heuristic algorithms, artificial potential fields, and deep reinforcement learning (DRL) [Burmeister and Constapel, 2021]. For instance, Sutton et al. [2011] realized autonomous obstacle avoidance by fusing multiple sensors based on fuzzy control. However, their techniques are only validated using stationary obstacles in realworld experiments. Heuristic algorithms for collision avoidance are usually computationally efficient and simple to implement [He et al., 2022]. Though heuristic algorithms may not be
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 63
Figure 1: Schematic illustration of the architecture and workflow of the autonomous collision prevention system of uncrewed surface vehicles (USVs) using deep learning-based object detection.
able to handle obstacle avoidance in complex situations, they can be used along with other methods to obtain promising results [Han et al., 2020]. Recently, studies about obstacle avoidance using DRL-based methods have emerged. Wang et al. [2023] developed an autonomous obstacle method using DRL based on approximate representation. The developed method improved the large-scale learning efficiency and was successfully tested in two scenarios involving dynamic obstacles. The advantage of the DRL-based methods is that they can adapt to complex and dynamic environments by learning from experience. However, DRL-based methods are computationally expensive, and it may be challenging to transfer a DRL algorithm trained in a simulation or under specific conditions to a real-world environment due to the reality gap. Reports show that 56% of collisions at sea are caused by violations of collision regulations [Statheros et al., 2008]. Other than the mentioned examples, several protocol-based algorithms have been developed. For example, Kuwata et al. [2014] successfully developed a velocity obstacle method which can avoid both dynamic and static hazards while also conforming to government regulations.
64 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
In the present study, three deep learning-based object detection algorithms, namely, YOLO v5, SSD, and Retina Net were benchmarked using a maritime dataset including various types of vessels and buoys. Subsequently, the algorithm with the best performance (e.g., YOLO v5m) was converted into a TensorRT™ engine and deployed within a DeepStream pipeline on a Nvidia Jetson™ device for real-time detection tasks. A catamaran was converted into a USV by installing required sensors (e.g., camera and LiDAR) and devices (i.e., ArduPilot controller, Nvidia Jetson™, and Global Positioning System (GPS)) to perform real-world obstacle avoidance tests. In the tests, a Livox LiDAR was used in conjugation with DeepStream pipelines for accurate depth estimation. The information obtained by the object detection algorithm and LiDAR was then compressed into a topic message and sent to the ArduPilot controller to realize autonomous collision avoidance based on the Bendy Ruler algorithm [ArduPilot, n.d.]. The Robotic Operation System (ROS) was used as a backbone system to manage the sensors and controller and to transmit messages. A schematic illustration of the designed system is shown in Figure 1. The rest of the technical paper is organized in the following way. Section 2 introduces the collected maritime dataset by Marine Thinking Inc. and elaborates on algorithm training and benchmark results. Section 3 discusses the techniques of the USV regarding autonomous navigation and autonomous obstacle avoidance. In Section 4, the setup of field experiments is discussed, and test results are presented. The paper is concluded in Section 5.
2. DEEP LEARNING-BASED MARITIME OBJECT DETECTION 2.1 Maritime Dataset A total of 7,897 maritime images were collected so far from different open online resources, including Kaggle, Marine Traffic, and Google Images. Twelve different marine vessels and buoys were considered in the collected images. Only images taken during the daytime are collected for the present study. Most of the images contain a single vessel instance, but for small-size vessels (e.g., inflatable, sail, and kayak) and buoys, there are about 300 images with multiple instances. The resolutions of the collected images vary drastically but are all higher than 640 × 640. The data annotation was completed using the Computer Vision Annotation Tool [Sekachev, 2020] by a data labeller. Every instance in the collected images was annotated using a rectangular bounding box with a given label. After annotation, the images with associated annotation files were exported and divided into two subsets, namely, a training dataset and a validation dataset, for deep learning algorithm training. The training
and validation datasets were strictly separated to ensure the algorithm was not exposed to the validation data during the training phase. The training dataset consisted of images having approximately 70% of the total instances for each subclass, and the rest of the images were used for validation. The number of instances of each subclass in the training and validation set is shown in Table 1. Table 1 shows that there are 6,169 and 2,654 instances in the training and validation datasets, respectively. Also, it is noticed that the datasets are highly imbalanced. For instance, there are 1,215 instances of passenger vessels but only 273 instances of icebreakers in the total dataset. This is a matter of fact that some types of vessels are not commonly seen in the ocean, such as research and supply ships. Figure 2 shows several sample images from the collected dataset. It is noticed that different types of ships have distinct characteristics. For instance, cargo ships usually have loaded containers (Figure 2a), and military vessels are equipped with weapons and battle units (Figure 2e); inflatable vessels and tugboats are small but manoeuvrable (Figures 2d and 2g). For large-size ships such as cargo
Table 1: Number of instances for each subclass in the training and validation datasets. The training datasets contains approximately 70% of the total instances.
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 65
(a) Cargo (1360 x 644)
(b) Passenger (1440 x 960)
(c) Research (1199 x 799)
(d) Tugboat (2560 x 1730)
(e) Military (1200 x 800)
(f) Sail (1280 x 850)
(g) Inflatable (1280 x 720)
(h) Kayak and Buoy (1280 x 853)
Figure 2: Sample images of the collected maritime dataset. Note that the images have been resized to 1.5 × 2.4 inches for illustration, and the original image resolution is shown in parentheses.
66 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
and military, each image usually contains only a single instance (Figures 2a, 2b, 2c, and 2e). However, multiple instances may be found in an image for small-size vessels or buoys (Figures 2f and 2h). Figure 2 also shows the resolutions of the images are distinct. 2.2 Deep Learning-based Object Detection Algorithms There are two major groups of deep learning detection algorithms, namely, one-stage and two-stage algorithms. Two-stage algorithms divide the object detection tasks into two sub-steps. In the first step, a regional proposal algorithm is used to predict potential regions that may have instances. In the second step, a classifier is used to classify the instances in the proposed regions. One of the prominent two-stage algorithms is Faster R-CNN [Ren et al., 2016]. Though in many cases two-stage algorithms show higher accuracy and precision, they usually have high detection latencies (i.e., the FPS on video streams is less than 10). Thus, two-stage algorithms are not quite suitable for real-time object detection tasks. One-stage algorithms incorporate the regional proposal and classification into a unifying procedure and usually consist of three sub-components, namely, backbone, neck, and prediction head. In the present study, the three most used singlestage object detection algorithms, SSD, Retina Net, and YOLO v5, are benchmarked using the collected maritime dataset from Marine Thinking Inc. The VGG16 network was adopted as the backbone of the SSD algorithm, and the Feature Pyramid Network was used as the backbone of the Retina Net. For the YOLO v5 algorithm, the backbone network was CSPDarknet53. Different backbones can be selected to optimize the performance of the
algorithms; however, this is beyond the scope of the present study. The functionality of the neck is to integrate and merge the features extracted by the backbone. The prediction head is the final part of the network, and the outputs of the three algorithms are similar, namely, a bounding box with a predicted label and associated confidence level. For the sake of being concise, the details of the architecture of each algorithm will not be discussed here. The readers are referred to publications related to these algorithms. Note that the latest version of the YOLO algorithm is YOLO v8, which was also tested using the collected dataset. Though, for the same size of the architecture, the performance of YOLO v8 was improved by about 5% compared to YOLO v5, no obvious improvement was found in real-time video tests with YOLO v8. Also, the YOLO v8 was found about 30% slower than the YOLO v5 by examining the FPS of video streams. Therefore, the YOLO v8 algorithm was not adopted in the present study. 2.3 Algorithms Training and Benchmark The SSD and Retina Net were imported from the Torchvision library and were trained and validated using the PyTorch framework. The YOLO v5 codes were downloaded from the repository of Ultralytics. Algorithms were trained and validated on a laptop with an Nvidia RTX3060 discrete GPU and 32 GB RAM using the maritime dataset described in Section 2.1. An image batch size of 4 was used for all algorithms considering the memory of the GPU. The image resolution was resized to 640 × 640 before feeding to the algorithm for both training and validation. All algorithms were trained 100 epochs with an initial learning rate of 0.01, and the learning rate was changed
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 67
Table 2: Performance of Single-shot Detector (SSD), Reina Net, You Only Look Once (YOLO) v5s, and YOLO v5m algorithms for images in the validation dataset. The number 16 in the parathesis indicates the frame per second (FPS) was evaluated using an fp16 TensorRT™ engine of trained algorithms.
to 0.001 after 50 epochs. The rest of the hyperparameters were kept by default. In computer vision, Intersection over Union (IoU) is defined by comparing the ground truth bounding box to the predicted bounding box [Everingham et al., 2010]. An IoU of 0.5 indicates that, between the ground truth and the predicted bounding box, the ratio of the intersection area to the union area is 0.5. The mean Average Precision (mAP) is usually calculated using different thresholds of IoU to benchmark the performance of object detection algorithms [Lin et al., 2014]. For instance, mAP50 indicates a mean AP value calculated over all images using an IoU value larger than 0.5, and mAP50:95 represents a mean AP value obtained by further averaging over calculated mAP for IoU from 0.5 to 0.95 with a step size of 0.05. Both mAP50 and mAP50:95 were evaluated in the present paper for studied algorithms. The benchmark results of the three algorithms are shown in Table 2. It is noticed that the mAP of SSD algorithms, including both mAP50 and mAP50:95, are significantly lower than the other three algorithms, indicating poorer detection results. The detection accuracy is close among Retina Net and two different-sized YOLO v5 algorithms, with a mAP50 > 0.82 and a mAP50:95 > 0.64. However, the latency of the YOLO v5s and YOLO v5m algorithms is much slower than
68 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Retina Net. The FPS of Retina Net is just about 12.97, which is much less than the requirement of real-time tasks. Though the YOLO v5 algorithms were evaluated using a 16-bit floating point engine, the original 32-bit algorithm is still much more efficient. There is no clear standard regarding the minimum requirement of FPS for real-time object detection tasks. We herein expected the FPS to be equal to or higher than 60, with which no obvious latency can be observed by human eyes. For the YOLO v5s and YOLO v5m algorithms, as their latencies are sufficiently low for real-time tasks, the medium-sized algorithm with a higher accuracy was selected for autonomous obstacle avoidance tests. The YOLO v5m algorithm is further evaluated and the results are shown in Figures 3 and 4, and Table 3. Figure 3a shows that the box, object, and class losses continuously decrease during training. The object loss and class loss reached a stage with a level of 0.01 and 0.005, respectively. The box loss could continue decreasing at a slow speed. However, it was noticed that mAP50 and mAP50:95 values stopped increasing (Figure 3b), and hence, the training was terminated at 100 epochs. Figure 4 shows images with the original labels and prediction results on a validation batch. It is shown in Figure 4 that generally the algorithm predicted most instances correctly with high confidence (Figures 4a to 4c). Also, the algorithms can successfully capture multiple
Figure 3: Evolution of the validation loss during the training of YOLO v5m: (a) box, object, and class losses versus epoch, and (b) mean Average Precision (mAP) versus epoch.
Figure 4: Sample validation results for You Only Look Once (YOLO) v5m. The left column shows the images with original annotations, and the right column presents the images with predicted labels. The Journal of Ocean Technology, Vol. 18, No. 4, 2023 69
Table 3: Precision, Recall, and mAP50 and mAP50:95 of You Only Look Once (YOLO) v5m of each subclass and all images in the validation dataset.
instances in an image (Figure 4d). Though the algorithm did not capture all five instances, the accurate detection of the four cases is sufficient for object avoidance purposes. The YOLO v5m algorithm failed in the last case (Figure 4e), as a supply ship was predicted as a tugboat. However, the algorithm did detect the location of the instance correctly, which would be in favour of obstacle avoidance. Table 3 shows the performance of YOLO v5m algorithms on each subclass in the validation dataset. Among all thirteen classes, YOLO v5m did not perform well on fishing, recreational, inflatable, and kayak subclasses, with a mAP50:95 less than 0.6. However, the mAP50 value of these four classes is high, indicating that many instances are detected by the algorithm with low confidence. Image augmentation could be done in the future on these four subclasses to further improve the algorithm performance.
standard for representing machine learning algorithms, facilitates smooth interoperability across various deep learning frameworks. To accomplish this conversion, we leveraged the ONNX exporter available in the YOLO v5 repository, which allowed us to export the algorithm in the ONNX format seamlessly. Once the ONNX version of the algorithm was generated, we integrated the YOLO v5m algorithm into the DeepStream framework by converting the ONNX format of the algorithm into a TensorRT engine. We then developed a customized DeepStream Docker image based on the official DeepStream 6.1 Docker image, integrating the optimized TensorRT engine to conduct efficient inference on incoming video streams. The Docker image was deployed on an onboard Jetson device, which continuously received live video streams and sensor information during the autonomous navigation of the USV.
2.4 Conversion and Deployment of Trained Algorithms We employed the ONNX format as the medium for converting the YOLO v5m algorithm into a TensorRT engine. ONNX, being an open
3. AUTONOMOUS COLLISION AVOIDANCE SYSTEM FOR USVS
70 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
The integrated autonomous collision avoidance system for USVs includes a ROS node acting
as an intermediary between the deep learningbased object detection system (YOLO v5m), RGB camera, LiDAR, and the ArduPilot control unit (as shown in Figure 1). The ROS node first processes data from bounding box results of YOLO v5m to extract the information of detected obstacles, and then the node sends out commands to LiDAR for accurate depth estimation. Subsequently, the obtained obstacle data (i.e., class, location, and distance) is compressed as LaserScan messages defined by ROS. The LaserScan messages are then transmitted to the ArduPilot system, which serves as the USV’s autopilot controller. The communication between ROS and ArduPilot is established through a suitable communication interface, such as MAVLink, enabling seamless data transfer. The LaserScan messages are then utilized by the ArduPilot system for obstacle avoidance during autonomous missions. The ArduPilot firmware enables autonomous navigation and path replanning (for obstacle avoidance) through control signal adjustment. Control signal adjustment is the process through which the ArduPilot system regulates the actuation signals to the control surfaces or propulsion system of the USV. The core functionalities of control signal adjustment within the ArduPilot firmware included two major components, for autonomous navigation and obstacle avoidance, respectively. 3.1 Autonomous Navigation ArduPilot leverages the GPS and onboard sensors to enable autonomous navigation. First, ArduPilot reads GPS and onboard sensor information to obtain the current position, heading, and velocity of the USV with respect to the predefined mission plan (e.g., desired navigation path and cruise speed) and
calculates the errors between them, as shown in Equation (1): (1a) (1b) (1c) where P, θ, and V denote the position, heading, and velocity of the USV, respectively. Then, the calculated errors between the desired and current values (Equations 1a, 1b, and 1c) are passed to the control module. Subsequently, the ArduPilot performs control signal adjustment for navigation based on the errors, given by Equation (2): (2a) (2b) (2c) where Kp, Kθ, and KV are the proportional gains for position control signal Cp, heading control signal Cθ, and velocity control signal CV, respectively. By dynamically and continuously adjusting the control signal using Equation (2) based on Equation (1), the ArduPilot system can steer the vehicle following the predefined path with a desired cruise speed. 3.2 Bendy Ruler Object Avoidance Algorithm The obstacle avoidance is realized through the Bendy Ruler object avoidance algorithm [ArduPilot, n.d.]. The Bendy Ruler object avoidance algorithm is a simple yet effective obstacle avoidance method for autonomous vehicles. It is based on the concept of using a
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 71
“virtual bendy ruler” to ensure safe navigation around obstacles. The algorithm begins by initializing the bendy ruler path. The start point P0 is set to the current position of the USV, and the endpoint PN is set to the destination waypoint. The virtual bendy ruler can then be represented as a path connecting a series of points, Pi, where i = 0 to N. The virtual bendy ruler is flexible and can deform around obstacles while keeping its shape. Each segment, Si, between two consecutive points, Pi and Pi+1, can be represented as a line equation: (3) Where (x,y) is any point on the segment, (xi,yi) and (xi+1, yi+1) are the coordinates of the consecutive points Pi and Pi+1. When an obstacle is detected, the algorithm responds by displacing the point nearest to the obstacle. Assuming an obstacle is represented by a circle with its centre at (xobs,yobs) and radius Robs, the algorithm calculates the distance from each point, Pi, to the obstacle centre (xobs,yobs). If the distance is less than or equal to the obstacle radius, Robs, it will displace the point to create a safe path around the obstacle. The distance from Pi to the obstacle centre, di, can be given by: (4) If di < Robs, then the point Pi is displaced away from the obstacle centre by a distance of (Robs - di). Let ( ) be the new position of point ) can be Pi after displacement, and ( calculated by:
72 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
(5a) (5b) To encourage the modified path to regain its original shape and smoothness, the algorithm applies a relaxation process. The relaxation moves the new point towards the average position of its neighbours, effectively minimizing deformations: (6a) (6b) By smoothly updating the displacement points ( ), the algorithms can guide the USV to avoid obstacles without abrupt manoeuvring and significantly deviating from the predefined paths. 3.3 Depth Estimation The accurate depth estimation was performed using a Livox LiDAR to evaluate and update (xobs,yobs) in Equations (4) and (5) for the Bendy Ruler algorithm. Whenever an obstacle was detected by the YOLO v5m algorithm, a first rough estimation would be made to get an overall direction and a relative size of the obstacle. For instance, if the x-coordinate of the centre of the predicted bounding box is less than 0.3, it indicates the obstacle is located at the left-hand side of the vessel. A bounding box occupying a whole captured frame reflects that the obstacle is either huge or already close to the vessel. Recall that the YOLO algorithm always normalizes the size of the bounding box based on the resolution of the camera. Then, based on the information provided by the object detection algorithm, the ROS node
activated LiDAR to send out multiple laser beams forming a sector for accurate depth estimation. The LiDAR and RGB cameras were mounted on the same onboard location, and all faced straight ahead of the vessel. By checking the information on reflected laser beams, the accurate distance between the obstacle and the vessel was obtained. With the measured distance, the size of the obstacle could then be estimated based on the size of the bounding box from the YOLO v5m. Note that the LiDAR can work in a 3D mode for 3D situational awareness. However, under such mode, 50 million laser beams are generated each second, leading to a large amount of not useful information, which can jam the data network rapidly. Thus, this mode was not adopted in the present study. 4. USV COLLISION AVOIDANCE EXPERIMENT 4.1 Experimental Setup The USV is built by incorporating the required sensors and modules into a Hobie 16 catamaran used for racing and sailing. An ArduPilot controller with an onboard Jetson Orin composed the core control module of the USV. A compass, GPS, and an inertial measurement unit were installed to provide required navigation information, such as position, speed, and heading, to the ArduPilot controller. Other onboard sensors include an RGB camera and a Livox LiDAR, facilitating the capability of autonomous obstacle avoidance as discussed in Section 3. A remote controller (Skydroid H16) was used to plan the mission path of the USV (e.g., define the navigation waypoints on a downloaded nautical chart). The remote controller can also stop the operation of the USV
in case of emergencies such as if the USV failed to avoid the obstacle and a collision might occur. For the purposes of this study, the performance of obstacle avoidance was tested for two scenarios: a slow-moving obstacle and a static obstacle. For the experiment with a slowmoving obstacle, an inflatable boat raft with a gas outboard motor was used during a calm sunny day. The inflatable boat was slowly moving across the planned path of the USV. If the USV did not perform collision avoidance, it would hit the inflatable ship and cause damage. For the experiment with a static obstacle, the edge of a stationary dock was used, and tests were performed on a rainy day. The dock edge is located in the planned path of the USV. Both types of tests were performed in a nearshore region at Dartmouth, NS, Canada. The wind was mild and hence no large waves were observed on the water surface. Based on the standard of the World Meteorological Organization, the test environments have a Sea State 1 (e.g., wave height < 0.1 m). 4.2 Experimental Results Figure 5 shows the moving trajectory of the USV on the monitor of the remote controller. It is shown that there is an obstacle located in the front of the USV on the moving path, and the vessel started to adjust its route (Figure 5a). Then, the USV manoeuvred to avoid the obstacle based on the Bendy Ruler algorithm by continuously adjusting its positions (Figure 5b). Last, the USV moved back to the predefined path and continued cruising to the targeted area (Figure 5c.). Figure 6 shows recorded images of the first test from a third camera. In the beginning,
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 73
a
b
c
Figure 5: Observation of autonomous obstacle avoidance on the control panel of the uncrewed surface vehicle (USV): (a) detect the obstacle on the navigation path, (b) replan the path to avoid a collision, and (c) steer back to the planned path. The yellow star indicates the approximate location of the obstacle.
a
b
c Figure 6: A successful autonomous obstacle avoidance test of the uncrewed surface vehicle (USV) on a slowly moving inflatable ship: (a) detect the obstacle on the navigation path, (b) replan the path to avoid a collision, and (c) steer back to the planned path.
the USV cruise followed the predefined path, and an inflatable ship was controlled to cross the planned path of the USV. If no action was taken, two ships would collide with each other (Figure 6a). The obstacle avoidance module
74 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
detected the potential risk of collision and started to manoeuvre the USV gradually to avoid the inflatable ship (Figure 6b). The USV successfully avoided the collision and continued cruising by steering back to the predefined path.
Figure 7 shows recorded images of the second test from a third camera. The planned path of the USV would have it collide with the wooden dock straight ahead (Figure 7a). Before collision occurred, the USV detected the dock (i.e., as a buoy) and started to steer following the modified path provided by Bendy Ruler algorithm (Figure 7b). After the USV avoided the obstacle, it started to cruise back to the predefined path (Figure 7c). 4.3 Discussion These two tests show that the Bendy Ruler algorithm has the capability of real-time obstacle avoidance, and the algorithm is particularly effective in situations where a vehicle needs to navigate through non-cluttered environments with static or moderately moved obstacles without a sudden change of navigation direction and speed. In the present study, all the tests were performed in environments of Sea State 1. For instance, no strong wind or large waves disturbed the manoeuvre of the USV. The proposed system will be tested in harsher situations (e.g., Sea State 2) to further prove its robustness and efficiency. There are some other advantages of the present system. First, all the components of the proposed autonomous obstacle avoidance system can be purchased from the market at a moderate cost (i.e., less than $10,000), which makes the present system industrial production-ready and cost-effective. Second, both ROS and ArduPilot are open-sourced, and hence, the present methodology has a good extensibility and can be further enhanced by combining with other methods and algorithms. The choice of obstacle avoidance algorithm depends on the specific application and
requirements of the autonomous vehicle system. Theoretically, there are more sophisticated algorithms that can handle complex scenarios with narrow passages or densely packed obstacles, including the Rapidly Exploring Random Tree [LaValle, 1998] and Probabilistic Roadmap [Geraerts and Overmars, 2022] algorithms. However, such algorithms are theoretically powerful (i.e., studied in many simulations) but not widely adopted in industrial-level USVs due to their complexity and difficulties of implementation (i.e., incorporating them into the firmware of the control system). An object detection algorithm was used to detect obstacles in the present study. There are two major concerns for using DL-based object detection. On the one hand, specific protocols/ commands could be made in the future to enhance the present system based on the detected types of vessels. For example, small vessels are usually speedier and may require USVs to respond rapidly or even temporarily stop moving forward. On the other hand, object detection results (e.g., bounding boxes with labels) could be fused with information from the Automatic Identification System (AIS) to facilitate a visualization environment for end users. This is a consideration from the perspective of industrial production instead of academic research. Undefined objects, such as rocks and floating ice, may be encountered in real-world scenarios. An image segmentation algorithm can be used in conjunction with the object detection algorithm to improve the performance of the system regarding this perspective. Currently, Marine Thinking is training and validating an image segmentation algorithm, which will be tested in future work.
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 75
a
b
c Figure 7: A successful autonomous obstacle avoidance test of the uncrewed surface vehicle (USV) on a static obstacle: (a) detect the obstacle on the navigation path, (b) replan the path to avoid a collision, and (c) steer back to the planned path.
76 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
5. CONCLUSION In the present study, three different deep learning-based object detection algorithms were benchmarked using a maritime dataset collected by Marine Thinking Inc. Among the tested algorithms, the YOLO v5m algorithm yielded the best performance and was then deployed into an onboard computing system for obstacle avoidance tests. A catamaran was converted into a USV by installing required onboard devices and sensors, such as GPS, Nvidia Jetson Orin, RGB camera, and LiDAR. The functionality of autonomous obstacle avoidance for the USV is realized using Bendy Ruler algorithms along with perception information provided by the onboard sensors and object detection algorithm. Two real-world tests were performed in a nearshore region of Dartmouth, NS, Canada, under a Sea State 1 condition. The USV successfully avoided collision with obstacles in both tests, and the results prove the capability of the present system to avoid collision with both static and moderately moving obstacles (i.e., < 2 m/s). The present architecture is cost-effective and industrial production-ready, as all the components of the designed architecture can be purchased from the market with moderate costs (i.e., total cost less than $10,000). In the future, the information provided by onboard sensors and the object detection algorithm will be fused with the information from the AIS, which can be used to facilitate a visualization environment for end users. The AIS information (i.e., location, speed, and type) can also serve as a redundancy for the existing system. In addition, a maritime image segmentation algorithm will be incorporated into the present system. Image segmentation
will be helpful for USVs to detect undefined objects (i.e., rocks and floating ice) and be useful in places where objects stay or move closely (i.e., in crowded waterways). ACKNOWLEDGMENT We sincerely acknowledge the funding support from the National Research Council (NRC) for our Industrial Research Assistance Program uncrewed surface vehicle development project. This financial contribution was instrumental in the successful completion of the project and paper. Additionally, we express our gratitude to Tim Jackson and Dean Pelley, our Industrial Technology Advisors from NRC, for their invaluable industry knowledge and advice, which significantly contributed to the progress and success of our USV development endeavour. Their guidance and expertise were instrumental in shaping this project and paper. REFERENCES Ardupilot [n.d.]. Object avoidance with Bendy Ruler. Retrieved from: https://ardupilot.org copter/docs/common-oa-bendyruler.html. Burmeister, H.-C. and Constapel, M. [2021]. Autonomous collision avoidance at sea: a survey. Frontiers in Robotics and AI, Vol. 8. Everingham, M.; Van Gool, L.; Williams, C.K.I.; Winn, J.; and Zisserman, A. [2010]. The PASCAL Visual Object Classes (VOC) Challenge. No. 88, pp. 303-338. Geraerts, R. and Overmars, M. [2002]. A comparative study of probabilistic roadmap planners. Proceedings, Workshop on the Algorithmic Foundations of Robotics (WAFR’02). GMI [2022]. Unmanned surface vehicle
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 77
market. Retrieved from: https://www. gminsights.com/industry-analysis/unmannedsurface-vehicle-market. Han, J.; Cho, Y.; Kim, J.; Kim, J.; Son, N.-s.; and Kim, S.Y. [2020]. Autonomous collision detection and avoidance for ARAGON USV: development and field tests. Journal of Field Robotics. He, Z.; Liu, C.; Chu, X; Negenborn, R.R.; and Wu, Q. [2022]. Dynamic anti-collision A-star algorithm for multi-ship encounter situations. Applied Ocean Research, Vol. 118. Johnston, P. and Poole, M. [2017]. Marine surveillance capabilities of the AutoNaut wave-propelled unmanned surface vessel (USV). OCEANS 2017 - Aberdeen. Kuwata, Y.; Wolf, M.T.; Zarzhitsky, D.; and Huntsberger, T.L. [2014]. Safe maritime autonomous navigation with COLREGs, using velocity obstacles. IEEE Transaction on Journal of Oeanic Engineering, Vol. 39, No. 1, pp. 110-119. LaValle, S.M. [1998]. Rapidly-exploring random trees: a new tool for path planning. Iowa State University. Lecun, Y.; Bottou, L.; Bengio, Y.; and Haffner, P. [1998]. Gradient-based learning applied to document recognition. Proceedings, IEEE 1998. Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Dollár, P.; and Zitnick, C.L. [2014]. Microsoft COCO: Common Objects in Context. In: ECCV 2014. Lin, T.-Y.; Goyal, P.; Girshick, R.; He, K.; and Dollár, P. [2017]. Focal loss for dense object detection. arXiv. Liu, Z.; Zhang, Y.; Yu, X.; and Yuan, C. [2016A]. Unmanned surface vehicles: an overview of developments and challenges. Annual Reviews in Control, Vol. 41, pp. 71-93.
78 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.-Y.; and Berg, A.C. [2016B]. SSD: single shot multiBox detector. arXiv. Mahony, N.O.; Campbell, S.; Carvalho, A.; Harapanahalli, S.; Hernandez, G.V.; Krpalkova, L.; Riordan, D.; and Walsh, J. [2019]. Deep learning vs. traditional computer vision. arXiv. Naeem, W.; Irwin, G.W.; and Yang, A. [2012]. COLREGs-based collision avoidance strategies for unmanned surface vehicles. Mechatronics, Vol. 22, No. 6, pp. 669-678. Redmon, J.; Divvala, S.; Girshick, R.; and Farhadi, A. [2015]. You only look once: unified, real-time object detection. arXiv. Ren, S.; He, K.; Girshick, R.; and Sun, J. [2016]. Faster R-CNN: towards real-time object detection with region proposal networks. arXiv. Sekachev, B. [2020]. opencv/cvat: v1.1.0, Zenodo. Statheros, T.; Howells, G.; and Maier, K. [2008]. Autonomous ship collision avoidance navigation concepts, technologies and techniques. Journal of Navigation, Vol. 61, No. 1, pp. 129-142. Sutton, R.; Sharma, S.; and Xao, T. [2011]. Adaptive navigation systems for an unmanned surface vehicle. Journal of Marine Engineering and Technology, Vol. 10, No. 3. Wang, J.; Ren, F.; Li, Z.; Liu, Z.; Zheng, X.; and Yang, Y. [2016]. Unmanned surface vessel for monitoring and recovering of spilled oil on water. OCEANS 2016 Shanghai. Wang, C.; Zhang, X.; Yang, Z.; Bashir, M.; and Lee, K. [2023]. Collision avoidance for autonomous ship using deep reinforcement learning and prior
knowledge-based approximate representation. Frontiers in Marine Science, Vol. 9. Yan, R.-j.; Pang, S.; Sun, H.-b.; and Pang, Y.-j. [2010]. Development and missions of unmanned surface vehicle. Journal of Marine Science and Application, Vol. 9, No. 4, pp. 451-457. Zhao, Z.-Q.; Zheng, P.; Xu, S.-t.; and Wu, X. [2019]. Object detection with deep learning: a review. arXiv. SUPPLEMENTARY MATERIALS
Supplementary_Obstacle_Avoidance_ Case I.mp4 Supplementary_Obstacle_Avoidance_ Case II.mp4
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 79
SADIE LANGE AND HANNAH BUTTON ENGINEERING CADETS ATLANTIC TOWING LIMITED ST. JOHN'S, NL, CANADA Marine Institute (MI) students Sadie Lange (left in photo) and Hannah Button are enrolled in MI’s Marine Engineering Program. As students, they learn about the mechanical operations of vessels – how to maintain and control ship systems and operate machinery. Currently, they are completing their sea time (work term) requirement as engine room cadets on board the Atlantic Shrike. In this role, they shadow and work with engineers to complete day-to-day maintenance. This involves checking parameters on systems – including battery systems and hybrid propulsion systems – to ensure they are in working order.
80 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
The Atlantic Shrike is an offshore supply vessel that uses a diesel-electric propulsion system as well as a zero emissions battery system. This energy storage system enables decreased fuel consumption and carbon emissions and reduces the vessel’s maintenance requirements without compromising operational performance. Using the battery during standby mode and operations at offshore rigs increases fuel efficiency. In port, the vessel runs the batteries on cyclic mode, meaning the generator will charge the battery system up to a preset percentage. Then the generator shuts down and the battery powers all of the ship’s systems while discharging to a preset percentage. Another application involves peak shavings (or load shedding) when the vessel is at sea. Peak shavings are essentially smoothing out the load demand spikes by using the battery
to feed back into the system while the engines are running. This reduces the overall cost of load demand as the engines can run at a more consistent fuel consumption rate. In addition to reduced fuel consumption and carbon emissions, the hybrid propulsion system makes less noise when using the battery system due to generators not running and provides for better monitoring on the power management system. As part of the engineering crew, Ms. Lange and Ms. Button help perform preventative maintenance and repairs to this battery system. The system is monitored remotely by Vard Energy connecting to the ship’s battery system and trending the battery’s performance and characteristics in different modes. Ms. Lange and Ms. Button chose this career path because of their mechanical backgrounds and interests. Working on board the Atlantic Shrike as cadets allows them to learn more about electrical and economically advantageous systems. They relish the opportunity to work with seasoned engineers and to be involved with new technologies. This sea time offers a great hands-on learning experience that will help them in their future careers. www.atlantictowing.com www.mi.mun.ca slange@wave.mi.mun.ca hbutton@wave.mi.mun.ca
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 81
Defining a Collision Dr. Ramzi Madi explores the legal framework governing autonomous vessels in the UAE. Who should read this paper?
This paper is of interest to those working with autonomous vessels, particularly in the UAE. This includes developers of autonomous technology, users and owners of autonomous vessels, law professionals, and others.
Dr. Ramzi Madi
Why is it important?
There is an increasing interest in the concept of autonomous vessels around the globe. Such vessels will affect the transportation and logistics industry, enhance maritime safety and security, and contribute to environmental sustainability through increased efficiency, improved safety measures, and reduced emissions. In this paper, the research delves into the provisions of marine collisions in the UAE, providing clear definitions and outlining relevant conditions. It identifies and challenges the current legal framework and sheds light on potential areas for improvement.
About the authors
Professor Ramzi Madi earned his Doctorate in Law from the University of Aberdeen, with a specialization in commercial law and intellectual property. His research interests encompass maritime law, intellectual property, commercial law, and biotechnology law. Currently, Dr. Madi serves as a professor and deputy dean of the College of Law at Al Ain University.
82 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
TO WHAT EXTENT IS A COLLISION WITH AN AUTONOMOUS VESSEL CONSIDERED A MARINE COLLISION IN LIGHT OF UAE LAW? Ramzi Madi College of Law, Al Ain University, Abu Dhabi, UAE; ramzi.madi@aau.ac.ae ABSTRACT This paper comprehensively explores autonomous vessels, employing a methodology grounded in legal analysis to achieve its research objectives. The primary purpose of this research is to enhance the understanding of the legal framework governing autonomous vessels, specifically within the context of UAE Maritime Commercial Law No. (26) [1981]. The study places specific emphasis on international conventions, with notable attention given to the Convention for the Unification of Certain Rules of Law with respect to Collisions between Vessels [1910]. In addition to elucidating the legal status, the research delves into the provisions of marine collisions, providing clear definitions and outlining relevant conditions. The exploration extends to various categories of marine collisions, encompassing those attributed to the fault of a single vessel, shared faults, collisions caused by force majeure, and uncertainties arising from doubt or unknown causes. The significance of this research lies in identifying gaps and challenges within the current legal framework and shedding light on potential areas of improvement. The findings underscore a compelling need for legislative adjustments within the existing UAE Maritime Commercial Law No. (26) of 1981. Alternatively, the enactment of a new law, similar to Law No. (9) [2023], is proposed to govern the operation of autonomous vessels in the Emirate of Dubai.
KEYWORDS Collision; Autonomous vessels; Maritime commercial law; UAE
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 83
INTRODUCTION The United Arab Emirates, particularly Dubai, has shown significant interest in regulating autonomous vehicles through the issuance of Law No. (9) of 2023, Regulating the Operation of Autonomous Vehicles in the Emirate of Dubai. This legislation sets a strong foundation for the integration of autonomous vehicles in Dubai’s transportation, aligning with Sheikh Mohammed bin Rashid Al Maktoum’s goal of achieving 25% smarter and driverless transportation trips by 2030 [RTA, 2023]. The rapid development of transportation technology and driving systems has not overlooked maritime transportation. There has been an increasing interest in the concept of autonomous vessels. Autonomous vessels will significantly affect the transportation and logistics industry, enhance maritime safety and security, and contribute to environmental sustainability through increased efficiency, improved safety measures, and reduced emissions [Goerlandt and Pulsifer, 2022; Issa et al., 2022]. The UAE, with its 12 commercial trading ports (excluding oil ports) and its status as a major player in ship fuel provisioning, is a key participant in maritime activities [UAE Government portal, n.d.]. It is also home to the Abu Dhabi Ports Group, responsible for pioneering the Maqta Gateway, the UAE’s first digital port community utilizing blockchain technology [Alwatan, 2022]. Additionally, it serves as the central hub for DP World, a leading force in intelligent trade. This entity has introduced innovative platforms like the Dubai Trade Portal for streamlined
84 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
transnational commerce and the SeaRates platform for immediate container shipping rate quotations. Recently, the group launched the CARGOES.com platform, providing comprehensive solutions for financing and shipping goods globally [Alwatan, 2022]. Prior research conducted in the field of uncrewed ships in the UAE has played a substantial role in deepening the understanding of different aspects of this field. Notable studies include Alharmoodi and Abdul-Hay’s [2023] research titled “The impact of technology in shipbuilding on the conditions for maritime collision in the UAE maritime law and international agreements” published in the Journal of Law Sciences. This study investigates how modern shipbuilding technology affects collision scenarios involving floating facilities and unique maritime collision cases. It assesses UAE legislation and international agreements in light of recent shipbuilding advancements. With advancements in shipbuilding leading to an increased quantity of ships being built, speeds, and advanced systems, the risk of maritime collisions has increased, accompanied by legal complexities. According to the study, 80% of collisions result from inadequate training in using modern ship technologies. The study concludes that current laws, both in the UAE and globally, do not adequately address the impact of modern technology on collision scenarios, necessitating the establishment of new legal regulations for both traditional and autonomous vessels. Another significant study is Alkasem’s [2020] work titled “The legal system of unmanned
ships: An analytical study of Emirati law,” a master’s thesis at the United Arab Emirates University. This study addresses the legal complexities surrounding uncrewed ships, including their status and liability for actions. It highlights unique features like autonomy and decision-making abilities, advocating for their consideration in legal frameworks. The study investigates the existing legal structure in the UAE concerning uncrewed ships, with a specific focus on highlighting issues related to liability. It stresses the urgency of adapting existing laws to suit this technology until comprehensive regulations are developed. This paper employs a methodology grounded in legal analysis to achieve its research objectives. It thoroughly investigates the definition of autonomous vessels and assesses the extent to which they qualify as vessels under international conventions and commercial law. Furthermore, it delves into provisions regarding marine collisions to address the central inquiry of whether a collision involving an autonomous vessel is classified as a marine collision in accordance with legal principles. The study extensively examines legal sources, with a primary focus on the UAE Maritime Commercial Law No. (26) of 1981, and supplements this with detailed scrutiny of various international conventions, notably the Brussels Collision Convention (1910). Moreover, the research incorporates legal clauses into its analysis, fortifying the presented findings and arguments. The conclusion of the study draws upon this exhaustive analysis, underscoring the imperative need for legal updates to establish explicit provisions pertaining to autonomous vessels.
THE DEFINITION OF AUTONOMOUS VESSELS AND THE DEGREES OF AUTONOMY The terminology regarding uncrewed ships lacks uniformity. There exist numerous guides that delineate levels of autonomy, but they lack standardization and hold more of an informal status in legal terms. It is important to recognize that the terms “unmanned,” “uncrewed,” “autonomous,” and “automated” are distinct, though they can overlap in meaning [Achnioti, 2021]. Tøndel [2017] defined an “uncrewed vessel” as “a vessel which is not operated by an onboard master and crew, and covers all vessels from those remotely operated to the fully autonomous.” It is also defined “as a vessel that is able to navigate from point A to point B, without requiring the support from a crew aboard the ship” [Deketelaere and Maes, 2017]. “Autonomy” is defined by the Maritime Autonomous Surface Ships (MASS) UK Industry Conduct Principles and Code of Practice (v6) [2022]. “In the context of ships, autonomy (e.g. as in “Autonomous Ship”) means that the ship can operate without human intervention, related to one or more ship functions, for the full or limited periods of the ship operations or voyage.” The term “autonomous vessels” consists of two words: “autonomous” and “vessels.” The word “vessels” will be discussed later in light of the international conventions and the UAE Maritime Commercial Law No. (26) of 1981.
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 85
A significant discourse surrounds the concept of “autonomy.” It is widely acknowledged as a fundamental element in legal, technical, and ethical discussions in this era of advancing technology and its various applications. Within the realm of transportation, particularly in maritime contexts, “autonomy” stands as a prominent subject of deliberation for international bodies, industry players, manufacturers, scholars, and end users [Johansson et al., 2023]. The term “autonomous” derives its roots from the Greek words “autos,” meaning “self,” and “nomos,” meaning “rule/regulation.” Existing scholarly literature alludes to the fact that autonomous vessels, uncrewed ships, and uncrewed vessels are overarching terms denoting watercraft guided by self-regulating algorithm. In common parlance, any entity possessing self-regulation capabilities can autonomously interact with its surroundings, make decisions, and navigate through various environments. The question of whether “human-in-the-loop” coexists with “autonomy” (the capacity for self-governance) remains a topic of ongoing discussion. This ideally occurs with reference to the four degrees of autonomy outlined by the International Maritime Organization’s (IMO) MASS working group [Johansson et al., 2023]. The degrees of autonomy [IMO, n.d.] delineated for the scoping exercise were:
•
Degree one: A vessel equipped with automated processes and decision support. Seafarers are present on board to oversee and manage shipboard systems and operations. While some tasks may be automated and occasionally unsupervised,
86 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
•
•
•
seafarers are on hand to assume control. Degree two: A ship controlled remotely with seafarers present on board. The vessel’s operations are directed from a separate location. Seafarers are on site to assume control and manage shipboard systems and operations. Degree three: A remotely controlled ship with no seafarers on board. Operations and control of the vessel are managed from a separate location. Degree four: A fully autonomous ship. The ship’s operating system is capable of independently making decisions and executing actions without external intervention.
Degrees one and two are not viewed as controversial as they both include seafarers on board, unlike degrees three and four. Furthermore, the author favours the term “autonomous” for its inclusivity in encompassing various levels of autonomy. TO WHAT EXTENT ARE AUTONOMOUS VESSELS CONSIDERED VESSELS IN ACCORDANCE WITH INTERNATIONAL CONVENTIONS AND UAE MARITIME COMMERCIAL LAW NO. (26) OF 1981? International law lacks a standardized definition for the term “ship.” Each convention establishes its own criteria for applicability tailored to its specific objectives. Moreover, no international convention explicitly covers autonomous vessels [Ehlers and Paschke, 2018]. The renowned United Nations Convention on the Law of the Sea [1982], while making
reference to ships or vessels in several articles, notably refrains from providing a specific definition for them. Rule three of the Convention on the International Regulations for Preventing Collisions at Sea (COLREGs) [1972] defines “vessel” as “(a) …. every description of watercraft, including non-displacement craft and seaplanes, used or capable of being used as a means of transportation on water.” However, other international conventions define “ship.” Article 1 (d) of the International Convention for the Unification of Certain Rules of Law relating to Bills of Lading [1924] defines “ship” as “any vessel used for the carriage of goods by sea.” Moreover, the Athens Convention relating to the Carriage of Passengers and their Luggage by Sea [1974] provides in Article 1(3) a definition for a “ship,” specifying that it “means only a seagoing vessel, excluding an air-cushion vehicle.” Moreover, the International Convention for the Prevention of Pollution from Ships (MARPOL) [1973] identifies a “ship” in Article 2 (4) as “a vessel of any type whatsoever operating in the marine environment and includes hydrofoil boats, air-cushion vehicles, submersibles, floating craft and fixed or floating platforms.” Furthermore, Article 1 (1) of the International Convention on Civil Liability for Oil Pollution Damage [1969] defines “ships” as “any sea going vessel and any seaborn craft of any type whatsoever, actually carrying oil in bulk as cargo.” Observing the preceding definitions, it can be noted that they apply to autonomous vessels.
Article (11) of the UAE Maritime Commercial Law No. (26) of 1981 defines “a vessel” as: “1 – … any structure normally operating, or set for operating, in the field of maritime navigation, regardless of its power, tonnage, or the purpose of navigation thereof. 2 – In implementing the provisions, hovercrafts used for commercial or noncommercial purposes shall be deemed vessels. 3 – All appurtenances of the vessel necessary for operation shall be deemed parts of such vessel and the same nature.” The definition explicitly states that for a vessel to be classified as a vessel, it must meet the following two criteria: Firstly, it must be suitable for maritime navigation, i.e., “seaworthiness” [Boczek, 1969], meaning it must possess the necessary specifications and equipment to engage in its designated maritime activities. A vessel cannot be designated as such if it is not deemed fit for maritime navigation, as it must have the capability to confront maritime hazards. There is an exception for vessels in the construction phase, which can still be referred to as a ship [Radwan, 2018]. This is permissible under the law, which grants specific genuine rights, including the vessel’s mortgage during construction, as per Article (101) of the UAE Maritime Commercial Law No. (26) of 1981. The term “seaworthiness” not only encompasses the physical condition of a ship [Delgado and Pablo, 2018] but also includes other factors such as its cargo, route, and purpose, in addition to the crew on board. It is arguable to such an extent that numerous
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 87
authors may contend that an autonomous vessel is essentially an ill-equipped vessel (degree three and degree four). This could potentially imply its lack of seaworthiness, leading to arguable contractual violations early in the agreement [Delgado and Pablo, 2018]. Given that the human factor stands out as a key distinction between a conventional vessel and some autonomous vessels, it appears imperative to emphasize the carrier’s responsibility regarding the vessel’s equipment. This responsibility extends beyond merely ensuring crew complement; it also entails ensuring the crew’s professional competence instead of the quantity on board [Delgado and Pablo, 2018]. This responsibility of the carrier is outlined in Article (3) of the International Convention for the Unification of Certain Rules of Law relating to Bills of Lading [1924]. It states: “1. The carrier shall be bound before and at the beginning of the voyage to exercise due diligence to: a) Make the ship seaworthy. b) Properly man, equip and supply the ship. c) Make the holds, refrigerating and cool chambers, and all other parts of the ship in which goods are carried, fit and safe for their reception, carriage and preservation.” Acknowledging the ongoing debate surrounding the human element as a pivotal differentiation between a traditional vessel and an autonomous one (degree three and degree four), it is important to emphasize that if the shore-based controller (degree three) possesses the necessary competence and training to proficiently and safely operate
88 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
both the vessel and its equipment (holding valid certificates as per the Flag State’s legal requirements), this circumstance will not amount to a violation of the seaworthiness obligation, even in the absence of human presence on board [Delgado and Pablo, 2018]. The statement provided, though reasonable in its logic, contradicts a significant legal provision in the UAE Maritime Commercial Law No. (26) of 1981, one that explicitly states in Article (164): “No vessel registered in the State may set sail unless it has onboarded the required officers, marine engineers, and licensed crew. A ministerial decision shall be issued specifying the number of officers, marine engineers, and licensed crew who must be onboard the vessel, the qualifications they must possess, and health provisions onboard the vessel, provided that this decision does not in conflict with international navigational standards.” It mandates that no vessel can embark on a voyage unless it has the necessary officers, marine engineers, and licensed crew members on board. This Article aims to ensure that vessels meet certain staffing and safety criteria before being allowed to set sail. Accordingly, if we consider that an autonomous vessel is a vessel according to the definition mentioned in Article (11) of the UAE Maritime Commercial Law No. (26) of 1981, then this matter requires the presence of officers, marine engineers, and licensed crew on board, which is not available in degrees three and four. Therefore, the solution of the shore-based controller that would not constitute a breach of the obligation to maintain seaworthiness,
even if there is no human presence on board, will not be applicable in light of Article (164) of the UAE Maritime Commercial Law No. (26) of 1981. Secondly, according to Article (11) of UAE Maritime Commercial Law No. (26) of 1981, whether a facility is designated as a vessel for maritime navigation depends on its operating location in navigation. It must be situated on the sea and intended for maritime navigation. Therefore, vessels intended for river or inland navigation do not fall under this classification [Radwan, 2018]. At first glance, it is evident that autonomous watercraft meet these two criteria and therefore are vessels. They are a “structure normally operating, or set for operating, in the field of maritime navigation” and are often situated on the sea. However, as having a human crew is obligatory according to Article (164) of the UAE Maritime Commercial Law No. (26) of 1981, it becomes necessary to revise Article (164) to cover the four degrees of autonomy. Questions may arise regarding how the Ministry of Energy and Infrastructure in the UAE has issued full navigation licensing for Fugro’s uncrewed surface vessel, the Fugro Pegasus. As stated by the company, this 12-metre Blue Essence over-the-horizon vessel is reportedly the pioneer in its category to achieve full registration in the UAE [Offshore, 2023]. According to Hannes Swiegers, the director of remote operations, Middle East and India: “There were no existing regulations for uncrewed vessels, so we formed a working group with UAE authorities and local partners to facilitate the process” [Offshore, 2023].
PROVISIONS OF MARINE COLLISION The Definition of Marine Collision The Brussels Collision Convention (1910) defines “collision” in Article (1): “where a collision occurs between sea-going vessels or between sea-going vessel and vessels of inland navigation, the compensation due for damages caused to the vessels, or to any things or persons on board thereof, shall be settled following provisions, in whatever waters the collision takes place.” Moreover, Article (11), “this Convention does not apply to ships of war or Government ships appropriated exclusively to a public service.” In UAE, Articles (318) to (326) of the UAE Maritime Commercial Law No. (26) of 1981 address provisions related to collisions at sea. However, these provisions do not apply universally to all such incidents, but only to those that meet the criteria outlined in Article (318). In cases not covered by the UAE Maritime Commercial Law No. (26) of 1981, the provisions regarding torts in the UAE Civil Transaction Law No. 5 [1985] take precedence. Therefore, it is crucial to establish the specific scope in which the UAE Maritime Commercial Law No. (26) of 1981 provisions are applicable [Mohamed, 2009]. The meaning of “collision” is explained in Article (318) of the UAE Maritime Commercial Law No. (26) of 1981, which states: “1 – Should there be a collision between maritime vessels, or between such vessels and boats navigating in the domestic waters,
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 89
compensations due for damages incurred to vessels, objects and persons present on board the ship shall be settled following the provisions outlined in the present chapter, without regard to the legal system of the water where the collision occurs, except for floating devices moored at a fixed anchor. To apply the provisions of the present Article, a floating device shall be deemed a maritime vessel or a boat for domestic navigation, as the case may be. 2 – Even if a material collision does not occur, the said provisions shall apply to the compensation of the damages caused by a vessel to another or objects or persons present onboard thereof, should such damages arise from the movement or the negligence of performance of a movement by the vessel, or the failure to observe the provisions set by the national legislation or the ratified international conventions concerning the seas movement regulations. 3 – The provisions of maritime collision shall apply even if a vessel in collision is dedicated to public service by the State or an entity or institution thereof.” It should be noted that collisions between two autonomous vessels in a fully automated navigation setting will not pose any unique queries about the applicable liability framework as they operate under identical conditions [Gonzalez and Fernanda, 2019]. The Conditions of Marine Collision There are two conditions that we can infer from the definitions in Article (1) of the Brussels Collision Convention (1910) and Article (318) of the UAE Maritime Commercial Law No. (26) of 1981, and they are as follows:
90 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Collision Between Maritime Vessels or Between Such Vessels and Boats The UAE Maritime Commercial Law No. (26) of 1981 stipulates that for a marine accident to meet the classification of a collision, it necessitates the involvement of two floating facilities. Hence, if a vessel collides with a fixed object like a wharf, breakwater, rocks, coral reef, an oil extraction platform, or a person working at sea, such an incident would not qualify as a collision. A collision is also not deemed to have occurred if the vessel impacts a wreck [Alharmoodi and Abdul-Hay, 2023]. A collision can take place involving two or more vessels. This can happen if the striking vessel collides with multiple vessels simultaneously or if a collision between two vessels leads to the impacted vessel colliding with a third facility. It is immaterial whether the two vessels are owned by the same individual (sister vessels). Furthermore, the rules governing maritime collision still apply even if a vessel involved in the collision is designated for public service by the State or an associated entity or institution. Moreover, if a collision involves two boats, it falls outside the scope of maritime collision regulations, regardless of where the incident occurred, whether in marine or inland waters. In contrast, collisions between vessels or a vessel and a boat are classified as collisions, even if they occur in inland waters [Alharmoodi and Abdul-Hay, 2023]. It should be noted that there are several differences between vessels and boats, primarily in size. Vessels are commonly defined as large oceangoing entities, whereas boats are notably smaller in size [Raunek, 2020].
Article 318 (1) of the UAE Maritime Commercial Law No. (26) of 1981 states: “… For the purposes of applying the provisions of the present Article, a floating device shall be deemed a maritime vessel or a boat for domestic navigation, as the case may be.” Since an autonomous vessel cannot be categorized as a fixed object, it may be that an autonomous vessel (particularly, degrees three and four) is a floating device and shall be deemed a maritime vessel for the purposes of applying the provisions of the Article 318 about maritime collision. Collision Does Not Require a Material Contact A collision does not necessitate physical contact as mentioned clearly in Article 318 (1) of the UAE Maritime Commercial Law No. (26) of 1981 that states: “Even if a material collision does not occur …” This is also confirmed by Article (13) of the Brussels Collision Convention (1910); it states: “… either by the execution or non-execution of a maneuver or by the non-observance of the regulations, even if no collision had taken place.” Some examples where collision does not require actual physical contact include proceeding at excessive speed, causing the vessel to sink, compelling it to go out of the fairway and run aground, or negligently dragging down on the vessel to compel it to slip its anchor and chain and put to sea to avoid a collision [Mohamed, 2009]. Types of Marine Collision Collision Resulting from the Fault of a Single Vessel Article (320) of the UAE Maritime
Commercial Law No. (26) of 1981 states: “Should the collision arise for an error of a vessel; such vessel must compensate the damages resulting from the collision.” This is further affirmed by Article (3) of the Brussels Collision Convention (1910); it states: “If the collision is caused by the fault of one of the vessels, liability to make good the damages attaches to the one which has committed the fault.” The passage above implies that, if the collision results from a fault on the part of one of the vessels, liability is attributed to the vessel itself, rather than the person overseeing it. This perspective supports the notion that operating of an autonomous vessel (degrees three and four) would not change the legal framework of accountability that applies to other vessels. This framework is founded on the concept of fault, as the “vessel” is held responsible for the collision, with no mention of the master or any other crew member in this provision [Delgado and Pablo, 2018]. Collision Resulting from a Joint Fault A collision can occur due to the fault of all vessels participating in the incident – a socalled both-to-blame collision. Article (4) of the Brussels Collision Convention (1910) is similar to Article (321) of the UAE Maritime Commercial Law No. (26) of 1981; it states: “1 – Should the collision be common, the liability of each vessel shall be assessed in proportion to the error made thereby. Should the circumstances hinder the possibility to
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 91
determine the degree of error of each vessel, the liability shall be apportioned equally between them. 2 – Vessels shall be severally liable within the limits of the proportion referred to in the preceding paragraph and such for damages occurring to vessels, goods, belongings or other monies of the crew or any other person onboard the vessel. 3 – The liability shall be joint should the damage cause the death or injury of a person onboard the vessel. The vessel paying more than the share thereof shall have the right of recourse with regard to other vessels.” Hence, the overall loss incurred due to the collision is assessed, and this loss is distributed proportionally according to each vessel’s degree of responsibility. For instance, if vessel A sustains damages worth 100 and vessel B incurs damages worth 200, and both are deemed equally responsible, then each vessel bears a loss of (100 + 200) ÷ 2 = 150. This means that A, apart from covering its own loss, must also compensate B for 50 of its loss. However, in the case where B is considered two-thirds at fault, the outcome will be that A is accountable for 100, and B for 200, of the total loss. In this scenario, there is no transfer of funds between the vessels [Mohamed, 2009]. In the Case of a Collision Resulting from a Force Majeure, Uncertainty May Arise Due to Doubt or the Unknown Causes A force majeure refers to a situation beyond the parties’ control, such as an unforeseeable event that could lead to a securely anchored vessel dragging its anchor and colliding with another vessel [Mohamed, 2009].
92 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Article (287) of the UAE Civil Transaction Law No. 5 of 1985 refers to force majeure. A person is not held responsible for providing compensation if they can demonstrate that the factors beyond their control caused the harm; one of these factors is force majeure. Article (287) states: “In the absence of a provision in the law or an agreement to the contrary, a person is not liable for reparation if he proves that the prejudice resulted from a cause beyond his control such as a heavenly blight, unforeseen circumstances, force majeure, the fault of others or of the victim.” Article (2) of the Brussels Collision Convention (1910) bears a resemblance to Article (319) of the UAE Maritime Commercial Law No. (26) of 1981; it states: “1 – Should the collision arise from a force majeure, should there be a doubt as to the causes thereof, or should the causes thereof be unknown, each vessel shall bear the losses incurring thereto. 2 – The preceding provision shall also apply should the vessels or one of them be anchored at the time of the occurrence of the collision.” The UAE legislator in Article (319) of the UAE Maritime Commercial Law No. (26) of 1981 treated the three cases equally in regard to their resulting outcomes. In other words, each vessel is responsible for the losses it incurs. International Regulations for Preventing Collisions at Sea 1972 (COLREGs) It is significant to note that the COLREGs were created in the 1970s, a time when the idea of autonomous vessels did not exist. Therefore, making broad conclusions solely
because there is no rule against autonomous vessels may go against the principles of interpreting conventions [Achnioti, 2021]. In light of the preceding information, the “rules of navigation” outlined in this convention and how they apply to autonomous vessels will be briefly examined below. Rule 1 stipulates that the Regulations are applicable to “all vessels upon the high seas and in all waters connected therewith navigable by seagoing vessels.” If autonomous vessels are deemed to meet the criteria of “seagoing vessels,” they are obligated to adhere to the COLREGs [Achnioti, 2021]. Rule 2 stands out as the most “human-centric” among all COLREGs rules and arguably serves as the backbone of the entire set of regulations [Achnioti, 2021]. It states: “(a). Nothing in these Rules shall exonerate any vessel, or the owner, master or crew thereof, from the consequences of any neglect to comply with these Rules or of the neglect of any precaution which may be required by the ordinary practice of seamen, or by the special circumstances of the case. (b). In construing and complying with these Rules due regard shall be had to all dangers of navigation and collision and to any special circumstances, including the limitations of the vessels involved, which may make a departure from these Rules necessary to avoid immediate danger.” The Rule suggests that human judgment is crucial to decide when it is necessary to deviate from COLREGs. This poses a significant challenge for uncrewed technology. In autonomous vessels, a key question is how the duty of prudent seamanship can be fulfilled
without any crew on board. Additionally, how does the distinction between remote-controlled and autonomous systems impact compliance with the Rule? To fulfil the duty effectively, autonomous vessels must be able to go beyond the literal interpretation of the law in specific situations to avoid collisions. Whether this is achievable remains uncertain [Achnioti, 2021]. Rule 3(k) asserts that: “Vessels shall be deemed to be in sight of one another only when one can be observed visually from the other.” This rule carries substantial importance as it governs the application of steering and sailing rules and establishes obligations between vessels under various visibility conditions. For autonomous vessels, the critical question is whether visual observation of another vessel is achievable [Achnioti, 2021]. Rule 5 states that: “Every vessel shall at all times maintain a proper look-out by sight and hearing as well as by all available means appropriate in the prevailing circumstances and conditions so as to make a full appraisal of the situation and of the risk of collision.” The Rule explicitly demands human visual and aural observations from individuals on board a vessel, aligning with the requirement for prudent seamanship. Notably, non-compliance with Rule 5 would impact adherence to Rule 2. The Rule places a crucial emphasis on the application of human perception, posing a potential obstacle for autonomous vessels. The key inquiry revolves around whether fulfilling the duty to maintain a proper look-out can be accomplished solely through camera and aural sensing equipment or if it necessitates the presence of individuals on board [Achnioti, 2021].
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 93
Rule 6 mandates that: “Every vessel shall at all times proceed at a safe speed so that she can take proper and effective action to avoid collision and be stopped within a distance appropriate to the prevailing circumstances and conditions …” Determining a safe speed involves considering various factors, such as visibility and traffic density. This requirement, as outlined in the Rule, applies universally to all vessels, including autonomous vessels. The concept of a safe speed is relative and relies on prudent seamanship, varying based on specific circumstances and individual ships. To adhere to the imperative of maintaining a safe speed consistently, it is crucial to continuously assess changes in circumstances and the environment, making any necessary adjustments to speed [Achnioti, 2021]. Moving to Rule 8, it delineates the actions to be taken when avoiding a collision. It specifies that these actions must align with COLREGs, be positive, executed in ample time, and undertaken with due regard for the principles of good seamanship [Achnioti, 2021]. CONCLUSION Ultimately, the study’s results underscore the critical need for standardized terminology in the field of autonomous vessels. The terminology surrounding uncrewed vessels lacks uniformity, with various guides outlining levels of autonomy lacking standardization in legal terms. The IMO’s working group on MASS defines four degrees of autonomy, with degrees one and two not considered controversial, as they require seafarers on board. However, degrees
94 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
three and four, which do not mandate human presence, raise important debates about the human element. The assertion that a shorebased controller with proper competence can operate the vessel without violating seaworthiness obligations contradicts the UAE Maritime Commercial Law No. (26) of 1981, which mandates a required crew presence on board. Therefore, revising Article (11) to address all four degrees of autonomy in autonomous vessels is necessary. Moreover, collisions between vessels, irrespective of whether one or more of them are autonomous, does not affect liability. The author contends that autonomous vessels, particularly those classified as degrees three and four, should be considered maritime vessels to apply Article (318) on maritime collision as they cannot be categorized as fixed objects. This perspective emphasizes that liability for a collision lies with the vessel itself, rather than the overseeing personnel. As determined by the definitions in Article (1) of the Brussels Collision Convention (1910) and Article (318) of the UAE Maritime Commercial Law No. (26) of 1981, the conditions for a marine collision can be inferred. These conditions encompass collisions between maritime vessels or between such vessels and boats, and notably, a collision does not necessarily necessitate physical contact. Further findings have revealed that there are three distinct categories of marine collisions: those stemming from the sole fault of one vessel, collisions arising from shared fault, and collisions occurring due to force majeure,
which may introduce uncertainty due to the presence of doubt or unknown factors. Ultimately, there is a pressing need for legislative adjustments for the UAE Maritime Commercial Law No. (26) of 1981, or enacting a new law, similar to Law No. (9) of 2023, regulating the operation of autonomous vehicles in the Emirate of Dubai to govern the operation of autonomous vessels. REFERENCES Abu Al-Faraj, M.S. [2020]. Autonomous vessels: legal challenges: a comparative analytical study. Journal of Legal and Economic Studies, Vol. 6, Iss. 2. Achnioti, E. [2021]. To what extent can unmanned ships comply with COLREGs 1972 and how will the liability of such vessels be assessed? Southampton Student Law Review, Vol. 11, No. 1. Alharmoodi, M. and Abdul-Hay, I.E-D. [2023]. The impact of technology in shipbuilding on the conditions for maritime collision in the UAE maritime law and international agreements. University of Sharjah (UoS) Journal of Law Sciences, Vol. 19, No. 4. https://doi.org/10.36394/jls.v19.i4.11. Alkasem, S.F. [2020]. The legal system of unmanned ships: an analytical study of Emirati law. United Arab Emirates University. Master’s thesis. Alwatan [2022]. The future of autonomous ships in maritime shipping. Energy and infrastructure discussed in a virtual seminar on the challenges and future visions to enhance autonomous ships. Alwatan. https://alwatan.ae/archives/929646. Athens Convention relating to the Carriage of
Passengers and their Luggage by Sea [1974]. International Maritime Organization. www.imo.org/en/About/ Conventions/Pages/Athens-Conventionrelating-to-the-Carriage-of-Passengers-andtheir-Luggage-by-Sea-(PAL).aspx. Boczek, B. [1969]. The nationality of ships. By Herman Meyers. Foreword by D.H.N. Johnson. The Hague: Martinus Nijhoff, 1967. pp. xiii, 395. Index. Gld. 42. American Journal of International Law, Vol. 63, No. 1, pp. 167-170. https://doi. org/10.2307/2197210. Convention for the Unification of Certain Rules of Law with respect to Collisions between Vessels [1910]. Brussels. www.admiraltylaw guide.com/conven/collisions1910.html. Convention on the International Regulations for Preventing Collisions at Sea (COLREGs) [1972]. www.imo.org/en/About/Conventions/ Pages/COLREG.aspx. Deketelaere, P. and Maes, F. [2017]. The legal challenges of unmanned vessels. Ghent University. Master of Science in Maritime Science. https://libstore.ugent. be/fulltxt/RUG01/002/349/671/RUG01002349671_2017_0001_AC.pdf. Delgado, R. and Pablo, J. [2018]. The legal challenges of unmanned ships in the private maritime law: what laws would you change? Maritime, Port and Transport Law between Legacies of the Past and Modernization, Vol. 5, l Diritto marittimo – Quaderni, Italy. https://ssrn.com/abstract=3297487. Ehlers, P. and Paschke, M. [2018]. Maritime law – current developments and perspectives. Publication on the occasion of the 35th anniversary of the Institute for the Law of the Sea and Maritime Law (Hamburg). LIT Verlag Münster.
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 95
Goerlandt, F. and Pulsifer, K. [2022]. An exploratory investigation of public perceptions towards autonomous urban ferries. Safety Science, Vol. 145. https://doi. org/10.1016/j.ssci.2021.105496. Gonzalez, N. and Fernanda, P.M. [2019]. Legal challenges of liability in collisions arising from the development of autonomous and unmanned shipping – international and Norwegian perspective. Master thesis, University of Oslo. IMO International Maritime Organization [n.d.]. Autonomous shipping. www.imo.org/en/ MediaCentre/HotTopics/Pages/Autonomousshipping.aspx. International Convention for the Prevention of Pollution from Ships (MARPOL) [1973]. London, adoption on November 2, 1973. www.imo.org/en/about/Conventions/Pages/ International-Convention-for-the-Preventionof-Pollution-from-Ships-(MARPOL). aspx#:~:text=The%20International%20 Convention%20for%20the,2%20 November%201973%20at%20IMO. International Convention for the Unification of Certain Rules of Law relating to Bills of Lading [1924]. Brussels, August 25, 1924. www.jus.uio.no/english/services/ library/treaties/07/7-04/hague-rules.html. International Convention on Civil Liability for Oil Pollution Damage [1969]. Brussels, November 29, 1969. www.imo.org/en/ About/Conventions/Pages/InternationalConvention-on-Civil-Liability-for-OilPollution-Damage-(CLC).aspx#:~:text= International%20Convention%20on%20 Civil%20Liability%20for%20Oil%20 Pollution%20Damage%20(CLC),-Home& text=The%20Civil%20Liability%20 Convention%20was,casualties%20
96 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
involving%20oil%2Dcarrying%20ships. Issa, M.; Ilinca, A.; Ibrahim, H.; and Rizk, P. [2022]. Maritime autonomous surface ships: problems and challenges facing the regulatory process. Sustainability, Vol. 14, No. 23. https://doi.org/10.3390/su142315630. Johansson, T.; Fernández, J.E.; Dalaklis, D.; Pastra, A.; and Skinner, J. et al. [2023]. Autonomous vessels in maritime affairs: law and governance implications. Germany: Springer International Publishing. https:// doi.org/10.1007/978-3-031-24740-8. Law No. (9) [2023]. Regulating the operation of autonomous vehicles in the Emirate of Dubai. https://dlp.dubai.gov.ae/Legislation %20Reference/2023/Law%20No.%20(9)% 20of%202023%20Regulating%20the%20 Operation%20of%20Autonomous%20 Vehicles.pdf. MASS UK Industry Conduct Principles and Code of Practice (v6) [2022]. https:// massworld.news/maritimeuk-launches-v6mass-code-of-practice/. Mohamed, A.H. [2009]. Maritime collision under UAE maritime law: a comparative study. Journal Sharia and Law, Vol. 2009, No. 37, Article 9. https://scholarworks.uaeu. ac.ae/cgi/viewcontent.cgi?article=1418& context=sharia_and_law. Offshore [2023]. UAE authorizes Fugro USV for subsea inspections. www.offshore-mag. com/rigs-vessels/article/14292399/uaeauthorizes-fugro-usv-for-subsea-inspections. Radwan, F.N. [2018]. Al-Wajeez in explanation of maritime commercial law of the United Arab Emirates. Brighter Horizon Publishers. Raunek [2020]. 7 differences between a ship and a boat. Marine Insight. www.marine insight.com/types-of-ships/7-differencesbetween-a-ship-and-a-boat.
RTA Roads and Transport Authority [2023]. Self-driving transport (SDT). https://www. rta.ae/wps/portal/rta/ae/home/sdt?lang=en. UAE [n.d.]. United Arab Emirates’ Government portal. https://u.ae/en/information-andservices/infrastructure/civic-facilities/ seaports#:~:text=The%20UAE%20has% 2012%20commercial,tonnage%20of%2080 %20million%20tonnes.&text=Major%20 seaports%20in%20the%20UAE,cargo%20 port%20for%2040%20years. Tøndel, E. [2017]. Maritime law in the wake of the unmanned vessel. https://www.world servicesgroup.com/publications.asp?action =article&artid=8859. UAE Civil Transaction Law No. 5 [1985]. https://legaladviceme.com/legislation/126/ uae-federal-law-5-of-1985-on-civiltransactions-law-of-united-arab-emirates. UAE Maritime Commercial Law No. (26) [1981]. https://www.ecolex.org/details/ legislation/federal-law-no-26-of-1981-onmaritime-commercial-law-lex-faoc068181. United Nations Convention on the Law of the Sea [1982]. Montego Bay, December 10, 1982. https://www.imo.org/en/ourwork/ legal/pages/unitednationsconventiononthe lawofthesea.aspx.
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 97
River Ice Monitoring Researchers Amani, Mahdavi, and Jin use an ISODATA algorithm applied to satellite datasets to detect ice over a portion of the Churchill River. Who should read this paper?
Dr. Meisam Amani
Those interested in learning how satellite observations and advanced remote sensing techniques can be used for river ice monitoring should read this paper. It will be of interest to operators in and near rivers and those who are concerned with flooding and climate change.
Why is it important?
The study uses a variety of satellite datasets to produce river ice maps. The proposed approach shows how a combination of various satellite data can be used for frequent and effective monitoring and management of rivers. The produced river ice products can be used for monitoring coastal areas. They can also be used within meteorological and oceanographic models to map different met-ocean parameters.
Dr. Sahel Mahdavi
Dr. Shuanggen Jin
About the authors
Dr. Meisam Amani is the remote sensing team lead at WSP E&I Canada Limited, where he manages and leads various industrial, governmental, and academic projects related to remote sensing and geospatial technologies. Over the past 15 years, he has utilized various remote sensing datasets (e.g., UAV, optical, LiDAR, SAR, scatterometer, radiometer, and altimeter) along with different machine learning and big data processing algorithms to provide effective solutions for industrial challenges. A list of his research works, including over 120 peer reviewed journal and conference papers, can be found at www.researchgate.net/profile/Meisam_Amani3. Dr. Sahel Mahdavi is currently affiliated with the remote sensing team at WSP E&I Canada Limited. She has almost 14 years of academic and industrial background in remote sensing and is familiar with a wide array of topics relevant to remote sensing and GIS and their applications in various environmental aspects. She has authored more than 60 peer reviewed articles. Dr. Shuanggen Jin is currently vice-president and professor at Henan Polytechnic University, China, and professor at Shanghai Astronomical Observatory, CAS, Shanghai, China. His main research areas include satellite navigation, remote sensing, and space/planetary exploration. He has published over 500 papers in peer reviewed journals and proceedings, 20 patents/ software copyrights, and 12 books/monographs with more than 12,000 citations and H-index >57.
98 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
RIVER ICE MONITORING USING UNSUPERVISED ISODATA ALGORITHM AND DIFFERENT OPTICAL AND SAR SATELLITE DATASETS: A CASE STUDY FROM THE CHURCHILL RIVER IN LABRADOR, CANADA Meisam Amani1,2, Sahel Mahdavi2, Shuanggen Jin1,3 1 School of Surveying and Land Information Engineering, Henan Polytechnic University, Jiaozuo, China 2 WSP Environment and Infrastructure Canada Limited, Ottawa, ON, Canada; meisam.amani@ wsp.com 3 Shanghai Astronomical Observatory, Chinese Academy of Sciences, Shanghai, China ABSTRACT Monitoring river ice in Canada is important for various applications, including assessing the danger of floods. For this purpose, satellites provide a valuable solution to reduce cost and improve safety. For example, all-weather day and night acquisition capability of synthetic aperture radar (SAR) systems facilitate frequent monitoring of river ice. However, since a large volume of satellite data are required for river ice monitoring, utilizing commercial satellites is expensive. In this study, an Iterative Self-Organizing Data Analysis Technique (ISODATA) algorithm was applied to four types of satellite datasets, including two optical (i.e., Landsat-8 and Sentinel-2) and two SAR (Sentinel-1 and Radarsat Constellation Mission (RCM)) systems to detect ice over a portion of the Churchill River in Labrador, Canada. The results showed the high potential of the spaceborne solutions for river ice detection, though optical satellites could not be used to map river ice in regions experiencing cloud cover. Finally, several suggestions are provided to use the proposed method for producing up to date information about ice over all rivers in Canada using advanced machine learning and cloud computing models.
KEYWORDS River ice; Satellite; Machine learning; Canada; Remote sensing
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 99
INTRODUCTION Rivers play a pivotal role in redistributing water throughout land [Duguay et al., 2014]. Ice monitoring on rivers is important in many applications, including discharge capacity, ecosystem and microclimate, potential danger of flood, construction of dams, and navigation [Duguay et al., 2014; Łoś et al., 2019]. Most rivers cover large areas and are located in remote regions. Although in-situ measurements of ice and snow over rivers provide the most accurate data, they are expensive and laborious [Duguay et al., 2014; Smith, 1997; Zhang et al., 2021]. Moreover, frequent observation from river ice is not feasible nor cost-effective using field surveys and in-situ sensors. On the other hand, satellite observations are ideal for river ice monitoring because of their repeated and consistent coverage capabilities [Smith, 1997; Amani et al., 2022]. In recent years, considerable improvements have been achieved on river ice mapping and monitoring using satellite data. For example, ice concentration, extent, thickness, and phenology have been estimated effectively using remote sensing methods [Smith, 1997; Zhang et al., 2021; Mermoz et al., 2009; Mermoz et al., 2014; Weber et al., 2003; Murfitt and Duguay, 2021]. It should be noted that accurate estimation of ice thickness is challenging using only satellite data. However, satellite images could be employed as ancillary inputs along with sparse in-situ ice thickness data and statistical models to estimate ice thickness [Mermoz et al., 2009]. Two types of remote sensing datasets have been used widely for river ice studies: optical satellite and synthetic aperture radar (SAR) images. Optical satellites are considerably
100 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
valuable for river ice monitoring due to their relatively lower noise level, higher spatial resolution, and a shorter revisit time. Optical data can be interpreted simply by means of visual investigations to determine ice concentration and extent [Murfitt and Duguay, 2021]. Additionally, the spectral characteristics of ice can be studied efficiently using optical data. However, it should be noted that optical satellite images are affected by clouds, which is a common problem in many countries, such as Canada. Consequently, most studies concerning river ice monitoring use SAR data. Spaceborne SAR systems provide valuable datasets for river ice monitoring due to their all-weather, as well as day and night acquisition capabilities [Zhang et al., 2021; Amani et al., 2022; Stonevicius et al., 2022]. SAR signals can also penetrate through dry snow and characterize the ice below [Murfitt and Duguay, 2021; Mahdavi, 2017]. Moreover, the difference between river ice and water can be detected by SAR data as a result of SAR’s sensitivity to roughness [Smith, 1997]. It is finally worth noting that a combination of SAR intensity and texture information provides a high potential for accurate river ice monitoring [Zhang et al., 2021]. SAR configuration plays an important role in determining its ability for river ice monitoring. Therefore, wavelength and polarization, among other SAR system parameters, should be chosen wisely for the most optimal results. The SAR wavelength determines the signal penetration depth and its sensitivity to surface roughness [Łoś et al., 2016]. In this regard, C-band SAR data have been widely used in the literature for discriminating river ice from open water.
This is mainly because C-band has a high sensitivity to wet surfaces. However, C-band SAR may have difficulty in distinguishing thin ice from open water. On the other hand, L-band SAR data would be a better option if the objective is to map ice thickness and internal structure of river ice because it has a high penetration into river ice. X-band has also been used for studying small-scale features on the river ice surface, such as cracks and ridges [Duguay et al., 2014; Murfitt and Duguay, 2021; Stonevicius et al., 2022]. Regarding polarization types, the horizontal transmission and vertical reception channel has proved effective in river ice monitoring [Łoś et al., 2019; Mermoz et al., 2009]. Additionally, quadpolarimetric SAR data provided more accurate results in river ice characterization compared to single-polarized SAR data [Amani et al., 2022]. For example, Mermoz et al. [2009] compared single-polarized with quad-polarimetric SAR data, and reported full-polarimetric data were significantly better than single-polarized SAR data for this purpose. It should be noted that some single and dual-polarimetric SAR data, such as Sentinel-1, have the advantage of being open access, while quad-polarimetric data are usually not free for all users. Although SAR data have been used frequently for river ice mapping and monitoring compared to optical satellite data, each of them has several advantages and limitations. Therefore, a combination of optical satellite and SAR data could provide the most accurate results for river ice studies. Since a large volume of satellite data is required for river ice monitoring purposes, commercial satellites are not usually cost-effective solutions. Therefore, open access optical and SAR data, which
have suitable spatial and temporal resolutions, have been applied extensively to study river ice. Currently, there are various optical (e.g., Landsat-8/9 and Sentinel-2) and SAR (e.g., Sentinel-1, Radarsat-2, and Radarsat Constellation Mission (RCM)) satellites, which provide open access data and can be used for mapping and monitoring river ice. Canada is characterized by numerous rivers and harsh winters, which makes monitoring river ice imperative. Monitoring river ice and producing frequent river ice maps are important for various applications, including flood prediction, navigation safety, hydropower management, and climate change. For example, river ice is sensitive to changes in temperature and precipitation patterns, making it an important indicator of climate change. Additionally, extensive flooding can be caused by the development and break-up of ice on large rivers. As discussed, satellite observations, especially SAR data, can be used efficiently for river ice monitoring and help decision-makers to understand climate change and avoid undesired events, such as floods. Therefore, in this study, the capability of different optical and satellite datasets is investigated for mapping river ice in Canada. MATERIALS AND METHODS River ice mapping was conducted over a portion of the Churchill River (area = 82 Km2), near Happy Valley-Goose Bay in Labrador, Canada (see Figure 1). In this study, the open access imagery collected by the Landsat-8 and Sentinel-2 optical satellites, as well as those acquired by the
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 101
Figure 1: Study area: a portion of the Churchill River near Happy Valley-Goose Bay in Labrador, Canada.
RCM and Sentinel-1 SAR satellites were used for river ice mapping. The characteristics of these satellites are provided in Table 1 and Table 2. The Sentinel-1, RCM, Sentinel-2, and Landsat-8 images were acquired on December 27, 2019; February 5, 2021; October 5, 2021; and March 9, 2021, respectively. As mentioned in the Introduction section, C-band is the best option when the objective is discriminating river ice from open water and, thus, C-band SAR data were used in this study. For Landsat-8 and Sentinel-2 imagery, a pan sharpening algorithm was applied to produce images with a spatial resolution of 15 and 10 metres for all spectral bands, respectively. Finally, all spectral bands of each satellite were layer-stacked to produce one image for each date. Regarding RCM and Sentinel-1 SAR imagery, an adaptive speckle filter was applied to the images to remove noise. Then, terrain correction and georeferencing algorithms were implemented. After preprocessing the imagery, an Iterative Self-Organizing Data Analysis Technique
102 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
(ISODATA) unsupervised classification algorithm was applied to discriminate river ice from water bodies and other landcovers. The ISODATA algorithm was used in this study because in-situ data were not available, and its implementation was simple. ISODATA is designed to automatically classify imagery into groups or clusters based on their similarities. It works iteratively by assigning initial cluster centres and then adjusting them based on the input data. The steps for implementing the algorithm are described below:
1. Several values (e.g., 3 to 8) were
considered as the initial cluster number and based on the results, the optimum numbers were selected for each satellite image. For example, the initial cluster number for the Landsat-8 image was 4. The initial centres of the clusters were then randomly assigned by the algorithm. 2. Each pixel in an image was assigned to the nearest cluster centre based on the Euclidean distance. This step formed the initial clusters. 3. The cluster centres were recalculated based
Table 1: The characteristics of the optical satellites used in this study for river ice mapping.
Table 2: The characteristics of the synthetic aperture radar (SAR) systems used in this study for river ice mapping.
on the mean pixel values assigned to each cluster. The mean values represented the centroid of each cluster. 4. The similarity between clusters were investigated by comparing with the image and it was decided whether the clusters should be merged or split. 5. Steps 2 to 4 were repeated iteratively until the stopping criterion, which was the maximum number of iterations, was met. 6. Once the algorithm reached the maximum number of iterations, each pixel was assigned to its corresponding cluster based on the updated cluster centres, and the final classification was obtained. RESULTS AND DISCUSSION Figure 2 illustrates the produced ice maps over Churchill River using different optical
and SAR imagery. There was no ice on October 5, 2021, on Churchill River and, thus, the ice map was not produced for this data using Sentinel-2 imagery (see Figure 2 (g)). It should be noted that the white areas in Figure 2 (g) are sandbanks. Based on the results, approximately 50 km2 (61%), 22 km2 (27%), and 68 km2 (83%) of the study area (82 km2) were covered by ice on December 27, 2019; February 5, 2021; and March 9, 2021, respectively. Based on visual interpretation of the results, it was observed that river ice could be detected effectively by both optical and SAR data. Although both types of satellite data were efficient in detecting river ice, it should be noted that optical data cannot detect ice in the presence of clouds, which is common in most parts of Labrador. This would be the
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 103
Figure 2: The left column (a, c, e, and g) illustrates the Sentinel-1, RCM, Landsat-8, and Sentinel-2 imagery acquired on December 27, 2019; February 5, 2021; March 9, 2021; and October 5, 2021, respectively. The right column (b, d, and f) also shows the corresponding river ice maps. The blue colour indicates ice.
main limitation of optical satellite imagery for ice mapping and monitoring. It was also observed that the ISODATA unsupervised algorithm could detect river ice with an acceptable level of accuracy based on the visual interpretation, and there was no need to develop more advanced machine learning models for this purpose. Moreover, since an unsupervised algorithm was used, there was
104 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
no need for in-situ data to train the model. This, in turn, reduces cost, time, computation, and safety risks considerably, which are important for practical projects in harsh environments such as Labrador. Although this study provided a simple and effective method for river ice mapping using a variety of satellite datasets, the proposed
method has several limitations. Some of these limitations along with several solutions and suggestions are discussed below to be used in future studies. Since ground truth data were not available, the validation of the maps was only conducted based on visual interpretation of the imagery. It is suggested to collect in-situ data in future studies for better validation of the results. The main objective of this study was to illustrate the applications of different satellite data for river ice mapping. To this end, we selected the Churchill River as a case study. However, it is suggested to apply the proposed model to other water bodies with different characteristics. This will allow better understanding of the robustness of the model at different conditions. As mentioned in the datasets’ section, all imagery used in this study were open access for all users. Utilizing these free data could further decrease the cost of projects, especially if the objective is operational monitoring of river ice. In fact, it is suggested to develop a platform to automatically download and process these open access satellite data for effective monitoring of river ice in Churchill River and other water bodies in Canada. By using an automatic platform and the proposed classification algorithm, frequent and up-to-date river ice maps can be produced and can be used as inputs in different applications, such as flood risk and climate change modelling. In this study, different satellite data were used for ice detection. However, many studies require classification of different ice types. Therefore,
it is suggested to develop remote sensing and machine learning models to classify ice types. To this end, it is important to first collect enough and accurate in-situ data in order to develop robust models and accurate maps. Finally, it is suggested to apply the proposed model to all rivers in Canada and produce up to date ice maps. To this end, cloud computing platforms, such as Google Earth Engine, which contain many open access satellite data and open source machine learning models, could be employed [Amani et al., 2020]. CONCLUSION In this study, an unsupervised classification algorithm along with different types of open access satellite datasets were investigated for river ice detection. The results showed that the proposed approach had a high potential for accurate river ice mapping and monitoring. Using open access data acquired by satellites with a decent revisit time (e.g., six days) is a very cost-effective and safe approach to produce frequent river ice maps. It is suggested to develop a web platform for automatically downloading these free satellite datasets, processing them, and producing up to date river maps using machine learning and cloud computing models over any river in Canada. Subsequently, the developed models can be applied to investigate the break-up and freeze-up times in future studies. REFERENCES Amani, M. et al. [2020]. Google Earth Engine cloud computing platform for remote sensing big data applications: a comprehensive
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 105
review. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 13, pp. 5326-5350. doi: 10.1109/JSTARS.2020.3021052. Amani, M. et al. [2022]. Remote sensing systems for ocean: a review (Part 2: Active Systems). IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 15, pp. 1421-1453. doi: 10.1109/JSTARS.2022.3141980. Duguay, C.R.; Bernier, M.; Gauthier, Y.; and Kouraev, A. [2014]. Remote sensing of lake and river ice. In: Remote Sensing of the Cryosphere, Chichester, UK: John Wiley & Sons Ltd., pp. 273-306. Łoś, H. et al. [2016]. Comparison of C-Band and X-Band polarimetric SAR data for river ice classification on the Peace River. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. XLI-B7, pp. 543-548. doi: 10.5194/isprs-archives-XLI-B7-543-2016. Łoś, H.; Osińska-Skotak, K.; Pluto-Kossakowska, J.; Bernier, M.; Gauthier, Y.; and Pawłowski, B. [2019]. Performance evaluation of quad-pol data compare to dual-pol SAR data for river ice classification. European Journal of Remote Sensing, Vol. 52, No. sup1, pp. 79-95. doi: 10.1080/22797254. 2018.1540914. Mahdavi, S. [2017]. Effects of changing environmental conditions on synthetic aperture radar backscattering coefficient, scattering mechanisms, and class separability in a forest area. Journal of Applied Remote Sensing, Vol. 11, No. 03, p. 1. doi: 10.1117/1.JRS.11.036015. Mermoz, S.; Allain, S.; Bernier, M.; Pottier, E.; and Gherboudj, I. [2009]. Classification of river ice using polarimetric SAR data.
106 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Canadian Journal of Remote Sensing, Vol. 35, No. 5, pp. 460-473. doi: 10.5589/m09-034. Mermoz, S.; Allain-Bailhache, S.; Bernier, M.; Pottier, E.; Van Der Sanden, J.J.; and Chokmani, K. [2014]. Retrieval of river ice thickness from C-Band PolSAR data. IEEE Transactions on Geoscience and Remote Sensing, Vol. 52, No. 6, pp. 30523062. doi: 10.1109/TGRS.2013.2269014. Murfitt, J. and Duguay, C.R. [2021]. 50 years of lake ice research from active microwave remote sensing: progress and prospects. Remote Sensing of Environment, Vol. 264, p. 112616. doi: 10.1016/j.rse.2021.112616. Smith, L.C. [1997]. Satellite remote sensing of river inundation area, stage, and discharge: a review. Hydrological Processes, Vol. 11, No. 10, pp. 1427-1439. doi: 10.1002/ (SICI)1099-1085(199708)11:10<1427:: AID-HYP473>3.0.CO;2-S. Stonevicius, E.; Uselis, G.; and Grendaite, D. [2022]. Ice detection with Sentinel-1 SAR backscatter threshold in long sections of temperate climate rivers. Remote Sensing, Vol. 14, No. 7, p. 1627. doi: 10.3390/ rs14071627. Weber, F.; Nixon, D.; and Hurley, J. [2003]. Semi-automated classification of river ice types on the Peace River using RADARSAT-1 synthetic aperture radar (SAR) imagery. Canadian Journal of Civil Engineering, Vol. 30, No. 1, pp. 11-27. doi: 10.1139/l02-073. Zhang, X. et al. [2021]. River ice monitoring and change detection with multi-spectral and SAR images: application over Yellow River. Multimedia Tools and Applications, Vol. 80, No. 19, pp. 28989-29004. doi: 10.1007/s11042-021-11054-0.
Q&A with
Tor Erik
Jensen
Legal professional. Assistant professor, University of South-Eastern Norway. Visiting assistant professor, BI Norwegian Business School. Consultant, Jensen Business Development and Consultancy. Expert, Norwegian Agency for Quality Assurance in Education (Nokut). Former member, Shipping and Offshore Network (Bulkforum). Law degree, University of Oslo. MBA, Norwegian School of Economics.
108 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
What is your occupation? Assistant professor lecturing on maritime law, chartering, and marine insurance in the Department of Maritime Operations at the University of SouthEastern Norway (USN).
What do you like most about working in this field? Meeting people and trying to develop a mutual understanding of the problems of relevance.
Why did you choose this occupation? I was invited by the former USN dean to discuss an assistant professor position as they needed an industrialist from the shipping industry, which aligned quite well with my background.
What are some of the biggest challenges your job presents? Substantially, the difficulties with really delivering new insights and problem solving. Substantive work is time-consuming and organizational processes, changes, and leaders with different priorities may distract us from the point of delivering new insights.
Where has your career taken you? Literally around the globe, meeting clients, academics, and students. On a personal level, my career has enabled me to develop in-depth knowledge in a position from where I do my best to share it.
What technological advancements have you witnessed? From analogue phones and fax machines to the digitalization and automatization of shipping over the past three decades, now beginning to take form and shape with potential disruptive effects.
If you had to choose another career, what would it be? Law suits me fine but has a wide spectrum of positions. Realistically, I would most probably be working somewhere within the shipping industry. A little less realistically, it would be something more physical such as professional yachting or sailing and competing globally.
What does the future hold for this industry? Transportation by ships will be needed, but different competencies and skills may be required as well as a change in the value chain. Political turmoil and environmental challenges and solutions may require different trading patterns. The demand for sustainability is the greatest challenge of our time and laws/rules and regulations is one important driving factor. For me this gives opportunities; lack of understanding may increase as the need for new regulation increases and potential market instability may increase the need for analyses and work from scholars. How do we not only change to and demand a cleaner shipping industry, but also develop the mechanisms of which some already are in place by the International Maritime Organization, Port States, and Flag States for companies to obey effectively?
What is your personal motto? Independence and integrity. What hobbies do you enjoy? Sailing, skiing, and hiking when time permits. Some exercises and cultural events. Where do you like to vacation? On a sailing boat, at the family cabin, and hiking. Who inspires you? Great performers, Greek philosophers, and modern authors. What has been the highlight of your career so far? Apart from invitations to conferences and sharing knowledge, I think the success of students defines my achievements, not just in the classrooms and their exams, but to have contributed – if only a little – to their development and abilities to deliver good, honest work in their professions.
What new technologies would you like to see? Artificial intelligence, automatization, and autonomous ships in a variety of types, sizes, and operations. Alternative fuels need to be developed along with machinery systems and distribution. What advice do you have for those just starting their careers? Be honest, work hard, and learn the basics, whether you go for a specialization of commerce, technology, or law. This gives you a platform from where you can develop your own career.
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 109
Trade Winds Unlocking the Future of Maritime Transport
The One Sea Association The maritime industry has always stood at the forefront of mechanical engineering. The world now stands again on the brink of the next Industrial Revolution with a scale and scope so complex that it is difficult to fully understand the impact this will have on the maritime industry and transport system. There should be no question that decisions related to new technologies, concepts, systems and their fusion will require new knowledge and insights. The exponential increase in computing power, availability of vast amounts of data, and access to global digital platforms for research and development will significantly accelerate innovation. It has also become increasingly apparent that understanding and integrating these new technologies requires new forms of collaboration, particularly given the speed at which innovation and disruption occur. The One Sea Association is a non-profit global alliance of leading commercial developers, manufacturers, digital solution providers, integrators, and operators of automated, autonomous, and remote-control technologies for the maritime transport chain. The association promotes implementing and creating the conditions needed for automated and autonomous maritime transport systems and fosters societal acceptance and confidence in such technology. One Sea was founded as an ecosystem in September 2016, and it brought global pioneers and ICT companies together to promote and coordinate the development of an autonomous maritime transport system and the creation of the circumstances needed for it. Since its foundation, One Sea has gained broad acceptance as an expert in autonomous shipping and as a trusted advisor on regulatory 110 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
matters related to maritime autonomous surface ships (MASS) in addition to the safety, efficiency, environmental, and social gains it promises. The One Sea ecosystem was reorganized as an association in July 2019. A Driving Force for Global Regulations One of the critical purposes of the One Sea Association is to support the development of the international legal framework needed to enable the implementation of new technologies in the maritime transport chain. The association has become one of the driving forces for global regulations, rules, and standards for MASS, and has been working closely with the International Maritime Organization (IMO), Member States, and other industry stakeholders to align and harmonize the development of regulations, standards, interfaces, and the testing and approval regimes necessary to deliver a safe and commercially viable autonomous logistics system. A System of Systems A ship is a highly sophisticated and complex machine that has a multitude of systems on board, each with its own operational parameters and integrations that support and optimize vessel operations, communications, and life at sea. When thinking about a ship, one way to look at it is as a system of systems, and One Sea is of the opinion that this is the best way to understand and promote the commercialization and widespread adoption of solutions arising from research and development activities. Encouragingly, where MASS is concerned, this is an approach that has gained support among industry members and organizations. However, further efforts are needed for this perspective to become a mainstream view in the market and among legislators.
Automation, Autonomy, and the Environment The use of autonomous and automated technologies within the maritime transport chain can provide a wide range of benefits, from enhancing safety at sea, improving ship performance, and protecting the environment. Finding solutions that can help decarbonize shipping is a major priority for the IMO, and its emissions regulations together with the EU’s Green Deal have set significant greenhouse gas reduction targets for the maritime industry.
Enhancing Public Awareness One Sea’s vision is to see technology being used to create an optimized transport chain that will enhance maritime transport safety, efficiency, and sustainability, while also improving seafarers’ working conditions, well-being, and social welfare. This is a vision shared by One Sea and its members who believe in the importance of respecting and aligning technology development with generally accepted principles and goals.
When discussing decarbonization, there is an understandable temptation to focus on alternative fuels and the challenges of emissions trading.
One of the positive societal impacts from the work One Sea has conducted to date has been the increase in general understanding of what maritime automation and autonomy entail and the potential benefits such technologies can offer. Enhancing public awareness and societal acceptance has been a significant aspect of the work, reflecting the fact that increased technology adoption will reshape the role of humans. Through the association’s actions and collaborations with industry stakeholders at all levels, One Sea has also been able to dispel misconceptions about maritime automation, autonomy, and its development path as interest and adoption of such technologies continue to gather pace.
However, digitalization, automation/autonomy, and the efficient use of data resources can kick start the transition towards greener shipping and will provide results before the broad transition to renewable fuels is possible. One Sea’s members, for example, have verified how fuel savings can be secured by using autonomous functionality across multiple trials. As environmental efficiency in the maritime transport chain is not only about ship emissions, One Sea has also strengthened its dialogue and collaboration with stakeholders related to port operations with a view to further advance collaboration across the entire maritime transport system.
Sinikka Hartonen is the secretary general with One Sea Association. sinikka.hartonen@one-sea.org | www.one-sea.org
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 111
Trade Winds Revolutionizing Maritime Sustainability
DANAOS
SmartShip’s Innovative Approach
Figure 1: The SmartShip system is a data analytics, decision support, and circular economy-based multi-layer optimization platform.
Why SmartShip? The maritime industry faces a critical challenge in reducing greenhouse gas emissions, with the International Maritime Organization targeting a 50% reduction in international shipping emissions by 2050 compared to 2008. Achieving this goal necessitates transition to more sustainable policies. Incorporating energy-saving devices and adopting necessary investment strategies are vital for enhancing existing operations and complying with the new regulatory environmental schema. While expensive retrofit solutions hold promise, they also pose challenges and uncertainties regarding their technical effectiveness and return on investment. This is the reason that you should first look at the way you are doing things, how you operate your vessels today, and try to achieve cost-effective improvements before 112 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
you move to heavy investments. To do so, it is crucial to examine policies and decisions through a digital transition and a datadriven approach, which is precisely what the SmartShip system offers. The SmartShip project stands as an example of innovation, seeking to redefine the maritime sector’s approach to sustainability. The main objective of SmartShip is to facilitate a profound transformation, aligning the industry with international emissions targets and embracing circular economy principles. What is SmartShip Project? At its core, SmartShip is a data-driven holistic framework, designed to optimize energy efficiency, reduce emissions, and enhance fuel consumption in maritime operations (Figure 1). The SmartShip system offers a data analytics,
decision support, and circular economy-based multi-layer optimization platform. It introduces a new way of approaching ship design, operation, and maintenance, aligning with the global trend for greener and more sustainable fleet management. What is the SmartShip System? SmartShip takes advantage of cutting-edge tools and frameworks to build and develop the project’s holistic system by researching the following fields:
• • • • • •
Energy efficiency management and monitoring Emissions control in the maritime sector Cloud-based and Internet of Things (IoT) systems usage in the maritime industry Sensor technologies for maritime applications Algorithms and optimization techniques, including Life Cycle Assessment Advanced data analytics and decision support systems
The SmartShip system is characterized by three foundational components, each playing a pivotal role in its success: 1. Data sourcing (IoT): At the base of the SmartShip architecture, this component considers tools, communication protocols, and network topology for data retrieving, pre-processing at the edge, and finally, transferring information to the SmartShip core for further processing and analysis. 2. SmartShip core system: The heart of the ecosystem, the SmartShip Core, is where data is processed, analyzed, and visualized to support decision-making for critical maritime operational procedures defined in the project’s use cases: a) Weather routing optimization: This use case provides optimized routing advice to onboard masters, considering various factors, including weather conditions and individual vessel characteristics, to reduce fuel consumption and enhance efficiency. b) Route monitoring: Complementing
the first, this use case continuously monitors vessel progress along their planned routes, ensuring they stay on course and documenting any deviations. c) Condition-based maintenance: Leveraging technology-driven fleet performance monitoring, this use case optimizes maintenance practices through historical data analysis, predictive analytics, and strategies to ensure peak performance. 3. User applications: The top layer facilitates practical utilization of insights derived from data analysis. It allows onshore and onboard users to use data insights for informed decisions, fostering interaction between shore and vessels, enhancing sustainability. A Circular by Design System and its Benefits About 47% of maritime businesses use IoT and advanced data analytics techniques to measure and forecast fuel consumption. Increased IoT uptake will improve the monitoring of ship components and improve longevity and performance, while real-time monitoring will enhance the scheduling of maintenance when necessary. Embedding circular economy principles in the different components of the SmartShip system is the novel proposition of the project, which brings benefit to the existing data driven approaches in the industry. Circularity characteristics to consider in the development of the platform are:
• • • •
Circular attributes: Location, Condition, and Availability Circular design: Modularity, Scalability, Functionality Data collectors’ requirements: End-toend security/privacy, Dependability, Operability Trust: Trustworthiness, Confidentiality, Security
The circular design concept of the SmartShip system comes with many notable benefits including: The Journal of Ocean Technology, Vol. 18, No. 4, 2023 113
•
•
•
• •
Digital transformation of vessels: Maximize sustainable vessel utilization and ensure long-lasting durability of the assets. Value driven: Extract value from the large amount of data generated by digital transformation of vessels, ensuring an effective flow of information for natural capital rebuilding. Eliminate waste: Achieve waste reduction through data reusability and lean management in decision-making for fleet operation and maintenance. Green thinking and sustainability: Minimize energy consumption per unit by effectively combining technologies. Integrated framework: Extends across the entire fleet and vessel’s lifetime, pairing value drivers for efficiency through comparative analysis.
SmartShip EU Project received funding from the European Union’s Horizon 2020 research and innovation programme under the MarieSlodowska Curie grant agreement no. 823916.
A specialist in project management and business development, Fotis Oikonomou has gained experience in leading and managing projects in a diverse range of industries including logistics, port, shipping, and technology. He joined DANAOS as a senior consultant and research officer facilitating the planning and execution of research projects. He is participating in a number of EU research projects focused on applying innovative solutions in the maritime industry.
114 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
MARINE INSTITUTE
ALUMNI SERVICES AND BENEFITS
CAREER SERVICES
• Special events • Reunions • Degree frames • Alumni and campus news • Group rates on insurance services • Alumni credit card • Retail discounts • University library access
• Resume/CV critique and evaluation • Mock interviews • Job search skills – personal branding, networking, job search resources • Resume referral services • Job postings • Access to career events
Stay connected and up to date on new alumni services, events, latest news and how you can become involved.
RECONNECT
NOW!
www.mi.mun.ca/alumni/reconnectwithmi alumni@mi.mun.ca 1 800 563 5799, ext. 0367
linkedin.com/edu/FisheriesandMarineInstituteofMemorialUniversity facebook.com/MarineInstituteAlumni twitter.com/MarineInstitute youtube.com/marineinstitutepr instagram.com/marine.institute
Trade Winds Research and Scientific Expeditions
Sea Centric Inc. Sea Centric Inc. (formerly Real-Time Ocean Data Services) is a family-owned research vessel charter company based in St. John’s, NL. It owns and operates the R/V Patrick and William and the R/V Connor Murphy (Figure 1). Since its inception in 2019, the company has grown exponentially due to high demand of vessel availability, accessible technology to collect important scientific data, and the Blue Economy push to put more funds and effort into protecting our ocean. The R/V Patrick and William was originally used solely as a crab fishing vessel. The company was asked to charter scientists to tag icebergs with drones outside of its fishing season in 2019. This charter is what started the obsession with ocean technology and developing a platform to house it. One of the biggest journeys for Sea Centric Inc. was researching and purchasing high end technology to offer its clients on the R/V Connor Murphy boasting the best available technology on a rock-solid vessel platform. One of the most important pieces of technological equipment we have purchased to date is the VEEM Gyrostabilizer through Wajax (Figure 2). By purchasing this technological equipment, the company adds such a high level of stability to the vessel that important operations can be performed 10-12 months of the year instead of eight. The next retrofit of the R/V Connor Murphy, set to happen in fall 2023 to spring 2024, is to add 10 m into her midships to make her a fully realized, state-of-the-art research vessel that can work internationally for long durations from working in the Arctic to off season contracts in South America, United States, or Europe. Following is an equipment description provided by Wajax.
116 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Figure 1: The R/V Connor Murphy boasts the best available technology on a rock-solid research vessel platform.
SEA CENTRIC
VEEM Marine and Wajax have joined forces to help Sea Centric Inc. meet its project requirements. VEEM Marine, part of VEEM Ltd., is a marine technology company with over 55 years of precision manufacturing experience and decades working with leading boat builders and naval architects that manufactures leading propulsion systems and gyrostabilizers at its facility in Perth, Western Australia. Supporting Canadian industries since 1858, Wajax is one of Canada’s leading suppliers of marine power products, stabilization systems, as well as other specialized equipment. In early 2023, VEEM put its experience and expertise at the service of Sea Centric, which was looking for
a stabilization system capable of meeting the unique challenges posed by the operation of a salvage vessel converted for research and study expeditions (R/V Connor Murphy). The vessel operation would often be confronted with rough seas and ice conditions. With the repurposing of the vessel, Sea Centric was particularly conscious of the need to provide the most stable platform possible to ensure that the crew and research personnel could perform their duties efficiently with the highest degree of comfort and safety. This is a particularly important objective for a research vessel as the work often involves close focus and screen time away from fresh air and a visible horizon, a well-known recipe for fatigue and motion sickness, particularly for those not used to living and working at sea. With no external hull appendages exposed to damage or drag in the water, and the ability to deliver smooth, quiet, and continuous stabilization at all speeds, including at zero knots, a gyrostabilizer stood out as a solution that could be retrofitted with relative ease thanks to its self-contained, single module configuration. Launch and retrieval of delicate equipment can also be carried out with greater safety in a wider weather window while vessel mounted instruments collect higher quality data with fewer passes with effective stabilization. Reducing a vessel’s rolling motions effectively provides the motion characteristics of a much larger vessel. This means lower capital, crew, and operating costs to deliver the same results as a larger vessel platform. Considering the crew welfare objectives and operational productivity benefits, Sea Centric decided that installing a system with the highest angular momentum, the power measure for Gyros, that would fit in the space available was the right approach. There are now six VEEM Marine Gyro models designed to cater to vessels ranging from 20 m to over 100 m.
Figure 2: The VEEM Gyrostabilizer is installed on the R/V Connor Murphy, offering the most stable platform possible for crew and research personnel. WAJAX
All models use a vacuum housed flywheel to reduce heat generation and power consumption and are fully maintainable in the vessel. Wajax’s and Sea Centric’s teams worked closely with technical support from VEEM Marine to select the VG140SD, the more powerful of VEEM Marine’s mid-size Gyros. The selection process was aided by VEEM Marine’s GYROsim modelling that predicts root mean square (RMS) (average) roll angle reduction of up to 89% for lightship configuration and close to 60% RMS roll reduction even when fully loaded in 3 m seas. Sea Centric’s charter clients will undoubtedly experience an increase in productivity, on station time, comfort, and safety during their future research and scientific expeditions aboard the Connor Murphy. Both VEEM and Wajax are excited to be a part of the journey by bringing the first of this excellent technology to Canada and to Sea Centric Inc.’s, R/V Connor Murphy. For more information: www.miawpukekhorizon.com megan.murphy@miawpukekhorizon.com Megan Murphy works as the business development manager for Miawpukek Horizon Maritime Services.
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 117
Trade Winds Navigating the Future of Autonomous Shipping
Maritime Research and Innovation UK Maritime Research and Innovation UK (MarRI-UK) was established in 2019 with the support of the government and leading industrial and academic organizations. These include Department for Transport, National Shipbuilding Office, Royal Navy, Society of Maritime Industry, Babcock, BAE Systems, BMT, Lloyds Register, QinetiQ, Shell, Newcastle University, University College London, University of Plymouth, University of Southampton, and University of Strathclyde. The maritime industry is of vital importance to the UK – generating £6Bn for the economy, enabling 90% of visible UK trade, and directly supporting 227,100 jobs in its own right. It has a key role in enhancing national competitiveness, securing a resilient future for the UK, and enabling sustainable development. Its scale offers major opportunities for economic growth, exports, and carbon reduction. Sir John Parker’s review of UK shipbuilding emphasized the role of MarRI-UK as a key institution to drive maritime innovation in alignment with the UK Government’s Maritime 2050 plan. To date, we have coalesced the UK’s research community, two government departments, and 24 members into a single group to accelerate the delivery of world leading research and innovation through collaboration. Including the trade body partners’ network, we proudly represent over one thousand UK maritime organizations. In fostering collaboration with open access between industry, academia, and government, MarRI-UK acts as a catalyst for technological advancements, promoting economic growth, skills development, and job creation. Hosted by University of Strathclyde (the largest maritime research centre in Europe 118 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
and the highest rated in the Western World), we have created an internationally distinctive maritime research and innovation grouping with UK’s best maritime research universities. MarRI-UK is strategically positioned to drive UK government policy outcomes, with the strategic objectives shown in Figure 1. One notable strength is the UK’s historical expertise in shipbuilding, which can be harnessed for sustainable practices and the development of eco-friendly vessels. The refreshed National Shipbuilding Strategy serves as a cornerstone for fostering growth and competitiveness in the maritime sector. The strategy aims to revitalize the nation’s shipbuilding capabilities, leveraging regional expertise and bolstering the industry’s contribution to the national economy. The focus is not only on strengthening the UK’s position domestically but also on enhancing its global standing in the maritime market. The latest challenges in the maritime sector include the need for improved sustainability, changing regulatory demands, and the evolving landscape of geopolitical and economic factors. However, within these challenges lie opportunities for innovation, technological advancement, skills development, and economic prosperity, which could be explored through MarRI-UK members. Based on our industry members’ interest, MarRI-UK’s current research program areas are focused on clean maritime; autonomy; and integrated data, information, and knowledge management support. Autonomous ships, or smart ships, leverage advanced technologies such as artificial intelligence (AI), machine learning, the Internet of Things, and automation to enhance
Figure 1: MarRI-UK strategic objectives.
various aspects of maritime operations. While autonomous ships offer several benefits, they also face challenges that need to be addressed for successful implementation. A few main challenges are listed below, which we hope to address through our members’ individual research activities and our collaborative research program.
•
•
•
•
•
Regulatory Frameworks: The maritime industry has traditionally been heavily regulated, and integrating autonomous technologies requires new or adapted regulations. Cybersecurity: As autonomous ships rely on interconnected systems and data exchange, they are susceptible to cybersecurity threats. Integration of Technologies: Integrating diverse technologies such as sensors, AI, and automation systems seamlessly is a complex task. Data Management and Analytics: Autonomous ships generate vast amounts of data, and efficiently managing, analyzing, and extracting actionable insights from this data is a challenge. Human Element: The transition to autonomous ships raises concerns about the role of human operators, their training, and the potential need for new skill sets.
•
Infrastructure and Connectivity: Reliable and high-speed connectivity is essential for autonomous ship operations, and the required infrastructure may not be uniformly available globally.
To overcome the above challenges, collaboration is vital. This involves bringing together government, academia, industry experts, investors, and regulatory bodies to establish clear and standardized regulations, and to invest in and develop the necessary technologies and infrastructure. Furthermore, collaboration enables the sharing of knowledge, expertise, and resources among different technology providers, ensuring smooth integration and interoperability of systems. It also facilitates the development of standardized data formats, protocols, and analytics tools, allowing for effective data management and utilization across the industry. Additionally, collaboration is crucial for developing training programs, guidelines, and standards that address the evolving roles of human operators in the context of autonomous ship technologies. For more information: https://marri-uk.org info@marri-uk.org Dr. Wenjuan Wang is the MarRI-UK program manager.
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 119
Remotely Controlling USVs Towards a Lowcarbon Future by Henry Robinson As maritime stakeholders increasingly look to uncrewed vessel technology to complete a wider range of offshore survey, surveillance, and inspection tasks, the systems that are used to control these flexible platforms are developing fast. Uncrewed surface vessels (USVs; Figure 1) offer a low impact, non-invasive, and costeffective solution for an industry that has set itself ambitious net zero emission goals, and the technology is now well established on commercial projects around the world. The control of a crewed vessel to achieve a high level of manoeuvrability, accuracy, and safety usually relies on an experienced helmsperson using multiple skills and senses to provide feedback and respond to sea states and other environmental conditions. Achieving the same performance with a USV requires in-depth understanding of vessel dynamics, including hull design, payload, and propulsion and steering control systems. There are many propulsion solutions for USVs, including electric, internal combustion, vectored thrusters, propellers and rudders, waterjets, Voith Schneider propulsors, wave propulsion, and wind power. Each of these propulsion setups has characteristics suited to different applications. As a result, USV designers must carefully consider the end user application or mission when specifying the propulsion and steering configuration. This article explores the various systems available and how uncrewed vessel performance can be maximized.
120 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Low-speed Seabed Survey For the low-speed manoeuvring required to survey the seabed, it is important to control thrust right down to near zero, and reverse, as well as to maintain the ability to steer even at minimal thrust levels. With a single outboard engine using conventional internal combustion, this is not possible. Consequently, low speed manoeuvring and station keeping often involves repeatedly going in and out of gear, using pulses of thrust followed by counterthrust for steering. Even electric motors can have significant dead-bands, as most motors have a minimum speed for start-up. Brushless motors are difficult to control at very low speeds unless they are fitted with phase feedback sensors; even then, a minimum speed of around 50 RPM is considered quite low. The lowest speeds are generally achieved using motors fitted with reduction gears, or brushed motors, some of which can drive down to less than 10 RPM. The problem of minimum engine speed can be overcome using other mechanisms such as waterjets, where the reversing bucket is used to vector the thrust smoothly all the way through to zero. Typically, the engine is left in idle RPM, giving accurate vectored thrust control around the zero-thrust point. If more vigorous steering is needed, the vectored thrust can be boosted by running the waterjet at higher RPM. Station Keeping Manoeuvres Sideways manoeuvres are needed for stable
Figure 1: Uncrewed surface vessel. ALTI
station keeping. For example, when operating an underwater remotely operated vehicle, the USV platform it launched from needs to maintain both heading and position. Any lateral forces from current, wind, or waves must be countered using sideways thrust. Using twin propellers, steering can be controlled by means of differential thrust; if the thrusters are steerable, this provides a third control parameter. In principle, this might mean that twin steered propulsors can be used to control three variables – forward and lateral speeds components and yaw rate. In practice, this is extremely hard to control and tends to have very limited effectiveness with lateral movement.
For USVs that need to be able to move sideways, another control parameter is needed. This could come in the form of a lateral thruster (bow thruster) or can also be achieved using independently steered main propellers. Steered propellers are more agile than fixed propellers with rudders because steering ability is maintained in reverse, where rudders are much less effective. New Control Challenges During some 25 years of operation, Dynautics has encountered a wide range of vessels, missions, and combinations of propulsors and steering devices, for both surface and subsurface vessels, and has developed methods to combine control parameters
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 121
DYNAUTICS
in different ways to suit the vessel and the mission requirement. The challenge has always been to incorporate new strategies in a manner that is as generic as possible, so that the system becomes easier to adapt to new configurations as and when they are encountered (Figure 2). These could include different ways of coordinating the propeller RPM, gearbox, reversing bucket, and steering nozzle to exploit the full manoeuvrability of waterjets or methods to switch safely between autopilot and crew control on vessels that have both uncrewed and crewed capabilities. A good example of a recent new challenge was a USV designed to hold station in strong currents, while maintaining any prescribed heading, so that the onboard sensors (which were fixed in relation to the boat) could be directed at a target on the seabed from any angle. The vessel had been designed with multiple steerable thrusters so that, conceptually at least, it could thrust and manoeuvre in any direction. However, the ability to drive sideways had only been achieved at the expense of stability in forward motion; indeed, it was impossible for
122 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Figure 2: Dynautics SPECTRE marine autopilots and remote control systems offer heading, depth, height, and altitude control as well as dynamic positioning.
a human pilot to drive it forward at any speed above minimum without veering off to one side or the other. The solution was to use a carefully tuned thrust vectoring algorithm which adjusted thrusts and directions very rapidly. Using this predictive algorithm, the autopilot was able to maintain stability up to full speed. Successful control has also been achieved using both fixed and vectored twin-thruster arrangements to achieve maximum manoeuvrability. This technique has been used on several USVs to transform vessel performance. Even vessels with a turning circle of many boat lengths have been adapted to be able to turn within their own length, and to operate in stronger environmental conditions.
Dr. Henry Robinson is the founder and CEO of Dynautics Limited. He has been working in marine electronics and autopilots for the last 25 years.
Perspective viewpoint
Autonomous Vessels in Maritime Affairs This book examines law and governance implications in relation to maritime autonomous surface ships. Adopting a multi-disciplinary approach, it focuses on a wide array of timely, topical, and thorny issues, including naval warfare and security, seaworthiness and technoregulatory assessments, global environmental change, autonomous passenger transportation, as well as liability and insurance. It also considers selected national and regional developments. The book provides an insight into the role of innovation diplomacy as the driving force that could expedite the transition from automation to autonomy. After navigating through the complex law and governance landscape, it concludes by assessing critical findings for further consideration. The book will appeal to scholars and students of maritime technology, law, and governance.
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 123
HST MARINE
Hybrid Vessel Performance Data HST Marine is leveraging Reygar’s BareFLEET technology to understand and report on the performance of its hybrid crew transfer vessels (CTVs). The company currently has four hybrid CTVs in operation with three more soon to enter service, all of which have BareFLEET installed. These vessels are a mix of both controllable and fixed pitch propeller systems that take power from either a high-efficiency electric motor or the main engine, allowing them to operate almost silently and with zero emissions in electric only mode. New features, developed by Reygar within the BareFLEET technology package,
124 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
allow HST Marine to closely monitor the performance of hybrid vessels including a breakdown of electric versus diesel power consumption while carrying out different tasks offshore. BareFLEET also monitors the electrical power consumption of the hybrid drive, with specific usage and performance statistics now included alongside conventional diesel engine performance data. These features enable HST Marine to evaluate the environmental performance of hybrid CTVs against conventional vessels and to make adjustments for further improvement. www.reygar.co.uk www.hst-marine.com
Turnings what's new
Data Collection at Sea The United States Defense Advanced Research Projects Agency has awarded Glas Ocean Electric a contract to develop and demonstrate a low-cost data acquisition system that can be easily installed on a range of vessels including fishing vessels, workboats, ferries, and pleasure craft. Glas Ocean Electric’s PerforMarine™ is a platform that collects vessel and environmental data and then, using artificial intelligence, helps operators optimize vessel performance, reduce fuel costs, and cut emissions. PerforMarine collects data from commonly installed vessel sensors such as depth sounders and anemometers. Additional sensors can be added and with artificial intelligence they can fully describe the vessel movement. Real-time and predicted data such as vessel position, sea conditions, and weather are integrated with the software and enhance the capability of PerforMarine. www.GlasOceanElectric.com
GLAS OCEAN ELECTRIC
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 125
DVL-INS Systems for Dynamic Positioning by Derek Lynch Ships operating with dynamic positioning (DP) systems are governed by international regulatory requirements. The International Maritime Organization (IMO) has strict rules for DP ships, classified as DP Class 1, DP Class 2, and DP Class 3, each setting different levels of redundancy and independence for the ship systems – depending on how critical station keeping is to the ship’s safe operation.
it requires the deployment and recovery of transponders. In shallow water there is a relatively narrow cone of operation before the need for more transponders, with associated operational time/cost factors. Taut-wire – derives vessel movement relative to a deployed seabed clump weight on the end of a (taut) wire. However, generally big and expensive, they take up valuable deck space, restrict vessel movement – particularly in shallow water – and need careful management if there are cables or infrastructure below. Laser and radar ranging systems – simple range and bearing principle but require a suitable structure nearby for mounting target prisms or responders.
DP Class 2 is the most common classification for ships operating in the offshore energy sector. The rules call for three independent position reference systems based on a minimum of two different principles for deriving the ship’s position reliably to within three metres.
It can be seen, particularly in shallow water operations typical of offshore wind farm developments, nearshore work, and much of the Middle East, the selection of suitable systems is very limited. Consequently, there is often an over reliance on multiple GNSS systems with common vulnerabilities and failure points.
Acceptable solutions to meet this requirement are limited to a small number of specialized systems. However, each has vulnerabilities and limitations, driving the need for diversity. The most common solutions used include the following:
The International Marine Contractors Association DP Committee identified a trend of DP station keeping incidents relating to position reference sensors – often as a result of selecting systems inappropriate for the task or operating conditions. So, what to do about it?
Global Navigation Satellite System (GNSS) with augmentation subscription services – precise, accurate, widely available, and easy to use, it is often the first choice. However, signals can be interrupted and blocked by nearby structures and are susceptible to interference.
Subsea vehicles also require precise and reliable navigation, especially in GNSSdenied environments. Acoustic, inertial, and sonar technologies are used, but most vehicles operate close to the seabed, allowing the use of Doppler Velocity Log (DVL) Inertial Navigation Systems (INS), like Sonardyne’s SPRINT-Nav, for accurate navigation.
Hydro acoustic positioning systems – provides reliable ship position relative to a seabed reference transponder. However,
126 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Sonardyne has over 50 years’ experience in providing subsea positioning and
Reverberations then and now navigation solutions, based on our core acoustic, doppler, and inertial technologies, for customers operating in the harshest of environments where precision, accuracy, and reliability is paramount. Realizing the value of tightly integrating high grade INS with other aiding sensors, we were the first to market a hybrid acoustic-inertial family of products for subsea applications. Our SPRINT-Nav family of all-in-one DVL-INS instruments has provided reliable navigation for ROVs, AUVs, and USVs for more than a decade. Renowned for its performance, it is installed on hundreds of vessels. SPRINT-Nav has several advantages, including:
• • • •
High accuracy and position update rate, comparable to GNSS Independence from GNSS No need for targets or sensors Proven track record in subsea navigation
The question was raised: can SPRINT-Nav provide a viable solution for a shallow water DP position reference system? Due to IMO regulations, we needed to validate the integrity of SPRINT-Nav performance in the DP domain. So, we set out to prove its effectiveness in shallow water DP applications. The first trial was conducted with a customer in the United Arab Emirates. A regular user of SPRINT-Nav on their subsea vehicles meant they were familiar with its capability. We installed a unit on a DP2 multi-purpose support vessel, operating in water depths of around 10 m, from Mussafah Port, Abu Dhabi. System testing and DP trials were carried out in line with regulatory class requirements and performance comparisons made with
Figure 1: Position plots.
onboard survey grade GNSS systems. All data was recorded for deep analysis by our system engineers. The system performance was flawless over a period of two weeks and completely satisfied all requirements. Seeking to experience different DP operating scenarios, another customer made their DP2 geotechnical drilling vessel available for trials. Operating in the Baltic Sea performing geotechnical surveys for offshore wind developers, the vessel is continuously in DP. This install was carried out at the end of 2021 and has now become our standard test platform. We have changed variations of the SPRINT-Nav unit for comparison without any failures or out of spec performance. Extensive amounts of data have been collected, allowing our developers great insight into how the system performs. Here are some samples of the data we were able to retrieve and analyze: Figure 1: Position plots show a tight cluster, with no appreciable drifting, spikes, or outliers (the plot shows +/- 0.5m movement over a 24hour duration).
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 127
Figure 2: DVL bottom track residuals.
Figure 3: DVL signal quality metrics.
Figure 2: DVL bottom track residuals showing exceptional performance and no dropouts, noise, or other anomalies.
without DP station keeping events be in sight for offshore wind, coastal management, and other shallow water operators?
Figure 3: DVL signal quality metrics showing very strong signal levels – anything over 10 dB is good and 40 dB is excellent. We can see increased signal to noise during drilling and higher levels of thruster activity but no dropouts or degrading of position. Finally, the cross correlation against stored replica is exceptional. Further trials are planned in other areas including offshore wind installation, shallow water cable laying, and platform supply operations. The field trials have shown that SPRINTNav is a viable solution for shallow water DP operations, offering a robust and independent position reference system not reliant on GNSS. Could operating in shallow water
128 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
Derek Lynch is the business development manager, energy, with Sonardyne International Ltd. He brings over 30 years’ experience of working within the marine dynamic positioning and navigation systems sectors having held senior product management, sales, marketing, and commercial positions.
Informative Cutting Edge Provocative Challenging Thought Provoking International
thejot.net
Smart Ships Unleashed Mixed Reality as the Transformative Core by Iain Whyte The marine sector has always been at the forefront of technological innovation to enhance efficiency, safety, and sustainability for vessel operations and fleet management. More recently, the development of concepts around smart ships represents the next step in that development, drawing together the evolving power of digital technologies, artificial intelligence, data analytics, and Internet of Things sensor technologies enabling owners and operators to optimize their operations. Smart in the marine sector, while used widely, is not uniquely defined, nor should it be seen as just a synonym for autonomy. For the purposes of this article, smart will focus on the fusion and integration of digital technologies, automation, and data analytics to enable operating processes that contribute to enhanced operational efficiency, sustainability, and safety goals. At the heart of it all is data, with vessels, in essence, becoming large data centres which, while facilitating more automation, will still require human input and interaction. One of the technologies that can help us make sense of this wealth of data, visualize it in meaningful ways, and optimize human performance in the marine sector is mixed reality (MR). MR is a transformative technology that blends the real-world space we operate in with digital elements that augment and inform our understanding, offering users a rich and immersive experience while operating safely in their real-world environment. In the marine sector, MR applications predominantly leverage information repositories, remote expertise, 130 The Journal of Ocean Technology, Vol. 18, No. 4, 2023
real-time data, and visualization to inform our situational awareness as well as enhance decision-making and operational processes. Why now? In the marine sector, several converging factors underline the growing recognition of MR as a valuable tool. These factors centre on safety concerns, rising operational and systems complexity, and stringent greenhouse gas emissions targets. Additionally, the increasing deployment of digital technologies and an evolving interest in this direction – further supported by improved connectivity through low-earth orbit satellite communication networks – all play a role. Despite the unique challenges and opportunities within the marine sector, MR can yield significant benefits. Enhancing Operational Efficiency Maintenance and repairs in the marine sector often require highly specialized knowledge and resources. With MR, remote experts can provide real-time, step-by-step guidance to onboard crew, reducing downtime and minimizing the need to fly in specialists. This not only improves operational efficiency but also saves costs and reduces the carbon footprint associated with air travel. Training and skills currency is a critical aspect of maritime operations. MR enables crew members to learn by visualizing equipment and complex systems, making it easier to understand and work on those systems. The industry’s requirement to comply with maritime regulations is critical. MR can assist in ensuring that owner operators can efficiently resolve some classification issues remotely by providing real-time information and guidance while reducing the delays in inspectors travelling to the vessel. Safety Benefits Much of maritime safety relies on realtime monitoring of ship equipment and conditions. MR can enable crew members to
Homeward Bound commentary
LEEWAY MARINE
visualize and monitor parameters like engine performance, stability, and weather conditions in real time, identifying and addressing issues promptly. Collaboration, shared situational awareness, and decision support are central to MR applications, allowing crew members to communicate seamlessly, share critical information, and work together both on board and with onshore support teams, reducing misunderstandings and accident risks.
While MR use in the marine sector is growing, implementation has challenges, including the investment required in digital infrastructure, crew training, content development and management, cybersecurity, data privacy, and service resilience. Despite these common digital challenges, MR in the marine sector enhances efficiency, safety, and sustainability, and aligns well with evolving sector priorities.
While smart ships are integral to the future of Sustainability Benefits the marine sector – offering a paradigm shift in the way vessels are designed, operated, and Mixed reality significantly contributes to maintained – they will need solutions that sustainability, encompassing operational, social, and environmental aspects. integrate human operators, managers, and key decision-makers in those operations. MR Remote MR monitoring empowers shipping solutions offer tangible benefits, and we can companies to globally optimize routes and expect MR applications to become further operations, reducing costs and related integrated into daily marine operations. Smart environmental impact. Additionally, MR is a ships will play a pivotal role in shaping the useful tool in identifying maintenance and future of maritime operations, contributing repair needs early and supports conditions-based to a more sustainable industry, and are now maintenance systems that can extend vessel poised to play an increasingly critical role in lifespans, minimizing environmental effects. the future of the marine sector. Addressing workforce challenges related to an aging workforce and a shortage of skilled personnel, MR can help bridge the gap by connecting new or less experienced crew members with newly retired experienced mariners, tapping into their years of experience and reducing the learning curve for new crew members.
Iain Whyte is a former UK Royal Navy Officer and is currently director, defence (naval) and marine solutions, at Kognitiv Spark, a Canadian mixed reality software company.
The Journal of Ocean Technology, Vol. 18, No. 4, 2023 131
Parting Notes
Fast Friends Grayson Shallow Age 6 St. John's, NL
The ocean is the foundation of life. It is crucial that both the young and the old respect it and preserve it. This piece depicting turtle friends enjoying a swim together was created by a little artist/ocean lover in his 2023 summer camp as part of a weeklong celebration of all things ocean and the wonder within it. In order for the ocean, and its countless inhabitants, to thrive, it is important to take the time to talk to children about the impact that humans have on the environment, both good and bad, keeping the dialogue open as to what we can and should do to protect all life and the places they call home. Ocean preservation benefits us all.
132 The Journal of Ocean Technology, Vol. 18, No. 4, 2023