Assessing competence in ECDIS navigation

Contents Presence in marine simulator training and research .......................................... 5 Simulator based training (SBT) and quality assurance .................................... 10 The use of marine simulators in developing the concepts and technology of e-navigation .................................................................................................... 18 The IBS as part of the bridge team ................................................................. 27 Research on standardization of marine simulator training and assessment ...... 32 Effective simulator training in preparation for icebreaking operations and ice management assessment ................................................................................. 38 BRM course for deck cadets of maritime institutions ..................................... 46 Navigation simulator’s role in maritime english teaching ............................... 52 Assessing competence in ECDIS navigation .................................................. 60 How does the use of marine navigation simulators vary in the training and examination of ship masters, globally? ........................................................... 76 Simulation training of boat handling: Contributions of problem solving style, I INSLC16 Proceedings Dalian, China July 12-16, 2010 ASSESSING COMPETENCE IN ECDIS NAVIGATION Christian Hempstead (U.S. Merchant Marine Academy, USA) Abstract: The STCW training requirements for ECDIS adopted in 2010 intend that watchstanders are prepared to navigate safely while applying ECDIS. The commonplace static review of ECDIS principles and functions must now include some measures of mastery in adapting the ECDIS to dynamic contexts. This means that the training must occur predominantly in an underway environment. Through navigation simulation, the most effective training can be accomplished on type-approved ECDIS embedded in ownship functionality. Ideally, the training environment would include several semi-isolated integrated visual ownships for solo navigation using ECDIS. Each trainee should be able to practice many aspects of navigation with ECDIS. The development of the necessary skills should then be similarly assessed. Proof that the skillful use of ECDIS can enhance the safety of navigation requires a solo environment and simulation exercises designed to require ECDIS-based solutions. The fundamental rule must be stated in the negative: Any unsafe navigational event might have been avoided through a more effective use of ECDIS. Groundings, allisions, collisions or even near misses should by definition invalidate otherwise satisfactory skill assessments. In this paper it is suggested that evaluation of ECDIS navigation follows a competency-based matrix consisting of a quality scale of 1 to 3 for approximately 50 specific skill sets. These skill sets are organized into task groups, and are closely aligned with the new KUP and evaluation criteria listed in STCW-2010 Tables A-II/1 and A-II/2. Scoring is tallied according to the task groups with the application of weighted averages as a method for recognizing the higher degree of relevance of certain task groups in ECDIS navigation. INTRODUCTION Above all, the new competence of ECDIS in STCW-2010 is succinctly stated: “Use of ECDIS to maintain the safety of navigation.” This not only aligns ECDIS with other navigational aids included in STCW, but it also places ECDIS qualification in the service of navigation. For this to be the outcome, the training scheme and methods of assessing that qualification must take the same approach, namely, in the service of navigational safety. It must be stated at the outset that ECDIS by its very nature is a centralizing instrument with the unique function of integrating many aspects of navigation. To learn ECDIS outside the context on such integration and outside the context of underway navigation is to risk developing a very unwelcome competence in what should be called “anti-navigation”. This paper will attempt to demonstrate an effective method of training and assessing ECDIS competence that is closely aligned with the detail and intent of STCW-2010. To accomplish this, the ~ 59 ~ INSLC16 Proceedings Dalian, China July 12-16, 2010 new STCW language is provided, since it is likely to be unfamiliar at this early date in the probable scheme of implementation (2012 through 2017 is generally expected). This is followed by a brief description of the form and function of an ECDIS navigation simulation arrangement optimally suited to accomplish these requirements. Then a sequence of underway exercises will be presented in order to illustrate the central theme of training in navigation with ECDIS. In connection with the exercise examples, specific tasks of ECDIS navigation are identified within a set of ten task groups. A matrix of ECDIS-related tasks in the same groups is then presented in order to streamline the effort of assessing the application of skills in ECDIS navigation. The matrix also provides a means of guidance for the training process, as well as a means of adjusting the weight or relative importance for task groups depending on the desired emphasis of the training outcome. As a culmination of the scoring, a method is provided to apply the weighted averages of the task groups into a tally that represents the achieved skills in ECDIS navigation with reliability and validity. It should be noted also at the outset that, despite the commonplace use of rubrics and matrices for scoring demonstrated competencies across a broad spectrum of training and educational disciplines, this method is not yet being applied in ECDIS navigation training, to the best of the author’s knowledge. The discussion in this paper about ECDIS simulator standards, ECDIS navigation exercises, the task groups, competency matrix, and scoring method are all derived from the author’s own experience over the last decade in ECDIS course development and training, and especially through several years of implementing visual solo navigation training labs specifically for ECDIS navigation at the U.S. Merchant Marine Academy in Kings Point, New York [2]. The author also had significant input into the language initially and eventually written into the ECDIS section of the Comprehensive Review of the STCW Code [1][5]. 1. STCW-2010 REQUIREMENTS ECDIS TRAINING FOR The STCW-2010 training requirements for ECDIS on vessels subject to SOLAS are significantly different from the previous Code. First, the new competence is succinctly stated: “Use of ECDIS to maintain the safety of navigation.” Second, there are distinct and appropriate difference in requirements under the Functions of Navigation at the Operational Level (Table A-II/1) and Navigation at the Management Level (Table A-II/2). New ECDIS training requirements also apply to navigational watch officers and masters on ships < 500GT through Table A-II/3, but the description of KUP is limited to “Thorough knowledge of and ability to use ECDIS.” Third, there is specific detail provided in both KUP (Knowledge, understanding and proficiency) and Criteria (Criteria for evaluating competence) categories. Fourth, the requirements apply in all three Tables to STCW-endorsed seafarers who serve on ships fitted with ECDIS [1]. (As of this writing, the Manila Conference of June 2010 has not yet occurred, so final documents are not available, nor are dates that the requirements will enter into force.) The new STCW ECDIS training requirements are reproduced below, and will be referenced and discussed in the subsequent sections of this paper. 1.1. Table A-II/1: Navigation using ECDIS Knowledge, understanding and proficiency Knowledge of the capability and limitations of ECDIS operations including: .1 a thorough understanding of Electronic Navigational Chart (ENC) data, data accuracy, pres- ~ 60 ~ INSLC16 Proceedings Dalian, China July 12-16, 2010 entation rules, display options and other chart data formats .2 the dangers of over reliance .3 familiarity with the functions of ECDIS required by performance standards in force Proficiency in operation, interpretation, and analysis of information obtained from ECDIS, including: .1 use of functions that are integrated with other navigation systems in various installations, including proper functioning and adjustment to desired settings .2 safe monitoring and adjustment of information including own position, sea area display, mode and orientation, chart data displayed, route monitoring, user-created information layers, contacts (when interfaced with AIS and/or radar tracking) and radar overlay functions (when interfaced) .3 confirmation of vessel position by alternate means .4 efficient use of settings to ensure conformance to operational procedures, including alarm parameters for anti-grounding, proximity to contacts and special areas, completeness of chart data and chart update status, and backup arrangements .5 adjustment of settings and values to suit the present conditions .6 situational awareness while using ECDIS including safe water and proximity of hazards, set and drift, chart data and scale selection, suitability of route, contact detection and management, and integrity of sensors Criteria for evaluating competence Monitors information on ECDIS in a manner that contributes to safe navigation Information obtained from ECDIS (including radar overlay and/or radar tracking functions, when fitted) is correctly interpreted and analyzed taking into account the limitations of the equip- ment, all connected sensors (including radar and AIS where interfaced), and prevailing circumstances and conditions Safety of navigation is maintained through adjustments made to the ship’s course and speed through ECDIS-controlled track keeping functions (when fitted) Communication is clear, concise and acknowledged at all times in a seamanlike manner 1.2. Table A-II/2: Navigation using ECDIS Knowledge, understanding and proficiency Management of operational procedures, system files and data, including: .1 manage procurement, licensing and updating of chart data and system software to conform to established procedures .2 system and information updating including the ability to update ECDIS system version in accordance with vendor’s product development .3 create and maintain system configuration and backup files .4 create and maintain log files in accordance with established procedures .5 create and maintain route plan files in accordance with established procedures .6 use ECDIS logbook and track history functions for inspection of system functions, alarm settings and user responses .7 use ECDIS playback functionality for passage review, route planning and review of system functions Criteria for evaluating competence Operational procedures for using ECDIS are established, applied, and monitored Actions taken to minimize risk to safety of navigation ~ 61 ~ INSLC16 Proceedings Dalian, China July 12-16, 2010 1.3. Commentary on baseline competency The intent and specific criteria of the revised STCW Code on ECDIS is plain to see. Any Officer in Charge of a Navigational Watch (OIC) must be able to monitor information on ECDIS in a manner that contributes to safe navigation, while any Chief Mate or Master must be able to also establish, apply and monitor operation procedures for using ECDIS. Where these skills are acquired and assessed using approved ECDIS simulator training, the simulator and the associated ECDIS course of instruction must be able to produce a navigational environment where unsafe events will occur when ECDIS information is incorrectly monitored and when ECDIS operating procedures are inadequately applied. In other words, it is the unsafe navigational events in a given scenario, where preventable by proper operation of ECDIS, which should determine the baseline competency. Interwoven with this is the opportunity to practice and apply the very many specific ECDIS functions that add up to proper operation, and which will help avoid distraction by ECDIS through complete familiarity. The methods for demonstrating this competence require either underway experience or ECDIS simulator training. The existing IMO Model Course 1.27 is long overdue for revision for its lack of specifying simulation standards, its lack of formalizing competency standards, and its lack of specific learning goals for time allotted to practical exercises [3]. Any proposed revision should establish the high importance of replicating that underway navigational experience with ECDIS in simulator training. It is fair to state that ECDIS simulator train- ing that does not meet the "underway standard" cannot possibly satisfy the letter or intent of the revised STCW Code. 2. TOWARD ECDIS-BASED STANDARDS FOR NAVIGATION SIMULATION 2.1. Simulation in STCW As given under Section A-I/12, “Standards governing the use of simulators”, the STCW-95 Code requires that any simulator used for mandatory simulator-based training "be capable of simulating the operating capabilities of shipboard equipment concerned, to a level of physical realism appropriate to training objectives and assessment objectives." The content of this section of the Code has not been altered from STCW-95 in the comprehensive review of 2010 [4]. As given under Section B-I/12, “Guidance regarding the use of simulators”, performance standards for non-mandatory simulation equipment used for training and/or assessment of competence or demonstration of skills in navigation and watchkeeping and in ship handling and maneuvering should "provide a realistic visual scenario by day or by night ..." This is in addition to meeting all applicable performance standards set out in section A-I/12. Here also, the content has not been altered from STCW-95. However, as given under Section B-I/12, “Guidance regarding the use of simulators”, significant details regarding training and assessment in the operational use of the ECDIS have been added [5]. This section recognizes the necessity of gaining practical skills on individual ECDIS simulators. Its source is an IMO Circular from 2001 [6]. That guidance predates the cost-effective technology now available for individual ownship ~ 62 ~ INSLC16 Proceedings Dalian, China July 12-16, 2010 workstations integrated with the visual scene and the full emulation of all instrumentation plus the inclusion of type-approved ECDIS. Nonetheless, its intent to promote navigational understanding and skill with ECDIS is fully relevant. "The main objective of simulator exercises is to ensure that trainees understand their responsibilities in the operational use of ECDIS in all safety-relevant aspects and are thoroughly familiar with the system and equipment used" [6, para. 47, as proposed]. own live ECDIS to operate and their own vessel to operate, the navigational functions of which are fully integrated. This approach is necessary for the progressive development of skills and for the assessment of their achievement in the underway context of navigation. There is, in this approach to the training, no place for multiple users on one ECDIS, or an ECDIS that is not receiving underway sensor input, or an underway environment devoid of an integrated visualization of the scene. 2.2. ECDIS simulation 2.3. Toward ECDIS simulation standards ECDIS is unlike any other navigational aid in its ability to capture the user's attention for extended periods of time. Absorption in menus, functional tasks, and potential ambiguities in information can swiftly undermine the safety of navigation even where the user has the best intentions of standing a vigilant watch. ECDIS is an encyclopedia of live information both graphic and textual, in the medium of a PC display whose near total capture of the digital age has evolved on the basis of surfing, browsing, interacting, storing, retrieving - in general, of being captivating. Effective ECDIS training demands that each user develops the practical discipline of approaching the ECDIS with a single question to resolve at any one time - confirm position, present track tendency, confirm progress in a critical turn, next course to steer, ETA to upcoming waypoint, chart information, contact data, adjustment to display, adapting the route to changes in plans, etc. Developing this single point query approach helps ensure that the ECDIS serves as an aid to navigation, such that its reliability can be efficiently verified. For this form of training, each trainee in an ECDIS course following such a revised approach should be, through simulation, provided with their The following description of ECDIS simulation is meant to apply in each setting where ECDIS training is conducted: ECDIS simulation equipment shall be capable of simulating the operational capabilities of ECDIS which meet all applicable performance standards adopted by the Organization, and shall incorporate facilities to: 1) handle ENC data, licenses and update files 2) interface with the following emulated or OEM equipment: a) position indicator, including emulation of fix quality and, in the instance of GNSS, satellite constellation b) alternative position source, preferably a second GNSS unit c) heading indicator, both true and magnetic, including graphic course recording d) speed indicator e) depth indicator f) ARPA tracked target data g) AIS, including control of static data and messaging h) radar data including emulated raw video, cursor, EBL and VRM i) autopilot capable of control by heading (course), COG, and track, where moni- ~ 63 ~ INSLC16 Proceedings 3) 4) 5) 6) 7) 8) 9) 10) Dalian, China July 12-16, 2010 tored track may be provided through both instructor control and alternatively through ECDIS at ownship provide radar overlay, with functions operating independently from ownship radar provide audio for navigation and assessment systems when fitted provide VHF communications between all ownships and instructor permit all ownships to interact with one another, depending on the exercise design provide for viewing visual scene by scrolling in all directions horizontally and vertically, or horizontally without scrolling where fixed visual channels cover 360 degrees provide for taking accurate visual bearing provide adequate and well-lit surface for plotting navigation information on paper charts as the required means of back-up required for single ECDIS installation permit simultaneous navigation on paper charts associated with area databases as appropriate to ECDIS watchstanding graded to NTPro 5000 in 2011). Each lab is designed to run any combination of interacting ownships, depending on the exercise design. Each ownship in the integrated navigation labs consists of the visual scene on a 42” display, conning including autopilot with track control and other navigation instruments, hardware conning controls, ECDIS and radar, VHF, and chart table. Sound is also integrated. Each ownship is in a 9ft x 5ft cubicle, where the conning officer stands. There is room for a team of two. The coaching on equipment and exercise review are conducted in the open lab. The real learning takes place in the ownships. The integration means there is always a visual scene for the underway context in the classrooms and in the semi-isolated ownship labs. The multiple ownships allow for a wide variety of scenarios, from single ownship exercises run simultaneously in parallel, to a single exercise with all ownships interacting as well as with target vessels from the instructor. 2.4. Integrated navigation labs at USMMA ECDIS is an integrating device, and its use must be mastered in solo watchstanding. It follows that ECDIS simulation must suit solo watchstanding training, and this certainly demands the inclusion of the visual scene at all times. In order to accomplish the goals of training and assessment included in any appropriate revision to the ECDIS Model Course, ECDIS simulation should adhere to the foregoing as performance standards. In place at the United States Merchant Marine Academy (USMMA) at Kings Point, New York, are 2 integrated navigation labs each with 16 isolated ownships (32 total), a classroom (open lab) with 24 workstations, and another classroom with 18 workstations. All four labs have the same fully integrated simulation software package installed (Transas’ NTPro 4000, version 4.62, to be up- There is a fundamental difference between the demonstration and practice of ECDIS functionalities, on the one hand, and the unassisted application of ECDIS while faced with navigational decision making, on the other hand. These are separate learning experiences and should for the most part be conducted separately. Nonetheless, all ECDIS workstations should have the same fully integrated simulation software package. Whether the lesson is being conducted in a classroom or open environment or the exercise is being conducted in the semi-isolation of a multiple ownship navigation lab, each network is designed to run any combination of interacting ownships, depending on the exercise design [2]. ~ 64 ~ INSLC16 Proceedings 3. UNDERWAY Dalian, China July 12-16, 2010 EXERCISES FOR ECDIS • • • • NAVIGATION It is well established in practical training schemes that development of complex skill sets should follow a certain progression to allow for reinforcement and to achieve a stated and measurable outcome with consistency. As applied to a 40-hour training scheme in ECDIS navigation, the progression should ideally follow the sequence: 1) Elements of ECDIS (10.5 hours) 2) Watchstanding with ECDIS (8 hours) 3) ECDIS route planning (7 hours) 4) ECDIS charts, targets & system (10 hours) 5) ECDIS responsibility (4.5 hours) Sample exercises are outlined below for each of these stages. 3.1. Elements of ECDIS (2.5 hours of solo underway exercises) Exercise 1, option (A): Piloting in open waters for 1 ownship (assign 12 times simultaneously) Task groups: • Safe navigation through skills integration • Adjust heading to maintain track Outcomes: • Navigate with ECDIS • 1-person watch • Cooperative navigation • Visual lookout • Limited maneuvering Exercise 1, option (B): Piloting in open waters for 1 ownship (assign 12 times simultaneously) Task groups: • Safe navigation through skills integration Outcomes: • Navigate with ECDIS • Monitor SOG & COG Apply standing orders: > 1 nm off land Keep vessel in counter current 1-person watch Visual lookout 3.2. Watchstanding with ECDIS (2 hours of solo underway exercises) Exercise 2, option (A): Piloting in open waters for 12 ownships (in a single exercise of inbound/outbound in TSS) Task groups: • Safe navigation through skills integration • Adjust heading to maintain track Outcomes: • Navigate with ECDIS • 1-person watch • Cooperative navigation • Visual lookout • Maneuvering Exercise 2, option (B): Piloting in open waters for 6 ownships (inbound in TSS, assigned 2 times) Task groups: • Safe navigation through skills integration • Adjust heading to maintain track in strong leeway Outcomes: • Navigate with ECDIS • Detect leeway by radar and visual plotting • 1-person watch • Cooperative navigation • Visual lookout • Maneuvering Exercise 2, option (C): Piloting in open waters for 12 ownships (large area TSS) Task groups: • Safe navigation through skills integration • Adjust heading to maintain track • Observe TSS rules • Alarm zones (triggered by maneuvers) ~ 65 ~ INSLC16 Proceedings Dalian, China July 12-16, 2010 Outcomes: • Navigate with ECDIS • 1-person watch • Cooperative navigation • Visual lookout • Maneuvering 3.3. ECDIS route planning (2 hours of solo underway exercises) Exercise 3, option (A): Piloting semi-confined waters for 12 ownships Task groups: • Safe navigation through skills integration • Adapt instruments to conditions • Adapt track-keeping to workload Outcomes: • Navigate with ECDIS • 1-person watch • Cooperative navigation • Visual lookout • Maneuvering in Exercise 3, option (B): Anchoring in semi-confined waters for 6 ownships (assign 2 times) Task groups: • Safe navigation through skills integration • Create and apply route • Create ECDIS user layer • Adjust heading to maintain track • Observe anchorage rules • Apply ECDIS user layer for anchoring Outcomes: • Navigate with ECDIS • 1-person watch • Cooperative navigation • Visual lookout • Maneuvering 3.4. ECDIS charts, targets & system (2 hours of solo underway exercises) Exercise 4, option (A): Piloting in open waters for 6 ownships 2-person watches (assign 1 time) Task groups: • Safe navigation through skills integration • Adapt ECDIS route for Search & Rescue • Find man overboard Outcomes: • Share ECDIS route • 2-person watch • Cooperative navigation • Team & communications • Visual lookout • Maneuvering Exercise 4, option (B): Piloting in confined waters for 12 ownships Task groups: • Safe navigation through skills integration • Adapt instruments to conditions • Adapt track-keeping to workload • Create and apply route • Adjust heading to maintain track Outcomes: • Navigate with ECDIS • 1-person watch • Cooperative navigation • Visual lookout • Maneuvering Exercise 4, option (C): Piloting in open waters for 6 ownships (assign 2 times) Task groups: • Safe navigation through skills integration • Adapt instruments to conditions • Create and apply route • Adapt track-keeping to workload • Adjust heading to maintain track • Observe TSS rules • Alarm zones Outcomes: • Navigate with ECDIS • 1-person watch ~ 66 ~ INSLC16 Proceedings Dalian, China July 12-16, 2010 • Cooperative navigation • Visual lookout • Maneuvering 3.5. ECDIS underway evaluation (2.5 hours of solo underway) Exercise 5, Evaluation: Piloting in semi-confined waters for 6 ownships solo watch (assign 1 time) Task groups: • ECDIS Competency Assessment (use of all ECDIS functions is observed) • Safe navigation through skills integration • Adapt instruments to conditions • Create route (from a prior lesson) and apply route • Modify route underway • Adapt track-keeping to workload • Comply with VTS and TSS rules Outcomes: • Navigate with ECDIS • 1-person watch • Cooperative navigation • Visual lookout • Maneuvering The culminating exercise is run as a formal demonstration of competency, and is scored according to a detailed matrix. This scoring methodology will be examined in Sections 6 & 7. 3.6. Explanation of hours The tasks and task groups in the foregoing scheme of exercises progress in complexity in a combination of ways. The later tasks themselves are more demanding of time and involve more steps. The groupings of tasks involve progressively more variety. The application of later tasks occurs in more demanding underway circumstances. The outcomes, while retaining consistency, become progressively more challenging to maintain as the navigational workload increases. The significance of this process in mastering ECDIS is that navigational safety remains the central outcome, and can therefore stand as a baseline measure of ECDIS competency. Typically, a professional training course is designed to suit a 40-hour work week. This is minimally feasible with gaining proficiency in ECDIS navigation, particularly when the basic skill sets of navigation are already familiar, such as plotting positions determined visually and by radar, maintaining a rough log, track keeping in coastal waterways, and maneuvering for traffic. When these types of skills are unfamiliar, they may be learned quite optimally during training in ECDIS navigation, but more time is needed. For that reason, added exercise options are provided for each stage in the training scheme. Whether maintaining the 40 hour schedule or a 50 hour schedule (such as USMMA), the percentage of solo underway time, including the evaluation, should be kept to about 30% (or 12 of 40 hours, and 15 of 50 hours). Preparation time for solo trainees in the simulator should be limited to 20 minutes prior to getting underway. This allows for route familiarization and instrument setups. In the instance of the underway evaluation, the ECDIS route plan should be created and saved in classroom time, and evaluated separately for completeness and accuracy. Each trainee installs their own validated route in the preparation period of the underway evaluation. This act personalizes that exercise, and requires that the trainee modifies a portion of the route for the actual position of each one’s ownship. This accomplishes the A-II/2 Management level competency: “create and maintain route plan files in accordance with established procedures”. ~ 67 ~ INSLC16 Proceedings Dalian, China July 12-16, 2010 4. ECDIS TASK GROUPS AND OUTCOMES faced with ECDIS Specific learning objectives can be achieved by clustering task elements that support the overarching goal of safe navigation. The purpose of grouping tasks is to help the instructor and each trainee maintain a focus on the intended outcomes for an exercise. This approach allows tasks to shift in importance as an exercise unfolds. Trainees make decisions based on information that may become apparent depending on the skill with the given tasks. It is crucial that the process of integrated ECDIS navigation training is at no time reduced to a set of particular functions performed on ECDIS to the neglect prudent watchstanding. Concentration on the specific tasks for a given stage in the ECDIS training is necessary, but not to the exclusion of other desired outcomes, such as looking out, plotting on paper charts as the backup system, keeping a detailed rough log, and scanning source sensors. Task grouping requires more underway exercise time than ECDIS courses are typically designed for. But the result is a measurably appropriate reliance on ECDIS precisely due to the mastery of outcomes constituting safe navigation. 4.1. Explanation of ECDIS task groups The task groups shown in Table 1 are meant to isolate significant skill sets directly related to effective ECDIS navigation. The groups that focus on ECDIS account for using critical interfaces on ECDIS (1), procedural setups (3), relevant info (4), all position options (5), and making adjustments (6). The other groups account for externals critical to effective ECDIS navigation. Table 1 No. 1 ECDIS Task Groups Task Group Use all navigation systems inter- Focus On ECDIS 2 3 4 5 6 7 8 Verify settings of interfaced sensors Check that setting conform to procedures (exercise briefing) Monitor information on ECDIS for safe navigation Verify position by alternate means Adjust settings to suit conditions and adapt to changing conditions Use ECDIS-managed track control autopilot Other sensors On ECDIS On ECDIS On ECDIS On ECDIS On autopilot Maneuver according to accepted Conning & navigational practice and with situational regard to COLREGS awareness Situational 9 Responses to role play with regards to ECDIS use awareness with VTS on ECDIS 10 Manage AIS and assess environ- On ECDIS mental conditions (optional) The last group (On ECDIS: manage AIS and assess environmental conditions) is considered optional for two reasons: First, AIS messaging and environmental factors such as tidal current cycles may not suit all evaluation exercise scenarios, and second, not all Integrated Multiple Ownship Simulators can be expected to include this functionality, although they should since these are commonly a vital and integral aspect of navigation in confined and semi-confined waters (see Section 3.3 above regarding ECDIS simulation standards). The scoring of an Optional Task Group will be discussed in Section 7 below. 4.2. Weighted averaging for Task Groups It is anticipated that a non-normative weighted average would be preferred, such that (1) would be some value > 1, and that the weights for the Task Groups could be altered in order to create a ~ 68 ~ INSLC16 Proceedings Dalian, China July 12-16, 2010 scoring system that accurately represents the desired balance of emphasis between tasks performed directly on ECDIS and other navigational tasks not performed on ECDIS. The formula used for weighted averages is standard for non-blank values, where the divisor of 10 represents the number of Task Group weights as presented in Section 5.1 and in Table 2 below: ℎ = ∑ (1) The weights chosen for the instance of the underway competency score sheet presented in Table 2 and Figure 1 below emphasize the relative importance of skills performed on ECDIS, but still allows for the consideration in the scoring of important navigational tasks not performed directly on ECDIS, but nonetheless central to maintaining the safety of navigation. The application of weighted averages in the evaluation tally results in a change in a trainee’s overall score percentile, while the non-zero average score itself remains unaffected. The change in percentile is slight when the weight of one or another Task Group is varied even by a factor of 2 or 3. For example, when the weight of a Task Group is increased from 1.0 to 2.0, the percentile score for a high-scoring trainee increases from 91.1% to 91.8%. However, the percentile score increases by only 0.2% for a trainee scoring moderately in that particular Task Group. 4.3. Explanation of outcomes Perhaps the most significant challenge facing instructional design for ECDIS use is the recognition that the mastery of navigational functionality of the device does not equate to skillful and safe navigation with ECDIS. Because ECDIS is an integrative device by its structure and purpose and placement in integrated bridge systems, the use of it and its use in the various acts of navigation cannot be separated for very long. The consequence of training on ECDIS outside of its navigational context is to develop skills that depend upon not having any other navigational distractions. There can be no argument that this is the most undesirable outcome imaginable for training in the use of ECDIS. Regrettably, this particular approach, common to part-task training everywhere, has taken hold in most instances of ECDIS certified training worldwide. With the advent of STCW-2010, however, the fundamental competence is now defined as a navigation function at both levels: Maintain the safety of navigation through the use of ECDIS [1]. As with many other STCW navigational competencies, this desired outcome of ECDIS use must be demonstrated as a solo act, and that fact demands that safe navigation with ECDIS is practiced and ultimately evaluated in a solo navigational environment. When ECDIS competence is kept in its integrated context, it becomes clear that many general aspects of navigation must be evaluated, as well as the use of specific ECDIS functions at the right time and place. Indirectly, the behavior that is being assessed is the trainee’s ability to divide attention evenly amongst all navigational aids and systems, not least being the visual scene. Taking the STCW criteria for evaluating ECDIS competence as desired outcomes, then Table A-II/1 requires: • Monitoring appropriate information • Interpreting that information correctly • Controlling the vessel’s speed and autopilot • Utilizing effective communications, and Table A-II/2 requires (in the same evaluation): • Making and using operational procedures • Making navigation safe by minimizing risk Such training should encourage the application of judgment and development of situational ~ 69 ~ INSLC16 Proceedings Dalian, China July 12-16, 2010 awareness. Route monitored Route waypoints named & 5. ECDIS COMPETENCY MATRIX displayed Route XTE zones set for 5.1. Competency score sheet passage Ship Time Zone set The score sheet shown in Table 2 below is meant for each trainee in the underway evaluation. Instructions: Score 3 (all), 2 (some), 1 (not enough), leave blank if n/a. Scoring 0 on [*] (in Task group 8) represents a Critical Fault. In that instance, the trainee should retake the evaluation because it is always plausible that better use of ECDIS would have mitigated the dangers related to close quarters and proximity to shoal water. An important consideration used in this scoring scheme is to avoid a penalty for un-scored tasks, unless the entire Task Group is un-scored. Paper chart backup: route, DR, notes, etc. 4 12 13 14 15 16 Monitor information on ECDIS for safe navigation Weight: 1.5 EC look ahead occasional 17 Adjust EC scale occasional 18 EC chart choice 19 Route Data panel displayed Query tracked targets, chart objects Acknowledge alarms 5 20 21 22 Verify position by alternate means Weight: 1.0 Table 2 ECDIS Underway Score Sheet Radar cursor on EC 23 PS2 into EC (if on) 24 Use all navigation systems interfaced with ECDIS Radar overlay occasional 25 Weight: 1.5 Paper chart: plot visual & Tasks by group 1 score Comment No. Tag targets 1 radar fixes Acquire radar targets 2 6 DGPS on PS1 3 2 Adjust settings to suit conditions and adapt to changing conditions Weight: 1.5 Verify settings of interfaced sensors Set day/night palette Weight: 1.0 GPS HDOP recorded 4 Custom layers set for low Sounder set to DBK 5 clutter Speed log set to BT 6 Dual panel - full screen plus PS2 initialized 7 Multipanel Radar displays route 8 Route plan schedule set for AP settings adjusted for 9 Track Control 3 26 ETAs ETA & speed made good Check that setting conform to procedures (exercise displayed on Multipanel briefing and Standing Orders on Rough Log sheet) CU/NU and RM/TM modes Weight: 1.5 choices 27 28 29 30 31 32 Anti-grounding parameters 10 Revise route as needed 33 Vector length set 11 Add Info layers used & 34 ~ 70 ~ INSLC16 Proceedings Dalian, China July 12-16, 2010 weighted averages for each Task Group (see Section 7 below). revised Radar overlay used 7 35 Use ECDIS-managed track control autopilot Weight: 0.5 Track Control occasional Track Control turns use safe radius on EC 8 36 37 Maneuver according to accepted navigational practice and with regard to COLREGS Weight: 1.0 Effective VHF use 38 Proactive passing 39 [*] Safe CPA always 40 Safe turns executed 41 Safe speed always 42 [*] Safe DBK always 43 Navigation lights on 44 Fog signals used 45 Rough Log complete 46 9 Responses to role play with regards to ECDIS use Weight: 1.0 Provide ETA on request Provide revised plan on request Provide requested environmental info Call VTS on request 47 48 Fig. 1 Sample underway score sheet 49 50 10 Manage AIS and assess environmental conditions (optional) Weight: 0.5 AIS messaging managed from EC RIB targets tracked Display tide & current info on EC Query tide & port info on EC 51 52 53 54 The scoring could be kept simple with the use of integers, or decimal values could be used for finer differentiation. Consistency in either case is essential. The values recorded in the underway score sheet are applied in the evaluation tally with Figure 1 is a sample score sheet for eight trainees each assigned to solo navigation in one underway evaluation exercise where all eight ownships are fully interactive (see Section 4.5 above). In this instance, the scenario was placed in a wide, deep and lengthy waterway with a Traffic Separation Scheme (TSS) and some crossing ferries and additional target vessels in the TSS. Four ownships were northbound, while the other four were southbound. Each trainee created a validated route prior to the evaluation. The trainees set up their integrated systems and paper charts for 30 minutes. Underway time lasted about 2 hours. ~ 71 ~ 5.2. Instructor’s methods of observation INSLC16 Proceedings Dalian, China July 12-16, 2010 As alluded to in Section 5.3 above, safe solo ECDIS navigation should include (but is not limited to) passage planning, underway transits (near-coastal, in TSS and their junctions, in-port, entering anchorages), track-keeping, use of Track Control in autopilot, and traffic management. In connection with ECDIS generally, this entails maintaining a visual lookout, situational awareness, use of backup systems, application of all available means, communication, log keeping, and slow-speed control of ownship. Specifically with regards to ECDIS, this means appropriate chart portfolios, electronic updates, route planning safety checks and route monitoring, sensors working and connected and used, anti-grounding alarms set, an informative and unambiguous display, user layers, settings adapted to changing conditions. In order for an instructor to score a given trainee’s application of such general as well as specific skills, some method of observation must be developed. This is very effectively done using remote control software (software that can display the screen of another computer), such that the instructor’s mouse pointer is not active on the trainee’s display. At an instructor’s station equipped with an additional monitor or two, a window into each trainee’s ECDIS can be opened. This allows the sequential monitoring of the trainees’ ECDIS use without any form of intrusion. Additionally, the instructor should observe automatically recorded events (depending on the simulator’s functionality), such as each trainee’s dangerous approaches to other vessels, use of autopilot, navigation lights, fog signals, depth sounder, AIS messaging, etc. The trainees should also be required to keep a rough log, which includes “Standing Orders” to be incorporated into ECDIS use as ensuring “conformance to operational procedures” (Table A-II/1), such as minimum CPA, speed requirements, backup plot in- terval, reminders to check on tides and currents, and to observe COLREGS, and to keep a vigilant visual lookout. The log can include entries for recording the alternative fix information, sensor fix quality, and other details that would augment the completeness and workload of paper chart plotting as the backup system. 6. SCORING FOR PROFICIENCY IN ECDIS NAVIGATION 6.1. Trainee scoring For each trainee, a score is delivered to each Task Group on the evaluation tally (Fig. 2) directly from the underway score sheet: = (2) ∗ As noted above, blank entries for specific tasks in the underway score sheet are disregarded in this initial averaging per Task Group (see Section 6.2). The intent is to provide a flexible scoring system in the event that certain devices are not included in the integrated simulation. On the other hand, simply entering a zero value in place of a blank will cause that entry to be included in the average, significantly penalizing omitted behaviors, and rendering the scoring system far more rigorous. Such a choice should be left to the training course developer, and should not be directly dictated by a scoring system. The formulation here yields to the scorer (0 or blank). Each trainee’s total score is represented by the average of non-zero values: =∑ ∑ [ .# ] (3) It is anticipated that Task Group #10 may be optional as noted in Section 5.1, resulting in no score given for any of the four tasks in that group. When that is not the case, those scores should be included, and the sum of weights should allow inclusion of the inclusion of the weight for Task ~ 72 ~ INSLC16 Proceedings Dalian, China July 12-16, 2010 Group #10. The achievement of each trainee by percentile will is determined by utilizing 3 as the maximum possible score for any given task in the score sheet: %= ( ) For each Task Group across all trainees: (5) Max score = For the non-zero values of each Task Group: (6) vg = Avg (3) For the tally of each Task Group by percentile: (4) %= 6.2. Tallying the Task Groups ( ) For the overall evaluation score: There is considerable value in tracking the average scores by Task Groups over a succession of underway evaluation exercises, in particular to aid in assessing and validating the outcomes of the ECDIS navigation training overall. For example, certain Task Groups may show under-performance over several iterations, indicating the need for revised emphasis of certain tasks in the underway context. Likewise, the overall score for the evaluation exercise can be determined and tracked. Fig. 2 Sample ECDIS evaluation tally =∑ ∑ [ .# ( ) ] (7) (8) 7. CONCLUSION In an effort to provide some validation of the foregoing, a sample proficiency evaluation matrix is examined, preceded by a description of suitable exercises, tasks, task groups, and navigational outcomes. The combination of written assessment for certain principles of electronic chart systems and their use in navigation, along with the evaluation of proficiency while in solo underway contexts should provide the high value appropriate to contemporary marine navigation training. There can be no doubt that adequate ECDIS simulation performance standards must be uniformly applied in order to achieve internationally consistent competence in ECDIS navigation. Demonstrating safe and effective use of ECDIS demands foremost the demonstration of safe navigation. The assessment of ECDIS use therefore requires a detailed analysis of ECDIS functions applied in the context of coastal and confined navigation in visual ship simulation. This is best accomplished through solo navigation exercises where an instructor can observe each trainee’s navigation using ECDIS in a non-disruptive yet detailed manner. ~ 73 ~ INSLC16 Proceedings Dalian, China July 12-16, 2010 REFERENCES [1] IMO STW. “Comprehensive Review of the STCW Convention and the STCW Code: Chapter II of the STCW Convention and Code.” STW 41/7/3, 29 September 2009, pp. 8-9, 21, 36 [2] Christian Hempstead. “Integrated Multiple Ownship Simulation: Where the learning takes place”. INSLC 15, July 2008 [3] IMO ECDIS Training. IMO Model Course 1.27: The Operational Use of Electronic Chart Display and Information Systems (ECDIS). ISBN 92-801-6112-1, International Maritime Organization, London, 2000 [4] IMO STCW. International Convention for Standards for Training, Certification and Watchkeeping (STCW); Consolidated text of the 1978 STCW Convention with the Final Act of the 1995 Conference of Parties, International Maritime Organization, London, 1995 [5] IMO STW. “Report to the Maritime Safety Committee”. STW 40/14/Add. 1, Annex 3, paragraphs 36 through 66, “Training and assessment in the operational use of ECDIS”, as proposed. International Maritime Organization, London, 2009. [6] IMO MSC. “Interim guidance on training and assessment in the operational use of the ECDIS simulators”. STCW.7/Circ.10, International Maritime Organization, London, 2001. Christian Hempstead, Master Mariner; MNI, MA, 19 years as deck officer with SeaRiver Maritime Inc. (formerly Exxon Shipping Co.) including 5 years responsible for type-approved ECDIS and other electronic chart systems as watchstanding navigation officer on board ship; 10 years USCG certified ECDIS instructor (at USMMA, GMATS, SUNY, MITAGS, & PMI), Transas Ltd certified instructor of NaviSailor ECS & ECDIS; developer and author of ECDIS certified training at USMMA, GMATS, MITAGS, & PMI (author of first approved ECDIS course in the US, 2000) including design and implementation of ECDIS simulation training environments; 6 years Associate Professor at USMMA including leading innovations in large-scale Integrated Multiple Ownship Simulation, and providing the means and ability to institute USCG certified training in ECDIS navigation for all deck cadets at USMMA beginning in 2011. AUTHOR’S INFORMATION AUTHOR’S BIOGRAPHY ~ 74 ~ Christian Hempstead United States Merchant Marine Academy Department of Marine Transportation 300 Steamboat Rd., Kings Point, NY 11024 Office: (516) 773-5286; Fax: (516) 773-5668 M: (360) 286-1932; E: hempsteadc@usmma.edu