PLOS ONE

RESEARCH ARTICLE

Exploring time series of hyperspectral images

for cold water coral stress response analysis
Daniel Langenkämper ID1*, Aksel Alstad Mogstad4, Ingrid Myrnes Hansen2,
Thierry Baussant ID3, Øystein Bergsagel2, Ingunn Nilssen4, Tone Karin Frost4,
Tim Wilhelm Nattkemper1
1 Biodata Mining Group, Bielefeld University, Bielefeld, Germany, 2 Ecotone AS, Trondheim, Norway,
3 NORCE Norwegian Research Centre, Randaberg, Norway, 4 Equinor ASA, Research and Technology,
Trondheim, Norway

a1111111111 * dlangenk@cebitec.uni-bielefeld.de
a1111111111
a1111111111
a1111111111
a1111111111 Abstract
Hyperspectral imaging (HSI) is a promising technology for environmental monitoring with a
lot of undeveloped potential due to the high dimensionality and complexity of the data. If
temporal effects are studied, such as in a monitoring context, the analysis becomes more
OPEN ACCESS challenging as time is added to the dimensions of space (image coordinates) and wave-
Citation: Langenkämper D, Mogstad AA, Hansen lengths. We conducted a series of laboratory experiments to investigate the impact of differ-
IM, Baussant T, Bergsagel Ø, Nilssen I, et al. ent stressor exposure patterns on the spectrum of the cold water coral Desmophyllum
(2022) Exploring time series of hyperspectral
pertusum. 65 coral samples were divided into 12 groups, each group being exposed to dif-
images for cold water coral stress response
analysis. PLoS ONE 17(8): e0272408. https://doi. ferent types and levels of particles. Hyperspectral images of the coral samples were col-
org/10.1371/journal.pone.0272408 lected at four time points from prior to exposure to 6 weeks after exposure. To investigate
Editor: Andrew Davies, University of Rhode Island, the relationships between the corals’ spectral signatures and controlled experimental
UNITED STATES parameters, a new software tool for interactive visual exploration was developed and
Received: March 29, 2022 applied, the HypIX (Hyperspectral Image eXplorer) web tool. HypIX combines principles
from exploratory data analysis, information visualization and machine learning-based
Accepted: July 19, 2022
dimension reduction. This combination enables users to select regions of interest (ROI) in
Published: August 8, 2022
all dimensions (2D space, time point and spectrum) for a flexible integrated inspection. We
Copyright: © 2022 Langenkämper et al. This is an propose two HypIX workflows to find relationships in time series of hyperspectral datasets,
open access article distributed under the terms of
namely morphology-based filtering workflow and embedded driven response analysis work-
the Creative Commons Attribution License, which
permits unrestricted use, distribution, and flow. With these HypIX workflows three users identified different temporal and spatial pat-
reproduction in any medium, provided the original terns in the spectrum of corals exposed to different particle stressor conditions. Corals
author and source are credited. exposed to particles tended to have a larger change rate than control corals, which was evi-
Data Availability Statement: The hyperspectral dent as a shifted spectrum. The responses, however, were not uniform for coral samples
data is available for download at https://ani.cebitec. undergoing the same exposure treatments, indicating individual tolerance levels. We also
uni-bielefeld.de/hypix.h5. The HypIX tool is
accesible at https://webserver.biodtmin.projects.bi.
observed a good inter-observer agreement between the three HyPIX users, indicating that
denbi.de/hypix using the username coral and the the proposed workflow can be applied to obtain reproducible HSI analysis results.
password hypercoral2020 and the source code is
available at https://github.com/
BiodataMiningGroup/HyPIX.

Funding: The study was financed by Equinor.

Equinor Ventures is one of the main shareholders
of Ecotone AS. IMH is a minor shareholder of

PLOS ONE | https://doi.org/10.1371/journal.pone.0272408 August 8, 2022 1 / 19

PLOS ONE Exploring time series of hyperspectral images for cold water coral stress response analysis

Ecotone AS. Ecotone AS is the owner of patent no. Introduction

NO/EP2286194 titled “Underwater Hyperspectral
Imaging“. Ecotone sells scientific instruments for Hyperspectral imaging (HSI) originally used for remote sensing [1] has nowadays been estab-
underwater use under the product name lished in multiple fields and applications, such as medicine [2], agriculture and forestry [3],
Underwater Hyperspectral Imager (UHI). Ecotone food quality [4, 5], structure monitoring [6], material research [7], mining applications [8] and
AS has two pending patent applications, IMH is environmental monitoring [9, 10]. Organisms in particular exhibit a specific reflectance spec-
involved as inventor. Equinor funded a project at
trum upon illumination, mainly caused by their pigment composition [11]. This is known as
the Biodata Mining Group, but these funding was in
no way linked to the outcome of this study. The their spectral signature and can be studied using HSI. For non-invasive monitoring of environ-
funders had no instructive role in study design, mental stress, HSI of selected organisms may have the potential to become one of the key
data collection and analysis, decision to publish, or tools. Most of the related studies have been carried out on photosynthetic organisms, especially
preparation of the manuscript. on terrestrial vegetation and crop [12, 13] and shallow water coastal habitats. However, there is
Competing interests: The study was financed by limited knowledge on the spectral responses of marine organisms in the deep sea to environ-
Equinor. Equinor Ventures is one of the main mental or physiological stress.
shareholders of Ecotone AS. IMH is a minor Underwater hyperspectral imagers (UHI) for recording HSI have been implemented on
shareholder of Ecotone AS. Ecotone AS is the
underwater vehicles in situ for mapping extent and distribution of habitats [14, 15] such as
owner of patent no. NO/EP2286194 titled
“Underwater Hyperspectral Imaging“. Ecotone sells
deep-water corals and coralligenous habitats [16], red calcareous algae and associated fauna
scientific instruments for underwater use under the [17] or deep sea megafauna [18]. In recent years, few multivariate image analysis applications
product name Underwater Hyperspectral Imager [19] have shown that objects of interest can be identified based on their spectral signatures [20,
(UHI). Ecotone AS has two pending patent 21]. Further, UHI has been used to visualize the extent of particulate mud and drill cuttings on
applications, IMH is involved as inventor. Equinor the seafloor [22] with HSI. The UHI system has been utilized to detect changes in the health
funded a project at the Biodata Mining Group, but
condition of Desmophyllum pertusum (Linnaeus 1758) exposed to the hydrocarbon-2-Methyl-
these funding was in no way linked to the outcome
of this study. This does not alter our adherence to naphthalene in laboratory experiments [21]. This was done in order to establish a basic under-
all the policies on sharing data and materials. All standing on how a potential subsea oil spill could affect the spectral properties of corals, and
other authors declare no competing interests. assess if hyperspectral imaging could be a valuable tool to detect such changes in the spectrum.
Although the HSI technology has been established in many fields and a small number of
successful UHI applications have been reported, methodological challenges remain. Marine
environmental monitoring in the field using UHI has mainly been constrained to identifica-
tion and quantification of organisms and habitats, as well as the extent of particle sedimenta-
tion. There are no established methods for monitoring the physiological condition of deep-
water marine organisms in situ using UHI. The rationale behind utilizing UHI for this purpose
is that there must be a link between the health of the organism and its spectral signature. From
nature, it is well known that coloration may be linked to health. For instance, for tropical cor-
als, it is established knowledge that coral bleaching is a process triggered by the loss of the sym-
biotic photosynthetic algae component due to environmental unfavorable conditions, and this
is clearly observable using spectroscopy. The beforementioned study (Letnes et al [21]) showed
a link between spectrum and exposure level to hydrocarbon. Yet there is a limitation that there
is an absence of established health metrics or indices that correlate physiological parameters
with spectral response. Establishing such metrics requires ex situ experimentation with con-
trolled stress exposures, and tools for interpreting resulting spectral images. A second limita-
tion is the absence of an established analytical approach for assessing spectral changes over the
same geographical area over time using UHI. Both of these two knowledge gaps are addressed
by the works presented here.
Since there is no established methodological approach we propose a new software-based
approach for the comparative analysis of such a large number of HSI data sets. The analyses of
HSI data in general is not straightforward due to volume and dimensionality of the data, i.e. the
huge number of wavelength acquired simultaneously, even for a small number of data sets
recorded at one time point. The sheer dimensionality of the data typically makes an ad-hoc
visual analysis with standard methods from exploratory data analysis unfeasible. In order to
reduce the volume of data, it is often (spatially) filtered beforehand, by a heuristics-driven

PLOS ONE | https://doi.org/10.1371/journal.pone.0272408 August 8, 2022 2 / 19

PLOS ONE Exploring time series of hyperspectral images for cold water coral stress response analysis

selection of e.g. a small number of regions of interest (ROI) in the lateral image domain
(through selection of a polygon or a frame in the pixel grid). While this is a valuable established
approach it limits the chances to find new relationships outside these ROIs. Another way to
make HSI data easier to interpret, is the selective filtering of the high dimensional spectral
domain to a relative small number of bandwidths (intervals), referred to as multispectral imag-
ing [21, 23]. This is usually motivated by the observation that HSI data often feature low intrin-
sic dimensionality. The downside of this strategy is that it requires some domain knowledge
about potentially interesting bandwidths a priori which is often not available. Sometimes
dimension reduction techniques such as principal component analysis (PCA, [24]), multidi-
mensional scaling, isometric mapping, or t-distributed stochastic neighbor embedding (t-SNE,
[25]) are proposed [26, 27]. Due to the low intrinsic dimensionality, the spectral data can be
well embedded in a two-dimensional data space, suitable for visualisation, which additionally
reduces noise. A downside can be that the pixel-associated relation between the original spectral
signal space and the embedding space can be lost if not covered by an appropriate software
solution and that the much lower dimensional representation can be prone to misinterpreta-
tions due to the non-lossless embedding. Thus, a comparative analysis of the spectral signatures
from different HSI data sets recorded at different time points must be supported by a flexible
interactive data visualization tool, integrating the spatial, the spectral and the temporal domain.
And since HSI data is not straightforward to interpret, an interdisciplinary approach to its anal-
ysis, integrating knowledge from disciplines, such as (bio-)chemistry, marine biology, statistics
and computer science is required. As a consequence it is important, that the visualization tool
features good usability and can be used online as a web-tool without technical requirements.
To address the problems described above, we present and analyse HSI data from controlled
dose-response experiments on D. pertusum cold water corals (CWC) in laboratory tanks. The
reef building coral D. pertusum has a wide geographical distribution and can be find in most
part of the world, with the highest reported density in Norwegian waters. The reef offers habi-
tat for a variety of species, and is described as a hot-spot for biodiversity. D. pertusum is
defined as a ‘Near threatened’ species, mainly due to mechanical damages on the reefs caused
by past and ongoing trawling. Further, there are concerns on the species ability to tolerate
increasing ocean temperatures and ocean acidification [28]. D. pertusum is also located in
areas with petroleum activities and several studies have therefore been performed to monitor
potential impact of and to establish threshold for drilling particulates to these corals [29–33].
The objective of this study was to investigate the future potential of non-invasive HSI technol-
ogy for detection of changes in the health status by exploring potential systematic changes in
the spectral composition with various exposure concentrations of drilling particulates of adult
CWC D. pertusum as a response to certain exposure levels of suspended drilling particulates.
Drill particulates are a collective expression for drill cuttings (formation rock particulates gen-
erated during drilling) and weighting agents added to the drilling fluids, such as barite and
bentonite (use to lubricate and control the well during drilling) [34]. To this end coral samples
of both the white and the orange D. pertusum phenotype were exposed to drill cuttings, ben-
tonite and barite in a laboratory setting, mimicking a realistic offshore drilling scenario. After
exposure, a small HSI time series was recorded for the CWC samples.
Following these considerations we developed a new tool for HSI analysis, referred to as
HyPIX, presented in this paper. With this tool we investigated the HSI data collected in our
experiments with the main objective to find response patterns in the coral spectra related
to exposure type, levels and/or time points in the corals. A second objective was to find a
HyPIX—based workflow to spatially locate such responses in the sample (in case these
changes are not observed for all sample pixels) and to describe observations in the HSI that are
reproducible.

PLOS ONE | https://doi.org/10.1371/journal.pone.0272408 August 8, 2022 3 / 19

PLOS ONE Exploring time series of hyperspectral images for cold water coral stress response analysis

Fig 1. Overview of the Hypix workflow. a) Corals are kept in glass aquariums. b) Different aquariums are exposed with either barite, bentonite,
drill cutting or kept as is for the control experiment. c) The exposure happens at a timepoint T0.5. Before exposure (T0), directly after exposure
(T0.5) and after 2 (T1) and 6 weeks (T2) of recovery hyperspectral images are taken. d) The images are preprocessed. e) The outline of the corals is
annotated in Biigle 2.0 and the individual corals are cut out for further processing. f) Data mining and machine learning methods are applied and
saved to a file for g) Visualization and analysis of the results in the Hypix system. For more details on each step please have a look at the respective
subsection in the Method section.
https://doi.org/10.1371/journal.pone.0272408.g001

Materials and methods

An overview of the Hypix workflow is shown in Fig 1. For details consult the following text.

Experimental setup
Samples of coral colonies were collected from Trondheimsfjorden, Trondheim, Norway April
2018 using remotely operated vehicle (ROV). The sampling was carried out outside national
parks or other types of protected areas. The corals were transported to NORCE marine facility,

PLOS ONE | https://doi.org/10.1371/journal.pone.0272408 August 8, 2022 4 / 19

PLOS ONE Exploring time series of hyperspectral images for cold water coral stress response analysis

Mekjarvik, Randaberg where they were acclimatized for three weeks according to Baussant
et al. 2017 [35]. The sampling of corals for current study was done in conjunction with sam-
pling for study reported by Baussant et al. 2022 [29], where further details on sampling are to
be found.
The corals were contained in tanks with running seawater from Byfjord (Rogaland, Nor-
way) in stable temperature conditions (7.5˚C) and dark surroundings. They were fed with
Artemia larvae approximately two times per week. Prior to the experiment, the sampling mate-
rial was distributed to smaller coral nubbins, each nubbin with approximately 5–10 polyps.
Both white and orange morphs were used.
All exposure experiments (controls, barite (bar), bentonite (ben) and drill cutting (DC))
together with the HSI measurements were conducted from June 2018 to February 2019. For
each experiment a separate glass aquarium (50cm × 30cm × 30cm) with a flow rate of
180±10mL � min−1 sea water was used. In each, 5 (bar/ben) or 6 (DC) coral nubbins of both
color morphotypes (white and orange) were placed. Each nubbin was approx. 5–15cm in size
and had 5–10 polyps. The corals needed to be visible for two camera observations (a digital
consumer time-lapse camera and the hyperspectral imaging), and were therefore placed on a
diagonal line (approx. 30˚, S1 Fig in the supplementary). The bottom and backside of each
aquarium was covered with black insulating material to facilitate HSI and time-lapse imagery
analysis.
DC exposure. The exposure to suspended DC particles on the corals was made as
reported in [29]. DC was added in pulses of 4 hours followed by 4 hours with no DC over a
total duration of 5 days. A peristaltic pump added 2mL. DC stock min−1 from two 30L stock
tanks to the experimental aquariums, where DC mixed with the waterflow supplied at 200
±30mL � min−1 to achieve peak exposure nominal concentrations of 10, 30, 50 and 100mg � L−1
(please see Fig 1 of [29] for more details). Actual peak concentrations were measured from
point seawater samples (250mL to 1L, depending on expected concentration) collected each
day from the aquariums during the 4-hour exposure cycle 1 hour after DC pump start to
insure steady-state equilibrium was reached. Samples were filtered over a GF/F Whatman fil-
ter, and the filter was dried (60˚C) overnight or until constant weight to obtain the total parti-
cle weight from which the DC concentrations were derived.
There was a deviation from the nominal to the actual measured DC concentrations by
water filtration. The mean actual DC concentrations were 4, 6, 18 and 41mg � L−1, respectively
corresponding to DC nominal concentrations of 10, 30, 50 and 100mg � L−1.
Barite and bentonite exposure. Barite and bentonite exposure experiments were carried
out with the same experimental setup and exposure scenario as for DC, but the 10mg � L−1 con-
centration was not tested. The actual concentrations measured in the different barite/bentonite
treatments were also lower than the target concentrations: For barite, actual measured concen-
trations were 5.8, 18 and 54.7mg � L−1 (corresponding to respectively 30, 50 and 100mg � L−1)
and for bentonite, this was 9.9, 17.1 and 44.1mg � L−1 (corresponding to respectively 30, 50 and
100mg � L−1).
Hyperspectral imaging. The underwater hyperspectral imager (UHI) applied in our
experiments for recording HSI, is a push-broom type underwater hyperspectral sensor with a
narrow slit, spectrograph, hyperspectral line camera placed in a waterproof housing. Since it is
a push-broom imager, either the object or camera needs to move, to enable imaging of a scene.
This was implemented by using a platform moving the camera along the vertical axis (S2 Fig
in the supplementary), capturing hyperspectral images perpendicular to the direction of move-
ment within the wavelength range 380 − 750nm (Setup modified from [21]). Recording was
controlled through the data acquisition software Immersion on a top -side computer, con-
nected to the sensor through a sub-sea ethernet cable. During image acquisition the scenery

PLOS ONE | https://doi.org/10.1371/journal.pone.0272408 August 8, 2022 5 / 19

PLOS ONE Exploring time series of hyperspectral images for cold water coral stress response analysis

was illuminated with halogen lamps from above (Osram Decostar 51 TITAN 50 W 12 V 60˚
GU5.3) with constant power supply. Radiometric correction was carried out on the raw hyper-
spectral image in order to correct for dark current and sensor specific noise, resulting in a radi-
ance image. However, for comparability with other studies reflectance was estimated for a
selection of pixels (ROIs) from all exposure treatments (see S1 Text and S3 Fig).
HSI data. A time series HSI dataset was obtained for all samples at four selected time
points throughout the experiment: Prior to exposure (T0), directly after the exposure (T0.5)
and two weeks (T1) and six weeks (T2) after the exposure, into the recovery period. As the HSI
applied in our setup described above recorded one large hyperspectral image of all corals for
each time point t 2 {T0, T0.5, T1, T2} including a lot of pixels with background a first step was
to split the large HSI showing five (barite/bentonite exporsure) or six corals (DC exposure)
into single HSI (each one showing one sample). Afterwards the individual coral samples were
outlined and annotated in the single HSI using BIIGLE 2.0 [36] annotation software. Each sin-
n
gle data set is referred to with Ht;c , with c denoting contaminant and concentration level, t:
time point (T0, T0.5, T1, T2) and n: sample ID (0, . . ., 4 or 5). Hcn refers to the time series of all
HSI recorded for sample n in exposure experiment c and Ht,c refers to all HSI recorded at one
n
time point with the same exposure concentration. The term Ht;c;x;y refers to the spectrum at
position (x, y) in the measurement of sample n at time point t and contamination c. The inten-
n
sity value for a given wavelength s at one position is given by Ht;c;x;y;s .

Analysis of HSI data: HypIX

In order to analyse the HSI data, i.e. to find relationships between changes in spectral signature
through time and the exposure levels applied, we present the new HSI exploration webtool
HypIX (Hyperspectral Image eXplorer) that was implemented and employed in this work.
HypIX and the data sets used in this study are available online at https://webserver.biodtmin.
projects.bi.denbi.de/hypix using the username coral and the password hypercoral2020. An
example display of the HyPIX interface is shown in Fig 2 for a first impression and overview.

Fig 2. Screenshot of the HypIX tool: In the top frame (a) the experimental conditions, concentration levels, dimension reduction algorithms and
individual coral samples (0, . . ., 4 or 5) can be chosen by the user. On the right (b) a pseudo image of each HSI of all four time steps is shown in the
image display, in this case the mean spectral response value (see Methods for details). On the right side in frame (b) the user can chose to apply image
normalization or use pseudocolor to change the images visualization. In frame (c), spectral signatures from all four time steps are shown in the spectra
display. This window initially shows the agglomerated spectral signatures from all four data sets color encoded (T0: blue, T0.5: yellow, T1: red, T2:
green. (d) The large frame on the left (d) shows the dimension reduction results for the four HSI data sets in the embedding display using again the
same color code for the four time points.
https://doi.org/10.1371/journal.pone.0272408.g002

PLOS ONE | https://doi.org/10.1371/journal.pone.0272408 August 8, 2022 6 / 19

PLOS ONE Exploring time series of hyperspectral images for cold water coral stress response analysis

After starting HyPIX, a data set (i.e. experimental run and sample) and other methodological
parameters are selected (see Fig 2a). Afterwards the data set can be explored and filtered in
three different domains simultaneously, namely the spatial (or lateral) image domain (cmp.
Fig 2b), the spectral domain (cmp. Fig 2c) and the data embedding domain (cmp. Fig 2d).
Methodological details behind the HyPIX modules, data pre-processing and the two HyPIX
workflows applied in our study are described in the following. In the formal description of the
processing steps we will from now on omit the experiment index c and time point index t as all
HSI were treated the same. Instead we will use spatial coordinates (x, y) and the spectral wave-
length s to describe the processing steps.
Spectral domain visualization. The spectra display (Fig 2(c)) shows the four spectra of
n
the four HSI (recorded at T0, T0.5, T1, T2) for one sample n. The spectra Hx;y are visualized
either for all pixels or just for a subset of pixels selected in the image display (Fig 2(b)) if partic-
ular morphological / structural elements are of interest. If a selection is made containing more
than a single spectrum an agglomeration spectrum H ^ ns;� is shown. If no selection is made at all,
an agglomeration spectrum of all spectra per sample n is shown. Different agglomeration
options can be used. When using arithmetic mean as agglomeration option it yields the mean
spectral signal intensity per sample
P n
x;y Hx;y;s
^ n
H s;mean ¼ ;
w�h

Analogously, instead of the arithmetic mean operation, the max(), min() or medium()
operation can be used for the agglomeration step, i.e. the maximum spectral intensity
H^ ns;max ¼ maxx;y ðHx;y;s
n
Þ, minimum spectral intensity as H ^ ns;median ¼ minx;y ðHx;y;s
n
Þ or median spec-
tral intensity respectively H ^ s;median ¼ minx;y ðHx;y;s Þ can be chosen in the HyPIX interface (see
n n

upper left in the frame Fig 2(b)).

Additionally, an optional normalization step can be applied to the spectra using the l1-
n
norm, i.e. for each spectrum Hx;y;s ~ nx;y;s with
we create a normalized spectrum H
n
Hx;y;s
~ nx;y;s ¼ P
H :
n
s0 jHx;y;s 0j

Image domain visualization. To visualize the data in the image domain, so users can
assess the morphology, we compute different representative pseudo grey-value images I nt;c for
n
each HSI Ht;c . In the following we will consider the case of one HSI and omit the indices t, c for
the sake of compact writing. Users can select the mean spectral image, i.e.
P n
n;mean s Hx;y;s
X
I x;y ¼ n
maxfHx;y;s g
S s
s

or the maximum spectral image I n;max

x;y
n
¼ maxs fHx;y;s g or the minimum spectral image
I n;min
x;y
n
¼ mins fHx;y;s g, or the median spectral image, i.e. I n;median
x;y
n
¼ medianfHx;y;s g.
In addition we provide the option to normalize the images, i.e. to linearly scale the values to
[0, 255] and a pseudocolor visualization option for the pseudo gray-valued image I nx;y;s . In this
case, the gray scale pseudo images are mapped to colors with a lookup table using the matplo-
tlib [37] spectral colormap. Further user options offered in HypIX are to increase the bright-
ness b of the images by multiplying b with the pseudogray/-color image, i.e. b � I nx;y;s . Instead
of showing an agglomerated pseudo grey-value image an image for a certain spectral channel ^s
can be shown, i.e. I nx;y;^s .

PLOS ONE | https://doi.org/10.1371/journal.pone.0272408 August 8, 2022 7 / 19

PLOS ONE Exploring time series of hyperspectral images for cold water coral stress response analysis

Embedding domain visualization. To visualize the data from an HSI time series in one
scatter plot (see embedding display in Fig 2(d)) each HSI time series is mapped to a two-
n
dimensional space using dimensionality reduction. Thus, Hx;y;s 2 Rw�h�S is transformed to
Dnx;y;d 2 Rw�h�2 , referred to as the embedding domain. As the computation of the two-dimen-
sional projection is time-consuming it cannot be done in real-time, so the embeddings must
be computed beforehand, saved and then loaded into HypIX on demand. Because of this the
normalization option mentioned above is used mandatorily. For dimensionality reduction we
use two different state-of-the-art methods namely t-distributed stochastic neighbor embedding
(t-SNE [25]) and Uniform Manifold Approximation and Projection (UMAP [38]). For the
UMAP method we have tested two different metrics, the euclidean metric and the cosine
angular metric. The cosine angular metric in contrast to the euclidean metric is independent
of the length of the vector and thus in our case neglects the amplitude of the spectra, thus look-
ing only at the distribution of the spectra.
HypIX interface and functions. After selecting the HSI data from one time series experi-
ment (i.e. exposure level or control), one sample (see frame (a) in Fig 2), HypIX displays
pseudo gray-valued images of the four HSI data sets in the image display (see (b) in Fig 2), an
agglomerated spectrum for each HSI data set in the spectra display (see (c) in Fig 2) and a
dimension reduction scatter plot that was computed for all spectra from the four HSI data sets
in the embedding display (see (d) in Fig 2). In the embedding display or in the spectral display
the data from different time points can be selected and de-selected in order to focus on the
comparison of for instance only two measurements.
The four displays are functionally linked to allow interactive dynamic selection, filtering
and highlighting of data subsets in one display with a simultaneous highlighting of the same
data in the other displays, which is also referred to as gating or link and brush in the informa-
tion visualization community [39] (cmp. Fig 2a–2c). The (de-)selection of data groups in one
display is propagated to the other different displays, i.e. if a subset of data is selected in the
embedding display (cmp. Fig 2a)) the location of this selection is highlighted as a ROI in the
image domain (Fig 2b)) and the mean spectra are depicted in the spectrum display (Fig 2c)).
The selection of single data points in the embedding display furthermore allows the inspection
of the spectrum of a single hyperspectral pixel (Fig 2d)).
In addition to this functionality, the HypIX tool also offers the possibility to show a PCA
biplot of the data chosen in the embedding display. The PCA biplot shows a PCA dimensional-
ity reduction to the two-dimensional plane as well as a visualization of the PCA loadings. The
dots correspond to individual coral pixels, while arrows correspond to the variable loadings of
individual wavelength variables (pseudo-colored according to the color they represent).
Although PCA is possibly less powerful than, e.g., UMAP when it comes do differentiating
between spectral samples in a two-dimensional data space, it arguably represents a means of
dimensionality reduction that is easier to interpret. By inspecting a biplot, it is for instance pos-
sible to coarsely relate observed spectral differences to the wavelengths causing them. This is a
useful property that potentially may guide more targeted analyses performed subsequently. For
a more in-depth discussion on PCA biplots please have a look at the respective literature [40].
All these HypIX functions can be used to conduct several different approaches to data
exploration. This allows an analysis of the data in three domains simultaneously and to develop
new workflows to guide users in the analysis of HSI data.
HypIX is built using the Dash library and the Python programming language. For data pro-
cessing we use numpy [41], scikit-learn [42], umap-learn [38] and h5py [43].
Morphology-based filtering workflow. This workflow is motivated by the standard
approach of first selecting ROIs in the spatial domain and investigating the average spectra

PLOS ONE | https://doi.org/10.1371/journal.pone.0272408 August 8, 2022 8 / 19

PLOS ONE Exploring time series of hyperspectral images for cold water coral stress response analysis

Fig 3. Local changes revealed with the interactive subselection of the hyperspectral viewer, otherwise shadowed by global analysis. Shown is the
bentonite sample with 100mg � L−1 concentration using a UMAP with cosine metric and coral 3. The normalization of spectra was activated and the
aggregation option was set to mean.
https://doi.org/10.1371/journal.pone.0272408.g003

from these ROIs and their differences or similarities through the time series. In HypIX, the
ROIs can be selected in the image display by drawing polygons or rectangles. Since the HSI
shown in this study are not spatially aligned (or registered), users need to be very careful to
select ROIs in each of the four pseudo-color images that show corresponding morphological
substructures (like calice) in all four images. After the ROI selection, the other two displays are
updated and the corresponding points are highlighted in the embedding display and the spec-
tra display now shows the agglomerated spectra from the four ROIs.
Embedding driven response analysis workflow. We introduce the concept of embed-
ding-driven ROIs in contrast to anatomy/morphology driven ROIs, as described above. The
idea is to detect areas of spectral shifts and changes not based on a spatial hypothesis but based
on patterns in the spectral data points from two or more time points. One example for results
obtained with such a workflow is shown in Fig 3 and explained in detail in the Results section.
First, a group of points is selected in the embedding display using the mouse and a lasso or
rectangle selection tool. Sometimes it is beneficial to limit the embedding display to data from
two time points only (e.g. comparing the spectral data recorded at T0 to that recorded at T2).
The average spectra from these selections are presented in the spectra display and the locations
of these spectra are highlighted in the image display. Using this workflow, regions in the
embedding display that are populated by data from only one time point can be selected. This
causes the image display to highlight these points. Now we can select the same regions for the
other time points in the image display. This in turn causes the embedding display and the spec-
tra display to be updated, showing local spectral differences over time.
HyPIX application experiment. Three users (one bioinformatician, two marine biolo-
gists), independently applied HypIX and the workflows described above to investigate the col-
lected HSI times series. Each time series was rated regarding the changes observed between the
initial time point T0 to the other time points T0.5 (right after exposure), T1 (after two weeks)
and T2 (after six weeks). Since many ROIs described only small parts of the samples, these
changes cannot be found using a holistic approach, e.g. by comparing average spectra com-
puted for entire samples. Therefore, a tool such a HypIX is required to find these changes.

PLOS ONE | https://doi.org/10.1371/journal.pone.0272408 August 8, 2022 9 / 19

PLOS ONE Exploring time series of hyperspectral images for cold water coral stress response analysis

n
Each user independently rated each HSI data Ht;c with t 6¼ T0 regarding the degree of change
n
in comparison with its initial pre-exposure state HSI HT0;c . All changes appeared only in spe-
n
cific parts of the corals, so the ROI Ht6¼T0;c describing this part showed a spectral signature con-
siderably different to the signature from the same ROI in the T0 measurement. The users’
rating decision was noted as rðHcn Þ 2 f1ðno changeÞ; 2; 3ðaverage change levelÞ; 4; 5ðhighest
change levelÞg and the detailed rating result is given in S1 and S2 Tables in the supplementary.
After all HSI have been evaluated by the users for each sample’s time series Hcn the average
change rating ^r nc 2 ½1; . . . ; 5� was computed from the three users’ ratings. In Fig 4 the average
change rates for the control group experiments and each exposure level experiment for ben-
tonite, barite and DC exposure are summarized as one star glyph per experimental condition,
i.e. exposure concentration (see graphical explanation of the star glyph in the box in Fig 4).

Results
The first observation in Fig 4 was that the users did not find many changes in the control
group but identified stronger changes in the exposure experiments. Although, the corals did
not all react with the same intensity for one exposure level, the trend that there were at least
two corals reacting stronger to the exposure is visualized by a larger area in Fig 4. In the control
group of the DC experiments some users noted a change in the embedding display for one
coral sample (see “Coral 5” in the upper right starplot in Fig 4). A re-investigation of this case
showed that in this case, the shift in the embedding space was caused by a mistake in the
description of coral mask in the T0 data set, which was not noted by some users. In T0, the
coral mask was too large and included non-coral pixels from the ground. As this was only the
case in one time point, this resulted in two shifted groups of points outside the main point
cloud in the embedding display. We decided not to repeat the experiment with a corrected
mask in order to follow the planned experimental set up as strictly as possible so the workflow
could be evaluated rigorously. Instead, in the supplementary S4 Fig we show the embedding
results with and without the wrong part of the mask.
The second observation in the bentonite/barite experiment was that not all five corals in
one exposure experiment showed a reaction to the exposure in the spectral signatures. Just a
subset of three to four corals showed an average change rate rðHcn Þ > 2:5. This kind of individ-
ual response behavior was an interesting observation as it motivates the investigation of single
samples instead of analysing all spectra from all samples together, which is usually the case.
The third observation was that the exposure concentration did not seem to impact the
degree of change in the barite exposure experiments but it seemed to play a role in the benton-
ite experiments.
In addition we observed a trend for the reaction speed of the corals for bentonite and barite
on the one hand and DC on the other hand. Many corals treated with bentonite and barite
showed an immediate reaction to the exposure at time point T0.5 but no further change at the
following time points. The corals exposed to DC solution reacted at T0.5 and then seemed to
go back to a state with lesser change when compared to T0.
In addition to these general observations based on the subjective ratings rðHcn Þ we found
more specific patterns through the interactive nature of HypIX, i.e. the link and brush between
embedding, image and spectra displays. In Fig 3 we showed an example where the subselection
of data in the embedding display and linkage between different displays led to the identifica-
tion of a local change. In the initial state (no filtering applied and all data from T0, . . ., T2 is
shown) the embedding and the spectral displays showed only small differences in point distri-
butions and the spectral composition looked exactly the same (cmp. Fig 3(a) and 3(d)). Hiding
data from two time points, i.e. T0.5 and T1 (cmp. Fig 3(b)) hinted to a ROI in the embedding

PLOS ONE | https://doi.org/10.1371/journal.pone.0272408 August 8, 2022 10 / 19

PLOS ONE Exploring time series of hyperspectral images for cold water coral stress response analysis

Fig 4. Subjective rating results of the changes from initial time point T0 to final time point T2 of the bentonite,
barite and DC experiment after six weeks. All values represent the average over all subjective ratings. Ratings range
from 1 (no change) to 5 (strong change). Analogues to the color scale of the Hyperspectral Viewer, yellow is T0.5
(directly after exposure), red is T1 (two weeks after exposure) and green is T2 (six weeks after exposure). The titles of
the subgraphs are the concentrations in mg � L−1 of the exposure depicted as the row title.
https://doi.org/10.1371/journal.pone.0272408.g004

PLOS ONE | https://doi.org/10.1371/journal.pone.0272408 August 8, 2022 11 / 19

PLOS ONE Exploring time series of hyperspectral images for cold water coral stress response analysis

Fig 5. Pseudocolor images of each timepoint. We can clearly spot differences between time point T0 and the other time points. a) start of
experiment (T0); b) immediately after exposure (T0.5); c) after 2 weeks of exposure (T1); d) after 6 weeks of exposure (T2).
https://doi.org/10.1371/journal.pone.0272408.g005

display (see Fig 3b upper left), where the data distributions differed between T0 (blue) and T2
(green). In a subsequent step we selected this suspicious area in the data display (cmp. Fig 3
(c)), which triggered the link and brush of the location view (cmp. Fig 3(e) and 3(f)), as well as
of the spectral view (cmp. Fig 3(g)). From the Fig 3(e) and 3(f) we saw that the data distribu-
tion represented indeed the same location, however, the spectral composition (cmp. Fig 3(g))
differed significantly from T0 to T2. The curves cross at approx. 630 nm, suggesting a change
in spectral properties towards a less red and more flattened spectrum. This example showed
that these changes cannot be analysed with a technique using a single modality. When we only
looked at the unfiltered spectra (cmp. Fig 3(d)) we could not point out any difference, because
it was shadowed by the mean of spectra. When we looked at the original data view (cmp. Fig 3
(a)), we could probably see a change in data distribution, although this was already quite hard
to see, because of the huge amount of data. Even if we could observe the change in data distri-
bution, we could only state that there might be a change and there is still the slight possibility
of a misleading embedding of the data causing this change in distribution. What this change
looked like in the spectral domain or where it was located is hidden from a mono-modal analy-
sis. In addition using t-SNE or UMAP with euclidean metric this change in data distribution
could not be spotted as clearly as with the UMAP using a cosine metric. In contrast our results
showed the potential of the multi-modal HypIX functions in HSI analysis.
Another option to quickly generate a hypothesis about changes from time point to time
point was the pseudocolor option (cmp. Fig 5). A change in color meant that there was likely a
change in the spectral composition, which could be validated using the spectral and the data
view in successive steps. Furthermore a single channel could be selected to be presented as a
pseudocolor image to analyse changes in a specific spectral band of interest.

Discussion
Hypix was evaluated by a focus group of users and they liked the look and feel. Problems that
arose were fixed in the final version of Hypix. Users particularly appreciated the link and
brush functionality, i.e., interactive filtering of data in one modality that links changes in the
other modalities. This opened new ways of exploratory data analysis in hyperspectral imaging,
which can lead to the generation of future research hypotheses.
The two proposed workflows led to results that were mostly consistent between the users
(standard deviation 0.39, also cmp. S5 Fig in the supplementary). The low degree of variation

PLOS ONE | https://doi.org/10.1371/journal.pone.0272408 August 8, 2022 12 / 19

PLOS ONE Exploring time series of hyperspectral images for cold water coral stress response analysis

indicates that the software and the proposed workflow can be applied for HSI analysis to
achieve results that are reproducible.
The fact that the method is vulnerable to mistakes in the definition of the mask (cmp.
“Coral 5” in the Control experiment of the DC exposure in Fig 4) may be considered accept-
able as any kind of analysis that uses a mask description would potentially suffer from such
kinds of error. One may even consider it a strength of this interactive visualization-based
approach that this imprecise mask definition was detected in the experiment.
A clear defined exposure-related spectral change was not ubiquitously observed for all the
coral samples in any of the treatments. It is therefore difficult to model a relationship with
respect to specific exposure levels and their general impact on coral color based on our experi-
mental output. A notable pattern, however, was that exposure to bentonite and barite had a
higher effect on the spectral properties of white corals than on orange ones (cmp. Fig 4). This
is particularly interesting considering that findings from a recent study by Büscher et al. [44]
indicate that the white D. pertusum phenotype may be less stress-resistant than the orange phe-
notype. For the white corals where a spectral change was observed, the change was typically
manifested as a red-shifted spectrum (i.e., lowered blue values and elevated red values, cmp.
Fig 6). The ultimate cause of this spectral shift is yet to be determined, and consequently a
topic that warrants further investigation.
Provided that our observed spectral shift is a response to the exposure treatment, the non-
uniform response of coral samples undergoing the exact same treatment indicates that individ-
ual corals may have individual tolerance levels. Further studies should seek to identify the rea-
son for these differences in tolerance. One can speculate if factors such as age, reproduction,
feeding availability and integrity of the coenosarc layer contributes to the fitness of the individ-
ual polyp.
In a recent tank study [29], D. pertusum was also exposed to various concentrations of parti-
cles associated with drilling operations. Notably, the study reported a significantly increased
ratio of organic carbon to organic nitrogen (OC:ON ratio) in the mucus of corals exposed to
bentonite concentrations >20mgL−1. Furthermore, a study on the effects of drilling particles

Fig 6. Screenshot of the spectral view of Hypix. Example for a red-shifted spectrum due to the exposure with a barite
concentration of 30mg � L−1 for coral 0. The blue part of the light spectrum is lowered while the red part is elevated at
T0.5, T1 and T2 compared to before the exposure at T0.
https://doi.org/10.1371/journal.pone.0272408.g006

PLOS ONE | https://doi.org/10.1371/journal.pone.0272408 August 8, 2022 13 / 19

PLOS ONE Exploring time series of hyperspectral images for cold water coral stress response analysis

on D. pertusum larvae found that bentonite affected larval behavior at lower concentrations
than both barite and DCs [31]. This was attributed to the bentonite particles being finer, mak-
ing them stick to the coral mucus more easily. The findings from both the aforementioned
studies are interesting, as the greatest spectral changes in the current study also were observed
for corals exposed to bentonite (see results obtained for the Bentonite 100 experiments, illus-
trated in Fig 4). A possible explanation for these observations is that the finer bentonite parti-
cles interfered with the coral mucus to a greater extent than both barite and DCs, and that this
manifested itself as a detectable spectral shift. However, as the trend was not ubiquitous
among coral samples exposed to high bentonite concentrations, this interpretation should cur-
rently be treated with caution. In the future, we recommend conducting particle exposure
studies of D. pertusum where hyperspectral signatures and coral mucus properties are mea-
sured simultaneously. This could provide further insight into D. pertusum’s response to dril-
ling operations and possibly help substantiate the observations related to bentonite exposure
made in the current study.
In general, the shifts observed in the HSI of the corals can be described as colour shifts
towards a more reddish colour. However, the shifts are too small to be recognized in the RGB
image by the human eye. Colour shifts in D. pertusum corals have been reported before by
Osterloff et al. [45], however on a much longer time scale (from May to September 2015) and
with another possible explanation, i.e. a seasonal difference in the food supply (copepods). In
the RGB images in this study of 2019, the colour shift was strong enough to be perceived com-
paring images recorded at time points four months apart. The underlying mechanisms of
color changes in this species are not well known and only a small number of studies have been
published on the topic so far. Letnes et al. [21] found a correlation between hydrocarbon expo-
sure, coral mortality, and coral color change. Elde et al. [11] found different concentrations of
pigments in the morphotypes. A color morph specific bacterial assemblage is reported by Neu-
linger et al. [46].
Besides the effects observed in the HSI data mentioned above, the rationale behind the
application of hyperspectral imaging to assess the health status of cold water corals, is that I)
the spectral properties of corals have been observed to change with declining health, and that
II) these changes are observable with underwater hyperspectral imaging. In the following we
will address these two points to provide more context for our findings.
Point I) is supported by the work of Letnes et al. [21] who showed a correlation between
coral mortality and colour of the species. The corals undergoing mortality exhibited a loss of
absorption properties at specific wavelength (560nm), which suggests a loss of pigment func-
tion. However the link between colour and levels of non-lethal health change is not well stud-
ied. Hyperspectral assessment from air is much used to assess the health of shallow water
tropical corals. These corals form a symbiotic relationship with dinoflagellates, and with the
loss of this dinoflagellate due to increasing water temperatures, the process of coral bleaching
occurs. The process is reversible to a certain point, however if the symbiosis is not restored the
coral will eventually die and the remaining carbonate skeleton will typically be inhabited by
macro algae. HSI is used to monitor the extent of live corals, bleached corals and macro algae
covered corals [47–49]. This is a related yet different scenario than health monitoring of the
azooxanthellate D. pertusum. We have a fairly good overview of the typical live spectrum and
the dead spectrum of the species [11, 21, 23, 50, 51], however, there is limited knowledge on
spectral properties of the intermediate health, which would be equivalent to the bleached, but
not dead tropical coral. Establishing further knowledge on the spectral responses is crucial if
HSI is to give an early warning of changes in environmental conditions.
Point II) is motivated by the observation that knowledge on how coral spectral properties
change with a declining health is rather limited. Thus, the discussion on the capability of HSI

PLOS ONE | https://doi.org/10.1371/journal.pone.0272408 August 8, 2022 14 / 19

PLOS ONE Exploring time series of hyperspectral images for cold water coral stress response analysis

to detect these changes, should be taken with precaution. However, the fact that HyPIX can be
applied to produce reproducible results obtained from visualizations can be interpreted as a
new step towards increasing the significance of HSI data by improving the HSI data interpreta-
tion. Hence, we propose that HSI does have a potential to function as a tool for monitoring
changing physiological conditions of marine organisms. Future studies should seek to under-
stand the actual mechanisms involved in color change of corals. Emphasis should also be put
on bringing the methodology from lab to field. An HSI field tool for detection of health
changes in deep-water marine organisms could be utilized for monitoring of effects of offshore
drilling operations and is also likely to be be valuable for other industries, for instance for
monitoring of environmental footprints from sea based aquaculture production, where there
are concerns regarding the the effects of organic wastes on filtering benthic fauna.
We show that this software-driven workflow shows great potential in the analysis of hyper-
spectral imagery. Although in this case the Hypix tool was tailored for this use case the method
can be applied to other hyperspectral or multispectral data as well. The two workflows pro-
posed in this research led to a streamlining of methods thus to reproducible results. We think
that therefore not only the software is a valuable asset but also the workflows and methodology
in general.

Supporting information
S1 Table. Individual subjective rating results for Bentonite and Barite experiments.
(PDF)
S2 Table. Individual subjective rating results for drill cutting experiments.
(PDF)
S1 Text. Extraction of spectral signatures for reflectance estimation.
(PDF)
S1 Fig. Laboratory set-up: Aquarium setup. Left: Photo of corals (white and orange) and
polyethylene reference plate inside aquarium. Right: Sketch showing position of time lapse
camera and underwater hyperspectral imager (UHI) outside the aquarium with coral nubbins.
(PDF)
S2 Fig. Laboratory set-up: Setup for acquisition of hyperspectral images Coral samples and
a polyethylene reference plate were placed at the bottom of aquarium. The hyperspectral
imager was placed on a scanning rig outside the aquarium, scanning through the glass.
(PDF)
S3 Fig. Examples of reflectance estimated spectra (380–750 nm) of Control corals and cor-
als exposed to drill cuttings, barite and bentonite. X-axis scale 400–800 nm. All spectra
shown are from coral polyp/calice.
(PDF)
S4 Fig. Identification of a false annotation of coral 5 in the drill cutting control experiment
with Hypix.
(PDF)
S5 Fig. Histogram of the differences between observers for all subjective ratings of all sub-
jects.
(PDF)

PLOS ONE | https://doi.org/10.1371/journal.pone.0272408 August 8, 2022 15 / 19

PLOS ONE Exploring time series of hyperspectral images for cold water coral stress response analysis

Author Contributions
Conceptualization: Daniel Langenkämper, Aksel Alstad Mogstad, Ingrid Myrnes Hansen,
Thierry Baussant, Øystein Bergsagel, Ingunn Nilssen, Tone Karin Frost, Tim Wilhelm
Nattkemper.
Data curation: Daniel Langenkämper.
Formal analysis: Daniel Langenkämper, Aksel Alstad Mogstad, Ingrid Myrnes Hansen,
Ingunn Nilssen, Tim Wilhelm Nattkemper.
Funding acquisition: Ingunn Nilssen, Tone Karin Frost, Tim Wilhelm Nattkemper.
Investigation: Daniel Langenkämper, Aksel Alstad Mogstad, Ingrid Myrnes Hansen, Thierry
Baussant, Øystein Bergsagel, Ingunn Nilssen, Tone Karin Frost, Tim Wilhelm Nattkemper.
Methodology: Daniel Langenkämper, Aksel Alstad Mogstad, Ingrid Myrnes Hansen, Thierry
Baussant, Ingunn Nilssen, Tone Karin Frost, Tim Wilhelm Nattkemper.
Project administration: Ingunn Nilssen, Tone Karin Frost, Tim Wilhelm Nattkemper.
Resources: Daniel Langenkämper, Ingrid Myrnes Hansen, Ingunn Nilssen, Tone Karin Frost.
Software: Daniel Langenkämper, Tim Wilhelm Nattkemper.
Supervision: Ingunn Nilssen, Tone Karin Frost, Tim Wilhelm Nattkemper.
Validation: Daniel Langenkämper, Aksel Alstad Mogstad, Ingrid Myrnes Hansen, Thierry
Baussant, Øystein Bergsagel, Tone Karin Frost, Tim Wilhelm Nattkemper.
Visualization: Daniel Langenkämper, Aksel Alstad Mogstad, Tim Wilhelm Nattkemper.
Writing – original draft: Daniel Langenkämper, Aksel Alstad Mogstad, Ingrid Myrnes Han-
sen, Thierry Baussant, Øystein Bergsagel, Ingunn Nilssen, Tone Karin Frost, Tim Wilhelm
Nattkemper.
Writing – review & editing: Daniel Langenkämper, Aksel Alstad Mogstad, Ingrid Myrnes
Hansen, Thierry Baussant, Øystein Bergsagel, Ingunn Nilssen, Tone Karin Frost, Tim Wil-
helm Nattkemper.

References
1. Goetz A. Three decades of hyperspectral remote sensing of the Earth: A personal view. Remote Sens-
ing of Environment. 2009; 113. https://doi.org/10.1016/j.rse.2007.12.014
2. Lu G, Fei B. Medical hyperspectral imaging: a review. Journal of biomedical optics. 2014; 19(1):010901.
https://doi.org/10.1117/1.JBO.19.1.010901 PMID: 24441941
3. Adão T, Hruška J, Pádua L, Bessa J, Peres E, Morais R, et al. Hyperspectral imaging: A review on
UAV-based sensors, data processing and applications for agriculture and forestry. Remote Sensing.
2017; 9(11):1110. https://doi.org/10.3390/rs9111110
4. Liu Y, Pu H, Sun DW. Hyperspectral imaging technique for evaluating food quality and safety during var-
ious processes: A review of recent applications. Trends in food science & technology. 2017; 69:25–35.
https://doi.org/10.1016/j.tifs.2017.08.013
5. Lu Y, Huang Y, Lu R. Innovative hyperspectral imaging-based techniques for quality evaluation of fruits
and vegetables: A review. Applied Sciences. 2017; 7(2):189. https://doi.org/10.3390/app7020189
6. Rizk P, Al Saleh N, Younes R, Ilinca A, Khoder J. Hyperspectral imaging applied for the detection of
wind turbine blade damage and icing. Remote Sensing Applications: Society and Environment. 2020;
18:100291. https://doi.org/10.1016/j.rsase.2020.100291
7. Dong X, Jakobi M, Wang S, Köhler MH, Zhang X, Koch AW. A review of hyperspectral imaging for nano-
scale materials research. Applied Spectroscopy Reviews. 2019; 54(4). https://doi.org/10.1080/
05704928.2018.1463235

PLOS ONE | https://doi.org/10.1371/journal.pone.0272408 August 8, 2022 16 / 19

PLOS ONE Exploring time series of hyperspectral images for cold water coral stress response analysis

8. Krupnik D, Khan S. Close-range, ground-based hyperspectral imaging for mining applications at various
scales: Review and case studies. Earth-science reviews. 2019; 198:102952. https://doi.org/10.1016/j.
earscirev.2019.102952
9. Stuart MB, McGonigle AJ, Willmott JR. Hyperspectral imaging in environmental monitoring: a review of
recent developments and technological advances in compact field deployable systems. Sensors. 2019;
19(14):3071. https://doi.org/10.3390/s19143071 PMID: 31336796
10. Brando VE, Dekker AG. Satellite hyperspectral remote sensing for estimating estuarine and coastal
water quality. IEEE transactions on geoscience and remote sensing. 2003; 41(6):1378–1387. https://
doi.org/10.1109/TGRS.2003.812907
11. Elde AC, Pettersen R, Bruheim P, Järnegren J, Johnsen G. Pigmentation and spectral absorbance sig-
natures in deep-water corals from the Trondheimsfjord, Norway. Marine drugs. 2012; 10(6):1400–1411.
https://doi.org/10.3390/md10061400 PMID: 22822381
12. Mishra P, Asaari MSM, Herrero-Langreo A, Lohumi S, Diezma B, Scheunders P. Close range hyper-
spectral imaging of plants: A review. Biosystems Engineering. 2017; 164:49–67. https://doi.org/10.
1016/j.biosystemseng.2017.09.009
13. Khan MJ, Khan HS, Yousaf A, Khurshid K, Abbas A. Modern trends in hyperspectral image analysis: A
review. IEEE Access. 2018; 6:14118–14129. https://doi.org/10.1109/ACCESS.2018.2812999
14. Montes-Herrera JC, Cimoli E, Cummings V, Hill N, Lucieer A, Lucieer V. Underwater Hyperspectral
Imaging (UHI): A Review of Systems and Applications for Proximal Seafloor Ecosystem Studies.
Remote Sensing. 2021; 13(17):3451. https://doi.org/10.3390/rs13173451
15. Liu B, Liu Z, Men S, Li Y, Ding Z, He J, et al. Underwater hyperspectral imaging technology and its appli-
cations for detecting and mapping the seafloor: a review. Sensors. 2020; 20(17):4962. https://doi.org/
10.3390/s20174962 PMID: 32887344
16. Foglini F, Grande V, Marchese F, Bracchi VA, Prampolini M, Angeletti L, et al. Application of hyperspec-
tral imaging to underwater habitat mapping, Southern Adriatic Sea. Sensors. 2019; 19(10). https://doi.
org/10.3390/s19102261 PMID: 31100805
17. Johnsen G, Ludvigsen M, Sørensen A, Aas LMS. The use of underwater hyperspectral imaging
deployed on remotely operated vehicles-methods and applications. IFAC-PapersOnLine. 2016;
49(23):476–481. https://doi.org/10.1016/j.ifacol.2016.10.451
18. Dumke I, Nornes SM, Purser A, Marcon Y, Ludvigsen M, Ellefmo SL, et al. First hyperspectral imaging
survey of the deep seafloor: High-resolution mapping of manganese nodules. Remote Sensing of Envi-
ronment. 2018; 209:19–30. https://doi.org/10.1016/j.rse.2018.02.024
19. Herold J, Loyek C, Nattkemper TW. Multivariate image mining. Wiley Interdisciplinary Reviews: Data
Mining and Knowledge Discovery. 2011; 1(1).
20. Tollefson J. Computers on the reef. Nature News. 2016; 537(7618):123. https://doi.org/10.1038/
537123a PMID: 27582223
21. Letnes PA, Hansen IM, Aas LMS, Eide I, Pettersen R, Tassara L, et al. Underwater hyperspectral clas-
sification of deep sea corals exposed to 2-methylnaphthalene. PloS one. 2019; 14(2):e0209960. https://
doi.org/10.1371/journal.pone.0209960 PMID: 30811426
22. Cochrane S, Ekehaug S, Pettersen R, Refit E, Hansen I, Aas L. Detection of deposited drill cuttings on
the sea floor-A comparison between underwater hyperspectral imagery and the human eye. Marine pol-
lution bulletin. 2019; 145:67–80. https://doi.org/10.1016/j.marpolbul.2019.04.031 PMID: 31590836
23. Liu H, Büscher JV, Köser K, Greinert J, Song H, Chen Y, et al. Automated Activity Estimation of the
Cold-Water Coral Lophelia pertusa by Multispectral Imaging and Computational Pixel Classification.
Journal of Atmospheric and Oceanic Technology. 2021; 38(2):141–154. https://doi.org/10.1175/
JTECH-D-19-0139.1
24. FRS KP. LIII. On lines and planes of closest fit to systems of points in space. The London, Edinburgh,
and Dublin Philosophical Magazine and Journal of Science. 1901; 2(11):559–572. https://doi.org/10.
1080/14786440109462720
25. Van der Maaten L, Hinton G. Visualizing data using t-SNE. Journal of machine learning research. 2008;
9(11).
26. Khulal U, Zhao J, Hu W, Chen Q. Nondestructive quantifying total volatile basic nitrogen (TVB-N) con-
tent in chicken using hyperspectral imaging (HSI) technique combined with different data dimension
reduction algorithms. Food Chemistry. 2016; 197:1191–1199. https://doi.org/10.1016/j.foodchem.2015.
11.084 PMID: 26675857
27. Mishra P, Nordon A, Tschannerl J, Lian G, Redfern S, Marshall S. Near-infrared hyperspectral imaging
for non-destructive classification of commercial tea products. Journal of Food Engineering. 2018;
238:70–77. https://doi.org/10.1016/j.jfoodeng.2018.06.015

PLOS ONE | https://doi.org/10.1371/journal.pone.0272408 August 8, 2022 17 / 19

PLOS ONE Exploring time series of hyperspectral images for cold water coral stress response analysis

28. Mortensen T. Koralldyr: Vurdering av øyekorall Desmophyllum pertusum for Norge. Rødlista for arter
2021. Artsdatabanken.;. https://www.artsdatabanken.no/lister/rodlisteforarter/2021/5718.
29. Baussant T, Arnberg M, Lyng E, Ramanand S, Bamber S, Berry M, et al. Identification of tolerance lev-
els on the cold-water coral Desmophyllum pertusum (Lophelia pertusa) from realistic exposure condi-
tions to suspended bentonite, barite and drill cutting particles. PLOS ONE. 2022; 17(2):1–24. https://
doi.org/10.1371/journal.pone.0263061 PMID: 35192627
30. Baussant T, Nilsen M, Ravagnan E, Westerlund S, Ramanand S. Effects of suspended drill cuttings on
the coral Lophelia pertusa using pulsed and continuous exposure scenarios. Journal of Toxicology and
Environmental Health, Part A. 2018; 81(10):361–382. https://doi.org/10.1080/15287394.2018.1444375
PMID: 29537937
31. Jaernegren J, Brooke S, Jensen H. Effects and recovery of larvae of the cold-water coral Lophelia per-
tusa (Desmophyllum pertusum) exposed to suspended bentonite, barite and drill cuttings. Marine Envi-
ronmental Research. 2020; 158:104996. https://doi.org/10.1016/j.marenvres.2020.104996
32. Purser A. A time series study of Lophelia pertusa and reef megafauna responses to drill cuttings expo-
sure on the Norwegian margin. PLoS One. 2015; 10(7):e0134076. https://doi.org/10.1371/journal.pone.
0134076 PMID: 26218658
33. Larsson AI, van Oevelen D, Purser A, Thomsen L. Tolerance to long-term exposure of suspended ben-
thic sediments and drill cuttings in the cold-water coral Lophelia pertusa. Marine pollution bulletin. 2013;
70(1-2):176–188. https://doi.org/10.1016/j.marpolbul.2013.02.033 PMID: 23510599
34. Neff JM. Estimation of bioavailability of metals from drilling mud barite. Integrated Environmental
Assessment and Management. 2008; 4(2):184–193. https://doi.org/10.1897/IEAM_2007-037 PMID:
17994916
35. Baussant T, Nilsen M, Ravagnan E, Westerlund S, Ramanand S. Physiological responses and lipid
storage of the coral Lophelia pertusa at varying food density. Journal of Toxicology and Environmental
Health, Part A. 2017; 80(5):266–284. https://doi.org/10.1080/15287394.2017.1297274 PMID:
28569653
36. Langenkämper D, Zurowietz M, Schoening T, Nattkemper TW. Biigle 2.0-browsing and annotating
large marine image collections. Frontiers in Marine Science. 2017; 4:83.
37. Hunter JD. Matplotlib: A 2D graphics environment. Computing in Science & Engineering. 2007; 9
(3):90–95. https://doi.org/10.1109/MCSE.2007.55
38. McInnes L, Healy J, Melville J. Umap: Uniform manifold approximation and projection for dimension
reduction. arXiv preprint arXiv:180203426. 2018;.
39. Ware C. Information visualization: perception for design. Morgan Kaufmann; 2019.
40. Gower JC, Lubbe SG, Le Roux NJ. Understanding biplots. John Wiley & Sons; 2011.
41. Harris CR, Millman KJ, van der Walt SJ, Gommers R, Virtanen P, Cournapeau D, et al. Array program-
ming with NumPy. Nature. 2020; 585(7825):357–362. https://doi.org/10.1038/s41586-020-2649-2
PMID: 32939066
42. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, et al. Scikit-learn: Machine
Learning in Python. Journal of Machine Learning Research. 2011; 12:2825–2830.
43. Collette A. Python and HDF5. O’Reilly; 2013.
44. Büscher JV, Wisshak M, Form AU, Titschack J, Nachtigall K, Riebesell U. In situ growth and bioerosion
rates of Lophelia pertusa in a Norwegian fjord and open shelf cold-water coral habitat. PeerJ. 2019; 7:
e7586. https://doi.org/10.7717/peerj.7586 PMID: 31579574
45. Osterloff J, Nilssen I, Järnegren J, Van Engeland T, Buhl-Mortensen P, Nattkemper TW. Computer
vision enables short-and long-term analysis of Lophelia pertusa polyp behaviour and colour from an
underwater observatory. Scientific reports. 2019; 9(1):1–12. https://doi.org/10.1038/s41598-019-
41275-1 PMID: 31036904
46. Neulinger SC, Järnegren J, Ludvigsen M, Lochte K, Dullo WC. Phenotype-specific bacterial communi-
ties in the cold-water coral Lophelia pertusa (Scleractinia) and their implications for the coral’s nutrition,
health, and distribution. Applied and Environmental Microbiology. 2008; 74(23):7272–7285. https://doi.
org/10.1128/AEM.01777-08 PMID: 18849454
47. Kutser T, Miller I, Jupp DL. Mapping coral reef benthic substrates using hyperspectral space-borne
images and spectral libraries. Estuarine, Coastal and Shelf Science. 2006; 70(3):449–460. https://doi.
org/10.1016/j.ecss.2006.06.026
48. Hedley J, Roelfsema C, Koetz B, Phinn S. Capability of the Sentinel 2 mission for tropical coral reef
mapping and coral bleaching detection. Remote Sensing of Environment. 2012; 120:145–155. https://
doi.org/10.1016/j.rse.2011.06.028

PLOS ONE | https://doi.org/10.1371/journal.pone.0272408 August 8, 2022 18 / 19

PLOS ONE Exploring time series of hyperspectral images for cold water coral stress response analysis

49. Hochberg EJ, Atkinson MJ. Capabilities of remote sensors to classify coral, algae, and sand as pure
and mixed spectra. Remote Sensing of Environment. 2003; 85(2):174–189. https://doi.org/10.1016/
S0034-4257(02)00202-X
50. Andersson IHK. Bio-optical diversity on cold water coral habitats. Master thesis NTNU, Norway. 2017;.
51. Pettersen R. Identification of Marine Organisms using Chemotaxonomy and Hyperspectral Imaging.
PhD thesis NTNU, Norway. 2013;.

PLOS ONE | https://doi.org/10.1371/journal.pone.0272408 August 8, 2022 19 / 19