Bak at Al 2023

Download as pdf or txt
Download as pdf or txt
You are on page 1of 13

Measurement 219 (2023) 113286

Contents lists available at ScienceDirect

Measurement
journal homepage: www.elsevier.com/locate/measurement

Measurement methodology for surface defects inventory of building wall


using smartphone with light detection and ranging sensor
Wioleta Błaszczak-Bąk a, *, Czesław Suchocki b, Tomasz Kozakiewicz a, Joanna Janicka a
a
Faculty of Geoengineering, University of Warmia and Mazury in Olsztyn, Poland
b
Faculty of Civil Engineering Environmental and Geodetic Sciences, Koszalin University of Technology, Poland

A R T I C L E I N F O A B S T R A C T

Keywords: The popularity of smartphones along with advances in sensor technologies has opened new possibilities for
Smartphone scientific application and low-cost observation. Particularly useful is the LiDAR sensor, which can be used to
LiDAR measure building elements.
TLS
The aim of study is to propose a new methodology of using the smartphone with LiDAR Apple iPhone XR, in
Crack
Target
examining wall cracks and resulting displacement. Due to the fact, that the measured walls were quasi-flat and
homogeneous, dedicated spatial measurement targets (SMTs) were used for measurement. These SMTs are
designed to change the geometry of the tested object. The methodology proposed in the study includes mea­
surements using a smartphone with LiDAR in variants with and without SMTs, and its analysis.
The test results are compared with the measurements using a terrestrial laser scanner (TLS). Analysis showed
that for detecting small cracks such as 0.5–1.0 mm, smartphone measurement with LiDAR and SMT is better than
TLS.

1. Introduction point clouds [8], geological fieldwork [9], and human body measure­
ments [10]. It should be noted that, smartphone photogrammetry is also
Light detection and ranging (LiDAR) is used in many areas to perform harnessing for engineering measurements. We can find many applica­
fast and precise object measurements. Laser scanning may be performed tion of hand-held smartphone photogrammetry such as: 3D surface re­
from an airplane (airborne laser scanning – ALS), an unmanned aerial constructions of geomorphological objects [11,12], rock engineering
vehicle (UAV laser scanning), and a moving vehicle (mobile laser [13,14], coastal engineering [15] or landslide monitoring [16,17].
scanning – MLS); it may also be in ground mode, such as in terrestrial Generally, the TLS technology is well established in the technical di­
laser scanning (TLS). In recent years, LiDAR sensor has been incorpo­ agnoses of buildings and structures [18,19,20,21]. Based on the high
rated into various devices. In 2020, Apple launched its first smartphone resolution of 3D point clouds, dimensioning structural elements [22,23]
equipped with a LiDAR sensor. Hence, LiDAR is becoming an increas­ and locating defects, such as cracks and cavities, are possible [24,25].
ingly accessible low-cost technology. The possibility of using a smart­ Generally, an average smartphone user does not regularly employ the
phone with a LiDAR sensor for various engineering applications has LiDAR feature. Nevertheless, 3D sensors can be extremely useful in a
already been tested by scientists. For example, it has been employed in number of situations. It can be used by conservators as an alternative to
cultural heritage documentations [1,2], surveys of coastal cliffs [3], TLS to measure building walls or wall elements during renovation and
inventories of building interiors and façades [4], building renovation restoration. The use of a smartphone with LiDAR for defect measure­
diagnostics [5], forest inventories [6,7], research on the effect of ments is particularly justified in the case of hard-to-reach locations. The
different surface materials on the quality of three-dimensional (3D) small device enables measurements in situations where a large

Abbreviations: SfM MVS, Structure-from-Motion Multi-View Stereo; GNSS, Global Positioning Satellite System; LiDAR, Light Detection and Ranging; ALS, Airborne
Laser Scanning; MLS, Mobile Laser Scanning; TLS, Terrestrial Laser Scanning; UAV, Unmanned Aerial Vehicle; OptD, Optimum Dataset; ToF, Time of Flight; dToF,
Direct Time of Flight; iToF, Indirect Time of Flight; SL, Structured Light; CMOS, Complementary Metal Oxide Semiconductor; CIS, CMOS Image Sensor; SMT, Spatial
Measurement Target.
* Corresponding author.
E-mail addresses: wioleta.blaszczak@uwm.edu.pl (W. Błaszczak-Bąk), czeslaw.suchocki@tu.koszalin.pl (C. Suchocki), tomasz.kozakiewicz@uwm.edu.pl
(T. Kozakiewicz), joanna.janicka@uwm.edu.pl (J. Janicka).

https://doi.org/10.1016/j.measurement.2023.113286
Received 22 February 2023; Received in revised form 25 June 2023; Accepted 4 July 2023
Available online 8 July 2023
0263-2241/© 2023 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
W. Błaszczak-Bąk et al. Measurement 219 (2023) 113286

terrestrial scanner cannot be set up or in positions, which the laser beam iPad Pro with a rear LiDAR scanner. This rear 3D sensing module uses
cannot reach. the first ever consumer dToF CMOS image sensor product with an in-
As a non-invasive data acquisition technology, TLS is difficult to use pixel connection. This supports Apple’s ARkit 3.5, a development kit
for detecting cracks, displacements, and deformations in the case of for augmented reality. The iToF depth sensors simultaneously measure
hard-to-reach locations. Therefore, the team in the current study the distance from a scene to every camera pixel. This is achieved by
appreciated the new possibilities of using smartphones with LiDAR.. illuminating the full scene with a flash of light and measuring the phase
However, during the measurement of defects, the research team found difference between the emitted light and its reflection [34].
that this technology is unreliable. Despite the excellent accuracy and The underlying 3D imaging technology is called SL and uses a set of
resolution of laser scanners built into smartphones, they do not yield the ‘dots’ projected onto the face of a user. Biometric face authentication
expected results. In the case of extremely diverse objects with a allows the rapid unlocking of the screen and eliminates the necessity for
complicated structure, the measurements were acceptable. However, using fingers and entering passwords or patterns [35,36].
when the surface was homogeneous, the measurements of the smart­ The SL scanning method is based on the principle of triangulation
phone with LiDAR were inaccurate. Note that such situations typically [37,38] in which incident laser lines are projected onto the object to be
occur: a flat wall may have few cracks or cavities. The main problem is scanned. The basic geometry of the active triangulation system is shown
the impossibility of registering the point clouds during the measurement in Fig. 1.
because of the extremely homogeneous surface. However, the benefits of In the Fig. 1, Z is the optical axis; O is the origin; P is a single point on
using a smartphone with LiDAR seemed evident. Accordingly, the the object; baseline is the distance between the optical centre and laser;
research team was motivated to develop a new measurement θ is the angle between the baseline and laser plane; and α is the angle
methodology. between the baseline and segment OP.
In LiDAR measurements, the size of the laser spot and the divergence In SL systems that employ triangulation methods to obtain 3D laser
of the laser beam have a direct impact on the details of collected data measurements, the parameters of baseline, θ, and α play important roles
[26,27]. A small laser spot facilitates the measurement of minute details. in determining the accuracy and precision of the measurements. The
In TLS the laser spot size is typically 3–5 mm. Note that in TLS, the baseline is a fundamental parameter that affects the scan resolution of
measurement of details, which are smaller than the size of the laser spot, the system. The parameter θ is crucial for determining the accuracy of
is difficult or even impossible to implement. The manufacturers of the point cloud measurements. A larger θ can help improve the accuracy
smartphones with LiDAR do not indicate the laser spot size in the because it increases the spatial coverage of the projected laser pattern on
technical specifications. However, in many experiments conducted by the object surface. With a wider coverage, more surface points are
the authors, the point cloud captured using a smartphone is more illuminated, allowing for better triangulation and more accurate scan
detailed than that obtained by TLS. This allows for the measurement of estimation. However, an excessively large θ can cause measurement
surface changes, such as cracks, on building walls. errors due to occlusions or reflections, as the laser pattern might not be
The main aim of this study is to use smartphones equipped with visible or may be distorted on certain parts of the object surface. The
LiDAR to inventory object defects. To achieve this goal, the development parameter α affects the accuracy of the measurement, especially in
of a new measurement methodology using smartphones with LiDAR and scenarios where the object surface has a complex geometry or is not
spatial measurement targets (SMTs) to enable the measurement of de­ perfectly planar. Properly selecting and configuring these parameters
fects on homogeneous surfaces is necessary. The task of SMT targets is to based on the specific application and object characteristics is crucial to
artificially deform a flat surface in order to enable the registration of a achieve accurate and reliable depth information [39,40].
point cloud in a one coordinate system in real time. Thus, the novelty of A light projector (typically a laser) is placed at a certain distance
the present study is the application of SMTs to improve the quality of from the centre of the projection of the pinhole camera. The projector
point clouds obtained by smartphones using LiDAR. The research is emits a plane of light intersecting the scene surfaces in a planar curve
conducted on two test objects using a TLS (Leica ScanStation C10) and called stripes, which is observed by the camera. If the position of the
smartphone (Apple iPhone XR). The TLS measurements are used as
reference of the data captured by the smartphone.
The measurement and analysis were carried out by a research team
from the University of Warmia and Mazury in Olsztyn and the University
of Technology in Koszalin, which for many years, has been using TLS
technology to assess the technical conditions of buildings. The authors
have conducted thorough and successful research programs dedicated to
the application of TLS technology to building structure diagnostics
[28,29,25]. In particular, defects are determined using the optimum
dataset method [30,31].

2. Materials and methods

Different scanning technologies are distinguished, including photo­


grammetry technology and time-of-flight (ToF) distance measurement
by LiDAR. These techniques are used in a variety of scanning systems
with different capabilities and limitations [32]. Based on the field of
application, the selection of a suitable scanner that satisfies the re­
quirements is important.
The operation of LiDAR in smartphones is based on two principles:
structured light (SL) and ToF. Two types of ToF technology are used for
smartphone LiDAR: indirect ToF (iToF) and direct ToF (dToF) [33]. The
dToF complementary metal oxide semiconductor (CMOS) image sensor
produced by Sony has a dedicated Lumentum vertical-cavity surface-
emitting laser. Following the first introduction of a 3D sensing module Fig. 1. SL systems use triangulation methods to obtain 3D laser measurements
for Face ID in the front side of the iPhone X. Apple was prepare the new (source: based on [37]).

2
W. Błaszczak-Bąk et al. Measurement 219 (2023) 113286

camera with respect to the laser plane is known, the 3D image can be When the surface is flat and homogeneous, the problem involves
recovered using simple triangulation [37]. unambiguously identifying the same point in every smartphone position
The term ‘3D imaging’ refers to techniques that can acquire true 3D and determining the dependence, R. Nevertheless, the presented math­
data, i.e., values of some properties of a 3D object (e.g., density distri­ ematical relationships include the assumption that the surface is
bution), as a function of 3D coordinates (x,y,z)[41]. Because the surface generally nonplanar and described in a 3D space. In the case of detecting
of a 3D object is generally nonplanar, it is described in 3D space, and the small cracks and defects on homogeneous surfaces, the introduction of
imaging problem is called 3D surface imaging. The measurement result significant changes in the measurement methodology to render the
may be regarded as a map of the depth (or range), z, as a function of a applied mathematical formulae effective is justified.
position (x, y) in the Cartesian coordinate system. It may be expressed as
follows:
2.1. New measurement concept using smartphone with LiDAR
{zij = (xi, yj), i = 1, 2, ⋯, L, j = 1, 2, ⋯, M} (1)
Many engineering structures or their fragments require detailed
This process is also referred to as 3D surface measurement, range
measurements and dedicated approaches. Despite its high scanning
finding, range sensing, depth mapping, and surface scanning. A more
resolution, TLS is not invariably adequate. Accordingly, this paper
general 3D surface imaging system can acquire a scalar value, such as
presents the use of a smartphone with LiDAR as a supplementary mea­
surface reflectance, associated with each point on the nonplanar surface.
surement approach. First, the measurement of homogeneous and flat
The set of values forms a point cloud, as follows:
surfaces with a smartphone using SMTs is suggested. Fig. 2 shows the
{Pi = (xi, yi, zi, fi), i = 1, 2, ⋯, N} (2) measurements without SMTs, and Fig. 3 shows measurements obtained
with SMTs for comparison.
where fi represents the value of the ith surface point in the dataset. Measurements with a smartphone are performed while moving in a
Similarly, a colour image on a surface is represented by fixed direction to cover the entire object under study. The movement of
{Pi = (xi, yi, zi, ri, gi, bi), i = 1, 2, ⋯, N} (3) the smartphone means that each scan is implemented at a different angle
and from a different distance. If the measured surface is flat and ho­
where ri, gi, andbi represent the red (R), green (G), and blue (B) colour mogeneous, then a new point cloud is acquired with each movement
components associated with the ith surface point, respectively. The (1–n movements) of the smartphone. Unfortunately, because no char­
spectral surface properties may also be described by vectors with large acteristic points on the measured surface exist, successive scans are not
dimensions. properly combined. Consequently, the first scan of a point on the surface
The geometric relationship between an imaging sensor, SL projector, does not correspond to the same point in the second scan. This results in
and object surface point can be expressed by the triangulation principle capturing several planes that are as many as the number of phone
as scanning stops made. In this case, several measured point clouds rep­
resenting the plane are shifted and twisted relative to each other.
R = besline
sinθ
(4) For the measurement to proceed correctly, the so-called tie points
sin(α + θ) must be registered. These points enable the correct connection of point
The key technique for triangulation-based 3D imaging is to differ­ clouds during subsequent scans. Tie points are located in the charac­
entiate a single projected light spot from an acquired image using a two- teristic elements of an object. For flat and homogeneous surfaces, the
dimensional projection pattern. In a more general sense, actively illu­ geometry of the object must be modified by adding dedicated SMTs.
minated SL patterns may include spatial variations in all (x, y, z) di­ Laser beams reflected by SMTs are associated with tie points, which are
rections, thus becoming true 3D SL projection systems. shown in red in Fig. 3.
By implementing the new measurement methodology, only one

Fig. 2. Measurements without SMTs.

3
W. Błaszczak-Bąk et al. Measurement 219 (2023) 113286

Fig. 3. New concept of measurements with SMT.

plane is obtained after the test. The acquired dataset representing the in Fig. 4.
object contains all points from the scans merged using tie points (located The presented methodology differs significantly in the point cloud
on the SMT). Incorrect points (noise) may exist in the dataset; however, acquisition stage. The TLS point cloud contains the coordinates of points
these points are definitely fewer than those in measurements obtained representing the wall, but small changes on the wall are not recorded in
without SMTs. Consequently, incorrect points may be considered irrel­ it (due to the divergence of the laser beam and the size of the laser spot).
evant in the development process. A acquisition point cloud from a smartphone with LiDAR may have a
Taking into account the existing approach to the measurements using problem with combining point clouds from individual smartphone
a smartphone with LiDAR (Fig. 2) and the proposal of a new approach to movements.
measurements (Fig. 3), the research methodology was adopted as shown In the new approach using SMT, this problem disappears, because

Fig. 4. Proposed measurement methodology for inventory of building wall defects using LiDAR built into smartphones against the background of exist­
ing approaches.

4
W. Błaszczak-Bąk et al. Measurement 219 (2023) 113286

the task of the targets is to change/distort the geometry of the object, so The SMT for smartphones with LiDAR used in this study has been
as to obtain one point cloud for a quasi-flat surface. proposed by a team from the Department of Geodesy at the University of
The next step presented methodology is to combine all measured Warmia and Mazury in Olsztyn. The target is designed and dimensioned
point clouds in the appropriate software, compare and analyse the re­ using AutoCAD and produced on a 3D printer in three variants. These
sults, as a consequence of which changes/cracks can be detected and variants are shown in Fig. 6.
dimensioned. The SMTs have three versions. The first is an AutoCAD project in
which the target is dimensioned. The other is a printed filled geometric
solid. The third is a photograph of the target produced using a Zortrax
2.2. Equipment
M200 3D printer.
Measurements using the front scanner allows rapid data acquisition
from a relatively small area. Such measurement may supplement exist­ 3. Research objects
ing data and those obtained from difficult-to-access locations. Hence,
this is an inexpensive and rapid means to obtain high-resolution point Two types of measurements are conducted to verify the usefulness of
clouds. However, this study has some limitations. During the tests, the proposed SMTs. The first is the use of iPhone XR, and the second is
extremely small cracks were not measured, and point cloud visualisation the use of the Leica ScanStation C10 laser scanner.
did not capture the small cracks observed in the photographs. This is The sensor in iPhone XR, specifically the front TrueDepth sensor
because each time the phone moves, it recalculates the coordinates such (called FaceID by Apple Inc.) manufactured by TSMC sensor company,
that each subsequent scan can be linked to the previous one. Moreover, was used for measurements. Several applications, such as 3D Scanner
if the measurement area is homogeneous and has an uncomplicated App, Pix4Dcatch, or ScandyPro, are available for the LiDAR sensor of
geometry, combining such measurements with existing measurements is iPhone XR. The paid application (registered 09/05/2022) ‘ScandyPro’
difficult because of the absence of characteristic points. Accordingly, in version 1.10.7 from Louisiana Entertainment was used in this study.
this study, to increase the efficiency of measurements using a smart­ Point cloud files are in *.las format. Exports were created in a mesh in *.
phone with LiDAR, the geometry of a measured object is modified. This ply format and exported as a ‘.ply’ file.
change consists of distorting or varying the geometry of the object near Scanning with the ScandyPro app was performed by moving along
the measured fracture. This can be achieved by placing an SMT on the and up and down the wall with the iPhone XR front sensor covering
wall as a reference point of the smartphone with LiDAR. Targets dedi­ every part of the object of interest; the smartphone was at a distance of
cated to total stations and laser scanners are not suitable for scanning approximately 10 cm from the wall. The built-in.
with a smartphone, especially when a surface with small area is scanned; 7-MP front camera captured additional images, which were used to
the target must not be larger than the area to be measured. To date, add texture to the scan once it was completed.
various measurement targets dedicated to particular groups of geodetic The TLS measurement was performed based on two stations with 1-
measurements are available. Two fundamental types are used. mm resolution and at a distance of approximately 2 m. The datasets were
(1) Natural. then imported into Cyclone where they were combined with three
This so-called type is recognisable in the scan based on the object’s planar targets into one point cloud and then exported as a ‘.pts’ file.
morphological peculiarities, such as edges, windows, and identifiable The objects selected for measurement and analysis using the in­
and particular elements. struments are presented in Fig. 7.
(2) Artificial. Object 1 is located in the historic part of the building of the UWM
Artificial targets are positioned within the area to be scanned and Department of Geodesy in Olsztyn (Oczapowskiego Street); the building
generally consist of materials that are particularly visible and reflect the is undergoing renovation. Because the conservator of monuments has to
laser scanner wavelength. The goal is to create points that can be facilely provide an expert opinion on the condition of the building, each crack is
recognised by the tool to facilitate data acquisition and recording. analysed. A fragment of the wall with a crack selected for analysis is
Artificial targets can be of various types, depending on their shape. shown in Fig. 8.
Examples of these targets are presented in Fig. 5. Object 2 is located in a multi-family apartment block in Bartąg
The main problem related to targets is locating them during the (Morenowa Street). It is a fragment of a wall in the underground parking
survey. Undoubtedly, selecting the appropriate targets and correctly area of the building, as presented in Fig. 9.
positioning them are extremely important. An inadequately positioned
target is not clearly visible and therefore potentially useless. The use of 4. Results and analysis
targets of various shapes is preferred, and utilisation of different sizes is
mainly advantageous. Depending on the type of survey, size of the area The next step was to register the point clouds of each object captured
surveyed, and other deciding factors, selecting the correct target for by the TLS and smartphone in the CloudCompare software using the
scanning to be as simple as possible is necessary. Cloud2Cloud module. The selected fragments containing changes have
Considering the foregoing, SMTs must be utilised when performing been transferred to CloudCompare such that the changes can be
measurements with a smartphone. In particular, if the measured surface demonstrated and dimensioned. To present the defect map, a reference
is flat and has no complicated shape, SMTs enable the smartphone with plane was fitted into the point cloud based on the mean sum error
LiDAR to detect changes. method [42]. Next, the distances, zi, of all points from the reference

Fig. 5. Examples of targets: (a) total station target; (b) planar targets (square and circular) for laser scanner; and (c) spherical target for laser scanner.

5
W. Błaszczak-Bąk et al. Measurement 219 (2023) 113286

Fig. 6. SMTs for smartphone with LiDAR: (a) No. 1, (b) No. 2, and (c) No. 3.

plane were calculated. The data obtained using the smartphone and TLS crack can be detected by analysing the radiometric information of the
are presented in Fig. 10. Analyses are performed using the intensity value point cloud (intensity) from the TLS measurement. The absorption of the
and distance (z). laser beam at the crack site is high (blue in Fig. 10(e)). The absorption
In the analysis of the point clouds obtained by the Apple iPhone XR and dispersion of a laser beam depend on several factors, such as colour,
measurements, the crack on object 1 was not visible (Fig. 10(b)). After roughness, surface humidity, and edge effects [27,43,44]. Additionally,
using SMTs, the geometry of the measured object changed and regis­ a detailed analysis was implemented by creating two profiles (AA and BB
tering the crack became possible (Fig. 10(c)). The analysis shows that in (Fig. 10(a))) from the narrow strips (0.005 m wide) in the area of interest
addition to the crack, a shift between the two surfaces resulting from the on the wall. The analysis is presented in the ‘Discussion’ section.
crack occurs. Based on the geometric data (x,y,z) of the point cloud Similar research is performed on object 2. The purpose of measuring
captured by TLS, the crack was not registered (Fig. 10(d)). The crack this object is to verify the possibility of registering surface changes, such
width did not exceed 2 mm and was smaller than the laser spot of Leica as the merging of two precast concrete elements. The change analysis of
ScanStation C10 (spot size: equal to 4.5 mm from a distance 0 to 50 m). object 2 is shown in Fig. 11.
This is confirmed by the fact that the smartphone’s LiDAR laser spot is Only partial surface changes were mapped based on the 3D geo­
smaller than that of the TLS. Hence, the diameter of the laser beam used metric data from the TLS measurement. The smartphone measurement
in the smartphone with LiDAR is smaller than the width of the crack. A without SMTs yielded significantly worse results than the TLS

6
W. Błaszczak-Bąk et al. Measurement 219 (2023) 113286

details, and curved surfaces; however, it is less important for flat and
homogeneous surfaces. The diameter of the laser spot is assumed to be
smaller than the size of the smallest expected element. The smaller the
spot, the better the results because errors are fewer; this is because the
spot falls on the border of two areas (such as at the junction of surfaces of
varying colours) with different reflectance values.
Laser beam divergence is the angular spread of the light beam. The
specifications of the laser beam radiating from the source consist of two
parameters: the size of the beam and its divergence. The beam size is the
output diameter expressed in millimetres. Divergence is the expansion of
the beam with increasing distance from the source (expressed in
milliradians).
Considering the above comparison between the measurements of
Apple iPhone XR and Leica ScanStation C10, interpreting the dispersion
of points on a flat surface is essential. The distributions of points on a flat
surface and cracks are presented in Figs. 12 and 13 in the form of pro­
files, respectively. The cross-sections are perpendicular to the cracks.
Fig. 7. Objects and equipment: (a) smartphone with LiDAR for measuring The defects at the cross-sections are dimensioned, as shown in
object 1 and (b) TLS for measuring object 2. Fig. 14.
In analysing the measurement results of object 1 using the TLS
measurement. In contrast, the results of smartphone measurements with (Figs. 12 and 13), the dispersion of points was observed to be larger than
SMTs are considerably detailed. The comparison of measurements that of the smartphone measurements. This is due the influence of dis­
(Fig. 11(d)) with the RGB photograph (Fig. 11(a)) leads to the conclu­ tance measurement accuracy; for the terrestrial scanner, this accuracy is
sion that most of the significant surface changes have been sufficiently 4 mm [45]. Consequently, wall changes cannot be correctly determined
mapped, as confirmed by field inspections. The selected points are based on the TLS data. The detection of cracks and geometry changes,
indicated in Fig. 11, (b) and (c), for comparison using Z coordinates. which are smaller than the dispersion of points, is not possible using
Leica ScanStation C10. Based on the data captured by Apple iPhone XR
5. Discussion without SMT, the defects on the wall were not well mapped. The
application of SMTs during the measurement significantly improved the
The next stage of this research focuses on a detailed analysis of the quality of the data and reliably mapped the changes on the wall surface
results obtained. The objective is to determine the advantages of (Figs. 12 and 14 (green colour)). The foregoing indicates that a shift
employing a smartphone with LiDAR using a dedicated SMT. Before occurred on the wall surface (1–2 mm) at the crack location. During field
presenting the analysis results, the effects of using a laser scanner for inspection, the measurements of Apple iPhone XR with SMT are found to
measurement are examined. be consistent (Fig. 13).
The difference in measurements between the TLS and smartphone To measure object 2, a different analysis involving a surface analysis
with LiDAR may be due to the laser spot size, laser beam divergence of fragments with significant changes is implemented. Fig. 15 shows an
phenomenon, and measurement distance between the target and LiDAR. analysed fragment of object 2 as a real image and as a point cloud
The laser spot size is crucial for measuring small objects, fine structural captured by Apple iPhone XR with SMT.

Fig. 8. Object 1: (a) real view and (b) with SMTs.

7
W. Błaszczak-Bąk et al. Measurement 219 (2023) 113286

Fig. 9. Object 2: (a) real view and (b) with SMTs.

Fig. 10. Object 1 crack analysis: (a) real view; smartphone point clouds obtained (b) without SMT and (c) with SMT; and (d) and (e) point clouds obtained by TLS.

At the location where changes A, B, and C are observed (Fig. 15, (a) determined. A plane was fitted independently into both point clouds
and (b)), the defects are directly dimensioned by a measuring stick and obtained by Leica ScanStation C10 and Apple iPhone XR with SMT.
calliper (Fig. 15, (e)–(g), respectively). The surface change dimensions Next, the distances (di) between the point clouds and planes were
obtained by direct measurements (dd) and 3D coordinates of the point determined. Fig. 16 illustrates the fitted planes of the point clouds. The
cloud (d3D) are compared. In all cases, these dimensions are within the fundamental di statistical values, such as the minimum, maximum,
same the measurement error, |dd − d3D| ≤ ±2 mm. The measurements range, and root mean square (RMS) error, are calculated and summar­
obtained employing the proposed measurement technology with SMT ised in the table attached to Fig. 16.
allowed the identification of changes that are not visible after the TLS The test results show variations in the accuracy of the point clouds
measurement (Fig. 11(b)). These changes are visible in the X, Y, and Z obtained by TLS and iPhone XR. The RMS error can be interpreted as the
axes. The Z coordinate determines the possibility of identifying defects. accuracy of the fitted plane in a point cloud. The data ranges for the
The results of the smartphone measurement with LiDAR and SMT Leica ScanStation C10 is 0.011 m; the RMS error is 0.001 m. The data
(Fig. 15) illustrate the changes in the object. Hence, dimensioning based range for the Apple iPhone XR with SMT is 0.0008 m; and the RMS error
on the 3D coordinates of the point cloud captured by Apple iPhone XR is 0.0001 m. Thus, the point cloud obtained using the smartphone is
with SMT yields satisfactory results. characterised by greater accuracy. Note that this precision only applies
Additionally, the flatness of a 0.10 × 0.10-m wall fragment was to close-range (i.e., 2 dm) measurements and when special artificial

8
W. Błaszczak-Bąk et al. Measurement 219 (2023) 113286

Fig. 11. Object 2 change analysis: (a) real view; (b) point cloud obtained by TLS; and smartphone point clouds obtained (c) without SMT and (d) with SMT.

SMTs are used. A similar study of room measurements using iPhone 13 • it is necessary to use SMTs temporarily glued to the wall,
Pro and the Z + F 5006 h scanner has also been conducted [4]. The RMS • each defect is measured independently, so point clouds are registered
errors of the planes fitted to the wall were 0.001 and 0.003 m for the Z + in a different coordinate system,
F 5006 h scanner and iPhone 13 Pro, respectively. These studies showed • a lot of small defects on the walls make the method can be time-
that the data obtained by a smartphone with LiDAR were characterised consuming.
by lower accuracy compared with TLS data. Note that the measurements
by iPhone 13 Pro were made from a distance of several metres and Compared to the existing methods of measuring changes on the walls
without special artificial targets. of buildings, the proposed methodology is highly effective, especially for
In summary, the implementation of close-range measurements using detecting small cracks. The equipment presented in the proposed
a smartphone with LiDAR for small areas using special SMTs can derive methodology (smartphone and SMT) is easily available and inexpensive
point clouds with higher accuracy than point clouds derived by TLS. This compared to TLS. Similar as in TLS, the result of the smartphone mea­
is particularly important for the low-cost inventory measurements of surement is a point cloud that provides full 3D information about surface
building structure defects, such as cracks and cavities. changes in a small area. The advantage of TLS is the possibility of
The novel aspects of the proposed methodology and their importance capturing one point cloud of the entire object, whereas during the
are summarised in Table 1. measurement with an iPhone, several point clouds are usually obtained
The methodology of iPhone laser scanning with SMTs provides many separately for the examined areas. Overall, iPhone laser scanning can be
benefits, but some limitations should be noted: a good alternative to TLS in many engineering tasks. Theoretical aspects
of the use of SMTs in iPhone laser measurements presented in the ‘Ma­
• measurements are usually made in small areas at close range, terials and method’ section have been successfully verified by conducted

9
W. Błaszczak-Bąk et al. Measurement 219 (2023) 113286

Fig. 12. Interpretation of dispersion of points in cross-section AA: grey colour (Apple iPhone XR without SMT); green colour (Apple iPhone with SMT); and red dots
(Leica ScanStation C10). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 13. Interpretation of dispersion of points in cross-section BB: grey colour (Apple iPhone XR without SMT); green colour (Apple iPhone with SMT); and red dots
(Leica ScanStation C10). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 14. Field measurements: cross-sections (a) AA and (b) BB.

experiments. 6. Conclusion

An increasing number of readily available devices, such as smart­


phones, have built-in sensors. Smartphones with LiDAR technology have

10
W. Błaszczak-Bąk et al. Measurement 219 (2023) 113286

Fig. 15. Analysed fragment of object 2: (a) real image, (b) point cloud, (c) analysis of changes A and B, (d) analysis of change C, and (e)–(g) direct measurement
of changes.

the potential to revolutionize defect measurement in various industries, 2. The resolution 5 mm in iPhone Xr and short distance of the
including construction, manufacturing, and engineering, because it is an smartphone from the object gave opportunity to measure really small
easily accessible, small and light tool. Accordingly, a new methodology cracks (even about 1 mm). When the width of the crack is smaller than
for implementing measurements using a smartphone with LiDAR has the TLS Leica ScanStation C10 laser spot (spot size equal 4.5 mm from
been proposed. The methodology employs spatial metrics targets for 0 to 50 m) the cracks will not be visible.
measuring flat and homogeneous surfaces. The measurement of homo­ 3. The smartphone LiDAR laser spot is much smaller than TLS
geneous surfaces without the use of specially dedicated SMTs do not 4. The results of smartphone measurement with the SMT targets are
yield accurate results. The proposed methodology can be used to mea­ the most detailed for measuring quasi-flat and homogeneous wall sur­
sure homogeneous objects with cracks, defects, or small changes faces. The results of the smartphone measurement without the SMT gave
resulting from the construction employed. The possibility of using a significantly worse results compared to the TLS measurement. It is
smartphone with LiDAR in the inventory of small defects in buildings therefore justified to use SMTs to measure homogeneous, flat surfaces.
has been demonstrated by the conducted research. 5. SMT can be reused many times and is an integral part of LiDAR
Based on the presented research, the following conclusions can be smartphone measurement planning
drawn: The proposed technology will be developed in subsequent studies to
1. The LiDAR sensor in the iPhone XR can be used for the measure­ verify the importance of SMT placement according to the type of defect
ments, specifically the front TrueDepth sensor made by TSMC sensor and number of SMTs to be used.
company, called by Apple as FaceID

11
W. Błaszczak-Bąk et al. Measurement 219 (2023) 113286

Fig. 16. 3D view of fitted planes in point clouds of flat wall fragment obtained by Leica ScanStation C10 and Apple iPhone XR with SMT.

References
Table 1
Novel aspects of proposed methodology and their importance. [1] L. Teppati Losè, A. Spreafico, F. Chiabrando, F. Giulio Tonolo, Apple LiDAR sensor
for 3d surveying: tests and results in the cultural heritage domain, Remote Sens. 14
Novelty Application Advantage
(2022) 1–30, https://doi.org/10.3390/rs14174157.
SMT • for smartphones with • accurate combination [2] A. Murtiyoso, P. Grussenmeyer, T. Landes, H. Macher, First assessments into the
LiDAR, of scans during use of commercial-grade solid state lidar for low cost heritage documentation, in:
• ToF technology for measurement, Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. - ISPRS Arch., 2021: pp.
homogeneous surface • only one 3D point cloud 599–604. doi:10.5194/isprs-archives-XLIII-B2-2021-599-2021.
measurements, is obtained, [3] G. Luetzenburg, A. Kroon, A.A. Bjørk, Evaluation of the Apple iPhone 12 Pro LiDAR
for an application in geosciences, Sci. Rep. 11 (2021) 1–10, https://doi.org/
• for modifying geometry of • ease of use,
10.1038/s41598-021-01763-9.
objects,
[4] J. Zaczek, Evaluation of the LiDAR in the Apple iPhone 13 Pro for use in Inventory
New • inventory of difficult-to- • small changes
Works, FIG Proc. (2022) 1–19.
measurements access locations, impossible to [5] M. Pedro, D. Calvetti, Exploring the potential of iPad-LiDAR technology for
methodology • examination of technical determine by TLS can building renovation diagnosis : a case study, Buildings 13 (2023) 1–18.
conditions of homogeneous be detected, [6] C. Gollob, T. Ritter, R. Kraßnitzer, A. Tockner, A. Nothdurft, Measurement of forest
surfaces by conservator of • low measurement cost. inventory parameters with apple ipad pro and integrated lidar technology, Remote
monuments, Sens. 13 (2021) 1–35, https://doi.org/10.3390/rs13163129.
• measurement of outcrops, [7] S. Tatsumi, K. Yamaguchi, N. Furuya, ForestScanner: A mobile application for
• acquisition of point clouds measuring and mapping trees with LiDAR-equipped iPhone and iPad, Methods
representing cultural Ecol. Evol. (2022) 1–7. doi:10.1111/2041-210X.13900.
heritage objects for 3D [8] M.I. Razali, A.N. Idris, M.H. Razali, W.M. Syafuan, Quality assessment of 3D point
modelling, clouds on the different surface materials generated from iPhone LiDAR Sensor, Int.
J. Geoinformatics. 18 (2022) 51–58, https://doi.org/10.52939/ijg.v18i4.2259.
[9] S. Tavani, A. Billi, A. Corradetti, M. Mercuri, A. Bosman, M. Cuffaro, T. Seers,
E. Carminati, Smartphone assisted fieldwork: Towards the digital transition of
CRediT authorship contribution statement geoscience fieldwork using LiDAR-equipped iPhones, Earth-Science Rev. 227
(2022) 599–604, https://doi.org/10.1016/j.earscirev.2022.103969.
Wioleta Błaszczak-Bąk: Conceptualization, Methodology, Valida­ [10] Z. Mikalai, D. Andrey, H.S. Hawas, H. Tetiana, S. Oleksandr, Human body
measurement with the iPhone 12 Pro LiDAR scanner, in: AIP Conf. Proc, American
tion, Formal analysis, Investigation, Resources, Writing – original draft, Institute of Physics Inc., 2022, https://doi.org/10.1063/5.0078310.
Writing – review & editing, Visualization, Supervision. Czesław [11] N. Micheletti, J.H. Chandler, S.N. Lane, Investigating the geomorphological
Suchocki: Conceptualization, Methodology, Software, Validation, potential of freely available and accessible structure-from-motion photogrammetry
using a smartphone, Earth Surf. Process. Landforms. 40 (2015) 473–486, https://
Formal analysis, Investigation, Resources, Writing – original draft, doi.org/10.1002/esp.3648.
Writing – review & editing, Visualization, Supervision. Tomasz Koza­ [12] A. Corradetti, T. Seers, M. Mercuri, C. Calligaris, A. Busetti, L. Zini, Benchmarking
kiewicz: Methodology, Software, Data curation. Joanna Janicka: Different SfM-MVS Photogrammetric and iOS LiDAR Acquisition Methods for the
Digital Preservation of a Short-Lived Excavation: A Case Study from an Area of
Formal analysis, Writing – review & editing. Sinkhole Related Subsidence, Remote Sens. 14 (2022) 1–20, https://doi.org/
10.3390/rs14205187.
Declaration of Competing Interest [13] P. An, K. Fang, Y. Zhang, Y. Jiang, Y. Yang, Assessment of the trueness and
precision of smartphone photogrammetry for rock joint roughness measurement,
Meas. J. Int. Meas. Confed. 188 (2022) 1–13, https://doi.org/10.1016/j.
The authors declare that they have no known competing financial measurement.2021.110598.
interests or personal relationships that could have appeared to influence [14] P. An, K. Fang, Q. Jiang, H. Zhang, Y. Zhang, Measurement of rock joint surfaces by
using smartphone structure from motion (SFM) photogrammetry, Sensors
the work reported in this paper.
(Switzerland). 21 (2021) 1–23, https://doi.org/10.3390/s21030922.
[15] M. Jaud, M. Kervot, C. Delacourt, S. Bertin, Potential of smartphone SfM
Data availability photogrammetry to measure coastal morphodynamics, Remote Sens. 11 (2019)
1–17, https://doi.org/10.3390/rs11192242.
[16] K. Fang, A. Dong, H. Tang, P. An, B. Zhang, M. Miao, B. Ding, X. Hu,
Data will be made available on request. Comprehensive assessment of the performance of a multismartphone measurement
system for landslide model test, Landslides 20 (2022) 845–864, https://doi.org/
10.1007/s10346-022-02009-z.

12
W. Błaszczak-Bąk et al. Measurement 219 (2023) 113286

[17] K. Fang, P. An, H. Tang, J. Tu, S. Jia, M. Miao, A. Dong, Application of a multi- Build. Res. Inf. 49 (2020) 197–215, https://doi.org/10.1080/
smartphone measurement system in slope model tests, Eng. Geol. 295 (2021), 09613218.2020.1729687.
https://doi.org/10.1016/j.enggeo.2021.106424. [32] W. Ameen, A.M. Al-Ahmari, S.H. Mian, Evaluation of handheld scanners for
[18] R. Nowak, T. Kania, R. Rutkowski, E. Ekiert, Research and TLS (LiDAR) automotive applications, Appl. Sci. 8 (2018) 1–16, https://doi.org/10.3390/
Construction Diagnostics of Clay Brick Masonry Arched Stairs, Materials (Basel). 15 app8020217.
(2022) 1–19, https://doi.org/10.3390/MA15020552. [33] SystemPlus Consulting, Apple iPad Pro LiDAR Module - System Plus Consulting
[19] L. Truong-Hong, D.F. Laefer, Application of Terrestrial Laser Scanner in Bridge (2020) 1–3. https://s3.i-micronews.com/uploads/2020/06/SP20557-Yole-Appl
Inspection: Review and an Opportunity, IABSE Symp. Rep. 102 (2014) 2713–2720, e-iPad-pro-Lidar-Module_flyer.pdf.
https://doi.org/10.2749/222137814814070190. [34] H.J. Jung, N. Brasch, A. Leonardis, N. Navab, B. Busam, Wild ToFu: Improving
[20] J. Siwiec, G. Lenda, Integration of terrestrial laser scanning and structure from Range and Quality of Indirect Time-of-Flight Depth with RGB Fusion in
motion for the assessment of industrial chimney geometry, Measurement 199 Challenging Environments, in: Proc. - 2021 Int. Conf. 3D Vision, 3DV 2021, 2021:
(2022) 1–12, https://doi.org/10.1016/J.MEASUREMENT.2022.111404. pp. 1–10. doi:10.1109/3DV53792.2021.00034.
[21] Y. Cao, J. Liu, S. Feng, D. Li, S. Zhang, H. Qi, G. Cheng, Y. Frank Chen, Towards [35] G. D’Ettorre, M. Farronato, E. Candida, V. Quinzi, C. Grippaudo, A comparison
automatic flatness quality assessment for building indoor acceptance via terrestrial between stereophotogrammetry and smartphone structured light technology for
laser scanning, Measurement. 203 (2022) 111862. doi:10.1016/J. three-dimensional face scanning, Angle Orthod. 93 (2022) 358–363, https://doi.
MEASUREMENT.2022.111862. org/10.2319/040921-290.1.
[22] L. Fu, Z. Xing, G. Cheng, D. Li, N. Cui, Y. Frank Chen, Terrestrial laser scanning [36] T. Bakirman, M.U. Gumusay, H.C. Reis, M.O. Selbesoglu, S. Yosmaoglu, M.C. Yaras,
assisted dimensional quality assessment for space frame components, D.Z. Seker, B. Bayram, Comparison of low cost 3D structured light scanners for face
Measurement. 204 (2022) 112067. doi:10.1016/J.MEASUREMENT.2022.112067. modeling, Appl. Opt. 56 (2017) 985–993, https://doi.org/10.1364/ao.56.000985.
[23] H. Yang, M. Omidalizarandi, X. Xu, I. Neumann, Terrestrial laser scanning [37] F. Isgrò, F. Odone, A. Verri, An open system for 3D data acquisition from multiple
technology for deformation monitoring and surface modeling of arch structures, sensor, in: Proc. - Int. Work. Comput. Archit. Mach. Perception, CAMP, 2005: pp.
Compos. Struct. 169 (2017) 173–179, https://doi.org/10.1016/j. 1–6. doi:10.1109/camp.2005.13.
compstruct.2016.10.095. [38] X. Huang, Y. Zhang, Z. Xiong, High-speed structured light based 3D scanning using
[24] X. Xu, H. Yang, Intelligent crack extraction and analysis for tunnel structures with an event camera, Opt. Express. 29 (2021) 1–13, https://doi.org/10.1364/
terrestrial laser scanning measurement, Adv. Mech. Eng. 11 (2019) 1–7, https:// oe.437944.
doi.org/10.1177/1687814019872650. [39] T. Li, F. Duan, X. Fu, C. Liu, C. Liang, A. Chen, X. Li, A novel method for 3D
[25] P. Stałowska, C. Suchocki, M. Rutkowska, Crack detection in building walls based reconstruction in tight and restricted space based on improved line-structured
on geometric and radiometric point cloud information, Autom. Constr. 134 (2022) light, Meas. J. Int. Meas. Confed. 216 (2023) 1–10, https://doi.org/10.1016/J.
1–19, https://doi.org/10.1016/j.autcon.2021.104065. MEASUREMENT.2023.112930.
[26] G. Jacobs, Understanding Spot Size for Laser Scanning, Prof. Surv. Mag. (2006) [40] J. Cao, C. Li, C. Li, X. Zhang, D. Tu, High-reflectivity surface measurement in
1–13. structured-light technique by using a transparent screen, Meas. J. Int. Meas.
[27] C. Suchocki, J. Katzer, TLS technology in brick walls inspection, in: 2018 Balt. Confed. 196 (2022) 1–10, https://doi.org/10.1016/j.measurement.2022.111273.
Geod. Congr. (BGC Geomatics), IEEE, Olsztyn, 2018: pp. 359–363. doi:10.1109/ [41] J. Geng, Structured-light 3D surface imaging: a tutorial, Adv. Opt. Photonics. 3
BGC-Geomatics.2018.00074. (2011) 128–160, https://doi.org/10.1364/aop.3.000128.
[28] C. Suchocki, Comparison of time-of-flight and phase-shift TLS intensity data for the [42] S.-E. Chen, Laser Scanning Technology for Bridge Monitoring, chapter in book,
diagnostics measurements of buildings, Materials (Basel). 13 (2020) 1–18, https:// Laser Scanner Technology (2012), https://doi.org/10.5772/32794.
doi.org/10.3390/ma13020353. [43] C.J. Hawley, P.J. Gräbe, Water leakage mapping in concrete railway tunnels using
[29] C. Suchocki, M. Damięcka-Suchocka, J. Katzer, J. Janicka, J. Rapiński, LiDAR generated point clouds, Constr. Build. Mater. 361 (2022) 1–16, https://doi.
P. Stałowska, Remote Detection of Moisture and Bio-Deterioration of Building org/10.1016/J.CONBUILDMAT.2022.129644.
Walls by Time-Of-Flight and Phase-Shift Terrestrial Laser Scanners, Remote Sens. [44] Q. Wang, H. Sohn, J.C.P. Cheng, Development of high-accuracy edge line
12 (11) (2020) 1–15, https://doi.org/10.3390/rs12111708. estimation algorithms using terrestrial laser scanning, Autom. Constr. 101 (2019)
[30] C. Suchocki, W. Błaszczak-Bąk, Down-Sampling of Point Clouds for the Technical 59–71, https://doi.org/10.1016/j.autcon.2019.01.009.
Diagnostics of Buildings and Structures, Geosciences 9 (2) (2019) 1–14, https:// [45] Leica ScanStation C10 - Product Specifications, All-in-One Laser Scanner Any Appl.
doi.org/10.3390/geosciences9020070. (2011) 1–2.
[31] C. Suchocki, W. Błaszczak-Bąk, J. Janicka, A. Dumalski, Detection of defects in
building walls using modified OptD method for down-sampling of point clouds,

13

You might also like