Journal of Manufacturing Processes 57 (2020) 268–287

Contents lists available at ScienceDirect

Journal of Manufacturing Processes

journal homepage: www.elsevier.com/locate/manpro

Segmentation of workpiece surfaces with tool marks based on high T

definition metrology
Yaxiang Yina,b, Yiping Shaoc, Kun Wanga,b, Shichang Dua,b,*, Lifeng Xia,b
a
State Key Lab of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, 200240, China
b
School of Mechanical Engineering, Shanghai Jiao Tong University, No. 800 Dongchuan Road, Shanghai, 200240, China
c
College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou, China

ARTICLE INFO ABSTRACT

Keywords: Tool marks on face-milled surfaces contain huge information about the manufacturing processes. Direct surface
Surface segmentation segmentation based on tool marks is in favor of surface error sources diagnoses. However, traditional surface
Tool marks segmentation methods are prone to over-segmentation when partition surfaces with tool marks. This paper
High definition metrology proposes an improved segmentation approach to solve this problem. Based on surface topography measured by
high definition metrology (HDM), the surface segmentation methodology mainly involves four steps: automatic
subsurface selection, local thresh-holding, broken tool marks repairing and water segmentation (abbreviated as
“STRW” methodology). A novel concept called “periodic degree” is proposed and used as the criteria of sub-
surface selection. A binary image of tool marks is created by an adaptive local threshold. Broken tool marks are
identified by a distance threshold and repaired by a convex-hull based tool marks repairing algorithm. Finally,
water segmentation is applied to divide the surface into different regions and each region belongs to a unique
tool tooth trajectory. Three real cases from powertrain plant demonstrate the procedures of the methodology and
verify the effectiveness of the proposed methodology.

1. Introduction (corresponding to waviness) is critical for sealing performance [1].

Malburg first proposed a filtering method to relate profile waviness
The analysis of surface topography has been increasingly important parameters with sealing function [2]. Later Shao et al. extended the
in the field of mechanical manufacturing in recent years. Surface tex- filtering method to three-dimensional (3D) cases and proposed 3D
tures at different scales impact surface functionality in different ways. waviness-based leakage parameters [3]. Liao et al. filtered the waviness
For example, the intermedia scale of surface texture has significant subsurface from face-milled surface measured by high definition me-
impact on surface sealing performance. Periodic tool marks are the trology (HDM) using wavelet decomposition, and found that the wa-
main components of surface texture produced by milling. viness subsurface mainly consists of tool marks [4]. The researches
Face milling is a common machining technique to produce relative above strongly imply a relationship between tool marks and surface
smooth surfaces to guarantee surface sealing. Fig. 1 shows the surface sealing performance. Since tool marks are directly determined by mil-
appearance of face milling schematically. In this paper, tool mark is ling parameters, surface sealing can be improved by optimization of
conceptually defined as the local maximum of surface topography, and milling parameters.
the section view of the milled surface in Fig. 1 shows this concept. The The relationship between tool marks and milling parameters is in-
area between two adjacent tool marks is viewed as the region cut by vestigated mainly in two ways. The first one is surface simulation,
one tool tooth in one revolution. which means to input the milling parameters into models and simulate
In automobile engine assembly, the top surfaces of cylinder blocks the surface topography. The second one is surface monitoring, which
and bottom surfaces of cylinder heads are two key sealing surfaces means the measured surface topography is the input of models, and
produced by face milling. The topography of milled surfaces has direct milling parameters deduced from surface measurements are used for
relationship with its sealing performance. Based on experimental re- process monitoring. Surface simulation typically builds a dynamical
searches, Marie found that surface components at the intermedia scale model of milling processes and generates the surface topography by

Corresponding author at: School of Mechanical Engineering, Shanghai Jiao Tong University, No. 800 Dongchuan Road, Shanghai 200240, China.
⁎

E-mail addresses: yaxiang@sjtu.edu.cn (Y. Yin), syp123gh@zjut.edu.cn (Y. Shao), wangkun1224@sjtu.edu.cn (K. Wang), lovbin@sjtu.edu.cn (S. Du),
lfxi@sjtu.edu.cn (L. Xi).

https://doi.org/10.1016/j.jmapro.2020.06.035
Received 30 January 2020; Received in revised form 8 June 2020; Accepted 25 June 2020
Available online 03 July 2020
1526-6125/ © 2020 The Society of Manufacturing Engineers. Published by Elsevier Ltd. All rights reserved.
Y. Yin, et al. Journal of Manufacturing Processes 57 (2020) 268–287

Fig. 1. Surface appearance of face milling, single tooth.

Fig. 3. The top surface from a cylinder block of automobile engines.
simulation. For example, Shun et al. proposed a numerical model in-
tegrated data from ABAQUS and MATLAB to predict surface topography Fig. 3. The gray image is converted from point cloud of the surface
produced by face milling [5]. Sun et al. presented a unified simulation measured by HDM [9]. The gray pixel values are proportional to surface
model to predict 3D face-milled surface topography considering milling heights. Moreover, periodic tool marks are observable from image vi-
parameters and different kinds of initial set up errors [6]. Li et al. built a sion. Tool marks contain rich information about milling processes such
kinetic model of tool cutting to simulate the turning and milling surface as tool geometry, spindle tilt and milling parameters. Currently known
topography, and studied the effect of machined surface texture on its studies only focus on some overall statistical properties of surface
contact performance [7]. variations, however, there is more information deserving to mine by
For surface monitoring of milled surface topography, HDM is in- studying the tool marks one by one. For instance, the tool geometry and
dispensable for measuring the tool marks. A recently developed HDM its evolution during the machining processes can be revealed by the
instrument named Shapix has been applied in automobile engine plants cross-section view (perpendicular to tool tooth trajectory) of tool
successfully [8]. It can measure a surface in a large view and generate a marks, which is overlooked by traditional tool marks’ statistics. To
3D height map within 40 s. The measurement resolution in x-y-direc- conduct such delicate study on each tool mark, the first step is to par-
tion and z-direction are 150 µm and 0.05 µm respectively. Based on this tition the surface topography into distinct regions. And each region
novel measurement platform, many researches have been developed, belongs to a specific tool tooth trajectory and is bounded by tool marks.
including surface quality evaluation [9], monitoring and diagnosis Therefore, a suitable surface segmentation methodology is required to
[3,10], surface filtering [11–14], classification [15,16] and forecasting fulfill the above task of partitioning.
[17,18]. A recently published book provides deeper insights into HDM However, there is only one published surface segmentation stan-
based surface quality control and applications [19]. The HDM instru- dard: ISO 16610−85 and it is a branch of morphological areal filters
ment and an engine block face measured by HDM are shown in Fig. 2. [23]. It adopts watershed segmentation using wolf pruning to partition
With advanced HDM equipment, several milled surface monitoring a surface into distinct regions (hills and dales) separated by course lines
methods were presented. Nguyen et al. presented modeling and ex- and ridge lines and featured by points such as peak, pit and saddle
periments of a face milling process to correlate surface patterns ex- points. However, direct application of watershed algorithm to an image
tracted from straightened tool marks with cutting force variation [20]. is often disappointing. The image is over-segmented into a large
Later, Nguyen et al. proposed a method to monitor the spindle setup tilt number of small, irregular catchment basins that have no meanings. To
and deflection at each tool mark using HDM measured surface data overcome this problem, a large number of region merging algorithms
[21]. Wang et al. used 3D surface form indicators extracted from tool were proposed [24,25]. For example, Barré and Lopez proposed to
mark images converted from HDM measurement to monitor the wear of combine regions whose area is smaller than a certain threshold to its
wiper inserts [22]. neighbors until all the segmented regions are larger than the threshold
A surface from a cylinder block of automobile engines is shown in [25].

Fig. 2. Measurement by HDM.

269
Y. Yin, et al. Journal of Manufacturing Processes 57 (2020) 268–287

such as turning and milling surfaces, its segmentation results are often
unsatisfactory and have no relation with the machining processes. The
contribution of this paper is to solve this problem by proposing an
improved segmentation methodology. The methodology developed in
this paper could be an important complement and extension to ISO
16610−85 and it is a first trial to relate surface topography with ma-
chining processes by surface segmentation.
The surface segmentation methodology mainly consists of four
steps: automatic subsurface selection by periodic degree, local thresh-
holding to enhance tool marks, broken tool marks identification and
repairing, and watershed segmentation. This methodology is named as
“STRW” (subsurface selection, local thresh-holding, tool marks re-
pairing and watershed segmentation) according to the key operation of
each step.
The remainder of this paper is organized as follows. A detailed de-
scription of the proposed “STRW” methodology is presented in Section
2. In Section 3, three case studies demonstrate the effectiveness of the
proposed methodology and compares the segmentation result with
multiscale watershed segmentation. Section 4 discusses the limitation
of “STRW” methodology and future improvement direction.

2. The proposed methodology

The overview of “STRW” segmentation methodology is shown in

Fig. 4. It mainly consists of the following four steps.
Fig. 4. Framework of the proposed STRW surface segmentation methodology.
Step1: Since direct measured surface only shows the tool marks
pattern implicitly, for convenience of extracting tool marks in next
steps, it must be filtered first to get the subsurface that shows tool marks
The traditional watershed segmentation and its extensions have more clearly. However, the spatial frequency of tool marks is unknown,
gained success on some kinds of machined surfaces especially surfaces the most suitable subsurface for tool marks extraction can only be se-
with particle features. For example, a grinding wheel surface and a car lected manually according to engineers’ experiences. To achieve auto-
body panel shown in ISO 16610−85 have a satisfactory segmentation matic subsurface selection, a novel concept “periodic degree” is pro-
[23]. The concepts of watershed segmentation are originally from posed in this step and the subsurface with highest periodic degree is
geomorphology. It provides a new view of machined surface topo- proved to be the most suitable subsurface for tool marks extraction.
graphy and has a close relationship with surface functionality. How- Step2: To separate tool marks from its background clearly, thresh-
ever, it has a major weakness: for surfaces who have no particle features holding method is applied to the selected subsurface. Subsurface may

Fig. 5. Three types of function with different PD and their ACV.

270
Y. Yin, et al. Journal of Manufacturing Processes 57 (2020) 268–287

have a fluctuated form error and such fluctuation has a negative effect defined as Eq. (2). The “peak” is defined as any point that rises above
on tool marks extraction. To overcome this negative effect, a local forty-eight nearest neighbors, as shown in Fig. 6. The distance between
thresh-holding method is proposed in this step. Local threshold is de- two points in 2D cases are defined as the chessboard distance, as shown
termined only on the properties of a small neighborhood of a point, so it in Eq. (5). Therefore, the distance of yellow points, orange points and
can adapt to surface fluctuations. Therefore, a binary image of tool green points to P are 1, 2 and 3 respectively. So in 2D cases, a peak is
marks is created by applying a local thresh-holding method to the se- formally defined as Eq. (6), consistent with its 1D definition.
lected subsurface.
d (Pmn, Pij ) = max(|m i|, |n j|) (5)
Step3: After thresh-holding, the tool marks are separated from its
background. However, due to irregularities of real surfaces, breakage z ij Peaks ifz ij > z mn > z pq > z st , d (z ij , z mn) = 1, d (z ij, z pq) = 2, d (z ij , z st ) = 3 (6)
may occur in tool marks. Tool marks with breakage are identified by an
adaptive distance threshold, and they are repaired by a convex-hull By calculating the periodic degree of the auto-covariance function
based tool marks repairing algorithm. of different subsurfaces, the subsurface with maximum periodic degree
Step4: The final step is to partition the surface into regions that is selected as the most suitable subsurface for tool marks extraction.
belong to different tool tooth trajectories. Based on the former three To demonstrate this subsurface selection algorithm, a simulated
steps, direct application of watershed segmentation to the binary image surface (Eq. (7)) is tested,
after tool mark repairing can result in a satisfactory segmentation. z (x , y ) = 0.5 cos(2 x /6.4) + 0.1 cos(2 x /3) + 0.05x + 0.05y + 0.05randn (256) (7)
where cosine term of wavelength 6.4 is regarded as periodic tool marks,
2.1. Automatic subsurface selection by periodic degree cosine term of wavelength 3 is the simulated periodic noises, and the
last three terms are regarded as form error and random noises.
Direct measured surface only shows the tool marks pattern im- The area of simulated surface is 51.2 mm × 51.2 mm and sample
plicitly, for convenience of extracting tool marks, it must be filtered first spacing is 0.2 mm in horizontal directions. A biorthogonal wavelet
and to find the subsurface which shows tool marks most clearly. Since (bior4.4) decomposition is applied to this surface and it is decomposed
tool marks have a fixed frequency, the subsurface that contains tool into five levels. Fig. 7(a) shows the simulated surface in 3D form.
marks must have a dominant periodic feature. ISO 25178−2 has de- Fig. 7(b) ∼ (f) are images of decomposed subsurface D1 ∼ D5
fined many kinds of surface texture parameters, such as texture aspect (defineDi = DiH + DiV + DiD , and wavelet decomposition is a recursive
ratio (Str) and auto-correlation length (Sal) [26]. Sal is related to the form of Ai = Ai + 1 + Di + 1). From vision observation, Fig. 7(e) seems to
wavelength of the main frequency components of surfaces, and Str is a have a strongest periodic degree. The AACV of subsurfaces are plotted
measure of surface anisotropy. However, there is no existing parameter in Fig. 8, and the last three graphs are similar and the most periodic
to measure the periodic degree of a surface. subsurface cannot be determined from vision. The periodic degree de-
To quantify the periodic level of a profile or surface, a novel concept fined in this section are calculated and listed in Table 1, and D4 has the
called “periodic degree” (PD ) is proposed in this section. Three types of largest value, which is consistent with observation from Fig. 7. So in
function with different periodic level is shown in Fig. 5(a), (c) and (e). It this step, subsurface D4 is selected.
is obvious that the function 5(a) has a maximum periodic level while
function 5(e) has a minimum periodic level. The 1D auto-covariance 2.2. Binarization by local thresh-hold of subsurface
function ( ACV ) (defined in Eq. (1)) shown in Fig. 5 (b), (d) and (f)
reflects the periodicity difference clearly, so PD can be defined as a To separate tool marks from its background, an image thresh-
ratio of the secondary peak height (Hsp ) to the highest peak height holding method is applied, and the thresh-holding method is defined as
(Hhp ), as shown in Eq. (2). And the corresponding periodic degrees are Eq. (8).
PDa = 125.6/157.1 = 0.80, PDc = 2.44/15.05 = 0.16, PDe = 0,
consistent with vision observation. It is worth noting that the definition 1ifI (x , y ) > T
B (x , y ) =
of “peak” in this paper is a local maximum with an uphill path and a 0ifI (x , y ) T (8)
downhill path with respective least path length of three. According to
There are two common thresh-holding methods: global thresh-
this definition, Fig. 5(f) has no peaks, and in this situation, the periodic
holding and local thresh-holding. Classical global thresh-hold method is
degree is set to be zero.
the Ostu’s method [27]. An optimal threshold T* is found to achieve
1
nx i
maximum inter-class variance. An improved adaptive approach is to
ACV (i ) = y (i ) y (i + i ), 0 i nx 1 compute a threshold at every point (x , y ) in the image based on speci-
nx i=1 (1)
fied properties computed in a neighborhood of (x , y ) . In this algorithm,
Hsp the standard deviation and mean of the heights in the neighborhood of
PD = a point are used to determine the local threshold. Let xy and m xy denote
Hhp (2)
the standard deviation and mean value of the heights of adjacent points
yi > yi 1 > yi 2 > yi 3 centered at (x , y ) . The local threshold is defined as Eq. (9), where a and
yi Peak if b are nonnegative constants.
yi > yi + 1 > yi + 2 > yi + 3 (3)
Txy = a + bm xy (9)
The definition of auto-covariance function, corresponding “peak” xy

and periodic degree can be extended to 2D cases. The areal auto-cov- To illustrate the adaptability of local thresh-holding method, the
ariance function ( AACV ) for 3D surfaces is defined by Eq. (4). 2 x
( )
surface simulated by z (x , y ) = 0.5 cos 6.4 + 0.05x + 0.05y is taken
1
nx i ny j for an example. Fig. 9(a) shows the image form of this surface and it has
AACV (j , i ) =
nx•ny
z (j, i) z (j + j , i + i ), 0 j ny 1, 0 i nx 1 clear periodic marks with a gradient background. Fig. 9(b) shows the
i=1 j =1
image after global thresh-holding using Ostu’s method and it doesn’t
(4) capture the periodic marks. Fig. 9(c) shows the image after local thresh-
The 2D periodicity degree is defined the same as 1D cases, namely, holding (the neighborhood region is a window size of 7 × 7, and

271
Y. Yin, et al. Journal of Manufacturing Processes 57 (2020) 268–287

from adjacent marks. The elements IRD(i,j) represents the minimum

distance between region labeled i and another region labeled j and it is
defined by Eq. (10).
IRD (i , j) = min (x pi xqj )2 + (ypi yqj )2 (p = 1, 2, , Ni, q = 1, 2, , Nj, i j)

(10)
where Ni is the number of points in region Xi and Nj is the number of
points in another region Xj . And min(IRD) is a vector containing the
minimum value of each column from the matrix IRD. The maximum of
min(IRD) is approximately the distance between adjacent tool marks. So
the adaptive distance threshold DT to distinguish broken marks from
adjacent marks can be set to be a proportion of max(min(IRD)) ac-
cording to engineering experiences (a proportion coefficient C between
0.5 and 0.9 is recommended), and DT is defined by Eq. (11).
DT = C × max(min(IRD )) (11)
If min(IRD(i)) is less than DT, then the region labeled i can be as-
signed to the group of broken tool marks. Otherwise, the region labeled
Fig. 6. A 2D peak and its neighbors.
i should be assigned to the group of continuous tool marks. By repairing
the group of broken tool marks and adding the group of continuous tool
a = 1.5, b = 1), it captures the periodic marks successfully.
marks, the image of fully repaired tool marks could be obtained. This
thought of dividing and repairing is summarized in Fig. 11.
2.3. Broken tool marks identification and repairing The “tool mark repairing” algorithm is designed to deal with the sub
image containing all broken tool marks. A “convex hull” generation
After local thresh-holding, the tool marks may not be perfect as algorithm is adopted to help the tool marks repairing.
expected because of some machining errors. A common situation is the In mathematics, the convex hull of a points set S in the Euclidean
unexpected breakage of continuous tool marks as shown in Fig. 10(a). plane is the smallest convex set that contains S and it is also the set of all
Unexpected breakage of continuous tool marks will bring obstacles to convex combinations of points in set S. The formal definition of convex
correct segmentation. Naturally, it is desired to develop an algorithm to hull could be expressed as formula (12),
automatically repair the breakage as Fig. 10(b) shows.
To repair the breakage of tool marks, the broken parts of marks |X| |X|
Convhull (X ) = i xi i: 0 = 1,x i X
must be identified firstly. Since the size of breakage gap of tool marks is i i
i=1 i=1 (12)
usually smaller than the distance of adjacent tool marks. The inter-re-
gion distance (IRD) matrix can be utilized to distinguish broken marks where |X | denotes the number of points in set X . Fig. 12(a) shows an

Fig. 7. Simulated surface and decomposed subsurfaces.

272
Y. Yin, et al. Journal of Manufacturing Processes 57 (2020) 268–287

Fig. 8. AACV plots of subsurfaces.

example of a set of points and its convex hull. The idea of repairing the shows the simulated tool marks with breakage, and the sub image
breakage by convex hull is illustrated in Fig. 12(b). First, find the edge BW_broken and sub image BW_connected are shown in Fig. 14(b) and (c)
points near the breakage gap, then construct a convex hull of the points respectively. The convex hulls used to repair the breakage are plotted
set, finally fill the convex hull region. The quickhull algorithm for with red boxes in Fig. 14(d). The sub image of repaired tool marks
convex hull [28] is adopted to generate the convex hull connecting the BW_repaired and corresponding full image BW_full are shown in
broken parts. The detailed procedures of tool mark repairing algorithm Fig. 14(e) and (f) respectively.
are demonstrated in Fig. 13.
To illustrate the repairing ability of the proposed algorithm, a
simple example of broken marks is taken for demonstration. Fig. 14(a) 2.4. Watershed segmentation

After repairing broken tool marks, the periodic tool marks pattern
Table 1 has been clearly shown. However, for surface segmentation, dividing
Periodic degree of decomposed subsurface D1 ∼ D5. line of one-pixel width is desired to achieve better segmentation ac-
Subsurface D1 D2 D3 D4 D5 curacy. The task of this final step is to thin tool marks to one-pixel
width, and keep the thinned tool marks as close to the centerline of the
Periodic degree 0.013 0.086 0.539 0.742 0.693 origin tool marks as possible.
Fortunately, classical water segmentation could help to fulfill this
task. Fig. 15(a) shows a simulated diagonal tool mark, and the dividing

Fig. 9. Simulated surface and its binary form by different thresh-holding methods.

273
Y. Yin, et al. Journal of Manufacturing Processes 57 (2020) 268–287

Fig. 10. Unexpected breakage of tool marks and their repairing scheme.

Fig. 11. The work-flow of the broken tool marks identification and repairing.

274
Y. Yin, et al. Journal of Manufacturing Processes 57 (2020) 268–287

Fig. 12. The concept of convex hull and its application to repair tool marks’ breakage.

Fig. 13. Tool mark repair algorithm.

line by watershed segmentation is central diagonal except for two ends, CELL-O Machining Center using a CBN milling cutter with a diameter of
as shown in Fig. 15(b). 200 mm. The cutting speed was 1300 rpm, the depth of cut was 0.5 mm,
and feed rate was 3360 mm/min. The HDM tool used to measure the
3. Case study cylinder block in this case is ShaPix3D® 3000 series, with a vertical
resolution of 0.05 µm and lateral resolution of 150 µm respectively. Its
3.1. Case study Ⅰ height measurement range is ± 5mm and the vertical accuracy is 1 µm .
Its field of view is 280 mm × 280 mm, and the maximum number of
3.1.1. STRW algorithm experiment sampling points is 4 million in each view. For workpieces whose sizes
Surfaces produced by milling and turning have apparent periodic exceed the field of view, the HDM could stitch multiple point clouds to
tool marks. In this case, the top surface of a B12 cylinder block from an generate the full view of the surfaces.
automobile factory is taken as a case to study the proposed surface A B12 cylinder block is shown in Fig. 16(a) and a sample of its top
segmentation methodology. The milling process was carried in an EX- surface is shown in Fig. 16(b). The partial enlarged view of Fig. 16(b)

275
Y. Yin, et al. Journal of Manufacturing Processes 57 (2020) 268–287

Fig. 14. Simulated tool marks with breakage and its repairing procedures.

Fig. 15. A diagonal tool mark and its watershed segmentation.

shows obvious periodic tool mark features. For the convenience of The local thresh-holding result of D4 is shown in Fig. 19(a). The
showing the segmentation procedures, a small sample of the surface is neighborhood region is defined as a square window of size 7 by 7. And
taken for study, as shown in Fig. 16(b). the threshold is T = 1.4 + m (in Eq. (9), aandb are experienced values
A biorthogonal wavelet (bior4.4) is applied to decompose the and may change according to different surface generation methods). In
sampled surface data to different sub-bands, which is shown in Fig. 17. Fig. 19(a), some white blocks appear due to noises. To remove these
To select the most suitable subsurface for segmentation, the areal small noises, the white blocks whose area is less than a certain
auto-covariance function of each subsurface is calculated and shown in threshold are discarded. In this case, the threshold value is 10. And the
Fig. 18. It is clear that the central point (corresponding to zero offset) is image after noise removal is shown in Fig. 19(b). The labels of each
the highest peak in every AACV plot. From the direct vision of Fig. 17, region is shown in Fig. 19(c).
it could be inferred that subsurface D4 has the strongest periodic degree. As noticed in Fig. 19(b), there is a breakage occurring in the right-
The proposed periodic degree values listed in Table 2 also agree with top tool mark. To detect the breakage, the IRD matrix is calculated and
this observation. So the subsurface D4 is selected for next steps. listed in Table 3. The last row shows the minimum value of each

276
Y. Yin, et al. Journal of Manufacturing Processes 57 (2020) 268–287

Fig. 16. B12 cylinder block and a sample from its top surface.

Fig. 17. Decomposed subsurfaces of sampled surface in case1.

column and represents the minimum distance of each region to its boxes in Fig. 20(d). The sub image of repaired tool marks BW_repaired is
nearest neighbor. shown in Fig. 20(e). Adding the repaired tool mark image BW_repaired
Set C = 0.5, the distance threshold determined from the last row is with the image of connected tool marks BW_connected, the fully re-
DT = 50 %×max(min(IRD)) = 50 %×13.34 = 6.67. According to DT, paired tool mark image BW_full is obtained and shown in Fig. 20(f).
Region 4 and Region 5 are broken tool marks and need to be re-con- Applying the classical watershed algorithm to the final repaired tool
nected. So Fig. 19(b) can be partitioned into two sub images: one is the mark image, the segmentation results superimposed on original surface
broken tool marks BW_broken and the other one is connective tool topography and selected subsurface are shown in Fig. 21(a) and (b)
marks BW_connected. For the broken tool marks shown in Fig. 20(b), respectively. Fig. 21(c) labels each partitioned region with a color for
“tool mark repair” algorithm proposed in section 2.3 is applied to repair better demonstration. The watershed lines are specially labelled with
the breakage. The convex hull to repair the breakage is shown by red red color to enhance the vision.

277
Y. Yin, et al. Journal of Manufacturing Processes 57 (2020) 268–287

Fig. 18. AACV of subsurface D1 to D5in case1.

Table 2 Table 3
Periodic degree of subsurface D1 to D5in case1. The inter region distance matrix IRDin case1.
Subsurface D1 D2 D3 D4 D5 Distance Region1 Region2 Region3 Region4 Region5

Periodic degree 0.038 0.181 0.300 0.398 0.045 Region1 12.81 31.06 10.77 12.04
Region2 12.81 13.34 30.41 30.41
Region3 31.06 13.34 50.01 51.86
Region4 10.77 30.41 50.01 5.66
A cross section perpendicular to tool marks is plotted in Fig. 22. The
Region5 12.04 30.41 51.86 5.66
points on the tool marks are marked with A, B, C and D. It is clear that min(IRD) 10.77 12.81 13.34 5.66 5.66
A, B, C and D are local peak points, which is consistent with the concept
of tool marks in Fig. 1. Generally, the tool shape and scar will be ne-
gative replicated in surface and repeated at the feed spacing. This 3.1.2. Comparison with multiscale watershed segmentation algorithm
phenomenon also holds in this case, the shapes of profile AB, BC and CD To compare the segmentation algorithm proposed in this paper with
are similar to each other. The average feed spacing calculated from classical methods, the multiscale watershed algorithm proposed by
Fig. 22(b) is f = = 2.9mm . The cutting speed was
AB + BC + CD 2.7 + 3 + 3
3
= 3
Barré and Lopez [25] is applied to the same surface data. The multiscale
1300 rpm, and feed rate was 3360 mm/min. The theoretical feed spa- watershed algorithm adopts series of area threshold to perform the
3360 2.9 2.58
cing is f = 1300 = 2.58mm . The relative error is r = 2.58 = 12.4%. multiscale watershed segmentation and gains better results in some
cases than traditional watershed algorithm. The idea is to fill up non-

Fig. 19. Local thresh-holding of D4 and its noise removal in case1.

278
Y. Yin, et al. Journal of Manufacturing Processes 57 (2020) 268–287

Fig. 20. Tool marks repairing procedures in case1.

Fig. 21. Final surface segmentation result in case1.

Fig. 22. A cross-section view of tool marks in case1.

279
Y. Yin, et al. Journal of Manufacturing Processes 57 (2020) 268–287

3.2. Case study Ⅱ

A sample from the top surface of a N12 cylinder block is tested in

this case. The N12 block was milled with a cutting speed 1200 rpm, a
depth of cut 0.6 mm, and a feed rate of 3000 mm/min. Fig. 25 shows a
N12 cylinder block and the height map of its top surface measured by
Shapix. Similar to case 1, a sample was taken for segmentation. The
wavelet (bior4.4) decomposed the sample into subsurfaces (shown in
Fig. 26), and their AACV plots are shown in Fig. 27. The periodic de-
grees of each subsurface are listed in Table 4. Since D4 has a maximum
of periodic degree, D4 is selected for next steps.
D4 is locally thresh-held by T = 1.2 + m to generate a binary
image. Each region of the binary image is labelled and shown in
Fig. 28(a). To detect and repair the breakage of tool marks, the IRD
matrix is calculated and listed in Table 5. Set C = 0.8, the distance
threshold determined from the last row is DT = 0.8×max(min(IRD)) =
0.8 × 10.63 = 8.50. According to DT, Region 1, 3, 4 and 5 are broken
tool marks and need to be re-connected. The details of the repairing
procedures are shown in Fig. 28.
Fig. 23. Multiscale watershed segmentation algorithm. Applying the classical watershed algorithm to the final repaired tool
mark image, the segmentation results are shown in Fig. 29.
A cross section perpendicular to tool marks is plotted in Fig. 30. The
significant catchment basins whose area is less than the threshold and
points on the tool marks are marked with A, B and C. Although A, B and
then determine watershed lines for the modified surface to obtain a new
C are not exactly local peak points, their position are very close to local
segmentation. The pseudocode of this algorithm is shown in Fig. 23.
peaks. The shapes of profile AB and BC are also similar to each other.
Fig. 24 shows the multiscale watershed segmentation results and the
The average feed spacing calculated from Fig. 30(b) is
area threshold A is from 1 to 57.7, equally spaced by interval 8.1.
= 2 = 2.52mm . The theoretical feed spacing is
AB + BC 5.04
f=
Compared with Fig. 21, it is clear that Fig. 24 has no effective seg- 2
2.52 2.5
= 2.5mm . The relative error is r = 2.5 = 0.8%.
3000
mentation since the watershed lines are messy and have no physical f= 1200
meanings. And the multiscale segmentation has no trend to ap- Fig. 31 shows the multiscale watershed segmentation results and the
proaching the segmented result in Fig. 21 neither. So it could be con- area threshold A is from 1 to 44.4, equally spaced by interval 6.2.
cluded that traditional watershed segmentation cannot partition the Definitely, the multiscale watershed segmentation cannot partition the
surfaces with periodic tool marks effectively. surfaces with periodic tool marks into different tool tooth trajectories.
Since the B12 engine block top surface is only one of the many types
of milled surfaces, other two engineering surfaces are studied in this
3.3. Case study Ⅲ
section to further validate the effectiveness of the proposed metho-
dology.
To verify the effectiveness of the proposed algorithm furtherly, a
sample from the top surface of a bearing cap is taken for study in this

Fig. 24. Results of multiscale watershed segmentation in case1.

280
Y. Yin, et al. Journal of Manufacturing Processes 57 (2020) 268–287

Fig. 25. N12 cylinder block and a sample from its top surface.

Fig. 26. Decomposed subsurfaces of sampled surface in case2.

case. The top surface of the bearing cap was milled with a cutting speed breakage of tool marks, the IRD matrix is calculated and listed in
of 1400 rpm, a depth of cut 0.5 mm and a feed rate of 3800 mm/min. Table 7. Set C = 0.8, the distance threshold determined from the last
Fig. 32(a) shows a bearing cap. The left top surface was measured by row is DT = 0.8×max(min(IRD)) = 0.8 × 10.63 = 8.50. According to
HDM, and the height map is shown in Fig. 32(b). A sample is taken form DT, Region 2, 3 and 5 are broken tool marks and need to be re-con-
the measurement to test the proposed methodology. nected. The details of the repairing procedures are shown in Fig. 35.
The wavelet (bior4.4) decomposed subsurfaces and their AACV The final watershed segmentation results are shown in Fig. 36.
plots are shown in Fig. 33 and Fig. 34 respectively. The periodic degrees A cross section perpendicular to tool marks is plotted in Fig. 37. The
of each subsurface are listed in Table 6. D4 is selected again, since its points on the tool marks are marked with A, B and C. Although point B
periodic degree is maximum. is not exactly a local peak, its position is very close to a local peak. The
The local thresh-holding parameters are the same as case 2. The shapes of profile AB and BC are also similar to each other. The average
labels of each mark are shown in Fig. 35(a). To detect and repair the feed spacing calculated from Fig. 37(b) is f = = 2 = 2.55mm .
AB + BC 5.1
2

281
Y. Yin, et al. Journal of Manufacturing Processes 57 (2020) 268–287

Fig. 27. AACV of subsurface D1 to D5 in case2.

Table 4 Table 5
Periodic degree of subsurface D1 to D5 in case2. The inter region distance matrix IRD in case2.
Subsurface D1 D2 D3 D4 D5 Distance Region1 Region2 Region3 Region4 Region5

Periodic degree 0.111 0.173 0.273 0.447 0.221 Region1 12.04 5.66 27.80 31.76
Region2 12.04 10.63 11.40 11.31
Region3 5.66 10.63 29.83 27.46
Region4 27.80 11.40 29.83 7.07
Region5 31.76 11.31 27.46 7.07
The theoretical feed spacing is f =
3800
1400
= 2.71mm . The relative error is min(IRD) 5.66 10.63 5.66 7.07 7.07
2.71 2.55
r= 2.71
= 5.9%.
Fig. 38 shows the multiscale watershed segmentation results and the

Fig. 28. Tool marks labelling and repairing procedures in case2.

282
Y. Yin, et al. Journal of Manufacturing Processes 57 (2020) 268–287

Fig. 29. Final surface segmentation result in case2.

Fig. 30. A cross-section view of tool marks in case2.

Fig. 31. Results of multiscale watershed segmentation in case2.

area threshold A is from 1 to 50, equally spaced by interval 7. Similar to segmentation image is close to the theoretical values calculated from
the conclusion of case 1 and 2, the multiscale watershed segmentation cutting parameters, with an average relative error about 6.4 %. And the
fails to partition the surfaces with periodic tool marks. multiscale watershed segmentation fails to partition the surfaces cor-
In all three cases, the points on tool marks are local peaks or very rectly in all three cases. Therefore, the conclusion that “STRW” seg-
close to local peaks; each profile segment representing negative replica mentation methodology could effectively divide the milling surfaces
of tool shape are similar to each other; the calculated feed spacing from with periodic tool marks is drawn.

283
Y. Yin, et al. Journal of Manufacturing Processes 57 (2020) 268–287

Fig. 32. A bearing cap and a sample from its top surface.

Fig. 33. Decomposed subsurfaces of sampled surface in case3.

284
Y. Yin, et al. Journal of Manufacturing Processes 57 (2020) 268–287

Fig. 34. AACV of subsurface D1 to D5 in case3.

Table 7
The inter region distance matrix IRDin case3.
Table 6
Periodic degree of subsurface D1 to D5 in case3. Distance Region1 Region2 Region3 Region4 Region5

Subsurface D1 D2 D3 D4 D5
Region1 15 9.22 27.29 11.31
Region2 15 7.81 18.25 24.41
Periodic degree 0.081 0.109 0.237 0.392 0.189
Region3 9.22 7.81 10.63 7.81
Region4 27.29 18.25 10.63 12.37
Region5 11.31 24.41 7.81 12.37
min(IRD) 9.22 7.81 7.81 10.63 7.81

Fig. 35. Tool marks labelling and repairing procedures in case3.

285
Y. Yin, et al. Journal of Manufacturing Processes 57 (2020) 268–287

Fig. 36. Final surface segmentation result in case3.

Fig. 37. A cross section view of tool marks in case3.

Fig. 38. Results of multiscale watershed segmentation in case3.

286
Y. Yin, et al. Journal of Manufacturing Processes 57 (2020) 268–287

4. Conclusion surface variation in face milling large-scale workpiece with complex geometry.
ASME Trans Manuf Sci Eng 2019;141(3):031009.
[6] Jin S, Liu S, Zhang XP, Chen K. A unified prediction model of 3D surface topography
This paper proposes an improved surface segmentation metho- in face milling considering multi-error sources. Int J Adv Manuf Technol
dology called “STRW” to partition surfaces with periodic tool marks 2019;102(1):705–17.
into distinct regions, and each region belongs to a unique tool tooth [7] Li PY, Zhai YF, Huang SK, Wang QD, Fu WP, Yang HP. Investigation of the contact
performance of machined surface morphology. Tribol Int 2017;107:125–34.
trajectory. In the first step, the measured surface is decomposed into [8] Huang ZH, Shih AJ, Ni J. Laser interferometry hologram registration for three-di-
different subsurfaces, and a novel concept called “periodic degree” is mensional precision measurements. ASME Trans Manuf Sci Eng
proposed and used as the criteria of subsurface selection. Then the se- 2006;128(4):887–96.
[9] Wang M, Xi LF, Du SC. 3D surface form error evaluation using high definition
lected subsurface is locally thresh-held to separate tool marks from its metrology. Precis Eng 2014;38(1):230–6.
background. Broken tool marks may occur due to surface irregularities, [10] Shao YP, Yin YX, Du SC, Xi LF. A surface connectivity based approach for leakage
and they are identified by an adaptive distance threshold and repaired channel prediction in static sealing interface. J Tribol 2019;141(6):062201.
[11] Du SC Liu CP, Huang DL. A shearlet-based separation method of 3D engineering
by a convex-hull based tool marks repairing algorithm. Finally, the
surface using high definition metrology. Precis Eng 2015;40:55–73.
classical watershed segmentation is applied to partition the surface into [12] Wang M, Shao YP, Du SC Xi LF. A diffusion filter for discontinuous surface mea-
distinct meaningful regions. The “STRW” segmentation methodology sured by high definition metrology. Int J Precis Eng Manuf 2015;16(10):2057–62.
could be an important complement to ISO 16610−85 and it provides a [13] Shao YP, Wang K, Du SC, Xi LF. High definition metrology enabled three dimen-
sional discontinuous surface filtering by extended tetrolet transform. J Manuf Syst
new way to relate surface topography with machining processes. 2018;49:75–92.
The proposed “STRW” methodology has well application to con- [14] Du SC, Liu T, Huang DL, Li GL. A fast and adaptive bi-dimensional empirical mode
tinuous surfaces with periodic tool marks. For machined surfaces decomposition approach for filtering of workpiece surfaces using high definition
metrology. J Manuf Syst 2018;46:247–63.
without periodic tool marks, their segmentation may require special [15] Du SC, Huang DL, Wang H. An adaptive support vector machine-based workpiece
treatment according to their machining processes. For milled surface surface classification system using high-definition metrology. IEEE Trans Instrum
that has big holes and bores as Fig. 3 shows, additional treatment is Meas 2015;64(10):2590–604.
[16] Du SC, Liu CP, Xi LF. A selective multiclass support vector machine ensemble
required to link the tool marks separated by holes and bores. For the classifier for engineering surface classification using high definition metrology.
further researches, tool marks on the whole surface of the cylinder ASME Trans Manuf Sci Eng 2015;137(1):011003.
block will be classified and parameterized to study the relationship [17] Wang M, Du SC, Xi LF. Predicting machined surface topography based on high
definition metrology. 15th IFAC Symposium on Information Control Problems in
between milling processes parameters and tool marks properties. Manufacturing INCOM 2015:1013–7.
[18] Shao YP, Du SC, Xi LF. 3D machined surface topography forecasting with space-
Declaration of Competing Interest time multioutput support vector regression using High definition metrology. 37th
Computers and Information in Enginnering ConferenceCleveland, Ohio, USA
2017:V001T002A069.
The authors declare that they have no known competing financial [19] Shichang Du LX. High definition metrology based surface quality control and ap-
interests or personal relationships that could have appeared to influ- plications. Singapore: Springer; 2019.
[20] Nguyen HT, Wang H, Hu SJ. Characterization of cutting force induced surface shape
ence the work reported in this paper. variation in face milling using high-definition metrology. ASME Trans Manuf Sci
Eng 2013;135(4):041014.
Acknowledgement [21] Nguyen HT, Wang H, Hu SJ. Modeling cutter tilt and cutter-spindle stiffness for
machine condition monitoring in face milling using high-definition surface me-
trology. Int J Adv Manuf Technol 2014;70(5–8):1323–35.
The authors would like to thank Coherix Company for providing the [22] Wang M, Ken T, Du SC, Xi LF. Tool wear monitoring of wiper inserts in multi-insert
HDM equipment. This work was supported by the National Natural face milling using three-dimensional surface form indicators. ASME Trans Manuf
Sci Eng 2015;137(3):031006.
Science Foundation of China (Grant No. 51775343 and No. 51535007). [23] ISO. Geometrical product specifications(GPS)-filtration Part 85: morphological
areal filter: segmentation. Geneva,Switzerland: International Organization for
References Standardization; 2013. Standard No. ISO 16610-85:2013.
[24] Bleau A, Leon LJ. Watershed-based segmentation and region merging. Comput Vis
Image Underst 2000;77(3):317–70.
[1] Marie C, Lasseux D. Experimental leak-rate measurement through a static metal [25] Barré F, Lopez J. Watershed lines and catchment basins: a new 3D-motif method. Int
seal. J Fluids Eng 2006;129(6):799–805. J Mach Tools Manuf 2000;40(8):1171–84.
[2] Malburg MC. Surface profile analysis for conformable interfaces. ASME Trans Manuf [26] ISO. Geometrical product specifications (GPS)-Surface texture: Areal-Part2: terms,
Sci Eng 2003;125(3):624–7. definitions and texture parameters. Geneva,Switzerland: International Organization
[3] Shao YP, Yin YX, Du SC, Xia TB, Xi LF. Leakage monitoring in static sealing interface for Standardization; 2012. Standard No. ISO 25178-2:2012.
based on three dimensional surface topography Indicator. ASME Trans Manuf Sci [27] Otsu N. A threshold selection method from gray-level histograms. IEEE Trans Syst
Eng 2018;140:101003. Man Cybern 1979;9(1):62–6.
[4] Liao Y, Stephenson DA, Ni J. Multiple-scale wavelet decomposition, 3D surface [28] Barber CB, Dobkin DP, Huhdanpaa H. The quickhull algorithm for convex hulls.
feature exaction and applications. ASME Trans Manuf Sci Eng 2012;134(1):011005. ACM Trans Math Softw 1996;22(4):469–83.
[5] Liu S, Jin S, Zhang XP, Chen K, Tian A, Xi LF. A coupled model for the prediction of

287