Functional Data Analysis With PACE: Kehui Chen

Functional Data Analysis with PACE
Kehui Chen
Department of Statistics,
University of California, Davis
JSM, 2012
Outline
General introduction of PACE
Illustrative examples for various functional regression programs
Overview of PACE
Implements various methods of Functional Data Analysis (FDA).

Provides analysis for sparsely or densely sampled random
trajectories and time courses.

The core program is based on the Principal Analysis by
Conditional Expectation (PACE) algorithm.

The most updated version is PACE 2.15, written in Matlab, along
with an R version in development.
Development of PACE
Supported by various NSF grants.
Coordinated by Hans-Georg Muller and Jane-Ling Wang.
PACE 1.0 was written by Fang Yao in 2005, and subsequent
major improvements were made by Bitao Liu.

Contributors and developers include (alphabetical order):
Dong Chen, Kehui Chen, Jeng-Min Chiou, Joel Dubin,

Andrew Farris, Andrea Gottlieb, Jinjiang He, Ci-Ren Jiang,
Yu-Ru Su, Rona Tang, Wenwen Tao, Shuang Wu,
Cong Xu, Matt Yang, Wenjing Yang, Xiaoke Zhang.
Functional Principal Component Analysis

X(t) is a second order random process,
mean function (t) L2 (T ),

continuous covariance function G(s, t) = cov(X(s), X(t)).
G(s, t) =
k=1 k k (s)k (t), eigenvalues 1 2 , , k , 0,
eigenfunctions k (t) form an orthogonal basis.

Karhunen-Lo`eve expansion
X(t) = (t) + k k (t)

k=1
Best linear expansion with p components:

p
X(t) (t) + k k (t).

k=1
Dense and Sparse Designs
Very densely and regularly observed data: empirical mean and

R
covariance, and k =
(X(t) (t))k (t)dt.
Densely recorded but irregular design, or contaminated with
error: pre-smoothing for individual curves.

Sparse random design (longitudinal data): pre-smoothing is
problematic.
PACE works for both dense and sparse data.
The Core Program FPCA

Pool all the sample Yij = Xi (tij ) + ij , 1 i n, 1 j mi , and
estimate mean and covariance by local linear smoothing. One
(two) dimensional nonparametric rate for sparse data, and n

rate for dense data.
Conditional expectation method to estimate the components ik .
For sparse case, best linear unbiased prediction; for dense data, it
is asymptotically
equivalent to the numerical approximation of
R
ik = T (Xi (t) (t))k (t)dt.
Yao et al. (2005), Hall et al. (2006), Li and Hsing (2010), Cai
and Yuan (2010).
Local Linear Smoothing Estimators

Mean function is given by (t)
= a 0 , where
n mi
(a0 , a 1 ) = arg min {[Yij a0 a1 (tij t)]2 Kh (tij t)}.

i=1 j=1
1 , t2 ) = a 0 , where
Covariance function is given by G(t
n
(a0 , a 1 , a 2 ) = arg min {[Yijc Yilc a0 a1 (tij t1 )

i=1 j6=l
a2 (til t2 )]2 Kb (tij t1 )Kb (til t2 )}.
Covariance Estimation
G(t,t)+2
G(s,t)
Principal Analysis by Conditional Expectation
Xi = (Xi (ti1 ), . . . , Xi (timi ))T , Yi = (Yi1 , . . . , Yimi )T ,
i = ((ti1 ), . . . , (timi ))T , ik = (k (ti1 ), . . . , k (timi ))T , by

Gaussianity
E[ik |Yi ] = k ikT 1
Yi (Yi i ),
where Yi = cov(Yi , Yi ) = cov(Xi , Xi ) + 2 Imi .
The method is robust and works well for non-Gaussian data.
Functional Regression in PACE
Linear regression and diagnostics

Quadratic (Polynomial) regression
Additive modeling
Generalized responses
Quantile and conditional distribution modeling
Function to scalar; function to function
Illustrative Example: Meat Spectral Data

FPCreg, FPCdiag: Let X c (t) = X c (t) (t)
Z
E(Y|X) = +
X c (t) (t)dt

Z
E(Y|X) = +
X c (t) (t)dt
FPCQuadReg: (Yao and Muller 2010, Horvath and Reeder, 2012)

Z
ZZ
E(Y|X) = +
X c (t) (t)dt +
(s, t)X c (s)X c (t)dsdt

Z
E(Y|X) = +
X c (t) (t)dt
FPCQuadReg: (Yao and Muller 2010, Horvath and Reeder, 2012)

Z
ZZ
E(Y|X) = +
X c (t) (t)dt +
FPCquantile (Chen and Muller 2012. JRSSB.)

1
P(Y y|X) = E(I(Y y)|X) = g ((t) +
X c (t) (y, t)dt)
Predictor Functions: Spectral Data

5.5
Absorbance
4.5
3.5
2.5
2
850
900
950
Spectrum Channel
1000
1050
Coefficient of Linear Regression

Confidence bands for Beta
1200
1000
800
600
400
200
200
400
600
800
850
900
950
x
1000
E(Y|X) = + X c (t) (t)dt

R
1050
Residual Plot for Linear Regression
10
Residual
10
10
20
30
Fitted
40
50
60
Coefficients of Quadratic Regression
10
5
3
2
0
1
0
1
2
1050
1000
1050
10
1000
950
950
900
900
15
850
900
950
1000
1050
E(Y|X) = + X c (t) (t)dt +

R
850
RR
850
Residual Plot for Quadratic Regression

5
4
3
2
Residual
1
0
1
2
3
4
5
10
15
20
25
30
Fitted
35
40
45
50
55
Quantiles
50
45
Predicted Quantiles
40
35
30
25
20
15
true
median
0.1 th
0.9 th
10
10
15
20
25
30
Fat Content
35
40
45
50
Illustrative Example: Traffic Data
Velocity on I-880
Velocity (mph)
10:25:26
14:15:41
70
70
60
60
50
50
40
40
30
30
20
20
10
10
21
22
23
24
25
26
27
21
22
23
Velocity (mph)
16:33:50
25
26
27
26
27
12:29:56
70
70
60
60
50
50
40
40
30
30
20
20
10
21
24
10
22
23
24
25
Postmile
26
27
21
22
23
24
25
Postmile
Prediction for Response Functions

Y and X are both functions

FPCfam: E(Y(t)|X) = Y (t) +

k=1 j=1 fjk (k )j (t)


k=1 j=1 fjk (k )j (t)
FPCpredBands (Chen and Muller 2012): Global prediction bands
for Y conditional on X


k=1 j=1 fjk (k )j (t)
For Gaussian process: E(Y|X) and cov(Y|X)


k=1 j=1 fjk (k )j (t)
For Gaussian process: E(Y|X) and cov(Y|X)
Common principal component assumption
Additive assumption
cov(Y(t1 ), Y(t2 ) | X)
2
= GYY (t1 , t2 ) +
j=1 {k=1 gjk (k ) k=1 fjk (k ) }j (t1 )j (t2 )
Modeling the Prediction Bands
Global prediction bands for Gaussian case:
P((t) DX (t) YX (t) (t) + DX (t) | X) 1

where DX (t) = C {var(Y(t)|X)}1/2
For more general random processes:
E {P(LX (t) YX (t) UX (t) | X)} 1

Find C by the empirical coverage
Mobile Century Data
Joint UC Berkeley - Nokia project (Herrera et al., 2010)

Students were hired to drive on a segment of highway I-880 and
send data (time, location, and speed) back through GPS enabled
mobile phones.
The follow-up project Mobile Millennium is generating more
data.
Estimated 90% Prediction Regions

20
0
20
40
60
80
20
0
20
40
60
80
Relative Speed (mph)
50
100
150
200
250
300
20
0
20
40
60
80
50
100
150
200
250
300
50
100
150
200
250
300
50
100
150
200
250
300
20
0
20
40
60
80
0
50
100
150
200
250
300
20
0
20
40
60
80
20
0
20
40
60
80
0
50
100
150
200
Time (sec)
250
300
Time (sec)
Other Important Tools in PACE

Modeling of derivatives (linear and nonlinear empirical
dynamics)
Modeling of functional errors (variance processes, volatility
processes)
Time-synchronization based on pairwise warping
Functional manifold analysis
Modeling of functional correlations
Distance based methods (curve clustering)
Stringing method
Get Started with PACE
Get Started with PACE
User Friendly: help files, examples, documentation, references.

p = setOptions()
p2 = setOptions(0 bwmu0 , 3)
Various options for bandwidth selection, number of components,
different designs, errors, pre-binning options.

The code and descriptions can be downloaded from
http://anson.ucdavis.edu/~mueller/data/programs.html.
THANK YOU!
Yao, F., Muller, H.G., Wang, J.L. (2005), Functional data analysis for sparse
longitudinal data. J. American Statistical Association, 100, 577-590.
Yao, F., Muller, H.G., Wang, J.L. (2005), Functional Linear Regression
Analysis for Longitudinal Data. Annals of Statistics, 33, 2873-2903.
Chiou, J., Muller, H.G. (2007), Diagnostics for functional regression via
residual processes. Computational Statistics and Data Analysis, 51,
4849-4863.
Muller, H.G., Yao, F. (2010), Functional quadratic regression. Biometrika 97,

49-64.
Muller, H.-G. and Yao, F. (2008), Functional additive models, J. of the

American Statistical Association, 103, 1534-1544.
Muller, H.-G. and Stadtmuller, U. (2005), Generalized functional linear

models, Annals of Statistics, 33, 774805.
Chen, K. and Muller, H.-G. (2012), Conditional quantile analysis when

covariates are functions, with application to growth data, J. of the Royal
Statistical Society: Series B, 74, 67-89.
Liu, B., Muller, H.G. (2009), Estimating derivatives for samples of sparsely
observed functions, with application to on-line auction dynamics. J. American
Statistical Association, 104, 704-717.
Muller, H.G., Yao, F. (2010), Empirical dynamics for longitudinal data. Annals
of Statistics, 38, 3458C3486.
Muller, H.G., Stadtmuller, U., Yao, F. (2006), Functional variance processes. J.

of the American Statistical Association, 101, 1007-1018.
Muller, H.G., Sen, R., Stadtmuller, U. (2011), Functional Data Analysis for
Volatility. J. Econometrics 165, 233-245.
Tang, R., Muller, H.G. (2008), Pairwise curve synchronization for

high-dimensional data.Biometrika, 95, 875-889
Chen, D., Muller, H.G. (2012), Nonlinear manifold representations for

functional data. Annals of Statistics, 40, 1-29.
Yang, W., Mller, H.G. Muller, H.G., Stadtmller, U. (2011), Functional singular
component analysis. J. Royal Statistical Society B, 73, 303C-324.
Dubin, J., Muller, H.G. (2005), Dynamical correlation for multivariate

longitudinal data. J. American Statistical Association, 100, 872-881.
Peng, J., Muller, H.G. (2008), Distance-based clustering of sparsely observed

stochastic processes, with applications to online auctions. Annals of Applied
Statistics, 2, 1056-1077.
Chen, K., Chen, K., Muller, H.G., Wang, J.L. (2011), Stringing
high-dimensional data for functional analysis. J. American Statistical
Association, 106, 275-284.

Functional Data Analysis With PACE: Kehui Chen

Uploaded by

Copyright:

Available Formats

Functional Data Analysis With PACE: Kehui Chen

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Functional Data Analysis With PACE: Kehui Chen

Uploaded by

Copyright:

Available Formats

Functional Data Analysis with PACE

General introduction of PACE

Illustrative examples for various functional regression programs

Implements various methods of Functional Data Analysis (FDA).

trajectories and time courses.

Conditional Expectation (PACE) algorithm.

with an R version in development.

major improvements were made by Bitao Liu.

Dong Chen, Kehui Chen, Jeng-Min Chiou, Joel Dubin,

Functional Principal Component Analysis

mean function (t) L2 (T ),

eigenfunctions k (t) form an orthogonal basis.

X(t) = (t) + k k (t)

Best linear expansion with p components:

X(t) (t) + k k (t).

Dense and Sparse Designs

Very densely and regularly observed data: empirical mean and

(X(t) (t))k (t)dt.

Densely recorded but irregular design, or contaminated with

error: pre-smoothing for individual curves.

The Core Program FPCA

estimate mean and covariance by local linear smoothing. One

(two) dimensional nonparametric rate for sparse data, and n

and Yuan (2010).

Local Linear Smoothing Estimators

(a0 , a 1 ) = arg min {[Yij a0 a1 (tij t)]2 Kh (tij t)}.

(a0 , a 1 , a 2 ) = arg min {[Yijc Yilc a0 a1 (tij t1 )

a2 (til t2 )]2 Kb (tij t1 )Kb (til t2 )}.

Principal Analysis by Conditional Expectation

Xi = (Xi (ti1 ), . . . , Xi (timi ))T , Yi = (Yi1 , . . . , Yimi )T ,

i = ((ti1 ), . . . , (timi ))T , ik = (k (ti1 ), . . . , k (timi ))T , by

Functional Regression in PACE

Linear regression and diagnostics

Illustrative Example: Meat Spectral Data

Illustrative Example: Meat Spectral Data

FPCQuadReg: (Yao and Muller 2010, Horvath and Reeder, 2012)

(s, t)X c (s)X c (t)dsdt

Illustrative Example: Meat Spectral Data

FPCQuadReg: (Yao and Muller 2010, Horvath and Reeder, 2012)

(s, t)X c (s)X c (t)dsdt

FPCquantile (Chen and Muller 2012. JRSSB.)

P(Y y|X) = E(I(Y y)|X) = g ((t) +

X c (t) (y, t)dt)

Predictor Functions: Spectral Data

Coefficient of Linear Regression

E(Y|X) = + X c (t) (t)dt

Residual Plot for Linear Regression

Coefficients of Quadratic Regression

E(Y|X) = + X c (t) (t)dt +

(s, t)X c (s)X c (t)dsdt

Residual Plot for Quadratic Regression

Illustrative Example: Traffic Data

Prediction for Response Functions

Prediction for Response Functions

FPCfam: E(Y(t)|X) = Y (t) +

Prediction for Response Functions

FPCfam: E(Y(t)|X) = Y (t) +

FPCpredBands (Chen and Muller 2012): Global prediction bands

Prediction for Response Functions