Department of Management

Science and Technology

Seminar: Researchers training

Basics on Regression Analysis

Irini Voudouris

Before starting.
Statistics is a science assisting you to make
decisions under uncertainties
Deficiencies of non-statistician instructors teaching
statistics lead students to develop phobias for the
sweet science of statistics

Since everybody in the world thinks he can teach

statistics even though he does not know any, I shall
put myself in the position of teaching biology even
though I do not know any

Prof. Herman Chernoff

Since I do not think I can teach statistics, I will

limit myself to describing my experiences as a
statistics user
Irini Voudouris
2

Overview

Part 1: Basics of Econometrics

Part 2: Basics of Regression Analysis

Econometrics is
Methodology of Econometrics
Elements of Econometrics

Regression analysis: some basic ideas, terminology

10 assumptions underlying the linear regression model
Goodness of fit
Regression modeling selection process
What if

Part 3: Regression Analysis in praxis

Planning, development and maintenance of the model

An example: Determinants of atypical work
3

Econometrics is

an amalgam of economic theory, mathematical

economics, economic statistics, and
mathematical statistics
Gujarati

the quantitative analysis of actual economic

phenomena based on the concurrent
development of theory and observation, related
by appropriate methods of inference
Samuelson, Koopmans & Stone

using sample data on observable variables to

learn about the functional relationships among
economic variables
Abbott
5

Econometrics consists

mainly of:
estimating relationships from sample data
testing hypotheses about how variables are
related

the existence of relationships between variables

the direction of the relationships between the
dependent variable and its hypothesized observable
determinants
the magnitude of the relationships between a
dependent variable and the independent variables thought
to determine it

Methodology of
Econometrics

Statement of theory or hypothesis

Specification of the mathematical &
econometric model of the theory
Obtaining the data
Estimation of the parameters of the
econometric model
Hypothesis testing
Forecasting or prediction
Using the model for control or policy
purposes

Elements of Econometrics

Specification of the econometric model that

we think (hope) generated the sample data. It
consists of:

An economic model: specifies the dependent

variable to be explained and the independent
variables thought to be related to the dependent

Suggested or derived from theory

Sometimes obtained from informal intuition/ observation

A statistical model: specifies the statistical elements

of the relationship under investigation, in particular the
statistical properties of the random variables in the
relationship
8

Elements of Econometrics

Collecting and coding the sample data

Most economic data is observational, or non-experimental

Sample data consist of observations on randomly selected
members of populations (individual persons, households or
families, firms, industries, provinces or states, countries)

Estimation consists of using the sample data on the

observable variables to compute estimates of the
numerical values of all the unknown parameters in the
model
Inference consists of using the parameter estimates
computed from sample data to test hypotheses about the
numerical values of the unknown population
parameters that describe the behavior of the population
from which the sample was selected

Suggestions for further

reading

Damodar N. Gujarati,
Basic Econometrics, 3d eds, Mc Graw Hill,
1995

Adrian C. Darnell & J. Lynne Evans,

The limits of Econometrics, Edward Elgar
Publishing Ltd., Hants, England, 1990

10

11

Regression analysis

Regression models:

one variable called the dependent variable is

expressed as a linear function of one or more
other variables, called the explanatory
variables
it is assumed implicitly that causal
relationships, if any, between dependent and
explanatory variables flow in one direction
only, namely, from the explanatory variables to
the dependent variables

12

Regression Analysis: Some

basic ideas

Key idea: the statistical dependence of one variable on one or more other
variables

Objective: to estimate and/or predict the mean or average value of the

dependent variable on the basis of the known or fixed values of the explanatory
variables

Yi = f(X2i ,, Xki) + ui = 1 + 2X2i + +kXki + ui

Population Regression Function (PRF)

The PRF is an idealized concept

Generally one uses the stochastic sample regression (SRF) function to estimate
the PRF
Ordinary least squares (OLS) is the most used method of constructing the SFR

Sample Data: A random sample of N members of the population for which

the observed values of Y and Xkis are measured

Sample size is critical because it influences the confidence level of conclusions

The larger the sample, the higher the associated confidence

BUT larger samples require more effort & time

13

Regression Analysis:
Terminology
Where
Yi , Xi , ui variables of the regression model

Yi dependent or explained variable or regressand

Xi independent or explanatory variable or
regressor

Ui random error term

Yi, Xi are observable variables

it is unobservable variable
It is called residual for sample observation

1 and 2 parameters of the regression model

1 intercept coefficient,
2 slope coefficient of X

they are called regression coefficients

the true population values of 1 and 2 are unknown
They are called estimators of the regression coefficients for
14
the sample observation

10 assumptions underlying the

linear regression model

Linear regression model

X values are fixed in repeated sampling

More technically X is assumed to be non-stochastic

Zero mean value of random error term ui

Homoscedasticity or equal variance of ui

It is always linear in the coefficients being estimated, not

necessarily linear in the variables
A scatter-plot is essential to examining the relationship
between the two variables

It means that Y populations corresponding to various X values

have the same variance

No autocorrelation between the random error terms

In other words ui & uj are uncorrelated

15

10 assumptions underlying the

linear regression model

Zero covariance between ui and Xi

The number of observations n must be greater than the

number of parameters to be estimated

The X values in a given sample must not be all the same

The regression model is correctly specified

Alternatively the number of observations must be greater than

the number of explanatory variables

Variability in X values

The error term and explanatory variable are uncorrelated

There is no bias or error in the model used in empirical

analysis

There is no perfect multi-collinearity

There are no perfect linear relationships among the

explanatory variables, the variables should be independent
16

Goodness of fit

The coefficient of determination r2 (2 variable

case) or R2 (multiple variable case) is a
summary measure that tells how well the
regression line fits the data

It measure the proportion or percentage of the total

variation in Y explained by the regression model
It is nonnegative quantity (0 R2 1)

The coefficient of correlation r is a measure of

the degree of association between two
variables

Quantity closely related but conceptually very much

different from r2
17

Regression vs Correlation
Correlation analysis

Primary objective: to measure the degree of linear association

between two variables
Symmetry in the way variables are treated

Both variables are assumed to be random

It does not imply any cause and effect relationship

The correlation between two random variables is often due only to
the fact that both variables are correlated with the same third
variable

Regression analysis

Primary objective: to estimate or predict the average value of one

variable on the basis of fixed values of other variables
Asymmetry in the way dependent & explanatory variables are
treated

Dependent variable is random (normal probability distribution)

Independent variables are assumed to have fixed values

It implies causality
18

Regression modeling selection

process

When you have more than one regression

equation based on data, to select the "best
model", you should compare:

R2 or adjusted R2, i.e., the percentage of

variance in Y accounted for variance in X
captured by the model
Standard deviation of error terms, i.e., observed
y-value/ predicted y-value for each X
The T-statistic of individual parameters
The values of the parameters and its content to
content underpinnings
19

20

What ifwe have qualitative

explanatory variables?

Introduce dummy variables Di (taking values: 1,0)

dummies can be used in regression models just as easily as

quantitative variables
If there is only dummy explanatory variables use ANOVA model
(Analysis of Variance)
Possibly consider Interactions effects

There may be interaction between two dummies so that their effect on

mean Y is not simply additive but also multiplicative

The dummy variable technique must be handled carefully

The number of dummy variables must be less than the number of

classifications of each qualitative variable
The coefficient attached to the dummy variable must be always
interpreted in relation to the base, that is the group that gets the
value of zero
Weight the number of dummies introduced against the number of
observations
21

What if we have dummy

dependent variable?

Use a dummy dependent variable

regression model

Logistic regression model

Probit
Logit
Tobit

22

What if
In regression analysis involving time series

the regression includes not only current

but also lagged values of the explanatory
variables (Xs)?

Distributed lag model

the model includes one or more lagged

values of the dependent variable (Yt-1)
among its explanatory variables?

Auto-regressive (dynamic) model

23

What if
a regression model has been estimated
using the available data sample and an
additional data sample become available?

Use the analysis of covariance (ANCOVA) to

test if previous model is still valid or the two
separate models are equivalent or not

24

25

Planning the model

Define the problem; select response; suggest variables.

Collect data
Check the quality of data

plot; try models

Find the basic statistics, correlation matrix and first regression runs
Establish goal: The final equation should have

Are the proposed variables fundamental to the problem, are they variables?
Are they measurable/countable?
Can one get a complete set of observations at the same time?
Is the problem potentially solvable?

adjusted R2 = 0.8
coefficient of variation of less than 0.10
appropriate number of predictors
estimated coefficients must be significant at m = 0.05
no pattern in the residuals

Are goals and budget acceptable?

26

Development of the model

Check the regression conditions

Remove outliers they may have major impact

Examine all the points in the scatter diagram is Y a linear
function of X? Consider transformation
The distribution of the residual must be normal
The residuals should have mean equal to zero, and
constant standard deviation
The residuals constitute a set of random variables
Check for residuals autocorrelation

Consult experts for criticism

Plot new variable and examine same fitted model

Durbin-Watson (D-W) (values [0,4])

No correlation ~2

Also transformed predictor variable may be used

Are goals met?

Have you found "the best" model?

27

Validation & maintenance of

the model
Are parameters stable over the sample
space?
Is there a lack of fit?

Are the coefficients reasonable?

Are any obvious variables missing?
Is the equation usable for control or for
prediction?

Maintenance of the Model

Need to have control chart to check the model

periodically by statistical techniques
28

An example: Determinants of
atypical work

Hypotheses drawn from three

perspectives:

the integration perspective

the work environment perspective
the capacity planning perspective

Identification of three groups of variables

29

Hypotheses
Factor
category
Integration
costs

Explanatory Factors

Environment

Firm

Atypical
workers

Training costs
Fringe benefits
Technological complexity
Interpersonal complexity
Specific know-how

+
-

Employment variability
Demands predictability
Unionization
Difficulty of finding
workers on the external
labor market
Industry capital intensive
Size
Number of non-frequent
tasks
Difficulty of monitoring
the employees

+
+
-

30

Research Methodology

An empirical evaluation of the factors

affecting:

firms decision to use temporaries, independent

contractors & subcontractors

Logistic Regression

the proportion of the firms labor force

belonging to temporaries, independent
contractors & subcontractors

Regression

31

Data selection

Data from 75 companies belonging to 4

industrial sectors:

construction (29%)
metallurgy (30%)
oil/ chemical (25%)
electrical/ electronical (16%)

Data selection based on:

semi-directed personal interviews

questionnaires
complementary data through internal documents
32

Data selection

Examination of 3 types of atypical workers:

temporaries
independent contractors
subcontractors

for 6 task categories:

engineers
managers, financiers, counsels
technologists
administrative personnel
workers
subsidiary personnel

450 observations

33

Measures
Independent variables

Measures

Benefit costs

Two measures based on a proportional

scale
1 indicator based on an additive 5
points scale of 3 items
4 measures based on a proportional
scale
1 indicator based on an additive 5
point scale of 3 items
2 measures based on a proportional
scale
1 indicator based on a 5 points additive
scale of 4 items
1 indicator based on a 5 points additive
scale of 5 items
1 indicator based on a 5 points additive
scale of 3 items
1 indicator based on a 5 points additive
scale of 2 items
1 measure based on a proportional
scale
1 indicator based on a 5 points additive
scale of 3 items
1 item

Training
Employment
variability
Demands
predictability
Unionization
Monitoring problems
Interpersonal
complexity
Technological
complexity
Firm specific
how
Size
Low frequency
Difficult to find

know-

Cronbachs
Alpha

= 0,8175

= 0,844

= 0,94
= 0,9364
= 0,9048
= 0,8736

= 0,9048

34

Results: Determinants of
Temporaries use
Benefit costs
Training
Employment variability
Demands predictability
Difficult to find
Unionization
Monitoring problems
Size
Low frequency
Interpersonal complexity
Technological complexity
Firm specific know-how
Sample, N=

2
R2

Logistic
regression

Regression

-0,052
-2,260***
0,760*
0,985*
-1,394**
0,037***
-3,450***
0,001
-0,090
-5,007***
-0,142
-1,196**
450
453,032**
*
----

0,078
-3,145**
1,477*
-0,048
-0,021
-0,060**
-16,163***
-0,072
2,141**
0,047
-0,014
-0,052
140
---0,723

*** significant at 0,001, ** significant at 0,01, * significant at 0,05

35

Inference: Temporaries
Mostly low-skilled workers performing well
defined elementary tasks
Training costs & monitoring problems are
the most influential factors
Companies tend to resort to temporaries
when their integration is easy & cheap

This policy can increase the volume

flexibility
36

Results: Determinants of
Independent Contractors use
Benefit costs
Training
Employment variability
Demands predictability
Difficult to find
Unionization
Monitoring problems
Size
Low frequency
Interpersonal complexity
Technological complexity
Firm specific know-how
Sample, N=

2
R2

Logistic
regression

Regression

0,0508
0,513*
1,118***
-0,128
-0,814*
0,009
-0,830**
-0,006*
1,126***
0,064
-0,408
0,196
450
398,407***

0,915**
-0,013
-0,046
-5,897***
-6,076***
-0,083**
-5,010***
-0,064**
7,326***
0,062
-0,014
0,059
160
----

----

0,635

*** significant at 0,001, ** significant at 0,01, * significant at 0,05

37

Inference: Independent
Contractors
Mostly high skilled workers often used for
the realization of fixed term projects
Low frequency, difficulty to find
& monitoring problems are the most
influential factors

This policy can increase firms capacity to

adapt to unanticipated conditions implying
new set of competencies
38

Results: Determinants of
Subcontractors use
Benefit costs
Training
Employment variability
Demands predictability
Difficult to find
Unionization
Monitoring problems
Size
Low frequency
Interpersonal complexity
Technological complexity
Firm specific know-how
Sample, N=

2
R2

Logistic
regression

Regression

-0,0302
0,1004
1,080***
0,252
-0,008
0,034***
0,470
0,0004
0,520**
0,344
-0,206
-0,473*
450
248,149**
*

0,065
-0,039
-0,018
-0,124
-5,062*
0,611
-0,018
-0,109
5,205*
-0,057
0,003
-0,050

0,392

*** significant at 0,001, ** significant at 0,01, * significant at 0,05

39

Inference: Subcontractors
Both low- & medium-skilled workers for
the realization of peripheral as well as of
specialized tasks
Low frequency is the most influential
factor

This policy can increase both the volume

flexibility and the creation of new set of
competencies
40

Conclusion
Multiple types of atypical workers may be found
in the same firm, for the accomplishment of
different tasks and different types of atypical
work appear to be influenced by different factors

temporaries are used mainly as a source of

quantitative flexibility
independent contractors are sources of qualitative
flexibility
subcontractors, though not very developed is however
present and used as a source of both types of
flexibility

41