Hexp C13

Download as pdf or txt
Download as pdf or txt
You are on page 1of 67

國立高雄師範大學物理系

高能實驗物理
GH107
Error estimation (I)
2024/5/24

1
課程進度
週別 內 容 日期 參 考 資 料
1 Introduction: Size, energy, cross section 2月23日
2 History of High-Energy Experiments 3月1日
3 Accelerator and Collisions 3月8日 Class at NKNU
4 Particle Detector I 3月15日
5 Particle Detector II 3月29日
6 Particle Detector III 4月1日(一)下午
︵ 7 Particle Detector IV 4月12日 Class at NKNU
下 8 Front End Electronics 4月19日

9 Trigger & Data acquisition system 4月22日(一)下午

︶ 10 Data Analysis I: Calibration & Data Reduction 5月6日(一)下午
11 Data Analysis II: Monte-Carlo Simulation 5月10日
12 Data Analysis III: ROOT Analysis 5月20日(一)下午
13 Data Analysis IV: Error estimation (I) 5月24日
14 Data Analysis V: Error estimation (II) 5月31日 Class at NKNU
15 Oral Presentation 6月7日 ONLINE
16 Oral Presentation 6月13日

2
INTRODUCTION

3
Making the connection
Reality

The imperfect measurement of a (set of)


Raw Data interactions in the detector

A unique happening:
Reconstructed eg. Run 23458, event 1345
Events which contains a Z → μ+μ- decay
Analysis : We “confront theory with experiment” by comparing
the measured quantity (observable) with the prediction.

cross sections (probabilities for interactions),


Observables branching ratios (BR), ratios of BRs, specific
lifetimes, ...

Theory A small number of general equations, with some


4
parameters (poorly or not known at all)
5
Why do we do experiments?
• Parameter determination: determine the numerical
value of some physical quantity.
• Hypothesis testing: test whether a particular theory is
consistent with our data.

6
7
8
9
10
11
12
Statistics: Probability Distribution

13
14
15
Measurements Compared to Standard Model
https://atlas.web.cern.ch/Atlas/GROUPS/PHYSICS/CombinedSummaryPlots/SM/index.html#ATLAS_b_SMSummary_FiducialXsect

16
Problems to deal with
--- Things to Learn
• How to assign the errors of data points?
– How to identify a signal? How to determine its
significance?
– How to determine the parameters and associated
errors of the theoretical modeling?
– How to reject a theory?
– How to make a claim of discovery?

17
Why estimate errors?
• We are concerned not only with the answer
but also with its accuracy.
• For example, speed of light 2.998x108 m/sec
– (2.990.15) x108:
– (3.090.01) x108:
– (3.092) x108:
Q1: which measurement has the best accuracy?
Q2: which measurement is not consistent with the standard value of speed of light?

18
Source of Errors
• Random (Statistic) error: the inability of any
measuring device to give infinitely accurate
answers.
• Systematic error: uncertainty.

19
Statistic Errors

20
Statistic Errors

21
Statistic Errors

22
Systematic Errors
https://ebookcentral.proquest.com/lib/sinciatw/detail.action?docID=1315866

• badly known detector acceptances or trigger efficiencies;


• incorrect detector calibrations;
• badly known detector resolutions;
• badly known background;
• uncertainties in the simulation or underlying theoretical models;
• uncertainties on input parameters like cross sections, branching
fractions, lifetimes,
• the luminosity, and so on (often called ‘external uncertainties’);
• computational and software errors;
• personal biases towards a specific outcome of an analysis;
• other usually unknown effects on the measurement.

23
PROBABILITY DISTRIBUTIONS

24
Probability Density Functions (PDF)
Probabilistic treatment of possible outcomes

25
PDF vs. Distributions
pdf = histogram with
infinite data sample,
zero bin width,
normalized to unit area.

26
Probability Distribution

27
Expectation values
Consider continuous r.v. x with pdf f (x). r.v. = random variable

Define expectation (mean) value as


Notation (often): ~ “centre of gravity” of pdf.
For a function y(x) with pdf g(y),
(equivalent)

Variance:

Notation:

Standard deviation:
s ~ width of pdf, same units as x.
28
Expectation Value:
E [k ] =  kP(k , p) = 0 + p = p
k =0,1

Variation:
V [k ] =  (k − E[k ])
k =0,1
2
P( k , p ) = (0 − p ) 2 (1 − p ) +(1 − p ) 2 p = p(1 − p ) 29
30
31
Error estimation in each single bin of the counting experiments. 32
33
Probability distribution of measured/constructed physics quantities. 34
35
36
37
38
Some distributions
Distribution/pdf Example use in HEP
Binomial Branching ratio
Multinomial Histogram with fixed N
Poisson Number of events found
Uniform Monte Carlo method
Exponential Decay time
Gaussian Measurement error
Chi-square Goodness-of-fit
Cauchy Mass of resonance
Landau Ionization energy loss
G. Cowan 39
40
41
Error propagation − special cases

That is, if the xi are uncorrelated:


add errors quadratically for the sum (or difference),
add relative errors quadratically for product (or ratio).

But correlations can change this completely...

42
Error propagation − special cases (2)

Consider with

Now suppose r = 1. Then

i.e. for 100% correlation, error in difference → 0.

43
44
PARAMETER DETERMINATION

45
Parameter estimation
The parameters of a pdf are constants that characterize
its shape, e.g.
a pdf example

random variable parameter

Suppose we have a sample of observed values:

We want to find some function of the data to estimate the


parameter(s):
← estimator written with a hat

Sometimes we say ‘estimator’ for the function of x1, ..., xn;


‘estimate’ for the value of the estimator with a particular data set.
G. Cowan 2011 CERN Summer Student Lectures on Statistics / Lecture 4 46
Properties of estimators
If we were to repeat the entire measurement, the estimates
from each would follow a pdf:
best

large biased
variance

We want small (or zero) bias (systematic error):


→ average of repeated measurements should tend to true value.
And we want a small variance (statistical error):
→ small bias & variance are in general conflicting criteria
G. Cowan 2011 CERN Summer Student Lectures on Statistics / Lecture 4 47
An estimator for the mean (expectation value)

Parameter:

Estimator: (‘sample mean’)

We find:

G. Cowan 2011 CERN Summer Student Lectures on Statistics / Lecture 4 48


49
L( S = 0.5; n = 5) = 0.0001579
L( S = 5; n = 5) = 0.175467
L( S = 20; n = 5) = 0.0000549

50
51
The likelihood function
Suppose the outcome of an experiment is: x1, ..., xn, which
is modeled as a sample from a joint pdf with parameter(s) q :

Now evaluate this with the data sample obtained and regard it as
a function of the parameter(s). This is the likelihood function:

(xi constant)

If the xi are independent observations of x ~ f(x;q), then,

G. Cowan 2011 CERN Summer Student Lectures on Statistics / Lecture 4 52


Maximum likelihood estimators
If the hypothesized q is close to the true value, then we expect
a high probability to get data like that which we actually found.

So we define the maximum likelihood (ML) estimator(s) to be


the parameter value(s) for which the likelihood is maximum.
ML estimators not guaranteed to have any ‘optimal’
properties, (but in practice they’re very good).
G. Cowan 2011 CERN Summer Student Lectures on Statistics / Lecture 4 53
ML example: parameter of exponential pdf

Consider exponential pdf,

and suppose we have data,

The likelihood function is

The value of t for which L(t) is maximum also gives the


maximum value of its logarithm (the log-likelihood function):

G. Cowan 2011 CERN Summer Student Lectures on Statistics / Lecture 4 54


ML example: parameter of exponential pdf (2)
Find its maximum by setting

Monte Carlo test:


generate 50 values
using t = 1:

We find the ML estimate:

G. Cowan 2011 CERN Summer Student Lectures on Statistics / Lecture 4 55


Variance of estimators: graphical method
Expand ln L (q) about its maximum:

First term is ln Lmax, second term is zero, for third term use
information inequality (assume equality):

i.e.,

→ to get , change q away from until ln L decreases by 1/2.

G. Cowan 2011 CERN Summer Student Lectures on Statistics / Lecture 4 56


Example of variance by graphical method

ML example with
exponential:
Not quite parabolic ln L
since finite sample size
(n = 50).

G. Cowan 2011 CERN Summer Student Lectures on Statistics / Lecture 4 57


The method of least squares
Suppose we measure N values, y1, ..., yN,
assumed to be independent Gaussian
r.v.s with

Assume known values of the control


variable x1, ..., xN and known variances

We want to estimate q, i.e., fit the curve to the data points.


The likelihood function is

G. Cowan 2011 CERN Summer Student Lectures on Statistics / Lecture 4 58


The method of least squares (2)
The log-likelihood function is therefore

So maximizing the likelihood is equivalent to minimizing

Minimum of this quantity defines the least squares estimator


Often minimize c2 numerically (e.g. program MINUIT).

G. Cowan 2011 CERN Summer Student Lectures on Statistics / Lecture 4 59


60
Goodness-of-fit with least squares
The value of the c2 at its minimum is a measure of the level
of agreement between the data and fitted curve:

An estimator example

It can therefore be employed as a goodness-of-fit statistic to


test the hypothesized functional form l(x; q).

We can show that if the hypothesis is correct, then the statistic


t = c2min follows the chi-square pdf,

where the number of degrees of freedom is


nd = number of data points − number of fitted parameters
G. Cowan 2011 CERN Summer Student Lectures on Statistics / Lecture 4 61
62
63
64
65
References
• Lecture notes of Roger Barlow:
http://www.hep.man.ac.uk/u/roger/Lecture1.ppt
http://www.hep.man.ac.uk/u/roger/Lecture2.ppt
http://www.hep.man.ac.uk/u/roger/Lecture3.ppt
http://www.hep.man.ac.uk/u/roger/Lecture4.ppt
http://www.hep.man.ac.uk/u/roger/Lecture5.ppt
• 2012 CERN Summer Student Lectures :
https://indico.cern.ch/conferenceDisplay.py?confId=190030
• 2011 CERN Summer Student Lectures : Glen Cowan, Physics Department,
Royal Holloway, http://www.pp.rhul.ac.uk/~cowan/stat_cern.html
• Louis Lyons, “Statistics for Nuclear and Particle Physicists”, Cambridge
1986; http://arxiv.org/abs/1301.1273 .
• 2016 CERN Summer Student Lectures :
https://indico.cern.ch/event/521953/
https://indico.cern.ch/event/521954/
https://indico.cern.ch/event/521955/
https://indico.cern.ch/event/521956/ .

66
References
• Foundation of Statistics - 1/3:
https://indico.cern.ch/event/817550/
• Foundation of Statistics - 2/3:
https://indico.cern.ch/event/817553/
• Foundation of Statistics - 3/3:
https://indico.cern.ch/event/817623/

67

You might also like