Uncertainty Estimation (II): Combined & Expanded Uncertainties. Cover Factor kp

5. Estimation of Combined & Expanded Uncertainty. Cover factor kp

A physical measurement, however simple, has a model associated with the actual process. The physical model is represented by a mathematical model that in many cases implies approximations.

5.1. Uncertainties Propagation Law (UPL), or "Ley de Propagación de Incertidumbres (LPI)”. Combined Uncertainty

In most cases, the measurand Y is not measured directly, but is determined from other N quantities X1, X2, ..., XN , by a functional relationship f:.

The function f does not express both a physical law and the measurement process and must contain all the magnitudes that contribute to the final result, including corrections, even if these are of null value, to be able to consider the uncertainties of such corrections..
In principle, by means of type A evaluation or type B evaluation, we would be able to know the distribution functions of each of the input magnitudes, and we could derive from this the distribution function of the indirect magnitude.
Taking Taylor's first-order series around the expected value into account, we can obtain the uncertainty propagation law, which facilitates the estimation of variance.

The terms ci , cj are the coefficients of sensitivity and indicate the weight of each of the different input quantities in the output variable, represented by the measurement function. The second term is the covariance term in which the influence of input quantities on others appears, in case they are correlated. If the input magnitudes are independent, the equation can be simplified, the second term disappearing, leaving only the first.

Example 1:
UPL or “LPI” for models with shape:

In this case we obtain:

A tipical example for this shape model can be the Young’s modulus for metals elasticity.
E = σ/ε
When it is not possible to write the measurement model in an explicit way, the sensitivity coefficients can not be calculated analytically, but numerically, by introducing small changes xi + Δxi into the input quantities and observing the changes that occur in the output quantities.

5.2. Uncertainty Propagation Law “LPI” limitations.

Uncertainty Propagation Law can be applied when:

1. Only an output magnitude appears in the mathematical model.
2. The mathematical model is an explicit model, ie, Y = f (Xi).
3. The mathematical expectation, the typical uncertainties and the mutual uncertainties of the input magnitudes could be calculated.
4. The model is a good approximation to a linear development around the best estimator of the input magnitudes.

When it comes to nonlinear models, we can perform the second order approximation of the Taylor series, or even obtain the values of mathematical expectation and variance without approximations, directly, much more complex solutions mathematically than the law of propagation of uncertainties.

After the elaboration of the GUM guide, we have worked on additional guides to this one, for the evaluation of uncertainties by other methods. One of them has influenced uncertainty calculation using the Montecarlo method.

The basic idea of this method, useful for both linear and nonlinear models, is that assuming a model Y = f (Xi), where all input quantities can be described by their distribution functions, a mathematical algorithm programmed to generate a sequence of values τi = (x1, .......xN) where each xi is randomly generated from its distribution function (extractions from its distribution function). The value of y1 obtained from each sequence τi is calculated using the measurement model, the process being repeated a large number of times, on the order of 10E5 or 10E6. This high number of repetitions allows to obtain a distribution function for the magnitude y, and he calculation of their mathematical expectation and their standard deviation in mathematical form, which will lead us to the results of the best combined estimator and associated uncertainty.

In this process the results of the measurement of the input parameters are not really used to give a measurement result, but to establish with them the distribution function of the input magnitudes and to be able to randomly generate values of the function magnitude of those input quantities. As it is an automated generation, many more values can be generated than if the measurement were actually performed and with all of them finding the distribution function of the magnitude that depends on those input magnitudes. Mathematically known, the results and associated uncertainties are then obtained.

5.3. Expanded Uncertainty

The purpose of the combined standard uncertainty is to characterize the quality of the measurements. In practice what is needed is to know the interval within which it is reasonable to suppose, with a high probability of not being wrong, that the infinite values that can be "reasonably" attributed to the measurand are found. We might ask if we could use the combined standard uncertainty to define that interval (y-u , y+u). In this case, the probability that the true value of the measurand is inside the range (y-u, y+u) is low since, assuming that the distribution function of the measurand “y” is a normal function, we are talking about 68.3% of probability.

To increase this probability to more useful values for later decision making, we can multiply the combined uncertainty by a number called the “cover factor” kp and to use the interval (y-uc(y) kp , y+ uc(y) kp).

The product kp uc(y) = Up is called expanded uncertainty, where kp is the coverage factor for a confidence level p.

Mathematically this means that:

 Hence the area of the density function associated with Y within this interval is:

The interval we want to know is ( y - Up , y + Up ).
The relation between p y kp depends, of course, on the density function f(Y) that is obtained from the information accumulated during the measurement process.

 

5.4. Statistical Distributions

5.4.1. Rectangular distribution

For a confidence level p we calculate the integral of the distribution function:

Standard deviation is then:

In order to calculate the coverage factor:

Operating shows:

Example:
Given a certain magnitude t, it is known to be described by a symmetrical rectangular distribution whose limits are 96 and 104. Determine the coverage factor and the expanded uncertainty for a 99% confidence level.

In this case we have a = 4, μ = 100, y σ = 2,31, and that for a confidence level of 99% the coverage factor is 1.71. Then, the expanded uncertainty is U99 = kp u = 1,71×2,31=3,949

5.4.2.- Triangular distribution

For a confidence level p, the integral of the density function is calculated:

As the standard deviation is σ= a/√6 Then kp=√6(1-√(1-p) :
Operating gives:

5.4.3. Normal distribution
In most measurement processes the distribution that best describes what is observed is the normal distribution.

Normal distribution with μ = 0 y σ = 1 is termed standard or standard distribution.
The integration required for the determination of confidence intervals is more complicated. However, in the case of the standard normal distribution there are tables that allow the calculation to be carried out in a simple way.

The way to obtain the confidence intervals is by typing. If we have a variable or magnitude Y that is distributed according to an N(µ,σ), we define a normal variable typified as Z=(Y-µ)/σ.
The integration of the standard distribution N (0,1) is in the tables. From these the confidence intervals are determined.

Example:
If, for a confidence level p the confidence interval defined for a distribution N (0,1) is (-kp, kp); As Z is a distribution N (0,1):

In a normal distribution the cover factor kp, for a confidence level p is:

The expanded uncertainty is calculated by Up (y) = kp · uc (y).

5.4.4.- Student's T-distribution

Student's T-distribution is used to make hypothesis tests when the sample size is small.
Let Z be a random variable of expectation µz and standard deviation σz of which n observations are made, estimating a mean z value and an experimental standard deviation s(z).

It is possible to define the following variable whose distribution is the T-Student with ν freedom degrees

The number of degrees of freedom is ν = n -1 for an amount estimated by the arithmetic mean of n independent observations.

If n independent observations are used to make a least square fit of m parameters, the number of degrees of freedom is ν = n -m

For a T-Student variable with ν freedom degrees, el the interval for the confidence level p is (-tp (ν) , tp (v)).

Factor tp (ν) is found inside T-Student tables.

Example:
Suppose that the measurand Y is simply a magnitude that is estimated by the arithmetic mean X of n independent observations, where s(X) is the experimental standard deviation:
The best estimate of Y is y = X with associated uncertainty uc (y) = s(X)
The variable is distributed according to the t-Student. Thus:

Cover Factor is tp(ν) and Expanded Uncertainty is Up (ν) = tp (ν) uc (y).

 5.5. Expanded Uncertainty determination after measurement.

The problem that arises is to determine the expanded uncertainty after a measurement has been made.
If it is a direct measure, we can act in different ways:
- Uncertainty determined according to type A assessment of uncertainty
The cover factor for a given confidence level is obtained from the Student's T-distribution with n-1 degrees of freedom, where n is the number of measurements, or the normal distribution, if the number of repetitions is sufficiently large.
- Uncertainty determined if the distribution Y is rectangular, centered on the estimator y

The cover factor is obtained from the rectangular distribution.

If it is a matter of calculating the expanded uncertainty for an indirect measure, it is somewhat complicated to calculate the distribution function, which must be performed by special analytical or numerical methods. But we can think of simplifying:

A first approximation would be to calculate the probability distribution function by convolution of the probability distributions of the input quantities.

A second approximation would be to assume the indirect measure as a linear function of the input quantities

 5.5.1. Central limit theorem

The central limit theorem in its different versions ensures that the sum of independent and equidistributed random variables converges to a normal one. On paper convergence is commonly very fast, but actual experiments make one despair before seeing the Gaussian bell. There is no contradiction in this, for example we can understand the probability 1/2 to obtain face as a limit when we throw infinite times a coin and we can not demand that after 20 or 30 runs we have a precise approximation counting the percentage of hits.
The following graphs show the histograms of the sum of the 10 dice scores compared to the corresponding normal when the experiment is repeated one hundred, one thousand and ten thousand times. Naturally they come from a computer simulation.

Sum of scores of ten dice thrown a hundred times.

Sum of scores of ten dice thrown a thousand times.

Sum of scores of ten dice thrown ten thousand times.
Suppose an indirect measure is a linear function of the input quantities according to equation 33:

• The central limit theorem says:
“The distribution associated with Y will approximate a normal distribution of expectation

and variance

with E(Xi) the expectetation of Xi and σ2(Xi) the variance of Xi. This happens if Xi are independents and σ2(Y) is much larger than any other individual component ci σ2(Y)”
Conditions of the theorem are fulfilled when the combined uncertainty uc(y) is not dominated by typical uncertainty components obtained by type A evaluations based on few observations or type B evaluations based on rectangular distributions.

Convergence towards the normal distribution will be the faster the greater the number N of variables involved, the more normal these are, and when there is no dominant.
Consequently, a first approximation to determine the expanded uncertainty defining a confidence level interval p will be to use the cover factor proper of a normal distribution, kp.

If the number of random readings is small, then the value of uA derived may be inaccurate, and the distribution of the random component is best represented by the t-Student distribution. We would overestimate the uncertainty, especially if q was small and uA y uB were comparable in size.

5.5.2. Effective degrees of freedom. Welch-Satterthwaite approach

The problem can be solved by using the Welch-Satterthwaite approximation formula, which calculates the number of effective degrees of freedom of the combination of the t-Student distribution with a Gaussian distribution. The resulting distribution will be treated as a t-Student distribution with a calculated number of freedom degrees.

or:

If relative uncertainties are used, the number of effective degrees of freedom is:

In evaluations type B, to calculate the effective degrees of freedom, we will use the approximation given by the following formula:

This formula is obtained by considering the "uncertainty" of uncertainty. The greater the number of degrees of freedom, the greater the confidence in uncertainty. In practice, in type B evaluations vi→∞

In Type A evaluations the calculation of the number of degrees of freedom will depend on the statistic used to evaluate the most probable value.

References:

• [1]Vocabulario Internacional de Metrología VIM, 3ª edición 2008 (español).
• [2]Metrología Abreviada, traducción al español de 3ª edición. Edición digital.
• [3]Evaluación de datos de medición. Guía para la expresión de la incertidumbre de medida. Edición digital.
• [4]Evaluación de datos de medición. Suplemento 1 de la Guía para la expresión de la incertidumbre de medida. Propagación de incertidumbres utilizando el método de Monte Carlo.
• [5]UNE-EN ISO 14253-1:1999: “Especificación geométrica de productos (GPS). Inspección mediante medición de piezas y equipos de medida. Parte 1: Reglas de decisión para probar la conformidad o no conformidad con las especificaciones”.
• [6]LIRA Ignacio. “Evaluating the Measurement Uncertainty”, Bristol, IoP Publishing Ltd., 2002
• [7]JGCM 102:2011. Evaluation of measurement data – Supplement 2 to the “Guide to the expression of uncertainty in measurement” – Extension to any number of output quantities.
• [8] JCGM 106:2012. Evaluation of measurement data - The role of measurement uncertainty in conformity assessment.
• [9] Estimación de incertidumbres. Mª Mar Pérez Hernández: Head of the Length Primary Laboratory at Spanish Center of Metrology

Next: Uncertainty Estimation (III): Examples of uncertainty calculation: resolution, drift, influence of temperature on measurements, steps to follow to estimate the result of a measurement & Conclusions

Uncertainty Estimation (I): GUM Guidelines

Uncertainty of the result of a measurement reflects the lack of exact knowledge of the value of the measurand. The result of a measurement after correction for recognized systematic effects is still only an estimation of the value of the measurand because of the uncertainty arising from random effects and/from imperfect correction of the result for systematic effects.

This article aims to give a brief description with examples of the steps in determining uncertainties.

“La incertidumbre del resultado de una medida refleja la falta de conocimiento sobre el verdadero valor del mensurando. El resultado de medir tal mensurando tras aplicar las correcciones debidas a efectos sistemáticos es todavía una estimación debido a la incertidumbre proveniente de los efectos aleatorios y a la falta de conocimiento completo de las correcciones aplicadas por los efectos sistemáticos.”

“En este artículo se pretende dar una breve descripción, acompañada de ejemplos, de los pasos a seguir en la determinación de incertidumbres.”

1.- Introduction

One measurement without a quantitative indication of the quality of the result is useless, this indication is what we will call uncertainty. The word "uncertainty" means doubt, doubt about the validity of the result of a measure and reflects the impossibility of knowing exactly the measurand value.

Uncertainty estimation is not a simple work in which there is consensus. Work is continuing and guidelines are being developed.

Thanks to a working group, in 1993 the ISO presented the first edition of the Guide to Expressing Uncertainty of Measurement (GUM). This Guide sets out general rules for assessing and expressing uncertainty in measurement, not how this estimation can be used to make decisions. For this last one is the norm UNE-EN ISO 14253-1: 1999 Geometric specification of products (GPS). Inspection by measuring parts and measuring equipment. Part 1: Decision rules for testing compliance or non-compliance with specifications. Although the existence of this guide has made us much easier, we must not lose sight of the fact that the evaluation of uncertainty is not a purely mathematical process that must be performed after each measurement, but rather more complex, such as the guide itself clarifies.

"Although this Guide provides an action framework for the assessment of uncertainty, it can never substitute for critical reflection, intellectual honesty and professional competence. The assessment of uncertainty is neither a routine task nor something purely mathematical; Depends on the detailed knowledge of the nature of the measurand and measurement. The quality and usefulness of the uncertainty associated with the outcome of a measurement ultimately depends on the knowledge, critical analysis and integrity of those who contribute to its assessment." (Guide for Expression of Measurement Uncertainty, Section 3.4.8)

“Aunque la presente Guía proporciona un marco de actuación para la evaluación de la incertidumbre, este no puede nunca sustituir a la reflexión crítica, la honradez intelectual y la competencia profesional. La evaluación de la incertidumbre no es ni una tarea rutinaria ni algo puramente matemático; depende del conocimiento detallado de la naturaleza del mensurando y de la medición. La calidad y utilidad de la incertidumbre asociada al resultado de una medición dependen en último término del conocimiento, análisis crítico e integridad de aquellos que contribuyen a su evaluación”. (Guía para la expresión de la incertidumbre de medida, Sección 3.4.8)

According to the GUM definition, uncertainty is the "parameter associated with the outcome of a measure, which characterizes the dispersion of values that can reasonably be attributed to the measurand."
The base documents for the estimation of uncertainties are as follows, the first being the fundamental reference document:
⎯ Evaluation of measurement data – Guide to the expression of uncertainty in measurement
⎯ Evaluation of measurement data – Supplement 1 to the “Guide to the expression of uncertainty in measurement” – Propagation of distributions using a Monte Carlo method
⎯ Evaluation of measurement data – Supplement 2 to the “Guide to the expression of uncertainty in measurement” – Extension to any number of output quantities

2.- Uncertainty sources.

Inside measurement practices there are many potential sources of uncertainty, including:
a) incomplete measurand definition;
b) imperfect realization of the measurand definition;
c) non-representative measurand sample;
d) inadequate knowledge of the effects of environmental conditions over the measurement, or imperfect measurement of such environmental conditions;
e) biased reading of analogue instruments by the operator;
f) finite instrument resolution or discrimination threshold;
g) inaccurate values of measurement standards or reference materials;
h) inaccurate values of constants and other parameters obtained from external sources, used inside data processing algorithm;
i) approximations and assumptions established in the measurement procedure method;
j) variations in the repetition of measurand observations under apparently identical conditions.

These sources are not necessarily independent, and some of them, a) to i), can contribute to source j).

Of course, an unrecognized systematic effect cannot be taken into account in the evaluation of the uncertainty of the result of a measurement but contributes to its error.

Instrumental uncertainty is the measure uncertainty component that comes from the instrument or measurement system used and is obtained by its calibration. In the case of a primary pattern it is usually obtained from key inter-laboratory comparisons participations.

Uncertainty assessment associated with a measurement is fundamental to be able to subsequently check the product conformity or accordingly to the specs applicable to it, whether of design, legal or othersourcetype (UNE-EN ISO 14253 -1).

3.- Uncertainty & Probability.

In order to begin to evaluate the uncertainty of a measurement in an operational way, the values obtained inside the measurement from the point of view of the probability theory must be seen.
The set of all possible outcomes of a random experiment is called the sample space. The function that assigns a real number to each element of the sample space is the random variable. The range of this function is the set of all possible values of this variable.

In our case we are interested in knowing a magnitude X after the realization of an experiment.

Denoting x to any of the elements of the sample space, we will define as function probability density pdf the function f(x) in the range of (0, ∞) such that the infinitesimal probability dp that the value of the variable is in between the Values x and x + dx is f (x) dx. Hence the probability that a random variable takes values i between the xa and xb limits is:

The distribution function of a measured quantity describes our knowledge of the reality of that magnitude.
For our purpose, the most interesting parameters of the distribution function are: mathematical expectation or expected value, variance and covariance in the case of two related magnitudes.
Given a magnitude Xi whose probability density function is f(xi), the expected value E(Xi) is defined as:

The variance of a random variable or a probability distribution is defined as:

As properties we can emphasize that the mathematical expected value is a linear operator and the variance is not.The best estimator of Xi will be the one that minimizes the expression:

with respect to xi'. The minimum appears when xi´= E(Xi)
On the other hand the variance of a density function gives us idea of the dispersion of the values. Uncertainty is defined as the positive square root of the variance.

4.- Evaluating standard uncertainty

Following the GUM we can group the components of uncertainty into two categories according to the evaluation method, "type A" and "type B". Classification in type A and type B does not imply any difference in nature between the components of these types, it consists only of two different ways of evaluating the components of uncertainty, and both are based on probability distributions.

4.1.- Type A evaluation of standard uncertainty

The uncertainty type A evaluation is used when n independent observations are made from one of the input quantities Xi under the same measurement conditions.
In most cases, the best available estimate of the mathematical expectation q of a random variable q, from which n independent observations qk have been obtained under the same measurement conditions, is the arithmetic mean of the n observations:

The individual observations qk differ in value because of random variations in the influence quantities, or random effects. The experimental variance of the observations, which estimates the variance σ2 of the probability distribution of q, is given by

This estimate of variance and its positive square root s(qk), termed the experimental standard deviation characterize the variability of the observed values qk , or more specifically, their dispersion about their mean q
The best estímate of σ σ²(q)=σ²/n , the variance of the mean, is given by:

that, together with the experimental standard deviation of the mean s(q), quantify how well q estimates the mathematical expectation of q

Thus, for an input quantity Xi determined from n independent repeated observations Xi,k, the standard uncertainty u(xi) of its estimate xi = Xi is u(xi)=s(Xi)=s(xi/sqrt(n)) called a Type A variance and a Type A standard uncertainty, respectively.

In some situations other statistical methods may be used, such as the method of least squares or analysis of variance. For example, the use of calibration models based on the least squares method is useful for evaluating the uncertainties arising from random variations in the short and long term of the results of comparisons of materialized reference measurement standard of unknown value such as blocks or masses reference measurement standards, with reference standards of known value. The components of uncertainty can be evaluated in these cases by statistical analysis of the data obtained using experimental designs consisting of sequences of measurements of the measurand for a certain number of values different from the quantities on which it depends.

4.2.- Type B evaluation of standard uncertainty

For an estimate xi of an input quantity Xi that has not been obtained from repeated observations, the associated estimated variance u²(xi) or the standard uncertainty u(xi) is evaluated by scientific judgement based on all of the available information on the possible variability of Xi . The pool of information may include:

⎯ previous measurement data;
⎯ experience with or general knowledge of the behaviour and properties of relevant materials and instruments;
⎯ manufacturer's specifications;
⎯ data provided in calibration and other certificates;
⎯ uncertainties assigned to reference data taken from handbooks;

For convenience, u²(xi) and u(xi) evaluated in this way are sometimes called a Type B variance and a Type B standard uncertainty, respectively.

According to the source from which this type B uncertainty is obtained, it will be estimated differently. Some examples of type B evaluation are:

4.2.1.- Uncertainty due to calibrated standard or instrument
The typical uncertainty is obtained by dividing the expanded uncertainty given in the calibration certificate of the standard by the indicated coverage factor:

4.2.2.- Uncertainty due to resolution
One of the sources of uncertainty of an instrument is the resolution of its indicating device, if it is a digital instrument, or the uncertainty due to the read resolution, if it is an analog instrument. In the case of the analogue instrument, the resolution depends on the operator or the media used in the reading (optical amplification, for instance)
If the resolution of the indicating device is δx, the input signal value producing a given indication X can be placed with equal probability at any point within the range from (X-δx/2) to (X+δx/2). The input signal can then be described by means of a rectangular distribution of range δx and variance u2=(δx)²/12, which represents a typical uncertainty for any indication of:

4.2.3.- Uncertainty due to working measurement standard drift
The drift of a measurement standard is not easy to determine in many cases, and is an independent and characteristic parameter of each measurement standard. Its value depends, among other factors, on the conditions of use and maintenance, the frequency of use, the accuracy of the measuring instrument, the period between calibrations, etc.
For its calculation, it is possible to start from the history of successive calibrations of the working standard and to estimate a variation of the certified value δp. For the evaluation of the uncertainty we will be able to apply a type of rectangular or triangular distribution according to the knowledge that we have of the history of the working measurement standard.

Bibliography:

• Vocabulario Internacional de Metrología VIM, 3ª edición 2008 (español).
• Metrología Abreviada, traducción al español de 3ª edición. Edición digital.
• Evaluación de datos de medición. Guía para la expresión de la incertidumbre de medida. Edición digital.
• Evaluación de datos de medición. Suplemento 1 de la Guía para la expresión de la incertidumbre de medida. Propagación de incertidumbres utilizando el método de Monte Carlo.
• UNE-EN ISO 14253-1:1999: “Especificación geométrica de productos (GPS). Inspección mediante medición de piezas y equipos de medida. Parte 1: Reglas de decisión para probar la conformidad o no conformidad con las especificaciones”.
• LIRA Ignacio. “Evaluating the Measurement Uncertainty”, Bristol, IoP Publishing Ltd., 2002
• JGCM 102:2011. Evaluation of measurement data – Supplement 2 to the “Guide to the expression of uncertainty in measurement” – Extension to any number of output quantities.
• JCGM 106:2012. Evaluation of measurement data - The role of measurement uncertainty in conformity assessment

NEXT: Uncertainty Estimation (II): Combined and Expanded Uncertainties. Cover Factor, k