That uncertain feeling

The reliability of decisions based on measurements depends on knowing the ‘uncertainty’ of such results. If the uncertainty of measurement is underestimated, erroneous decisions may be made that can have large financial consequences.

In everyday life, we are used to the doubt that arises when estimating how large or small things are. For example, if someone asks, ‘what do you think the temperature of this room is?’, we might say, ‘about 23°C’. The use of the word ‘about’ implies that we know the room is not exactly 23°C but is somewhere near it. We recognise that there is some doubt about the value of the temperature that we have estimated.

We could, of course, be a bit more specific. We could say, ‘it is 23°C give or take a couple of degrees’. The term ‘give or take’ implies that there is still doubt about the estimate, but now we are assigning limits to the extent of that doubt. We have given some quantitative information about the doubt, or uncertainty, of our estimate.

This estimate of the room temperature has been based on a subjective evaluation. It is not entirely a guess, as we may have experience of exposure to similar and known environments. However, in order to make a more objective measurement it is necessary to make use of a measuring instrument of some kind; in this case we can use a thermometer.

However, no measuring instrument is perfect and there will always be uncertainties involved. Other uncertainties can arise in a measurement situation. These may include the effects due to the environment, the operator and from the quantity being measured. If this quantity relates to a material that is not uniform in its characteristics, and only part of that material has been analysed, the result may be an imperfect description of the material as a whole.

Uncertainty in sampling

It is sometimes the case that a particular characteristic of a substance has to be deduced by analysing a sample taken from it. Clearly, it is important to define, as closely as possible, the characteristic intended to be deduced from the sampling process – this is known as the sampling target. This means that it will be necessary to devise a sampling plan that captures, as well as possible, information that is representative of the target.

The sampling plan is critical to the validity of the results. It must address both spatial and temporal variations in the sampling target. For example, river water may be sampled to determine whether any pollutants from a nearby factory are present.

“Risk management is based on having confidence in the decision-making process

If the sample were taken at a time when the factory was closed for maintenance, no pollutants would be generated and the result might therefore not represent the situation at other times of year. Similarly, if the samples were taken upstream of the factory it is unlikely that the analysis would reveal the true situation.

Even if the spatial and temporal samples are taken in an optimal manner, there will still be uncertainty, due to the fact that samples can never be fully representative of the sampling target. For example, suppose that it was decided to estimate the average height of all people in the UK. It would be impractical to measure everybody’s height, of course, so an estimate could be made by measuring a sample of the population. Now, suppose only one person was sampled – clearly, the chance of this person being an exact average of the whole population is minimal. If, for example, a small child had been sampled the estimate of the average population height would be grossly in error.

It would therefore be necessary to take a large number of samples so that there is a good chance of capturing information that is sufficient to represent the population as a whole. Furthermore, the sampling plan must ensure that all members of the population have an equal chance of being selected. This is known as random sampling. If, for example, the sampling had been performed at a local primary school then a large number of small children would be included and the estimate would be biased towards their average height. Therefore the sampling must be truly random, rather than arbitrary.

The only way to achieve a perfect result would be to sample everybody’s height. Even if 99% of the population were measured, there would still be uncertainty in the estimate of the sampling target, as the remaining one per cent would not have an average height that exactly represented that of the whole population. So it can be seen that sampling itself introduces uncertainty, no matter how good the sampling plan and process is. This sampling error can be a significant source of uncertainty.

Risk assessment

Risk management is based on having confidence in the decision-making process and in the reliability of the decisions that an organisation makes. This reliability can only exist when there is confidence that the information on which the decisions are based is valid. Thus, not only does measurement uncertainty – including sampling uncertainty – have to be taken into account, but the sampling plan has to be such that the relevant information has been captured in a reliable manner. Otherwise the organisation could make decisions that appeared appropriate but carried a degree of risk that was not recognised.

If, for example, the management of the factory mentioned above had sampled the river water in an inappropriate place, or at the wrong time of year, they would have a false impression of the amount by which they were polluting the environment. Risk-based decisions, even if made in good faith and considering all the available information, may therefore have an inappropriate outcome. This can have huge consequences, both financially and with respect to the organisation’s reputation, if the true situation is then discovered.

Decisions have to be made taking into account a balance of cost and risk. It may be tempting to minimise the costs of sampling and testing, but is it really worth the risk if the sampling is carried out inappropriately and the information upon which the decision has to be made is unreliable?