– We know that when we take the average of a sample, it is probably not exactly the same as the average of the population.

– Confidence intervals help us determine the likely range of the population parameter.

* For example, if my 95% confidence interval is 5 +/- 2, then I have 95% confidence that the mean of the population is between 3 and 7.

Why Do We Need Confidence Intervals? confidence interval calculator

– Sample statistics, such as Mean and Standard Deviation, are only estimates of the population’s parameters.

– Because there is variability in these estimates from sample to sample, we can quantify our uncertainty using statistically-based confidence intervals.

– Confidence intervals provide a range of plausible values for the population parameters (m and s).

– Any sample statistic will vary from one sample to another and, therefore, from the true population or process parameter value.

Confidence Interval for the Mean (Mu) with Population Standard Deviation (Sigma) Unknown:

– A very important point to remember is that for this example we assumed that we knew the population standard deviation, and many times that is not the case. Often, we have to estimate both the mean and the standard deviation from the sample.

– When Sigma is not known, we use the t-distribution rather than the normal (z) distribution.

The t-distribution:

– In many cases, the true population Sigma is not known, so we must use our sample standard deviation (s) as an estimate for the population standard deviation (s).

– Since there is less certainty (not knowing Mu or Sigma ), the t-distribution essentially “relaxes” or “expands” our confidence intervals to allow for this additional uncertainty.

– In other words, for a 95% confidence interval, you would multiply the standard error by a number greater than 1.96, depending on the sample size.

– 1.96 comes from the normal distribution, but the number we will use in this case will come from the t-distribution.

What Is This t-Distribution?

– The t-distribution is actually a family of distributions.

– They are similar in shape to the normal distribution (symmetric and bell-shaped), although wider, and flatter in the tails.

– How wide and flat the specific t-distribution is depends on the sample size. The smaller the sample size, the wider and flatter the distribution tails.

– As sample size increases, the t-distribution approaches the exact shape of the normal distribution.

Sample Size Concerns

– If we sample only one item, how close do we expect to get to the true population mean?

– How well do you think this one item represents the true mean?

– How much ability do we have to draw conclusions about the mean?

– What if we sample 900 items? Now, how close would we expect to get to the true population mean?

Three concepts affect the conclusions drawn from a single sample data set of (n) items:

– Variation in the underlying population (sigma)

– Risk of drawing the wrong conclusions (alpha, beta)

– How small a Difference is significant (delta)

– These 3 factors work together. Each affects the others.

Variation: When there’s greater variation, a larger sample is needed to have the same level of confidence that the test will be valid. More variation diminishes our confidence level.

Risk: If we want to be more confident that we are not going to make a decision error or miss a significant event, we must increase the sample size.

Difference: If we want to be confident that we can identify a smaller difference between two test samples, the sample size must increase.

– Larger samples improve our confidence level.

– Lower confidence levels allow smaller samples.

– All of these translate into a specific confidence interval for a given parameter, set of data, confidence level and sample size.

– They also translate into what types of conclusions result from hypothesis tests.

– Testing for larger differences between the samples, reduces the size of the sample. This is known as delta (D).

Type I Error

– Alpha Risk or Producer Risk is the risk of rejecting the null, and taking action, when none was necessary

– It is the alpha value you choose.

– The confidence level is one minus the alpha level.

– Most non-critical business processes choose an alpha of 5% with a Confidence Level of 95

Type II Error

– Beta Risk or – Consumer Risk is the risk of accepting the null when you should have rejected it. No action is taken when there should have been action.

– The Type II Error is determined from the circumstances of the situation.

– If alpha is made very small, then beta increases (all else being equal).

– Requiring overwhelming evidence to reject the null increases the chances of a type II error.

– To minimize beta, while holding alpha constant, requires increased sample sizes.

– One minus beta is the probability of rejecting the null hypothesis when it is false. This is referred to as the Power of the test.