Sampling in Statistics

Copyright 2022 

Stephanie Glen

All Rights Reserved

Intro to Sampling

Samples

In statistics, you’ll be working with samples. A sample is just a part of a population. If you want to find out how much the average American earns, you aren’t going to want to survey everyone in the population (over 300 million people), so you would choose a small number of people in the population. For example, you might select 10,000 people.

Technically, you can’t just choose any 10,000 people. For it to be statistical (i.e., one that you can use in statistics), the actual size must be found using a statistical method. Ten thousand people might not be the optimal amount for valid survey results: you may need more, or less. There are many, many ways to find sample sizes, including using data from prior experiments or using an online sample size calculator. How you find a sample size can be quite complex, depending on what you want to do with your data.

If you’ve decided to assemble your sample from scratch (for example, you aren’t using prior data), then you need to choose a sampling method. Which sampling method you use depends on what resources and information you have available.

For example, the national draft worked by drawing random birth dates, a method called simple random sampling. For that to work, the government needed a list of every potential draftee’s name and date of birth. The draft could also have used systematic sampling, drawing the nth name from a list (for example, every 100th name). For that to have worked, all the names must first have been compiled on a list.

What is a Sample Size?

A sample size is a part of the population chosen for a survey or experiment. For example, you might take a survey of dog owner’s brand preferences. You won’t want to survey all the millions of dog owners in the country (either because it’s too expensive or time consuming), so you take a sample size. That may be several thousand owners. The sample size is a representation of all dog owner’s brand preferences. If you choose your sample wisely, it will be a good representation.

When Error can Creep in

When you only survey a small sample of the population, uncertainty creeps into your statistics. If you can only survey a certain percentage of the true population, you can never be 100% sure that your statistics are a complete and accurate representation of the population. This uncertainty is called sampling error and is usually measured by a confidence level. A confidence level is the probability a parameter value falls within a specified range of values. Loosely speaking, it tells you how confident you are that your results will contain the true value for the population—even if you (or someone else) were to repeat your experiment. For example, you might state that your results are at a 90% confidence level. That means if you were to repeat your survey over and over, 90% of the time you would get the same results.

Sampling Distribution

A sampling distribution is a graph of a statistic for your sample data. While, technically, you could choose any statistic to paint a picture, some common ones you’ll come across are:

• Mean (the average)

• Mean absolute value of the deviation from the mean

• Range (a measure of spread)

• Standard deviation of the sample (a measure of spread)

• Unbiased estimate of variance

• Variance of the sample

Up until a certain point in statistics, you plot graphs for a set of numbers. For example, you might have graphed a data set and found it follows the shape of a normal distribution with a mean score of 100. Where probability distributions differ is that you aren’t working with a single set of numbers; you’re dealing with multiple statistics for multiple sets of numbers. If you find that concept hard to grasp: you aren’t alone.

While most people can imagine what the graph of a set of numbers looks like, it’s much more difficult to imagine what stacks of, say, averages look like.

An explanation…

Let’s start with a mean, like heights of students in the above cartoon. As you probably know, heights (and many other natural phenomenon) follow a bell curve shape. So, if you surveyed your class, you’d probably find a few short people, a few tall people, and most people would fall in between.

Let’s say the average height was 5’9″. Survey all the classes in your school and you’ll probably get somewhere close to the average. If you had 10 classes of students, you might get 5’9″, 5’8″, 5’10, 5’9″, 5’7″, 5’9″, 5’9″, 5’10, 5’7″, and 5’9″. If you graph all those averages, you’re probably going to get a graph that resembles the sporkahedron. For other data sets, you might get a flatlined distribution, resembling a flat-roofed building.

It’s almost impossible to predict what that graph will look like, but the Central Limit Theorem tells us that if you have a ton of data, it’ll eventually look like a bell curve. That’s the basic idea: you take your average (or another statistic, like the variance) and you plot those statistics on a graph.

The mean of the sampling distribution of the means is just math-speak for plotting a graph of averages (like I outlined above) and then finding the average of that set of data.

Mean of the sampling distribution of the mean

In a nutshell, the mean of the sampling distribution of the mean is the same as the population mean (what you would expect to find as an average if you were to get data from the entire population). For example, if your population mean (μ) is 99, then the mean of the sampling distribution of the mean, μm, is also 99 (if you have a sufficiently large sample size).

The Central Limit Theorem.

Roughly stated, the central limit theorem tells us that if we have many independent, identically distributed variables, the distribution will approximately follow a bell shape. It doesn’t matter what the underlying distribution is.

Here’s a simple example of the theory: when you roll a single die, your odds of getting any number (1, 2, 3, 4, 5, or 6) are the same (1/6). The mean for any roll is (1 + 2 + 3 + 4 + 5 + 6) / 6 = 3.5. The results from a one-die roll are shown in the first figure below: it looks like a uniform distribution. However, as the sample size is increased (two dice, three dice…), the distribution of the mean looks more and more like a normal distribution. That is what the central limit theorem predicts.