Answer keys for every guide's practice and quiz are at the back.
NC Math 3 · NC.M3.S-IC.1 · S-IC.3 · S-IC.6
Sampling, Study Design, and Reading Reports
Study notes
Name ____________________ Date ____________
Most of the time we cannot ask everyone. We study a sample and use it to say something about a whole population. This guide shows why a random sample lets you do that, how the three kinds of study (sample surveys, observational studies and experiments) use randomization differently, and what each one lets you conclude. Then you will practice evaluating a report: who collected the data, how, and how the graph shows it.
From a random sample to a population (S-IC.1)
Population, sample, parameter, statistic
The population is the whole group you want to know about. The sample is the part of it you actually measure.
A number that describes the whole population is a parameter. A number that describes a sample is a statistic. You usually cannot get the parameter, so you use the statistic to estimate it. This is called making an inference.
The sample is called random when chance, not a person's choice, decides who is in it. In a simple random sample, every member of the population and every group of the same size has an equal chance of being picked.
Term
Meaning
Example
Population
Everyone or everything you want to know about
All 2,000 students at a school
Sample
The part of the population you measure
25 students picked by random number
Parameter
A number about the whole population
40% of all 2,000 students ride the bus
Statistic
A number about the sample
9 of the 25 sampled students ride the bus: 9 ÷ 25 = 36%
Example
A school has 2,000 students. A random sample of 25 is asked how they get to school, and 9 ride the bus. Name the population, the sample, and the statistic. Population: all 2,000 students. Sample: the 25 students picked. Statistic: the sample proportion 9 ÷ 25 = 0.36 = 36%. The parameter is the percent of all 2,000 students who ride the bus. It is not known from this sample alone, but the 36% is the estimate of it.
Trap: Calling the sample's percent "the percent for the school". A statistic is only an estimate of the parameter, and a different sample would give a different statistic.
Why the sample must be random
If a person chooses who is in the sample, the choice can lean toward one kind of member without anyone meaning to. Then the sample's statistic is pushed in one direction, and every new sample is pushed the same way. That push is bias.
Random selection gives every member a fair chance, so the sample has no built-in lean. A random sample is more likely to look like the population, but it is never guaranteed to match it.
The inference only goes as far as the population the sample was drawn from. A random sample of students at one school says something about that school, not about every school in the state.
Sampling variability: different samples, different statistics
In this simulation, the population is 2,000 students and exactly 40% (800 students) ride the bus. We took 24 different random samples of 25 students. Each dot is one sample, placed at the percent of that sample who ride the bus. The dashed line marks the true value, 40%.
The dots are not all at 40%. They run from 24% to 52%, and only 4 of the 24 samples hit exactly 40%. All of the samples were taken the same fair way, so this spread is just chance. It is called sampling variability.
The dots pile up around 40%, with fewer further away. That is why one random sample is a reasonable estimate of the parameter. It is also why you should not expect it to be exactly right.
Each dot is one random sample of 25 students. Horizontal axis: percent of the sample who ride the bus.
A non-random sample misses in one direction
Now suppose the 25 students are always picked from the 300 students standing at the bus loop (210 of them, or 70%, ride the bus). We again took 24 samples, with the same dashed line at the true 40%.
The spread from sample to sample is still there, 44% to 84%. But the whole group of dots sits to the right of 40%. All 24 samples are above the true value. Taking more samples this way would not fix it, because the method itself leans.
Same population and same size of sample, but the students were picked at the bus loop instead of at random.
Trap: Thinking a big sample cures bias. Sampling variability shrinks when you ask more people, but a biased method stays biased however many people you ask.
Three kinds of study (S-IC.3)
Survey, observational study, experiment
A sample survey asks a sample questions to estimate something about a population, such as a percent who favor a plan. The purpose is to describe the population.
An observational study records what happens to subjects without changing anything. The researcher does not choose who gets which condition. The purpose is to look for patterns and associations.
An experiment applies a treatment to subjects on purpose and compares the results with a group that did not get it. The purpose is to find out whether the treatment causes a difference.
There are two different jobs for chance. Random selection means chance decides who is in the sample. It lets you generalize to the population. Random assignment means chance decides who gets which treatment. It lets you make cause-and-effect conclusions.
Design
Random how?
Can conclude
Sample survey
Random selection of who is asked
Describes the population (an estimate). No cause and effect.
Observational study
Random selection of subjects, when possible. Nothing is assigned.
An association (a pattern) only. No cause and effect.
Experiment
Random assignment of subjects to treatments (and random selection too, if possible)
Cause and effect, when the groups were assigned at random. It applies to the population only as far as the subjects were randomly chosen from it.
Why only an experiment can show cause and effect
In an observational study, the groups form themselves. People who take vitamin D may also exercise more, eat better or see a doctor more often. These other factors, called confounding variables, are mixed in with the thing you are looking at. You cannot tell which one caused the difference.
In an experiment, chance puts each subject in a group. The groups are alike on average in every way, including the ways nobody measured. If the results are different afterward, the treatment is the reasonable explanation.
A control group (no treatment, or a placebo, which is a fake treatment) gives you something to compare with.
Example
Researchers find that students who sleep 8 hours score higher on tests. Another team takes 60 volunteer students, uses a random number generator to put 30 on an 8-hour schedule and 30 on a 6-hour schedule for a week, then gives all 60 the same test. Which is which, and which can show that sleep causes higher scores? The first is an observational study: the researchers only recorded sleep and scores, and the students chose their own sleep. It can show an association, but not that sleep caused the scores (students who sleep more might also study more). The second is an experiment: the researchers assigned the sleep schedules, and chance decided who got which. If the scores are different by more than chance, the sleep schedule is the reasonable cause. Because the 60 were volunteers and not a random sample of all students, the result applies most safely to students like them.
Trap: Mixing up the two kinds of randomness. A random sample (selection) lets you generalize to a population. Random assignment lets you claim cause and effect. A study can have either one, both or neither.
Evaluating reports (S-IC.6)
Three questions to ask about any report, then a verdict
Source: who collected and paid for the data? Is the group trying to sell something or win an argument? Is the original data available, or only a headline?
Design: was it a survey, an observational study or an experiment? How were the subjects chosen, and were groups assigned at random? How big was the sample?
Display: does the graph or table show the data fairly? Check the axes, the intervals and the shapes.
Then decide what the report really supports. A claim of cause and effect needs a randomized experiment. A claim about a whole population needs a random sample of that population.
Spotting bias in how the sample was collected
A fix for the first three is the same: choose the sample by random selection from a list that includes the whole population. A fix for the fourth is to word the question neutrally.
Bias
What went wrong
Example
Voluntary response
People choose themselves to answer. Those with strong opinions respond most.
A radio call-in poll about a new stadium
Convenience
The sample is whoever is easiest to reach.
Surveying only the first 30 students into the cafeteria
Undercoverage
Some part of the population has no chance of being chosen.
A survey made only by landline phone, which leaves out people who have no landline
Leading question
The wording pushes toward one answer.
"Don't you agree the wasteful stadium should be stopped?"
Misleading graphs: a truncated axis
Two brands report 52% and 50% satisfied customers. In this graph the vertical axis starts at 48, not 0, so bar A is 4 units tall and bar B is only 2. A looks twice as tall as B, but the numbers differ by just 2 percentage points.
Always read the axis numbers. A bar's height should be in proportion to the value, which only works when the axis starts at 0.
Truncated axis: the same data as the next graph. Look at the numbers on the left.
Trap: Judging by how the bars look. Here 52 ÷ 50 = 1.04, so A is 4% larger, not 100% larger.
The same data, drawn fairly
With the axis starting at 0, the bars are in proportion to the values and the small difference looks small.
If a small change really matters, a report can say so in words and still show the whole axis.
Axis starts at 0: bar heights are in proportion to 52 and 50.
Other misleading displays
Unequal intervals: a histogram or a number line whose steps are not the same size, such as 0–10, 10–20, 20–50, 50–100, bends the picture. Equal steps on the axis should be equal amounts.
3D pictographs: a picture whose height, width and depth all grow makes a doubled value look much bigger. If a glass icon is twice as tall and twice as wide to show a value that is twice as big, its area is 2 × 2 = 4 times as big, and a 3D version suggests even more.
Others to watch for: a graph with no axis labels or units, a cropped time period, and a headline that claims more than the data show.
Example
A web page for a vitamin company says, "Survey: 9 of 10 customers feel more energetic!" The survey was an option on the company's own site, and a 3D picture of a glass is drawn about twice as large for the vitamin as for the competitor. Critique the report. Source: the company that sells the product, so it has a reason to show good news. Design: an online survey that customers chose to answer is voluntary response, and it only includes customers, not people who stopped buying. It is a survey or an observational study, not an experiment, so even a real 90% could not show the vitamin caused the energy. Display: a doubled 3D icon exaggerates the difference. The report also gives no sample size.
Memory hook: Random selection lets you generalize. Random assignment lets you say "causes". Before you trust a report, ask: who, how, and how shown?
NC Math 3 · NC.M3.S-IC.1 · S-IC.3 · S-IC.6
Sampling, Study Design, and Reading Reports
Practice
Name ____________________ Date ____________
1. A school has 1,200 students. The principal chooses 100 of them using a random number generator and asks each how they get to school. 37 say they walk. (a) Name the population and the sample. (b) What is the statistic, and what does it estimate? (c) If she chose 100 different students, would she expect the same statistic?
2. Classify each study as a sample survey, an observational study or an experiment. (a) A polling group phones 800 randomly chosen registered voters and asks which candidate they prefer. (b) A researcher looks at medical records of 2,000 adults and compares the blood pressure of those who chose to take a daily vitamin with those who did not. (c) A doctor randomly divides 60 patients into two groups of 30. One group takes a new pill and the other takes a placebo, and she compares their blood pressure after a month.
3. Name the main source of bias in each plan. (a) A radio station asks listeners to call in and say whether they support a new stadium. (b) A student surveys the first 30 classmates who walk into the cafeteria about lunch quality. (c) A town surveys residents about internet use by calling only landline numbers from an old phone book. (d) A survey asks, "Don't you agree that the wasteful, overpriced stadium should be stopped?"
4. A teacher wants to test whether a new tutoring app improves quiz scores. Eighty students volunteer. (a) Describe how to use randomization. (b) If the app group scores clearly higher, can she say the app caused the improvement? (c) Can she say it works for all students in the state?
5. A newspaper reports that students who eat breakfast have a higher mean GPA than students who skip it. Researchers looked at school records for 300 students that show whether each one eats breakfast and what their GPA is. The principal says, "Let's require breakfast to raise GPAs." (a) What kind of study is this? (b) Is the principal's causal conclusion justified? (c) Give a reason the data could show this pattern even if breakfast has no effect.
6. A school has 2,000 students and 40% ride the bus. Five different random samples of 25 students gave these numbers of bus riders: 7, 11, 9, 13, 10. (a) Find each sample proportion. (b) Find the range of the five proportions. (c) Does the spread mean someone sampled incorrectly?
7. A magazine compares two phone brands. The graph shows Brand A at 63% satisfied and Brand B at 60% satisfied, and Brand A's bar looks 4 times as tall. (a) What is misleading about the graph? (b) How many times as tall should Brand A's bar be if the axis starts at 0? (c) How could the graph be fixed?
8. A website for a snack company says: "Taste test: 8 out of 10 people prefer our chips!" It explains that the 10 people were the first 10 customers to walk up to the company's booth at a fair. The report shows a picture of a chip bag for each brand, and the bag for the company's brand is drawn twice as tall and twice as wide as the competitor's. (a) Identify the source, the design and the display. (b) Name the biggest problem with each. (c) Can the report say 80% of all people prefer the chips?
NC Math 3 · NC.M3.S-IC.1 · S-IC.3 · S-IC.6
Sampling, Study Design, and Reading Reports
Quiz
Name ____________________ Date ____________
1. A forester measures the trunk diameters of a simple random sample of 36 of the 2,400 oak trees in a state forest. The mean diameter of the sample is 14.2 inches. (a) Name the population. Is 14.2 a parameter or a statistic? (b) A second forester takes her own simple random sample of 36 oaks from the same forest and gets a mean of 15.1 inches. Did one of them make a mistake? (c) Can the first forester use the 14.2 to describe oak trees in other states?
2. A district has 6,000 students. To estimate the percent who would ride a late bus, a researcher takes 30 random samples of 20 students and 30 random samples of 200 students. Each dot is the percent who said yes in one sample. The two dot plots use the same scale, but they are not labeled by sample size. (a) Which plot used the samples of 200 students? How can you tell? (b) About what percent of all 6,000 students would ride the late bus? (c) Suppose the samples of 200 had been taken only from students waiting in the band room, instead of at random. Would plot Y still center on the true percent? Explain.
3. For each study, say whether it used random selection, random assignment, both or neither. Then say what it can conclude. Study 1: A hospital randomly picks 300 of last year's patient records and compares the recovery times of patients whose doctors chose physical therapy with those whose doctors did not. Study 2: A shelter has 24 volunteer dogs. A coin flip decides which 12 dogs get food A and which 12 get food B for two months, and then the dogs' weights are compared.
4. A gardener wants to know whether a new plant food makes tomato plants grow taller. She puts the plant food on the 10 plants along the sunny south fence and none on the 10 plants along the shady north fence. At the end of the summer the fed plants are 8 cm taller on average. (a) Why can't she conclude that the plant food caused the difference? (b) Name a confounding variable. (c) Describe a better design.
5. Name the main source of bias in each plan, and say how to fix it. (a) A TV show asks viewers to text YES or NO about building a new park, and reports the percent of texts that say YES. (b) A researcher polls a city's residents, using a list of homeowners from the county tax office. (c) A survey asks, "Given how dangerous texting while driving is, don't you agree it should be banned?"
6. A gym newsletter prints the graph below, where the bar heights are the numbers of members in each age group. Its headline says, "Most of our members are between 30 and 50!" (a) What about the graph makes the headline look true? (b) What fraction of the members are actually between 30 and 50? (c) Which age group has the most members per year of age, and how could the graph be redrawn fairly?
NC Math 3 · NC.M3.S-IC.4 · S-IC.5
Simulation, Margin of Error, and Comparing Groups
Study notes
Name ____________________ Date ____________
A sample is only a piece of a population, so the number you get changes every time you sample. Simulation lets you see how much it changes. You will use it two ways. First, to estimate a population mean or proportion and find a margin of error. Second, to decide whether a difference between two groups in an experiment is real or could just be luck of the draw.
The big idea: repeat it many times
One sample is one guess
Ask 100 students a yes/no question and 56 say yes. The sample proportion is 0.56. Ask a different 100 students and you might get 0.52 or 0.61.
A statistic is a number computed from a sample, like a sample proportion or a sample mean. Its value changes from sample to sample. Simulation shows how much.
A simulation uses random numbers to imitate taking a sample (or running an experiment) again and again. You record the statistic each time, then look at the spread of all the results.
Decide what to imitate. For an estimate, imitate sampling from a population that looks like your sample. For two groups, imitate the case where the group makes no difference.
Do one random trial and record the statistic.
Repeat many times. This guide uses 1000 trials.
Graph all the results in a dot plot or histogram. Look at the center and the spread.
Standard
Question
What you simulate
What you read off
S-IC.4
How well does a sample estimate the population?
Many samples of the same size
Spread of the sample statistics, which gives the margin of error
S-IC.5
Is a difference between two groups real?
Many shuffles of the group labels
Where the observed difference falls in the shuffled results
Estimating a proportion and its margin of error (S-IC.4)
Simulate many samples and look at the spread
Margin of error Use the SD of the simulated statistics, not the SD of the original data. — margin of error ≈ 2 × (SD of the simulated sample statistics)
The estimate — estimate = sample statistic ± margin of error
A poll of 100 students finds that 56 prefer a later start time, so the sample proportion is 0.56. We want to estimate the proportion of all students.
To simulate, act as if the whole population has a proportion of 0.56. Draw 1000 random samples of size 100 from it and record each sample proportion. The graph shows all 1000 results. Each bar counts how many samples gave that proportion.
The 1000 results are centered at about 0.559 and have a standard deviation (SD) of about 0.050. The dashed line marks the observed 0.56.
The thick green bar runs from 2 SD below the center to 2 SD above it. In this simulation, 952 of the 1000 samples (about 95%) landed inside it. The few orange bars outside are the unusual samples.
Example
Use the simulation above to estimate the proportion of all students who prefer a later start time. SD of the 1000 simulated proportions ≈ 0.050.
Margin of error ≈ 2 × 0.050 = 0.10.
Estimate: 0.56 ± 0.10, so from 0.46 to 0.66.
We estimate that between 46% and 66% of all students prefer a later start.
Trap: Using the SD of the original yes/no answers, or just using the sample proportion with no margin at all. The margin of error comes from the spread of the simulated sample proportions.
Estimating a mean works the same way
Nothing changes when the statistic is a sample mean. Simulate many samples, compute the mean of each, and find the SD of those simulated means.
Example
A sample of 40 cereal boxes has a mean weight of 12.1 oz. A simulation of 1000 samples of size 40 gives sample means with an SD of 0.15 oz. Estimate the population mean weight. Margin of error ≈ 2 × 0.15 = 0.30 oz.
Estimate: 12.1 ± 0.3 oz, so from 11.8 oz to 12.4 oz.
We estimate that the mean weight of all boxes is between 11.8 oz and 12.4 oz.
Bigger samples give smaller margins of error
Samples of 25
Same population (proportion 0.56), but now each sample has only 25 people. A sample of 25 with 14 yes answers also gives 0.56.
The 1000 simulated proportions are centered at about 0.558, but they are spread wide. The SD is about 0.095.
Margin of error ≈ 2 × 0.095 ≈ 0.19, so the estimate is 0.56 ± 0.19, from 0.37 to 0.75.
Samples of 100
Both graphs use the same scale, so you can compare them directly. With samples of 100 the results bunch up tightly around the center.
The SD is about 0.050, so the margin of error is about 0.10. The estimate is 0.56 ± 0.10, from 0.46 to 0.66.
Sample size n
SD of simulated proportions
Margin of error (2 SD)
Estimate
25
0.095
0.19
0.56 ± 0.19
100
0.050
0.10
0.56 ± 0.10
Example
How did the margin of error change when the sample size went from 25 to 100? The sample size was multiplied by 4 (25 × 4 = 100).
The margin of error went from 0.19 to 0.10, about half.
A bigger sample gives a smaller margin of error, so a more precise estimate.
Trap: Thinking that doubling the sample size cuts the margin of error in half. To cut it in half you need about 4 times as many people. Doubling the sample only shrinks the margin of error by about 30%.
Is the difference between two groups real? (S-IC.5)
An experiment with two groups
Ten plants were randomly given fertilizer and ten were not. After a month the growth in centimeters was measured. The fertilized plants grew a mean of 16.7 cm and the control plants grew a mean of 14.4 cm.
The observed difference in means is 16.7 − 14.4 = 2.3 cm. The question: did the fertilizer cause the extra growth, or could random assignment alone produce a gap like that?
Shuffle the labels (re-randomization)
Suppose the fertilizer did nothing. Then each plant's growth would be the same whichever group it landed in, and the labels "fertilizer" and "control" would be just random tags. Shuffling the tags imitates running the experiment again under that assumption.
Write all 20 growth values in one pool.
Shuffle the pool and deal 10 values to a fake "fertilizer" group and 10 to a fake "control" group.
Record the difference in the two group means.
Repeat 1000 times.
Graph the 1000 differences and mark the observed difference on the graph.
Where does the observed difference fall?
Decision rule Compare the observed difference with the shuffled differences. — beyond about 2 SD, or rarely reached by shuffling → evidence the groups really differ
The 1000 shuffled differences are centered at about 0, which makes sense: shuffling destroys any real group effect. Their SD is about 0.92, so the middle 95% runs from about −1.8 to 1.8 (the green bar).
The observed difference of 2.3 is far out in the tail, about 2.5 SD from the center. Only 19 of the 1000 shuffles gave a difference as far from 0 as 2.3 in either direction (orange bars and beyond).
A difference in the tails, beyond about 2 SD or rarely reached by shuffling, is hard to explain by chance alone. That is evidence of a real difference.
Example
Use the simulation to decide whether the fertilizer made a difference in plant growth. SD of the shuffled differences ≈ 0.92, so 2 SD ≈ 1.84.
Observed difference = 2.3, which is more than 1.84, so it is beyond 2 SD.
Only 19 of 1000 shuffles (1.9%) were that extreme.
Conclusion: there is evidence that the fertilizer increased plant growth.
When the difference is not in the tail
If the observed difference lands in the middle of the shuffled results (within about 2 SD, or reached by many of the shuffles), then random assignment alone could easily have produced it.
In that case the data do not give evidence of a real difference. This is not the same as showing the groups are equal.
Trap: Writing "this proves the fertilizer works" or "there is no difference." A simulation gives evidence, not proof. Say "there is evidence that…" for a result in the tail and "there is not enough evidence that…" for a result in the middle.
Memory hook: Estimating: simulate samples, margin of error ≈ 2 SD, bigger n means smaller margin. Comparing: shuffle the labels, see where the observed difference lands, tail means evidence.
NC Math 3 · NC.M3.S-IC.4 · S-IC.5
Simulation, Margin of Error, and Comparing Groups
Practice
Name ____________________ Date ____________
1. A sample of 36 pizza deliveries has a mean delivery time of 28 minutes. A simulation of 1000 samples of size 36 gives sample means with a standard deviation of 1.5 minutes. Give an estimate of the population mean delivery time with its margin of error, as an interval.
2. In a poll of 200 students, 130 said they have a part-time job, so the sample proportion is 0.65. A simulation of 1000 samples of size 200 gives sample proportions with a standard deviation of 0.034. Estimate the proportion of all students with a part-time job, with a margin of error.
3. A simulation with samples of size 50 gives a margin of error of about 0.14. About what margin of error would you expect from a simulation with samples of size 200? Explain.
4. A poll of 100 voters finds 54% support a new bus line. A simulation of 1000 samples of size 100 gives sample proportions with a standard deviation of 0.05. A reporter writes: "A majority of all voters support the bus line." Does the estimate back up that claim? Explain.
5. In an experiment, 15 students were randomly assigned to be tutored and 15 were not. The tutored group's mean test score was 6 points higher than the other group's. To see if that could be chance, the group labels were shuffled 1000 times and the difference in means recorded each time. The table shows how the shuffled results compared. What do you conclude?
Shuffled difference
Number of the 1000 shuffles
As far from 0 as 6 points (either direction)
12
Closer to 0 than 6 points
988
6. Ten students studied with flashcards (group A) and ten studied with a summary sheet (group B), then took a memory quiz scored out of 20. Group A averaged 13.5 and group B averaged 13.1, an observed difference of 0.4. The graph shows the differences from 1000 shuffles of the group labels. The shuffled differences have a mean of 0.00 and a standard deviation of 0.69. Is the 0.4 evidence that the study method matters?
7. In the plant experiment, a student writes: "The observed difference of 2.3 cm is beyond 2 standard deviations of the shuffled differences, so this proves the fertilizer works." (a) What do the shuffles assume? (b) Rewrite the student's conclusion with correct wording.
8. Twelve students were randomly split into two groups of six. Group 1 listened to music while studying and group 2 did not. On the quiz, group 1's mean was 1.9 points higher. After 1000 shuffles of the group labels, the shuffled differences had a mean of 0 and a standard deviation of 1.0. A student concludes: "Music has no effect on quiz scores." Is that conclusion justified? What can you say?
NC Math 3 · NC.M3.S-IC.4 · S-IC.5
Simulation, Margin of Error, and Comparing Groups
Quiz
Name ____________________ Date ____________
1. A random sample of 50 laptops of one model has a mean battery life of 7.4 hours. The 50 individual battery lives have a standard deviation of 1.3 hours. A simulation of 1000 samples of size 50 gives sample means with a standard deviation of 0.18 hours. Estimate the mean battery life of all laptops of this model, with its margin of error, as an interval.
2. In a random sample of 250 residents, 90 say they support a new library tax, so the sample proportion is 0.36. The graph shows the sample proportions from 1000 simulated samples of size 250, drawn as if the population proportion were 0.36. Their standard deviation is 0.030. (a) Estimate the proportion of all residents who support the tax, with a margin of error. (b) A council member says, "At least 45% of residents support it." Does the estimate back that up?
3. A poll of 150 people gives a sample proportion of 0.50. A simulation of 1000 samples of size 150 gives a margin of error of about 0.08. A student says: "If we double the poll to 300 people, the margin of error will be cut in half, to 0.04." (a) Is the student right? (b) About what sample size would give a margin of error of 0.04? (c) About what margin of error should you expect for 300 people?
4. A teacher randomly assigned 16 students to two groups of 8 to solve the same puzzle. Group A got a hint sheet and group B did not. The table shows each student's solving time in seconds. To see whether the hint could have mattered, the group labels were shuffled 1000 times and the difference in means (A − B) was recorded each time. The graph shows the 1000 shuffled differences, which have a mean of −0.01 and a standard deviation of 1.92. (a) Find the observed difference in means (A − B). (b) Is it beyond 2 standard deviations of the shuffled differences? (c) What do you conclude?
Group
Times (seconds)
A (hint sheet)
41, 38, 45, 36, 41, 37, 43, 39
B (no hint)
47, 44, 50, 42, 45, 44, 46, 42
5. Seedlings were randomly assigned to grow under blue light (5 seedlings) or white light (5 seedlings). Their heights after two weeks, in cm, were: blue 12, 15, 14, 16, 13; white 11, 13, 12, 15, 14. (a) Find the observed difference in means (blue − white). (b) In one shuffle of the 10 heights, the fake blue group got 11, 14, 15, 12, 16 and the fake white group got the other five. Find the difference in means for this shuffle. (c) After 1000 shuffles, 424 of them gave a difference at least as far from 0 as the observed difference. Do the data give evidence that light color matters?
6. A hospital randomly assigned 40 patients to two groups of 20. Group 1 got a new exercise program and group 2 got standard care. After a month, 15 of the 20 patients in group 1 and 7 of the 20 in group 2 had recovered. To test whether the program matters, the 22 recovered and 18 not-recovered patients were shuffled into two fake groups of 20, 1000 times, and the difference in recovery proportions (group 1 − group 2) was recorded each time. Those differences had a mean of 0.00 and a standard deviation of 0.16, and only 24 of the 1000 shuffles gave a difference as far from 0 as the observed one. (a) Find the observed difference in proportions. (b) Compare it with 2 standard deviations. (c) State the conclusion in words.
NC Math 3 · NC.M3.S-IC.1 · S-IC.3 · S-IC.6
Sampling, Study Design, and Reading Reports
Practice answer key
Name ____________________ Date ____________
1. (a) Population: all 1,200 students. Sample: the 100 chosen students. (b) The statistic is 37 ÷ 100 = 0.37 = 37%. It estimates the parameter, the percent of all 1,200 students who walk. (c) Not necessarily: a different random sample would give a statistic that is close but probably not the same.
The population is everyone the principal wants to know about, which is all 1,200 students. The sample is only the 100 who were asked. A number about the sample is a statistic, here the proportion 37 ÷ 100 = 0.37. The parameter is the same kind of number for all 1,200 students, and the principal does not know it. The statistic is her estimate of it. Because the sample was chosen by chance, a second sample of 100 would include different students, and sampling variability means its percent would likely come out a bit different, such as 33% or 41%.
(a) The pollsters select a random sample and only ask questions. Nothing is assigned. That is a sample survey. (b) The researcher only looks at records. The adults chose for themselves whether to take the vitamin, so no treatment was assigned. That is an observational study. (c) The doctor assigns the treatments (pill or placebo) and chance decides who gets which. That is an experiment. The key question is: did the researchers impose a treatment? Only in (c) did they.
(a) Listeners choose to call, and people with strong opinions are the most likely to call. That is voluntary response. (b) The first 30 students in the door are just the easiest ones to reach, and they may eat early for a reason. That is a convenience sample. (c) Residents with no landline, or not in the old phone book, have no chance to be picked, which is undercoverage. They may also use the internet differently from the people who are called. (d) The words "wasteful" and "overpriced" push the answer, so it is a leading question.
4. (a) Randomly assign 40 of the volunteers to use the app and 40 to not use it, then compare scores. (b) Yes, because assignment was random. (c) No: the 80 were volunteers, not a random sample of all students, so the result only applies safely to students like them.
Question
Needs
Does she have it?
Did the app cause the change?
Random assignment
Yes
Does it work for all students in the state?
Random selection from all students
No (volunteers)
(a) She should not let students pick their group, because the more motivated might choose the app. Instead chance assigns them: for example, numbers drawn at random put 40 in each group. (b) With random assignment, the two groups are alike on average in motivation, past grades and everything else, so if the app group does much better than chance alone would explain, the app is the reasonable cause. (c) Generalizing needs random selection from the population. These students volunteered, so they may differ from other students. Cause and effect comes from random assignment, and generalizing comes from random selection, and she has only the first.
5. (a) Observational study. (b) No. (c) Other factors could explain it, such as students who eat breakfast having a more regular routine or sleep schedule.
(a) Nobody assigned breakfast or no breakfast. The students already had their own habits and the researchers only looked at records, so this is an observational study. (b) An observational study can only show an association. Cause and effect needs random assignment. (c) A confounding variable can be mixed in with breakfast: for example, students with a steady sleep schedule, a calm home or a parent who checks homework may be the ones who eat breakfast and also do well in school. Then breakfast could look helpful even if it does nothing. To test the principal's claim, you would randomly assign some students to eat breakfast and others not to, and compare.
(a) Divide each count by 25: 7 ÷ 25 = 0.28, 11 ÷ 25 = 0.44, 9 ÷ 25 = 0.36, 13 ÷ 25 = 0.52, 10 ÷ 25 = 0.40. (b) The largest is 52% and the smallest is 28%, so the range is 24 percentage points. (c) Every sample was chosen at random, so nobody made a mistake. Different samples of the same population give different statistics. They tend to be near the true 40%, and here they run from 28% to 52%, much like the dots in the simulation above. Only one of the five equals 40% exactly.
7. (a) The vertical axis starts at 59, not 0, so the bar heights are not in proportion to the values. (b) 63 ÷ 60 = 1.05 times as tall. (c) Start the axis at 0 so the bar heights match the values.
The axis labels run from 59 to 64, which means each bar is only drawn from 59 up. Bar A is 63 − 59 = 4 units tall and Bar B is 60 − 59 = 1 unit tall, so A looks 4 ÷ 1 = 4 times as tall. But the values are 63 and 60, and 63 ÷ 60 = 1.05, so A is only 5% larger, a difference of 3 percentage points. A bar chart compares bar heights, so it should start at 0. The fixed graph shows the two bars almost the same height.
8. (a) Source: the snack company's own website. Design: a taste test of the first 10 people at its booth. Display: a pictograph with the company's bag drawn twice as tall and wide. (b) Source: the company has a reason to show good news. Design: a tiny convenience sample, not random. Display: doubling both dimensions makes the area 4 times as large. (c) No.
The source is the company that sells the chips, so you cannot assume the report is neutral, and it does not tell you who ran the test. The design is a convenience sample: the first 10 people to walk up to the booth are not chosen at random, and people at the booth may already like the brand. Ten people is also a very small sample, so a different 10 could give a very different result. The display is a pictograph in which the bag is twice as tall and twice as wide, so its area is 2 × 2 = 4 times as big. The eye reads area, so the picture makes the company's bag look 4 times as big as the other one. A pictograph should change only one dimension, so the area matches the value. Because the sample was not random and the group was tiny, the 80% describes only those 10 people. The report cannot say that 80% of all people prefer the chips.
NC Math 3 · NC.M3.S-IC.1 · S-IC.3 · S-IC.6
Sampling, Study Design, and Reading Reports
Quiz answer key
Name ____________________ Date ____________
1. (a) Population: all 2,400 oak trees in the forest. 14.2 is a statistic (a sample mean). (b) No: this is ordinary sampling variability. (c) No: the sample was drawn only from this forest.
(a) The population is every tree the forester wants to know about, all 2,400 oaks in the forest. A number computed from the sample, like the sample mean 14.2, is a statistic. The parameter would be the mean diameter of all 2,400 oaks, which nobody measured. (b) Both samples were chosen by chance, so neither forester did anything wrong. Different random samples contain different trees, so their means differ: here by 15.1 − 14.2 = 0.9 inch. This is sampling variability. Both numbers are estimates of the same parameter. (c) An inference only reaches as far as the population the sample was drawn from. Trees in other states had no chance of being picked, so the result does not describe them.
2. (a) Plot Y: its dots are packed much closer together. (b) About 35%. (c) No. A biased method leans one way no matter how many students are asked.
(a) Bigger random samples vary less from sample to sample, so their statistics bunch together. Plot Y's dots run from about 28% to 42%, while plot X's run from 10% to 65%. So Y is the samples of 200 and X is the samples of 20. (b) Both plots center at about 35%, and a random sample is not leaning either way, so about 35% of all students would ride the late bus. (c) The band room is not a random selection of the students. If band members like the late bus more or less than other students, every sample would lean the same way, and the dots would center away from the true percent. A larger sample makes the dots closer together but does not fix a leaning method.
3. Study 1: random selection only. It can describe last year's patients at that hospital, but it cannot show that physical therapy caused the difference in recovery. Study 2: random assignment only. It can show that the food caused a difference in weight for dogs like these, but it cannot be generalized to all dogs.
Study
Random selection?
Random assignment?
Can conclude
1: patient records
Yes
No
Association; describes last year's patients; no cause and effect
2: shelter dogs
No
Yes
Cause and effect for dogs like these; not all dogs
Study 1 used chance to pick the records, so what it finds can be generalized to last year's patients at that hospital. But the researchers did not assign anything: doctors chose who got therapy, and doctors may choose the patients with worse injuries, or the ones who will follow the plan. Those other factors are mixed in, so only an association can be claimed. Study 2 used chance to decide who gets which food, so the two groups are alike on average in everything else, and a difference in weight can reasonably be blamed on the food. But the dogs were volunteers, not a random sample of all dogs, so the result is safest for dogs like these.
4. (a) She did not assign the food at random: the fed plants were also the sunnier plants. (b) Amount of sunlight. (c) Randomly assign which 10 of the 20 plants get the food, treat all of them the same otherwise, and compare.
(a) She did apply a treatment, but she chose which plants got it, and her choice put the food and the sunny spot together. If the fed plants grew taller, it could be the food, the sun, or both, and there is no way to separate them. (b) Sunlight is the confounding variable: it differs between the groups and could affect height. (c) Chance should decide the groups, for example by drawing 10 names out of a hat. Then sunny and shady plants end up spread about evenly across both groups, so a clear difference in height afterward can reasonably be blamed on the plant food.
5. (a) Voluntary response; choose people at random instead of letting them choose themselves. (b) Undercoverage (renters are left out); choose at random from a list of all residents. (c) Leading question; ask it in neutral words.
(a) Viewers pick themselves, and people with strong opinions are the most likely to bother texting. That is voluntary response. A fix is to choose the people by random selection from a list that includes everyone. (b) Residents who rent do not own property, so they are not on the tax list and have no chance of being picked. That is undercoverage. A fix is to select at random from a list that includes all residents. (c) The words "how dangerous" and "don't you agree" push toward YES. That is a leading question. A fix is neutral wording, such as "Do you think texting while driving should be banned?"
6. (a) The 30–50 bar is five times as wide as each of the three younger groups' bars, so it takes up the most room. (b) 40 of 152 members, about 26%, which is not most. (c) Ages 22–26, with 9 members per year of age. Redraw with equal-width age groups or members per year.
(a) The age groups are not the same size: the three younger groups are 4 years wide, while 30–50 (and 50–70) are 20 years wide. The eye reads the whole area of a bar, so the wide bar seems to hold most of the members even though its height, 40, is only a little taller than the others. (b) The counts are 28 + 36 + 32 + 40 + 16 = 152 members, and 40 ÷ 152 ≈ 0.26, so only about 26% are between 30 and 50. (c) Divide each count by the width of its group: 28 ÷ 4 = 7, 36 ÷ 4 = 9, 32 ÷ 4 = 8, 40 ÷ 20 = 2 and 16 ÷ 20 = 0.8 members per year of age. The most crowded ages are 22–26. A fair graph uses equal-width age groups, or plots members per year of age, as in the answer graph.
NC Math 3 · NC.M3.S-IC.4 · S-IC.5
Simulation, Margin of Error, and Comparing Groups
Practice answer key
Name ____________________ Date ____________
1. 28 ± 3 minutes, so from 25 to 31 minutes.
The margin of error is about 2 standard deviations of the simulated sample means: 2 × 1.5 = 3 minutes. The estimate is the sample statistic plus or minus the margin: 28 ± 3, which gives 25 to 31 minutes.
2. 0.65 ± 0.07, so from 0.58 to 0.72.
The sample proportion is 130 ÷ 200 = 0.65. The margin of error is about 2 × 0.034 = 0.068, which rounds to 0.07. The estimate is 0.65 ± 0.07, so we estimate that between 58% and 72% of all students have a part-time job.
3. About 0.07, half as big.
The sample size is multiplied by 4 (50 × 4 = 200). A bigger sample gives a smaller margin of error, and a sample 4 times as large cuts the margin to about half: 0.14 ÷ 2 = 0.07. Doubling to 100 would not be enough to halve it.
4. No. The estimate is 0.54 ± 0.10, from 0.44 to 0.64, and that includes values below 0.50.
The margin of error is about 2 × 0.05 = 0.10, so the estimate for all voters is 0.54 ± 0.10, from 0.44 to 0.64. A majority means more than 0.50. Because the interval includes 0.50 and values below it, the true proportion could be a minority. The poll does not give enough evidence for the claim.
5. Only 12 of 1000 shuffles (1.2%) gave a difference as extreme as 6, so there is evidence that the tutoring made a real difference in scores.
Shuffling imitates the case where tutoring does nothing. In that case a difference of 6 points almost never happens: just 12 shuffles out of 1000. A difference that is reached this rarely is in the tail of the shuffled results. Chance alone is a poor explanation, so we conclude there is evidence that tutoring really raised scores. We say "evidence", not "proof", because 12 shuffles did reach it.
6. No. 0.4 is well inside 2 SD (1.38), so it is not evidence of a real difference.
Two SD of the shuffled differences is 2 × 0.69 = 1.38. The observed 0.4 is only about 0.6 SD from the center, deep in the middle of the shuffled results. In fact 673 of the 1000 shuffles gave a difference at least as far from 0 as 0.4. A gap this small happens all the time by chance, so the data do not give evidence that the study method matters. That does not prove the two methods are equal.
7. (a) They assume the fertilizer makes no difference, so the labels are just random tags. (b) "There is evidence that the fertilizer increased plant growth."
(a) Each shuffle deals the 20 growth values into two fake groups at random, which is exactly what would happen if the fertilizer did nothing. The shuffled differences show how much difference chance alone produces. (b) A result in the tail, 2.3 cm is about 2.5 SD out, with only 19 of 1000 shuffles as extreme, makes chance a poor explanation. That is evidence, not proof, so the right wording is "evidence that the fertilizer increased growth." Proof is too strong because chance could still, rarely, produce a gap like 2.3.
8. No. 1.9 is within 2 SD (2.0), so there is not enough evidence of a real difference, but that does not show music has no effect.
Two SD of the shuffled differences is 2 × 1.0 = 2.0. The observed 1.9 is just inside that, not in the tail, so chance alone could produce a gap this big fairly often. The right statement is "there is not enough evidence that music affects scores." Saying "music has no effect" claims too much: the groups were tiny, and a real effect of this size could be hidden by the large shuffle spread. A larger experiment might settle it.
NC Math 3 · NC.M3.S-IC.4 · S-IC.5
Simulation, Margin of Error, and Comparing Groups
Quiz answer key
Name ____________________ Date ____________
1. 7.4 ± 0.36 hours, so from 7.04 to 7.76 hours.
The margin of error comes from the spread of the simulated sample means, not from the spread of the 50 individual battery lives. Margin of error ≈ 2 × 0.18 = 0.36 hours. Using the 1.3 instead would give 2 × 1.3 = 2.6 hours, far too wide, because single laptops vary much more than the mean of 50 of them. The estimate is the sample mean plus or minus the margin: 7.4 ± 0.36, so from 7.04 to 7.76 hours.
2. (a) 0.36 ± 0.06, so from 0.30 to 0.42. (b) No: every value in the estimate is below 0.45.
(a) The sample proportion is 90 ÷ 250 = 0.36. The margin of error is about 2 × 0.030 = 0.06, so the estimate is 0.36 ± 0.06, from 0.30 to 0.42. We estimate that between 30% and 42% of all residents support the tax. (b) The claim says 0.45 or more, but the whole estimate stops at 0.42. The graph agrees: only 2 of the 1000 simulated samples came out at 0.45 or higher. A claim of "at least 40%" would be different, because 0.40 is inside the estimate and could not be ruled out.
3. (a) No. (b) About 600 people. (c) About 0.06.
(a) The margin of error does not shrink in step with the sample size. Doubling the sample only shrinks it by about 30%. (b) To cut the margin in half you need about 4 times as many people: 4 × 150 = 600, and 0.08 ÷ 2 = 0.04. (c) For 300 people the margin is about 30% smaller than 0.08: 0.08 × 0.70 = 0.056, which is about 0.06. Simulations of 1000 samples agree: sample sizes 150, 300 and 600 give margins of error of about 0.08, 0.06 and 0.04.
4. (a) 40.0 − 45.0 = −5.0 seconds. (b) Yes: 2 SD = 3.84 and −5.0 is farther from 0 than that. (c) There is evidence that the hint sheet shortened solving time.
(a) Group A's mean is 320 ÷ 8 = 40.0 and group B's mean is 360 ÷ 8 = 45.0, so A − B = −5.0. The minus sign only says that A was faster. (b) 2 SD = 2 × 1.92 = 3.84. The observed difference is 5.0 away from 0, farther than 3.84, about 2.6 SD out. A difference on the left side of the graph counts just like one on the right. Only 5 of the 1000 shuffles gave a difference as far from 0 as 5.0 in either direction. (c) Chance alone is a poor explanation, so there is evidence that the hint sheet shortened solving time. It is evidence, not proof.
5. (a) 14.0 − 13.0 = 1.0 cm. (b) 13.6 − 13.4 = 0.2 cm. (c) No: 424 of 1000 shuffles (42%) were that extreme, so there is not enough evidence.
(a) Blue: 70 ÷ 5 = 14.0. White: 65 ÷ 5 = 13.0. The difference is 1.0 cm. (b) The fake blue group has mean 68 ÷ 5 = 13.6. The other five heights are 12, 13, 13, 14, 15, with mean 67 ÷ 5 = 13.4. This shuffle's difference is 0.2 cm, much smaller than 1.0, just by luck of the deal. (c) Shuffling imitates light color making no difference. Under that assumption, 424 of 1000 shuffles (42%) gave a gap as big as 1.0, so a gap like that happens very often by chance. The data do not give enough evidence that light color matters. That does not show the colors are the same, because 5 seedlings per group is small.
6. (a) 0.75 − 0.35 = 0.40. (b) 2 SD = 0.32, and 0.40 is beyond it. (c) There is evidence that the new exercise program raised the recovery rate.
(a) Group 1: 15 ÷ 20 = 0.75. Group 2: 7 ÷ 20 = 0.35. The difference is 0.75 − 0.35 = 0.40. (b) 2 SD = 2 × 0.16 = 0.32. The observed 0.40 is beyond that, about 2.5 SD from the center. (c) Each shuffle imitates the program making no difference, with 22 of the 40 patients recovering whichever group they land in. Only 24 of 1000 shuffles (2.4%) gave a gap as far from 0 as 0.40, so chance alone is a poor explanation. We say there is evidence that the program raised the recovery rate, not that it is proven.