Applying a Level of Significance
In this activity, you’ll study the data from two experiments. For each data set, you will determine whether the difference of means between the treatment group and the control group is statistically significant and interpret the result in the context of the situation.

Question 1
Fitness experts advise people who want to lose weight to drink protein shakes after workout sessions, because it helps build muscle mass and speeds up the weight-loss process. To test the effectiveness of protein shakes in losing weight, 200 adults between the ages of 25 and 40 years were divided equally into two groups: a treatment group and a control group. The treatment group was asked to drink a protein shake after every workout session for a period of 6 months. The subjects’ weights were noted before and after the treatment.

The results showed that the mean change in the amount of weight lost by the treatment group is 8 points more than that of the control group. To test whether the results could be explained by random chance, researchers created a table that summarizes the results of 1,000 re-randomizations of the data with differences of means rounded to the nearest 2 points.

Consider the significance level to be set at 5%, so results less than 5% can be considered statistically significant.

Treatment Group Mean − Control Group Mean Frequency
-12 1
-10 10
-8 28
-6 58
-4 125
-2 184
0 201
2 186
4 114
6 57
8 26
10 8
12 2
Part A
Determine the probability of the treatment group’s mean being greater than the control group’s mean by 8 points or more.

Font Sizes

Characters used: 0 / 15000
Part B
Is the result statistically significant? Explain your answer.

Font Sizes

Characters used: 0 / 15000
Part C
What does your answer mean in the context of drinking protein shakes while on a weight-loss program?

Font Sizes

Characters used: 0 / 15000
Question 2
According to a research study, people are able to better manage their time when they wear watches than when they use mobile phones or other gadgets to keep track of time.

To test this, 300 college students were randomized into a treatment group and a control group. The treatment group was asked to wear a watch every day, and the control group was asked to avoid wearing a watch for 50 days. The number of occasions when students from both groups arrived later than their scheduled time was noted.

The results of the experiment showed that the mean score of the treatment group was exactly 15 points less than the mean score of the control group. To test whether the results could be explained by random chance, the researchers created the following table, which summarizes the results of 1,000 randomizations of the data with differences of means rounded to the nearest 5 points.

Consider the significance level to be set at 5%, so results lower than 5% can be considered statistically significant.

Treatment Group Mean − Control Group Mean Frequency
-25 10
-20 18
-15 50
-10 127
-5 195
0 226
5 180
10 117
15 48
20 17
25 12
Part A
Determine the probability of the treatment group’s mean being lower than the control group’s mean by 15 points or more.

Font Sizes

Characters used: 0 / 15000
Part B
Is the result statistically significant? Explain your answer.

Font Sizes

Characters used: 0 / 15000
Part C
What does your answer suggest about wearing a watch and managing time?

Font Sizes