Methods for Two Categorical Variables¶

Statistical Inference: Conducting a Simulation Study with Playing Cards¶

The above descriptive analysis tells us what we have learned about the 50 subjects in the study. Our next goal is to make inferences beyond what happened in the study (i.e., we want to make general statements about the population). Does the higher proportion of yawners in the seeded group provide convincing evidence that being seeded with a yawn actually makes a person more likely to yawn? Note that it is possible that random chance alone could have led to this large of a difference. That is, while it is possible that the yawn seeding had no effect and the MythBusters happened to observe more yawners in the seeded group just by chance, the key question is whether it is probable.

We will answer this question by replicating the experiment over and over again, but in a situation where we know that yawn seeding has no effect (the null model). We’ll start with 14 yawners and 36 non-yawners, and we’ll randomly assign 34 of these 50 subjects to the seeded group and the remaining 16 to the non-seeded group.

Note that we could use playing cards to replicate this experiment: let 14 red cards represent the yawners, and let 36 black cards represent the non-yawners. Shuffle the cards well, and randomly deal out 34 to be the seeded group. This is the first run of our simulation study. Construct the contingency table to show the number of yawners and non-yawners in each group from the first run of our simulation study.

Next, note that if you know the number of yawners in the Seeded group, then you can fill in the rest of the cells in the contingency table. So, for the rest of the simulated runs, we need only keep track of the number of yawners in the Seeded group. Record the number of yawners in the Seeded group from your first simulated experiment in the table below. Then, repeat this randomization process nine more times, recording your results in the table.

Next, create a dotplot of the results:

Number of Yawners Randomly Assigned to the Seeded Group

Statistical Inference: Conducting a Simulation Study with Tinkerplots¶

Note that we could use technology to simulate this process much more efficiently. This will enable us to easily obtain many more simulated results so that we can be more confident in our answer to Question 3 above.

Open the file YawnExperiment.tp from the course website. Note that each mixer contains 50 “subjects.” The first mixer contains 14 yawners and 36 non-yawners; the second mixer contains 34 subjects in the Seeded group and 16 in the Non-seeded group.

Make sure the Repeat value is set to 50 (the number of subjects in the study), and set BOTH mixers to sample Without Replacement (as shown below).

Click Run, and you will see the results of the first simulated trial:

To plot these results, drag a new Plot to the workspace. Drag the predictor variable (Group) to the x-axis and the response variable (Yawn) to the y-axis, as shown below.

Click on n to count the number in each cell, and you can also vertically stack the dots using the Stack tool.

Note that we are interested in recording the number of yawners in the Seeded group. Place your cursor over this number (circled in red on the below graph), right-click, and choose Collect Statistic. Set the Collect value to 99 in order to obtain a total of 100 simulated results, and click Collect.

Finally, to graph these outcomes, highlight the “count_Yawn_yes_Group_Seeded” column and drag a new plot into the workspace. Drag a point all the way to the right to organize the dots, and use the Stack tool to vertically stack the dots. Finally, right-click on either endpoint on the x-axis and set “Bin Width” to 1, and use the count tool (n) to count the dots in each bin.

You should see something similar to the following:

Carrying out Fisher’s Exact Test¶

This test is based on the probability of observing data at least as extreme as the actual observed data. The procedure is carried out as follows.

Example 3.2: Vested Interest and Task Performance¶

This example is from Investigating Statistical Concepts, Applications, and Methods by Beth Chance and Allan Rossman. 2006. Thomson-Brooks/Cole.*

“A study published in the Journal of Personality and Social Psychology (Butler and Baumeister, 1998) investigated a conjecture that having an observer with a vested interest would decrease subjects’ performance on a skill-based task. Subjects were given time to practice playing a video game that required them to navigate an obstacle course as quickly as possible. They were then told to play the game one final time with an observer present. Subjects were randomly assigned to one of two groups. One group (A) was told that the participant and observer would each win $3 if the participant beat a certain threshold time, and the other group (B) was told only that the participant would win the prize if the threshold were beaten. The threshold was chosen to be a time that they beat in 30% of their practice turns. The following results are very similar to those found in the experiment: 3 of the 12 subjects in group A beat the threshold, and 8 of 12 subjects in group B achieved success.”

The data can be entered into JMP as follows:

To create the contingency table and the mosaic plot in JMP, select Analyze > Fit Y by X. As always, place the response variable in the Y, Response box and the explanatory variable in the X, Factor box.

Click OK, and JMP returns the following:

Keep in mind that a p-value is always the probability of seeing results at least as extreme as the observed data. So, JMP uses what is known as the hypergeometric distribution to find the probability of seeing the observed data table and EACH table that is more extreme, assuming that there is no difference in the two groups being compared. The sum of these probabilities is the p-value from Fisher’s exact test.

Example 3.3: Claritin and Nervousness¶

An advertisement by the Schering Corporation in 1999 for the allergy drug Claritin mentioned that in a pediatric randomized clinical trial, symptoms of nervousness were shown by 4 of 188 patients on Claritin and 2 of 262 patients taking a placebo.

The data can be found in the file Claritin.JMP.

Next, use JMP to carry out Fisher’s Exact Test for these data.

Observational Studies vs. Designed Experiments¶

Reconsider the “Vested Interest and Task Performance” example. Fisher’s exact test provided evidence that the proportion of successes was in fact smaller for the vested interest group (p-value = .0498). Now, the question is this: can we conclude that having a vested interest really is the cause of the decreased performance?

The answer to this question lies in whether the experiment itself was a designed experiment or an observational study.

Observational Study

An *observational study* involves collecting and analyzing data without randomly assigning treatments to experimental units.

Designed Experiment

On the other hand, in a *designed experiment*, a treatment is randomly imposed on individual subjects in order to observe whether the treatment causes a change in the response.

Note that the “Vested Interest and Task Performance” study is an example of a designed experiment since participants were randomly assigned to the two groups. We were trying to show that having a vested interest caused a decreased task performance. The small p-value rules out observing the decreased performance in the vested interest group simply by chance, and the randomization of subjects to treatment groups should have balanced out any other factors that might explain the difference. So, the most likely explanation left is that having a vested interest really is the cause of the decreased task performance.

Example 3.4: Alcoholism and Depression¶

Past research has suggested a high rate of alcoholism in families of patients with primary unipolar depression. A study of 210 families of females with primary unipolar depression found that 89 had alcoholism present. A set of 299 control families found 94 present.

The data can be found in the file Depression.JMP:

To analyze these data, choose Analyze > Fit Y by X.

The JMP output is shown below:

Find the p-value:¶

Recall that the Chi-Square test uses a distribution known as the chi-square (χ:sup:2) distribution. The chi-square distribution takes on only positive numbers and is a continuous distribution. In addition, this distribution is indexed by its degrees of freedom (or df). For this test, this is given by df = (r - 1)(c - 1). When the null hypothesis is true, the test-statistic follows the chi-square distribution with df = (r - 1)(c - 1). .

The following graph shows the chi-square distribution with df = 1. The p-value is found by plotting the chi-square test statistic on the x-axis and calculating the area under the curve above the test statistic.

Note that the output for the chi-square test automatically appears when you select Analyze > Fit Y by X.

Example 3.5: Mice and Tobacco Smoke¶

Ten mice (6–8 weeks old) were randomly assigned to one of two groups; five were exposed to simulated environmental tobacco smoke for 6 h/day, 5 days/week for 5 months. The other 5 mice were kept in clean air during this time period. Then, all of the mice were allowed to recover for a further 4 months in filtered air before being killed for analysis of lung tumor incidence. The results are shown below.

Step 1: Convert the Research Hypothesis into H:sub:`o` and H:sub:`a`.

H₀: The proportion of mice that develop a lung tumor when exposed to tobacco does not differ from the proportion that develop a lung tumor when not exposed to tobacco.

H_a: The proportion of mice that develop a lung tumor differs between those exposed to tobacco smoke and those not exposed.

Equivalently, we can state the hypotheses as follows:

H₀:

H_a:

Step 2: Find the p-value for the data.

p-value from Fisher’s exact test: _____________

p-value from Chi-Square test: _____________

Step 3: Write a conclusion addressing the original research hypothesis.

Example 3.6: Gender and Health Insurance¶

In 2010, the U.S. Congress passed the historic health care reform bill that will provide some type of coverage for the 32 million Americans currently without health care insurance. Prior to this, several studies were carried out to investigate how widespread the lack of medical coverage really was and to determine which segments of the population were most at risk. One such study compared the proportion of uninsured across gender using data from the U.S. Census Bureau Current Population Survey, and the results are shown below.

Example 3.7: Adoption and Suicide Attempts¶

In September of 2013, researchers from the University of Minnesota published a study in the journal Pediatrics. This study was described in a Fox News article titled “Adopted teens may be at higher risk of suicide.” This article described the results as follows:

[The researchers] examined data from an existing University of Minnesota study of 692 adopted children and 540 non-adopted siblings in Minnesota… All of the adopted kids, who were between 11 and 21 years old during the study period, had been taken in by their families before age two. Almost three quarters of the adopted children were born abroad, most of the foreign-born children were from South Korea and 60 percent of those were girls. At the beginning of the study, and again about three years later, the researchers asked participating families if the children had made a suicide attempt. Over the three years of the study, 56 children attempted suicide at least once, according to the family members’ reports. Of those kids, 47 were adopted and nine were not adopted.

Example 3.8: Smoking and Lung Cancer¶

The data in the contingency table shown below are the results of a case-control study published back in 1950 (this was one of the first studies linking smoking to lung cancer). Note that in this study, the researchers found one control subject for every case of lung cancer in their study.

Now, suppose that the researchers had found two control subjects for every case and that the results were as shown below.

Example 3.9: When is the Relative Risk Close to the Odds Ratio?¶

Consider the following hypothetical example comparing survival rates between two groups.

Now, suppose the results were as shown below, instead.

Example 3.10: Importance of Considering Absolute Risk¶

In 1995, the UK Committee on Safety of Medicines issued a warning that birth control pills increased the risk of life-threating blood clots. The data on which these conclusions were drawn were very similar to the following outcomes.

Example 3.11: Support for Iraq War and Political Affiliation¶

In March of 2003, the Pew Internet & American Life Project commissioned Princeton Survey Research Associates to develop and carry out a survey of what Americans thought about the recent war in Iraq. Some of the results of this survey of over 1,400 American adults are given in the JMP data file IraqWar.JMP. (Source: McClave & Sincich, Problem 13.33) Responses to the following questions were recorded:

1. Do you support or oppose the Iraq War?
2. Do you ever go online to access the Internet or World Wide Web?
3. Do you consider yourself a Republican, Democrat, or Independent?
4. In general, would you describe your political views as very conservative, conservative, moderate, liberal, or very liberal?
5. What is your race?
6. Do you live in a suburban, rural, or urban community?

Note that this investigation requires us to focus on only two of the measured variables: Support for the Iraq War and Political Affiliation. First, let’s summarize the data using JMP. Select Analyze > Fit Y by X and enter the following:

JMP returns the contingency table and mosaic plot:

Carrying out the chi-square test of independence:¶

Step 1: Convert the research question into H_0 and H_a:

H_o:

H_a:

Step 2: Calculate a test statistic and p-value from the data.

Observed counts:

Finding the expected counts:

Once we have the expected counts, we can calculate the test statistic as follows:

Finding the p-value:¶

The Chi-Square test uses a distribution known as the chi-square \chi^2 distribution. The chi-square distribution takes on only positive numbers and is a continuous distribution. In addition, this distribution is indexed by its degrees of freedom (or df). For this test, this is given by

You can use the following function in EXCEL to find the p-value: =CHIDIST(test statistic, df).

The following graph shows the chi-square distribution with df = 2. The p-value is found by plotting the chi-square test statistic on the x-axis and calculating the area under the curve above the test statistic.

The output for the chi-square test is also given in the JMP output:

Step 3: Write a conclusion in the context of the problem

Finally, since we have determined that there is a significant association between support for the Iraq war and political affiliation, we can describe this relationship based on the patterns displayed in the mosaic plot.

Example 3.12: Opinions on Disciplining Children across Regions of U.S.¶

On September 16-17, 2014, an NBC News/Marist Poll surveyed a random sample of 606 adults nationwide. Respondents were asked the following question: “Do you think it is right or wrong for parents to discipline their children by striking them - either with a paddle, switch, or belt?”

The results were separated according to the Region of the U.S. in which the respondent lived (Northeast, Midwest, South, or West). The data are summarized in the following table.

First, since we don’t have access to the raw data, we can enter the data from the contingency table into JMP as follows (see the file Discipline.jmp posted on the course website).

We can then use the Analyze > Fit Y by X menu to obtain the desired output:

Step 1: Convert the research question into H_0 and H_a:

H_o:

H_a:

Step 2: Find the test statistic and p-value from the data.

Step 3: Write a conclusion in the context of the problem.

Finally, note that if we determine a significant relationship exists between the two variables of interest, we should take the time to describe this relationship.

Also, we can look at each cell’s contribution to the test statistic to informally determine what is driving the significance of this relationship: