- Home
- News
- Install
- Data
- Design
- Basics
- Probability
- Probability calculator
- Central Limit Theorem
- Means
- Single mean
- Compare means
- Proportions
- Single proportion
- Compare proportions
- Tables
- Goodness of fit
- Cross-tabs
- Correlation
- Model
- Estimate
- Linear regression (OLS)
- Logistic regression (GLM)
- Multinomial logistic regression (MNL)
- Neural Network
- Naive Bayes
- Trees
- Classification and regression trees
- Random Forest
- Gradient Boosted Trees
- Evaluate
- Evaluate regression
- Evaluate classification
- Recommend
- Collaborative Filtering
- Decide
- Decision analysis
- Simulate
- Multivariate
- Maps
- (Dis)similarity
- Attributes
- Factor
- Pre-factor
- Factor
- Cluster
- Hierarchical
- K-clustering
- Conjoint
- Conjoint
- R
- Tutorials
- About
Compare the means of two or more variables or groups in the data
The compare means t-test is used to compare the mean of a variable inone group to the mean of the same variable in one, or more, othergroups. The null hypothesis for the difference between the groups in thepopulation is set to zero. We test this hypothesis using sampledata.
We can perform either a one-tailed test (i.e., less thanor greater than) or a two-tailed test (see the ‘Alternativehypothesis’ dropdown). We use one-tailed tests to evaluate if theavailable data provide evidence that the difference in sample meansbetween groups is less than (or greater than ) zero.
Example: Professor salaries
We have access to the nine-month academic salary for AssistantProfessors, Associate Professors and Professors in a college in the U.S(2008-09). The data were collected as part of an on-going effort by thecollege’s administration to monitor salary differences between male andfemale faculty members. The data has 397 observations and the following6 variables.
- rank = a factor with levels AsstProf, AssocProf, and Prof
- discipline = a factor with levels A (“theoretical” departments) or B(“applied” departments)
- yrs.since.phd = years since PhD
- yrs.service = years of service
- sex = a factor with levels Female and Male
- salary = nine-month salary, in dollars
The data are part of the CAR package and are linked to the book: FoxJ. and Weisberg, S. (2011) An R Companion to Applied Regression, SecondEdition Sage.
Suppose we want to test if professors of lower rank earn lowersalaries compared to those of higher rank. To test this hypothesis wefirst select professor rank and select salaryas the numerical variable to compare across ranks. In theChoose combinations box select all available entries toconduct pair-wise comparisons across the three levels. Note thatremoving all entries will automatically select all combinations. We areinterested in a one-sided hypothesis (i.e., less than).
The first two blocks of output show basic information about the test(e.g., selected variables and confidence levels) and summary statistics(e.g., mean, standard deviation, margin or error, etc. per group). Thefinal block of output shows the following:
Null hyp.is the null hypothesis andAlt. hyp.the alternative hypothesisdiffis the difference between the sample means for twogroups (e.g., 80775.99 - 93876.44 = -13100.45). If the null hypothesisis true we expect this difference to be small (i.e., close to zero)p.valueis the probability of finding a value asextreme or more extreme thandiffif the null hypothesis istrue
If we check Show additional statistics the followingoutput is added:
Pairwise mean comparisons (t-test)Data : salary Variables : rank, salary Samples : independent Confidence: 0.95 Adjustment: None rank mean n n_missing sd se me AsstProf 80,775.985 67 0 8,174.113 998.627 1,993.823 AssocProf 93,876.438 64 0 13,831.700 1,728.962 3,455.056 Prof 126,772.109 266 0 27,718.675 1,699.541 3,346.322 Null hyp. Alt. hyp. diff p.value se t.value df 0% 95% AsstProf = AssocProf AsstProf < AssocProf -13100.45 < .001 1996.639 -6.561 101.286 -Inf -9785.958 *** AsstProf = Prof AsstProf < Prof -45996.12 < .001 1971.217 -23.334 324.340 -Inf -42744.474 *** AssocProf = Prof AssocProf < Prof -32895.67 < .001 2424.407 -13.569 199.325 -Inf -28889.256 ***Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
seis the standard error (i.e., the standard deviationof the sampling distribution ofdiff)t.valueis the t statistic associated withdiffthat we can compare to a t-distribution (i.e.,diff/se)dfis the degrees of freedom associated with thestatistical test. Note that the Welch approximation is used for thedegrees of freedom0% 95%show the 95% confidence interval around thedifference in sample means. These numbers provide a range within whichthe true population difference is likely to fall
Testing
There are three approaches we can use to evaluate the nullhypothesis. We will choose a significance level of 0.05.1 Ofcourse, each approach will lead to the same conclusion.
p.value
Because each of the p.values is smaller than thesignificance level we reject the null hypothesis for each evaluated pairof professor ranks. The data suggest that associate professors make morethan assistant professors and professors make more than assistant andassociate professors. Note also the ’***’ that are used as an indicatorfor significance.
confidence interval
Because zero is not contained in any of theconfidence intervals we reject the null hypothesis for each evaluatedcombination of ranks. Because our alternative hypothesis isLess than the confidence interval is actually an upperbound for the difference in salaries in the population at a 95%confidence level (i.e., -9785.958, -42744.474, and -28889.256)
t.value
Because the calculated t.values (-6.561, -23.334, and -13.569) aresmaller than the corresponding criticalt.value we reject the null hypothesis for each evaluated combination ofranks. We can obtain the critical t.value by using the probabilitycalculator in the Basics menu. Using the test for assistantversus associate professors as an example, we find that for at-distribution with 101.286 degrees of freedom (see df) thecritical t.value is 1.66. We choose 0.05 as the lower probability boundbecause the alternative hypothesis is Less than.
In addition to the numerical output provided in the Summarytab we can also investigate the association between rankand salary visually (see the Plot tab). The screenshot below shows a scatter plot of professor salaries and a bar chartwith confidence interval (black) and standard error (blue) bars.Consistent with the results shown in the Summary tab there isclear separation between the salaries across ranks. We could also chooseto plot the sample data as a box plot or as a set of density curves.
Multiple comparison adjustment
The more comparisons we evaluate the more likely we are to find a“significant” result just by chance even if the null hypothesis is true.If we conduct 100 tests and set our significance levelat 0.05 (or 5%) we can expect to find 5 p.values smaller than or equalto 0.05 even if the are no associations in the population.
Bonferroni adjustment ensures the p.values are scaled appropriatelygiven the number of tests conducted.This XKCD cartoonexpresses the need for this type of adjustments very clearly.
Stats speak
This is a comparison of means test of the nullhypothesis that the true population difference in meansis equal to 0. Using a significance level of 0.05, wereject the null hypothesis for each pair of ranks evaluated, andconclude that the true population difference in meansis less than 0.
The p.value for the test of differences in salaries between assistantand associate professors is < .001. This is theprobability of observing a sample difference in meansthat is as or more extreme than the sample difference inmeans from the data if the null hypothesis is true. In thiscase, it is the probability of observing a sample difference inmeans that is less than (or equal to)-13100.45 if the true population difference inmeans is 0.
The 95% confidence bound is -9785.958. If repeatedsamples were taken and the 95% confidence bound computed for each one,the true population mean would be below the lower bound in 95% of thesamples
1 The significance level, often denotedby \(\alpha\), is the highestprobability you are willing to accept of rejecting the null hypothesiswhen it is actually true. A commonly used significance level is 0.05 (or5%)
Report > Rmd
Add code toReport> Rmd to (re)create the analysis by clicking the icon on the bottomleft of your screen or by pressing ALT-enter on yourkeyboard.
If a plot was created it can be customized using ggplot2commands (e.g.,plot(result, plots = "scatter", custom = TRUE) + labs(title = "Compare means")).SeeData> Visualize for details.
R-functions
For an overview of related R-functions used by Radiant to evaluatemeans seeBasics> Means
The key function from the stats package used in thecompare_means tool is t.test.
Video Tutorials
Copy-and-paste the full command below into the RStudio console (i.e.,the bottom-left window) and press return to gain access to all materialsused in the hypothesis testing module of theRadiantTutorial Series:
usethis::use_course("https://www.dropbox.com/sh/0xvhyolgcvox685/AADSppNSIocrJS-BqZXhD1Kna?dl=1")- This video shows how to conduct a compare means hypothesis test
- Topics List:
- Calculate summary statistics by groups
- Setup a hypothesis test for compare means in Radiant
- Use the p.value and confidence interval to evaluate the hypothesistest
© Vincent Nijs (2019) 
