A Guide to Statistical Tests: Chi-Square, t-Test, and Z-Test

In population and family health studies, raw data alone rarely tells a complete story. Whether you’re examining vaccination coverage, comparing birth weights across regions, or studying healthcare utilization patterns, you need tools that can separate genuine differences from random fluctuations. This is where statistical tests come in. They help researchers move from “this group looks different” to “this group is significantly different, and here’s the evidence.” The three workhorses you’ll encounter most often are the Chi-square test, the t-test, and the Z-test, each suited to a different type of question and data.

Why statistical tests matter in health research 🔗

Health researchers rarely study entire populations. Instead, they collect data from samples and use that information to draw conclusions about broader groups. The problem is that any two samples will show some differences, even if they come from identical populations, simply because of random variation. Statistical tests are formal procedures that help us decide whether an observed difference is large enough to be considered real or whether it could plausibly have arisen by chance.

Every test follows the same basic logic. You begin with a null hypothesis (usually stating “there is no difference” or “there is no association”) and an alternative hypothesis that contradicts it. You then calculate a test statistic from your data and compare it against a known distribution to obtain a p-value. By convention, a p-value below 0.05 leads researchers to reject the null hypothesis and conclude that the observed effect is unlikely to be due to chance.

The choice of test depends on two things: the type of data you have collected and what you’re trying to compare. Categorical data (like male/female, vaccinated/unvaccinated, urban/rural) calls for one family of tests, while continuous data (like blood pressure, weight, or income) calls for another.

<[Image: A flowchart showing how researchers choose between Chi-square, t-test, and Z-test based on data type and sample size]

The Chi-square test: comparing categorical data 🔗

The Chi-square (χ²) test, developed by Karl Pearson, is one of the most widely used nonparametric tools for analyzing categorical data. It does not assume that your data follows a normal distribution, which makes it especially useful for survey responses, demographic groupings, and any situation where outcomes are counted rather than measured on a continuous scale.

The test works by comparing the frequencies you actually observed in your data with the frequencies you would expect if there were no real association between the variables. A large gap between observed and expected counts produces a large Chi-square statistic, which signals that the variables are probably linked.

Types of Chi-square tests 🔗

There are three main variants of the Chi-square test that researchers should know. The Goodness-of-fit test checks whether the distribution of a single categorical variable matches a predefined pattern. The Test of independence examines whether two categorical variables collected from one sample are related to each other. The Test of homogeneity compares the distribution of a categorical variable across several populations to see whether they share a similar pattern, as explained in this overview of Chi-square applications in healthcare.

A healthcare utilization example 🔗

Consider a study on preventive health check-ups in a rural district of Tamil Nadu. Researchers wanted to know whether utilization differed by age, education, occupation, and vaccination status. Using the Chi-square test, they found that younger participants, those with stable occupations, and those with higher education were significantly more likely to undergo preventive health check-ups. The variables were categorical (age groups, education levels, employment status), and the outcome was binary (had a check-up or not). Chi-square was the natural choice.

Another illustration comes from a nationwide survey on cancer awareness, where Pearson’s Chi-square test was used to identify links between demographic factors and awareness of cancer screening among the general population. These kinds of associations are critical for designing targeted health campaigns.

Assumptions and cautions 🔗

For Chi-square results to be reliable, a few conditions must be met. Observations should be independent, categories must be mutually exclusive, and expected frequencies in each cell of the contingency table should generally be five or more. When expected counts fall below this threshold, researchers often turn to Fisher’s exact test instead. Random sampling and adequate sample size are also essential for valid conclusions.

The t-test: working with small samples 🔗

When your data is continuous and you want to compare means rather than proportions, the Chi-square test no longer applies. Instead, you need a test built for averages. The t-test is the go-to choice when your sample size is small (typically n ≤ 30) and the population standard deviation is unknown, which describes most real-world research situations.

The t-test uses the t-distribution, which resembles the normal distribution but has heavier tails. These heavier tails account for the extra uncertainty that comes with estimating variability from a small sample rather than knowing it from the whole population.

Three types of t-tests 🔗

The one-sample t-test compares the mean of a single sample to a known or hypothesized value. For example, you might check whether the average haemoglobin level of pregnant women in a clinic differs from the national average. The independent two-sample t-test compares the means of two unrelated groups, such as the average birth weight of babies born to mothers receiving iron supplementation versus those who did not. The paired t-test compares two measurements taken from the same individuals, such as blood pressure before and after a lifestyle intervention.

A practical scenario 🔗

Suppose a researcher wants to evaluate whether a new community-based nutrition programme improves the body mass index (BMI) of adolescent girls. They recruit 20 participants, measure their BMI before the programme, and measure it again after six months. Because the same individuals are measured twice and the sample is small, a paired t-test is appropriate. The test will reveal whether the average change in BMI is large enough to be considered a real effect rather than random variation.

Similarly, when testing the effectiveness of a new drug for reducing blood pressure on 15 patients, the t-test handles the small sample size and unknown population variance gracefully, as demonstrated in this comparison of Z-test and t-test scenarios.

The Z-test: large samples and known parameters 🔗

The Z-test is the older sibling of the t-test. It is used when you have a large sample size (n ≥ 30) and the population variance is known, allowing the use of the standard normal distribution to evaluate the test statistic. In population health research, Z-tests are common when working with national surveys, census data, or any setting where historical estimates of variability are available.

Types and applications 🔗

Like the t-test, the Z-test has variants. A one-sample Z-test compares a sample mean to a known population mean, useful when checking whether the average household income in a particular district differs from a national benchmark. A two-sample Z-test compares the means of two large independent samples, ideal for comparing average ages of first childbirth between two states using NFHS data.

When dealing with proportions in very large samples, a Z-test for proportions is also widely used. For example, comparing the proportion of fully immunised children between two regions in the National Family Health Survey, which collected information from over 636,000 households across the country in its fifth round, would lend itself to a Z-test because the sample sizes are enormous.

Why sample size matters 🔗

The Central Limit Theorem tells us that as sample size grows, the distribution of sample means approaches a normal distribution, regardless of the underlying population distribution. This is why the Z-test becomes reliable for large samples. With small samples, this approximation breaks down, and the t-test’s heavier-tailed distribution provides a more honest reflection of the uncertainty involved.

Choosing the right test 🔗

The decision tree for picking among these three tests is straightforward once you know what to look for. Start by identifying the nature of your data. If you are working with counts or frequencies in categories, the Chi-square test is the answer. If you are working with means of continuous data, ask two follow-up questions: is the sample size large or small, and do you know the population standard deviation?

For small samples (n less than 30) with an unknown population variance, use the t-test. For large samples (n greater than or equal to 30) where the population variance is known or can be reliably estimated, the Z-test is appropriate. In practice, when sample sizes are very large, the t-distribution and the Z-distribution become almost identical, so the choice between them matters less. However, statistical software typically defaults to the t-test when the population variance is not given, which is the safer assumption in most real-world research.

Combining tests in real studies 🔗

Most published health research uses more than one test. A study on preventive check-ups in Tamil Nadu used Chi-square for categorical comparisons and independent t-tests for assessing motivators and barriers. This layered approach allows researchers to tell a complete story, moving from “who uses these services” to “what do they think about them.”

Common pitfalls to avoid 🔗

Statistical tests are powerful but easy to misuse. A p-value below 0.05 does not prove that an effect is large or important; it only suggests the result is unlikely to be due to chance. With very large samples, even tiny differences can become statistically significant without being practically meaningful. Conversely, with small samples, real effects can be missed because the test lacks power to detect them.

Another common mistake is choosing the wrong test for the data. Applying a Chi-square test to continuous variables, or a t-test to clearly non-normal data with a small sample, can lead to misleading conclusions. Always check the assumptions of any test before reporting the results, and consider non-parametric alternatives like the Mann-Whitney U test or Fisher’s exact test when assumptions are violated.

What do you think? If you were designing a study to compare contraceptive use between two districts, which of these three tests would you reach for first, and what additional information would you want before running it? And when does a “statistically significant” result genuinely matter for public health policy versus when does it just look impressive on paper?

References

1. https://journals.lww.com/jpcs/fulltext/2015/01010/chi_square_test_and_its_application_in_hypothesis.17.aspx
2. https://www.journalijar.com/article/50832/chi-square-tests-a-quick-guide-for-health-researchers/
3. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10874466/
4. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11953554/
5. https://www.geeksforgeeks.org/data-science/t-test-vs-z-test/
6. https://www.geeksforgeeks.org/data-science/z-test-vs-t-test/
7. https://www.datacamp.com/tutorial/t-test-vs-z-test
8. https://pmc.ncbi.nlm.nih.gov/articles/PMC10657051/