Goals

After reading this article, you will be able to do the following:

• Understand how hypothesis testing is used to build and test scientific theories.
• Apply questions about the world using hypothesis testing.
• Evaluate hypothesis testing’s strengths and weaknesses.

Introduction

In this article we will cover the most common approach to building theories in the sciences: the hypothesis test. While this approach has variations, depending on specific disciplines and philosophical foundations, you will learn how it can be applied to social science methods. It is not the only approach used in the sciences to test our ideas, but it is the most widely used.

The Role of Hypotheses in Research

Hypothesis means below or under a theory. It is a way to test your ideas against the world, to try and disprove them while building up data to support a theory. It is also the most commonly used approach in testing science ideas, but by no means the only method. While the hypotheses can certainly be used in qualitative research, their origin and most common use is in what we consider quantitative research (Bhattacherjee et al, n.d.) Even there, other approaches do exist.

Origins of Hypothesis Testing

First, we will discuss this approach’s origins and how it evolved over time. Evidence of early scientific approaches has existed throughout written history, and a component has always been to record variables. Variables are simply the environmental aspects that you are trying to measure, such as behavior, temperature, light, or emotions. When you use numbers to measure variables, you are using a quantitative method. Think quantity, like asking how many bananas I have in my boat. As opposed to qualitative research, think quality, like me interviewing you about your experience eating one of these bananas.

Another important aspect and benefit of using hypotheses in research is to define how you measure your variables; this is called operational definition. An operational definition is a description of the variable you are studying, based on how you are measuring it (Price et al., 2015.) If I must count bananas, it seems straightforward to count them one by one. But if we think about it, that would be tedious and time-consuming. So, we might take a different approach depending on the available resources and ways to measure. If you have access to a large scale, you could estimate your banana numbers by measuring by the pound. If you didn’t have a scale, you could still make your task more reasonable by counting banana bunches or boatloads instead of one by one.

Let’s say we do have a scale. Our banana-yield operational definition would look something like, “banana-crop yields were estimated based on pounds per boat.” Worrying so much about how we define our variables seems almost as tedious as counting bananas, but it’s part of science’s recipe—how we ask our questions will affect our answers. Plus, your peer scientists need to understand your approach and how it compares to theirs.

Hypothesis testing originated in quantitative approaches, specifically in applying statistics to agriculture in the late nineteenth to early twentieth century. Statisticians like Karl Pearson and R.A. Fisher sought to quantify and predict how best to grow crops (not bananas but let’s pretend). To do this, the early scientist made an intervention, say, trying a new fertilizer, on a crop and predicted that it would work better than what was used in the past (Salsburg, 2001).

Prediction is the hypothesis’ key component, which at the most basic definition is a testable prediction. Remember that scientific theories are built on hypotheses. A good theory must both explain and predict outcomes. In our fertilizer example, the farmer explains how the banana crops grow because of such and such nutrients and predicts the fertilizer’s effectiveness.

But it might not. Maybe the farmer was way off base and actually just killed all the bananas with the experimental fertilizer. While that’s bad for your stomach and wallet, it’s good for your hypothesis. Why? Because it was falsifiable. Falsifiability is a relatively new concept, but it’s considered critical to good scientific theories and hypotheses. It was first described by philosopher of science Karl Popper in the 1930s, whose seminal text relating to the idea was not even translated into English until the 1950s! But in short, he argued that for a scientific theory to be robust and valid, it had to have the ability to be shown to be false. This does not mean that it will be, but that it could be (Carlson, 2021).

Let’s look at Popper’s famous thought experiment. In his classic example, he asks us to imagine that we have never seen a swan before. We find ourselves sitting alongside a pond, and we see a bevy of swans, all of which have white feathers. You would be reasonable in creating the theory that “All swans are white.” Now imagine a black swan showing up. Your theory must now change to adapt to the new information, and it can. You can now say, “Most swans are white,” but “Some swans are black.” Note how your first theory wasn’t completely wrong, but your new theory is much closer to the truth. That is what we want in science—a process to continually get our theories closer to objective reality. If, after finding a black swan you argue that it could not possibly be counted as a swan, since you say swans can only be white, you are limiting your world understanding. You are not allowing it to be open to change when valid, new evidence comes along.

Null Hypothesis and Confirmation Bias

In the ideal form, hypothesis testing is set up to try and disprove itself. You, as the researcher, put forward a prediction and test it against your collected data. In traditional statistical approaches to hypothesis testing, you do this by setting up what is called a null hypothesis, which is the antagonist to your initial hypothesis. Bear with me here, because this gets a bit convoluted, but let’s go back to our fertilizer example. If our hypothesis is “This new fertilizer will increase our banana yield by 10 percent” then our null is essentially, “No, it won’t!” Or more technically, “This new fertilizer will not increase banana yields.” Our job as scientists is then to accept when we fail to reject the null.

This approach’s purpose is to get away from the innate drive towards confirmation bias, which is when we only look for information that supports our theories and easily ignore that which does not. This approach to disproving is commendable, but there are criticisms. While there are a few concerns, the biggest reason why this null hypothesis approach isn’t ideal is that it works best for purely inferential statistical analysis. It also often does not work as well out in real-world research, or in research where there is already much evidence because it insists on the null being a naïve pure rejection. But the hypothesis as a testable prediction is still useful.

An Example

We’ve now built hypotheses from the ground up and discussed their place in building theories. But you aren’t going to immediately go through the world thinking in precise predictions. Really, the beauty of social and behavioral research is that the world and our own selves are the laboratory. We can’t move through the world without finding fertile ground for research questions.

Say you find yourself stuck in traffic. Not much fun, but you could easily pass the time coming up with many fascinating research questions. What drives people to engage in road rage? How do they pass the time? What are the negative health outcomes of sitting in traffic for hours every day? Does my state really have the worst drivers? Why are cities built around cars instead of other transport systems? Why won’t this person let me merge!

All these questions could be interesting research lines, but it is research that may take you years to answer. That’s because hypothesis testing seeks to test reduced and specific predictions that slowly build up into a supported theory, while also continually testing said theories.

So, let’s apply a scientific approach to our commuter culture and traffic questions by creating a hypothesis. Let’s take the question about why people seem to engage in rude behavior in traffic. A good starting research question may be, “Do driving norms vary in different states?” The first step in creating a good hypothesis is to predict a specific outcome. So, we could predict that people drive more aggressively in their cars in cities than rural areas. The null hypothesis would be that there is no difference in aggressive driving across areas.

Next, let’s operationally define our variables: aggressive driving and location. Remember that how we define variables is based on how we can measure them. So, to measure aggressive driving, look at police reports of road rage incidents, or you could stand on roadways and document the honking, yelling, or vulgar hand gestures that you observe. You also need to define the difference between urban and rural areas. So, you could take definitions and data from the United States Census count or another organization like Housing and Urban Development.

Conclusion

Understanding and applying hypothesis testing in scientific research is crucial for advancing knowledge while mitigating confirmation bias. As you engage with hypotheses, remember Popper’s insight: scientific theories evolve through falsifiability, where predictions are rigorously tested against evidence. Just as identifying a black swan refines our understanding of swans, embracing new data refines our theories closer to objective truth. While hypothesis testing has its limitations, particularly in complex, real-world contexts, its structured approach remains invaluable for both qualitative and quantitative research methodologies. Embrace hypothesis testing not just as a prediction tool, but as a tool to continually refine our world understanding.

Practice

Now that you have a background in hypothesis testing and how a hypothesis is created, let’s practice everything we’ve learned with a few examples one might be likely to encounter in social research. First, pick an example research question below. Second, design a novel hypothesis that might come from your research question. Third, be sure to operationally define which variables you have in your hypothesis, and think about how you would measure them:

1. What are the economic factors that lead people to experience homelessness?
2. Why are educational outcomes different across races?
3. How does election redistricting affect voter participation?

References

Bhattacherjee, A., Toleman, M., Rowling, S., & Andersen, N. (Eds.) (n.d.). Social Science

Research: Principles, Methods, and Practices (Revised edition).

Carlson, E.A. (2021) What Is Science? A Guide for Those Who Love It, Hate It, Or Fear It. World Scientific Publishing. 9789811230103

Price, P., Jhangiani, R., & Chiang, I. (2015). Research Methods of Psychology – 4th American Edition. Victoria, B.C.: BCcampus.

Salsburg, D. (2001). The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century. Macmillan.