# Reading research beyond the headline Canonical page: https://thejasonsamuel.com/reading-research [Home](https://thejasonsamuel.com/) > [Essays](https://thejasonsamuel.com/essays) # Reading research beyond the headline Restored from Instagram posts · 22 August 2023 today I am going to teach you how research which is published in journals without being properly peer reviewed (not the exception) can be skewed. in any research study, the experimental method is the most powerful of all research methods because it is the only one that can determine cause-and-effect relationships. thus, if it is important to know whether one variable produces or causes another variable to change, then the experiment is the only method that should be used. *[a qualification: “only” is too absolute. the question to ask is what assumptions make a causal interpretation credible, rather than dismissing a result solely because its design is not a randomised experiment.]* Two characteristics define an experiment: 1. manipulation of one or more independent variables in the experiment. 2. random assignment of subjects to experimental and control conditions of our choice. If either of these characteristics is missing, a research project cannot be called an experiment; instead, it is called a quasi-experiment, a study, a survey, or an investigation. people confuse these with an experiment and often the reason why you believe someone peddling hear-say without actual journalism credentials as investigative journalism. study: (when defined as a noun) - the activity of learning about something survey: the act of examining a process or questioning a selected sample of individuals to obtain data about a service, product, or process investigation: an official examination of the facts about a situation, crime, etc. whatever facts that come out of an investigations are always situation responses or circumstantial. if I'd have to quote Arthur Conan Doyle, "once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth" "there's nothing deceptive than an obvious fact" In an experiment, the researcher intentionally manipulates one or more aspects of the question of interest, called the independent variable, and measures the changes that occur as a result of that manipulation, called the dependent variable. now we have clarified what these two variables are and how they are derived. Suppose we were interested in finding out whether wearing a suit to an interview is better for men than wearing informal clothes, we could study this issue by observing job applicants at a specific company and comparing the interview scores or hiring data of people with suits with those of people wearing informal clothing. We might find that the better-dressed applicants received higher scores, but we could not conclude that wearing a suit caused the higher scores; something other than the suit may be the cause there. it might be due to the fact that they were talented by themselves, their portfolios made the talking, the recruiter took a liking or recommendations worked in, or it can be anything. Perhaps applicants who own suits are more socially skilled than other applicants; it then might have been social skill and not dress style that led to the higher interview scores. If we want to determine that dress style affects interview scores, we have to manipulate the variable of interest and hold all other variables as constant as possible (which is practically impossible - you must be getting the idea now) okay i am back. let's say we do this as an experiment. how would we do it ? take 100 people and randomly assign half (about 50) of them to wear suits in this hot weather and assign the other 50 to wear whatever they want. shorts, jeans, tees, veshti. Each subject then goes through an interview with an HR. post the interview, we compare the interview scores of our two groups. In this case, the independent variable is the type of dress and the dependent variable is the interview score. Even though this particular research design is not very sophisticated and has many problems with its fundamental assertions, the fact that we manipulated the applicant’s dress style gives us greater confidence that dress style was the cause of higher interview scores. this in fact has no relation with what a person is. presentable appearance of a person always takes importance in giving a false sense of confidence in what the other person can do, and more than often it's a risk that everyone is willing to take. Even though the results of experiments provide more confidence regarding cause-and-effect relationships, ethical and practical considerations do not always make experimental designs possible. want to figure if a research paper is legit? - read the number of samples they took and how they conducted the study and the rationale supporting the size of the study. (an other point - even though twitter created its own metric mDAU for its valuation, the sample size that they took is mathematically right - which affects elon's ability to cite that there are many bots (ofcourse they are though). - often research papers and studies are funded. find who funded them. want to see the research done on a drug ? you can't trust the pharma who funded it. the joke is that, FDA peeps often don't go through the entire research study that was done and often go through the summary of what was given by Pharma people themselves (correct me if the info was wrong) *[a source gap: the claim about what FDA reviewers read needs evidence from the review process. funding is a reason to examine methods and conflicts of interest closely, not a substitute for assessing the study itself.]* this also brings forth various fakes and name sake stuff which sells you that you are the luxury when they really aren't. let's take an example. watches. so you'd understand this more clearly. [← Previous essay](https://thejasonsamuel.com/the-platform-that-farms-your-creativity)[All essays](https://thejasonsamuel.com/essays)[Next essay →](https://thejasonsamuel.com/hiring-through-conversations)All rights reserved. Copyright © 2026 Jason Samuel