A Guide to Understanding High Quality Research

For many readers, one of the most challenging aspects of research published in academic journals is understanding whether or not a particular study is of good quality. Even professional scientists debate the strengths and weaknesses of most published studies. We have also seen some media stories that contain contradicting information (such as the fish oil and prostate cancer debacle).

When reading research that makes claims about the efficacy of a product, it is important to be able to distinguish a meaningful clinical study from one that is potentially flawed. A well-designed clinical study—often referred to as a “valid” or “sound” study—uses rigorous methodology to help ensure a high degree of confidence in its conclusions. A poorly designed study may generate results that are unjustified or downright false. While the key to a good (or bad) study often lies in the details of the research design, there are a few simple questions that can be asked to help understand scientific research.

Here are 9 Questions to Ask to Determine a Credible Clinical Study:

1) Who funded the study?

There are only a few sources of funding for scientific research. Corporations, non-profits, and governmental agencies are the primary sources. Corporate-funded studies are often given a bad rap. However, some of the most important nutrition research conducted to date has been funded by corporations wanting to determine the efficacy of their own products. For consumers, the results of these studies provide information that we all want to know so that we can make the best choices based on valid results.

2) Who is doing the research?

Even with corporate-funded research, it is important for the primary scientific investigator to be neutral (i.e., third party) from any corporate ties so that there is no undue corporate influence in the development and implementation of the study. The affiliations of the researchers are typically found at the end of the study, near the references, so any corporate influences can clearly be identified. While both industry and government-funded studies can provide important clinical information, industry-funded studies can sometimes be perceived as biased. If the study is conducted on company employees, rather than an unaffiliated group of patients, there is a stronger likelihood for corporate bias. However, if the criteria that follow are met, the potential for corporate bias can easily be reduced.

3) Is the study published?

To be deemed credible, studies should always be published in a scientific journal. An ad placed by a manufacturer in a trade journal that has the appearance of a “study” does not constitute a published study.

4) If so, in what journal was the study published?

It is important to determine if a study is published in a “peer-reviewed” journal. Peer-reviewed means that experts in a given field have read, reviewed, and determined that the research is worthwhile and unbiased before publishing it.

5) What type of study is it?

The “gold standard” for research is a study that is: randomized, placebo-controlled, and double-blinded. Small pilot studies are also valuable pieces of research that can help determine whether a larger scope project is worth funding. These studies provide critical fact-finding information.

6) Was the trial controlled?

It’s important that the treatment group of patients (for example, subjects taking fish oil) is compared to a control group. For instance, the control group can either be a placebo (e.g., sugar pill) or an accepted treatment (such as standard of care). The control acts as a standard against which the experimental treatment can be measured. The use of a control group eliminates the influence of the “placebo effect,” which is improvement that occurs simply because the patients think they are being treated when they are not, and expect to feel better as a result. This is also why the best studies are double-blinded, meaning that during the study neither subjects nor researchers know which group is receiving treatment, and which group is not. This way there is less chance that subjects will experience a placebo effect and less chance that researchers will believe that they see one.

7) Was the trial randomised?

It’s important that subjects are randomly assigned to either the treatment group or a control group. Otherwise, it’s impossible to distinguish what effects are due to treatment and what effects resulted naturally from differences in the patients, or if there was, in fact, a “placebo effect.”

8) Did the trial include a large enough sample size?

The “sample size” refers to the number of patients taking part in the study. It is important that the sample of individuals chosen to study truly represents the larger population about which the research will draw conclusions. Large studies are generally more representative of the general population and produce statistically stronger results.

9) Was the dosage adequate?

Sometimes we see studies that generate a positive response, but may have a dose that far exceeds what most people would likely take in a day, or would likely continue to take over time. On the other hand, sometimes we see a negative, or null result (no effect) that may stem from the dose being inadequate or just too low to have a meaningful clinical outcome.

It is perhaps impossible to perform any study that is truly devoid of human bias, or error. However, the above factors should always be taken into consideration when looking at the results of clinical research. The process of peer review exists to eliminate as much error as possible, whether intentional or deliberately misleading. While data is often subjective, the interpretation of it should not be. Don’t be afraid to ask your product manufacturer these very important questions to ensure that their research is not biased, and that you can rely on it to make informed decisions about your health.

For a database of credible research on omega-3, visit www.omega-research.com