FOOD, DIET AND HEALTH: HOW YOU CAN CHECK THE EVIDENCE
Low
carb diets. Low fat diets. Owning a dog. Eating leafy greens. Fermented
foods. Dairy foods. Eggs. Salt. Sugar. We are bombarded with conflicting
messages about food, diet and health in the media and on-line.
Universities and research organisations keen to promote their staff and
their findings in turn bombard the media with often inflated press
releases. How do we work out who or what to believe? In Food for
Thought, Dr Alan Barclay explains the different kinds of evidence and
how compelling it is, or isn’t.
THE STUDY: HOW COMPELLING IS THE EVIDENCE?
1. RANDOMISED CONTROLLED TRIALS (RCTS) ARE CONSIDERED THE “GOLD STANDARD”,
providing the highest level of evidence, as they can prove that
intervention A improves health outcome B, while all other known factors
(known as confounders such as age, gender, body mass index, etc.) have
been accounted for by the randomisation process. The process of studying
people tends to improve their health independent of the intervention
itself, because people know that they are being monitored and are more
conscious of their health, and are consequently being more careful about
what they eat and drink, so having a control group is vital. Only
randomised controlled trials are able to show that a particular
intervention causes a particular outcome.
2. OBSERVATIONAL STUDIES PROVIDE MEDIUM LEVEL EVIDENCE
because scientists are simply observing and measuring people’s
behaviour at a point in time, or over a particular time frame, without
randomising them to groups and providing different dietary
interventions. The best epidemiological evidence comes from large
prospective cohort studies where large groups (typically thousands) of
people have a medical check-up, their dietary patterns are measured, and
they are followed up regularly for long periods of time (e.g., 5–25
years).
Observational studies can’t provide as high a
level of evidence as RCTs can as it is not possible to control for all
confounders (e.g. people who are already overweight may drink more
beverages than those who aren’t, as fluid requirements are proportional
to body size, and being overweight is an independent risk factor for
developing many chronic diseases), and our tools of observation (e.g. a
food frequency questionnaire for measuring a person’s usual food and
drink intake) are imperfect. Observational studies are only able to
prove that event A is associated with outcome Z. It’s possible that
unknown or unmeasured intermediary factors (B, C, D, E, etc) are
involved. They are not able to prove that event A causes outcome Z –
only RCTs can.
3. ANIMAL STUDIES ONLY PROVIDE LOW LEVELS OF SCIENTIFIC EVIDENCE,
however, they can be used to generate hypotheses that can be tested in
human populations (either using RCTs or observational studies) and to
investigate hypothesised physiological mechanisms in experiments that
cannot be ethically conducted in people. They are also used to determine
the toxic dose of novel ingredients, like food additives, for example,
and results are extrapolated to people using a large safety factor
(typically 100 x).
HOW SCIENTISTS REVIEW EVIDENCE: 3 THINGS YOU NEED TO KNOW
1. SYSTEMATIC LITERATURE REVIEWS are based on careful searches of
scientific databases (e.g., PubMed, EMBASE, CINAHL, and Cochrane
Library) with pre-determined search terms looking for all of the
research published on a particular topic over a long period of time
(ideally with no time constraints). Once all studies have been
identified, researchers then go through each paper’s reference lists to
make sure as best as possible that they have not missed any additional
evidence. The data from each paper is then extracted and the results
summarised in a table. The quality of each study is also rated or
graded. Strong conclusions can be drawn from the summarised data.
2. META-ANALYSES
can be performed when three or more similarly designed studies on a
particular health outcome have been published in scientific journals.
The outcome data from each study is entered into specialised software
and weighted according to the study size and statistical significance. A
final summary statistic is given that indicates whether an intervention
is effective, and if so, how effective.
3. SYSTEMATIC LITERATURE REVIEWS AND META-ANALYSES OF RANDOMISED CONTROLLED TRIALS are considered the highest level of evidence. Cochrane reviews
are a good example of this method. You can also do systematic reviews
and meta-analyses of observational studies. However, because the
underlying study design is not as robust as the randomised controlled
trials, they are not considered to be as high a level of evidence as a
Cochrane review, for example.
WHAT DOES IT ALL MEAN?
If the latest study broadcast in the news is:
- A systematic literature review and meta-analysis of randomised controlled trials then the results are worth taking notice of if the people involved are similar to you, and live under similar circumstances.
- A systematic literature review and meta-analysis of observational studies then the results are interesting, but a randomised controlled trial in humans that studied the same effect would be necessary to prove that the relationship was causal.
- A systematic literature review and meta-analysis based on an animal study or in vitro (test-tube) study, then more research in people is needed to prove the hypothesis.
