Bad Science

Pseudoscience in Institutions

The book reveals a troubling trend of pseudoscience infiltrating the UK education system. Practices like Brain Gym, which make scientifically unsupported claims about improving brain function, have become widely accepted in schools despite a lack of evidence. Teachers and administrators are often swayed by the technical jargon used to describe these methods, failing to critically evaluate their legitimacy.

This allows pseudoscientific ideas to spread unchecked, undermining the public's understanding of science and evidence-based approaches. The prevalence of such practices in schools, from primary to secondary levels and even government agencies, demonstrates a concerning gap in critical thinking and scientific literacy. Addressing this issue requires a concerted effort to improve science education and empower teachers to recognize and reject unsubstantiated claims, no matter how "sciencey" they may sound.

Here are key examples from the context that support the insight about pseudoscience infiltrating UK institutions:

• Brain Gym: This is a program of purported "brain exercises" being taught in "hundreds if not thousands of mainstream state schools throughout the country." The exercises make claims like "wiggling your head up and down will increase blood flow to the frontal lobes" and "rubbing your fingers together in a special way will improve 'energy flow' through the body" - all without scientific evidence.

• Government Promotion: Brain Gym is "promoted by local education authorities, funded by the government, and the training counts as continuing professional development for teachers." It is even "promoted on the Department for Education and Skills website."

• Teacher Acceptance: The author notes that "the same teacher who explains to your children how blood is pumped around the lungs and then the body by the heart is also telling them" the pseudoscientific claims of Brain Gym, "without challenging or questioning it."

• Jargon and Complexity: The author suggests the reason teachers accept Brain Gym is because it uses "clever long phrases like 'reticular formation' and 'increased oxidation'" which make the nonsense sound more scientific and complex.

• Lack of Critical Evaluation: The author states that debunking Brain Gym "does not require high-end, specialist knowledge" yet teachers still defend it, indicating a lack of critical evaluation of the claims.

In summary, the context shows how unsupported pseudoscientific practices have become widely accepted in the UK education system, promoted by authorities and uncritically adopted by teachers, due to the use of technical jargon and complexity rather than scientific evidence.

The Power of Placebo

The placebo effect is a powerful phenomenon that demonstrates how our beliefs, expectations, and the presentation of treatments can profoundly influence health outcomes. This effect extends far beyond traditional medicine, impacting our everyday health practices.

The placebo effect shows that the meaning we ascribe to a treatment, rather than just the treatment's pharmacological properties, can trigger physiological changes in the body. For example, studies have found that the color, number, and ceremony surrounding a placebo pill can impact its effectiveness. Even administering a drug that should induce nausea can reduce symptoms if the patient believes it will help.

This insight challenges the notion that the placebo effect is "all in the mind." In reality, the placebo response involves complex interactions between the mind and body, with the brain triggering physiological changes. Everyone is susceptible to the placebo effect - it is a universal human response, not limited to certain personality types.

The placebo effect also highlights how cultural context shapes our health beliefs and experiences. The effectiveness of a treatment can vary dramatically based on the cultural meanings and expectations surrounding it. What works in one setting may not translate to another with different cultural preconditions. Understanding the power of the placebo effect is crucial for developing effective, personalized healthcare approaches.

Here are some examples from the context that illustrate the power of the placebo effect:

• Placebo-controlled trials: The context discusses how placebo-controlled trials can be used to determine the size of the placebo effect. For example, a study compared the ulcer healing rate in trials where the placebo was 2 sugar pills per day versus 4 sugar pills per day, and found that 4 pills were more effective - demonstrating that the "placebo effect" is influenced by the presentation of the treatment.

• Pill color: An experiment found that pink sugar pills were more effective at maintaining concentration than blue sugar pills during a boring lecture, even though the pills contained no active ingredients. This shows how the cultural meanings associated with different colors can impact the placebo effect.

• Dose-response: The context explains that there appears to be a "dose-response" relationship with placebos, where more "treatment" (e.g. more pills) can elicit a stronger placebo effect. This challenges the notion that placebos simply have a binary "works/doesn't work" effect.

• Homeopathy in cholera epidemics: During 19th century cholera epidemics, the London Homeopathic Hospital had much lower mortality rates than the Middlesex Hospital, not because homeopathy had a specific effect, but likely due to the placebo effect - homeopathic treatments did no harm, unlike harmful medical practices at the time.

• Medicalizing problems: The context cautions that while placebos can be beneficial for conditions where mainstream medicine has little to offer, they also carry risks of "medicalizing problems" and reinforcing beliefs about illness that may be unhelpful.

In summary, the examples illustrate how patient beliefs, the presentation of treatments, and the cultural meanings associated with treatments can profoundly influence health outcomes, even when no active medical intervention is present. This highlights the power of the placebo effect.

Scientific Misrepresentation

The misrepresentation of science in the media and academia has serious consequences. When scientific data is cherry-picked, distorted, or presented without proper context, it can mislead the public and hinder scientific progress.

One major issue is publication bias, where negative or inconclusive research findings are suppressed, while positive results are over-emphasized or even duplicated to appear more substantial. This deprives doctors, policymakers, and the public of the full picture, leading to poor decisions.

Another problem is the tendency to focus on sensational, unsubstantiated claims rather than the nuanced reality of scientific research. The media often presents preliminary findings or fringe theories as established facts, creating a false impression of scientific instability and uncertainty.

Ultimately, accurately communicating scientific evidence is crucial. Providing the public with the methods, results, and limitations of research, rather than just the conclusions, empowers them to critically evaluate claims. Upholding scientific integrity benefits everyone, leading to better-informed decisions and advancing human knowledge.

Here are some key examples from the context that illustrate the misrepresentation of science in the media and academia:

• Wacky stories: The context describes how the media often report on "wacky" or sensational scientific stories that have not been properly published or peer-reviewed, such as claims that "watching Richard and Judy improves children's IQ" or that "in the future, all men will have big willies." These stories distort the public's understanding of how real science works.

• Breakthrough stories: The media also frequently report on "breakthrough" scientific findings, without providing the necessary context and caveats. This "reinforces the idea that science is only about tenuous, new, hotly contested data and spectacular breakthroughs," when in reality, established scientific principles like Archimedes' laws of buoyancy have stood the test of time.

• Scare stories: The media often rely on unpublished scientific research, such as the recent "headline stories on new MMR research," to generate alarming "scare" stories that can mislead the public.

• Confirmation bias: The context explains how people, including journalists and researchers, tend to seek out and emphasize information that confirms their pre-existing beliefs, while downplaying contradictory evidence. This "dangerous" bias can lead to the selective reporting and distortion of scientific findings.

• Lack of statistical literacy: The media frequently fail to properly explain the statistical methods and evidence behind scientific claims, instead simply reporting the researchers' conclusions. This prevents the public from critically evaluating the quality of the underlying research.

In summary, the key insight is that the media's tendency to sensationalize, cherry-pick, and misrepresent scientific information can significantly undermine public understanding of science and hinder scientific progress.

Ethics and Efficacy in Alternative Therapies

The book examines the ethical and scientific issues surrounding alternative therapies. It emphasizes the importance of evidence-based medicine and exposes the flaws in popular but unproven treatments like homeopathy.

The book demonstrates how many alternative therapy studies are poorly designed, allowing biases and errors to skew the results in favor of the treatment. Rigorous, unbiased trials consistently show that homeopathic remedies perform no better than placebos. Yet homeopaths often cherry-pick the limited positive studies and ignore the overwhelming negative evidence.

This raises ethical concerns. While placebos can provide benefits in some cases, the book argues that knowingly promoting ineffective treatments as cures is intellectually dishonest and can have harmful consequences. Alternative therapists who denigrate mainstream medicine and discourage proven treatments like vaccines are acting irresponsibly.

The book advocates for a more thoughtful, evidence-based approach to evaluating alternative therapies. Rather than blindly accepting claims of ancient wisdom or quantum effects, it calls for carefully designed studies that measure the true benefits of the entire therapeutic process, not just the purported mechanisms. This balanced, scientific perspective is crucial for ensuring patient safety and public health.

Here are some key examples from the context that support the key insight about the ethics and efficacy of alternative therapies:

• Homeopathy: The context explains how homeopathy was developed by Samuel Hahnemann based on the unproven principle of "like cures like", and how he further claimed that diluting substances would "potentise" their healing powers. Despite numerous trials showing homeopathy performs no better than placebo, homeopaths continue to make unsupported claims about its efficacy.

• Placebo Effect: The context discusses the ethical quandaries around using placebos, noting that while homeopathic sugar pills may provide some benefits through the placebo effect, this comes with risks like "medicalizing problems" and promoting the idea that "a pill is an appropriate response to a social problem." It also notes how homeopaths sometimes irresponsibly discourage use of mainstream medicine.

• Cherry-Picking Evidence: The context provides the example of Linus Pauling selectively citing research to support his claims about vitamin C, illustrating how alternative therapists often "cherry-pick" favorable studies while dismissing contradictory evidence. This contrasts with the systematic, evidence-based approach of meta-analyses.

• Antioxidants: The context suggests the prevailing "nutritionist" view on the benefits of antioxidants is oversimplified, highlighting the need for a more nuanced, evidence-based understanding of complex health claims.

The key point is that the book emphasizes the importance of rigorous, unbiased evaluation of alternative therapies' claims and effects, rather than accepting unsupported or cherry-picked evidence. It cautions against the ethical risks of using placebos deceptively, and advocates for a more sophisticated, evidence-based approach to complex health topics.

Statistical Misinterpretation

The book highlights how statistical misinterpretation can lead to flawed conclusions. One common pitfall is the misuse of relative risk, where small changes are exaggerated to create a false sense of significance. Another issue is data dredging, where researchers sift through data to find patterns that may be spurious.

To avoid these traps, the book emphasizes the importance of robust statistical literacy. Interpreting research accurately requires understanding the limitations of statistical methods and being cautious about drawing conclusions from limited or cherry-picked data. Without this critical perspective, people are vulnerable to being misled by misleading statistics and unsupported claims.

The book underscores that sound statistical reasoning is essential for making sense of complex information and separating genuine insights from cognitive biases and logical fallacies. By developing stronger statistical skills, readers can learn to scrutinize research findings more effectively and avoid falling prey to common statistical misinterpretations.

Here are some examples from the context that illustrate the key insight about statistical misinterpretation:

• The experiment where subjects were asked to determine if a child's arrival time at school was affected by rewards and punishments, even though the arrival times were completely random. The subjects still concluded that reprimands were more effective, demonstrating how people see causal relationships where there are none.

• The example of people believing in "winning streaks" in sports, when statistical analysis has shown no such patterns. This shows how people perceive patterns in random sequences of events.

• The study on beliefs about the death penalty, where subjects selectively found flaws in research data that went against their pre-existing views, while accepting data that supported their views. This demonstrates how people's assessment of evidence is biased by their prior beliefs.

• The discussion of how anecdotal "success stories" about alternative medicine have high "availability" and emotional impact, making them more memorable and persuasive than abstract statistical data, even if the data does not support the anecdotes.

• The experiment where subjects asked confirmatory questions when trying to determine if someone was an "extrovert", rather than questions that could disprove the hypothesis. This illustrates the bias towards seeking out confirmatory information.

These examples highlight how people's intuitive reasoning and interpretation of data can be systematically flawed, emphasizing the need for robust statistical literacy to avoid such pitfalls.

Industry Influence on Research

The book exposes how commercial interests can corrupt scientific research and public health guidance. It highlights the pervasive influence of pharmaceutical and food supplement industries on the evidence base.

These industries often distort research findings to promote their products. They may selectively publish favorable studies, hide negative data, or use surrogate outcomes that don't reflect real-world benefits. This allows them to make exaggerated marketing claims not supported by rigorous science.

The book cautions that anyone can call themselves a "nutritionist" and make unsupported health claims, similar to tactics used by drug companies. It encourages readers to be skeptical of such claims and to demand high-quality, unbiased evidence before accepting advice about supplements or diets. Systematic reviews of the full body of research, not cherry-picked studies, should guide public health policies and individual health decisions.

Here are specific examples from the context that illustrate the key insight about industry influence on research:

• The case of the pharmaceutical company refusing to release information on clinical trials to academics is cited as an example of a company withholding data to avoid scrutiny of its claims.

• The founder of the company Equazen, Cathra Kelliher, is noted to have been influenced by David Horrobin, who is described as having research results that "wind up being presented directly to the media" rather than undergoing rigorous academic review.

• The context discusses how "media nutritionists" make claims about "placebo-controlled randomised control trials" and "surrogate outcomes" that are misleading, as the trials may only show changes in blood markers rather than real-world health benefits. This is described as a tactic used by "drug companies" to promote their products.

• The context states that "animal data" is often misapplied by "media nutritionists" to make unsupported claims about human health, without accounting for the limitations of applying animal research to people.

• The example of Linus Pauling's selective use of research on vitamin C and the common cold is provided as a case study of "cherry-picking" in the academic literature to support a predetermined position, which the context states is a common practice in "alternative therapies" and "nutrition-ism."

Quotes

Let's take a look at some key quotes from "Bad Science" that resonated with readers.

You cannot reason people out of a position that they did not reason themselves into.

People often hold beliefs or opinions based on emotional or intuitive connections, rather than logical reasoning. It's challenging to change their minds through rational argumentation because their initial beliefs weren't founded on reason in the first place. This phenomenon highlights the importance of critically evaluating evidence and using sound reasoning when forming opinions.

You are a placebo responder. Your body plays tricks on your mind. You cannot be trusted.

Our perceptions can be influenced by our beliefs and expectations, making it difficult to distinguish between real and imagined effects. This means that our bodies can respond to treatments or substances even if they have no actual medical benefit, simply because we believe they will work. As a result, our minds can play tricks on us, making it essential to approach health claims with a critical and nuanced perspective.

These corporations run our culture, and they riddle it with bullshit.

Large companies have a significant influence on the way we live and think, shaping our culture and values. They often spread false or misleading information to serve their own interests, making it difficult to distinguish truth from fiction. As a result, our daily lives are filled with deceptive messages and half-truths, which can be damaging to individuals and society as a whole.

Chapter Notes

INTRODUCTION

• Pseudoscience in Education: The chapter highlights the concerning trend of pseudoscientific practices, such as "Brain Gym" exercises, being taught in British state schools, despite a lack of scientific evidence supporting their effectiveness.

• Credence Given to Quacks: The chapter suggests that the mainstream media often gives credence to the claims of quacks and individuals who lack a basic understanding of science and evidence, leading to the public being repeatedly misled.

• Lack of Evidence-Based Medicine Education: The chapter argues that the education system has failed to adequately teach students about evidence-based medicine, statistics, and the science behind what can kill or cure them, leaving a significant gap in the public's understanding of these critical topics.

• Homeopathy as a Model for Teaching Evidence-Based Medicine: The chapter proposes using homeopathy as a model for teaching the principles of evidence-based medicine, including the importance of proper trial design and the role of the placebo effect in understanding the effectiveness of treatments.

• Nutritionists as Alternative Therapists: The chapter suggests that nutritionists, despite being perceived as scientific experts, often make errors and promote pseudoscientific claims that undermine the public's understanding of evidence-based practices.

• Pharmaceutical Industry Tricks: The chapter indicates that the pharmaceutical industry employs various tactics to mislead doctors and patients about the efficacy and safety of their products.

• Media Misunderstanding of Science: The chapter criticizes the media's tendency to promote misunderstandings of science, statistics, and evidence, which contributes to the public's confusion and susceptibility to misinformation.

• Consequences of Misunderstanding Evidence: The chapter warns that the widespread misunderstanding of evidence can have grave consequences, including people being imprisoned, derided, or even dying due to poor decision-making based on flawed information.

• Empowering the Reader: The chapter aims to provide the reader with the tools and knowledge necessary to critically evaluate claims, understand the nature of evidence, and engage in informed discussions on a wide range of health-related topics.

1 Matter

Here are the key takeaways from the chapter:

• Pseudoscience vs. the Scientific Method: Many people are eager to share their views on science despite never having done an experiment themselves. They treat "science" as a monolith rather than a method of testing ideas through experimentation and observation.

• Detox Treatments as Theater: Detox treatments like Aqua Detox and Hopi Ear Candles are examples of "pseudoscience" that rely on the appearance of science rather than actual scientific evidence. The changes in water color or wax buildup are simply theater, not evidence of toxin removal.

• Controlled Experiments: Conducting controlled experiments, where the only difference between conditions is the factor being tested, is crucial for determining the validity of pseudoscientific claims. The chapter provides examples of how to set up such experiments for detox treatments.

• Shifting Goalposts: When presented with evidence that contradicts their claims, proponents of pseudoscience often shift the goalposts or retreat into untestable positions, rather than revising their ideas based on the new findings.

• Detox as a Cultural Phenomenon: Detox is better understood as a cultural product than a scientific concept. It blends common sense with medicalised fantasy, and reflects a desire for ritual purification and redemption from unhealthy behaviors.

• Detox vs. Healthy Lifestyle Changes: While there is nothing wrong with making healthy lifestyle changes, the detox industry presents these as quick fixes, rather than addressing the long-term impact of lifestyle factors on health.

• Pseudoscience and Ritual Purification: Detox rituals are part of a broader human tendency to seek purification and redemption through ritual, which has existed in various forms across cultures and religions throughout history.

2 Brain Gym

Here are the key takeaways from the chapter:

• Brain Gym is a widespread pseudoscientific program in the UK education system: Brain Gym is a program that claims to "enhance the experience of whole brain learning" through a series of complicated exercises, but is riddled with nonsensical and unsubstantiated claims. It is widely promoted and used in state schools across the UK, despite being scientifically unsound.

• Brain Gym makes dubious claims about the body and brain: For example, it claims that "water is best absorbed by the body when provided in frequent small amounts", that "processed foods do not contain water", and that certain exercises can "stimulate the carotid arteries" or "connect the electrical circuits in the body". These claims are easily falsifiable and have no scientific basis.

• The appeal of Brain Gym is due to the use of technical-sounding language: Research has shown that people are more likely to find explanations convincing if they contain irrelevant neuroscientific jargon, even if the explanation itself is nonsensical. Brain Gym exploits this tendency by using complex-sounding terminology to make its claims seem more credible.

• Pseudoscience can be used to promote otherwise sensible practices: While Brain Gym advocates some reasonable practices like exercise breaks and hydration, it wraps these in a layer of pseudoscience. This can make people more receptive to the sensible advice, but also risks instilling a dependence on dubious authority figures and undermining critical thinking.

• The proliferation of Brain Gym highlights issues with the education system: The widespread acceptance of Brain Gym in schools suggests a lack of critical thinking and scientific literacy among teachers, as well as a failure of oversight and regulation by educational authorities. This is a concerning trend that can have long-term consequences for students' understanding of science.

3 The Progenium XY Complex

• Moisturizing Creams and Their Ingredients: Moisturizing creams typically contain three main groups of ingredients:

  • Powerful chemicals like alpha-hydroxy acids, high levels of vitamin C, or molecular variations of vitamin A, which can make skin appear more youthful but also cause irritation, stinging, and redness.
  • Cooked and mashed-up vegetable proteins, which can temporarily tighten skin and reduce the appearance of fine wrinkles as the cream dries.
  • A long list of esoteric, scientifically-sounding ingredients that are often included without clear evidence of their effectiveness.

• Absorption and Efficacy of Ingredients: The author argues that many of the claims made about the ability of cosmetic ingredients to be absorbed through the skin and have a meaningful impact on appearance are not supported by evidence. The skin is designed to be relatively impermeable, and the body has complex homeostatic mechanisms that would likely counteract any attempts to "hack" it with topical applications.

• Semantic Games and Regulatory Compliance: Cosmetic companies often make claims about their products that are carefully worded to be suggestive and compelling, while still being technically and legally defensible. The author suggests that many of these claims are not supported by rigorous, publicly available scientific evidence.

• Moral Concerns: The author acknowledges that people are free to spend their money on cosmetic products, but argues that the industry's marketing tactics can be problematic. They sell the idea that science is incomprehensible and that expensive products can be a shortcut to a healthy appearance, rather than promoting healthy lifestyle choices. This can be particularly concerning when targeting young women, who are underrepresented in the sciences.

• The Barbie Liberation Organization: The author references the Barbie Liberation Organization, a feminist direct-action group that switched the voice circuits of Teen Talk Barbies and GI Joe dolls in the 1990s, as an example of challenging the problematic messaging around gender and science that is often perpetuated by the cosmetics industry.

4 Homeopathy

• Homeopathy is a paradigmatic example of alternative therapy: It claims historical authority, has a sciencey-sounding framework without evidence, and its proponents assert the pills work despite extensive research showing they perform no better than placebo.

• Homeopathy was devised by Samuel Hahnemann: He theorized that substances which induce symptoms in healthy individuals can be used to treat the same symptoms in sick people (the "like cures like" principle). He also believed that diluting substances would "potentise" their healing powers.

• Homeopathic "provings": Groups of volunteers take doses of a remedy and record their experiences, which are then compiled into a "symptom picture" for that remedy. This process has obvious flaws, as the experiences may not be caused by the remedy.

• Homeopathic dilutions are extremely high: A typical 30C dilution is one part in 10^60, which is far beyond the number of atoms in the universe. Homeopaths claim water has a "memory" that retains the healing properties, but this is scientifically implausible.

• Evaluating the efficacy of treatments: Subjective symptoms make it difficult to determine if a treatment is effective. Factors like the placebo effect and regression to the mean can lead to the mistaken belief that a treatment is working.

• Placebo-controlled trials: These are the gold standard for evaluating treatments. They involve randomly assigning patients to receive either the treatment or an inert placebo, with neither the patients nor the researchers knowing who received what.

• Importance of proper trial design: Trials must be properly randomized and blinded to avoid bias. Poorly designed trials tend to show positive results for homeopathy, while well-designed trials find homeopathy performs no better than placebo.

• Meta-analyses: These combine the results of multiple trials on the same topic, providing a more robust evaluation of the overall evidence. Meta-analyses have consistently shown homeopathy is no more effective than placebo.

• Pragmatic trials: These compare homeopathic treatment to usual care, without blinding, to evaluate the real-world effectiveness of the entire homeopathic process, not just the pills.

• Homeopaths' response to negative evidence: They often cherry-pick positive trials, misrepresent meta-analyses, and respond with anger and threats rather than engaging with the evidence in a constructive manner.

5 The Placebo Effect

Here are the key takeaways from the chapter:

• The Placebo Effect is a Powerful Phenomenon: The placebo effect refers to the beneficial effects that a treatment can have simply due to the patient's belief in the treatment, rather than any inherent pharmacological properties of the treatment itself. The chapter provides numerous examples demonstrating the potency of the placebo effect, even in dramatic medical contexts like surgery and pain relief.

• Placebo Effects are Influenced by Treatment Characteristics: The chapter shows that factors like the color, shape, and packaging of a placebo treatment can significantly influence its effectiveness. Treatments that appear more "medical" or "scientific" tend to elicit stronger placebo responses.

• Doctors' Beliefs and Behaviors Affect Outcomes: The chapter describes experiments showing that doctors' beliefs about a treatment, even if not communicated to the patient, can influence the patient's response. Doctors who convey confidence and positivity about a treatment tend to produce better outcomes.

• Placebo Explanations Can Also Be Beneficial: The chapter discusses how simply providing patients with a confident diagnosis or explanation for their condition, even if it is not scientifically grounded, can improve outcomes. This suggests that the "placebo effect" extends beyond just physical treatments.

• The Placebo Effect is Culturally-Specific: The chapter argues that the placebo effect is heavily influenced by cultural context and expectations. What works as a placebo in one setting may not work in another, as the meaning and symbolism of the treatment is culturally-dependent.

• Ethical Considerations Around the Placebo Effect: The chapter explores the ethical dilemmas around utilizing the placebo effect, particularly in the context of alternative therapies that may undermine public health measures or provide false hope. It suggests the need for an "ethical placebo" that enhances treatments without misleading patients.

• The Placebo Effect Challenges Simplistic Biomedical Models: The chapter argues that the placebo effect reveals the limitations of a purely biomedical model of health and medicine. It suggests that the mind-body connection and the role of meaning are crucial aspects of healing that science is still working to fully understand.

6 The Nonsense

• Misrepresentation of Evidence: Nutritionists often make claims that are not supported by the available scientific evidence. They may extrapolate wildly from laboratory studies to make claims about humans, or from observational data to make intervention claims, and they may also cherry-pick the literature to support their arguments.

• Distinction between Lying and Bullshitting: The author argues that nutritionists are often not outright liars, but rather "bullshitters" who do not care about the truth and simply make claims to impress or persuade the audience, rather than to accurately represent the evidence.

• Confounding Variables in Observational Studies: Observational studies that find associations between diet and health outcomes often fail to account for confounding variables, such as differences in lifestyle, socioeconomic status, and other factors that may also influence the outcomes.

• Extrapolation from Laboratory to Humans: Nutritionists often cite laboratory studies on cells or animals as evidence for claims about the effects of nutrients or foods on human health, without considering the limitations of extrapolating from these simplified models to the complex human system.

• Surrogate Outcomes and Limitations of Animal Studies: Nutritionists may cite changes in blood markers or other surrogate outcomes from studies as evidence of health benefits, without considering the limitations of using these as proxies for real-world outcomes. Similarly, they may cite animal studies without acknowledging the differences between animal models and human biology.

• Systematic Reviews and the Importance of Rigorous Evidence Synthesis: The author highlights the importance of systematic reviews, which rigorously search, evaluate, and synthesize the available evidence, as a way to overcome the cherry-picking and misrepresentation of evidence that is common in nutrition research and reporting.

• The Antioxidant Example and the Limitations of Reductionist Thinking: The chapter uses the example of antioxidants to illustrate how a plausible theoretical mechanism can lead to overly simplistic dietary recommendations, which are then contradicted by the results of large-scale, rigorous clinical trials.

• The Role of Industry Influence and Lobbying: The author suggests that the food supplement industry, like other industries with vested interests, engages in tactics to "manufacture doubt" and undermine the credibility of research that challenges their products, which can contribute to the persistence of misleading nutrition claims.

7 Dr Gillian McKeith PhD

• Gillian McKeith's Credentials: Gillian McKeith claims to have a PhD from Clayton College of Natural Health, a non-accredited correspondence course college, and a masters degree from the same institution. She has also falsely claimed to have a PhD from the American College of Nutrition. The Advertising Standards Authority ruled that her use of the title "doctor" was misleading, as her qualifications did not meet the expected standards.

• Misunderstanding of Basic Science: McKeith demonstrates a poor understanding of fundamental scientific concepts, such as photosynthesis and the nutritional energy content of plants. She makes claims that are easily disproven by basic biology knowledge, suggesting a lack of scientific rigor in her work.

• Questionable Research and Clinical Practices: McKeith claims to conduct clinical research and trials, but the evidence she cites is often from obscure magazines and books rather than peer-reviewed scientific journals. Her methods of examining patients and diagnosing conditions are also questionable and lack the rigor of proper medical practice.

• Aggressive Response to Criticism: When her work has been criticized by scientists and medical professionals, McKeith has responded aggressively, threatening legal action and making false claims about the nature of the criticism. This suggests a lack of willingness to engage in constructive scientific discourse.

• Ineffective Dietary Advice: While McKeith promotes the general advice to eat more fresh fruits and vegetables, which is supported by evidence, her more specific dietary recommendations and claims about the benefits of her products are not well-supported by scientific research. The author argues that the overcomplicated and unjustified nature of her advice can be confusing and even harmful to consumers.

• Societal Factors in Health: The author suggests that individual dietary changes are difficult to maintain and that broader societal factors, such as social inequality and the food environment, may be more important determinants of population health. He argues that these systemic issues are often overlooked in favor of more sensational, individualized approaches promoted by figures like McKeith.

8 'Pill Solves Complex Social Problem'

• Medicalisation: The process of expanding the biomedical remit into domains where it may not be helpful or necessary, often driven by commercial interests. This can involve "disease-mongering" - inventing new diseases for which existing treatments can be marketed.

• Placebo Effect: The phenomenon where the mere act of taking a pill, or being part of a study, can lead to improvements in performance or health outcomes, regardless of the actual efficacy of the intervention. This needs to be accounted for in well-designed trials.

• Hawthorne Effect: The finding that simply being part of a study can improve performance or outcomes, independent of the intervention being tested. This is another factor that needs to be considered in trial design.

• Importance of Proper Trial Design: Trials without a placebo control group are prone to producing false positive results due to the placebo and Hawthorne effects. Proper randomized, placebo-controlled trials are necessary to establish the true efficacy of an intervention.

• Misrepresentation of Research in the Media: Companies and researchers often use media coverage to promote the benefits of their products, making claims that are not supported by the actual research evidence. This can mislead the public and policymakers.

• Withholding of Research Data: There is a concerning trend of researchers and companies withholding full details of their studies from the public, making it difficult to independently evaluate the validity of their claims.

• Ethical Concerns: Conducting poorly designed trials, especially on children, and then failing to publish the results raises serious ethical issues around the exploitation of research participants.

• Preference for "Pill Solutions": There is a societal tendency to prefer simple, technological "pill solutions" to complex social problems, rather than addressing underlying social, educational, or lifestyle factors.

• Parallels with the Pharmaceutical Industry: The tactics used by the food supplement industry, such as media manipulation and withholding of data, are similar to those employed by the pharmaceutical industry, despite the "alternative" branding.

• Lack of Regulation: The food supplement industry is largely unregulated, allowing companies to make unsupported claims about the benefits of their products, in contrast to the stricter regulations on pharmaceutical advertising.

9 Professor Patrick Holford

• Cherry-picking of Evidence: The author highlights how Patrick Holford selectively chooses research studies that support his claims, while ignoring the broader body of evidence that contradicts his assertions. This practice of "cherry-picking" is a common tactic used by Holford and other "nutritionists" to make their claims appear more convincing.

• Misinterpretation of Research: The author demonstrates how Holford often misinterprets the findings of research studies, making claims that go far beyond what the actual data supports. This is exemplified by Holford's assertion that vitamin C is more effective than the HIV drug AZT, despite the research paper he cites not making any such comparison.

• Questionable References and Sources: The author closely examines the references and sources Holford uses to support his claims, and finds that many of them are either self-published, from dubious or discredited sources, or simply do not actually support the claims made.

• Lack of Academic Rigor: The author's in-depth analysis of Holford's work reveals a consistent lack of academic rigor, with issues such as inaccurate citations, cherry-picking of evidence, and overextrapolation of findings. This raises concerns about Holford's suitability to be appointed as a visiting professor and to supervise research.

• Penetration of Pseudoscience into Academia: The author expresses concern about the ease with which figures like Holford, who promote pseudoscientific "nutritionism," are able to gain academic credibility and influence, despite a lack of genuine scientific evidence to support their claims. This highlights a broader issue of how such ideas are able to infiltrate the academic system.

• Tactics to Avoid Criticism: The author notes that Holford often responds to criticism by employing tactics such as making accusations of bias, issuing legal threats, and overwhelming critics with a "cloud of sciencey material." This serves to discourage further scrutiny of his work.

• Importance of Critical Appraisal: The author emphasizes the need for a critical and rigorous approach to evaluating scientific claims, particularly in the field of nutrition, where pseudoscientific ideas can gain widespread acceptance. This underscores the importance of maintaining academic standards and integrity.

10 The Doctor Will Sue You Now

Here are the key takeaways from the chapter:

• Matthias Rath's Libel Lawsuit: Rath sued the author and the Guardian newspaper for libel, which cost over £500,000 to defend. This lawsuit provided the author with extensive knowledge about Rath and his activities.

• Rath's Vitamin Pill Marketing in Europe: Rath made false and misleading claims about his vitamin pills, such as that they could cure cancer and were more effective than chemotherapy. He faced limited regulatory action in Europe for these claims.

• Rath's Vitamin Pill Marketing in South Africa: Rath took his vitamin pill marketing to South Africa, making false claims that his vitamins could treat and cure AIDS, and that antiretroviral drugs were poisonous and ineffective. This had devastating consequences.

• South African Government's AIDS Denialism: The South African government under President Thabo Mbeki and Health Minister Manto Tshabalala-Msimang promoted AIDS denialism and opposed the rollout of antiretroviral treatment, leading to hundreds of thousands of preventable deaths.

• Rath's Attacks on the Treatment Action Campaign: Rath launched a campaign against the Treatment Action Campaign, a grassroots organization that fought for access to antiretroviral treatment in South Africa, accusing them of being part of a pharmaceutical industry conspiracy.

• Lack of Criticism from the Alternative Therapy Movement: The author notes that the alternative therapy movement, including prominent figures and organizations, have failed to criticize or distance themselves from Rath's harmful activities, demonstrating a systemic inability for critical self-appraisal.

• Cultural and Intellectual Dangers: The author suggests that the biggest dangers posed by the issues covered in the book are cultural and intellectual, rather than just the specific harms caused by individual actors.

11 Is Mainstream Medicine Evil?

• Pharmaceutical Industry Influence on Research: The pharmaceutical industry has a significant influence on what gets researched, how it is researched, how the results are reported, analyzed, and interpreted. This is due to the fact that 90% of clinical drug trials and 70% of trials reported in major medical journals are conducted or commissioned by the pharmaceutical industry.

• Tricks Used by Pharmaceutical Companies: Pharmaceutical companies employ various tactics to manipulate research data, such as:

  • Studying the drug in the "winners" (i.e., younger, healthier patients) to make the drug appear more effective
  • Comparing the drug against a useless control (e.g., placebo) to guarantee a positive result
  • Using an inadequate dose of the competing drug to make their own drug appear superior
  • Failing to ask about or report certain side effects (e.g., sexual side effects of SSRI antidepressants)
  • Using "surrogate outcomes" (e.g., reduced cholesterol) instead of real-world outcomes (e.g., cardiac deaths)
  • Selectively publishing or suppressing negative results

• Publication Bias: Positive trials are more likely to be published than negative trials, leading to an overestimation of the effectiveness of treatments. This is due to factors such as researchers' lack of motivation to publish negative results and the difficulty in getting negative results published in high-impact journals.

• Duplicate Publication: Pharmaceutical companies can publish the same positive results multiple times in different journals, creating the illusion of a larger body of evidence supporting their drug.

• Hiding Harm: Pharmaceutical companies have been known to downplay or suppress evidence of harmful side effects of their drugs, such as the increased risk of heart attacks associated with the painkiller Vioxx.

• Clinical Trials Registry: A public, open, and properly enforced clinical trials registry could help address many of the issues with pharmaceutical industry research by requiring the pre-registration of trial protocols and the reporting of all results, regardless of their nature.

• Direct-to-Consumer Advertising: Direct-to-consumer advertising of drugs, which is allowed in the United States but not in the UK, has been shown to increase patient requests for and doctor prescriptions of the advertised drugs, even when the evidence for their efficacy is weak.

12 How the Media Promote the Public Misunderstanding of Science

Here are the key takeaways from the chapter:

• Wacky Science Stories: The media frequently publish "wacky" science stories that are empty, fanciful, and masquerade as legitimate science. These stories are often sponsored by PR companies and used to promote products or fill pages cheaply. Examples include formulae for the "perfect" boiled egg, TV sitcom, or joke.

• Misrepresentation of Research: The media often present the conclusions or hypotheses of research studies without providing the actual evidence or methodology. This creates a distorted view of science as "groundless, incomprehensible, and contradictory."

• Reliance on Authority Figures: Instead of presenting scientific evidence, the media often rely on "authority figures" like scientists to make claims, without examining the reasoning or evidence behind those claims. This leaves the door open for questionable authorities to gain credibility.

• Misunderstanding of Statistics: The media frequently misunderstand and misrepresent statistical findings, leading to misleading or inaccurate reporting on scientific studies.

• Lack of Scientific Expertise: Many journalists who cover science have backgrounds in the humanities and lack a deep understanding of scientific concepts and methods. This contributes to the distorted portrayal of science in the media.

• Structural Issues in Journalism: The author argues that the media's approach to science reporting is driven by structural issues in journalism, such as the need to fill pages quickly and the pressure to produce "clickbait" stories, rather than a genuine interest in accurately representing science.

• Commercialization of Science: The media often uncritically report on studies that are sponsored by commercial interests, blurring the line between legitimate science and promotional content.

• Gradual Progress in Science: The author notes that modern scientific progress often occurs through gradual, incremental advances, rather than dramatic "breakthroughs." This makes it challenging for the media to report on science in a way that fits their preferred narrative of sudden, revolutionary discoveries.

13 Why Clever People Believe Stupid Things

Here are the key takeaways from the chapter:

• Randomness and Pattern Recognition: Humans have an innate tendency to perceive patterns in random sequences of events, even when no such patterns exist. This is demonstrated by experiments where people identify "streaks" or "runs of luck" in random sequences of X's and O's, despite the sequences being truly random.

• Regression to the Mean: When things are at their extremes, they tend to settle back towards the average or "mean" over time. This can lead to the mistaken belief that an intervention (e.g. a homeopathic remedy) caused an improvement, when the improvement was simply due to regression to the mean.

• Confirmation Bias: People have a tendency to seek out and overvalue information that confirms their existing beliefs or hypotheses, while downplaying or ignoring information that contradicts them. This is demonstrated in experiments where people find flaws in research that goes against their views, but accept research that supports their views.

• Availability Heuristic: People's judgments and perceptions are disproportionately influenced by information that is more "available" or salient, such as dramatic anecdotes or emotionally-charged stories, rather than by more abstract statistical information.

• Social Conformity: People's beliefs and behaviors are strongly influenced by the beliefs and behaviors of those around them, even in the face of clear evidence to the contrary, as demonstrated by the Asch experiments on social conformity.

• Other Cognitive Biases: Humans also exhibit other cognitive biases, such as an overly positive self-assessment, the tendency to attribute our own successes to internal factors and our failures to external factors, and the tendency to assume that positive characteristics cluster together.

The key point is that our intuitive reasoning is subject to a variety of systematic biases and flaws, which can lead us to believe "stupid things" even when we are otherwise "clever" people. The scientific method and the use of statistics are valuable tools for overcoming these biases and arriving at more accurate understandings of the world.

14 Bad Stats

Here are the key takeaways from the chapter:

• Natural Frequencies: Natural frequencies, which use concrete numbers to describe risk, are more intuitive and informative than other statistical measures like relative risk increase or absolute risk increase. Journalists and experts often fail to communicate risk using natural frequencies, leading to misunderstandings.

• Relative Risk Increase: Journalists often choose to report the "relative risk increase" when describing a change in risk, as it sounds more dramatic than the actual change in risk. For example, a 2% increase in risk may be reported as a 50% relative risk increase.

• Data Dredging and Multiple Comparisons: When analyzing large datasets with many variables, researchers must correct for the increased likelihood of finding "significant" results by chance alone (the "multiple comparisons problem"). Failing to do so can lead to false positive findings.

• Clustering and Statistical Significance: When data points are not independent (e.g. students in the same school), the statistical significance of findings is reduced. Researchers must "correct for clustering" to avoid overstating the significance of their results.

• Prosecutor's Fallacy: Equating the rarity of an event (e.g. double SIDS) with the probability of innocence is a logical fallacy. The relevant question is the relative likelihood of the competing explanations (double SIDS vs. double murder), not the overall rarity of the observed event.

• Ecological Fallacy: Inferring individual-level relationships from group-level data can lead to erroneous conclusions. For example, the risk of SIDS for a family cannot be directly inferred from population-level SIDS rates.

• Unlikely Events Happen: Even extremely unlikely events, like winning the lottery, do occur by chance. Mistaking such unlikely events as evidence of wrongdoing or a pattern is a common statistical pitfall, as seen in the case of nurse Lucia de Berk.

• Circular Reasoning in Data Collection: Investigators can inadvertently introduce bias by selectively collecting data based on their initial suspicions, a form of circular reasoning that undermines the validity of the statistical analysis.

• Multiplication of P-values: Incorrectly combining p-values by simply multiplying them, rather than using appropriate statistical methods, can artificially inflate the significance of findings and lead to erroneous conclusions, as seen in the Lucia de Berk case.

15 Health Scares

Here are the key takeaways from the chapter:

• Modest tweaks in risk can have magnified effects when followed by a large population: When advice or information is followed by a large number of people, even small errors can lead to significant harm. This is illustrated by the example of Dr. Spock's advice on how babies should sleep.

• Journalists have a special responsibility when reporting on health-related issues: Journalists can mislead readers in various ways, such as cherry-picking evidence, manipulating statistics, or pitting emotion against facts. The MRSA swabs hoax and the MMR story are used as illustrative examples.

• The MRSA swabs hoax: A private laboratory, Chemsol Consulting, run by Dr. Christopher Malyszewicz, provided false positive MRSA results to undercover journalists, leading to a series of sensationalized media stories about the prevalence of MRSA in hospitals. Microbiologists at various hospitals found that the Chemsol results were inaccurate, but their concerns were ignored by the media.

• Malyszewicz's lack of qualifications: Despite being presented as a "respected MRSA specialist" and "the UK's top MRSA expert" by the media, Malyszewicz had no formal microbiology qualifications or training. He had a non-accredited PhD from the US, which was not recognized in the UK.

• The "Unskilled and Unaware of It" phenomenon: The media's response to the concerns raised by microbiologists about Malyszewicz's methods is an example of the "Unskilled and Unaware of It" phenomenon, where incompetent individuals are unable to recognize their own incompetence.

• The media's tendency to overstate their role in exposing health risks: The chapter debunks the myth that the media played a heroic role in exposing the dangers of thalidomide, showing that the medical community was responsible for the initial discovery and subsequent action.

• The media's dismissive attitude towards science: The chapter suggests that some media professionals, feeling intellectually intimidated by science, may conclude that it is all "arbitrary, made-up nonsense" and that they can simply pick and choose results to suit their agenda.

• Sympathy for Malyszewicz: The author expresses a degree of sympathy for Malyszewicz, viewing him as a "Walter Mitty figure" who was overwhelmed by the media attention and the consequences of his actions, ultimately leading to his tragic death.

16 The Media's MMR Hoax

Here are the key takeaways from the chapter:

• The MMR Scare was a Prototypical Health Scare: The MMR scare had all the elements of a typical health scare - misinformation, hysteria, and venal incompetence from the media. It serves as a model for understanding how health scares emerge and propagate.

• Vaccine Scares are Geographically Isolated: Vaccine scares tend to be confined to specific regions and do not propagate globally, suggesting they are driven more by local social and political concerns rather than genuine scientific evidence.

• The Wakefield Study was Flawed and Misreported: The 1998 Lancet paper by Andrew Wakefield that claimed a link between MMR and autism was a case series with major methodological flaws. However, the media vastly overinterpreted and sensationalized the findings.

• Subsequent Research Found No Link Between MMR and Autism: Large, well-designed studies in the years after the Wakefield paper consistently found no evidence of a causal link between the MMR vaccine and autism. However, the media largely ignored these reassuring findings.

• The Media Prioritized Emotive Narratives over Scientific Evidence: The media coverage focused heavily on personal anecdotes and emotive appeals from parents, while downplaying or ignoring the actual scientific evidence on MMR safety.

• Lack of Science Reporting Expertise Contributed to Poor Coverage: The MMR story was often covered by generalist reporters rather than science/health specialists, leading to a poor understanding and representation of the underlying scientific evidence.

• Vaccine Hesitancy Caused by the Media Scare has Led to Disease Resurgence: The drop in MMR vaccination rates due to the media scare has resulted in a resurgence of measles, mumps, and rubella - diseases that can have serious health consequences, especially for unvaccinated children.

• Vaccines are a Victim of Their Own Success: As vaccine-preventable diseases have become rare, people have forgotten their severity, making them more susceptible to vaccine hesitancy driven by health scares.

Discussion

What do you think of "Bad Science"? Share your thoughts with the community below.