Why Psychedelic Experiences Are Difficult to Compare Scientifically

Psychedelic research methodology is the set of tools scientists use to measure something that only exists inside a person's head: what a psychedelic session actually felt like. Ask ten people to describe a psilocybin session and you will get ten different answers. Some reach for metaphors, some go quiet, one starts drawing. The experience only becomes data once someone tries to put it into words, and something always gets lost in translation.

This article stays in the lab. If you are after the botanical side, our wiki guide to magic truffle varieties and the wiki comparison of psilocybin strains cover that ground. Here we look at questionnaires, blinding, placebos and people, and at why two studies on "the same" substance can land on different planets.

Why subjective experience is hard to measure

Subjective experience is hard to measure because it only becomes data once a participant translates it into answers on a questionnaire, usually hours after the fact. Researchers rely on validated instruments: the 5D-ASC and its 11-dimension version (Altered States of Consciousness rating scales), the 30-item Mystical Experience Questionnaire (MEQ30), the Ego Dissolution Inventory and the Challenging Experience Questionnaire. Each asks people to rate statements about perception, emotion and sense of self.

AZARIUS · Why subjective experience is hard to measure

Three problems stack up. First, rating is retrospective: by the time someone fills in the form, the session is a memory being reconstructed, not a live readout. Second, language has limits. Someone who has read Huxley may describe ego dissolution very differently from someone who has not, even if the underlying state is similar. Third, each scale was built to capture particular features, such as mystical-type qualities or altered perception, and can miss others entirely. The same MEQ30 score in two trials is not automatically the same phenomenon. Our guide to psilocybin effects and what people report shows how wide that variety is.

Expectancy effects and the blinding problem

Blinding rarely holds in psychedelic trials, because most participants can tell within an hour whether they received the active drug. Muthukumaraswamy, Forsyth and Lumley (2021, Expert Review of Clinical Pharmacology) argued that effect sizes in psychedelic trials are likely overestimated for exactly this reason: participants work out their allocation, and the trial setup itself raises strong expectations. They also pointed out that most published trials never measured expectancy or how well the blind held.

AZARIUS · Expectancy effects and the blinding problem

Aday and colleagues (2022, Psychopharmacology) made a similar case in a paper titled "Great Expectations". They recommended that trials routinely measure what participants expect and whether they guessed their condition. The self-blinding microdosing study by Szigeti and colleagues (2021, eLife) shows why this matters. Its 191 participants prepared their own capsules, some containing a microdose and some empty, without knowing which they took each week. Wellbeing improved in both groups, with no significant difference between them. It does not prove microdosing does nothing, but it shows how far expectancy can move self-reports. The microdosing placebo debate article looks at that study in more depth.

Placebo controls and their limits

No placebo fully mimics a psychedelic, which leaves every comparison in the field slightly lopsided. Researchers use inert placebos (an empty capsule) and active placebos, which produce some noticeable effect so the difference is less obvious. Options have included niacin, which causes a harmless skin flush, very low doses of psilocybin, methylphenidate and diphenhydramine. Griffiths and colleagues (2006, Psychopharmacology) used methylphenidate as the comparator in their landmark psilocybin study, because its alert, stimulated feeling could plausibly be mistaken for "something".

None of these fully solve the problem. A niacin flush does not make the walls breathe, and methylphenidate does not dissolve anyone's sense of self. A low dose of psilocybin is a milder version of the same drug, which blurs the comparison in a different way. So when a paper reports that psilocybin outperformed placebo, ask which placebo, and whether participants could plausibly tell the difference.

Why participants differ so much

Participants differ so much that the same dose can produce a profound mystical-type experience in one person and mild visual drift in another. Studerus, Gamma, Kometer and Vollenweider (2012, PLoS ONE) pooled data from 23 experimental studies: 409 psilocybin sessions in 261 healthy volunteers. Dose was the strongest predictor of almost every outcome. Beyond dose, a high score on the personality trait "absorption" and an emotionally excitable state just before the session were linked to more pleasant and mystical-type experiences. Younger age and sessions inside a PET scanner were linked to more anxious reactions.

AZARIUS · Why participants differ so much

Body weight is often assumed to matter. Many recent trials give a fixed dose rather than adjusting it to body weight. A pooled analysis by Garcia-Romeu and colleagues (2021, Journal of Psychopharmacology) covered 288 participants weighing between 49 and 113 kg and found no significant link between weight and subjective effects. As a post hoc analysis, though, it is a hint rather than a final answer. Genetic and metabolic differences in how people process psilocin may also play a role, but the human data here is still thin. Small, homogeneous samples deserve caution.

How study environments shape results

The study environment is part of the intervention, not a neutral backdrop. Hartogsohn (2016, Journal of Psychopharmacology) argued that the surroundings and the participant's mindset should be treated as research variables in their own right, closely tied to the placebo response. A trial in a softly lit room with a curated playlist, two trained guides and weeks of preparation is a different intervention from an online survey about a festival last summer. Our set and setting guide covers why context matters. For psychedelic research methodology, the problem is that settings differ from trial to trial, which makes results harder to compare.

AZARIUS · How study environments shape results

Naturalistic surveys carry their own baggage, especially selection bias: people who had a rough time may be less likely to fill one in. Clinical trials, meanwhile, screen out people with a personal or family history of psychosis. That produces cleaner safety data but limits what the findings say about everyone else. Neither design is wrong. They answer different questions. If you want to understand difficult sessions specifically, our guide to preventing and managing challenging experiences is the place to start.

What this means for reading psychedelic research

Good psychedelic research methodology is easy to spot once you look past the headline to a handful of unglamorous details. Here is the short checklist we use:

1. Sample size. A trial with a few dozen participants is a signal, not a conclusion.
2. Blinding check. Did the authors ask participants to guess their condition? If not, assume many could.
3. Comparator. Inert placebo, active placebo or waitlist? Each tells you something different.
4. Funding and conflicts. Industry funding does not make a study wrong, but it is worth noting.
5. Preregistration. Was the main outcome fixed before the trial started, or chosen afterwards?
6. Uncertainty language. Does the paper state its limits, or read like a press release?

Our psilocybin clinical trials research overview applies this lens to the best-known trials, and our blog on psilocybin and the brain covers the neuroscience. None of this means psychedelic science is broken. The field is young, its tools are blunt, and the honest answer to "does it work?" is often: the signal looks interesting, and here is what we still don't know.