What none of these trials can tell us yet

Started 18 January 2026 · 18 replies · 133 hearts · read 1.9k times

PatrickLimerick, IE · 18 January 2026

6 months ago

Starting a thread on this because it keeps coming up in ones and twos: What none of these trials can tell us yet

Evidence question. I would rather have an honest "we do not know" than a confident answer.

The claim I keep seeing is stated with enormous confidence and I cannot find a single primary source for it anywhere.

I have a practical question that turns entirely on a technical detail, and I have not been able to get a straight answer to the technical detail.

Putting this here because I do not want to be sold anything. I want to know what is actually established and where the evidence runs out.

Thank you for reading. Careful answers welcome; confident ones I can already get elsewhere.

AmmiKarachi, PK

I would encourage anybody nervous about joining to come and say nothing for a few sessions. That is what I did for four months.

OrphBath, UK

What I have taken from this group is mostly humility about how much is genuinely settled.

9me too 7

GwilymCaernarfon, UK

Something worth knowing: the difference between a trial and an observational study changes what you can conclude, and I did not know that two years ago.

SunilFacilitatorBirmingham, UK

A standing request in this circle: when you bring a paper, say what the methods section actually did. It is the part everybody skips and the part that decides what the paper is worth.

OskarWarsaw, PL

One paper a fortnight, read properly, has taught me more than three years of reading summaries of papers.

8me too 2

TildaCambridge, UK

That is a very clear way of putting something I have never been able to put clearly.

PerrineStrasbourg, FR

Tilda has already said most of this and I want to add the one part that was different for me.

Nothing here is medical advice, and a paper is not a recommendation for any individual person including the ones in it.

TheaBergen, NO

The session where we read a paper about something none of us takes was the most useful, because nobody was defending a position.

GilTel Aviv, IL

My experience is that the group is much better at this than I am alone, because somebody always spots the thing I skimmed.

DevManchester, UK

#10

Coming back to what Thea wrote further up, because it is the bit I keep thinking about.

Something practical: I write three sentences about each paper afterwards and I now have two years of notes I actually refer back to.

TiborBratislava, SK

#11

Coming back to say the fortnightly rhythm has held for two years and it is the thing on this site I would least like to lose.

HattieExeter, UK

#12

Building on what Dev wrote rather than answering the original question.

The thing I look at first now is who was actually in the study, because that is usually where the answer to my question lives.

8me too 3

BeaDublin, IE

#13

I have read this twice now. Thank you for writing it.

NadjaBelgrade, RS

#14

What I found is that the abstract and the results section quite often tell slightly different stories, and the difference is the interesting part.

PrunellaBury St Edmunds, UK

#15

I had to learn that an effect being statistically significant and it mattering to a person are two entirely separate questions.

PádraigGalway, IE

#16

Something practical: I read the methods before the conclusions now, which somebody here suggested and which changed how much I trust what I read.

SunilFacilitatorBirmingham, UK

#17

Facilitator note: this circle is not a place to arrange anything with anybody, and it never has been. It is a place to read carefully.

10me too 1

PetuaKampala, UG

#18

Sunil, thank you for that. It gives me somewhere to start from.

What I would say is that an honest "we do not know yet" is worth more than a confident answer, and the papers say it more often than the coverage does.