A peer support community, independent and not for sale. Since February 2024.

GLP CircleA peer support community

Research reading group

A small study that got enormous coverage and should not have

Started 23 December 2024 · 21 replies · 262 hearts · read 2.8k times

FabianBasel, CH · 23 December 2024
1.6 years ago

A small study that got enormous coverage and should not have — putting it in its own thread so it does not get lost in the check-in.

Evidence question. I would rather have an honest "we do not know" than a confident answer.

Putting this here because I do not want to be sold anything. I want to know what is actually established and where the evidence runs out.

I would like to understand the limits of this properly rather than the headline, because the headline is the part everybody already knows.

I will summarise whatever the thread concludes and add it to my notes, in case it is useful to somebody later.

87

JunoBrighton, UK
#1

The registry entry is worth checking against the published paper, and on one occasion the outcomes we were reading were not the ones originally listed.

5

SunilFacilitatorBirmingham, UK
#2

Facilitator note: an honest "we do not know" is a better answer here than a confident one, and this circle exists to make that answer sayable.

23me too 5

DevManchester, UK
#3

This is the post I wish had existed when I was looking for it.

14

OskarWarsaw, PL
#4

Sunil, thank you for that. It gives me somewhere to start from.

Somebody here asked what would change our minds, at the start of a session, and it improved the discussion enormously.

11

GilTel Aviv, IL
#5

My reading has got slower and better. I get through a quarter of what I used to and I understand considerably more of it.

11

HattieExeter, UK
#6

Building on what Dev wrote rather than answering the original question.

Something worth knowing: the difference between a trial and an observational study changes what you can conclude, and I did not know that two years ago.

10

PrunellaBury St Edmunds, UK
#7

What I have taken from this group is mostly humility about how much is genuinely settled.

15me too 3

TarqWindsor, UK
#8

I would encourage anybody nervous about joining to come and say nothing for a few sessions. That is what I did for four months.

10

NilsUppsala, SE
#9

What helped was somebody explaining confidence intervals to me properly, once, in plain terms, and I have read differently ever since.

10

PetuaKampala, UG
#10

That is a very clear way of putting something I have never been able to put clearly.

12

NadjaBelgrade, RS
#11

Reading and nodding. Nothing to add, and I wanted you to know somebody had read it.

6

RuthieLeicester, UK
#12

I had to learn that an effect being statistically significant and it mattering to a person are two entirely separate questions.

11

SaoirseKilkenny, IE
#13

What I would say is that an honest "we do not know yet" is worth more than a confident answer, and the papers say it more often than the coverage does.

10

PádraigGalway, IE
#14

The trial that gets quoted most in general coverage is not the one that answers most of the questions people here actually have.

10me too 9

PamSouthport, UK
#15

One paper a fortnight, read properly, has taught me more than three years of reading summaries of papers.

15me too 5

JuniperMachynlleth, UK
#16

What Pam described happened to me too, in a slightly different order.

Coming back to say the fortnightly rhythm has held for two years and it is the thing on this site I would least like to lose.

12

AnnaAthens, GR
#17

Pam, thank you for that. It gives me somewhere to start from.

What I found is that the abstract and the results section quite often tell slightly different stories, and the difference is the interesting part.

16

BeaDublin, IE
#18

The thing I look at first now is who was actually in the study, because that is usually where the answer to my question lives.

9

GwilymCaernarfon, UK
#19

The session where we read a paper about something none of us takes was the most useful, because nobody was defending a position.

11

SunilFacilitatorBirmingham, UK
#20

Noticing the drift from what the evidence shows to what people believe. Both are interesting and only one of them is what this circle is for.

26me too 1

TheaBergen, NO
#21

Something practical: I read the methods before the conclusions now, which somebody here suggested and which changed how much I trust what I read.

15