mjbat7 t1_iv6ymuv wrote on November 5, 2022 at 7:56 PM

Reply to comment by DucksNQuackers in BRAIN SEROTONIN RELEASE IS REDUCED IN PATIENTS WITH DEPRESSION: A [11C]Cimbi-36 PET STUDY WITH A D-AMPHETAMINE CHALLENGE. - Biological Psychiatry by chilladipa

Hijackingthe top comment for visibility. Copying comments from last time this was posted.

Probably could just ignore this result unless it's replicated.

"BPND was significantly reduced in the HC group (1.04±0.31 vs 0.87±0.24 , p < 0.001) but not in MDE (0.97 ± 0.25 vs 0.92 ± 0.22, ns)".

Those differences are apparently p-value significant, but the error bars overlap widely in each group.

Grisward t1_iv7msd6 wrote on November 5, 2022 at 10:50 PM

Can you describe “error bars” and why they are more important to you than the P-value? What metric would you want to see?

mjbat7 t1_iv8rky6 wrote on November 6, 2022 at 3:58 AM

I used the term 'error bars' to describe the +/- values in the listed results. Frustratingly so far I haven't been able to access the original article in full text, so I can't be sure exactly what those 'error bars' represent. Presumably they're quoting the standard error in the samples, in which case, there appears to be a lot of error in their samples, which suggests that each estimate would have wide confidence intervals, which may overlap. Obviously overlapping confidence intervals don't necessarily mean that the difference isn't real, but if they overlap widely you'd certainly be quite skeptical of the results.

More specifically, let's look at some of the data in the abstract.

In the pre-treatment groups, the baseline BPND was 1.04 +/-0.31 vs 0.97 +/- 0.25. Is this a statistically significant difference? I don't know, but my guess would be that it isn't. Post exposure, the two groups' BPND was 0.87 +/- 0.24 vs 0.92 +/- 0.22. is this a statistically significant difference? Once again, I don't know, but my guess would be that it isn't.

If depressed people weren't measurably different from healthy controls in the baseline or post-treatment measures, then the fact that there is a within-group difference in the healthy controls but not in the depressed patients is much less convincing and seems more likely to be a statistical fluke.

Grisward t1_iv8ubsu wrote on November 6, 2022 at 4:24 AM

I get what you’re saying, and to be frank I appreciate the skepticism, and the drive to reality-check the numbers. Sometimes the numbers are over- or under-stated, or not reasonable effects.

In this case the abstract actually says it’s a significant change:

> “was significantly reduced in the HC group (1.04±0.31 vs 0.87±0.24 , p < 0.001)”

They include the P-value. You’re right it’s hard to know what the +/- means, typically that’s standard deviation, and not reflective of confidence intervals. Literally the confidence interval at 95% by definition would be smaller than the difference, that is if it used the same model used by the test.

It’s a whole thing about reporting and displaying confidence intervals, versus standard deviations, etc. Sometimes it’s straightforward to report standard deviation, but is not reflective of whatever statistical model was actually used. Frustrating, but not usually an author issue, also a journal guidance issue.

Oh, and the reason standard deviation does not indicate whether effect size is significant is in part that it doesn’t account for the number of individuals, nor the actual statistical model. Standard deviation is a simple university summary.