Viewing a single comment thread. View all comments

elehman839 t1_jdmt4om wrote

Reply to comment by The_One_Who_Slays in A recently submitted paper has demonstrated that Stable Diffusion can accurately reconstruct images from fMRI scans, effectively allowing it to "read people's minds". by iboughtarock

The claims are interesting, but far more modest than people here seem to realize. This is what they say about their evaluation process:

we conducted two-way identification experiments: examined whether the image reconstructed from fMRI was more similar to the corresponding original image than randomly picked reconstructed image. See Appendix B for details and additional results.

So, if I understand correctly, they claim that if you take a randomly-generated image and an image generated by their system from an fMRI scan, then their generated image more closely matches what the subject actually saw than the randomly-generated image only 80% of the time.

This is statistically significant (random guessing would give only 50%), but the practical significance seems pretty low. In particular, that's waaaay far form a pixel-perfect image of what you're dreaming. The paper has only cherry-picked examples. The full evaluation results are apparently in Appendix B, which I can not locate. (I'm wondering wether the randomly-generated images had some telling defect, for example.) Also, the paper seems measured, but this institution seems to very aggressively seek press coverage.

4

The_One_Who_Slays t1_jdn2ac5 wrote

I mean, Rome wasn't built in a day. Just the fact that it's possible to do speaks volume. As for seeking press coverage, it's understandable: could be them trying to secure more funding by getting more publicity, could be them being genuinely passionate about their tech, could be both. The time will tell.

Still, it's an interesting application for the image gen technology, it's never even crossed my mind to my surprise.

2

elehman839 t1_jdollr0 wrote

If anyone cares: I found Appendix B, but there wasn't much more helpful information. In particular, I don't understand how the randomly-generated images in their evaluation process were produced. And, as far as I can tell, the significance of the paper comes down to that detail.

  • If the randomly-generated images were systematically defective in any way, then the 80% result is meaningless.
  • On the other hand, if these randomly-generated images are fairly close to the image shown to the person in the fMRI-- but just differing in some subtle ways-- then 80% would be absolutely amazing.

Sooo... I think there's something moderately cool here, but I don't see a way to conclude more (or less) from than that from their paper. Frustrating. :-/

2

The_One_Who_Slays t1_jdoodjv wrote

Yeah, some public trials would come in handy there. Show, don't tell, and all that.

1