IMO the biggest problem is not the ML side, it's the people-workflow-deployment side (as other commenters have pointed out) -- a radiologist has spent 18 years of schooling plus a residency, how can you position AI to help them do their job better rather than presumptively challenge their expertise? The cost of error (particularly false negatives) is orders of magnitude higher than the benefit of getting the vast majority of true-negatives correct, what's the right way to tune the model?

Also, based on my experience in the space, there's plenty of training data out there, but outside of routine preventive scans like mammograms, many diagnoses don't have a lot of "clean" negative examples, because doctors won't order a CT unless they have reason to suspect something is wrong. This is unlikely to change, since the radiation exposure from the scan poses individual harm to the patient without providing individual benefit.