[D] Interpolation in medical imaging?

Submitted by Delacroid t3_y4kvq2 in MachineLearning

I have been wondering if there has been research on the field of interpolating between slices of medical imaging procedures.

For example taking a brain MRI and trying to predict an intermediate slice given the other two surrounding ones as inputs.

I imagine that a generative model like a cGAN would be useful for this context. After a dive on the literature I haven't been able to find good articles on the topic, however my background is not in ML.

Thanks in advance

14

Comments

You must log in or register to comment.

eigenham t1_iseq5zz wrote

Medical imaging community doesn't like when you make up new data (which makes sense when you think about the use case). That said, sure there's work on interpolation, but probably a lot of what you're looking for is hiding in the literature as "super resolution imaging". There's a bunch of hand wavy work and a few groups doing really good validation studies (just look for the authors from the biggest and most famous institutions, because the sad truth is you need money and resources to properly validate).

35

tdgros t1_isey85z wrote

https://arxiv.org/pdf/2209.07162.pdf This recent paper released a dataset of 100k brain MRI images generated with a diffusion model. So things are moving a bit...

7

eigenham t1_isfkldi wrote

Yeah that's definitely one of the bigger groups/institutions in this field. You can expect groups like theirs to push the bounds early, and only some of those efforts gain enough traction for actual acceptance in clinical research/practice, so while this is a good effort, the real sign of movement will be when these methods start showing up in clinical journals

2

Delacroid OP t1_iseuqrb wrote

Thank you very much because I didn't know that super resolution also dealt with interpolating. I thought it was only for improving quality of image as going from 360p to 720p. I will try to use the term in my search and see what I get.

2

freezelikeastatue t1_iseurai wrote

Excellent summation! However, you do draw an interesting point, don’t use it to make up data, use it to predict growth, especially for tumors in the brain. I’m not sure you have the proper data sets to feed a model to predict biological growth, but I think that would be an application for your specific use case provided above.

2

tdgros t1_isez24z wrote

Filling in a missing slice could be called an "inpainting problem".

There is this line of work that should fit your description: https://arxiv.org/pdf/2202.04200.pdf (there are older similar approaches as well). There are approaches using GANs as well. I can't say if they're popular for medical imaging data, but they're quite general.

5

Red-Portal t1_isg2apg wrote

Interpolation is quite different from inpainting. Inpainting is about filling out information that is outright missing, but super-resolution and interpolation is about filling out only the missing "high-frequency information."

1

tdgros t1_isg3vwi wrote

You are welcome to call it what you want, I'm pretty sure you see the similarities and why I suggested maskGIT.

0

Red-Portal t1_isg4hjv wrote

No I don't? Because the usual methods used for frame interpolation or super-resolution are not only quite well established but completely different.

1

tdgros t1_isg5iy8 wrote

In OP's setting, imho you can use the term you want: inpainting because it's a large missing area, SR because some people see SR as filling in new rows and columns (I don't, I prefer to see it as inverting the lens degradation) and interpolation because it just means "adding things between other things", at least in my native language. I'm not sure what usual methods you are referring to, but you could suggest them to OP!

0

Delacroid OP t1_iseztbz wrote

Thanks, I never thought about the analogy with an impainting problem.

Edit: grammar

0

radio_wave t1_isga74f wrote

Interpolation happens all the time in medical imaging data, but it is most often within the image and not across slices. Filling a missing slice between two slices is not routinely done. There is some work where you collect thick slices and attempt to create multiple thinner slices from them.

3

marcus_hk t1_isgbpv9 wrote

If you have a dense 3D image, as in CT, then there is really no distinction between "within image" and "across slices" because these are the same thing, just along a different axis. Of course with sparse MRI slices, though, you're right.

2

radio_wave t1_isgct2o wrote

Yup, modern CT and a fair bit of MRI is fully 3D imaging where this is less of a consideration. But something like 60 or 70% (anecdotal stats, am an MR physicist) of MR imaging is 2D slice by slice imaging where there will be value for some better interpolation approaches. And even for 3D imaging, non-isotropic scanning (typically thicker slices compared to in-plane resolution) is clinically common to keep scan times short and interpolation could be handy

1

TimelyStill t1_isin6sh wrote

For anatomical imaging where you want to estimate the size or growth of a structure of known shape it's reasonably common to perform simple interpolation between slices. There's also a lot of research being done regarding superresolution imaging and compressed sensing, and there are commercially available sequences for both.

It's also important to keep in mind that magnetic resonance imaging isn't acquired in the same three-dimensional space we're used to, but in k-space, which is the Fourier transform of the image measured. It's very common to acquire only, say, the first 70% of the lines of a k-space grid and then either mirror the first 30% of lines (since k-space is largely symmetrical and has the most important pixels concentrated in its center) or to zero-fill a border around the acquired image. Underampling is typically performed in this way rather than skipping entire slices.

For standard anatomical MRI in humans the resolution problem is largely 'solved' since 3D pulse sequences at good resolutions don't take that much time. For diffusion imaging with high angular resolution and fMRI it's a different question, and many undersampling methods are often combined. Parallel imaging, k-space undersampling, slice interpolation....With diffusion imaging there's also the possibility to interpolate in the 'angular domain'.

So to answer your question briefly: there is continuous research being done to shorten the duration of MRI sequences, but understanding said research requires a lot of domain knowledge since, unlike optical imaging, MRI images are acquired in the frequency domain, and often have one or more other dimensions (angular, relaxation time, diffusion coefficient etc).

1