Using GANs to generate defective data

Submitted by Tekno-12345 t3_11qanvr in deeplearning

Hey guys,

I'm working on implementing a model to detect defects on the labels of bottles.

The model should be able to spot bubbles, folds, and miss-labels.

But I'm short on actual defective data, so I'm thinking about making some artificial data using GANs.

I gave it a shot with simple image processing, but the model couldn't generalize well.

Got any ideas or suggestions on how I could make this work?

Would really appreciate some help.

​

Bubble example

11

Comments

You must log in or register to comment.

mcottondesign t1_jc2dygc wrote

You can increase the amount of your existing defect images but flipping, rotating, cropping the images in a pre-processing step.

It isn’t a perfect answer but it is a great way to augment the limited data you already have.

7

gradientic t1_jc430rz wrote

Not really direct answer to your question, but the general problem you are trying to solve is called image anomaly detection, there are well know approaches that try to solve this problem, some of them in an unsupervised manner (assuming that you have a significant dataset of images without anomalies - learn inlier look for outliers) - check https://towardsdatascience.com/an-effective-approach-for-image-anomaly-detection-7b1d08a9935b for some ideas and pointers (sry if pointing obvious things)

5

Tekno-12345 OP t1_jc5vjac wrote

My problem with AD models is that they are very sensitive, even if trained on noise.
Any small diversion will be detected as defective.

1

HarissaForte t1_jc6sser wrote

> Any small diversion will be detected as defective.

If this small diversion is smaller than the diversion within your train dataset (+ the noise) then it should not be detected.

Did you try different splits or a CV ?

1

Tekno-12345 OP t1_jc9oymg wrote

You're right, I'll try to incorporate more representative data.
I'll also try different splits
Thanks

1

Kuchenkiller t1_jc4t81a wrote

If you have enough images of defects but are just lacking the labeling (probably easier to come by) one approach is to generate random morphological structures on your bottles (e.g. just some random circles and ellipses) and then apply cycleGAN or CUT to transform from this "segmented" image domain to the domain of real images. As said, you still need a lot of data but don't need labelling. Just generating useful data from noise (basic GAN idea) can work in theory but is extremely hard to train. I had way more success with the domain transfer approach (my case in medical imaging)

2

Kuchenkiller t1_jc4tir0 wrote

Also, what is the reason for deep learning if you don't have the data? Have classical methods been unsuccessful?

3

Tekno-12345 OP t1_jc5wf2t wrote

Thanks for the reply.
Yes classical methods did not generalize well on foreign data.
I don't have experience on GANs and wanted some experienced opinions.

I have some leads now, I'll try them out and get back to your comment if they did not work out.

2

HarissaForte t1_jc66t3m wrote

> a model to detect defects

… as in object detection?

1

Tekno-12345 OP t1_jc6drqj wrote

It could be..
However, such models require the data I am seeking

1

HarissaForte t1_jc6gder wrote

I don't get it… what do you have so far?

How many of

  • image alone,
  • image + labels,
  • image + bbox,
  • image + mask ?
1

Tekno-12345 OP t1_jc6jl0q wrote

I have a lot of good images with no defects.
And very few images with defects.

In order to train a good object detector for this task, a huge amount of defects will be needed.

1

HarissaForte t1_jc6qhid wrote

Well first it really is a use case for anomaly detection, but I see there's a discussion on it in another comment.

I still don't know if you have (classification) labels, bbox, or masks for the images with defect… if you do not have any bbox or mask, I suggest you try creating mask annotations, as this will increase the "power" of each sample (instead of 1 label per image you get HxW labels for each pixel location).

It should be fast as:

  1. You say you do not have many images with defect.
  2. It seems the mask for defect-less bottles can be generated with a simple thresholding.
  3. Looking at your example you can create a bubble annotation in one click with a proper annotation tool (I can't tell for the other defect)

Then you ca try a simple supervised training. You could have a nice surprise since your picture are taken in a controlled, standardized environment (it's not like you'd have pictures of bottles on a beach or in the jungle).

If not then you will be able to use the segmentation data to create patches of defects that you can use on your defect-free bottle to create new data. It will be much easier than messing around with GANs and might give you good results.

1

Tekno-12345 OP t1_jc9pgbl wrote

Thanks for the good ideas, i'll try them out.
What do you mean by the last part? How can I create patches of defects using the segmentation data?
You mean by classical methods?

1

HarissaForte t1_jc9s369 wrote

For example you use a bubble mask to "cut" out a bubble patch from an image. Cutting can be a mix of rectangular selection + transparency where the mask equals zero.

Then you do some random change (flip, aspect ratio…) to this patch.

Then you take a defect free image, and you randomly choose a location where to paste this defect using the bottle mask. You can paste by simply over-writing on the image, or you can do a linear interpolation between the patch image and the defect-free image so it's smoother.

I assume you have very similar images, since they're from a standardized inspection process. Also the bubbles are a very simple pattern. So this could work quite well.

1

Tekno-12345 OP t1_jc9zmeh wrote

Ow I see,
I have already done something similar but the results were not convincing.
Maybe I'll try it again using the masks.

1