Generate Faces that Trick Facial Recognition with GANs in PyTorch
My goal here is to create an easily adaptable framework to generate faces that look realistic, but also trick a facial recognition classifier. The example we will work through is the task of generating realistic faces that always classify as your face — despite them not being your face (or anyone’s face for that matter).
This is actually a tricky task because it involves updating the generator in two ways.
- Update the generator to make realistic images
- Update the generator adversarially to classify as your face
As you might expect, this will require two loss functions to update simultaneously. And if you thought updating a GAN was already a delicate procedure…you have not seen anything yet.
You can imagine the update as a kind of push and pull between the two loss functions at each iteration. In one update the generator updates its weights to create more realistic faces, but if it tries to update too far in this direction it will start making faces that do not trick the facial recognition classifier. The same is true in the opposite direction, if the generator starts making images simply to trick the facial recognition classifier it might find some garbled mess of pixels that make the classifier think it’s you but doesn’t look like a face anymore.
The trick comes with keeping track of the gradient steps at each update. And for this we will need to make use of hooks in…
