3D Models To Defeat Face Recognition

A new paper this month at USENIX Security demonstrated that widely available techniques can be used to create realistic enough renderings of faces that they can fool some security systems that recognize users’ faces [1].

As a sign of how far I am from current in these matters, I hadn’t ever heard of this authentication method, and now it is defeated.

Specifically, some systems have the option to use a phone or web cam to scan your face, which is checked against the list of authorized personnel to grant access. Obviously, this approach must reject simple ploys like holding up a photograph or wearing a mask. The basic idea is to use image cues to detect “liveness”, i.e., that this is a real, 3D person being scanned. If so, then the assumption is that the identification is valid.

What Xu and colleagues from U. North Carolina did was to collect images of a person from social media and other public sources and combine them to create a realistic 3D model using techniques developed for creating Virtual Reality content. The model is highly accurate, and can have very high resolution textures which give a very realistic appearance.

The researchers note that there are so many ways for images of people to appear on the public internet, it is likely that an attacker can obtain a sample for any target. They also note that the reconstruction techniques they use are within each of many attackers. In other words, this is a very plausible possible attack.

The question is, can such a model fool the system?

They put their 3D models into commonly use Virtual Reality software, which projects a realistic scene onto a mobile device, using its gyroscopes and accelerometers to keep the image registered with the scene or video background. If this scene is shown to the authentication system (i.e., by holding up the phone to the camera), what happens?

In their experiment, the systems were fooled in many cases!

Evidently, the VR model is “live” enough to defeat the current detection algorithms.


The researchers argue that in the current systems the “liveness” checks, at least implicitly, assume that realistic rendering is too difficult or expensive for attackers to deploy. Clearly, these assumptions are wrong.

There are, of course, ways to fix the systems so they could never be hacked this way. For one thing, image recognition should be combined with other methods (duh!). The visual detector could generate test lighting patterns (e.g., project a moving pattern of light across the face and compare to recorded samples) and infraread heat maps of the faces. And so on. Of course, these methods add cost and complexity to the system, and may be slower and less convenient for users. Such is life.

Very nice work.

  1. Yi Xu, True Price, Jan-Michael Frahm, and Fabian Monrose, Virtual U: Defeating Face Liveness Detection by Building Virtual Models from Your Public Photos, in {25th USENIX Security Symposium (USENIX Security 16). 2016, USENIX Association: Austin, TX. p. 497–512. {https://www.usenix.org/conference/usenixsecurity16/technical-sessions/presentation/xu},

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s