Jan Vermeer (1632–75) is widely considered one of the most important painters of the Dutch Golden Age and his Girl with a pearl earring in the Mauritshuis in The Hague has long been praised for its freshness, immediacy, and—our central interest—its effects of light. We ask perhaps the simplest technical questions about the light: What is the direction of illumination? How accurately or consistently did Vermeer render the effects of light? Below are several techniques used to address such questions, presented in this video and this paper:
Model-independent algorithms do not require any knowledge or assumption about the three-dimensional form or model of the scene in question. The two model-independent methods we explored are cast-shadow analysis and occluding-contour analysis.
The simplest method for inferring the direction of illumination is cast-shadow analysis, that is, drawing a straight line from a point on a cast shadow through the corresponding point on the occluder. This line, extended, should pass through the two-dimensional position of the illuminant. If there is a set of such cast shadow lines in normal configuration (i.e., not all parallel), the lines will meet at the position of the illuminant. For Girl with a pearl earring, about the only usable cast shadow is that due to her nose.
The occluding-contour algorithm uses the pattern of luminance along the outer boundary or "occluding contour" of a three-dimensional object to infer the direction toward the illumination. The central insight underlying the algorithm is that along an occluding contour the normal to the surface is known exactly: it is perpendicular to our line of sight and perpendicular to the contour. Like a flagpole on the horizon.
The algorithm gives reliable results if and only if the object is Lambertian (diffusely reflecting like cloth or skin, not shiny and specular like glass or metal), is of constant reflectivity or albedo, and for which the light source is fairly small and distant so that the light striking a contour segment is nearly parallel.
Model-dependent algorithms require knowledge or assumptions about the three-dimensional form or model of the scene in question. Some are weakly model dependent, for instance we can assume quite naturally that the pearl had cylindrical symmetry. Others are strongly model dependent, for instance the full computer graphics model of the face, nose, eyes, cheeks, costume, and so on.
We assumed, naturally enough, that the pearl had cylindrical symmetry and sampled the contour of the pearl in the painting and then created a three-dimensional model having that contour. Then we adjusted the position of the virtual illuminant until the rendered virtual pearl appeared as similar to the painting as possible. The location of the illuminant was, in this way, our best estimate of the position of the illuminant in Vermeer's studio.
One can infer the direction of illumination by the position of the glint off an eyeball. This has been used in forensic photography where if the position of glints for two figures differ significantly, then one of them was likely composited into the photograph. Because the cornea is a bump on the eyeball, we must first determine the direction of this bump. We do this by fitting an oval to the iris (colored) part of the eye. For example, if that oval is a perfect circle, then the eye is pointing directly toward us (or the camera or the artist). This is the case for the Girl with a pearl earring.
There are techniques from computer vision called "shape from shading" that relate the three-dimensional shape of a face to the two dimensional image for a given lighting distribution. These algorithms can work "backward": If one has a three-dimensional model and the two-dimensional image, one can infer what direction of illumination was used when creating that image. For Girl with a pearl earring, however, we don't have such a three-dimensional model. What should we do to estimate the directly of illumination? We took a generic three-dimensional face model often used in computer vision research. We temporarily assumed this was the shape of the girl's face and computed the direction of illumination that best explained the image we find in the painting. Of course this direction will be in error because this is not her true shape of her face. So we next assumed the direction was right and then "fixed" the albedo on the model so as to better approximate the image found in the painting. This is an improved model of her face. Then we used this improved shape model to re-estimate the direction of illumination. This is an example of the so-called expectation-maximization (or EM) algorithm. (Technically speaking, it is the generalized expectation-maximization algorithm.)
We built a full three-dimensional computer graphics model of the girl's face, headdress, smock, and so on, and adjusted the direction of virtual illumination until the rendered image matched the painting as closely as possible. The direction of virtual illumination is thus an estimate of the direction of illumination in Vermeer's studio.
The excellent agreement of the direction of illumination estimated by our six techniques testifies to Vermeer's mastery in rendering the effects of light and the reliability of our methods themselves. They also strongly argue that Vermeer had an actual human figure before him, that is, that he wasn't painting from memory.
Suppose there were excellent agreement between the estimates for one of Vermeer's paintings, but disagreement in another painting. That would suggest that the second painting was executed under varying studio conditions, or without an actual model present. Or suppose five of our methods agreed closely, but one differed significantly, in particular, the direction from the pearl. That would suggest that the pearl was added later, perhaps by a different artist or under different studio conditions.