HDRI

Radiance Map

Introduction

An important step of HDRI is the creation of a radiance map that contains the scene radiance (light intensities of points in the scene) as compared to superficial "pixel values". To understand the basic principle of this, let us first take a look at some vocabulary.

Radiometry/Photometry and Light

In radiometry, radiance is the amount of light emitted from (or passing through) a certain area and going into a certain direction, so scene radiance is the amount of light emitted (or reflected) from a visible object point of the scene and going into the direction of the camera. Irradiance is the amount of light that hits a certain area of a surface such as the image sensor of a camera (sensor irradiance). The sensor exposure is the amount of light that falls on a certain area of the sensor in a certain period of time, the exposure time. These terms relate as follows:

sensor exposure = sensor irradiance * exposure time

Sensor exposure is what the image sensor actually measures. This is done by converting the photons of light into electrons that are collected in the photo diodes until the electric signals are read from the diodes and converted into digital values.

The terms luminance/illuminance are found in the field of photometry and basically mean the same as radiance/irradiance, light intensities, but with respect to how the human visual system perceives them.

Basic Idea

We want to create a radiance map, an image with scene radiance as values for the pixels. What we have is a series of images of the scene with varying exposure times. For now, let us assume that sensor exposure and corresponding pixel values of the resulting image are proportional. This is what we did for our first, simple solution.

Using this assumption, the corresponding sensor irradiance for a given pixel can be calculated (using the formula above) by dividing the pixel value by the exposure time. The result is a value that is proportional to the irradiance, which is in turn proportional to the radiance of the corresponding point. So, if we do this for all pixels, we receive an HDR image where all pixel values are equal to the scene radiance (except for a scaling factor): the radiance map.

Basic Algorithm

Due to the small range of values (0..255) in the given LDR images, the results of the calculation above are not very precise. And values of 0 or 255 do not even contain any useable information, because those pixels are under- or overexposed as their irradiance was under or over the measurable threshold and we do not know by how much.

In order to get better values, we calculate the average of all radiance values for a certain pixel of all LDR images, while ignoring over- and underexposed ones. This is shown in the pictures below: We want to calculate the radiance of the scene at the pixel position marked with a red border and we have three LDR images. The respective pixel of the third image is overexposed and ignored, while the radiance corresponding to that pixel in the first and second image is calculated. The result is the average of the two values.

There are some other approaches to cummulating the computed radiance values at one pixel position, such as various weighted average methods.

Results

The following images are LDR source images with exposure times ranging from 1/10 second to 8 seconds:

This is the resulting radiance map, shown with a simple tone mapping technique: linear contrast compression. All HDR values are mapped to the LRD range of 0 to 255, the smallest to 0, the greatest to 255 and everything else in proportion. The second image has an additional gamma correction of 0.2, the third 0.3.

For more sophisticated approaches to showing an HDR image on an LDR display, take a look at tone mapping.

Non-Linearity

For the basic algorithm, we assumed a linear relationship between pixel values of LDR images and corresponding exposures. Unfortunately, this is not actually the case. In reality, due to various technical reasons, this relationship is not linear, as we can see on the graph at the right hand side. This graph (taken from Debevec97) shows the opto-electronic conversion function (or camera function for short) of a specific camera (DCS460).

For a radiance map that represents the real scene radiances, this non-linearity must be taken into account. First, the camera function must be calculated in a preceding calibration step. Then the inverse of this function must be used in the creation of the radiance map. This is explained in the aforementioned Debevec97 paper.

Originally, we planned to implement this in our solution, but because of time issues, we had to settle for the simple algorithm explained above.

Image Alignment

Another problem ignored so far is the aligment of images. The radiance map algorithm assumes that one pixel coordinate corresponds to the same world coordinate in all LDR images. This is only true, if the camera does not even slightly move between the shots. With a hand-held camera, this is virtually impossible, but even with a tripod, it is not trivial, because touching the camera (to release the shutter) or wind can induce small movements.

For best results, the images must be aligned to meet the assumption of corresponding pixels. This is a well known problem of computer vision involving finding corresponding points and applying appropriate transformations. Several HDR-specific solutions have been proposed, such as using Mean Threshold Bitmaps (Ward03), a kind of feature detector invariant to differences in exposure time.

Object Movement

Besides camera movement, object movement also leads to problems. When an object moves between the shots, then, for a certain position, the object will be present in some images and absent in others. The radiance map algorithm will then calculate the average radiance, resulting in a mixture of background and object radiance, thereby creating transparent, ghostly objects called ghosting artifacts. A number of solutions have been proposed to reduce or remove such artifacts.