Matching real world with CGI

"How we can composite the 3d object with an still image or a video footages with exactly the same lighting and composition"

I think Ambient Occlusion is a good solution for achieving this particular goal. For example immense amount of planning and R&D must be required in order to match the real scene lighting setup to CG lighting set up and if we are matching models then again there is lot of planning is required.

Now industry is shifting towards IBMR for the same. (Image-Based Modeling and Rendering) IBMR method is pretty simple in concept it takes reference from set of photographs and with complex algorithms it generates final image this technology is still at its primitive stage though its gaining more popularity today. IBMR methods gave more importance to lighting than the actual shape of the model. And there is lot more happening in IBMR which helps the artist to match real world lighting in CG world. Almost all major VFX blockbusters in Hollywood use IMBR in their pipeline

Ref: http://www.debevec.org/Items/NewScientist/

http://research.microsoft.com/china/papers/Review_Image_Rendering.pdf

So lighting, as you know it is very important in geometry based modeling when it comes to compositing them to the CG scene or the real footage. Lighting in Computer is darker than real world and it doesn’t show the highlights clearly. So with these lighting normal lighting setup our models will look flat (will have that soft plastic feel) so on SIGGRAPH 2002 our good guys at ILM presented technique called ambient occlusion with this technique we got control over all the attributes of the scene or object and we may adjust the levels of each in the post according to our needs

Here you can see the advantages very well!!!

Ref: http://www-viz.tamu.edu/students/bmoyer/617/ambocc/

In lighting Context, as you may know one of the new and exciting technology is HDRI (High Dynamic Range Images) it is basically a technique where the ratio between brightest and darkest regions are stored in one single image file. This information is very useful for faking the real world lighting in 3d app and even AE 7 supports HDRI. There are lot of movies which had used this technique for example Spider man 2, Batman begins, Troy and lot more.

Ref: http://en.wikipedia.org/wiki/HDRI

http://www.debevec.org/Research/HDR/

Studying real world lighting is really challenging job because in Matrix sequels technical directors hired $700,000 machine which is actually used by US Defense to calculate how real world light acts on Agent Smith’s highly anisotropic silky cloth material in order to recreate the same on CG. They calculated BRDF (Bidirectional Reflectance Distribution Function) BRDF is calculated because same model or scene reacts differently in lights sources from different positions and angles

BRDF (It gives the reflectance of a target as a function of illumination geometry and viewing geometry. The BRDF depends on wavelength and is determined by the structural and optical properties of the surface, such as shadow-casting, multiple scattering, mutual shadowing, transmission, reflection, absorption and emission by surface elements, facet orientation distribution and facet density.)

For more information

Ref: http://www-modis.bu.edu/brdf/brdfexpl.html

And another example is for creating Doc Ock's CG skin in spider man 2 Sony image works created Light Stage system which captures reflectance field of a human face photographically. This resulting image was to used for model under different lighting condition and from various light points and angles and this same technique was used for creating various HDR environment maps.

After lighting the next important factor is dept which calculating 3 dimensional information from 2 dimensional images for this purpose a method called technique called Photogrammetry is used. In this technique different points on an object are determined by measurements made in two or more photographic images taken from different positions. Hollywood movie Fight Club is perfect example for this technique using this technique high speed high-speed photo-realistic camera movement down the side of, around and through buildings, as evinced at the film's beginning and also in the 'galaxy of trash' pull-back. And finally Photogrammetry and HDR are very closely related subjects Pixel Lieberation Front ( PLF ) used the Photogrammetry side of the initial research very effectively in films such as Fight Club.

Ref: http://www.univie.ac.at/Luftbildarchiv/wgv/intro.htm