In this post, we will continue our intro into designing for AR by adding our models into the demo AR app and apply lighting, shadows, and animations!
This post is from the multi-part series on ARKit, where we talk about designing for AR, building a demo app, exploring and testing many of the features of ARKit. We previously wrote on getting started with ARKit .
A rather important thing to mention is that there’s a small amount of coding involved in designing for AR. Despite the fact that this might sound scary to some, I believe it’s our responsibility as designers to account for the entire lifespan of our creations through the journey of building a product, even though they might be far from our immediate reach. I’m strongly in support of going the extra mile and making an effort to learn new working methods, tools, or even a programming language, in order to ensure perfect delivery of our designs.
Lighting the model in Xcode
If you've followed our previous post, by now you should have a running AR demo app. Let’s start by copying the .dae file as well as the UVW map into the art.scnassets folder. If the files are exported correctly, we should see our model in Xcode, along with its texture, in a 3D environment similar to the one of C4D.
This is the time to add a light source to our object. In Xcode’s Object Library panel on the bottom right, you can see a few types of light sources that you can add to your scene. A Directional light shining from 80–90 degrees overhead seems to be the most realistic one. In addition to the main Directional light, you may choose to add a secondary Directional light to illuminate certain parts of the object that aren’t directly illuminated by the main Directional light, in order to avoid completely black areas. Adjust the intensity of the light source until you get something that looks relatively realistic.
A crucial part of making virtual objects appear as if they’re part of the real world is to include shadows. Drop shadows—i.e. shadows that are cast onto the surface where the object is resting on—is what makes it look tangible and creates the illusion that it is in fact anchored to a surface.
To add a shadow, while still having the main Directional light selected, go into the Attribute Inspector panel and make sure Casts shadows is enabled, and the mode is set to Deferred.
So now that the light source can cast shadows, we need a surface on which to cast these shadows. From the Object Library, add a Plane object to your scene, which essentially is a surface without any thickness. Let’s adjust the size and placement of the Plane object to be right under your model (or put 0, 0, 0 on the object’s xyz coordinates), and now we should see its shadow cast on the Plane. Now there’s a very obvious problem; the Plane object has a texture, but the end goal is to have our model’s shadow cast on whatever surface is resting on, not a surface embedded in the model itself.
Fixing that is as easy as going to the Material Inspector while having the plane selected, and simply unchecking all color channels from Write To Color property under the Transparency settings. In addition to that, choose Constant on the Lighting Model dropdown menu. Now we should see a shadow under the 3D object while the surface is invisible, which is what we were going for. The surface might also appear darkened, but this effect won’t be visible in AR.
Xcode might ask you to convert the .dae file to a .scn file. That’s okay for now, but it might prove to be a problem if your models also animate. However, there is a way to fix that. We’ll be exploring that in a bit.
The default settings will cast a hard shadow on the surface, or in other words a shadow with clear lines. A more diffused shadow will look more realistic in most lighting conditions. To adjust diffusion, go back to the Attributes Inspector while having the Directional Light selected, and adjust the shadow sample radius and sample count until you get something that looks good. You can also decrease the transparency of the shadow colour for an optimal look.
Lighting based on environment
The final step to achieving basic realism with ARKit is lighting your model in accordance with the environment. Imagine if you placed an object in a relatively dark room—the object would look quite dark too, right? The opposite goes for objects placed in a bright environment. This is something we have to take into account when designing for AR.
The ARKit API calculates and provides a Light Estimate value, which is the estimated luminance of a frame in lumens.
self.sceneView.autoenablesDefaultLighting = false; let estimate: ARLightEstimate! estimate = self.sceneView.session.currentFrame?.lightEstimate
Now that we have an estimated luminance value of our real world lights, it’s easy to adjust the intensity of our virtual light source. This way we ensure that there’s consistent lighting between the model and the environment around it.
let light: SCNLight! light = self.lightNodeModel?.light light.intensity = estimate.ambientIntensity
The model is now ready to be placed in the real world!
Everything that has been mentioned above applies to .dae files that include animations in them. And that's great… until we try to add a plane and Xcode asks us to convert the file to .scn, and as soon as we accept, the animation doesn’t play anymore. For some reason, the .scn format isn’t maintaining the animations of the .dae file. So we have to find another way to add our light sources and shadows, while still maintaining the animations of the model.
A fairly small amount of coding is required. Adding a light source and plane to your .dae file won’t require us to convert the file to .scn, but altering these objects’ properties will. Who’s to say we can’t do that programmatically, though?
let plane = self.planeNodeModel?.geometry! plane?.firstMaterial?.writesToDepthBuffer = true plane?.firstMaterial?.colorBufferWriteMask =  plane?.firstMaterial?.lightingModel = .constant
This piece of code is the equivalent of unchecking all colour channels from the drawing the texture of the plane and setting the lighting model to constant, thus drawing only the shadows of any objects cast on it. For that to work, the plane must have a material applied to it. C4D doesn’t apply a material to a new geometry by default, so make sure you apply a basic material, (it doesn’t matter how it looks, as it’s not going to be visible).
Your model is now lit properly, anchored to the surface realistically, while also doing all sorts of crazy dance moves.
Now we've learned everything you need to know in order to design for a basic AR app. Previously we learned to create 3D models and export them properly, and now we're able to add them in an AR app and place them in the world adequately.
Any comments or questions? Hit me up on Twitter @BashaChris