]

Thorne Brandt

Mobile Menu

Nums

Aura Reading

Kinect Installation

December 16th, 2016

This was an installation for Digital Art Demo Space as part of the pre-show for Nothing Up My Sleeve show that would project a unique distinctly textured 'aura' onto each individual person and allow Hannah Simon Kim to provide personalized interpretations of the shapes.

This trick was accomplished using Unity scripts, a Microsoft Kinect hidden above the projector, and a lot of calibrating. The complete source code is here. However, it should be noted that it will require recalibration for a new location. Contact me if you'd like some advice with this.

private Dictionary<ulong, GameObject> _Bodies = new Dictionary<ulong, GameObject>();
private Dictionary<ulong, Color> _Colors = new Dictionary<ulong, Color>();
private Dictionary<ulong, float> _offColors = new Dictionary<ulong, float>();

The important code takes place within three distinct dictionaries. The first being the bodies detected, so that multiple individuals will retain their distinct "spiritual identity" (represented as a ulong) which serves as a key for the second two arrays, "_Colors" and "_offColors." I'll explain what _offColors are later.


void Update (){

...

   foreach(var body in data)
   {
       if (body == null)
       {
           continue;
       }

       if(body.IsTracked)
       {
           if(!_Bodies.ContainsKey(body.TrackingId))
           {
               _Bodies[body.TrackingId] = CreateBodyObject(body.TrackingId);
               _Colors[body.TrackingId] = Random.ColorHSV(0f, 1f, 1f, 1f, 0.5f, 1f);
               _offColors[body.TrackingId] = Random.Range(.02f, .2f);
           }

           RefreshBodyObject(body, _Bodies[body.TrackingId]);
       }
   }
}

When a body is initiated, it's tracking identity is used to create random main Color for the _Colors dictionary, and likewise, a random float is assigned for offColor. We're projecting on a black background, so we need the majority of main colors to be bright enough. We also don't want to bum most people out for having black auras. We're able accomplish the color range we ant via a handy Random.ColorHSV() method, which ouputs a Color instance within a specificed range.



private void RefreshBodyObject(Kinect.Body body, GameObject bodyObject){
    for (Kinect.JointType jt = Kinect.JointType.SpineBase; jt <= Kinect.JointType.ThumbRight; jt++)
    {
        Kinect.Joint sourceJoint = body.Joints[jt];
        Kinect.Joint? targetJoint = null;

        if(_BoneMap.ContainsKey(jt))
        {
            targetJoint = body.Joints[_BoneMap[jt]];
        }

        GameObject jointObj = bodyObject.transform.FindChild(jt.ToString()).gameObject;
        jointObj.transform.localPosition = GetVector3FromJoint(sourceJoint);
        assignColor(jointObj, _Colors[body.TrackingId]);
        assignSomeRandom(jointObj, _offColors[body.TrackingId]);
    }
}


private void assignRandomColor(GameObject jointObj){
    Color randomColor = Random.ColorHSV(0f, 1f, 1f, 1f, 1f, 1f);
    assignColor(jointObj, randomColor);
}

private void assignSomeRandom(GameObject jointObj, float freq){
    if(Random.value < freq){
        assignRandomColor(jointObj);
    }
}

During the first pass, I wasn't satisfied with one iteration of random colors. No matter how I adjusted the specific thresholds for the color spectrum, it never felt organic. I decided to add another filter that would control a degree of blotchy contrasting, potentially dark colors. This was ok, because some darkness in your aura reveals character. I knew I was getting somewhere when testing the variability became addicting. Each shape started to obtain a sense of character and individuality. One aura would elicit a peaceful lake, while the next would resemble a calico cat vomiting up jelly beans. This was an enjoyable process that truly felt like painting with code.


These colors were used to tint variations of this grayscale texture that I meticulously handpainted /sarcasm A particle emitter using this colored texture is attached to each joint of the body. Resolume's "trail" effect provided the final touch.