Webcam Sculpting
This virtual webcam sculpting tool is meant to be experienced in a space in which the feed from the HTC Vive is projected onto the space directly where the user is standing so that a disorienting feedback loop is created.
Our HTC Vive arrived a few days before a chiptunes show was scheduled Digital Art Demo Space. I wanted to create an app for the show that would allow audience members to continue to watch performances in the physical space while they were experiencing VR. I decided to experiment with texturing objects from the images captured from the camera attached to the Vive headset. This is accomplished with the handy WebCamTexture() class in Unity.
void Start(){
WebCamTexture webcamTexture = new WebCamTexture();
Renderer renderer = GetComponent();
renderer.material.mainTexture = webcamTexture;
webcamTexture.Play();
}
This script can be placed on any mesh that you want. A potential problem is that other devices can create WebCamTextures that override the visual information you want. You can specifiy a specific webcam with webcamTexture.deviceName = "myWebCamName"
I wanted the controls to be intuitive without directions or interface. A simple sensation of blowing air into a cube-shaped balloon that is released and slowly floats away
public float triggerVelocity = 0.0f;
public float oldTriggerVelocity = 0.0f;
public float triggerReleaseThreshold = 0.1f;
void Update(){
triggerVelocity = controller.GetAxis(triggerButton).x;
}
void detectChange() {
float triggerChange = oldTriggerVelocity - triggerVelocity;
oldTriggerVelocity = triggerVelocity;
triggerButtonRelease = false;
if (!triggerButtonReleased) {
if (triggerChange != 0f) {
if (triggerChange > triggerReleaseThreshold) {
triggerButtonRelease = true;
triggerButtonReleased = true;
}
}
}
else {
if (triggerVelocity == 0) {
triggerButtonReleased = false;
}
}
}
I wanted the size of the cube to be sensitive to how hard you were pressing and not lose size when the user planned on releasing. A lot of tweaking made me satisfied that a rate of change greater than 0.1 in the trigger velocity meant that the user was releasing the trigger. If this change happend on the upswing, the triggerChange variable would be negative.
if (controller.triggerButtonRelease && controller.holdingObject){
releaseObject(controller);
}
else{
if (controller.triggerVelocity > 0.2f) {
if (!controller.holdingObject) {
createObjectAtController(controller);
}else{
moveObjectToController(controller);
rotateObjectToController(controller);
growObject(controller.triggerVelocity);
}
}
}
if (controller.triggerVelocity < 0.05f && controller.holdingObject){
releaseObject(controller);
}
We move the growing of the webcam bubble into a parent controller class.
private List instances;
private GameObject currentInstance;
void createInstances(GameObject _prefab)
{
for (int i = 0; i < numInstances; i++)
{
GameObject obj = createInstance(_prefab);
obj.transform.position = holdingStation;
shrinkObjectToInitialSize(obj);
instances.Add(obj);
}
currentInstance = instances[0];
}
It's important that when a user has the ability to create new objects on the fly, and those objects are render intensive, that they aren't actually creating *new* objects, but pulling from a pool that has been pre-created.
void growObject(float triggerVelocity){
currentSize += ((triggerVelocity * triggerVelocity )/200f);
if(triggerVelocity < .2f){
float gravity = (.2f - triggerVelocity)/100f;
currentSize -= gravity;
}
currentSize = Mathf.Clamp(currentSize, 0, 1.15f);
setObjectSize(currentSize);
Vector3 targetScale = new Vector3(currentSize, currentSize, currentSize);
currentInstance.transform.localScale = targetScale;
}
"Gravity" in this context, is the subtle shrinking of the object when the player is slowly holding back on the object. This has no practical purpose other than haptic pleasure. One problem is that because of this shrinking potential, the GameObject ends up being released at a much smaller scale than we wanted it to be.
private float[] savedSizes;
private int numSavedSizes = 3;
void saveSizeHistory(float size){
int frameIndex = frames % savedSizes.Length;
savedSizes[frameIndex] = size;
}
void setInstanceToMaxSavedSize() {
float maxSize = savedSizes[0];
foreach (float size in savedSizes) {
if (size > maxSize) {
maxSize = size;
}
}
setObjectSize(maxSize);
}
void releaseObject(WandController controller){
controller.holdingObject = false;
setInstanceToMaxSavedSize();
startAnimation(currentInstance);
}
We fix this by making a small cache of saved sizes, and then pulling from the max size when we release the object.
This project was featured as part of the virtual reality gallery scanva at Subchroma 2016