logo

Face Cap V1.5

Live mode information:

  1. Face Cap transmits data over a local (wifi) network using OSC.
  2. The live mode data receiver application is to be created by the user.
  3. For testing a demo receiver application is provided for Mac & Windows.

Scheduled Updates:

  1. Configurable port settings.
  2. Example Unity project.

Live mode receiver demo application v0.1:

  1. Windows.
  2. Mac.

Instructions:

  1. Make sure both devices are connected to the same (Wifi) network.
  2. Disable VPN.
  3. Allow the demo application through the firewall.
  4. Enter the I.P of the receiver device into Face Cap live mode.
  5. If you experience lag check your network (kill uploads/downloads etc), or in case of using a Macbook make sure the power is plugged in (network can be throttled when running on battery only).

What is OSC?

  1. Open Sound Control (OSC) is a protocol for communication among computers, sound synthesizers, and other multimedia devices that is optimized for modern networking technology.

Why OSC?

  1. OSC is easy to implement.
  2. OSC libraries are available for Unity, VVVV, Processing, Open framworks etc.
  3. OSC allows the data Face Cap sends to be used to drive pretty much anything. Use the data to do eye tracking or drive a synthesizer with your face etc.

Face Cap OSC Addresses & data formatting:

  1. Head position: /HT + 3 Floats (x,y,z)
  2. Head rotation in degrees: /HR + 3 Floats (x,y,z)
  3. Head rotation as quaternion: /HRQ + 4 Floats (x,y,z,w)
  4. Eye left rotation : /ELR + 2 Floats (x,y)
  5. Eye right rotation : /ERR + 2 Floats (x,y)
  6. Blendshape parameters : /W + 1 Int + 1 Float (blendshape index, value)

Blendshape index (for reference):

  1. 00 : brow inner up
  2. 01 : brow down left
  3. 02 : brow down right
  4. 03 : brow outer up left
  5. 04 : brow outer up right
  6. 05 : eye look up left
  7. 06 : eye look up right
  8. 07 : eye look down left
  9. 08 : eye look down right
  10. 09 : eye look in left
  11. 10 : eye look in right
  12. 11 : eye look out left
  13. 12 : eye look out right
  14. 13 : eye blink left
  15. 14 : eye blink right
  16. 15 : eye squint left
  17. 16 : eye squint right
  18. 17 : eye wide left
  19. 18 : eye wide right
  20. 19 : cheek puff
  21. 20 : cheek squint left
  22. 21 : cheek squint right
  23. 22 : nose sneer left
  24. 23 : nose sneer right
  25. 24 : jaw open
  26. 25 : jaw forward
  27. 26 : jaw left
  28. 27 : jaw right
  29. 28 : mouth funnel
  30. 29 : mouth pucker
  31. 30 : mouth left
  32. 31 : mouth right
  33. 32 : mouth roll upper
  34. 33 : mouth roll lower
  35. 34 : mouth shrug upper
  36. 35 : mouth shrug lower
  37. 36 : mouth close
  38. 37 : mouth smile left
  39. 38 : mouth smile right
  40. 39 : mouth frown left
  41. 40 : mouth frown right
  42. 41 : mouth dimple left
  43. 42 : mouth dimple right
  44. 43 : mouth upper up left
  45. 44 : mouth upper up right
  46. 45 : mouth lower down left
  47. 46 : mouth lower down right
  48. 47 : mouth press left
  49. 48 : mouth press right
  50. 49 : mouth stretch left
  51. 50 : mouth stretch right
  52. 51 : tongue out

Notes:

  1. Y is up.
  2. You might have to correct axis/scale data depending on your application. For Example in Unity the z axis is -z resulting in mirrored rotations etc.
IOS link