Augmenta TUIO

👋

We use the standard TUIO 1.1 protocol, with additional /scene and /fusion OSC messages from the OSC protocol. Those messages can either be sent on the same network port or on a different one depending on your needs.

TUIO Protocol does not give access to the complete data collection from the Augmenta protocol. Two points need to be highlighted :

TUIO protocol doesn't support sending both the bounding box and the centroid position and orientation at the same time. The bounding box information is only sent in Blob descriptors mode. The centroid information is sent in the cursor and object descriptors mode.

We chose to use the class ID field (symbol i) of TUIO to represent Augmenta's slotID. This means that if you are interested in using the slotID, you will have to use the Object descriptor mode.

In Augmenta Fusion, as well as in the Augmenta Simulator there is no such thing as a scene depth (on the Z axis). To match with the TUIO spec, we provide the sceneDepth parameter that defaults to a value of 10 in the TUIO output of those software. It can be adjusted in the user interface of both software. This is also the reason why the Z value (velocity on z axis) is a fixed 0, there is no such data computed for Augmenta objects.

To carry this sceneDepth parameter to the clients we add a value to the scene message ONLY when it is sent from a TUIO output. The scene message sent in parallel of the TUIO data looks like this:

plain text
/scene   arg0 arg1 ... argN

    0: frame (int)                         // Scene frame number
    1: objectCount (int)                   // Number of objects
    2: scene.width (float)                 // Size (in m)
    3: scene.height (float)
    4: sceneDepth (float)

Here is the correspondence table between Augmenta data and TUIO data:

TUIO	Augmenta (cursor TUIO mode)	Augmenta (object TUIO mode)	Augmenta (blob TUIO mode)
s	id	id	id
i	-	slotID	-
x, y, z	centroid.x, centroid.y, (1/2 height)/sceneDepth	centroid.x, centroid.y, (1/2 height)/sceneDepth	boundingRect.x, boundingRect.y, (1/2 height)/sceneDepth
a, b, c	orientation, 0, 0	orientation, 0, 0	boundingRect.rotation, 0, 0
w, h, d	-	-	boundingRect.width, boundingRect.height, height/sceneDepth
f, v	-	-	(bR.width x bR.height), (bR.width x bR.height x height)
X, Y, Z	velocity.x, velocity.y, 0	velocity.x, velocity.y, 0	velocity.x, velocity.y, 0
A, B, C	0, 0, 0	0, 0, 0	0, 0, 0
m	0	0	0
r	0	0	0

in the table above bR is a shorthand for boundingRect.

Please note: As in the TUIO standard, the velocity is normalized by the size of the scene (as all the other position-dependant parameters such as centroid, bounds, etc...). This means that for non-square scenes, you will have a speed value scaled differently for x and y. If you want to do meaningful operations based on speed, you will have to multiply the x value by scene.width and the y value by scene.height.

Reminder of TUIO 1.1 semantic types:

TUIO Data	Description	Type
s	Session ID (temporary object ID)	`int32`
i	Class ID (e.g. marker ID)	`int32`
x, y, z	Position	`float32` [0,1]
a, b, c	Angle	`float32` [0,2PI]
w, h, d	Dimension	`float32` [0,1]
f, v	Area, Volume	`float32` [0,1]
X, Y, Y	Velocity vector (motion speed and direction)	`float32`
A, B, C	Rotation velocity vector (rotation speed and direction)	`float32`
m	Motion acceleration	`float32`
r	Rotation acceleration	`float32`

Presets are implemented in Augmenta Fusion to help you choose the best TUIO format:

Presets	Supported descriptors	Supported dimensions	Default	Description
None	Cursor, Blob, Object	2D, 2.5D, 3D	-	-
Minimal	Cursor	2D	2D Cursor	-
Best	Blob	2.5D, 3D	2.5D Blob	Best fit for Augmenta's data
SlotID	Object	2.5D	2.5D Object	Supports the SlotID attribute
Notch	Cursor, Object, Blob	2D	2D Blob	Supported formats in Notch
Touch Designer	Cursor, Object, Blob	2D, 2.5D, 3D	3D Blob	Supported formats in Touch Designer

Note : By default, OSC and TUIO are sent on the same port for Touch Designer's preset as you can receive both OSC messages and TUIO messages on the same connection. For Notch's preset they are sent on separate ports.