building demo app
I decided to edit (again) this post instead of creating a new one, since the demo application under discussion is still under development.
In order to show the main features of the framework, how it works and how it can be used, I’ve started to develop a demo application just for this purpose. It’s available to download via svn. However without an (upcoming) documentation it’s still hard to understand many things of the framework.
Features of the demo (in short):
– Add/move/remove tangible objects. These objects, (internally related to TuioObject so from now on I’ll refer to them simply as tuio objects) are listeners of gesture events. They’re represented by coloured squares and their round area of interaction as it is explained below.
– Visual feedback showing how fingers are grouped together. A group will be considered as producing a gesture (or more). A different color is associated to each group of fingers. The traces have a losing colored tail and their head shows the id number of the trace. The head also says if the cursor has been removed, i.e. the finger released (black circle) of not (grey circle). The fingers composing a group are linked by lines towards the centroid calculated on the present fingers and the recently removed ones. Finally, a semitransparent round area indicates where another finger can be put on the surface in order to be added to the same group of the finger.
– There are various kind of gestures that can be recognize:
a) The gesture of ‘pinching’ (like a zoom with two or more fingers). This is *local* and that means that refers to a single tuio object, which is the listener (target) of the gesture event. ‘Local gestures’ have to be produced closed to the target. A parameter in settings.xml specifies that the gesture have to remain within the radius of interaction of the object (instead of only starting in that area). In the demo, the length of the radius is connected to the pinching gesture, so that can be modified.
b) A ‘multitrace’ gesture. This is *global* and that means that the gesture can have multiple targets, not necessarily tuio objects. By dragging one or more fingers from (proximally) an object to another, these are linked together by a line (the thickness is proportional to the number of fingers used for the gesture). This is a sort of drag&drop.
c) Another global gesture is to touch (or drag over) the central point of a link with one finger to remove such link. Internally the gesture recognizer module makes use of a client’s object (the list of current links) as a resource (passed in the GR’s configurator).
d) Lazoing (sorrounding with a trace) one or more tuio objects (global gesture) will select (highlight) them.
e) Tapping (clicking with one finger) close to one tuio object will unselect it.
f) Hovering (hold fingers in a point of the surface) with at least three fingers close to a tuio object will mark it as “requesting a link on channel n” where n is the number of fingers. If two tuio objects are requesting a link on the same channel, they will be linked together.
g) Drawing a circle (the recognition algorithm is to fix) will make a GUI panel to appear/disappear.
h) Moreover the GUI panel has touch-sensible buttons (finger down/up/tap), including a set of three radio buttons which can be selected by dragging a finger over them.
– Visual feedback showing the possible local targets of the Local Gesture Recognizers for a group. The centroid of the group is linked with the central point of the possibly listening object. If a LGR succeeds in its recognition and it’s exclusive, i.e. stops the next GRs in the priority list from processing, then its relative target is focused by a thicker link. In the demo example the LGRs (recognizing the pinching gesture) have been registered with the same priority number, so they’ll work in parallel, but being set as exclusive, the priority goes to the LGR associated with the closest target. In other words, when you pinch, you’re calling one pinching GR for each close target, but only the closest target will receive the gesture event.
The demo runs terribly slow because of Windows Forms that has to render the graphics of the fingers and the objects. Pressing T will hide the tuio graphics and it works fine (on Windows!), however you’ll see nothing but the GUI panel. Mono has problems with Windows Forms. Next demo will be much better!