As shown in the above figure hands (includes palm and fingers) were detected by the Leap Motion Controller and data was sent to the Processing software. The processing software is capable of detecting, extracting data, identifying, decision making and other output controlling. This depends on the generated program. Once control decisions are made according to the application requirements, control algorithms, processing can instruct controller or other program to what should be done.
Above figure (right) shows the development of tracking two hands using processing. The skeleton of two hands were generated and shown in the above figure (left). The main design requirement of the application is to detect the fingertip and to extract the position, orientation and velocity data from it.
Therefore it is developed to track the middle, index and the thumb fingers. The tracked fingertips and wrist was recreated in the Processing framework as square objects and a circular object. The index finger was used to extract position, orientation and velocity data for the robot.
Then four vectors were drawn from the Leap Motion Controller coordinate frame to each fingertip and to the wrist. Then using the following relationship, a relationship to wrist to, fingertip distance was developed. The equation for the middle finger is as follows,
Vwrist + Wmiddle = Vmiddle
Wmiddle = Vmiddle - Vwrist
The same equation, was repeated for each finger and then took the average value as follows,
K = | Wmiddle |+| Windex |+| Wthumb | / 3
By that K parameter was generated, that will use to signal the solenoid valve. To control smoothly, the feedback was taken from the flex sensor attached to the soft finger and then was controlled according to the following pseudo code.
P = Flex Sensor
K is Initial Value
P is initial Value
P is Serial Analog Value Read
If P is Smaller Than or Equal K
Serial Digital Write High
Else
Serial Digital Write Low