The Leap Motion device is out! Great news for all who waited for it.
For me as an Eclipse 4 enthusiast and a technology lover it was obvious to bring both worlds together. Controlling an Eclipse 4 application with gestures. What nice opportunities this could bring up? Well the future will tell.
But first the main challenge needs to be solved, bringing the native libraries for the Leap Motion device into the OSGi context of an Eclipse 4 application. As a registered Leap Motion developer I was able to get my hands on that and solve it a while ago. Unfortunately I didn’t had time to write about it earlier. But now the Leap Motion device is out, I definitely should do so. So here is the blog post on how to bring the Leap Motion device into an Eclipse 4 application, so you are able to start control it with gestures.
The main idea I had was to create an OSGi service which allows to inject the Leap Motion Controller instance whereever necessary. Knowing about the ExtendedObjectSupplier
, creating the OSGi service is rather easy. Of course there is a tutorial about it written by Lars Vogel you can find here.
First we need to specify the annotation that should be used to inject the Controller
. Let’s name it LeapController
package com.beone.leapmotion; import java.lang.annotation.Documented; import java.lang.annotation.ElementType; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target; import javax.inject.Qualifier; @Qualifier @Documented @Target({ElementType.FIELD, ElementType.PARAMETER}) @Retention(RetentionPolicy.RUNTIME) public @interface LeapController { }
The next step is to create the ExtendedObjectSupplier
for the annotation.
package com.beone.leapmotion.impl; import org.eclipse.e4.core.di.suppliers.ExtendedObjectSupplier; import org.eclipse.e4.core.di.suppliers.IObjectDescriptor; import org.eclipse.e4.core.di.suppliers.IRequestor; import com.leapmotion.leap.Controller; public class LeapControllerObjectSupplier extends ExtendedObjectSupplier { static { //load the native libraries for the Leap Motion //device in the specified order System.loadLibrary("Leap"); System.loadLibrary("LeapJava"); } /** * The Leap Motion Controller that will be provided by this * object supplier. */ private Controller controller; @Override public Object get( IObjectDescriptor descriptor, IRequestor requestor, boolean track, boolean group) { if (this.controller == null) { this.controller = new Controller(); } return this.controller; } }
As the libraries for Leap Motion are native libraries, they need to be loaded when the LeapControllerObjectSupplier
is loaded by the classloader. This is done in the static init block as suggested in the wiki.
Now we register our service as an OSGi declarative service. To do this we create the file OSGI-INF/leapcontrollersupplier.xml
with the following content.
<?xml version="1.0" encoding="UTF-8"?> <scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.1.0" name="org.eclipse.leapmotion.leapcontrollersupplier"> <implementation class="org.eclipse.leapmotion.impl.LeapControllerObjectSupplier"/> <property name="dependency.injection.annotation" type="String" value="org.eclipse.leapmotion.LeapController"/> <service> <provide interface="org.eclipse.e4.core.di.suppliers.ExtendedObjectSupplier"/> </service> </scr:component>
Now it is time to talk about the plugin project setup. There are four settings to be made in the MANIFEST.MF you need to set in order to make things work:
- Add the
LeapJava.jar
to theBundle-ClassPath
so the Leap Motion Controller can be resolved - Add the
Service-Component
parameter that points to theleapcontrollersupplier.xml
- Add
javax.inject
andorg.eclipse.e4.core.di
as required bundles - Set the
Bundle-ActivationPolicy
to lazy so the libraries get loaded correctly when everything necessary for the OSGi services are ready.
To find out about the last setting cost me a lot of time. Finally Lars Vogel pointed out to that fact, so big thanks again for the tipp.
After adding all those settings to the MANIFEST.MF it should look similar to this
Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: Leapmotion Bundle-SymbolicName: com.beone.leapmotion;singleton:=true Bundle-Version: 1.0.0.qualifier Bundle-Vendor: BeOne Stuttgart GmbH Bundle-RequiredExecutionEnvironment: JavaSE-1.7 Bundle-ClassPath: LeapJava.jar, . Export-Package: com.leapmotion.leap, com.beone.leapmotion Service-Component: OSGI-INF/leapcontrollersupplier.xml Bundle-ActivationPolicy: lazy Require-Bundle: javax.inject, org.eclipse.e4.core.di
Also ensure that the necessary files are added in your build.properties
so they get exported.
Now you might ask yourself “Where are the native libraries located that we need to load?”. Well, for a clean separation we create separate fragment projects for each platform. This is similar to SWT. As an example we create a fragment project for a platform running on an x86 architecture, using a win32 operating system and the win32 windowing system, and name it com.beone.leapmotion.win32.win32.x86
. In the top level of the project we need to put the DLL files for that platform out of the LeapMotion SDK. These are the ones we load in the static initializer of the LeapControllerObjectSupplier.
In the MANIFEST.MF of the fragment project we need to set the Eclipse-PlatformFilter
to the corresponding platform. This way we ensure that the fragment is only resolved if the application is running on the matching platform (for more information have a look in the Eclipse Help). For the platform specified above, the MANIFEST.MF could look like this
Manifest-Version: 1.0 Eclipse-PlatformFilter: (& (osgi.ws=win32) (osgi.os=win32) (osgi.arch=x86)) Bundle-ManifestVersion: 2 Bundle-Name: %fragmentName Bundle-SymbolicName: com.beone.leapmotion.win32.win32.x86;singleton:=true Bundle-Version: 1.0.0.qualifier Bundle-Vendor: %providerName Fragment-Host: com.beone.leapmotion;bundle-version="1.0.0" Bundle-RequiredExecutionEnvironment: JavaSE-1.7
You can find the possible values for the Eclipse-PlatformFilter in the OSGi Specification.
Now you are able to create Eclipse 4 based applications that are controllable with the Leap Motion device. Your product simply needs to add the Plugins and Fragments created before. Also you need to ensure to start the plugins org.eclipse.core.runtime
and org.eclipse.equinox.ds
with a start level lower than 4. This way it is ensured the OSGi services are started correctly.
The following example is a simple part with a label. The label gets updated if you perform the swipe gesture, simply showing in which direction the swipe was performed.
package com.beone.leapmotion.e4.example.part; import javax.annotation.PostConstruct; import javax.annotation.PreDestroy; import javax.inject.Inject; import org.eclipse.e4.core.contexts.IEclipseContext; import org.eclipse.e4.ui.di.Focus; import org.eclipse.e4.ui.di.UISynchronize; import org.eclipse.swt.SWT; import org.eclipse.swt.widgets.Composite; import org.eclipse.swt.widgets.Label; import com.beone.leapmotion.LeapController; import com.leapmotion.leap.Controller; import com.leapmotion.leap.Frame; import com.leapmotion.leap.Gesture; import com.leapmotion.leap.GestureList; import com.leapmotion.leap.Listener; import com.leapmotion.leap.SwipeGesture; public class LeapMotionExamplePart { private Listener listener; private Label label; @Inject private UISynchronize uiSync; @PostConstruct public void postConstruct(Composite parent, @LeapController Controller leapController, IEclipseContext context) { this.label = new Label(parent, SWT.NONE); this.listener = new LeapMotionListener(); leapController.addListener(this.listener); } @PreDestroy public void preDestroy(@LeapController Controller leapController) { leapController.removeListener(this.listener); } @Focus public void onFocus() { if (this.label != null) { this.label.setFocus(); } } private class LeapMotionListener extends Listener { String labelText = ""; @Override public void onInit(Controller controller) { System.out.println("Initialized"); } @Override public void onConnect(Controller controller) { System.out.println("Connected"); controller.enableGesture(Gesture.Type.TYPE_SWIPE); } @Override public void onDisconnect(Controller controller) { System.out.println("Disconnected"); } @Override public void onExit(Controller controller) { System.out.println("Exited"); } @Override public void onFrame(Controller controller) { // Get the most recent frame and report some basic information Frame frame = controller.frame(); GestureList gestures = frame.gestures(); for (int i = 0; i < gestures.count(); i++) { Gesture gesture = gestures.get(i); switch (gesture.type()) { case TYPE_SWIPE: SwipeGesture swipe = new SwipeGesture(gesture); float x = swipe.startPosition().getX() - swipe.position().getX(); labelText = "Swipe drawed in direction " + (x > 0 ? "left" : "right"); break; default: System.out.println("Unknown gesture type."); break; } uiSync.syncExec(new Runnable() { @Override public void run() { label.setText(labelText); } }); } } } }
Note that the UI updates need to be performed in the UI thread!
I would love to share my whole project infrastructure in Git or somewhere else. But I’m not aware of the policies for sharing the Leap Motion SDK out of the Leap Motion Developer Portal. If someone from the Leap Motion Team reads this post and could provide me with informations on how to share my project, I would love to do so. It would be even greater if the OSGi bundles containing the native libraries for several platforms would be provided by Leap Motion directly, so not every developer needs to create the plugins himself. Of course I would love to provide any input you need to do so.
7 Responses to Leap Motion in Eclipse 4