Open Source Motion Controller Revolutionizes Human-Computer Interactions

The beauty of open sourcing development resources is that a single library or wrapper, once released to the world, can be integrated and built out by thousands of other people. And because the future of any new platform depends on what people can do with it, over the past few months Leap Motion has released a steady stream of open source assets and examples to help devs get started with our v2 tracking beta.

At the same time, developers—from creative coders and hardware hackers to web and game devs—have been releasing their projects into the wild since we first released our v1 SDK. Using the extra dimensions of data provided by motion control, along with openly available frameworks like Three.js, developers have created everything from art installations and musical experiments to motion-controlled robots and educational games.

In this post, we’ll look at some of the key resources and hotspots for open source projects. But first, a quick refresher.

What is the Leap Motion Controller?

Leap Motion ControllerOver the past 30 years, computers have become smaller, faster, and cheaper at a rate that boggles the mind. But how we interact with technology hasn’t changed much—and what can be accomplished with a lump of clay in five minutes still takes hours in a 3D CAD application. The modern human-computer interface limits what we can achieve with virtually every application that involves multi-dimensional input.

The Leap Motion Controller is a step towards breaking down this barrier with motion control, using the original human interface: the hand. It’s a small USB peripheral that tracks your hands within an interaction zone of eight cubic feet. While the hardware is fairly simple—featuring three infrared lights and a pair of cameras—the software makes possible high-precision tracking with near-zero latency.

Using the Leap Motion ControllerWhenever hands, fingers, or tools (and now forearms) are being tracked, the device provides a stream of frame, motion, and gesture data. Using the Leap Motion SDK, developers can access this data through a robust set of APIs for C#, C++, JavaScript, Python, Objective-C, and Java. The SDK is available for Windows, Mac, and Linux Ubuntu.

LeapJS

Imagine building a real-time web-based 3D data visualization app, or a multiplayer racing game that takes advantage of the browser’s innate accessibility. Our JavaScript framework is constantly evolving, and it’s easy for anyone to get started. While developers can use our CDN to integrate Leap Motion into their web apps, LeapJS also has an active GitHub repo with a variety of subprojects. After registering your name, email, and GitHub account in our contributors list, you can start making pull requests for your changes.

LeapJS also provides a powerful and flexible extension framework that makes it easy to build and use custom plugins. They’re configurable, can extend frame objects with new functionality or data, and can emit custom events from the controller. Most importantly, the rigged hand lets you add an onscreen hand to your web app with just a few lines of code.

Unity

To help developers start building 3D applications for our v2 tracking, we recently released V2 Skeletal Assets for Unity for free under the Apache 2.0 license. These include:

• Quick and easy Leap Motion integration for your existing Unity projects.
• An interaction engine that allows you to grab, rotate, and scale objects, as well as pass them from hand to hand.
• Customizable hand models and a rigged hand.
• Several demo scenes and interactions.

Our Unity Store page includes links to other free and open source assets, including a unique menu system and a v1 demo pack with more elaborate examples. For more information on getting set up with Unity Free or Pro, see our getting started guide.

Developer Portal

Along with extensive API documentation, Leap Motion’s developer portal includes a variety of examples, wrappers, and libraries—most of which are community-generated. While many v1 libraries are still being brought up to take advantage of the new V2 tracking capabilities, we’re committed to backwards compatibility over the long term. Check out the v2 gallery for a variety of examples and experiments.

Community Projects

If you’re looking to connect with other developers, Leap Motion’s community forums are a hub for questions, projects, and collaborations of all stripes. You can also find open source community projects through the Leap Motion blog, which includes a free and open source category. Here are some recent projects featured on the blog:

• Sculpt sounds in real time (SuperCollider)
• Play a 3D motion harp (Python)
• Explore a coral reef and learn about the environment (Unity)
• Experiment with camera controls (JS)
• Learn coding by fighting zombies (JS)
• Hack an analog drone’s radio signals (JS)

Dig in, and happy hacking!