Natural User Interfaces


  • Project Vision Statements due in 2 weeks (Friday, Oct. 21)
  • Time slot sign up sheets for working with the NJSM collection will be posted next week
  • Google Daydream
  • Google Pixel

Augmented Reality Revisited

Aurasma Exercise

  • Break up into teams of 2, 3 or 4
  • Head over to the Aurasma website and signup
  • Using Aurasma Studio, create a a new Aura
  • The follow the instructions to create a demo AR app
  • Use photos or images from the web to create your demos

Design Notebooks

Let’s take a few minutes to review some of your Design Notebooks for this week.

Natural User Interfaces

According to Wikipedia

“…a user interface that is (1) effectively invisible, or becomes invisible with successive learned interactions, to its users, and (2) is based on nature or natural elements”


The Best Interface is No Interface

NUI in the news

Types of NUIs

NUIs (Natural User Interfaces) are generally divided into several categories:

  • Multi-touch interfaces
  • Gestural Interfaces
  • Speech Interfaces
  • Physical Object Interfaces




Microsoft’s NUI Lab


Kinect = Microsoft software + PrimeSense Hardware

PrimeSense open sourced Interface and middleware libraries

Adafruit Bounty + Josh Blake’s OpenKinect


Kinect + Processing Demo


Physical Object Sensors

      • Thread
      • Paint
      • Fruits & Veggies
      • Water
      • Playdough

MakeyMakey Demo

Touchboard Demo

Projection Mapping

The Electric Rise and Fall of Nikola Tesla

Projection Mapping Tools

Choosing your implementation options

– Who’s your audience?

  • Are you providing the interface (kiosk, installation, loaner device, etc)? Or are they providing their own (cell phone, tablet, desktop, etc)?
  • Diffusion of Innovation curve
  • Gartner Hype Cycle
  • Analytics (like Google Analytics, or tallies by security staff)

– What are the characteristics of the exhibition space?

  • Wifi
  • Interference
  • Lighting

– What’s easiest, least invasive, least effort for your users (how can you make the technology “disappear”)?

– What’s simplest to implement?

– What is most maintainable/durable?

– What fits in your budget


  • Break up into 3 or 4 groups
  • Using Scratch or another program of your choice, create an interactive piece that maps the space bar, arrow keys and click events to navigating some content.
  • After you’ve created the content and mapped the keyboard events, create touch pads using tin foil or other conductive material for users to use to navigate your app
  • Hook up the MakeyMakey to test it out


Vision Document

  • Begin thinking about your project concept in earnest
  • Review the Vision Document Template and start filling out the sections based on your interests at the NJSM and with the technology trends thus far in the class
  • Come prepared next week to share your project concept in more detail

Games and Gamification

  • Read “Games and Gamification” in the NMC Horizon Report: Museum Edition 2015 pp. 38-39
  • Create a new slide in your Design Notebook and title it “Games & Gamification”
  • Write a brief paragraph defining gamification in your own words, and then list your favorite examples of games and gamification from the reading

Submitting Your Work

This week’s homework assignment is due next week before class. When you’ve completed them, post a comment on this page (Augmented Reality), including a link to your Design Notebook.

One thought on “Natural User Interfaces”

Leave a Reply

Your email address will not be published. Required fields are marked *