Natural User Interfaces


  • Project Vision Statements due in 2 weeks (Friday, Oct. 16)
  • Sign up sheets for working with the NJSM collection files will be posted next week
  • Connected Worlds
  • Dickie Cox guest speaking this week!

Design Notebooks

Let’s take a few minutes to review some of your Design Notebooks for this week.

Natural User Interfaces

According to Wikipedia

“…a user interface that is (1) effectively invisible, or becomes invisible with successive learned interactions, to its users, and (2) is based on nature or natural elements”

NUI in the news

Types of NUIs

NUIs (Natural User Interfaces) are generally divided into several categories:

  • Multi-touch interfaces
  • Gestural Interfaces
  • Speech interfaces
  • Physical Object interfaces


Making a Multi-touch Screen


Microsoft’s NUI Lab

Remotes & Smart Phones
Depth Cameras

Kinect = Microsoft software + PrimeSense Hardware

PrimeSense open sourced Interface and middleware libraries

Adafruit Bounty + Josh Blake’s OpenKinect

Alternative Depth Cameras

Kinect + Processing Demo


Physical Object Sensors

      • Thread
      • Paint
      • Fruits & Veggies
      • Water
      • Playdough

MakeyMakey Demo

Touchboard Demo

Hover Demo

Choosing your implementation options

– Who’s your audience?

  • Are you providing the interface (kiosk, installation, loaner device, etc)? Or are they providing their own (cell phone, tablet, desktop, etc)?
  • Diffusion of Innovation curve
  • Gartner Hype Cycle
  • Analytics (like Google Analytics, or tallies by security staff)

– What are the characteristics of the exhibition space?

  • Wifi
  • Interference
  • Lighting

– What’s easiest, least invasive, least effort for your users (how can you make the technology “disappear”)?

– What’s simplest to implement?

– What is most maintainable/durable?

– What fits in your budget


  • Break up into 3 or 4 groups
  • Using Scratch or another program of your choice, create an interactive piece that maps the space bar, arrow keys and click events to navigating some content.
  • After you’ve created the content and mapped the keyboard events, create touch pads using tin foil or other conductive material for users to use to navigate your app
  • Hook up the MakeyMakey to test it out


Vision Document

  • Begin thinking about your project concept in earnest
  • Review the Vision Document Template and start filling out the sections based on your interests at the NJSM and with the technology trends thus far in the class
  • Come prepared next week to share your project concept in more detail

Games and Gamification

  • Read “Games and Gamification” in the NMC Horizon Report: Museum Edition 2015 pp. 38-39
  • Create a new slide in your Design Notebook and title it “Games & Gamification”
  • Write a brief paragraph defining gamification in your own words, and then list your favorite examples of games and gamification from the reading

Submitting Your Work

This week’s homework assignment is due next week before class. When you’ve completed them, post a comment on this page (Augmented Reality), including a link to your Design Notebook.

15 thoughts on “Natural User Interfaces”

Leave a Reply

Your email address will not be published. Required fields are marked *