Natural User Interfaces

Announcements

• Project Vision Statements due in 2 weeks (Friday, Oct. 21)
• Time slot sign up sheets for working with the NJSM collection will be posted next week
• Google Daydream
• Google Pixel

Augmented Reality Revisited

Aurasma Exercise

• Break up into teams of 2, 3 or 4
• Head over to the Aurasma website and signup
• Using Aurasma Studio, create a a new Aura
• The follow the instructions to create a demo AR app
• Use photos or images from the web to create your demos

Design Notebooks

Let’s take a few minutes to review some of your Design Notebooks for this week.

Natural User Interfaces

According to Wikipedia

“…a user interface that is (1) effectively invisible, or becomes invisible with successive learned interactions, to its users, and (2) is based on nature or natural elements”

No-UI UI

The Best Interface is No Interface

NUI in the news

• In Wired Design: http://www.wired.com/design/2013/10/watch-a-computer-orchestra-you-conduct-by-waving-your-hands/

Types of NUIs

NUIs (Natural User Interfaces) are generally divided into several categories:

• Multi-touch interfaces
• Gestural Interfaces
• Speech Interfaces
• Physical Object Interfaces

Multi-touch

• Smart Phones
• Tablets
• Muti-touch Screens
  • Jeff Han, NYU
  • CNN 2008 Election
  • CNN Magic Wall
  • InteractiveLab 

MAKING A MULTI-TOUCH SCREEN

• The Mill
• Hammer.js 

Gesture

Microsoft’s NUI Lab

REMOTES & SMART PHONES

• Wii Remote
• MSA Remote for iPhone
• Unified Remote

DEPTH CAMERAS

Kinect = Microsoft software + PrimeSense Hardware

• Microsoft Kinect
• Microsoft SDK (Depth, Sound & Rotation)

PrimeSense open sourced Interface and middleware libraries

• OpenNI
• SimpleOpenNI Processing Library

Adafruit Bounty + Josh Blake’s OpenKinect

• OpenKinect was killed by Apple two years ago after purchasing PrimeSense
• Shiffman’s Kinect Processing Library

ALTERNATIVE DEPTH CAMERAS

• Asus Xtion
• Leap3D
• Creative Senz3D

Kinect + Processing Demo

Speech

• Siri
• Amazon Echo
• Amazon Alexa SDK & Voice Service
• Google Home
• Microsoft Cortana
• Shazam
• Ford Sync

Physical Object Sensors

MICRO-CONTROLLERS

• Arduino
• Raspberry Pi
• MakeyMakey
• Touch Board
• Capacitive Sensing Library
• MPR121 Capacitive Touch Sensor
• Hover

EXAMPLES

• Banana Piano
• Living Wall
• Connected Worlds

CONDUCTIVE MATERIALS

• Thread
• Paint
• Fruits & Veggies
• Water
• Playdough

MakeyMakey Demo

Touchboard Demo

Projection Mapping

The Electric Rise and Fall of Nikola Tesla

Projection Mapping Tools

• Touch Designer
• Mad Mapper
• VPT

Choosing your implementation options

– Who’s your audience?

• Are you providing the interface (kiosk, installation, loaner device, etc)? Or are they providing their own (cell phone, tablet, desktop, etc)?
• Diffusion of Innovation curve
• Gartner Hype Cycle
• Analytics (like Google Analytics, or tallies by security staff)

– What are the characteristics of the exhibition space?

• Wifi
• Interference
• Lighting

– What’s easiest, least invasive, least effort for your users (how can you make the technology “disappear”)?

– What’s simplest to implement?

– What is most maintainable/durable?

– What fits in your budget

Exercise

• Break up into 3 or 4 groups
• Using Scratch or another program of your choice, create an interactive piece that maps the space bar, arrow keys and click events to navigating some content.
• After you’ve created the content and mapped the keyboard events, create touch pads using tin foil or other conductive material for users to use to navigate your app
• Hook up the MakeyMakey to test it out

Assignment

Vision Document

• Begin thinking about your project concept in earnest
• Review the Vision Document Template and start filling out the sections based on your interests at the NJSM and with the technology trends thus far in the class
• Come prepared next week to share your project concept in more detail

Games and Gamification

• Read “Games and Gamification” in the NMC Horizon Report: Museum Edition 2015 pp. 38-39
• Create a new slide in your Design Notebook and title it “Games & Gamification”
• Write a brief paragraph defining gamification in your own words, and then list your favorite examples of games and gamification from the reading

Submitting Your Work

This week’s homework assignment is due next week before class. When you’ve completed them, post a comment on this page (Augmented Reality), including a link to your Design Notebook.