Twilio Voice Recordings

Announcements

Design Notebooks

Let’s take a few minutes to review your Design Notebook entries from last week.

Partner Meetings

Project Teach

Tracy Perron has provided contact info for the Nursing School students working on site at Project Teach. I’ll share that contact info with the Team Teach directly.

Columbus School

Tracy Perron has arranged for Mission Slim-possible to have direct access to the school nurse at Columbus. I’ll share her contact info with the team.

History Department

Team Infinitime should make direct contact with the history department to collect user research.

Twilio Voice Recordings

Twilio’s Text-to-Speech engine is cool, but…

  • Language limitations (even Alice supports only 26 languages/dialects)
  • Sometimes we want greater flexibility with specific voices, i.e. celebrities
  • Synthesized speech just isn’t as authentic/compelling/trusted as natural human speech

Today we’ll be using Twilio’s <Play> and <Record> TwiML verbs.

  1. Download the source code for this week’s examples from the class Github Organization.
  2. Unzip the  repository on your local machine and upload the contents to a new folder in the ‘www’ folder on your TCNJ web account.
  3. Let’s try out the examples one by one.

Basic Voice Playback

Instead of using Twilio’s text-to-speech engine, let’s play back a recorded message.

  1. After you’ve uploaded the basic-voice.xml file to your TCNJ web account, log into Twilio, click into your number detail page, and edit the Voice request url for your number to point to ‘basic-voice-playback.xml’.
  2. Call your Twilio number.
  3. What happened?
<!-- basic-voice-playback.xml -->
<?xml version="1.0" encoding="UTF-8"?>
<Response>
    <Play>http://www.mediamesis.net/imm/3hellos2.wav</Play>
</Response>

Twilio <Play> documentation

Gathering Input

Now let’s use the <Play> verb with <Gather>.

  1. Upload matrix.xml and checkpill.php to your TCNJ web account and edit the Voice request url in the Twilio Dashboard for your number to point to your edited file.
  2. Call your Twilio number.
  3. What happened?
 <!-- matrix.xml -->
<?xml version="1.0" encoding="UTF-8"?>
<Response>
    <Gather action="./check_pill.php" method="get" finishOnKey="*">
        <Play>http://www.mediamesis.net/imm/redpill.mp3</Play>
        <Say>Press 1 to take the red pill.</Say>
        <Say>Press 2 to take the blue pill.</Say>
    </Gather>
</Response>
 /* checkpill.php */
<?php
$pill = $_REQUEST['Digits'];

header("content-type: text/xml");
echo "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n";

if($pill == 1){ // red pill
    echo "<Response>";
    echo "<Play>http://www.mediamesis.net/imm/bluepill.mp3</Play>";
    echo "</Response>";
} else if($pill = 2) { // blue pill
    echo "<Response>";
    echo "<Play>http://www.mediamesis.net/imm/matrix66.mp3</Play>"
    echo "</Response>";
} else { // other
    echo "<Response>";
    echo "<Say>Sorry, I didn't understand your selection. Please try again.</Say>"; 
    echo "<Redirect method=\"POST\">http://yourserver/matrix.xml</Redirect>";
    echo "</Response>";
}

“Please Leave a Message…”

Now let’s try building a simple answering machine.

  1. Upload voicemail.xml and record.php to your TCNJ web account and edit the Voice request url in the Twilio Dashboard for your number to point to your edited file.
  2. Call your Twilio number.
  3. What happened?
<!-- voicemail.xml -->
<?xml version="1.0" encoding="UTF-8"?>
<Response>
   <Say>Leave a message after the beep. Press the pound key when you're done recording.</Say>
  <Record 
      action="http://www.tcnj.edu/~thompsom/imm-470-03/voice-recordings/record.php"
      method="GET"
      maxLength="20"
      finishOnKey="#"
   />
</Response>
/* recording.php */
<?php
header("content-type: text/xml");
echo "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n";

$recording = $_REQUEST['RecordingUrl']; 
$duration = $_REQUEST['RecordingDuration'];
?>
<Response>
 <Say>Thanks for calling. Your message was <?php echo $duration; ?> seconds long. Here's the message you left.</Say>
 <Play><?php echo $recording; ?></Play>
</Response>

Twilio <Record> Documentation

Save a voicemail to SoundCloud

Workflow for Creating Voice Menu Recordings

Basic flow

  1. Start with a flow diagram
  2. Prototype it using <Say>
  3. Create a script for the voice talent
  4. Record the messages
  5. Implement using <Play>

What about foreign languages/dialects?

Challenges

Twilio

Review the Twilio Audio Recording Best Practices.

Using Audacity or other audio recording software, create a voice menu + leave-a-message app that presents the caller with information about an upcoming IMM event (e.g. HackTCNJ) and prompts them to record a message about the event.

Create menu options for:

  • Date, time and location
  • An overview description of the event
  • A sponsor listing
  • Request the caller leave-a-message about the event

Amazon Echo

Amazon Lambda + Amazon Developer Console

TCNJ Webserver + Amazon Developer Console

Deploy the Count Example skill (uses php on TCNJ web account)

  1. Create an Amazon Developer Account
  2. Create an Amazon Alexa Skill in the Developer Console
  3. Follow the Amazon Alexa with PHP Hello World tutorial

Cordova

Deploy a demo app to a connected iPhone

Timeline.js

  • Experiment with adding Soundcloud files to a Timeline.js timeline
  • Try uploading a Twilio voice recording to Soundcloud using the save2sc.php script

Assignment

Reading

Design Notebooks

  • Create a new slide in your notebook and title it “Year of the bot”
  • Close read the The search for the killer bot article
  • Write a paragraph or two commenting on the premise of the article. What evidence have you personally experienced that confirms or refutes the primary assertion of the article.

Questions for Your Partner Organization (team)

  • Create a list of questions for your Partner Organization about their services and most pressing communication problems.
  • Create another slide in your Design Notebook, titling it “Interview Questions” and today’s date., and drop in your list of questions from above.
  • Next week, you’ll combine your questions with those from your teammates to create a master list.

*Submit the link to your Design Notebook to Canvas before we meet again next week

Team Infinitime Line

logo

Partner

We will be most likely partnering with the History department on our project.

Team

Matt Newman, Developer: Focuses on developing project from the ground up. Emphasis on database structure and php programming.

Matt Cherry, Designer: Focuses on designing project from the ground up. Emphasis on visuals, color scheme, and user experience.

Project Concept

People often don’t realize the underlying connections that exist between historical events. Our idea is based off the age old saying that “History repeats itself”. We are making timelines that will allow people to visualize history more accurately.

Live Website Demo

Final Presentation (Google Presentation)

TextIt.in Flow

Design Notebooks

Mission Slim-Possible!

HealthNut-01

Partner

Culture of Health

Team Members

  • Jamie
  • Elizabeth
  • Angela
  • Shay

Project Concept

“…Our goal is to promote a healthy lifestyle for children through interactive games and exercises, while providing an application to educate parents on health conscious choices, recipes, and information…”

Click for more information on our Project Concept

 

Vision Document

An iterative development, that is constantly updated. This is an overview of our project specifications and includes the following information:

  • Project Overview
  • User Personas
  • Timeline
  • Budget
  • Flowcharts and Diagrams

Vision Document


System Persona

  • A two-fold approach in order to help promote healthy lifestyle choices for children and parents within the community of Trenton.
  • Developing two projects, the first of which, will be utilizing Amazon Echo’s skills kit, as well as Pandorabots, in order to help communicate healthy food choices and exercise options to children in a fun, interactive and comprehensible manner.
  • Our second project will consist of an informational app oriented towards parents that will communicate healthy lifestyle choices, including proper diet, nutrition, exercise, and mindfulness.
  • The content incorporated in both of our projects will deal with healthy food alternatives, forms of exercise, and balanced daily nutritional needs.
  • As our projects rely heavily on communicating up-to-date information related to health and wellness, we will pull from a variety of sources, such as webmd.com, nutrition.gov, kids.usa.gov, eatright.org and cdc.gov.
  • Distill information in a way that can be understood by Trenton’s parents and children.
  • We will generate relevant visuals to supplement health-related content and stick to basic language in order to convey meaningful information to parents and children.
  • The parents and children of Trenton may not be well-versed in health and exercise terminology, such as macro/micronutrients, or glycemic index, so we have to find a way to simplify these general terms.
  • Based on our varied target audience, we plan to use differing “voices” or “personas” in each of our projects.
    • When developing our Echo-based application and chatbot, which is geared towards children, we will appropriately shift our tone to one that promotes an experience that is more interactive, kind, friendly, fun, as well as educational.

System Flowcharts

Flow_1
Amazon Alexa Flow Chart

 

healthnut_general_flow
HealthNut ChatBot Flowchart

 

 

Project Timeline

Project Timeline
Project Timeline

 


Live Demos Available

HealthNut Chat Bot:

How to interact:

  • Sign up for a free account here.
  • Log in and you will see a menu at the top of the screen. Click on “Clubhouse”

Screen Shot 2016-03-31 at 2.44.31 PM

  • In the search bar, type “healthnut”:

Screen Shot 2016-03-31 at 2.46.43 PM

  • To interact, click “talk”:

Screen Shot 2016-03-31 at 2.47.02 PM

  • Have fun!

Project Documentation

Final Presentations

Team TEACH

Partner:

Project TEACH

http://www.nj.gov/dcf/families/educational/teach/

Team Members:

Mike Lore – medicines
Dennis Bereznyak – medicines
Israel Dominguez – focus on Spanish translations
Lucy Clementoni – milestones
Jenna Vitale – milestones

Project Concept:

We will create an SMS app to help the mothers in Project TEACH identify alternative names of medications in order to help stop accidental double dosing.. We will also have a milestone system in place where mothers can text in their child’s age and receive information such as the ideal weight and what type of foods they should be feeding their child.
Full Project Concept

Our specific project will focus on the medicines. Our partners will focus on the milestones, and for the purposes of this project we’ll keep them separate, as a demonstration of a platform that can be added to.

Interview Questions

Project Teach Interview Questions

Vision Document

Link to vision document

System Persona

Link to System Persona

Project Design & Implementation Documentation

Link to design diagram – milestones AND medicines

Link to design diagram – medicine specific flow

Link to Documentation

Final Presentation

Twilio Voice

Announcements

  • This Week’s Brown Bag Lunch: The Evolution of Television
  • Next Week – User Persona & User Stories
  • In 2 Weeks – site visits to partner organizations

Team Pages

Let’s take a few minutes to complete setting up your team pages.

Now that we’ve got your team pages set up, get together with your team and create the following headings on your page:

Partner

Include which project your team will be partnering with.

Team

List your team members, along with what role each will perform on the team.

Project Concept

Include a link to your team’s project concept document. Be sure to share it as “anyone with the link may view”.

Design Notebooks: Project Concepts

Now that our team pages are set up, let’s go around the room and have each team briefly present their project concept, using their team page as their starting point.

As a group, let’s discuss each project concept.

Voice Menus

SMS is great, but…

  • It only supports 160 characters
  • It assumes the device supports a language you understand well enough
  • And it assumes you can read and write

What languages do the most common devices support?

What countries support local phone numbers for Twilio?

Twilio Voice

In addition to SMS, Twilio supports voice. You can use voice using Twilio’s text-to-speech engine, or use recorded messages. Today we’ll be using the text-to-speech engine.

  1. Download the source code for this week’s examples from the class Github Organization.
  2. Unzip the  repository on your local machine and upload the contents to a new folder in the ‘www’ folder on your TCNJ web account.
  3. Let’s try out the examples one by one.

Basic Voice

  1. After you’ve uploaded the basic-voice.xml file to your TCNJ web account, log into Twilio, click into your number detail page, and edit the Voice request url for your number to point to ‘basic-voice.xml’.
  2. Call your Twilio number.
  3. What happened?
<!-- basic-voice.xml -->
<?xml version="1.0" encoding="UTF-8"?>
<Response>
    <Say>Hello Twilio!</Say>
</Response>

TwiML Say Documentation

VOICE ATTRIBUTE

Let’ experiment. Take a look at the documentation for the <Say> command paying particular attention to the ‘voice’ attribute.

  1. Try changing the voice for your app.
  2. Re-upload the your file to your TCNJ web account and edit the Voice request url for your number to point to your edited file.
  3. Call your Twilio number.
  4. What happened?
LANGAUGE ATTRIBUTE
  1. Once you’ve successfully changed the voice, try changing the language.
  2. Re-upload the your file to your TCNJ web account and edit the Voice request url for your number to point to your edited file.
  3. Call your Twilio number.
  4. What happened?

Twilio Voice Request Variables

Similar to our experience last week sending SMS messages, TwiML for Voice also supports a number of request variables we can access. Like last week, we can use this to create a simple phone book so we can recognize callers.

  1. Change (or add) the phone record in the phone book array in your source code.
  2. Re-upload the your file to your TCNJ web account and edit the Voice request url for your number to point to voice-request-vars.php.
  3. Call your Twilio number.
  4. What happened?
<!-- voice-request-vars.php -->
<?php
$from = $_REQUEST["From"];

// make an associative array of senders we know, indexed by phone number
$phonebook = array(
    "+12152642459"=>"Professor Thompson",
);

$name = "étranger";

// if the caller is in our phonebook, then greet them by name
// otherwise, refer to them as étranger
if(isset($phonebook[$_REQUEST['From']])) {
    $name = $phonebook[$_REQUEST['From']];
}
header("content-type: text/xml");
echo "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n";
?>
<Response>
    <Say voice="alice" language="fr-FR">Allô <?php echo $name ?>.</Say>
<Response>

TwiML Voice Request Variables

Gathering Input

Now let’s see if we can gather information interactively from the user.

  1. Upload numbergame.xml and checknumber.php to your TCNJ web account and edit the Voice request url for your number to point to your edited file.
  2. Call your Twilio number.
  3. What happened?
 <!-- numbergame.xml -->
<?xml version="1.0" encoding="UTF-8"?>
<Response>
    <Gather action="./checknumber.php" method="get" finishOnKey="*">
        <Say>I'm thinking of a number between 1 and 10, enter the number then press star.</Say>
    </Gather>
</Response>
 /* checknumber.php */
<?php
$number = 5;
$guess = $_REQUEST['Digits'];

header("content-type: text/xml");
echo "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n";

if($guess == $number){
    echo "<Response>";
    echo "<Say>Correct. The number was ".$number.". You win!</Say>";
    echo "</Response>";
} else {
    echo "<Response>";
    echo "<Say>Incorrect. Guess again.</Say>";
    echo "<Redirect method="POST">http://yourserver/numbergame.xml</Redirect>";
    echo "</Response>";
}

TwiML Gather Documentation

Navigating a Tree

Now let’s navigate a simple voice menu.

  1. Upload class-menu.xml and class-menu-processor.php to your TCNJ web account and edit the Voice request url for your number to point to your edited file.
  2. Call your Twilio number.
  3. What happened?
  4. How could you improve the user’s experience?
 <!-- class-menu.xml -->
<?xml version="1.0" encoding="UTF-8"?>
<Response>
    <Gather action="./class-menu-processor.php" numDigits="1">
        <Say>Welcome to Interactive Design for the Developing World, Spring 2015.</Say>
        <Say>For the course description, press 1.</Say>
        <Say>For the course location and meeting time, press 2.</Say>
        <Say>To leave a message for the instructor, press 3.</Say>         
    </Gather>
    <!-- Timeout defaults to 5 seconds, if customer doesn't input anything, redirect to prompt and try again. -->
    <Say>Sorry, I didn't get your response.</Say>
    <Redirect>./class-menu.xml</Redirect>
</Response>
/* class-menu-processor.php */
<?php 

$selection = (int) $_REQUEST['Digits'];
header('Content-type: text/xml');

echo '<?xml version="1.0" encoding="UTF-8"?>'; 
echo '<Response>';
 
switch($selection){
    case 1:
        echo "<Say>For many in the United States...</Say>";
    break;
 
    case 2:
        echo "<Say>I M M 470 0 3, meets every Friday morning, 9 30 a m to 12 20 p m, in A I M M 2 26.</Say>";
     break;
 
    case 3:
        echo "<Dial>215-264-2459</Dial>";
    break;
 
    default:
        echo "<Say>I didn't understand your selection.</Say><Redirect>./class-menu.xml</Redirect>";
    break;
}
 
echo '</Response>';
?>

TwiML Redirect Documentation

 Challenges

Twilio SMS or Voice

  • Using this influenza diagnostic decision tree, create a Twilio SMS or Voice Menu app a user can call or text with their symptoms and receive a recommended course of action in return
  • Hint: If using Twilio Voice TwiML, consider using the <redirect> verb

Amazon Echo

Continue working with IFTTT

Try creating a new recipe using SMS, Voice, or another connector from IFTTT.

Cordova

Have a look at this Cordova Sensors Plugin

See if you can install it and get one of the three demo apps working on your phone.

Assignment

Reading

Design Notebooks

VUI (Voice User Interface) Design Principles

  • Create a new slide in your Design Notebooks and label it “Best VUI  Design Practices”
  • Read Page 42 in the Freedom Fone User & Advocacy Guide
  • Review the class-menu.xml voice menu example from class
  • Based on what you read in the Freedom Fone guide, critique the flow for the class-menu.xml voice menu
  • Read Alexa Skills Kit Voice Design Best Practices
  • Compare the voice menu recommendations from Freedom Fone with those from Amazon for the Alexa. How do they compare?
  • Identify a couple of differences and similarities between designing for DTFM (dial tone buttons) and speech recognition (Alexa)

VUI Design Process

  • Create a new slide in your Design Notebook and title it “VUI Design Process”
  • Read pages 31-41 in Freedom Fone User & Advocacy Guide
  • Based on what you’ve done so far in class, and the design process recommendations in the Freedom Fone documentation, write a short paragraph describing what should your team’s next step be in designing your system.

*Submit links to your Design Notebooks in Canvas prior to next week’s class

Twilio SMS

Announcements

Design Notebooks

Lets take a few minutes to review your Design Notebooks

Teams & Projects

Based on your interests, lets organize into teams and begin to formulate project ideas.

Let’s also take a few minutes to set up our team pages on the class blog.

Twilio SMS

Log in to your Twilio account and navigate to your number configuration screen.

Let’s take another look at TwiML and run through some examples using the <message> verb.

Example

A Twilio phonebook – Try swapping in phone numbers for friends and family members, have them text your Twilio number.

Challenges

Twilio SMS

  • Create an SMS app that parses through the <Body> Twilio parameter, looking for a particular keyword (hint: use strpos())
  • If the sender’s message contains a particular word, send one response, otherwise send a different response

Amazon Alexa

  • If you don’t already have one, create an account on IFTTT have a look around to get a feel for it
  • Log out and then log back in using Prof. Thompson’s account
  • Navigate to the IFTTT Amazon Alexa Channel
  • Use  IFTTT to message your Twilio number using SMS
  • When you receive an SMS from Alexa, forward the message to someone you know (hint: try using the <to> attribute of the TwiML <message> verb)

Cordova

Assignment

Read

Project Concepts

  • Meet with your team members to iron out the specific problem your prototype will address
  • Together with your team members, craft brief a problem statement (3 sentences or so)

Design Notebooks

Project Concepts

  • Individually, create a new slide in your Design Notebook and title it “Project Concept”
  • Include a link to your team’s problem statement in your slide

Project Teams

  • Create a second slide and title it “Project Team”
  • Identify what role you would like to play on your team and write a brief paragraph outlining how you would like to approach fulfilling your role
  • List your team members and what (general) role each will be fulfilling on the team (e.g. designer, programmer, content developer, etc)

Voice Applications

  • Create a 3rd slide and label it “Voice Applications”
  • Identify three examples from the case studies or possible scenarioes you read about in the Freedom Fone User Guide that appeal to you
  • Do any of them bear similarities to your team’s project concept? Write a paragraph outlining how they do or don’t

*Submit links to your Design Notebooks in Canvas prior to next week’s class

Partner Overview Presentations

Announcements

Design Notebooks

Let’s take a few minutes to review a few of your Design Notebooks.

Twilio & TCNJ Web Account Setup

Twilio

Let’s head over to Twilio and create our accounts, it’s free.

Once you’ve created an account, go ahead and set up a new phone number, just make sure the number supports both SMS  & MMS.

Let’s take a couple of minutes to look over some of the features of your new number.

TCNJ Web Account

Take a minute to confirm that you have a functioning TCNJ web account:

Open a browser and type in the following URL:

http://www.tcnj.edu/~your_username

Does the URL resolve? If so, you’re good to go. Otherwise, follow the following instructions:

Setting up a web account on TCNJ’s servers

Your First SMS App

Create a TwiML file

  1. Open up a new file in your favorite web development text editor
  2. Navigate to https://github.com/IMM-470-03-Spring-2015/sms/blob/master/hello.xml and copy the code to your new file
  3. Save the file as hello.xml
  4. Using Filezilla or another FTP client, log in to your web account and create a folder to hold your SMS code
  5. Upload your hello.xml file to your SMS app directory

Configure Your Twilio Phone Number

  1. Navigate back to your Twilio account and click Phone Numbers->Manage
  2. Click on your phone number to bring up it’s configuration page and navigate down to the Messaging section
  3. Type in the url to your TwiML file in the Request URL field
  4. Click Save at the bottom of the page
  5. Text Your Twilio Number!
  6. What happened?

Amazon Echo Unboxing

Can you guys help me unbox the class Amazon Echo?

Let’s set it up!

Partner Presentations

Welcome Katie, Tracy & team!

Assignment

Reading

Design Notebooks

Partner Projects

  • Create a new slide in your Design Notebooks and title it Partner Projects
  • Order the projects from 1 to 3 in the order that they appeal to you
  • Write a paragraph or two detailing why your top choice appeals most to you

Interfaces

  • Create a new slide in your design notebook and title it Interfaces
  • Consider the reading from this week, the two blog posts in particular
  • Write a paragraph for each, reflecting on the basic premise of each article
  • Do you agree with each? Do your own personal experiences map to the reading?
  • Finally, did you read any of the other links? The Cordova documentation, the Alexa documentation or the Xfinity documentation?
  • Write a paragraph reflecting on why it interested you.

*submit links to your Design Notebooks in Canvas