User Testing, Continued

Top line progress

Over the past two weeks, I’ve continued to user test the workflow guide with designers and began user testing with low vision/blind electronics students.  I’ve done some research on other ASSETS abstracts in preparation of writing my own.

A detailed account of my process and progress is below.

Design Feedback

Debriefing the usability testing with Keerthana:

  • There’s a floating Ohm symbol in the bottom of the template
  • Name the red box as the height of the schematic
  • The guides don’t save, draw them with the line tool
  • “Gray text can be sentence case”
  • The SVG flattened my combined Braille and text boxes
  • Switch #4 and #5 in the Workflow Guide
  • Add another step on how to save and name and format
    • If your schematic is Analog_In, the title is “Analog In”
  • Be more specific as to what a Key Template is
  • Be more specific as to what a “label” is. Is it the file name?
  • Next time have the designer read the style guide

Debriefing the usability testing of tactile schematics with Antonio:

  • Put testing questions in order of the schematics in binder
  • First, have a page of all the symbols in the binder
  • Pot
    • Ground is too big
    • Explain microcontroller input
  • Arduino Switch
    • Put ground label next to ground symbol
    • The pins aren’t representative of how many are on an Arduino
    • The connection dots could be bigger
  • LED Resistor in Series
    • The key says LED twice
    • Put ground label on bottom next to symbol
  • Emily’s LED Resistor in Series
    • Likes a tighter resistor like Emily’s
    • Likes in and out microcontroller to use a leader line
  • Transistor
    • The connection dot is JUST RIGHT
  • Battery
    • Battery label is sentence case
    • Use leader line for battery symbol
  • Make a tactile thank you with the gift card

Debriefing the usability testing of tactile schematics with Chancey:

  • What is the bar under +V?
  • Make sure the inconsistent lines are fixed by designers
  • Show the symbol library first
  • Show the battery loop 2nd
  • Potentiometer:
    • Label resistor
  • Inside Pot
    • If the carbon resistor is thick, make it solid
    • Label the dots
  • Analog in:
    • Grey line before ground feels like it disappeared (too light)
  • Arduino Switch
    • Why does it say “3V3”? (3.3V)
    • S1 should be “Switch 1”
  • LED resistor
    • Spell out “mm” in key

Usability Testing: Two Types of Tasks

The hierarchy for how these schematics are tested is:

  1. Are they readable?
  2. Can they be used to build a circuit?
  3. Can they be reproduced by other designers?

You could argue that 3 and 4 could be swapped, but the main purpose of reading them is to understand and to apply. Currently, my testing script focuses on whether or not the schematics are comprehensible. It’s hard to prove, however, that they are as useful as complex audio descriptions. In order to do so, I would have to have someone build the circuit themselves.  The midterm feedback panel of professionals all encouraged me to “close the loop”, having the participant build the circuit out. Fortunately, they insisted that it only needs to be one participant. In my case, I will ask Antonio.

In the broader context of this research, I would want to expand on my usability testing to include physical computing materials. This would translate to two sections of tasks:

  1. Comprehension tasks: finding the resistor or reading what type of microcontroller was used, and explaining the circuit back to me in their own words.
  2. Application tasks: constructing a working circuit using the diagram as a reference.

Debriefing the usability testing with Vidia:

  • What are the standards for the symbol abbreviations?
  • Make a visual style guide for designers
  • Remove “and other modules” from workflow guide
  • Fix all the grey lines in the SVG
  • URL/link as link, so don’t have to copy and paste
  • Sketch doesn’t have the artboard when downloaded from file
  • Not clear what this means : “Use to save space if there are > 3 labels or if there’s already abbreviations”
    • To create more whitespace?
  • Not clear about the naming : “If your schematic is “analog_in.svg”, the title would be “Analog In””
    • So when I translate it, should it be capital or not?
  • It’s better to also put the elements size in the guideline on the right
    • Title
      • Braille: black (#000000) 30pt
      • Regular font: gray (#D1D2D4) Helvatica Regular 30pt
  • It would be nice if you set width & height for the artboard so if it’s opened in Sketch, I know exactly what the paper size is for to print
  • The groupings are cluttered in Sketch
  • Do I need to change the color of the line connection? Bc if it’s not grey then it’s not gonna be printed?

Debriefing the usability testing with David:

  • He didn’t want to read all the words in the workflow guide, he just wanted to get started
  • A more visual guide would be helpful
  • The red guides remained when he was done

Types of Annotations

The more captions and annotations you add to a graphic, the more cluttered it gets. Researchers have experimented in various ways to solve this issue. One being using an iPad and image recognition to create audio annotations. When the user moves their finger to a part of a tactile educational model, the screen reader reads out annotations on that element.

Other research has explored use of audio and haptic feedback as a form of annotation, combatting the concern with Braille annotation illiteracy.

Other ASSETS Papers on Tactile Graphics

  1. FluxMarker: Enhancing Tactile Graphics with Dynamic Tactile Markers
    1. Pain Points:
      1. Traditional tactile graphics are static
      2. Dynamic displays are prohibitively expensive
    2. Solution: Inexpensive, movable tactile markers.
    3. Pros:
      1. User observation was clearly defined: interviews with 4 blind participants asking about challenges with current tactile options.
      2. User tasks were clearly defined: analyze a prototype using a map
      3. Participants were clearly defined: 6 blind users
      4. This abstract is more fleshed out than the first one, the focus being on the process
  2. A Haptic Memory Game using the STRESS Tactile Display
    1. What was their research process? Who were their personas/participants? The effectiveness is unclear for me, reads more like a proof of concept.
  3. Evaluating Author and User Experience for an AudioHaptic System for Annotation of Physical Models
    1. Uses computer, webcam, and markers to hear labels spoken when a 3D model hotspot is touched.
    2. Pain point: Affixing a braille label to an object is bound by limited space and Braille illiteracy.
    3. Solution: audio-haptic labels
    4. Pros:
      1. The user study description
      2. The outcome and next steps descriptions

Weekly Status

Phase 1 | Observation 1/29/19-  2/12/19 |

  1. Organize existing research for a status check and onboarding Amy Hurst.
  2. Complete a competitive analysis on other electronics/microcontroller education techniques for low vision readers.
  3. Begin IRB approval process.
  4. Register and take IRB training
  5. Look into how similar the best practices are for other graphics, like AFB
  6. Make Tom and Amy’s edits to the style guide
  7. Add visuals to style guide
  8. Write How-To guide

Phase 2 | Ideation 2/12/19-  2/26/19 |

  1. Consider user testing methodologies.
  2. Develop personas.
  3. Test style guide with Emily.
  4. Design the main 6 schematics for testing.
  5. Prepare IRB documents.
  6. Create the inventory of tactile graphics with cataloguing in order of PComp labs
  7. CITI Investigators Common Rule training/Refresher course.

Phase 3 | Rapid Prototyping 2/26/19-  3/12/19 |

  1. Define tasks for testing.
  2. Write user testing script.
  3. Go through Emily Lin’s itemized feedback and apply to Style Guide
  4. Continue to refine Style Guide
  5. Finish Cayuse submission
  6. Attend NYU IRB info session on 2/26 from 2-4pm
  7. Update Usability Testing questions
  8. Test with Krizia
  9. Prepare binder of printed tactile graphics for testing.
  10. IRB approval

Phase 4 | User Feedback | 3/12/19- 4/2/19 |

  1. Make a doc of designs: me, Emily, Krizia, side by side
  2. Make a questions list for Chancey
  3. Write user testing script
  4. User test Style Guide again with another designer (Azalea)
  5. Confirm feedback meeting/schedule user test date with Chancey
  6. Contact Research and Development at the NFB to connect with a user test population.
  7. Contact Josh Miele, Ken Perry of the APH for design feedback
  8. Design a Grove Shield schematic for Chancey Fleet and her Arduino group
  9. User testing on and off NYU campus
  10. Debrief by listing the most serious issues.
  11. Order the issues by how serious they are- fix the top ones first.

Phase 5 | Iteration | 4/2/19- 4/16/19 |

  1. Repeat Phase 3 and 4 as many times as necessary to optimize the graphics for a larger user set.

Phase 6 | Implementation | 4/23/19- 4/30/19 |

  1. Prepare presentation of designer’s journey.
  2. Print and bind a book of the completed tactile graphics.
  3. Build an online hub that includes:
  • Style guide
  • Workflow guide
  • Research abstract
  • Downloadable book of all the finished tactile graphics.

Leave a Reply

Your email address will not be published. Required fields are marked *