Design Strategies: Interface Design
2010_ To investigate the affordance of gesture in interface design I developed a scenario and viewing matrix to understand patron and system behaviors and patterns. Through these methods I learned designing gestures was not the proper way to move forward, and concepted the Virtual Vitrine—a spin on a traditional vitrine that allows patrons to move past the experience of looking at artifacts to engaging and experiencing them. Breaking museum convention enabled a richer understanding of artifacts in their original contexts, intended purposes, and their materiality, form, scale, and history. The Virtual Vitrine assimilates natural gestures corresponding to specific artifacts, and through these gestures rich, dynamic information is revealed. (Top screen view only)
Content
PERSONA:
Zed Xander is visiting the MAD and comes across the new virtual vitrine. At first he is not sure what to do with the ojbect, but as he moves around it he sees the arm hole label and follows the instructions; Interact with our collection by placing your arms here. INITIAL INTERACTION PROVOKES PLAY Zed proceeds to place his arms into the holes and as he gets closer, he sees the top of the vitrine come to life. Artifacts zoom by underneath his virtual arms until they come to a stop. Zed sees a dark, 3D space with artifacts streaming out in all directions. As he begins to move his hands from side to side Zed notices the artifacts pan as a group depending on his hand positions; farther left he moves, the artifacts move faster right; farther right he moves, artifacts move faster left. When he moves his hands down (palms down), he notices he can see more and more artifacts, even though they are tiny. When his hands moves up (palms up), he sees less artifacts, but greater detail. SPECIFIC GOALS ELICIT EXPERIMENTING WITH GESTURES After several moments of playing with the pan and zoom function, Zed decides to interact with an artifact. While zoomed in and viewing 25 or so artifacts, he points to one, it highlights. He immediately points to another and notices it highlights, but the other de-highlights. His immediate reaction is to select one with his left hand and the other with his right; the same thing happens. After a few attempts, the system visually prompts a different hand gesture; showing selecting with 2-fingers making multiple selections. Zed gets it and decides to move on anyway. He selects the original artifact, pauses to think about how he could zoom in and recalls the earlier gesture for zooming, 2 HANDS, palms up, moving up, except this time he does it faster and the artifact quickly comes into view while the rest of the artifacts go down and slightly out of focus. CURIOSITY REVEALS SOMETHING UNEXPECTED Once in view Zed attempts to use 2 hands to handle and rotate the artifact; it works perfectly. A little curious, Zed attempts to ‘feel’ the artifacts texture by rubbing the side with 3-fingers. When he does this, he notices bits of type starting to form on the surface under his hand. He stops, pulls his hand back and then attempts to grab the type by pinching and pulling on it with 1-finger and his thumb with no avail. Immediately he pinches with 2 fingers and his thumb, as he pulls back, he notices information coming directly out of the artifact, but is unable to read it because of its orientation. With his left hand in the 5-finger grab gesture he used earlier to rotate the artifact and the other in a 2-finger pinch and pull on the end of the information, he rotates the artifact and text, making it legible. When Zed attempts to put the information back by closing his hands together, nothing happens, but when he pulls them apart, the artifact breaks into segments, revealing the individual pieces that make up the artifact. Amazed, he closes his hands and the artifact goes back together, then opens it up again. Does the previously retrieved information stay visible, disappear at the first break apart or slowly fade? Stopping with the artifact disassembled, Zed zooms in and out of specific pieces, examining them in superfine detail. MULTIPLE CHOICES REVEAL SYSTEM RECOMMENDATIONS Putting the artifact back together and zooming back out, Zed begins to browse, eventually zooming in and, remembering to use 2-fingers, selects multiple artifacts. Upon the third selection, Zed is subtly prompted by the system with two options; start a collection or compare artifacts. Zed decides to start a collection. After making the choice with 1-finger a visually distinct 3D rectangular tube zooms from left to right the width of the vitrine directly in front of the closest vitrine wall to Zed. The system prompts Zed to make a stiff “C” gesture to add the items to the new collection space. Making the “C” gesture Zed’s selected artifacts quickly move into the 3D area, lining up in order of selection. Zed selects three more artifacts to add, but then decides against one and deselects it by abruptly spreading his 2-finger select gesture. How does he know to do this? After making his selections Zed chooses to examine his collection. Recognizing a texture on the corner of the rectangular area from conventional software programs, Zed pinches it with 1-finger and pulls. As he does so, the main library quickly moves down and out of focus and his collection opens, filling the central area with the expansion of the 3D rectangular tube.
Zed Xander is a 27 year old accountant who visits museums from time to time.
SCENARIO:
Zed Xander discovers the MAD Virtual Vitrine
Intersection of BEHAVIORS and HAND GESTURES in the MAD Virtual Vitrine interface. Content: MAD collection Context: MAD gallery; life-size virtual vitrine
Hand
Arm combined w/speciļ¬c HAND gestures
2 HANDS: flat hands over entire space moving side-to-side and up-and-down shift focused plane on all axes; requires hand tensity recognition; a variation in palm direction controls y-axis movement; proximity to vitrine walls determines x- and z-axis movement—this works b/c the entire database is present.
Recognizes axes depths Proximity of hands to vitrine walls increases speed (OS physics) Recognizes proximity of HAND to HAND
Browsing
What about one palm up and one palm down moving in opposite directions? Does imply something is selected? What if the screen splits and multiple views are displayed simultaneously; artifact and rich content; comparing multiple artifacts? If comparing multiple artifacts, gesture could hold one zoomed in and the other out. 1-finger selects artifact - visible change 2-fingers - enables multiple selections - repeats visible change of 1-finger selection 3-fingers - (rub: reveals embedded information) 5-fingers - (grab:) w/1 HAND: stabilization of selection—used w/2nd HAND; w/2 HANDS: enables full axes rotation 1-finger w/thumb pinch and pull - control of GUI (?) 2-finger w/thumb pinch and pull - control of deeper information w/in artifacts 2-finger abrupt spread - deselect (?)
Selecting
Select multiple artifacts and store them: select the artifact and bring it toward the wall the user is standing in front of—literally bring the artifacts to you. Artifacts arrange in the new 3D space according to: order of selection or predetermined selection reasons determined by user. After making multiple selections, the user makes a stiff “C” gesture with either hand to add to collection.
Collecting
Single artifact selected and user wants to zoom in and interact with only this artifact: 3-finger rub w/visual change; followed by a 2 finger pinch and pull to reveal imbedded info. 2 HANDS; 5-finger grab and pull to break apart artifacts
Object Interaction
How is text input for login, naming and organizing collections, searching? Does it make sense to type conventionally in this space?
User Input
Zed Xander is visiting the MAD and comes across the new virtual vitrine. At first he is not sure what to do with the ojbect, but as he moves around it he sees the arm hole label and follows the instructions; Interact with our collection by placing your arms here. INITIAL INTERACTION PROVOKES PLAY Zed proceeds to place his arms into the holes and as he gets closer, he sees the top of the vitrine come to life. Artifacts zoom by underneath his virtual arms until they come to a stop. Zed sees a dark, 3D space with artifacts streaming out in all directions. As he begins to move his hands from side to side Zed notices the artifacts pan as a group depending on his hand positions; farther left he moves, the artifacts move faster right; farther right he moves, artifacts move faster left. When he moves his hands down (palms down), he notices he can see more and more artifacts, even though they are tiny. When his hands moves up (palms up), he sees less artifacts, but greater detail. SPECIFIC GOALS ELICIT EXPERIMENTING WITH GESTURES After several moments of playing with the pan and zoom function, Zed decides to interact with an artifact. While zoomed in and viewing 25 or so artifacts, he points to one, it highlights. He immediately points to another and notices it highlights, but the other de-highlights. His immediate reaction is to select one with his left hand and the other with his right; the same thing happens. After a few attempts, the system visually prompts a different hand gesture; showing selecting with 2-fingers making multiple selections. Zed gets it and decides to move on anyway. He selects the original artifact, pauses to think about how he could zoom in and recalls the earlier gesture for zooming, 2 HANDS, palms up, moving up, except this time he does it faster and the artifact quickly comes into view while the rest of the artifacts go down and slightly out of focus. CURIOSITY REVEALS SOMETHING UNEXPECTED Once in view Zed attempts to use 2 hands to handle and rotate the artifact; it works perfectly. A little curious, Zed attempts to ‘feel’ the artifacts texture by rubbing the side with 3-fingers. When he does this, he notices bits of type starting to form on the surface under his hand. He stops, pulls his hand back and then attempts to grab the type by pinching and pulling on it with 1-finger and his thumb with no avail. Immediately he pinches with 2 fingers and his thumb, as he pulls back, he notices information coming directly out of the artifact, but is unable to read it because of its orientation. With his left hand in the 5-finger grab gesture he used earlier to rotate the artifact and the other in a 2-finger pinch and pull on the end of the information, he rotates the artifact and text, making it legible. When Zed attempts to put the information back by closing his hands together, nothing happens, but when he pulls them apart, the artifact breaks into segments, revealing the individual pieces that make up the artifact. Amazed, he closes his hands and the artifact goes back together, then opens it up again. Does the previously retrieved information stay visible, disappear at the first break apart or slowly fade? Stopping with the artifact disassembled, Zed zooms in and out of specific pieces, examining them in superfine detail. MULTIPLE CHOICES REVEAL SYSTEM RECOMMENDATIONS Putting the artifact back together and zooming back out, Zed begins to browse, eventually zooming in and, remembering to use 2-fingers, selects multiple artifacts. Upon the third selection, Zed is subtly prompted by the system with two options; start a collection or compare artifacts. Zed decides to start a collection. After making the choice with 1-finger a visually distinct 3D rectangular tube zooms from left to right the width of the vitrine directly in front of the closest vitrine wall to Zed. The system prompts Zed to make a stiff “C” gesture to add the items to the new collection space. Making the “C” gesture Zed’s selected artifacts quickly move into the 3D area, lining up in order of selection. Zed selects three more artifacts to add, but then decides against one and deselects it by abruptly spreading his 2-finger select gesture. How does he know to do this? After making his selections Zed chooses to examine his collection. Recognizing a texture on the corner of the rectangular area from conventional software programs, Zed pinches it with 1-finger and pulls. As he does so, the main library quickly moves down and out of focus and his collection opens, filling the central area with the expansion of the 3D rectangular tube.
Zed Xander is a 27 year old accountant who visits museums from time to time.
SCENARIO:
Zed Xander discovers the MAD Virtual Vitrine
Intersection of BEHAVIORS and HAND GESTURES in the MAD Virtual Vitrine interface. Content: MAD collection Context: MAD gallery; life-size virtual vitrine
Hand
Arm combined w/speciļ¬c HAND gestures
2 HANDS: flat hands over entire space moving side-to-side and up-and-down shift focused plane on all axes; requires hand tensity recognition; a variation in palm direction controls y-axis movement; proximity to vitrine walls determines x- and z-axis movement—this works b/c the entire database is present.
Recognizes axes depths Proximity of hands to vitrine walls increases speed (OS physics) Recognizes proximity of HAND to HAND
Browsing
What about one palm up and one palm down moving in opposite directions? Does imply something is selected? What if the screen splits and multiple views are displayed simultaneously; artifact and rich content; comparing multiple artifacts? If comparing multiple artifacts, gesture could hold one zoomed in and the other out. 1-finger selects artifact - visible change 2-fingers - enables multiple selections - repeats visible change of 1-finger selection 3-fingers - (rub: reveals embedded information) 5-fingers - (grab:) w/1 HAND: stabilization of selection—used w/2nd HAND; w/2 HANDS: enables full axes rotation 1-finger w/thumb pinch and pull - control of GUI (?) 2-finger w/thumb pinch and pull - control of deeper information w/in artifacts 2-finger abrupt spread - deselect (?)
Selecting
Select multiple artifacts and store them: select the artifact and bring it toward the wall the user is standing in front of—literally bring the artifacts to you. Artifacts arrange in the new 3D space according to: order of selection or predetermined selection reasons determined by user. After making multiple selections, the user makes a stiff “C” gesture with either hand to add to collection.
Collecting
Single artifact selected and user wants to zoom in and interact with only this artifact: 3-finger rub w/visual change; followed by a 2 finger pinch and pull to reveal imbedded info. 2 HANDS; 5-finger grab and pull to break apart artifacts
Object Interaction
How is text input for login, naming and organizing collections, searching? Does it make sense to type conventionally in this space?
User Input
