Test Script Thank you for agreeing to participate! This study will have three stages: 1. Pre-Study Questionnaire (6) 2. Task Session (you will be working with different features of a whiteboarding app) a. Session 1 (7 tasks) b. Session 2(5 tasks) 3. Post-Study Questionnaire This is your Starbucks gift card. [Provide compensation] Stage 1: Pre-Study Questionnaire Before we jump in, I’ll ask a few questions about your current habits on your experience in remote online meetings for work. (Moderator to help build rapport) 1. What are your remote meetings typically about at work? Length and content? 2. Which online collaboration app have you used for remote meetings at work? Zoom Google Hangouts BlueJeans Webex Others:__________________ 3. Which device(s) have you used for remote collaboration at work? Laptop Desktop Ipad Mobile Others:__________________ 4. When you are using the app what feature(s) are you typically using at work? Video Call Chatboard Whiteboard Audio Call Call Recording Screen Sharing Screen Recording Others:___________________ If the participant use screen share to share their work on another tool during the meeting, ask what that tool is and the purpose 5. Do you sketch or write things out while explaining or talking to other people during remote collaboration meetings for the purpose of demonstrating ideas? ( Yes No ) 6. How do you do that today with the tools you said you use? Stage 2: Test Session Moderator: Thanks for answering all the questions, now we are moving to the next session where we are going to show you an online collaborative tool. [Show participant the Webex meeting app that is logged in and ready for joining a meeting] Simply look at it and don’t interact with it yet. We’ll ask some questions about what you saw and then we’ll move onto using it. Just a kind reminder that we are not testing you so there is no wrong way to do any of the tasks and there will be no wrong answers. Do you have any questions so far? [If not, move on.] Stage 2-1 Participant Initiating Whiteboard Sharing Scenario: Let's pretend that you have scheduled an online meeting with your colleague, Jane, to discuss a new product idea. You have logged into your online meeting account and have joined a meeting. Jane is speaking and now you are on the main homepage of this online meeting. Do you have any questions so far? [If not, move on.] Task 1: Explain Meeting Landing Page [Participant in the meeting and on the meeting page.] Moderator: Please meet your college Jane. Collaborator: Hi! Moderator: Simply look at this page and don’t interact with it yet. Could you explain what you see on the screen? Moderator: Thanks for answering that, now we are moving to our next task. Task 2: Create a Whiteboard [Staring on the meeting page with meeting on-going] Moderator: Before we get started, just trying to remind you that please try to think out loud throughout the process since it will provide us with resourceful insights. Also remember at all times that we are not testing you so there is no wrong way to do any of the tasks and there will be no wrong answers. Collaborator: Hi! For this meeting we are going to brainstorm color options for house painting. I think we could go with some bright colors like red, yellow, blue and purple, what do you think? Moderator reads scenario: Now that Jane has brought up some project ideas and you wish to quickly take notes or draw. Listen to what Jane says, then show me how you might get started. [Note how the participant goes to the whiteboard] ❏ Clicks on the toggle menu on the right ❏ Selects the whiteboard feature ❏ Clicks on a new whiteboard button ❏ Clicks on one of the existing whiteboard ❏ Other: _______________ 1. On a scale of 1 - 5, please rate how easy or difficult it is to complete this task. 1 is very difficult, 5 is very easy. ( 1 2 3 4 5 ) 2. Why did you give it that rating? Task 3 - Explain Whiteboard Landing Page [Staring on the whiteboard canvas] Moderator: Simply look at this page and don’t interact with it yet, could you explain what you see on the screen? ❏ Participant identifies the whiteboard ❏ Participant identifies the exit button ❏ Participant identifies the pen tool icon ❏ Participant identifies the color palette options ❏ Participant identifier the eraser ❏ Participant identifies the sticky note icon ❏ Participant identifies the new whiteboard icon ❏ Participant identifies the share/ save icon ❏ Other notes: __________________________ Moderator: Thanks for answering that, now we are moving to our next task. Task 4 - Do participants understand the whiteboard is NOT being shared [Staring on the whiteboard canvas] Moderator: What do you think Jane can see on her screen? ❏ Participant feels jane can see the whiteboard ❏ Participant feels jane can see and edit the whiteboard ❏ Participant isn’t aware that the whiteboard isn’t shared and is surprised to know that it isn’t ❏ Participant has to be told that it isn’t shared ❏ Others:________________________ Task 5 - Share Whiteboard [Staring on the whiteboard canvas] Moderator reads scenario: Let’s say now you want Jane to work on the whiteboard with you. Could you show me how you’d do it? ❏ Participant successfully finds the share button ❏ Participant clicks on share ❏ Participant understands what sharing means in this context ❏ Participant got stuck ❏ Couldn’t understand why he has the share? ❏ Was not able to find the share button ❏ Others:________________________ Moderator: 1. Now what do you expect your teammate to see? 2. What do you expect your teammate to be able to do with this whiteboard you shared? 3. On a scale of 1 - 5, please rate how easy or difficult it is to complete this task. 1 is very difficult, 5 is very easy. ( 1 2 3 4 5 ) 4. Why did you give it that rating? Task 6 - Add Sticky Note on Whiteboard [Staring on the whiteboard canvas] ow that you have created the whiteboard and are working on it with Moderator reads the scenario: N Jane. You want to brainstorm around the new project idea with Jane. Please communicate with Jane and work on it together. Collaborator: Could you put down the color options we discussed on sticky notes so we can better organize them and move them around? Moderator: Please try to think out loud throughout the process since it will provide us with resourceful insights. Also remember at all times that we are not testing you so there is no wrong way to do any of the taska and there will be no wrong answers. ❏ Participant finds the sticky note icon ❏ Participant creates a new sticky note ❏ Participant writes/types on the sticky notes ❏ Participant clicks on done ❏ Participant cannot add a sticky notes ❏ Participant did not know there is a sticky note feature ❏ Participant failed to find the sticky note feature ❏ Participant did not know how to create a sticky notes ❏ Other: _______________ Moderator: On a scale of 1 - 5, please rate how easy or difficult it is to complete this task. 1 is very difficult, 5 is very easy. ( 1 2 3 4 5 ) Moderator: Why did you give it that rating? Task 7 - Draw on Whiteboard [Participant on the whiteboard canvas] ou want to draw some sketches to better illustrate the product ideas you Moderator reads the scenario: Y came up with Jane. Show me how you’d sketch. [Collaborator drew the house and put the sticky notes on the house, againsting the walls. ] Collaborator:I just drew the house and put the color we discussed next to where I’d like the house to be painted. Could you color the house? ❏ Participant uses the pen to draw shapes ❏ Participant uses the eraser (optional) ❏ Participant cannot use the tools ❏ Participant expects to see more options for the drawing tools ❏ Participant did not know how to select the pen ❏ Participant did not know how to change pen color ❏ Participant did not know how to use the eraser ❏ Other: _______________ 1. On a scale of 1 - 5, please rate how easy or difficult it is to complete it. 1 is very difficult, 5 is very easy. ( 1 2 3 4 5 ) 2. Why did you give it that rating? Task 8 - Save Whiteboard [Participant on the whiteboard canvas] Scenario: Let’s say you want to keep a copy of the whiteboard on your tablet to refer to it for working on the project. Could you show me how you’d do it? ❏ Participant find the download button ❏ Participant clicks download successfully ❏ Participant cannot find the save button ❏ Note what path he follows and try to understand why the participant is confused 1. On a scale of 1 - 5, please rate how easy or difficult it is to complete it. 1 is very difficult, 5 is very easy. ( 1 2 3 4 5 ) 2. Why did you give it that rating? Stage 2-2 Participant Receiving Whiteboard Sharing Moderator: Thanks for doing all the previous tasks, now we are moving to the next session where we are going to ask you some questions about what you saw and then we’ll move onto using it. Do you have any questions so far? [If not, move on.] Task 1 - Explain Expectations Receiving a Shared Whiteboard [Start on the meeting homepage] Moderator reads the scenario: you are now in a new meeting with your Colleague Jane on the same project. Simply look at this page and don’t interact with it yet. [Jane: I am using the whiteboard and just drew a house] Could you explain what you see on the screen? Task 2 - Collaborate on Whiteboard [Start on the meeting homepage while the collaborate is sharing the whiteboard] Scenario: Now Jane needs your help on her drawing, please go ahead and help her. [Jane: I just added sticky notes with new color options I think would go better. Could you help me draw color on the house? ] ❏ Participant travels back to the whiteboard options ❏ Participant finds the whiteboard the collaborator created ❏ Participant opens the whiteboard the collaborator created ❏ Participant successfully works on the whiteboard simultaneously ❏ Participant fails to complete tasks ❏ Participant thought it will be happen automatically ❏ Participant thought he will be notified when the board was created ❏ Participant didn’t think of going back to repository ❏ Participant couldn’t figure out which whiteboard was created ❏ Others: ________________________ 1. On a scale of 1 - 5, please rate how easy or difficult it is to complete it. 1 is very difficult, 5 is very easy. ( 1 2 3 4 5 ) 2. Why did you give it that rating? Task 3 - Exit the Whiteboard and the Meeting [Start on the whiteboard canvas] Scenario: Now you and Jane have done using the whiteboard and you want to leave the whiteboard and chat with Jane face to face in the video call. Could you show me how you would do it? ❏ Participant finds the whiteboard the collaborator created ❏ Participant opens the whiteboard the collaborator created ❏ Participant fails to complete tasks ❏ Participant didn’t think of going back to repository ❏ Participant couldn’t figure out which whiteboard was the right one ❏ Others: ________________________ 1. On a scale of 1 - 5, please rate the difficulty in completing this task. 1 is very difficult, 5 is very easy. ( 1 2 3 4 5 ) 2. Why did you give it that rating? Scenario: Now that your meeting is over and you want to leave, could you show me how you’d do it ? ❏ Participant finds the “X (End Call)” button to exit the meeting ❏ Participant taps on the red window button to exit the meeting ❏ Participant couldn’t figure out how to exist the meeting Task 4 - Revisit Whiteboard Scenario: Now you want to go back to check the whiteboard you and Jane worked on together. Could you show me how you’d do it ? ❏ Participant travels back to the whiteboard options ❏ Participant finds the whiteboard the collaborator created ❏ Participant opens the whiteboard the collaborator created ❏ Participant fails to complete tasks ❏ Participant didn’t think of going back to repository ❏ Participant couldn’t figure out which whiteboard was the right one ❏ Others: ________________________ 3. On a scale of 1 - 5, please rate how easy or difficult it is to complete this task. 1 is very difficult, 5 is very easy. ( 1 2 3 4 5 ) 4. Why did you give it that rating? Task 5 - Delete Whiteboard [Start on the whiteboard canvas] Scenario: Now you don't need the whiteboard anymore and you want to get rid of it, could you show me how you’d do it ? ❏ Participant travels back to the whiteboard options ❏ Participant finds the whiteboard the collaborator created ❏ Participant deletes the whiteboard the collaborator created ❏ Participant fails to complete tasks ❏ Participant didn’t think of going back to repository ❏ Participant couldn’t figure out which whiteboard was created ❏ Others: ________________________ 1. On a scale of 1 - 5, please rate how easy or difficult it is to complete this task. 1 is very difficult, 5 is very easy. ( 1 2 3 4 5 ) 2. Why did you give it that rating? Stage 3: Post-Study Questionnaire Now the tasks are over, please take a step back and think about your overall experience with the system and I’ll ask you some follow-up questions. Overall, how satisfied are you with this whiteboard feature in online collaboration? Rate satisfaction on a scale of 1 - 5 ( Not at all satisfied) 1 2 3 4 5 (Very satisfied) Overall, how effectively does this whiteboard feature help you in online collaboration? Rate effectiveness on a scale of 1 - 5 ( Not at all effective) 1 2 3 4 5 (Very effective) Overall, how easy or difficult is the whiteboard to use in online collaboration? Rate ease of use on a scale of 1 - 5 ( Not at all easy to use) 1 2 3 4 5 (Very easy to use) Overall, how useful or not useful is whiteboard to use in online collaboration? Rate usefulness on a scale of 1 - 5 ( Not at all useful) 1 2 3 4 5 (Very useful) Overall, how well does the whiteboard feature keep you informed about what's going on? Rate wellness on a scale of 1 - 5 ( Not well at all) 1 2 3 4 5 (Very well) Overall, how clear is the information (such as icons, visuals, and buttons) presented on this whiteboard? Rate clarity on a scale of 1 - 5 ( Not at all clear) 1 2 3 4 5 (Very clear) Does this whiteboard feature have all the functions and capabilities you expect it to have? a. Why or why not? Please provide examples if you can.
Enter the password to open this PDF file:
-
-
-
-
-
-
-
-
-
-
-
-