IP3 video of testing session

Interactive Prototype III: Testing

Outcomes

Users were asked to play a basic version of the game using a modified stick to tap coloured balls of play dough and then provide feedback.  The coloured dots on the screen corresponded to coloured balls of dough.  Users had 25 seconds to complete 25 dots. The screen displayed a countdown timer, the user's high score, and the background colour changed every 5 seconds from yellow to navy to simulate sunset.

Users played the game multiple times, generally three times.  All wanted to beat their previous score by playing again.  Less than half the users (3/7) tested noticed the background colour changing.

I measured user feedback through observation, in-person questions and an online questionnaire. The data from these measures can be found here.

I wanted to know:

• how the background colour changing simulate the change in time, specifically from daytime to nighttime (in lieu of an actual sun being on the stage), and
• if users like being able to see their high score on the screen (they had previously remembered them).

Reflections

The format of the prototype worked well with the 'wacky stick' being a reliable game controller that allowed me to test other aspects of the game without resorting back to keyboard controls.

It was useful to be near the users when they filled out their survey to see their surprise that the background colour changed, some even went back and started playing agin just to see that it was true.  I received solid feedback and agreement that adding the high score to the stage was useful.  Users also commented on how fun the game was without prompting.

The testing protocol went smoothly, I felt much more confident at this last prototype delivering consistent instructions to users without relying on a script and talking through the various aspects of the game.Feedback received indicated that I gave clear instructions but I could have possibly asked different questions like "did the session flow in a logical way".

Effectiveness

The prototype worked really well and I received clear feedback on the use of high scores and changing colour background.  Many users did not notice the coloured background but still found the game well paced and responded to running out of time.  I am interested in whether the changing background is necessary to show the passage of time at all or if it could/should be more obvious (i.e. through other elements).

Constraints

By having an easier-to-build but more reliable controller for this prototype, I was less reliant on the input method for testing.  This meant I could successfully test the items I wanted to without interference.  It was interesting how users responded so positively to the stick controller, it has made me reconsider the input method entirely.

Implications

Changes to my concept

• possibly add other indicators of time passing like music getting faster or time flashing
• test different input methods (e.g. foot pads vs wacky stick)

Future prototypes

 Future prototypes
• on-screen feedback for wrong control pressed (not just sound)
• add movement to cat (e.g. jumping on press)
• add sun rising and setting to stage (would this make the changing background colour more noticeable, does it need to be noticeable)
• make start/stop easier (enter or other control)
• add in the extra bits (power-ups, levels, etc) 

Future testing sessions

• Wider range of users, maybe children

Interactive Prototype III: Testing Data

Here is the raw data from the testing sessions that were run for the Interactive Prototype 3 of Meow Meow Cat.  The data forms two parts, the observations recorded during the session and the questionnaire users filled out.

Testing session: 9 November, in-class session

Users:

• 7 total
• 4 male, 3 female

Observations

User #1 

• played 3 games 
• “I have to memorise where the different colour are” 
• “oh damn, I hit the wrong button"
• times (seconds): 
• 8, 7, 11

User #2

• played 3 games 
• Do you want another go -> “Yes!”
• On the second go, user reset game themselves which caused an error.  (Code is set up so you have to end game then reset)
• times (seconds): 
• 16, x, 11

User #3

• played 2 games 
• Body language response to being told that they have 25 seconds to complete 25 dots – apprehensive, tense, excited
• After 1st go -> I ned to see if I can beat my time
• times (seconds): 
• 13, 13

User #4

• played 3 games 
• times (seconds): 
• 12, 9, 13

User #5

• played 2 games 
• “This tech is hardcore” in response to the stick
• “oh, this is fun”
• didn’t notice background colour change
• times (seconds): 
• 8, 12

User #6

• played 3 games 
• “play another round? Definitely!” 
• “see if I can beat my highest score"
• times (seconds): 
• 11, 15, 16

User #7

• played 3 games 
• “did the background change?”
• “my first time was my best – weird”
• Clarified how the scoring worked
• times (seconds): 
• 13, 10, 9

Observation overview

• Stick/play dough combination worked really well, users seemed to find it fun to use/play
• Many commented that they didn’t notice the background colour change until they were prompted in the survey and some even went back to check
• Most wanted to play to beat their score
• Game played 19 times
• Times:
• Highest Score: 16 seconds
• Lowest Score: 7 seconds
• Average Score: 11.5 seconds

Questionnaire results

Total responses: 7

Q1. Were you aware that time was running out during the game? 

Yes - 6 users
No - 1 user

Q2. What indicated this to you?

• The number in the top left corner and by the score at the end
• Timer at top
• The counter at the top of the screen
• There was a timer in the top corner. Which I did not actually look at, and then the background colour would also change. 
• Was there a ticking clock going on in the background, that made me fele like the pressure was on? 
• I was too absorbed in the game
• When you told me.

Q3. Do you remember your best time?

Yes - 7 users

Q4. Was it helpful to have an on-screen record of your best time?

Yes - 7 users

Q5. Did you notice the background colour changing?

Yes - 3 users

No - 4 users

Q6. What did the background colour represent to you?

• i was too focused on the dots!
• I thought it stayed the same color yellow
• Getting closer to the end of the time allowed and the end of the game - increasing urgency.
• Stress! It made me think I was running out of time, but I was not sure how many times it would change or how much longer I had left when the colour changed.
• I didn't notice it was changing, I was focused on the colours.
• The questionable state of western foreign relationships within the middle-east

Q7. How would you describe the instructions for this testing session?
Scale: 1 'Clear - it was easy to understand what to do' to 5 'Confusing - I did not understand what to do'

1: Clear - 7 users

Q8. Any other comments?

• It was very fun and made me want to keep playing to improve my score
• Really liked your game. It was lots of fun. I tested all your prototypes and have definitely gotten quicker!
• It was fun - a good quick game.
• Whacky Stick! Perhaps some sort of instruction on how many colour changes there would be before time ran out or if the background music was to increase in speed to indicate that time was getting closer and closer to running out.
• There was really fun, simple yet clean and effective. 
• fun game
• I loved the fact that is was so fast paced. After trying my luck the first time I definitely wanted to beat my highest my score.

Questionnaire overview

Most or all users could agree that:

• my instructions were clear (7/7)
• they were aware time was running out (6/7)
• the timer showed them this (5/7)
• they could remember their best time (7/7)
• they liked having an on-screen record of their best time (7/7)
• it was fun (6/7 mentions in free comments)

However, it was almost half and half about the changing coloured background.

• 4/7 did not notice the change at all (many were surprised when they got to this question)
• those that did notice said it added to their sense of urgency
• Suggestions include:adding other elements to show end of time approaching 
• increase music speed
• graphic of sun setting over 'sky
• flashing timer of 5 seconds to go

The 'whacky stick' interaction also seemed to be popular and successful even though it was not specifically tested in this prototype.