IP3 video of testing session

Interactive Prototype III: Testing

Outcomes

Users were asked to play a basic version of the game using a modified stick to tap coloured balls of play dough and then provide feedback.  The coloured dots on the screen corresponded to coloured balls of dough.  Users had 25 seconds to complete 25 dots. The screen displayed a countdown timer, the user's high score, and the background colour changed every 5 seconds from yellow to navy to simulate sunset.

Users played the game multiple times, generally three times.  All wanted to beat their previous score by playing again.  Less than half the users (3/7) tested noticed the background colour changing.

I measured user feedback through observation, in-person questions and an online questionnaire. The data from these measures can be found here.

I wanted to know:

• how the background colour changing simulate the change in time, specifically from daytime to nighttime (in lieu of an actual sun being on the stage), and
• if users like being able to see their high score on the screen (they had previously remembered them).

Reflections

The format of the prototype worked well with the 'wacky stick' being a reliable game controller that allowed me to test other aspects of the game without resorting back to keyboard controls.

It was useful to be near the users when they filled out their survey to see their surprise that the background colour changed, some even went back and started playing agin just to see that it was true.  I received solid feedback and agreement that adding the high score to the stage was useful.  Users also commented on how fun the game was without prompting.

The testing protocol went smoothly, I felt much more confident at this last prototype delivering consistent instructions to users without relying on a script and talking through the various aspects of the game.Feedback received indicated that I gave clear instructions but I could have possibly asked different questions like "did the session flow in a logical way".

Effectiveness

The prototype worked really well and I received clear feedback on the use of high scores and changing colour background.  Many users did not notice the coloured background but still found the game well paced and responded to running out of time.  I am interested in whether the changing background is necessary to show the passage of time at all or if it could/should be more obvious (i.e. through other elements).

Constraints

By having an easier-to-build but more reliable controller for this prototype, I was less reliant on the input method for testing.  This meant I could successfully test the items I wanted to without interference.  It was interesting how users responded so positively to the stick controller, it has made me reconsider the input method entirely.

Implications

Changes to my concept

• possibly add other indicators of time passing like music getting faster or time flashing
• test different input methods (e.g. foot pads vs wacky stick)

Future prototypes

 Future prototypes
• on-screen feedback for wrong control pressed (not just sound)
• add movement to cat (e.g. jumping on press)
• add sun rising and setting to stage (would this make the changing background colour more noticeable, does it need to be noticeable)
• make start/stop easier (enter or other control)
• add in the extra bits (power-ups, levels, etc) 

Future testing sessions

• Wider range of users, maybe children

Interactive Prototype III: Testing Data

Here is the raw data from the testing sessions that were run for the Interactive Prototype 3 of Meow Meow Cat.  The data forms two parts, the observations recorded during the session and the questionnaire users filled out.

Testing session: 9 November, in-class session

Users:

• 7 total
• 4 male, 3 female

Observations

User #1 

• played 3 games 
• “I have to memorise where the different colour are” 
• “oh damn, I hit the wrong button"
• times (seconds): 
• 8, 7, 11

User #2

• played 3 games 
• Do you want another go -> “Yes!”
• On the second go, user reset game themselves which caused an error.  (Code is set up so you have to end game then reset)
• times (seconds): 
• 16, x, 11

User #3

• played 2 games 
• Body language response to being told that they have 25 seconds to complete 25 dots – apprehensive, tense, excited
• After 1st go -> I ned to see if I can beat my time
• times (seconds): 
• 13, 13

User #4

• played 3 games 
• times (seconds): 
• 12, 9, 13

User #5

• played 2 games 
• “This tech is hardcore” in response to the stick
• “oh, this is fun”
• didn’t notice background colour change
• times (seconds): 
• 8, 12

User #6

• played 3 games 
• “play another round? Definitely!” 
• “see if I can beat my highest score"
• times (seconds): 
• 11, 15, 16

User #7

• played 3 games 
• “did the background change?”
• “my first time was my best – weird”
• Clarified how the scoring worked
• times (seconds): 
• 13, 10, 9

Observation overview

• Stick/play dough combination worked really well, users seemed to find it fun to use/play
• Many commented that they didn’t notice the background colour change until they were prompted in the survey and some even went back to check
• Most wanted to play to beat their score
• Game played 19 times
• Times:
• Highest Score: 16 seconds
• Lowest Score: 7 seconds
• Average Score: 11.5 seconds

Questionnaire results

Total responses: 7

Q1. Were you aware that time was running out during the game? 

Yes - 6 users
No - 1 user

Q2. What indicated this to you?

• The number in the top left corner and by the score at the end
• Timer at top
• The counter at the top of the screen
• There was a timer in the top corner. Which I did not actually look at, and then the background colour would also change. 
• Was there a ticking clock going on in the background, that made me fele like the pressure was on? 
• I was too absorbed in the game
• When you told me.

Q3. Do you remember your best time?

Yes - 7 users

Q4. Was it helpful to have an on-screen record of your best time?

Yes - 7 users

Q5. Did you notice the background colour changing?

Yes - 3 users

No - 4 users

Q6. What did the background colour represent to you?

• i was too focused on the dots!
• I thought it stayed the same color yellow
• Getting closer to the end of the time allowed and the end of the game - increasing urgency.
• Stress! It made me think I was running out of time, but I was not sure how many times it would change or how much longer I had left when the colour changed.
• I didn't notice it was changing, I was focused on the colours.
• The questionable state of western foreign relationships within the middle-east

Q7. How would you describe the instructions for this testing session?
Scale: 1 'Clear - it was easy to understand what to do' to 5 'Confusing - I did not understand what to do'

1: Clear - 7 users

Q8. Any other comments?

• It was very fun and made me want to keep playing to improve my score
• Really liked your game. It was lots of fun. I tested all your prototypes and have definitely gotten quicker!
• It was fun - a good quick game.
• Whacky Stick! Perhaps some sort of instruction on how many colour changes there would be before time ran out or if the background music was to increase in speed to indicate that time was getting closer and closer to running out.
• There was really fun, simple yet clean and effective. 
• fun game
• I loved the fact that is was so fast paced. After trying my luck the first time I definitely wanted to beat my highest my score.

Questionnaire overview

Most or all users could agree that:

• my instructions were clear (7/7)
• they were aware time was running out (6/7)
• the timer showed them this (5/7)
• they could remember their best time (7/7)
• they liked having an on-screen record of their best time (7/7)
• it was fun (6/7 mentions in free comments)

However, it was almost half and half about the changing coloured background.

• 4/7 did not notice the change at all (many were surprised when they got to this question)
• those that did notice said it added to their sense of urgency
• Suggestions include:adding other elements to show end of time approaching 
• increase music speed
• graphic of sun setting over 'sky
• flashing timer of 5 seconds to go

The 'whacky stick' interaction also seemed to be popular and successful even though it was not specifically tested in this prototype.

Week 13: Building IP3

For this prototype, I decided to keep iterating the concept and test different features.

I started by going over what I wanted to achieve from this prototype and features I had not yet developed including:

• change in timer from counting up (record how long it takes a user to complete a level) to counting down (users only have a set time to complete the level).
• change background colour to simulate sunset and passing of time
• add high-scores to stage
• add additional cue to which dot users are meant to press

I also thought about which controllers I would use and how the interaction would take place.  The floor-dots were great but unreliable and I didn't want to re-create them for this prototype.  I also didn't want to go back to using keyboard arrow controllers.  So, I looked for a simple way of having physical controllers for the game and went with playdough balls that corresponded to the colours of the dots on screen.  

When looking for colours for the background changes, I initially looked to blend yellow to navy.  They meant the 'middle' colours were really murky so I changed the colour steps to imitate sunset colours but also tried to avoid colours that were too close to the dot colours.  To help further with contrast, I also added a white outline to some dots to ensure they could still be easily seen on the screen. 

initial colour palette

final colour palette 

I was able to change the colours using a switch statement tied to the timer. The challenge I had wth adding the colours was resetting back to the start colour at the end of game.  To help me problem solve this, I had to draw out how the 'end game' functions interacted with each other as I had initially written this a few weeks ago so writting comments in code was definitely helpful.

 

Week 12: Interactive Prototype 3 - Final Prototype

What is the key function or interaction for your concept?

The user moves the character (Meow Meow Cat) forward through a series of coloured dots.  Users have a set amount of time to complete the game and are given visual cues through a timer and a changing background (from Daytime to nighttime).

I wanted to included a physical element to this prototype but without rebuilding the floor-dot-buttons from the previous prototype (or using the keyboard as an input).  Instead, I have continued to use the Makey Makey device but have substituted the dots you press with your feet for playdough you tap with your hands.

How does it work?

The user taps corresponding coloured balls of playdough to those displayed on screen.   A Makey Makey is used to connect the playdough controls with the computer.  A countdown timer is used to show users how long they have to complete a basic level (25 dots).  The background screen colour also changes colour with the timer to indicate the passage of time (sunshine to nighttime).  User's high scores are also displayed in the top left corner

What do you want to know about your prototype?

• Does the background colour changing simulate the change in time, specifically from daytime to nighttime (in lieu of an actual sun being on the stage)
• Do users like being able to see their high score on the screen (they had previously remembered them)

How do you want IP3 to work?

1. A testing computer is set-up connected to playdough through a Makey Makey device.
1. Four coloured balls of playdough have corresponding dots on them to the dots in the game. 
2. A wire is run from each ball of playdough to the Makey Makey.
3. A stick to tap the playdough is coated with reflective material and is connected to the grounding wire attached to the Makey Makey.
4. When the balls are tapped,  a signal is sent from the dots to the Makey Makey to the computer.  Each four colours corresponds to an arrow key.
2. User sits in front of the playdough and screen, ready to begin and is given verbal instruction:
1. There are four coloured balls in front of you
2. With the stick, tap the corresponding colour to the colour on the screen to move your character forward
3. Level ends when you reach the end of the row of dots
4. You only have 25 seconds to complete a level.  The background colour will change every 5 seconds to show the passage of time from day-time to night-time.
3. There is a start button that the tester will click to begin game
4. Game can end by either reaching the end of the row of dots or by pressing the 'end game' button.
5. Screen includes character on far left-hand side, a row of coloured dots (four colours with patterns), timer in top left corner, 'Begin game' and 'stop' buttons on top right-hand corner, 'High score' display in top right-hand corner below buttons.
6. User taps coloured playdough to move character forward
7. High-scores are records on the screen at the end of each game
8. Users can restart and replay as many times as they like 
9. User can stop play at any time

What is not included

• Appearance of sun or moon moving across the 'sky - only appearance of change through background colour
• More than one level (and level progression)
• Written instructions/walk through

Week 11: Testing IP2 videos

As the in-class testing session had some problems, here are two videos (apologies for the vertical filming orientation) of the first testing session with the two formations.

Interactive Prototype II: Testing

Outcomes

Users were asked to tap four coloured dots on the floor with their feet to play a basic version of the game and provide feedback.  The coloured dots on the screen corresponded to coloured dots on a floor.

Users played the game a minimum of four times (maximum of 7). Most user wanted to get the best time possible and kept playing until they felt they had achieved that.  Users would keep track of their best scores in their head.

I measured user feedback through observation, in-person questions and an online questionnaire. The data from these measures can be found here.

I wanted to know:

• What formation of dots is both challenging (users need to use both feet, can't press multiple dots at once) and fun (is not so hard that user gives up early on or does not want to play)
• How long it takes a user to complete a basic level (25 dots) and how that time improves over multiple goes.

Reflections

The physical format for the prototype was a good way to get more detailed feedback on how the game is played.  It allowed me to observe users' interaction and how physical it was as they became tired and 'puffed' trying to tap all the dots and beat their times.  It was interesting to see how competitive users were with themselves and how even though the timer was there for my benefit for tracking purposes, users manually took note of their times to improve upon.

Testing with more users who weren't from class was again useful.  It was different getting feedback from people who had no prior knowledge of the concept but equally valuable in class as I was able to get comparison feedback (e.g. this element worked better than last time).  A mix of new and previous users led to more insight.

Effectiveness

When it was fully functional, the prototype worked really well and I was able to answer the specific questions I had about the formation on the input. It was very clear both verbally and through the survey which formation was preferred (square).

The first session (not in class) was very successful but the in-class one had problems with one of the dots only working intermittently.  This was a significant obstacle in getting more users to participate in testing. I pulled apart the prototype after the session and am still not sure where the connectedness  failed.  It is a downside of quick prototyping like this that it wasn't robust enough to withstand transport and multiple testing sessions.

Constraints

The constraint of testing the physical input in the session meant that when it didn't work, I couldn't test.  There were no backup inputs as using a keyboard would not have answered the questions I was testing.  It was quite disappointing to have to stop testing because of this.

Implications

Changes to my concept

• concept is unchanged

Future prototypes

• Interactive Prototype 3  (IP3)
• add high-score to screen for users to see 
• change timer from count up to count down
• make background change colour with timer (sunshine to night timer)
• add additional visual cue to stage to make it clearer which dot is next
•  Future prototypes
• on-screen feedback for wrong control pressed (not just sound)
• add movement to cat
• change timer from count up to count down
• add sun rising and setting to stage
• make start/stop easier (enter or other control)
• add in the extra bits (power-ups, levels, etc) 

Future testing sessions

• possible use other input methods to test other elements of game to avoid having input breakdowns affect testing session (if inputs are not the element being tested)