Testing – The most hated and most important part of game development.

Hello and welcome back!

Today's topic is all about testing. I consider this more of an introduction to testing, but it will help to get you started on what to look for, how to look for it and why.

My apologies, but this is a bit of a long one. But I believe it's worth it!

When I talk to people about my previous job as a game tester for 2K games people usually say something like 'Oh my god that must have been a dream job!' as to which I reply 'My dear friend, let me explain the horrors that is game testing...'.

Game testing is not a glamorous thing, as any one who's done it will tell ya. However, that doesn't mean it isn't very important. In what follows I'll explain what it means to test games, how to do it and why it is so crucial. But first let me explain what happens when you don't test, under test or ignore your testers. I'll explain these from personal experience.

Lets start with not testing. I have worked for a game company we'll call 'G-Company'. They are terrible and most of my examples will be from working there. We produced a game (who's name I won't post here) and they wanted this action/adventure mobile game produced in a couple of months. They also did not plan for any testing. We made the game and it looked pretty decent. The art was nice and much of it was thought out a moderate amount. But then we handed it to people saying 'Here is our finished game.'. The people were excited to play, but then quickly turned around with an onslaught of complaints. “How do I do this?” “How do I do that?” “Where do I go?” “Why am I doing this?” “Who is that?” “What is this?”. There was a whole dungeon in the middle of the game that was so confusing and difficult to navigate through due to poor level design, poor camera handling, poor controls and no testing that most people gave up even trying the rest of the game there. This level was also riddled with glitches allowing the player to fall through the world, teleport to the end boss and countless others.

Overall much of these would have been easy fixes if we had tested while developing. Simple story adjustments, ensuring collision boxes were placed correctly and much more would have been a breeze to catch and fix if only we had tested.

What about under testing or ignoring your testers? I worked on another game and this one I was pretty proud of. It was a simple match 3 game in a 3D space so you could rotate the 'board'. I wasn't going to let this game take a dive like the last simply because we didn't test. This time I planned and pushed for testing hard. I wouldn't let up, no matter how much every one else thought it wasn't needed or a waste of time.

Finally testing could start so I and a few other co-workers ripped into the game. We tested for hours on end, every day until the game was released and then we tested more. We tore the game apart and reported every little issue. Everything from small graphics bugs, to big story points. We also sent it to people outside of the dev team, but still part of the company, to test.
The results? Many of the bugs were caught, changed and improved the experience. The game was much stronger than our previous one and I believe testing was a big part of that. But there were plenty of issues. Although we tested and improved the game much of the bugs and issues we found either were ignored by our management and CEO or the programmers, who got it in their heads that testers are only trying to get them more work and criticize them. Because of our management ignoring us there were plenty of story elements that just didn't make sense, didn't influence the game and had no impact at all. Later, because of this, players didn't care about nor like the story, often finding it confusing, boring or just dumb. People started to say having no story would improve the game, but the CEO wouldn't have it.

As for the programmers, let me say this to you all – And really, this goes for you artists too, you're not off the hook. - Testers aren't picking on you. They want to see and play an amazing game just as much as you. They want to enjoy the experience and feel they got their monies worth just like you. If they give you bug it's because something is wrong with your code, your art, and/or your game. They aren't thinking about how much they could make you work more. How much they could hurt your feelings by ripping apart your baby. They are making it a better product for every one and not making it personal for you.

Does that mean you should just take it? Maybe not. Not all bugs are bugs and not all bugs are priority or in the direction you may want the game to go in. This is where you should have a conversation with your tester(s). Find out what the real issue is and pick the problem apart. Testers can't always pin point what it is about the game or experience that they don't like, so they just mark it and move on in hopes the rest of the dev team can figure it out. Starting up a conversation with the tester will reveal much more about their experience and why it wasn't optimal. In turn this will improve your product ten fold. You are all a team and communication is a hug aspect of team work.

Also, As a side note, I'd like to share an instance while working at 2K. I was testing the game Fantastic Four, Rise of the Silver Surfer. That game was crap and terrible. I was so bored playing it for 14 hours a day, every day that I fell asleep playing it and I happened to hold the control stick forward while playing it. When I awoke I found the end credits. I had beat the game while sleeping. I wrote a bug report saying the game is too easy and that I beat it while just holding the control stick forward. The company stated 'not a bug' and ignore it. Soon reviews poured in after the game launched and sure enough a review stated 'I beat the game while sleeping.'.


Alright, so now we know the downfalls of not testing or not caring about the testing process. But how do you become a more meaningful tester yourself?

To begin with play games! Best way to learn is to see how other games do various things well or poorly. I don't simply mean play some of your favorite games and call it work. No, really dig into them. When you play your favorite game ask yourself 'Why do I like this one much more than its competitors?”, “What's different about this game than the others?”. “At what points do I put down this game and why do I pick it back up?” ,”How did this game overcome this obstacle and how did other games approach the same issue?”

Ask these questions and more. Think about why you are playing, what keeps you going or what makes you stop. Then play games you don't really like and ask the same questions. Try and figure out the thought process of the devs and their target market. Do you fit in that market? If so, why did it miss the mark on you?

Now that you've thought about these over arcing questions, its time to dig. Test the limits of your games. Try and break the game and see what happens. I'm going to give an example with Fallout 4 and if you've never played it there is mild spoilers. Nothing too big, but I obviously will be talking about the game. In this example I'll be addressing the very beginning of the game.



****** Mild Spoilers******


















At the beginning of the game bombs are falling and its time for you to get to your vault. This is the first intense moment of the game, but how do you know this? Whats the shift in mood? Well before this point you were just having a slow morning as you wake up and get ready for the day. Your spouse and son are taking it easy and your robot attends to the house. You have time to explore the house and take in the atmosphere with no pressure to move on or do anything specific.

Eventually a news report comes on and your robot sounds alarmed and beckons you to join him. The tone is much more serious now instead of the care free emotions that had just surrounded you. Next you need to evacuate and armed men point you in the direction you need to go. Exits are blocked and all the civilians are running to one location. It's clear where you need to go because of all the pointing and crowds. But what if you don't?

What if you instead try to leave this neighborhood? How does the game stop you from just exploring the waste land? Well in this instance if you run too far off the beaten path the nukes fall and you die. The game doesn't let you really explore at this point. It gives you the impression that you can explore with your house. You search rooms, check out comic books, talk to your family, etc. you feel like you have no limits. But you can never leave your house until the scripted point. Then you must follow the arrows to the next scripted point.

Feelings, level design, sound, and more make you – the player- do different things at different times.
Notice, also, that if you try to leave the starting location you can never see the rest of the world. The world design limits how far you can see, thus letting you wonder whats over that hill. This would help limit what needs to be processed, built and designed but makes it feel like there is much much more to the pre-bombed world.





















*****End mild spoilers*****






In other words what are you not supposed to do in a game? Now do it. Can you? How did the game stop you? Were you surprised? Did it keep up the suspension of disbelief, or did you become ever more aware that you are just playing a game?

But testing isn't all about breaking the game, it's also making it easier to use and more intuitive. I'll compare 2 games I enjoy and how you move about the map. In Divinity Original Sin you move the camera by using W,A,S and D, but in Pillars of Eternity the camera is moved using the arrow keys. As a PC player I find the WASD far more intuitive and it took me time to get used to the arrow keys. However before I got into PC I enjoyed the arrow keys much more and found that more intuitive. Both are good games, but I like Pillars a bit less just because of how unintuitive the controls are for me.

Testing is a lot of things all at once. How fun is the game? Do the controls make sense? Does the game work as intended? Can it prevent players from doing what isn't intended? Is it intuitive? Does it get its theme and emotion across? Is it easy to navigate? Is it easy to learn? At what points is it boring and how can you improve it?

It's all these and more. Testing is a huge part in games and makes a mediocre product into something that really shines. It teaches you what works and what doesn't and it makes you a better developer all around.

I hope this was helpful and if you'd like more examples of good and bad game design let me know!
Also, if you have a topic you'd like me to cover please tell me and I'd be happy to write something up about it.

I'm sure I'll be covering this topic again as it is very important and there is always something to be said about it. However next time I'll be moving on to 'Save step game design'. Save step is a form of planning and developing a product that can be considered done early if all goes wrong.
Take care