Month: May 2016

Unit Testing Games: Tip #1

I previously discussed some benefits of Test-Driven Development for Games. Next, I’ll offer some tips for effectively adopting TDD and unit testing.

Keep tests small

Game teams strive to keep the size of their game assets to a minimum. This conserves disk space, and allows packing more content into a single release. Your unit tests should also be small, though for different reasons.

Small tests aid in comprehension and provide real-world documentation that is always up-to-date. Naming functions is hard, and sometimes the name doesn’t properly convey behavior. Unit tests identify exactly HOW the function should be called and if/what it should return.

A passive benefit of unit testing is forcing you to examine your interfaces. Spaghetti code usually inflates the size of your tests. Functions with multiple dependencies are hard to understand and refactor. If the dependencies require sizable setup, you lose even more flexibility.

Larger tests also translates to more written code, which can be a barrier to a team accepting the process.

The Weapon class in this code sample is very coupled to other game objects. The fire() function in particular has implicit dependencies on the Player class (someone must be holding the weapon), the World class for line-of-sight checks, and the Sound Manager class to play sounds.

def fire(self, target):
    # Must fire at a valid target player
    if target is None:
        raise MissingTargetException()

    # Weapon must be held by someone to be fired
    if self._player is None:
        raise NoWeaponOwnerException()

    # If no ammo, play "empty click" sound and return
    if self._current_ammo == 0:
        sound_manager.play(self._empty_sound)
        return

    # Query current map to verify line-of-sight to target
    # Player carring this weapon is cached on construction
    if world.get_current_map().check_los(self._player, target):
        # Decrease ammo
        self._current_ammo = self._current_ammo - 1

        # Apply damage to target
        target.take_damage(self._damage)

        # Play weapon "fire" sound
        sound_manager.play(self._fire_sound)

    else:
        # Play "error" sound indicating no line of sight
        sound_manager.play(self._not_visible_sound)

    return

Let’s add test coverage to the fire() method. We can start by adding a test which verifies the ammo count is correctly reduced by one.

The ‘setup’ portion of the test must instantiate the dependencies.

def test_weapon_fire(self):
    # setup
    # =============================

    # Instantiate player object because Weapon is required to belong to someone,
    # and uses Player for line-of-sight checks.
    player = Player("user_player")

    # fire() requires a valid target
    target_player = Player("some_enemy")

    # Weapon plays sounds, and requires a World instance for line-of-sight
    sound_mgr = sound_manager()
    world_mgr = world()

    sound_mgr.init()
    world_mgr.init("some_map")

    # Weapon will load required sounds and art assets on its own
    weapon = Weapon("rifle")

    # Weapon has circular reference to owning Player
    player.add_weapon(weapon)


    # execute
    # =============================

    # Reduce ammo
    weapon.fire(target)
    actual_ammo_post_fire = weapon.get_current_ammo()


    # assert
    # =============================
    # Verify expected results
    
    # Verify we lost ammo on fire
    self.assertEqual(weapon.get_max_ammo() - 1, actual_ammo_post_fire)

Comments aside, you can see how convoluted things get. The number of implicit dependencies add overhead to object creation and test comprehension. Remember, a benefit of unit testing is providing real-world usage and documentation. A new engineer will need a baseline understanding of multiple systems to work on Weapons.

If the Weapon class were decoupled from players, and world functionality, things get simpler. This alternate fire() function no longer requires the World class for line-of-sight checks. You could argue this should be handled by an A.I. system.

Attachment to a Player instance is also removed, and fire() can now operate without a valid target. There is still an implicit link to the Sound Manager, though it’s not required. The dependency overhead is reduced to zero so you can now test the function in isolation. The goal is to test that ammo is decreased, not that the sound system works or that the target can take damage. This simplifies our system and test.

def fire(self, start, target):
    # If no ammo, play "empty click" sound and return
    if self._current_ammo == 0:
        if sound_manager:
            sound_manager.play(self._empty_sound)
        return

    # Decrease ammo
    self._current_ammo = self._current_ammo - 1

    # Play weapon "fire" sound
    if sound_manager:
        sound_manager.play(self._fire_sound)

    # Apply damage to target
    if target:
        target.take_damage(self._damage)

    return

The test below operates on the second Weapon class. Having fewer responsibilities allows for more maintainable code and simpler testing. This is much easier for a new engineer to grok.

def test_weapon_fire(self):
    # setup
    # =============================

    # This version will NOT implicitly load assets on construction
    weapon = Weapon("rifle")
    start_position = vector(2.0, 0.0, 2.0) 

    # execute
    # =============================

    # Reduce ammo
    # Note: we no longer require a target
    weapon.fire(start_position, None)
    actual_ammo_post_fire = weapon.get_current_ammo()

    # assert
    # =============================

    # Verify we lost ammo on fire
    self.assertEqual(weapon.get_max_ammo() - 1, actual_ammo_post_fire)

Poor interfaces and tightly coupled code usually lead to long and convoluted tests. Writing tests first force you to consider how your interfaces will be used and help create more maintainable code.

Intro to Test-Driven Development for Games

Video game companies rely heavily on QA testers during development, especially during crunch time. This is generally a time consuming process as it requires human interaction with a running instance of the game. To help reduce the bug counts and lower manual testing costs, companies already utilize forms of automated testing to catch problems. These tests are typically reactive, providing feedback after things fail in the main code repository.

Developers can use proactive measures such as Test-Driven Development, or TDD, to supplement their testing. TDD centers around frequent iteration of the code base, and supporting these changes with automated unit tests (tests that verify a small piece of functionality). The tests run in isolation (no game instance required), and without human interaction. Software Engineers write unit tests that verify functionality before writing code for a feature, adding confidence that features performs as expected. This greatly complements the QA team by reducing their workload.

Additionally, TDD improves code health by encouraging programming to interfaces instead of implementations, which helps make systems more modular. Evidence also suggests it can actually increase development speed.

The TDD process consists of 5 steps, with the last step repeating as needed:

  1. Write a test that verifies some functionality
  2. Run the test and watch it fail (since the code isn’t there yet)
  3. Write the least amount of code to make the test pass
  4. Watch test pass
  5. (repeat) Refactor code as needed, making sure the test continues to pass

For example, let’s say you need a function that adds 2 numbers. Your first step is to create a valid, but failing test for the function. It is valid because it will pass if the function behaves properly. It is failing because the assert at the end will not execute/compile due to the missing function. This first test should highlight the “success path” for the function and not be concerned about failure cases. The success test may be the most important for your function, as it demonstrates intended operation. Prove your function works as expected before trying to examine when it doesn’t. You will add expected failure cases later.

One pattern you can follow with your tests is setup->execute->assert. You setup any state or dependencies you need for the code you want tested, execute the code in question, then assert your expectations were met.

def test_add(self):
  # setup
  value1 = 2
  value2 = 3
  expected = 5

  # execute
  actual = add(value1, value2)

  # assert expected value is correct
  self.assertEqual(expected, actual)

Next, let’s write the “add” function

def add(a, b):
  return 5

As step 3 states, write the least amount of code necessary to pass the test. The test expects a return value of “5”, so the least amount of code to satisfy the test is to return “5”.

Step 4, the test will now pass.

At step 5, you refactor the function now, making sure the test continues to pass

def add(a, b):
  return a + b

After this change, you can now update the unit test to add another test. This also highlights inline testing instead of the setup-execute-assert pattern.

def test_add(self):
  # no setup

  # execute/assert
  self.assertEqual(5, add(2, 3))
  self.assertEqual(2, add(1, 1))

This is a trivial example, but it showcases standard TDD and how both the code and tests will evolve. I usually write tests first, but I rarely take the minimalist approach on the first pass. When starting out, it is good practice to start with the “proper” TDD to allow developers to get acquainted with the process.

Later, we will dive deeper into the characteristics of testing, examine how to apply them to game code, and discuss ways to implement testing into your current workflow.