Introduction
Making your own game framework can be a daunting task, with no clear end in sight. It's more complex than your typical C++ project and requires you to clean about many things: creating a stable API, creating shared and static libraries with CMake, handling dependencies, and much more.
Things get even more complicated when you want to build a cross-platform project in C++. This is where the complexities of CMake truly come out. Not to mention, C++ itself is a special language with quirks not found in others I'm used to, like Java and Python.
Nevertheless, I took it upon myself to build a cross-platform game framework with C++ and CMake. I wanted to create something different from the available C++ game frameworks—something that offers more to developers familiar with higher-level frameworks. Penguin Framework aims to provide a higher level of abstraction than similar frameworks in C++.
But before I can offer that abstraction, I have to build the intermediate objects that enable it. That is, abstracting lower-level systems such as window creation and mathematical game objects.
When creating such systems, it's important to validate that they work as intended. This is where tests come in, as they allow you to verify that your functions and classes behave as expected.
There are many ways to write tests, but for Penguin Framework, since I was mainly interested in testing class creation and function calls, unit tests were the perfect option.
So, I embarked on a journey to create unit tests for all the classes I had recently finalized: Vector2, Rect2 (and their integer counterparts), Circle2, Colour, String, and the Window class.
The math and string classes were quite easy to write unit tests for using GoogleTest, and I quickly achieved 100% coverage for them.
The window, however… was another issue.
The Initial Problem
The Vector2/Vector2i, Rect2/Rect2i, Circle2, Colour, and String classes all had one thing in common: they relied on no external dependencies. This meant that their functions would work as intended if implemented correctly.
It also meant that I didn't need to link against the SDL library. Test executables would still run fine on their own.
But that wasn't the case for the Window class. Since it depended on SDL, I needed to link the test executable against the SDL library. After doing so, I expected it to work like the others.
And it didn't.
The tests failed before they even ran, because of the test fixture itself. SDL was not being correctly set up, and I had no clue why. Here's what the test fixture initially looked like:
class WindowTestFixture : public ::testing::Test {
protected:
void SetUp() override {
ASSERT_EQ(SDL_Init(SDL_INIT_VIDEO), true) << "SDL video subsystem could not be initialized because of: " << SDL_GetError();
}
void TearDown() override {
SDL_Quit();
}
};
After some debugging, I found out that SDL_Init() was returning a nonzero value, but not a negative one like -1. It was returning a positive number: specifically, 1.
This confused me, and SDL_GetError() wasn't even reporting any error.
I knew SDL worked in a non-testing environment. I had created a game with my older Penguin2D code that relied on SDL. So I knew the issue had to do with the testing environment.
The testing environment had no display, while the non-testing environment did. This likely meant that graphics drivers were either missing or disabled. That was most likely why SDL wasn't initializing as intended.
Since I wanted to test SDL in a headless environment (i.e., no GUI), I needed a way to make SDL initialize without needing access to a display.
Luckily, SDL makes it pretty easy to set up a "dummy" environment using the SDL_SetHint function. I set the video driver to this dummy value before calling SDL_Init(). So, I updated my test fixture with the following line of code:
SDL_SetHint(SDL_HINT_VIDEO_DRIVER, "dummy");And it worked.
Only one line of code was added, and my tests were running.
So we're done, right?
Unfortunately, no…
The Unexpected Challenge
My tests were running, but not all of them were passing. Some functions failed due to my own implementation errors, so I fixed the issues in the .cpp file and the tests passed as expected.
However, I noticed something interesting: functions like set_title() or enable_borders() worked fine, but functions like maximize() or resize() consistently failed. As I looked through the implementation of the Window class, I realized what the issue was.
State-Changing vs Property-Changing
Property-changing functions (e.g., set_title(), enable_borders()) just modify metadata. These work fine in a dummy context.
State-changing functions (e.g., maximize(), minimize(), restore()) rely on real OS interaction. These either silently fail or do nothing when using the dummy video driver.
The internal flags in those state-changing functions correctly changed (or remained consistent) throughout the function calls, but the underlying SDL function itself failed. So I wasn't really guaranteeing that the function executed successfully, just that the internal flags were updated.
What Unit Tests Can and Can't Do
Unit tests are great for verifying intent—i.e., did your logic try to maximize the window? But they can't verify outcomes that rely on external systems, like a real display manager.
How I Adapted
I updated my unit tests to verify internal flag changes, not SDL's external behavior. I also added clear notes to the state-changing tests, indicating that their actual functionality must be verified at the integration level. I also updated my GitHub project roadmap to include plans for integration tests to validate real OS interactions.
Conclusion
Not all bugs are in your logic. Some bugs I wouldn't have found without unit testing, but I also wouldn't have realized the limitations of unit testing without this experience. Sometimes, the test environment limits what's possible.
Unit tests should focus on validating logic, not external effects. Validating those external effects has to be done with other types of tests, like integration tests or example games.