Google Test and Visual Studio

As you may surmise from the title, this post discusses integrating Google Test into Visual Studio. It will not discuss why you might want to use Google Test rather than Microsoft Test; I leave that to you to decide.

To fully use Google Test with Visual Studio 2015, you need two components:

  • Google Test (and Google Mock, which is included in the download), and
  • A test runner.

Installing and Using Google Test

To use Google Test, you should build a static library for Google Test as a separate project in the solution for the code you want to test. You should also create a separate project containing the unit tests.

Installing Google Test

You need to perform this step only once:

  1. Download Google Test from github. Place it in its own directory, I chose C:\googletest, so if you place it in a different location, make the appropriate changes in the instructions given in the rest of this post.

Using Google Test

Perform the following steps for each solution that you use Google Test with:

  1. If you have not already generated a solution for your code, do so now. For this post, I created a solution containing an empty project called MyProject.
  2. Add Google Test to the solution: Add a new Win32 Project to the solution. Name it GTest or something similar. Set the Application type as Static library, and clear the checkmark for Precompiled header. The Application Settings page of the Win32 Application Wizard should look like this:
    Click the Finish button to create the project.
  3. In the Visual Studio Solution Explorer, select the GTest project. Add the files and from C:\googletest\googletest\src as Existing Items. DO NOT copy the files to GTest project directory.
  4. As with all software projects, you must determine if you will be sharing the code for your solution with others. If you are, or you intend to use Google Test in multiple solutions, then I recommend setting user-wide settings in Visual Studio as outlined in this step; otherwise, you can set the include directories directly (also specified in this step).
    You must add …\googletest and …\googletest/googletest to the project’s Include Directories.  You do so using one of the following methods (not both!):
    (a) Create a User Macro that contains the location of your googletest directory. You need to do this only once for each configuration that you build. The settings are used for every solution and project that you build from now on. See User-Wide Settings in Visual Studio 2015 for how to do this.
    I created a macro called GTEST that points to C:\googletest\googletest, then added the following to the Include Directories:
    . $(GTEST)
    . $(GTEST)/include
    (b) Add c:\googletest\googletest and c:\googletest\googletest\include to the Include Directories for the GTest project. You must perform this step for every project and every configuration in every solution that uses Google Test.
  5. Now build the project or solution. There should be no build errors in the GTest project.
  6. If you performed step 4(a), above, and not step 4(b), open Windows Explorer and navigate to the directory containing the GTest project. Open the GTest.vcxproj file in a text editor; WordPad works well. Near the bottom of the file you will see the following lines:
     <ClCompile Include="..\..\..\..\..\..\googletest\googletest\src\" />
     <ClCompile Include="..\..\..\..\..\..\googletest\googletest\src\" />

    Change these lines to:

     <ClCompile Include="$(GTEST)\src\" />
     <ClCompile Include="$(GTEST)\src\" />

    Save the file and rebuild the solution. There should be no errors.

  7. You need some code to test. For my simple MyProject, I added a file called arithmetic.h that contains the following:
    #pragma once
    int square(const int value)
        return value * value;
  8. Add a new Win32 project for the unit tests. Name the project after the project containing the code under test and append Test or Tests. This is required later for test discovery by the test runner we will install later. (This is the default naming scheme for test executables; if you want to use a different naming scheme, refer to Christian Soltenborn’s comments on this post. Christian is the developer of GoogleTestAdapter, which you will install later.) In the Application Settings page of the Win32 Application Wizard, select Console application as the Application type, and uncheck the Precompiled header checkbox. Click Finish.
  9. In the Solution Explorer, select the References item for your tests project. Select the Add Reference… menu item from either the context menu or the Project dropdown menu. This opens the Add Reference dialog. Select the Solution item under Projects in the dialog. Now select both GTest and the project containing the code that you will be testing. For MyProject, the dialog will look like this:

    Click the OK button.

  10. Open the MyProjectTests.cpp file and replace any text in that file with the following code:
    #include "gtest/gtest.h"
    #include "../MyProject/arithmetic.h"
    TEST(SquareTests, Square)
        EXPECT_EQ(0, square(0));
        EXPECT_EQ(-4, square(-2));

    This code tests the square function that we created in arithmetic.h, above. If you performed step 4(b), above, you must repeat that step for your tests project.

  11. Rebuild the solution.
  12. Set MyProjectTests as the startup project. Start the program without debugging (using the Start Without Debugging menu item in the Debug dropdown menu, or by pressing Ctrl-F5). This will open a console window and run the tests. The output should look like this:

    Note that there is a failed test. The lines between [RUN ] SquareTests.Square and [ FAILED ] SquareTests.Square (3 ms) give both the location of the failed test and the expected and actual values. This indicates that either the code being tested is in error or the test itself is in error. In this case, it is the test that is in error. We will correct this later.

Installing  A Test Runner

As currently configured, Visual Studio and Google Test allow you to run tests, but you must look at every error and manually go to the test that errors. The MS Testing Framework is integrated into a test explorer. It would be nice if Google Test could also use the test explorer.

This is where test runners come in. There are two test runners for Google Test that are available as Visual Studio extensions:

  1. Google Test Runner for Visual Studio 2015; and,
  2. Google Test Adapter

To load either extension, click on the Tools->Extensions and Updates… menu item. This opens the Extensions and Updates dialog. Select Online and then search for google test.
I first tried Google Test Runner, but I got the following output:

------ Run test started ------
Found 1 tests, resolving symbols
Loading symbols from C:\Users\jimor_000\Source\Repos\test\x64\Debug\MyProjectTests.exe
From C:\Users\jimor_000\Source\Repos\test\x64\Debug\MyProjectTests.exe, found 1 symbols in 55 ms
In C:\Users\jimor_000\Source\Repos\test\x64\Debug, running: C:\Users\jimor_000\Source\Repos\test\x64\Debug\MyProjectTests.exe --gtest_output="xml:C:\Users\jimor_000\AppData\Local\Temp\tmp610C.tmp"
Opened results from C:\Users\jimor_000\AppData\Local\Temp\tmp610C.tmp
Could not load file or assembly 'FSharp.Core, Version=, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
at FSharp.Data.Runtime.XmlRuntime.GetChildrenArray(XmlElement value, String nameWithNS)
at FSharp.Data.Runtime.XmlRuntime.ConvertArray[R](XmlElement xml, String nameWithNS, Func`2 f)
at GoogleTestRunner.ResultParser.getResults(IMessageLogger logger, String outputPath, FSharpList`1 testCases)
at GoogleTestRunner.GoogleTestExecutor.runOnce(IFrameworkHandle framework, IRunContext runContext, FSharpList`1 allCases, FSharpList`1 cases, String executable, Boolean runAll)
at GoogleTestRunner.GoogleTestExecutor.runTests(FSharpList`1 allCases, IEnumerable`1 cases, IRunContext runContext, IFrameworkHandle framework, Boolean runAll)
No test is available in C:\Users\jimor_000\Source\Repos\test\x64\Debug\MyProjectTests.exe. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.
========== Run test finished: 0 run (0:00:01.7510369) ==========

Google Test Runner is written in F# and requires a version of FSharp.Core that is not installed with Visual Studio 15 Update 3. I posted an Issue and the response was that the developers have no time to maintain the extension, so users should switch to Google Test Adapter.

Installing Google Test Adapter

Google Test Adapter is a complete rewrite of Google Test Runner in C#. According to its developer, it is also now more feature complete. See the comment by Christian Soltenborn. It is actively being developed and maintained, so there should be no support problems like there are with Google Test Runner.

Here is how to install Google Test Adapter:

  1. Select the Tools->Extensions and Updates… menu item. This opens the Extensions and Updates dialog.
  2. Select Online, then search for google test.
  3. Download, then install Google Test Adapter.
  4. Restart Visual Studio.

Using Google Test Adapter

  1. If the Test Explorer window is not open, open it by selecting the Test->Windows->Test Explorer menu item.
  2. Immediately after Google Test Adapter was installed, no tests will be visible in the Test Explorer window. To populate the test explorer with tests, you must rebuild your solution. Do so now. Remember, you must name your test project to end in either Test or Tests for the tests to be discovered.
  3. For MyProject, the Test Explorer window now looks like this:
    Note the single test called SquareTests.Square. If several tests are created, then all tests will be listed.
  4. To execute tests, select one of the Run items in the Test Explorer window, or one of the Test->Run-> menu items. Here is the result:
    Note the links to the source and to the stack trace. Selecting the stack trace item takes you to the test that failed.
  5. The square of -2 is 4, not -4, so the test is in error, not the square function. Correcting this bug, then building the solution again and rerunning the test gives:
    indicating that the test now passes.

That’s it. You can now continue to add more tests and more code to test.

You should also see the GoogleTestAdapter documentation on GitHub for information on additional capabilities.