Google Test and Visual Studio

As you may surmise from the title, this post discusses integrating Google Test into Visual Studio. It will not discuss why you might want to use Google Test rather than Microsoft Test; I leave that to you to decide.

To fully use Google Test with Visual Studio 2015, you need two components:

  • Google Test (and Google Mock, which is included in the download), and
  • A test runner.

Installing and Using Google Test

To use Google Test, you should build a static library for Google Test as a separate project in the solution for the code you want to test. You should also create a separate project containing the unit tests.

Installing Google Test

You need to perform this step only once:

  1. Download Google Test from github. Place it in its own directory, I chose C:\googletest, so if you place it in a different location, make the appropriate changes in the instructions given in the rest of this post.

Using Google Test

Perform the following steps for each solution that you use Google Test with:

  1. If you have not already generated a solution for your code, do so now. For this post, I created a solution containing an empty project called MyProject.
  2. Add Google Test to the solution: Add a new Win32 Project to the solution. Name it GTest or something similar. Set the Application type as Static library, and clear the checkmark for Precompiled header. The Application Settings page of the Win32 Application Wizard should look like this:
    Click the Finish button to create the project.
  3. In the Visual Studio Solution Explorer, select the GTest project. Add the files and from C:\googletest\googletest\src as Existing Items. DO NOT copy the files to GTest project directory.
  4. As with all software projects, you must determine if you will be sharing the code for your solution with others. If you are, or you intend to use Google Test in multiple solutions, then I recommend setting user-wide settings in Visual Studio as outlined in this step; otherwise, you can set the include directories directly (also specified in this step).
    You must add …\googletest and …\googletest/googletest to the project’s Include Directories.  You do so using one of the following methods (not both!):
    (a) Create a User Macro that contains the location of your googletest directory. You need to do this only once for each configuration that you build. The settings are used for every solution and project that you build from now on. See User-Wide Settings in Visual Studio 2015 for how to do this.
    I created a macro called GTEST that points to C:\googletest\googletest, then added the following to the Include Directories:
    . $(GTEST)
    . $(GTEST)/include
    (b) Add c:\googletest\googletest and c:\googletest\googletest\include to the Include Directories for the GTest project. You must perform this step for every project and every configuration in every solution that uses Google Test.
  5. Now build the project or solution. There should be no build errors in the GTest project.
  6. If you performed step 4(a), above, and not step 4(b), open Windows Explorer and navigate to the directory containing the GTest project. Open the GTest.vcxproj file in a text editor; WordPad works well. Near the bottom of the file you will see the following lines:
     <ClCompile Include="..\..\..\..\..\..\googletest\googletest\src\" />
     <ClCompile Include="..\..\..\..\..\..\googletest\googletest\src\" />

    Change these lines to:

     <ClCompile Include="$(GTEST)\src\" />
     <ClCompile Include="$(GTEST)\src\" />

    Save the file and rebuild the solution. There should be no errors.

  7. You need some code to test. For my simple MyProject, I added a file called arithmetic.h that contains the following:
    #pragma once
    int square(const int value)
        return value * value;
  8. Add a new Win32 project for the unit tests. Name the project after the project containing the code under test and append Test or Tests. This is required later for test discovery by the test runner we will install later. (This is the default naming scheme for test executables; if you want to use a different naming scheme, refer to Christian Soltenborn’s comments on this post. Christian is the developer of GoogleTestAdapter, which you will install later.) In the Application Settings page of the Win32 Application Wizard, select Console application as the Application type, and uncheck the Precompiled header checkbox. Click Finish.
  9. In the Solution Explorer, select the References item for your tests project. Select the Add Reference… menu item from either the context menu or the Project dropdown menu. This opens the Add Reference dialog. Select the Solution item under Projects in the dialog. Now select both GTest and the project containing the code that you will be testing. For MyProject, the dialog will look like this:googletest2
    Click the OK button.
  10. Open the MyProjectTests.cpp file and replace any text in that file with the following code:
    #include "gtest/gtest.h"
    #include "../MyProject/arithmetic.h"
    TEST(SquareTests, Square)
        EXPECT_EQ(0, square(0));
        EXPECT_EQ(-4, square(-2));

    This code tests the square function that we created in arithmetic.h, above. If you performed step 4(b), above, you must repeat that step for your tests project.

  11. Rebuild the solution.
  12. Set MyProjectTests as the startup project. Start the program without debugging (using the Start Without Debugging menu item in the Debug dropdown menu, or by pressing Ctrl-F5). This will open a console window and run the tests. The output should look like this:googletest3
    Note that there is a failed test. The lines between [RUN ] SquareTests.Square and [ FAILED ] SquareTests.Square (3 ms) give both the location of the failed test and the expected and actual values. This indicates that either the code being tested is in error or the test itself is in error. In this case, it is the test that is in error. We will correct this later.

Installing  A Test Runner

As currently configured, Visual Studio and Google Test allow you to run tests, but you must look at every error and manually go to the test that errors. The MS Testing Framework is integrated into a test explorer. It would be nice if Google Test could also use the test explorer.

This is where test runners come in. There are two test runners for Google Test that are available as Visual Studio extensions:

  1. Google Test Runner for Visual Studio 2015; and,
  2. Google Test Adapter

To load either extension, click on the Tools->Extensions and Updates… menu item. This opens the Extensions and Updates dialog. Select Online and then search for google test.
I first tried Google Test Runner, but I got the following output:

------ Run test started ------
Found 1 tests, resolving symbols
Loading symbols from C:\Users\jimor_000\Source\Repos\test\x64\Debug\MyProjectTests.exe
From C:\Users\jimor_000\Source\Repos\test\x64\Debug\MyProjectTests.exe, found 1 symbols in 55 ms
In C:\Users\jimor_000\Source\Repos\test\x64\Debug, running: C:\Users\jimor_000\Source\Repos\test\x64\Debug\MyProjectTests.exe --gtest_output="xml:C:\Users\jimor_000\AppData\Local\Temp\tmp610C.tmp"
Opened results from C:\Users\jimor_000\AppData\Local\Temp\tmp610C.tmp
Could not load file or assembly 'FSharp.Core, Version=, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
at FSharp.Data.Runtime.XmlRuntime.GetChildrenArray(XmlElement value, String nameWithNS)
at FSharp.Data.Runtime.XmlRuntime.ConvertArray[R](XmlElement xml, String nameWithNS, Func`2 f)
at GoogleTestRunner.ResultParser.getResults(IMessageLogger logger, String outputPath, FSharpList`1 testCases)
at GoogleTestRunner.GoogleTestExecutor.runOnce(IFrameworkHandle framework, IRunContext runContext, FSharpList`1 allCases, FSharpList`1 cases, String executable, Boolean runAll)
at GoogleTestRunner.GoogleTestExecutor.runTests(FSharpList`1 allCases, IEnumerable`1 cases, IRunContext runContext, IFrameworkHandle framework, Boolean runAll)
No test is available in C:\Users\jimor_000\Source\Repos\test\x64\Debug\MyProjectTests.exe. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.
========== Run test finished: 0 run (0:00:01.7510369) ==========

Google Test Runner is written in F# and requires a version of FSharp.Core that is not installed with Visual Studio 15 Update 3. I posted an Issue and the response was that the developers have no time to maintain the extension, so users should switch to Google Test Adapter.

Installing Google Test Adapter

Google Test Adapter is a complete rewrite of Google Test Runner in C#. According to its developer, it is also now more feature complete. See the comment by Christian Soltenborn. It is actively being developed and maintained, so there should be no support problems like there are with Google Test Runner.

Here is how to install Google Test Adapter:

  1. Select the Tools->Extensions and Updates… menu item. This opens the Extensions and Updates dialog.
  2. Select Online, then search for google test.
  3. Download, then install Google Test Adapter.
  4. Restart Visual Studio.

Using Google Test Adapter

  1. If the Test Explorer window is not open, open it by selecting the Test->Windows->Test Explorer menu item.
  2. Immediately after Google Test Adapter was installed, no tests will be visible in the Test Explorer window. To populate the test explorer with tests, you must rebuild your solution. Do so now. Remember, you must name your test project to end in either Test or Tests for the tests to be discovered.
  3. For MyProject, the Test Explorer window now looks like this:
    Note the single test called SquareTests.Square. If several tests are created, then all tests will be listed.
  4. To execute tests, select one of the Run items in the Test Explorer window, or one of the Test->Run-> menu items. Here is the result:
    Note the links to the source and to the stack trace. Selecting the stack trace item takes you to the test that failed.
  5. The square of -2 is 4, not -4, so the test is in error, not the square function. Correcting this bug, then building the solution again and rerunning the test gives:
    indicating that the test now passes.

That’s it. You can now continue to add more tests and more code to test.

You should also see the GoogleTestAdapter documentation on GitHub for information on additional capabilities.

This post was updated on August 1, 2017 to add the following section.

Running Tests Automatically As Part of Build Process

Visual Studio Enterprise Edition

The following only works if you are using Visual Studio Enterprise Edition:

  1. Do either of the following, as they are equivalent:
    (a) Select the Test->Test Settings->Run Tests After Build menu item.
    (b) In Test Explorer, click the Run Tests After Build button at the top of the window. This button is shown in the image below. It is the one with the yellow background indicating that it is selected.
  2. Rebuild the solution. The tests should run.

Other Editions of Visual Studio

Even though the menu item and button mentioned for the Enterprise Edition may be active, they only work for the Enterprise Edition. To run tests in other editions of Visual Studio, do the following:

  1. Open the properties dialog for the project containing the tests.
  2. Select Build Events->Post-Build Event.
  3. Set the Configuration that you want the tests to run for.
  4. Set the Platform that you want to tests to run for.
  5. For the command line, enter the location and name of the test executable {“$TargetDir)MyProjectTests.exe” in this case}. Enter a description, such as “Run Unit Tests”. This text is for documentation purposes only; it does not appear anywhere. Make sure that Use In Build is set to “Yes”.
  6. The properties dialog should now look similar to this:
    Click the OK Button to close the dialog.
  7. Rebuild the solution. The Output window will contain:
    1>------ Build started: Project: MyProjectTests, Configuration: Debug Win32 ------
    1>MyProjectTests.vcxproj -> C:\Users\jimor_000\Source\Repos\MyProject\Debug\MyProjectTests.exe
    1>MyProjectTests.vcxproj -> C:\Users\jimor_000\Source\Repos\MyProject\Debug\MyProjectTests.pdb (Partial PDB)
    1>Running main() from
    1>[==========] Running 1 test from 1 test case.
    1>[----------] Global test environment set-up.
    1>[----------] 1 test from SquareTests
    1>[ RUN ] SquareTests.Square
    1>c:\users\jimor_000\source\repos\myproject\myprojecttests\myprojecttests.cpp(7): error : Expected: -4
    1>To be equal to: square(-2)
    1> Which is: 4
    1>[ FAILED ] SquareTests.Square (1 ms)
    1>[----------] 1 test from SquareTests (1 ms total)
    1>[----------] Global test environment tear-down
    1>[==========] 1 test from 1 test case ran. (1 ms total)
    1>[ PASSED ] 0 tests.
    1>[ FAILED ] 1 test, listed below:
    1>[ FAILED ] SquareTests.Square
    1> 1 FAILED TEST
    1>C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\Microsoft.CppCommon.targets(133,5): error MSB3073: The command "C:\Users\jimor_000\Source\Repos\MyProject\Debug\MyProjectTests.exe
    1>C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\Microsoft.CppCommon.targets(133,5): error MSB3073: :VCEnd" exited with code 1.
    1>Done building project "MyProjectTests.vcxproj" -- FAILED.
    ========== Build: 0 succeeded, 1 failed, 2 up-to-date, 0 skipped ==========
  8. If any tests fail, run all tests in the Test Explorer window. When you do this, the following message box is displayed:
    This is displayed because running the tests was part of the build process for the test project. If the tests ran, then the test program was created. You can therefore safely click on the Yes button to run the tests.