Google Test and Visual Studio

As you may surmise from the title, this post discusses integrating Google Test into Visual Studio. It will not discuss why you might want to use Google Test rather than Microsoft Test; I leave that to you to decide.

To fully use Google Test with Visual Studio 2015, you need two components:

  • Google Test (and Google Mock, which is included in the download), and
  • A test runner.

Installing and Using Google Test

To use Google Test, you should build a static library for Google Test as a separate project in the solution for the code you want to test. You should also create a separate project containing the unit tests.

Installing Google Test

You need to perform this step only once:

  1. Download Google Test from github. Place it in its own directory, I chose C:\googletest, so if you place it in a different location, make the appropriate changes in the instructions given in the rest of this post.

Using Google Test

Perform the following steps for each solution that you use Google Test with:

  1. If you have not already generated a solution for your code, do so now. For this post, I created a solution containing an empty project called MyProject.
  2. Add Google Test to the solution: Add a new Win32 Project to the solution. Name it GTest or something similar. Set the Application type as Static library, and clear the checkmark for Precompiled header. The Application Settings page of the Win32 Application Wizard should look like this:
    googletest1
    Click the Finish button to create the project.
  3. In the Visual Studio Solution Explorer, select the GTest project. Add the files gtest_main.cc and gtest-all.cc from C:\googletest\googletest\src as Existing Items. DO NOT copy the files to GTest project directory.
  4. As with all software projects, you must determine if you will be sharing the code for your solution with others. If you are, or you intend to use Google Test in multiple solutions, then I recommend setting user-wide settings in Visual Studio as outlined in this step; otherwise, you can set the include directories directly (also specified in this step).
    You must add …\googletest and …\googletest/googletest to the project’s Include Directories.  You do so using one of the following methods (not both!):
    (a) Create a User Macro that contains the location of your googletest directory. You need to do this only once for each configuration that you build. The settings are used for every solution and project that you build from now on. See User-Wide Settings in Visual Studio 2015 for how to do this.
    I created a macro called GTEST that points to C:\googletest\googletest, then added the following to the Include Directories:
    . $(GTEST)
    . $(GTEST)/include
    (b) Add c:\googletest\googletest and c:\googletest\googletest\include to the Include Directories for the GTest project. You must perform this step for every project and every configuration in every solution that uses Google Test.
  5. Now build the project or solution. There should be no build errors in the GTest project.
  6. If you performed step 4(a), above, and not step 4(b), open Windows Explorer and navigate to the directory containing the GTest project. Open the GTest.vcxproj file in a text editor; WordPad works well. Near the bottom of the file you will see the following lines:
     <ClCompile Include="..\..\..\..\..\..\googletest\googletest\src\gtest-all.cc" />
     <ClCompile Include="..\..\..\..\..\..\googletest\googletest\src\gtest_main.cc" />

    Change these lines to:

     <ClCompile Include="$(GTEST)\src\gtest-all.cc" />
     <ClCompile Include="$(GTEST)\src\gtest_main.cc" />

    Save the file and rebuild the solution. There should be no errors.

  7. You need some code to test. For my simple MyProject, I added a file called arithmetic.h that contains the following:
    #pragma once
    int square(const int value)
    {
        return value * value;
    }
  8. Add a new Win32 project for the unit tests. Name the project after the project containing the code under test and append Test or Tests. This is required later for test discovery by the test runner we will install later. (This is the default naming scheme for test executables; if you want to use a different naming scheme, refer to Christian Soltenborn’s comments on this post. Christian is the developer of GoogleTestAdapter, which you will install later.) In the Application Settings page of the Win32 Application Wizard, select Console application as the Application type, and uncheck the Precompiled header checkbox. Click Finish.
  9. In the Solution Explorer, select the References item for your tests project. Select the Add Reference… menu item from either the context menu or the Project dropdown menu. This opens the Add Reference dialog. Select the Solution item under Projects in the dialog. Now select both GTest and the project containing the code that you will be testing. For MyProject, the dialog will look like this:

    googletest2
    Click the OK button.

  10. Open the MyProjectTests.cpp file and replace any text in that file with the following code:
    #include "gtest/gtest.h"
    #include "../MyProject/arithmetic.h"
    
    TEST(SquareTests, Square)
    {
        EXPECT_EQ(0, square(0));
        EXPECT_EQ(-4, square(-2));
    }

    This code tests the square function that we created in arithmetic.h, above. If you performed step 4(b), above, you must repeat that step for your tests project.

  11. Rebuild the solution.
  12. Set MyProjectTests as the startup project. Start the program without debugging (using the Start Without Debugging menu item in the Debug dropdown menu, or by pressing Ctrl-F5). This will open a console window and run the tests. The output should look like this:

    googletest3
    Note that there is a failed test. The lines between [RUN ] SquareTests.Square and [ FAILED ] SquareTests.Square (3 ms) give both the location of the failed test and the expected and actual values. This indicates that either the code being tested is in error or the test itself is in error. In this case, it is the test that is in error. We will correct this later.

Installing  A Test Runner

As currently configured, Visual Studio and Google Test allow you to run tests, but you must look at every error and manually go to the test that errors. The MS Testing Framework is integrated into a test explorer. It would be nice if Google Test could also use the test explorer.

This is where test runners come in. There are two test runners for Google Test that are available as Visual Studio extensions:

  1. Google Test Runner for Visual Studio 2015; and,
  2. Google Test Adapter

To load either extension, click on the Tools->Extensions and Updates… menu item. This opens the Extensions and Updates dialog. Select Online and then search for google test.
I first tried Google Test Runner, but I got the following output:

------ Run test started ------
Found 1 tests, resolving symbols
Loading symbols from C:\Users\jimor_000\Source\Repos\test\x64\Debug\MyProjectTests.exe
From C:\Users\jimor_000\Source\Repos\test\x64\Debug\MyProjectTests.exe, found 1 symbols in 55 ms
In C:\Users\jimor_000\Source\Repos\test\x64\Debug, running: C:\Users\jimor_000\Source\Repos\test\x64\Debug\MyProjectTests.exe --gtest_output="xml:C:\Users\jimor_000\AppData\Local\Temp\tmp610C.tmp"
Opened results from C:\Users\jimor_000\AppData\Local\Temp\tmp610C.tmp
Could not load file or assembly 'FSharp.Core, Version=4.3.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
at FSharp.Data.Runtime.XmlRuntime.GetChildrenArray(XmlElement value, String nameWithNS)
at FSharp.Data.Runtime.XmlRuntime.ConvertArray[R](XmlElement xml, String nameWithNS, Func`2 f)
at GoogleTestRunner.ResultParser.getResults(IMessageLogger logger, String outputPath, FSharpList`1 testCases)
at GoogleTestRunner.GoogleTestExecutor.runOnce(IFrameworkHandle framework, IRunContext runContext, FSharpList`1 allCases, FSharpList`1 cases, String executable, Boolean runAll)
at GoogleTestRunner.GoogleTestExecutor.runTests(FSharpList`1 allCases, IEnumerable`1 cases, IRunContext runContext, IFrameworkHandle framework, Boolean runAll)
No test is available in C:\Users\jimor_000\Source\Repos\test\x64\Debug\MyProjectTests.exe. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.
========== Run test finished: 0 run (0:00:01.7510369) ==========

Google Test Runner is written in F# and requires a version of FSharp.Core that is not installed with Visual Studio 15 Update 3. I posted an Issue and the response was that the developers have no time to maintain the extension, so users should switch to Google Test Adapter.

Installing Google Test Adapter

Google Test Adapter is a complete rewrite of Google Test Runner in C#. According to its developer, it is also now more feature complete. See the comment by Christian Soltenborn. It is actively being developed and maintained, so there should be no support problems like there are with Google Test Runner.

Here is how to install Google Test Adapter:

  1. Select the Tools->Extensions and Updates… menu item. This opens the Extensions and Updates dialog.
  2. Select Online, then search for google test.
  3. Download, then install Google Test Adapter.
  4. Restart Visual Studio.

Using Google Test Adapter

  1. If the Test Explorer window is not open, open it by selecting the Test->Windows->Test Explorer menu item.
  2. Immediately after Google Test Adapter was installed, no tests will be visible in the Test Explorer window. To populate the test explorer with tests, you must rebuild your solution. Do so now. Remember, you must name your test project to end in either Test or Tests for the tests to be discovered.
  3. For MyProject, the Test Explorer window now looks like this:
    testexplorer
    Note the single test called SquareTests.Square. If several tests are created, then all tests will be listed.
  4. To execute tests, select one of the Run items in the Test Explorer window, or one of the Test->Run-> menu items. Here is the result:
    testexplorer2
    Note the links to the source and to the stack trace. Selecting the stack trace item takes you to the test that failed.
  5. The square of -2 is 4, not -4, so the test is in error, not the square function. Correcting this bug, then building the solution again and rerunning the test gives:
    testexplorer3
    indicating that the test now passes.

That’s it. You can now continue to add more tests and more code to test.

You should also see the GoogleTestAdapter documentation on GitHub for information on additional capabilities.

Advertisements

6 thoughts on “Google Test and Visual Studio

  1. Thanks for providing a tutorial on the Google Test Adapter! I hope it’s fine if I link this page from the Google Test Adapter homepage…

    Here’s a minor suggestion for improvement: the tutorial refers to the default naming scheme for test executables; this can however be overridden by configuring a test discovery regex, either through VS Options or (preferably) through a solution settings file to be shared via source control.

    One other note: Google Test Adapter is by now a complete rewrite (and much more feature complete), but has started as a C# port of the Google Test Runner by Markus Lindqvist.

    Like

    • Thank you for this contribution, it is very help. 2 remarks however:

      — I think step 2 has to be repeated as part of step 8 (otherwise the GTest includes are not available in the test project with the test code).
      I had a lot of compiler errors, which disappeared after asserting step 2 within step 8.

      — when building the solution (which contains those 3 projects) the following error occurs (copied from my output pane):
      “The following errors and warnings occured during test discovery (enable debug mode for more information):
      ERROR: Couldn’t find the msdia.dll to parse *.pdb files. You will not get any source locations for your tests.
      Warning: Could not find source location for test SquareTests.Square
      Cannot discover test extensions installed by NuGet. The NuGet service is not available. Tests may not be found due to missing test adapter extensions.”

      I installed GTA (succesfully) using Tools->Extensions and Updates… and the consecutive actions as stated above…..

      These errors/warnings do not seem to impede the test execution however….. the tests pass fine, debugger is launched….
      Are those errors relevant? What must be done to prevent them? What’s is missing in my configuration/setup( I thought the NuGet facilty was installed as part of the “overall” GTA tooling)?

      Regards,
      Herman

      Like

      • Hi Herman. Thanks for the input.
        (1) While step 2 (now step 4) made reference to adding c:\googletest\googletest and c:\googletest\googletest\include directly to the Include Directories, the rest of the post was written assuming user-wide settings. I have now corrected this by specifying what needs to be done in both cases.
        (2) msdia.dll is a file that is installed as part of Visual C++. The only thing I can think of is that it was not properly installed when you installed Visual Studio.

        Hope this helps.

        Like

      • Hi usingcpp,

        thanks for your reaction and the clarifications. In the meantime I’ve installed another VS version (2013, in addition to the 2017 version I installed earlier) and contrary to the 2017 version I performed a complete installation of all “default” modules. The “msdia.dll msg” didn’t occur anymore, which also solves the problem for the 2017 version because apparently the msdia.dll is included in the full installation and is also picked up by the 2017 environment thereafter. So the full installation solved the issue for me as you already suggested.

        Regards,
        Herman

        Like

  2. This is very helpful.

    In step 6 where it says “Near the bottom of the file you will see the following lines:
    Change these lines to:”, it doesn’t actually say what we should see or what we should change it to. What should go there?

    Also, should the test results show up in Visual Studio 2015’s Test Results and Test Runs windows? I’m not seeing them there.

    Like

    • Thanks for pointing this out, Richard. I have corrected this.
      I think the lines disappeared when I changed the theme for this blog. At least that is what I am blaming it on. 🙂

      Once you install GoogleTestAdapter, the results should show up in the Test Explorer window as shown in the post.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s