Visual Studio is a powerful Integrated Development Environment for developing software for the Windows OS environment, and with its latest release (2015, Update 1), a number of tablet and phone OSes as well. As with all IDEs , the times required to build a solution using Visual Studio increases with the number of source files and projects in a solution. There are a number of different techniques that can be used to improve this build time. This post will discuss one of these techniques: using multiple processes and processors for the build.
MSBuild.exe is the builder used by Visual Studio to build solutions. MSBuild has a command line switch that tells MSBuild to use multiple processors if available to build the loaded solution. There is also a commercial product, Incredibuild, that integrates with Visual Studio to provide a different way of building the solution and can offload work to additional computers on the network.
Visual Studio and Multiple Processors
Visual Studio provides a property for each C++ project that controls whether MSBuild uses multiple processes to build the various projects in a solution. This setting is found on the Properties dialog for each project. The image below shows the Properties Pages for a C++ project. The setting is found under the C/C++ -> General item, and is called Multi-processor Compilation (the last item in the panel on the right of the dialog). There are three choices:
- Yes (/MP)
- <inherit from parent or project defaults>
No obviously means use a single process to perform the compiles, Yes means use as many processes as possible, and the third says to inherit the setting. You may use the technique discussed in User Wide Settings in Visual Studio 2015 to set a default setting for all new projects and for any projects that inherit the setting.
Let’s now look at the Yes setting. Using this switch, a number of processes are created to perform the build. The number of processes created, by default, is equal to the number of virtual processors on the computer performing the build (number of CPUs times the number of cores per CPU [times 2 if the cores support hyperthreading]). You can determine this number by starting Task Manager and selecting the Performance tab and the CPU item. The lower right of the displayed panel contains information about the CPUs, with the Logical processors line containing the number of logical (virtual) processors as shown in this image:
As you can see, on my computer, I have a total of 8 logical processors, so by default, Visual Studio would use up to 8 processes to build a solution.
It is possible to limit the number of processes. More information about this option is available in /MP (Build With Multiple Processes).
Visual Studio and Incredibuild
Incredibuild is a commercial product that accelerates builds by taking advantage of the multiple logical processors in the host computer, and if available, the processors in other computers on your network. At the start of each build, Incredibuild performs an analysis to determine the best way to build the Visual Studio solution, and divides the compiles and links among the available processors.
One advantage of Incredibuild over a number of other build accelerators is that Incredibuild integrates right into Visual Studio. As noted in the last review, above, one disadvantage is that linking files compiled with Incredibuild with those compiles with the normal build process may create executable files that crash. More information is provided in that review.
Build Accelerator Measurements
It’s nice to talk about multi-processor builds and Incredibuild, but how do they really compare? Some people claim great improvements in build time for very large solutions, and others say build times are worse with Incredibuild. I will perform a set of experiments to measure the build times for two open source software solutions that are the size that a typical user would encounter in his personal programming activities. Note: You should not extrapolate these results to much larger projects; you should repeat these measurements on your own software and hardware.
For each of the open source solutions, six separate measurements will be taken:
- Solution rebuild without using multi-processes;
- Solution rebuild using multi-processes;
- Solution rebuild using Incredibuild;
- Incremental build without using multi-processes;
- Incremental build using multi-processes; and,
- Incremental build using Incredibuild.
All measurements are made on a single laptop that contains the following components. This is typical for a higher end laptop built within the last two years. Specifically, the laptop contains:
- Intel Core i7-4710MQ CPU with 8 logical processors. More information about the CPU is displayed in the first image above this text.
- 8 GB DDR3 RAM running at 1600 MHz.
- 1 TB hard drive spinning at 5400 RPM.
- Visual Studio 2015 Community Edition (Update 1)
- Incredibuild 7.1
- Other software is running on the laptop during these tests, such as a web browser, an email client program, and a number of tasks that run in the background. This mix would be typical for a developer.
Source Code for the Measurements
The two open source software solutions used for these measurements are:
- wxWidgets 3.1.0. wxWidgets contains 24 projects that generate 23 x64 Debug lib files.
- libtins. libtins contains 4 projects that generate one lib file. The solution is built using the x86 Debug configuration.
Rebuild Test Procedure
- Visual Studio is started and the solution is loaded.
- The solution is built and the build time is measured. For the Incredibuild builds, the time is taken from the Incredibuild monitor panel; for other builds, the time is measured using a clock. In both cases, the times should be accurate to within one second.
- Visual Studio is terminated.
- Steps 1, 2, and 3 are repeated five times for each build type and for each solution.
The procedure above was chosen for two reasons:
- The first time that a rebuild is performed after it is loaded takes longer than subsequent builds. While I could have performed one or two solution builds before measuring build times, I chose to use the procedure outlined above. In either case, measured values relative to one another would be the same, only the actual measured values change.
- Additional software is running on the computer used for these tests. While no direct user interaction was performed with other programs during these tests, the other software may run a variable amount. For example, an email may be received by the email client program. The tests are performed a number of times to average out these variations.
Incremental Build Test Procedure
- Visual Studio is started and the solution is loaded.
- The solution is rebuilt to ensure that all object, lib, and executable files are compatible. See the discussion of advantages and disadvantages of Incredibuild in the section above titled “Visual Studio and Incredibuild”.
- A single blank line is added or removed at the end of one of the source files. Addition and removal are alternated between runs. The source file chosen resulted in the compilation of that file and the linking of a single lib file.
- A build is performed and the time taken is measured. For the Incredibuild builds, the time is taken from the Incredibuild monitor panel; for other builds, the time is measured using a clock.
- Steps 3 and 4 are repeated for a total of five measurements.
- Steps 2 through 5 are repeated for each of the build types (single process, multi-processes, and Incredibuild.
The graphs above shows the rebuild times in seconds as indicated for the build types.
This graph shows the Incredibuild incremental build times for wxWidgets. The build times using a single process were less than 1 second, and the build times using multiple processes was just over 1 second. The tests were not repeated for libtins; however, you can assume that the results would be similar since only a single source file was modified.
Discussion of Results
The procedure for the rebuilds measurements was chosen because it would typically mimic the procedure used for the integration of daily or more frequent changes in an agile development process.
It is interesting to note that the variation in rebuild times for the Incredibuild builds was the greatest. Also, the Incredibuild times were greater than the build times for the multi-process builds. The Incredibuild method starts with a “Preparing Build Tasks” task that takes several seconds to complete; the similar task in MSBuild for the multi-process builds takes much less time, but no doubt results in a different distribution of build tasks over the logical processors.
The build times for single process builds are longer than the build times for multi-process builds as would be expected. The relative difference in build times for libtins are what I would have expected; the surprising results are for the wxWidgets rebuilds. There are many times more files being compiled and libraries being built so I would have expected the difference between the single process and multi-process build times for wxWidgets to be relatively even greater, yet the differences are relatively minor. Also, the single-process build times for wxWidgets beat the Incredibuild build times for the same solution.
One of the possibly major advantages for Incredibuild is the splitting of build tasks across multiple computers on a network. It was not possible to test this, so no conclusion can be drawn about the build times using that scenario.
From the reviews and from testimonials on the Incredibuild web site, Incredibuild seems to shine for very large projects; that is, for projects that take many minutes to build. A very large solution was not selected for these tests because that does not fit the types of solution that I work on. As mentioned above, you may want to repeat these tests for your own solutions.
The incremental build times for single process and multi-processes were difficult to measure using a clock. However, the single process bulid times were definitely less than one second, and the multi-process build times were more than one second, but considerably less than two seconds.
Is this what one would expect? Shouldn’t the multi-process build time be less than the single process build time? In this case, no. Remember that a single source file is being compiled, and a single lib file is being linked. The linking cannot be performed until the source file has been compiled, so no parallel processing can be performed. The extra time required for the potential multi-process build is the time that MSBuild takes to determine that no parallel processing can be performed.
The extra time required for the Incredibuild builds was divided between determining the build tasks and the time taken to compile the source file. The time required to determine the build tasks is understandable; however, the much longer compile time was unexpected.
What Was Not Tested
The computer that the tests were run on has a single CPU with 4 cores and hyper-V support for a total of 8 logical processors. All 8 logical processes shared the same cache, so this may slow down the build processes. A second physically separate CPU would be expected to improve build times more than multiple cores on a single CPU, but this could not be tested.
Similarly, one of the potential advantages of Incredibuild is its ability to use the idle cycles of the CPUs on other computers on the same network to perform some of the build tasks. This also could not be tested.
- For those situations where multiple tasks (compiles, links) can be performed at the same time, setting the multi-processor switch in Visual Studio will reduce build times over the times of single-process builds.
- For situations where multiple tasks cannot be performed concurrently, there is a penalty for using the multi-processor switch. However, the number of times that multiple tasks cannot be performed concurrently relative to the number of times that multiple tasks can be performed concurrently dictates that setting the multi-processor switch is the recommended procedure.
- Incredibuild increases the build times for the standard “make a small change, build, and test” process that most developers use, and therefore Incredibuild should not be used for these development steps.
- For these open source projects and these specific test conditions, Incredibuild provided no improvement over the times for multi-process rebuilds.That does not indicate that Incredibuild does not improve the rebuild times for many other, and especially larger projects. There may be great advantages to using Incredibuild for integration builds, but you will have to perform your own tests.
- The open source projects used in these tests are typical of the type of software that I develop. Because Incredibuild provided no improvement in the build times for either of these projects, I will not be using Incredibuild in my build procedures.