question

GlenSand-7727 avatar image
0 Votes"
GlenSand-7727 asked RLWA32-6355 commented

Parallel compilation for projects with dependencies on each other in Visual studio

In my company, we are using Visual Studio 2019 and we would like to reduce the build times of our C++ solution. It is about 21 projects and it takes roughly 8.5 minutes for a debug build and a lot more for a release build. I am wondering if there is a way to make projects with dependencies on each other compile in parallel. I am already aware and make use of multiprocess compilation (/MP flag). Indeed, each project in the solution is built using multiple cores. Also when two projects, let's say A and B, don't depend on each other they build in parallel. However with our current settings, if A depends on B, the build of B has to complete before A starts. As far as I understand, the compilation of files in A can start even before B finishes, and only linking of A has to really wait until B finishes (let's assume that A generates either a .dll or a .exe). Is there a way to achieve this?

c++
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.


In addition to your query, are you also using Precompiled Headers correctly to reduce the build time?


0 Votes 0 ·

Yes, sure.
Among other things, we use something like unity build on precompiled headers for projects that rarely change.
Also to fasten the link-time I added:: /DEBUG:FASTLINK, and related parameters
Now I'm working on implementing Pimpl and reducing the size of the files included in the headers.

0 Votes 0 ·

1 Answer

JeanineZhang-MSFT avatar image
0 Votes"
JeanineZhang-MSFT answered RLWA32-6355 commented

Hi,

The /MP option causes the compiler to create one or more copies of itself, each in a separate process. As far as I'm concerned, this is based on each project, and this will not let all projects build in parallel. To do that you have to pass in -M[:num] or -maxcpucount (max cpu count) to msbuild.exe. For more details, I suggest you could refer to the Doc: https://docs.microsoft.com/zh-cn/visualstudio/msbuild/msbuild-command-line-reference?view=vs-2019&redirectedfrom=MSDN&viewFallbackFrom=vs-2015

I am wondering if there is a way to make projects with dependencies on each other compile in parallel.

If the two projects that depend on each other, are in the same solution and the dependencies between each other are clearly marked, then the cpp files can be compiled at the same time, but not linked at the same time. The linking must occur in order of how they depend on one another.

Best Regards,

Jeanine


If the response is helpful, please click "Accept Answer" and upvote it.

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.


· 11
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@JeanineZhang-MSFT Apparently there are differences between how Visual Studio builds projects and how MSBuild acts when it is run outside of Visual Studio. According to Visual Studio builds vs. MSBuild.exe builds

120447-vsbuild.png

So, I think that @GlenSand-7727 is asking about altering how dependencies are evaluated and handled for Visual Studio builds, including building inside the IDE.

@GlenSand-7727 , please correct me if I misunderstand your question.


1 Vote 1 ·
vsbuild.png (34.0 KiB)

Yes, I wanted to learn how to build projects in parallel in Visual Studio. Your answer helped me, before I did not understand what is the difference between these assembly methods. However, I still don't know if the assembly can be made more parallel. I know for sure that I need it, I experimented with dependencies, and I realized that I can win another 50 seconds if I do what I want.

0 Votes 0 ·

Can you give us an example of the dependencies that you are working with? For example, does one project reference another (a dependency is created) because it needs an import library for functions imported from a dll?

0 Votes 0 ·
Show more comments

Hi Jeanine,
You wrote: "If the two projects that depend on each other, are in the same solution and the dependencies between each other are clearly marked". So, I can't understand "clearly marked", what does it mean? And how to properly set dependency between projects. I use the standard mechanism now (right-click on a project -> Build Dependencies -> Project Dependencies). And in this case, my project is being built consistently.

ЗP.S -maxcpucount is set to zero (via Visual Studio of course), and it means that scheduler uses all the available cores and create as much separate process as possible.
P.P.S And the number of available parallel builds is equal to available logic cores(12).

Glen

0 Votes 0 ·

@GlenSand-7727

What do you mean"my project is being built consistently"? Compile and build are different.

-maxcpucount is set to zero

The value must be an integer greater then zero. You could without specifying a value, MSBuild will use up to the number of processors on the computer.

0 Votes 0 ·

About "my project is being built consistently":

Let's imagine that I have 2 projects. Y depends on X. Project Y starts compiling only when the linking of project X completes.
I want project Y to start compiling at the same time as project X. But they must be linked strictly sequentially. I want the linker on the Y project to start immediately after the completion of the linking of the X project

0 Votes 0 ·
Show more comments