question

SachinMyneni-6188 avatar image
0 Votes"
SachinMyneni-6188 asked SachinMyneni-5019 commented

Cross Compiling for Windows using WSL/WSL2

We wanted to replace the slow cygwin toolset with WSL for building our Windows executable.
Executables are built on Windows 10 VMs. Access is either through RDP for manual tests or using SSH as part of our CI system.


Here is my process:


  1. Call setup script from DOS window: "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" amd64

  2. Compile using cl.exe:

    C:\Users\osqa>"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" amd64

    C:\Users\osqa>cl.exe -version
    Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24215.1 for x64
    Copyright (C) Microsoft Corporation. All rights reserved.

    cl : Command line warning D9002 : ignoring unknown option '-version'
    cl : Command line error D8003 : missing source filename

    C:\Users\osqa>
    C:\Users\osqa>cd workarea\osqa_ca-w10-bld-05_os_211\os

    C:\Users\osqa\workarea\osqa_ca-w10-bld-05_os_211\os>cl.exe hello.c
    Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24215.1 for x64
    Copyright (C) Microsoft Corporation. All rights reserved.

    hello.c
    Microsoft (R) Incremental Linker Version 14.00.24215.1
    Copyright (C) Microsoft Corporation. All rights reserved.

    /out:hello.exe
    hello.obj

    C:\Users\osqa\workarea\osqa_ca-w10-bld-05_os_211\os>hello.exe
    Hello World


This works fine in the DOS window when log into the system using RDP or SSH using OpenSSH.
This also works fine in WSL if I RDP into the VM. But it does not work in WSL if I SSH into the VM: Note that I ssh into the windows VM, call the vcvarsall.bat script and then start WSL and then invoke the compiler. I do not ssh directly into the WSL.



 prog\osqa@CA-W10-BLD-05 C:\Users\osqa\workarea\osqa_ca-w10-bld-05_os_211\os>"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" amd64
    
 prog\osqa@CA-W10-BLD-05 C:\Users\osqa\workarea\osqa_ca-w10-bld-05_os_211\os>cl.exe hello.c
 Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24215.1 for x64
 Copyright (C) Microsoft Corporation.  All rights reserved.
    
 hello.c
 Microsoft (R) Incremental Linker Version 14.00.24215.1
 Copyright (C) Microsoft Corporation.  All rights reserved.
    
 /out:hello.exe
 hello.obj
    
 prog\osqa@CA-W10-BLD-05 C:\Users\osqa\workarea\osqa_ca-w10-bld-05_os_211\os>hello.exe
 Hello World
    
 prog\osqa@CA-W10-BLD-05 C:\Users\osqa\workarea\osqa_ca-w10-bld-05_os_211\os>wsl
 osqa@ca-w10-bld-05:/c/Users/osqa/workarea/osqa_ca-w10-bld-05_os_211/os$ rm hello.exe
 osqa@ca-w10-bld-05:/c/Users/osqa/workarea/osqa_ca-w10-bld-05_os_211/os$ cl.exe --version
 osqa@ca-w10-bld-05:/c/Users/osqa/workarea/osqa_ca-w10-bld-05_os_211/os$ cl.exe hello.c
 osqa@ca-w10-bld-05:/c/Users/osqa/workarea/osqa_ca-w10-bld-05_os_211/os$ echo $?
 66
 osqa@ca-w10-bld-05:/c/Users/osqa/workarea/osqa_ca-w10-bld-05_os_211/os$ gcc hello.c
 osqa@ca-w10-bld-05:/c/Users/osqa/workarea/osqa_ca-w10-bld-05_os_211/os$ echo $?
 0
 osqa@ca-w10-bld-05:/c/Users/osqa/workarea/osqa_ca-w10-bld-05_os_211/os$ ./hello.exe
 -bash: ./hello.exe: No such file or directory
 osqa@ca-w10-bld-05:/c/Users/osqa/workarea/osqa_ca-w10-bld-05_os_211/os$ ./a.out
 Hello World
 osqa@ca-w10-bld-05:/c/Users/osqa/workarea/osqa_ca-w10-bld-05_os_211/os$

As one can see cl.exe just returns back with an error '66'.
For shells error 66 typically means: cannot open input.

77373-image.png




Environment:


Host machine: Microsoft Windows [Version 10.0.19041.804]
OpenSSH: I've setup OpenSSH using instructions from here: https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse
WSL: Ubuntu 20.04.2 LTS in WSL2. I've also tried WSL1, Ubuntu 18.04 with the same results.


Tests done so far:
I ran the command 'whoami' in DOS window, when logged in through each of RDP and SSH. In both case the results are identical so the user seems to be the same in case of both RDP and SSH.



remote-desktop-clientwindows-10-application-compatibilitywindows-server-virtualization
image.png (17.5 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

TeemoTang-MSFT avatar image
0 Votes"
TeemoTang-MSFT answered

Hi,
In fact, for Cross-compile issue, you’d better ask for help from github forum.
https://github.com/topics/cross-compiler
The reason why we recommend posting appropriately is you will get the most qualified pool of respondents, and other partners who read the forums regularly can either share their knowledge or learn.
Thanks for your understanding and cooperating.
Look at this similar case:
[Help requested] Cross-compile from WSL to Windows
https://github.com/veandco/go-sdl2/issues/336


If the Answer is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.


5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

SachinMyneni-6188 avatar image
0 Votes"
SachinMyneni-6188 answered

Hello @TeemoTang-MSFT

The reason why I posted here and not github is that the issue, while showing up in cross-compilation, seems to have to do with user identity and permissions.

Note that I can get the cross-compilation to work fine when I RDP into the system. But not when I ssh into the system.

If you still think this should go into github cross-compiler forum, I will post it there.

Thank you,
-Sachin

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

SachinMyneni-6188 avatar image
0 Votes"
SachinMyneni-6188 answered SachinMyneni-5019 commented

Another update: As I mentioned in my OP this is primarily for our CI system. We use a windows domain account that is not and administrator on all the build systems.
I discovered that if I made this account an 'administrator' on the build system (added to administrator group), this whole thing works!
I can ssh into the system (as the build account), start wsl and cross-compile

So this is definitely a permissions issue.

I would rather not leave the build-account with Administrator privileges on all build systems. I guess I can narrow my question to:
What is the specific privilege that is required for ssh + wsl + windows_interop to work?

Thank you,
-Sachin

· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello Sachin, You could use process monitor to capture permission trace for analysis. From my experience, WSL usually needs administrator privilege for use

0 Votes 0 ·

Hello @TeemoTang-MSFT ,
I will try process monitor (haven't used it in the past but will try to figure out)..
But I am not sure why WSL will accept my account as Administrator when I login using RDP but not as an Administrator when I SSH into Windows.
There still seems to be something different when one logs in using SSH.

Thank you,
-Sachin

0 Votes 0 ·

Hello @TeemoTang-MSFT
I generated the Process Monitor output files in CSV and PML formats.
I've noted used it before so I am not sure what I am looking for.
I am attaching the CSV formatted output files (renamed as .txt). The one with "_good" in the name is when the account has Admin privileges and compilation can succeed.
Note that the output is from Intel's ICL compiler but the behavior was the same with Microsoft's cl.exe.

81588-icl-wcl-good.txt


0 Votes 0 ·
icl-wcl-good.txt (1.2 MiB)

Here is the one with the failed case.

81664-wsl-icl.txt


0 Votes 0 ·
wsl-icl.txt (98.5 KiB)