Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

msbuild nuget package unable to open vs2017 csproj files #2369

Closed
weltkante opened this issue Jul 31, 2017 · 72 comments
Closed

msbuild nuget package unable to open vs2017 csproj files #2369

weltkante opened this issue Jul 31, 2017 · 72 comments

Comments

@weltkante
Copy link

I'm referencing the nuget packages Microsoft.Build and Microsoft.Build.Tasks.Core. It doesn't matter if I chose stable or prerelease, both versions run into the same error.

I'm trying to open a csproj file for evaluation:

var projectFile = @"C:\projects\test\test.csproj";
var project = new Microsoft.Build.Evaluation.Project(projectFile);
var projectName = Path.GetFileNameWithoutExtension(projectFile);
var outputFile = project.GetPropertyValue("TargetPath");
var outputName = Path.GetFileName(outputFile);

I'm getting Microsoft.Build.Exceptions.InvalidProjectFileException: 'The tools version "15.0" is unrecognized. Available tools versions are "12.0", "14.0", "2.0", "3.5", "4.0".'

When searching the web for this error message I'm only coming up with things related to the VS 2017 installation and how to look up the msbuild packaged with it; unfortunately this doesn't help me with my instance of the exception because I'm invoking msbuild as a library and not as a process.

Do I have to tell the nuget assemblies where the VS 2017 installation is? How do I do this? (I was assuming the nuget assemblies can work stand alone, but if a VS installation is required that works too, it's just not discoverable for me what to do here.)

@Kryptos-FR
Copy link

I'm having a similar issue. I noticed that it usually happens when there are multiple side-by-side installation of Visual Studio 2017 (e.g. stable release and preview, or Community and Pro) and/or Build Tools for Visual Studio 2017 (as a nickname "2" installation).

Our users complain that they can't build their project with our system anymore but it looks like the issue is in Microsoft Build itself. This is a serious blocker and I hope that it can be fixed ASAP.

@Kryptos-FR
Copy link

Kryptos-FR commented Aug 16, 2017

Seems to be related to #2427.

Isn't this code supposed to find all installed and supported versions of MSBuild?

var projectCollection = new Microsoft.Build.Evaluation.ProjectCollection();
var toolsets = projectCollection.Toolsets; // is missing 15.0
var project = new Microsoft.Build.Evaluation.Project(null, null, projectCollection); // throws an exception here
// same happens with default constructor of project:
var project = new Microsoft.Build.Evaluation.Project();

In my case, after updating to Visual Studio 15.3, this doesn't find the latest (15.0) tools version. It was working fine before the update. Seems to me that this can break at any time after an update. This is a critical issue.
cc @rainersigwald

@rainersigwald
Copy link
Member

Do I have to tell the nuget assemblies where the VS 2017 installation is? How do I do this?

Ideally you wouldn't have to, but it does look like something is wrong with our code to support that.

#2030 tracks us providing an easy way to do that. There's a link from there to #1784 (comment), which links to debanne/dotnet-builder#1 where @AndyGerlicher wrote some example code to find and load MSBuild.

I'll see if I can reproduce this to figure out what's going on (and probably some other workarounds).

@rainersigwald
Copy link
Member

@weltkante
Copy link
Author

I probably should have noted that in my original report, but I'm already using the Microsoft.VisualStudio.Setup.Configuration.Interop nuget library to figure out which VS 2017 versions are installed. So figuring out where VS or MSBuild is located is not the problem, I'm already doing that.

My problem is that I'm linking against the MSBuild nuget package and want to evaluate projects in-process programmatically without invoking a separate MSBuild process (which would make no sense anyways since I'm extracting information from the projects, not running a build).

So, assuming I already have the information where v15 is located, any idea how I can pass this on to MSBuild? Or is this impossible and I have to wait for a fix in the nuget MSBuild package?

(Sorry if this information was in the linked issues, I tried looking through them, but couldn't find anything related.)

@rainersigwald
Copy link
Member

@weltkante Yes, if you have that information already, there's an easy workaround.

Set the environment variables VSINSTALLDIR and VisualStudioVersion before calling into MSBuild APIs.

I just confirmed on my machine with this hardcoded value on top of your example code:

Environment.SetEnvironmentVariable("VSINSTALLDIR", @"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community");
Environment.SetEnvironmentVariable("VisualStudioVersion", @"15.0");

The problem is arising because MSBuild attempts to build from the installed version of Visual Studio where possible, but we're failing to locate it now (still not sure why; continuing to look). Setting those environment variables lets an alternate codepath through the find-toolset code take over.

@rainersigwald
Copy link
Member

Ok, here's the source of the problem:

https://github.com/Microsoft/msbuild/blob/ab090d1255caa87e742cbdbc6d7fe904ecebd975/src/Shared/BuildEnvironmentHelper.cs#L232-L235

The returned instance of VS (on this machine) has version 15.3.26730.3, which doesn't match CurrentVisualStudioVersion which is 15.0.

Need to track down an Update 2 machine to see if this is a recent change in the return value that we need to ping the setup folks about (I don't see how it could have been working otherwise).

Also need to make our code more robust. Not quite sure how yet.

@weltkante
Copy link
Author

weltkante commented Aug 17, 2017

Your example didn't work for me, but looking at the source, trying to set MSBUILD_EXE_PATH worked for me (using the latest nuget package 15.3.409):

Environment.SetEnvironmentVariable("MSBUILD_EXE_PATH", @"C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\15.0\Bin\MSBuild.exe");

No idea why it doesn't pick up the other two variables you used.

(Also, for the record, I created the issue when I was on v15.2 - so this has been broken before the update to v15.3 ~ at the point where I created the issue I had no preview installed, but I did have a preview a few weeks/months before writing that code; not sure if it was a 15.2 or 15.3 preview though)

@weltkante
Copy link
Author

weltkante commented Aug 17, 2017

Ok figured it out why your variables didn't work for me.

Apparently having msbuild anywhere in the process name will freak out msbuild ;)

https://github.com/Microsoft/msbuild/blob/ab090d1255caa87e742cbdbc6d7fe904ecebd975/src/Shared/BuildEnvironmentHelper.cs#L327-L334

(Notice the IndexOf() >= 0 part)

I had named my sandbox project to repro the bug MsBuildIssueRepro and apparently that makes MsBuild think my process is a full MsBuild installation.

Renaming the project allows your variables to work, too.

Reading through the source I believe most of the TryFromXXX Methods should check if their guess was correct before returning a non-null value. Otherwise they are ending the chain of try-methods early with a wrong guess.

@rainersigwald
Copy link
Member

@weltkante "MSBuild libraries get confused if your program has msbuild in its name" is #2194. We discovered this week that it's bitten every member of the core MSBuild team for exactly this kind of sample project. You nailed the cause!

Glad to hear setting the environment variables worked for you. I should also note that if you run your application from a "Developer Command Prompt for VS2017" it'll set those for you, so any MSBuild API consumer should work from there.

@rainersigwald
Copy link
Member

Ok, I set up a VM with different VSes:
image

And our enumeration code returns:

Count = 3
 {Microsoft.Build.Shared.VisualStudioInstance}
  "Visual Studio Community 2017"
  "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Community"
  {15.2.26430.4}
 {Microsoft.Build.Shared.VisualStudioInstance}
  "Visual Studio Professional 2017"
  "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional"
  {15.3.26730.3}
 {Microsoft.Build.Shared.VisualStudioInstance}
  "Visual Studio Enterprise 2017"
  "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Enterprise"
  {15.0.26403.0}

So it does look like this broke with 15.2.

@Kryptos-FR
Copy link

Kryptos-FR commented Aug 18, 2017

@rainersigwald As far as I know us final user don't have a way to chose which version of VS we want to install. I know it is not directly related to MSBuild but it would be nice to ask the VS team to offer an option in the VS installer. When the whole team is still on a previous version (say 15.2) and you have a newcomer you want him or her to also use the same version.

Answering your question on #2427:

Please include details about how you're referencing MSBuild and what you deploy with your application.

We ask our user to either install VS or the Build Tools. We actually have a step in our installer setup that will call the VS installer in case none are detected (using vs_buildtools.exe that is deployed with our setup).
Then from our code we just use the API from the several Microsoft.Build nuget packages to either create or load .props or .csproj using similar code than #2369 (comment). So we don't deploy MSBuild ourselves (and as you said in the other issue we shouldn't have to do it), we just include the Microsoft.Build dlls with our application.

Also I'll have to check but I'm not sure if Microsoft.VisualStudio.Setup.Configuration.Interop is able to return the instance(s) of Build Tools. We were not using this library to find out MSBuild path, only the installed version of VS so that we can provide a built-in option to open the project with VS.

So it does look like this broke with 15.2.

It is a bit weird. It still works at home for me with 15.2. But a coworker (who still have 15.2) did modify its VS install to include other packages (like support for UWP and such) and it started to fail.
It also work on a VM with a fresh install of everything (VS or Build Tools) until 15.2 (included). But I did notice that sometimes it will start to fail when more than one instance was installed side by side (not sure why). So maybe Build Tools for 15.2 was still returning the correct version at some point.

adamskt added a commit to IntelliTect/Coalesce that referenced this issue Aug 18, 2017
@rainersigwald rainersigwald added this to the MSBuild 15.5 milestone Aug 18, 2017
rainersigwald added a commit to rainersigwald/msbuild that referenced this issue Aug 18, 2017
Fixes dotnet#2369 by relaxing the check to find any Visual Studio 15.*
install. This is required because Visual Studio 2017 Update 3 reports as
15.3.something, which wouldn't be found by the old code. Before Visual
Studio 15.2, a bug in VS setup caused it to report 15.0 (erroneously,
but compatibly with the old MSBuild behavior).
@parkycai
Copy link

@rainersigwald When will this fix be applied to nuget package? thx!

@dasMulli
Copy link
Contributor

Maybe somehow related: What happens if a program using the vs shell like the VS 2017 Team Explorer is installed that isn't really a complete VS / build environment?
e.g. https://stackoverflow.com/questions/45760787 sees this error:

MSBuild auto-detection: using msbuild version '15.3.409.57025' from 'C:\Program Files (x86)\Microsoft Visual Studio\2017\TeamExplorer\MSBuild\15.0\bin'.

@artiomchi
Copy link

A potentially related issue.. Once upgraded to 15.3, MSBuild can't properly open 2017 csproj files. The error we're getting is:

Failure: Msbuild failed when processing the file 'D:\Development\R4MVC\src\R4MvcHostApp\R4MvcHostApp.csproj' with message: The SDK 'Microsoft.NET.Sdk.Web' specified could not be found. D:\Development\R4MVC\src\R4MvcHostApp\R4MvcHostApp.csproj

Running process monitor, I can see that the app is trying to access [path_to_running_exe]\Sdks\Microsoft.NET.Sdk.Web\Sdk and is obviously not finding that. Adding various combinations of environment settings from the comments above have caused it to crash with several different error messages, but none of the suggestions have ultimately fixed the issue.

I'm perfectly happy to provide any help in tracing this, as we have this replicated on multiple machines, and by multiple users.

@rainersigwald
Copy link
Member

@artiomchi Yes, that sounds related. Can you share the errors when you set VSINSTALLDIR and VisualStudioVersion? I expect that to work.

@rainersigwald
Copy link
Member

(I should also note that most of the team including me is out on vacation until the first week of January, so responses may be delayed, but I want to help you get to the bottom of this.)

@raffaeler
Copy link

raffaeler commented Dec 23, 2017

Thank you @rainersigwald ... I made many tests by analyzing my test solution with your code.

  • your solution worked
  • your solution after updating all the latest nuget (not the prerelease) packages worked

so I updated the non-roslyn packages to the latest (the output is similar to this, but I didn't capture the one of my update):
image
(they are just roslyn dependencies, the only one I directly use is the tuple package)
Then I updated the Roslyn from 1.32 to 2.6.1 and now works.

Now the only difference between mine and your solutions is that I also reference Microsoft.Build.* packages while you don't (should I remove them?)

Where is the wizardry? Is the compilation process dependent from a specific version of system.composition package?

P.S. yes, I have binding redirects in my solution which were automatically updated by the nuget process.

@matkoch
Copy link

matkoch commented Jan 8, 2018

@rainersigwald my use case is just to evaluate MSBuild properties, like Configuration, TargetFrameworks, PackageReferences, included files, and so on. My goal is that this works cross platform with .NET Framework, Mono, and of course also .NET Core tooling (without any VS installed at all). Roslyn/C# is completely out of scope.

Just a moment ago, I created a simple repro with netcoreapp2.0, references to Microsoft.Build and Microsoft.Build.Utilities.Core and this code:

var projectCollection = new ProjectCollection();
var project = new Project(@"..\ConsoleApp10.csproj",
        new Dictionary<string, string>(),
        projectCollection.DefaultToolsVersion);

var lookup = project.Items.ToLookup(x => x.ItemType, x => x.EvaluatedInclude);

I tried this w/ and w/o binding redirects. Always fails with:

The SDK 'Microsoft.NET.Sdk' specified could not be found.

@matkoch
Copy link

matkoch commented Jan 20, 2018

Ping @rainersigwald. Could you please also re-open this issue? It is obviously not solved.

@ulrichb
Copy link

ulrichb commented Feb 2, 2018

@rainersigwald This is a blocking issue for me too. /cc @matkoch

@matkoch
Copy link

matkoch commented Mar 2, 2018

@rainersigwald kindly pinging....

@flagbug
Copy link

flagbug commented Apr 6, 2018

I'm having the same problem that @matkoch is having. I'm working on a cross-platform tool that analyzes outdated packages (https://github.com/goenning/dotnet-outdated) and really just want to read the package references of a simple project file.

The problem is that this always throws The SDK 'Microsoft.NET.Sdk.Web' specified could not be found. for some reason

@p-kaczynski
Copy link

Having same problem, but only when using .netcore2.0, works fine in .net framework.

@p-kaczynski
Copy link

Literally changing from <TargetFramework>netcoreapp2.0</TargetFramework> to <TargetFramework>net461</TargetFramework> fixes the issue for me using

<PackageReference Include="Microsoft.Build" Version="15.6.85" />
    <PackageReference Include="Microsoft.Build.Utilities.Core" Version="15.6.85" />

and new Project(path)

@CodeTroopers
Copy link

CodeTroopers commented Jul 4, 2018

I had the same issue with a TFS Build Agent (Version 2010) after installing Build Tools for Visual Studio 2017 and configuring MSBuild arguments to : /toolsversion:15.0 /property:VisualStudioVersion=15.0.
It seems this registery key is missing : HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\MSBuild\ToolsVersions\15.0 with the string value MSBuildToolsPath and cause the error :

'The tools version "15.0" is unrecognized. Available tools versions are "12.0", "14.0", "2.0", "3.5", "4.0".'

@manish10sharma
Copy link

Hello I am having the same issue.
MSB4062 The "Microsoft.CodeAnalysis.BuildTasks.Csc" task could not be loaded from the assembly C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin\Roslyn\Microsoft.Build.Tasks.CodeAnalysis.dll. Could not load file or assembly 'Microsoft.Build.Utilities.Core, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified. Confirm that the declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask. MSBuild C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin\Roslyn\Microsoft.CSharp.Core.targets 52 false Error

Can Anyone tell me solution if found?
I am using 2017 15.8.2 Version.

Please help.
Thanks

@kjkrum
Copy link

kjkrum commented Nov 6, 2018

I'm got the same error when opening the csproj of one project from another project. Both projects were freshly created today in VS2017 15.8.8. Updating to 15.8.9 did not help.

using (var col = new ProjectCollection())
{
	var proj = col.LoadProject(projectFile); // kaboom!
}

Installing the latest Microsoft.Build and Microsoft.Build.Utilities.Core from nuget solved the problem. I didn't even need to fiddle around with environment variables. Why does VS still ship with a broken MSBuild?

@weltkante
Copy link
Author

@kjkrum

Why does VS still ship with a broken MSBuild?

As far as I know VS no longer ships MSBuild (it ships via nuget). VS has its own private copy of MSBuild which is not exposed to your project build so you can't reference it.

If you reference MSBuild through the GAC or Assembly Reference list (i.e. not through nuget) you likely get some outdated version which previously shipped with the .NET framework or an older VS and thus still is visible there for backwards compatibility.

@rainersigwald
Copy link
Member

You can and should use the VS copy of MSBuild, but since it's no longer in the GAC you must locate it. https://docs.microsoft.com/en-us/visualstudio/msbuild/updating-an-existing-application has details on what's required, and https://github.com/Microsoft/MSBuildLocator/ is a package used to make the process easier.

@kjkrum
Copy link

kjkrum commented Nov 7, 2018

@rainersigwald That's an awful lot of dicking around for something that used to be a checkbox.

@aolszowka
Copy link

This is still present. The work around provided by @rainersigwald and documented here https://docs.microsoft.com/en-us/visualstudio/msbuild/updating-an-existing-application?view=vs-2017 (specifically using MSBuildLocator.RegisterDefaults();) worked for us.

@kurtnelle
Copy link

Any solution for this. I'm having a similar issue with trying to create a Raspberry Pi Blink project. The project creation fails with the error message "Unable to read the project file "Blink12.vcxproj" The tools version "15.0" is unrecognized. Available tools versions are "2.0","3.5","4.0"

@rainersigwald
Copy link
Member

@kurtnelle If https://docs.microsoft.com/en-us/visualstudio/msbuild/updating-an-existing-application doesn't help you, please open a new issue with more details about how you're using the API and what's going wrong.

@GeirGrusom
Copy link

Seriously this is a huge issue for me. Old version of the libraries don't work with 2019 only systems and new versions doesn't work with 2017 only systems. I even made a shim that would use new versions on systems with 2019 installed but old in 2017 systems. It seemed like it worked but the integration tests that looked like they worked everywhere else fails on the build server. I don't want that stupid hack to begin with and now it unsurprisingly fails in the most important configuration.

I've been struggling with this for almost a month. The error message is non-sensical and unhelpful, and this is obviously a major breaking change in the library.
Why does upgrading to new versions, with no other changes at all, break simply loading the project in 2017? Just calling "LoadFromXmlReader" throws this error. Setting toolsversion and subtoolsetversion seems to have absolutely no effect what.so.ever.

This thing makes me extremely unproductive and it makes me look incompetent.

@rainersigwald
Copy link
Member

@GeirGrusom can you please file a new issue, including details on

Old version of the libraries don't work with 2019 only systems and new versions doesn't work with 2017 only systems.

What libraries?

Why does upgrading to new versions, with no other changes at all, break simply loading the project in 2017?

Can you provide more details on exactly how you're loading projects and what the system environment is (upgrading from what to what)?

@GeirGrusom
Copy link

I'm very sorry. I was very stressed on friday, but that's no excuse for unprofessional behavior.

Short about the appliation: It's used for building solutions, versioning and running the test suite on them. I would argue the value of this in the first place but that's unfortunately not particularily productive. It first invokes vswhere to find out where Visual Studio is located and uses that to actually build the application. This failed earlier because it wouldn't try to use Visual Studio 2019 if a project was made with 2017 and specified that so I updated the vswhere implementation to find the latest if an exact match could not be found. This seems to work: projects build just fine.

However the issues show up when running the test suites. It uses Microsoft.Build to load the project files in order to find the output DLL files and dependencies the projects might have. On the old version this worked fine for 2017 but in 2019 it gave this odd error:

error MSB1040: ToolsVersion is not valid. The tools version "15.0" is unrecognized. Available tools versions are "2.0", "3.5", "4.0".

Build libraries are Microsoft.Build and Microsoft.Build.Utilities.Core at version 15.9.20.

I tried to just change "tools" version but this error shows up with different versions except if I put 4.0, at which point it tells me that it can't find Microsoft.NET.Sdk. I upgrade to 16.4.0 on both libraries and now it works in 2019, but on systems with 2017 installed I'm back to the drawing board. So I made this shim that will force it on a newer version as explained and now in all environments I've tested it it works but it is hacky and not exactly very maintenable. It also fails to build on the build server with the same error when it tries to run integration tests with this issues error description.

It seems to me that there's some confusion as to what "toolsVersion" actually mean. Take Microsoft.Build.Utilities.ToolLocationHelper.GetPathToBuildTools(string). In the documentation it looks to me like it should return the location of MSBuild, but it doesn't do that at all. Sometimes it seems to return the entry assembly path, and other times .NET Framework paths, and on my development machine it returns exactly what I would expect.

What does toolsVersion mean? Is it CLR version? MSBuild version? Visual Studio version? The documentaiton on MSBuild...Project.LoadFromStream is not exactly helpful as it just says "that's the tools version" more or less. If I state nothing it seems like MSBuild will put toolsversion from the <Project /> element or whatever SDK it ends up loading and that's what's causing this odd error message. Change the toolsversion and it will complain about wrong toolsversions until you hit 4.0, 3.5 or 2.0 at which point it will complain that Microsoft.NET.Sdk could not be found. Is MSBuild doing the wrong thing here, or is the error description wrong?

I'll see if I can make a trivial reproduction.

@GeirGrusom
Copy link

Couldn't reproduce the issue in a trivial solution. Only new project does not properly load in VS2019 but I got everything to work on build so I don't think I want to waste anymore of your time on this.

Thanks, and again, sorry.

@rainersigwald
Copy link
Member

@GeirGrusom I think you might like to use MSBuildLocator in your application. It handles the scenarios you're currently using VSWhere for, and you should be able to use a single application to load both 2017 and 2019 projects (as long as you reference MSBuild 15.x when you build).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests