Microsoft has reported a remarkable 15 times speed-up in package restore when using Nuget 6.12, introduced with the .NET 9.0 SDK (Software Development Kit), but some developers have run into bugs that require switching back to the old behavior.
Nuget is the package manager for .NET and whenever a project is built, run or tested, there is an implicit call to a comment called “dotnet restore” which checks all the dependencies on external libraries and downloads them if needed. The command also checks for compatibility issues and, by default since .NET 8, security vulnerabilities.
Microsoft uses .NET internally and a team tasked with migrating thousands of .NET Framework projects to .NET Core ran into performance issues, with the restore command taking more than 30 minutes in some cases, causing what principal software engineer Jeff Kluge called “wasted time and frustration for developers.”
The problem was reported to another internal team, responsible for Nuget, resulting in a GitHub issue: “Nuget’s dependency graph resolution algorithm should scale better for large graphs.”
The existing algorithm, it turned out, was remarkably inefficient. Individual projects ended up with millions of nodes in the graph. “Walking millions of nodes several times causes restore times for 2,500 projects to be around 15 minutes,” the issue stated.
According to Kluge, the original dependency resolution algorithm was only intended as a temporary one, but has continued in use despite Nuget now being more than 14 years old.
A new improved algorithm was developed, which is “optimized to only walk nodes once, resolving the tree as it goes.” Instead of creating 1.6 million nodes in one of Micrsoft’s examples, it created just 1,200 nodes, and restore times were reduced to 2 minutes. Taken together with earlier optimizations, this was a remarkable improvement on the original 30 minute wait.
Unfortunately there can be side-effects. According to an issue posted last week, while the new resolver ends up with the same resulting packages as the old one, it may also download extra packages that are not used. It is all to do with versioning. A developer ran into this exact issue with jQuery versions. The jQuery.Validation package depended on jQuery 1.4.4, but separately jQuery 1.10.2 was specified. This should resolve to just jQuery 1.10.2 being used, but with the new dependency resolver, the restore failed when trying to download jQuery 1.4.4, which was not available. The message was “verify the package exists on the feed and try again.”
The solution is to set a property in the MSBuild configuration to “RestoreUseLegacyDependencyResolver,” which unfortunately also means the old slow restore times.
Despite these early issues, this will be a significant benefit for .NET developers and illustrates the value of “dogfooding” – that Microsoft itself uses the tool internally, increasing the likelihood that problems such as this get fixed.