RANT: Microsoft.UniversalStore.HardwareWorkflow.SubmissionBuilder.dll

There's naive code, there's bad code, there's unoptimized code, and then there's Microsoft's code.

If you ever had to deal with Windows drivers, you might be aware of Inf2Cat command-line tool which is part of Windows Driver Kit. It is used to generate driver catalog files (.CAT) which are then digitally signed by Microsoft (or your trusted certificate) so you can install the devices listed in accompanying .INF files.

Inf2Cat.exe in latest versions of WDK relies on the following .NET assemblies to do that one job, all of them named Microsoft.UniversalStore.HardwareWorkflow.*.dll where * is one of:

• Cabinets
• Catalogs
• InfReader
• SubmissionBuilder

They all reference each other, and they all share one common feature — they have horrible performance. Did I say horrible? I meant criminal. As in, someone should be criminally liable for it, and no, I am not exaggerating.

If you want to follow through to confirm my findings, go grab a coffee, a sedative, pillow, and a blanket. Maybe also a teddy bear so you can cry in the corner when you're done.

Take for example NVIDIA Drivers — they are big, unpacked version sits at 2.72 GB just in Display.Driver folder which is our target. Let's say you want to disable driver telemetry so you do a search for all NvSupportTelemetry = 1 lines in all 43 .INF files which total 51.75 MB, flip that 1 to 0 and run Inf2Cat.exe on the folder to generate a new nv_disp.cat file.

And then you wait...

And wait...

And wait some more...

And you start thinking "Maybe my PC is really slow" but then you go and look at your gear and remind yourself that Xeon w5-2455X running at 4.6 GHz with 64 GB RAM and Samsung 990 Pro NVME drive has no business taking this much time generating a puny 5.07 MB catalog file.

Then the despair starts setting in, and you go and pull another espresso, and shaky like a hamster in a wheel you wait...

And wait...

And after (checks stopwatch) 32 min 2 sec it is finally done. And so are you, because after 1,922 sec of your monster PC appearing mostly idle you forgot what you were doing, why, and you are questioning your life choices.

And if you are anything like me, after 5 of those 32 minutes you would have grabbed dnSpy or ILDasm to check WHAT THE FUCK IS GOING ON IN THERE. And that's exactly what I did and I was not amused.

1. It takes 16 sec just for file enumeration + per-file copy into %TEMP% cache (267 files) — it runs all files though expand.exe. Files which in 99% of cases aren't even compressed because nobody compresses individual driver files anymore. Files which if they were compressed would have an underscore as the last character in the file extension (i.e. .ex_).
2. It takes 31 min 30 sec for .INF processing (43 files) — it builds a 256.85 MB mutable System.Xml.XmlDocument as its working store, then uses full-document XPath (SelectNodes("/driverPackage/file[@name=...]")) as a lookup table. Since XmlDocument has no index, every call is a linear DOM walk. Those calls are nested several loops deep (ProcessCopyFiles -> per file-list item -> per source-disks item -> SelectNodes, plus a second token-resolved SelectNodes on a miss) and repeated per referenced file per .INF. The work is ~O(N³) in package size, single-threaded, with nothing hoisted.
3. It takes another 16 sec for finalization and actual .CAT file output.

Even the point #1 above is already bad enough — spawning 267 child processes just to expand (or if not compressed, copy) files to work folder is sheer architectural insanity. There are so many ways to speed that up if it's still necessary to do (which I highly doubt), that I won't waste your time listing them all but what comes to mind is:

• Build a list of files (Dictionary<string, bool>)
• Read first 4 bytes of each and if SZDD or MSCF mark as compressed.
• Do any other checks if needed (test for unsupported multi-volume cabs for example, but those aren't used for decades as floppies are all but extinct by now)
• If all files are compressed use a work folder, if none are compressed work in-place. If some are compressed, expand those to a work folder, and reference the rest from the original — don't copy 2.72 GB for nothing like an idiot.
• Use a fucking library to expand files instead of launching expand.exe processes like it's 2000s.
• Run expansion in parallel on several files — servers have NVME storage for the last 10 years at least.

Etc, etc, you get the idea.

This part already shows that absolutely no thought whatsoever went into writing of that code. If you dare decompile it to verify my claims you will see that it looks like something someone who learned programming from Stack Overflow has been hammering at until it barely worked using techniques they knew (hey, XML is all the rage these days so why not use XML DOM instead of trie, right?) and left it at that.

A word of warning though, don't stare too long into that abyss, because it will absolutely stare back at you and it will come back to haunt you in your dreams. And if by any chance you are on sedatives or anti-depressants look at it at your own risk.

OK, so point #1 is embarrassing but it can be waved off as an intern-level incompetence. Point #2 however crosses the line into complete software engineering malpractice.

The code parsing those .INF files is the kind of code whose creation should be considered criminal negligence at the minimum — people who write code like this should lose a license to work in the field and go to jail for the damage their code has caused.

"So what's the damage?" I hear you ask. Good that you asked, here's the timing (on the same hardware, with the same set of driver files) of CATBuilder C++ library I co-wrote with Claude few days ago:

Yes, you read that right — that's 575x faster. Suddenly that SubmissionBuilder part of the Microsoft's library name sounds like a typo — SubmissionBeater would be a much better fit.

So what's the damage? If say Azure Portal is using the same code to process submissions for driver signing or even Universal Store apps as the name would suggest, that's 575x less submissions in the same amount of time. It is also 1 core of a multi-core server chugging molasses through a tight straw at 100% utilization for 32 min not allowing the OS to park it to save power, and the same probably goes for storage.

Potential blast radius includes, but is not limited to:

1. Increased energy use (so much for all those sleep studies in Windows 10 and 11, so much for "we are green")
2. Queue bottleneck for hardware developers who have to take a lunch break when their CI/CO pipeline hits the "submit for catalog signing" phase

Also, my poor nerves. Would somebody think of my poor nerves?!? I hope Raymond Chen reads this, so he can be as enraged as I was when I looked into the abyss and some of my neurons permanently shut themselves down in protest.

Closing thoughts

I believe it's about time that software engineering got the same level of scrutiny and liability as civil engineering. If only there was some union of countries on this continent which could pass some laws instead of only being concerned with endless arms races, thought policing, and warmongering.