If you’ve ever fooled around with the PSAPI library under Windows NT to build a list of processes running on the system (using the EnumProcesses() API), you probably know that you won’t get the 16-bit Windows Applications that way. Instead, all you’ll get is the list of NTVDM.EXE instances on the system.
Well, there’s a [relatively] simple way to get around that limitation, but the answer is not some hidden functions on PSAPI or the Performance Counters in the registry. The answer is to look for the Win16 debugging facilities built onto NT itself. The thing goes like this:
The WinNT NTVDM subsystem provides a set of services that allow a Win32 process to debug 16-bit Windows applications running on the system. For this, there’s a library called VDMDBG.DLL, for which all functions are exported through the vdmdbg.h and vdmdbg.lib files you’ll find in the Platform SDK. Among those are some calls that allow us to get the list of 16-bit processes running under a Virtual Machine in the NT subsystem. Most of this function are documented in VDMDbg.hlp, which comes with the SDK, with one notable exception noted later.
Briefly, What you have to do is call the VDMEnumProcessWOW() function, which will call a CALLBACK function in your program for every process on the machine which are Win16 subsystems, which are usually instances of NTVDM.EXE. On the first version of NT, there could only be one Win16 subsystem executing on the machine, but on later releases, there can be more than one, which provides address space separation. For each Win16 Subsystem, you have to enumerate the 16-bit processes running inside them, which are referred to as Tasks. For this, we can use the VDMEnumTaskWOW() call.
The problem with this function is that in the callback procedure we only get the Thread ID in the Win16 subsystem under which the task is running, as well as win16 handles to the process and the task. To be able to get the filename of the running executable, you would need to call VDMModuleFirst(). The problem is that to be able to call this functions, WOWDEB.EXE needs to be running inside the win16 subsystem that runs the application. Getting this to work is a pain the a**, and I really couldn’t do it.
However, it turned out that there was an even easier way. There’s an undocumented call that appears on vdmdbg.h, called VDMEnumTaskWOWEx() which works just like the original, with the notable exception that the callback also receives the module name and the full path of the executable as parameters.
To show how this is done, I built a simple, dialog based application written in plan C. It’s small, and doesn’t do anything interesting besides showing the 16-bit processes running on the system, but it does show how to do it in a clear manner.
You can download a zipped file with the project files (written on VC++ 6.0) from here.
And finally, here’s a snapshot of the sample’s screen: