• Start PMON from NT Resource Kit.
• Monitor paged / non-paged pool usage. If these increase over time, there is a memory leak.
• Monitor commit counters. If this increases over several hours, there is a probable leak.
• Monitor the Commit Charge column. The process with the leak will increase.

To release the leaked memory, use Resource Kit utility:

empty { PID | progname }

where PID is Product Identification number. EMPTY frees the working set of a task or process

Microsoft TechNet related articles:

How to Use Dh.exe to Troubleshoot User-Mode Memory Leaks
How to Use Poolmon to Troubleshoot Kernel Mode Memory Leaks
Using Performance Monitor To Identify A Pool Leak
MS-DOS-Based Apps Starting Non-MS-DOS-Based Apps
WM_DDE_EXECUTE API Causes a Memory Leak in the WOW Subsystem
SetTimer() API Causes Memory Leak in the WOW Subsystem
Memory Leak in Ntfs.sys
Memory Leak in SERVICES.EXE Causes Performance Degradation

There are many examples. Many are fixed in various service pack releases. Many are not. If your box gets sluggish or unreliable, this is an area worth checking.