tracing registry corruption issue

1. Hopper test reported sth like and paused:
@18:15:34.838 MTTF: RegCreateKeyEx failed to create the key SOFTWARE\Microsoft\Hopper, error code 14, hKey 0x0

2. Setting up a DEBUGBREAK at the place frequently check registry and catch the issue;

3. CeDebugX reported:
======================================================= Fixed-sized heap in filesys.exe is full =======================================================
Process : filesys.exe Proc Id : 0x00000001 Heap : 0x040b0000 Heap max size : 1048576 Free bytes : 64 Largest free block : 64
This does not necessarily indicate a bug in itself. However, it may be thecause of other problems, e.g. if allocation attempts on this heap are failing,and should be examined.

Tags for bug matcher:+DEFECT:LOW_MEMORY:FIXED_HEAP_FULL:filesys.exe:

it means that windows registry handle pool(0x040b0000, 1M) got full;

dump out the whole 1m memory and analyzed the handle, found out that most of handles are belong to device.exe; Sample:

cedebugx50>dumpitem 0x040B7098 filesys.exe
Creating heap list...
pItem : 0x040b7078
Type : Normal Alloc
Total size : 96 (including header)
Data size : 88 (allocation size)
Heap ptr : 0x040b0030

addr value
========================== header
0x040b7078 : 0x00000060 size
0x040b707c : 0x040b0030 region ptr
========================== data
0x040b7080 : 0x040b7080 <-- pCur
0x040b7084 : 0xe3a2641e <-- hProc : application handle
0x040b7088 : 0x000003f5 <-- bDepth : 3; bMagic: 0xf5
0x040b708c : 0x040b7020 <-- pNext
0x040b7090 : 0x00000001 <-- dwFlags
0x040b7094 : 0x000540f0 <--pHives
0x040b7098 : 0x80000002 <-- HKLM
0x040b709c : 0x00000008 <-- System
0x040b70a0 : 0x000000c4 <-- State
0x040b70a4 : 0x00000106 <-- ???
0x040b70a8 : 0x00000000
0x040b70ac : 0x00000000
0x040b70b0 : 0x00000000
0x040b70b4 : 0x00000000
0x040b70b8 : 0x00000000
0x040b70bc : 0x00000000
0x040b70c0 : 0x00000000
0x040b70c4 : 0x00000000
0x040b70c8 : 0x00000000
0x040b70cc : 0x00000000
0x040b70d0 : 0x00000000
0x040b70d4 : 0x00000000

4. use tools to test different key value and find the reprents in handle heap, here is the sample.
0x040b3c18 80000002 20001551 00000000 00000000 .... Q... .... .... <------- Explorer
0x040b5538 80000002 200010FA 00000000 00000000 .... .... .... .... <------- ExtModems
0x040b5598 80000002 000002B2 00000000 00000000 .... .... .... .... <------- HARDWARE
0x040b55f8 80000002 00000033 00000000 00000000 .... 3... .... .... <------- Ident
0x040b5658 80000002 20000001 00000000 00000000 .... .... .... .... <------- init
0x040b56b8 80000002 2000009F 00000000 00000000 .... .... .... .... <------- Loader
0x040b5718 80000002 20000118 00000000 00000000 .... .... .... .... <------- MUI
0x040b5778 80000002 00000001 00000000 00000000 .... .... .... .... <------- nls
0x040b57d8 80000002 2000164E 00000000 00000000 .... N... .... .... <------- Platform
0x040b5838 80000002 00000014 00000000 00000000 .... .... .... .... <------- Security
0x040b5898 80000002 000000CA 00000000 00000000 .... .... .... .... <------- Services
0x040b58f8 80000002 000002F4 00000000 00000000 .... .... .... .... <------- Snd
0x040b5958 80000002 00000028 00000000 00000000 .... (... .... .... <------- Software
0x040b59b8 80000002 00000008 00000000 00000000 .... .... .... .... <------- System
0x040b5a78 80000002 00000023 00000000 00000000 .... #... .... .... <------- TAPI
0x040b5d18 80000002 2000297D 00000000 00000000 .... }).. .... .... <------- Audio
0x040b5dd8 80000002 20000DC2 00000000 00000000 .... .... .... .... <------- AudioCompressionManager
0x040b5e38 80000002 20001C4A 00000000 00000000 .... J... .... .... <------- ButtonApps
0x040b5e98 80000002 0000001E 00000000 00000000 .... .... .... .... <------- Comm
0x040b5ef8 80000002 00000301 00000000 00000000 .... .... .... .... <------- ControlPanel
0x040b5f58 80000002 00000003 00000000 00000000 .... .... .... .... <------- Drivers
0x040b5fb8 80000002 20000DC0 00000000 00000000 .... .... .... .... <------- Drivers32
0x040b6018 80000002 00000023 00000000 00000000 .... #... .... .... <------- TAPI
0x040b6078 80000002 00000023 00000024 00000000 .... #... $... .... <------- TAPI\TSP
0x040b60d8 80000002 00000023 00000024 20001EC5 .... #... $... .... <------- TAPI\TSP\CellTSP.dll
0x040b6138 80000002 00000023 00000024 00000025 .... #... $... %... <------- TAPI\TSP\Unimodem.dll
0x040b6198 80000002 20001642 00000000 00000000 .... B... .... .... <------- Telephony
0x040b61f8 80000002 20000001 00000000 00000000 .... .... .... .... <------- init
0x040b6258 80000002 2000058B 00000000 00000000 .... .... .... .... <------- Windows CE Services
0x040b62b8 80000002 20001559 00000000 00000000 .... Y... .... .... <------- Windows CE Tools
0x040b6318 80000002 00000008 20001468 00000000 .... .... h... .... <------- System\ActiveSync
0x040b6378 80000002 00000008 200001C7 00000000 .... .... .... .... <------- System\Autoupdate
0x040b63d8 80000002 00000008 00000009 00000000 .... .... .... .... <------- System\Ceddk
0x040b6438 80000002 00000008 20002CCE 00000000 .... .... .,.. .... <------- System\ChoosePicture
0x040b6498 80000002 00000008 00000047 00000000 .... .... G... .... <------- System\CurrentControlSet
0x040b64f8 80000002 00000008 2000294C 00000000 .... .... L).. .... <------- System\D3DM
0x040b6558 80000002 00000008 20000543 00000000 .... .... C... .... <------- System\DOWNLOADINSTALLSERVICE
0x040b65b8 80000002 00000008 0000004F 00000000 .... .... O... .... <------- System\ErrorReporting
0x040b6618 80000002 00000008 2000000D 00000000 .... .... .... .... <------- System\Events
0x040b6678 80000002 00000008 20001238 00000000 .... .... 8... .... <------- System\Explorer
0x040b66d8 80000002 00000008 000000C6 00000000 .... .... .... .... <------- System\FileSys
0x040b6738 80000002 00000008 000000B5 00000000 .... .... .... .... <------- System\GDI
0x040b6798 80000002 00000008 000000BC 00000000 .... .... .... .... <------- System\GWE
0x040b67f8 80000002 00000008 200000BE 00000000 .... .... .... .... <------- System\ImageUpdate
0x040b6858 80000002 00000008 000000C0 00000000 .... .... .... .... <------- System\IME
0x040b68b8 80000002 00000008 0000034F 00000000 .... .... O... .... <------- System\Inbox
0x040b6918 80000002 00000008 000000FC 00000000 .... .... .... .... <------- System\IPSecVPN
0x040b6978 80000002 00000008 00000019 00000000 .... .... .... .... <------- System\Loader
0x040b69d8 80000002 00000008 200017ED 00000000 .... .... .... .... <------- System\Notifications
0x040b6a38 80000002 00000008 2000008E 00000000 .... .... .... .... <------- System\ObjectStore
0x040b6a98 80000002 00000008 20000126 00000000 .... .... &... .... <------- System\OOM
0x040b6af8 80000002 00000008 0000035E 00000000 .... .... ^... .... <------- System\Pictures
0x040b6b58 80000002 00000008 0000033E 00000000 .... .... >... .... <------- System\PIMSources
0x040b6bb8 80000002 00000008 2000011B 00000000 .... .... .... .... <------- System\Platform
0x040b6c18 80000002 00000008 00000368 00000000 .... .... h... .... <------- System\Shell
0x040b6c78 80000002 00000008 200005B2 00000000 .... .... .... .... <------- System\SQM
0x040b6cd8 80000002 00000008 000000C4 00000000 .... .... .... .... <------- System\State
0x040b6d38 80000002 00000008 0000000B 00000000 .... .... .... .... <------- System\StorageManager
0x040b6d98 80000002 00000008 00000376 00000000 .... .... v... .... <------- System\Uptime
0x040b6df8 80000002 00000008 20001374 00000000 .... .... t... .... <------- System\Versions
0x040b6e58 80000002 00000008 200002DE 00000000 .... .... .... .... <------- System\WelcomeCenter
0x040b6eb8 80000002 00000008 20001468 00000000 .... .... h... .... <------- System\ActiveSync
0x040b6f18 80000002 00000008 000000C4 00000000 .... .... .... .... <------- System\State
0x040b6f78 80000002 00000008 000000C4 000000F6 .... .... .... .... <------- System\State\ActiveSync
0x040b6fd8 80000002 00000008 000000C4 00000102 .... .... .... .... <------- System\State\Battery
0x040b7038 80000002 00000008 000000C4 00000370 .... .... .... p... <------- System\State\Connections
0x040b7098 80000002 00000008 000000C4 00000106 .... .... .... .... <------- System\State\Connectivity
0x040b70f8 80000002 00000008 000000C4 0000036F .... .... .... o... <------- System\State\DateTime
0x040b7158 80000002 00000008 000000C4 00000110 .... .... .... .... <------- System\State\Hardware
0x040b71b8 80000002 00000008 000000C4 000000C8 .... .... .... .... <------- System\State\Phone
0x040b7218 80000002 00000008 000000C4 20000E5E .... .... .... ^... <------- System\State\Power
0x040b7278 80000002 00000008 000000C4 0000010D .... .... .... .... <------- System\State\Shell
0x040b72d8 80000002 00000008 000000C4 000000D8 .... .... .... .... <------- System\State\VoIP
0x040b7338 80000002 00000008 000000C4 00000000 .... .... .... .... <------- System\State