We need to know the exact version of OpenMPT you're using (upper-right corner of the About dialog) to be able to do anything with that memory dump.

OpenMPT (amd64 (64 bit))
Version 1.31.00.20-r18475 TEST

There it is

Apparently the Windows-internal tree view code gets stuck in an infinite loop when trying to expand some tree view nodes. Similar to this problem apparently https://social.msdn.microsoft.com/Forums/en-US/3fd6c4a2-b5c7-4334-b11a-e909b11e8bdc/quottreeviewsortquot-stuck-under-windows-vista-?forum=netfxbcl (at least the hang is in the same function)

alright, if you need anything from me, just ask.
I will leave you to it otherwise.

I can reproduce it locally now after adding a few more soundfonts to the tree and typing slowly. The only obvious fix that I see at the moment is no longer trying to expand the child folders in soundfonts, which is probably rather annoying when refining a search because every newly typed character will collapse all the soundfont sub-folders again.

Ok. Then it is a good thing we caught this before putting the feature into a stable release, I am glad I could help in a little way. Good luck Saga and the rest of whomever might be capable and interested in looking into this, and thank you.

Apparently the bug goes deeper than just child node expansion. Deleting items can also cause the hang, so just disabling the auto-expansion of bank folders doesn't fix anything. Right now I don't really have any idea how to approach this problem. Apparently several people have found similar problems before when using tree views but the component has stayed broken since at least Windows Vista!

Without knowing exactly what is broken, my best guess is that in both cases, the hang happens when calculating the visual size of the tree and whether scrollbars are needed - similar to issue 0001490.

Ok after a lot of thought, I my final thought on it is we are going to have to work around it somehow since microsoft has left that nonsense in since windows vista. Just wow that they give so few fucks about their buggy code that they just ignore it. I am sorry I have presented you with a problem that might actually be unsolvable, but I hope you find a creative work around.
The only thing I can think of is maybe doing it in stages - rendering the full tree first with the unusable options turned off (like you did with the search initially) and then spreading the load of cutting the list in chunks that will not result in an infinite loop. But obviously I am not a programmer and I am just making a suggestion that makes some sense in my head but probably makes no sense in practice (I imagine something like this could take far more processing than it rightfully should if the tree refresh on microsoft's side was working properly).
Good luck, and I hope you guys find a solution some day.

[edit]
If this isnt an option and we cant find a solution that works then I guess our only option will be to go back to the full list with the non-search result options turned off like it was initially. In that case I am sorry for suggesting the change, and as much as I would prefer the list becomes shorter, I am more than willing to live without it for the sake of stability.

I've tried everything I could think of, but there is simply no reliable workaround. As of r18670, soundfont search has been removed.

The only way this feature could find its way back would be to do the filtering in a completely different place outside of the treeview. But I don't see that happening anytime before the 1.31 release.