View Issue Details

IDProjectCategoryView StatusLast Update
0001030OpenMPTPlugins / VSTpublic2020-01-12 14:45
ReporterSolus Assigned Tomanx  
PrioritynormalSeveritycrashReproducibilityalways
Status resolvedResolutionfixed 
Platformx64OSWindowsOS Version10
Product VersionOpenMPT 1.27.00.* (old testing) 
Target VersionOpenMPT 1.27.01.00 / libopenmpt 0.3.1 (upgrade first)Fixed in VersionOpenMPT 1.27.01.00 / libopenmpt 0.3.1 (upgrade first) 
Summary0001030: VSTi Synth1 64bit crashes / not working
Description

The 64bit version of Synth1 crashes by activation, giving a "The plugin threw an exception (C0000005) in processReplacing." message.
"Use plugin bridge" fixes the crash, but the plugin doesn't really work anyway.
The 32bit version works fine. (but let's face it: 32bit is dead - time to move on... ;) )

Steps To Reproduce
  • open the 64bit version of Synth 1 and play a note.
TagsNo tags attached.
Has the bug occurred in previous versions?
Tested code revision (in case you know it)

Relationships

related to 0001291 resolvedmanx Separate plugin bridge versions for legacy / buggy plugins 
has duplicate 0001035 closed Synth1 error when in poly mode 

Activities

Saga Musix

Saga Musix

2017-09-23 12:50

administrator   ~0003235

Synth1 works just fine here, both the 32-bit and 64-bit version. It could of course be another Windows 10 specific issue, but I only have a 32-bit VM to test.
Are you using the latest Synth1 version? Do any of the Windows compatibility options help?

Solus

Solus

2017-09-24 22:28

reporter   ~0003243

Well, it's V1.13 beta3 from 2014. Should be the latest.
But how can I use Windows compatibility options on a dll ?

Saga Musix

Saga Musix

2017-09-24 22:29

administrator   ~0003244

You don't, you enable them on mptrack.exe. And please attach the generated crash dump to have a look at.

Solus

Solus

2017-09-25 00:01

reporter   ~0003245

Compatibility options on mptrack.exe don't do anything for the plugin. And as only the plugin itself crashes, and not openmpt, there is no crash dump (isn't it?).
I will test the plugin tommorow on a different win10/64-system...

Solus

Solus

2017-09-25 08:57

reporter   ~0003246

Okay, so it seems related to the version of OpenMPT.
I first installed OpenMPT 1.26.14.00 on the other system and Synth1 64bit worked.
Switching to an 1.27 / 1.28 build made the crash return.

Saga Musix

Saga Musix

2017-09-25 11:05

administrator   ~0003247

Last edited: 2017-09-25 11:05

And as only the plugin itself crashes, and not openmpt, there is no crash dump (isn't it?).

Sorry, I forgot that you actually got an error message from OpenMPT's plugin handler and not from the crash handler. (If the process crashed, it would not matter which part of it crashed).

I cannot really think of anything right now that would cause the crash that would be new to OpenMPT 1.27, so if you have the time, please perform a regression test:
This is done by downloading various old builds and performing a binary search to find the bug.
The first archive 1.27 version at https://builds.openmpt.org/builds-archive/auto/openmpt-win64/ is r6782 and the latest is r8649. Calculate the average of those two values (7715), find this (or a revision close to it, not every revision will be available on buildbot) and download it. Does it crash? -> Repeat the step by taking the lower bound (6782) and the new upper bound (7715), if it doesn't crash, repeat the step by taking the new lower bound (7715) and the upper bound (8649). This way, you should be able to find the revision at which the fault was introduced in about 14 steps.

Solus

Solus

2017-09-25 12:49

reporter   ~0003248

Synth1 still works:
openmpt-win64-1.27.00.06-r6997.7z

Synth1 crashes:
openmpt-win64-1.27.00.07-r6999.7z

Saga Musix

Saga Musix

2017-09-25 13:03

administrator   ~0003249

Interesting. Starting with r6999, test builds were compiled using VS2015 instead of VS2010. No other VST-related changes happened there... (cc @manx)

manx

manx

2017-09-25 13:31

administrator   ~0003250

@Solus:
We will provide special builds for you once we have decided which way to pursue first.

Technical details:

Yes, compiler changed in that revision.

So, the root cause might be a compiler bug, in which case testing the with disabled optimization around VST plugin handling as well as testing with VS2017 might help.

Another thing that could cause the problem is alignment of the actual float buffers which could be a problem if Synth1 uses SSE2 here (in which case it would be a Synth1 bug). As far as I can tell, these buffers are 4 byte aligned, however, I do not see any way how any possible additional better alignment could change between compiler versions by chance.

However, IMixPlugin contains bool bitfields right after the mix buffer, which frankly rings a ton of alarm bells, given out history with this language feature.

manx

manx

2017-09-25 13:35

administrator   ~0003251

In any case, I cannot really see how the possible problems I have describe could actually only affect Win10 and Synth1.

manx

manx

2017-09-25 14:18

administrator   ~0003252

Could you please try each of

Saga Musix

Saga Musix

2017-09-25 15:06

administrator   ~0003253

As far as I can tell, these buffers are 4 byte aligned

Just a small clarification: Plugin buffers provided by OpenMPT are already manually aligned to 16 bytes (see PluginMixBuffer.h) since basically forever.

manx

manx

2017-09-25 15:11

administrator   ~0003254

Oh, yes, missed that.

Solus

Solus

2017-09-25 18:12

reporter   ~0003255

The plugin crashes with all three versions.

Saga Musix

Saga Musix

2017-09-25 18:43

administrator   ~0003256

Last edited: 2017-09-25 18:44

For what it's worth, I installed a Win10 x64 VM and can reproduce the issue there, but I have no dev environment set up in that VM yet to further analyze the problem; it's very well possible that we cannot fix this if Synth1 makes any assumptions e.g. about memory allocation that are no longer true with VS2015/2017.

Saga Musix

Saga Musix

2017-09-25 19:02

administrator   ~0003257

Last edited: 2017-09-25 19:10

Here's another try: https://sagagames.de/stuff/mptrack.exe
This version removes OpenMPT's "safety net" around the plugin processing code, so now it should actually crash the entire process when using Synth1. You should be able to obtain a memory dump from that crash.

Solus

Solus

2017-09-25 19:16

reporter   ~0003258

Well, now the plugin doesn't crash anymore. It plays the instrument, but only one note at the time. The behaviour is similar to what the plugin does, when I activate "use plugin bridge".

Saga Musix

Saga Musix

2017-09-25 19:18

administrator   ~0003259

This is really strange. I can now confirm that Windows 8.1 is also affected.

Saga Musix

Saga Musix

2017-09-25 21:18

administrator   ~0003260

Last edited: 2017-09-25 21:27

Looking at the disassembly of the crash location a bit more, it looks like a pointer trunction issue to me. In one specific example, Synth1 tries to read from address 49656AB0 but the real address should be A749656AB0 (only the lower 32 bits are preserved). Windows 8 probably changed the process memory layout, and MSVC10's might be using a different heap layout, hiding the issue at hand.
In other words, there is nothing we can do here, but I have notified the Synth1 developer of this issue. Other 64-bit hosts compiled with a recent MSVC compiler should have the same issue as OpenMPT. It's impossible for us at this stage to return to MSVC10.

Well, now the plugin doesn't crash anymore. It plays the instrument, but only one note at the time. The behaviour is similar to what the plugin does, when I activate "use plugin bridge".

In both cases the program actually crashes but is unable to display its crash message for some reason. So the crash is not gone with the bridge, it's just "less visible".

Saga Musix

Saga Musix

2017-09-25 22:55

administrator   ~0003261

Last edited: 2017-09-25 22:57

Other developers have faced the same issue with Synth1: https://github.com/teotigraphix/Framework4Bitwig/issues/3
Maybe now that I sent a mail to Ichiro, he will do something about it. ;)

A possible semi-solution would be to disable ASLR at least for the plugin bridge. I wouldn't feel good about disabling it in the main process.

manx

manx

2017-09-26 10:14

administrator   ~0003262

Yes, as a work-around, we could either disable ASLR completely for the PluginBridge, or tame it done to Windows 7 level as described here:
https://blogs.msdn.microsoft.com/oldnewthing/20150709-00/?p=45181
https://docs.microsoft.com/en-us/cpp/build/reference/highentropyva

Longer term, it might even make sense to have 2 versions of the plugin bridge, 1 with all the normal features enabled and 1 with extremely conservative settings. Another aspect which might require providing multiple bridge versions is high DPI support, although that can be implemented during process startup. ASLR however can not (at least, as far as I know).

Saga Musix

Saga Musix

2017-09-26 10:30

administrator   ~0003263

Another aspect which might require providing multiple bridge versions is high DPI support, although that can be implemented during process startup

OpenMPT already sets this value at process startup, and it's not possible to set it in the plugin bridge again because it's a child window of OpenMPT - I tried that a while ago, and bridged plugins always inherited OpenMPT's DPI (non-)awareness.

manx

manx

2017-09-26 10:33

administrator   ~0003264

OpenMPT already sets this value at process startup, and it's not possible to set it in the plugin bridge again because it's a child window of OpenMPT - I tried that a while ago, and bridged plugins always inherited OpenMPT's DPI (non-)awareness.

In that case, not making the bridge and plugin windows child windows may be another option.

Anyway, this branch of the discussion is getting off-topic for this issue.

manx

manx

2017-09-26 11:16

administrator   ~0003265

@Solus:
We applied the work-around (disabling High Entropy address space layout randomization) in r8984 . 64bit Synth1 will still not work in OpenMPT, however, it should work when using the Plugin Bridge now.
That version will show up in about an hour at https://builds.openmpt.org/ .

Solus

Solus

2017-09-27 12:14

reporter   ~0003266

Yes, this works. Thanks again for your ongoing and always enthusiastic work on MPT !

Issue History

Date Modified Username Field Change
2017-09-23 09:58 Solus New Issue
2017-09-23 12:50 Saga Musix Note Added: 0003235
2017-09-24 22:28 Solus Note Added: 0003243
2017-09-24 22:29 Saga Musix Note Added: 0003244
2017-09-25 00:01 Solus Note Added: 0003245
2017-09-25 08:57 Solus Note Added: 0003246
2017-09-25 11:05 Saga Musix Note Added: 0003247
2017-09-25 11:05 Saga Musix Note Edited: 0003247
2017-09-25 12:49 Solus Note Added: 0003248
2017-09-25 13:03 Saga Musix Note Added: 0003249
2017-09-25 13:31 manx Note Added: 0003250
2017-09-25 13:35 manx Note Added: 0003251
2017-09-25 14:18 manx Note Added: 0003252
2017-09-25 15:06 Saga Musix Note Added: 0003253
2017-09-25 15:11 manx Note Added: 0003254
2017-09-25 18:12 Solus Note Added: 0003255
2017-09-25 18:43 Saga Musix Note Added: 0003256
2017-09-25 18:44 Saga Musix Note Edited: 0003256
2017-09-25 19:02 Saga Musix Note Added: 0003257
2017-09-25 19:03 Saga Musix Note Edited: 0003257
2017-09-25 19:10 Saga Musix Note Edited: 0003257
2017-09-25 19:16 Solus Note Added: 0003258
2017-09-25 19:18 Saga Musix Note Added: 0003259
2017-09-25 21:18 Saga Musix Note Added: 0003260
2017-09-25 21:22 Saga Musix Note Edited: 0003260
2017-09-25 21:26 Saga Musix Note Edited: 0003260
2017-09-25 21:27 Saga Musix Note Edited: 0003260
2017-09-25 22:55 Saga Musix Note Added: 0003261
2017-09-25 22:57 Saga Musix Note Edited: 0003261
2017-09-26 10:14 manx Note Added: 0003262
2017-09-26 10:30 Saga Musix Note Added: 0003263
2017-09-26 10:33 manx Note Added: 0003264
2017-09-26 11:16 manx Note Added: 0003265
2017-09-27 12:14 Solus Note Added: 0003266
2017-09-27 12:14 manx Target Version => OpenMPT 1.27.01.00 / libopenmpt 0.3.1 (upgrade first)
2017-09-27 12:15 manx Assigned To => manx
2017-09-27 12:15 manx Status new => resolved
2017-09-27 12:15 manx Resolution open => fixed
2017-09-27 12:15 manx Fixed in Version => OpenMPT 1.27.01.00 / libopenmpt 0.3.1 (upgrade first)
2017-09-27 21:21 Saga Musix Relationship added has duplicate 0001035
2020-01-12 14:45 Saga Musix Relationship added related to 0001291