View Issue Details

IDProjectCategoryView StatusLast Update
0001548OpenMPTPlugins / VSTpublic2022-01-13 21:16
Reporterexelotl Assigned ToSaga Musix  
PrioritynormalSeveritymajorReproducibilityalways
Status resolvedResolutionfixed 
Platformx64OSWindowsOS Version10
Product VersionOpenMPT 1.30.01.00 / libopenmpt 0.6.0 (current stable) 
Target VersionOpenMPT 1.30.02.00 / libopenmpt 0.6.1 (upcoming stable)Fixed in VersionOpenMPT 1.30.02.00 / libopenmpt 0.6.1 (upcoming stable) 
Summary0001548: Can't load song after adding RetroPlug VST to it.
Description

I added the "RetroPlug" VST plugin to a module, loaded a .gb file through the plugin's UI, then saved my project. Now OpenMPT won't recognise it as a valid module.

I have attached an example of an unreadable module that was created this way.

Attempting to open the file gives a popup with the following error message:

Opening "C:\Users\exelotl\Music\test.it" failed. This can happen if no more modules can be opened or if the file type was not recognised (currently there are 0 documents open).

Steps To Reproduce
  1. Download RetroPlug v0.3.4 (vst2-win64) and extract it to your VstPlugins directory.

  2. Download a compatible freeware Game Boy ROM such as mGB or LSDj.

  3. Launch OpenMPT and create a new project.

  4. Open the Plugin Manager and create a new instance of RetroPlug in the FX01 slot.

  5. Open the RetroPlug UI (via the "editor" button in OpenMPT).

  6. The UI says "Double click to load a rom..." - do this and select the Game Boy ROM you downloaded in step 2.

  7. Save your project, then close it.

  8. Open the project.

Additional Information
  • I tried saving in several formats (.xm, .it, .mptm) and none of them could be loaded afterwards.
  • I tried old versions of RetroPlug, and the problem still occurred, which leads me to believe there's something unique about the way RetroPlug is saving that causes this.
  • I tried other VSTs, including ones that store and load external filepaths (sforzando, DSK SF2), but they didn't cause any problem.
  • I tried RetroPlug in another DAW (Renoise), and no similar problem occurred.
TagsNo tags attached.
Has the bug occurred in previous versions?Yes (tested as far back as 1.26)
Tested code revision (in case you know it)

Activities

Saga Musix

Saga Musix

2022-01-11 22:18

administrator   ~0004988

Apparently this plugin stores its data in ZIP format. This data is then embedded in the OpenMPT module, and somehow the unzip implementation is extremely eager to find this ZIP file hidden in the middle of the module file. I guess we might have to find a way for minizip to be less eager detecting such files, because the file neither starts with the PK signature (which doesn't matter) nor does it end with an end-of-central-directory signature.

If you try to open your MPTM file in e.g. 7-zip, you will see the contents of the plugin's data chunk, which shows that it's not just minizip that's so eager to find it.

Saga Musix

Saga Musix

2022-01-11 22:25

administrator   ~0004989

For the time being, you can circumvent this issue by loading a sample that's at least a few KB in size (10KB or so should be enough, I didn't verify how far into the file minizip tries to find the ZIP signature). This will avoid the ZIP-inside-MPTM being detected by minizip and the file will load as a regular MPTM file. Obviously this is not a long-term fix but it allows you to use the plugin at least.

exelotl

exelotl

2022-01-11 23:36

reporter   ~0004990

Ah thanks! That all makes sense, good to know there's a workaround for the time being. :)

exelotl

exelotl

2022-01-12 00:39

reporter   ~0004991

Update: unfortunately adding samples doesn't seem to solve it (even over 10MB worth of samples, at which point 7-zip stops detecting it as a zip). I guess minizip has no limit on how much of the file it searches? Or the limit is super high.

Saga Musix

Saga Musix

2022-01-12 08:29

administrator   ~0004992

Last edited: 2022-01-12 21:07

I did add a sample and it worked fine here, but just to be clear the module needs to be in IT or MPTM format for that workaround, for XM it won't work because samples are stored before plugin data, not after. From my understanding of minizip's source code, the global comment following the central directory index may be at most 65535 bytes long, so any sample longer than that should work around the problem, but I found it to work properly even with smaller samples.

I'll report this issue to the plugin developer as well, because any other application that can handle both music files and zip files will have the same issue. Maybe the plugin developer can change their chunk storage format.

One way forward for OpenMPT would be to expect ZIP files to start with the "PK" magic bytes before passing the ZIP file even on to minizip, similar to how we do it with RAR files. This would remove the possibility to extract modules from self-extracting ZIP files (which exploit exactly the behaviour you encountered here), but I guess that's not OpenMPT's main functionality as well, so it's something we can do without.

exelotl

exelotl

2022-01-12 16:01

reporter   ~0004993

Ahh right, yes I was using XM, verified that the sample data was at the front and mistakenly thought that was desirable x)

Saga Musix

Saga Musix

2022-01-12 22:07

administrator   ~0004995

Reported to the plugin author: https://github.com/tommitytom/RetroPlug/issues/25

I guess from here there are two ways forward to fixing this in OpenMPT without requiring a plugin fix (and hardening against similar plugins that we don't know about yet):

  • Don't pass files to minizip if they don't start with PK magic bytes - removes support for self-extracting ZIPs
  • Restructure code so that first module formats are tried, and only if they fail handle archive formats.
Saga Musix

Saga Musix

2022-01-13 20:50

administrator   ~0004996

As of r16517, OpenMPT first tries to load files as modules, and only if that files it tries again loading them as an archive and extracting modules from that archive.
This fixes modules using RetroPlug, but also crazy modules that contain ZIP files as raw samples (I wouldn't be surprise if that has been done already).

Issue History

Date Modified Username Field Change
2022-01-11 22:02 exelotl New Issue
2022-01-11 22:18 Saga Musix Note Added: 0004988
2022-01-11 22:25 Saga Musix Note Added: 0004989
2022-01-11 23:36 exelotl Note Added: 0004990
2022-01-12 00:39 exelotl Note Added: 0004991
2022-01-12 08:29 Saga Musix Note Added: 0004992
2022-01-12 16:01 exelotl Note Added: 0004993
2022-01-12 21:07 Saga Musix Note Edited: 0004992
2022-01-12 22:07 Saga Musix Note Added: 0004995
2022-01-12 22:07 Saga Musix Assigned To => Saga Musix
2022-01-12 22:07 Saga Musix Status new => assigned
2022-01-13 20:50 Saga Musix Note Added: 0004996
2022-01-13 21:16 Saga Musix Status assigned => resolved
2022-01-13 21:16 Saga Musix Resolution open => fixed
2022-01-13 21:16 Saga Musix Fixed in Version => OpenMPT 1.30.02.00 / libopenmpt 0.6.1 (upcoming stable)
2022-01-13 21:16 Saga Musix Target Version => OpenMPT 1.30.02.00 / libopenmpt 0.6.1 (upcoming stable)