View Issue Details

IDProjectCategoryView StatusLast Update
0000672OpenMPTFile Format Supportpublic2021-03-20 14:16
ReporterAmaroq_Dricaldari Assigned ToSaga Musix  
PrioritynoneSeverityfeatureReproducibilityN/A
Status resolvedResolutionfixed 
Platformx86OSWindowsOS Version7
Product VersionOpenMPT 1.24.02.* (old testing) 
Target VersionOpenMPT 1.30.01.00 / libopenmpt 0.6.0 (upgrade first)Fixed in VersionOpenMPT 1.30.01.00 / libopenmpt 0.6.0 (upgrade first) 
Summary0000672: SymMOD support
Description

Grant OpenMPT the ability to open modules composed in Symphonie Pro

Additional Information

I have provided a link to a Google Drive download, which contains the Symphonie Pro source code and a SymMOD.

https://drive.google.com/file/d/0B1URcZnhifwbaVVGQzN4OWF5RTg/view?usp=sharing

TagsNo tags attached.
Has the bug occurred in previous versions?
Tested code revision (in case you know it)

Activities

Saga Musix

Saga Musix

2015-04-11 22:25

administrator   ~0002031

I won't be going through 700kb of 68k assembly to write a mod loader, seriously. If there is no textual format description, you are pretty much out of luck.

Amaroq_Dricaldari

Amaroq_Dricaldari

2015-04-11 22:36

reporter   ~0002032

Could you leave this open in case somebody else wants to look into it?

Saga Musix

Saga Musix

2015-04-11 22:42

administrator   ~0002033

I didn't even close it but do you seriously think there are thousands of people out there just waiting to translate 50,000 of mostly uncommented assembly code to C++? Let me give you a realistic answer: No, there aren't. In fact I couldn't even find a hint where the player code would be hidden in this gigantic pile of code, if it's in there at all.

Revenant

Revenant

2017-02-20 21:49

reporter   ~0002886

http://aminet.net/package/mus/play/SymphoniePlayer

This is a Java-based player for Symphonie modules (including source) by the author of the original tracker, which contains much easier-to-follow code for loading modules. I haven't used this tracker at all so I don't know how easily convertible the modules are, but if somebody wants to attempt it, check out src/symreader/ImportSongSymphonie.java.

Revenant

Revenant

2017-02-23 02:43

reporter   ~0002888

After looking at the player some more, I think I can take a crack at writing a symmod loader sometime soon.

Before I do, though, is the fact that there would now be more 32 MODTYPE flags going to be a problem?

Saga Musix

Saga Musix

2017-02-23 10:40

administrator   ~0002889

Technically we could probably use a 64-bit enum. However, I wouldn't be too happy with that either. Many formats don't really need this flag (especially all the MOD-like ones) so I would like to solve this issue in a different way. However, this shouldn't stop you from writing a new loader, as I will take care of that.

manx

manx

2017-02-23 21:15

administrator   ~0002890

In particular, MOD_TYPE_WAV and MOD_TYPE_UAX are completely bogus anyway. I think I already have a working copy lying around that does remove them. I guess I should finish that work now.

Revenant

Revenant

2017-02-24 01:10

reporter   ~0002893

Is it even currently possible for a module to have more than one of the type flags set at the same time? It doesn't seem like this is ever actually done anywhere, unless I'm missing something.

manx

manx

2017-02-24 09:15

administrator   ~0002894

Last edited: 2017-02-24 11:51

AFAICS it is not possible to have multiple MOD_TYPE flags set for a single module, but I think it had been possible (or had been planned to be possible) in the past.
In any case, changing that requires modifying a ton of places doing if(GetType() & (MOD_TYPE_IT | MOD_TYPE_MPT | MOD_TYPE_DBM)) or similar. It is also worth noting that converting these bitwise flag bits tests to an actual logical tests of enum values would produce way more code. This could have performance implications if we are actually checking these in the mixer code (which I think we luckily do not).
In any case, the easy step is removing the bogus type flags, and the next easy step would be just using 64 bits instead of 32 bits. The long-term solution is to actually use enum values instead of flags, but I do not think there is currently any pressure to do that right now.

Saga Musix

Saga Musix

2017-02-24 11:02

administrator   ~0002895

Last edited: 2017-02-24 12:46

There never was more than one of those flags set at once, but I wanted to leave the possibility to do so, if ever required (though with m_playBehaviour we probably do not need that nowadays). As seen in manx' example, we use these flags to quickly check module capabilities of several formats at once, and the code would become much less readable if we had to do this using e.g. std::bitset or tons of GetType() == MOD_TYPE_IT || GetType() == MOD_TYPE_MPT || ...).

Revenant

Revenant

2017-02-24 18:42

reporter   ~0002896

Alright, that makes sense. I went ahead and started using 64-bit MODTYPE for the time being, but I guess it won't be a problem once the unneeded values are removed and I can switch to using one of those instead.

Saga Musix

Saga Musix

2017-02-24 20:53

administrator   ~0002897

Last edited: 2017-02-25 18:55

UAX/WAV MODTYPEs have been killed, so updating to the latest revision should make it easy to write a new loader. :)

Slightly off-topic, I have added a new document to the repository, doc/module_formats.md, which contains general hints on writing module loaders (you may recognize some of those from the discussions on your STP loader). I have been dealing with this code for a very long time so things that seem obvious to me might be missing. If you have any specific hints that should be added to this document (e.g. stuff that was not obvious to you when starting to write those loaders), let me know!

Revenant

Revenant

2017-02-24 22:59

reporter   ~0002898

The list looks pretty good to me so far. One more thing that might be worth mentioning is how FileReader can be used to treat a portion of another file or a region of memory as its own independent file, which has been useful in loading samples and other data from Symphonie modules (in addition to raw samples they can also contain WAV, IFF, etc. files, and they also sometimes have to be decompressed into memory before being passed to the normal sample loading functions).

There's probably more I'm forgetting from my previous two loaders, but that was recently on my mind anyway.

Saga Musix

Saga Musix

2019-01-28 18:21

administrator   ~0003836

@Revenant are there any updates on your porting progress? There is some interest in implementing support for this format in HippoPlayer and that player already uses libopenmpt as one of its playback sources, so I guess it would be nice to have this format playable via libopenmpt.

Revenant

Revenant

2019-01-28 19:33

reporter   ~0003837

I'd made a fair amount of progress some time ago, though I guess I'm probably a good year and a half or so behind on upstream commits at this point, so I'll have to get up to date and make sure everything still works okay.

I can't promise 100% support for all modules, since there are simply some features of the tracker that would be a bigger effort to implement correctly (namely the stuff involving 'dynamic instruments' or whatever they're called, which allow stuff like synthesizing multiple other samples/notes into new instruments at load time), but I seem to remember that most of the modules I tried (from modland) were playing back fairly well (in many cases even noticeably more accurately than the official Java-based player mentioned earlier), so I think it's pretty close to usable aside from missing a few less commonly used features.

Anyway, I'll try to get up to date with the current upstream sometime soon and do some more testing with the modules I have available and see how much I think still needs to be done on this.

Saga Musix

Saga Musix

2019-10-27 20:47

administrator   ~0004125

How's the loader going? :) If you think that the only thing that's required is making it work with upstream (there have probably been a few changes here and there that will require some minor changes), I can help with that.

Revenant

Revenant

2020-06-11 22:12

reporter   ~0004380

Ouch, where does the time go...

I'll see what I can do sometime within the next week or so.

Saga Musix

Saga Musix

2020-12-26 11:41

administrator   ~0004568

Hi Revenant, are there any updates on the loader?

Revenant

Revenant

2020-12-26 18:21

reporter   ~0004569

I didn't really have time to try bringing my changes up to date after all, so once I get home from Christmas I will probably just push my existing changes to GitHub or something along with some notes. Hopefully after that it should at least be not too much more work to get things going again later.

Revenant

Revenant

2021-01-03 02:15

reporter   ~0004603

https://github.com/devinacker/openmpt/commit/ea89d3090ecc2509b7505f41575b2f40a74bca0b

This should be everything I had been working on back in 2017. It's probably still a little messy and experimental but hopefully still workable. I can try doing more with it whenever I get time.

Saga Musix

Saga Musix

2021-01-03 12:51

administrator   ~0004605

Thank you! I will have a look ASAP.

Saga Musix

Saga Musix

2021-01-03 17:07

administrator   ~0004607

FWIW, here's just the loader, brought up to current standards so it compiles and works fine:

Load_symmod.7z (10,654 bytes)
Saga Musix

Saga Musix

2021-01-26 17:56

administrator   ~0004617

Some general updates including bugfixes. I also started looking at the echo/delay DSP code in the Java player, which we may also be able to handle.

Load_symmod-2.7z (11,558 bytes)
Saga Musix

Saga Musix

2021-01-26 18:00

administrator   ~0004618

I should add that there are so many loose ends though that it's unlikely that this will make it into OpenMPT 1.30 (which will already feature two new module formats). I'll rather spend the time to make it sound as good as possible (including DSP) and have it in 1.31.

Revenant

Revenant

2021-01-26 21:02

reporter   ~0004619

Thanks, that sounds reasonable to me. I'll check out your changes sometime soon.

I think the only other thing of note I had been doing here (aside from the loader itself) was adding support for 16-bit and/or stereo IFF samples, which some of these modules do use, though I don't remember which ones anymore.

Saga Musix

Saga Musix

2021-01-26 21:17

administrator   ~0004620

Yep, I already put the 16-bit stuff to be committed separately (would be useful even without SymMOD support), I will commit that soon.

Saga Musix

Saga Musix

2021-01-31 21:43

administrator   ~0004624

Last edited: 2021-01-31 22:44

This DSP stuff is rather confusing... I will summarize what I have found but if this contradicts anything you might have found yourself, please let me know!

  • The Java DSP doesn't really map to the tracker echo / delay settings at all. It has echo, cross-echo, delay and cross-delay while the echo in the tracker alone has normal, cross, cross2 and center settings. I tried starting from the Java code but I quickly dropped it.
  • cross and cross2 seem to the same thing except that the feedback parameters are 100,50,25,... and 50,75,87.5,... respectively.
  • While you can manually enable and modify effects from the "System Control" window, those settings are not stored in the file at all.
  • The Echo DSP is straight-forward, the Delay DSP seems... strange? It generally shows the same parameters but when it works at all, it does so in a weird way. From my observation, it only clears and updates the delay line if it's set up in the "System Control" window. I will just treat it the same as the Echo DSP for now.
Saga Musix

Saga Musix

2021-02-19 21:28

administrator   ~0004644

For the one SymMOD out there that uses Transwaves... they are now implemented. Including many more changes as usual.

Load_symmod-3.7z (13,120 bytes)
Saga Musix

Saga Musix

2021-02-28 17:55

administrator   ~0004670

Last edited: 2021-03-01 16:34

Lots of progress since the last update... I think all loadtime sample manipulation (fading, overamplification, filters) is now implemented more or less correctly. Virtual instruments are mixed at loadtime. Tons of other findings from studying the original assembly code in more detail.

Load_symmod-4.7z (18,393 bytes)
Revenant

Revenant

2021-02-28 18:43

reporter   ~0004671

Nice, thanks again for all the updates! I'll take a look at the latest loader soon.

Saga Musix

Saga Musix

2021-03-14 16:47

administrator   ~0004677

Current state with a few more bugfixes. I'm inclined to add this to OpenMPT 1.30 after all - most SymMODs seem to be played with sufficient accuracy (especially considering that there isn't really any alternative out there apart from the tracker itself), and I'm currently specifically fuzzing the SymMOD loader to make sure it's safe.

Load_symmod-5.7z (18,623 bytes)
Saga Musix

Saga Musix

2021-03-20 14:15

administrator   ~0004682

This loader is now finally in trunk as of r14381! Many thanks again to Revenant for your fantastic work on getting this started. It's probably not perfect yet, so anything that doesn't sound quite right yet should be reported in a new issue.

Issue History

Date Modified Username Field Change
2015-04-11 22:20 Amaroq_Dricaldari New Issue
2015-04-11 22:21 Amaroq_Dricaldari Additional Information Updated
2015-04-11 22:25 Saga Musix Note Added: 0002031
2015-04-11 22:30 Saga Musix Priority low => none
2015-04-11 22:30 Saga Musix Reproducibility unable to reproduce => N/A
2015-04-11 22:36 Amaroq_Dricaldari Note Added: 0002032
2015-04-11 22:42 Saga Musix Note Added: 0002033
2017-02-20 21:49 Revenant Note Added: 0002886
2017-02-23 02:43 Revenant Note Added: 0002888
2017-02-23 10:40 Saga Musix Note Added: 0002889
2017-02-23 21:15 manx Note Added: 0002890
2017-02-24 01:10 Revenant Note Added: 0002893
2017-02-24 09:15 manx Note Added: 0002894
2017-02-24 11:02 Saga Musix Note Added: 0002895
2017-02-24 11:51 manx Note Edited: 0002894
2017-02-24 12:45 Saga Musix Note Edited: 0002895
2017-02-24 12:46 Saga Musix Note Edited: 0002895
2017-02-24 18:42 Revenant Note Added: 0002896
2017-02-24 20:53 Saga Musix Note Added: 0002897
2017-02-24 22:59 Revenant Note Added: 0002898
2017-02-25 18:55 Saga Musix Note Edited: 0002897
2019-01-28 18:21 Saga Musix Note Added: 0003836
2019-01-28 19:33 Revenant Note Added: 0003837
2019-10-27 20:47 Saga Musix Note Added: 0004125
2020-06-11 22:12 Revenant Note Added: 0004380
2020-12-26 11:41 Saga Musix Note Added: 0004568
2020-12-26 18:21 Revenant Note Added: 0004569
2021-01-03 02:15 Revenant Note Added: 0004603
2021-01-03 08:10 manx Target Version => OpenMPT 1.?? (long term goals)
2021-01-03 12:51 Saga Musix Note Added: 0004605
2021-01-03 16:33 Saga Musix Assigned To => Saga Musix
2021-01-03 16:33 Saga Musix Status new => assigned
2021-01-03 17:07 Saga Musix Note Added: 0004607
2021-01-03 17:07 Saga Musix File Added: Load_symmod.7z
2021-01-26 17:56 Saga Musix Note Added: 0004617
2021-01-26 17:56 Saga Musix File Added: Load_symmod-2.7z
2021-01-26 18:00 Saga Musix Note Added: 0004618
2021-01-26 21:02 Revenant Note Added: 0004619
2021-01-26 21:17 Saga Musix Note Added: 0004620
2021-01-30 10:40 manx Target Version OpenMPT 1.?? (long term goals) => OpenMPT 1.31.01.00 / libopenmpt 0.7.0 (upgrade first)
2021-01-31 21:43 Saga Musix Note Added: 0004624
2021-01-31 21:45 Saga Musix Note Edited: 0004624
2021-01-31 22:44 Saga Musix Note Edited: 0004624
2021-02-19 21:28 Saga Musix Note Added: 0004644
2021-02-19 21:28 Saga Musix File Added: Load_symmod-3.7z
2021-02-28 17:55 Saga Musix Note Added: 0004670
2021-02-28 17:55 Saga Musix File Added: Load_symmod-4.7z
2021-02-28 18:43 Revenant Note Added: 0004671
2021-03-01 16:34 Saga Musix Note Edited: 0004670
2021-03-14 16:47 Saga Musix Note Added: 0004677
2021-03-14 16:47 Saga Musix File Added: Load_symmod-5.7z
2021-03-14 16:47 Saga Musix Target Version OpenMPT 1.31.01.00 / libopenmpt 0.7.0 (upgrade first) => OpenMPT 1.30.01.00 / libopenmpt 0.6.0 (upgrade first)
2021-03-20 14:15 Saga Musix Note Added: 0004682
2021-03-20 14:16 Saga Musix Status assigned => resolved
2021-03-20 14:16 Saga Musix Resolution open => fixed
2021-03-20 14:16 Saga Musix Fixed in Version => OpenMPT 1.30.01.00 / libopenmpt 0.6.0 (upgrade first)