View Issue Details

IDProjectCategoryView StatusLast Update
0001507OpenMPTPlayback Compatibilitypublic2024-05-03 22:07
ReporterSaga Musix Assigned ToSaga Musix  
PrioritynormalSeverityfeatureReproducibilityN/A
Status assignedResolutionopen 
Target VersionOpenMPT 1.32 / libopenmpt 0.8 (goals) 
Summary0001507: Automatic module playback regression suite
Description

Similar to what libxmp has, it would be great to be able to automatically verify that the test cases from https://resources.openmpt.org/player_tests/ don't break when working on the player engine.

Some libraries do this by hashing the rendered song output and storing just that hash, but I think this approach is not really suitable for OpenMPT with its multitude of output options (resampling, dither, ...) and lack of guarantee of any sort of bit-identical rendering. What could make more sense (and is more similar to what libxmp is doing, and could actually be a lot more helpful with debugging regressions) is capturing the state of the core mixer variables for each channel (the stuff at the very beginning of the ModChannel struct, i.e. currently played sample, position, increment, L/R volume, filter coefficients) once per tick to a file and then comparing this when rendering the modules in the test suite.

For this to work, the CSoundFile PRNG seed would have to be forced to be deterministic when recording and verifying the tests. Another thing to figure out would be how to handle NNA channels because the order in which NNA channel indices are allocated is an implementation detail that should be allowed to change without having to recreate all test outputs. Every NNA channel could be identified by a pair of (source channel, point in time when it was moved) as this combination would be unique (a channel can only be moved once to a background channel in each tick).
For OPL channels, it would also be great to be able to capture the individual channel state in the wrapper object.

As running the test suite would be potentially quite time-consuming, it shouldn't be part of the tests that are run when launching an OpenMPT debug build.

TagsNo tags attached.
Has the bug occurred in previous versions?
Tested code revision (in case you know it)

Activities

Saga Musix

Saga Musix

2024-05-03 22:07

administrator   ~0005945

Step 1 is done in r20701: A class for capturing the playback state and being able to compare two dumps or convert them to TSV format is present, and can be invoked from OpenMPT DEBUG builds. Automation of this process is for a later step.

Issue History

Date Modified Username Field Change
2021-09-24 10:35 Saga Musix New Issue
2021-09-24 13:29 Saga Musix Description Updated
2021-11-07 14:44 Saga Musix Assigned To => Saga Musix
2021-11-07 14:44 Saga Musix Status new => assigned
2021-11-14 01:32 Saga Musix Target Version => OpenMPT 1.30.01.00 / libopenmpt 0.6.0 (upgrade first)
2021-12-23 15:04 Saga Musix Target Version OpenMPT 1.30.01.00 / libopenmpt 0.6.0 (upgrade first) => OpenMPT 1.31.01.00 / libopenmpt 0.7.0 (upgrade first)
2023-04-29 11:11 Saga Musix Target Version OpenMPT 1.31.01.00 / libopenmpt 0.7.0 (upgrade first) => OpenMPT 1.32 / libopenmpt 0.8 (goals)
2024-05-03 22:07 Saga Musix Note Added: 0005945