|
|
If the song message is empty, we fall back to using the instrument names, if in turn they are completely empty, we fall back to using sample names. Although this heuristics work for most modules, it obviously cannot work for all modules, and I do not see an easy way to solve this.
It might be the case that the module in question only contains whitespace characters in the instrument names, and we could probably include that aspect in the heuristics.
Can you name/provide specific modules where the current heuristic fails? |
|
|
|
Yes, (-DF-RND.IT. https://www.dropbox.com/s/gyn60pu6ej569jj/%28-DF-RND.IT?dl=1 |
|
|
|
It is more common in IT files to use sample texts rather than instrument texts for a song message, but of course this is not true for all modules. I guess we have to expose both texts if no comments are found in order to support both possible scenarios. The foobar plugin should probably also somehow expose the sample/instrument texts separately if it doesn't do so already, just like xmp-openmpt. |
|
|
|
Update. Here is a more permanent link to the test case I provided in earlier comments. ftp://ftp.modland.com/pub/modules/Impulsetracker/Corpse/coop-Dark%20Freddy/rainy%20day.it |
|
|
|
Slightly related to this, a lot of Protracker IFF files are affected by this as well, though the <message tag there is just exposing several space characters rather than the sample text, example: ftp://ftp.modland.com/pub/modules/Protracker%20IFF/DJ%20Pie/heaven.ptm |
|
|
|
We could probably trim the message (and end up with an empty message in that case). I'm not sure if that's always a good idea though, e.g. when the message contains some ascii art and it's important that it appears in a rectangular shape (e.g. if the user renders text with a fixed background color). Either way it's a different issue that would not be solved by the proposed fix for the original issue at hand. |
|
|
|
Regarding the original issue, I think it probably makes sense to, depending on module format, try either instrument or sample names first in case the message is empty or non-existent. For IT it is probably more common to use sample names, for XM, it is always instrument names. I am also not opposed to, again depending on module format, just include both instrument and sample names in case of no real message (if that makes sense for some formats).
It might even make sense to just always include everything in the "message" metadata (with ordering depending on the format).
Regarding trimming anything, I am strongly opposed. If the names contain any spaces we should keep them as much as possible. The goal here is not to provide a somehow "optimized" look of what is stored in the module, but instead provide an as identical as possible representation of what is stored in the file. |
|
|
|
Or we do all of that and provide "message", "message_heuristics", "message_all", "message_raw", "message_trimmed" ;)
That's probably not very useful. However, no matter what single variation we choose, I think there will always be some files where the chosen solution will not be optimal. |
|
|
|
I don't think it makes much sense to attach sample/instrument texts to the message metadata if there is already a message text (weird messages like the one in heaven.ptm notwithstanding). Generally I think if a player wants to list both the message and sample/instr texts I think that the player developer should decide the best way to represent them (e.g. with separators between different sections). To be clear, this means of course that foo_openmpt should handle this situation better, and I think that libopenmpt already does the right thing in this case.
But if there is no message, it makes sense to just include both instrument and sample texts for most formats. Maybe not so much for XM and clones where samples belong to instruments, but for IT-like formats where sample and instrument lists are completely separate, it makes sense to list both. |
|
|
|
Should this be closed since foo_openmpt is discontinued as of 0.5? |
|
|
|
No, this is still relevant for libopenmpt in general. |
|
|
|
message-heuristic-v1.patch (9,716 bytes)
Index: libopenmpt/libopenmpt_impl.cpp
===================================================================
--- libopenmpt/libopenmpt_impl.cpp (revision 12440)
+++ libopenmpt/libopenmpt_impl.cpp (working copy)
@@ -1145,6 +1145,40 @@
"warnings",
};
}
+std::string module_impl::get_message_instruments() const {
+ std::string retval;
+ std::string tmp;
+ bool valid = false;
+ for ( INSTRUMENTINDEX i = 1; i <= m_sndFile->GetNumInstruments(); ++i ) {
+ std::string instname = m_sndFile->GetInstrumentName( i );
+ if ( !instname.empty() ) {
+ valid = true;
+ }
+ tmp += instname;
+ tmp += "\n";
+ }
+ if ( valid ) {
+ retval = tmp;
+ }
+ return retval;
+}
+std::string module_impl::get_message_samples() const {
+ std::string retval;
+ std::string tmp;
+ bool valid = false;
+ for ( SAMPLEINDEX i = 1; i <= m_sndFile->GetNumSamples(); ++i ) {
+ std::string samplename = m_sndFile->GetSampleName( i );
+ if ( !samplename.empty() ) {
+ valid = true;
+ }
+ tmp += samplename;
+ tmp += "\n";
+ }
+ if ( valid ) {
+ retval = tmp;
+ }
+ return retval;
+}
std::string module_impl::get_metadata( const std::string & key ) const {
if ( key == std::string("type") ) {
return mpt::ToCharset(mpt::Charset::UTF8, m_sndFile->m_modFormat.type );
@@ -1172,35 +1206,55 @@
} else if ( key == std::string("message") ) {
std::string retval = m_sndFile->m_songMessage.GetFormatted( SongMessage::leLF );
if ( retval.empty() ) {
- std::string tmp;
- bool valid = false;
- for ( INSTRUMENTINDEX i = 1; i <= m_sndFile->GetNumInstruments(); ++i ) {
- std::string instname = m_sndFile->GetInstrumentName( i );
- if ( !instname.empty() ) {
- valid = true;
- }
- tmp += instname;
- tmp += "\n";
+ switch ( m_sndFile->m_modFormat.messageHeuristic ) {
+ case ModMessageHeuristicOrder::Instruments:
+ retval = get_message_instruments();
+ break;
+ case ModMessageHeuristicOrder::Samples:
+ retval = get_message_samples();
+ break;
+ case ModMessageHeuristicOrder::InstrumentsSamples:
+ if ( retval.empty() ) {
+ retval = get_message_instruments();
+ }
+ if ( retval.empty() ) {
+ retval = get_message_samples();
+ }
+ break;
+ case ModMessageHeuristicOrder::SamplesInstruments:
+ if ( retval.empty() ) {
+ retval = get_message_samples();
+ }
+ if ( retval.empty() ) {
+ retval = get_message_instruments();
+ }
+ break;
+ case ModMessageHeuristicOrder::BothInstrumentsSamples:
+ {
+ std::string message_instruments = get_message_instruments();
+ std::string message_samples = get_message_samples();
+ if ( !message_instruments.empty() ) {
+ retval += std::move( message_instruments );
+ }
+ if ( !message_samples.empty() ) {
+ retval += std::move( message_samples );
+ }
+ }
+ break;
+ case ModMessageHeuristicOrder::BothSamplesInstruments:
+ {
+ std::string message_instruments = get_message_instruments();
+ std::string message_samples = get_message_samples();
+ if ( !message_samples.empty() ) {
+ retval += std::move( message_samples );
+ }
+ if ( !message_instruments.empty() ) {
+ retval += std::move( message_instruments );
+ }
+ }
+ break;
}
- if ( valid ) {
- retval = tmp;
- }
}
- if ( retval.empty() ) {
- std::string tmp;
- bool valid = false;
- for ( SAMPLEINDEX i = 1; i <= m_sndFile->GetNumSamples(); ++i ) {
- std::string samplename = m_sndFile->GetSampleName( i );
- if ( !samplename.empty() ) {
- valid = true;
- }
- tmp += samplename;
- tmp += "\n";
- }
- if ( valid ) {
- retval = tmp;
- }
- }
return mod_string_to_utf8( retval );
} else if ( key == std::string("message_raw") ) {
std::string retval = m_sndFile->m_songMessage.GetFormatted( SongMessage::leLF );
Index: libopenmpt/libopenmpt_impl.hpp
===================================================================
--- libopenmpt/libopenmpt_impl.hpp (revision 12440)
+++ libopenmpt/libopenmpt_impl.hpp (working copy)
@@ -146,6 +146,8 @@
std::size_t read_wrapper( std::size_t count, float * left, float * right, float * rear_left, float * rear_right );
std::size_t read_interleaved_wrapper( std::size_t count, std::size_t channels, std::int16_t * interleaved );
std::size_t read_interleaved_wrapper( std::size_t count, std::size_t channels, float * interleaved );
+ std::string get_message_instruments() const;
+ std::string get_message_samples() const;
std::pair< std::string, std::string > format_and_highlight_pattern_row_channel_command( std::int32_t p, std::int32_t r, std::int32_t c, int command ) const;
std::pair< std::string, std::string > format_and_highlight_pattern_row_channel( std::int32_t p, std::int32_t r, std::int32_t c, std::size_t width, bool pad ) const;
static double could_open_probability( const OpenMPT::FileReader & file, double effort, std::unique_ptr<log_interface> log );
Index: soundlib/Load_it.cpp
===================================================================
--- soundlib/Load_it.cpp (revision 12440)
+++ soundlib/Load_it.cpp (working copy)
@@ -1257,6 +1257,7 @@
m_modFormat.type = (GetType() == MOD_TYPE_MPT) ? U_("mptm") : U_("it");
m_modFormat.madeWithTracker = std::move(madeWithTracker);
m_modFormat.charset = m_dwLastSavedWithVersion ? mpt::Charset::Windows1252 : mpt::Charset::CP437;
+ m_modFormat.messageHeuristic = ModMessageHeuristicOrder::Samples;
return true;
}
Index: soundlib/Load_itp.cpp
===================================================================
--- soundlib/Load_itp.cpp (revision 12440)
+++ soundlib/Load_itp.cpp (working copy)
@@ -399,6 +399,7 @@
m_modFormat.type = U_("itp");
m_modFormat.madeWithTracker = U_("OpenMPT ") + mpt::ufmt::val(m_dwLastSavedWithVersion);
m_modFormat.charset = mpt::Charset::Windows1252;
+ m_modFormat.messageHeuristic = ModMessageHeuristicOrder::Samples;
return true;
#endif // MPT_EXTERNAL_SAMPLES
Index: soundlib/Load_mo3.cpp
===================================================================
--- soundlib/Load_mo3.cpp (revision 12440)
+++ soundlib/Load_mo3.cpp (working copy)
@@ -831,11 +831,13 @@
mpt::ustring originalFormatType;
mpt::ustring originalFormatName;
+ ModMessageHeuristicOrder messageHeuristic = ModMessageHeuristicOrder::Default;
if(fileHeader.flags & MO3FileHeader::isIT)
{
SetType(MOD_TYPE_IT);
originalFormatType = U_("it");
originalFormatName = U_("Impulse Tracker");
+ messageHeuristic = ModMessageHeuristicOrder::Samples;
} else if(fileHeader.flags & MO3FileHeader::isS3M)
{
SetType(MOD_TYPE_S3M);
@@ -856,6 +858,7 @@
SetType(MOD_TYPE_XM);
originalFormatType = U_("xm");
originalFormatName = U_("FastTracker 2");
+ messageHeuristic = ModMessageHeuristicOrder::InstrumentsSamples;
}
if(fileHeader.flags & MO3FileHeader::linearSlides)
@@ -1948,6 +1951,7 @@
m_modFormat.charset = mpt::Charset::ISO8859_1;
else
m_modFormat.charset = mpt::Charset::CP437;
+ m_modFormat.messageHeuristic = messageHeuristic;
if(unsupportedSamples)
{
Index: soundlib/Load_xm.cpp
===================================================================
--- soundlib/Load_xm.cpp (revision 12440)
+++ soundlib/Load_xm.cpp (working copy)
@@ -1032,6 +1032,7 @@
m_modFormat.originalType = U_("xm");
m_modFormat.madeWithTracker = std::move(madeWithTracker);
m_modFormat.charset = m_dwLastSavedWithVersion ? mpt::Charset::Windows1252 : mpt::Charset::CP437;
+ m_modFormat.messageHeuristic = ModMessageHeuristicOrder::InstrumentsSamples;
} else
{
m_modFormat.formatName = mpt::format(U_("FastTracker 2 v%1.%2"))(fileHeader.version >> 8, mpt::ufmt::hex0<2>(fileHeader.version & 0xFF));
@@ -1038,6 +1039,7 @@
m_modFormat.type = U_("xm");
m_modFormat.madeWithTracker = std::move(madeWithTracker);
m_modFormat.charset = m_dwLastSavedWithVersion ? mpt::Charset::Windows1252 : mpt::Charset::CP437;
+ m_modFormat.messageHeuristic = ModMessageHeuristicOrder::InstrumentsSamples;
}
return true;
Index: soundlib/Sndfile.cpp
===================================================================
--- soundlib/Sndfile.cpp (revision 12440)
+++ soundlib/Sndfile.cpp (working copy)
@@ -1576,6 +1576,21 @@
Patterns.OnModTypeChanged(oldType);
m_modFormat.type = mpt::ToUnicode(mpt::Charset::UTF8, GetModSpecifications().fileExtension);
+ switch(newType)
+ {
+ case MOD_TYPE_MPT:
+ m_modFormat.messageHeuristic = ModMessageHeuristicOrder::Samples;
+ break;
+ case MOD_TYPE_IT:
+ m_modFormat.messageHeuristic = ModMessageHeuristicOrder::Samples;
+ break;
+ case MOD_TYPE_XM:
+ m_modFormat.messageHeuristic = ModMessageHeuristicOrder::InstrumentsSamples;
+ break;
+ default:
+ m_modFormat.messageHeuristic = ModMessageHeuristicOrder::Default;
+ break;
+ }
}
#endif // MODPLUG_TRACKER
Index: soundlib/Sndfile.h
===================================================================
--- soundlib/Sndfile.h (revision 12440)
+++ soundlib/Sndfile.h (working copy)
@@ -247,6 +247,17 @@
};
+enum class ModMessageHeuristicOrder
+{
+ Instruments,
+ Samples,
+ InstrumentsSamples,
+ SamplesInstruments,
+ BothInstrumentsSamples,
+ BothSamplesInstruments,
+ Default = InstrumentsSamples,
+};
+
struct ModFormatDetails
{
mpt::ustring formatName; // "FastTracker 2"
@@ -255,6 +266,7 @@
mpt::ustring originalFormatName; // "FastTracker 2" in the case of converted formats like MO3 or GDM
mpt::ustring originalType; // "xm" in the case of converted formats like MO3 or GDM
mpt::Charset charset = mpt::Charset::UTF8;
+ ModMessageHeuristicOrder messageHeuristic = ModMessageHeuristicOrder::Default;
};
|
|
|
|
Other XM-like formats: MDL, IMF
To avoid duplicating code between loaders and Sndfile.cpp (mod conversion), I would rather have this information in a function in CSoundFile (using a switch statement similar to that in added in the mod conversion code). This information is inherently bound to how the file format is structured, so it doesn't make much sense to be able to differentiate it on a per-tracker basis like charsets etc.. This avoids the danger of having inconsistent values between loaders and after mod conversion (which is only applicable to OpenMPT), and all the information regarding this feature is centered in a single place. |
|
|
|
Patch is even shorter this way.
message-heuristic-v2.patch (6,679 bytes)
Index: libopenmpt/libopenmpt_impl.cpp
===================================================================
--- libopenmpt/libopenmpt_impl.cpp (revision 12440)
+++ libopenmpt/libopenmpt_impl.cpp (working copy)
@@ -1145,6 +1145,40 @@
"warnings",
};
}
+std::string module_impl::get_message_instruments() const {
+ std::string retval;
+ std::string tmp;
+ bool valid = false;
+ for ( INSTRUMENTINDEX i = 1; i <= m_sndFile->GetNumInstruments(); ++i ) {
+ std::string instname = m_sndFile->GetInstrumentName( i );
+ if ( !instname.empty() ) {
+ valid = true;
+ }
+ tmp += instname;
+ tmp += "\n";
+ }
+ if ( valid ) {
+ retval = tmp;
+ }
+ return retval;
+}
+std::string module_impl::get_message_samples() const {
+ std::string retval;
+ std::string tmp;
+ bool valid = false;
+ for ( SAMPLEINDEX i = 1; i <= m_sndFile->GetNumSamples(); ++i ) {
+ std::string samplename = m_sndFile->GetSampleName( i );
+ if ( !samplename.empty() ) {
+ valid = true;
+ }
+ tmp += samplename;
+ tmp += "\n";
+ }
+ if ( valid ) {
+ retval = tmp;
+ }
+ return retval;
+}
std::string module_impl::get_metadata( const std::string & key ) const {
if ( key == std::string("type") ) {
return mpt::ToCharset(mpt::Charset::UTF8, m_sndFile->m_modFormat.type );
@@ -1172,35 +1206,55 @@
} else if ( key == std::string("message") ) {
std::string retval = m_sndFile->m_songMessage.GetFormatted( SongMessage::leLF );
if ( retval.empty() ) {
- std::string tmp;
- bool valid = false;
- for ( INSTRUMENTINDEX i = 1; i <= m_sndFile->GetNumInstruments(); ++i ) {
- std::string instname = m_sndFile->GetInstrumentName( i );
- if ( !instname.empty() ) {
- valid = true;
- }
- tmp += instname;
- tmp += "\n";
+ switch ( m_sndFile->GetMessageHeuristic() ) {
+ case ModMessageHeuristicOrder::Instruments:
+ retval = get_message_instruments();
+ break;
+ case ModMessageHeuristicOrder::Samples:
+ retval = get_message_samples();
+ break;
+ case ModMessageHeuristicOrder::InstrumentsSamples:
+ if ( retval.empty() ) {
+ retval = get_message_instruments();
+ }
+ if ( retval.empty() ) {
+ retval = get_message_samples();
+ }
+ break;
+ case ModMessageHeuristicOrder::SamplesInstruments:
+ if ( retval.empty() ) {
+ retval = get_message_samples();
+ }
+ if ( retval.empty() ) {
+ retval = get_message_instruments();
+ }
+ break;
+ case ModMessageHeuristicOrder::BothInstrumentsSamples:
+ {
+ std::string message_instruments = get_message_instruments();
+ std::string message_samples = get_message_samples();
+ if ( !message_instruments.empty() ) {
+ retval += std::move( message_instruments );
+ }
+ if ( !message_samples.empty() ) {
+ retval += std::move( message_samples );
+ }
+ }
+ break;
+ case ModMessageHeuristicOrder::BothSamplesInstruments:
+ {
+ std::string message_instruments = get_message_instruments();
+ std::string message_samples = get_message_samples();
+ if ( !message_samples.empty() ) {
+ retval += std::move( message_samples );
+ }
+ if ( !message_instruments.empty() ) {
+ retval += std::move( message_instruments );
+ }
+ }
+ break;
}
- if ( valid ) {
- retval = tmp;
- }
}
- if ( retval.empty() ) {
- std::string tmp;
- bool valid = false;
- for ( SAMPLEINDEX i = 1; i <= m_sndFile->GetNumSamples(); ++i ) {
- std::string samplename = m_sndFile->GetSampleName( i );
- if ( !samplename.empty() ) {
- valid = true;
- }
- tmp += samplename;
- tmp += "\n";
- }
- if ( valid ) {
- retval = tmp;
- }
- }
return mod_string_to_utf8( retval );
} else if ( key == std::string("message_raw") ) {
std::string retval = m_sndFile->m_songMessage.GetFormatted( SongMessage::leLF );
Index: libopenmpt/libopenmpt_impl.hpp
===================================================================
--- libopenmpt/libopenmpt_impl.hpp (revision 12440)
+++ libopenmpt/libopenmpt_impl.hpp (working copy)
@@ -146,6 +146,8 @@
std::size_t read_wrapper( std::size_t count, float * left, float * right, float * rear_left, float * rear_right );
std::size_t read_interleaved_wrapper( std::size_t count, std::size_t channels, std::int16_t * interleaved );
std::size_t read_interleaved_wrapper( std::size_t count, std::size_t channels, float * interleaved );
+ std::string get_message_instruments() const;
+ std::string get_message_samples() const;
std::pair< std::string, std::string > format_and_highlight_pattern_row_channel_command( std::int32_t p, std::int32_t r, std::int32_t c, int command ) const;
std::pair< std::string, std::string > format_and_highlight_pattern_row_channel( std::int32_t p, std::int32_t r, std::int32_t c, std::size_t width, bool pad ) const;
static double could_open_probability( const OpenMPT::FileReader & file, double effort, std::unique_ptr<log_interface> log );
Index: soundlib/Sndfile.cpp
===================================================================
--- soundlib/Sndfile.cpp (revision 12440)
+++ soundlib/Sndfile.cpp (working copy)
@@ -1581,6 +1581,34 @@
#endif // MODPLUG_TRACKER
+ModMessageHeuristicOrder CSoundFile::GetMessageHeuristic() const
+{
+ ModMessageHeuristicOrder result = ModMessageHeuristicOrder::Default;
+ switch(GetType())
+ {
+ case MOD_TYPE_MPT:
+ result = ModMessageHeuristicOrder::Samples;
+ break;
+ case MOD_TYPE_IT:
+ result = ModMessageHeuristicOrder::Samples;
+ break;
+ case MOD_TYPE_XM:
+ result = ModMessageHeuristicOrder::InstrumentsSamples;
+ break;
+ case MOD_TYPE_MDL:
+ result = ModMessageHeuristicOrder::InstrumentsSamples;
+ break;
+ case MOD_TYPE_IMF:
+ result = ModMessageHeuristicOrder::InstrumentsSamples;
+ break;
+ default:
+ result = ModMessageHeuristicOrder::Default;
+ break;
+ }
+ return result;
+}
+
+
bool CSoundFile::SetTitle(const std::string &newTitle)
{
if(m_songName != newTitle)
Index: soundlib/Sndfile.h
===================================================================
--- soundlib/Sndfile.h (revision 12440)
+++ soundlib/Sndfile.h (working copy)
@@ -247,6 +247,17 @@
};
+enum class ModMessageHeuristicOrder
+{
+ Instruments,
+ Samples,
+ InstrumentsSamples,
+ SamplesInstruments,
+ BothInstrumentsSamples,
+ BothSamplesInstruments,
+ Default = InstrumentsSamples,
+};
+
struct ModFormatDetails
{
mpt::ustring formatName; // "FastTracker 2"
@@ -674,6 +685,8 @@
#endif // MODPLUG_TRACKER
}
+ ModMessageHeuristicOrder GetMessageHeuristic() const;
+
void SetPreAmp(uint32 vol);
uint32 GetPreAmp() const { return m_MixerSettings.m_nPreAmp; }
|
|
|
|
r12446 |
|
