View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0001559 | OpenMPT | General | public | 2022-01-30 17:48 | 2025-12-12 19:57 |
| Reporter | Saga Musix | Assigned To | manx | ||
| Priority | normal | Severity | minor | Reproducibility | N/A |
| Status | assigned | Resolution | open | ||
| Target Version | OpenMPT 1.33 / libopenmpt 0.9 (goals) | ||||
| Summary | 0001559: Don't use WinAPI functions for INI reading/writing | ||||
| Description | Currently OpenMPT uses
As a consequence of the first point, it's possible that one OpenMPT instance takes a long time to shut down, and a newly launched instance may not see the settings as saved by the previous instance. One particularly worrying example is the following scenario:
A custom INI implementation should avoid this scenario. While the file is being written, other OpenMPT instances should wait until the file is no longer locked, rather than reading incomplete settings. | ||||
| Tags | No tags attached. | ||||
| Has the bug occurred in previous versions? | |||||
| Tested code revision (in case you know it) | |||||
I do not think this would be necessary. Atomically writing the file is the easier approach, because we do not (and do not want to, and cannot) guarantee settings consistency across different instances running concurrently anyway. Writing the file in 1 go should also be plenty fast enough to avoid the mentioned race condition. If the second instance is started while the first one is still actively running, there is not much we can do anyway, except for maybe flushing the configuration file after the welcome dialog (which would probably be a good idea even with the old implementation). There are a couple of open questions regarding our own implementation:
The INI implementation that I have in an older code base answers these questions as:
In case the answer to 3 is "no", we should use a different file name in order to not break downgrades completely. "OpenMPT.ini" would make sense here. |
|
|
While strict backwards compatibility would be nice to have, I think it should not delay this effort. I think under normal circumstances 3 should not matter at all, only when a user hand-edited the file. As we lost the ability to automatically back up the current OpenMPT configuration during upgrades, using a new settings name file would probably the safest choice. |
|
This mainly happened by accident. Using GetPrivateProfileSection would be required anyway for converting from the existing Win32 INI functions to a custom implementation, because we cannot rely on all settings being read eagerly into out internal data structures (i.e. currently not selected Sound Devices). Adding the writing was only a small extra step. There is another potential problem that I noticed while working on this: We are passing the INI file name to MFC via speedup-ini-settings-v1.patch (7,831 bytes)
Index: mptrack/Settings.cpp
===================================================================
--- mptrack/Settings.cpp (revision 24333)
+++ mptrack/Settings.cpp (working copy)
@@ -140,6 +140,16 @@
backend->RemoveSection(section);
}
+bool SettingsContainer::BackendsCanWriteMultipleSettings() const
+{
+ return backend->CanWriteMultipleSettings();
+}
+
+void SettingsContainer::BackendsWriteMultipleSettings(const std::map<SettingPath, SettingValue> &settings)
+{
+ backend->WriteMultipleSettings(settings);
+}
+
SettingValue SettingsContainer::ReadSetting(const SettingPath &path, const SettingValue &def) const
{
ASSERT(theApp.InGuiThread());
@@ -242,6 +252,20 @@
{
ASSERT(theApp.InGuiThread());
ASSERT(!CMainFrame::GetMainFrame() || (CMainFrame::GetMainFrame() && !CMainFrame::GetMainFrame()->InNotifyHandler())); // This is a slow path, use CachedSetting for stuff that is accessed in notify handler.
+ if(BackendsCanWriteMultipleSettings())
+ {
+ std::map<SettingPath, SettingValue> settings;
+ for(auto &[path, value] : map)
+ {
+ if(value.IsDirty())
+ {
+ settings.insert(std::make_pair(path, value.GetRefValue()));
+ value.Clean();
+ }
+ }
+ BackendsWriteMultipleSettings(settings);
+ return;
+ }
for(auto &[path, value] : map)
{
if(value.IsDirty())
@@ -407,7 +431,12 @@
IniFileSettingsBackend::IniFileSettingsBackend(const mpt::PathString &filename)
: filename(filename)
{
- return;
+#if defined(UNICODE)
+ if(mpt::osinfo::windows::Version::Current().IsAtLeast(mpt::osinfo::windows::Version::WinXP))
+ {
+ ConvertToUnicode();
+ }
+#endif
}
IniFileSettingsBackend::~IniFileSettingsBackend()
@@ -493,10 +522,143 @@
RemoveSectionRaw(section);
}
+std::set<mpt::winstring> IniFileSettingsBackend::ReadSections() const
+{
+ std::set<mpt::winstring> result;
+ const std::vector<TCHAR> sectionsstr = [&]()
+ {
+ std::vector<TCHAR> buf;
+ buf.resize(1024);
+ while(true)
+ {
+ DWORD bufused = ::GetPrivateProfileSectionNames(buf.data(), mpt::saturate_cast<DWORD>(buf.size()), filename.AsNative().c_str());
+ if(bufused >= (buf.size() - 2))
+ {
+ buf.resize(mpt::exponential_grow(buf.size()));
+ continue;
+ }
+ buf.resize(bufused);
+ break;
+ };
+ return buf;
+ }();
+ const std::vector<mpt::winstring> sections = mpt::split(mpt::winstring(sectionsstr.data(), sectionsstr.size()), mpt::winstring(_T("\0"), 1));
+ for(const auto §ion : sections)
+ {
+ result.insert(section);
+ }
+ return result;
+}
+std::map<mpt::winstring, mpt::winstring> IniFileSettingsBackend::ReadSection(const mpt::winstring §ion) const
+{
+ std::map<mpt::winstring, mpt::winstring> result;
+ const std::vector<TCHAR> keyvalsstr = [&]()
+ {
+ std::vector<TCHAR> buf;
+ buf.resize(1024);
+ while(true)
+ {
+ DWORD bufused = ::GetPrivateProfileSection(mpt::ToWin(section).c_str(), buf.data(), mpt::saturate_cast<DWORD>(buf.size()), filename.AsNative().c_str());
+ if(bufused >= (buf.size() - 2))
+ {
+ buf.resize(mpt::exponential_grow(buf.size()));
+ continue;
+ }
+ buf.resize(bufused);
+ break;
+ };
+ return buf;
+ }();
+ const std::vector<mpt::winstring> keyvals = mpt::split(mpt::winstring(keyvalsstr.data(), keyvalsstr.size()), mpt::winstring(_T("\0"), 1));
+ for(const auto &keyval : keyvals)
+ {
+ const auto equalpos = keyval.find(_T("="));
+ if(equalpos == mpt::winstring::npos)
+ {
+ continue;
+ }
+ if(equalpos == 0)
+ {
+ continue;
+ }
+ result.insert(std::make_pair(keyval.substr(0, equalpos), keyval.substr(equalpos + 1)));
+ }
+ return result;
+}
+mpt::winstring IniFileSettingsBackend::FormatValueAsIni(const SettingValue &value)
+{
+ switch(value.GetType())
+ {
+ case SettingTypeBool:
+ return mpt::tfmt::val(value.as<bool>());
+ break;
+ case SettingTypeInt:
+ return mpt::tfmt::val(value.as<int32>());
+ break;
+ case SettingTypeFloat:
+ return mpt::tfmt::val(value.as<double>());
+ break;
+ case SettingTypeString:
+ return mpt::ToWin(value.as<mpt::ustring>());
+ break;
+ case SettingTypeBinary:
+ {
+ std::vector<std::byte> data = value.as<std::vector<std::byte>>();
+ uint8 checksum = 0;
+ for(const std::byte x : data) {
+ checksum += mpt::byte_cast<uint8>(x);
+ }
+ return mpt::ToWin(mpt::encode_hex(mpt::as_span(data)) + mpt::ufmt::HEX0<2>(checksum));
+ }
+ break;
+ case SettingTypeNone:
+ default:
+ return mpt::ustring();
+ break;
+ }
+}
+void IniFileSettingsBackend::WriteSection(const mpt::winstring §ion, const std::map<mpt::winstring, mpt::winstring> keyvalues)
+{
+ mpt::winstring keyvals;
+ for(const auto &[key, val] : keyvalues)
+ {
+ keyvals.append(key);
+ keyvals.append(_T("="));
+ keyvals.append(val);
+ keyvals.append(mpt::winstring(_T("\0"), 1));
+ }
+ keyvals.append(mpt::winstring(_T("\0"), 1));
+ ::WritePrivateProfileSection(section.c_str(), keyvals.c_str(), filename.AsNative().c_str());
+}
+void IniFileSettingsBackend::WriteMultipleSettings(const std::map<SettingPath, SettingValue> &settings)
+{
+ std::map<mpt::ustring, std::map<SettingPath, SettingValue>> sectionssettings;
+ for(const auto &[path, value] : settings)
+ {
+ sectionssettings[path.GetRefSection()][path] = value;
+ }
+ for(const auto &[section, sectionsettings] : sectionssettings)
+ {
+ std::map<mpt::winstring, mpt::winstring> workingsectionsettings = ReadSection(mpt::ToWin(section));
+ for(const auto &[path, value] : sectionsettings)
+ {
+ workingsectionsettings[mpt::ToWin(path.GetRefKey())] = FormatValueAsIni(value);
+ }
+ WriteSection(mpt::ToWin(section), workingsectionsettings);
+ }
+}
+
+bool IniFileSettingsBackend::CanWriteMultipleSettings() const
+{
+ return true;
+}
+
+
+
IniFileSettingsContainer::IniFileSettingsContainer(const mpt::PathString &filename)
: IniFileSettingsBackend(filename)
, SettingsContainer(this)
Index: mptrack/Settings.h
===================================================================
--- mptrack/Settings.h (revision 24333)
+++ mptrack/Settings.h (working copy)
@@ -408,6 +408,8 @@
virtual void WriteSetting(const SettingPath &path, const SettingValue &val) = 0;
virtual void RemoveSetting(const SettingPath &path) = 0;
virtual void RemoveSection(const mpt::ustring §ion) = 0;
+ virtual bool CanWriteMultipleSettings() const = 0;
+ virtual void WriteMultipleSettings(const std::map<SettingPath, SettingValue> &settings) = 0;
protected:
virtual ~ISettingsBackend() = default;
};
@@ -448,6 +450,8 @@
void BackendsWriteSetting(const SettingPath &path, const SettingValue &val);
void BackendsRemoveSetting(const SettingPath &path);
void BackendsRemoveSection(const mpt::ustring §ion);
+ bool BackendsCanWriteMultipleSettings() const;
+ void BackendsWriteMultipleSettings(const std::map<SettingPath, SettingValue> &settings);
void NotifyListeners(const SettingPath &path);
SettingValue ReadSetting(const SettingPath &path, const SettingValue &def) const;
bool IsDefaultSetting(const SettingPath &path) const;
@@ -711,6 +715,14 @@
virtual void WriteSetting(const SettingPath &path, const SettingValue &val) override;
virtual void RemoveSetting(const SettingPath &path) override;
virtual void RemoveSection(const mpt::ustring §ion) override;
+private:
+ std::set<mpt::winstring> ReadSections() const;
+ std::map<mpt::winstring, mpt::winstring> ReadSection(const mpt::winstring §ion) const;
+ static mpt::winstring FormatValueAsIni(const SettingValue &value);
+ void WriteSection(const mpt::winstring §ion, const std::map<mpt::winstring, mpt::winstring> keyvalues);
+public:
+ virtual bool CanWriteMultipleSettings() const override;
+ virtual void WriteMultipleSettings(const std::map<SettingPath, SettingValue> &settings) override;
const mpt::PathString& GetFilename() const { return filename; }
};
|
|
This is for storing the position and visibility of toolbars. It would probably be possible to serialize them outselves by overwriting some functions, then MFC would not be involved at all anymore. |
|
|
speedup-ini-settings-v3-wip.patch (17,925 bytes)
Index: mptrack/Settings.cpp
===================================================================
--- mptrack/Settings.cpp (revision 24343)
+++ mptrack/Settings.cpp (working copy)
@@ -140,6 +140,16 @@
backend->RemoveSection(section);
}
+bool SettingsContainer::BackendsCanWriteMultipleSettings() const
+{
+ return backend->CanBatchedSettings();
+}
+
+void SettingsContainer::BackendsWriteMultipleSettings(const std::map<SettingPath, SettingValue> &settings)
+{
+ backend->WriteMultipleSettings(settings);
+}
+
SettingValue SettingsContainer::ReadSetting(const SettingPath &path, const SettingValue &def) const
{
ASSERT(theApp.InGuiThread());
@@ -242,6 +252,26 @@
{
ASSERT(theApp.InGuiThread());
ASSERT(!CMainFrame::GetMainFrame() || (CMainFrame::GetMainFrame() && !CMainFrame::GetMainFrame()->InNotifyHandler())); // This is a slow path, use CachedSetting for stuff that is accessed in notify handler.
+ if(BackendsCanWriteMultipleSettings())
+ {
+ std::map<SettingPath, SettingValue> settings;
+ for(auto &[path, value] : map)
+ {
+ if(value.IsDirty())
+ {
+ settings.insert(std::make_pair(path, value.GetRefValue()));
+ }
+ }
+ BackendsWriteMultipleSettings(settings);
+ for(auto &[path, value] : map)
+ {
+ if(value.IsDirty())
+ {
+ value.Clean();
+ }
+ }
+ return;
+ }
for(auto &[path, value] : map)
{
if(value.IsDirty())
@@ -405,11 +435,22 @@
IniFileSettingsBackend::IniFileSettingsBackend(const mpt::PathString &filename)
- : filename(filename)
+ : IniFileSettingsBackend(filename, CachePolicy::Cached)
{
return;
}
+IniFileSettingsBackend::IniFileSettingsBackend(const mpt::PathString &filename, CachePolicy cachePolicy)
+ : filename(filename)
+{
+ OPENMPT_PROFILE_FUNCTION(Profiler::Settings);
+ if(cachePolicy == CachePolicy::Cached)
+ {
+ ConvertToUnicode();
+ cache = ReadAllSectionsRaw();
+ }
+}
+
IniFileSettingsBackend::~IniFileSettingsBackend()
{
return;
@@ -469,14 +510,41 @@
SettingValue IniFileSettingsBackend::ReadSetting(const SettingPath &path, const SettingValue &def) const
{
OPENMPT_PROFILE_FUNCTION(Profiler::Settings);
- switch(def.GetType())
+ if(cache.has_value())
{
- case SettingTypeBool: return SettingValue(ReadSettingRaw(path, def.as<bool>()), def.GetTypeTag()); break;
- case SettingTypeInt: return SettingValue(ReadSettingRaw(path, def.as<int32>()), def.GetTypeTag()); break;
- case SettingTypeFloat: return SettingValue(ReadSettingRaw(path, def.as<double>()), def.GetTypeTag()); break;
- case SettingTypeString: return SettingValue(ReadSettingRaw(path, def.as<mpt::ustring>()), def.GetTypeTag()); break;
- case SettingTypeBinary: return SettingValue(ReadSettingRaw(path, def.as<std::vector<std::byte> >()), def.GetTypeTag()); break;
- default: return SettingValue(); break;
+ const auto sectionit = (*cache).find(mpt::ToWin(path.GetRefSection()));
+ if(sectionit != (*cache).end())
+ {
+ const std::map<mpt::winstring, std::optional<mpt::winstring>> §ion = sectionit->second;
+ const auto it = section.find(mpt::ToWin(path.GetRefKey()));
+ if(it != section.end())
+ {
+ if(it->second.has_value())
+ {
+ return ParseValueFromIni(it->second.value(), def);
+ } else
+ {
+ return def;
+ }
+ } else
+ {
+ return def;
+ }
+ } else
+ {
+ return def;
+ }
+ } else
+ {
+ switch(def.GetType())
+ {
+ case SettingTypeBool: return SettingValue(ReadSettingRaw(path, def.as<bool>()), def.GetTypeTag()); break;
+ case SettingTypeInt: return SettingValue(ReadSettingRaw(path, def.as<int32>()), def.GetTypeTag()); break;
+ case SettingTypeFloat: return SettingValue(ReadSettingRaw(path, def.as<double>()), def.GetTypeTag()); break;
+ case SettingTypeString: return SettingValue(ReadSettingRaw(path, def.as<mpt::ustring>()), def.GetTypeTag()); break;
+ case SettingTypeBinary: return SettingValue(ReadSettingRaw(path, def.as<std::vector<std::byte> >()), def.GetTypeTag()); break;
+ default: return SettingValue(); break;
+ }
}
}
@@ -493,12 +561,22 @@
case SettingTypeBinary: WriteSettingRaw(path, val.as<std::vector<std::byte> >()); break;
default: break;
}
+ if(cache.has_value())
+ {
+ (*cache)[mpt::ToWin(path.GetRefSection())][mpt::ToWin(path.GetRefKey())] = FormatValueAsIni(val);
+ }
}
void IniFileSettingsBackend::RemoveSetting(const SettingPath &path)
{
OPENMPT_PROFILE_FUNCTION(Profiler::Settings);
- RemoveSettingRaw(path);
+ if(cache.has_value())
+ {
+ (*cache)[mpt::ToWin(path.GetRefSection())][mpt::ToWin(path.GetRefKey())] = std::nullopt;
+ } else
+ {
+ RemoveSettingRaw(path);
+ }
}
void IniFileSettingsBackend::RemoveSection(const mpt::ustring §ion)
@@ -505,19 +583,255 @@
{
OPENMPT_PROFILE_FUNCTION(Profiler::Settings);
RemoveSectionRaw(section);
+ if(cache.has_value())
+ {
+ const auto it = (*cache).find(mpt::ToWin(section));
+ if(it != (*cache).end())
+ {
+ (*cache).erase(it);
+ }
+ }
}
+std::set<mpt::winstring> IniFileSettingsBackend::ReadSectionNamesRaw() const
+{
+ std::set<mpt::winstring> result;
+ const std::vector<TCHAR> sectionsstr = [&]()
+ {
+ std::vector<TCHAR> buf;
+ buf.resize(mpt::IO::BUFFERSIZE_SMALL);
+ while(true)
+ {
+ DWORD bufused = ::GetPrivateProfileSectionNames(buf.data(), mpt::saturate_cast<DWORD>(buf.size()), filename.AsNative().c_str());
+ if(bufused >= (buf.size() - 2))
+ {
+ std::size_t newsize = mpt::exponential_grow(buf.size());
+ buf.resize(0);
+ buf.resize(newsize);
+ continue;
+ }
+ if(bufused >= 1)
+ {
+ bufused -= 1; // terminating \0
+ }
+ buf.resize(bufused);
+ break;
+ };
+ return buf;
+ }();
+ const std::vector<mpt::winstring> sections = mpt::split(mpt::winstring(sectionsstr.data(), sectionsstr.size()), mpt::winstring(_T("\0"), 1));
+ for(const auto §ion : sections)
+ {
+ result.insert(section);
+ }
+ return result;
+}
+std::map<mpt::winstring, std::optional<mpt::winstring>> IniFileSettingsBackend::ReadNamedSectionRaw(const mpt::winstring §ion) const
+{
+ std::map<mpt::winstring, std::optional<mpt::winstring>> result;
+ const std::vector<TCHAR> keyvalsstr = [&]()
+ {
+ std::vector<TCHAR> buf;
+ buf.resize(mpt::IO::BUFFERSIZE_SMALL);
+ while(true)
+ {
+ DWORD bufused = ::GetPrivateProfileSection(mpt::ToWin(section).c_str(), buf.data(), mpt::saturate_cast<DWORD>(buf.size()), filename.AsNative().c_str());
+ if(bufused >= (buf.size() - 2))
+ {
+ std::size_t newsize = mpt::exponential_grow(buf.size());
+ buf.resize(0);
+ buf.resize(newsize);
+ continue;
+ }
+ if(bufused >= 1)
+ {
+ bufused -= 1; // terminating \0
+ }
+ buf.resize(bufused);
+ break;
+ };
+ return buf;
+ }();
+ const std::vector<mpt::winstring> keyvals = mpt::split(mpt::winstring(keyvalsstr.data(), keyvalsstr.size()), mpt::winstring(_T("\0"), 1));
+ for(const auto &keyval : keyvals)
+ {
+ const auto equalpos = keyval.find(_T("="));
+ if(equalpos == mpt::winstring::npos)
+ {
+ continue;
+ }
+ if(equalpos == 0)
+ {
+ continue;
+ }
+ result.insert(std::make_pair(keyval.substr(0, equalpos), std::make_optional(keyval.substr(equalpos + 1))));
+ }
+ return result;
+}
+std::map<mpt::winstring, std::map<mpt::winstring, std::optional<mpt::winstring>>> IniFileSettingsBackend::ReadAllSectionsRaw() const
+{
+ std::map<mpt::winstring, std::map<mpt::winstring, std::optional<mpt::winstring>>> result;
+ const std::set<mpt::winstring> sectionnames = ReadSectionNamesRaw();
+ for(const mpt::winstring §ionname : sectionnames)
+ {
+ result.insert(std::make_pair(sectionname, ReadNamedSectionRaw(sectionname)));
+ }
+ return result;
+}
+mpt::winstring IniFileSettingsBackend::FormatValueAsIni(const SettingValue &value)
+{
+ switch(value.GetType())
+ {
+ case SettingTypeBool:
+ return mpt::tfmt::val(value.as<bool>());
+ break;
+ case SettingTypeInt:
+ return mpt::tfmt::val(value.as<int32>());
+ break;
+ case SettingTypeFloat:
+ return mpt::tfmt::val(value.as<double>());
+ break;
+ case SettingTypeString:
+ return mpt::ToWin(value.as<mpt::ustring>());
+ break;
+ case SettingTypeBinary:
+ {
+ std::vector<std::byte> data = value.as<std::vector<std::byte>>();
+ uint8 checksum = 0;
+ for(const std::byte x : data) {
+ checksum += mpt::byte_cast<uint8>(x);
+ }
+ return mpt::ToWin(mpt::encode_hex(mpt::as_span(data)) + mpt::ufmt::HEX0<2>(checksum));
+ }
+ break;
+ case SettingTypeNone:
+ default:
+ return mpt::winstring();
+ break;
+ }
+}
+
+SettingValue IniFileSettingsBackend::ParseValueFromIni(const mpt::winstring &str, const SettingValue &def)
+{
+ switch(def.GetType())
+ {
+ case SettingTypeBool:
+ return SettingValue(mpt::parse_or<bool>(mpt::trim(str), def.as<bool>()), def.GetTypeTag());
+ break;
+ case SettingTypeInt:
+ return SettingValue(mpt::parse_or<int32>(mpt::trim(str), def.as<int32>()), def.GetTypeTag());
+ break;
+ case SettingTypeFloat:
+ return SettingValue(mpt::parse_or<double>(mpt::trim(str), def.as<double>()), def.GetTypeTag());
+ break;
+ case SettingTypeString:
+ return SettingValue(mpt::ToUnicode(str), def.GetTypeTag());
+ break;
+ case SettingTypeBinary:
+ {
+ mpt::ustring ustr = mpt::trim(mpt::ToUnicode(str));
+ if((ustr.length() % 2) != 0)
+ {
+ return SettingValue(def.as<std::vector<std::byte>>(), def.GetTypeTag());
+ }
+ std::vector<std::byte> data = mpt::decode_hex(ustr);
+ if(data.size() < 1)
+ {
+ return SettingValue(def.as<std::vector<std::byte>>(), def.GetTypeTag());
+ }
+ const uint8 storedchecksum = mpt::byte_cast<uint8>(data[data.size() - 1]);
+ data.resize(data.size() - 1);
+ uint8 calculatedchecksum = 0;
+ for(const std::byte x : data) {
+ calculatedchecksum += mpt::byte_cast<uint8>(x);
+ }
+ if(calculatedchecksum != storedchecksum)
+ {
+ return SettingValue(def.as<std::vector<std::byte>>(), def.GetTypeTag());
+ }
+ return SettingValue(data, def.GetTypeTag());
+ }
+ break;
+ default:
+ return SettingValue();
+ break;
+ }
+}
+
+void IniFileSettingsBackend::WriteSectionRaw(const mpt::winstring §ion, const std::map<mpt::winstring, std::optional<mpt::winstring>> &keyvalues)
+{
+ mpt::winstring keyvals;
+ for(const auto &[key, val] : keyvalues)
+ {
+ if(val.has_value())
+ {
+ keyvals.append(key);
+ keyvals.append(_T("="));
+ keyvals.append(val.value());
+ keyvals.append(mpt::winstring(_T("\0"), 1));
+ }
+ }
+ keyvals.append(mpt::winstring(_T("\0"), 1));
+ ::WritePrivateProfileSection(section.c_str(), keyvals.c_str(), filename.AsNative().c_str());
+}
+
+void IniFileSettingsBackend::WriteMultipleSettings(const std::map<SettingPath, SettingValue> &settings)
+{
+ OPENMPT_PROFILE_FUNCTION(Profiler::Settings);
+ std::map<mpt::ustring, std::map<SettingPath, SettingValue>> sectionssettings;
+ for(const auto &[path, value] : settings)
+ {
+ sectionssettings[path.GetRefSection()][path] = value;
+ }
+ for(const auto &[section, sectionsettings] : sectionssettings)
+ {
+ // we need to re-read the section before writing it out in order to not overwrite settings written by MFC
+ std::map<mpt::winstring, std::optional<mpt::winstring>> workingsectionsettings = ReadNamedSectionRaw(mpt::ToWin(section));
+ // apply deleted settings from cache
+ for(const auto &[key, value] : (*cache)[mpt::ToWin(section)])
+ {
+ if(!value.has_value())
+ {
+ workingsectionsettings[key] = std::nullopt;
+ }
+ }
+ for(const auto &[path, value] : sectionsettings)
+ {
+ workingsectionsettings[mpt::ToWin(path.GetRefKey())] = FormatValueAsIni(value);
+ }
+ WriteSectionRaw(mpt::ToWin(section), workingsectionsettings);
+ if(cache.has_value())
+ {
+ (*cache)[mpt::ToWin(section)] = std::move(workingsectionsettings);
+ }
+ }
+}
+
+bool IniFileSettingsBackend::CanBatchedSettings() const
+{
+ return cache.has_value();
+}
+
+
+
IniFileSettingsContainer::IniFileSettingsContainer(const mpt::PathString &filename)
- : IniFileSettingsBackend(filename)
+ : IniFileSettingsBackend(filename, CachePolicy::Cached)
, SettingsContainer(this)
{
return;
}
+IniFileSettingsContainer::IniFileSettingsContainer(const mpt::PathString &filename, CachePolicy cachePolicy)
+ : IniFileSettingsBackend(filename, cachePolicy)
+ , SettingsContainer(this)
+{
+ return;
+}
+
IniFileSettingsContainer::~IniFileSettingsContainer()
{
return;
Index: mptrack/Settings.h
===================================================================
--- mptrack/Settings.h (revision 24343)
+++ mptrack/Settings.h (working copy)
@@ -408,6 +408,8 @@
virtual void WriteSetting(const SettingPath &path, const SettingValue &val) = 0;
virtual void RemoveSetting(const SettingPath &path) = 0;
virtual void RemoveSection(const mpt::ustring §ion) = 0;
+ virtual bool CanBatchedSettings() const = 0;
+ virtual void WriteMultipleSettings(const std::map<SettingPath, SettingValue> &settings) = 0;
protected:
virtual ~ISettingsBackend() = default;
};
@@ -448,6 +450,8 @@
void BackendsWriteSetting(const SettingPath &path, const SettingValue &val);
void BackendsRemoveSetting(const SettingPath &path);
void BackendsRemoveSection(const mpt::ustring §ion);
+ bool BackendsCanWriteMultipleSettings() const;
+ void BackendsWriteMultipleSettings(const std::map<SettingPath, SettingValue> &settings);
void NotifyListeners(const SettingPath &path);
SettingValue ReadSetting(const SettingPath &path, const SettingValue &def) const;
bool IsDefaultSetting(const SettingPath &path) const;
@@ -684,11 +688,26 @@
};
+enum class CachePolicy
+{
+ Direct,
+ Cached,
+};
+
class IniFileSettingsBackend : public ISettingsBackend
{
+public:
private:
const mpt::PathString filename;
+ std::optional<std::map<mpt::winstring, std::map<mpt::winstring, std::optional<mpt::winstring>>>> cache;
+#ifdef ENABLE_TESTS
+public:
+#else
private:
+#endif
+ std::set<mpt::winstring> ReadSectionNamesRaw() const;
+ std::map<mpt::winstring, std::optional<mpt::winstring>> ReadNamedSectionRaw(const mpt::winstring §ion) const;
+ std::map<mpt::winstring, std::map<mpt::winstring, std::optional<mpt::winstring>>> ReadAllSectionsRaw() const;
std::vector<std::byte> ReadSettingRaw(const SettingPath &path, const std::vector<std::byte> &def) const;
mpt::ustring ReadSettingRaw(const SettingPath &path, const mpt::ustring &def) const;
double ReadSettingRaw(const SettingPath &path, double def) const;
@@ -699,12 +718,17 @@
void WriteSettingRaw(const SettingPath &path, double val);
void WriteSettingRaw(const SettingPath &path, int32 val);
void WriteSettingRaw(const SettingPath &path, bool val);
+ void WriteSectionRaw(const mpt::winstring §ion, const std::map<mpt::winstring, std::optional<mpt::winstring>> &keyvalues);
void RemoveSettingRaw(const SettingPath &path);
void RemoveSectionRaw(const mpt::ustring §ion);
+private:
+ static mpt::winstring FormatValueAsIni(const SettingValue &value);
+ static SettingValue ParseValueFromIni(const mpt::winstring &str, const SettingValue &def);
static mpt::winstring GetSection(const SettingPath &path);
static mpt::winstring GetKey(const SettingPath &path);
public:
- IniFileSettingsBackend(const mpt::PathString &filename);
+ [[deprecated]] IniFileSettingsBackend(const mpt::PathString &filename);
+ IniFileSettingsBackend(const mpt::PathString &filename, CachePolicy cachePolicy);
~IniFileSettingsBackend() override;
void ConvertToUnicode(const mpt::ustring &backupTag = mpt::ustring());
virtual SettingValue ReadSetting(const SettingPath &path, const SettingValue &def) const override;
@@ -711,6 +735,9 @@
virtual void WriteSetting(const SettingPath &path, const SettingValue &val) override;
virtual void RemoveSetting(const SettingPath &path) override;
virtual void RemoveSection(const mpt::ustring §ion) override;
+ virtual bool CanBatchedSettings() const override;
+ virtual void WriteMultipleSettings(const std::map<SettingPath, SettingValue> &settings) override;
+public:
const mpt::PathString& GetFilename() const { return filename; }
};
@@ -717,7 +744,8 @@
class IniFileSettingsContainer : private IniFileSettingsBackend, public SettingsContainer
{
public:
- IniFileSettingsContainer(const mpt::PathString &filename);
+ [[deprecated]] IniFileSettingsContainer(const mpt::PathString &filename);
+ IniFileSettingsContainer(const mpt::PathString &filename, CachePolicy cachePolicy);
~IniFileSettingsContainer() override;
};
Index: test/test.cpp
===================================================================
--- test/test.cpp (revision 24343)
+++ test/test.cpp (working copy)
@@ -2737,9 +2737,35 @@
VERIFY_EQUAL(inifile.ReadSetting(SettingPath{U_("S1"), U_("bar1")}, U_("empty")).as<mpt::ustring>(), U_("a"));
VERIFY_EQUAL(inifile.ReadSetting(SettingPath{U_("S1"), U_("bar2")}, U_("empty")).as<mpt::ustring>(), U_("empty"));
}
+ {
+ IniFileSettingsBackend inifile{filename};
+ const std::set<mpt::winstring> a = inifile.ReadSectionNamesRaw();
+ for(const mpt::winstring & s : a)
+ {
+ std::map<mpt::winstring, std::optional<mpt::winstring>> b = inifile.ReadNamedSectionRaw(s);
+ }
+ }
DeleteFile(mpt::support_long_path(filename.AsNative()).c_str());
}
+ {
+ const mpt::PathString filename = theApp.GetConfigPath() + P_("test.ini");
+ std::vector<std::byte> data;
+ for(std::size_t i = 0; i < 10; ++i)
+ {
+ data.push_back(mpt::byte_cast<std::byte>(static_cast<uint8>(i)));
+ }
+ {
+ IniFileSettingsBackend inifile{filename, CachePolicy::Cached};
+ inifile.WriteMultipleSettings({{SettingPath(U_("Test"), U_("Data")), data}});
+ }
+ {
+ IniFileSettingsBackend inifile{filename, CachePolicy::Cached};
+ VERIFY_EQUAL(inifile.ReadSetting(SettingPath(U_("Test"), U_("Data")), SettingValue(data)).as<std::vector<std::byte>>(), data);
+ }
+ DeleteFile(mpt::support_long_path(filename.AsNative()).c_str());
+ }
+
#endif // MODPLUG_TRACKER
}
|
|
|
speedup-ini-settings-v6-wip.patch (20,093 bytes)
Index: common/Profiler.h
===================================================================
--- common/Profiler.h (revision 24593)
+++ common/Profiler.h (working copy)
@@ -54,7 +54,7 @@
#if defined(MODPLUG_TRACKER)
-//#define USE_PROFILER
+#define USE_PROFILER
#endif
Index: mptrack/Mptrack.cpp
===================================================================
--- mptrack/Mptrack.cpp (revision 24593)
+++ mptrack/Mptrack.cpp (working copy)
@@ -932,7 +932,7 @@
const mpt::PathString &CTrackApp::GetSongSettingsFilename() const
{
- return m_pSongSettingsIniFile->GetFilename();
+ return m_pSongSettingsIniFile->Filename();
}
Index: mptrack/SettingsIni.cpp
===================================================================
--- mptrack/SettingsIni.cpp (revision 24593)
+++ mptrack/SettingsIni.cpp (working copy)
@@ -13,6 +13,7 @@
#include "SettingsIni.h"
+#include "mpt/binary/hex.hpp"
#include "mpt/io_file/fstream.hpp"
#include "mpt/io_file/outputfile.hpp"
#include "mpt/parse/parse.hpp"
@@ -22,11 +23,33 @@
#include "../common/mptStringBuffer.h"
#include <algorithm>
+#include <utility>
+
OPENMPT_NAMESPACE_BEGIN
+
+WindowsIniFileBase::WindowsIniFileBase(mpt::PathString filename)
+ : filename(std::move(filename))
+ , file(filename)
+{
+ return;
+}
+
+const mpt::PathString &WindowsIniFileBase::Filename() const
+{
+ return filename;
+}
+
+mpt::PathString WindowsIniFileBase::GetFilename() const
+{
+ return filename;
+}
+
+
+
std::vector<std::byte> ImmediateWindowsIniFileSettingsBackend::ReadSettingRaw(const SettingPath &path, const std::vector<std::byte> &def) const
{
std::vector<std::byte> result = def;
@@ -126,12 +149,17 @@
::WritePrivateProfileString(mpt::ToWin(section).c_str(), NULL, NULL, filename.AsNative().c_str());
}
+void CachedBatchedWindowsIniFileSettingsBackend::RemoveSectionRaw(const mpt::ustring §ion)
+{
+ ::WritePrivateProfileString(mpt::ToWin(section).c_str(), NULL, NULL, filename.AsNative().c_str());
+}
-mpt::winstring ImmediateWindowsIniFileSettingsBackend::GetSection(const SettingPath &path)
+
+mpt::winstring IniFileHelpers::GetSection(const SettingPath &path)
{
return mpt::ToWin(path.GetSection());
}
-mpt::winstring ImmediateWindowsIniFileSettingsBackend::GetKey(const SettingPath &path)
+mpt::winstring IniFileHelpers::GetKey(const SettingPath &path)
{
return mpt::ToWin(path.GetKey());
}
@@ -138,18 +166,36 @@
-ImmediateWindowsIniFileSettingsBackend::ImmediateWindowsIniFileSettingsBackend(const mpt::PathString &filename)
- : filename(filename)
- , file(filename)
+ImmediateWindowsIniFileSettingsBackend::ImmediateWindowsIniFileSettingsBackend(mpt::PathString filename)
+ : WindowsIniFileBase(std::move(filename))
{
+ OPENMPT_PROFILE_FUNCTION(Profiler::Settings);
return;
}
+CachedBatchedWindowsIniFileSettingsBackend::CachedBatchedWindowsIniFileSettingsBackend(mpt::PathString filename)
+ : WindowsIniFileBase(std::move(filename))
+{
+ OPENMPT_PROFILE_FUNCTION(Profiler::Settings);
+ ConvertToUnicode();
+ cache = ReadAllSectionsRaw();
+}
+
+void CachedBatchedWindowsIniFileSettingsBackend::InvalidateCache()
+{
+ cache = ReadAllSectionsRaw();
+}
+
ImmediateWindowsIniFileSettingsBackend::~ImmediateWindowsIniFileSettingsBackend()
{
return;
}
+CachedBatchedWindowsIniFileSettingsBackend::~CachedBatchedWindowsIniFileSettingsBackend()
+{
+ return;
+}
+
static void WriteFileUTF16LE(mpt::IO::atomic_shared_file_ref &file, const std::wstring &str)
{
static_assert(sizeof(wchar_t) == 2);
@@ -160,7 +206,7 @@
file.write(mpt::byte_cast<mpt::const_byte_span>(mpt::as_span(inifile.str())));
}
-void ImmediateWindowsIniFileSettingsBackend::ConvertToUnicode(const mpt::ustring &backupTag)
+void WindowsIniFileBase::ConvertToUnicode(const mpt::ustring &backupTag)
{
// Force ini file to be encoded in UTF16.
// This causes WINAPI ini file functions to keep it in UTF16 encoding
@@ -209,6 +255,39 @@
}
}
+SettingValue CachedBatchedWindowsIniFileSettingsBackend::ReadSetting(const SettingPath &path, const SettingValue &def) const
+{
+ OPENMPT_PROFILE_FUNCTION(Profiler::Settings);
+ const auto sectionit = cache.find(path.GetRefSection());
+ if(sectionit != cache.end())
+ {
+ if(sectionit->second.has_value())
+ {
+ const std::map<mpt::ustring, std::optional<mpt::ustring>> §ion = sectionit->second.value();
+ const auto it = section.find(path.GetRefKey());
+ if(it != section.end())
+ {
+ if(it->second.has_value())
+ {
+ return ParseValueFromIni(it->second.value(), def);
+ } else
+ {
+ return def;
+ }
+ } else
+ {
+ return def;
+ }
+ } else
+ {
+ return def;
+ }
+ } else
+ {
+ return def;
+ }
+}
+
void ImmediateWindowsIniFileSettingsBackend::WriteSetting(const SettingPath &path, const SettingValue &val)
{
OPENMPT_PROFILE_FUNCTION(Profiler::Settings);
@@ -236,22 +315,271 @@
RemoveSectionRaw(section);
}
+std::set<mpt::ustring> CachedBatchedWindowsIniFileSettingsBackend::ReadSectionNamesRaw() const
+{
+ std::set<mpt::ustring> result;
+ const std::vector<TCHAR> sectionsstr = [&]()
+ {
+ std::vector<TCHAR> buf;
+ buf.resize(mpt::IO::BUFFERSIZE_SMALL);
+ while(true)
+ {
+ DWORD bufused = ::GetPrivateProfileSectionNames(buf.data(), mpt::saturate_cast<DWORD>(buf.size()), filename.AsNative().c_str());
+ if(bufused >= (buf.size() - 2))
+ {
+ std::size_t newsize = mpt::exponential_grow(buf.size());
+ buf.resize(0);
+ buf.resize(newsize);
+ continue;
+ }
+ if(bufused >= 1)
+ {
+ bufused -= 1; // terminating \0
+ }
+ buf.resize(bufused);
+ break;
+ };
+ return buf;
+ }();
+ const std::vector<mpt::winstring> sections = mpt::split(mpt::winstring(sectionsstr.data(), sectionsstr.size()), mpt::winstring(_T("\0"), 1));
+ for(const auto §ion : sections)
+ {
+ result.insert(mpt::ToUnicode(section));
+ }
+ return result;
+}
+std::map<mpt::ustring, std::optional<mpt::ustring>> CachedBatchedWindowsIniFileSettingsBackend::ReadNamedSectionRaw(const mpt::ustring §ion) const
+{
+ std::map<mpt::ustring, std::optional<mpt::ustring>> result;
+ const std::vector<TCHAR> keyvalsstr = [&]()
+ {
+ std::vector<TCHAR> buf;
+ buf.resize(mpt::IO::BUFFERSIZE_SMALL);
+ while(true)
+ {
+ DWORD bufused = ::GetPrivateProfileSection(mpt::ToWin(section).c_str(), buf.data(), mpt::saturate_cast<DWORD>(buf.size()), filename.AsNative().c_str());
+ if(bufused >= (buf.size() - 2))
+ {
+ std::size_t newsize = mpt::exponential_grow(buf.size());
+ buf.resize(0);
+ buf.resize(newsize);
+ continue;
+ }
+ if(bufused >= 1)
+ {
+ bufused -= 1; // terminating \0
+ }
+ buf.resize(bufused);
+ break;
+ };
+ return buf;
+ }();
+ const std::vector<mpt::winstring> keyvals = mpt::split(mpt::winstring(keyvalsstr.data(), keyvalsstr.size()), mpt::winstring(_T("\0"), 1));
+ for(const auto &keyval : keyvals)
+ {
+ const auto equalpos = keyval.find(_T("="));
+ if(equalpos == mpt::winstring::npos)
+ {
+ continue;
+ }
+ if(equalpos == 0)
+ {
+ continue;
+ }
+ result.insert(std::make_pair(mpt::ToUnicode(keyval.substr(0, equalpos)), std::make_optional(mpt::ToUnicode(keyval.substr(equalpos + 1)))));
+ }
+ return result;
+}
+std::map<mpt::ustring, std::optional<std::map<mpt::ustring, std::optional<mpt::ustring>>>> CachedBatchedWindowsIniFileSettingsBackend::ReadAllSectionsRaw() const
+{
+ std::map<mpt::ustring, std::optional<std::map<mpt::ustring, std::optional<mpt::ustring>>>> result;
+ const std::set<mpt::ustring> sectionnames = ReadSectionNamesRaw();
+ for(const mpt::ustring §ionname : sectionnames)
+ {
+ result.insert(std::make_pair(sectionname, std::make_optional(ReadNamedSectionRaw(sectionname))));
+ }
+ return result;
+}
+mpt::ustring IniFileHelpers::FormatValueAsIni(const SettingValue &value)
+{
+ switch(value.GetType())
+ {
+ case SettingTypeBool:
+ return mpt::ufmt::val(value.as<bool>());
+ break;
+ case SettingTypeInt:
+ return mpt::ufmt::val(value.as<int32>());
+ break;
+ case SettingTypeFloat:
+ return mpt::ufmt::val(value.as<double>());
+ break;
+ case SettingTypeString:
+ return value.as<mpt::ustring>();
+ break;
+ case SettingTypeBinary:
+ {
+ std::vector<std::byte> data = value.as<std::vector<std::byte>>();
+ uint8 checksum = 0;
+ for(const std::byte x : data) {
+ checksum += mpt::byte_cast<uint8>(x);
+ }
+ return mpt::encode_hex(mpt::as_span(data)) + mpt::ufmt::HEX0<2>(checksum);
+ }
+ break;
+ case SettingTypeNone:
+ default:
+ return mpt::ustring();
+ break;
+ }
+}
-IniFileSettingsContainer::IniFileSettingsContainer(const mpt::PathString &filename)
- : ImmediateWindowsIniFileSettingsBackend(filename)
- , SettingsContainer(this)
+
+SettingValue IniFileHelpers::ParseValueFromIni(const mpt::ustring &str, const SettingValue &def)
{
+ switch(def.GetType())
+ {
+ case SettingTypeBool:
+ return SettingValue(mpt::parse_or<bool>(mpt::trim(str), def.as<bool>()), def.GetTypeTag());
+ break;
+ case SettingTypeInt:
+ return SettingValue(mpt::parse_or<int32>(mpt::trim(str), def.as<int32>()), def.GetTypeTag());
+ break;
+ case SettingTypeFloat:
+ return SettingValue(mpt::parse_or<double>(mpt::trim(str), def.as<double>()), def.GetTypeTag());
+ break;
+ case SettingTypeString:
+ return SettingValue(str, def.GetTypeTag());
+ break;
+ case SettingTypeBinary:
+ {
+ if((str.length() % 2) != 0)
+ {
+ return SettingValue(def.as<std::vector<std::byte>>(), def.GetTypeTag());
+ }
+ std::vector<std::byte> data = mpt::decode_hex(str);
+ if(data.size() < 1)
+ {
+ return SettingValue(def.as<std::vector<std::byte>>(), def.GetTypeTag());
+ }
+ const uint8 storedchecksum = mpt::byte_cast<uint8>(data[data.size() - 1]);
+ data.resize(data.size() - 1);
+ uint8 calculatedchecksum = 0;
+ for(const std::byte x : data) {
+ calculatedchecksum += mpt::byte_cast<uint8>(x);
+ }
+ if(calculatedchecksum != storedchecksum)
+ {
+ return SettingValue(def.as<std::vector<std::byte>>(), def.GetTypeTag());
+ }
+ return SettingValue(data, def.GetTypeTag());
+ }
+ break;
+ default:
+ return SettingValue();
+ break;
+ }
+}
+
+void CachedBatchedWindowsIniFileSettingsBackend::WriteSectionRaw(const mpt::ustring §ion, const std::map<mpt::ustring, std::optional<mpt::ustring>> &keyvalues)
+{
+ OPENMPT_PROFILE_FUNCTION(Profiler::Settings);
+ mpt::winstring keyvals;
+ for(const auto &[key, val] : keyvalues)
+ {
+ if(val.has_value())
+ {
+ keyvals.append(mpt::ToWin(key));
+ keyvals.append(_T("="));
+ keyvals.append(mpt::ToWin(val.value()));
+ keyvals.append(mpt::winstring(_T("\0"), 1));
+ }
+ }
+ keyvals.append(mpt::winstring(_T("\0"), 1));
+ ::WritePrivateProfileSection(mpt::ToWin(section).c_str(), keyvals.c_str(), filename.AsNative().c_str());
+}
+
+void CachedBatchedWindowsIniFileSettingsBackend::WriteRemovedSections(const std::set<mpt::ustring> &removeSections)
+{
+ OPENMPT_PROFILE_FUNCTION(Profiler::Settings);
+ for(const auto §ion : removeSections)
+ {
+ const auto it = cache.find(section);
+ if(it != cache.end())
+ {
+ it->second = std::nullopt;
+ }
+ }
+ for(const auto &[section, keyvalues] : cache)
+ {
+ if(!keyvalues.has_value())
+ {
+ RemoveSectionRaw(section);
+ }
+ }
+ for(const auto §ion : removeSections)
+ {
+ cache.erase(section);
+ }
+}
+
+void CachedBatchedWindowsIniFileSettingsBackend::WriteMultipleSettings(const std::map<SettingPath, std::optional<SettingValue>> &settings)
+{
+ OPENMPT_PROFILE_FUNCTION(Profiler::Settings);
+ std::map<mpt::ustring, std::map<SettingPath, std::optional<SettingValue>>> sectionssettings;
+ for(const auto &[path, value] : settings)
+ {
+ sectionssettings[path.GetRefSection()][path] = value;
+ }
+ for(const auto &[section, sectionsettings] : sectionssettings)
+ {
+ if(!cache[section].has_value())
+ {
+ cache[section].emplace();
+ }
+ std::map<mpt::ustring, std::optional<mpt::ustring>> &workingsectionsettings = cache[section].value();
+ for(const auto &[path, value] : sectionsettings)
+ {
+ if(value.has_value())
+ {
+ workingsectionsettings[path.GetRefKey()] = FormatValueAsIni(value.value());
+ } else
+ {
+ workingsectionsettings.erase(path.GetRefKey());
+ }
+ }
+ WriteSectionRaw(section, workingsectionsettings);
+ }
+}
+
+
+
+IniFileSettingsContainer::IniFileSettingsContainer(mpt::PathString filename)
+ : ImmediateWindowsIniFileSettingsBackend(std::move(filename))
+ , SettingsContainer(static_cast<ISettingsBackendFlavour<SettingsBatching::Single>*>(this))
+{
return;
}
+BatchedIniFileSettingsContainer::BatchedIniFileSettingsContainer(mpt::PathString filename)
+ : CachedBatchedWindowsIniFileSettingsBackend(std::move(filename))
+ , SettingsContainer(static_cast<ISettingsBackendFlavour<SettingsBatching::Section>*>(this))
+{
+ return;
+}
+
IniFileSettingsContainer::~IniFileSettingsContainer()
{
return;
}
+BatchedIniFileSettingsContainer::~BatchedIniFileSettingsContainer()
+{
+ return;
+}
+
OPENMPT_NAMESPACE_END
Index: mptrack/SettingsIni.h
===================================================================
--- mptrack/SettingsIni.h (revision 24593)
+++ mptrack/SettingsIni.h (working copy)
@@ -13,12 +13,12 @@
#include "openmpt/all/BuildSettings.hpp"
-
#include "Settings.h"
#include "mpt/io_file_atomic/atomic_file.hpp"
#include <map>
+#include <optional>
#include <set>
#include <vector>
@@ -25,49 +25,112 @@
OPENMPT_NAMESPACE_BEGIN
+
+// Version 1: ANSI
+// Version 2: UTF-16LE with BOM
+// Version 3: UTF8 with BOM (not supported by Windows)
+// Version 4: UTF8 without BOM and explicit version
+
+
+class IniFileHelpers
+{
+protected:
+ static mpt::ustring FormatValueAsIni(const SettingValue &value);
+ static SettingValue ParseValueFromIni(const mpt::ustring &str, const SettingValue &def);
+ static mpt::winstring GetSection(const SettingPath &path);
+ static mpt::winstring GetKey(const SettingPath &path);
+};
+
+
+class WindowsIniFileBase
+{
+protected:
+ const mpt::PathString filename;
+ mpt::IO::atomic_shared_file_ref file;
+protected:
+ WindowsIniFileBase(mpt::PathString filename);
+ ~WindowsIniFileBase() = default;
+public:
+ void ConvertToUnicode(const mpt::ustring &backupTag = mpt::ustring());
+public:
+ const mpt::PathString &Filename() const;
+ mpt::PathString GetFilename() const;
+};
+
+
class ImmediateWindowsIniFileSettingsBackend
: virtual public ISettingsBackend
, public ISettingsBackendFlavour<SettingsBatching::Single>
+ , public WindowsIniFileBase
+ , protected IniFileHelpers
{
private:
- const mpt::PathString filename;
- mpt::IO::atomic_shared_file_ref file;
-private:
std::vector<std::byte> ReadSettingRaw(const SettingPath &path, const std::vector<std::byte> &def) const;
mpt::ustring ReadSettingRaw(const SettingPath &path, const mpt::ustring &def) const;
double ReadSettingRaw(const SettingPath &path, double def) const;
int32 ReadSettingRaw(const SettingPath &path, int32 def) const;
bool ReadSettingRaw(const SettingPath &path, bool def) const;
+ void RemoveSectionRaw(const mpt::ustring §ion);
+ void RemoveSettingRaw(const SettingPath &path);
void WriteSettingRaw(const SettingPath &path, const std::vector<std::byte> &val);
void WriteSettingRaw(const SettingPath &path, const mpt::ustring &val);
void WriteSettingRaw(const SettingPath &path, double val);
void WriteSettingRaw(const SettingPath &path, int32 val);
void WriteSettingRaw(const SettingPath &path, bool val);
- void RemoveSettingRaw(const SettingPath &path);
- void RemoveSectionRaw(const mpt::ustring §ion);
- static mpt::winstring GetSection(const SettingPath &path);
- static mpt::winstring GetKey(const SettingPath &path);
public:
- ImmediateWindowsIniFileSettingsBackend(const mpt::PathString &filename);
+ ImmediateWindowsIniFileSettingsBackend(mpt::PathString filename);
~ImmediateWindowsIniFileSettingsBackend() override;
- void ConvertToUnicode(const mpt::ustring &backupTag = mpt::ustring());
+public:
virtual SettingValue ReadSetting(const SettingPath &path, const SettingValue &def) const override;
+ virtual void RemoveSection(const mpt::ustring §ion) override;
+ virtual void RemoveSetting(const SettingPath &path) override;
virtual void WriteSetting(const SettingPath &path, const SettingValue &val) override;
- virtual void RemoveSetting(const SettingPath &path) override;
- virtual void RemoveSection(const mpt::ustring §ion) override;
- const mpt::PathString& GetFilename() const { return filename; }
};
-#ifndef IniFileSettingsBackend
+
+class CachedBatchedWindowsIniFileSettingsBackend
+ : virtual public ISettingsBackend
+ , public ISettingsBackendFlavour<SettingsBatching::Section>
+ , public WindowsIniFileBase
+ , protected IniFileHelpers
+{
+private:
+ std::map<mpt::ustring, std::optional<std::map<mpt::ustring, std::optional<mpt::ustring>>>> cache;
+private:
+ std::set<mpt::ustring> ReadSectionNamesRaw() const;
+ std::map<mpt::ustring, std::optional<mpt::ustring>> ReadNamedSectionRaw(const mpt::ustring §ion) const;
+ std::map<mpt::ustring, std::optional<std::map<mpt::ustring, std::optional<mpt::ustring>>>> ReadAllSectionsRaw() const;
+ void RemoveSectionRaw(const mpt::ustring §ion);
+ void WriteSectionRaw(const mpt::ustring §ion, const std::map<mpt::ustring, std::optional<mpt::ustring>> &keyvalues);
+public:
+ CachedBatchedWindowsIniFileSettingsBackend(mpt::PathString filename);
+ ~CachedBatchedWindowsIniFileSettingsBackend() override;
+public:
+ virtual void InvalidateCache() override;
+ virtual SettingValue ReadSetting(const SettingPath &path, const SettingValue &def) const override;
+ virtual void WriteRemovedSections(const std::set<mpt::ustring> &removeSections) override;
+ virtual void WriteMultipleSettings(const std::map<SettingPath, std::optional<SettingValue>> &settings) override;
+};
+
+
#define IniFileSettingsBackend ImmediateWindowsIniFileSettingsBackend
-#endif
-class IniFileSettingsContainer : private IniFileSettingsBackend, public SettingsContainer
+
+class IniFileSettingsContainer : private ImmediateWindowsIniFileSettingsBackend, public SettingsContainer
{
public:
- IniFileSettingsContainer(const mpt::PathString &filename);
+ IniFileSettingsContainer(mpt::PathString filename);
~IniFileSettingsContainer() override;
};
+class BatchedIniFileSettingsContainer : private CachedBatchedWindowsIniFileSettingsBackend, public SettingsContainer
+{
+public:
+ BatchedIniFileSettingsContainer(mpt::PathString filename);
+ ~BatchedIniFileSettingsContainer() override;
+};
+
+
+
OPENMPT_NAMESPACE_END
Index: test/test.cpp
===================================================================
--- test/test.cpp (revision 24593)
+++ test/test.cpp (working copy)
@@ -2741,6 +2741,14 @@
VERIFY_EQUAL(inifile.ReadSetting(SettingPath{U_("S1"), U_("bar1")}, U_("empty")).as<mpt::ustring>(), U_("a"));
VERIFY_EQUAL(inifile.ReadSetting(SettingPath{U_("S1"), U_("bar2")}, U_("empty")).as<mpt::ustring>(), U_("empty"));
}
+ {
+ IniFileSettingsBackend inifile{filename};
+ const std::set<mpt::winstring> a = inifile.ReadSectionNamesRaw();
+ for(const mpt::winstring & s : a)
+ {
+ std::map<mpt::winstring, std::optional<mpt::winstring>> b = inifile.ReadNamedSectionRaw(s);
+ }
+ }
DeleteFile(mpt::support_long_path(filename.AsNative()).c_str());
}
@@ -2801,6 +2809,23 @@
}
#endif
+ {
+ std::vector<std::byte> data;
+ for(std::size_t i = 0; i < 10; ++i)
+ {
+ data.push_back(mpt::byte_cast<std::byte>(static_cast<uint8>(i)));
+ }
+ {
+ IniFileSettingsBackend inifile{filename, CachePolicy::Cached};
+ inifile.WriteMultipleSettings({{SettingPath(U_("Test"), U_("Data")), data}});
+ }
+ {
+ IniFileSettingsBackend inifile{filename, CachePolicy::Cached};
+ VERIFY_EQUAL(inifile.ReadSetting(SettingPath(U_("Test"), U_("Data")), SettingValue(data)).as<std::vector<std::byte>>(), data);
+ }
+ DeleteFile(mpt::support_long_path(filename.AsNative()).c_str());
+ }
+
#endif // MODPLUG_TRACKER
}
|
|
|
Full "read all settings, write all settings" cycle is down from ~1300ms to ~10ms with the attached patch. speedup-ini-settings-v8-wip.patch (38,324 bytes)
Index: mptrack/Mptrack.h
===================================================================
--- mptrack/Mptrack.h (revision 24610)
+++ mptrack/Mptrack.h (working copy)
@@ -35,6 +35,7 @@
class TrackerSettings;
class ImmediateWindowsIniFileSettingsBackend;
class BatchedWindowsIniFileSettingsBackend;
+class CachedIniFileSettingsBackend;
template <typename Backend> class FileSettingsContainer;
#ifndef IniFileSettingsBackend
#define IniFileSettingsBackend ImmediateWindowsIniFileSettingsBackend
Index: mptrack/SettingsIni.cpp
===================================================================
--- mptrack/SettingsIni.cpp (revision 24610)
+++ mptrack/SettingsIni.cpp (working copy)
@@ -13,25 +13,44 @@
#include "SettingsIni.h"
+#include "mpt/base/integer.hpp"
#include "mpt/binary/hex.hpp"
#include "mpt/io_file/fstream.hpp"
#include "mpt/io_file/outputfile.hpp"
#include "mpt/parse/parse.hpp"
+#include "mpt/base/bit.hpp"
+#include "mpt/io_read/filecursor.hpp"
+#include "mpt/io_read/filecursor_memory.hpp"
+#include "mpt/io_read/filereader.hpp"
+#include "mpt/format/message_macros.hpp"
+#include "mpt/string/types.hpp"
+#include "mpt/string_transcode/transcode.hpp"
+
#include "../common/misc_util.h"
#include "../common/mptFileIO.h"
#include "../common/mptStringBuffer.h"
#include <algorithm>
+#include <list>
+#include <map>
+#include <optional>
+#include <set>
+#include <string>
+#include <string_view>
#include <utility>
+#include <vector>
+#include <cstring>
+
+
OPENMPT_NAMESPACE_BEGIN
-WindowsIniFileBase::WindowsIniFileBase(mpt::PathString filename_)
+IniFileBase::IniFileBase(mpt::PathString filename_)
: filename(std::move(filename_))
, file(filename)
{
@@ -38,6 +57,24 @@
return;
}
+const mpt::PathString &IniFileBase::Filename() const
+{
+ return filename;
+}
+
+mpt::PathString IniFileBase::GetFilename() const
+{
+ return filename;
+}
+
+
+
+WindowsIniFileBase::WindowsIniFileBase(mpt::PathString filename_)
+ : IniFileBase(std::move(filename_))
+{
+ return;
+}
+
void WindowsIniFileBase::ConvertToUnicode(const mpt::ustring &backupTag)
{
// Force ini file to be encoded in UTF16.
@@ -80,18 +117,86 @@
#endif
}
-const mpt::PathString &WindowsIniFileBase::Filename() const
+
+
+IniProbeResult IniFileHelpers::Probe(mpt::const_byte_span data)
{
- return filename;
+ IniProbeResult result;
+ const std::array<std::byte, 4> bom_utf32be = {mpt::byte_cast<std::byte>(uint8{0x00}), mpt::byte_cast<std::byte>(uint8{0x00}), mpt::byte_cast<std::byte>(uint8{0xfe}), mpt::byte_cast<std::byte>(uint8{0xff})};
+ const std::array<std::byte, 4> bom_utf32le = {mpt::byte_cast<std::byte>(uint8{0xff}), mpt::byte_cast<std::byte>(uint8{0xfe}), mpt::byte_cast<std::byte>(uint8{0x00}), mpt::byte_cast<std::byte>(uint8{0x00})};
+ const std::array<std::byte, 2> bom_utf16be = {mpt::byte_cast<std::byte>(uint8{0xfe}), mpt::byte_cast<std::byte>(uint8{0xff})};
+ const std::array<std::byte, 2> bom_utf16le = {mpt::byte_cast<std::byte>(uint8{0xff}), mpt::byte_cast<std::byte>(uint8{0xfe})};
+ const std::array<std::byte, 3> bom_utf8 = {mpt::byte_cast<std::byte>(uint8{0xEF}), mpt::byte_cast<std::byte>(uint8{0xBB}), mpt::byte_cast<std::byte>(uint8{0xBF})};
+ if((data.size() >= bom_utf32be.size()) && (std::memcmp(data.data(), bom_utf32be.data(), bom_utf32be.size()) == 0))
+ {
+ result.encoding = IniEncoding::UTF32BE;
+ result.data_offset = 4;
+ } else if((data.size() >= bom_utf32le.size()) && (std::memcmp(data.data(), bom_utf32le.data(), bom_utf32le.size()) == 0))
+ {
+ result.encoding = IniEncoding::UTF32LE;
+ result.data_offset = 4;
+ } else if((data.size() >= bom_utf16be.size()) && (std::memcmp(data.data(), bom_utf16be.data(), bom_utf16be.size()) == 0))
+ {
+ result.encoding = IniEncoding::UTF16BE;
+ result.data_offset = 2;
+ } else if((data.size() >= bom_utf16le.size()) && (std::memcmp(data.data(), bom_utf16le.data(), bom_utf16le.size()) == 0))
+ {
+ result.encoding = IniEncoding::UTF16LE;
+ result.data_offset = 2;
+ } else if((data.size() >= bom_utf8.size()) && (std::memcmp(data.data(), bom_utf8.data(), bom_utf8.size()) == 0))
+ {
+ result.encoding = IniEncoding::UTF8;
+ result.data_offset = 3;
+ } else
+ {
+ result.encoding = IniEncoding::ANSI;
+ result.data_offset = 0;
+ }
+
+ return result;
}
-mpt::PathString WindowsIniFileBase::GetFilename() const
+std::list<std::pair<SettingPath, SettingValue>> IniFileHelpers::CreateIniHeader(IniVersion version)
{
- return filename;
+ std::list<std::pair<SettingPath, SettingValue>> result;
+ switch(version.major)
+ {
+ case 1:
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!Format")}, U_("com.microsoft.fileformat.ini")));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!Encoding")}, U_("ANSI")));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!CaseSensitive")}, static_cast<int32>(0)));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!VersionMajor")}, static_cast<int32>(version.major)));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!VersionMinor")}, static_cast<int32>(version.minor)));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!VersionPatch")}, static_cast<int32>(version.patch)));
+ break;
+ case 2:
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!Format")}, U_("com.microsoft.fileformat.ini")));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!Encoding")}, U_("UTF-16LE")));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!CaseSensitive")}, static_cast<int32>(0)));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!VersionMajor")}, static_cast<int32>(version.major)));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!VersionMinor")}, static_cast<int32>(version.minor)));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!VersionPatch")}, static_cast<int32>(version.patch)));
+ break;
+ case 3:
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!Format")}, U_("org.winehq.fileformat.ini")));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!Encoding")}, U_("UTF-8")));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!CaseSensitive")}, static_cast<int32>(0)));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!VersionMajor")}, static_cast<int32>(version.major)));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!VersionMinor")}, static_cast<int32>(version.minor)));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!VersionPatch")}, static_cast<int32>(version.patch)));
+ break;
+ case 4:
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!Format")}, U_("org.openmpt.fileformat.ini")));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!Encoding")}, U_("UTF-8")));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!CaseSensitive")}, static_cast<int32>(1)));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!VersionMajor")}, static_cast<int32>(version.major)));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!VersionMinor")}, static_cast<int32>(version.minor)));
+ result.emplace_back(std::make_pair(SettingPath{U_("!Type"), U_("!VersionPatch")}, static_cast<int32>(version.patch)));
+ break;
+ }
+ return result;
}
-
-
mpt::winstring IniFileHelpers::GetSection(const SettingPath &path)
{
return mpt::ToWin(path.GetSection());
@@ -558,4 +663,695 @@
+static inline bool is_hex(mpt::uchar c)
+{
+ return (MPT_UCHAR('0') <= c && c <= MPT_UCHAR('9')) || (MPT_UCHAR('a') <= c && c <= MPT_UCHAR('f')) || (MPT_UCHAR('A') <= c && c <= MPT_UCHAR('F'));
+}
+
+static mpt::ustring escape(mpt::ustring_view text)
+{
+ mpt::ustring result;
+ const std::size_t len = text.length();
+ result.reserve(len);
+ for(std::size_t pos = 0; pos < len; ++pos)
+ {
+ const mpt::uchar c = text[pos];
+ if((c == MPT_UCHAR('^')) || (c == MPT_UCHAR(';')) || (c == MPT_UCHAR('[')) || (c == MPT_UCHAR(']')) || (c == MPT_UCHAR('=')) || (c == MPT_UCHAR('\"')))
+ {
+ result.push_back(MPT_UCHAR('^'));
+ result.push_back(c);
+ } else if(mpt::char_value(c) < 32)
+ {
+ result.push_back(MPT_UCHAR('^'));
+ result.push_back(MPT_UCHAR('x'));
+ result.append(mpt::ufmt::hex0<2>(mpt::char_value(c)));
+ } else if((0x80 <= mpt::char_value(c)) && (mpt::char_value(c) <= 0x9f))
+ {
+ result.push_back(MPT_UCHAR('^'));
+ result.push_back(MPT_UCHAR('x'));
+ result.append(mpt::ufmt::hex0<2>(mpt::char_value(c)));
+ } else
+ {
+ result.push_back(c);
+ }
+ }
+ return result;
+}
+
+static mpt::ustring unescape(mpt::ustring_view text)
+{
+ mpt::ustring result;
+ const std::size_t len = text.length();
+ result.reserve(len);
+ for(std::size_t pos = 0; pos < len; ++pos)
+ {
+ std::size_t rem = len - pos;
+ if(text[pos] == MPT_UCHAR('^'))
+ {
+ if((rem >= 2) && (text[pos+1] == MPT_UCHAR('^')))
+ {
+ result.push_back(MPT_UCHAR('^'));
+ pos += 1;
+ } else if((rem >= 2) && (text[pos+1] == MPT_UCHAR(';')))
+ {
+ result.push_back(MPT_UCHAR(';'));
+ pos += 1;
+ } else if((rem >= 2) && (text[pos+1] == MPT_UCHAR('[')))
+ {
+ result.push_back(MPT_UCHAR('['));
+ pos += 1;
+ } else if((rem >= 2) && (text[pos+1] == MPT_UCHAR(']')))
+ {
+ result.push_back(MPT_UCHAR(']'));
+ pos += 1;
+ } else if((rem >= 2) && (text[pos+1] == MPT_UCHAR('=')))
+ {
+ result.push_back(MPT_UCHAR('='));
+ pos += 1;
+ } else if((rem >= 2) && (text[pos+1] == MPT_UCHAR('\"')))
+ {
+ result.push_back(MPT_UCHAR('\"'));
+ pos += 1;
+ } else if((rem >= 4) && (text[pos+1] == MPT_UCHAR('x')) && is_hex(text[pos+2]) && is_hex(text[pos+3]))
+ {
+ result.push_back(static_cast<mpt::uchar>(mpt::parse_hex<uint8>(mpt::ustring{text.substr(pos + 2, 2)})));
+ pos += 3;
+ } else if((rem >= 6) && (text[pos+1] == MPT_UCHAR('u')) && is_hex(text[pos+2]) && is_hex(text[pos+3]) && is_hex(text[pos+4]) && is_hex(text[pos+5]))
+ {
+ result.append(mpt::transcode<mpt::ustring>(std::u16string(1, static_cast<char16_t>(mpt::parse_hex<uint16>(mpt::ustring{text.substr(pos + 2, 4)})))));
+ pos += 5;
+ } else if((rem >= 10) && (text[pos+1] == MPT_UCHAR('U')) && is_hex(text[pos+2]) && is_hex(text[pos+3]) && is_hex(text[pos+4]) && is_hex(text[pos+5]) && is_hex(text[pos+6]) && is_hex(text[pos+7]) && is_hex(text[pos+8]) && is_hex(text[pos+9]))
+ {
+ result.append(mpt::transcode<mpt::ustring>(std::u32string(1, static_cast<char32_t>(mpt::parse_hex<uint32>(mpt::ustring{text.substr(pos + 2, 8)})))));
+ pos += 9;
+ } else
+ {
+ result.push_back(text[pos]);
+ }
+ } else
+ {
+ result.push_back(text[pos]);
+ }
+ }
+ return result;
+}
+
+static inline std::size_t find_unescaped_first(mpt::ustring_view text, mpt::uchar c)
+{
+ const std::size_t len = text.length();
+ for(std::size_t pos = 0; pos < len; ++pos)
+ {
+ if(text[pos] == c)
+ {
+ return pos;
+ }
+ std::size_t rem = len - pos;
+ if(text[pos] == MPT_UCHAR('^'))
+ {
+ if((rem >= 2) && (text[pos+1] == MPT_UCHAR('^')))
+ {
+ pos += 1;
+ } else if((rem >= 2) && (text[pos+1] == MPT_UCHAR(';')))
+ {
+ pos += 1;
+ } else if((rem >= 2) && (text[pos+1] == MPT_UCHAR('[')))
+ {
+ pos += 1;
+ } else if((rem >= 2) && (text[pos+1] == MPT_UCHAR(']')))
+ {
+ pos += 1;
+ } else if((rem >= 2) && (text[pos+1] == MPT_UCHAR('=')))
+ {
+ pos += 1;
+ } else if((rem >= 2) && (text[pos+1] == MPT_UCHAR('\"')))
+ {
+ pos += 1;
+ } else if((rem >= 4) && (text[pos+1] == MPT_UCHAR('x')) && is_hex(text[pos+2]) && is_hex(text[pos+3]))
+ {
+ pos += 3;
+ } else if((rem >= 6) && (text[pos+1] == MPT_UCHAR('u')) && is_hex(text[pos+2]) && is_hex(text[pos+3]) && is_hex(text[pos+4]) && is_hex(text[pos+5]))
+ {
+ pos += 5;
+ } else if((rem >= 10) && (text[pos+1] == MPT_UCHAR('U')) && is_hex(text[pos+2]) && is_hex(text[pos+3]) && is_hex(text[pos+4]) && is_hex(text[pos+5]) && is_hex(text[pos+6]) && is_hex(text[pos+7]) && is_hex(text[pos+8]) && is_hex(text[pos+9]))
+ {
+ pos += 9;
+ }
+ }
+ }
+ return mpt::ustring_view::npos;
+}
+
+static inline std::size_t find_unescaped_last(mpt::ustring_view text, mpt::uchar c)
+{
+ std::size_t result = mpt::ustring_view::npos;
+ const std::size_t len = text.length();
+ for(std::size_t pos = 0; pos < len; ++pos)
+ {
+ if(text[pos] == c)
+ {
+ result = pos;
+ }
+ std::size_t rem = len - pos;
+ if(text[pos] == MPT_UCHAR('^'))
+ {
+ if((rem >= 2) && (text[pos+1] == MPT_UCHAR('^')))
+ {
+ pos += 1;
+ } else if((rem >= 2) && (text[pos+1] == MPT_UCHAR(';')))
+ {
+ pos += 1;
+ } else if((rem >= 2) && (text[pos+1] == MPT_UCHAR('[')))
+ {
+ pos += 1;
+ } else if((rem >= 2) && (text[pos+1] == MPT_UCHAR(']')))
+ {
+ pos += 1;
+ } else if((rem >= 2) && (text[pos+1] == MPT_UCHAR('=')))
+ {
+ pos += 1;
+ } else if((rem >= 2) && (text[pos+1] == MPT_UCHAR('\"')))
+ {
+ pos += 1;
+ } else if((rem >= 4) && (text[pos+1] == MPT_UCHAR('x')) && is_hex(text[pos+2]) && is_hex(text[pos+3]))
+ {
+ pos += 3;
+ } else if((rem >= 6) && (text[pos+1] == MPT_UCHAR('u')) && is_hex(text[pos+2]) && is_hex(text[pos+3]) && is_hex(text[pos+4]) && is_hex(text[pos+5]))
+ {
+ pos += 5;
+ } else if((rem >= 10) && (text[pos+1] == MPT_UCHAR('U')) && is_hex(text[pos+2]) && is_hex(text[pos+3]) && is_hex(text[pos+4]) && is_hex(text[pos+5]) && is_hex(text[pos+6]) && is_hex(text[pos+7]) && is_hex(text[pos+8]) && is_hex(text[pos+9]))
+ {
+ pos += 9;
+ }
+ }
+ }
+ return result;
+}
+
+static std::vector<mpt::ustring> split_any_line_ending(mpt::ustring_view text)
+{
+ std::vector<mpt::ustring> result;
+ const std::size_t len = text.length();
+ std::size_t beg = 0;
+ for(std::size_t pos = 0; pos < len; ++pos)
+ {
+ std::size_t rem = len - pos;
+ if((rem >= 2) && (text[pos+0] == MPT_UCHAR('\r')) && (text[pos+1] == MPT_UCHAR('\n')))
+ {
+ result.emplace_back(text.substr(beg, pos - beg));
+ pos += 2;
+ beg = pos;
+ } else if((rem >=1) && text[pos+0] == MPT_UCHAR('\n'))
+ {
+ result.emplace_back(text.substr(beg, pos - beg));
+ pos += 1;
+ beg = pos;
+ } else if((rem >=1) && text[pos+0] == MPT_UCHAR('\r'))
+ {
+ result.emplace_back(text.substr(beg, pos - beg));
+ pos += 1;
+ beg = pos;
+ }
+ }
+ if(!result.empty() || ((len - beg) > 0))
+ {
+ result.emplace_back(text.substr(beg, len - beg));
+ }
+ return result;
+}
+
+static inline mpt::ustring_view trim_whitespace(mpt::ustring_view text)
+{
+ mpt::ustring_view::size_type beg_pos = text.find_first_not_of(MPT_ULITERAL(" \t\r\n"));
+ if(beg_pos != mpt::ustring_view::npos)
+ {
+ text = text.substr(beg_pos);
+ } else
+ {
+ text = text.substr(text.length(), 0);
+ }
+ mpt::ustring_view::size_type end_pos = text.find_last_not_of(MPT_ULITERAL(" \t\r\n"));
+ if(end_pos != mpt::ustring_view::npos)
+ {
+ text = text.substr(0, end_pos + 1);
+ } else
+ {
+ text = text.substr(0, 0);
+ }
+ return text;
+}
+
+static inline mpt::ustring_view remove_quotes(mpt::ustring_view text)
+{
+ if(text.length() <= 1)
+ {
+ return text;
+ }
+ mpt::ustring_view::size_type beg_quote = find_unescaped_first(text, MPT_UCHAR('\"'));
+ mpt::ustring_view::size_type end_quote = find_unescaped_last(text, MPT_UCHAR('\"'));
+ if((beg_quote == 0) && (end_quote == (text.length() - 1)))
+ {
+ return text.substr(1, text.length() - 2);
+ }
+ return text;
+}
+
+void CachedIniFileSettingsBackend::ReadFileIntoCache()
+{
+ cache.clear();
+ const std::vector<std::byte> filedataraw = file.read();
+ IniProbeResult probe = Probe(mpt::as_span(filedataraw));
+ const mpt::span filedata = mpt::as_span(filedataraw).subspan(probe.data_offset);
+ mpt::ustring filetext;
+ switch(probe.encoding)
+ {
+ case IniEncoding::UTF32BE:
+ {
+ if((filedata.size() % sizeof(char32_t)) != 0)
+ {
+ MPT_LOG_GLOBAL(LogError, "Settings", MPT_UFORMAT_MESSAGE("{}: {}")(filename, U_("UTF32 encoding detected, but file size is not a multiple of 4.")));
+ }
+ MPT_MAYBE_CONSTANT_IF(mpt::endian_is_big())
+ {
+ std::u32string utf32data;
+ std::size_t count = filedata.size() / sizeof(char32_t);
+ utf32data.resize(count);
+ std::memcpy(utf32data.data(), filedata.data(), count * sizeof(char32_t));
+ filetext = mpt::transcode<mpt::ustring>(utf32data);
+ } else
+ {
+ auto fc = mpt::IO::make_FileCursor<mpt::PathString>(filedata);
+ std::u32string utf32data;
+ utf32data.reserve(fc.GetLength() / sizeof(char32_t));
+ while(!fc.EndOfFile())
+ {
+ utf32data.push_back(static_cast<char32_t>(mpt::IO::FileReader::ReadInt32BE(fc)));
+ }
+ filetext = mpt::transcode<mpt::ustring>(utf32data);
+ }
+ }
+ break;
+ case IniEncoding::UTF32LE:
+ {
+ if((filedata.size() % sizeof(char32_t)) != 0)
+ {
+ MPT_LOG_GLOBAL(LogError, "Settings", MPT_UFORMAT_MESSAGE("{}: {}")(filename, U_("UTF32 encoding detected, but file size is not a multiple of 4.")));
+ }
+ MPT_MAYBE_CONSTANT_IF(mpt::endian_is_little())
+ {
+ std::u32string utf32data;
+ std::size_t count = filedata.size() / sizeof(char32_t);
+ utf32data.resize(count);
+ std::memcpy(utf32data.data(), filedata.data(), count * sizeof(char32_t));
+ filetext = mpt::transcode<mpt::ustring>(utf32data);
+ } else
+ {
+ auto fc = mpt::IO::make_FileCursor<mpt::PathString>(filedata);
+ std::u32string utf32data;
+ utf32data.reserve(fc.GetLength() / sizeof(char32_t));
+ while(!fc.EndOfFile())
+ {
+ utf32data.push_back(static_cast<char32_t>(mpt::IO::FileReader::ReadInt32LE(fc)));
+ }
+ filetext = mpt::transcode<mpt::ustring>(utf32data);
+ }
+ }
+ break;
+ case IniEncoding::UTF16BE:
+ {
+ if((filedata.size() % sizeof(char16_t)) != 0)
+ {
+ MPT_LOG_GLOBAL(LogError, "Settings", MPT_UFORMAT_MESSAGE("{}: {}")(filename, U_("UTF16 encoding detected, but file size is not a multiple of 2.")));
+ }
+ MPT_MAYBE_CONSTANT_IF(mpt::endian_is_big())
+ {
+ std::u16string utf16data;
+ std::size_t count = filedata.size() / sizeof(char16_t);
+ utf16data.resize(count);
+ std::memcpy(utf16data.data(), filedata.data(), count * sizeof(char16_t));
+ filetext = mpt::transcode<mpt::ustring>(utf16data);
+ } else
+ {
+ auto fc = mpt::IO::make_FileCursor<mpt::PathString>(filedata);
+ std::u16string utf16data;
+ utf16data.reserve(fc.GetLength() / sizeof(char16_t));
+ while(!fc.EndOfFile())
+ {
+ utf16data.push_back(static_cast<char16_t>(mpt::IO::FileReader::ReadInt16BE(fc)));
+ }
+ filetext = mpt::transcode<mpt::ustring>(utf16data);
+ }
+ }
+ break;
+ case IniEncoding::UTF16LE:
+ {
+ if((filedata.size() % sizeof(char16_t)) != 0)
+ {
+ MPT_LOG_GLOBAL(LogError, "Settings", MPT_UFORMAT_MESSAGE("{}: {}")(filename, U_("UTF16 encoding detected, but file size is not a multiple of 2.")));
+ }
+ MPT_MAYBE_CONSTANT_IF(MPT_OS_WINDOWS)
+ {
+ std::wstring utf16data;
+ std::size_t count = filedata.size() / sizeof(wchar_t);
+ utf16data.resize(count);
+ std::memcpy(utf16data.data(), filedata.data(), count * sizeof(wchar_t));
+ filetext = mpt::transcode<mpt::ustring>(utf16data);
+ } else MPT_MAYBE_CONSTANT_IF(mpt::endian_is_little())
+ {
+ std::u16string utf16data;
+ std::size_t count = filedata.size() / sizeof(char16_t);
+ utf16data.resize(count);
+ std::memcpy(utf16data.data(), filedata.data(), count * sizeof(char16_t));
+ filetext = mpt::transcode<mpt::ustring>(utf16data);
+ } else
+ {
+ auto fc = mpt::IO::make_FileCursor<mpt::PathString>(filedata);
+ std::u16string utf16data;
+ utf16data.reserve(fc.GetLength() / sizeof(char16_t));
+ while(!fc.EndOfFile())
+ {
+ utf16data.push_back(static_cast<char16_t>(mpt::IO::FileReader::ReadInt16LE(fc)));
+ }
+ filetext = mpt::transcode<mpt::ustring>(utf16data);
+ }
+ }
+ break;
+ case IniEncoding::UTF8:
+ filetext = mpt::transcode<mpt::ustring>(mpt::common_encoding::utf8, mpt::buffer_cast<std::string>(filedata));
+ break;
+ case IniEncoding::ANSI:
+ filetext = mpt::transcode<mpt::ustring>(mpt::logical_encoding::locale, mpt::buffer_cast<std::string>(filedata));
+ break;
+ }
+#if 0
+ mpt::ustring lineending;
+ switch(probe.lineending)
+ {
+ case IniLineEnding::CRLF:
+ lineending = MPT_USTRING("\r\n");
+ break;
+ case IniLineEnding::LF:
+ lineending = MPT_USTRING("\n");
+ break;
+ case IniLineEnding::CR:
+ lineending = MPT_USTRING("\r");
+ break;
+ }
+ const std::vector<mpt::ustring> lines = mpt::split<mpt::ustring>(filetext, lineending);
+#else
+ const std::vector<mpt::ustring> lines = split_any_line_ending(filetext);
+#endif
+ mpt::ustring current_section;
+ mpt::ustring last_key;
+ bool empty_line_after_last_key = false;
+ std::vector<mpt::ustring> running_comments;
+ std::size_t line_number = 0;
+ bool last_line_empty = false;
+ auto store_previous_comments = [&]()
+ {
+ if(!empty_line_after_last_key && (running_comments.size() > 0))
+ {
+ comments[std::make_pair(current_section, last_key)].after = std::move(running_comments);
+ running_comments = {};
+ }
+ };
+ for(const auto &raw_line : lines)
+ {
+ line_number += 1;
+ last_line_empty = (raw_line.length() == 0);
+ mpt::ustring_view line = trim_whitespace(raw_line);
+ std::size_t line_len = line.length();
+ if(line_len == 0)
+ { // empty line
+ store_previous_comments();
+ empty_line_after_last_key = true;
+ continue;
+ }
+ if(find_unescaped_first(line, MPT_UCHAR(';')) == 0)
+ { // comment
+ running_comments.push_back(mpt::ustring{trim_whitespace(line.substr(1))});
+ continue;
+ }
+ if(find_unescaped_first(line, MPT_UCHAR('[')) == 0)
+ { // section start
+ mpt::ustring_view::size_type opening_pos = 0;
+ mpt::ustring_view::size_type closing_pos = find_unescaped_first(line, MPT_UCHAR(']'));
+ if(closing_pos == mpt::ustring_view::npos)
+ {
+ store_previous_comments();
+ empty_line_after_last_key = true;
+ MPT_LOG_GLOBAL(LogError, "Settings", MPT_UFORMAT_MESSAGE("{}({}): No closing bracket in section header. ('{}').")(filename, line_number, raw_line));
+ continue;
+ }
+ mpt::ustring_view remainder = line.substr(closing_pos + 1);
+ if(trim_whitespace(remainder).length() > 0)
+ {
+ MPT_LOG_GLOBAL(LogWarning, "Settings", MPT_UFORMAT_MESSAGE("{}({}): Junk after section header. ('{}').")(filename, line_number, raw_line));
+ }
+ mpt::ustring_view escaped_section = trim_whitespace(line.substr(opening_pos + 1, closing_pos - opening_pos - 1));
+ mpt::ustring section = unescape(escaped_section);
+ if(cache.find(section) != cache.end())
+ {
+ store_previous_comments();
+ empty_line_after_last_key = true;
+ current_section.clear();
+ MPT_LOG_GLOBAL(LogWarning, "Settings", MPT_UFORMAT_MESSAGE("{}({}): Ignoring duplicate section '{}'. ('{}').")(filename, line_number, escape(section), raw_line));
+ continue;
+ }
+ MPT_LOG_GLOBAL(LogInformation, "Settings", MPT_UFORMAT_MESSAGE("{}({}): [{}]")(filename, line_number, escape(section)));
+ comments[std::make_pair(section, mpt::ustring{})].before = std::move(running_comments);
+ running_comments = {};
+ cache[section].emplace();
+ current_section = std::move(section);
+ last_key.clear();
+ empty_line_after_last_key = false;
+ continue;
+ }
+ mpt::ustring_view::size_type equal_pos = find_unescaped_first(line, MPT_UCHAR('='));
+ if(equal_pos == mpt::ustring_view::npos)
+ { // syntax error
+ store_previous_comments();
+ empty_line_after_last_key = true;
+ MPT_LOG_GLOBAL(LogError, "Settings", MPT_UFORMAT_MESSAGE("{}({}): Syntax error: Invalid token. ('{}').")(filename, line_number, raw_line));
+ continue;
+ }
+ // key value
+ if(current_section.empty())
+ {
+ store_previous_comments();
+ empty_line_after_last_key = true;
+ MPT_LOG_GLOBAL(LogWarning, "Settings", MPT_UFORMAT_MESSAGE("{}({}): Ignoring key-value without section. ('{}').")(filename, line_number, raw_line));
+ continue;
+ }
+ mpt::ustring_view escaped_key = trim_whitespace(line.substr(0, equal_pos));
+ mpt::ustring_view escaped_val = remove_quotes(trim_whitespace(line.substr(equal_pos + 1)));
+ if(escaped_key.empty())
+ {
+ store_previous_comments();
+ empty_line_after_last_key = true;
+ MPT_LOG_GLOBAL(LogError, "Settings", MPT_UFORMAT_MESSAGE("{}({}): Syntax error: Empty key. ('{}').")(filename, line_number, raw_line));
+ continue;
+ }
+ mpt::ustring key = unescape(escaped_key);
+ mpt::ustring val = unescape(escaped_val);
+ if(cache[current_section].value().find(key) != cache[current_section].value().end())
+ {
+ store_previous_comments();
+ empty_line_after_last_key = true;
+ MPT_LOG_GLOBAL(LogWarning, "Settings", MPT_UFORMAT_MESSAGE("{}({}): Ignoring duplicate key '{}'. ('{}').")(filename, line_number, escape(key), raw_line));
+ continue;
+ }
+ MPT_LOG_GLOBAL(LogInformation, "Settings", MPT_UFORMAT_MESSAGE("{}({}): {}={}")(filename, line_number, escape(key), escape(val)));
+ comments[std::make_pair(current_section, key)].before = std::move(running_comments);
+ running_comments = {};
+ cache[current_section].value()[key] = std::make_optional(std::move(val));
+ last_key = std::move(key);
+ empty_line_after_last_key = false;
+ }
+ store_previous_comments();
+ if(!last_line_empty)
+ {
+ MPT_LOG_GLOBAL(LogNotification, "Settings", MPT_UFORMAT_MESSAGE("{}({}): {}")(filename, line_number, U_("No newline after last line.")));
+ }
+}
+
+void CachedIniFileSettingsBackend::MergeSettingsIntoCache(const std::set<mpt::ustring> &removeSections, const std::map<SettingPath, std::optional<SettingValue>> &settings)
+{
+ for(const auto §ion : removeSections)
+ {
+ cache.erase(section);
+ }
+ std::map<mpt::ustring, std::map<SettingPath, std::optional<SettingValue>>> sectionssettings;
+ for(const auto &[path, value] : settings)
+ {
+ sectionssettings[path.GetRefSection()][path] = value;
+ }
+ for(const auto &[section, sectionsettings] : sectionssettings)
+ {
+ if(!cache[section].has_value())
+ {
+ cache[section].emplace();
+ }
+ std::map<mpt::ustring, std::optional<mpt::ustring>> &workingsectionsettings = cache[section].value();
+ for(const auto &[path, value] : sectionsettings)
+ {
+ if(value.has_value())
+ {
+ workingsectionsettings[path.GetRefKey()] = FormatValueAsIni(value.value());
+ } else
+ {
+ workingsectionsettings.erase(path.GetRefKey());
+ }
+ }
+ }
+}
+
+void CachedIniFileSettingsBackend::WriteCacheIntoFile()
+{
+ mpt::ustring filetext;
+ const std::list<std::pair<SettingPath, SettingValue>> header = CreateIniHeader(IniVersion{4, 0, 0});
+ for(const auto &path : header)
+ {
+ cache.erase(path.first.GetRefSection());
+ }
+ const mpt::ustring newline = MPT_USTRING("\r\n");
+ mpt::ustring last_section;
+ for(const auto &[path, value] : header)
+ {
+ if(path.GetRefSection() != last_section)
+ {
+ last_section = path.GetRefSection();
+ filetext += MPT_UFORMAT_MESSAGE("[{}]{}")(escape(path.GetRefSection()), newline);
+ }
+ filetext += MPT_UFORMAT_MESSAGE("{}={}{}")(escape(path.GetRefKey()), escape(FormatValueAsIni(value)), newline);
+ }
+ for(const auto &[section, sectionsettings] : cache)
+ {
+ filetext += newline;
+ auto sectioncomments = comments.find(std::make_pair(section, mpt::ustring{}));
+ if(sectioncomments != comments.end())
+ {
+ for(const auto &comment : sectioncomments->second.before)
+ {
+ filetext += MPT_UFORMAT_MESSAGE(";{}{}")(comment, newline);
+ }
+ }
+ filetext += MPT_UFORMAT_MESSAGE("[{}]{}")(escape(section), newline);
+ if(sectioncomments != comments.end())
+ {
+ for(const auto &comment : sectioncomments->second.after)
+ {
+ filetext += MPT_UFORMAT_MESSAGE(";{}{}")(comment, newline);
+ }
+ if(sectioncomments->second.after.size() > 0)
+ {
+ filetext += newline;
+ }
+ }
+ if(sectionsettings.has_value())
+ {
+ for(const auto &[key, value] : sectionsettings.value())
+ {
+ if(value.has_value())
+ {
+ auto keycomments = comments.find(std::make_pair(section, key));
+ if(keycomments != comments.end())
+ {
+ for(const auto &comment : keycomments->second.before)
+ {
+ filetext += MPT_UFORMAT_MESSAGE(";{}{}")(comment, newline);
+ }
+ }
+ const mpt::ustring escaped_value = escape(value.value());
+ if(escaped_value != trim_whitespace(escaped_value))
+ {
+ filetext += MPT_UFORMAT_MESSAGE("{}=\"{}\"{}")(escape(key), escaped_value, newline);
+ } else
+ {
+ filetext += MPT_UFORMAT_MESSAGE("{}={}{}")(escape(key), escaped_value, newline);
+ }
+ if(keycomments != comments.end())
+ {
+ for(const auto &comment : keycomments->second.after)
+ {
+ filetext += MPT_UFORMAT_MESSAGE(";{}{}")(comment, newline);
+ }
+ if(keycomments->second.after.size() > 0)
+ {
+ filetext += newline;
+ }
+ }
+ }
+ }
+ }
+ }
+ const std::array<std::byte, 3> utf8_bom = {mpt::byte_cast<std::byte>(uint8{0xef}), mpt::byte_cast<std::byte>(uint8{0xbb}), mpt::byte_cast<std::byte>(uint8{0xbf})};
+ std::vector<std::byte> filedata;
+ mpt::append(filedata, utf8_bom);
+ mpt::append(filedata, mpt::byte_cast<mpt::const_byte_span>(mpt::as_span(mpt::transcode<std::string>(mpt::common_encoding::utf8, filetext))));
+ file.write(filedata);
+}
+
+void CachedIniFileSettingsBackend::InvalidateCache()
+{
+ OPENMPT_PROFILE_FUNCTION(Profiler::Settings);
+ std::lock_guard l(file);
+ ReadFileIntoCache();
+}
+
+SettingValue CachedIniFileSettingsBackend::ReadSetting(const SettingPath &path, const SettingValue &def) const
+{
+ OPENMPT_PROFILE_FUNCTION(Profiler::Settings);
+ const auto sectionit = cache.find(path.GetRefSection());
+ if(sectionit != cache.end())
+ {
+ if(sectionit->second.has_value())
+ {
+ const std::map<mpt::ustring, std::optional<mpt::ustring>> §ion = sectionit->second.value();
+ const auto it = section.find(path.GetRefKey());
+ if(it != section.end())
+ {
+ if(it->second.has_value())
+ {
+ return ParseValueFromIni(it->second.value(), def);
+ } else
+ {
+ return def;
+ }
+ } else
+ {
+ return def;
+ }
+ } else
+ {
+ return def;
+ }
+ } else
+ {
+ return def;
+ }
+}
+
+void CachedIniFileSettingsBackend::WriteAllSettings(const std::set<mpt::ustring> &removeSections, const std::map<SettingPath, std::optional<SettingValue>> &settings)
+{
+ OPENMPT_PROFILE_FUNCTION(Profiler::Settings);
+ std::lock_guard l(file);
+ ReadFileIntoCache();
+ MergeSettingsIntoCache(removeSections, settings);
+ WriteCacheIntoFile();
+}
+
+CachedIniFileSettingsBackend::CachedIniFileSettingsBackend(mpt::PathString filename_)
+ : IniFileBase(std::move(filename_))
+{
+ OPENMPT_PROFILE_FUNCTION(Profiler::Settings);
+ std::lock_guard l(file);
+ ReadFileIntoCache();
+}
+
+CachedIniFileSettingsBackend::~CachedIniFileSettingsBackend()
+{
+ return;
+}
+
+
OPENMPT_NAMESPACE_END
Index: mptrack/SettingsIni.h
===================================================================
--- mptrack/SettingsIni.h (revision 24610)
+++ mptrack/SettingsIni.h (working copy)
@@ -15,11 +15,15 @@
#include "Settings.h"
+#include "mpt/base/integer.hpp"
+#include "mpt/string/types.hpp"
#include "mpt/io_file_atomic/atomic_file.hpp"
+#include <list>
#include <map>
#include <optional>
#include <set>
+#include <utility>
#include <vector>
@@ -26,9 +30,49 @@
OPENMPT_NAMESPACE_BEGIN
+struct IniVersion
+{
+ uint8 major = 0;
+ uint8 minor = 0;
+ uint8 patch = 0;
+};
+
+
+enum class IniEncoding
+{
+ UTF32BE,
+ UTF32LE,
+ UTF16BE,
+ UTF16LE,
+ UTF8,
+ ANSI,
+};
+
+
+enum class IniLineEnding
+{
+ CRLF,
+ LF,
+ CR,
+};
+
+
+struct IniProbeResult
+{
+ IniVersion version;
+ IniEncoding encoding = IniEncoding::ANSI;
+ IniLineEnding lineending = IniLineEnding::CRLF;
+ bool case_sensitive = false;
+ bool has_bom = false;
+ mpt::IO::Offset data_offset = 0;
+};
+
+
class IniFileHelpers
{
protected:
+ static IniProbeResult Probe(mpt::const_byte_span data);
+ static std::list<std::pair<SettingPath, SettingValue>> CreateIniHeader(IniVersion version);
static mpt::winstring GetSection(const SettingPath &path);
static mpt::winstring GetKey(const SettingPath &path);
static mpt::ustring FormatValueAsIni(const SettingValue &value);
@@ -36,19 +80,34 @@
};
-class WindowsIniFileBase
+// Version 1: ANSI
+// Version 2: UTF-16LE with BOM
+// Version 3: UTF8 with BOM (not supported by Windows)
+// Version 4: UTF8 without BOM and explicit version
+
+
+class IniFileBase
{
protected:
const mpt::PathString filename;
mpt::IO::atomic_shared_file_ref file;
protected:
+ IniFileBase(mpt::PathString filename_);
+ ~IniFileBase() = default;
+public:
+ const mpt::PathString &Filename() const;
+ mpt::PathString GetFilename() const;
+};
+
+
+class WindowsIniFileBase
+ : public IniFileBase
+{
+protected:
WindowsIniFileBase(mpt::PathString filename_);
~WindowsIniFileBase() = default;
public:
void ConvertToUnicode(const mpt::ustring &backupTag = mpt::ustring());
-public:
- const mpt::PathString &Filename() const;
- mpt::PathString GetFilename() const;
};
@@ -105,6 +164,36 @@
};
+class CachedIniFileSettingsBackend
+ : public ISettingsBackend<SettingsBatching::All>
+ , public IniFileBase
+ , protected IniFileHelpers
+{
+private:
+ struct comments
+ {
+ std::vector<mpt::ustring> before;
+ std::vector<mpt::ustring> after;
+ };
+private:
+ std::map<mpt::ustring, std::optional<std::map<mpt::ustring, std::optional<mpt::ustring>>>> cache;
+ std::map<std::pair<mpt::ustring, mpt::ustring>, comments> comments;
+private:
+
+private:
+ void ReadFileIntoCache();
+ void MergeSettingsIntoCache(const std::set<mpt::ustring> &removeSections, const std::map<SettingPath, std::optional<SettingValue>> &settings);
+ void WriteCacheIntoFile();
+public:
+ CachedIniFileSettingsBackend(mpt::PathString filename_);
+ ~CachedIniFileSettingsBackend() override;
+public:
+ virtual void InvalidateCache() override;
+ virtual SettingValue ReadSetting(const SettingPath &path, const SettingValue &def) const override;
+ virtual void WriteAllSettings(const std::set<mpt::ustring> &removeSections, const std::map<SettingPath, std::optional<SettingValue>> &settings) override;
+};
+
+
#ifndef IniFileSettingsBackend
#define IniFileSettingsBackend ImmediateWindowsIniFileSettingsBackend
#endif
Index: test/test.cpp
===================================================================
--- test/test.cpp (revision 24610)
+++ test/test.cpp (working copy)
@@ -3031,6 +3031,17 @@
#ifdef MODPLUG_TRACKER
+ {
+ const mpt::PathString in = P_("D:\\stuff\\in.ini");
+ const mpt::PathString out = P_("D:\\stuff\\out.ini");
+ CopyFile(in.AsNative().c_str(), out.AsNative().c_str(), FALSE);
+ {
+ FileSettingsContainer<CachedIniFileSettingsBackend> ini{out};
+ ini.Write<mpt::ustring>(U_("Dummy"), U_("Value1"), U_("\r\n"));
+ ini.Write<mpt::ustring>(U_("Dummy"), U_("Value2"), U_(" \r\n "));
+ }
+ }
+
VERIFY_EQUAL(SettingPath(U_("a"),U_("b")) < SettingPath(U_("a"),U_("c")), true);
VERIFY_EQUAL(!(SettingPath(U_("c"),U_("b")) < SettingPath(U_("a"),U_("c"))), true);
@@ -3042,6 +3053,8 @@
TestIniSettingsBackendRead<BatchedWindowsIniFileSettingsBackend>(filename);
+ TestIniSettingsBackendRead<CachedIniFileSettingsBackend>(filename);
+
{
IniFileSettingsContainer conf{filename};
int32 foobar = conf.Read(U_("Test"), U_("bar"), 23);
@@ -3134,7 +3147,7 @@
{
{
- IniFileSettingsContainer conf{filename};
+ FileSettingsContainer<ImmediateWindowsIniFileSettingsBackend> conf{filename};
conf.Write<mpt::ustring>(SettingPath(MPT_USTRING("Test"), MPT_USTRING("spacespacecharspacespace")), spacespacecharspacespace);
conf.Write<mpt::ustring>(SettingPath(MPT_USTRING("Test"), MPT_USTRING("threespaces")), threespaces);
conf.Write<mpt::ustring>(SettingPath(MPT_USTRING("Test"), MPT_USTRING("tab")), tab);
@@ -3143,7 +3156,7 @@
conf.Write<mpt::ustring>(SettingPath(MPT_USTRING("Test"), MPT_USTRING("cc0")), cc0);
}
{
- IniFileSettingsContainer conf{filename};
+ FileSettingsContainer<ImmediateWindowsIniFileSettingsBackend> conf{filename};
//VERIFY_EQUAL(conf.Read<mpt::ustring>(SettingPath(MPT_USTRING("Test"), MPT_USTRING("spacespacecharspacespace"))), spacespacecharspacespace);
//VERIFY_EQUAL(conf.Read<mpt::ustring>(SettingPath(MPT_USTRING("Test"), MPT_USTRING("threespaces"))), threespaces);
//VERIFY_EQUAL(conf.Read<mpt::ustring>(SettingPath(MPT_USTRING("Test"), MPT_USTRING("tab"))), tab);
@@ -3160,6 +3173,28 @@
DeleteFile(mpt::support_long_path(filename.AsNative()).c_str());
}
+ {
+ {
+ FileSettingsContainer<CachedIniFileSettingsBackend> conf{filename};
+ conf.Write<mpt::ustring>(SettingPath(MPT_USTRING("Test"), MPT_USTRING("spacespacecharspacespace")), spacespacecharspacespace);
+ conf.Write<mpt::ustring>(SettingPath(MPT_USTRING("Test"), MPT_USTRING("threespaces")), threespaces);
+ conf.Write<mpt::ustring>(SettingPath(MPT_USTRING("Test"), MPT_USTRING("tab")), tab);
+ conf.Write<mpt::ustring>(SettingPath(MPT_USTRING("Test"), MPT_USTRING("tokens")), tokens);
+ conf.Write<mpt::ustring>(SettingPath(MPT_USTRING("Test"), MPT_USTRING("xcrlfy")), xcrlfy);
+ conf.Write<mpt::ustring>(SettingPath(MPT_USTRING("Test"), MPT_USTRING("cc0")), cc0);
+ }
+ {
+ FileSettingsContainer<CachedIniFileSettingsBackend> conf{filename};
+ VERIFY_EQUAL(conf.Read<mpt::ustring>(SettingPath(MPT_USTRING("Test"), MPT_USTRING("spacespacecharspacespace"))), spacespacecharspacespace);
+ VERIFY_EQUAL(conf.Read<mpt::ustring>(SettingPath(MPT_USTRING("Test"), MPT_USTRING("threespaces"))), threespaces);
+ VERIFY_EQUAL(conf.Read<mpt::ustring>(SettingPath(MPT_USTRING("Test"), MPT_USTRING("tab"))), tab);
+ VERIFY_EQUAL(conf.Read<mpt::ustring>(SettingPath(MPT_USTRING("Test"), MPT_USTRING("tokens"))), tokens);
+ VERIFY_EQUAL(conf.Read<mpt::ustring>(SettingPath(MPT_USTRING("Test"), MPT_USTRING("xcrlfy"))), xcrlfy);
+ VERIFY_EQUAL(conf.Read<mpt::ustring>(SettingPath(MPT_USTRING("Test"), MPT_USTRING("cc0"))), cc0);
+ }
+ DeleteFile(mpt::support_long_path(filename.AsNative()).c_str());
+ }
+
#endif // MODPLUG_TRACKER
}
|
|
|
The patch does not replace the old INI backend, it just adds the new, but does not connect it yet.
|
|
|
|
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2022-01-30 17:48 | Saga Musix | New Issue | |
| 2022-01-30 17:51 | Saga Musix | Target Version | => OpenMPT 1.31.01.00 / libopenmpt 0.7.0 (upgrade first) |
| 2022-10-22 13:45 | manx | Assigned To | => manx |
| 2022-10-22 13:45 | manx | Status | new => acknowledged |
| 2023-04-10 08:24 | manx | Target Version | OpenMPT 1.31.01.00 / libopenmpt 0.7.0 (upgrade first) => OpenMPT 1.32.01.00 / libopenmpt 0.8.0 (upgrade first) |
| 2024-10-26 18:05 | manx | Target Version | OpenMPT 1.32.01.00 / libopenmpt 0.8.0 (upgrade first) => OpenMPT 1.33 / libopenmpt 0.9 (goals) |
| 2025-08-16 16:27 | manx | Note Added: 0006447 | |
| 2025-08-30 13:00 | Saga Musix | Note Added: 0006454 | |
| 2025-10-18 17:35 | manx | Note Added: 0006514 | |
| 2025-10-18 17:35 | manx | File Added: speedup-ini-settings-v1.patch | |
| 2025-10-19 18:35 | Saga Musix | Note Added: 0006515 | |
| 2025-10-25 18:52 | manx | Note Added: 0006517 | |
| 2025-10-25 18:52 | manx | File Added: speedup-ini-settings-v3-wip.patch | |
| 2025-12-11 19:57 | manx | Note Added: 0006533 | |
| 2025-12-11 19:57 | manx | File Added: speedup-ini-settings-v6-wip.patch | |
| 2025-12-11 20:12 | manx | Status | acknowledged => assigned |
| 2025-12-12 19:31 | manx | Note Added: 0006534 | |
| 2025-12-12 19:31 | manx | File Added: speedup-ini-settings-v8-wip.patch | |
| 2025-12-12 19:53 | manx | Note Added: 0006535 | |
| 2025-12-12 19:57 | manx | Note Added: 0006536 |