mdbx: merge branch master into stable.

This commit is contained in:
Леонид Юрьев (Leonid Yuriev) 2023-04-18 16:29:05 +03:00
commit 4151e0e348
No known key found for this signature in database
GPG Key ID: 518BD10B927E8686
16 changed files with 686 additions and 247 deletions

View File

@ -4,7 +4,51 @@ ChangeLog
English version [by Google](https://gitflic-ru.translate.goog/project/erthink/libmdbx/blob?file=ChangeLog.md&_x_tr_sl=ru&_x_tr_tl=en)
and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/gitflic.ru/project/erthink/libmdbx/blob?file=ChangeLog.md).
## v0.12.4 (Арта-333) от 2023-03-03
## v0.12.5 "Динамо" от 2023-04-18
Стабилизирующий выпуск с исправлением обнаруженных ошибок и устранением
недочетов, в день 100-летнего юбилея спортивного общества [«Динамо»](https://ru.wikipedia.org/wiki/Динамо_(спортивное_общество)).
```
16 files changed, 686 insertions(+), 247 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
```
Благодарности:
- Max <maxc0d3r@protonmail.com> за сообщение о проблеме экспорта из DSO/DLL
устаревших функций API.
- [`@calvin3721`](https://t.me/calvin3721) за сообщение о проблеме работы
`MainDB` с флагами не по-умолчанию.
Исправления:
- Поправлен экспорт из DSO/DLL устаревших функций,
которые заменены на inline в текущем API.
- Устранено использование неверного компаратора при создании или пересоздании
`MainDB` с флагами/опциями предполагающим использование специфического
компаратора (не по-умолчанию).
Мелочи:
- Удалена дублирующая диагностика внутри `node_read_bigdata()`.
- Исправлены ссылки в описании `mdbx_env_set_geometry()`.
- Добавлен отдельный тест `extra/upsert_alldups` для специфического
сценария замены/перезаписи одним значением всех multi-значений
соответствующих ключу, т.е. замена всех «дубликатов» одним значением.
- В C++ API добавлены варианты `buffer::key_from()` с явным именованием по типу данных.
- Добавлен отдельный тест `extra/maindb_ordinal` для специфического
сценария создания `MainDB` с флагами требующими использования
компаратора не по-умолчанию.
- Рефакторинг проверки "когерентности" мета-страниц.
- Корректировка `osal_vasprintf()` для устранения предупреждений статических анализаторов.
-------------------------------------------------------------------------------
## v0.12.4 "Арта-333" от 2023-03-03
Стабилизирующий выпуск с исправлением обнаруженных ошибок, устранением
недочетов и технических долгов. Ветка 0.12 считается готовой к
@ -26,7 +70,7 @@ Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
- Masatoshi Fukunaga <https://gitflic.ru/user/mah0x211> за сообщение о проблеме
`put(MDBX_UPSERT+MDBX_ALLDUPS)` для случая замены всех значений в subDb.
Исправления (без корректировок новых функций):
Исправления:
- Устранен регресс после коммита 474391c83c5f81def6fdf3b0b6f5716a87b78fbf,
приводящий к возврату ERROR_SHARING_VIOLATION в Windows при открытии БД
@ -83,7 +127,7 @@ Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
-------------------------------------------------------------------------------
## v0.12.3 (Акула) от 2023-01-07
## v0.12.3 "Акула" от 2023-01-07
Выпуск с существенными доработками и новой функциональностью в память о закрытом open-source
[проекте "Акула"](https://erigon.substack.com/p/winding-down-support-for-akula-project).
@ -226,7 +270,7 @@ Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
-------------------------------------------------------------------------------
## v0.12.2 (Иван Ярыгин) от 2022-11-11
## v0.12.2 "Иван Ярыгин" от 2022-11-11
Выпуск с существенными доработками и новой функциональностью
в память о российском борце [Иване Сергеевиче Ярыгине](https://ru.wikipedia.org/wiki/Ярыгин,_Иван_Сергеевич).
@ -374,7 +418,7 @@ Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
Мелочи:
- Исторические ссылки cвязанные с удалённым на ~~github~~ проектом перенаправлены на [web.archive.org](https://web.archive.org/web/https://github.com/erthink/libmdbx).
- Синхронизированны конструкции CMake между проектами.
- Синхронизированы конструкции CMake между проектами.
- Добавлено предупреждение о небезопасности RISC-V.
- Добавлено описание параметров `MDBX_debug_func` и `MDBX_debug_func`.
- Добавлено обходное решение для минимизации ложно-положительных
@ -396,7 +440,7 @@ Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
-------------------------------------------------------------------------------
## v0.12.1 (Positive Proxima) at 2022-08-24
## v0.12.1 "Positive Proxima" at 2022-08-24
The planned frontward release with new superior features on the day of 20 anniversary of [Positive Technologies](https://ptsecurty.com).
@ -438,10 +482,54 @@ Fixes:
Not a release but preparation for changing feature set and API.
===============================================================================
## v0.11.14 "Sergey Kapitsa" at 2023-02-14
The stable bugfix release in memory of [Sergey Kapitsa](https://en.wikipedia.org/wiki/Sergey_Kapitsa) on his 95th birthday.
```
22 files changed, 250 insertions(+), 174 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
```
Fixes:
- backport: Fixed insignificant typo of `||` inside `#if` byte-order condition.
- backport: Fixed `SIGSEGV` or an erroneous call to `free()` in situations where
errors occur when reopening by `mdbx_env_open()` of a previously used
environment.
- backport: Fixed `cursor_put_nochecklen()` internals for case when dupsort'ed named subDb
contains a single key with multiple values (aka duplicates), which are replaced
with a single value by put-operation with the `MDBX_UPSERT+MDBX_ALLDUPS` flags.
In this case, the database becomes completely empty, without any pages.
However exactly this condition was not considered and thus wasn't handled correctly.
See [issue#8](https://gitflic.ru/project/erthink/libmdbx/issue/8) for more information.
- backport: Fixed extra assertion inside `override_meta()`, which could
lead to false-positive failing of the assertion in a debug builds during
DB recovery and auto-rollback.
- backport: Refined the `__cold`/`__hot` macros to avoid the
`error: inlining failed in call to always_inline FOO(...): target specific option mismatch`
issue during build using GCC >10.x for SH4 arch.
Minors:
- backport: Using the https://libmdbx.dqdkfa.ru/dead-github
for resources deleted by the Github' administration.
- backport: Fixed English typos.
- backport: Fixed proto of `__asan_default_options()`.
- backport: Fixed doxygen-description of C++ API, especially of C++20 concepts.
- backport: Refined `const` and `noexcept` for few C++ API methods.
- backport: Fixed copy&paste typo of "Getting started".
- backport: Update MithrilDB status.
- backport: Resolve false-posirive `used uninitialized` warning from GCC >10.x
while build for SH4 arch.
-------------------------------------------------------------------------------
## v0.11.13 at (Swashplate) 2022-11-10
## v0.11.13 at "Swashplate" 2022-11-10
The stable bugfix release in memory of [Boris Yuryev](https://ru.wikipedia.org/wiki/Юрьев,_Борис_Николаевич) on his 133rd birthday.
@ -469,7 +557,10 @@ Minors:
- Use `--dont-check-ram-size` for small-tests make-targets (backport).
## v0.11.12 (Эребуни) at 2022-10-12
-------------------------------------------------------------------------------
## v0.11.12 "Эребуни" at 2022-10-12
The stable bugfix release.
@ -491,7 +582,10 @@ Minors:
- Removed needless `LockFileEx()` inside `mdbx_env_copy()` (backport).
## v0.11.11 (Тендра-1790) at 2022-09-11
-------------------------------------------------------------------------------
## v0.11.11 "Тендра-1790" at 2022-09-11
The stable bugfix release.
@ -507,7 +601,10 @@ Fixes:
- Fixed derived C++ builds by removing `MDBX_INTERNAL_FUNC` for `mdbx_w2mb()` and `mdbx_mb2w()`.
## v0.11.10 (the TriColor) at 2022-08-22
-------------------------------------------------------------------------------
## v0.11.10 "the TriColor" at 2022-08-22
The stable bugfix release.
@ -537,8 +634,10 @@ Minors:
- Minor clarified `iov_page()` failure case.
-------------------------------------------------------------------------------
## v0.11.9 (Чирчик-1992) at 2022-08-02
## v0.11.9 "Чирчик-1992" at 2022-08-02
The stable bugfix release.
@ -547,7 +646,7 @@ The stable bugfix release.
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
```
Acknowledgements:
Acknowledgments:
- [Alex Sharov](https://github.com/AskAlexSharov) and Erigon team for reporting and testing.
- [Andrew Ashikhmin](https://gitflic.ru/user/yperbasis) for contributing.
@ -579,11 +678,11 @@ Minors:
-------------------------------------------------------------------------------
## v0.11.8 (Baked Apple) at 2022-06-12
## v0.11.8 "Baked Apple" at 2022-06-12
The stable release with an important fixes and workaround for the critical macOS thread-local-storage issue.
Acknowledgements:
Acknowledgments:
- [Masatoshi Fukunaga](https://github.com/mah0x211) for [Lua bindings](https://github.com/mah0x211/lua-libmdbx).
@ -632,7 +731,7 @@ Minors:
-------------------------------------------------------------------------------
## v0.11.7 (Resurrected Sarmat) at 2022-04-22
## v0.11.7 "Resurrected Sarmat" at 2022-04-22
The stable risen release after the Github's intentional malicious disaster.
@ -678,7 +777,7 @@ Minors:
- Switched to using `MDBX_EPERM` instead of `MDBX_RESULT_TRUE` to indicate that the geometry cannot be updated.
- Added `NULL` checking during memory allocation inside `mdbx_chk`.
- Resolved all warnings from MinGW while used without CMake.
- Added inheretable `target_include_directories()` to `CMakeLists.txt` for easy integration.
- Added inheritable `target_include_directories()` to `CMakeLists.txt` for easy integration.
- Added build-time checks and paranoid runtime assertions for the `off_t` arguments of `fcntl()` which are used for locking.
- Added `-Wno-lto-type-mismatch` to avoid false-positive warnings from old GCC during LTO-enabled builds.
- Added checking for TID (system thread id) to avoid hang on 32-bit Bionic/Android within `pthread_mutex_lock()`.
@ -695,7 +794,7 @@ The stable release with the complete workaround for an incoherence flaw of Linux
Nonetheless the cause for this trouble may be an issue of Intel CPU cache/MESI.
See [issue#269](https://libmdbx.dqdkfa.ru/dead-github/issues/269) for more information.
Acknowledgements:
Acknowledgments:
- [David Bouyssié](https://github.com/david-bouyssie) for [Scala bindings](https://github.com/david-bouyssie/mdbx4s).
- [Michelangelo Riccobene](https://github.com/mriccobene) for reporting and testing.
@ -715,12 +814,15 @@ Minors:
- Clarified error messages of a signature/version mismatch.
-------------------------------------------------------------------------------
## v0.11.5 at 2022-02-23
The release with the temporary hotfix for a flaw of Linux unified page/buffer cache.
See [issue#269](https://libmdbx.dqdkfa.ru/dead-github/issues/269) for more information.
Acknowledgements:
Acknowledgments:
- [Simon Leier](https://github.com/leisim) for reporting and testing.
- [Kai Wetlesen](https://github.com/kaiwetlesen) for [RPMs](http://copr.fedorainfracloud.org/coprs/kwetlesen/libmdbx/).
@ -744,11 +846,14 @@ Minors:
- Minor fixes Doxygen references, comments, descriptions, etc.
-------------------------------------------------------------------------------
## v0.11.4 at 2022-02-02
The stable release with fixes for large and huge databases sized of 4..128 TiB.
Acknowledgements:
Acknowledgments:
- [Ledgerwatch](https://github.com/ledgerwatch), [Binance](https://github.com/binance-chain) and [Positive Technologies](https://www.ptsecurity.com/) teams for reporting, assistance in investigation and testing.
- [Alex Sharov](https://github.com/AskAlexSharov) for reporting, testing and provide resources for remote debugging/investigation.
@ -789,9 +894,12 @@ Minors:
- Using the `-fno-semantic interposition` option to reduce the overhead to calling self own public functions.
-------------------------------------------------------------------------------
## v0.11.3 at 2021-12-31
Acknowledgements:
Acknowledgments:
- [gcxfd <i@rmw.link>](https://github.com/gcxfd) for reporting, contributing and testing.
- [장세연 (Чан Се Ен)](https://github.com/sasgas) for reporting and testing.
@ -822,9 +930,12 @@ Minors:
- For compatibility reverted returning `MDBX_ENODATA`for some cases.
-------------------------------------------------------------------------------
## v0.11.2 at 2021-12-02
Acknowledgements:
Acknowledgments:
- [장세연 (Чан Се Ен)](https://github.com/sasgas) for contributing to C++ API.
- [Alain Picard](https://github.com/castortech) for [Java bindings](https://github.com/castortech/mdbxjni).
@ -848,6 +959,9 @@ Minors:
- Remove unneeded `#undef P_DIRTY`.
-------------------------------------------------------------------------------
## v0.11.1 at 2021-10-23
### Backward compatibility break:
@ -862,12 +976,12 @@ This change is mostly invisible:
- previously versions are unable to read/write a new DBs;
- but the new release is able to handle an old DBs and will silently upgrade ones.
Acknowledgements:
Acknowledgments:
- [Alex Sharov](https://github.com/AskAlexSharov) for reporting and testing.
-------------------------------------------------------------------------------
===============================================================================
## v0.10.5 at 2021-10-13 (obsolete, please use v0.11.1)
@ -880,7 +994,7 @@ Unfortunately, the `v0.10.5` accidentally comes not full-compatible with previou
This cannot be fixed, as it requires fixing past versions, which as a result we will just get a current version.
Therefore, it is recommended to use `v0.11.1` instead of `v0.10.5`.
Acknowledgements:
Acknowledgments:
- [Noel Kuntze](https://github.com/Thermi) for immediately bug reporting.
@ -896,9 +1010,12 @@ Minors:
- Refined providing information for the `@MAIN` and `@GC` sub-databases of a last committed modification transaction's ID.
-------------------------------------------------------------------------------
## v0.10.4 at 2021-10-10
Acknowledgements:
Acknowledgments:
- [Artem Vorotnikov](https://github.com/vorot93) for support [Rust wrapper](https://github.com/vorot93/libmdbx-rs).
- [Andrew Ashikhmin](https://github.com/yperbasis) for contributing to C++ API.
@ -916,9 +1033,12 @@ Minors:
- In debugging builds fixed a too small (single page) by default DB shrink threshold.
-------------------------------------------------------------------------------
## v0.10.3 at 2021-08-27
Acknowledgements:
Acknowledgments:
- [Francisco Vallarino](https://github.com/fjvallarino) for [Haskell bindings for libmdbx](https://hackage.haskell.org/package/libmdbx).
- [Alex Sharov](https://github.com/AskAlexSharov) for reporting and testing.
@ -944,9 +1064,12 @@ Minors:
- Fixed CMake warning about compatibility with 3.8.2
-------------------------------------------------------------------------------
## v0.10.2 at 2021-07-26
Acknowledgements:
Acknowledgments:
- [Alex Sharov](https://github.com/AskAlexSharov) for reporting and testing.
- [Andrea Lanfranchi](https://github.com/AndreaLanfranchi) for reporting bugs.
@ -991,9 +1114,12 @@ Fixes:
- Fixed [test framework keygen-related issue](https://libmdbx.dqdkfa.ru/dead-github/issues/127).
-------------------------------------------------------------------------------
## v0.10.1 at 2021-06-01
Acknowledgements:
Acknowledgments:
- [Alexey Akhunov](https://github.com/AlexeyAkhunov) and [Alex Sharov](https://github.com/AskAlexSharov) for bug reporting and testing.
- [Andrea Lanfranchi](https://github.com/AndreaLanfranchi) for bug reporting and testing related to WSL2.
@ -1015,9 +1141,12 @@ Fixes:
- Re-Fixed WSL1/WSL2 detection with distinguishing (https://libmdbx.dqdkfa.ru/dead-github/issues/97).
-------------------------------------------------------------------------------
## v0.10.0 at 2021-05-09
Acknowledgements:
Acknowledgments:
- [Mahlon E. Smith](https://github.com/mahlonsmith) for [Ruby bindings](https://rubygems.org/gems/mdbx/).
- [Alex Sharov](https://github.com/AskAlexSharov) for [mdbx-go](https://github.com/torquem-ch/mdbx-go), bug reporting and testing.
@ -1101,12 +1230,12 @@ Fixes:
- Fixed building by MinGW for Windows (https://libmdbx.dqdkfa.ru/dead-github/issues/155).
-------------------------------------------------------------------------------
===============================================================================
## v0.9.3 at 2021-02-02
Acknowledgements:
Acknowledgments:
- [Mahlon E. Smith](http://www.martini.nu/) for [FreeBSD port of libmdbx](https://svnweb.freebsd.org/ports/head/databases/mdbx/).
- [장세연](http://www.castis.com) for bug fixing and PR.
@ -1162,9 +1291,12 @@ Fixes:
- Fixed operation on systems with unusual small/large page size, including PowerPC (https://libmdbx.dqdkfa.ru/dead-github/issues/157).
-------------------------------------------------------------------------------
## v0.9.2 at 2020-11-27
Acknowledgements:
Acknowledgments:
- Jens Alfke (Mobile Architect at [Couchbase](https://www.couchbase.com/)) for [NimDBX](https://github.com/snej/nimdbx).
- Clément Renault (CTO at [MeiliSearch](https://www.meilisearch.com/)) for [mdbx-rs](https://github.com/Kerollmops/mdbx-rs).
@ -1212,6 +1344,9 @@ Fixes:
- Added handling `EXCEPTION_POSSIBLE_DEADLOCK` condition for Windows.
-------------------------------------------------------------------------------
## v0.9.1 2020-09-30
Added features:
@ -1258,6 +1393,9 @@ Fixes:
- Now C++ compiler optional for building by CMake.
-------------------------------------------------------------------------------
## v0.9.0 2020-07-31 (not a release, but API changes)
Added features:
@ -1271,7 +1409,7 @@ Deprecated functions and flags:
Please use the value-to-key functions to provide keys that are compatible with the built-in libmdbx comparators.
-------------------------------------------------------------------------------
===============================================================================
## 2020-07-06
@ -1411,6 +1549,8 @@ Deprecated functions and flags:
- API description.
- Checking for non-local filesystems to avoid DB corruption.
-------------------------------------------------------------------------------
===============================================================================
For early changes see the git commit history.

4
mdbx.h
View File

@ -3057,6 +3057,8 @@ LIBMDBX_API int mdbx_env_get_fd(const MDBX_env *env, mdbx_filehandle_t *fd);
* it is reasonable to know some details in order to make optimal decisions
* when choosing parameters.
*
* \see mdbx_env_info_ex()
*
* Both \ref mdbx_env_set_geometry() and legacy \ref mdbx_env_set_mapsize() are
* inapplicable to read-only opened environment.
*
@ -3166,7 +3168,7 @@ LIBMDBX_API int mdbx_env_get_fd(const MDBX_env *env, mdbx_filehandle_t *fd);
* \note Actual values may be different than your have specified because of
* rounding to specified database page size, the system page size and/or the
* size of the system virtual memory management unit. You can get actual values
* by \ref mdbx_env_sync_ex() or see by using the tool `mdbx_chk` with the `-v`
* by \ref mdbx_env_info_ex() or see by using the tool `mdbx_chk` with the `-v`
* option.
*
* Legacy \ref mdbx_env_set_mapsize() correspond to calling

View File

@ -356,6 +356,9 @@ class cursor_managed;
__cpp_lib_memory_resource >= 201603L && _GLIBCXX_USE_CXX11_ABI)
/// \brief Default polymorphic allocator for modern code.
using polymorphic_allocator = ::std::pmr::string::allocator_type;
using default_allocator = polymorphic_allocator;
#else
using default_allocator = legacy_allocator;
#endif /* __cpp_lib_memory_resource >= 201603L */
/// \brief Default singe-byte string.
@ -2663,41 +2666,65 @@ public:
return buffer(::std::move(src));
}
static buffer key_from(const double ieee754_64bit) {
static buffer key_from_double(const double ieee754_64bit) {
return wrap(::mdbx_key_from_double(ieee754_64bit));
}
static buffer key_from(const double ieee754_64bit) {
return key_from_double(ieee754_64bit);
}
static buffer key_from(const double *ieee754_64bit) {
return wrap(::mdbx_key_from_ptrdouble(ieee754_64bit));
}
static buffer key_from(const uint64_t unsigned_int64) {
static buffer key_from_u64(const uint64_t unsigned_int64) {
return wrap(unsigned_int64);
}
static buffer key_from(const int64_t signed_int64) {
static buffer key_from(const uint64_t unsigned_int64) {
return key_from_u64(unsigned_int64);
}
static buffer key_from_i64(const int64_t signed_int64) {
return wrap(::mdbx_key_from_int64(signed_int64));
}
static buffer key_from(const int64_t signed_int64) {
return key_from_i64(signed_int64);
}
static buffer key_from_jsonInteger(const int64_t json_integer) {
return wrap(::mdbx_key_from_jsonInteger(json_integer));
}
static buffer key_from(const float ieee754_32bit) {
static buffer key_from_float(const float ieee754_32bit) {
return wrap(::mdbx_key_from_float(ieee754_32bit));
}
static buffer key_from(const float ieee754_32bit) {
return key_from_float(ieee754_32bit);
}
static buffer key_from(const float *ieee754_32bit) {
return wrap(::mdbx_key_from_ptrfloat(ieee754_32bit));
}
static buffer key_from(const uint32_t unsigned_int32) {
static buffer key_from_u32(const uint32_t unsigned_int32) {
return wrap(unsigned_int32);
}
static buffer key_from(const int32_t signed_int32) {
static buffer key_from(const uint32_t unsigned_int32) {
return key_from_u32(unsigned_int32);
}
static buffer key_from_i32(const int32_t signed_int32) {
return wrap(::mdbx_key_from_int32(signed_int32));
}
static buffer key_from(const int32_t signed_int32) {
return key_from_i32(signed_int32);
}
};
template <class ALLOCATOR, class CAPACITY_POLICY,

View File

@ -3298,9 +3298,8 @@ static int __must_check_result page_split(MDBX_cursor *mc,
MDBX_val *const newdata,
pgno_t newpgno, const unsigned naf);
static int coherency_timeout(uint64_t *timestamp, pgno_t pgno);
static bool coherency_check_meta(const MDBX_env *env,
const volatile MDBX_meta *meta, bool report);
static int coherency_timeout(uint64_t *timestamp, intptr_t pgno,
const MDBX_env *env);
static int __must_check_result validate_meta_copy(MDBX_env *env,
const MDBX_meta *meta,
MDBX_meta *dest);
@ -4680,7 +4679,7 @@ static void iov_callback4dirtypages(iov_ctx_t *ctx, size_t offset, void *data,
WARNING("catch delayed/non-arrived page %" PRIaPGNO " %s", wp->mp_pgno,
"(workaround for incoherent flaw of unified page/buffer cache)");
do
if (coherency_timeout(&ctx->coherency_timestamp, wp->mp_pgno) !=
if (coherency_timeout(&ctx->coherency_timestamp, wp->mp_pgno, env) !=
MDBX_RESULT_TRUE) {
ctx->err = MDBX_PROBLEM;
break;
@ -8244,14 +8243,6 @@ __cold int mdbx_env_sync_ex(MDBX_env *env, bool force, bool nonblock) {
return env_sync(env, force, nonblock);
}
#ifndef LIBMDBX_NO_EXPORTS_LEGACY_API
__cold int mdbx_env_sync(MDBX_env *env) { return __inline_mdbx_env_sync(env); }
__cold int mdbx_env_sync_poll(MDBX_env *env) {
return __inline_mdbx_env_sync_poll(env);
}
#endif /* LIBMDBX_NO_EXPORTS_LEGACY_API */
/* Back up parent txn's cursors, then grab the originals for tracking */
static int cursor_shadow(MDBX_txn *parent, MDBX_txn *nested) {
tASSERT(parent, parent->mt_cursors[FREE_DBI] == nullptr);
@ -8600,20 +8591,26 @@ static bool coherency_check(const MDBX_env *env, const txnid_t txnid,
(!freedb_mod_txnid && freedb_root &&
likely(magic_and_version == MDBX_DATA_MAGIC)))) {
if (report)
WARNING("catch invalid %sdb.mod_txnid %" PRIaTXN
" for meta_txnid %" PRIaTXN " %s",
"free", freedb_mod_txnid, txnid,
"(workaround for incoherent flaw of unified page/buffer cache)");
WARNING(
"catch invalid %sdb.mod_txnid %" PRIaTXN " for meta_txnid %" PRIaTXN
" %s",
"free", freedb_mod_txnid, txnid,
(env->me_stuck_meta < 0)
? "(workaround for incoherent flaw of unified page/buffer cache)"
: "(wagering meta)");
ok = false;
}
if (unlikely(txnid < maindb_mod_txnid ||
(!maindb_mod_txnid && maindb_root &&
likely(magic_and_version == MDBX_DATA_MAGIC)))) {
if (report)
WARNING("catch invalid %sdb.mod_txnid %" PRIaTXN
" for meta_txnid %" PRIaTXN " %s",
"main", maindb_mod_txnid, txnid,
"(workaround for incoherent flaw of unified page/buffer cache)");
WARNING(
"catch invalid %sdb.mod_txnid %" PRIaTXN " for meta_txnid %" PRIaTXN
" %s",
"main", maindb_mod_txnid, txnid,
(env->me_stuck_meta < 0)
? "(workaround for incoherent flaw of unified page/buffer cache)"
: "(wagering meta)");
ok = false;
}
if (likely(freedb_root && freedb_mod_txnid)) {
@ -8623,11 +8620,12 @@ static bool coherency_check(const MDBX_env *env, const txnid_t txnid,
const txnid_t root_txnid = freedb_root->mp_txnid;
if (unlikely(root_txnid != freedb_mod_txnid)) {
if (report)
WARNING(
"catch invalid root_page %" PRIaPGNO " mod_txnid %" PRIaTXN
" for %sdb.mod_txnid %" PRIaTXN " %s",
freedb_root_pgno, root_txnid, "free", freedb_mod_txnid,
"(workaround for incoherent flaw of unified page/buffer cache)");
WARNING("catch invalid root_page %" PRIaPGNO " mod_txnid %" PRIaTXN
" for %sdb.mod_txnid %" PRIaTXN " %s",
freedb_root_pgno, root_txnid, "free", freedb_mod_txnid,
(env->me_stuck_meta < 0) ? "(workaround for incoherent flaw of "
"unified page/buffer cache)"
: "(wagering meta)");
ok = false;
}
}
@ -8638,11 +8636,12 @@ static bool coherency_check(const MDBX_env *env, const txnid_t txnid,
const txnid_t root_txnid = maindb_root->mp_txnid;
if (unlikely(root_txnid != maindb_mod_txnid)) {
if (report)
WARNING(
"catch invalid root_page %" PRIaPGNO " mod_txnid %" PRIaTXN
" for %sdb.mod_txnid %" PRIaTXN " %s",
maindb_root_pgno, root_txnid, "main", maindb_mod_txnid,
"(workaround for incoherent flaw of unified page/buffer cache)");
WARNING("catch invalid root_page %" PRIaPGNO " mod_txnid %" PRIaTXN
" for %sdb.mod_txnid %" PRIaTXN " %s",
maindb_root_pgno, root_txnid, "main", maindb_mod_txnid,
(env->me_stuck_meta < 0) ? "(workaround for incoherent flaw of "
"unified page/buffer cache)"
: "(wagering meta)");
ok = false;
}
}
@ -8654,15 +8653,16 @@ static bool coherency_check(const MDBX_env *env, const txnid_t txnid,
return ok;
}
__cold static int coherency_timeout(uint64_t *timestamp, pgno_t pgno) {
__cold static int coherency_timeout(uint64_t *timestamp, intptr_t pgno,
const MDBX_env *env) {
if (likely(timestamp && *timestamp == 0))
*timestamp = osal_monotime();
else if (unlikely(!timestamp || osal_monotime() - *timestamp >
osal_16dot16_to_monotime(65536 / 10))) {
if (pgno)
ERROR("bailout waiting for %" PRIaPGNO " page arrival %s", pgno,
if (pgno >= 0 && pgno != env->me_stuck_meta)
ERROR("bailout waiting for %" PRIuSIZE " page arrival %s", pgno,
"(workaround for incoherent flaw of unified page/buffer cache)");
else
else if (env->me_stuck_meta < 0)
ERROR("bailout waiting for valid snapshot (%s)",
"workaround for incoherent flaw of unified page/buffer cache");
return MDBX_PROBLEM;
@ -8683,23 +8683,25 @@ __cold static int coherency_timeout(uint64_t *timestamp, pgno_t pgno) {
/* check with timeout as the workaround
* for https://libmdbx.dqdkfa.ru/dead-github/issues/269 */
__hot static int coherency_check_readed(const MDBX_env *env,
const txnid_t txnid,
const volatile MDBX_db *dbs,
const volatile MDBX_meta *meta,
uint64_t *timestamp) {
const bool report = !(timestamp && *timestamp);
if (unlikely(!coherency_check(env, txnid, dbs, meta, report)))
return coherency_timeout(timestamp, 0);
__hot static int coherency_check_head(MDBX_txn *txn, const meta_ptr_t head,
uint64_t *timestamp) {
/* Copy the DB info and flags */
txn->mt_geo = head.ptr_v->mm_geo;
memcpy(txn->mt_dbs, head.ptr_c->mm_dbs, CORE_DBS * sizeof(MDBX_db));
txn->mt_canary = head.ptr_v->mm_canary;
if (unlikely(!coherency_check(txn->mt_env, head.txnid, txn->mt_dbs,
head.ptr_v, *timestamp == 0)))
return coherency_timeout(timestamp, -1, txn->mt_env);
return MDBX_SUCCESS;
}
static int coherency_check_written(const MDBX_env *env, const txnid_t txnid,
const volatile MDBX_meta *meta,
uint64_t *timestamp) {
const intptr_t pgno, uint64_t *timestamp) {
const bool report = !(timestamp && *timestamp);
const txnid_t head_txnid = meta_txnid(meta);
if (unlikely(head_txnid < MIN_TXNID || (head_txnid < txnid))) {
if (unlikely(head_txnid < MIN_TXNID || head_txnid < txnid)) {
if (report) {
env->me_lck->mti_pgop_stat.incoherence.weak =
(env->me_lck->mti_pgop_stat.incoherence.weak >= INT32_MAX)
@ -8710,16 +8712,18 @@ static int coherency_check_written(const MDBX_env *env, const txnid_t txnid,
bytes2pgno(env, ptr_dist(meta, env->me_map)),
"(workaround for incoherent flaw of unified page/buffer cache)");
}
return coherency_timeout(timestamp, 0);
return coherency_timeout(timestamp, pgno, env);
}
return coherency_check_readed(env, head_txnid, meta->mm_dbs, meta, timestamp);
if (unlikely(!coherency_check(env, head_txnid, meta->mm_dbs, meta, report)))
return coherency_timeout(timestamp, pgno, env);
return MDBX_SUCCESS;
}
static bool coherency_check_meta(const MDBX_env *env,
static bool check_meta_coherency(const MDBX_env *env,
const volatile MDBX_meta *meta, bool report) {
uint64_t timestamp = 0;
return coherency_check_written(env, 0, meta, report ? &timestamp : nullptr) ==
MDBX_SUCCESS;
return coherency_check_written(env, 0, meta, -1,
report ? &timestamp : nullptr) == MDBX_SUCCESS;
}
/* Common code for mdbx_txn_begin() and mdbx_txn_renew(). */
@ -8831,10 +8835,6 @@ static int txn_renew(MDBX_txn *txn, const unsigned flags) {
/* Snap the state from current meta-head */
txn->mt_txnid = head.txnid;
txn->mt_geo = head.ptr_v->mm_geo;
memcpy(txn->mt_dbs, head.ptr_c->mm_dbs, CORE_DBS * sizeof(MDBX_db));
txn->mt_canary = head.ptr_v->mm_canary;
if (likely(env->me_stuck_meta < 0) &&
unlikely(meta_should_retry(env, &troika) ||
head.txnid < atomic_load64(&env->me_lck->mti_oldest_reader,
@ -8852,8 +8852,7 @@ static int txn_renew(MDBX_txn *txn, const unsigned flags) {
continue;
}
rc = coherency_check_readed(env, head.txnid, txn->mt_dbs, head.ptr_v,
&timestamp);
rc = coherency_check_head(txn, head, &timestamp);
jitter4testing(false);
if (likely(rc == MDBX_SUCCESS))
break;
@ -8923,14 +8922,12 @@ static int txn_renew(MDBX_txn *txn, const unsigned flags) {
const meta_ptr_t head = meta_recent(env, &txn->tw.troika);
uint64_t timestamp = 0;
while ("workaround for https://libmdbx.dqdkfa.ru/dead-github/issues/269") {
rc = coherency_check_readed(env, head.txnid, head.ptr_v->mm_dbs,
head.ptr_v, &timestamp);
rc = coherency_check_head(txn, head, &timestamp);
if (likely(rc == MDBX_SUCCESS))
break;
if (unlikely(rc != MDBX_RESULT_TRUE))
goto bailout;
}
txn->mt_canary = head.ptr_c->mm_canary;
eASSERT(env, meta_txnid(head.ptr_v) == head.txnid);
txn->mt_txnid = safe64_txnid_next(head.txnid);
if (unlikely(txn->mt_txnid > MAX_TXNID)) {
@ -8955,10 +8952,6 @@ static int txn_renew(MDBX_txn *txn, const unsigned flags) {
env->me_txn = txn;
txn->mt_numdbs = env->me_numdbs;
memcpy(txn->mt_dbiseqs, env->me_dbiseqs, txn->mt_numdbs * sizeof(unsigned));
/* Copy the DB info and flags */
memcpy(txn->mt_dbs, head.ptr_c->mm_dbs, CORE_DBS * sizeof(MDBX_db));
/* Moved to here to avoid a data race in read TXNs */
txn->mt_geo = head.ptr_c->mm_geo;
if ((txn->mt_flags & MDBX_WRITEMAP) == 0 || MDBX_AVOID_MSYNC) {
rc = dpl_alloc(txn);
@ -9171,13 +9164,6 @@ int mdbx_txn_renew(MDBX_txn *txn) {
return rc;
}
#ifndef LIBMDBX_NO_EXPORTS_LEGACY_API
int mdbx_txn_begin(MDBX_env *env, MDBX_txn *parent, MDBX_txn_flags_t flags,
MDBX_txn **ret) {
return __inline_mdbx_txn_begin(env, parent, flags, ret);
}
#endif /* LIBMDBX_NO_EXPORTS_LEGACY_API */
int mdbx_txn_set_userctx(MDBX_txn *txn, void *ctx) {
int rc = check_txn(txn, MDBX_TXN_FINISHED);
if (unlikely(rc != MDBX_SUCCESS))
@ -9261,6 +9247,9 @@ int mdbx_txn_begin_ex(MDBX_env *env, MDBX_txn *parent, MDBX_txn_flags_t flags,
(MDBX_GOOFY_MSVC_STATIC_ANALYZER && base > size) ? size : base);
txn->mt_dbs = ptr_disp(txn, base);
txn->mt_cursors = ptr_disp(txn->mt_dbs, sizeof(MDBX_db) * env->me_maxdbs);
#if MDBX_DEBUG
txn->mt_cursors[FREE_DBI] = nullptr; /* avoid SIGSEGV in an assertion later */
#endif /* MDBX_DEBUG */
txn->mt_dbistate = ptr_disp(txn, size - env->me_maxdbs);
txn->mt_dbxs = env->me_dbxs; /* static */
txn->mt_flags = flags;
@ -11124,10 +11113,6 @@ static __always_inline bool check_dbi(MDBX_txn *txn, MDBX_dbi dbi,
return dbi_import(txn, dbi);
}
#ifndef LIBMDBX_NO_EXPORTS_LEGACY_API
int mdbx_txn_commit(MDBX_txn *txn) { return __inline_mdbx_txn_commit(txn); }
#endif /* LIBMDBX_NO_EXPORTS_LEGACY_API */
/* Merge child txn into parent */
static __inline void txn_merge(MDBX_txn *const parent, MDBX_txn *const txn,
const size_t parent_retired_len) {
@ -12509,7 +12494,7 @@ __cold static MDBX_page *meta_model(const MDBX_env *env, MDBX_page *model,
model_meta->mm_dbs[MAIN_DBI].md_root = P_INVALID;
meta_set_txnid(env, model_meta, MIN_TXNID + num);
unaligned_poke_u64(4, model_meta->mm_sign, meta_sign(model_meta));
eASSERT(env, coherency_check_meta(env, model_meta, true));
eASSERT(env, check_meta_coherency(env, model_meta, true));
return ptr_disp(model, env->me_psize);
}
@ -12680,7 +12665,7 @@ static int sync_locked(MDBX_env *env, unsigned flags, MDBX_meta *const pending,
goto fail;
}
meta_set_txnid(env, pending, txnid);
eASSERT(env, coherency_check_meta(env, pending, true));
eASSERT(env, check_meta_coherency(env, pending, true));
}
}
}
@ -12726,7 +12711,7 @@ static int sync_locked(MDBX_env *env, unsigned flags, MDBX_meta *const pending,
rc = (flags & MDBX_SAFE_NOSYNC) ? MDBX_RESULT_TRUE /* carry non-steady */
: MDBX_RESULT_FALSE /* carry steady */;
}
eASSERT(env, coherency_check_meta(env, pending, true));
eASSERT(env, check_meta_coherency(env, pending, true));
/* Steady or Weak */
if (rc == MDBX_RESULT_FALSE /* carry steady */) {
@ -12831,7 +12816,7 @@ static int sync_locked(MDBX_env *env, unsigned flags, MDBX_meta *const pending,
/* LY: 'commit' the meta */
meta_update_end(env, target, unaligned_peek_u64(4, pending->mm_txnid_b));
jitter4testing(true);
eASSERT(env, coherency_check_meta(env, target, true));
eASSERT(env, check_meta_coherency(env, target, true));
} else {
/* dangerous case (target == head), only mm_sign could
* me updated, check assertions once again */
@ -12904,8 +12889,9 @@ static int sync_locked(MDBX_env *env, unsigned flags, MDBX_meta *const pending,
uint64_t timestamp = 0;
while ("workaround for https://libmdbx.dqdkfa.ru/dead-github/issues/269") {
rc =
coherency_check_written(env, pending->unsafe_txnid, target, &timestamp);
rc = coherency_check_written(env, pending->unsafe_txnid, target,
bytes2pgno(env, ptr_dist(target, env->me_map)),
&timestamp);
if (likely(rc == MDBX_SUCCESS))
break;
if (unlikely(rc != MDBX_RESULT_TRUE))
@ -12934,7 +12920,7 @@ static int sync_locked(MDBX_env *env, unsigned flags, MDBX_meta *const pending,
pending->mm_geo.upper, impilict_shrink);
if (rc != MDBX_SUCCESS && rc != MDBX_EPERM)
goto fail;
eASSERT(env, coherency_check_meta(env, target, true));
eASSERT(env, check_meta_coherency(env, target, true));
}
MDBX_lockinfo *const lck = env->me_lck_mmap.lck;
@ -13173,10 +13159,10 @@ __cold static intptr_t get_reasonable_db_maxsize(intptr_t *cached_result) {
return *cached_result;
}
__cold LIBMDBX_API int
mdbx_env_set_geometry(MDBX_env *env, intptr_t size_lower, intptr_t size_now,
intptr_t size_upper, intptr_t growth_step,
intptr_t shrink_threshold, intptr_t pagesize) {
__cold int mdbx_env_set_geometry(MDBX_env *env, intptr_t size_lower,
intptr_t size_now, intptr_t size_upper,
intptr_t growth_step,
intptr_t shrink_threshold, intptr_t pagesize) {
int rc = check_env(env, false);
if (unlikely(rc != MDBX_SUCCESS))
return rc;
@ -13428,7 +13414,6 @@ mdbx_env_set_geometry(MDBX_env *env, intptr_t size_lower, intptr_t size_now,
ENSURE(env, pagesize == (intptr_t)env->me_psize);
MDBX_meta meta;
memset(&meta, 0, sizeof(meta));
const MDBX_geo *current_geo;
if (!inside_txn) {
eASSERT(env, need_unlock);
const meta_ptr_t head = meta_recent(env, &env->me_txn0->tw.troika);
@ -13436,14 +13421,13 @@ mdbx_env_set_geometry(MDBX_env *env, intptr_t size_lower, intptr_t size_now,
uint64_t timestamp = 0;
while ("workaround for "
"https://libmdbx.dqdkfa.ru/dead-github/issues/269") {
meta = *head.ptr_c;
rc = coherency_check_readed(env, head.txnid, meta.mm_dbs, &meta,
&timestamp);
rc = coherency_check_head(env->me_txn0, head, &timestamp);
if (likely(rc == MDBX_SUCCESS))
break;
if (unlikely(rc != MDBX_RESULT_TRUE))
goto bailout;
}
meta = *head.ptr_c;
const txnid_t txnid = safe64_txnid_next(head.txnid);
if (unlikely(txnid > MAX_TXNID)) {
rc = MDBX_TXN_FULL;
@ -13451,11 +13435,10 @@ mdbx_env_set_geometry(MDBX_env *env, intptr_t size_lower, intptr_t size_now,
goto bailout;
}
meta_set_txnid(env, &meta, txnid);
current_geo = &meta.mm_geo;
} else {
current_geo = &env->me_txn->mt_geo;
}
const MDBX_geo *const current_geo =
&(env->me_txn ? env->me_txn : env->me_txn0)->mt_geo;
/* update env-geo to avoid influences */
env->me_dbgeo.now = pgno2bytes(env, current_geo->now);
env->me_dbgeo.lower = pgno2bytes(env, current_geo->lower);
@ -13558,28 +13541,6 @@ bailout:
return rc;
}
#ifndef LIBMDBX_NO_EXPORTS_LEGACY_API
__cold int mdbx_env_set_mapsize(MDBX_env *env, size_t size) {
return __inline_mdbx_env_set_mapsize(env, size);
}
__cold int mdbx_env_set_maxdbs(MDBX_env *env, MDBX_dbi dbs) {
return __inline_mdbx_env_set_maxdbs(env, dbs);
}
__cold int mdbx_env_get_maxdbs(const MDBX_env *env, MDBX_dbi *dbs) {
return __inline_mdbx_env_get_maxdbs(env, dbs);
}
__cold int mdbx_env_set_maxreaders(MDBX_env *env, unsigned readers) {
return __inline_mdbx_env_set_maxreaders(env, readers);
}
__cold int mdbx_env_get_maxreaders(const MDBX_env *env, unsigned *readers) {
return __inline_mdbx_env_get_maxreaders(env, readers);
}
#endif /* LIBMDBX_NO_EXPORTS_LEGACY_API */
__cold static int alloc_page_buf(MDBX_env *env) {
return env->me_pbuf ? MDBX_SUCCESS
: osal_memalign_alloc(env->me_os_psize,
@ -14383,9 +14344,9 @@ __cold static int __must_check_result override_meta(MDBX_env *env,
MDBX_meta *const model = page_meta(page);
meta_set_txnid(env, model, txnid);
if (txnid)
eASSERT(env, coherency_check_meta(env, model, true));
eASSERT(env, check_meta_coherency(env, model, true));
if (shape) {
if (txnid && unlikely(!coherency_check_meta(env, shape, false))) {
if (txnid && unlikely(!check_meta_coherency(env, shape, false))) {
ERROR("bailout overriding meta-%zu since model failed "
"freedb/maindb %s-check for txnid #%" PRIaTXN,
target, "pre", constmeta_txnid(shape));
@ -14409,7 +14370,7 @@ __cold static int __must_check_result override_meta(MDBX_env *env,
model->mm_dbs[MAIN_DBI].md_root != P_INVALID))
memcpy(&model->mm_magic_and_version, &shape->mm_magic_and_version,
sizeof(model->mm_magic_and_version));
if (unlikely(!coherency_check_meta(env, model, false))) {
if (unlikely(!check_meta_coherency(env, model, false))) {
ERROR("bailout overriding meta-%zu since model failed "
"freedb/maindb %s-check for txnid #%" PRIaTXN,
target, "post", txnid);
@ -15461,12 +15422,6 @@ __cold int mdbx_env_close_ex(MDBX_env *env, bool dont_sync) {
return rc;
}
#ifndef LIBMDBX_NO_EXPORTS_LEGACY_API
__cold int mdbx_env_close(MDBX_env *env) {
return __inline_mdbx_env_close(env);
}
#endif /* LIBMDBX_NO_EXPORTS_LEGACY_API */
/* Search for key within a page, using binary search.
* Returns the smallest entry larger or equal to the key.
* Updates the cursor index with the index of the found entry.
@ -16020,19 +15975,11 @@ static __noinline int node_read_bigdata(MDBX_cursor *mc, const MDBX_node *node,
if (!MDBX_DISABLE_VALIDATION) {
const MDBX_env *env = mc->mc_txn->mt_env;
const size_t dsize = data->iov_len;
if (unlikely(node_size_len(node_ks(node), dsize) <= env->me_leaf_nodemax) &&
mc->mc_dbi != FREE_DBI)
poor_page(mp, "too small data (%zu bytes) for bigdata-node", dsize);
const unsigned npages = number_of_ovpages(env, dsize);
if (unlikely(lp.page->mp_pages != npages)) {
if (lp.page->mp_pages < npages)
return bad_page(lp.page,
"too less n-pages %u for bigdata-node (%zu bytes)",
lp.page->mp_pages, dsize);
else if (mc->mc_dbi != FREE_DBI)
poor_page(lp.page, "extra n-pages %u for bigdata-node (%zu bytes)",
lp.page->mp_pages, dsize);
}
if (unlikely(lp.page->mp_pages < npages))
return bad_page(lp.page,
"too less n-pages %u for bigdata-node (%zu bytes)",
lp.page->mp_pages, dsize);
}
return MDBX_SUCCESS;
}
@ -21835,8 +21782,8 @@ __cold int mdbx_env_copy(MDBX_env *env, const char *dest_path,
return rc;
}
LIBMDBX_API int mdbx_env_copyW(MDBX_env *env, const wchar_t *dest_path,
MDBX_copy_flags_t flags) {
__cold int mdbx_env_copyW(MDBX_env *env, const wchar_t *dest_path,
MDBX_copy_flags_t flags) {
#endif /* Windows */
int rc = check_env(env, true);
@ -22042,12 +21989,6 @@ __cold int mdbx_env_get_fd(const MDBX_env *env, mdbx_filehandle_t *arg) {
return MDBX_SUCCESS;
}
#ifndef LIBMDBX_NO_EXPORTS_LEGACY_API
__cold int mdbx_env_stat(const MDBX_env *env, MDBX_stat *stat, size_t bytes) {
return __inline_mdbx_env_stat(env, stat, bytes);
}
#endif /* LIBMDBX_NO_EXPORTS_LEGACY_API */
static void stat_get(const MDBX_db *db, MDBX_stat *st, size_t bytes) {
st->ms_depth = db->md_depth;
st->ms_branch_pages = db->md_branch_pages;
@ -22219,13 +22160,6 @@ __cold int mdbx_dbi_dupsort_depthmask(MDBX_txn *txn, MDBX_dbi dbi,
return (rc == MDBX_NOTFOUND) ? MDBX_SUCCESS : rc;
}
#ifndef LIBMDBX_NO_EXPORTS_LEGACY_API
__cold int mdbx_env_info(const MDBX_env *env, MDBX_envinfo *info,
size_t bytes) {
return __inline_mdbx_env_info(env, info, bytes);
}
#endif /* LIBMDBX_NO_EXPORTS_LEGACY_API */
__cold static int fetch_envinfo_ex(const MDBX_env *env, const MDBX_txn *txn,
MDBX_envinfo *arg, const size_t bytes) {
@ -22449,7 +22383,7 @@ static __inline MDBX_cmp_func *get_default_datacmp(unsigned flags) {
static int dbi_bind(MDBX_txn *txn, const MDBX_dbi dbi, unsigned user_flags,
MDBX_cmp_func *keycmp, MDBX_cmp_func *datacmp) {
/* LY: so, accepting only three cases for the table's flags:
/* Accepting only three cases:
* 1) user_flags and both comparators are zero
* = assume that a by-default mode/flags is requested for reading;
* 2) user_flags exactly the same
@ -22471,6 +22405,10 @@ static int dbi_bind(MDBX_txn *txn, const MDBX_dbi dbi, unsigned user_flags,
/* make sure flags changes get committed */
txn->mt_dbs[dbi].md_flags = user_flags & DB_PERSISTENT_FLAGS;
txn->mt_flags |= MDBX_TXN_DIRTY;
/* обнуляем компараторы для установки в соответствии с флагами,
* либо заданных пользователем */
txn->mt_dbxs[dbi].md_cmp = nullptr;
txn->mt_dbxs[dbi].md_dcmp = nullptr;
} else {
return /* FIXME: return extended info */ MDBX_INCOMPATIBLE;
}
@ -22889,12 +22827,6 @@ int mdbx_dbi_flags_ex(MDBX_txn *txn, MDBX_dbi dbi, unsigned *flags,
return MDBX_SUCCESS;
}
#ifndef LIBMDBX_NO_EXPORTS_LEGACY_API
int mdbx_dbi_flags(MDBX_txn *txn, MDBX_dbi dbi, unsigned *flags) {
return __inline_mdbx_dbi_flags(txn, dbi, flags);
}
#endif /* LIBMDBX_NO_EXPORTS_LEGACY_API */
static int drop_tree(MDBX_cursor *mc, const bool may_have_subDBs) {
int rc = page_search(mc, NULL, MDBX_PS_FIRST);
if (likely(rc == MDBX_SUCCESS)) {
@ -23404,16 +23336,6 @@ __cold static txnid_t kick_longlived_readers(MDBX_env *env,
return oldest;
}
#ifndef LIBMDBX_NO_EXPORTS_LEGACY_API
__cold int mdbx_env_set_syncbytes(MDBX_env *env, size_t threshold) {
return __inline_mdbx_env_set_syncbytes(env, threshold);
}
__cold int mdbx_env_set_syncperiod(MDBX_env *env, unsigned seconds_16dot16) {
return __inline_mdbx_env_set_syncperiod(env, seconds_16dot16);
}
#endif /* LIBMDBX_NO_EXPORTS_LEGACY_API */
__cold int mdbx_env_set_hsr(MDBX_env *env, MDBX_hsr_func *hsr) {
int rc = check_env(env, false);
if (unlikely(rc != MDBX_SUCCESS))
@ -24487,16 +24409,6 @@ int mdbx_dbi_sequence(MDBX_txn *txn, MDBX_dbi dbi, uint64_t *result,
/*----------------------------------------------------------------------------*/
#ifndef LIBMDBX_NO_EXPORTS_LEGACY_API
__cold MDBX_NOTHROW_CONST_FUNCTION intptr_t mdbx_limits_pgsize_min(void) {
return __inline_mdbx_limits_pgsize_min();
}
__cold MDBX_NOTHROW_CONST_FUNCTION intptr_t mdbx_limits_pgsize_max(void) {
return __inline_mdbx_limits_pgsize_max();
}
#endif /* LIBMDBX_NO_EXPORTS_LEGACY_API */
__cold intptr_t mdbx_limits_dbsize_min(intptr_t pagesize) {
if (pagesize < 1)
pagesize = (intptr_t)mdbx_default_pagesize();
@ -24600,16 +24512,6 @@ uint32_t mdbx_key_from_ptrfloat(const float *const ieee754_32bit) {
return float2key(ieee754_32bit);
}
#ifndef LIBMDBX_NO_EXPORTS_LEGACY_API
MDBX_NOTHROW_CONST_FUNCTION uint64_t mdbx_key_from_int64(const int64_t i64) {
return __inline_mdbx_key_from_int64(i64);
}
MDBX_NOTHROW_CONST_FUNCTION uint32_t mdbx_key_from_int32(const int32_t i32) {
return __inline_mdbx_key_from_int32(i32);
}
#endif /* LIBMDBX_NO_EXPORTS_LEGACY_API */
#define IEEE754_DOUBLE_MANTISSA_SIZE 52
#define IEEE754_DOUBLE_EXPONENTA_BIAS 0x3FF
#define IEEE754_DOUBLE_EXPONENTA_MAX 0x7FF
@ -25418,6 +25320,109 @@ __cold void global_ctor(void) {
#endif /* #if 0 */
}
/*------------------------------------------------------------------------------
* Legacy API */
#ifndef LIBMDBX_NO_EXPORTS_LEGACY_API
LIBMDBX_API int mdbx_txn_begin(MDBX_env *env, MDBX_txn *parent,
MDBX_txn_flags_t flags, MDBX_txn **ret) {
return __inline_mdbx_txn_begin(env, parent, flags, ret);
}
LIBMDBX_API int mdbx_txn_commit(MDBX_txn *txn) {
return __inline_mdbx_txn_commit(txn);
}
LIBMDBX_API __cold int mdbx_env_stat(const MDBX_env *env, MDBX_stat *stat,
size_t bytes) {
return __inline_mdbx_env_stat(env, stat, bytes);
}
LIBMDBX_API __cold int mdbx_env_info(const MDBX_env *env, MDBX_envinfo *info,
size_t bytes) {
return __inline_mdbx_env_info(env, info, bytes);
}
LIBMDBX_API int mdbx_dbi_flags(MDBX_txn *txn, MDBX_dbi dbi, unsigned *flags) {
return __inline_mdbx_dbi_flags(txn, dbi, flags);
}
LIBMDBX_API __cold int mdbx_env_sync(MDBX_env *env) {
return __inline_mdbx_env_sync(env);
}
LIBMDBX_API __cold int mdbx_env_sync_poll(MDBX_env *env) {
return __inline_mdbx_env_sync_poll(env);
}
LIBMDBX_API __cold int mdbx_env_close(MDBX_env *env) {
return __inline_mdbx_env_close(env);
}
LIBMDBX_API __cold int mdbx_env_set_mapsize(MDBX_env *env, size_t size) {
return __inline_mdbx_env_set_mapsize(env, size);
}
LIBMDBX_API __cold int mdbx_env_set_maxdbs(MDBX_env *env, MDBX_dbi dbs) {
return __inline_mdbx_env_set_maxdbs(env, dbs);
}
LIBMDBX_API __cold int mdbx_env_get_maxdbs(const MDBX_env *env, MDBX_dbi *dbs) {
return __inline_mdbx_env_get_maxdbs(env, dbs);
}
LIBMDBX_API __cold int mdbx_env_set_maxreaders(MDBX_env *env,
unsigned readers) {
return __inline_mdbx_env_set_maxreaders(env, readers);
}
LIBMDBX_API __cold int mdbx_env_get_maxreaders(const MDBX_env *env,
unsigned *readers) {
return __inline_mdbx_env_get_maxreaders(env, readers);
}
LIBMDBX_API __cold int mdbx_env_set_syncbytes(MDBX_env *env, size_t threshold) {
return __inline_mdbx_env_set_syncbytes(env, threshold);
}
LIBMDBX_API __cold int mdbx_env_get_syncbytes(const MDBX_env *env,
size_t *threshold) {
return __inline_mdbx_env_get_syncbytes(env, threshold);
}
LIBMDBX_API __cold int mdbx_env_set_syncperiod(MDBX_env *env,
unsigned seconds_16dot16) {
return __inline_mdbx_env_set_syncperiod(env, seconds_16dot16);
}
LIBMDBX_API __cold int mdbx_env_get_syncperiod(const MDBX_env *env,
unsigned *seconds_16dot16) {
return __inline_mdbx_env_get_syncperiod(env, seconds_16dot16);
}
LIBMDBX_API __cold MDBX_NOTHROW_CONST_FUNCTION intptr_t
mdbx_limits_pgsize_min(void) {
return __inline_mdbx_limits_pgsize_min();
}
LIBMDBX_API __cold MDBX_NOTHROW_CONST_FUNCTION intptr_t
mdbx_limits_pgsize_max(void) {
return __inline_mdbx_limits_pgsize_max();
}
LIBMDBX_API MDBX_NOTHROW_CONST_FUNCTION uint64_t
mdbx_key_from_int64(const int64_t i64) {
return __inline_mdbx_key_from_int64(i64);
}
LIBMDBX_API MDBX_NOTHROW_CONST_FUNCTION uint32_t
mdbx_key_from_int32(const int32_t i32) {
return __inline_mdbx_key_from_int32(i32);
}
#endif /* LIBMDBX_NO_EXPORTS_LEGACY_API */
/******************************************************************************/
/* *INDENT-OFF* */
/* clang-format off */

View File

@ -1,6 +1,6 @@
.\" Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_CHK 1 "2023-03-03" "MDBX 0.12.4"
.TH MDBX_CHK 1 "2023-04-18" "MDBX 0.12.5"
.SH NAME
mdbx_chk \- MDBX checking tool
.SH SYNOPSIS

View File

@ -1,8 +1,8 @@
.\" Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_COPY 1 "2023-03-03" "MDBX 0.12.4"
.TH MDBX_COPY 1 "2023-04-18" "MDBX 0.12.5"
.SH NAME
mdbx_copy \- MDBX environment copy tool
.SH SYNOPSIS

View File

@ -1,7 +1,7 @@
.\" Copyright 2021-2023 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2014-2021 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_DROP 1 "2023-03-03" "MDBX 0.12.4"
.TH MDBX_DROP 1 "2023-04-18" "MDBX 0.12.5"
.SH NAME
mdbx_drop \- MDBX database delete tool
.SH SYNOPSIS

View File

@ -1,8 +1,8 @@
.\" Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_DUMP 1 "2023-03-03" "MDBX 0.12.4"
.TH MDBX_DUMP 1 "2023-04-18" "MDBX 0.12.5"
.SH NAME
mdbx_dump \- MDBX environment export tool
.SH SYNOPSIS

View File

@ -1,8 +1,8 @@
.\" Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_LOAD 1 "2023-03-03" "MDBX 0.12.4"
.TH MDBX_LOAD 1 "2023-04-18" "MDBX 0.12.5"
.SH NAME
mdbx_load \- MDBX environment import tool
.SH SYNOPSIS

View File

@ -1,8 +1,8 @@
.\" Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_STAT 1 "2023-03-03" "MDBX 0.12.4"
.TH MDBX_STAT 1 "2023-04-18" "MDBX 0.12.5"
.SH NAME
mdbx_stat \- MDBX environment status tool
.SH SYNOPSIS

View File

@ -316,17 +316,16 @@ MDBX_INTERNAL_FUNC int osal_vasprintf(char **strp, const char *fmt,
va_list ap) {
va_list ones;
va_copy(ones, ap);
int needed = vsnprintf(nullptr, 0, fmt, ap);
const int needed = vsnprintf(nullptr, 0, fmt, ones);
va_end(ones);
if (unlikely(needed < 0 || needed >= INT_MAX)) {
*strp = nullptr;
va_end(ones);
return needed;
}
*strp = osal_malloc(needed + (size_t)1);
if (unlikely(*strp == nullptr)) {
va_end(ones);
#if defined(_WIN32) || defined(_WIN64)
SetLastError(MDBX_ENOMEM);
#else
@ -335,9 +334,7 @@ MDBX_INTERNAL_FUNC int osal_vasprintf(char **strp, const char *fmt,
return -1;
}
int actual = vsnprintf(*strp, needed + (size_t)1, fmt, ones);
va_end(ones);
const int actual = vsnprintf(*strp, needed + (size_t)1, fmt, ap);
assert(actual == needed);
if (unlikely(actual < 0)) {
osal_free(*strp);
@ -351,7 +348,7 @@ MDBX_INTERNAL_FUNC int osal_vasprintf(char **strp, const char *fmt,
MDBX_INTERNAL_FUNC int osal_asprintf(char **strp, const char *fmt, ...) {
va_list ap;
va_start(ap, fmt);
int rc = osal_vasprintf(strp, fmt, ap);
const int rc = osal_vasprintf(strp, fmt, ap);
va_end(ap);
return rc;
}

View File

@ -64,9 +64,23 @@ if(${CMAKE_SYSTEM_NAME} STREQUAL "Windows")
endif()
if(UNIX AND NOT SUBPROJECT)
add_executable(pcrf_test pcrf/pcrf_test.c)
target_include_directories(pcrf_test PRIVATE "${PROJECT_SOURCE_DIR}")
target_link_libraries(pcrf_test ${TOOL_MDBX_LIB})
add_executable(test_extra_pcrf extra/pcrf/pcrf_test.c)
target_include_directories(test_extra_pcrf PRIVATE "${PROJECT_SOURCE_DIR}")
target_link_libraries(test_extra_pcrf ${TOOL_MDBX_LIB})
add_executable(test_extra_upsert_alldups extra/upsert_alldups.c)
target_include_directories(test_extra_upsert_alldups PRIVATE "${PROJECT_SOURCE_DIR}")
target_link_libraries(test_extra_upsert_alldups ${TOOL_MDBX_LIB})
if(MDBX_BUILD_CXX)
add_executable(test_extra_maindb_ordinal extra/maindb_ordinal.c++)
target_include_directories(test_extra_maindb_ordinal PRIVATE "${PROJECT_SOURCE_DIR}")
target_link_libraries(test_extra_maindb_ordinal ${TOOL_MDBX_LIB})
if(MDBX_CXX_STANDARD)
set_target_properties(test_extra_maindb_ordinal PROPERTIES
CXX_STANDARD ${MDBX_CXX_STANDARD} CXX_STANDARD_REQUIRED ON)
endif()
endif()
endif()
################################################################################

View File

@ -0,0 +1,59 @@
#include "mdbx.h++"
#include <iostream>
#include <unistd.h>
int main(int argc, const char *argv[]) {
(void)argc;
(void)argv;
unlink("." MDBX_DATANAME);
unlink("." MDBX_LOCKNAME);
mdbx::env_managed env(".", mdbx::env_managed::create_parameters(),
mdbx::env::operate_parameters());
using buffer =
mdbx::buffer<mdbx::default_allocator, mdbx::default_capacity_policy>;
auto txn = env.start_write();
auto map = txn.create_map(nullptr, mdbx::key_mode::ordinal,
mdbx::value_mode::single);
#if 0 /* workaround */
txn.commit();
env.close();
env = mdbx::env_managed(".", mdbx::env_managed::create_parameters(),
mdbx::env::operate_parameters());
txn = env.start_write();
#endif
txn.insert(map, buffer::key_from_u64(UINT64_C(8) << 8 * 0), buffer("a"));
txn.insert(map, buffer::key_from_u64(UINT64_C(7) << 8 * 1), buffer("b"));
txn.insert(map, buffer::key_from_u64(UINT64_C(6) << 8 * 2), buffer("c"));
txn.insert(map, buffer::key_from_u64(UINT64_C(5) << 8 * 3), buffer("d"));
txn.insert(map, buffer::key_from_u64(UINT64_C(4) << 8 * 4), buffer("e"));
txn.insert(map, buffer::key_from_u64(UINT64_C(3) << 8 * 5), buffer("f"));
txn.insert(map, buffer::key_from_u64(UINT64_C(2) << 8 * 6), buffer("g"));
txn.insert(map, buffer::key_from_u64(UINT64_C(1) << 8 * 7), buffer("h"));
txn.commit();
txn = env.start_read();
auto cursor = txn.open_cursor(map);
#if defined(__cpp_lib_string_view) && __cpp_lib_string_view >= 201606L
if (cursor.to_first().value.string_view() == "a" &&
cursor.to_next().value.string_view() == "b" &&
cursor.to_next().value.string_view() == "c" &&
cursor.to_next().value.string_view() == "d" &&
cursor.to_next().value.string_view() == "e" &&
cursor.to_next().value.string_view() == "f" &&
cursor.to_next().value.string_view() == "g" &&
cursor.to_next().value.string_view() == "h" &&
!cursor.to_next(false).done && cursor.eof()) {
std::cout << "OK\n";
return EXIT_SUCCESS;
}
std::cerr << "Fail\n";
return EXIT_FAILURE;
#else
std::cerr << "Skipped since no std::string_view\n";
return EXIT_SUCCESS;
#endif /* __cpp_lib_string_view >= 201606L */
}

195
test/extra/upsert_alldups.c Normal file
View File

@ -0,0 +1,195 @@
//
// libmdbx/test/extra/upsert_alldups.c
//
// Created by Masatoshi Fukunaga <https://gitflic.ru/user/mah0x211>
// on 2023-01-31.
//
#include "mdbx.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
static int dump(MDBX_cursor *cur) {
MDBX_val key = {};
MDBX_val data = {};
int rc = mdbx_cursor_get(cur, &key, &data, MDBX_FIRST);
while (rc == 0) {
printf("(%.*s) = (%.*s)\n", (int)key.iov_len, (const char *)key.iov_base,
(int)data.iov_len, (const char *)data.iov_base);
rc = mdbx_cursor_get(cur, &key, &data, MDBX_NEXT);
}
return rc;
}
static int clear(MDBX_cursor *cur) {
MDBX_val key = {};
MDBX_val data = {};
int rc = mdbx_cursor_get(cur, &key, &data, MDBX_FIRST);
while (rc == 0) {
rc = mdbx_cursor_del(cur, MDBX_ALLDUPS);
if (rc) {
return rc;
}
rc = mdbx_cursor_get(cur, &key, &data, MDBX_NEXT);
}
return (rc == MDBX_NOTFOUND) ? 0 : rc;
}
static int put(MDBX_txn *txn, MDBX_dbi dbi, const char *k, const char *v,
MDBX_put_flags_t flags) {
MDBX_val key = {.iov_base = (void *)k, .iov_len = strlen(k)};
MDBX_val data = {.iov_base = (void *)v, .iov_len = strlen(v)};
return mdbx_put(txn, dbi, &key, &data, flags);
}
int main(int argc, const char *argv[]) {
(void)argc;
(void)argv;
char *errmsg = NULL;
MDBX_env *env = NULL;
MDBX_txn *txn = NULL;
MDBX_cursor *cur = NULL;
MDBX_dbi dbi = 0;
unlink("." MDBX_DATANAME);
unlink("." MDBX_LOCKNAME);
int rc = 0;
if ((rc = mdbx_env_create(&env))) {
errmsg = "failed to mdbx_env_create: %s\n";
goto Fail;
}
if ((rc = mdbx_env_open(
env, ".", MDBX_NOSUBDIR | MDBX_COALESCE | MDBX_LIFORECLAIM, 0644))) {
errmsg = "failed to mdbx_env_open: %s\n";
goto Fail;
}
if ((rc = mdbx_txn_begin(env, NULL, 0, &txn))) {
errmsg = "failed to mdbx_txn_begin: %s\n";
goto Fail;
}
if ((rc = mdbx_dbi_open(txn, NULL, MDBX_DUPSORT | MDBX_CREATE, &dbi))) {
errmsg = "failed to mdbx_dbi_open: %s\n";
goto Fail;
}
if ((rc = mdbx_cursor_open(txn, dbi, &cur))) {
errmsg = "failed to mdbx_cursor_open: %s\n";
goto Fail;
}
#define DUMP() \
do { \
if ((rc = dump(cur)) && rc != MDBX_NOTFOUND) { \
errmsg = "failed to mdbx_cursor_get(FIRST): %s\n"; \
goto Fail; \
} \
puts(""); \
} while (0)
#define PUTVAL(k, v, flags) \
do { \
if ((rc = put(txn, dbi, k, v, flags))) { \
errmsg = "failed to mdbx_put: %s\n"; \
goto Fail; \
} \
} while (0)
puts("TEST WITH MULTIPLE KEYS ====================");
// UPSERTING
// MDBX_UPSERT:
// Key is absent → Insertion (Insertion)
// Key exist → Wanna to add new values (Overwrite by single new value)
puts("insert multiple keys and values {");
puts(" foo = bar, baz, qux");
puts(" hello = world");
puts("}");
PUTVAL("foo", "bar", MDBX_UPSERT);
PUTVAL("foo", "baz", MDBX_UPSERT);
PUTVAL("foo", "qux", MDBX_UPSERT);
PUTVAL("hello", "world", MDBX_UPSERT);
DUMP();
//
// above code will output the fllowing;
//
// insert multiple values {
// foo = bar, baz, qux
// hello = world
// }
// (foo) = (bar)
// (foo) = (baz)
// (foo) = (qux)
// (hello) = (world)
//
// UPSERTING
// MDBX_UPSERT + MDBX_ALLDUPS:
// Key exist → Replace all values with a new one (Overwrite by single new
// value)
puts("overwrite by single new value: MDBX_UPSERT + MDBX_ALLDUPS {");
puts(" foo = baa");
puts("}");
PUTVAL("foo", "baa", MDBX_UPSERT | MDBX_ALLDUPS);
DUMP();
// above code will output the fllowing;
// overwrite by single new value {
// foo = baa
// }
// (foo) = (baa)
// (hello) = (world)
if ((rc = clear(cur))) {
errmsg = "failed to clear: %s\n";
goto Fail;
}
DUMP();
puts("TEST WITH A SINGLE KEY ====================");
// UPSERTING
// MDBX_UPSERT:
// Key is absent → Insertion (Insertion)
// Key exist → Wanna to add new values (Overwrite by single new value)
puts("insert single key and multiple values {");
puts(" foo = bar, baz, qux");
puts("}");
PUTVAL("foo", "bar", MDBX_UPSERT);
PUTVAL("foo", "baz", MDBX_UPSERT);
PUTVAL("foo", "qux", MDBX_UPSERT);
DUMP();
//
// above code will output the fllowing;
//
// insert: foo = bar, baz, qux
// foo = bar
// foo = baz
// foo = qux
// UPSERTING
// MDBX_UPSERT + MDBX_ALLDUPS:
// Key exist → Replace all values with a new one (Overwrite by single new
// value)
puts("overwrite by single new value: MDBX_UPSERT + MDBX_ALLDUPS {");
puts(" foo = baa");
puts("}");
PUTVAL("foo", "baa", MDBX_UPSERT | MDBX_ALLDUPS);
DUMP();
// above code outputs nothing.
// all data associated with key has been deleted.
// Is it a bug? Or, am I misunderstanding how to use it?
if ((rc = mdbx_txn_commit(txn))) {
errmsg = "failed to mdbx_txn_commit: %s\n";
goto Fail;
}
if ((rc = mdbx_env_close(env))) {
errmsg = "failed to mdbx_env_close: %s\n";
goto Fail;
}
return 0;
Fail:
printf(errmsg, mdbx_strerror(rc));
return EXIT_FAILURE;
}