1. Explicitly check and handle a race/collision case with `find_oldest_reader()`.
2. Handle "recovery mode" (me_stuck_meta >= 0) by the same code as for regular latch.
3. Add bailout error message for buggy compiler and/or hardware (paranoid).
1. Fix regression `assert: oldest >= lck->mti_oldest_reader.weak` after d4bf0a3332c7b05331ab0a87e3cd65b0903edc3c.
2. Add explicit check, kick and notice for stuck reader.
3. Made more e2k-frendly.
Кроме небольшого рефакторинга здесь реализуется более регулярный способ
обхода дерева при копировании с компактификаций. В частности, полная
инициализация курсоров позволяет выполнять больше проверок/контроля
структуры БД и избавиться от флажка CC_COPYING.
Beside a small refactoring, a more regular way of traversing the tree
when copying with compactification is implemented here. In particular,
full initialization of cursors allows to perform more checks/control of
the DB structure and get rid of the CC_COPYING flag.
Здесь основная часть изменений преобразующих отладочную проверку страниц
в регулярный и доступный пользователю осторожный/безопасный режим работы
с потенциально поврежденной БД.
Here the major part of the changes that transform a debugging check of
pages into a regular and user-accessible careful/safe mode for working
with a potentially corrupted database.
Chunking long list of retired pages during huge transactions commit
to avoid use sequences of pages:
- splits a long retired page-number-list into chunks
which fits one per single overflow/large page;
- this requires a few unique id for keys
for create such records into GC/freeDB;
- just use the necessary subsequent IDs following the current
transaction ID and then take the last of ones to update a meta-page.
Thus avoids using/allocating/searching a sequence of free pages
but just increase txnid more than one during the commit
a huge write transaction with a long retired-pages-list.
The stable release with an important fixes and workaround for the critical macOS thread-local-storage issue.
Acknowledgements:
-----------------
- [Masatoshi Fukunaga](https://github.com/mah0x211) for [Lua bindings](https://github.com/mah0x211/lua-libmdbx).
New:
----
- Added most of transactions flags to the public API.
- Added `MDBX_NOSUCCESS_EMPTY_COMMIT` build option to return non-success result (`MDBX_RESULT_TRUE`) on empty commit.
- Reworked validation and import of DBI-handles into a transaction.
Assumes these changes will be invisible to most users, but will cause fewer surprises in complex DBI cases.
- Added ability to open DB in without-LCK (exclusive read-only) mode in case no permissions to create/write LCK-file.
Fixes:
------
- A series of fixes and improvements for automatically generated documentation (Doxygen).
- Fixed copy&paste bug with could lead to `SIGSEGV` (nullptr dereference) in the exclusive/no-lck mode.
- Fixed minor warnings from modern Apple's CLANG 13.
- Fixed minor warnings from CLANG 14 and in-development CLANG 15.
- Fixed `SIGSEGV` regression in without-LCK (exclusive read-only) mode.
- Fixed `mdbx_check_fs_local()` for CDROM case on Windows.
- Fixed nasty typo of typename which caused false `MDBX_CORRUPTED` error in a rare execution path,
when the size of the thread-ID type not equal to 8.
- Fixed write-after-free memory corruption on latest `macOS` during finalization/cleanup of thread(s) that executed read transaction(s).
> The issue was suddenly discovered by a [CI](https://en.wikipedia.org/wiki/Continuous_integration)
> after adding an iteration with macOS 11 "Big Sur", and then reproduced on recent release of macOS 12 "Monterey".
> The issue was never noticed nor reported on macOS 10 "Catalina" nor others.
> Analysis shown that the problem caused by a change in the behavior of the system library (internals of dyld and pthread)
> during thread finalization/cleanup: now a memory allocated for a `__thread` variable(s) is released
> before execution of the registered Thread-Local-Storage destructor(s),
> thus a TLS-destructor will write-after-free just by legitime dereference any `__thread` variable.
> This is unexpected crazy-like behavior since the order of resources releasing/destroying
> is not the reverse of ones acquiring/construction order. Nonetheless such surprise
> is now workarounded by using atomic compare-and-swap operations on a 64-bit signatures/cookies.
- Fixed Elbrus/E2K LCC 1.26 compiler warnings (memory model for atomic operations, etc).
Minors:
-------
- Refined `release-assets` GNU Make target.
- Added logging to `mdbx_fetch_sdb()` to help debugging complex DBI-handels use cases.
- Added explicit error message from probe of no-support for `std::filesystem`.
- Added contributors "score" table by `git fame` to generated docs.
- Added `mdbx_assert_fail()` to public API (mostly for backtracing).
- Now C++20 concepts used/enabled only when `__cpp_lib_concepts >= 202002`.
- Don't provide nor report package information if used as a CMake subproject.
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
The stable risen release after the Github's intentional malicious disaster.
We have migrated to a reliable trusted infrastructure
-----------------------------------------------------
The origin for now is at [GitFlic](https://gitflic.ru/project/erthink/libmdbx)
since on 2022-04-15 the Github administration, without any warning nor
explanation, deleted _libmdbx_ along with a lot of other projects,
simultaneously blocking access for many developers.
For the same reason ~~Github~~ is blacklisted forever.
GitFlic already support Russian and English languages, plan to support more,
including 和 中文. You are welcome!
New:
----
- Added the `tools-static` make target to build statically linked MDBX tools.
- Support for Microsoft Visual Studio 2022.
- Support build by MinGW' make from command line without CMake.
- Added `mdbx::filesystem` C++ API namespace that corresponds to `std::filesystem` or `std::experimental::filesystem`.
- Created [website](https://libmdbx.website.yandexcloud.net/) for online auto-generated documentation.
- Used `todo4recovery://erased_by_github/` for dead (or temporarily lost) resources deleted by ~~Github~~.
- Added `--loglevel=` command-line option to the `mdbx_test` tool.
- Added few fast smoke-like tests into CMake builds.
Fixes:
------
- Fixed a race between starting a transaction and creating a DBI descriptor that could lead to `SIGSEGV` in the cursor tracking code.
- Clarified description of `MDBX_EPERM` error returned from `mdbx_env_set_geometry()`.
- Fixed non-promoting the parent transaction to be dirty in case the undo of the geometry update failed during abortion of a nested transaction.
- Resolved linking issues with `libstdc++fs`/`libc++fs`/`libc++experimental` for C++ `std::filesystem` or `std::experimental::filesystem` for legacy compilers.
- Added workaround for GNU Make 3.81 and earlier.
- Added workaround for Elbrus/LCC 1.25 compiler bug of class inline `static constexpr` member field.
- [Fixed](https://github.com/ledgerwatch/erigon/issues/3874) minor assertion regression (only debug builds were affected).
- Fixed detection of `C++20` concepts accessibility.
- Fixed detection of Clang's LTO availability for Android.
- Fixed build for ARM/ARM64 by MSVC.
- Fixed non-x86 Windows builds with `MDBX_WITHOUT_MSVC_CRT=ON` and `MDBX_BUILD_SHARED_LIBRARY=ON`.
Minors:
-------
- Resolve minor MSVC warnings: avoid `/INCREMENTAL[:YES]` with `/LTCG`, `/W4` with `/W3`, the `C5105` warning.
- Switched to using `MDBX_EPERM` instead of `MDBX_RESULT_TRUE' to indicate that the geometry cannot be updated.
- Added `NULL` checking during memory allocation inside `mdbx_chk`.
- Resolved all warnings from MinGW while used without CMake.
- Added inheretable `target_include_directories()` to `CMakeLists.txt` for easy integration.
- Added build-time checks and paranoid runtime assertions for the `off_t` arguments of `fcntl()` which are used for locking.
- Added `-Wno-lto-type-mismatch` to avoid false-positive warnings from old GCC during LTO-enabled builds.
- Added checking for TID (system thread id) to avoid hang on 32-bit Bionic/Android within `pthread_mutex_lock()`.
- Reworked `MDBX_BUILD_TARGET` of CMake builds.
- Added `CMAKE_HOST_ARCH` and `CMAKE_HOST_CAN_RUN_EXECUTABLES_BUILT_FOR_TARGET`.
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
Fixes https://github.com/ledgerwatch/erigon/issues/3874.
This was a minor regression after the c4a5325aafd3f03ce7520731b9da7253d7d178f0
that affects only debug builgs (with enabled assertions) and only when the added
code catch a incoherency of unified page/buffer cache.
