The stable bugfix release.
It is planned that this will be the last release of the v0.11 branch.
Acknowledgements:
-----------------
- [Alex Sharov](https://github.com/AskAlexSharov) and Erigon team for reporting and testing.
- [Andrew Ashikhmin](https://gitflic.ru/user/yperbasis) for contributing.
New:
----
- Ability to customise `MDBX_LOCK_SUFFIX`, `MDBX_DATANAME`, `MDBX_LOCKNAME` just by predefine ones during build.
- Added to [`mdbx::env_managed`](https://libmdbx.dqdkfa.ru/group__cxx__api.html#classmdbx_1_1env__managed)'s methods a few overloads with `const char* pathname` parameter (C++ API).
Fixes:
------
- Fixed hang copy-with-compactification of a corrupted DB
or in case the volume of output pages is a multiple of `MDBX_ENVCOPY_WRITEBUF`.
- Fixed standalone non-CMake build on MacOS (`#include AvailabilityMacros.h>`).
- Fixed unexpected `MDBX_PAGE_FULL` error in rare cases with large database page sizes.
Minors:
-------
- Minor fixes Doxygen references, comments, descriptions, etc.
- Fixed copy&paste typo inside `meta_checktxnid()`.
- Minor fix `meta_checktxnid()` to avoid assertion in debug mode.
- Minor fix `mdbx_env_set_geometry()` to avoid returning `EINVAL` in particular rare cases.
- Minor refine/fix batch-get testcase for large page size.
- Added `--pagesize NN` option to long-stotastic test script.
- Updated Valgrind-suppressions file for modern GCC.
- Fixed `has no symbols` warning from Apple's ranlib.
18 files changed, 318 insertions(+), 178 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
1. Explicitly check and handle a race/collision case with `find_oldest_reader()`.
2. Handle "recovery mode" (me_stuck_meta >= 0) by the same code as for regular latch.
3. Add bailout error message for buggy compiler and/or hardware (paranoid).
1. Fix regression `assert: oldest >= lck->mti_oldest_reader.weak` after d4bf0a3332c7b05331ab0a87e3cd65b0903edc3c.
2. Add explicit check, kick and notice for stuck reader.
3. Made more e2k-frendly.
Кроме небольшого рефакторинга здесь реализуется более регулярный способ
обхода дерева при копировании с компактификаций. В частности, полная
инициализация курсоров позволяет выполнять больше проверок/контроля
структуры БД и избавиться от флажка CC_COPYING.
Beside a small refactoring, a more regular way of traversing the tree
when copying with compactification is implemented here. In particular,
full initialization of cursors allows to perform more checks/control of
the DB structure and get rid of the CC_COPYING flag.
Здесь основная часть изменений преобразующих отладочную проверку страниц
в регулярный и доступный пользователю осторожный/безопасный режим работы
с потенциально поврежденной БД.
Here the major part of the changes that transform a debugging check of
pages into a regular and user-accessible careful/safe mode for working
with a potentially corrupted database.
Chunking long list of retired pages during huge transactions commit
to avoid use sequences of pages:
- splits a long retired page-number-list into chunks
which fits one per single overflow/large page;
- this requires a few unique id for keys
for create such records into GC/freeDB;
- just use the necessary subsequent IDs following the current
transaction ID and then take the last of ones to update a meta-page.
Thus avoids using/allocating/searching a sequence of free pages
but just increase txnid more than one during the commit
a huge write transaction with a long retired-pages-list.
The stable release with an important fixes and workaround for the critical macOS thread-local-storage issue.
Acknowledgements:
-----------------
- [Masatoshi Fukunaga](https://github.com/mah0x211) for [Lua bindings](https://github.com/mah0x211/lua-libmdbx).
New:
----
- Added most of transactions flags to the public API.
- Added `MDBX_NOSUCCESS_EMPTY_COMMIT` build option to return non-success result (`MDBX_RESULT_TRUE`) on empty commit.
- Reworked validation and import of DBI-handles into a transaction.
Assumes these changes will be invisible to most users, but will cause fewer surprises in complex DBI cases.
- Added ability to open DB in without-LCK (exclusive read-only) mode in case no permissions to create/write LCK-file.
Fixes:
------
- A series of fixes and improvements for automatically generated documentation (Doxygen).
- Fixed copy&paste bug with could lead to `SIGSEGV` (nullptr dereference) in the exclusive/no-lck mode.
- Fixed minor warnings from modern Apple's CLANG 13.
- Fixed minor warnings from CLANG 14 and in-development CLANG 15.
- Fixed `SIGSEGV` regression in without-LCK (exclusive read-only) mode.
- Fixed `mdbx_check_fs_local()` for CDROM case on Windows.
- Fixed nasty typo of typename which caused false `MDBX_CORRUPTED` error in a rare execution path,
when the size of the thread-ID type not equal to 8.
- Fixed write-after-free memory corruption on latest `macOS` during finalization/cleanup of thread(s) that executed read transaction(s).
> The issue was suddenly discovered by a [CI](https://en.wikipedia.org/wiki/Continuous_integration)
> after adding an iteration with macOS 11 "Big Sur", and then reproduced on recent release of macOS 12 "Monterey".
> The issue was never noticed nor reported on macOS 10 "Catalina" nor others.
> Analysis shown that the problem caused by a change in the behavior of the system library (internals of dyld and pthread)
> during thread finalization/cleanup: now a memory allocated for a `__thread` variable(s) is released
> before execution of the registered Thread-Local-Storage destructor(s),
> thus a TLS-destructor will write-after-free just by legitime dereference any `__thread` variable.
> This is unexpected crazy-like behavior since the order of resources releasing/destroying
> is not the reverse of ones acquiring/construction order. Nonetheless such surprise
> is now workarounded by using atomic compare-and-swap operations on a 64-bit signatures/cookies.
- Fixed Elbrus/E2K LCC 1.26 compiler warnings (memory model for atomic operations, etc).
Minors:
-------
- Refined `release-assets` GNU Make target.
- Added logging to `mdbx_fetch_sdb()` to help debugging complex DBI-handels use cases.
- Added explicit error message from probe of no-support for `std::filesystem`.
- Added contributors "score" table by `git fame` to generated docs.
- Added `mdbx_assert_fail()` to public API (mostly for backtracing).
- Now C++20 concepts used/enabled only when `__cpp_lib_concepts >= 202002`.
- Don't provide nor report package information if used as a CMake subproject.
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
The stable risen release after the Github's intentional malicious disaster.
We have migrated to a reliable trusted infrastructure
-----------------------------------------------------
The origin for now is at [GitFlic](https://gitflic.ru/project/erthink/libmdbx)
since on 2022-04-15 the Github administration, without any warning nor
explanation, deleted _libmdbx_ along with a lot of other projects,
simultaneously blocking access for many developers.
For the same reason ~~Github~~ is blacklisted forever.
GitFlic already support Russian and English languages, plan to support more,
including 和 中文. You are welcome!
New:
----
- Added the `tools-static` make target to build statically linked MDBX tools.
- Support for Microsoft Visual Studio 2022.
- Support build by MinGW' make from command line without CMake.
- Added `mdbx::filesystem` C++ API namespace that corresponds to `std::filesystem` or `std::experimental::filesystem`.
- Created [website](https://libmdbx.website.yandexcloud.net/) for online auto-generated documentation.
- Used `todo4recovery://erased_by_github/` for dead (or temporarily lost) resources deleted by ~~Github~~.
- Added `--loglevel=` command-line option to the `mdbx_test` tool.
- Added few fast smoke-like tests into CMake builds.
Fixes:
------
- Fixed a race between starting a transaction and creating a DBI descriptor that could lead to `SIGSEGV` in the cursor tracking code.
- Clarified description of `MDBX_EPERM` error returned from `mdbx_env_set_geometry()`.
- Fixed non-promoting the parent transaction to be dirty in case the undo of the geometry update failed during abortion of a nested transaction.
- Resolved linking issues with `libstdc++fs`/`libc++fs`/`libc++experimental` for C++ `std::filesystem` or `std::experimental::filesystem` for legacy compilers.
- Added workaround for GNU Make 3.81 and earlier.
- Added workaround for Elbrus/LCC 1.25 compiler bug of class inline `static constexpr` member field.
- [Fixed](https://github.com/ledgerwatch/erigon/issues/3874) minor assertion regression (only debug builds were affected).
- Fixed detection of `C++20` concepts accessibility.
- Fixed detection of Clang's LTO availability for Android.
- Fixed build for ARM/ARM64 by MSVC.
- Fixed non-x86 Windows builds with `MDBX_WITHOUT_MSVC_CRT=ON` and `MDBX_BUILD_SHARED_LIBRARY=ON`.
Minors:
-------
- Resolve minor MSVC warnings: avoid `/INCREMENTAL[:YES]` with `/LTCG`, `/W4` with `/W3`, the `C5105` warning.
- Switched to using `MDBX_EPERM` instead of `MDBX_RESULT_TRUE' to indicate that the geometry cannot be updated.
- Added `NULL` checking during memory allocation inside `mdbx_chk`.
- Resolved all warnings from MinGW while used without CMake.
- Added inheretable `target_include_directories()` to `CMakeLists.txt` for easy integration.
- Added build-time checks and paranoid runtime assertions for the `off_t` arguments of `fcntl()` which are used for locking.
- Added `-Wno-lto-type-mismatch` to avoid false-positive warnings from old GCC during LTO-enabled builds.
- Added checking for TID (system thread id) to avoid hang on 32-bit Bionic/Android within `pthread_mutex_lock()`.
- Reworked `MDBX_BUILD_TARGET` of CMake builds.
- Added `CMAKE_HOST_ARCH` and `CMAKE_HOST_CAN_RUN_EXECUTABLES_BUILT_FOR_TARGET`.
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
Fixes https://github.com/ledgerwatch/erigon/issues/3874.
This was a minor regression after the c4a5325aafd3f03ce7520731b9da7253d7d178f0
that affects only debug builgs (with enabled assertions) and only when the added
code catch a incoherency of unified page/buffer cache.
The error was that the array of pointers in the transaction zeroed by the
value of env->me_numdbs and before txn->mt_numdbs was set to env->me_numdbs.
Thus, a cursor pointer(s) in the starting transaction could uninitialized if
another thread opened a dbi-handle(s) between the two aforementioned events.
The stable release with the complete workaround for an incoherence flaw of Linux unified page/buffer cache.
Nonetheless the cause for this trouble may be an issue of Intel CPU cache/MESI.
See [issue#269](https://github.com/erthink/libmdbx/issues/269) for more information.
Acknowledgements:
-----------------
- [David Bouyssié](https://github.com/david-bouyssie) for [Scala bindings](https://github.com/david-bouyssie/mdbx4s).
- [Michelangelo Riccobene](https://github.com/mriccobene) for reporting and testing.
Fixes:
------
- [Added complete workaround](https://github.com/erthink/libmdbx/issues/269) for an incoherence flaw of Linux unified page/buffer cache.
- [Fixed](https://github.com/erthink/libmdbx/issues/272) cursor reusing for read-only transactions.
- Fixed copy&paste typo inside `mdbx::cursor::find_multivalue()`.
Minors:
-------
- Minor refine C++ API for convenience.
- Minor internals refines.
- Added `lib-static` and `lib-shared` targets for make.
- Added minor workaround for AppleClang 13.3 bug.
- Clarified error messages of a signature/version mismatch.
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
