From 70e80067760ab24dff5fbc7d8e5f49194378262e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 10 Nov 2022 15:54:31 +0300 Subject: [PATCH] =?UTF-8?q?mdbx-docs:=20=D0=BF=D0=B5=D1=80=D0=B5=D0=BD?= =?UTF-8?q?=D0=B0=D0=BF=D1=80=D0=B0=D0=B2=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20?= =?UTF-8?q?github-=D1=81=D1=81=D1=8B=D0=BB=D0=BE=D0=BA=20=D0=BD=D0=B0=20we?= =?UTF-8?q?b-archive.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ChangeLog.md | 130 ++++++++++++++++++------------------- README.md | 6 +- TODO.md | 22 +++---- mdbx.h++ | 2 +- src/core.c | 20 +++--- src/mdbx_chk.c | 2 +- src/osal.c | 12 ++-- test/valgrind_suppress.txt | 2 +- 8 files changed, 98 insertions(+), 98 deletions(-) diff --git a/ChangeLog.md b/ChangeLog.md index fda8ce33..d162ba94 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -353,7 +353,7 @@ New: - Support build by MinGW' make from command line without CMake. - Added `mdbx::filesystem` C++ API namespace that corresponds to `std::filesystem` or `std::experimental::filesystem`. - Created [website](https://libmdbx.dqdkfa.ru/) for online auto-generated documentation. - - Used `https://web.archive.org/web/20220414235959/https://github.com/erthink/` for dead (or temporarily lost) resources deleted by ~~Github~~. + - Used `https://web.archive.org/web/https://github.com/erthink/libmdbx` for dead (or temporarily lost) resources deleted by ~~Github~~. - Added `--loglevel=` command-line option to the `mdbx_test` tool. - Added few fast smoke-like tests into CMake builds. @@ -393,7 +393,7 @@ Minors: The stable release with the complete workaround for an incoherence flaw of Linux unified page/buffer cache. Nonetheless the cause for this trouble may be an issue of Intel CPU cache/MESI. -See [issue#269](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/269) for more information. +See [issue#269](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/269) for more information. Acknowledgements: @@ -402,8 +402,8 @@ Acknowledgements: Fixes: - - [Added complete workaround](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/269) for an incoherence flaw of Linux unified page/buffer cache. - - [Fixed](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/272) cursor reusing for read-only transactions. + - [Added complete workaround](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/269) for an incoherence flaw of Linux unified page/buffer cache. + - [Fixed](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/272) cursor reusing for read-only transactions. - Fixed copy&paste typo inside `mdbx::cursor::find_multivalue()`. Minors: @@ -418,7 +418,7 @@ Minors: ## v0.11.5 at 2022-02-23 The release with the temporary hotfix for a flaw of Linux unified page/buffer cache. -See [issue#269](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/269) for more information. +See [issue#269](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/269) for more information. Acknowledgements: @@ -428,10 +428,10 @@ Acknowledgements: Fixes: - - [Added hotfix](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/269) for a flaw of Linux unified page/buffer cache. - - [Fixed/Reworked](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/pull/270) move-assignment operators for "managed" classes of C++ API. + - [Added hotfix](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/269) for a flaw of Linux unified page/buffer cache. + - [Fixed/Reworked](https://web.archive.org/web/https://github.com/erthink/libmdbx/pull/270) move-assignment operators for "managed" classes of C++ API. - Fixed potential `SIGSEGV` while open DB with overrided non-default page size. - - [Made](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/267) `mdbx_env_open()` idempotence in failure cases. + - [Made](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/267) `mdbx_env_open()` idempotence in failure cases. - Refined/Fixed pages reservation inside `mdbx_update_gc()` to avoid non-reclamation in a rare cases. - Fixed typo in a retained space calculation for the hsr-callback. @@ -464,15 +464,15 @@ New features, extensions and improvements: Fixes: - Fixed handling `MDBX_opt_rp_augment_limit` for GC's records from huge transactions (Erigon/Akula/Ethereum). - - [Fixed](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/258) build on Android (avoid including `sys/sem.h`). - - [Fixed](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/pull/261) missing copy assignment operator for `mdbx::move_result`. + - [Fixed](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/258) build on Android (avoid including `sys/sem.h`). + - [Fixed](https://web.archive.org/web/https://github.com/erthink/libmdbx/pull/261) missing copy assignment operator for `mdbx::move_result`. - Fixed missing `&` for `std::ostream &operator<<()` overloads. - Fixed unexpected `EXDEV` (Cross-device link) error from `mdbx_env_copy()`. - Fixed base64 encoding/decoding bugs in auxillary C++ API. - Fixed overflow of `pgno_t` during checking PNL on 64-bit platforms. - - [Fixed](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/260) excessive PNL checking after sort for spilling. + - [Fixed](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/260) excessive PNL checking after sort for spilling. - Reworked checking `MAX_PAGENO` and DB upper-size geometry limit. - - [Fixed](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/265) build for some combinations of versions of MSVC and Windows SDK. + - [Fixed](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/265) build for some combinations of versions of MSVC and Windows SDK. Minors: @@ -499,10 +499,10 @@ Acknowledgements: New features, extensions and improvements: - - [Added](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/236) `mdbx_cursor_get_batch()`. - - [Added](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/250) `MDBX_SET_UPPERBOUND`. + - [Added](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/236) `mdbx_cursor_get_batch()`. + - [Added](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/250) `MDBX_SET_UPPERBOUND`. - C++ API is finalized now. - - The GC update stage has been [significantly speeded](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/254) when fixing huge Erigon's transactions (Ethereum ecosystem). + - The GC update stage has been [significantly speeded](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/254) when fixing huge Erigon's transactions (Ethereum ecosystem). Fixes: @@ -513,12 +513,12 @@ Minors: - Fixed returning `MDBX_RESULT_TRUE` (unexpected -1) from `mdbx_env_set_option()`. - Added `mdbx_env_get_syncbytes()` and `mdbx_env_get_syncperiod()`. - - [Clarified](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/pull/249) description of `MDBX_INTEGERKEY`. + - [Clarified](https://web.archive.org/web/https://github.com/erthink/libmdbx/pull/249) description of `MDBX_INTEGERKEY`. - Reworked/simplified `mdbx_env_sync_internal()`. - - [Fixed](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/248) extra assertion inside `mdbx_cursor_put()` for `MDBX_DUPFIXED` cases. + - [Fixed](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/248) extra assertion inside `mdbx_cursor_put()` for `MDBX_DUPFIXED` cases. - Avoiding extra looping inside `mdbx_env_info_ex()`. - Explicitly enabled core dumps from stochastic tests scripts on Linux. - - [Fixed](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/253) `mdbx_override_meta()` to avoid false-positive assertions. + - [Fixed](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/253) `mdbx_override_meta()` to avoid false-positive assertions. - For compatibility reverted returning `MDBX_ENODATA`for some cases. @@ -534,10 +534,10 @@ Acknowledgements: Fixes: - - [Fixed compilation](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/pull/239) with `devtoolset-9` on CentOS/RHEL 7. - - [Fixed unexpected `MDBX_PROBLEM` error](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/242) because of update an obsolete meta-page. - - [Fixed returning `MDBX_NOTFOUND` error](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/243) in case an inexact value found for `MDBX_GET_BOTH` operation. - - [Fixed compilation](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/245) without kernel/libc-devel headers. + - [Fixed compilation](https://web.archive.org/web/https://github.com/erthink/libmdbx/pull/239) with `devtoolset-9` on CentOS/RHEL 7. + - [Fixed unexpected `MDBX_PROBLEM` error](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/242) because of update an obsolete meta-page. + - [Fixed returning `MDBX_NOTFOUND` error](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/243) in case an inexact value found for `MDBX_GET_BOTH` operation. + - [Fixed compilation](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/245) without kernel/libc-devel headers. Minors: @@ -554,7 +554,7 @@ Minors: The database format signature has been changed to prevent forward-interoperability with an previous releases, which may lead to a -[false positive diagnosis of database corruption](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/238) +[false positive diagnosis of database corruption](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/238) due to flaws of an old library versions. This change is mostly invisible: @@ -606,7 +606,7 @@ Acknowledgements: Fixes: - Fixed possibility of looping update GC during transaction commit (no public issue since the problem was discovered inside [Positive Technologies](https://www.ptsecurity.ru)). - - Fixed `#pragma pack` to avoid provoking some compilers to generate code with [unaligned access](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/235). + - Fixed `#pragma pack` to avoid provoking some compilers to generate code with [unaligned access](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/235). - Fixed `noexcept` for potentially throwing `txn::put()` of C++ API. Minors: @@ -632,7 +632,7 @@ Extensions and improvements: Fixes: - - Always setup `madvise` while opening DB (fixes https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/231). + - Always setup `madvise` while opening DB (fixes https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/231). - Fixed checking legacy `P_DIRTY` flag (`0x10`) for nested/sub-pages. Minors: @@ -653,11 +653,11 @@ Acknowledgements: - [Lionel Debroux](https://github.com/debrouxl) for fuzzing tests and reporting bugs. - [Sergey Fedotov](https://github.com/SergeyFromHell/) for [`node-mdbx` NodeJS bindings](https://www.npmjs.com/package/node-mdbx). - [Kris Zyp](https://github.com/kriszyp) for [`lmdbx-store` NodeJS bindings](https://github.com/kriszyp/lmdbx-store). - - [Noel Kuntze](https://github.com/Thermi) for [draft Python bindings](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/commits/python-bindings). + - [Noel Kuntze](https://github.com/Thermi) for [draft Python bindings](https://web.archive.org/web/https://github.com/erthink/libmdbx/commits/python-bindings). New features, extensions and improvements: - - [Allow to predefine/override `MDBX_BUILD_TIMESTAMP` for builds reproducibility](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/201). + - [Allow to predefine/override `MDBX_BUILD_TIMESTAMP` for builds reproducibility](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/201). - Added options support for `long-stochastic` script. - Avoided `MDBX_TXN_FULL` error for large transactions when possible. - The `MDBX_READERS_LIMIT` increased to `32767`. @@ -665,7 +665,7 @@ New features, extensions and improvements: - Minimized the size of poisoned/unpoisoned regions to avoid Valgrind/ASAN stuck. - Added more workarounds for QEMU for testing builds for 32-bit platforms, Alpha and Sparc architectures. - `mdbx_chk` now skips iteration & checking of DB' records if corresponding page-tree is corrupted (to avoid `SIGSEGV`, ASAN failures, etc). - - Added more checks for [rare/fuzzing corruption cases](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/217). + - Added more checks for [rare/fuzzing corruption cases](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/217). Backward compatibility break: @@ -677,18 +677,18 @@ Backward compatibility break: Fixes: - Fixed excess meta-pages checks in case `mdbx_chk` is called to check the DB for a specific meta page and thus could prevent switching to the selected meta page, even if the check passed without errors. - - Fixed [recursive use of SRW-lock on Windows cause by `MDBX_NOTLS` option](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/203). - - Fixed [log a warning during a new DB creation](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/205). - - Fixed [false-negative `mdbx_cursor_eof()` result](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/207). - - Fixed [`make install` with non-GNU `install` utility (OSX, BSD)](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/208). - - Fixed [installation by `CMake` in special cases by complete use `GNUInstallDirs`'s variables](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/209). - - Fixed [C++ Buffer issue with `std::string` and alignment](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/191). + - Fixed [recursive use of SRW-lock on Windows cause by `MDBX_NOTLS` option](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/203). + - Fixed [log a warning during a new DB creation](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/205). + - Fixed [false-negative `mdbx_cursor_eof()` result](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/207). + - Fixed [`make install` with non-GNU `install` utility (OSX, BSD)](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/208). + - Fixed [installation by `CMake` in special cases by complete use `GNUInstallDirs`'s variables](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/209). + - Fixed [C++ Buffer issue with `std::string` and alignment](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/191). - Fixed `safe64_reset()` for platforms without atomic 64-bit compare-and-swap. - Fixed hang/shutdown on big-endian platforms without `__cxa_thread_atexit()`. - - Fixed [using bad meta-pages if DB was partially/recoverable corrupted](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/217). + - Fixed [using bad meta-pages if DB was partially/recoverable corrupted](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/217). - Fixed extra `noexcept` for `buffer::&assign_reference()`. - Fixed `bootid` generation on Windows for case of change system' time. - - Fixed [test framework keygen-related issue](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/127). + - Fixed [test framework keygen-related issue](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/127). ## v0.10.1 at 2021-06-01 @@ -709,10 +709,10 @@ New features: Fixes: - Fixed minor "foo not used" warnings from modern C++ compilers when building the C++ part of the library. - - Fixed confusing/messy errors when build library from unfit github's archives (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/197). + - Fixed confusing/messy errors when build library from unfit github's archives (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/197). - Fixed `#​e​l​s​i​f` typo. - - Fixed rare unexpected `MDBX_PROBLEM` error during altering data in huge transactions due to wrong spilling/oust of dirty pages (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/195). - - Re-Fixed WSL1/WSL2 detection with distinguishing (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/97). + - Fixed rare unexpected `MDBX_PROBLEM` error during altering data in huge transactions due to wrong spilling/oust of dirty pages (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/195). + - Re-Fixed WSL1/WSL2 detection with distinguishing (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/97). ## v0.10.0 at 2021-05-09 @@ -735,7 +735,7 @@ New features: and conjointly with the `MDBX_ENV_CHECKPID=0` and `MDBX_TXN_CHECKOWNER=0` options can yield up to 30% more performance compared to LMDB. - Using float point (exponential quantized) representation for internal 16-bit values - of grow step and shrink threshold when huge ones (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/166). + of grow step and shrink threshold when huge ones (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/166). To minimize the impact on compatibility, only the odd values inside the upper half of the range (i.e. 32769..65533) are used for the new representation. - Added the `mdbx_drop` similar to LMDB command-line tool to purge or delete (sub)database(s). @@ -744,7 +744,7 @@ New features: - The internal node sizes were refined, resulting in a reduction in large/overflow pages in some use cases and a slight increase in limits for a keys size to ≈½ of page size. - Added to `mdbx_chk` output number of keys/items on pages. - - Added explicit `install-strip` and `install-no-strip` targets to the `Makefile` (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/pull/180). + - Added explicit `install-strip` and `install-no-strip` targets to the `Makefile` (https://web.archive.org/web/https://github.com/erthink/libmdbx/pull/180). - Major rework page splitting (af9b7b560505684249b76730997f9e00614b8113) for - An "auto-appending" feature upon insertion for both ascending and descending key sequences. As a result, the optimality of page filling @@ -752,7 +752,7 @@ New features: inserting ordered sequences of keys, - A "splitting at middle" to make page tree more balanced on average. - Added `mdbx_get_sysraminfo()` to the API. - - Added guessing a reasonable maximum DB size for the default upper limit of geometry (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/183). + - Added guessing a reasonable maximum DB size for the default upper limit of geometry (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/183). - Major rework internal labeling of a dirty pages (958fd5b9479f52f2124ab7e83c6b18b04b0e7dda) for a "transparent spilling" feature with the gist to make a dirty pages be ready to spilling (writing to a disk) without further altering ones. @@ -768,7 +768,7 @@ New features: - Support `make help` to list available make targets. - Silently `make`'s build by default. - Preliminary [Python bindings](https://github.com/Thermi/libmdbx/tree/python-bindings) is available now - by [Noel Kuntze](https://github.com/Thermi) (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/147). + by [Noel Kuntze](https://github.com/Thermi) (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/147). Backward compatibility break: @@ -783,22 +783,22 @@ Backward compatibility break: Fixes: - - Fixed performance regression due non-optimal C11 atomics usage (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/160). - - Fixed "reincarnation" of subDB after it deletion (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/168). + - Fixed performance regression due non-optimal C11 atomics usage (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/160). + - Fixed "reincarnation" of subDB after it deletion (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/168). - Fixed (disallowing) implicit subDB deletion via operations on `@MAIN`'s DBI-handle. - - Fixed a crash of `mdbx_env_info_ex()` in case of a call for a non-open environment (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/171). - - Fixed the selecting/adjustment values inside `mdbx_env_set_geometry()` for implicit out-of-range cases (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/170). - - Fixed `mdbx_env_set_option()` for set initial and limit size of dirty page list ((https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/179). - - Fixed an unreasonably huge default upper limit for DB geometry (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/183). + - Fixed a crash of `mdbx_env_info_ex()` in case of a call for a non-open environment (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/171). + - Fixed the selecting/adjustment values inside `mdbx_env_set_geometry()` for implicit out-of-range cases (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/170). + - Fixed `mdbx_env_set_option()` for set initial and limit size of dirty page list ((https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/179). + - Fixed an unreasonably huge default upper limit for DB geometry (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/183). - Fixed `constexpr` specifier for the `slice::invalid()`. - - Fixed (no)readahead auto-handling (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/164). + - Fixed (no)readahead auto-handling (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/164). - Fixed non-alloy build for Windows. - Switched to using Heap-functions instead of LocalAlloc/LocalFree on Windows. - - Fixed `mdbx_env_stat_ex()` to returning statistics of the whole environment instead of MainDB only (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/190). + - Fixed `mdbx_env_stat_ex()` to returning statistics of the whole environment instead of MainDB only (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/190). - Fixed building by GCC 4.8.5 (added workaround for a preprocessor's bug). - Fixed building C++ part for iOS <= 13.0 (unavailability of `std::filesystem::path`). - Fixed building for Windows target versions prior to Windows Vista (`WIN32_WINNT < 0x0600`). - - Fixed building by MinGW for Windows (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/155). + - Fixed building by MinGW for Windows (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/155). ------------------------------------------------------------------------------- @@ -821,7 +821,7 @@ Removed options and features: New features: - Package for FreeBSD is available now by Mahlon E. Smith. - - New API functions to get/set various options (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/128): + - New API functions to get/set various options (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/128): - the maximum number of named databases for the environment; - the maximum number of threads/reader slots; - threshold (since the last unsteady commit) to force flush the data buffers to disk; @@ -834,7 +834,7 @@ New features: - maximal part of the dirty pages may be spilled when necessary; - minimal part of the dirty pages should be spilled when necessary; - how much of the parent transaction dirty pages will be spilled while start each child transaction; - - Unlimited/Dynamic size of retired and dirty page lists (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/123). + - Unlimited/Dynamic size of retired and dirty page lists (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/123). - Added `-p` option (purge subDB before loading) to `mdbx_load` tool. - Reworked spilling of large transaction and committing of nested transactions: - page spilling code reworked to avoid the flaws and bugs inherited from LMDB; @@ -844,22 +844,22 @@ New features: - Added `MDBX_ENABLE_REFUND` and `MDBX_PNL_ASCENDING` internal/advanced build options. - Added `mdbx_default_pagesize()` function. - Better support architectures with a weak/relaxed memory consistency model (ARM, AARCH64, PPC, MIPS, RISC-V, etc) by means [C11 atomics](https://en.cppreference.com/w/c/atomic). - - Speed up page number lists and dirty page lists (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/132). + - Speed up page number lists and dirty page lists (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/132). - Added `LIBMDBX_NO_EXPORTS_LEGACY_API` build option. Fixes: - - Fixed missing cleanup (null assigned) in the C++ commit/abort (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/pull/143). + - Fixed missing cleanup (null assigned) in the C++ commit/abort (https://web.archive.org/web/https://github.com/erthink/libmdbx/pull/143). - Fixed `mdbx_realloc()` for case of nullptr and `MDBX_WITHOUT_MSVC_CRT=ON` for Windows. - - Fixed the possibility to use invalid and renewed (closed & re-opened, dropped & re-created) DBI-handles (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/146). - - Fixed 4-byte aligned access to 64-bit integers, including access to the `bootid` meta-page's field (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/153). + - Fixed the possibility to use invalid and renewed (closed & re-opened, dropped & re-created) DBI-handles (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/146). + - Fixed 4-byte aligned access to 64-bit integers, including access to the `bootid` meta-page's field (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/153). - Fixed minor/potential memory leak during page flushing and unspilling. - Fixed handling states of cursors's and subDBs's for nested transactions. - Fixed page leak in extra rare case the list of retired pages changed during update GC on transaction commit. - - Fixed assertions to avoid false-positive UB detection by CLANG/LLVM (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/153). - - Fixed `MDBX_TXN_FULL` and regressive `MDBX_KEYEXIST` during large transaction commit with `MDBX_LIFORECLAIM` (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/123). + - Fixed assertions to avoid false-positive UB detection by CLANG/LLVM (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/153). + - Fixed `MDBX_TXN_FULL` and regressive `MDBX_KEYEXIST` during large transaction commit with `MDBX_LIFORECLAIM` (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/123). - Fixed auto-recovery (`weak->steady` with the same boot-id) when Database size at last weak checkpoint is large than at last steady checkpoint. - - Fixed operation on systems with unusual small/large page size, including PowerPC (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/157). + - Fixed operation on systems with unusual small/large page size, including PowerPC (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/157). ## v0.9.2 at 2020-11-27 @@ -897,11 +897,11 @@ Fixes: - Fixed copy&paste typos. - Fixed minor false-positive GCC warning. - Added workaround for broken `DEFINE_ENUM_FLAG_OPERATORS` from Windows SDK. - - Fixed cursor state after multimap/dupsort repeated deletes (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/121). + - Fixed cursor state after multimap/dupsort repeated deletes (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/121). - Added `SIGPIPE` suppression for internal thread during `mdbx_env_copy()`. - - Fixed extra-rare `MDBX_KEY_EXIST` error during `mdbx_commit()` (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/131). - - Fixed spilled pages checking (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/126). - - Fixed `mdbx_load` for 'plain text' and without `-s name` cases (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/136). + - Fixed extra-rare `MDBX_KEY_EXIST` error during `mdbx_commit()` (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/131). + - Fixed spilled pages checking (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/126). + - Fixed `mdbx_load` for 'plain text' and without `-s name` cases (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/136). - Fixed save/restore/commit of cursors for nested transactions. - Fixed cursors state in rare/special cases (move next beyond end-of-data, after deletion and so on). - Added workaround for MSVC 19.28 (Visual Studio 16.8) (but may still hang during compilation). diff --git a/README.md b/README.md index 3f78a26b..d25189b8 100644 --- a/README.md +++ b/README.md @@ -353,7 +353,7 @@ named mutexes are used. Historically, _libmdbx_ is a deeply revised and extended descendant of the [Lightning Memory-Mapped Database](https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database). At first the development was carried out within the -[ReOpenLDAP](https://web.archive.org/web/20220414235959/https://github.com/erthink/ReOpenLDAP) project. About a +[ReOpenLDAP](https://web.archive.org/web/https://github.com/erthink/ReOpenLDAP) project. About a year later _libmdbx_ was separated into a standalone project, which was [presented at Highload++ 2015 conference](http://www.highload.ru/2015/abstracts/1831.html). @@ -435,7 +435,7 @@ unexpected or broken down. ### Testing The amalgamated source code does not contain any tests for or several reasons. -Please read [the explanation](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/214#issuecomment-870717981) and don't ask to alter this. +Please read [the explanation](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/214#issuecomment-870717981) and don't ask to alter this. So for testing _libmdbx_ itself you need a full source code, i.e. the clone of a git repository, there is no option. The full source code of _libmdbx_ has a [`test` subdirectory](https://gitflic.ru/project/erthink/libmdbx/tree/master/test) with minimalistic test "framework". @@ -618,7 +618,7 @@ Bindings | Rust | [libmdbx-rs](https://github.com/vorot93/libmdbx-rs) | [Artem Vorotnikov](https://github.com/vorot93) | | Rust | [mdbx](https://crates.io/crates/mdbx) | [gcxfd](https://github.com/gcxfd) | | Java | [mdbxjni](https://github.com/castortech/mdbxjni) | [Castor Technologies](https://castortech.com/) | -| Python (draft) | [python-bindings](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/commits/python-bindings) branch | [Noel Kuntze](https://github.com/Thermi) +| Python (draft) | [python-bindings](https://web.archive.org/web/https://github.com/erthink/libmdbx/commits/python-bindings) branch | [Noel Kuntze](https://github.com/Thermi) | .NET (obsolete) | [mdbx.NET](https://github.com/wangjia184/mdbx.NET) | [Jerry Wang](https://github.com/wangjia184) | diff --git a/TODO.md b/TODO.md index 66b0fff9..70016f38 100644 --- a/TODO.md +++ b/TODO.md @@ -11,19 +11,19 @@ For the same reason ~~Github~~ is blacklisted forever. So currently most of the links are broken due to noted malicious ~~Github~~ sabotage. - - [Move most of `mdbx_chk` functional to the library API](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/204). - - [Replace SRW-lock on Windows to allow shrink DB with `MDBX_NOTLS` option](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/210). - - [More flexible support of asynchronous runtime/framework(s)](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/200). - - [Migration guide from LMDB to MDBX](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/199). - - [Support for RAW devices](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/124). - - [Support MessagePack for Keys & Values](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/115). - - [Engage new terminology](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/137). + - [Move most of `mdbx_chk` functional to the library API](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/204). + - [Replace SRW-lock on Windows to allow shrink DB with `MDBX_NOTLS` option](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/210). + - [More flexible support of asynchronous runtime/framework(s)](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/200). + - [Migration guide from LMDB to MDBX](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/199). + - [Support for RAW devices](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/124). + - [Support MessagePack for Keys & Values](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/115). + - [Engage new terminology](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/137). - Packages for [Astra Linux](https://astralinux.ru/), [ALT Linux](https://www.altlinux.org/), [ROSA Linux](https://www.rosalinux.ru/), etc. Done ---- - - [Simple careful mode for working with corrupted DB](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/223). - - [Engage an "overlapped I/O" on Windows](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/224). - - [Large/Overflow pages accounting for dirty-room](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/192). - - [Get rid of dirty-pages list in MDBX_WRITEMAP mode](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/193). + - [Simple careful mode for working with corrupted DB](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/223). + - [Engage an "overlapped I/O" on Windows](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/224). + - [Large/Overflow pages accounting for dirty-room](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/192). + - [Get rid of dirty-pages list in MDBX_WRITEMAP mode](https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/193). diff --git a/mdbx.h++ b/mdbx.h++ index dddc1880..bc3e1802 100644 --- a/mdbx.h++ +++ b/mdbx.h++ @@ -287,7 +287,7 @@ namespace mdbx { // To enable all kinds of an compiler optimizations we use a byte-like type // that don't presumes aliases for pointers as does the `char` type and its // derivatives/typedefs. -// Please see todo4recovery://erased_by_github/libmdbx/issues/263 +// Please see https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/263 // for reasoning of the use of `char8_t` type and switching to `__restrict__`. using byte = char8_t; #else diff --git a/src/core.c b/src/core.c index 494da680..00f1b46d 100644 --- a/src/core.c +++ b/src/core.c @@ -4443,7 +4443,7 @@ static void iov_callback4dirtypages(iov_ctx_t *ctx, size_t offset, void *data, osal_flush_incoherent_mmap(env->me_map + offset, bytes, env->me_os_psize); const MDBX_page *const rp = (const MDBX_page *)(env->me_map + offset); /* check with timeout as the workaround - * for todo4recovery://erased_by_github/libmdbx/issues/269 */ + * for https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/269 */ if (unlikely(memcmp(wp, rp, bytes))) { ctx->coherency_timestamp = 0; WARNING("catch delayed/non-arrived page %" PRIaPGNO " %s", wp->mp_pgno, @@ -6598,7 +6598,7 @@ static __inline bool is_gc_usable(const MDBX_txn *txn) { return false; /* avoid (recursive) search inside empty tree and while tree is - updating, todo4recovery://erased_by_github/libmdbx/issues/31 */ + updating, https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/31 */ if (txn->mt_dbs[FREE_DBI].md_entries == 0) return false; @@ -6797,7 +6797,7 @@ static pgr_t page_alloc_slowpath(const MDBX_cursor *mc, const size_t num, /* Stop reclaiming to avoid large/overflow the page list. * This is a rare case while search for a continuously multi-page region * in a large database. - * todo4recovery://erased_by_github/libmdbx/issues/123 */ + * https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/123 */ NOTICE("stop reclaiming to avoid PNL overflow: %zu (current) + %zu " "(chunk) -> %zu", MDBX_PNL_GETSIZE(txn->tw.relist), gc_len, @@ -7972,7 +7972,7 @@ __cold int mdbx_thread_unregister(const MDBX_env *env) { return MDBX_SUCCESS; } -/* check against todo4recovery://erased_by_github/libmdbx/issues/269 */ +/* check against https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/269 */ static bool coherency_check(const MDBX_env *env, const txnid_t txnid, const volatile MDBX_db *dbs, const volatile MDBX_meta *meta, bool report) { @@ -8073,7 +8073,7 @@ __cold static int coherency_timeout(uint64_t *timestamp, pgno_t pgno) { } /* check with timeout as the workaround - * for todo4recovery://erased_by_github/libmdbx/issues/269 */ + * for https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/269 */ __hot static int coherency_check_readed(const MDBX_env *env, const txnid_t txnid, const volatile MDBX_db *dbs, @@ -8310,7 +8310,7 @@ static int txn_renew(MDBX_txn *txn, const unsigned flags) { const meta_ptr_t head = meta_recent(env, &txn->tw.troika); uint64_t timestamp = 0; while ( - "workaround for todo4recovery://erased_by_github/libmdbx/issues/269") { + "workaround for https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/269") { rc = coherency_check_readed(env, head.txnid, head.ptr_v->mm_dbs, head.ptr_v, ×tamp); if (likely(rc == MDBX_SUCCESS)) @@ -11953,7 +11953,7 @@ static int sync_locked(MDBX_env *env, unsigned flags, MDBX_meta *const pending, } uint64_t timestamp = 0; - while ("workaround for todo4recovery://erased_by_github/libmdbx/issues/269") { + while ("workaround for https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/269") { rc = coherency_check_written(env, pending->unsafe_txnid, target, ×tamp); if (likely(rc == MDBX_SUCCESS)) @@ -12475,7 +12475,7 @@ mdbx_env_set_geometry(MDBX_env *env, intptr_t size_lower, intptr_t size_now, uint64_t timestamp = 0; while ("workaround for " - "todo4recovery://erased_by_github/libmdbx/issues/269") { + "https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/269") { meta = *head.ptr_c; rc = coherency_check_readed(env, head.txnid, meta.mm_dbs, &meta, ×tamp); @@ -13818,7 +13818,7 @@ __cold int mdbx_env_openW(MDBX_env *env, const wchar_t *pathname, } else { #if MDBX_MMAP_INCOHERENT_FILE_WRITE /* Temporary `workaround` for OpenBSD kernel's flaw. - * See todo4recovery://erased_by_github/libmdbx/issues/67 */ + * See https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/67 */ if ((flags & MDBX_WRITEMAP) == 0) { if (flags & MDBX_ACCEDE) flags |= MDBX_WRITEMAP; @@ -21169,7 +21169,7 @@ __cold static int fetch_envinfo_ex(const MDBX_env *env, const MDBX_txn *txn, const size_t size_before_pgop_stat = offsetof(MDBX_envinfo, mi_pgop_stat); /* is the environment open? - * (todo4recovery://erased_by_github/libmdbx/issues/171) */ + * (https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/171) */ if (unlikely(!env->me_map)) { /* environment not yet opened */ #if 1 diff --git a/src/mdbx_chk.c b/src/mdbx_chk.c index 311695c8..57ea1631 100644 --- a/src/mdbx_chk.c +++ b/src/mdbx_chk.c @@ -1171,7 +1171,7 @@ int main(int argc, char *argv[]) { envflags &= ~MDBX_RDONLY; #if MDBX_MMAP_INCOHERENT_FILE_WRITE /* Temporary `workaround` for OpenBSD kernel's flaw. - * See todo4recovery://erased_by_github/libmdbx/issues/67 */ + * See https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/67 */ envflags |= MDBX_WRITEMAP; #endif /* MDBX_MMAP_INCOHERENT_FILE_WRITE */ break; diff --git a/src/osal.c b/src/osal.c index 494290a2..71046f6c 100644 --- a/src/osal.c +++ b/src/osal.c @@ -1311,7 +1311,7 @@ MDBX_INTERNAL_FUNC int osal_openfile(const enum osal_openfile_purpose purpose, flags |= O_CLOEXEC; #endif /* O_CLOEXEC */ - /* Safeguard for todo4recovery://erased_by_github/libmdbx/issues/144 */ + /* Safeguard for https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/144 */ #if STDIN_FILENO == 0 && STDOUT_FILENO == 1 && STDERR_FILENO == 2 int stub_fd0 = -1, stub_fd1 = -1, stub_fd2 = -1; static const char dev_null[] = "/dev/null"; @@ -1349,7 +1349,7 @@ MDBX_INTERNAL_FUNC int osal_openfile(const enum osal_openfile_purpose purpose, errno = EACCES /* restore errno if file exists */; } - /* Safeguard for todo4recovery://erased_by_github/libmdbx/issues/144 */ + /* Safeguard for https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/144 */ #if STDIN_FILENO == 0 && STDOUT_FILENO == 1 && STDERR_FILENO == 2 if (*fd == STDIN_FILENO) { WARNING("Got STD%s_FILENO/%d, avoid using it by dup(fd)", "IN", @@ -2192,7 +2192,7 @@ MDBX_INTERNAL_FUNC int osal_munmap(osal_mmap_t *map) { VALGRIND_MAKE_MEM_NOACCESS(map->address, map->current); /* Unpoisoning is required for ASAN to avoid false-positive diagnostic * when this memory will re-used by malloc or another mmapping. - * See todo4recovery://erased_by_github/libmdbx/pull/93#issuecomment-613687203 + * See https://web.archive.org/web/https://github.com/erthink/libmdbx/pull/93#issuecomment-613687203 */ MDBX_ASAN_UNPOISON_MEMORY_REGION(map->address, (map->filesize && map->filesize < map->limit) @@ -2271,7 +2271,7 @@ MDBX_INTERNAL_FUNC int osal_mresize(const int flags, osal_mmap_t *map, /* Unpoisoning is required for ASAN to avoid false-positive diagnostic * when this memory will re-used by malloc or another mmapping. - * See todo4recovery://erased_by_github/libmdbx/pull/93#issuecomment-613687203 + * See https://web.archive.org/web/https://github.com/erthink/libmdbx/pull/93#issuecomment-613687203 */ MDBX_ASAN_UNPOISON_MEMORY_REGION(map->address, map->limit); status = NtUnmapViewOfSection(GetCurrentProcess(), map->address); @@ -2552,7 +2552,7 @@ retry_mapview:; /* Unpoisoning is required for ASAN to avoid false-positive diagnostic * when this memory will re-used by malloc or another mmapping. * See - * todo4recovery://erased_by_github/libmdbx/pull/93#issuecomment-613687203 + * https://web.archive.org/web/https://github.com/erthink/libmdbx/pull/93#issuecomment-613687203 */ MDBX_ASAN_UNPOISON_MEMORY_REGION( map->address, @@ -2574,7 +2574,7 @@ retry_mapview:; /* Unpoisoning is required for ASAN to avoid false-positive diagnostic * when this memory will re-used by malloc or another mmapping. * See - * todo4recovery://erased_by_github/libmdbx/pull/93#issuecomment-613687203 + * https://web.archive.org/web/https://github.com/erthink/libmdbx/pull/93#issuecomment-613687203 */ MDBX_ASAN_UNPOISON_MEMORY_REGION( map->address, (map->current < map->limit) ? map->current : map->limit); diff --git a/test/valgrind_suppress.txt b/test/valgrind_suppress.txt index 2a95ff0f..2e67a56b 100644 --- a/test/valgrind_suppress.txt +++ b/test/valgrind_suppress.txt @@ -31,7 +31,7 @@ fun:wipe_steady* } -# memcmp() inside iov_write() as workaround for todo4recovery://erased_by_github/libmdbx/issues/269 +# memcmp() inside iov_write() as workaround for https://web.archive.org/web/https://github.com/erthink/libmdbx/issues/269 { iov-pagecheck-1 Memcheck:Cond