1830 Commits

Author SHA1 Message Date
Леонид Юрьев (Leonid Yuriev)
e3a09db3da mdbx: always coalescing GC records, regardless to MDBX_COALESCE flag. 2022-06-27 12:37:29 +03:00
Леонид Юрьев (Leonid Yuriev)
ece2fe2514 mdbx: split page_alloc() and page_new() to fast- and slow/rare- parts. 2022-06-27 12:37:29 +03:00
Леонид Юрьев (Leonid Yuriev)
f1ccc717b4 mdbx: add update-gc context (extracted from bigfoot). 2022-06-27 11:27:05 +03:00
Леонид Юрьев (Leonid Yuriev)
f5280ebf6e mdbx: bump version to 0.12.0 (not a release but preparation for changing feature set and API). 2022-06-25 15:14:46 +03:00
Леонид Юрьев (Leonid Yuriev)
bd80e01eda
mdbx: release v0.11.8
The stable release with an important fixes and workaround for the critical macOS thread-local-storage issue.

Acknowledgements:
-----------------

 - [Masatoshi Fukunaga](https://github.com/mah0x211) for [Lua bindings](https://github.com/mah0x211/lua-libmdbx).

New:
----

 - Added most of transactions flags to the public API.
 - Added `MDBX_NOSUCCESS_EMPTY_COMMIT` build option to return non-success result (`MDBX_RESULT_TRUE`) on empty commit.
 - Reworked validation and import of DBI-handles into a transaction.
   Assumes  these changes will be invisible to most users, but will cause fewer surprises in complex DBI cases.
 - Added ability to open DB in without-LCK (exclusive read-only) mode in case no permissions to create/write LCK-file.

Fixes:
------

 - A series of fixes and improvements for automatically generated documentation (Doxygen).
 - Fixed copy&paste bug with could lead to `SIGSEGV` (nullptr dereference) in the exclusive/no-lck mode.
 - Fixed minor warnings from modern Apple's CLANG 13.
 - Fixed minor warnings from CLANG 14 and in-development CLANG 15.
 - Fixed `SIGSEGV` regression in without-LCK (exclusive read-only) mode.
 - Fixed `mdbx_check_fs_local()` for CDROM case on Windows.
 - Fixed nasty typo of typename which caused false `MDBX_CORRUPTED` error in a rare execution path,
   when the size of the thread-ID type not equal to 8.
 - Fixed write-after-free memory corruption on latest `macOS` during finalization/cleanup of thread(s) that executed read transaction(s).
   > The issue was suddenly discovered by a [CI](https://en.wikipedia.org/wiki/Continuous_integration)
   > after adding an iteration with macOS 11 "Big Sur", and then reproduced on recent release of macOS 12 "Monterey".
   > The issue was never noticed nor reported on macOS 10 "Catalina" nor others.
   > Analysis shown that the problem caused by a change in the behavior of the system library (internals of dyld and pthread)
   > during thread finalization/cleanup: now a memory allocated for a `__thread` variable(s) is released
   > before execution of the registered Thread-Local-Storage destructor(s),
   > thus a TLS-destructor will write-after-free just by legitime dereference any `__thread` variable.
   > This is unexpected crazy-like behavior since the order of resources releasing/destroying
   > is not the reverse of ones acquiring/construction order. Nonetheless such surprise
   > is now workarounded by using atomic compare-and-swap operations on a 64-bit signatures/cookies.
 - Fixed Elbrus/E2K LCC 1.26 compiler warnings (memory model for atomic operations, etc).

Minors:
-------

 - Refined `release-assets` GNU Make target.
 - Added logging to `mdbx_fetch_sdb()` to help debugging complex DBI-handels use cases.
 - Added explicit error message from probe of no-support for `std::filesystem`.
 - Added contributors "score" table by `git fame` to generated docs.
 - Added `mdbx_assert_fail()` to public API (mostly for backtracing).
 - Now C++20 concepts used/enabled only when `__cpp_lib_concepts >= 202002`.
 - Don't provide nor report package information if used as a CMake subproject.

Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2022-06-12 23:47:18 +03:00
Леонид Юрьев (Leonid Yuriev)
3817236b68 mdbx: avoid memory-model from variables for C11 atomics (workaround for lcc 1.26). 2022-06-12 20:04:35 +03:00
Леонид Юрьев (Leonid Yuriev)
c082eb7d8a mdbx: use _tlv_atexit() on Darwin/MacOSX. 2022-06-11 21:16:29 +03:00
Леонид Юрьев (Leonid Yuriev)
abac366eac mdbx: rework/fix rthc-cleanup using pthread' tsd to avoid write-after-free (critical). 2022-06-11 21:16:29 +03:00
Леонид Юрьев (Leonid Yuriev)
63f8eb253d mdbx: do not enable ASAN.detect_leaks in macOS since unsupported. 2022-06-10 17:26:10 +03:00
Леонид Юрьев (Leonid Yuriev)
117be44c44 mdbx: fix nasty typename typo.
Спасибо Станиславу Очеретному за сообщение о проблеме.
2022-06-08 20:57:42 +03:00
Леонид Юрьев (Leonid Yuriev)
cdb16c9f00 mdbx: minor fix/clarify internal comment. 2022-06-07 18:13:49 +03:00
Леонид Юрьев (Leonid Yuriev)
77f56541d0 mdbx-windows: refine/fix mdbx_check_fs_local() for CDROM case. 2022-06-04 02:08:24 +03:00
Леонид Юрьев (Leonid Yuriev)
73fbf5c8fa mdbx: fix minor typo in comment. 2022-06-04 02:07:19 +03:00
Леонид Юрьев (Leonid Yuriev)
d1e67645a2 mdbx: refine handling EACCESS while trying create LCK-file on a read-only filesystem. 2022-06-04 02:04:55 +03:00
Леонид Юрьев (Leonid Yuriev)
a4da10bc62 mdbx: minor fix LCK-initialization. 2022-06-03 22:40:18 +03:00
Леонид Юрьев (Leonid Yuriev)
e795fe7c3e mdbx: fix without-LCK (exclusive + readonly) mode. 2022-06-03 22:39:11 +03:00
Леонид Юрьев (Leonid Yuriev)
b6138c39f0 mdbx: reorganize internal includes. 2022-06-02 20:35:40 +03:00
Леонид Юрьев (Leonid Yuriev)
535ad41ae6 mdbx: made internal noop macro compatible with MSVC. 2022-06-02 20:35:40 +03:00
Леонид Юрьев (Leonid Yuriev)
106f39327e mdbx: add mdbx_assert_fail() to public API. 2022-06-02 20:35:40 +03:00
Леонид Юрьев (Leonid Yuriev)
501691a3c0 mdbx: add __assert_fail() macro for Android. 2022-06-02 20:35:40 +03:00
Леонид Юрьев (Leonid Yuriev)
d9407ee648 mdbx: minor refine page_alloc() internal logging. 2022-05-31 17:48:23 +03:00
Леонид Юрьев (Leonid Yuriev)
c7bde8be8d mdbx: rework internal DBI-handles serials, validation and import into a txn (squashed). 2022-05-31 17:48:10 +03:00
Леонид Юрьев (Leonid Yuriev)
70d9efdce4 mdbx: fix minor CLANG 14/15 warnings. 2022-05-17 13:36:38 +03:00
Леонид Юрьев (Leonid Yuriev)
21e1dc3248 mdbx: fix minor warnings from modern Apple's CLANG. 2022-05-15 23:42:57 +03:00
Леонид Юрьев (Leonid Yuriev)
5cd4190f2d mdbx: fix copy&paste bug.
Resolves https://gitflic.ru/project/erthink/libmdbx/issue/5
2022-05-12 14:21:19 +03:00
Леонид Юрьев (Leonid Yuriev)
405de05ca9 mdbx: return MDBX_RESULT_TRUE on empty transaction commit. 2022-05-12 13:11:59 +03:00
Леонид Юрьев (Leonid Yuriev)
548d0a50b8 mdbx: minor refine logging inside page_alloc(). 2022-05-04 19:07:33 +03:00
Леонид Юрьев (Leonid Yuriev)
f57ca4fbf6 mdbx: minor refine logging inside fetch_sdb(). 2022-05-04 18:00:54 +03:00
Леонид Юрьев (Leonid Yuriev)
9230201ca9 mdbx: fix minor warning for MDBX_TXN_INVALID. 2022-05-03 14:16:19 +03:00
Леонид Юрьев (Leonid Yuriev)
6e4094c714 mdbx-docs: refine Build options for Doxygen. 2022-05-03 12:05:16 +03:00
Леонид Юрьев (Leonid Yuriev)
838f8d8fab mdbx: move most of transactions flags to public API. 2022-05-03 11:49:28 +03:00
Леонид Юрьев (Leonid Yuriev)
447d6bfca5 mdbx: add logging to mdbx_fetch_sdb() to help users debugging complex DBI use cases. 2022-05-01 12:02:40 +03:00
Леонид Юрьев (Leonid Yuriev)
40ec559c8c
mdbx: release v0.11.7
The stable risen release after the Github's intentional malicious disaster.

We have migrated to a reliable trusted infrastructure
-----------------------------------------------------

The origin for now is at [GitFlic](https://gitflic.ru/project/erthink/libmdbx)
since on 2022-04-15 the Github administration, without any warning nor
explanation, deleted _libmdbx_ along with a lot of other projects,
simultaneously blocking access for many developers.
For the same reason ~~Github~~ is blacklisted forever.

GitFlic already support Russian and English languages, plan to support more,
including 和 中文. You are welcome!

New:
----

 - Added the `tools-static` make target to build statically linked MDBX tools.
 - Support for Microsoft Visual Studio 2022.
 - Support build by MinGW' make from command line without CMake.
 - Added `mdbx::filesystem` C++ API namespace that corresponds to `std::filesystem` or `std::experimental::filesystem`.
 - Created [website](https://libmdbx.website.yandexcloud.net/) for online auto-generated documentation.
 - Used `todo4recovery://erased_by_github/` for dead (or temporarily lost) resources deleted by ~~Github~~.
 - Added `--loglevel=` command-line option to the `mdbx_test` tool.
 - Added few fast smoke-like tests into CMake builds.

Fixes:
------

 - Fixed a race between starting a transaction and creating a DBI descriptor that could lead to `SIGSEGV` in the cursor tracking code.
 - Clarified description of `MDBX_EPERM` error returned from `mdbx_env_set_geometry()`.
 - Fixed non-promoting the parent transaction to be dirty in case the undo of the geometry update failed during abortion of a nested transaction.
 - Resolved linking issues with `libstdc++fs`/`libc++fs`/`libc++experimental` for C++ `std::filesystem` or `std::experimental::filesystem` for legacy compilers.
 - Added workaround for GNU Make 3.81 and earlier.
 - Added workaround for Elbrus/LCC 1.25 compiler bug of class inline `static constexpr` member field.
 - [Fixed](https://github.com/ledgerwatch/erigon/issues/3874) minor assertion regression (only debug builds were affected).
 - Fixed detection of `C++20` concepts accessibility.
 - Fixed detection of Clang's LTO availability for Android.
 - Fixed build for ARM/ARM64 by MSVC.
 - Fixed non-x86 Windows builds with `MDBX_WITHOUT_MSVC_CRT=ON` and `MDBX_BUILD_SHARED_LIBRARY=ON`.

Minors:
-------

 - Resolve minor MSVC warnings: avoid `/INCREMENTAL[:YES]` with `/LTCG`, `/W4` with `/W3`, the `C5105` warning.
 - Switched to using `MDBX_EPERM` instead of `MDBX_RESULT_TRUE' to indicate that the geometry cannot be updated.
 - Added `NULL` checking during memory allocation inside `mdbx_chk`.
 - Resolved all warnings from MinGW while used without CMake.
 - Added inheretable `target_include_directories()` to `CMakeLists.txt` for easy integration.
 - Added build-time checks and paranoid runtime assertions for the `off_t` arguments of `fcntl()` which are used for locking.
 - Added `-Wno-lto-type-mismatch` to avoid false-positive warnings from old GCC during LTO-enabled builds.
 - Added checking for TID (system thread id) to avoid hang on 32-bit Bionic/Android  within `pthread_mutex_lock()`.
 - Reworked `MDBX_BUILD_TARGET` of CMake builds.
 - Added `CMAKE_HOST_ARCH` and `CMAKE_HOST_CAN_RUN_EXECUTABLES_BUILT_FOR_TARGET`.

Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2022-04-22 23:28:56 +03:00
Леонид Юрьев (Leonid Yuriev)
f341129afa mdbx-windows: fix for non-x86 with MDBX_WITHOUT_MSVC_CRT=ON and MDBX_BUILD_SHARED_LIBRARY=ON. 2022-04-22 18:31:49 +03:00
Леонид Юрьев (Leonid Yuriev)
1ec5687d36 mdbx: fix build for ARM/ARM64 by MSVC. 2022-04-22 17:18:58 +03:00
Леонид Юрьев (Leonid Yuriev)
bb377fd20e mdbx: merge branch master into devel. 2022-04-21 15:44:23 +03:00
Леонид Юрьев (Leonid Yuriev)
a2aa6667e1 mdbx: avoid 32-bit Bionic/Android hang within pthread_mutex_lock(). 2022-04-21 15:41:25 +03:00
Леонид Юрьев (Leonid Yuriev)
6d15836171 mdbx: use todo4recovery://erased_by_github/ for dead (or temporarily lost) resources. 2022-04-21 15:34:53 +03:00
Леонид Юрьев (Leonid Yuriev)
ed23956e11 mdbx: basic cleanup from dead github and another corrupted services. 2022-04-20 06:42:16 +03:00
Леонид Юрьев (Leonid Yuriev)
eb8bc865d1 mdbx: fix and check width of off_t for Android. 2022-04-19 20:00:29 +03:00
Леонид Юрьев (Leonid Yuriev)
4628ac6863 mdbx: add paranoid assertions for off_t args of fcntl(). 2022-04-19 19:43:42 +03:00
Леонид Юрьев (Leonid Yuriev)
9fa4e21165 mdbx: fix assertion regression.
Fixes https://github.com/ledgerwatch/erigon/issues/3874.

This was a minor regression after the c4a5325aafd3f03ce7520731b9da7253d7d178f0
that affects only debug builgs (with enabled assertions) and only when the added
code catch a incoherency of unified page/buffer cache.
2022-04-11 21:52:46 +03:00
Леонид Юрьев (Leonid Yuriev)
7b95720f59 mdbx++: add support for legacy experimental/filesystem. 2022-04-05 22:20:39 +03:00
Леонид Юрьев (Leonid Yuriev)
6f37c8e57f mdbx: resolve all warnings from MinGW. 2022-03-30 18:13:08 +03:00
Leonid Yuriev
21da42d23d mdbx: fix built-in status of MDBX_WITHOUT_MSVC_CRT option. 2022-03-30 18:10:03 +03:00
Леонид Юрьев (Leonid Yuriev)
2497437060 mdbx: mark the parent transaction dirty in case the undo of the geometry update failed during abortion of a nested transaction. 2022-03-29 18:57:35 +03:00
Леонид Юрьев (Leonid Yuriev)
33a9395afe mdbx: use MDBX_EPERM to indicate that the geometry cannot be updated instead of `MDBX_RESULT_TRUE'. 2022-03-29 18:57:26 +03:00
Leonid Yuriev
27a513682a mdbx: disable C5105 warning for Visual Studio > 15.7 (MSVC > 19.14). 2022-03-28 23:57:24 +03:00
Леонид Юрьев (Leonid Yuriev)
eb532b8907 mdbx-tools: add check for log-level. 2022-03-28 23:57:24 +03:00
Леонид Юрьев (Leonid Yuriev)
6f06641bf2 mdbx-tools: handle NULL from memory allocation functions. 2022-03-28 23:44:31 +03:00