Compare commits

...

106 Commits

Author SHA1 Message Date
Леонид Юрьев (Leonid Yuriev)
40ec559c8c mdbx: release v0.11.7
The stable risen release after the Github's intentional malicious disaster.

We have migrated to a reliable trusted infrastructure
-----------------------------------------------------

The origin for now is at [GitFlic](https://gitflic.ru/project/erthink/libmdbx)
since on 2022-04-15 the Github administration, without any warning nor
explanation, deleted _libmdbx_ along with a lot of other projects,
simultaneously blocking access for many developers.
For the same reason ~~Github~~ is blacklisted forever.

GitFlic already support Russian and English languages, plan to support more,
including 和 中文. You are welcome!

New:
----

 - Added the `tools-static` make target to build statically linked MDBX tools.
 - Support for Microsoft Visual Studio 2022.
 - Support build by MinGW' make from command line without CMake.
 - Added `mdbx::filesystem` C++ API namespace that corresponds to `std::filesystem` or `std::experimental::filesystem`.
 - Created [website](https://libmdbx.website.yandexcloud.net/) for online auto-generated documentation.
 - Used `todo4recovery://erased_by_github/` for dead (or temporarily lost) resources deleted by ~~Github~~.
 - Added `--loglevel=` command-line option to the `mdbx_test` tool.
 - Added few fast smoke-like tests into CMake builds.

Fixes:
------

 - Fixed a race between starting a transaction and creating a DBI descriptor that could lead to `SIGSEGV` in the cursor tracking code.
 - Clarified description of `MDBX_EPERM` error returned from `mdbx_env_set_geometry()`.
 - Fixed non-promoting the parent transaction to be dirty in case the undo of the geometry update failed during abortion of a nested transaction.
 - Resolved linking issues with `libstdc++fs`/`libc++fs`/`libc++experimental` for C++ `std::filesystem` or `std::experimental::filesystem` for legacy compilers.
 - Added workaround for GNU Make 3.81 and earlier.
 - Added workaround for Elbrus/LCC 1.25 compiler bug of class inline `static constexpr` member field.
 - [Fixed](https://github.com/ledgerwatch/erigon/issues/3874) minor assertion regression (only debug builds were affected).
 - Fixed detection of `C++20` concepts accessibility.
 - Fixed detection of Clang's LTO availability for Android.
 - Fixed build for ARM/ARM64 by MSVC.
 - Fixed non-x86 Windows builds with `MDBX_WITHOUT_MSVC_CRT=ON` and `MDBX_BUILD_SHARED_LIBRARY=ON`.

Minors:
-------

 - Resolve minor MSVC warnings: avoid `/INCREMENTAL[:YES]` with `/LTCG`, `/W4` with `/W3`, the `C5105` warning.
 - Switched to using `MDBX_EPERM` instead of `MDBX_RESULT_TRUE' to indicate that the geometry cannot be updated.
 - Added `NULL` checking during memory allocation inside `mdbx_chk`.
 - Resolved all warnings from MinGW while used without CMake.
 - Added inheretable `target_include_directories()` to `CMakeLists.txt` for easy integration.
 - Added build-time checks and paranoid runtime assertions for the `off_t` arguments of `fcntl()` which are used for locking.
 - Added `-Wno-lto-type-mismatch` to avoid false-positive warnings from old GCC during LTO-enabled builds.
 - Added checking for TID (system thread id) to avoid hang on 32-bit Bionic/Android  within `pthread_mutex_lock()`.
 - Reworked `MDBX_BUILD_TARGET` of CMake builds.
 - Added `CMAKE_HOST_ARCH` and `CMAKE_HOST_CAN_RUN_EXECUTABLES_BUILT_FOR_TARGET`.

Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2022-04-22 23:28:56 +03:00
Леонид Юрьев (Leonid Yuriev)
42f1abd7e9 mdbx-cmake: fix/refine --allow-multiple-definition as workaround for std::filesystem and LCC < 1.25.23 2022-04-22 23:03:58 +03:00
Леонид Юрьев (Leonid Yuriev)
b327cafe1a mdbx-cmake: seeding cmake tests. 2022-04-22 22:07:42 +03:00
Леонид Юрьев (Leonid Yuriev)
fb4bd6158f mdbx-cmake: fix extra message space (cosmetics). 2022-04-22 21:35:05 +03:00
Леонид Юрьев (Leonid Yuriev)
a878b47343 mdbx-cmake: refine fetch_version macro for no-git-tags case (sync cmake modules). 2022-04-22 21:32:38 +03:00
Леонид Юрьев (Leonid Yuriev)
f866ad2fa3 mdbx: update ChangeLog. 2022-04-22 21:03:11 +03:00
Леонид Юрьев (Leonid Yuriev)
f341129afa mdbx-windows: fix for non-x86 with MDBX_WITHOUT_MSVC_CRT=ON and MDBX_BUILD_SHARED_LIBRARY=ON. 2022-04-22 18:31:49 +03:00
Леонид Юрьев (Leonid Yuriev)
03e7e3be74 mdbx-cmake: avoid tests with mdbx_chk when MDBX_BUILD_TOOLS=OFF. 2022-04-22 18:07:51 +03:00
Леонид Юрьев (Leonid Yuriev)
7770cae4b5 mdbx-cmake: provide CMAKE_HOST_ARCH and CMAKE_HOST_CAN_RUN_EXECUTABLES_BUILT_FOR_TARGET. 2022-04-22 17:45:28 +03:00
Леонид Юрьев (Leonid Yuriev)
039ebe1f14 mdbx-cmake: rework forming of MDBX_SYSTEM_ARCH identity. 2022-04-22 17:30:11 +03:00
Leonid Yuriev
5eeb260c08 mdbx-cmake: rework making MDBX_BUILD_TARGET value. 2022-04-22 17:28:56 +03:00
Леонид Юрьев (Leonid Yuriev)
1ec5687d36 mdbx: fix build for ARM/ARM64 by MSVC. 2022-04-22 17:18:58 +03:00
Леонид Юрьев (Leonid Yuriev)
e3b0602664 mdbx-cmake: set loglevel=verbose for smoke test to speedup CI. 2022-04-22 09:32:47 +03:00
Леонид Юрьев (Leonid Yuriev)
b1101fc33e mdbx-cmake: fix detection of Clang's LTO availability for Android. 2022-04-21 22:11:57 +03:00
Леонид Юрьев (Leonid Yuriev)
bdf5fb7a72 mdbx++: avoid clang++ minor/extra warnings. 2022-04-21 21:53:46 +03:00
Леонид Юрьев (Leonid Yuriev)
e00f827de7 mdbx++: fix/refine detection of C++20 concepts accessibility. 2022-04-21 21:53:29 +03:00
Леонид Юрьев (Leonid Yuriev)
abc2341cb4 cmake-test: avoid run tests in case cross-compilation without emulator. 2022-04-21 20:35:45 +03:00
Леонид Юрьев (Leonid Yuriev)
03f9ed8820 mdbx-cmake: add a few smoke-like tests. 2022-04-21 20:35:45 +03:00
Леонид Юрьев (Leonid Yuriev)
e875d2128e mdbx-test: add --loglevel= option. 2022-04-21 20:35:41 +03:00
Леонид Юрьев (Leonid Yuriev)
bb377fd20e mdbx: merge branch master into devel. 2022-04-21 15:44:23 +03:00
Леонид Юрьев (Leonid Yuriev)
a2aa6667e1 mdbx: avoid 32-bit Bionic/Android hang within pthread_mutex_lock(). 2022-04-21 15:41:25 +03:00
Леонид Юрьев (Leonid Yuriev)
51d66494fd mdbx-cmake: add -Wno-lto-type-mismatch to avoid false-positive warnings from GCC < 9.x 2022-04-21 15:40:53 +03:00
Леонид Юрьев (Leonid Yuriev)
ed9e51d31d mdbx: update ChangeLog. 2022-04-21 15:34:56 +03:00
Леонид Юрьев (Leonid Yuriev)
2921711638 mdbx: minor refine note about migration from dead github. 2022-04-21 15:34:56 +03:00
Леонид Юрьев (Leonid Yuriev)
6d15836171 mdbx: use todo4recovery://erased_by_github/ for dead (or temporarily lost) resources. 2022-04-21 15:34:53 +03:00
Леонид Юрьев (Leonid Yuriev)
3f840ecd89 mdbx-doc: reorder restrictions section. 2022-04-20 23:55:25 +03:00
Леонид Юрьев (Leonid Yuriev)
9aa53d1616 mdbx: update ChangeLog and captions. 2022-04-20 23:55:25 +03:00
Леонид Юрьев (Leonid Yuriev)
35b5abc103 mdbx: minor fix/refix Doxygen descriptions. 2022-04-20 18:04:04 +03:00
Леонид Юрьев (Leonid Yuriev)
8ece0dfa93 mdbx-docs: update Doxygen.in 2022-04-20 18:00:54 +03:00
Леонид Юрьев (Leonid Yuriev)
ed23956e11 mdbx: basic cleanup from dead github and another corrupted services. 2022-04-20 06:42:16 +03:00
Леонид Юрьев (Leonid Yuriev)
1a471ed04b mdbx: github is dead and blacklisted forever.
Основной репозиторий проекта перемещен на
https://gitflic.ru/project/erthink/libmdbx, так как 15 апреля
2022 администрация Github без предупреждения и без объяснения
причин удалила libmdbx вместе с массой других проектов,
одновременно заблокировав доступ многим разработчикам. По той
же причине Github навсегда занесен в черный список.

   На случай если это была случайность или ошибка мы ждали 5
суток (три рабочих дня), но чуда не случилось. Github умер, как
и многие декларируемые либеральные ценности (свобода слова,
презумпция невиновности и право на суд, неприкосновенность
личности и частной собственности и т.д.).

---

   The origin repository of the project has been moved to
https://gitflic.ru/project/erthink/libmdbx since on April 15,
2022, the Github administration, without warning and without
explanation, deleted libmdbx along with a lot of other
projects, simultaneously blocking access to many developers.
For the same reason Github is blacklisted forever.

   In case it was an accident or a mistake, we waited 5 days
(three working days), but no miracle happened. So Github is
died, as well as many declared liberal values (freedom of
speech, presumption of innocence and right to trial,
inviolability of the person and private property, etc).

Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2022-04-20 02:24:31 +03:00
Леонид Юрьев (Leonid Yuriev)
eb8bc865d1 mdbx: fix and check width of off_t for Android. 2022-04-19 20:00:29 +03:00
Леонид Юрьев (Leonid Yuriev)
682ff99f1c mdbx-make: add smoke-assertion, test-assertion, long-test-assertion targets. 2022-04-19 19:43:42 +03:00
Леонид Юрьев (Leonid Yuriev)
4628ac6863 mdbx: add paranoid assertions for off_t args of fcntl(). 2022-04-19 19:43:42 +03:00
Леонид Юрьев (Leonid Yuriev)
bb8f431817 mdbx-cmake: use target_include_directories() instead of include_directories(). 2022-04-13 11:05:50 +03:00
Леонид Юрьев (Leonid Yuriev)
8fb63c3675 mdbx: minor refine API descriptions. 2022-04-13 11:05:03 +03:00
Леонид Юрьев (Leonid Yuriev)
3792dd1007 mdbx: now we accept donations only in Russian Rubles. 2022-04-12 15:08:49 +03:00
Леонид Юрьев (Leonid Yuriev)
15cc7d5ed3 mdbx: update ChangeLog. 2022-04-12 10:20:50 +03:00
Леонид Юрьев (Leonid Yuriev)
9fa4e21165 mdbx: fix assertion regression.
Fixes https://github.com/ledgerwatch/erigon/issues/3874.

This was a minor regression after the c4a5325aaf
that affects only debug builgs (with enabled assertions) and only when the added
code catch a incoherency of unified page/buffer cache.
2022-04-11 21:52:46 +03:00
Леонид Юрьев (Leonid Yuriev)
3872c0ab74 mdbx: update ChangeLog. 2022-04-08 01:01:01 +03:00
Леонид Юрьев (Leonid Yuriev)
d71b293de5 mdbx: fix spelling. 2022-04-08 01:00:58 +03:00
Леонид Юрьев (Leonid Yuriev)
5ebcb90620 mdbx-cmake: rework linking detection for std::filesystem. 2022-04-07 21:57:49 +03:00
Леонид Юрьев (Leonid Yuriev)
39a4a89650 mdbx++: declare mdbx::filesystem if available. 2022-04-07 18:00:23 +03:00
Леонид Юрьев (Leonid Yuriev)
68ac48235e mdbx-ci: add if-failure step to github-actions to dump all *.err logs. 2022-04-06 12:44:30 +03:00
Леонид Юрьев (Leonid Yuriev)
eb3fc985d6 mdbx-make: refine GNUmakefile to avoid bug of GNU Make 3.81 2022-04-06 12:43:58 +03:00
Леонид Юрьев (Leonid Yuriev)
c9dfb7d8c2 mdbx-cmake: add workaround for LCC 1.25 bug of class inline static constexpr member field definition. 2022-04-06 12:43:58 +03:00
Леонид Юрьев (Leonid Yuriev)
437cd0d3d7 mdbx-cmake: minor refine version detection of Elbrus LCC compiler. 2022-04-06 12:43:58 +03:00
Леонид Юрьев (Leonid Yuriev)
f054ceeab8 mdbx-cmake: add linking with stdc++fs for LCC < 1.26 2022-04-06 12:43:58 +03:00
Леонид Юрьев (Leonid Yuriev)
7b95720f59 mdbx++: add support for legacy experimental/filesystem. 2022-04-05 22:20:39 +03:00
Леонид Юрьев (Leonid Yuriev)
64e23c9be0 mdbx-make: add probe4 prefix for error-log files. 2022-04-05 13:07:54 +03:00
Леонид Юрьев (Leonid Yuriev)
cabead30b5 mdbx: minor cosmetics. 2022-03-31 00:31:49 +03:00
Леонид Юрьев (Leonid Yuriev)
688d4495c5 mdbx: update ChangeLog. 2022-03-30 19:12:42 +03:00
Леонид Юрьев (Leonid Yuriev)
b6d36e2235 mdbx-ci: add 'build by make' step info mingw action. 2022-03-30 18:50:34 +03:00
Леонид Юрьев (Leonid Yuriev)
ebf7bf7583 mdbx-make: support for MinGW. 2022-03-30 18:13:44 +03:00
Леонид Юрьев (Leonid Yuriev)
6f37c8e57f mdbx: resolve all warnings from MinGW. 2022-03-30 18:13:08 +03:00
Leonid Yuriev
21da42d23d mdbx: fix built-in status of MDBX_WITHOUT_MSVC_CRT option. 2022-03-30 18:10:03 +03:00
Леонид Юрьев (Leonid Yuriev)
2497437060 mdbx: mark the parent transaction dirty in case the undo of the geometry update failed during abortion of a nested transaction. 2022-03-29 18:57:35 +03:00
Леонид Юрьев (Leonid Yuriev)
33a9395afe mdbx: use MDBX_EPERM to indicate that the geometry cannot be updated instead of `MDBX_RESULT_TRUE'. 2022-03-29 18:57:26 +03:00
Leonid Yuriev
7654c9d9a1 mdbx-test: handle MDBX_RESULT_TRUE from mdbx_env_set_geometry() under Windows. 2022-03-28 23:57:24 +03:00
Leonid Yuriev
27a513682a mdbx: disable C5105 warning for Visual Studio > 15.7 (MSVC > 19.14). 2022-03-28 23:57:24 +03:00
Leonid Yuriev
92cb0cc0db mdbx-cmake: forcing /INCREMENTAL:NO while using LTO/Interprocedutal optimization. 2022-03-28 23:57:24 +03:00
Leonid Yuriev
cf32f4cdb3 mdbx-cmake: removing /W3 when adding /W4 to avoid stupid MSVC warnings. 2022-03-28 23:57:24 +03:00
Leonid Yuriev
57978b0f7f mdbx-cmake: add remove_flag and remove_compile_flag macros. 2022-03-28 23:57:24 +03:00
Леонид Юрьев (Leonid Yuriev)
eb532b8907 mdbx-tools: add check for log-level. 2022-03-28 23:57:24 +03:00
Леонид Юрьев (Leonid Yuriev)
6f06641bf2 mdbx-tools: handle NULL from memory allocation functions. 2022-03-28 23:44:31 +03:00
Леонид Юрьев (Leonid Yuriev)
1ccc9b3e3b mdbx-make: add missed "--match=v[0-9]*" to git describe. 2022-03-28 09:19:39 +03:00
Леонид Юрьев (Leonid Yuriev)
ec0379ad93 mdbx-cmake: add missed "--match=v[0-9]*" to git describe. 2022-03-28 09:16:15 +03:00
Леонид Юрьев (Leonid Yuriev)
18789654fc mdbx: update ChangeLog. 2022-03-27 16:01:04 +03:00
Леонид Юрьев (Leonid Yuriev)
bac546bdfa mdbx-make: add tools-static target. 2022-03-27 14:01:18 +03:00
Леонид Юрьев (Leonid Yuriev)
a6b506be45 mdbx-test: minor refine Valgrind-suppressions for gcc-isra. 2022-03-25 13:54:34 +03:00
Леонид Юрьев (Leonid Yuriev)
6380f2e844 mdbx: minor reformat (cosmetics). 2022-03-25 13:47:37 +03:00
Леонид Юрьев (Leonid Yuriev)
fef90d2a3c mdbx: add explicit memset() for debugging. 2022-03-25 13:47:37 +03:00
Леонид Юрьев (Leonid Yuriev)
d522069080 mdbx: fix rare SIGSEGV in cursor tracking code.
The error was that the array of pointers in the transaction zeroed by the
value of env->me_numdbs and before txn->mt_numdbs was set to env->me_numdbs.
Thus, a cursor pointer(s) in the starting transaction could uninitialized if
another thread opened a dbi-handle(s) between the two aforementioned events.
2022-03-25 13:05:46 +03:00
Леонид Юрьев (Leonid Yuriev)
d5e4c198d8 mdbx: release v0.11.6
The stable release with the complete workaround for an incoherence flaw of Linux unified page/buffer cache.
Nonetheless the cause for this trouble may be an issue of Intel CPU cache/MESI.
See [issue#269](https://github.com/erthink/libmdbx/issues/269) for more information.

Acknowledgements:
-----------------

 - [David Bouyssié](https://github.com/david-bouyssie) for [Scala bindings](https://github.com/david-bouyssie/mdbx4s).
 - [Michelangelo Riccobene](https://github.com/mriccobene) for reporting and testing.

Fixes:
------

 - [Added complete workaround](https://github.com/erthink/libmdbx/issues/269) for an incoherence flaw of Linux unified page/buffer cache.
 - [Fixed](https://github.com/erthink/libmdbx/issues/272) cursor reusing for read-only transactions.
 - Fixed copy&paste typo inside `mdbx::cursor::find_multivalue()`.

Minors:
-------

 - Minor refine C++ API for convenience.
 - Minor internals refines.
 - Added `lib-static` and `lib-shared` targets for make.
 - Added minor workaround for AppleClang 13.3 bug.
 - Clarified error messages of a signature/version mismatch.

Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2022-03-24 20:44:21 +03:00
Леонид Юрьев (Leonid Yuriev)
415d0d1dfb mdbx: update ChangeLog. 2022-03-24 19:19:53 +03:00
Леонид Юрьев (Leonid Yuriev)
50d5b2345e mdbx-test: add Valgrind-suppressions for bcmp/memcmp of page-check inside mdbx_iov_write().
Related to https://github.com/erthink/libmdbx/issues/269.
2022-03-24 12:11:50 +03:00
Леонид Юрьев (Leonid Yuriev)
d13534967a mdbx++: fix copy&paste typo inside mdbx::cursor::find_multivalue(). 2022-03-24 12:11:50 +03:00
Леонид Юрьев (Leonid Yuriev)
43070c7b26 mdbx: minor fix meta_checktxnid() for Valgrind/ASAN usage.
Related to https://github.com/erthink/libmdbx/issues/269.
2022-03-24 12:11:50 +03:00
Леонид Юрьев (Leonid Yuriev)
45f8197635 mdbx: clarify/refine error messages of a signature/version mismatch. 2022-03-24 00:19:34 +03:00
Леонид Юрьев (Leonid Yuriev)
3db02d2236 mdbx++: explicitly define noexcept copy-assignment ops as workaround for AppleClang > 13.3 bug.
Try workaround for https://github.com/erthink/libmdbx/issues/278
2022-03-23 18:06:28 +03:00
Леонид Юрьев (Leonid Yuriev)
b79f6712e3 mdbx: refine override_meta() to preserve format-signature for legacy/zero mod_txnid. 2022-03-23 00:48:41 +03:00
Леонид Юрьев (Leonid Yuriev)
de63041b7d mdbx: add MDBX_DBG_DONT_UPGRADE flag. 2022-03-23 00:48:41 +03:00
Леонид Юрьев (Leonid Yuriev)
a5c064c33e mdbx: preserve (don't upgrade) format-signature during turn to a specified meta-page while recovery. 2022-03-23 00:48:41 +03:00
Your Name
9c832c24a6 mdbx: minor fix error-message space. 2022-03-23 00:48:41 +03:00
Леонид Юрьев (Leonid Yuriev)
c4a5325aaf mdbx: check-and-retry against page arrival as a workaround for unified page/buffer cache incoherency.
Part 2 of 2 of the workaround for https://github.com/erthink/libmdbx/issues/269.
2022-03-08 17:09:29 +03:00
Леонид Юрьев (Leonid Yuriev)
93cf99a07c mdbx: drop the previous temporary workaround for unified page/buffer cache incoherency.
Related to https://github.com/erthink/libmdbx/issues/269.
2022-03-06 13:22:57 +03:00
Леонид Юрьев (Leonid Yuriev)
00ed61c685 mdbx: check-and-retry a mvcc-snapshot for unified page/buffer cache coherency.
Part 1 of 2 of the workaround for https://github.com/erthink/libmdbx/issues/269.
2022-03-06 13:22:38 +03:00
Леонид Юрьев (Leonid Yuriev)
f84d9f6208 mdbx++: minor refine mdbx::cursor::erase(). 2022-03-06 00:13:11 +03:00
Леонид Юрьев (Leonid Yuriev)
9569b864ff mdbx-test: fix quoting for tr args inside long_stochastic.sh 2022-03-05 15:37:32 +03:00
Леонид Юрьев (Leonid Yuriev)
8b69be8def mdbx: update ChangeLog. 2022-03-05 15:14:20 +03:00
Леонид Юрьев (Leonid Yuriev)
4c619e32f7 mdbx: move origin to https://gitflic.ru/project/erthink/libmdbx. 2022-03-05 14:39:02 +03:00
Леонид Юрьев (Leonid Yuriev)
ed86d9c066 mdbx: add link to Scala bindings.
Resolves https://github.com/erthink/libmdbx/issues/274.
2022-03-05 14:39:02 +03:00
Леонид Юрьев (Leonid Yuriev)
6fbaa54d3e mdbx: remove PayPal from links for sponsorship. 2022-03-05 14:39:02 +03:00
Леонид Юрьев (Leonid Yuriev)
9d8fc7b984 mdbx: minor mix result caching inside the get_reasonable_db_maxsize().
Change-Id: I9763ebb323d4f66e702e07f708e269e2479880ac
2022-03-05 14:31:02 +03:00
Леонид Юрьев (Leonid Yuriev)
8c2efe3aaa mdbx++: add full mdbx::env_managed::geometry() for convenience.
Change-Id: Ib132c776bfc503336582b672de1c44fcac286936
2022-03-05 14:31:02 +03:00
Леонид Юрьев (Leonid Yuriev)
4b130bd82c mdbx-make: add lib-static and lib-shared targets.
Resolve https://github.com/erthink/libmdbx/issues/275.
2022-03-05 14:28:15 +03:00
Леонид Юрьев (Leonid Yuriev)
44fb240955 mdbx: merge branch issue-269 into the devel branch. 2022-03-04 17:46:43 +03:00
Леонид Юрьев (Leonid Yuriev)
bfea3ca9fb mdbx: minor refine mdbx_page_search() internals. 2022-03-04 17:38:25 +03:00
Леонид Юрьев (Leonid Yuriev)
7ade182d64 mdbx: minor simplify/refine mdbx_env_set_geometry() internals. 2022-03-04 17:38:25 +03:00
Леонид Юрьев (Leonid Yuriev)
4adc7aa58d mdbx: add few checks to mdbx_validate_meta(). 2022-03-04 11:53:14 +03:00
Леонид Юрьев (Leonid Yuriev)
110cf09cf8 mdbx: simplify core-dbs update inside mdbx_txn_commit_ex(). 2022-03-04 11:49:54 +03:00
Леонид Юрьев (Leonid Yuriev)
fb25648b9c mdbx: force mdbx_page_get() to inline. 2022-03-04 11:49:01 +03:00
Леонид Юрьев (Leonid Yuriev)
78170a5750 mdbx: minor reduce/lowering debug "update oldest" logging. 2022-03-04 11:41:17 +03:00
Леонид Юрьев (Leonid Yuriev)
71d07b3a8e mdbx: adding cursors tracking/lists for read-only transactions.
Briefly, this commit fixes a missed flaw:
 - Cursor tracking is required to replacing shaded pages and adjusting the positions in writing transactions;
 - Thus, historically, an internal linked list was maintained for a read-write transactions, but not for a read-only.
   For this reason, the API for using cursors should be different for writing and reading transactions;
 - However, the libmdbx's API has been significantly improved, including the ability to reuse cursors and a uniform cursors behavior for any kind of transactions.
   My mistake is that due to working with MithrilDB, I forgot to make a same changes to libmdbx.

Fixes https://github.com/erthink/libmdbx/issues/272.
2022-03-03 15:00:28 +03:00
Леонид Юрьев (Leonid Yuriev)
96c93ac2f1 mdbx++: refine mdbx::env::operate_parameters() and related. 2022-03-03 14:13:10 +03:00
Леонид Юрьев (Leonid Yuriev)
6d61b18325 mdbx: fix auxiliary mdbx_txn_valgrind().
Fixed https://github.com/erthink/libmdbx/issues/273.
2022-02-27 15:42:48 +03:00
53 changed files with 1935 additions and 3397 deletions

View File

@@ -1,22 +0,0 @@
version: 2
jobs:
build:
branches:
ignore:
- gh-pages
docker:
- image: circleci/buildpack-deps:20.04
environment:
- TESTDB: /tmp/test.db
- TESTLOG: /tmp/test.log
steps:
- checkout
- run: ulimit -c unlimited && MDBX_BUILD_OPTIONS="-DNDEBUG=1 -DMDBX_FORCE_ASSERTIONS=1" make test-ubsan
- run:
command: |
mkdir -p /tmp/artifacts
mv -t /tmp/artifacts $TESTLOG $TESTDB core.*
when: on_fail
- store_artifacts:
path: /tmp/artifacts
destination: test-artifacts

View File

@@ -1,6 +0,0 @@
freebsd_instance:
image_family: freebsd-12-1-snap
task:
install_script: pkg install -y gmake bash git
script: git fetch --tags --force && gmake MDBX_BUILD_OPTIONS="-DNDEBUG=1 -DMDBX_FORCE_ASSERTIONS=1" check

12
.github/FUNDING.yml vendored
View File

@@ -1,12 +0,0 @@
# These are supported funding model platforms
github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username
otechie: # Replace with a single Otechie username
custom: ['https://paypal.me/erthink', 'https://www.blockchain.com/en/eth/address/0x19291d8658f762f3baceae1700c0b9466572ceab', 'https://www.blockchain.com/en/btc/address/152u2KXNWWGHQS3qiBEoQaveWyPvaSWYGC']

View File

@@ -1,11 +0,0 @@
\.def$
^AUTHORS$
^\.github/FUNDING\.yml$
^\.github/actions/spelling/
^\.github/workflows/ci\.yml$
^\.github/workflows/coverity\.yml$
^\.github/workflows/doxygen-github-pages\.yml$
^\.gitignore$
^\.travis\.yml$
^packages/buildroot/
^CMakeSettings\.json$

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +0,0 @@
# numbers
(?:[\\0][xX]|[uU]\+|#)[0-9a-fA-FgGrR]{2,}[uU]?[lL]{0,2}\b
# avoid false positive
-{1,2}force\W
# flags
(?!\w)-{1,2}[fDW]

View File

@@ -1,65 +0,0 @@
name: MinGW
on:
pull_request:
push:
branches: mingw
paths-ignore:
- '.circleci/**'
- '.github/actions/spelling/**'
- 'docs/**'
- 'packages**'
- .cirrus.yml
- .clang-format
- .gitignore
- .travis.yml
- AUTHORS
- COPYRIGHT
- ChangeLog.md
- LICENSE
- README.md
- appveyor.yml
jobs:
build:
runs-on: [windows-latest]
steps:
- uses: actions/checkout@v2
- name: fetch tags
run: git fetch --unshallow --tags --prune --force
- name: append PATH
run: echo 'C:/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/bin' >> $GITHUB_PATH
- name: update mingw64
# wanna version >= 10.2
run: choco upgrade mingw -y --no-progress && refreshenv
- name: configure-dll
shell: bash
run: |
mkdir build-dll && cd build-dll && \
cmake -G "MinGW Makefiles" -DMDBX_BUILD_SHARED_LIBRARY:BOOL=ON -DMDBX_INSTALL_STATIC:BOOL=OFF -DMDBX_ENABLE_TESTS:BOOL=ON ..
- name: build-dll
shell: bash
run: cd build-dll && cmake --build .
- name: test-dll
shell: bash
run: |
export "PATH=/c/programdata/chocolatey/lib/mingw/tools/install/mingw64/bin:$PATH" && \
cd build-dll && \
./mdbx_test.exe --progress --console=no --pathname=test.db --dont-cleanup-after basic && \
./mdbx_chk.exe -nvv test.db
- name: configure-static
shell: bash
run: |
mkdir build-static && cd build-static && \
cmake -G "MinGW Makefiles" -DMDBX_BUILD_SHARED_LIBRARY:BOOL=OFF -DMDBX_INSTALL_STATIC:BOOL=ON -DMDBX_ENABLE_TESTS:BOOL=ON ..
- name: build-static
shell: bash
run: |
cd build-static && cmake --build .
- name: test-static
shell: bash
run: |
export "PATH=/c/programdata/chocolatey/lib/mingw/tools/install/mingw64/bin:$PATH" && \
cd build-static && \
./mdbx_test.exe --progress --console=no --pathname=test.db --dont-cleanup-after basic && \
./mdbx_chk.exe -nvv test.db

View File

@@ -1,41 +0,0 @@
name: Android
on:
pull_request:
push:
branches-ignore:
- coverity_scan
paths-ignore:
- '.circleci/**'
- '.github/actions/spelling/**'
- 'docs/**'
- 'packages**'
- .cirrus.yml
- .clang-format
- .gitignore
- .travis.yml
- AUTHORS
- COPYRIGHT
- ChangeLog.md
- LICENSE
- README.md
- appveyor.yml
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: fetch tags
run: git fetch --unshallow --tags --prune --force
- uses: nttld/setup-ndk@v1
id: setup-ndk
with:
ndk-version: r21e
add-to-path: true
- name: configure
env:
ANDROID_NDK_HOME: ${{ steps.setup-ndk.outputs.ndk-path }}
run: cmake --version && cmake --toolchain "${{ steps.setup-ndk.outputs.ndk-path }}/build/cmake/android.toolchain.cmake"
- name: build
run: cmake --build .

View File

@@ -1,37 +0,0 @@
name: CI
on:
pull_request:
push:
branches-ignore:
- coverity_scan
paths-ignore:
- '.circleci/**'
- '.github/actions/spelling/**'
- 'docs/**'
- 'packages**'
- .cirrus.yml
- .clang-format
- .gitignore
- .travis.yml
- AUTHORS
- COPYRIGHT
- ChangeLog.md
- LICENSE
- README.md
- appveyor.yml
jobs:
build:
runs-on: ${{ matrix.os }}
strategy:
matrix:
#, windows-latest
os: [ubuntu-latest, macos-latest, ubuntu-18.04]
steps:
- uses: actions/checkout@v2
- name: fetch tags
run: git fetch --unshallow --tags --prune --force
- name: make check
run: make MDBX_BUILD_OPTIONS="-DNDEBUG=1 -DMDBX_FORCE_ASSERTIONS=1" --keep-going all && MALLOC_CHECK_=7 MALLOC_PERTURB_=42 make MDBX_BUILD_OPTIONS="-DNDEBUG=1 -DMDBX_FORCE_ASSERTIONS=1" --keep-going check
shell: bash

View File

@@ -1,55 +0,0 @@
name: Coverity
on:
push:
branches: coverity_scan
paths-ignore:
- '.circleci/**'
- '.github/actions/spelling/**'
- 'docs/**'
- 'packages**'
- .cirrus.yml
- .clang-format
- .gitignore
- .travis.yml
- AUTHORS
- COPYRIGHT
- ChangeLog.md
- LICENSE
- README.md
- appveyor.yml
env:
COVERITY_SCAN_PROJECT_NAME: 'ReOpen/libmdbx'
COVERITY_SCAN_TOKEN: ${{ secrets.COVERITY_SCAN_TOKEN }}
COVERITY_SCAN_BUILD_COMMAND: 'make MDBX_BUILD_OPTIONS=-DMDBX_DEBUG=2 CXXSTD=-std=gnu++17 build-test'
COVERITY_SCAN_NOTIFICATION_EMAIL: 'leo@yuriev.ru'
COVERITY_UNSUPPORTED_COMPILER_INVOCATION: 1
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: fetch tags
run: git fetch --unshallow --tags --prune --force
- name: Download Coverity Build Tool
run: |
wget -q https://scan.coverity.com/download/cxx/linux64 --post-data "token=$COVERITY_SCAN_TOKEN&project=$COVERITY_SCAN_PROJECT_NAME" -O cov-analysis-linux64.tar.gz
mkdir cov-analysis-linux64
tar xzf cov-analysis-linux64.tar.gz --strip 1 -C cov-analysis-linux64
- name: Build with cov-build
run: |
export PATH=`pwd`/cov-analysis-linux64/bin:$PATH
cov-build --dir cov-int $COVERITY_SCAN_BUILD_COMMAND
- name: Submit the result to Coverity Scan
run: |
tar czvf libmdbx.tgz cov-int
curl \
--form project=$COVERITY_SCAN_PROJECT_NAME \
--form token=$COVERITY_SCAN_TOKEN \
--form email=$COVERITY_SCAN_NOTIFICATION_EMAIL \
--form file=@libmdbx.tgz \
--form version=$GITHUB_SHA \
--form description="GithubActionCI $GITHUB_ACTION" \
https://scan.coverity.com/builds?project=$COVERITY_SCAN_PROJECT_NAME

View File

@@ -1,29 +0,0 @@
name: doxygen-github-pages
on:
push:
branches: master
jobs:
build:
runs-on: ubuntu-20.04
steps:
- name: Set Actions Allow Unsecure Commands (temporary)
run: |
echo "ACTIONS_ALLOW_UNSECURE_COMMANDS=true" >> $GITHUB_ENV
- name: Checkout code
uses: actions/checkout@v2.3.1
with:
fetch-depth: 0
- name: Install doxygen
run: sudo apt install doxygen graphviz fonts-freefont-ttf
- name: Build html docs
run: make doxygen && cp -R .circleci docs/html/
- name: Deploy gh-pages
uses: JamesIves/github-pages-deploy-action@3.5.7
with:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
BRANCH: gh-pages
FOLDER: docs/html
CLEAN: true
SINGLE_COMMIT: true

View File

@@ -1,58 +0,0 @@
# Based on the https://github.com/actions/upload-release-asset example
on:
push:
# Sequence of patterns matched against refs/tags
tags:
- 'v*' # Push events to matching v*, i.e. v1.0, v20.15.10
name: Upload Release Asset
jobs:
build:
name: Upload Release Asset
runs-on: ubuntu-20.04
steps:
- name: Checkout code
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Build assets
run: |
make release-assets
- id: name
run: |
echo "::set-output name=tarball::$(ls *.tar.gz)"
echo "::set-output name=zip::$(ls *.zip)"
echo "::set-output name=suffix_unix::$(sed 's|^\([0-9]\{1,\}\.[0-9]\{1,\}\.[0-9]\{1,\}\)\.\(.*\)$|\1|' dist/VERSION.txt)"
echo "::set-output name=suffix_dos::$(sed 's|^\([0-9]\{1,\}\.[0-9]\{1,\}\.[0-9]\{1,\}\)\.\(.*\)$|\1|' dist/VERSION.txt | tr . _)"
- name: Create Release
id: create_release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: ${{ github.ref }}
release_name: Release ${{ github.ref }}
draft: true
prerelease: true
- name: Upload tarball
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ${{ steps.name.outputs.tarball }}
asset_name: libmdbx-amalgamated-${{ steps.name.outputs.suffix_unix }}.tar.gz
# asset_label: Amalgamated source tarball
asset_content_type: application/tar+gzip
- name: Upload zip
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ${{ steps.name.outputs.zip }}
asset_name: libmdbx-amalgamated-${{ steps.name.outputs.suffix_dos }}.zip
# asset_label: Amalgamated source zip-archive
asset_content_type: application/zip

View File

@@ -1,14 +0,0 @@
name: Spell checking
on:
push:
branches: spelling
pull_request:
jobs:
build:
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 5
- uses: check-spelling/check-spelling@main

View File

@@ -1,89 +0,0 @@
language: c cpp
sudo: false
env:
global:
- secure: "M+W+heGGyRQJoBq2W0uqWVrpL4KBXmL0MFL7FSs7f9vmAaDyEgziUXeZRj3GOKzW4kTef3LpIeiu9SmvqSMoQivGGiomZShqPVl045o/OUgRCAT7Al1RLzEZ0efSHpIPf0PZ6byEf6GR2ML76OfuL6JxTVdnz8iVyO2sgLE1HbX1VeB+wgd/jfMeOBhCCXskfK6MLyZihfMYsiYZYSaV98ZDhDLSlzuuRIgzb0bMi8aL6AErs0WLW0NelRBeHkKPYfAUc85pdQHscgrJw6Rh/zT6+8BQ/q5f4IgWhiu4xoRg3Ngl7SNoedRQh93ADM3UG2iGl6HDFpVORaXcFWKAtuYY+kHQ0HB84BRYpQmeBuXNpltsfxQ3d1Q3u0RlE45zRvmr2+X1mFnkcNUAWISLPbsOUlriDQM8irGwRpho77/uYnRC00bJsHW//s6+uPf9zrAw1nI4f0y3PAWukGF/xs6HAI3FZPsuSSnx18Tj3Opgbc9Spop+V3hkhdiJoPGpNKTkFX4ZRXfkPgoRVJmtp4PpbpH0Ps/mCriKjMEfGGi0HcVCi0pEGLXiecdqJ5KPg5+22zNycEujQBJcNTKd9shN+R3glrbmhAxTEzGdGwxXXJ2ybwJ2PWJLMYZ7g98nLyX+uQPaA3BlsbYJHNeS5283/9pJsd9DzfHKsN2nFSc="
addons:
apt:
sources:
- ubuntu-toolchain-r-test
packages:
- cmake
- clang-format
update: true
matrix:
include:
- os: linux
dist: focal
compiler: gcc
env: CC=cc CXX=c++
- os: linux
dist: focal
compiler: clang
env: CC=clang CXX=clang++
- os: linux
dist: bionic
compiler: gcc
env: CC=cc CXX=c++
- os: linux
dist: bionic
compiler: clang
env: CC=clang CXX=clang++
- os: linux
dist: xenial
compiler: gcc
env: CC=cc CXX=c++
- os: linux
dist: xenial
compiler: clang
env: CC=clang CXX=clang++
- os: osx
osx_image: xcode11.3
env: CC=cc CXX=c++
- os: osx
osx_image: xcode9.4
env: CC=cc CXX=c++
before_script: |
if [ "${TRAVIS_BRANCH}" = "coverity_scan" ]; then
# call Coverity Scan manually of addons.coverity_scan for first job only
if [ "${TRAVIS_JOB_NUMBER}" = "${TRAVIS_BUILD_NUMBER}.1" ]; then
export COVERITY_SCAN_BRANCH=1
echo -n | openssl s_client -connect scan.coverity.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | sudo tee -a /etc/ssl/certs/ca-
curl -s 'https://scan.coverity.com/scripts/travisci_build_coverity_scan.sh' -o coverity_scan.sh
else
echo 'echo "Skip CoverityScan for unrelated os/compiler"' > coverity_scan.sh
fi
fi
script: |
${CC} --version
${CXX} --version
git fetch --unshallow --tags --prune --force || exit 1
if [ ! -s ./coverity_scan.sh ]; then
make --keep-going all && MALLOC_CHECK_=7 MALLOC_PERTURB_=42 make --keep-going check
else
COVERITY_UNSUPPORTED_COMPILER_INVOCATION=1 \
COVERITY_SCAN_PROJECT_NAME="ReOpen/libmdbx" \
COVERITY_SCAN_NOTIFICATION_EMAIL="leo@yuriev.ru" \
COVERITY_SCAN_BUILD_COMMAND_PREPEND="" \
COVERITY_SCAN_BUILD_COMMAND="make MDBX_BUILD_OPTIONS=-DMDBX_DEBUG=2 CXXSTD=-std=gnu++17 build-test" \
COVERITY_SCAN_BRANCH_PATTERN="$TRAVIS_BRANCH" \
bash ./coverity_scan.sh || cat cov-int/scm_log.txt
fi
after_script: |
if [ "${TRAVIS_BRANCH}" != "coverity_scan" -a "${TRAVIS_JOB_NUMBER}" = "${TRAVIS_BUILD_NUMBER}.1" ] && make reformat && [[ -n $(git diff) ]]; then
echo "You must run 'make reformat' before submitting a pull request"
echo "-------------------------------------------------------------------------------"
git diff
sleep 1
echo "-------------------------------------------------------------------------------"
sleep 1
exit -1
fi
echo "-------------------------------------------------------------------------------"
sleep 1

View File

@@ -14,7 +14,7 @@
##
## libmdbx = { Revised and extended descendant of Symas LMDB. }
## Please see README.md at https://github.com/erthink/libmdbx
## Please see README.md at https://gitflic.ru/project/erthink/libmdbx
##
## Libmdbx is superior to LMDB in terms of features and reliability,
## not inferior in performance. libmdbx works on Linux, FreeBSD, MacOS X
@@ -112,7 +112,10 @@ else()
endif()
endif()
if(MDBX_AMALGAMATED_SOURCE AND DEFINED MDBX_ENABLE_TESTS AND MDBX_ENABLE_TESTS)
if(NOT MDBX_AMALGAMATED_SOURCE)
include(CTest)
option(MDBX_ENABLE_TESTS "Build libmdbx tests." ${BUILD_TESTING})
elseif(DEFINED MDBX_ENABLE_TESTS AND MDBX_ENABLE_TESTS)
message(WARNING "MDBX_ENABLE_TESTS=${MDBX_ENABLE_TESTS}: But amalgamated source code don't includes tests.")
set(MDBX_ENABLE_TESTS OFF)
endif()
@@ -527,6 +530,11 @@ if(MDBX_BUILD_CXX AND NOT CMAKE_CXX_COMPILER_LOADED)
message(FATAL_ERROR "MDBX_BUILD_CXX=${MDBX_BUILD_CXX}: The C++ compiler is required to build the C++API.")
endif()
if(MDBX_BUILD_CXX)
# determine library for C++ std::filesystem
probe_libcxx_filesystem()
endif()
# Get version
fetch_version(MDBX "${CMAKE_CURRENT_SOURCE_DIR}" FALSE)
message(STATUS "libmdbx version is ${MDBX_VERSION}")
@@ -603,7 +611,7 @@ endmacro()
macro(libmdbx_setup_libs TARGET MODE)
target_link_libraries(${TARGET} ${MODE} Threads::Threads)
if(${CMAKE_SYSTEM_NAME} STREQUAL "Windows")
target_link_libraries(${TARGET} ${MODE} ntdll)
target_link_libraries(${TARGET} ${MODE} ntdll advapi32)
if(MDBX_NTDLL_EXTRA_IMPLIB AND MDBX_WITHOUT_MSVC_CRT)
target_link_libraries(${TARGET} ${MODE} ntdll_extra)
endif()
@@ -612,8 +620,12 @@ macro(libmdbx_setup_libs TARGET MODE)
elseif(${CMAKE_SYSTEM_NAME} STREQUAL "Android")
target_link_libraries(${TARGET} ${MODE} log)
endif()
if(MDBX_CXX_STANDARD GREATER_EQUAL 17 AND LIBCXX_FILESYSTEM AND MDBX_BUILD_CXX)
target_link_libraries(${TARGET} ${MODE} ${LIBCXX_FILESYSTEM})
if(LIBCXX_FILESYSTEM AND MDBX_BUILD_CXX)
if(CMAKE_COMPILER_IS_ELBRUSCXX AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 1.25.23
AND NOT CMAKE_VERSION VERSION_LESS 3.13)
target_link_options(${TARGET} PUBLIC "-Wl,--allow-multiple-definition")
endif()
target_link_libraries(${TARGET} PUBLIC ${LIBCXX_FILESYSTEM})
endif()
endmacro()
@@ -632,6 +644,7 @@ if(MDBX_BUILD_SHARED_LIBRARY)
else()
set_target_properties(mdbx-static PROPERTIES OUTPUT_NAME mdbx)
endif()
target_include_directories(mdbx-static INTERFACE "${CMAKE_CURRENT_SOURCE_DIR}")
################################################################################
@@ -653,6 +666,7 @@ if(MDBX_BUILD_SHARED_LIBRARY)
set_target_properties(mdbx PROPERTIES LINK_FLAGS "-fvisibility=hidden")
endif()
list(APPEND MDBX_BUILD_FLAGS ${CMAKE_SHARED_LINKER_FLAGS})
target_include_directories(mdbx INTERFACE "${CMAKE_CURRENT_SOURCE_DIR}")
endif()
if(MDBX_BUILD_SHARED_LIBRARY AND MDBX_LINK_TOOLS_NONSTATIC)
@@ -847,16 +861,33 @@ endif()
# make a build-target triplet
if(CMAKE_C_COMPILER_TARGET)
set(MDBX_BUILD_TARGET "${CMAKE_C_COMPILER_TARGET}")
elseif(CMAKE_C_PLATFORM_ID AND NOT CMAKE_C_PLATFORM_ID STREQUAL CMAKE_SYSTEM_NAME)
string(STRIP "${CMAKE_C_PLATFORM_ID}-${CMAKE_SYSTEM_NAME}" MDBX_BUILD_TARGET)
elseif(CMAKE_LIBRARY_ARCHITECTURE)
string(STRIP "${CMAKE_LIBRARY_ARCHITECTURE}-${CMAKE_SYSTEM_NAME}" MDBX_BUILD_TARGET)
elseif(CMAKE_GENERATOR_PLATFORM AND NOT CMAKE_C_PLATFORM_ID STREQUAL CMAKE_SYSTEM_NAME)
string(STRIP "${CMAKE_GENERATOR_PLATFORM}-${CMAKE_SYSTEM_NAME}" MDBX_BUILD_TARGET)
elseif(CMAKE_SYSTEM_ARCH)
string(STRIP "${CMAKE_SYSTEM_ARCH}-${CMAKE_SYSTEM_NAME}" MDBX_BUILD_TARGET)
else()
string(STRIP "${CMAKE_SYSTEM_PROCESSOR}-${CMAKE_SYSTEM_NAME}" MDBX_BUILD_TARGET)
if(CMAKE_C_COMPILER_ARCHITECTURE_ID)
string(STRIP "${CMAKE_C_COMPILER_ARCHITECTURE_ID}" MDBX_BUILD_TARGET)
elseif(CMAKE_GENERATOR_PLATFORM AND NOT CMAKE_GENERATOR_PLATFORM STREQUAL CMAKE_SYSTEM_NAME)
string(STRIP "${CMAKE_GENERATOR_PLATFORM}" MDBX_BUILD_TARGET)
elseif(CMAKE_SYSTEM_ARCH)
string(STRIP "${CMAKE_SYSTEM_ARCH}" MDBX_BUILD_TARGET)
elseif(CMAKE_LIBRARY_ARCHITECTURE)
string(STRIP "${CMAKE_LIBRARY_ARCHITECTURE}" MDBX_BUILD_TARGET)
elseif(CMAKE_SYSTEM_PROCESSOR)
string(STRIP "${CMAKE_SYSTEM_PROCESSOR}" MDBX_BUILD_TARGET)
else()
set(MDBX_BUILD_TARGET "unknown")
endif()
if(CMAKE_C_COMPILER_ABI
AND NOT (CMAKE_C_COMPILER_ABI MATCHES ".*${MDBX_BUILD_TARGET}.*" OR MDBX_BUILD_TARGET MATCHES ".*${CMAKE_C_COMPILER_ABI}.*"))
string(APPEND MDBX_BUILD_TARGET "-${CMAKE_C_COMPILER_ABI}")
endif()
if(CMAKE_C_PLATFORM_ID
AND NOT (CMAKE_SYSTEM_NAME
AND (CMAKE_C_PLATFORM_ID MATCHES ".*${CMAKE_SYSTEM_NAME}.*" OR CMAKE_SYSTEM_NAME MATCHES ".*${CMAKE_C_PLATFORM_ID}.*"))
AND NOT (CMAKE_C_PLATFORM_ID MATCHES ".*${CMAKE_C_PLATFORM_ID}.*" OR MDBX_BUILD_TARGET MATCHES ".*${CMAKE_C_PLATFORM_ID}.*"))
string(APPEND MDBX_BUILD_TARGET "-${CMAKE_C_COMPILER_ABI}")
endif()
if(CMAKE_SYSTEM_NAME)
string(APPEND MDBX_BUILD_TARGET "-${CMAKE_SYSTEM_NAME}")
endif()
endif()
# provide build-type

View File

@@ -3,24 +3,103 @@ ChangeLog
### TODO
- [Engage an "overlapped I/O" on Windows](https://github.com/erthink/libmdbx/issues/224).
- [Simple careful mode for working with corrupted DB](https://github.com/erthink/libmdbx/issues/223).
- [Move most of `mdbx_chk` functional to the library API](https://github.com/erthink/libmdbx/issues/204).
- [Replace SRW-lock on Windows to allow shrink DB with `MDBX_NOTLS` option](https://github.com/erthink/libmdbx/issues/210).
- [More flexible support of asynchronous runtime/framework(s)](https://github.com/erthink/libmdbx/issues/200).
- [Migration guide from LMDB to MDBX](https://github.com/erthink/libmdbx/issues/199).
- [Get rid of dirty-pages list in MDBX_WRITEMAP mode](https://github.com/erthink/libmdbx/issues/193).
- [Large/Overflow pages accounting for dirty-room](https://github.com/erthink/libmdbx/issues/192).
- [Support for RAW devices](https://github.com/erthink/libmdbx/issues/124).
- [Support MessagePack for Keys & Values](https://github.com/erthink/libmdbx/issues/115).
- [Engage new terminology](https://github.com/erthink/libmdbx/issues/137).
- [Engage an "overlapped I/O" on Windows](todo4recovery://erased_by_github/libmdbx/issues/224).
- [Simple careful mode for working with corrupted DB](todo4recovery://erased_by_github/libmdbx/issues/223).
- [Move most of `mdbx_chk` functional to the library API](todo4recovery://erased_by_github/libmdbx/issues/204).
- [Replace SRW-lock on Windows to allow shrink DB with `MDBX_NOTLS` option](todo4recovery://erased_by_github/libmdbx/issues/210).
- [More flexible support of asynchronous runtime/framework(s)](todo4recovery://erased_by_github/libmdbx/issues/200).
- [Migration guide from LMDB to MDBX](todo4recovery://erased_by_github/libmdbx/issues/199).
- [Get rid of dirty-pages list in MDBX_WRITEMAP mode](todo4recovery://erased_by_github/libmdbx/issues/193).
- [Large/Overflow pages accounting for dirty-room](todo4recovery://erased_by_github/libmdbx/issues/192).
- [Support for RAW devices](todo4recovery://erased_by_github/libmdbx/issues/124).
- [Support MessagePack for Keys & Values](todo4recovery://erased_by_github/libmdbx/issues/115).
- [Engage new terminology](todo4recovery://erased_by_github/libmdbx/issues/137).
- Packages for [Astra Linux](https://astralinux.ru/), [ALT Linux](https://www.altlinux.org/), [ROSA Linux](https://www.rosalinux.ru/), etc.
## v0.11.7 at 2022-04-22
The stable risen release after the Github's intentional malicious disaster.
#### We have migrated to a reliable trusted infrastructure
The origin for now is at [GitFlic](https://gitflic.ru/project/erthink/libmdbx)
since on 2022-04-15 the Github administration, without any warning nor
explanation, deleted _libmdbx_ along with a lot of other projects,
simultaneously blocking access for many developers.
For the same reason ~~Github~~ is blacklisted forever.
GitFlic already support Russian and English languages, plan to support more,
including 和 中文. You are welcome!
New:
- Added the `tools-static` make target to build statically linked MDBX tools.
- Support for Microsoft Visual Studio 2022.
- Support build by MinGW' make from command line without CMake.
- Added `mdbx::filesystem` C++ API namespace that corresponds to `std::filesystem` or `std::experimental::filesystem`.
- Created [website](https://libmdbx.website.yandexcloud.net/) for online auto-generated documentation.
- Used `todo4recovery://erased_by_github/` for dead (or temporarily lost) resources deleted by ~~Github~~.
- Added `--loglevel=` command-line option to the `mdbx_test` tool.
- Added few fast smoke-like tests into CMake builds.
Fixes:
- Fixed a race between starting a transaction and creating a DBI descriptor that could lead to `SIGSEGV` in the cursor tracking code.
- Clarified description of `MDBX_EPERM` error returned from `mdbx_env_set_geometry()`.
- Fixed non-promoting the parent transaction to be dirty in case the undo of the geometry update failed during abortion of a nested transaction.
- Resolved linking issues with `libstdc++fs`/`libc++fs`/`libc++experimental` for C++ `std::filesystem` or `std::experimental::filesystem` for legacy compilers.
- Added workaround for GNU Make 3.81 and earlier.
- Added workaround for Elbrus/LCC 1.25 compiler bug of class inline `static constexpr` member field.
- [Fixed](https://github.com/ledgerwatch/erigon/issues/3874) minor assertion regression (only debug builds were affected).
- Fixed detection of `C++20` concepts accessibility.
- Fixed detection of Clang's LTO availability for Android.
- Fixed build for ARM/ARM64 by MSVC.
- Fixed non-x86 Windows builds with `MDBX_WITHOUT_MSVC_CRT=ON` and `MDBX_BUILD_SHARED_LIBRARY=ON`.
Minors:
- Resolve minor MSVC warnings: avoid `/INCREMENTAL[:YES]` with `/LTCG`, `/W4` with `/W3`, the `C5105` warning.
- Switched to using `MDBX_EPERM` instead of `MDBX_RESULT_TRUE' to indicate that the geometry cannot be updated.
- Added `NULL` checking during memory allocation inside `mdbx_chk`.
- Resolved all warnings from MinGW while used without CMake.
- Added inheretable `target_include_directories()` to `CMakeLists.txt` for easy integration.
- Added build-time checks and paranoid runtime assertions for the `off_t` arguments of `fcntl()` which are used for locking.
- Added `-Wno-lto-type-mismatch` to avoid false-positive warnings from old GCC during LTO-enabled builds.
- Added checking for TID (system thread id) to avoid hang on 32-bit Bionic/Android within `pthread_mutex_lock()`.
- Reworked `MDBX_BUILD_TARGET` of CMake builds.
- Added `CMAKE_HOST_ARCH` and `CMAKE_HOST_CAN_RUN_EXECUTABLES_BUILT_FOR_TARGET`.
## v0.11.6 at 2022-03-24
The stable release with the complete workaround for an incoherence flaw of Linux unified page/buffer cache.
Nonetheless the cause for this trouble may be an issue of Intel CPU cache/MESI.
See [issue#269](todo4recovery://erased_by_github/libmdbx/issues/269) for more information.
Acknowledgements:
- [David Bouyssié](https://github.com/david-bouyssie) for [Scala bindings](https://github.com/david-bouyssie/mdbx4s).
- [Michelangelo Riccobene](https://github.com/mriccobene) for reporting and testing.
Fixes:
- [Added complete workaround](todo4recovery://erased_by_github/libmdbx/issues/269) for an incoherence flaw of Linux unified page/buffer cache.
- [Fixed](todo4recovery://erased_by_github/libmdbx/issues/272) cursor reusing for read-only transactions.
- Fixed copy&paste typo inside `mdbx::cursor::find_multivalue()`.
Minors:
- Minor refine C++ API for convenience.
- Minor internals refines.
- Added `lib-static` and `lib-shared` targets for make.
- Added minor workaround for AppleClang 13.3 bug.
- Clarified error messages of a signature/version mismatch.
## v0.11.5 at 2022-02-23
The stable release with the hotfix/workaround for a flaw of Linux 4.19 (at least) unified page/buffer cache.
See [issue#269](https://github.com/erthink/libmdbx/issues/269) for more information.
The release with the temporary hotfix for a flaw of Linux unified page/buffer cache.
See [issue#269](todo4recovery://erased_by_github/libmdbx/issues/269) for more information.
Acknowledgements:
@@ -30,10 +109,10 @@ Acknowledgements:
Fixes:
- [Added workaround](https://github.com/erthink/libmdbx/issues/269) for a flaw of Linux 4.19 (at least) unified page/buffer cache.
- [Fixed/Reworked](https://github.com/erthink/libmdbx/pull/270) move-assignment operators for "managed" classes of C++ API.
- [Added hotfix](todo4recovery://erased_by_github/libmdbx/issues/269) for a flaw of Linux unified page/buffer cache.
- [Fixed/Reworked](todo4recovery://erased_by_github/libmdbx/pull/270) move-assignment operators for "managed" classes of C++ API.
- Fixed potential `SIGSEGV` while open DB with overrided non-default page size.
- [Made](https://github.com/erthink/libmdbx/issues/267) `mdbx_env_open()` idempotence in failure cases.
- [Made](todo4recovery://erased_by_github/libmdbx/issues/267) `mdbx_env_open()` idempotence in failure cases.
- Refined/Fixed pages reservation inside `mdbx_update_gc()` to avoid non-reclamation in a rare cases.
- Fixed typo in a retained space calculation for the hsr-callback.
@@ -66,15 +145,15 @@ New features, extensions and improvements:
Fixes:
- Fixed handling `MDBX_opt_rp_augment_limit` for GC's records from huge transactions (Erigon/Akula/Ethereum).
- [Fixed](https://github.com/erthink/libmdbx/issues/258) build on Android (avoid including `sys/sem.h`).
- [Fixed](https://github.com/erthink/libmdbx/pull/261) missing copy assignment operator for `mdbx::move_result`.
- [Fixed](todo4recovery://erased_by_github/libmdbx/issues/258) build on Android (avoid including `sys/sem.h`).
- [Fixed](todo4recovery://erased_by_github/libmdbx/pull/261) missing copy assignment operator for `mdbx::move_result`.
- Fixed missing `&` for `std::ostream &operator<<()` overloads.
- Fixed unexpected `EXDEV` (Cross-device link) error from `mdbx_env_copy()`.
- Fixed base64 encoding/decoding bugs in auxillary C++ API.
- Fixed overflow of `pgno_t` during checking PNL on 64-bit platforms.
- [Fixed](https://github.com/erthink/libmdbx/issues/260) excessive PNL checking after sort for spilling.
- [Fixed](todo4recovery://erased_by_github/libmdbx/issues/260) excessive PNL checking after sort for spilling.
- Reworked checking `MAX_PAGENO` and DB upper-size geometry limit.
- [Fixed](https://github.com/erthink/libmdbx/issues/265) build for some combinations of versions of MSVC and Windows SDK.
- [Fixed](todo4recovery://erased_by_github/libmdbx/issues/265) build for some combinations of versions of MSVC and Windows SDK.
Minors:
@@ -101,10 +180,10 @@ Acknowledgements:
New features, extensions and improvements:
- [Added](https://github.com/erthink/libmdbx/issues/236) `mdbx_cursor_get_batch()`.
- [Added](https://github.com/erthink/libmdbx/issues/250) `MDBX_SET_UPPERBOUND`.
- [Added](todo4recovery://erased_by_github/libmdbx/issues/236) `mdbx_cursor_get_batch()`.
- [Added](todo4recovery://erased_by_github/libmdbx/issues/250) `MDBX_SET_UPPERBOUND`.
- C++ API is finalized now.
- The GC update stage has been [significantly speeded](https://github.com/erthink/libmdbx/issues/254) when fixing huge Erigon's transactions (Ethereum ecosystem).
- The GC update stage has been [significantly speeded](todo4recovery://erased_by_github/libmdbx/issues/254) when fixing huge Erigon's transactions (Ethereum ecosystem).
Fixes:
@@ -115,12 +194,12 @@ Minors:
- Fixed returning `MDBX_RESULT_TRUE` (unexpected -1) from `mdbx_env_set_option()`.
- Added `mdbx_env_get_syncbytes()` and `mdbx_env_get_syncperiod()`.
- [Clarified](https://github.com/erthink/libmdbx/pull/249) description of `MDBX_INTEGERKEY`.
- [Clarified](todo4recovery://erased_by_github/libmdbx/pull/249) description of `MDBX_INTEGERKEY`.
- Reworked/simplified `mdbx_env_sync_internal()`.
- [Fixed](https://github.com/erthink/libmdbx/issues/248) extra assertion inside `mdbx_cursor_put()` for `MDBX_DUPFIXED` cases.
- [Fixed](todo4recovery://erased_by_github/libmdbx/issues/248) extra assertion inside `mdbx_cursor_put()` for `MDBX_DUPFIXED` cases.
- Avoiding extra looping inside `mdbx_env_info_ex()`.
- Explicitly enabled core dumps from stochastic tests scripts on Linux.
- [Fixed](https://github.com/erthink/libmdbx/issues/253) `mdbx_override_meta()` to avoid false-positive assertions.
- [Fixed](todo4recovery://erased_by_github/libmdbx/issues/253) `mdbx_override_meta()` to avoid false-positive assertions.
- For compatibility reverted returning `MDBX_ENODATA`for some cases.
@@ -136,10 +215,10 @@ Acknowledgements:
Fixes:
- [Fixed compilation](https://github.com/erthink/libmdbx/pull/239) with `devtoolset-9` on CentOS/RHEL 7.
- [Fixed unexpected `MDBX_PROBLEM` error](https://github.com/erthink/libmdbx/issues/242) because of update an obsolete meta-page.
- [Fixed returning `MDBX_NOTFOUND` error](https://github.com/erthink/libmdbx/issues/243) in case an inexact value found for `MDBX_GET_BOTH` operation.
- [Fixed compilation](https://github.com/erthink/libmdbx/issues/245) without kernel/libc-devel headers.
- [Fixed compilation](todo4recovery://erased_by_github/libmdbx/pull/239) with `devtoolset-9` on CentOS/RHEL 7.
- [Fixed unexpected `MDBX_PROBLEM` error](todo4recovery://erased_by_github/libmdbx/issues/242) because of update an obsolete meta-page.
- [Fixed returning `MDBX_NOTFOUND` error](todo4recovery://erased_by_github/libmdbx/issues/243) in case an inexact value found for `MDBX_GET_BOTH` operation.
- [Fixed compilation](todo4recovery://erased_by_github/libmdbx/issues/245) without kernel/libc-devel headers.
Minors:
@@ -156,7 +235,7 @@ Minors:
The database format signature has been changed to prevent
forward-interoperability with an previous releases, which may lead to a
[false positive diagnosis of database corruption](https://github.com/erthink/libmdbx/issues/238)
[false positive diagnosis of database corruption](todo4recovery://erased_by_github/libmdbx/issues/238)
due to flaws of an old library versions.
This change is mostly invisible:
@@ -205,7 +284,7 @@ Acknowledgements:
Fixes:
- Fixed possibility of looping update GC during transaction commit (no public issue since the problem was discovered inside [Positive Technologies](https://www.ptsecurity.ru)).
- Fixed `#pragma pack` to avoid provoking some compilers to generate code with [unaligned access](https://github.com/erthink/libmdbx/issues/235).
- Fixed `#pragma pack` to avoid provoking some compilers to generate code with [unaligned access](todo4recovery://erased_by_github/libmdbx/issues/235).
- Fixed `noexcept` for potentially throwing `txn::put()` of C++ API.
Minors:
@@ -231,7 +310,7 @@ Extensions and improvements:
Fixes:
- Always setup `madvise` while opening DB (fixes https://github.com/erthink/libmdbx/issues/231).
- Always setup `madvise` while opening DB (fixes todo4recovery://erased_by_github/libmdbx/issues/231).
- Fixed checking legacy `P_DIRTY` flag (`0x10`) for nested/sub-pages.
Minors:
@@ -252,11 +331,11 @@ Acknowledgements:
- [Lionel Debroux](https://github.com/debrouxl) for fuzzing tests and reporting bugs.
- [Sergey Fedotov](https://github.com/SergeyFromHell/) for [`node-mdbx` NodeJS bindings](https://www.npmjs.com/package/node-mdbx).
- [Kris Zyp](https://github.com/kriszyp) for [`lmdbx-store` NodeJS bindings](https://github.com/kriszyp/lmdbx-store).
- [Noel Kuntze](https://github.com/Thermi) for [draft Python bindings](https://github.com/erthink/libmdbx/commits/python-bindings).
- [Noel Kuntze](https://github.com/Thermi) for [draft Python bindings](todo4recovery://erased_by_github/libmdbx/commits/python-bindings).
New features, extensions and improvements:
- [Allow to predefine/override `MDBX_BUILD_TIMESTAMP` for builds reproducibility](https://github.com/erthink/libmdbx/issues/201).
- [Allow to predefine/override `MDBX_BUILD_TIMESTAMP` for builds reproducibility](todo4recovery://erased_by_github/libmdbx/issues/201).
- Added options support for `long-stochastic` script.
- Avoided `MDBX_TXN_FULL` error for large transactions when possible.
- The `MDBX_READERS_LIMIT` increased to `32767`.
@@ -264,7 +343,7 @@ New features, extensions and improvements:
- Minimized the size of poisoned/unpoisoned regions to avoid Valgrind/ASAN stuck.
- Added more workarounds for QEMU for testing builds for 32-bit platforms, Alpha and Sparc architectures.
- `mdbx_chk` now skips iteration & checking of DB' records if corresponding page-tree is corrupted (to avoid `SIGSEGV`, ASAN failures, etc).
- Added more checks for [rare/fuzzing corruption cases](https://github.com/erthink/libmdbx/issues/217).
- Added more checks for [rare/fuzzing corruption cases](todo4recovery://erased_by_github/libmdbx/issues/217).
Backward compatibility break:
@@ -276,18 +355,18 @@ Backward compatibility break:
Fixes:
- Fixed excess meta-pages checks in case `mdbx_chk` is called to check the DB for a specific meta page and thus could prevent switching to the selected meta page, even if the check passed without errors.
- Fixed [recursive use of SRW-lock on Windows cause by `MDBX_NOTLS` option](https://github.com/erthink/libmdbx/issues/203).
- Fixed [log a warning during a new DB creation](https://github.com/erthink/libmdbx/issues/205).
- Fixed [false-negative `mdbx_cursor_eof()` result](https://github.com/erthink/libmdbx/issues/207).
- Fixed [`make install` with non-GNU `install` utility (OSX, BSD)](https://github.com/erthink/libmdbx/issues/208).
- Fixed [installation by `CMake` in special cases by complete use `GNUInstallDirs`'s variables](https://github.com/erthink/libmdbx/issues/209).
- Fixed [C++ Buffer issue with `std::string` and alignment](https://github.com/erthink/libmdbx/issues/191).
- Fixed [recursive use of SRW-lock on Windows cause by `MDBX_NOTLS` option](todo4recovery://erased_by_github/libmdbx/issues/203).
- Fixed [log a warning during a new DB creation](todo4recovery://erased_by_github/libmdbx/issues/205).
- Fixed [false-negative `mdbx_cursor_eof()` result](todo4recovery://erased_by_github/libmdbx/issues/207).
- Fixed [`make install` with non-GNU `install` utility (OSX, BSD)](todo4recovery://erased_by_github/libmdbx/issues/208).
- Fixed [installation by `CMake` in special cases by complete use `GNUInstallDirs`'s variables](todo4recovery://erased_by_github/libmdbx/issues/209).
- Fixed [C++ Buffer issue with `std::string` and alignment](todo4recovery://erased_by_github/libmdbx/issues/191).
- Fixed `safe64_reset()` for platforms without atomic 64-bit compare-and-swap.
- Fixed hang/shutdown on big-endian platforms without `__cxa_thread_atexit()`.
- Fixed [using bad meta-pages if DB was partially/recoverable corrupted](https://github.com/erthink/libmdbx/issues/217).
- Fixed [using bad meta-pages if DB was partially/recoverable corrupted](todo4recovery://erased_by_github/libmdbx/issues/217).
- Fixed extra `noexcept` for `buffer::&assign_reference()`.
- Fixed `bootid` generation on Windows for case of change system' time.
- Fixed [test framework keygen-related issue](https://github.com/erthink/libmdbx/issues/127).
- Fixed [test framework keygen-related issue](todo4recovery://erased_by_github/libmdbx/issues/127).
## v0.10.1 at 2021-06-01
@@ -308,10 +387,10 @@ New features:
Fixes:
- Fixed minor "foo not used" warnings from modern C++ compilers when building the C++ part of the library.
- Fixed confusing/messy errors when build library from unfit github's archives (https://github.com/erthink/libmdbx/issues/197).
- Fixed confusing/messy errors when build library from unfit github's archives (todo4recovery://erased_by_github/libmdbx/issues/197).
- Fixed `#elsif` typo.
- Fixed rare unexpected `MDBX_PROBLEM` error during altering data in huge transactions due to wrong spilling/oust of dirty pages (https://github.com/erthink/libmdbx/issues/195).
- Re-Fixed WSL1/WSL2 detection with distinguishing (https://github.com/erthink/libmdbx/issues/97).
- Fixed rare unexpected `MDBX_PROBLEM` error during altering data in huge transactions due to wrong spilling/oust of dirty pages (todo4recovery://erased_by_github/libmdbx/issues/195).
- Re-Fixed WSL1/WSL2 detection with distinguishing (todo4recovery://erased_by_github/libmdbx/issues/97).
## v0.10.0 at 2021-05-09
@@ -334,7 +413,7 @@ New features:
and conjointly with the `MDBX_ENV_CHECKPID=0` and `MDBX_TXN_CHECKOWNER=0` options can yield
up to 30% more performance compared to LMDB.
- Using float point (exponential quantized) representation for internal 16-bit values
of grow step and shrink threshold when huge ones (https://github.com/erthink/libmdbx/issues/166).
of grow step and shrink threshold when huge ones (todo4recovery://erased_by_github/libmdbx/issues/166).
To minimize the impact on compatibility, only the odd values inside the upper half
of the range (i.e. 32769..65533) are used for the new representation.
- Added the `mdbx_drop` similar to LMDB command-line tool to purge or delete (sub)database(s).
@@ -343,7 +422,7 @@ New features:
- The internal node sizes were refined, resulting in a reduction in large/overflow pages in some use cases
and a slight increase in limits for a keys size to ≈½ of page size.
- Added to `mdbx_chk` output number of keys/items on pages.
- Added explicit `install-strip` and `install-no-strip` targets to the `Makefile` (https://github.com/erthink/libmdbx/pull/180).
- Added explicit `install-strip` and `install-no-strip` targets to the `Makefile` (todo4recovery://erased_by_github/libmdbx/pull/180).
- Major rework page splitting (af9b7b560505684249b76730997f9e00614b8113) for
- An "auto-appending" feature upon insertion for both ascending and
descending key sequences. As a result, the optimality of page filling
@@ -351,7 +430,7 @@ New features:
inserting ordered sequences of keys,
- A "splitting at middle" to make page tree more balanced on average.
- Added `mdbx_get_sysraminfo()` to the API.
- Added guessing a reasonable maximum DB size for the default upper limit of geometry (https://github.com/erthink/libmdbx/issues/183).
- Added guessing a reasonable maximum DB size for the default upper limit of geometry (todo4recovery://erased_by_github/libmdbx/issues/183).
- Major rework internal labeling of a dirty pages (958fd5b9479f52f2124ab7e83c6b18b04b0e7dda) for
a "transparent spilling" feature with the gist to make a dirty pages
be ready to spilling (writing to a disk) without further altering ones.
@@ -367,7 +446,7 @@ New features:
- Support `make help` to list available make targets.
- Silently `make`'s build by default.
- Preliminary [Python bindings](https://github.com/Thermi/libmdbx/tree/python-bindings) is available now
by [Noel Kuntze](https://github.com/Thermi) (https://github.com/erthink/libmdbx/issues/147).
by [Noel Kuntze](https://github.com/Thermi) (todo4recovery://erased_by_github/libmdbx/issues/147).
Backward compatibility break:
@@ -382,22 +461,22 @@ Backward compatibility break:
Fixes:
- Fixed performance regression due non-optimal C11 atomics usage (https://github.com/erthink/libmdbx/issues/160).
- Fixed "reincarnation" of subDB after it deletion (https://github.com/erthink/libmdbx/issues/168).
- Fixed performance regression due non-optimal C11 atomics usage (todo4recovery://erased_by_github/libmdbx/issues/160).
- Fixed "reincarnation" of subDB after it deletion (todo4recovery://erased_by_github/libmdbx/issues/168).
- Fixed (disallowing) implicit subDB deletion via operations on `@MAIN`'s DBI-handle.
- Fixed a crash of `mdbx_env_info_ex()` in case of a call for a non-open environment (https://github.com/erthink/libmdbx/issues/171).
- Fixed the selecting/adjustment values inside `mdbx_env_set_geometry()` for implicit out-of-range cases (https://github.com/erthink/libmdbx/issues/170).
- Fixed `mdbx_env_set_option()` for set initial and limit size of dirty page list ((https://github.com/erthink/libmdbx/issues/179).
- Fixed an unreasonably huge default upper limit for DB geometry (https://github.com/erthink/libmdbx/issues/183).
- Fixed a crash of `mdbx_env_info_ex()` in case of a call for a non-open environment (todo4recovery://erased_by_github/libmdbx/issues/171).
- Fixed the selecting/adjustment values inside `mdbx_env_set_geometry()` for implicit out-of-range cases (todo4recovery://erased_by_github/libmdbx/issues/170).
- Fixed `mdbx_env_set_option()` for set initial and limit size of dirty page list ((todo4recovery://erased_by_github/libmdbx/issues/179).
- Fixed an unreasonably huge default upper limit for DB geometry (todo4recovery://erased_by_github/libmdbx/issues/183).
- Fixed `constexpr` specifier for the `slice::invalid()`.
- Fixed (no)readahead auto-handling (https://github.com/erthink/libmdbx/issues/164).
- Fixed (no)readahead auto-handling (todo4recovery://erased_by_github/libmdbx/issues/164).
- Fixed non-alloy build for Windows.
- Switched to using Heap-functions instead of LocalAlloc/LocalFree on Windows.
- Fixed `mdbx_env_stat_ex()` to returning statistics of the whole environment instead of MainDB only (https://github.com/erthink/libmdbx/issues/190).
- Fixed `mdbx_env_stat_ex()` to returning statistics of the whole environment instead of MainDB only (todo4recovery://erased_by_github/libmdbx/issues/190).
- Fixed building by GCC 4.8.5 (added workaround for a preprocessor's bug).
- Fixed building C++ part for iOS <= 13.0 (unavailability of `std::filesystem::path`).
- Fixed building for Windows target versions prior to Windows Vista (`WIN32_WINNT < 0x0600`).
- Fixed building by MinGW for Windows (https://github.com/erthink/libmdbx/issues/155).
- Fixed building by MinGW for Windows (todo4recovery://erased_by_github/libmdbx/issues/155).
## v0.9.3 at 2021-02-02
@@ -417,7 +496,7 @@ Removed options and features:
New features:
- Package for FreeBSD is available now by Mahlon E. Smith.
- New API functions to get/set various options (https://github.com/erthink/libmdbx/issues/128):
- New API functions to get/set various options (todo4recovery://erased_by_github/libmdbx/issues/128):
- the maximum number of named databases for the environment;
- the maximum number of threads/reader slots;
- threshold (since the last unsteady commit) to force flush the data buffers to disk;
@@ -430,7 +509,7 @@ New features:
- maximal part of the dirty pages may be spilled when necessary;
- minimal part of the dirty pages should be spilled when necessary;
- how much of the parent transaction dirty pages will be spilled while start each child transaction;
- Unlimited/Dynamic size of retired and dirty page lists (https://github.com/erthink/libmdbx/issues/123).
- Unlimited/Dynamic size of retired and dirty page lists (todo4recovery://erased_by_github/libmdbx/issues/123).
- Added `-p` option (purge subDB before loading) to `mdbx_load` tool.
- Reworked spilling of large transaction and committing of nested transactions:
- page spilling code reworked to avoid the flaws and bugs inherited from LMDB;
@@ -440,22 +519,22 @@ New features:
- Added `MDBX_ENABLE_REFUND` and `MDBX_PNL_ASCENDING` internal/advanced build options.
- Added `mdbx_default_pagesize()` function.
- Better support architectures with a weak/relaxed memory consistency model (ARM, AARCH64, PPC, MIPS, RISC-V, etc) by means [C11 atomics](https://en.cppreference.com/w/c/atomic).
- Speed up page number lists and dirty page lists (https://github.com/erthink/libmdbx/issues/132).
- Speed up page number lists and dirty page lists (todo4recovery://erased_by_github/libmdbx/issues/132).
- Added `LIBMDBX_NO_EXPORTS_LEGACY_API` build option.
Fixes:
- Fixed missing cleanup (null assigned) in the C++ commit/abort (https://github.com/erthink/libmdbx/pull/143).
- Fixed missing cleanup (null assigned) in the C++ commit/abort (todo4recovery://erased_by_github/libmdbx/pull/143).
- Fixed `mdbx_realloc()` for case of nullptr and `MDBX_WITHOUT_MSVC_CRT=ON` for Windows.
- Fixed the possibility to use invalid and renewed (closed & re-opened, dropped & re-created) DBI-handles (https://github.com/erthink/libmdbx/issues/146).
- Fixed 4-byte aligned access to 64-bit integers, including access to the `bootid` meta-page's field (https://github.com/erthink/libmdbx/issues/153).
- Fixed the possibility to use invalid and renewed (closed & re-opened, dropped & re-created) DBI-handles (todo4recovery://erased_by_github/libmdbx/issues/146).
- Fixed 4-byte aligned access to 64-bit integers, including access to the `bootid` meta-page's field (todo4recovery://erased_by_github/libmdbx/issues/153).
- Fixed minor/potential memory leak during page flushing and unspilling.
- Fixed handling states of cursors's and subDBs's for nested transactions.
- Fixed page leak in extra rare case the list of retired pages changed during update GC on transaction commit.
- Fixed assertions to avoid false-positive UB detection by CLANG/LLVM (https://github.com/erthink/libmdbx/issues/153).
- Fixed `MDBX_TXN_FULL` and regressive `MDBX_KEYEXIST` during large transaction commit with `MDBX_LIFORECLAIM` (https://github.com/erthink/libmdbx/issues/123).
- Fixed assertions to avoid false-positive UB detection by CLANG/LLVM (todo4recovery://erased_by_github/libmdbx/issues/153).
- Fixed `MDBX_TXN_FULL` and regressive `MDBX_KEYEXIST` during large transaction commit with `MDBX_LIFORECLAIM` (todo4recovery://erased_by_github/libmdbx/issues/123).
- Fixed auto-recovery (`weak->steady` with the same boot-id) when Database size at last weak checkpoint is large than at last steady checkpoint.
- Fixed operation on systems with unusual small/large page size, including PowerPC (https://github.com/erthink/libmdbx/issues/157).
- Fixed operation on systems with unusual small/large page size, including PowerPC (todo4recovery://erased_by_github/libmdbx/issues/157).
## v0.9.2 at 2020-11-27
@@ -493,11 +572,11 @@ Fixes:
- Fixed copy&paste typos.
- Fixed minor false-positive GCC warning.
- Added workaround for broken `DEFINE_ENUM_FLAG_OPERATORS` from Windows SDK.
- Fixed cursor state after multimap/dupsort repeated deletes (https://github.com/erthink/libmdbx/issues/121).
- Fixed cursor state after multimap/dupsort repeated deletes (todo4recovery://erased_by_github/libmdbx/issues/121).
- Added `SIGPIPE` suppression for internal thread during `mdbx_env_copy()`.
- Fixed extra-rare `MDBX_KEY_EXIST` error during `mdbx_commit()` (https://github.com/erthink/libmdbx/issues/131).
- Fixed spilled pages checking (https://github.com/erthink/libmdbx/issues/126).
- Fixed `mdbx_load` for 'plain text' and without `-s name` cases (https://github.com/erthink/libmdbx/issues/136).
- Fixed extra-rare `MDBX_KEY_EXIST` error during `mdbx_commit()` (todo4recovery://erased_by_github/libmdbx/issues/131).
- Fixed spilled pages checking (todo4recovery://erased_by_github/libmdbx/issues/126).
- Fixed `mdbx_load` for 'plain text' and without `-s name` cases (todo4recovery://erased_by_github/libmdbx/issues/136).
- Fixed save/restore/commit of cursors for nested transactions.
- Fixed cursors state in rare/special cases (move next beyond end-of-data, after deletion and so on).
- Added workaround for MSVC 19.28 (Visual Studio 16.8) (but may still hang during compilation).
@@ -513,7 +592,7 @@ Fixes:
Added features:
- Preliminary C++ API with support for C++17 polymorphic allocators.
- [Online C++ API reference](https://erthink.github.io/libmdbx/) by Doxygen.
- [Online C++ API reference](https://erased_by_github_and_to_be_restored.todo/libmdbx/) by Doxygen.
- Quick reference for Insert/Update/Delete operations.
- Explicit `MDBX_SYNC_DURABLE` to sync modes for API clarity.
- Explicit `MDBX_ALLDUPS` and `MDBX_UPSERT` for API clarity.
@@ -558,7 +637,7 @@ Fixes:
Added features:
- [Online C API reference](https://erthink.github.io/libmdbx/) by Doxygen.
- [Online C API reference](https://erased_by_github_and_to_be_restored.todo/libmdbx/) by Doxygen.
- Separated enums for environment, sub-databases, transactions, copying and data-update flags.
Deprecated functions and flags:

View File

@@ -6,6 +6,30 @@
#
################################################################################
#
# Basic internal definitios. For a customizable variables and options see below.
#
$(info // The GNU Make $(MAKE_VERSION))
SHELL := $(shell env bash -c 'echo $$BASH')
MAKE_VERx3 := $(shell printf "%3s%3s%3s" $(subst ., ,$(MAKE_VERSION)))
make_lt_3_81 := $(shell expr "$(MAKE_VERx3)" "<" " 3 81")
ifneq ($(make_lt_3_81),0)
$(error Please use GNU Make 3.81 or above)
endif
make_ge_4_1 := $(shell expr "$(MAKE_VERx3)" ">=" " 4 1")
SRC_PROBE_C := $(shell [ -f mdbx.c ] && echo mdbx.c || echo src/osal.c)
SRC_PROBE_CXX := $(shell [ -f mdbx.c++ ] && echo mdbx.c++ || echo src/mdbx.c++)
UNAME := $(shell uname -s 2>/dev/null || echo Unknown)
define cxx_filesystem_probe
int main(int argc, const char*argv[]) {
mdbx::filesystem::path probe(argv[0]);
if (argc != 1) throw mdbx::filesystem::filesystem_error(std::string("fake"), std::error_code());
return mdbx::filesystem::is_directory(probe.relative_path());
}
endef
#
################################################################################
#
# Use `make options` to list the available libmdbx build options.
#
# Note that the defaults should already be correct for most platforms;
@@ -13,38 +37,48 @@
# in README and source code (see src/options.h) if you do.
#
SHELL := env bash
# install sandbox
DESTDIR ?=
INSTALL ?= install
# install prefixes (inside sandbox)
prefix ?= /usr/local
mandir ?= $(prefix)/man
# lib/bin suffix for multiarch/biarch, e.g. '.x86_64'
suffix ?=
INSTALL ?= install
# toolchain
CC ?= gcc
CXX ?= g++
CFLAGS_EXTRA ?=
LD ?= ld
# build options
MDBX_BUILD_OPTIONS ?=-DNDEBUG=1
MDBX_BUILD_TIMESTAMP ?=$(shell date +%Y-%m-%dT%H:%M:%S%z)
CFLAGS ?= $(strip $(eval CFLAGS := -std=gnu11 -O2 -g -Wall -Werror -Wextra -Wpedantic -ffunction-sections -fPIC -fvisibility=hidden -pthread -Wno-error=attributes $$(shell PROBE=$$$$([ -f mdbx.c ] && echo mdbx.c || echo src/osal.c); for opt in -fno-semantic-interposition -Wno-unused-command-line-argument -Wno-tautological-compare; do [ -z "$$$$($(CC) '-DMDBX_BUILD_FLAGS="probe"' $$$${opt} -c $$$${PROBE} -o /dev/null >/dev/null 2>&1 || echo failed)" ] && echo "$$$${opt} "; done)$(CFLAGS_EXTRA))$(CFLAGS))
# Choosing C++ standard with deferred simple variable expansion trick
CXXSTD ?= $(eval CXXSTD := $$(shell PROBE=$$$$([ -f mdbx.c++ ] && echo mdbx.c++ || echo src/mdbx.c++); for std in gnu++23 c++23 gnu++2b c++2b gnu++20 c++20 gnu++2a c++2a gnu++17 c++17 gnu++1z c++1z gnu++14 c++14 gnu++1y c++1y gnu+11 c++11 gnu++0x c++0x; do $(CXX) -std=$$$${std} -c $$$${PROBE} -o /dev/null 2>std-$$$${std}.err >/dev/null && echo "-std=$$$${std}" && exit; done))$(CXXSTD)
CXXFLAGS = $(strip $(CXXSTD) $(filter-out -std=gnu11,$(CFLAGS)))
# TIP: Try append '--no-as-needed,-lrt' for ability to built with modern glibc, but then use with the old.
LIBS ?= $(strip -lm $(shell uname | grep -qi SunOS && echo "-lkstat") $(shell uname | grep -qi -e Darwin -e OpenBSD || echo "-lrt") $(shell uname | grep -qi Windows && echo "-lntdll"))
# probe and compose common compiler flags with variable expansion trick (seems this work two times per session for GNU Make 3.81)
CFLAGS ?= $(strip $(eval CFLAGS := -std=gnu11 -O2 -g -Wall -Werror -Wextra -Wpedantic -ffunction-sections -fPIC -fvisibility=hidden -pthread -Wno-error=attributes $$(shell for opt in -fno-semantic-interposition -Wno-unused-command-line-argument -Wno-tautological-compare; do [ -z "$$$$($(CC) '-DMDBX_BUILD_FLAGS="probe"' $$$${opt} -c $(SRC_PROBE_C) -o /dev/null >/dev/null 2>&1 || echo failed)" ] && echo "$$$${opt} "; done)$(CFLAGS_EXTRA))$(CFLAGS))
LDFLAGS ?= $(strip $(shell $(LD) --help 2>/dev/null | grep -q -- --gc-sections && echo '-Wl,--gc-sections,-z,relro,-O1')$(shell $(LD) --help 2>/dev/null | grep -q -- -dead_strip && echo '-Wl,-dead_strip'))
EXE_LDFLAGS ?= -pthread
# choosing C++ standard with variable expansion trick (seems this work two times per session for GNU Make 3.81)
CXXSTD ?= $(eval CXXSTD := $$(shell for std in gnu++23 c++23 gnu++2b c++2b gnu++20 c++20 gnu++2a c++2a gnu++17 c++17 gnu++1z c++1z gnu++14 c++14 gnu++1y c++1y gnu+11 c++11 gnu++0x c++0x; do $(CXX) -std=$$$${std} -c $(SRC_PROBE_CXX) -o /dev/null 2>probe4std-$$$${std}.err >/dev/null && echo "-std=$$$${std}" && exit; done))$(CXXSTD)
CXXFLAGS ?= $(strip $(CXXSTD) $(filter-out -std=gnu11,$(CFLAGS)))
# libraries and options for linking
EXE_LDFLAGS ?= -pthread
ifneq ($(make_ge_4_1),1)
# don't use variable expansion trick as workaround for bugs of GNU Make before 4.1
LIBS ?= $(shell $(uname2libs))
LDFLAGS ?= $(shell $(uname2ldflags))
LIB_STDCXXFS ?= $(shell echo '$(cxx_filesystem_probe)' | cat mdbx.h++ - | sed $$'1s/\xef\xbb\xbf//' | $(CXX) -x c++ $(CXXFLAGS) -Wno-error - -Wl,--allow-multiple-definition -lstdc++fs $(LIBS) $(LDFLAGS) $(EXE_LDFLAGS) -o /dev/null 2>probe4lstdfs.err >/dev/null && echo '-Wl,--allow-multiple-definition -lstdc++fs')
else
# using variable expansion trick to avoid repeaded probes
LIBS ?= $(eval LIBS := $$(shell $$(uname2libs)))$(LIBS)
LDFLAGS ?= $(eval LDFLAGS := $$(shell $$(uname2ldflags)))$(LDFLAGS)
LIB_STDCXXFS ?= $(eval LIB_STDCXXFS := $$(shell echo '$$(cxx_filesystem_probe)' | cat mdbx.h++ - | sed $$$$'1s/\xef\xbb\xbf//' | $(CXX) -x c++ $(CXXFLAGS) -Wno-error - -Wl,--allow-multiple-definition -lstdc++fs $(LIBS) $(LDFLAGS) $(EXE_LDFLAGS) -o /dev/null 2>probe4lstdfs.err >/dev/null && echo '-Wl,--allow-multiple-definition -lstdc++fs'))$(LIB_STDCXXFS)
endif
################################################################################
UNAME := $(shell uname -s 2>/dev/null || echo Unknown)
define uname2sosuffix
case "$(UNAME)" in
Darwin*|Mach*) echo dylib;;
@@ -52,6 +86,37 @@ define uname2sosuffix
*) echo so;;
esac
endef
define uname2ldflags
case "$(UNAME)" in
CYGWIN*|MINGW*|MSYS*|Windows*)
echo '-Wl,--gc-sections,-O1';
;;
*)
$(LD) --help 2>/dev/null | grep -q -- --gc-sections && echo '-Wl,--gc-sections,-z,relro,-O1';
$(LD) --help 2>/dev/null | grep -q -- -dead_strip && echo '-Wl,-dead_strip';
;;
esac
endef
# TIP: try add the'-Wl, --no-as-needed,-lrt' for ability to built with modern glibc, but then use with the old.
define uname2libs
case "$(UNAME)" in
CYGWIN*|MINGW*|MSYS*|Windows*)
echo '-lm -lntdll -lwinmm';
;;
*SunOS*|*Solaris*)
echo '-lm -lkstat -lrt';
;;
*Darwin*|OpenBSD*)
echo '-lm';
;;
*)
echo '-lm -lrt';
;;
esac
endef
SO_SUFFIX := $(shell $(uname2sosuffix))
HEADERS := mdbx.h mdbx.h++
LIBRARIES := libmdbx.a libmdbx.$(SO_SUFFIX)
@@ -59,8 +124,8 @@ TOOLS := mdbx_stat mdbx_copy mdbx_dump mdbx_load mdbx_chk mdbx_drop
MANPAGES := mdbx_stat.1 mdbx_copy.1 mdbx_dump.1 mdbx_load.1 mdbx_chk.1 mdbx_drop.1
TIP := // TIP:
.PHONY: all help options lib tools clean install uninstall check_buildflags_tag
.PHONY: install-strip install-no-strip strip libmdbx mdbx show-options
.PHONY: all help options lib libs tools clean install uninstall check_buildflags_tag tools-static
.PHONY: install-strip install-no-strip strip libmdbx mdbx show-options lib-static lib-shared
ifeq ("$(origin V)", "command line")
MDBX_BUILD_VERBOSE := $(V)
@@ -85,8 +150,9 @@ help:
@echo " make all - build libraries and tools"
@echo " make help - print this help"
@echo " make options - list build options"
@echo " make lib - build libraries"
@echo " make tools - built tools"
@echo " make lib - build libraries, also lib-static and lib-shared"
@echo " make tools - build the tools"
@echo " make tools-static - build the tools with statically linking with system libraries and compiler runtime"
@echo " make clean "
@echo " make install "
@echo " make uninstall "
@@ -132,7 +198,7 @@ show-options:
@echo " CC =`which $(CC)` | `$(CC) --version | head -1`"
@echo " CFLAGS =$(CFLAGS)"
@echo " CXXFLAGS =$(CXXFLAGS)"
@echo " LDFLAGS =$(LDFLAGS) $(LIBS) $(EXE_LDFLAGS)"
@echo " LDFLAGS =$(LDFLAGS) $(LIB_STDCXXFS) $(LIBS) $(EXE_LDFLAGS)"
@echo '$(TIP) Use `make help` to listing available targets.'
options:
@@ -172,9 +238,10 @@ else
endif
#< dist-cutoff-end
lib libmdbx mdbx: libmdbx.a libmdbx.$(SO_SUFFIX)
lib libs libmdbx mdbx: libmdbx.a libmdbx.$(SO_SUFFIX)
tools: $(TOOLS)
tools-static: $(addsuffix .static,$(TOOLS)) $(addsuffix .static-lto,$(TOOLS))
strip: all
@echo ' STRIP libmdbx.$(SO_SUFFIX) $(TOOLS)'
@@ -184,7 +251,8 @@ clean:
@echo ' REMOVE ...'
$(QUIET)rm -rf $(TOOLS) mdbx_test @* *.[ao] *.[ls]o *.$(SO_SUFFIX) *.dSYM *~ tmp.db/* \
*.gcov *.log *.err src/*.o test/*.o mdbx_example dist \
config.h src/config.h src/version.c *.tar* buildflags.tag
config.h src/config.h src/version.c *.tar* buildflags.tag \
mdbx_*.static mdbx_*.static-lto
MDBX_BUILD_FLAGS =$(strip $(MDBX_BUILD_OPTIONS) $(CXXSTD) $(CFLAGS) $(LDFLAGS) $(LIBS))
check_buildflags_tag:
@@ -196,13 +264,13 @@ check_buildflags_tag:
buildflags.tag: check_buildflags_tag
libmdbx.a: mdbx-static.o mdbx++-static.o
lib-static libmdbx.a: mdbx-static.o mdbx++-static.o
@echo ' AR $@'
$(QUIET)$(AR) rcs $@ $? $(HUSH)
libmdbx.$(SO_SUFFIX): mdbx-dylib.o mdbx++-dylib.o
lib-shared libmdbx.$(SO_SUFFIX): mdbx-dylib.o mdbx++-dylib.o
@echo ' LD $@'
$(QUIET)$(CXX) $(CXXFLAGS) $^ -pthread -shared $(LDFLAGS) $(LIBS) -o $@
$(QUIET)$(CXX) $(CXXFLAGS) $^ -pthread -shared $(LDFLAGS) $(LIB_STDCXXFS) $(LIBS) -o $@
#> dist-cutoff-begin
ifeq ($(wildcard mdbx.c),mdbx.c)
@@ -236,10 +304,19 @@ mdbx++-static.o: config.h mdbx.c++ mdbx.h mdbx.h++ $(lastword $(MAKEFILE_LIST))
@echo ' CC $@'
$(QUIET)$(CXX) $(CXXFLAGS) $(MDBX_BUILD_OPTIONS) '-DMDBX_CONFIG_H="config.h"' -ULIBMDBX_EXPORTS -c mdbx.c++ -o $@
mdbx_%: mdbx_%.c libmdbx.a
mdbx_%: mdbx_%.c mdbx-static.o
@echo ' CC+LD $@'
$(QUIET)$(CC) $(CFLAGS) $(MDBX_BUILD_OPTIONS) '-DMDBX_CONFIG_H="config.h"' $^ $(EXE_LDFLAGS) $(LIBS) -o $@
mdbx_%.static: mdbx_%.c mdbx-static.o
@echo ' CC+LD $@'
$(QUIET)$(CC) $(CFLAGS) $(MDBX_BUILD_OPTIONS) '-DMDBX_CONFIG_H="config.h"' $^ $(EXE_LDFLAGS) -static -Wl,--strip-all -o $@
mdbx_%.static-lto: mdbx_%.c config.h mdbx.c mdbx.h
@echo ' CC+LD $@'
$(QUIET)$(CC) $(CFLAGS) -Os -flto $(MDBX_BUILD_OPTIONS) '-DLIBMDBX_API=' '-DMDBX_CONFIG_H="config.h"' \
$< mdbx.c $(EXE_LDFLAGS) $(LIBS) -static -Wl,--strip-all -o $@
#> dist-cutoff-begin
else
################################################################################
@@ -248,6 +325,7 @@ else
.PHONY: build-test build-test-with-valgrind check cross-gcc cross-qemu dist doxygen gcc-analyzer long-test
.PHONY: reformat release-assets tags smoke test test-asan smoke-fault test-leak
.PHONY: smoke-singleprocess test-singleprocess test-ubsan test-valgrind memcheck
.PHONY: smoke-assertion test-assertion long-test-assertion
define uname2osal
case "$(UNAME)" in
@@ -289,10 +367,10 @@ reformat:
MAN_SRCDIR := src/man1/
ALLOY_DEPS := $(shell git ls-files src/)
git_DIR := $(shell if [ -d .git ]; then echo .git; elif [ -s .git -a -f .git ]; then grep '^gitdir: ' .git | cut -d ':' -f 2; else echo git_directory_is_absent; fi)
MDBX_GIT_VERSION = $(shell set -o pipefail; git describe --tags 2>&- | sed -n 's|^v*\([0-9]\{1,\}\.[0-9]\{1,\}\.[0-9]\{1,\}\)\(.*\)|\1|p' || echo 'Please fetch tags and/or use non-obsolete git version')
MDBX_GIT_VERSION = $(shell set -o pipefail; git describe --tags '--match=v[0-9]*' 2>&- | sed -n 's|^v*\([0-9]\{1,\}\.[0-9]\{1,\}\.[0-9]\{1,\}\)\(.*\)|\1|p' || echo 'Please fetch tags and/or use non-obsolete git version')
MDBX_GIT_REVISION = $(shell set -o pipefail; git rev-list `git describe --tags --abbrev=0`..HEAD --count 2>&- || echo 'Please fetch tags and/or use non-obsolete git version')
MDBX_GIT_TIMESTAMP = $(shell git show --no-patch --format=%cI HEAD 2>&- || echo 'Please install latest get version')
MDBX_GIT_DESCRIBE = $(shell git describe --tags --long --dirty=-dirty 2>&- || echo 'Please fetch tags and/or install non-obsolete git version')
MDBX_GIT_DESCRIBE = $(shell git describe --tags --long --dirty=-dirty '--match=v[0-9]*' 2>&- || echo 'Please fetch tags and/or install non-obsolete git version')
MDBX_VERSION_SUFFIX = $(shell set -o pipefail; echo -n '$(MDBX_GIT_DESCRIBE)' | tr -c -s '[a-zA-Z0-9]' _)
MDBX_BUILD_SOURCERY = $(shell set -o pipefail; $(MAKE) IOARENA=false CXXSTD= -s src/version.c >/dev/null && (openssl dgst -r -sha256 src/version.c || sha256sum src/version.c || shasum -a 256 src/version.c) 2>/dev/null | cut -d ' ' -f 1 || (echo 'Please install openssl or sha256sum or shasum' >&2 && echo sha256sum_is_no_available))_$(MDBX_VERSION_SUFFIX)
MDBX_DIST_DIR = libmdbx-$(MDBX_VERSION_SUFFIX)
@@ -303,11 +381,18 @@ MDBX_SMOKE_EXTRA ?=
check: DESTDIR = $(shell pwd)/@check-install
check: test dist install
smoke-assertion: MDBX_BUILD_OPTIONS=-DMDBX_FORCE_ASSERTIONS=1
smoke-assertion: smoke
test-assertion: MDBX_BUILD_OPTIONS=-DMDBX_FORCE_ASSERTIONS=1
test-assertion: smoke
long-test-assertion: MDBX_BUILD_OPTIONS=-DMDBX_FORCE_ASSERTIONS=1
long-test-assertion: smoke
smoke: build-test
@echo ' SMOKE `mdbx_test basic`...'
$(QUIET)rm -f $(TEST_DB) $(TEST_LOG).gz && (set -o pipefail; \
(./mdbx_test --table=+data.integer --keygen.split=29 --datalen.min=min --datalen.max=max --progress --console=no --repeat=$(TEST_ITER) --pathname=$(TEST_DB) --dont-cleanup-after $(MDBX_SMOKE_EXTRA) basic && \
./mdbx_test --mode=-writemap,-nosync-safe,-lifo --progress --console=no --repeat=12 --pathname=$(TEST_DB) --dont-cleanup-after $(MDBX_SMOKE_EXTRA) basic) \
./mdbx_test --mode=-writemap,-nosync-safe,-lifo --progress --console=no --repeat=$(TEST_ITER) --pathname=$(TEST_DB) --dont-cleanup-after $(MDBX_SMOKE_EXTRA) basic) \
| tee >(gzip --stdout >$(TEST_LOG).gz) | tail -n 42) \
&& ./mdbx_chk -vvn $(TEST_DB) && ./mdbx_chk -vvn $(TEST_DB)-copy
@@ -390,6 +475,15 @@ mdbx_%: src/mdbx_%.c libmdbx.a
@echo ' CC+LD $@'
$(QUIET)$(CC) $(CFLAGS) $(MDBX_BUILD_OPTIONS) '-DMDBX_CONFIG_H="config.h"' $^ $(EXE_LDFLAGS) $(LIBS) -o $@
mdbx_%.static: src/mdbx_%.c mdbx-static.o
@echo ' CC+LD $@'
$(QUIET)$(CC) $(CFLAGS) $(MDBX_BUILD_OPTIONS) '-DMDBX_CONFIG_H="config.h"' $^ $(EXE_LDFLAGS) $(LIBS) -static -Wl,--strip-all -o $@
mdbx_%.static-lto: src/mdbx_%.c src/config.h src/version.c src/alloy.c $(ALLOY_DEPS)
@echo ' CC+LD $@'
$(QUIET)$(CC) $(CFLAGS) -Os -flto $(MDBX_BUILD_OPTIONS) '-DLIBMDBX_API=' '-DMDBX_CONFIG_H="config.h"' \
$< src/alloy.c $(EXE_LDFLAGS) $(LIBS) -static -Wl,--strip-all -o $@
mdbx_test: $(TEST_OBJ) libmdbx.$(SO_SUFFIX)
@echo ' LD $@'
$(QUIET)$(CXX) $(CXXFLAGS) $(TEST_OBJ) -Wl,-rpath . -L . -l mdbx $(EXE_LDFLAGS) $(LIBS) -o $@

View File

@@ -3,9 +3,11 @@
all help options \
clean install install-no-strip install-strip strip tools uninstall \
bench bench-clean bench-couple bench-quartet bench-triplet re-bench \
lib libmdbx mdbx mdbx_chk mdbx_copy mdbx_drop mdbx_dump mdbx_load mdbx_stat \
lib libs lib-static lib-shared tools-static \
libmdbx mdbx mdbx_chk mdbx_copy mdbx_drop mdbx_dump mdbx_load mdbx_stat \
check dist memcheck cross-gcc cross-qemu doxygen gcc-analyzer reformat \
release-assets tags test build-test mdbx_test smoke smoke-fault smoke-singleprocess \
smoke-assertion test-assertion long-test-assertion \
test-asan test-leak test-singleprocess test-ubsan test-valgrind:
@CC=$(CC) \
CXX=`if test -n "$(CXX)" && which "$(CXX)" > /dev/null; then echo "$(CXX)"; elif test -n "$(CCC)" && which "$(CCC)" > /dev/null; then echo "$(CCC)"; else echo "c++"; fi` \

192
README.md
View File

@@ -1,35 +1,46 @@
<!-- Required extensions: pymdownx.betterem, pymdownx.tilde, pymdownx.emoji, pymdownx.tasklist, pymdownx.superfences -->
### The origin has been migrated to [GitFlic](https://gitflic.ru/project/erthink/libmdbx)
since on 2022-04-15 the Github administration, without any warning
nor explanation, deleted _libmdbx_ along with a lot of other projects,
simultaneously blocking access for many developers.
For the same reason ~~Github~~ is blacklisted forever.
GitFlic's developers plan to support other languages,
including English 和 中文, in the near future.
### Основной репозиторий перемещен на [GitFlic](https://gitflic.ru/project/erthink/libmdbx)
так как 15 апреля 2022 администрация Github без предупреждения и
объяснения причин удалила _libmdbx_ вместе с массой других проектов,
одновременно заблокировав доступ многим разработчикам.
По этой же причине ~~Github~~ навсегда занесен в черный список.
--------------------------------------------------------------------------------
*The Future will (be) [Positive](https://www.ptsecurity.com). Всё будет хорошо.*
> Please refer to the online [documentation](https://erthink.github.io/libmdbx/)
> with [`C` API description](https://erthink.github.io/libmdbx/group__c__api.html)
> and pay attention to the [`C++` API](https://github.com/erthink/libmdbx/blob/devel/mdbx.h%2B%2B).
> Please refer to the online [documentation](https://libmdbx.website.yandexcloud.net)
> with [`C` API description](https://libmdbx.website.yandexcloud.net/group__c__api.html)
> and pay attention to the [`C++` API](https://gitflic.ru/project/erthink/libmdbx/blob?file=mdbx.h%2B%2B#line-num-1).
> Questions, feedback and suggestions are welcome to the [Telegram' group](https://t.me/libmdbx).
> For NEWS take a look to the [ChangeLog](./ChangeLog.md).
> For NEWS take a look to the [ChangeLog](https://gitflic.ru/project/erthink/libmdbx/blob?file=ChangeLog.md).
[![https://t.me/libmdbx](https://raw.githubusercontent.com/wiki/erthink/libmdbx/img/telegram.png)](https://t.me/libmdbx)
[![Version](https://badge.fury.io/gh/erthink%2Flibmdbx.svg)](https://github.com/erthink/libmdbx/releases)
[![GithubCI](https://github.com/erthink/libmdbx/workflows/CI/badge.svg)](https://github.com/erthink/libmdbx/actions?query=workflow%3ACI)
[![AppveyorCI](https://ci.appveyor.com/api/projects/status/ue94mlopn50dqiqg/branch/master?svg=true)](https://ci.appveyor.com/project/leo-yuriev/libmdbx/branch/master)
[![CircleCI](https://circleci.com/gh/erthink/libmdbx/tree/master.svg?style=svg)](https://circleci.com/gh/erthink/libmdbx/tree/master)
[![CirrusCI](https://api.cirrus-ci.com/github/erthink/libmdbx.svg)](https://cirrus-ci.com/github/erthink/libmdbx)
[![Coverity Scan Status](https://scan.coverity.com/projects/12915/badge.svg)](https://scan.coverity.com/projects/reopen-libmdbx)
libmdbx
========
<!-- section-begin overview -->
_libmdbx_ is an extremely fast, compact, powerful, embedded,
transactional [key-value database](https://en.wikipedia.org/wiki/Key-value_database),
with [permissive license](./LICENSE).
_libmdbx_ is an extremely fast, compact, powerful, embedded, transactional
[key-value database](https://en.wikipedia.org/wiki/Key-value_database),
with [permissive license](https://gitflic.ru/project/erthink/libmdbx/blob?file=LICENSE).
_libmdbx_ has a specific set of properties and capabilities,
focused on creating unique lightweight solutions.
1. Allows **a swarm of multi-threaded processes to
[ACID]((https://en.wikipedia.org/wiki/ACID))ly read and update** several
[ACID](https://en.wikipedia.org/wiki/ACID)ly read and update** several
key-value [maps](https://en.wikipedia.org/wiki/Associative_array) and
[multimaps](https://en.wikipedia.org/wiki/Multimap) in a locally-shared
database.
@@ -62,6 +73,7 @@ neglected in favour of write performance.
7. Supports Linux, Windows, MacOS, Android, iOS, FreeBSD, DragonFly, Solaris,
OpenSolaris, OpenIndiana, NetBSD, OpenBSD and other systems compliant with
**POSIX.1-2008**.
<!-- section-end -->
Historically, _libmdbx_ is a deeply revised and extended descendant of the amazing
@@ -69,16 +81,19 @@ Historically, _libmdbx_ is a deeply revised and extended descendant of the amazi
_libmdbx_ inherits all benefits from _LMDB_, but resolves some issues and adds [a set of improvements](#improvements-beyond-lmdb).
<!-- section-begin mithril -->
The next version is under active non-public development from scratch and will be
released as _**MithrilDB**_ and `libmithrildb` for libraries & packages.
released as **MithrilDB** and `libmithrildb` for libraries & packages.
Admittedly mythical [Mithril](https://en.wikipedia.org/wiki/Mithril) is
resembling silver but being stronger and lighter than steel. Therefore
_MithrilDB_ is a rightly relevant name.
> _MithrilDB_ will be radically different from _libmdbx_ by the new
> database format and API based on C++17, as well as the [Apache 2.0
> License](https://www.apache.org/licenses/LICENSE-2.0). The goal of this
> revolution is to provide a clearer and robust API, add more features and
> new valuable properties of the database.
> _MithrilDB_ will be radically different from _libmdbx_ by the new
> database format and API based on C++17, as well as the [Apache 2.0
> License](https://www.apache.org/licenses/LICENSE-2.0). The goal of this
> revolution is to provide a clearer and robust API, add more features and
> new valuable properties of the database.
<!-- section-end -->
-----
@@ -188,6 +203,7 @@ databases"](https://github.com/coreos/bbolt#comparison-with-other-databases)
which is also (mostly) applicable to _libmdbx_.
<!-- section-end -->
<!-- section-begin improvements -->
Improvements beyond LMDB
@@ -203,45 +219,52 @@ the user's point of view.
## Added Features
1. Keys could be more than 2 times longer than _LMDB_.
> For DB with default page size _libmdbx_ support keys up to 2022 bytes
> and up to 32742 bytes for 64K page size. _LMDB_ allows key size up to
> 511 bytes and may silently loses data with large values.
> For DB with default page size _libmdbx_ support keys up to 2022 bytes
> and up to 32742 bytes for 64K page size. _LMDB_ allows key size up to
> 511 bytes and may silently loses data with large values.
2. Up to 30% faster than _LMDB_ in [CRUD](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete) benchmarks.
> Benchmarks of the in-[tmpfs](https://en.wikipedia.org/wiki/Tmpfs) scenarios,
> that tests the speed of the engine itself, showned that _libmdbx_ 10-20% faster than _LMDB_,
> and up to 30% faster when _libmdbx_ compiled with specific build options
> which downgrades several runtime checks to be match with LMDB behaviour.
>
> These and other results could be easily reproduced with [ioArena](https://github.com/pmwkaa/ioarena) just by `make bench-quartet` command,
> including comparisons with [RockDB](https://en.wikipedia.org/wiki/RocksDB)
> and [WiredTiger](https://en.wikipedia.org/wiki/WiredTiger).
> Benchmarks of the in-[tmpfs](https://en.wikipedia.org/wiki/Tmpfs) scenarios,
> that tests the speed of the engine itself, showned that _libmdbx_ 10-20% faster than _LMDB_,
> and up to 30% faster when _libmdbx_ compiled with specific build options
> which downgrades several runtime checks to be match with LMDB behaviour.
>
> These and other results could be easily reproduced with [ioArena](https://github.com/pmwkaa/ioarena) just by `make bench-quartet` command,
> including comparisons with [RockDB](https://en.wikipedia.org/wiki/RocksDB)
> and [WiredTiger](https://en.wikipedia.org/wiki/WiredTiger).
3. Automatic on-the-fly database size adjustment, both increment and reduction.
> _libmdbx_ manages the database size according to parameters specified
> by `mdbx_env_set_geometry()` function,
> ones include the growth step and the truncation threshold.
>
> Unfortunately, on-the-fly database size adjustment doesn't work under [Wine](https://en.wikipedia.org/wiki/Wine_(software))
> due to its internal limitations and unimplemented functions, i.e. the `MDBX_UNABLE_EXTEND_MAPSIZE` error will be returned.
> _libmdbx_ manages the database size according to parameters specified
> by `mdbx_env_set_geometry()` function,
> ones include the growth step and the truncation threshold.
>
> Unfortunately, on-the-fly database size adjustment doesn't work under [Wine](https://en.wikipedia.org/wiki/Wine_(software))
> due to its internal limitations and unimplemented functions, i.e. the `MDBX_UNABLE_EXTEND_MAPSIZE` error will be returned.
4. Automatic continuous zero-overhead database compactification.
> During each commit _libmdbx_ merges a freeing pages which adjacent with the unallocated area
> at the end of file, and then truncates unused space when a lot enough of.
> During each commit _libmdbx_ merges a freeing pages which adjacent with the unallocated area
> at the end of file, and then truncates unused space when a lot enough of.
5. The same database format for 32- and 64-bit builds.
> _libmdbx_ database format depends only on the [endianness](https://en.wikipedia.org/wiki/Endianness) but not on the [bitness](https://en.wiktionary.org/wiki/bitness).
> _libmdbx_ database format depends only on the [endianness](https://en.wikipedia.org/wiki/Endianness) but not on the [bitness](https://en.wiktionary.org/wiki/bitness).
6. LIFO policy for Garbage Collection recycling. This can significantly increase write performance due write-back disk cache up to several times in a best case scenario.
> LIFO means that for reuse will be taken the latest becomes unused pages.
> Therefore the loop of database pages circulation becomes as short as possible.
> In other words, the set of pages, that are (over)written in memory and on disk during a series of write transactions, will be as small as possible.
> Thus creates ideal conditions for the battery-backed or flash-backed disk cache efficiency.
> LIFO means that for reuse will be taken the latest becomes unused pages.
> Therefore the loop of database pages circulation becomes as short as possible.
> In other words, the set of pages, that are (over)written in memory and on disk during a series of write transactions, will be as small as possible.
> Thus creates ideal conditions for the battery-backed or flash-backed disk cache efficiency.
7. Fast estimation of range query result volume, i.e. how many items can
be found between a `KEY1` and a `KEY2`. This is a prerequisite for build
and/or optimize query execution plans.
> _libmdbx_ performs a rough estimate based on common B-tree pages of the paths from root to corresponding keys.
> _libmdbx_ performs a rough estimate based on common B-tree pages of the paths from root to corresponding keys.
8. `mdbx_chk` utility for database integrity check.
Since version 0.9.1, the utility supports checking the database using any of the three meta pages and the ability to switch to it.
@@ -254,12 +277,14 @@ Since version 0.9.1, the utility supports checking the database using any of the
or not, that allows to avoid copy-out before updates.
12. Extended information of whole-database, sub-databases, transactions, readers enumeration.
> _libmdbx_ provides a lot of information, including dirty and leftover pages
> for a write transaction, reading lag and holdover space for read transactions.
> _libmdbx_ provides a lot of information, including dirty and leftover pages
> for a write transaction, reading lag and holdover space for read transactions.
13. Extended update and delete operations.
> _libmdbx_ allows one _at once_ with getting previous value
> and addressing the particular item from multi-value with the same key.
> _libmdbx_ allows one _at once_ with getting previous value
> and addressing the particular item from multi-value with the same key.
14. Useful runtime options for tuning engine to application's requirements and use cases specific.
@@ -289,9 +314,10 @@ pre-opening is not needed.
4. Returning `MDBX_EMULTIVAL` error in case of ambiguous update or delete.
5. Guarantee of database integrity even in asynchronous unordered write-to-disk mode.
> _libmdbx_ propose additional trade-off by `MDBX_SAFE_NOSYNC` with append-like manner for updates,
> that avoids database corruption after a system crash contrary to LMDB.
> Nevertheless, the `MDBX_UTTERLY_NOSYNC` mode is available to match LMDB's behaviour for `MDB_NOSYNC`.
> _libmdbx_ propose additional trade-off by `MDBX_SAFE_NOSYNC` with append-like manner for updates,
> that avoids database corruption after a system crash contrary to LMDB.
> Nevertheless, the `MDBX_UTTERLY_NOSYNC` mode is available to match LMDB's behaviour for `MDB_NOSYNC`.
6. On **MacOS & iOS** the `fcntl(F_FULLFSYNC)` syscall is used _by
default_ to synchronize data with the disk, as this is [the only way to
@@ -318,14 +344,22 @@ named mutexes are used.
Historically, _libmdbx_ is a deeply revised and extended descendant of the
[Lightning Memory-Mapped Database](https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database).
At first the development was carried out within the
[ReOpenLDAP](https://github.com/erthink/ReOpenLDAP) project. About a
[ReOpenLDAP](todo4recovery://erased_by_github/ReOpenLDAP) project. About a
year later _libmdbx_ was separated into a standalone project, which was
[presented at Highload++ 2015
conference](http://www.highload.ru/2015/abstracts/1831.html).
Since 2017 _libmdbx_ is used in [Fast Positive Tables](https://github.com/erthink/libfpta),
Since 2017 _libmdbx_ is used in [Fast Positive Tables](https://gitflic.ru/project/erthink/libfpta),
and development is funded by [Positive Technologies](https://www.ptsecurity.com).
On 2022-04-15 the Github administration, without any warning nor
explanation, deleted _libmdbx_ along with a lot of other projects,
simultaneously blocking access for many developers. Therefore on
2022-04-21 we have migrated to a reliable trusted infrastructure.
The origin for now is at [GitFlic](https://gitflic.ru/project/erthink/libmdbx)
with backup at [ABF by ROSA Лаб](https://abf.io/erthink/libmdbx).
For the same reason ~~Github~~ is blacklisted forever.
## Acknowledgments
Howard Chu <hyc@openldap.org> is the author of LMDB, from which
originated the _libmdbx_ in 2015.
@@ -341,29 +375,25 @@ Usage
=====
<!-- section-begin usage -->
Currently, libmdbx is only available in a
[source code](https://en.wikipedia.org/wiki/Source_code) form.
Packages support for common Linux distributions is planned in the future,
since release the version 1.0.
## Never use tarballs nor zips automatically provided by Github !
Please don't use tarballs nor zips which are automatically provided by Github.
These archives do not contain version information and thus are unfit to build _libmdbx_.
Instead of ones just clone the git repository, either download a tarball or zip with the properly amalgamated source core.
Moreover, please vote for [ability of disabling auto-creation such unsuitable archives](https://github.community/t/disable-tarball).
## Source code embedding
_libmdbx_ provides two official ways for integration in source code form:
1. Using the amalgamated source code.
> The amalgamated source code includes all files required to build and
> use _libmdbx_, but not for testing _libmdbx_ itself.
> The amalgamated source code includes all files required to build and
> use _libmdbx_, but not for testing _libmdbx_ itself.
2. Adding the complete original source code as a `git submodule`.
> This allows you to build as _libmdbx_ and testing tool.
> On the other hand, this way requires you to pull git tags, and use C++11 compiler for test tool.
> This allows you to build as _libmdbx_ and testing tool.
> On the other hand, this way requires you to pull git tags, and use C++11 compiler for test tool.
_**Please, avoid using any other techniques.**_ Otherwise, at least
don't ask for support and don't name such chimeras `libmdbx`.
@@ -390,8 +420,7 @@ and build options respectively.
The only significant specificity is that git' tags are required
to build from complete (not amalgamated) source codes.
Executing **`git fetch --tags --force --prune`** is enough to get ones,
or `git fetch --unshallow --tags --prune --force` after the Github's
[`actions/checkout@v2`](https://github.com/actions/checkout) either set **`fetch-depth: 0`** for it.
and `--unshallow` or `--update-shallow` is required for shallow cloned case.
So just using CMake or GNU Make in your habitual manner and feel free to
fill an issue or make pull request in the case something will be
@@ -399,10 +428,10 @@ unexpected or broken down.
### Testing
The amalgamated source code does not contain any tests for or several reasons.
Please read [the explanation](https://github.com/erthink/libmdbx/issues/214#issuecomment-870717981) and don't ask to alter this.
Please read [the explanation](todo4recovery://erased_by_github/libmdbx/issues/214#issuecomment-870717981) and don't ask to alter this.
So for testing _libmdbx_ itself you need a full source code, i.e. the clone of a git repository, there is no option.
The full source code of _libmdbx_ has a [`test` subdirectory](https://github.com/erthink/libmdbx/tree/master/test) with minimalistic test "framework".
The full source code of _libmdbx_ has a [`test` subdirectory](https://gitflic.ru/project/erthink/libmdbx/tree/master/test) with minimalistic test "framework".
Actually yonder is a source code of the `mdbx_test` console utility which has a set of command-line options that allow construct and run a reasonable enough test scenarios.
This test utility is intended for _libmdbx_'s developers for testing library itself, but not for use by users.
Therefore, only basic information is provided:
@@ -413,7 +442,7 @@ Therefore, only basic information is provided:
- The `Makefile` provide several self-described targets for testing: `smoke`, `test`, `check`, `memcheck`, `test-valgrind`,
`test-asan`, `test-leak`, `test-ubsan`, `cross-gcc`, `cross-qemu`, `gcc-analyzer`, `smoke-fault`, `smoke-singleprocess`,
`test-singleprocess`, 'long-test'. Please run `make --help` if doubt.
- In addition to the `mdbx_test` utility, there is the script [`long_stochastic.sh`](https://github.com/erthink/libmdbx/blob/master/test/long_stochastic.sh),
- In addition to the `mdbx_test` utility, there is the script [`long_stochastic.sh`](https://gitflic.ru/project/erthink/libmdbx/blob/master/test/long_stochastic.sh),
which calls `mdbx_test` by going through set of modes and options, with gradually increasing the number of operations and the size of transactions.
This script is used for mostly of all automatic testing, including `Makefile` targets and Continuous Integration.
- Brief information of available command-line options is available by `--help`.
@@ -561,7 +590,7 @@ from the [ios-cmake](https://github.com/leetal/ios-cmake) project.
## API description
Please refer to the online [_libmdbx_ API reference](https://erthink.github.io/libmdbx/)
Please refer to the online [_libmdbx_ API reference](https://libmdbx.website.yandexcloud.net/docs)
and/or see the [mdbx.h++](mdbx.h%2B%2B) and [mdbx.h](mdbx.h) headers.
<!-- section-begin bindings -->
@@ -571,6 +600,7 @@ Bindings
| Runtime | Repo | Author |
| ------- | ------ | ------ |
| Scala | [mdbx4s](https://github.com/david-bouyssie/mdbx4s) | [David Bouyssié](https://github.com/david-bouyssie) |
| Haskell | [libmdbx-hs](https://hackage.haskell.org/package/libmdbx) | [Francisco Vallarino](https://github.com/fjvallarino) |
| NodeJS, [Deno](https://deno.land/) | [lmdbx-js](https://github.com/kriszyp/lmdbx-js) | [Kris Zyp](https://github.com/kriszyp/)
| NodeJS | [node-mdbx](https://www.npmjs.com/package/node-mdbx/) | [Сергей Федотов](mailto:sergey.fedotov@corp.mail.ru) |
@@ -580,7 +610,7 @@ Bindings
| Rust | [libmdbx-rs](https://github.com/vorot93/libmdbx-rs) | [Artem Vorotnikov](https://github.com/vorot93) |
| Rust | [mdbx](https://crates.io/crates/mdbx) | [gcxfd](https://github.com/gcxfd) |
| Java | [mdbxjni](https://github.com/castortech/mdbxjni) | [Castor Technologies](https://castortech.com/) |
| Python (draft) | [python-bindings](https://github.com/erthink/libmdbx/commits/python-bindings) branch | [Noel Kuntze](https://github.com/Thermi)
| Python (draft) | [python-bindings](todo4recovery://erased_by_github/libmdbx/commits/python-bindings) branch | [Noel Kuntze](https://github.com/Thermi)
| .NET (obsolete) | [mdbx.NET](https://github.com/wangjia184/mdbx.NET) | [Jerry Wang](https://github.com/wangjia184) |
<!-- section-end -->
@@ -619,7 +649,7 @@ Here showed sum of performance metrics in 3 benchmarks:
2. Performance gap is too high to compare in any meaningful way.
![Comparison #1: Integral Performance](https://raw.githubusercontent.com/wiki/erthink/libmdbx/img/perf-slide-1.png)
![Comparison #1: Integral Performance](https://libmdbx.website.yandexcloud.net/img/perf-slide-1.png)
--------------------------------------------------------------------------------
@@ -628,7 +658,7 @@ Here showed sum of performance metrics in 3 benchmarks:
Summary performance with concurrent read/search queries in 1-2-4-8
threads on the machine with 4 logical CPUs in HyperThreading mode (i.e. actually 2 physical CPU cores).
![Comparison #2: Read Scalability](https://raw.githubusercontent.com/wiki/erthink/libmdbx/img/perf-slide-2.png)
![Comparison #2: Read Scalability](https://libmdbx.website.yandexcloud.net/img/perf-slide-2.png)
--------------------------------------------------------------------------------
@@ -650,7 +680,7 @@ In the benchmark each transaction contains combined CRUD operations (2
inserts, 1 read, 1 update, 1 delete). Benchmark starts on an empty database
and after full run the database contains 10,000 small key-value records.
![Comparison #3: Sync-write mode](https://raw.githubusercontent.com/wiki/erthink/libmdbx/img/perf-slide-3.png)
![Comparison #3: Sync-write mode](https://libmdbx.website.yandexcloud.net/img/perf-slide-3.png)
--------------------------------------------------------------------------------
@@ -677,7 +707,7 @@ and after full run the database contains 100,000 small key-value
records.
![Comparison #4: Lazy-write mode](https://raw.githubusercontent.com/wiki/erthink/libmdbx/img/perf-slide-4.png)
![Comparison #4: Lazy-write mode](https://libmdbx.website.yandexcloud.net/img/perf-slide-4.png)
--------------------------------------------------------------------------------
@@ -701,7 +731,7 @@ In the benchmark each transaction contains combined CRUD operations (2
inserts, 1 read, 1 update, 1 delete). Benchmark starts on an empty database
and after full run the database contains 10,000 small key-value records.
![Comparison #5: Async-write mode](https://raw.githubusercontent.com/wiki/erthink/libmdbx/img/perf-slide-5.png)
![Comparison #5: Async-write mode](https://libmdbx.website.yandexcloud.net/img/perf-slide-5.png)
--------------------------------------------------------------------------------
@@ -725,10 +755,6 @@ All benchmark data is gathered by
[getrusage()](http://man7.org/linux/man-pages/man2/getrusage.2.html)
syscall and by scanning the data directory.
![Comparison #6: Cost comparison](https://raw.githubusercontent.com/wiki/erthink/libmdbx/img/perf-slide-6.png)
![Comparison #6: Cost comparison](https://libmdbx.website.yandexcloud.net/img/perf-slide-6.png)
<!-- section-end -->
--------------------------------------------------------------------------------
#### This is a mirror of the origin repository that was moved to [abf.io](https://abf.io/erthink/) because of discriminatory restrictions for Russian Crimea.

View File

@@ -1,113 +0,0 @@
version: 0.11.5.{build}
environment:
matrix:
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015
CMAKE_GENERATOR: Visual Studio 14 2015
TOOLSET: 140
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
CMAKE_GENERATOR: Visual Studio 16 2019
TOOLSET: 142
MDBX_BUILD_SHARED_LIBRARY: OFF
MDBX_WITHOUT_MSVC_CRT: OFF
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
CMAKE_GENERATOR: Visual Studio 16 2019
TOOLSET: 142
MDBX_BUILD_SHARED_LIBRARY: ON
MDBX_WITHOUT_MSVC_CRT: ON
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
CMAKE_GENERATOR: Visual Studio 16 2019
TOOLSET: 142
MDBX_BUILD_SHARED_LIBRARY: OFF
MDBX_WITHOUT_MSVC_CRT: ON
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
CMAKE_GENERATOR: Visual Studio 16 2019
TOOLSET: 142
MDBX_BUILD_SHARED_LIBRARY: ON
MDBX_WITHOUT_MSVC_CRT: OFF
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
CMAKE_GENERATOR: Visual Studio 15 2017
TOOLSET: 141
branches:
except:
- coverity_scan
configuration:
- Debug
# MSVC-2019 hangs during code generation/optimization due to its own internal errors.
# I have found out that the problem occurs because of the /Ob2 option (see https://github.com/erthink/libmdbx/issues/116).
# So the simplest workaround is to using RelWithDebiInfo configuration for testing (cmake will uses /Ob1 option), instead of Release.
# - Release
- RelWithDebInfo
platform:
- Win32
- x64
# MSVC-2019 may hang up during code generation/optimization due to its own internal errors.
matrix:
allow_failures:
- image: Visual Studio 2019
configuration: Release
# Enable RDP for troubleshooting
#init:
# - ps: iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-rdp.ps1'))
before_build:
- git clean -x -f -d
- git submodule sync
- git fetch --tags --prune --force
- git submodule update --init --recursive
- git submodule foreach --recursive git fetch --tags --prune --force
- cmake --version
build_script:
- ps: |
Write-Output "*******************************************************************************"
Write-Output "Configuration: $env:CONFIGURATION"
Write-Output "Platform: $env:PLATFORM"
Write-Output "Toolchain: $env:CMAKE_GENERATOR v$env:TOOLSET"
Write-Output "Options: MDBX_WITHOUT_MSVC_CRT=$env:MDBX_WITHOUT_MSVC_CRT MDBX_BUILD_SHARED_LIBRARY=$env:MDBX_BUILD_SHARED_LIBRARY"
Write-Output "*******************************************************************************"
md _build -Force | Out-Null
cd _build
$generator = $env:CMAKE_GENERATOR
if ($env:TOOLSET -lt 142) {
if ($env:PLATFORM -eq "x64") {
$generator = "$generator Win64"
}
& cmake -G "$generator" -D CMAKE_CONFIGURATION_TYPES="Debug;Release;RelWithDebInfo" -D MDBX_WITHOUT_MSVC_CRT:BOOL=$env:MDBX_WITHOUT_MSVC_CRT -D MDBX_BUILD_SHARED_LIBRARY:BOOL=$env:MDBX_BUILD_SHARED_LIBRARY ..
} else {
& cmake -G "$generator" -A $env:PLATFORM -D CMAKE_CONFIGURATION_TYPES="Debug;Release;RelWithDebInfo" -DMDBX_WITHOUT_MSVC_CRT:BOOL=$env:MDBX_WITHOUT_MSVC_CRT -D MDBX_BUILD_SHARED_LIBRARY:BOOL=$env:MDBX_BUILD_SHARED_LIBRARY ..
}
if ($LastExitCode -ne 0) {
throw "Exec: $ErrorMessage"
}
Write-Output "*******************************************************************************"
& cmake --build . --config $env:CONFIGURATION
if ($LastExitCode -ne 0) {
throw "Exec: $ErrorMessage"
}
Write-Output "*******************************************************************************"
test_script:
- ps: |
if (($env:PLATFORM -ne "ARM") -and ($env:PLATFORM -ne "ARM64")) {
& ./$env:CONFIGURATION/mdbx_test.exe --progress --console=no --pathname=test.db --dont-cleanup-after basic > test.log
Get-Content test.log | Select-Object -last 42
if ($LastExitCode -ne 0) {
throw "Exec: $ErrorMessage"
} else {
& ./$env:CONFIGURATION/mdbx_chk.exe -nvv test.db | Tee-Object -file chk.log | Select-Object -last 42
}
}
on_failure:
- ps: Push-AppveyorArtifact \projects\libmdbx\_build\test.log
- ps: Push-AppveyorArtifact \projects\libmdbx\_build\test.db
- ps: Push-AppveyorArtifact \projects\libmdbx\_build\chk.log

View File

@@ -89,9 +89,11 @@ if(CMAKE_C_COMPILER_LOADED)
if(tmp_lcc_marker GREATER -1 AND tmp_e2k_marker GREATER tmp_lcc_marker)
execute_process(COMMAND ${CMAKE_C_COMPILER} -print-version
OUTPUT_VARIABLE CMAKE_C_COMPILER_VERSION
RESULT_VARIABLE tmp_lcc_probe_result)
RESULT_VARIABLE tmp_lcc_probe_result
OUTPUT_STRIP_TRAILING_WHITESPACE)
set(CMAKE_COMPILER_IS_ELBRUSC ON)
set(CMAKE_C_COMPILER_ID "Elbrus")
message(STATUS "Detected Elbrus C compiler ${CMAKE_C_COMPILER_VERSION}")
else()
set(CMAKE_COMPILER_IS_ELBRUSC OFF)
endif()
@@ -113,9 +115,11 @@ if(CMAKE_CXX_COMPILER_LOADED)
if(tmp_lcc_marker GREATER -1 AND tmp_e2k_marker GREATER tmp_lcc_marker)
execute_process(COMMAND ${CMAKE_CXX_COMPILER} -print-version
OUTPUT_VARIABLE CMAKE_CXX_COMPILER_VERSION
RESULT_VARIABLE tmp_lxx_probe_result)
RESULT_VARIABLE tmp_lxx_probe_result
OUTPUT_STRIP_TRAILING_WHITESPACE)
set(CMAKE_COMPILER_IS_ELBRUSCXX ON)
set(CMAKE_CXX_COMPILER_ID "Elbrus")
message(STATUS "Detected Elbrus C++ compiler ${CMAKE_CXX_COMPILER_VERSION}")
else()
set(CMAKE_COMPILER_IS_ELBRUSCXX OFF)
endif()
@@ -178,42 +182,118 @@ if(NOT DEFINED CMAKE_TARGET_BITNESS)
endif()
endif()
if(CMAKE_COMPILER_IS_ELBRUSC OR CMAKE_COMPILER_IS_ELBRUSCXX OR CMAKE_SYSTEM_PROCESSOR MATCHES "e2k.*|E2K.*|elbrus.*|ELBRUS.*")
set(E2K TRUE)
set(CMAKE_SYSTEM_ARCH "Elbrus")
elseif((MSVC64 OR MINGW64) AND CMAKE_TARGET_BITNESS EQUAL 64)
set(X86_64 TRUE)
set(CMAKE_SYSTEM_ARCH "x86_64")
elseif(MINGW OR (MSVC AND NOT CMAKE_CROSSCOMPILING))
set(X86_32 TRUE)
set(CMAKE_SYSTEM_ARCH "x86")
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "amd64.*|x86_64.*|AMD64.*" AND CMAKE_TARGET_BITNESS EQUAL 64)
set(X86_64 TRUE)
set(CMAKE_SYSTEM_ARCH "x86_64")
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "i686.*|i386.*|x86.*")
set(X86_32 TRUE)
set(CMAKE_SYSTEM_ARCH "x86")
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "^(aarch64.*|AARCH64.*|ARM64.*)" AND CMAKE_TARGET_BITNESS EQUAL 64)
set(AARCH64 TRUE)
set(CMAKE_SYSTEM_ARCH "ARM64")
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "^(arm.*|ARM.*)")
set(ARM32 TRUE)
set(CMAKE_SYSTEM_ARCH "ARM")
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "^(powerpc|ppc)64le.*" AND CMAKE_TARGET_BITNESS EQUAL 64)
set(PPC64LE TRUE)
set(CMAKE_SYSTEM_ARCH "PPC64LE")
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "^(powerpc|ppc)64.*" AND CMAKE_TARGET_BITNESS EQUAL 64)
set(PPC64 TRUE)
set(CMAKE_SYSTEM_ARCH "PPC64")
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "^(powerpc|ppc).*")
set(PPC32 TRUE)
set(CMAKE_SYSTEM_ARCH "PPC")
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "^(mips|MIPS)64.*" AND CMAKE_TARGET_BITNESS EQUAL 64)
set(MIPS64 TRUE)
set(CMAKE_SYSTEM_ARCH "MIPS64")
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "^(mips|MIPS).*")
set(MIPS32 TRUE)
set(CMAKE_SYSTEM_ARCH "MIPS")
if(NOT CMAKE_SYSTEM_ARCH)
if(CMAKE_${CMAKE_PRIMARY_LANG}_COMPILER_ARCHITECTURE_ID)
set(CMAKE_SYSTEM_ARCH "${CMAKE_${CMAKE_PRIMARY_LANG}_COMPILER_ARCHITECTURE_ID}")
elseif(CMAKE_ANDROID_ARCH_ABI)
set(CMAKE_SYSTEM_ARCH "${CMAKE_ANDROID_ARCH_ABI}")
elseif(CMAKE_COMPILER_IS_ELBRUSC OR CMAKE_COMPILER_IS_ELBRUSCXX
OR CMAKE_${CMAKE_PRIMARY_LANG}_COMPILER_ID STREQUAL "LCC"
OR CMAKE_SYSTEM_PROCESSOR MATCHES "e2k.*|E2K.*|elbrus.*|ELBRUS.*")
set(E2K TRUE)
set(CMAKE_SYSTEM_ARCH "Elbrus")
elseif(MSVC64 OR MINGW64 OR MINGW OR (MSVC AND NOT CMAKE_CROSSCOMPILING))
if(CMAKE_TARGET_BITNESS EQUAL 64)
set(X86_64 TRUE)
set(CMAKE_SYSTEM_ARCH "x86_64")
else()
set(X86_32 TRUE)
set(CMAKE_SYSTEM_ARCH "x86")
endif()
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "amd64.*|[xXiI]86_64.*|AMD64.*|[iI][3-6]86.*|[xXiI]86.*")
if(CMAKE_TARGET_BITNESS EQUAL 64)
set(X86_64 TRUE)
set(CMAKE_SYSTEM_ARCH "x86_64")
else()
set(X86_32 TRUE)
set(CMAKE_SYSTEM_ARCH "x86")
endif()
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "^(aarch.*|AARCH.*|arm.*|ARM.*)")
if(CMAKE_TARGET_BITNESS EQUAL 64)
set(AARCH64 TRUE)
set(CMAKE_SYSTEM_ARCH "ARM64")
else()
set(ARM32 TRUE)
set(CMAKE_SYSTEM_ARCH "ARM")
endif()
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "^(powerpc|ppc).*")
if(CMAKE_TARGET_BITNESS EQUAL 64)
set(PPC64 TRUE)
set(CMAKE_SYSTEM_ARCH "PPC64")
else()
set(PPC32 TRUE)
set(CMAKE_SYSTEM_ARCH "PPC")
endif()
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "^(mips|MIPS).*")
if(CMAKE_TARGET_BITNESS EQUAL 64)
set(MIPS64 TRUE)
set(CMAKE_SYSTEM_ARCH "MIPS64")
else()
set(MIPS32 TRUE)
set(CMAKE_SYSTEM_ARCH "MIPS")
endif()
endif()
endif()
if(CMAKE_C_COMPILER_LOADED AND NOT CMAKE_C_COMPILER_ARCHITECTURE_ID)
set(CMAKE_C_COMPILER_ARCHITECTURE_ID "${CMAKE_SYSTEM_ARCH}")
endif()
if(CMAKE_CXX_COMPILER_LOADED AND NOT CMAKE_CXX_COMPILER_ARCHITECTURE_ID)
set(CMAKE_CXX_COMPILER_ARCHITECTURE_ID "${CMAKE_SYSTEM_ARCH}")
endif()
if(NOT CMAKE_HOST_ARCH)
if(CMAKE_VS_PLATFORM_TOOLSET_HOST_ARCHITECTURE)
set(CMAKE_HOST_ARCH "${CMAKE_VS_PLATFORM_TOOLSET_HOST_ARCHITECTURE}")
elseif(CMAKE_HOST_PROCESSOR MATCHES "e2k.*|E2K.*|elbrus.*|ELBRUS.*")
set(CMAKE_HOST_ARCH "Elbrus")
elseif(CMAKE_HOST_PROCESSOR MATCHES "amd64.*|[xXiI]86_64.*|AMD64.*")
set(CMAKE_HOST_ARCH "x86_64")
elseif(CMAKE_HOST_PROCESSOR MATCHES "[iI][3-6]86.*|[xXiI]86.*")
set(CMAKE_HOST_ARCH "x86")
elseif(CMAKE_HOST_PROCESSOR MATCHES "^(aarch64.*|AARCH64.*|ARM64.*)")
set(CMAKE_HOST_ARCH "ARM64")
elseif(CMAKE_HOST_PROCESSOR MATCHES "^(arm.*|ARM.*)")
set(CMAKE_HOST_ARCH "ARM")
elseif(CMAKE_HOST_PROCESSOR MATCHES "^(powerpc|ppc)64.*")
set(CMAKE_HOST_ARCH "PPC64")
elseif(CMAKE_HOST_PROCESSOR MATCHES "^(powerpc|ppc).*")
set(CMAKE_HOST_ARCH "PPC")
elseif(CMAKE_HOST_PROCESSOR MATCHES "^(mips|MIPS)64.*")
set(CMAKE_HOST_ARCH "MIPS64")
elseif(CMAKE_HOST_PROCESSOR MATCHES "^(mips|MIPS).*")
set(CMAKE_HOST_ARCH "MIPS")
else()
set(CMAKE_HOST_ARCH "${CMAKE_HOST_SYSTEM_PROCESSOR}")
endif()
endif()
if(NOT DEFINED CMAKE_HOST_CAN_RUN_EXECUTABLES_BUILT_FOR_TARGET)
if(CMAKE_CROSSCOMPILING AND CMAKE_CROSSCOMPILING_EMULATOR)
set(CMAKE_HOST_CAN_RUN_EXECUTABLES_BUILT_FOR_TARGET TRUE)
elseif(CMAKE_CROSSCOMPILING AND NOT CMAKE_CROSSCOMPILING_EMULATOR)
set(CMAKE_HOST_CAN_RUN_EXECUTABLES_BUILT_FOR_TARGET FALSE)
elseif(CMAKE_SYSTEM_NAME STREQUAL CMAKE_HOST_SYSTEM_NAME
AND ((CMAKE_SYSTEM_PROCESSOR STREQUAL CMAKE_HOST_PROCESSOR)
OR (CMAKE_SYSTEM_ARCH STREQUAL CMAKE_HOST_ARCH)
OR (WIN32 AND CMAKE_HOST_WIN32 AND X86_32 AND CMAKE_HOST_ARCH STREQUAL "x86_64")))
set(CMAKE_HOST_CAN_RUN_EXECUTABLES_BUILT_FOR_TARGET TRUE)
message(STATUS
"Assume СAN RUN A BUILT EXECUTABLES,"
" since host (${CMAKE_HOST_SYSTEM_NAME}-${CMAKE_HOST_ARCH})"
" match target (${CMAKE_SYSTEM_NAME}-${CMAKE_SYSTEM_ARCH})")
else()
if(CMAKE_C_COMPILER_LOADED)
include(CheckCSourceRuns)
check_c_source_runs("int main(void) { return 0; }" CMAKE_HOST_CAN_RUN_EXECUTABLES_BUILT_FOR_TARGET)
elseif(CMAKE_CXX_COMPILER_LOADED)
include(CheckCXXSourceRuns)
check_cxx_source_runs("int main(void) { return 0; }" CMAKE_HOST_CAN_RUN_EXECUTABLES_BUILT_FOR_TARGET)
endif()
if(NOT CMAKE_HOST_CAN_RUN_EXECUTABLES_BUILT_FOR_TARGET)
message(STATUS "Force CMAKE_CROSSCOMPILING to TRUE")
set(CMAKE_CROSSCOMPILING TRUE)
endif()
endif()
endif()
if(MSVC)
@@ -464,7 +544,7 @@ if(CMAKE_COMPILER_IS_CLANG)
AND NOT (CMAKE_HOST_SYSTEM_NAME STREQUAL "Linux"
AND CMAKE_SYSTEM_NAME STREQUAL "Linux"))
OR APPLE))
if(ANDROID AND CMAKE_SYSTEM_VERSION VERSION_LESS 22)
if(ANDROID AND CMAKE_${CMAKE_PRIMARY_LANG}_COMPILER_VERSION VERSION_LESS 12)
set(CLANG_LTO_AVAILABLE FALSE)
message(STATUS "Link-Time Optimization by CLANG/LLVM is available but unusable due https://reviews.llvm.org/D79919")
else()
@@ -605,6 +685,9 @@ macro(setup_compile_flags)
AND CMAKE_${CMAKE_PRIMARY_LANG}_COMPILER_VERSION VERSION_LESS 9)
# GCC < 9.x generates false-positive warnings for optimization attributes
add_compile_flags("C;CXX" "-Wno-attributes")
if(LTO_ENABLED)
add_compile_flags("C;CXX" "-Wno-lto-type-mismatch")
endif()
endif()
# In C a global variable without a storage specifier (static/extern) and
@@ -658,8 +741,8 @@ macro(setup_compile_flags)
if(NOT MSVC_VERSION LESS 1910)
add_compile_flags("CXX" "/Zc:__cplusplus")
endif()
add_compile_flags("C;CXX" "/W4")
add_compile_flags("C;CXX" "/utf-8")
remove_compile_flag("C;CXX" "/W3")
add_compile_flags("C;CXX" "/W4 /utf-8")
else()
if(CC_HAS_WALL)
add_compile_flags("C;CXX" "-Wall")
@@ -751,16 +834,12 @@ macro(setup_compile_flags)
add_compile_flags("C;CXX" "/GL")
foreach(linkmode IN ITEMS EXE SHARED STATIC MODULE)
set(${linkmode}_LINKER_FLAGS "${${linkmode}_LINKER_FLAGS} /LTCG")
string(REGEX REPLACE "^(.*)(/INCREMENTAL:NO *)(.*)$" "\\1\\3" ${linkmode}_LINKER_FLAGS "${${linkmode}_LINKER_FLAGS}")
string(REGEX REPLACE "^(.*)(/INCREMENTAL:YES *)(.*)$" "\\1\\3" ${linkmode}_LINKER_FLAGS "${${linkmode}_LINKER_FLAGS}")
string(REGEX REPLACE "^(.*)(/INCREMENTAL *)(.*)$" "\\1\\3" ${linkmode}_LINKER_FLAGS "${${linkmode}_LINKER_FLAGS}")
string(REGEX REPLACE "^(.*)(/INCREMENTAL)(:YES)?(:NO)?( ?.*)$" "\\1\\2:NO\\5" ${linkmode}_LINKER_FLAGS "${${linkmode}_LINKER_FLAGS}")
string(STRIP "${${linkmode}_LINKER_FLAGS}" ${linkmode}_LINKER_FLAGS)
foreach(config IN LISTS CMAKE_CONFIGURATION_TYPES ITEMS Release MinSizeRel RelWithDebInfo Debug)
string(TOUPPER "${config}" config_uppercase)
if(DEFINED "CMAKE_${linkmode}_LINKER_FLAGS_${config_uppercase}")
string(REGEX REPLACE "^(.*)(/INCREMENTAL:NO *)(.*)$" "\\1\\3" altered_flags "${CMAKE_${linkmode}_LINKER_FLAGS_${config_uppercase}}")
string(REGEX REPLACE "^(.*)(/INCREMENTAL:YES *)(.*)$" "\\1\\3" altered_flags "${altered_flags}")
string(REGEX REPLACE "^(.*)(/INCREMENTAL *)(.*)$" "\\1\\3" altered_flags "${altered_flags}")
string(REGEX REPLACE "^(.*)(/INCREMENTAL)(:YES)?(:NO)?( ?.*)$" "\\1\\2:NO\\5" altered_flags "${CMAKE_${linkmode}_LINKER_FLAGS_${config_uppercase}}")
string(STRIP "${altered_flags}" altered_flags)
if(NOT "${altered_flags}" STREQUAL "${CMAKE_${linkmode}_LINKER_FLAGS_${config_uppercase}}")
set(CMAKE_${linkmode}_LINKER_FLAGS_${config_uppercase} "${altered_flags}" CACHE STRING "Altered: '/INCREMENTAL' removed for LTO" FORCE)
@@ -826,20 +905,119 @@ macro(setup_compile_flags)
unset(MODULE_LINKER_FLAGS)
endmacro(setup_compile_flags)
if(CMAKE_CXX_COMPILER_LOADED)
# determine library for for std::filesystem
set(LIBCXX_FILESYSTEM "")
if(CMAKE_COMPILER_IS_CLANG)
if(CMAKE_CXX_COMPILER_VERSION VERSION_LESS 5.0)
set(LIBCXX_FILESYSTEM "c++experimental")
elseif(CMAKE_CXX_COMPILER_VERSION VERSION_LESS 9.0)
set(LIBCXX_FILESYSTEM "stdc++fs")
endif()
elseif(CMAKE_COMPILER_IS_GNUCXX AND NOT MINGW)
if(NOT CMAKE_CXX_COMPILER_VERSION VERSION_LESS 5.3 AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 9.0)
set(LIBCXX_FILESYSTEM "stdc++fs")
macro(probe_libcxx_filesystem)
if(CMAKE_CXX_COMPILER_LOADED AND NOT DEFINED LIBCXX_FILESYSTEM)
list(FIND CMAKE_CXX_COMPILE_FEATURES cxx_std_11 HAS_CXX11)
if(NOT HAS_CXX11 LESS 0)
include(CMakePushCheckState)
include(CheckCXXSourceCompiles)
cmake_push_check_state()
set(stdfs_probe_save_libraries ${CMAKE_REQUIRED_LIBRARIES})
set(stdfs_probe_save_flags ${CMAKE_REQUIRED_FLAGS})
set(stdfs_probe_save_link_options ${CMAKE_REQUIRED_LINK_OPTIONS})
unset(stdfs_probe_clear_cxx_standard)
if(NOT DEFINED CMAKE_CXX_STANDARD)
list(FIND CMAKE_CXX_COMPILE_FEATURES cxx_std_14 HAS_CXX14)
list(FIND CMAKE_CXX_COMPILE_FEATURES cxx_std_17 HAS_CXX17)
if(NOT HAS_CXX17 LESS 0
AND NOT (CMAKE_COMPILER_IS_CLANG AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 5))
set(CMAKE_CXX_STANDARD 17)
elseif(NOT HAS_CXX14 LESS 0)
set(CMAKE_CXX_STANDARD 14)
else()
set(CMAKE_CXX_STANDARD 11)
endif()
set(stdfs_probe_clear_cxx_standard ON)
endif()
if(CMAKE_COMPILER_IS_ELBRUSCXX AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 1.25.23)
if(CMAKE_VERSION VERSION_LESS 3.14)
set(CMAKE_REQUIRED_FLAGS ${stdfs_probe_save_flags} "-Wl,--allow-multiple-definition")
else()
set(CMAKE_REQUIRED_LINK_OPTIONS ${stdfs_probe_save_link_options} "-Wl,--allow-multiple-definition")
endif()
endif()
set(stdfs_probe_code [[
#if defined(__SIZEOF_INT128__) && !defined(__GLIBCXX_TYPE_INT_N_0) && defined(__clang__) && __clang_major__ < 4
#define __GLIBCXX_BITSIZE_INT_N_0 128
#define __GLIBCXX_TYPE_INT_N_0 __int128
#endif
#ifndef __has_include
#define __has_include(header) (0)
#endif
#if __has_include(<version>)
#include <version>
#endif
#include <cstdlib>
#include <string>
#if defined(__cpp_lib_string_view) && __cpp_lib_string_view >= 201606L
#include <string_view>
#endif
#if defined(__cpp_lib_filesystem) && __cpp_lib_filesystem >= 201703L
#include <filesystem>
#else
#include <experimental/filesystem>
#endif
#if (defined(__cpp_lib_filesystem) && __cpp_lib_filesystem >= 201703L && (!defined(__MAC_OS_X_VERSION_MIN_REQUIRED) || __MAC_OS_X_VERSION_MIN_REQUIRED >= 101500) && (!defined(__IPHONE_OS_VERSION_MIN_REQUIRED) || __IPHONE_OS_VERSION_MIN_REQUIRED >= 130100))
namespace fs = ::std::filesystem;
#elif defined(__cpp_lib_experimental_filesystem) && __cpp_lib_experimental_filesystem >= 201406L
namespace fs = ::std::experimental::filesystem;
#endif
int main(int argc, const char*argv[]) {
fs::path probe(argv[0]);
if (argc != 1) throw fs::filesystem_error(std::string("fake"), std::error_code());
return fs::is_directory(probe.relative_path());
}
]])
set(LIBCXX_FILESYSTEM "")
check_cxx_source_compiles("${stdfs_probe_code}" LIBCXX_FILESYSTEM_none)
if(LIBCXX_FILESYSTEM_none)
message(STATUS "No linking with additional library needed for std::filesystem")
else()
set(CMAKE_REQUIRED_LIBRARIES ${stdfs_probe_save_libraries} "stdc++fs")
check_cxx_source_compiles("${stdfs_probe_code}" LIBCXX_FILESYSTEM_stdcxxfs)
if(LIBCXX_FILESYSTEM_stdcxxfs)
set(LIBCXX_FILESYSTEM "stdc++fs")
message(STATUS "Linking with ${LIBCXX_FILESYSTEM} is required for std::filesystem")
else()
set(CMAKE_REQUIRED_LIBRARIES ${stdfs_probe_save_libraries} "c++fs")
check_cxx_source_compiles("${stdfs_probe_code}" LIBCXX_FILESYSTEM_cxxfs)
if(LIBCXX_FILESYSTEM_cxxfs)
set(LIBCXX_FILESYSTEM "c++fs")
message(STATUS "Linking with ${LIBCXX_FILESYSTEM} is required for std::filesystem")
else()
set(CMAKE_REQUIRED_LIBRARIES ${stdfs_probe_save_libraries} "c++experimental")
check_cxx_source_compiles("${stdfs_probe_code}" LIBCXX_FILESYSTEM_cxxexperimental)
if(LIBCXX_FILESYSTEM_cxxexperimental)
set(LIBCXX_FILESYSTEM "c++experimental")
message(STATUS "Linking with ${LIBCXX_FILESYSTEM} is required for std::filesystem")
else()
message(STATUS "No support for std::filesystem")
endif()
endif()
endif()
endif()
set(CMAKE_REQUIRED_LINK_OPTIONS ${stdfs_probe_save_link_options})
set(CMAKE_REQUIRED_FLAGS ${stdfs_probe_save_flags})
set(CMAKE_REQUIRED_LIBRARIES ${stdfs_probe_save_libraries})
if(stdfs_probe_clear_cxx_standard)
unset(CMAKE_CXX_STANDARD)
endif()
unset(stdfs_probe_clear_cxx_standard)
unset(stdfs_probe_save_link_options)
unset(stdfs_probe_save_flags)
unset(stdfs_probe_save_libraries)
unset(stdfs_probe_code)
unset(stdfs_probe_rc)
cmake_pop_check_state()
endif()
endif()
endif()
endmacro(probe_libcxx_filesystem)
cmake_policy(POP)

View File

@@ -36,6 +36,23 @@ macro(add_compile_flags languages)
unset(_flags)
endmacro(add_compile_flags)
macro(remove_flag varname flag)
string(REGEX REPLACE "^(.*)( ${flag} )(.*)$" "\\1 \\3" ${varname} ${${varname}})
string(REGEX REPLACE "^((.+ )*)(${flag})(( .+)*)$" "\\1\\4" ${varname} ${${varname}})
endmacro(remove_flag)
macro(remove_compile_flag languages flag)
foreach(_lang ${languages})
if(CMAKE_CXX_COMPILER_LOADED AND _lang STREQUAL "CXX")
remove_flag(${_lang}_FLAGS ${flag})
endif()
if(CMAKE_C_COMPILER_LOADED AND _lang STREQUAL "C")
remove_flag(${_lang}_FLAGS ${flag})
endif()
endforeach()
unset(_lang)
endmacro(remove_compile_flag)
macro(set_source_files_compile_flags)
foreach(file ${ARGN})
get_filename_component(_file_ext ${file} EXT)
@@ -75,15 +92,6 @@ macro(fetch_version name source_root_directory parent_scope)
set(${name}_GIT_REVISION 0)
set(${name}_GIT_VERSION "")
if(GIT AND EXISTS "${source_root_directory}/.git")
execute_process(COMMAND ${GIT} describe --tags --long --dirty=-dirty
OUTPUT_VARIABLE ${name}_GIT_DESCRIBE
OUTPUT_STRIP_TRAILING_WHITESPACE
WORKING_DIRECTORY ${source_root_directory}
RESULT_VARIABLE rc)
if(rc OR "${name}_GIT_DESCRIBE" STREQUAL "")
message(FATAL_ERROR "Please fetch tags and/or install latest version of git ('describe --tags --long --dirty' failed)")
endif()
execute_process(COMMAND ${GIT} show --no-patch --format=%cI HEAD
OUTPUT_VARIABLE ${name}_GIT_TIMESTAMP
OUTPUT_STRIP_TRAILING_WHITESPACE
@@ -96,7 +104,7 @@ macro(fetch_version name source_root_directory parent_scope)
WORKING_DIRECTORY ${source_root_directory}
RESULT_VARIABLE rc)
if(rc OR "${name}_GIT_TIMESTAMP" STREQUAL "%ci")
message(FATAL_ERROR "Please install latest version of git ('show --no-patch --format=%cI HEAD' failed)")
message(FATAL_ERROR "Please install latest version of git (`show --no-patch --format=%cI HEAD` failed)")
endif()
endif()
@@ -106,7 +114,7 @@ macro(fetch_version name source_root_directory parent_scope)
WORKING_DIRECTORY ${source_root_directory}
RESULT_VARIABLE rc)
if(rc OR "${name}_GIT_TREE" STREQUAL "")
message(FATAL_ERROR "Please install latest version of git ('show --no-patch --format=%T HEAD' failed)")
message(FATAL_ERROR "Please install latest version of git (`show --no-patch --format=%T HEAD` failed)")
endif()
execute_process(COMMAND ${GIT} show --no-patch --format=%H HEAD
@@ -115,58 +123,110 @@ macro(fetch_version name source_root_directory parent_scope)
WORKING_DIRECTORY ${source_root_directory}
RESULT_VARIABLE rc)
if(rc OR "${name}_GIT_COMMIT" STREQUAL "")
message(FATAL_ERROR "Please install latest version of git ('show --no-patch --format=%H HEAD' failed)")
message(FATAL_ERROR "Please install latest version of git (`show --no-patch --format=%H HEAD` failed)")
endif()
execute_process(COMMAND ${GIT} describe --tags --abbrev=0 "--match=v[0-9]*"
OUTPUT_VARIABLE last_release_tag
execute_process(COMMAND ${GIT} rev-list --tags --count
OUTPUT_VARIABLE tag_count
OUTPUT_STRIP_TRAILING_WHITESPACE
WORKING_DIRECTORY ${source_root_directory}
RESULT_VARIABLE rc)
if(rc)
message(FATAL_ERROR "Please install latest version of git ('describe --tags --abbrev=0 --match=v[0-9]*' failed)")
message(FATAL_ERROR "Please install latest version of git (`git rev-list --tags --count` failed)")
endif()
if (last_release_tag)
set(git_revlist_arg "${last_release_tag}..HEAD")
else()
execute_process(COMMAND ${GIT} tag --sort=-version:refname
OUTPUT_VARIABLE tag_list
if(tag_count EQUAL 0)
execute_process(COMMAND ${GIT} rev-list --all --count
OUTPUT_VARIABLE whole_count
OUTPUT_STRIP_TRAILING_WHITESPACE
WORKING_DIRECTORY ${source_root_directory}
RESULT_VARIABLE rc)
if(rc)
message(FATAL_ERROR "Please install latest version of git ('tag --sort=-version:refname' failed)")
message(FATAL_ERROR "Please install latest version of git (`git rev-list --all --count` failed)")
endif()
string(REGEX REPLACE "\n" ";" tag_list "${tag_list}")
set(git_revlist_arg "HEAD")
foreach(tag IN LISTS tag_list)
if(NOT last_release_tag)
string(REGEX MATCH "^v[0-9]+(\.[0-9]+)+" last_release_tag "${tag}")
set(git_revlist_arg "${tag}..HEAD")
if(whole_count GREATER 42)
message(FATAL_ERROR "Please fetch tags (no any tags for ${whole_count} commits)")
endif()
set(${name}_GIT_VERSION "0;0;0")
execute_process(COMMAND ${GIT} rev-list --count --all --no-merges
OUTPUT_VARIABLE ${name}_GIT_REVISION
OUTPUT_STRIP_TRAILING_WHITESPACE
WORKING_DIRECTORY ${source_root_directory}
RESULT_VARIABLE rc)
if(rc OR "${name}_GIT_REVISION" STREQUAL "")
message(FATAL_ERROR "Please install latest version of git (`rev-list --count --all --no-merges` failed)")
endif()
else(tag_count EQUAL 0)
execute_process(COMMAND ${GIT} describe --tags --long --dirty=-dirty "--match=v[0-9]*"
OUTPUT_VARIABLE ${name}_GIT_DESCRIBE
OUTPUT_STRIP_TRAILING_WHITESPACE
WORKING_DIRECTORY ${source_root_directory}
RESULT_VARIABLE rc)
if(rc OR "${name}_GIT_DESCRIBE" STREQUAL "")
if(_whole_count GREATER 42)
message(FATAL_ERROR "Please fetch tags (`describe --tags --long --dirty --match=v[0-9]*` failed)")
else()
execute_process(COMMAND ${GIT} describe --all --long --dirty=-dirty
OUTPUT_VARIABLE ${name}_GIT_DESCRIBE
OUTPUT_STRIP_TRAILING_WHITESPACE
WORKING_DIRECTORY ${source_root_directory}
RESULT_VARIABLE rc)
if(rc OR "${name}_GIT_DESCRIBE" STREQUAL "")
message(FATAL_ERROR "Please install latest version of git (`git rev-list --tags --count` and/or `git rev-list --all --count` failed)")
endif()
endif()
endforeach(tag)
endif()
execute_process(COMMAND ${GIT} rev-list --count "${git_revlist_arg}"
OUTPUT_VARIABLE ${name}_GIT_REVISION
OUTPUT_STRIP_TRAILING_WHITESPACE
WORKING_DIRECTORY ${source_root_directory}
RESULT_VARIABLE rc)
if(rc OR "${name}_GIT_REVISION" STREQUAL "")
message(FATAL_ERROR "Please install latest version of git ('rev-list --count ${git_revlist_arg}' failed)")
endif()
string(REGEX MATCH "^(v)?([0-9]+)\\.([0-9]+)\\.([0-9]+)(.*)?" git_version_valid "${${name}_GIT_DESCRIBE}")
if(git_version_valid)
string(REGEX REPLACE "^(v)?([0-9]+)\\.([0-9]+)\\.([0-9]+)(.*)?" "\\2;\\3;\\4" ${name}_GIT_VERSION ${${name}_GIT_DESCRIBE})
else()
string(REGEX MATCH "^(v)?([0-9]+)\\.([0-9]+)(.*)?" git_version_valid "${${name}_GIT_DESCRIBE}")
if(git_version_valid)
string(REGEX REPLACE "^(v)?([0-9]+)\\.([0-9]+)(.*)?" "\\2;\\3;0" ${name}_GIT_VERSION ${${name}_GIT_DESCRIBE})
else()
message(AUTHOR_WARNING "Bad ${name} version \"${${name}_GIT_DESCRIBE}\"; falling back to 0.0.0 (have you made an initial release?)")
set(${name}_GIT_VERSION "0;0;0")
endif()
endif()
execute_process(COMMAND ${GIT} describe --tags --abbrev=0 "--match=v[0-9]*"
OUTPUT_VARIABLE last_release_tag
OUTPUT_STRIP_TRAILING_WHITESPACE
WORKING_DIRECTORY ${source_root_directory}
RESULT_VARIABLE rc)
if(rc)
message(FATAL_ERROR "Please install latest version of git (`describe --tags --abbrev=0 --match=v[0-9]*` failed)")
endif()
if (last_release_tag)
set(git_revlist_arg "${last_release_tag}..HEAD")
else()
execute_process(COMMAND ${GIT} tag --sort=-version:refname
OUTPUT_VARIABLE tag_list
OUTPUT_STRIP_TRAILING_WHITESPACE
WORKING_DIRECTORY ${source_root_directory}
RESULT_VARIABLE rc)
if(rc)
message(FATAL_ERROR "Please install latest version of git (`tag --sort=-version:refname` failed)")
endif()
string(REGEX REPLACE "\n" ";" tag_list "${tag_list}")
set(git_revlist_arg "HEAD")
foreach(tag IN LISTS tag_list)
if(NOT last_release_tag)
string(REGEX MATCH "^v[0-9]+(\.[0-9]+)+" last_release_tag "${tag}")
set(git_revlist_arg "${tag}..HEAD")
endif()
endforeach(tag)
endif()
execute_process(COMMAND ${GIT} rev-list --count "${git_revlist_arg}"
OUTPUT_VARIABLE ${name}_GIT_REVISION
OUTPUT_STRIP_TRAILING_WHITESPACE
WORKING_DIRECTORY ${source_root_directory}
RESULT_VARIABLE rc)
if(rc OR "${name}_GIT_REVISION" STREQUAL "")
message(FATAL_ERROR "Please install latest version of git (`rev-list --count ${git_revlist_arg}` failed)")
endif()
string(REGEX MATCH "^(v)?([0-9]+)\\.([0-9]+)\\.([0-9]+)(.*)?" git_version_valid "${${name}_GIT_DESCRIBE}")
if(git_version_valid)
string(REGEX REPLACE "^(v)?([0-9]+)\\.([0-9]+)\\.([0-9]+)(.*)?" "\\2;\\3;\\4" ${name}_GIT_VERSION ${${name}_GIT_DESCRIBE})
else()
string(REGEX MATCH "^(v)?([0-9]+)\\.([0-9]+)(.*)?" git_version_valid "${${name}_GIT_DESCRIBE}")
if(git_version_valid)
string(REGEX REPLACE "^(v)?([0-9]+)\\.([0-9]+)(.*)?" "\\2;\\3;0" ${name}_GIT_VERSION ${${name}_GIT_DESCRIBE})
else()
message(AUTHOR_WARNING "Bad ${name} version \"${${name}_GIT_DESCRIBE}\"; falling back to 0.0.0 (have you made an initial release?)")
set(${name}_GIT_VERSION "0;0;0")
endif()
endif()
endif(tag_count EQUAL 0)
endif()
if(NOT ${name}_GIT_VERSION OR NOT ${name}_GIT_TIMESTAMP OR ${name}_GIT_REVISION STREQUAL "")

View File

@@ -1,4 +1,4 @@
# Doxyfile 1.8.17
# Doxyfile 1.9.1
# This file describes the settings to be used by the documentation system
# doxygen (www.doxygen.org) for a project.
@@ -227,6 +227,14 @@ QT_AUTOBRIEF = NO
MULTILINE_CPP_IS_BRIEF = NO
# By default Python docstrings are displayed as preformatted text and doxygen's
# special commands cannot be used. By setting PYTHON_DOCSTRING to NO the
# doxygen's special commands can be used and the contents of the docstring
# documentation blocks is shown as doxygen documentation.
# The default value is: YES.
PYTHON_DOCSTRING = YES
# If the INHERIT_DOCS tag is set to YES then an undocumented member inherits the
# documentation from any documented member that it re-implements.
# The default value is: YES.
@@ -263,12 +271,6 @@ TAB_SIZE = 4
ALIASES =
# This tag can be used to specify a number of word-keyword mappings (TCL only).
# A mapping has the form "name=value". For example adding "class=itcl::class"
# will allow you to use the command class in the itcl::class meaning.
TCL_SUBST =
# Set the OPTIMIZE_OUTPUT_FOR_C tag to YES if your project consists of C sources
# only. Doxygen will then generate output that is more tailored for C. For
# instance, some of the names that are used will be different. The list of all
@@ -310,18 +312,21 @@ OPTIMIZE_OUTPUT_SLICE = NO
# extension. Doxygen has a built-in mapping, but you can override or extend it
# using this tag. The format is ext=language, where ext is a file extension, and
# language is one of the parsers supported by doxygen: IDL, Java, JavaScript,
# Csharp (C#), C, C++, D, PHP, md (Markdown), Objective-C, Python, Slice,
# Csharp (C#), C, C++, D, PHP, md (Markdown), Objective-C, Python, Slice, VHDL,
# Fortran (fixed format Fortran: FortranFixed, free formatted Fortran:
# FortranFree, unknown formatted Fortran: Fortran. In the later case the parser
# tries to guess whether the code is fixed or free formatted code, this is the
# default for Fortran type files), VHDL, tcl. For instance to make doxygen treat
# .inc files as Fortran files (default is PHP), and .f files as C (default is
# Fortran), use: inc=Fortran f=C.
# default for Fortran type files). For instance to make doxygen treat .inc files
# as Fortran files (default is PHP), and .f files as C (default is Fortran),
# use: inc=Fortran f=C.
#
# Note: For files without extension you can use no_extension as a placeholder.
#
# Note that for custom extensions you also need to set FILE_PATTERNS otherwise
# the files are not read by doxygen.
# the files are not read by doxygen. When specifying no_extension you should add
# * to the FILE_PATTERNS.
#
# Note see also the list of default file extension mappings.
EXTENSION_MAPPING =
@@ -455,6 +460,19 @@ TYPEDEF_HIDES_STRUCT = YES
LOOKUP_CACHE_SIZE = 0
# The NUM_PROC_THREADS specifies the number threads doxygen is allowed to use
# during processing. When set to 0 doxygen will based this on the number of
# cores available in the system. You can set it explicitly to a value larger
# than 0 to get more control over the balance between CPU load and processing
# speed. At this moment only the input processing can be done using multiple
# threads. Since this is still an experimental feature the default is set to 1,
# which efficively disables parallel processing. Please report any issues you
# encounter. Generating dot graphs in parallel is controlled by the
# DOT_NUM_THREADS setting.
# Minimum value: 0, maximum value: 32, default value: 1.
NUM_PROC_THREADS = 1
#---------------------------------------------------------------------------
# Build related configuration options
#---------------------------------------------------------------------------
@@ -518,6 +536,13 @@ EXTRACT_LOCAL_METHODS = NO
EXTRACT_ANON_NSPACES = NO
# If this flag is set to YES, the name of an unnamed parameter in a declaration
# will be determined by the corresponding definition. By default unnamed
# parameters remain unnamed in the output.
# The default value is: YES.
RESOLVE_UNNAMED_PARAMS = YES
# If the HIDE_UNDOC_MEMBERS tag is set to YES, doxygen will hide all
# undocumented members inside documented classes or files. If set to NO these
# members will be included in the various overviews, but no documentation
@@ -555,11 +580,18 @@ HIDE_IN_BODY_DOCS = NO
INTERNAL_DOCS = NO
# If the CASE_SENSE_NAMES tag is set to NO then doxygen will only generate file
# names in lower-case letters. If set to YES, upper-case letters are also
# allowed. This is useful if you have classes or files whose names only differ
# in case and if your file system supports case sensitive file names. Windows
# (including Cygwin) ands Mac users are advised to set this option to NO.
# With the correct setting of option CASE_SENSE_NAMES doxygen will better be
# able to match the capabilities of the underlying filesystem. In case the
# filesystem is case sensitive (i.e. it supports files in the same directory
# whose names only differ in casing), the option must be set to YES to properly
# deal with such files in case they appear in the input. For filesystems that
# are not case sensitive the option should be be set to NO to properly deal with
# output files written for symbols that only differ in casing, such as for two
# classes, one named CLASS and the other named Class, and to also support
# references to files without having to specify the exact matching casing. On
# Windows (including Cygwin) and MacOS, users should typically set this option
# to NO, whereas on Linux or other Unix flavors it should typically be set to
# YES.
# The default value is: system dependent.
CASE_SENSE_NAMES = NO
@@ -798,7 +830,10 @@ WARN_IF_DOC_ERROR = YES
WARN_NO_PARAMDOC = NO
# If the WARN_AS_ERROR tag is set to YES then doxygen will immediately stop when
# a warning is encountered.
# a warning is encountered. If the WARN_AS_ERROR tag is set to FAIL_ON_WARNINGS
# then doxygen will continue running as if WARN_AS_ERROR tag is set to NO, but
# at the end of the doxygen process doxygen will return with a non-zero status.
# Possible values are: NO, YES and FAIL_ON_WARNINGS.
# The default value is: NO.
WARN_AS_ERROR = NO
@@ -829,13 +864,19 @@ WARN_LOGFILE =
# spaces. See also FILE_PATTERNS and EXTENSION_MAPPING
# Note: If this tag is empty the current directory is searched.
INPUT = overall.md intro.md usage.md mdbx.h mdbx.h++ options.h ChangeLog.md
INPUT = overall.md \
intro.md \
usage.md \
mdbx.h \
mdbx.h++ \
options.h \
ChangeLog.md
# This tag can be used to specify the character encoding of the source files
# that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses
# libiconv (or the iconv built into libc) for the transcoding. See the libiconv
# documentation (see: https://www.gnu.org/software/libiconv/) for the list of
# possible encodings.
# documentation (see:
# https://www.gnu.org/software/libiconv/) for the list of possible encodings.
# The default value is: UTF-8.
INPUT_ENCODING = UTF-8
@@ -848,13 +889,15 @@ INPUT_ENCODING = UTF-8
# need to set EXTENSION_MAPPING for the extension otherwise the files are not
# read by doxygen.
#
# Note the list of default checked file patterns might differ from the list of
# default file extension mappings.
#
# If left blank the following patterns are tested:*.c, *.cc, *.cxx, *.cpp,
# *.c++, *.java, *.ii, *.ixx, *.ipp, *.i++, *.inl, *.idl, *.ddl, *.odl, *.h,
# *.hh, *.hxx, *.hpp, *.h++, *.cs, *.d, *.php, *.php4, *.php5, *.phtml, *.inc,
# *.m, *.markdown, *.md, *.mm, *.dox (to be provided as doxygen C comment),
# *.doc (to be provided as doxygen C comment), *.txt (to be provided as doxygen
# C comment), *.py, *.pyw, *.f90, *.f95, *.f03, *.f08, *.f, *.for, *.tcl, *.vhd,
# *.vhdl, *.ucf, *.qsf and *.ice.
# *.py, *.pyw, *.f90, *.f95, *.f03, *.f08, *.f18, *.f, *.for, *.vhd, *.vhdl,
# *.ucf, *.qsf and *.ice.
FILE_PATTERNS = *.h
@@ -1069,16 +1112,22 @@ USE_HTAGS = NO
VERBATIM_HEADERS = YES
# If the CLANG_ASSISTED_PARSING tag is set to YES then doxygen will use the
# clang parser (see: http://clang.llvm.org/) for more accurate parsing at the
# cost of reduced performance. This can be particularly helpful with template
# rich C++ code for which doxygen's built-in parser lacks the necessary type
# information.
# clang parser (see:
# http://clang.llvm.org/) for more accurate parsing at the cost of reduced
# performance. This can be particularly helpful with template rich C++ code for
# which doxygen's built-in parser lacks the necessary type information.
# Note: The availability of this option depends on whether or not doxygen was
# generated with the -Duse_libclang=ON option for CMake.
# The default value is: NO.
CLANG_ASSISTED_PARSING = NO
# If clang assisted parsing is enabled and the CLANG_ADD_INC_PATHS tag is set to
# YES then doxygen will add the directory of each input to the include path.
# The default value is: YES.
CLANG_ADD_INC_PATHS = YES
# If clang assisted parsing is enabled you can provide the compiler with command
# line options that you would normally use when invoking the compiler. Note that
# the include paths will already be set by doxygen for the files and directories
@@ -1088,10 +1137,13 @@ CLANG_ASSISTED_PARSING = NO
CLANG_OPTIONS =
# If clang assisted parsing is enabled you can provide the clang parser with the
# path to the compilation database (see:
# http://clang.llvm.org/docs/HowToSetupToolingForLLVM.html) used when the files
# were built. This is equivalent to specifying the "-p" option to a clang tool,
# such as clang-check. These options will then be passed to the parser.
# path to the directory containing a file called compile_commands.json. This
# file is the compilation database (see:
# http://clang.llvm.org/docs/HowToSetupToolingForLLVM.html) containing the
# options used when the source files were built. This is equivalent to
# specifying the -p option to a clang tool, such as clang-check. These options
# will then be passed to the parser. Any options specified with CLANG_OPTIONS
# will be added as well.
# Note: The availability of this option depends on whether or not doxygen was
# generated with the -Duse_libclang=ON option for CMake.
@@ -1108,13 +1160,6 @@ CLANG_DATABASE_PATH =
ALPHABETICAL_INDEX = YES
# The COLS_IN_ALPHA_INDEX tag can be used to specify the number of columns in
# which the alphabetical index list will be split.
# Minimum value: 1, maximum value: 20, default value: 5.
# This tag requires that the tag ALPHABETICAL_INDEX is set to YES.
COLS_IN_ALPHA_INDEX = 5
# In case all classes in a project start with a common prefix, all classes will
# be put under the same header in the alphabetical index. The IGNORE_PREFIX tag
# can be used to specify a prefix (or a list of prefixes) that should be ignored
@@ -1285,10 +1330,11 @@ HTML_INDEX_NUM_ENTRIES = 100
# If the GENERATE_DOCSET tag is set to YES, additional index files will be
# generated that can be used as input for Apple's Xcode 3 integrated development
# environment (see: https://developer.apple.com/xcode/), introduced with OSX
# 10.5 (Leopard). To create a documentation set, doxygen will generate a
# Makefile in the HTML output directory. Running make will produce the docset in
# that directory and running make install will install the docset in
# environment (see:
# https://developer.apple.com/xcode/), introduced with OSX 10.5 (Leopard). To
# create a documentation set, doxygen will generate a Makefile in the HTML
# output directory. Running make will produce the docset in that directory and
# running make install will install the docset in
# ~/Library/Developer/Shared/Documentation/DocSets so that Xcode will find it at
# startup. See https://developer.apple.com/library/archive/featuredarticles/Doxy
# genXcode/_index.html for more information.
@@ -1330,8 +1376,8 @@ DOCSET_PUBLISHER_NAME = Publisher
# If the GENERATE_HTMLHELP tag is set to YES then doxygen generates three
# additional HTML index files: index.hhp, index.hhc, and index.hhk. The
# index.hhp is a project file that can be read by Microsoft's HTML Help Workshop
# (see: https://www.microsoft.com/en-us/download/details.aspx?id=21138) on
# Windows.
# (see:
# https://www.microsoft.com/en-us/download/details.aspx?id=21138) on Windows.
#
# The HTML Help Workshop contains a compiler that can convert all HTML output
# generated by doxygen into a single compiled HTML file (.chm). Compiled HTML
@@ -1361,7 +1407,7 @@ CHM_FILE =
HHC_LOCATION =
# The GENERATE_CHI flag controls if a separate .chi index file is generated
# (YES) or that it should be included in the master .chm file (NO).
# (YES) or that it should be included in the main .chm file (NO).
# The default value is: NO.
# This tag requires that the tag GENERATE_HTMLHELP is set to YES.
@@ -1406,7 +1452,8 @@ QCH_FILE =
# The QHP_NAMESPACE tag specifies the namespace to use when generating Qt Help
# Project output. For more information please see Qt Help Project / Namespace
# (see: https://doc.qt.io/archives/qt-4.8/qthelpproject.html#namespace).
# (see:
# https://doc.qt.io/archives/qt-4.8/qthelpproject.html#namespace).
# The default value is: org.doxygen.Project.
# This tag requires that the tag GENERATE_QHP is set to YES.
@@ -1414,8 +1461,8 @@ QHP_NAMESPACE = org.doxygen.Project
# The QHP_VIRTUAL_FOLDER tag specifies the namespace to use when generating Qt
# Help Project output. For more information please see Qt Help Project / Virtual
# Folders (see: https://doc.qt.io/archives/qt-4.8/qthelpproject.html#virtual-
# folders).
# Folders (see:
# https://doc.qt.io/archives/qt-4.8/qthelpproject.html#virtual-folders).
# The default value is: doc.
# This tag requires that the tag GENERATE_QHP is set to YES.
@@ -1423,16 +1470,16 @@ QHP_VIRTUAL_FOLDER = doc
# If the QHP_CUST_FILTER_NAME tag is set, it specifies the name of a custom
# filter to add. For more information please see Qt Help Project / Custom
# Filters (see: https://doc.qt.io/archives/qt-4.8/qthelpproject.html#custom-
# filters).
# Filters (see:
# https://doc.qt.io/archives/qt-4.8/qthelpproject.html#custom-filters).
# This tag requires that the tag GENERATE_QHP is set to YES.
QHP_CUST_FILTER_NAME =
# The QHP_CUST_FILTER_ATTRS tag specifies the list of the attributes of the
# custom filter to add. For more information please see Qt Help Project / Custom
# Filters (see: https://doc.qt.io/archives/qt-4.8/qthelpproject.html#custom-
# filters).
# Filters (see:
# https://doc.qt.io/archives/qt-4.8/qthelpproject.html#custom-filters).
# This tag requires that the tag GENERATE_QHP is set to YES.
QHP_CUST_FILTER_ATTRS =
@@ -1444,9 +1491,9 @@ QHP_CUST_FILTER_ATTRS =
QHP_SECT_FILTER_ATTRS =
# The QHG_LOCATION tag can be used to specify the location of Qt's
# qhelpgenerator. If non-empty doxygen will try to run qhelpgenerator on the
# generated .qhp file.
# The QHG_LOCATION tag can be used to specify the location (absolute path
# including file name) of Qt's qhelpgenerator. If non-empty doxygen will try to
# run qhelpgenerator on the generated .qhp file.
# This tag requires that the tag GENERATE_QHP is set to YES.
QHG_LOCATION =
@@ -1523,6 +1570,17 @@ TREEVIEW_WIDTH = 250
EXT_LINKS_IN_WINDOW = NO
# If the HTML_FORMULA_FORMAT option is set to svg, doxygen will use the pdf2svg
# tool (see https://github.com/dawbarton/pdf2svg) or inkscape (see
# https://inkscape.org) to generate formulas as SVG images instead of PNGs for
# the HTML output. These images will generally look nicer at scaled resolutions.
# Possible values are: png (the default) and svg (looks nicer but requires the
# pdf2svg or inkscape tool).
# The default value is: png.
# This tag requires that the tag GENERATE_HTML is set to YES.
HTML_FORMULA_FORMAT = png
# Use this tag to change the font size of LaTeX formulas included as images in
# the HTML documentation. When you change the font size after a successful
# doxygen run you need to manually remove any form_*.png images from the HTML
@@ -1562,7 +1620,7 @@ USE_MATHJAX = YES
# When MathJax is enabled you can set the default output format to be used for
# the MathJax output. See the MathJax site (see:
# http://docs.mathjax.org/en/latest/output.html) for more details.
# http://docs.mathjax.org/en/v2.7-latest/output.html) for more details.
# Possible values are: HTML-CSS (which is slower, but has the best
# compatibility), NativeMML (i.e. MathML) and SVG.
# The default value is: HTML-CSS.
@@ -1578,7 +1636,7 @@ MATHJAX_FORMAT = HTML-CSS
# Content Delivery Network so you can quickly see the result without installing
# MathJax. However, it is strongly recommended to install a local copy of
# MathJax from https://www.mathjax.org before deployment.
# The default value is: https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/.
# The default value is: https://cdn.jsdelivr.net/npm/mathjax@2.
# This tag requires that the tag USE_MATHJAX is set to YES.
MATHJAX_RELPATH = https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/
@@ -1592,7 +1650,8 @@ MATHJAX_EXTENSIONS =
# The MATHJAX_CODEFILE tag can be used to specify a file with javascript pieces
# of code that will be used on startup of the MathJax code. See the MathJax site
# (see: http://docs.mathjax.org/en/latest/output.html) for more details. For an
# (see:
# http://docs.mathjax.org/en/v2.7-latest/output.html) for more details. For an
# example see the documentation.
# This tag requires that the tag USE_MATHJAX is set to YES.
@@ -1639,7 +1698,8 @@ SERVER_BASED_SEARCH = NO
#
# Doxygen ships with an example indexer (doxyindexer) and search engine
# (doxysearch.cgi) which are based on the open source search engine library
# Xapian (see: https://xapian.org/).
# Xapian (see:
# https://xapian.org/).
#
# See the section "External Indexing and Searching" for details.
# The default value is: NO.
@@ -1652,8 +1712,9 @@ EXTERNAL_SEARCH = NO
#
# Doxygen ships with an example indexer (doxyindexer) and search engine
# (doxysearch.cgi) which are based on the open source search engine library
# Xapian (see: https://xapian.org/). See the section "External Indexing and
# Searching" for details.
# Xapian (see:
# https://xapian.org/). See the section "External Indexing and Searching" for
# details.
# This tag requires that the tag SEARCHENGINE is set to YES.
SEARCHENGINE_URL =
@@ -1817,9 +1878,11 @@ LATEX_EXTRA_FILES =
PDF_HYPERLINKS = YES
# If the USE_PDFLATEX tag is set to YES, doxygen will use pdflatex to generate
# the PDF file directly from the LaTeX files. Set this option to YES, to get a
# higher quality PDF documentation.
# If the USE_PDFLATEX tag is set to YES, doxygen will use the engine as
# specified with LATEX_CMD_NAME to generate the PDF file directly from the LaTeX
# files. Set this option to YES, to get a higher quality PDF documentation.
#
# See also section LATEX_CMD_NAME for selecting the engine.
# The default value is: YES.
# This tag requires that the tag GENERATE_LATEX is set to YES.
@@ -2123,8 +2186,6 @@ MACRO_EXPANSION = YES
EXPAND_ONLY_PREDEF = YES
EXPAND_AS_DEFINED = LIBMDBX_INLINE_API
# If the SEARCH_INCLUDES tag is set to YES, the include files in the
# INCLUDE_PATH will be searched if a #include is found.
# The default value is: YES.
@@ -2332,10 +2393,32 @@ UML_LOOK = NO
# but if the number exceeds 15, the total amount of fields shown is limited to
# 10.
# Minimum value: 0, maximum value: 100, default value: 10.
# This tag requires that the tag HAVE_DOT is set to YES.
# This tag requires that the tag UML_LOOK is set to YES.
UML_LIMIT_NUM_FIELDS = 10
# If the DOT_UML_DETAILS tag is set to NO, doxygen will show attributes and
# methods without types and arguments in the UML graphs. If the DOT_UML_DETAILS
# tag is set to YES, doxygen will add type and arguments for attributes and
# methods in the UML graphs. If the DOT_UML_DETAILS tag is set to NONE, doxygen
# will not generate fields with class member information in the UML graphs. The
# class diagrams will look similar to the default class diagrams but using UML
# notation for the relationships.
# Possible values are: NO, YES and NONE.
# The default value is: NO.
# This tag requires that the tag UML_LOOK is set to YES.
DOT_UML_DETAILS = NO
# The DOT_WRAP_THRESHOLD tag can be used to set the maximum number of characters
# to display on a single line. If the actual line length exceeds this threshold
# significantly it will wrapped across multiple lines. Some heuristics are apply
# to avoid ugly line breaks.
# Minimum value: 0, maximum value: 1000, default value: 17.
# This tag requires that the tag HAVE_DOT is set to YES.
DOT_WRAP_THRESHOLD = 17
# If the TEMPLATE_RELATIONS tag is set to YES then the inheritance and
# collaboration graphs will show the relations between templates and their
# instances.
@@ -2527,9 +2610,11 @@ DOT_MULTI_TARGETS = NO
GENERATE_LEGEND = YES
# If the DOT_CLEANUP tag is set to YES, doxygen will remove the intermediate dot
# If the DOT_CLEANUP tag is set to YES, doxygen will remove the intermediate
# files that are used to generate the various graphs.
#
# Note: This setting is not only used for dot files but also for msc and
# plantuml temporary files.
# The default value is: YES.
# This tag requires that the tag HAVE_DOT is set to YES.
DOT_CLEANUP = YES

View File

@@ -2,118 +2,6 @@ Restrictions & Caveats {#restrictions}
======================
In addition to those listed for some functions.
## Troubleshooting the LCK-file
1. A broken LCK-file can cause sync issues, including appearance of
wrong/inconsistent data for readers. When database opened in the
cooperative read-write mode the LCK-file requires to be mapped to
memory in read-write access. In this case it is always possible for
stray/malfunctioned application could writes thru pointers to
silently corrupt the LCK-file.
Unfortunately, there is no any portable way to prevent such
corruption, since the LCK-file is updated concurrently by
multiple processes in a lock-free manner and any locking is
unwise due to a large overhead.
The "next" version of libmdbx (\ref MithrilDB) will solve this issue.
\note Workaround: Just make all programs using the database close it;
the LCK-file is always reset on first open.
2. Stale reader transactions left behind by an aborted program cause
further writes to grow the database quickly, and stale locks can
block further operation.
MDBX checks for stale readers while opening environment and before
growth the database. But in some cases, this may not be enough.
\note Workaround: Check for stale readers periodically, using the
\ref mdbx_reader_check() function or the mdbx_stat tool.
3. Stale writers will be cleared automatically by MDBX on supported
platforms. But this is platform-specific, especially of
implementation of shared POSIX-mutexes and support for robust
mutexes. For instance there are no known issues on Linux, OSX,
Windows and FreeBSD.
\note Workaround: Otherwise just make all programs using the database
close it; the LCK-file is always reset on first open of the environment.
## Remote filesystems
Do not use MDBX databases on remote filesystems, even between processes
on the same host. This breaks file locks on some platforms, possibly
memory map sync, and certainly sync between programs on different hosts.
On the other hand, MDBX support the exclusive database operation over
a network, and cooperative read-only access to the database placed on
a read-only network shares.
## Child processes
Do not use opened \ref MDBX_env instance(s) in a child processes after `fork()`.
It would be insane to call fork() and any MDBX-functions simultaneously
from multiple threads. The best way is to prevent the presence of open
MDBX-instances during `fork()`.
The \ref MDBX_ENV_CHECKPID build-time option, which is ON by default on
non-Windows platforms (i.e. where `fork()` is available), enables PID
checking at a few critical points. But this does not give any guarantees,
but only allows you to detect such errors a little sooner. Depending on
the platform, you should expect an application crash and/or database
corruption in such cases.
On the other hand, MDBX allow calling \ref mdbx_env_close() in such cases to
release resources, but no more and in general this is a wrong way.
## Read-only mode
There is no pure read-only mode in a normal explicitly way, since
readers need write access to LCK-file to be ones visible for writer.
So MDBX always tries to open/create LCK-file for read-write, but switches
to without-LCK mode on appropriate errors (`EROFS`, `EACCESS`, `EPERM`)
if the read-only mode was requested by the \ref MDBX_RDONLY flag which is
described below.
The "next" version of libmdbx (\ref MithrilDB) will solve this issue for the "many
readers without writer" case.
## One thread - One transaction
A thread can only use one transaction at a time, plus any nested
read-write transactions in the non-writemap mode. Each transaction
belongs to one thread. The \ref MDBX_NOTLS flag changes this for read-only
transactions. See below.
Do not start more than one transaction for a one thread. If you think
about this, it's really strange to do something with two data snapshots
at once, which may be different. MDBX checks and preventing this by
returning corresponding error code (\ref MDBX_TXN_OVERLAPPING, \ref MDBX_BAD_RSLOT,
\ref MDBX_BUSY) unless you using \ref MDBX_NOTLS option on the environment.
Nonetheless, with the `MDBX_NOTLS` option, you must know exactly what you
are doing, otherwise you will get deadlocks or reading an alien data.
## Do not open twice
Do not have open an MDBX database twice in the same process at the same
time. By default MDBX prevent this in most cases by tracking databases
opening and return \ref MDBX_BUSY if anyone LCK-file is already open.
The reason for this is that when the "Open file description" locks (aka
OFD-locks) are not available, MDBX uses POSIX locks on files, and these
locks have issues if one process opens a file multiple times. If a single
process opens the same environment multiple times, closing it once will
remove all the locks held on it, and the other instances will be
vulnerable to corruption from other processes.
For compatibility with LMDB which allows multi-opening, MDBX can be
configured at runtime by `mdbx_setup_debug(MDBX_DBG_LEGACY_MULTIOPEN, ...)`
prior to calling other MDBX functions. In this way MDBX will track
databases opening, detect multi-opening cases and then recover POSIX file
locks as necessary. However, lock recovery can cause unexpected pauses,
such as when another process opened the database in exclusive mode before
the lock was restored - we have to wait until such a process releases the
database, and so on.
## Long-lived read transactions {#long-lived-read}
Avoid long-lived read transactions, especially in the scenarios with a
@@ -162,6 +50,7 @@ The "next" version of libmdbx (\ref MithrilDB) will completely solve this.
not apply to write transactions if the system clears stale writers, see
above.
## Large data items and huge transactions
MDBX allows you to store values up to 1 gigabyte in size, but this is
@@ -191,10 +80,123 @@ list of pages to be retired.
Both of these issues will be addressed in MithrilDB.
## Space reservation
An MDBX database configuration will often reserve considerable unused
memory address space and maybe file size for future growth. This does
not use actual memory or disk space, but users may need to understand
the difference so they won't be scared off.
\todo To write about the Read/Write Amplification Factors
## Remote filesystems
Do not use MDBX databases on remote filesystems, even between processes
on the same host. This breaks file locks on some platforms, possibly
memory map sync, and certainly sync between programs on different hosts.
On the other hand, MDBX support the exclusive database operation over
a network, and cooperative read-only access to the database placed on
a read-only network shares.
## Child processes
Do not use opened \ref MDBX_env instance(s) in a child processes after `fork()`.
It would be insane to call fork() and any MDBX-functions simultaneously
from multiple threads. The best way is to prevent the presence of open
MDBX-instances during `fork()`.
The \ref MDBX_ENV_CHECKPID build-time option, which is ON by default on
non-Windows platforms (i.e. where `fork()` is available), enables PID
checking at a few critical points. But this does not give any guarantees,
but only allows you to detect such errors a little sooner. Depending on
the platform, you should expect an application crash and/or database
corruption in such cases.
On the other hand, MDBX allow calling \ref mdbx_env_close() in such cases to
release resources, but no more and in general this is a wrong way.
## Read-only mode
There is no pure read-only mode in a normal explicitly way, since
readers need write access to LCK-file to be ones visible for writer.
So MDBX always tries to open/create LCK-file for read-write, but switches
to without-LCK mode on appropriate errors (`EROFS`, `EACCESS`, `EPERM`)
if the read-only mode was requested by the \ref MDBX_RDONLY flag which is
described below.
The "next" version of libmdbx (\ref MithrilDB) will solve this issue for the "many
readers without writer" case.
## Troubleshooting the LCK-file
1. A broken LCK-file can cause sync issues, including appearance of
wrong/inconsistent data for readers. When database opened in the
cooperative read-write mode the LCK-file requires to be mapped to
memory in read-write access. In this case it is always possible for
stray/malfunctioned application could writes thru pointers to
silently corrupt the LCK-file.
Unfortunately, there is no any portable way to prevent such
corruption, since the LCK-file is updated concurrently by
multiple processes in a lock-free manner and any locking is
unwise due to a large overhead.
The "next" version of libmdbx (\ref MithrilDB) will solve this issue.
\note Workaround: Just make all programs using the database close it;
the LCK-file is always reset on first open.
2. Stale reader transactions left behind by an aborted program cause
further writes to grow the database quickly, and stale locks can
block further operation.
MDBX checks for stale readers while opening environment and before
growth the database. But in some cases, this may not be enough.
\note Workaround: Check for stale readers periodically, using the
\ref mdbx_reader_check() function or the mdbx_stat tool.
3. Stale writers will be cleared automatically by MDBX on supported
platforms. But this is platform-specific, especially of
implementation of shared POSIX-mutexes and support for robust
mutexes. For instance there are no known issues on Linux, OSX,
Windows and FreeBSD.
\note Workaround: Otherwise just make all programs using the database
close it; the LCK-file is always reset on first open of the environment.
## One thread - One transaction
A thread can only use one transaction at a time, plus any nested
read-write transactions in the non-writemap mode. Each transaction
belongs to one thread. The \ref MDBX_NOTLS flag changes this for read-only
transactions. See below.
Do not start more than one transaction for a one thread. If you think
about this, it's really strange to do something with two data snapshots
at once, which may be different. MDBX checks and preventing this by
returning corresponding error code (\ref MDBX_TXN_OVERLAPPING, \ref MDBX_BAD_RSLOT,
\ref MDBX_BUSY) unless you using \ref MDBX_NOTLS option on the environment.
Nonetheless, with the `MDBX_NOTLS` option, you must know exactly what you
are doing, otherwise you will get deadlocks or reading an alien data.
## Do not open twice
Do not have open an MDBX database twice in the same process at the same
time. By default MDBX prevent this in most cases by tracking databases
opening and return \ref MDBX_BUSY if anyone LCK-file is already open.
The reason for this is that when the "Open file description" locks (aka
OFD-locks) are not available, MDBX uses POSIX locks on files, and these
locks have issues if one process opens a file multiple times. If a single
process opens the same environment multiple times, closing it once will
remove all the locks held on it, and the other instances will be
vulnerable to corruption from other processes.
For compatibility with LMDB which allows multi-opening, MDBX can be
configured at runtime by `mdbx_setup_debug(MDBX_DBG_LEGACY_MULTIOPEN, ...)`
prior to calling other MDBX functions. In this way MDBX will track
databases opening, detect multi-opening cases and then recover POSIX file
locks as necessary. However, lock recovery can cause unexpected pauses,
such as when another process opened the database in exclusive mode before
the lock was restored - we have to wait until such a process releases the
database, and so on.

View File

@@ -1,4 +1,3 @@
_The Future will (be) [Positive](https://www.ptsecurity.com). Всё будет хорошо._
\section toc Table of Contents
@@ -23,7 +22,7 @@ each of which is divided into several sections.
- The \ref mdbx.h++ header file reference
Please do not hesitate to point out errors in the documentation,
including creating [PR](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/proposing-changes-to-your-work-with-pull-requests) with corrections and improvements.
including creating [merge-request](https://gitflic.ru/project/erthink/libmdbx/merge-request) with corrections and improvements.
---

127
mdbx.h
View File

@@ -14,6 +14,12 @@ break down. _libmdbx_ supports Linux, Windows, MacOS, OSX, iOS, Android,
FreeBSD, DragonFly, Solaris, OpenSolaris, OpenIndiana, NetBSD, OpenBSD and other
systems compliant with POSIX.1-2008.
The origin has been migrated to
[GitFlic](https://gitflic.ru/project/erthink/libmdbx) since on 2022-04-15
the Github administration, without any warning nor explanation, deleted libmdbx
along with a lot of other projects, simultaneously blocking access for many
developers. For the same reason ~~Github~~ is blacklisted forever.
_The Future will (be) [Positive](https://www.ptsecurity.com). Всё будет хорошо._
@@ -324,7 +330,8 @@ typedef mode_t mdbx_mode_t;
#endif /* MDBX_DEPRECATED */
#ifndef __dll_export
#if defined(_WIN32) || defined(__CYGWIN__)
#if defined(_WIN32) || defined(_WIN64) || defined(__CYGWIN__) || \
defined(__MINGW__) || defined(__MINGW32__) || defined(__MINGW64__)
#if defined(__GNUC__) || __has_attribute(__dllexport__)
#define __dll_export __attribute__((__dllexport__))
#elif defined(_MSC_VER)
@@ -340,7 +347,8 @@ typedef mode_t mdbx_mode_t;
#endif /* __dll_export */
#ifndef __dll_import
#if defined(_WIN32) || defined(__CYGWIN__)
#if defined(_WIN32) || defined(_WIN64) || defined(__CYGWIN__) || \
defined(__MINGW__) || defined(__MINGW32__) || defined(__MINGW64__)
#if defined(__GNUC__) || __has_attribute(__dllimport__)
#define __dll_import __attribute__((__dllimport__))
#elif defined(_MSC_VER)
@@ -457,8 +465,13 @@ typedef mode_t mdbx_mode_t;
#ifndef MDBX_PRINTF_ARGS
#if defined(__GNUC__) || __has_attribute(__format__)
#if defined(__MINGW__) || defined(__MINGW32__) || defined(__MINGW64__)
#define MDBX_PRINTF_ARGS(format_index, first_arg) \
__attribute__((__format__(__gnu_printf__, format_index, first_arg)))
#else
#define MDBX_PRINTF_ARGS(format_index, first_arg) \
__attribute__((__format__(__printf__, format_index, first_arg)))
#endif /* MinGW */
#else
#define MDBX_PRINTF_ARGS(format_index, first_arg)
#endif
@@ -864,18 +877,23 @@ enum MDBX_debug_flags_t {
MDBX_DBG_JITTER = 4,
/** Include or not meta-pages in coredump files.
* May affect performance in \ref MDBX_WRITEMAP mode */
* \note May affect performance in \ref MDBX_WRITEMAP mode */
MDBX_DBG_DUMP = 8,
/** Allow multi-opening environment(s) */
MDBX_DBG_LEGACY_MULTIOPEN = 16,
/** Allow read and write transactions overlapping for the same thread */
/** Allow read and write transactions overlapping for the same thread. */
MDBX_DBG_LEGACY_OVERLAP = 32,
/** Don't auto-upgrade format signature.
* \note However a new write transactions will use and store
* the last signature regardless this flag */
MDBX_DBG_DONT_UPGRADE = 64,
#ifdef ENABLE_UBSAN
MDBX_DBG_MAX = ((unsigned)MDBX_LOG_MAX) << 16 |
63 /* avoid UBSAN false-positive trap by a tests */,
127 /* avoid UBSAN false-positive trap by a tests */,
#endif /* ENABLE_UBSAN */
/** for mdbx_setup_debug() only: Don't change current settings */
@@ -1403,6 +1421,7 @@ DEFINE_ENUM_FLAG_OPERATORS(MDBX_txn_flags_t)
* \anchor db_flags
* \see mdbx_dbi_open() */
enum MDBX_db_flags_t {
/** Variable length unique keys with usual byte-by-byte string comparison. */
MDBX_DB_DEFAULTS = 0,
/** Use reverse string comparison for keys. */
@@ -2098,19 +2117,19 @@ LIBMDBX_API int mdbx_env_get_option(const MDBX_env *env,
* the database files reside. In the case of directory it
* must already exist and be writable.
*
* \param [in] flags Special options for this environment. This parameter
* must be set to 0 or by bitwise OR'ing together one
* or more of the values described above in the
* \ref env_flags and \ref sync_modes sections.
* \param [in] flags Specifies options for this environment.
* This parameter must be bitwise OR'ing together
* any constants described above in the \ref env_flags
* and \ref sync_modes sections.
*
* Flags set by mdbx_env_set_flags() are also used:
* - \ref MDBX_NOSUBDIR, \ref MDBX_RDONLY, \ref MDBX_EXCLUSIVE,
* \ref MDBX_WRITEMAP, \ref MDBX_NOTLS, \ref MDBX_NORDAHEAD,
* \ref MDBX_NOMEMINIT, \ref MDBX_COALESCE, \ref MDBX_LIFORECLAIM.
* See \ref env_flags section.
* - \ref MDBX_ENV_DEFAULTS, \ref MDBX_NOSUBDIR, \ref MDBX_RDONLY,
* \ref MDBX_EXCLUSIVE, \ref MDBX_WRITEMAP, \ref MDBX_NOTLS,
* \ref MDBX_NORDAHEAD, \ref MDBX_NOMEMINIT, \ref MDBX_COALESCE,
* \ref MDBX_LIFORECLAIM. See \ref env_flags section.
*
* - \ref MDBX_NOMETASYNC, \ref MDBX_SAFE_NOSYNC, \ref MDBX_UTTERLY_NOSYNC.
* See \ref sync_modes section.
* - \ref MDBX_SYNC_DURABLE, \ref MDBX_NOMETASYNC, \ref MDBX_SAFE_NOSYNC,
* \ref MDBX_UTTERLY_NOSYNC. See \ref sync_modes section.
*
* \note `MDB_NOLOCK` flag don't supported by MDBX,
* try use \ref MDBX_EXCLUSIVE as a replacement.
@@ -2187,8 +2206,8 @@ typedef enum MDBX_env_delete_mode_t MDBX_env_delete_mode_t;
* \param [in] pathname The pathname for the database or the directory in which
* the database files reside.
*
* \param [in] mode Special deletion mode for the environment. This
* parameter must be set to one of the values described
* \param [in] mode Specifies deletion mode for the environment. This
* parameter must be set to one of the constants described
* above in the \ref MDBX_env_delete_mode_t section.
*
* \note The \ref MDBX_ENV_JUST_DELETE don't supported on Windows since system
@@ -2215,9 +2234,12 @@ LIBMDBX_API int mdbx_env_delete(const char *pathname,
* \param [in] dest The pathname of a file in which the copy will reside.
* This file must not be already exist, but parent directory
* must be writable.
* \param [in] flags Special options for this operation. This parameter must
* be set to 0 or by bitwise OR'ing together one or more
* of the values described here:
* \param [in] flags Specifies options for this operation. This parameter
* must be bitwise OR'ing together any of the constants
* described here:
*
* - \ref MDBX_CP_DEFAULTS
* Perform copy as-is without compaction, etc.
*
* - \ref MDBX_CP_COMPACT
* Perform compaction while copying: omit free pages and sequentially
@@ -2875,10 +2897,15 @@ LIBMDBX_API int mdbx_env_get_fd(const MDBX_env *env, mdbx_filehandle_t *fd);
* some possible errors are:
* \retval MDBX_EINVAL An invalid parameter was specified,
* or the environment has an active write transaction.
* \retval MDBX_EPERM Specific for Windows: Shrinking was disabled before
* and now it wanna be enabled, but there are reading
* threads that don't use the additional `SRWL` (that
* is required to avoid Windows issues).
* \retval MDBX_EPERM Two specific cases for Windows:
* 1) Shrinking was disabled before via geometry settings
* and now it enabled, but there are reading threads that
* don't use the additional `SRWL` (which is required to
* avoid Windows issues).
* 2) Temporary close memory mapped is required to change
* geometry, but there read transaction(s) is running
* and no corresponding thread(s) could be suspended
* since the \ref MDBX_NOTLS mode is used.
* \retval MDBX_EACCESS The environment opened in read-only.
* \retval MDBX_MAP_FULL Specified size smaller than the space already
* consumed by the environment.
@@ -3667,12 +3694,14 @@ typedef int(MDBX_cmp_func)(const MDBX_val *a,
* database is needed in the environment,
* this value may be NULL.
* \param [in] flags Special options for this database. This parameter must
* be set to 0 or by bitwise OR'ing together one or more
* of the values described here:
* be bitwise OR'ing together any of the constants
* described here:
*
* - \ref MDBX_DB_DEFAULTS
* Keys are arbitrary byte strings and compared from beginning to end.
* - \ref MDBX_REVERSEKEY
* Keys are strings to be compared in reverse order, from the end
* of the strings to the beginning. By default, Keys are treated as
* strings and compared from beginning to end.
* Keys are arbitrary byte strings to be compared in reverse order,
* from the end of the strings to the beginning.
* - \ref MDBX_INTEGERKEY
* Keys are binary integers in native byte order, either uint32_t or
* uint64_t, and will be sorted as such. The keys must all be of the
@@ -4803,26 +4832,26 @@ mdbx_get_datacmp(MDBX_db_flags_t flags);
/** \brief A callback function used to enumerate the reader lock table.
* \ingroup c_statinfo
*
* \param [in] ctx An arbitrary context pointer for the callback.
* \param [in] num The serial number during enumeration,
* starting from 1.
* \param [in] slot The reader lock table slot number.
* \param [in] txnid The ID of the transaction being read,
* i.e. the MVCC-snapshot number.
* \param [in] lag The lag from a recent MVCC-snapshot,
* i.e. the number of committed write transactions
* since the current read transaction started.
* \param [in] pid The reader process ID.
* \param [in] thread The reader thread ID.
* \param [in] bytes_used The number of last used page in the MVCC-snapshot
* which being read,
* i.e. database file can't shrinked beyond this.
* \param [in] bytes_retired The total size of the database pages that were
* retired by committed write transactions after
* the reader's MVCC-snapshot,
* i.e. the space which would be freed after
* the Reader releases the MVCC-snapshot
* for reuse by completion read transaction.
* \param [in] ctx An arbitrary context pointer for the callback.
* \param [in] num The serial number during enumeration,
* starting from 1.
* \param [in] slot The reader lock table slot number.
* \param [in] txnid The ID of the transaction being read,
* i.e. the MVCC-snapshot number.
* \param [in] lag The lag from a recent MVCC-snapshot,
* i.e. the number of committed write transactions
* since the current read transaction started.
* \param [in] pid The reader process ID.
* \param [in] thread The reader thread ID.
* \param [in] bytes_used The number of last used page
* in the MVCC-snapshot which being read,
* i.e. database file can't shrinked beyond this.
* \param [in] bytes_retained The total size of the database pages that were
* retired by committed write transactions after
* the reader's MVCC-snapshot,
* i.e. the space which would be freed after
* the Reader releases the MVCC-snapshot
* for reuse by completion read transaction.
*
* \returns < 0 on failure, >= 0 on success. \see mdbx_reader_list() */
typedef int(MDBX_reader_list_func)(void *ctx, int num, int slot, mdbx_pid_t pid,

129
mdbx.h++
View File

@@ -13,6 +13,14 @@
/// - AppleClang, but without C++20 concepts.
///
///
/// The origin has been migrated to https://gitflic.ru/project/erthink/libmdbx
/// since on 2022-04-15 the Github administration, without any warning nor
/// explanation, deleted libmdbx along with a lot of other projects,
/// simultaneously blocking access for many developers.
/// For the same reason Github is blacklisted forever.
///
#pragma once
/* Workaround for modern libstdc++ with CLANG < 4.x */
@@ -72,6 +80,8 @@
#if defined(__cpp_lib_filesystem) && __cpp_lib_filesystem >= 201703L
#include <filesystem>
#elif __has_include(<experimental/filesystem>)
#include <experimental/filesystem>
#endif
#include "mdbx.h"
@@ -199,14 +209,23 @@
#endif
#endif /* MDBX_CXX20_UNLIKELY */
#ifndef MDBX_CXX20_CONCEPT
#ifndef MDBX_HAVE_CXX20_CONCEPTS
#if defined(DOXYGEN) || \
(defined(__cpp_concepts) && __cpp_concepts >= 201907L && \
(!defined(__clang__) || \
(__clang_major__ >= 12 && !defined(__APPLE__) && \
!defined(__ANDROID_API__)) || \
__clang_major__ >= \
/* Hope Apple will fix concepts in AppleClang 14 */ 14))
(!defined(__clang__) || __has_include(<concepts>) || \
(defined(__cpp_lib_concepts) && __cpp_lib_concepts >= 202002L)))
#if __has_include(<concepts>) || \
(defined(__cpp_lib_concepts) && __cpp_lib_concepts >= 202002L)
#include <concepts>
#endif /* <concepts> */
#define MDBX_HAVE_CXX20_CONCEPTS 1
#else
#define MDBX_HAVE_CXX20_CONCEPTS 0
#endif
#endif /* MDBX_HAVE_CXX20_CONCEPTS */
#ifndef MDBX_CXX20_CONCEPT
#if MDBX_HAVE_CXX20_CONCEPTS
#define MDBX_CXX20_CONCEPT(CONCEPT, NAME) CONCEPT NAME
#else
#define MDBX_CXX20_CONCEPT(CONCEPT, NAME) typename NAME
@@ -214,13 +233,7 @@
#endif /* MDBX_CXX20_CONCEPT */
#ifndef MDBX_ASSERT_CXX20_CONCEPT_SATISFIED
#if defined(DOXYGEN) || \
(defined(__cpp_concepts) && __cpp_concepts >= 201907L && \
(!defined(__clang__) || \
(__clang_major__ >= 12 && !defined(__APPLE__) && \
!defined(__ANDROID_API__)) || \
__clang_major__ >= \
/* Hope Apple will fix concepts in AppleClang 14 */ 14))
#if MDBX_HAVE_CXX20_CONCEPTS
#define MDBX_ASSERT_CXX20_CONCEPT_SATISFIED(CONCEPT, TYPE) \
static_assert(CONCEPT<TYPE>)
#else
@@ -254,7 +267,7 @@ namespace mdbx {
// To enable all kinds of an compiler optimizations we use a byte-like type
// that don't presumes aliases for pointers as does the `char` type and its
// derivatives/typedefs.
// Please see https://github.com/erthink/libmdbx/issues/263
// Please see todo4recovery://erased_by_github/libmdbx/issues/263
// for reasoning of the use of `char8_t` type and switching to `__restrict__`.
using byte = char8_t;
#else
@@ -323,13 +336,21 @@ using filehandle = ::mdbx_filehandle_t;
__MAC_OS_X_VERSION_MIN_REQUIRED >= 101500) && \
(!defined(__IPHONE_OS_VERSION_MIN_REQUIRED) || \
__IPHONE_OS_VERSION_MIN_REQUIRED >= 130100))
#define MDBX_STD_FILESYSTEM_PATH
using path = ::std::filesystem::path;
namespace filesystem = ::std::filesystem;
#define MDBX_STD_FILESYSTEM_PATH ::mdbx::filesystem::path
#elif defined(__cpp_lib_experimental_filesystem) && \
__cpp_lib_experimental_filesystem >= 201406L
namespace filesystem = ::std::experimental::filesystem;
#define MDBX_STD_FILESYSTEM_PATH ::mdbx::filesystem::path
#endif /* MDBX_STD_FILESYSTEM_PATH */
#ifdef MDBX_STD_FILESYSTEM_PATH
using path = MDBX_STD_FILESYSTEM_PATH;
#elif defined(_WIN32) || defined(_WIN64)
using path = ::std::wstring;
#else
using path = ::std::string;
#endif
#endif /* mdbx::path */
/// \brief Transfers C++ exceptions thru C callbacks.
/// \details Implements saving exceptions before returning
@@ -491,13 +512,7 @@ static MDBX_CXX20_CONSTEXPR void *memcpy(void *dest, const void *src,
size_t bytes) noexcept;
//------------------------------------------------------------------------------
#if defined(DOXYGEN) || \
(defined(__cpp_concepts) && __cpp_concepts >= 201907L && \
(!defined(__clang__) || \
(__clang_major__ >= 12 && !defined(__APPLE__) && \
!defined(__ANDROID_API__)) || \
__clang_major__ >= \
/* Hope Apple will fix concepts in AppleClang 14 */ 14))
#if MDBX_HAVE_CXX20_CONCEPTS
template <typename T>
concept MutableByteProducer = requires(T a, char array[42]) {
@@ -520,7 +535,7 @@ concept SliceTranscoder = ImmutableByteProducer<T> &&
{ a.is_erroneous() } -> std::same_as<bool>;
};
#endif /* __cpp_concepts >= 201907L*/
#endif /* MDBX_HAVE_CXX20_CONCEPTS */
template <class ALLOCATOR = legacy_allocator,
typename CAPACITY_POLICY = default_capacity_policy,
@@ -2960,6 +2975,18 @@ public:
inline geometry &make_fixed(intptr_t size) noexcept;
inline geometry &make_dynamic(intptr_t lower = minimal_value,
intptr_t upper = maximal_value) noexcept;
MDBX_CXX11_CONSTEXPR geometry() noexcept {}
MDBX_CXX11_CONSTEXPR
geometry(const geometry &) noexcept = default;
MDBX_CXX11_CONSTEXPR geometry(intptr_t size_lower,
intptr_t size_now = default_value,
intptr_t size_upper = maximal_value,
intptr_t growth_step = default_value,
intptr_t shrink_threshold = default_value,
intptr_t pagesize = default_value) noexcept
: size_lower(size_lower), size_now(size_now), size_upper(size_upper),
growth_step(growth_step), shrink_threshold(shrink_threshold),
pagesize(pagesize) {}
};
/// \brief Operation mode.
@@ -2984,6 +3011,10 @@ public:
/// \copydoc MDBX_COALESCE
bool coalesce{false};
MDBX_CXX11_CONSTEXPR reclaiming_options() noexcept {}
MDBX_CXX11_CONSTEXPR
reclaiming_options(const reclaiming_options &) noexcept = default;
MDBX_CXX14_CONSTEXPR reclaiming_options &
operator=(const reclaiming_options &) noexcept = default;
reclaiming_options(MDBX_env_flags_t) noexcept;
};
@@ -2999,6 +3030,10 @@ public:
/// \copydoc MDBX_NOMEMINIT
bool disable_clear_memory{false};
MDBX_CXX11_CONSTEXPR operate_options() noexcept {}
MDBX_CXX11_CONSTEXPR
operate_options(const operate_options &) noexcept = default;
MDBX_CXX14_CONSTEXPR operate_options &
operator=(const operate_options &) noexcept = default;
operate_options(MDBX_env_flags_t) noexcept;
};
@@ -3016,6 +3051,19 @@ public:
env::operate_options options;
MDBX_CXX11_CONSTEXPR operate_parameters() noexcept {}
MDBX_CXX11_CONSTEXPR
operate_parameters(
const unsigned max_maps, const unsigned max_readers = 0,
const env::mode mode = env::mode::write_mapped_io,
env::durability durability = env::durability::robust_synchronous,
const env::reclaiming_options &reclaiming = env::reclaiming_options(),
const env::operate_options &options = env::operate_options()) noexcept
: max_maps(max_maps), max_readers(max_readers), mode(mode),
durability(durability), reclaiming(reclaiming), options(options) {}
MDBX_CXX11_CONSTEXPR
operate_parameters(const operate_parameters &) noexcept = default;
MDBX_CXX14_CONSTEXPR operate_parameters &
operator=(const operate_parameters &) noexcept = default;
MDBX_env_flags_t
make_flags(bool accede = true, ///< \copydoc MDBX_ACCEDE
bool use_subdirectory =
@@ -3027,7 +3075,6 @@ public:
reclaiming_from_flags(MDBX_env_flags_t flags) noexcept;
inline static env::operate_options
options_from_flags(MDBX_env_flags_t flags) noexcept;
operate_parameters(const env &);
};
/// \brief Returns current operation parameters.
@@ -3130,7 +3177,7 @@ public:
/// \brief Make a copy (backup) of an existing environment to the specified
/// path.
#ifdef MDBX_STD_FILESYSTEM_PATH
env &copy(const ::std::filesystem::path &destination, bool compactify,
env &copy(const MDBX_STD_FILESYSTEM_PATH &destination, bool compactify,
bool force_dynamic_size = false);
#endif /* MDBX_STD_FILESYSTEM_PATH */
#if defined(_WIN32) || defined(_WIN64)
@@ -3163,7 +3210,7 @@ public:
/// \brief Removes the environment's files in a proper and multiprocess-safe
/// way.
#ifdef MDBX_STD_FILESYSTEM_PATH
static bool remove(const ::std::filesystem::path &,
static bool remove(const MDBX_STD_FILESYSTEM_PATH &,
const remove_mode mode = just_remove);
#endif /* MDBX_STD_FILESYSTEM_PATH */
#if defined(_WIN32) || defined(_WIN64)
@@ -3408,7 +3455,7 @@ public:
/// \brief Open existing database.
#ifdef MDBX_STD_FILESYSTEM_PATH
env_managed(const ::std::filesystem::path &, const operate_parameters &,
env_managed(const MDBX_STD_FILESYSTEM_PATH &, const operate_parameters &,
bool accede = true);
#endif /* MDBX_STD_FILESYSTEM_PATH */
#if defined(_WIN32) || defined(_WIN64)
@@ -3423,11 +3470,13 @@ public:
env::geometry geometry;
mdbx_mode_t file_mode_bits{0640};
bool use_subdirectory{false};
MDBX_CXX11_CONSTEXPR create_parameters() noexcept = default;
create_parameters(const create_parameters &) noexcept = default;
};
/// \brief Create new or open existing database.
#ifdef MDBX_STD_FILESYSTEM_PATH
env_managed(const ::std::filesystem::path &, const create_parameters &,
env_managed(const MDBX_STD_FILESYSTEM_PATH &, const create_parameters &,
const operate_parameters &, bool accede = true);
#endif /* MDBX_STD_FILESYSTEM_PATH */
#if defined(_WIN32) || defined(_WIN64)
@@ -4037,8 +4086,8 @@ static MDBX_CXX17_CONSTEXPR size_t strlen(const char *c_str) noexcept {
#endif
}
static MDBX_CXX20_CONSTEXPR void *memcpy(void *dest, const void *src,
size_t bytes) noexcept {
MDBX_MAYBE_UNUSED static MDBX_CXX20_CONSTEXPR void *
memcpy(void *dest, const void *src, size_t bytes) noexcept {
#if defined(__cpp_lib_is_constant_evaluated) && \
__cpp_lib_is_constant_evaluated >= 201811L
if (::std::is_constant_evaluated()) {
@@ -4078,8 +4127,8 @@ static MDBX_CXX14_CONSTEXPR size_t check_length(size_t headroom,
return check_length(check_length(headroom) + check_length(payload));
}
static MDBX_CXX14_CONSTEXPR size_t check_length(size_t headroom, size_t payload,
size_t tailroom) {
MDBX_MAYBE_UNUSED static MDBX_CXX14_CONSTEXPR size_t
check_length(size_t headroom, size_t payload, size_t tailroom) {
return check_length(check_length(headroom, payload) + check_length(tailroom));
}
@@ -4763,7 +4812,12 @@ inline size_t env::limits::transaction_size_max(intptr_t pagesize) {
}
inline env::operate_parameters env::get_operation_parameters() const {
return env::operate_parameters(*this);
const auto flags = get_flags();
return operate_parameters(max_maps(), max_readers(),
operate_parameters::mode_from_flags(flags),
operate_parameters::durability_from_flags(flags),
operate_parameters::reclaiming_from_flags(flags),
operate_parameters::options_from_flags(flags));
}
inline env::mode env::get_mode() const {
@@ -5424,6 +5478,7 @@ inline size_t txn::put_multiple(map_handle map, const slice &key,
if (allow_partial)
break;
mdbx_txn_break(handle_);
MDBX_CXX17_FALLTHROUGH /* fallthrough */;
default:
MDBX_CXX20_UNLIKELY error::throw_exception(err);
}
@@ -5628,7 +5683,7 @@ inline cursor::move_result cursor::move(move_operation operation,
inline cursor::move_result cursor::find_multivalue(const slice &key,
const slice &value,
bool throw_notfound) {
return move(key_exact, key, value, throw_notfound);
return move(multi_find_pair, key, value, throw_notfound);
}
inline cursor::move_result cursor::lower_bound_multivalue(const slice &key,
@@ -5819,7 +5874,7 @@ inline bool cursor::erase(const slice &key, bool whole_multivalue) {
inline bool cursor::erase(const slice &key, const slice &value) {
move_result data = find_multivalue(key, value, false);
return data.done ? erase() : data.done;
return data.done && erase();
}
} // namespace mdbx

View File

@@ -9,7 +9,7 @@ This patch adds libmdbx v0.11.1:
focused on creating unique lightweight solutions.
- libmdbx surpasses the legendary LMDB (Lightning Memory-Mapped Database)
in terms of reliability, features and performance.
- https://github.com/erthink/libmdbx
- https://gitflic.ru/project/erthink/libmdbx
Signed-off-by: Leonid Yuriev <leo@yuriev.ru>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
@@ -71,7 +71,7 @@ index 0000000000..d13f73938f
+ libmdbx surpasses the legendary LMDB in terms of
+ reliability, features and performance.
+
+ https://github.com/erthink/libmdbx
+ https://gitflic.ru/project/erthink/libmdbx
+
+if BR2_PACKAGE_LIBMDBX
+
@@ -107,7 +107,7 @@ index 0000000000..c8b50f9ac3
--- /dev/null
+++ b/package/libmdbx/libmdbx.hash
@@ -0,0 +1,5 @@
+# Hashes from: https://github.com/erthink/libmdbx/releases/
+# Hashes from: https://gitflic.ru/project/erthink/libmdbx/releases/
+sha256 f954ba8c9768914a92c2b46aac0d66bec674dbb4d7b0f01e362ea2921746ddaa libmdbx-amalgamated-0.11.1.tar.gz
+
+# Locally calculated
@@ -126,7 +126,7 @@ index 0000000000..02d00b1a5a
+
+LIBMDBX_VERSION = 0.11.1
+LIBMDBX_SOURCE = libmdbx-amalgamated-$(LIBMDBX_VERSION).tar.gz
+LIBMDBX_SITE = https://github.com/erthink/libmdbx/releases/download/v$(LIBMDBX_VERSION)
+LIBMDBX_SITE = https://gitflic.ru/project/erthink/libmdbx/releases/download/v$(LIBMDBX_VERSION)
+LIBMDBX_SUPPORTS_IN_SOURCE_BUILD = NO
+LIBMDBX_LICENSE = OLDAP-2.8
+LIBMDBX_LICENSE_FILES = LICENSE

File diff suppressed because it is too large Load Diff

View File

@@ -107,6 +107,11 @@
# define __noop(...) do {} while(0)
#endif /* __noop */
#if defined(__fallthrough) && \
(defined(__MINGW__) || defined(__MINGW32__) || defined(__MINGW64__))
#undef __fallthrough
#endif /* __fallthrough workaround for MinGW */
#ifndef __fallthrough
# if defined(__cplusplus) && (__has_cpp_attribute(fallthrough) && \
(!defined(__clang__) || __clang__ > 4)) || __cplusplus >= 201703L

View File

@@ -47,7 +47,8 @@
/*----------------------------------------------------------------------------*/
/* Should be defined before any includes */
#ifndef _FILE_OFFSET_BITS
#if !defined(_FILE_OFFSET_BITS) && !defined(__ANDROID_API__) && \
!defined(ANDROID)
#define _FILE_OFFSET_BITS 64
#endif
@@ -88,6 +89,11 @@
#pragma warning(disable : 5045) /* Compiler will insert Spectre mitigation... \
*/
#endif
#if _MSC_VER > 1914
#pragma warning( \
disable : 5105) /* winbase.h(9531): warning C5105: macro expansion \
producing 'defined' has undefined behavior */
#endif
#pragma warning(disable : 4710) /* 'xyz': function not inlined */
#pragma warning(disable : 4711) /* function 'xyz' selected for automatic \
inline expansion */
@@ -117,6 +123,11 @@
#pragma GCC diagnostic ignored "-Wattributes"
#endif /* GCC < 9 */
#if (defined(__MINGW__) || defined(__MINGW32__) || defined(__MINGW64__)) && \
!defined(__USE_MINGW_ANSI_STDIO)
#define __USE_MINGW_ANSI_STDIO 1
#endif /* __USE_MINGW_ANSI_STDIO */
#include "../mdbx.h"
#include "defs.h"
@@ -945,6 +956,7 @@ struct MDBX_txn {
size_t mt_owner; /* thread ID that owns this transaction */
MDBX_canary mt_canary;
void *mt_userctx; /* User-settable context */
MDBX_cursor **mt_cursors;
union {
struct {
@@ -953,7 +965,6 @@ struct MDBX_txn {
} to;
struct {
/* In write txns, array of cursors for each DB */
MDBX_cursor **cursors;
pgno_t *reclaimed_pglist; /* Reclaimed GC pages */
txnid_t last_reclaimed; /* ID of last used record */
#if MDBX_ENABLE_REFUND

View File

@@ -175,14 +175,35 @@ __cold static void choice_fcntl() {
#ifndef OFF_T_MAX
#define OFF_T_MAX \
((sizeof(off_t) > 4 ? INT64_MAX : INT32_MAX) & ~(size_t)0xffff)
(((sizeof(off_t) > 4) ? INT64_MAX : INT32_MAX) & ~(size_t)0xffff)
#endif
static int lck_op(mdbx_filehandle_t fd, int cmd, int lck, off_t offset,
off_t len) {
static int lck_op(const mdbx_filehandle_t fd, int cmd, const int lck,
const off_t offset, off_t len) {
STATIC_ASSERT(sizeof(off_t) >= sizeof(void *) &&
sizeof(off_t) >= sizeof(size_t));
#ifdef __ANDROID_API__
STATIC_ASSERT_MSG((sizeof(off_t) * 8 == MDBX_WORDBITS),
"The bitness of system `off_t` type is mismatch. Please "
"fix build and/or NDK configuration.");
#endif /* Android */
mdbx_jitter4testing(true);
assert(offset >= 0 && len > 0);
assert((uint64_t)offset < (uint64_t)INT64_MAX &&
(uint64_t)len < (uint64_t)INT64_MAX &&
(uint64_t)(offset + len) > (uint64_t)offset);
assert((uint64_t)offset < (uint64_t)OFF_T_MAX &&
(uint64_t)len <= (uint64_t)OFF_T_MAX &&
(uint64_t)(offset + len) <= (uint64_t)OFF_T_MAX);
assert((uint64_t)((off_t)((uint64_t)offset + (uint64_t)len)) ==
((uint64_t)offset + (uint64_t)len));
for (;;) {
struct flock lock_op;
STATIC_ASSERT(sizeof(off_t) <= sizeof(lock_op.l_start) &&
sizeof(off_t) <= sizeof(lock_op.l_len) &&
OFF_T_MAX == (off_t)OFF_T_MAX);
memset(&lock_op, 0, sizeof(lock_op));
lock_op.l_type = lck;
lock_op.l_whence = SEEK_SET;
@@ -218,7 +239,7 @@ static int lck_op(mdbx_filehandle_t fd, int cmd, int lck, off_t offset,
}
#endif /* MDBX_USE_OFDLOCKS */
if (rc != EINTR || cmd == op_setlkw) {
mdbx_assert(nullptr, MDBX_IS_ERROR(rc));
assert(MDBX_IS_ERROR(rc));
return rc;
}
}
@@ -791,11 +812,32 @@ __cold static int mdbx_ipclock_failed(MDBX_env *env, mdbx_ipclock_t *ipc,
return rc;
}
#if defined(__ANDROID_API__) || defined(ANDROID) || defined(BIONIC)
MDBX_INTERNAL_FUNC int mdbx_check_tid4bionic(void) {
/* avoid 32-bit Bionic bug/hang with 32-pit TID */
if (sizeof(pthread_mutex_t) < sizeof(pid_t) + sizeof(unsigned)) {
pid_t tid = gettid();
if (unlikely(tid > 0xffff)) {
mdbx_fatal("Raise the ENOSYS(%d) error to avoid hang due "
"the 32-bit Bionic/Android bug with tid/thread_id 0x%08x(%i) "
"that dont fit in 16 bits, see "
"https://android.googlesource.com/platform/bionic/+/master/"
"docs/32-bit-abi.md#is-too-small-for-large-pids",
ENOSYS, tid, tid);
return ENOSYS;
}
}
return 0;
}
#endif /* __ANDROID_API__ || ANDROID) || BIONIC */
static int mdbx_ipclock_lock(MDBX_env *env, mdbx_ipclock_t *ipc,
const bool dont_wait) {
#if MDBX_LOCKING == MDBX_LOCKING_POSIX2001 || \
MDBX_LOCKING == MDBX_LOCKING_POSIX2008
int rc = dont_wait ? pthread_mutex_trylock(ipc) : pthread_mutex_lock(ipc);
int rc = mdbx_check_tid4bionic();
if (likely(rc == 0))
rc = dont_wait ? pthread_mutex_trylock(ipc) : pthread_mutex_lock(ipc);
rc = (rc == EBUSY && dont_wait) ? MDBX_BUSY : rc;
#elif MDBX_LOCKING == MDBX_LOCKING_POSIX1988
int rc = MDBX_SUCCESS;

View File

@@ -72,7 +72,7 @@ static
# pragma const_seg(push)
# pragma data_seg(push)
# ifdef _WIN64
# ifndef _M_IX86
/* kick a linker to create the TLS directory if not already done */
# pragma comment(linker, "/INCLUDE:_tls_used")
/* Force some symbol references. */
@@ -94,7 +94,7 @@ static
# pragma const_seg(pop)
#elif defined(__GNUC__)
# ifdef _WIN64
# ifndef _M_IX86
const
# endif
PIMAGE_TLS_CALLBACK mdbx_tls_anchor __attribute__((__section__(".CRT$XLB"), used)) = mdbx_module_handler;
@@ -754,6 +754,11 @@ MDBX_GetTickCount64 mdbx_GetTickCount64;
MDBX_RegGetValueA mdbx_RegGetValueA;
#endif /* xMDBX_ALLOY */
#if __GNUC_PREREQ(8, 0)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wcast-function-type"
#endif /* GCC/MINGW */
static void mdbx_winnt_import(void) {
const HINSTANCE hNtdll = GetModuleHandleA("ntdll.dll");
@@ -805,4 +810,8 @@ static void mdbx_winnt_import(void) {
}
}
#if __GNUC_PREREQ(8, 0)
#pragma GCC diagnostic pop
#endif /* GCC/MINGW */
#endif /* Windows LCK-implementation */

View File

@@ -1,6 +1,6 @@
.\" Copyright 2015-2022 Leonid Yuriev <leo@yuriev.ru>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_CHK 1 "2022-02-23" "MDBX 0.11.5"
.TH MDBX_CHK 1 "2022-04-22" "MDBX 0.11.7"
.SH NAME
mdbx_chk \- MDBX checking tool
.SH SYNOPSIS
@@ -96,4 +96,4 @@ if no quiet mode was requested.
.BR mdbx_load (1)
.BR mdbx_drop (1)
.SH AUTHOR
Leonid Yuriev <https://github.com/erthink>
Leonid Yuriev <https://gitflic.ru/user/erthink>

View File

@@ -2,7 +2,7 @@
.\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_COPY 1 "2022-02-23" "MDBX 0.11.5"
.TH MDBX_COPY 1 "2022-04-22" "MDBX 0.11.7"
.SH NAME
mdbx_copy \- MDBX environment copy tool
.SH SYNOPSIS
@@ -65,4 +65,4 @@ free during copying cannot be reused until the copy is done.
.BR mdbx_drop (1)
.SH AUTHOR
Howard Chu of Symas Corporation <http://www.symas.com>,
Leonid Yuriev <https://github.com/erthink>
Leonid Yuriev <https://gitflic.ru/user/erthink>

View File

@@ -1,7 +1,7 @@
.\" Copyright 2021-2022 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2014-2021 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_DROP 1 "2022-02-23" "MDBX 0.11.5"
.TH MDBX_DROP 1 "2022-04-22" "MDBX 0.11.7"
.SH NAME
mdbx_drop \- MDBX database delete tool
.SH SYNOPSIS

View File

@@ -2,7 +2,7 @@
.\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_DUMP 1 "2022-02-23" "MDBX 0.11.5"
.TH MDBX_DUMP 1 "2022-04-22" "MDBX 0.11.7"
.SH NAME
mdbx_dump \- MDBX environment export tool
.SH SYNOPSIS
@@ -91,4 +91,4 @@ utility to load the database using the correct comparison functions.
.BR mdbx_drop (1)
.SH AUTHOR
Howard Chu of Symas Corporation <http://www.symas.com>,
Leonid Yuriev <https://github.com/erthink>
Leonid Yuriev <https://gitflic.ru/user/erthink>

View File

@@ -2,7 +2,7 @@
.\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_LOAD 1 "2022-02-23" "MDBX 0.11.5"
.TH MDBX_LOAD 1 "2022-04-22" "MDBX 0.11.7"
.SH NAME
mdbx_load \- MDBX environment import tool
.SH SYNOPSIS
@@ -102,4 +102,4 @@ a diagnostic message being written to standard error.
.BR mdbx_drop (1)
.SH AUTHOR
Howard Chu of Symas Corporation <http://www.symas.com>,
Leonid Yuriev <https://github.com/erthink>
Leonid Yuriev <https://gitflic.ru/user/erthink>

View File

@@ -2,7 +2,7 @@
.\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_STAT 1 "2022-02-23" "MDBX 0.11.5"
.TH MDBX_STAT 1 "2022-04-22" "MDBX 0.11.7"
.SH NAME
mdbx_stat \- MDBX environment status tool
.SH SYNOPSIS
@@ -83,4 +83,4 @@ a diagnostic message being written to standard error.
.BR mdbx_drop (1)
.SH AUTHOR
Howard Chu of Symas Corporation <http://www.symas.com>,
Leonid Yuriev <https://github.com/erthink>
Leonid Yuriev <https://gitflic.ru/user/erthink>

View File

@@ -9,6 +9,11 @@
#define _CRT_SECURE_NO_WARNINGS
#endif /* _CRT_SECURE_NO_WARNINGS */
#if (defined(__MINGW__) || defined(__MINGW32__) || defined(__MINGW64__)) && \
!defined(__USE_MINGW_ANSI_STDIO)
#define __USE_MINGW_ANSI_STDIO 1
#endif /* __USE_MINGW_ANSI_STDIO */
#include "../mdbx.h++"
#include "defs.h"
@@ -1234,15 +1239,6 @@ env::operate_options::operate_options(MDBX_env_flags_t flags) noexcept
disable_readahead((flags & MDBX_NORDAHEAD) ? true : false),
disable_clear_memory((flags & MDBX_NOMEMINIT) ? true : false) {}
env::operate_parameters::operate_parameters(const env &env)
: max_maps(env.max_maps()), max_readers(env.max_readers()) {
const auto flags = env.get_flags();
mode = mode_from_flags(flags);
durability = durability_from_flags(flags);
reclaiming = reclaiming_from_flags(flags);
options = options_from_flags(flags);
}
bool env::is_pristine() const {
return get_stat().ms_mod_txnid == 0 &&
get_info().mi_recent_txnid == INITIAL_TXNID;
@@ -1251,9 +1247,9 @@ bool env::is_pristine() const {
bool env::is_empty() const { return get_stat().ms_leaf_pages == 0; }
#ifdef MDBX_STD_FILESYSTEM_PATH
env &env::copy(const ::std::filesystem::path &destination, bool compactify,
env &env::copy(const MDBX_STD_FILESYSTEM_PATH &destination, bool compactify,
bool force_dynamic_size) {
const path_to_pchar<::std::filesystem::path> utf8(destination);
const path_to_pchar<MDBX_STD_FILESYSTEM_PATH> utf8(destination);
error::success_or_throw(
::mdbx_env_copy(handle_, utf8,
(compactify ? MDBX_CP_COMPACT : MDBX_CP_DEFAULTS) |
@@ -1303,9 +1299,9 @@ path env::get_path() const {
}
#ifdef MDBX_STD_FILESYSTEM_PATH
bool env::remove(const ::std::filesystem::path &pathname,
bool env::remove(const MDBX_STD_FILESYSTEM_PATH &pathname,
const remove_mode mode) {
const path_to_pchar<::std::filesystem::path> utf8(pathname);
const path_to_pchar<MDBX_STD_FILESYSTEM_PATH> utf8(pathname);
return error::boolean_or_throw(
::mdbx_env_delete(utf8, MDBX_env_delete_mode_t(mode)));
}
@@ -1362,11 +1358,11 @@ __cold void env_managed::setup(unsigned max_maps, unsigned max_readers) {
}
#ifdef MDBX_STD_FILESYSTEM_PATH
__cold env_managed::env_managed(const ::std::filesystem::path &pathname,
__cold env_managed::env_managed(const MDBX_STD_FILESYSTEM_PATH &pathname,
const operate_parameters &op, bool accede)
: env_managed(create_env()) {
setup(op.max_maps, op.max_readers);
const path_to_pchar<::std::filesystem::path> utf8(pathname);
const path_to_pchar<MDBX_STD_FILESYSTEM_PATH> utf8(pathname);
error::success_or_throw(
::mdbx_env_open(handle_, utf8, op.make_flags(accede), 0));
@@ -1375,12 +1371,12 @@ __cold env_managed::env_managed(const ::std::filesystem::path &pathname,
MDBX_CXX20_UNLIKELY error::throw_exception(MDBX_INCOMPATIBLE);
}
__cold env_managed::env_managed(const ::std::filesystem::path &pathname,
__cold env_managed::env_managed(const MDBX_STD_FILESYSTEM_PATH &pathname,
const env_managed::create_parameters &cp,
const env::operate_parameters &op, bool accede)
: env_managed(create_env()) {
setup(op.max_maps, op.max_readers);
const path_to_pchar<::std::filesystem::path> utf8(pathname);
const path_to_pchar<MDBX_STD_FILESYSTEM_PATH> utf8(pathname);
set_geometry(cp.geometry);
error::success_or_throw(
::mdbx_env_open(handle_, utf8, op.make_flags(accede, cp.use_subdirectory),

View File

@@ -138,7 +138,8 @@ static void va_log(MDBX_log_level_t level, const char *msg, va_list args) {
out = stderr;
}
if (!quiet && verbose + 1 >= (unsigned)level) {
if (!quiet && verbose + 1 >= (unsigned)level &&
(unsigned)level < ARRAY_LENGTH(prefixes)) {
fflush(nullptr);
fputs(prefixes[level], out);
vfprintf(out, msg, args);
@@ -239,6 +240,8 @@ static void MDBX_PRINTF_ARGS(4, 5)
if (!p) {
p = mdbx_calloc(1, sizeof(*p));
if (unlikely(!p))
return;
p->caption = msg;
p->pr_next = problems_list;
problems_list = p;
@@ -616,6 +619,8 @@ static int handle_maindb(const uint64_t record_number, const MDBX_val *key,
}
name = mdbx_malloc(key->iov_len + 1);
if (unlikely(!name))
return MDBX_ENOMEM;
memcpy(name, key->iov_base, key->iov_len);
name[key->iov_len] = '\0';
userdb_count++;
@@ -1152,7 +1157,7 @@ int main(int argc, char *argv[]) {
envflags &= ~MDBX_RDONLY;
#if MDBX_MMAP_INCOHERENT_FILE_WRITE
/* Temporary `workaround` for OpenBSD kernel's flaw.
* See https://github.com/erthink/libmdbx/issues/67 */
* See todo4recovery://erased_by_github/libmdbx/issues/67 */
envflags |= MDBX_WRITEMAP;
#endif /* MDBX_MMAP_INCOHERENT_FILE_WRITE */
break;
@@ -1226,7 +1231,7 @@ int main(int argc, char *argv[]) {
mdbx_setup_debug((verbose < MDBX_LOG_TRACE - 1)
? (MDBX_log_level_t)(verbose + 1)
: MDBX_LOG_TRACE,
MDBX_DBG_LEGACY_OVERLAP, logger);
MDBX_DBG_LEGACY_OVERLAP | MDBX_DBG_DONT_UPGRADE, logger);
rc = mdbx_env_create(&env);
if (rc) {

View File

@@ -437,7 +437,7 @@ MDBX_INTERNAL_FUNC int mdbx_condpair_lock(mdbx_condpair_t *condpair) {
DWORD code = WaitForSingleObject(condpair->mutex, INFINITE);
return waitstatus2errcode(code);
#else
return pthread_mutex_lock(&condpair->mutex);
return mdbx_pthread_mutex_lock(&condpair->mutex);
#endif
}
@@ -507,7 +507,7 @@ MDBX_INTERNAL_FUNC int mdbx_fastmutex_acquire(mdbx_fastmutex_t *fastmutex) {
}
return MDBX_SUCCESS;
#else
return pthread_mutex_lock(fastmutex);
return mdbx_pthread_mutex_lock(fastmutex);
#endif
}
@@ -678,7 +678,7 @@ MDBX_INTERNAL_FUNC int mdbx_openfile(const enum mdbx_openfile_purpose purpose,
flags |= O_CLOEXEC;
#endif /* O_CLOEXEC */
/* Safeguard for https://github.com/erthink/libmdbx/issues/144 */
/* Safeguard for todo4recovery://erased_by_github/libmdbx/issues/144 */
#if STDIN_FILENO == 0 && STDOUT_FILENO == 1 && STDERR_FILENO == 2
int stub_fd0 = -1, stub_fd1 = -1, stub_fd2 = -1;
static const char dev_null[] = "/dev/null";
@@ -710,7 +710,7 @@ MDBX_INTERNAL_FUNC int mdbx_openfile(const enum mdbx_openfile_purpose purpose,
}
#endif /* O_DIRECT */
/* Safeguard for https://github.com/erthink/libmdbx/issues/144 */
/* Safeguard for todo4recovery://erased_by_github/libmdbx/issues/144 */
#if STDIN_FILENO == 0 && STDOUT_FILENO == 1 && STDERR_FILENO == 2
if (*fd == STDIN_FILENO) {
mdbx_warning("Got STD%s_FILENO/%d, avoid using it by dup(fd)", "IN",
@@ -1547,7 +1547,8 @@ MDBX_INTERNAL_FUNC int mdbx_munmap(mdbx_mmap_t *map) {
VALGRIND_MAKE_MEM_NOACCESS(map->address, map->current);
/* Unpoisoning is required for ASAN to avoid false-positive diagnostic
* when this memory will re-used by malloc or another mmapping.
* See https://github.com/erthink/libmdbx/pull/93#issuecomment-613687203 */
* See todo4recovery://erased_by_github/libmdbx/pull/93#issuecomment-613687203
*/
MDBX_ASAN_UNPOISON_MEMORY_REGION(map->address,
(map->filesize && map->filesize < map->limit)
? map->filesize
@@ -1619,11 +1620,12 @@ MDBX_INTERNAL_FUNC int mdbx_mresize(const int flags, mdbx_mmap_t *map,
* - extend read-only mapping;
* Therefore we should unmap/map entire section. */
if ((flags & MDBX_MRESIZE_MAY_UNMAP) == 0)
return MDBX_RESULT_TRUE;
return MDBX_EPERM;
/* Unpoisoning is required for ASAN to avoid false-positive diagnostic
* when this memory will re-used by malloc or another mmapping.
* See https://github.com/erthink/libmdbx/pull/93#issuecomment-613687203 */
* See todo4recovery://erased_by_github/libmdbx/pull/93#issuecomment-613687203
*/
MDBX_ASAN_UNPOISON_MEMORY_REGION(map->address, map->limit);
status = NtUnmapViewOfSection(GetCurrentProcess(), map->address);
if (!NT_SUCCESS(status))
@@ -1729,7 +1731,7 @@ retry_mapview:;
if (map->address && (size != map->current || limit != map->limit)) {
/* try remap with previously size and limit,
* but will return MDBX_UNABLE_EXTEND_MAPSIZE on success */
rc = (limit > map->limit) ? MDBX_UNABLE_EXTEND_MAPSIZE : MDBX_RESULT_TRUE;
rc = (limit > map->limit) ? MDBX_UNABLE_EXTEND_MAPSIZE : MDBX_EPERM;
size = map->current;
ReservedSize = limit = map->limit;
goto retry_file_and_section;
@@ -1753,8 +1755,7 @@ retry_mapview:;
if (flags & MDBX_RDONLY) {
map->current = (map->filesize > limit) ? limit : (size_t)map->filesize;
if (map->current != size)
rc =
(size > map->current) ? MDBX_UNABLE_EXTEND_MAPSIZE : MDBX_RESULT_TRUE;
rc = (size > map->current) ? MDBX_UNABLE_EXTEND_MAPSIZE : MDBX_EPERM;
} else {
if (map->filesize != size) {
rc = mdbx_ftruncate(map->fd, size);
@@ -1889,7 +1890,8 @@ retry_mapview:;
VALGRIND_MAKE_MEM_NOACCESS(map->address, map->current);
/* Unpoisoning is required for ASAN to avoid false-positive diagnostic
* when this memory will re-used by malloc or another mmapping.
* See https://github.com/erthink/libmdbx/pull/93#issuecomment-613687203
* See
* todo4recovery://erased_by_github/libmdbx/pull/93#issuecomment-613687203
*/
MDBX_ASAN_UNPOISON_MEMORY_REGION(
map->address,
@@ -1909,7 +1911,9 @@ retry_mapview:;
VALGRIND_MAKE_MEM_NOACCESS(map->address, map->current);
/* Unpoisoning is required for ASAN to avoid false-positive diagnostic
* when this memory will re-used by malloc or another mmapping.
* See https://github.com/erthink/libmdbx/pull/93#issuecomment-613687203 */
* See
* todo4recovery://erased_by_github/libmdbx/pull/93#issuecomment-613687203
*/
MDBX_ASAN_UNPOISON_MEMORY_REGION(
map->address, (map->current < map->limit) ? map->current : map->limit);
@@ -1941,6 +1945,9 @@ __cold MDBX_INTERNAL_FUNC void mdbx_osal_jitter(bool tiny) {
#if defined(_M_IX86) || defined(_M_X64) || defined(__i386__) || \
defined(__x86_64__)
const unsigned salt = 277u * (unsigned)__rdtsc();
#elif (defined(_WIN32) || defined(_WIN64)) && MDBX_WITHOUT_MSVC_CRT
static ULONG state;
const unsigned salt = (unsigned)RtlRandomEx(&state);
#else
const unsigned salt = rand();
#endif

View File

@@ -332,6 +332,14 @@ typedef pthread_mutex_t mdbx_fastmutex_t;
#define MDBX_WORDBITS 32
#endif /* MDBX_WORDBITS */
#if defined(__ANDROID_API__) || defined(ANDROID)
#if defined(_FILE_OFFSET_BITS) && _FILE_OFFSET_BITS != MDBX_WORDBITS
#error "_FILE_OFFSET_BITS != MDBX_WORDBITS" (_FILE_OFFSET_BITS != MDBX_WORDBITS)
#elif defined(__FILE_OFFSET_BITS) && __FILE_OFFSET_BITS != MDBX_WORDBITS
#error "__FILE_OFFSET_BITS != MDBX_WORDBITS" (__FILE_OFFSET_BITS != MDBX_WORDBITS)
#endif
#endif /* Android */
/*----------------------------------------------------------------------------*/
/* Compiler's includes for builtins/intrinsics */
@@ -474,7 +482,7 @@ typedef union bin128 {
#if defined(_WIN32) || defined(_WIN64)
typedef union MDBX_srwlock {
struct {
__anonymous_struct_extension__ struct {
long volatile readerCount;
long volatile writerCount;
};
@@ -722,6 +730,7 @@ MDBX_MAYBE_UNUSED static __inline uint32_t mdbx_getpid(void) {
#if defined(_WIN32) || defined(_WIN64)
return GetCurrentProcessId();
#else
STATIC_ASSERT(sizeof(pid_t) <= sizeof(uint32_t));
return getpid();
#endif
}
@@ -737,6 +746,20 @@ MDBX_MAYBE_UNUSED static __inline uintptr_t mdbx_thread_self(void) {
return (uintptr_t)thunk;
}
#if !defined(_WIN32) && !defined(_WIN64)
#if defined(__ANDROID_API__) || defined(ANDROID) || defined(BIONIC)
MDBX_INTERNAL_FUNC int mdbx_check_tid4bionic(void);
#else
static __inline int mdbx_check_tid4bionic(void) { return 0; }
#endif /* __ANDROID_API__ || ANDROID) || BIONIC */
MDBX_MAYBE_UNUSED static __inline int
mdbx_pthread_mutex_lock(pthread_mutex_t *mutex) {
int err = mdbx_check_tid4bionic();
return unlikely(err) ? err : pthread_mutex_lock(mutex);
}
#endif /* !Windows */
MDBX_INTERNAL_FUNC uint64_t mdbx_osal_monotime(void);
MDBX_INTERNAL_FUNC uint64_t
mdbx_osal_16dot16_to_monotime(uint32_t seconds_16dot16);
@@ -968,6 +991,8 @@ typedef LSTATUS(WINAPI *MDBX_RegGetValueA)(HKEY hkey, LPCSTR lpSubKey,
LPDWORD pcbData);
MDBX_INTERNAL_VAR MDBX_RegGetValueA mdbx_RegGetValueA;
NTSYSAPI ULONG RtlRandomEx(PULONG Seed);
#endif /* Windows */
#endif /* !__cplusplus */

View File

@@ -31,6 +31,7 @@ set(LIBMDBX_TEST_SOURCES
)
if(NOT MDBX_BUILD_CXX)
probe_libcxx_filesystem()
list(APPEND LIBMDBX_TEST_SOURCES "${MDBX_SOURCE_DIR}/mdbx.c++" ../mdbx.h++)
endif()
@@ -45,6 +46,14 @@ set_target_properties(mdbx_test PROPERTIES
INTERPROCEDURAL_OPTIMIZATION $<BOOL:${INTERPROCEDURAL_OPTIMIZATION}>)
target_setup_options(mdbx_test)
if(NOT MDBX_BUILD_CXX AND LIBCXX_FILESYSTEM)
if(CMAKE_COMPILER_IS_ELBRUSCXX AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 1.25.23
AND NOT CMAKE_VERSION VERSION_LESS 3.13)
target_link_options(mdbx_test PRIVATE "-Wl,--allow-multiple-definition")
endif()
target_link_libraries(mdbx_test ${LIBCXX_FILESYSTEM})
endif()
target_link_libraries(mdbx_test ${TOOL_MDBX_LIB} ${LIB_MATH} ${CMAKE_THREAD_LIBS_INIT})
if(${CMAKE_SYSTEM_NAME} STREQUAL "Windows")
target_link_libraries(mdbx_test winmm.lib)
@@ -55,3 +64,76 @@ if(UNIX AND NOT SUBPROJECT)
target_include_directories(pcrf_test PRIVATE "${PROJECT_SOURCE_DIR}")
target_link_libraries(pcrf_test ${TOOL_MDBX_LIB})
endif()
################################################################################
if (CMAKE_CROSSCOMPILING AND NOT CMAKE_CROSSCOMPILING_EMULATOR)
message(WARNING "No emulator to run cross-compiled tests")
add_test(NAME fake_since_no_crosscompiling_emulator COMMAND ${CMAKE_COMMAND} -E
echo "No emulator to run cross-compiled tests")
else()
string(RANDOM LENGTH 9 ALPHABET "1234567890" test_seed)
message(STATUS "The ${test_seed} will be used for seeding tests. Re-run cmake to re-seed it.")
add_test(NAME smoke COMMAND ${MDBX_OUTPUT_DIR}/mdbx_test
--loglevel=verbose
--keygen.seed=${test_seed}
--progress --console=no --pathname=smoke.db --dont-cleanup-after basic)
set_tests_properties(smoke PROPERTIES
TIMEOUT 600
RUN_SERIAL OFF)
if(MDBX_BUILD_TOOLS)
add_test(NAME smoke_chk COMMAND ${MDBX_OUTPUT_DIR}/mdbx_chk -nvv smoke.db)
set_tests_properties(smoke_chk PROPERTIES
DEPENDS smoke
TIMEOUT 60
REQUIRED_FILES smoke.db)
add_test(NAME smoke_chk_copy COMMAND ${MDBX_OUTPUT_DIR}/mdbx_chk -nvv smoke.db-copy)
set_tests_properties(smoke_chk_copy PROPERTIES
DEPENDS smoke
TIMEOUT 60
REQUIRED_FILES smoke.db-copy)
endif()
add_test(NAME dupsort_writemap COMMAND ${MDBX_OUTPUT_DIR}/mdbx_test
--loglevel=notice
--keygen.seed=${test_seed}
--table=+data.integer --keygen.split=29 --datalen.min=min --datalen.max=max --progress --console=no
--repeat=2 --pathname=dupsort_writemap.db --dont-cleanup-after basic)
set_tests_properties(dupsort_writemap PROPERTIES
TIMEOUT 600
RUN_SERIAL OFF)
if(MDBX_BUILD_TOOLS)
add_test(NAME dupsort_writemap_chk COMMAND ${MDBX_OUTPUT_DIR}/mdbx_chk -nvv dupsort_writemap.db)
set_tests_properties(dupsort_writemap_chk PROPERTIES
DEPENDS dupsort_writemap
TIMEOUT 60
REQUIRED_FILES dupsort_writemap.db)
add_test(NAME dupsort_writemap_chk_copy COMMAND ${MDBX_OUTPUT_DIR}/mdbx_chk -nvv dupsort_writemap.db-copy)
set_tests_properties(dupsort_writemap_chk_copy PROPERTIES
DEPENDS dupsort_writemap
TIMEOUT 60
REQUIRED_FILES dupsort_writemap.db-copy)
endif()
add_test(NAME uniq_nested COMMAND ${MDBX_OUTPUT_DIR}/mdbx_test
--loglevel=notice
--mode=-writemap,-nosync-safe,-lifo --progress --console=no --repeat=2 --pathname=uniq_nested.db --dont-cleanup-after basic)
set_tests_properties(uniq_nested PROPERTIES
TIMEOUT 1800
RUN_SERIAL OFF)
if(MDBX_BUILD_TOOLS)
add_test(NAME uniq_nested_chk COMMAND ${MDBX_OUTPUT_DIR}/mdbx_chk -nvv uniq_nested.db)
set_tests_properties(uniq_nested_chk PROPERTIES
DEPENDS uniq_nested
TIMEOUT 60
REQUIRED_FILES uniq_nested.db)
add_test(NAME uniq_nested_chk_copy COMMAND ${MDBX_OUTPUT_DIR}/mdbx_chk -nvv uniq_nested.db-copy)
set_tests_properties(uniq_nested_chk_copy PROPERTIES
DEPENDS uniq_nested
TIMEOUT 60
REQUIRED_FILES uniq_nested.db-copy)
endif()
endif()

View File

@@ -78,6 +78,12 @@ time from_ms(uint64_t ms) {
return result;
}
#if __GNUC_PREREQ(8, 0) && \
(defined(__MINGW__) || defined(__MINGW32__) || defined(__MINGW64__))
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wcast-function-type"
#endif /* GCC/MINGW */
time now_realtime() {
#if defined(_WIN32) || defined(_WIN64) || defined(_WINDOWS)
static void(WINAPI * query_time)(LPFILETIME);
@@ -133,4 +139,9 @@ time now_monotonic() {
#endif
}
#if __GNUC_PREREQ(8, 0) && \
(defined(__MINGW__) || defined(__MINGW32__) || defined(__MINGW64__))
#pragma GCC diagnostic pop
#endif /* GCC/MINGW */
} /* namespace chrono */

View File

@@ -106,7 +106,7 @@ bool parse_option<unsigned>(int argc, char *const argv[], int &narg,
while (true) {
if (!scan->verb)
failure("Unknown verb '%.*s', for option '==%s'\n", (int)len, list,
failure("Unknown verb '%.*s', for option '--%s'\n", (int)len, list,
option);
if (strlen(scan->verb) == len && strncmp(list, scan->verb, len) == 0) {
mask = strikethrough ? mask & ~scan->mask : mask | scan->mask;
@@ -257,6 +257,69 @@ bool parse_option(int argc, char *const argv[], int &narg, const char *option,
return false;
}
bool parse_option(int argc, char *const argv[], int &narg, const char *option,
logging::loglevel &loglevel) {
const char *value_cstr;
if (!parse_option(argc, argv, narg, option, &value_cstr))
return false;
if (strcmp(value_cstr, "min") == 0 || strcmp(value_cstr, "minimal") == 0 ||
strcmp(value_cstr, "fatal") == 0) {
loglevel = logging::failure;
return true;
}
if (strcmp(value_cstr, "error") == 0 || strcmp(value_cstr, "err") == 0) {
loglevel = logging::error;
return true;
}
if (strcmp(value_cstr, "warning") == 0 || strcmp(value_cstr, "warn") == 0) {
loglevel = logging::error;
return true;
}
if (strcmp(value_cstr, "default") == 0 || strcmp(value_cstr, "notice") == 0) {
loglevel = logging::notice;
return true;
}
if (strcmp(value_cstr, "verbose") == 0) {
loglevel = logging::verbose;
return true;
}
if (strcmp(value_cstr, "debug") == 0) {
loglevel = logging::debug;
return true;
}
if (strcmp(value_cstr, "trace") == 0) {
loglevel = logging::trace;
return true;
}
if (strcmp(value_cstr, "max") == 0 || strcmp(value_cstr, "maximal") == 0 ||
strcmp(value_cstr, "extra") == 0) {
loglevel = logging::extra;
return true;
}
char *suffix = nullptr;
unsigned long long raw = strtoull(value_cstr, &suffix, 0);
if ((suffix && *suffix) || errno) {
suffix = nullptr;
errno = 0;
raw = strtoull(value_cstr, &suffix, 10);
}
if ((!suffix || !*suffix) && !errno && raw < 8) {
loglevel = static_cast<logging::loglevel>(raw);
return true;
}
failure("Unknown log-level '%s', for option '--%s'\n", value_cstr, option);
}
bool parse_option(int argc, char *const argv[], int &narg, const char *option,
bool &value) {
const char *value_cstr = nullptr;

View File

@@ -133,6 +133,8 @@ inline bool parse_option_intptr(int argc, char *const argv[], int &narg,
int32_t(maxval), int32_t(default_value));
}
bool parse_option(int argc, char *const argv[], int &narg, const char *option,
logging::loglevel &);
//-----------------------------------------------------------------------------
struct keygen_params_pod {

View File

@@ -115,7 +115,7 @@ bool testcase_jitter::run() {
db_guard.get(), -1, -1,
coin4size ? upper_limit * 2 / 3 : upper_limit * 3 / 2, -1, -1, -1);
if (err != MDBX_SUCCESS && err != MDBX_UNABLE_EXTEND_MAPSIZE &&
err != MDBX_MAP_FULL && err != MDBX_TOO_LARGE)
err != MDBX_MAP_FULL && err != MDBX_TOO_LARGE && err != MDBX_EPERM)
failure_perror("mdbx_env_set_geometry-1", err);
}
}
@@ -126,7 +126,7 @@ bool testcase_jitter::run() {
db_guard.get(), -1, -1,
!coin4size ? upper_limit * 2 / 3 : upper_limit * 3 / 2, -1, -1, -1);
if (err != MDBX_SUCCESS && err != MDBX_UNABLE_EXTEND_MAPSIZE &&
err != MDBX_MAP_FULL && err != MDBX_TOO_LARGE)
err != MDBX_MAP_FULL && err != MDBX_TOO_LARGE && err != MDBX_EPERM)
failure_perror("mdbx_env_set_geometry-2", err);
}
@@ -142,7 +142,7 @@ bool testcase_jitter::run() {
err = mdbx_env_set_geometry(db_guard.get(), -1, -1, upper_limit, -1, -1,
-1);
if (err != MDBX_SUCCESS && err != MDBX_UNABLE_EXTEND_MAPSIZE &&
err != MDBX_MAP_FULL && err != MDBX_TOO_LARGE)
err != MDBX_MAP_FULL && err != MDBX_TOO_LARGE && err != MDBX_EPERM)
failure_perror("mdbx_env_set_geometry-3", err);
}

View File

@@ -144,9 +144,9 @@ case ${UNAME} in
mkdir -p $TESTDB_DIR && rm -f $TESTDB_DIR/*
if LC_ALL=C free | grep -q -i available; then
ram_avail_mb=$(($(LC_ALL=C free | grep -i Mem: | tr -s [:blank:] ' ' | cut -d ' ' -f 7) / 1024))
ram_avail_mb=$(($(LC_ALL=C free | grep -i Mem: | tr -s '[:blank:]' ' ' | cut -d ' ' -f 7) / 1024))
else
ram_avail_mb=$(($(LC_ALL=C free | grep -i Mem: | tr -s [:blank:] ' ' | cut -d ' ' -f 4) / 1024))
ram_avail_mb=$(($(LC_ALL=C free | grep -i Mem: | tr -s '[:blank:]' ' ' | cut -d ' ' -f 4) / 1024))
fi
;;

View File

@@ -24,6 +24,7 @@ MDBX_NORETURN void usage(void) {
"usage:\n"
" --help or -h Show this text\n"
"Common parameters:\n"
" --loglevel=[0-7]|[extra..fatal]"
" --pathname=... Path and/or name of database files\n"
" --repeat=N Set repeat counter\n"
" --threads=N Number of thread (unsupported for now)\n"
@@ -319,6 +320,13 @@ int main(int argc, char *const argv[]) {
config::duration, 1))
continue;
logging::loglevel loglevel;
if (config::parse_option(argc, argv, narg, "loglevel", loglevel)) {
logging::setup(loglevel, "main");
params.loglevel = static_cast<uint8_t>(loglevel);
continue;
}
const char *value = nullptr;
if (config::parse_option(argc, argv, narg, "case", &value)) {
fixup4qemu(params);

View File

@@ -75,7 +75,7 @@ void osal_setup(const std::vector<actor_config> &actors) {
if (!hEvent)
failure_perror("CreateEvent()", GetLastError());
hEvent = make_inheritable(hEvent);
log_trace("osal_setup: event %" PRIuPTR " -> %p", i, hEvent);
log_trace("osal_setup: event %u -> %p", i, hEvent);
events[i] = hEvent;
}

View File

@@ -50,7 +50,7 @@
#define rot64(v, s) _rotr64(v, s)
#define rot32(v, s) _rotr(v, s)
#if defined(_M_ARM64) || defined(_M_X64) || defined(_M_IA64)
#if defined(_M_X64) || defined(_M_IA64)
#pragma intrinsic(_umul128)
#define mul_64x64_128(a, b, ph) _umul128(a, b, ph)
#pragma intrinsic(__umulh)

View File

@@ -31,6 +31,20 @@
fun:mdbx_wipe_steady
}
# memcmp() inside mdbx_iov_write() as workaround for todo4recovery://erased_by_github/libmdbx/issues/269
{
write-page-check-bcmp
Memcheck:Cond
fun:bcmp
fun:mdbx_iov_write*
}
{
write-page-check-memcmp
Memcheck:Cond
fun:memcmp*
fun:mdbx_iov_write*
}
# single-page flush by pwrite()
{
pwrite-page-flush