Commit Graph

3449 Commits

Author SHA1 Message Date
Леонид Юрьев (Leonid Yuriev)
de63041b7d mdbx: add MDBX_DBG_DONT_UPGRADE flag. 2022-03-23 00:48:41 +03:00
Леонид Юрьев (Leonid Yuriev)
a5c064c33e mdbx: preserve (don't upgrade) format-signature during turn to a specified meta-page while recovery. 2022-03-23 00:48:41 +03:00
Your Name
9c832c24a6 mdbx: minor fix error-message space. 2022-03-23 00:48:41 +03:00
Леонид Юрьев (Leonid Yuriev)
c4a5325aaf mdbx: check-and-retry against page arrival as a workaround for unified page/buffer cache incoherency.
Part 2 of 2 of the workaround for https://github.com/erthink/libmdbx/issues/269.
2022-03-08 17:09:29 +03:00
Леонид Юрьев (Leonid Yuriev)
93cf99a07c mdbx: drop the previous temporary workaround for unified page/buffer cache incoherency.
Related to https://github.com/erthink/libmdbx/issues/269.
2022-03-06 13:22:57 +03:00
Леонид Юрьев (Leonid Yuriev)
00ed61c685 mdbx: check-and-retry a mvcc-snapshot for unified page/buffer cache coherency.
Part 1 of 2 of the workaround for https://github.com/erthink/libmdbx/issues/269.
2022-03-06 13:22:38 +03:00
Леонид Юрьев (Leonid Yuriev)
f84d9f6208 mdbx++: minor refine mdbx::cursor::erase(). 2022-03-06 00:13:11 +03:00
Леонид Юрьев (Leonid Yuriev)
9569b864ff mdbx-test: fix quoting for tr args inside long_stochastic.sh 2022-03-05 15:37:32 +03:00
Леонид Юрьев (Leonid Yuriev)
8b69be8def mdbx: update ChangeLog. 2022-03-05 15:14:20 +03:00
Леонид Юрьев (Leonid Yuriev)
4c619e32f7 mdbx: move origin to https://gitflic.ru/project/erthink/libmdbx. 2022-03-05 14:39:02 +03:00
Леонид Юрьев (Leonid Yuriev)
ed86d9c066 mdbx: add link to Scala bindings.
Resolves https://github.com/erthink/libmdbx/issues/274.
2022-03-05 14:39:02 +03:00
Леонид Юрьев (Leonid Yuriev)
6fbaa54d3e mdbx: remove PayPal from links for sponsorship. 2022-03-05 14:39:02 +03:00
Леонид Юрьев (Leonid Yuriev)
9d8fc7b984 mdbx: minor mix result caching inside the get_reasonable_db_maxsize().
Change-Id: I9763ebb323d4f66e702e07f708e269e2479880ac
2022-03-05 14:31:02 +03:00
Леонид Юрьев (Leonid Yuriev)
8c2efe3aaa mdbx++: add full mdbx::env_managed::geometry() for convenience.
Change-Id: Ib132c776bfc503336582b672de1c44fcac286936
2022-03-05 14:31:02 +03:00
Леонид Юрьев (Leonid Yuriev)
4b130bd82c mdbx-make: add lib-static and lib-shared targets.
Resolve https://github.com/erthink/libmdbx/issues/275.
2022-03-05 14:28:15 +03:00
Леонид Юрьев (Leonid Yuriev)
44fb240955 mdbx: merge branch issue-269 into the devel branch. 2022-03-04 17:46:43 +03:00
Леонид Юрьев (Leonid Yuriev)
bfea3ca9fb mdbx: minor refine mdbx_page_search() internals. 2022-03-04 17:38:25 +03:00
Леонид Юрьев (Leonid Yuriev)
7ade182d64 mdbx: minor simplify/refine mdbx_env_set_geometry() internals. 2022-03-04 17:38:25 +03:00
Леонид Юрьев (Leonid Yuriev)
4adc7aa58d mdbx: add few checks to mdbx_validate_meta(). 2022-03-04 11:53:14 +03:00
Леонид Юрьев (Leonid Yuriev)
110cf09cf8 mdbx: simplify core-dbs update inside mdbx_txn_commit_ex(). 2022-03-04 11:49:54 +03:00
Леонид Юрьев (Leonid Yuriev)
fb25648b9c mdbx: force mdbx_page_get() to inline. 2022-03-04 11:49:01 +03:00
Леонид Юрьев (Leonid Yuriev)
78170a5750 mdbx: minor reduce/lowering debug "update oldest" logging. 2022-03-04 11:41:17 +03:00
Леонид Юрьев (Leonid Yuriev)
71d07b3a8e mdbx: adding cursors tracking/lists for read-only transactions.
Briefly, this commit fixes a missed flaw:
 - Cursor tracking is required to replacing shaded pages and adjusting the positions in writing transactions;
 - Thus, historically, an internal linked list was maintained for a read-write transactions, but not for a read-only.
   For this reason, the API for using cursors should be different for writing and reading transactions;
 - However, the libmdbx's API has been significantly improved, including the ability to reuse cursors and a uniform cursors behavior for any kind of transactions.
   My mistake is that due to working with MithrilDB, I forgot to make a same changes to libmdbx.

Fixes https://github.com/erthink/libmdbx/issues/272.
2022-03-03 15:00:28 +03:00
Леонид Юрьев (Leonid Yuriev)
96c93ac2f1 mdbx++: refine mdbx::env::operate_parameters() and related. 2022-03-03 14:13:10 +03:00
Леонид Юрьев (Leonid Yuriev)
6d61b18325 mdbx: fix auxiliary mdbx_txn_valgrind().
Fixed https://github.com/erthink/libmdbx/issues/273.
2022-02-27 15:42:48 +03:00
Леонид Юрьев (Leonid Yuriev)
d01e44db0c
mdbx: release v0.11.5
The stable release with the hotfix/workaround for a flaw of Linux 4.19 (at least) unified page/buffer cache.
See [issue#269](https://github.com/erthink/libmdbx/issues/269) for more information.

Acknowledgements:
-----------------

 - [Simon Leier](https://github.com/leisim) for reporting and testing.
 - [Kai Wetlesen](https://github.com/kaiwetlesen) for [RPMs](http://copr.fedorainfracloud.org/coprs/kwetlesen/libmdbx/).
 - [Tullio Canepa](https://github.com/canepat) for reporting C++ API issue and contributing.

Fixes:
------

 - [Added workaround](https://github.com/erthink/libmdbx/issues/269) for a flaw of Linux 4.19 (at least) unified page/buffer cache.
 - [Fixed/Reworked](https://github.com/erthink/libmdbx/pull/270) move-assignment operators for "managed" classes of C++ API.
 - Fixed potential `SIGSEGV` while open DB with overrided non-default page size.
 - [Made](https://github.com/erthink/libmdbx/issues/267) `mdbx_env_open()` idempotence in failure cases.
 - Refined/Fixed pages reservation inside `mdbx_update_gc()` to avoid non-reclamation in a rare cases.
 - Fixed typo in a retained space calculation for the hsr-callback.

Minors:
-------

 - Reworked functions for meta-pages, split-off non-volatile.
 - Disentangled C11-atomic fences/barriers and pure-functions (with `__attribute__((__pure__))`) to avoid compiler misoptimization.
 - Fixed hypotetic unaligned access to 64-bit dwords on ARM with `__ARM_FEATURE_UNALIGNED` defined.
 - Reasonable paranoia that makes clarity for code readers.
 - Minor fixes Doxygen references, comments, descriptions, etc.

Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2022-02-23 20:06:30 +03:00
Леонид Юрьев (Leonid Yuriev)
464886ab61 mdbx++: rework/fix move-assignment operators for "managed" classes.
Replaces https://github.com/erthink/libmdbx/pull/270 and previous commit.
Fixed a half of https://github.com/torquem-ch/silkworm/issues/575.
2022-02-23 18:42:09 +03:00
Леонид Юрьев (Leonid Yuriev)
3c574fca99 mdbx++: fix move-assignment operators for "managed" classes.
The three same mistakes for `mdbx::env_managed`, `mdbx::txn_managed` and `mdbx::cursor_managed`.

A `derived_managed &operator=(derived_managed &&) = default;`
don't call an inherited `base::operator=()` since it hidded because the
`derived_managed(derived_managed &&) = default;` is also provided.

Replaces/overrides https://github.com/erthink/libmdbx/pull/270.
2022-02-23 17:58:56 +03:00
Леонид Юрьев (Leonid Yuriev)
6b45498985 mdbx: temporary workaround/hotfix for a flaw of Linux 4.19 (at least) unified page/buffer cache.
Temporary workaround for https://github.com/erthink/libmdbx/issues/269.
2022-02-22 23:58:57 +03:00
Леонид Юрьев (Leonid Yuriev)
1fd4101e2e mdbx: update ChangeLog. 2022-02-21 00:06:57 +03:00
Леонид Юрьев (Leonid Yuriev)
77f236db2a mdbx: clarify loglevel descriptions. 2022-02-17 02:30:16 +03:00
Леонид Юрьев (Leonid Yuriev)
a0728023cd mdbx: fix typo in a retained space calculation for the hsr-callback. 2022-02-16 15:44:06 +03:00
Леонид Юрьев (Leonid Yuriev)
3705d705d3 mdbx: refine/fix pages reserving inside mdbx_updage_gc() to avoid non-reclamation in a rare cases. 2022-02-16 12:18:53 +03:00
Леонид Юрьев (Leonid Yuriev)
72bc655ece mdbx++: fix env::is_empty() to check leaf-pages instead of branch ones.
Seems like an auto-completion mistake.
Thanks to https://igor@t.me/libmdbx for reporing.
2022-02-15 18:11:31 +03:00
Леонид Юрьев (Leonid Yuriev)
c5f1f73fca mdbx: avoid extra looping inside mdbx_page_alloc() for mdbx_prep_backlog(). 2022-02-15 03:17:05 +03:00
Леонид Юрьев (Leonid Yuriev)
925a673d57 mdbx: make mdbx_env_open() idempotence in failure cases.
Fixes https://github.com/erthink/libmdbx/issues/267.
2022-02-13 20:30:57 +03:00
Леонид Юрьев (Leonid Yuriev)
c27787eb31 mdbx: fix potential SIGSEGV for DB with non-default page size.
Related to https://github.com/erthink/libmdbx/issues/267.
2022-02-13 17:04:10 +03:00
Леонид Юрьев (Leonid Yuriev)
2b6fd968d2 mdbx: rework functions for meta-pages, split-off non-volatile, more const. 2022-02-09 22:56:29 +03:00
Леонид Юрьев (Leonid Yuriev)
ef7b4289c0 mdbx: rework unaligned access.
The three points:
 - disentangle C11-atomic fences/barriers and pure-functions (with `__attribute__((__pure__))`) to avoid compiler misoptimization;
 - fix hypotetic unaligned access to 64-bit dwords on ARM with `__ARM_FEATURE_UNALIGNED` defined;
 - reasonable paranoia that makes clarity for code readers.
2022-02-09 22:56:28 +03:00
Леонид Юрьев (Leonid Yuriev)
cbbfaf3202 mdbx-doc: fix doxygen-reference to the "Long-lived read transactions" paragraph. 2022-02-08 14:37:11 +03:00
Леонид Юрьев (Leonid Yuriev)
f6be8e3e9a mdbx: minor refine note about MDBX_WITHOUT_MSVC_CRT option. 2022-02-03 03:12:23 +03:00
Леонид Юрьев (Leonid Yuriev)
daa7f04f61
mdbx: release v0.11.4
The stable release with fixes for large and huge databases sized of 4..128 TiB.

Acknowledgements:
-----------------

 - Ledgerwatch, Binance and Positive Technologies teams for reporting, assistance in investigation and testing.
 - Alex Sharov for reporting, testing and provide resources for remote debugging/investigation.
 - Kris Zyp for Deno support.

New features, extensions and improvements:
------------------------------------------

 - Added treating the `UINT64_MAX` value as maximum for given option inside `mdbx_env_set_option()`.
 - Added `to_hex/to_base58/to_base64::output(std::ostream&)` overloads without using temporary string objects as buffers.
 - Added `--geometry-jitter=YES|no` option to the test framework.
 - Added support for [Deno](https://deno.land/) support by [Kris Zyp](https://github.com/kriszyp).

Fixes:
------

 - Fixed handling `MDBX_opt_rp_augment_limit` for GC's records from huge transactions (Erigon/Akula/Ethereum).
 - [Fixed](https://github.com/erthink/libmdbx/issues/258) build on Android (avoid including `sys/sem.h`).
 - [Fixed](https://github.com/erthink/libmdbx/pull/261) missing copy assignment operator for `mdbx::move_result`.
 - Fixed missing `&` for `std::ostream &operator<<()` overloads.
 - Fixed unexpected `EXDEV` (Cross-device link) error from `mdbx_env_copy()`.
 - Fixed base64 encoding/decoding bugs in auxillary C++ API.
 - Fixed overflow of `pgno_t` during checking PNL on 64-bit platforms.
 - [Fixed](https://github.com/erthink/libmdbx/issues/260) excessive PNL checking after sort for spilling.
 - Reworked checking `MAX_PAGENO` and DB upper-size geometry limit.
 - [Fixed](https://github.com/erthink/libmdbx/issues/265) build for some combinations of versions of  MSVC and Windows SDK.

Minors:
-------

 - Added workaround for CLANG bug [D79919/PR42445](https://reviews.llvm.org/D79919).
 - Fixed build test on Android (using `pthread_barrier_t` stub).
 - Disabled C++20 concepts for CLANG < 14 on Android.
 - Fixed minor `unused parameter` warning.
 - Added CI for Android.
 - Refine/cleanup internal logging.
 - Refined line splitting inside hex/base58/base64 encoding to avoid `\n` at the end.
 - Added workaround for modern libstdc++ with CLANG < 4.x
 - Relaxed txn-check rules for auxiliary functions.
 - Clarified a comments and descriptions, etc.
 - Using the `-fno-semantic interposition` option to reduce the overhead to calling self own public functions.

Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2022-02-02 20:56:28 +03:00
Леонид Юрьев (Leonid Yuriev)
18290489b2 mdbx-cmake: don't use -fno-semantic-interposition for CLANG. 2022-02-02 19:22:50 +03:00
Леонид Юрьев (Leonid Yuriev)
92ab17a644 mdbx-test: resolve minor MSVC /W4 warnings. 2022-02-02 17:11:45 +03:00
Леонид Юрьев (Leonid Yuriev)
ac02490c97 mdbx-cmake: sync the compiler.cmake module with other projects. 2022-02-02 16:50:49 +03:00
Леонид Юрьев (Leonid Yuriev)
d498c2580b mdbx: more refine mdbx_env_set_geometry() description. 2022-02-02 14:28:45 +03:00
Леонид Юрьев (Leonid Yuriev)
e740df6e50 mdbx: fix description of mdbx_env_set_geometry().
Oops, I just noticed that there are a strange typos in the description.
It looks like a search&replace or cut&paste mistake.

Related to https://github.com/LWJGL/lwjgl3/issues/724.
2022-02-02 14:09:03 +03:00
Leonid Yuriev
b6a0d11b99 mdbx: add note to using recent MSVC & Windows SDK versions. 2022-02-02 14:09:03 +03:00
Leonid Yuriev
65defdd0a8 mdbx-test: fix calculation default size_now of DB. 2022-02-02 14:09:03 +03:00
Leonid Yuriev
5aeb7ccf25 mdbx: fix extra casting inside mdbx_limits_dbsize_max(). 2022-02-02 14:09:03 +03:00