ISAR/libmdbx
3
0
Fork 0
mirror of https://github.com/isar/libmdbx.git synced 2025-01-06 20:24:13 +08:00
Code Issues Packages Projects Releases Wiki Activity
3,555 Commits 5 Branches 68 Tags 13 MiB
Commit Graph

1953 Commits

Author SHA1 Message Date
Леонид Юрьев (Leonid Yuriev)
a0728023cd mdbx: fix typo in a retained space calculation for the hsr-callback. 2022-02-16 15:44:06 +03:00
Леонид Юрьев (Leonid Yuriev)
3705d705d3 mdbx: refine/fix pages reserving inside mdbx_updage_gc() to avoid non-reclamation in a rare cases. 2022-02-16 12:18:53 +03:00
Леонид Юрьев (Leonid Yuriev)
72bc655ece mdbx++: fix env::is_empty() to check leaf-pages instead of branch ones. 
Seems like an auto-completion mistake.
Thanks to https://igor@t.me/libmdbx for reporing.
 2022-02-15 18:11:31 +03:00
Леонид Юрьев (Leonid Yuriev)
c5f1f73fca mdbx: avoid extra looping inside mdbx_page_alloc() for mdbx_prep_backlog(). 2022-02-15 03:17:05 +03:00
Леонид Юрьев (Leonid Yuriev)
925a673d57 mdbx: make mdbx_env_open() idempotence in failure cases. 
Fixes https://github.com/erthink/libmdbx/issues/267.
 2022-02-13 20:30:57 +03:00
Леонид Юрьев (Leonid Yuriev)
c27787eb31 mdbx: fix potential SIGSEGV for DB with non-default page size. 
Related to https://github.com/erthink/libmdbx/issues/267.
 2022-02-13 17:04:10 +03:00
Леонид Юрьев (Leonid Yuriev)
2b6fd968d2 mdbx: rework functions for meta-pages, split-off non-volatile, more const. 2022-02-09 22:56:29 +03:00
Леонид Юрьев (Leonid Yuriev)
ef7b4289c0 mdbx: rework unaligned access. 
The three points:
 - disentangle C11-atomic fences/barriers and pure-functions (with `__attribute__((__pure__))`) to avoid compiler misoptimization;
 - fix hypotetic unaligned access to 64-bit dwords on ARM with `__ARM_FEATURE_UNALIGNED` defined;
 - reasonable paranoia that makes clarity for code readers.
 2022-02-09 22:56:28 +03:00
Леонид Юрьев (Leonid Yuriev)
daa7f04f61
mdbx: release v0.11.4 
The stable release with fixes for large and huge databases sized of 4..128 TiB.

Acknowledgements:
-----------------

 - Ledgerwatch, Binance and Positive Technologies teams for reporting, assistance in investigation and testing.
 - Alex Sharov for reporting, testing and provide resources for remote debugging/investigation.
 - Kris Zyp for Deno support.

New features, extensions and improvements:
------------------------------------------

 - Added treating the `UINT64_MAX` value as maximum for given option inside `mdbx_env_set_option()`.
 - Added `to_hex/to_base58/to_base64::output(std::ostream&)` overloads without using temporary string objects as buffers.
 - Added `--geometry-jitter=YES|no` option to the test framework.
 - Added support for [Deno](https://deno.land/) support by [Kris Zyp](https://github.com/kriszyp).

Fixes:
------

 - Fixed handling `MDBX_opt_rp_augment_limit` for GC's records from huge transactions (Erigon/Akula/Ethereum).
 - [Fixed](https://github.com/erthink/libmdbx/issues/258) build on Android (avoid including `sys/sem.h`).
 - [Fixed](https://github.com/erthink/libmdbx/pull/261) missing copy assignment operator for `mdbx::move_result`.
 - Fixed missing `&` for `std::ostream &operator<<()` overloads.
 - Fixed unexpected `EXDEV` (Cross-device link) error from `mdbx_env_copy()`.
 - Fixed base64 encoding/decoding bugs in auxillary C++ API.
 - Fixed overflow of `pgno_t` during checking PNL on 64-bit platforms.
 - [Fixed](https://github.com/erthink/libmdbx/issues/260) excessive PNL checking after sort for spilling.
 - Reworked checking `MAX_PAGENO` and DB upper-size geometry limit.
 - [Fixed](https://github.com/erthink/libmdbx/issues/265) build for some combinations of versions of  MSVC and Windows SDK.

Minors:
-------

 - Added workaround for CLANG bug [D79919/PR42445](https://reviews.llvm.org/D79919).
 - Fixed build test on Android (using `pthread_barrier_t` stub).
 - Disabled C++20 concepts for CLANG < 14 on Android.
 - Fixed minor `unused parameter` warning.
 - Added CI for Android.
 - Refine/cleanup internal logging.
 - Refined line splitting inside hex/base58/base64 encoding to avoid `\n` at the end.
 - Added workaround for modern libstdc++ with CLANG < 4.x
 - Relaxed txn-check rules for auxiliary functions.
 - Clarified a comments and descriptions, etc.
 - Using the `-fno-semantic interposition` option to reduce the overhead to calling self own public functions.

Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
 2022-02-02 20:56:28 +03:00
Leonid Yuriev
5aeb7ccf25 mdbx: fix extra casting inside mdbx_limits_dbsize_max(). 2022-02-02 14:09:03 +03:00
Leonid Yuriev
79e1cc3bbc mdbx: add workaround for old MSVC and/or old Windows SDK. 
Resolves https://github.com/erthink/libmdbx/issues/265.

Change-Id: I295b5d9d5ecd670ccf258791bf87379a3ca17f21
 2022-02-01 13:28:59 +03:00
Leonid Yuriev
bd51e181fb mdbx: refine comment for mdbx_update_gc(). 2022-01-31 21:36:57 +03:00
Леонид Юрьев (Leonid Yuriev)
d93a13294a mdbx: explicit disable -Wattributes by #pragma for GCC < 9. 2022-01-27 17:20:12 +03:00
Leonid Yuriev
fe5199b9d1 mdbx: fix/rework PNL checking during search, refine/simplify spilled pages search. 
Fixes https://github.com/erthink/libmdbx/issues/260.

Change-Id: I8824f6af2f770c46668ee577aeeeb93b30b17a13
 2022-01-26 13:21:21 +03:00
Leonid Yuriev
9ae56332d7 mdbx: fix Coverity warnings. 2022-01-22 18:01:31 +03:00
Leonid Yuriev
03d828834b mdbx: schedule the release of v0.11.4 for 2022-01-28. 2022-01-21 15:43:17 +03:00
Leonid Yuriev
cd2c5f594c mdbx: rework checking MAX_PAGENO and DB upper-size limit. 
Related to https://github.com/erthink/libmdbx/issues/260.
 2022-01-21 02:25:27 +03:00
Leonid Yuriev
ddef217047 mdbx: relax txn-check rules for auxiliary functions. 2022-01-21 02:13:48 +03:00
Leonid Yuriev
3e28cc2a25 mdbx: allow mdbx_dbi_flags_ex() for errored transactions. 2022-01-21 02:13:48 +03:00
Leonid Yuriev
8779f665dc mdbx: fix excessive PNL checking after sort for spilling. 
This bug triggered only in the DEBUG builds or when the assertion checking is forcibly enabled.
It does not affect any core logic and cannot lead to DB corruption, data loss, and so on.

Fixes https://github.com/erthink/libmdbx/issues/260.
 2022-01-20 15:25:35 +03:00
Leonid Yuriev
52552ebfe3 mdbx++: fix MSVC warnings/errors. 2022-01-19 21:08:01 +03:00
Leonid Yuriev
a6acc9d1a3 mdbx++: fix minor comment typo. 
Change-Id: Ic169523218b366be0688ab92b76c4120df829ba3
 2022-01-17 11:06:27 +03:00
Leonid Yuriev
c588af6aca mdbx: update Copyright year. 
Change-Id: Ib3e4c0ac94882c4b2a1e167dd98e6e26dbdf48a4
 2022-01-17 11:06:14 +03:00
Leonid Yuriev
f2995ac75b mdbx++: add assertions for destination-overflow to from/to hex/base58/base64::write_bytes(). 
Change-Id: I5a1f23dcb1dc1942f9d72966a78080eacfacf546
 2022-01-15 18:52:12 +03:00
Leonid Yuriev
b5b0a9a284 mdbx++: add to_hex/to_base58/to_base64::output(std::ostream&) without using temporary objects/buffers/strings. 
Change-Id: Ideffd0e7f450307e14d780dcdeb2458c1c7e4c18
 2022-01-15 18:52:12 +03:00
Leonid Yuriev
b139d8165b mdbx++: refine line splitting inside to_hex/to_base58/to_base64 to avoid \n at the end. 
Change-Id: I53fb1f31ac5fc41567826a9035b0080fd62c8ade
 2022-01-15 18:51:56 +03:00
Leonid Yuriev
ef2e390b10 mdbx++: fix from_base64::write_bytes(). 
Change-Id: I6c90997bffab45019a77edcf2a9dd35ae80069d3
 2022-01-15 18:51:56 +03:00
Leonid Yuriev
51e6d4645c mdbx++: fix to_base64::write_bytes() for tail bytes. 
Change-Id: I60defa11b7e44dcf2ac840160ab0c833484c9d39
 2022-01-15 17:22:18 +03:00
Leonid Yuriev
2151be6cde mdbx: merge branch 'issue-260' into devel. 2022-01-15 15:31:45 +03:00
Leonid Yuriev
7a06cac680 mdbx: fix unexpected EXDEV (Cross-device link) error from mdbx_env_copy() 
Fallback to `write()` if `copy_file_range()` syscall returns `EXDEV`.
 2022-01-14 12:39:38 +03:00
Leonid Yuriev
1813bf9e53 mdbx: avoid 32-bit pgno_t overflow during checking PNL on 64-bit platforms. 
Related to https://github.com/erthink/libmdbx/issues/260.
 2022-01-13 15:42:00 +03:00
Leonid Yuriev
406cafb642 mdbx: treat the UINT64_MAX value as maximum for given option inside mdbx_env_set_option(). 2022-01-06 00:37:08 +03:00
Leonid Yuriev
b19ebf0c2e mdbx: fix minor unused parameter warning. 2022-01-05 17:47:45 +03:00
Leonid Yuriev
066bb696e3 mdbx: avoid including sys/sem.h when MDBX_LOCKING != MDBX_LOCKING_SYSV. 
Fixes https://github.com/erthink/libmdbx/issues/258.
 2022-01-05 17:47:09 +03:00
Leonid Yuriev
024900ee9c mdbx: fix/refine handling options.rp_augment_limit detent while loading GC records. 2022-01-05 11:03:21 +03:00
Leonid Yuriev
ab83d173ac mdbx: minor refine logging for internal GC-related ops. 2022-01-05 10:50:39 +03:00
Leonid Yuriev
d316bde561 mdbx: minor cleanup comment. 2022-01-05 10:45:23 +03:00
Leonid Yuriev
a6340c6b03 mdbx: minor fix open-MADV logging. 2022-01-05 10:45:15 +03:00
Leonid Yuriev
070ba5ded4 mdbx: check for GCC/clang in debug-includes (insignificant internals). 2022-01-04 13:41:25 +03:00
Leonid Yuriev
6866fa3eaa mdbx: revert returning MDBX_ENODATA from cursor-next. 2021-12-30 22:12:04 +03:00
Leonid Yuriev
9511bc491a mdbx: more unlikely(). 2021-12-30 19:40:22 +03:00
Leonid Yuriev
ae5df65af0 mdbx: refine page_alloc() and update_gc() to reduce looping. 
Resolves https://github.com/erthink/libmdbx/issues/254.
 2021-12-30 17:36:33 +03:00
Leonid Yuriev
2ba90e63b1 mdbx: remove obsolete current retired-list record from gc before reserve/backlogging free-pages to reduce looping. 2021-12-28 00:10:00 +03:00
Leonid Yuriev
b30ccbde9e mdbx: don't reserve/backlog free-pages if retired-list was already put into gc. 2021-12-28 00:10:00 +03:00
Leonid Yuriev
c3c088b8c3 mdbx: minor unify gc-update debug logging (cosmetics). 2021-12-28 00:10:00 +03:00
Leonid Yuriev
c70d2d62d1 mdbx: fix mdbx_override_meta() to avoid false-positive assertions. 
Fixed https://github.com/erthink/libmdbx/issues/253.
 2021-12-13 13:03:49 +03:00
Leonid Yuriev
4e44dc69f9 mdbx: add MDBX_SET_UPPERBOUND for cursors. 
Resolves https://github.com/erthink/libmdbx/issues/250.
 2021-12-13 04:24:34 +03:00
Leonid Yuriev
bee5df20fd mdbx: fix MDBX_SHRINK_ALLOWED collision with MDBX_ACCEDE inside mdbx_env_sync_internal(). 2021-12-13 04:12:43 +03:00
Leonid Yuriev
6f2c1e52ad mdbx: add mdbx_cursor_get_batch(). 
Resolve https://github.com/erthink/libmdbx/issues/236
 2021-12-11 05:58:39 +03:00
Leonid Yuriev
0265c847b8 mdbx++: remove preliminary label from C++ API. 2021-12-09 02:10:08 +03:00
Leonid Yuriev
f4057b2c3b mdbx: avoid extra looping inside mdbx_env_info_ex(). 2021-12-07 01:45:41 +03:00
Leonid Yuriev
839da86cac mdbx: minor fix handling MDBX_SET_LOWERBOUND to avoid MDBX_BAD_VALSIZE. 
Related to https://github.com/erthink/libmdbx/issues/248#issuecomment-986542620.
 2021-12-07 01:30:43 +03:00
Леонид Юрьев (Leonid Yuriev)
a899056fd1 mdbx: fix extra assertion. 
Fixes https://github.com/erthink/libmdbx/issues/248.
 2021-12-06 23:28:08 +03:00
Леонид Юрьев (Leonid Yuriev)
c484a92933 mdbx: minor refine prev commit. 2021-12-06 23:00:38 +03:00
Леонид Юрьев (Leonid Yuriev)
da855b13a3 mdbx: rework/simplify mdbx_env_sync_internal(). 2021-12-04 06:56:44 +03:00
Леонид Юрьев (Leonid Yuriev)
8ef8733ddc mdbx: minor refine mdbx_env_set_option(MDBX_opt_sync_bytes). 2021-12-04 06:55:41 +03:00
Leonid Yuriev
4b886e2a58 mdbx: don't return MDBX_RESULT_TRUE (-1) from mdbx_env_set_option(). 2021-12-03 15:33:22 +03:00
Леонид Юрьев (Leonid Yuriev)
d2b15b5958 mdbx: more unlikely(). 2021-12-02 20:22:37 +03:00
Леонид Юрьев (Leonid Yuriev)
d96bc98244 mdbx: minor refine/fix MDBX_ENODATA for compatibility. 
Related to https://github.com/erthink/libmdbx/issues/243
 2021-12-02 20:22:37 +03:00
Леонид Юрьев (Leonid Yuriev)
c2cab7a6a8 mdbx: using clang-format-13 (cosmetics). 2021-12-02 20:22:31 +03:00
Leonid Yuriev
0cc3aa3af8 mdbx: remove unneeded #undef P_DIRTY. 2021-11-26 17:51:37 +03:00
Leonid Yuriev
adac03729d mdbx: remove unneeded usage of <linux/sysctl.h>. 
Fixed https://github.com/erthink/libmdbx/issues/245
 2021-11-26 17:34:04 +03:00
Leonid Yuriev
76399bd643 mdbx: fix returning MDBX_NOTFOUND for non-exact seek case of MDBX_GET_BOTH. 
Fixed https://github.com/erthink/libmdbx/issues/243
 2021-11-25 19:19:04 +03:00
Leonid Yuriev
4f2aba2d22 mdbx: minor fix formatting (cosmetics). 2021-11-25 19:15:02 +03:00
Leonid Yuriev
a2141ceaac mdbx: slightly more cases to return MDBX_ENODATA. 2021-11-22 13:48:08 +03:00
Leonid Yuriev
8cb7c0f4fb mdbx: fix MDBX_PROBLEM while update an obsolete meta-page. 
Fixes https://github.com/erthink/libmdbx/issues/242
 2021-11-21 02:37:56 +03:00
Леонид Юрьев (Leonid Yuriev)
bfff6cfe85 mdbx: fix nasty sizeof(bytes) typo. 2021-11-19 16:21:44 +03:00
Leonid Yuriev
79a5802ad4 mdbx: add the note about "Visual Studio 2015 Update 3". 2021-11-11 20:38:39 +03:00
Leonid Yuriev
0e2ca3eb51 mdbx: more parallelable CMP2INT for E2K. 2021-11-10 03:17:59 +03:00
Leonid Yuriev
e488604448 mdbx: minor fix mdbx_jitter4testing() for case MDBX_DEBUG >= 2. 2021-11-10 03:05:51 +03:00
Leonid Yuriev
eaa77c91cd mdbx: drop obsolete internal mm_flags macro. 2021-11-10 02:43:43 +03:00
sasgas
74d5a42578 mdbx: fix compilation with devtoolset-9 on CentOS/RHEL 7. 
devtoolset is always using the old ABI
https://bugzilla.redhat.com/show_bug.cgi?id=1546704
https://stackoverflow.com/questions/49393888/how-can-i-use-the-new-c-11-abi-with-devtoolset-7-on-centos-rhel
 2021-11-09 13:29:23 +03:00
Leonid Yuriev
b46e5c4ce8 mdbx: simplify collection of page-ops statistic. 2021-11-07 22:14:33 +03:00
Leonid Yuriev
96139eef48 mdbx: a whole snapshot inside mdbx_env_info_ex(). 2021-11-07 21:18:10 +03:00
Leonid Yuriev
0c3a5da3cb mdbx: refine visibility of internal mdbx_osal_jitter(). 2021-11-07 19:48:14 +03:00
Leonid Yuriev
93a24abbab mdbx: fix minor/paranoid MSVC warning. 2021-10-24 02:28:28 +03:00
Leonid Yuriev
5babf0872e mdbx++: add ifndef-guard for _CRT_SECURE_NO_WARNINGS. 2021-10-22 20:14:12 +03:00
Leonid Yuriev
331232858a adds updating db-format signature during DB open. 
Fixes https://github.com/erthink/libmdbx/issues/238.
 2021-10-22 20:13:57 +03:00
Leonid Yuriev
fcb8cd2145
mdbx: alter DB-format' signature and change version to v0.11.x (not a release). 
Related to https://github.com/erthink/libmdbx/issues/238

Signed-off-by: Leonid Yuriev <leo@yuriev.ru>
 2021-10-21 15:17:18 +03:00
Leonid Yuriev
514910621e mdbx: return MDBX_CORRUPTED instead of MDBX_PAGE_NOTFOUND for invalid pages. 2021-10-15 01:11:20 +03:00
Leonid Yuriev
7251f47d5b mdbx: fix typo which lead any error conversion to the 1. 2021-10-14 20:03:37 +03:00
Leonid Yuriev
4632df5661 mdbx: cleanup non-persistent flags from meta-model. 2021-10-13 16:34:53 +03:00
Leonid Yuriev
11a5c50591 mdbx: fix turn_for_recovery() for possibility of meta-pages clashing after turn to a particular meta-page. 2021-10-13 16:34:53 +03:00
Leonid Yuriev
3092078709 mdbx: refine handling of weak or invalid meta-pages while a DB opening. 2021-10-13 16:34:53 +03:00
Leonid Yuriev
cbb71058ca mdbx: refine providing information for the @MAIN and @GC sub-databases of a last committed modification transaction's ID. 2021-10-13 16:34:25 +03:00
Leonid Yuriev
30745e0621 mdbx: refix #pragma pack for modern compilers and aligned-required arches (hotfix). 
Fix a regression after the https://github.com/erthink/libmdbx/issues/235
 2021-10-13 16:13:22 +03:00
Leonid Yuriev
64e6fa93fd mdbx: fix #pragma pack to avoid misalignment for some compilers. 
Fixes https://github.com/erthink/libmdbx/issues/235.
 2021-10-09 12:36:40 +03:00
Leonid Yuriev
fa0a38c1ac mdbx: avoid single-page (too small) shrink threshold by default when MDBX_DEBUG > 0. 2021-10-09 12:30:39 +03:00
Leonid Yuriev
fe0ec8ceca mdbx: fix and refine mdbx_update_gc() to avoid infinite looping possibility (squashed). 2021-10-09 12:29:10 +03:00
Leonid Yuriev
fe7186d549 mdbx: reflow comment (cosmetic). 2021-09-30 16:38:07 +03:00
Leonid Yuriev
d2cba98f70 mdbx: always setup madvise while opening DB. 
This partially revert 7da64b725d920b1602ec0d307581a6cdca997d05.

Hope fix https://github.com/erthink/libmdbx/issues/231
 2021-08-25 14:13:07 +03:00
Leonid Yuriev
99b75b5004 mdbx: fix/model minor Coverity issues. 2021-08-16 23:45:56 +03:00
Leonid Yuriev
42d545e579 mdbx: fix minor zero-length memcmp() UB. 2021-08-14 17:46:34 +03:00
Leonid Yuriev
b6ffec12e4 "mdbx: ignore legacy P_DIRTY flag (0x10) for nested/sub-pages. 2021-08-11 15:49:46 +03:00
Leonid Yuriev
65fa0cf8ed mdbx++: revive encode/decode to hex/base58/base64 (squashed). 2021-07-26 03:53:11 +03:00
Leonid Yuriev
046dc02f73 mdbx: make MDBX_STRINGIFYmacro public. 2021-07-26 03:53:11 +03:00
Leonid Yuriev
e731260056 mdbx: fix SIGSEGV regression while copying DB with enabled audit. 2021-07-26 02:24:15 +03:00
Leonid Yuriev
7759e52850 mdbx-windows: use MachineGuid of any size for bootid generation. 2021-07-19 12:07:45 +03:00
Leonid Yuriev
e7336e1d5e mdbx: add checks for unexpected LEAF2-pages. 
The fourth case of https://github.com/erthink/libmdbx/issues/217.
 2021-07-19 12:06:08 +03:00
Leonid Yuriev
68aef96f0a mdbx-tool: minor clarify mdbx_chk' logic key/data checks. 2021-07-19 12:06:08 +03:00
Leonid Yuriev
0b120b8fa9 mdbx-windows: fix bootid generation for case of change system' time. 2021-07-19 12:05:55 +03:00
Leonid Yuriev
28c36af67c mdbx: refine built-in ASAN option. 2021-07-16 16:04:09 +03:00
Leonid Yuriev
d67b9eaf17 mdbx: fix ASAN-regression after 1740043678951c5a1a088e6572722de438162ab2. 
Related to https://github.com/erthink/libmdbx/issues/217
 2021-07-16 16:03:49 +03:00
Leonid Yuriev
6034985686 mdbx: add MDBX_ASAN_(UN)POISON_MEMORY_REGION() macros. 2021-07-16 16:03:49 +03:00
Leonid Yuriev
7da64b725d mdbx: perform madvise only for the first process opens a DB. 2021-07-16 16:03:49 +03:00
Leonid Yuriev
8aaf5d071b mdbx: fix pagecheck(). 
Added a check that the data of the BIGDATA node (containing the target page number) is located within the boundaries of the page being checked.

The third case of https://github.com/erthink/libmdbx/issues/217.
 2021-07-16 15:55:18 +03:00
Leonid Yuriev
b6233ae2e5 mdbx-tools: minor fix/unify error counters. 2021-07-14 10:29:54 +03:00
Leonid Yuriev
fe5f008d39 mdbx-tools: skip iteration & checking records if corresponding tree is corrupted. 
Hope final for https://github.com/erthink/libmdbx/issues/217
 2021-07-14 03:59:56 +03:00
Leonid Yuriev
c8743cb9c4 mdbx: fix null-deref while override invalid meta-pages. 
Related to https://github.com/erthink/libmdbx/issues/217
 2021-07-14 03:59:52 +03:00
Leonid Yuriev
4de2dcebb5 mdbx: increase the MDBX_READERS_LIMIT to 32767. 
Fixes https://github.com/erthink/libmdbx/issues/219.
 2021-07-11 02:44:19 +03:00
Leonid Yuriev
678a80dd19 mdbx: fix hang/shutdown on big-endian platforms without __cxa_thread_atexit(). 
Change-Id: I1bf706abaaf42d5b40751d85ed7c7a83d02acaf5
 2021-07-11 02:25:39 +03:00
Leonid Yuriev
c18bf4f898 mdbx: minor clarify mdbx_mapresize(). 2021-07-11 02:25:39 +03:00
Leonid Yuriev
728e7f92b2 mdbx: minor fix mdbx_mresize() to preserve result code for read-only cases. 2021-07-11 02:25:39 +03:00
Leonid Yuriev
1740043678 mdbx: minimize the size of poisoned/unpoisoned regions to avoid ASAN hangs. 
More for second case of https://github.com/erthink/libmdbx/issues/217
 2021-07-11 02:25:26 +03:00
Leonid Yuriev
891d68838a mdbx: return MDBX_TOO_LARGE under Valgrind/ASAN if being opened DB is 100 larger than RAM. 
More for second case of https://github.com/erthink/libmdbx/issues/217
 2021-07-11 02:25:07 +03:00
Leonid Yuriev
108398c213 mdbx: refine rollback while opening weak/invalid DB. 
More for https://github.com/erthink/libmdbx/issues/217
 2021-07-11 02:24:51 +03:00
Leonid Yuriev
8bdee27248 mdbx: create/refactoring override_meta(). 2021-07-11 02:24:51 +03:00
Leonid Yuriev
00c6dc9788 mdbx: re-verify head and steady meta-pages while opening db by the first process. 
Basic fix for https://github.com/erthink/libmdbx/issues/217
 2021-07-11 02:24:06 +03:00
Leonid Yuriev
fb67682a79 mdbx: refine mdbx_validate_meta(). 2021-07-10 17:24:26 +03:00
Leonid Yuriev
5ed50a4739 mdbx: remove filesize arg from header/meta read functions (refactoring). 2021-07-10 16:10:30 +03:00
Leonid Yuriev
c30c3def8b mdbx: the filesize field of mdbx_mmap_t now is not specific for Windows. 2021-07-10 16:10:30 +03:00
Леонид Юрьев (Leonid Yuriev)
2f74f405ae mdbx: avoid returning MDBX_TXN_FULL error when possible. 2021-07-09 18:29:31 +03:00
Leonid Yuriev
39c5e66ed1 mdbx: fix safe64_reset() for platforms without atomic 64-bit compare-and-swap. 2021-07-09 17:44:27 +03:00
Leonid Yuriev
bd2bb51f0f mdbx++: rework buffer::silo to avoid use std::string. 2021-07-06 13:45:26 +03:00
Leonid Yuriev
cf5f31c577 mdbx: make __cold attribute first (cosmetic). 2021-07-03 01:51:04 +03:00
Leonid Yuriev
fa49e5a57b mdbx++: rename slice::from/to_FOO_bytes() to `slice::envisage_from/to_FOO_length()'. 2021-07-02 21:20:04 +03:00
Leonid Yuriev
5d4281fbbe mdbx: minor fix spelling. 2021-06-26 18:54:00 +03:00
Leonid Yuriev
a5e10b4ea1 mdbx: minor refine mdbx_cursor_eof(). 2021-06-18 01:17:48 +03:00
Leonid Yuriev
b8e621cb2f mdbx: mdbx: avoid log a warning about empty header during DB creation. 
Resolve https://github.com/erthink/libmdbx/issues/205.
 2021-06-18 01:09:26 +03:00
Leonid Yuriev
5db855d728 mdbx: fix false-negative mdbx_cursor_eof() result. 
Fixes https://github.com/erthink/libmdbx/issues/207.
 2021-06-17 21:44:24 +03:00
Leonid Yuriev
d7c06b1337 mdbx: partial fix for recursive SRW-lock with MDBX_NOTLS on Windows. 
Here are some changes to avoid recursive acquisition of SRW-lock,
which is still in use:
 - Read transactions don't acquire the shared SRW-lock with `MDBX_NOTLS.
 - Memory-mapping of DB is always kept while DB opened,
   therefore following limitations are:
 - DB file can't be shrinked while it used,
   including auto-shrink due to auto-compactification with corresponding geometry settings.
 - The upper limit of DB size can't be changed while DB is used.
 - The DB can grow within the upper size limit defined while opening by a first process,
   but this does not work under Wine since there is no `NtExtendSection()` function.

Partially fix https://github.com/erthink/libmdbx/issues/203
 2021-06-10 13:42:30 +03:00
Leonid Yuriev
0cd1eac6a8 mdbx: don't check other meta pages if one is specified for verification/recovery. 
Fixes https://github.com/erthink/libmdbx/issues/202
 2021-06-08 20:27:18 +03:00
Leonid Yuriev
f951f246b8 mdbx: allow to predefine/override MDBX_BUILD_TIMESTAMP for builds reproducibility. 
Resolve https://github.com/erthink/libmdbx/issues/201
 2021-06-02 14:50:22 +03:00
Leonid Yuriev
ea1fcc2246 mdbx: fix insignificantly Coverity warnings. 2021-05-28 00:54:11 +03:00
Leonid Yuriev
8915db5f83 mdbx-tools: cleanup MDBX_NOSUBDIR (needless). 2021-05-28 00:36:24 +03:00
Leonid Yuriev
a7167ce715 mdbx: allow open DB on 9P/WSL2 filesystem in exclusive mode. 
Related to https://github.com/erthink/libmdbx/issues/97
 2021-05-26 21:08:37 +03:00
Leonid Yuriev
3e0c7758ef mdbx: fallback to non-OFD locks after EINVAL. 
Related to https://github.com/erthink/libmdbx/issues/97
 2021-05-26 20:33:55 +03:00
Leonid Yuriev
b0830db25a mdbx: add 9P (WSL2 DrvFs) to blacklist of remote filesystems. 
Related to https://github.com/erthink/libmdbx/issues/97
 2021-05-26 20:33:08 +03:00
Leonid Yuriev
bcc546bdfa mdbx-windows: refix WSL1/WSL2 detection. 
Refix https://github.com/erthink/libmdbx/issues/97
 2021-05-25 15:18:15 +03:00
Leonid Yuriev
6bedb02ac0 mdbx: fix lru-counter overflow by module 2^31. 
2 of 2 fixes for https://github.com/erthink/libmdbx/issues/195
 2021-05-20 17:42:22 +03:00
Leonid Yuriev
bc6a690733 mdbx: simplify & fix mdbx_txn_keep() and mdbx_cursor_keep(). 
1 of 2 fixes for https://github.com/erthink/libmdbx/issues/195
 2021-05-20 15:10:38 +03:00
Leonid Yuriev
84b699a47c mdbx: fix elsif typo. 2021-05-18 22:44:31 +03:00
Leonid Yuriev
841fc15dd3 mdbx: little more likely/unlikely. 2021-05-12 19:24:19 +03:00
Leonid Yuriev
0875d16656 mdbx: fix/refine mdbx_env_info_ex() to distinguish a not-available and overflow cases for returned timestamps. 2021-05-12 17:51:31 +03:00
Leonid Yuriev
9c7c709b3e mdbx: avoid underflow during monotime/16dot16 conversion. 2021-05-12 17:44:18 +03:00
Leonid Yuriev
c5268f1da7 mdbx-tools: add support of page operations stat to mdbx_stat. 2021-05-12 14:41:09 +03:00
Leonid Yuriev
16d686bc42 mdbx: account msync/fsync write in page operations. 2021-05-12 14:40:58 +03:00
Leonid Yuriev
137f80e177 mdbx: refile rollback-writes inside mdbx_setup_dxb(). 2021-05-12 14:40:58 +03:00
Leonid Yuriev
7d8099fc77 mdbx: refine mdbx_wipe_steady(). 2021-05-12 14:40:58 +03:00
Leonid Yuriev
5731ad11c8 mdbx: don't use bool flags for mdbx_msync(). 2021-05-12 14:40:58 +03:00
Leonid Yuriev
76b9cb5dcc mdbx-tools: minor refine built-in usage/help. 2021-05-12 11:33:25 +03:00
Leonid Yuriev
9746dd20df mdbx-tools: refine/fix quiet mode. 2021-05-12 11:33:25 +03:00
Leonid Yuriev
ebab75642e mdbx: add public MDBX_MAYBE_UNUSED. 
This also should fix C++ mdbx API build by MSVC 2015.
 2021-05-11 21:07:40 +03:00
Leonid Yuriev
ed58ff9f81 mdbx++: add [[maybe_unused]] to internal functions which depends on platform and compiler features. 2021-05-11 16:53:47 +03:00
Leonid Yuriev
2071958f35 mdbx: remove extra notice/debug logging. 2021-05-11 00:24:01 +03:00
Leonid Yuriev
4fee14d1cf mdbx: logging all reasons of MDBX_PROBLEM. 
Related to https://github.com/erthink/libmdbx/issues/195
 2021-05-10 16:15:58 +03:00
Leonid Yuriev
244fab6c04 mdbx: fix extra conversion of an error during nested dupsort-DB update to MDBX_PROBLEM. 
Hope this fixes https://github.com/erthink/libmdbx/issues/195
 2021-05-10 16:15:03 +03:00
Leonid Yuriev
74ea79ac1b mdbx: minor reorder/shrink transaction fields. 2021-05-10 16:05:19 +03:00
Leonid Yuriev
0d100a898f mdbx: minor more const. 2021-05-10 15:53:07 +03:00
Леонид Юрьев (Leonid Yuriev)
aa1f6fbd5f
mdbx: release v0.10.0 
Acknowledgements:
-----------------
 - [Mahlon E. Smith](https://github.com/mahlonsmith) for [Ruby bindings](https://rubygems.org/gems/mdbx/).
 - [Alex Sharov](https://github.com/AskAlexSharov) for [mdbx-go](https://github.com/torquem-ch/mdbx-go), bug reporting and testing.
 - [Artem Vorotnikov](https://github.com/vorot93) for bug reporting and PR.
 - [Paolo Rebuffo](https://www.linkedin.com/in/paolo-rebuffo-8255766/), [Alexey Akhunov](https://github.com/AlexeyAkhunov) and Mark Grosberg for donations.
 - [Noel Kuntze](https://github.com/Thermi) for preliminary [Python bindings](https://github.com/Thermi/libmdbx/tree/python-bindings)

New features:
-------------
 - Added `mdbx_env_set_option()` and `mdbx_env_get_option()` for controls
   various runtime options for an environment (announce of this feature  was missed in a previous news).
 - Added `MDBX_DISABLE_PAGECHECKS` build option to disable some checks to reduce an overhead
   and detection probability of database corruption to a values closer to the LMDB.
   The `MDBX_DISABLE_PAGECHECKS=1` provides a performance boost of about 10% in CRUD scenarios,
   and conjointly with the `MDBX_ENV_CHECKPID=0` and `MDBX_TXN_CHECKOWNER=0` options can yield
   up to 30% more performance compared to LMDB.
 - Using float point (exponential quantized) representation for internal 16-bit values
   of grow step and shrink threshold when huge ones (https://github.com/erthink/libmdbx/issues/166).
   To minimize the impact on compatibility, only the odd values inside the upper half
   of the range (i.e. 32769..65533) are used for the new representation.
 - Added the `mdbx_drop` similar to LMDB command-line tool to purge or delete (sub)database(s).
 - [Ruby bindings](https://rubygems.org/gems/mdbx/) is available now by [Mahlon E. Smith](https://github.com/mahlonsmith).
 - Added `MDBX_ENABLE_MADVISE` build option which controls the use of POSIX `madvise()` hints and friends.
 - The internal node sizes were refined, resulting in a reduction in large/overflow pages in some use cases
   and a slight increase in limits for a keys size to ≈½ of page size.
 - Added to `mdbx_chk` output number of keys/items on pages.
 - Added explicit `install-strip` and `install-no-strip` targets to the `Makefile` (https://github.com/erthink/libmdbx/pull/180).
 - Major rework page splitting (af9b7b560505684249b76730997f9e00614b8113) for
     - An "auto-appending" feature upon insertion for both ascending and
       descending key sequences. As a result, the optimality of page filling
       increases significantly (more densely, less slackness) while
       inserting ordered sequences of keys,
     - A "splitting at middle" to make page tree more balanced on average.
 - Added `mdbx_get_sysraminfo()` to the API.
 - Added guessing a reasonable maximum DB size for the default upper limit of geometry (https://github.com/erthink/libmdbx/issues/183).
 - Major rework internal labeling of a dirty pages (958fd5b9479f52f2124ab7e83c6b18b04b0e7dda) for
   a "transparent spilling" feature with the gist to make a dirty pages
   be ready to spilling (writing to a disk) without further altering ones.
   Thus in the `MDBX_WRITEMAP` mode the OS kernel able to oust dirty pages
   to DB file without further penalty during transaction commit.
   As a result, page swapping and I/O could be significantly reduced during extra large transactions and/or lack of memory.
 - Minimized reading leaf-pages during dropping subDB(s) and nested trees.
 - Major rework a spilling of dirty pages to support [LRU](https://en.wikipedia.org/wiki/Cache_replacement_policies#Least_recently_used_(LRU))
   policy and prioritization for a large/overflow pages.
 - Statistics of page operations (split, merge, copy, spill, etc) now available through `mdbx_env_info_ex()`.
 - Auto-setup limit for length of dirty pages list (`MDBX_opt_txn_dp_limit` option).
 - Support `make options` to list available build options.
 - Support `make help` to list available make targets.
 - Silently `make`'s build by default.
 - Preliminary [Python bindings](https://github.com/Thermi/libmdbx/tree/python-bindings) is available now
   by [Noel Kuntze](https://github.com/Thermi) (https://github.com/erthink/libmdbx/issues/147).

Backward compatibility break:
-----------------------------
 - The `MDBX_AVOID_CRT` build option was renamed to `MDBX_WITHOUT_MSVC_CRT`.
   This option is only relevant when building for Windows.
 - The `mdbx_env_stat()` always, and `mdbx_env_stat_ex()` when called with the zeroed transaction parameter,
   now internally start temporary read transaction and thus may returns `MDBX_BAD_RSLOT` error.
   So, just never use deprecated `mdbx_env_stat()' and call `mdbx_env_stat_ex()` with transaction parameter.
 - The build option `MDBX_CONFIG_MANUAL_TLS_CALLBACK` was removed and now just a non-zero value of
   the `MDBX_MANUAL_MODULE_HANDLER` macro indicates the requirement to manually call `mdbx_module_handler()`
   when loading libraries and applications uses statically linked libmdbx on an obsolete Windows versions.

Fixes:
------
 - Fixed performance regression due non-optimal C11 atomics usage (https://github.com/erthink/libmdbx/issues/160).
 - Fixed "reincarnation" of subDB after it deletion (https://github.com/erthink/libmdbx/issues/168).
 - Fixed (disallowing) implicit subDB deletion via operations on `@MAIN`'s DBI-handle.
 - Fixed a crash of `mdbx_env_info_ex()` in case of a call for a non-open environment (https://github.com/erthink/libmdbx/issues/171).
 - Fixed the selecting/adjustment values inside `mdbx_env_set_geometry()` for implicit out-of-range cases (https://github.com/erthink/libmdbx/issues/170).
 - Fixed `mdbx_env_set_option()` for set initial and limit size of dirty page list ((https://github.com/erthink/libmdbx/issues/179).
 - Fixed an unreasonably huge default upper limit for DB geometry (https://github.com/erthink/libmdbx/issues/183).
 - Fixed `constexpr` specifier for the `slice::invalid()`.
 - Fixed (no)readahead auto-handling (https://github.com/erthink/libmdbx/issues/164).
 - Fixed non-alloy build for Windows.
 - Switched to using Heap-functions instead of LocalAlloc/LocalFree on Windows.
 - Fixed `mdbx_env_stat_ex()` to returning statistics of the whole environment instead of MainDB only (https://github.com/erthink/libmdbx/issues/190).
 - Fixed building by GCC 4.8.5 (added workaround for a preprocessor's bug).
 - Fixed building C++ part for iOS <= 13.0 (unavailability of  `std::filesystem::path`).
 - Fixed building for Windows target versions prior to Windows Vista (`WIN32_WINNT < 0x0600`).
 - Fixed building by MinGW for Windows (https://github.com/erthink/libmdbx/issues/155).

TODO for a next releases:
-------------------------
 - [Get rid of dirty-pages list in MDBX_WRITEMAP mode](https://github.com/erthink/libmdbx/issues/193).
 - [Large/Overflow pages accounting for dirty-room](https://github.com/erthink/libmdbx/issues/192).
 - [C++ Buffer issue](https://github.com/erthink/libmdbx/issues/191).
 - Finalize C++ API (few typos and trivia bugs are still likely for now).
 - [Support for RAW devices](https://github.com/erthink/libmdbx/issues/124).
 - [Test framework issue](https://github.com/erthink/libmdbx/issues/127).
 - [Support MessagePack for Keys & Values](https://github.com/erthink/libmdbx/issues/115).
 - [Engage new terminology](https://github.com/erthink/libmdbx/issues/137).
 - Packages for [Astra Linux](https://astralinux.ru/), [ALT Linux](https://www.altlinux.org/), [ROSA Linux](https://www.rosalinux.ru/), Fedora/RHEL, Debian/Ubuntu.
 2021-05-09 03:01:59 +03:00
Leonid Yuriev
43f1823cdc mdbx: minor alignment fix for lck-less stub to avoid false-positive alarm from UndefinedBehaviorSanitize. 
Change-Id: I8d9bc00ccfc7228cb42f9419cf77e8eadb92f918
 2021-05-08 22:43:16 +03:00
Leonid Yuriev
e75033b1f4 mdbx: drop some unused stuff. 
Change-Id: I136dcf891757d5c1a35a8082ee90e8cf07c016c6
 2021-05-08 18:40:14 +03:00
Leonid Yuriev
ecd3c06932 mdbx-windows: remove MDBX_CONFIG_MANUAL_TLS_CALLBACK build option and add MDBX_MANUAL_MODULE_HANDLER macro. 
Briefly:
 - Now constructor/destructor of "Thread Local Storage" handled automatically when possible.
 - Otherwise the MDBX_CONFIG_MANUAL_TLS_CALLBACK macro defined to 1 to indicate that mdbx_module_handle() should be called manually.
 - Corresponding build option MDBX_CONFIG_MANUAL_TLS_CALLBACK was removed.

Related to https://github.com/erthink/libmdbx/issues/155

Change-Id: Ic4e6a34b44f874676f0ab212ff473460e3d80559
 2021-05-08 18:39:18 +03:00
Leonid Yuriev
11a521f10a mdbx-windows: minor fix mdbx_free() for MDBX_WITHOUT_MSVC_CRT=ON. 
Related to https://github.com/erthink/libmdbx/issues/155

Change-Id: I378d9c47b51c55e1e61bc7f6269cbc8182265870
 2021-05-08 18:38:53 +03:00
Leonid Yuriev
5ae120af80 mdbx-windows: more crutches for MinGW. 
More for https://github.com/erthink/libmdbx/issues/155

Change-Id: I7de6122ff160372b2dcfd2a0a26e332cb52d0560
 2021-05-08 18:38:44 +03:00
Leonid Yuriev
33e8b19ea4 mdbx-windows: check _WIN32_WINNT definition. 
Related to https://github.com/erthink/libmdbx/issues/155

Change-Id: I49c294bd0fa055026b742a12a6f6ea9cd805cf02
 2021-05-08 18:38:36 +03:00
Leonid Yuriev
ab1fc94a5b mdbx-windows: fix build for Windows XP/2000 (_WIN32_WINNT < 0x0600). 
Related to https://github.com/erthink/libmdbx/issues/155

Change-Id: Ibd795817e05b6da39ef270ce7b55b31d963d07b0
 2021-05-08 18:38:26 +03:00
Leonid Yuriev
c15ad25073 mdbx: add minor workaround for E2K LCC's bug. 
Related to https://bugs.mcst.ru/bugzilla/show_bug.cgi?id=6011

Change-Id: I562dbcedb3c172e0646dcb03ef6d5cd3f14e02b9
 2021-05-07 19:05:10 +03:00
Leonid Yuriev
a14d6bcb11 mdbx: fix mis-defined MDBX_FORCE_ASSERTIONS. 
Change-Id: I6f7ad4fdd6aca99a4f908d634af0015a6e95c0d1
 2021-05-06 17:03:57 +03:00
Leonid Yuriev
1275bdb623 mdbx: add MDBX_opt_merge_threshold_16dot16_percent option. 
Change-Id: I416f85096e4b6fd21f2db622f07f31103cb9074a
 2021-05-06 02:05:33 +03:00
Leonid Yuriev
7d54518d60 mdbx: add workaround for GCC 4.8 preprocessor bug. 
Change-Id: I965cfbcd43596ef896a6b849dd043804bdb3f41f
 2021-05-03 15:00:53 +03:00
Leonid Yuriev
d6e67e3982 mdbx: minor retine options.h 
Change-Id: Ic340e6cea25df3e5dddf920ba851e39e714033e0
 2021-05-03 15:00:53 +03:00
Leonid Yuriev
c914d417d2 mdbx: refine Doxygen's API description. 
Change-Id: Ifd326eded287c68c6a95b6c9be22847d6efa5678
 2021-05-03 15:00:53 +03:00
Leonid Yuriev
df387ad943 mdbx: fix minor MSVC warnings. 
Change-Id: I9a2c774f6020bbe5c77e423a1f54110b12bdcb41
 2021-05-02 17:11:50 +03:00
Leonid Yuriev
732d3cd2d4 mdbx: fix/rework mdbx_env_stat_ex() to return stat of the whole env. 
Resolves https://github.com/erthink/libmdbx/issues/190

Included fix for the first version of this change.

Change-Id: I8357de24fda2ab4c93510574055b52d6e16b0c15
 2021-05-02 17:10:37 +03:00
Leonid Yuriev
a56c72d190 mdbx: minor refine internal audit. 
Change-Id: I0e177e9c9ecf1da97084c175f80b3e987391194c
 2021-05-02 15:14:30 +03:00
Leonid Yuriev
9ba8d5892a mdbx-tools: avoid use mdbx_env_stat_ex(). 
Change-Id: I484ead640967b3a6caa3c2536983a34956ae1827
 2021-05-01 21:20:18 +03:00
Leonid Yuriev
fe04c98327 mdbx-tools: cosmetic refine mdbx_chk. 
Change-Id: I815baf437d9e0af3c78f9df717b0fb669863993c
 2021-05-01 20:40:52 +03:00
Leonid Yuriev
0054f5388a mdbx-windows: use Heap-functions instead of LocalAlloc/LocalFree. 
Change-Id: I85f8a4c888bf98f4792f2a5e522d24ee671f060c
 2021-04-30 02:24:36 +03:00
Leonid Yuriev
161b00a4b6 mdbx: refine/rearrange build options. 
Change-Id: Ic27bf2b1f22e7ed9e6a1db9a1ed2496b1bb1239b
 2021-04-30 02:01:22 +03:00
Leonid Yuriev
6b6165cdeb mdbx: rename internal defines/macros to distinguish it from build options. 
Change-Id: I42fe240e6ddd5d34c8fec38633849ebdcce0e116
 2021-04-30 01:59:48 +03:00
Leonid Yuriev
d27e48ae1b mdbx-windows: rename MDBX_AVOID_CRT build option to MDBX_DISABLE_MSVC_CRT. 
Change-Id: I124c9d86d4a596718bd9671a7c1b538b84c2dd0f
 2021-04-30 01:59:48 +03:00
Leonid Yuriev
c3b2c1fdef mdbx: handling the case when all dirty pages are unspillable. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Iff7a123c1c171371624d8b7ae497941ec4622385
 2021-04-28 23:38:41 +03:00
Leonid Yuriev
aa5a39160a mdbx: internal build option for debuging dirtyroom reservation and spilling. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Ia315ae2cae5f5b5b82891843de22487db0139df3
 2021-04-28 23:38:38 +03:00
Leonid Yuriev
39230c9a22 mdbx: more debug/loggin for LRU spilling. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Ifcfabf8864830adf3c50fe8d01d24cccb950dcbb
 2021-04-28 23:38:34 +03:00
Leonid Yuriev
9fed78c92d mdbx: fix LRU spilling internal prio celling. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Ia4be12e063d1754a38889dc311eb1314b3676c21
 2021-04-28 23:38:32 +03:00
Leonid Yuriev
329a2a50e6 mdbx: account loose-pages during spilling and do purge ones if a lack of dirtyroom. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: I50c11195ad31615181f8359e123c1ffe2ebcfc30
 2021-04-28 23:38:29 +03:00
Leonid Yuriev
17116b9b46 mdbx: raise MDBX_TXN_FULL if spilled less that half of needed. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: I3c1a0e1bea6b6ad9bf41347a569e5833e82d8edc
 2021-04-28 23:38:26 +03:00
Leonid Yuriev
e57e42d0f8 mdbx: use single pointer to lck-less stub. 
Change-Id: I80542423386bd54cac574d1678af2620f147c769
 2021-04-28 23:38:26 +03:00
Leonid Yuriev
7d249c97ad mdbx: double dirtyroom reserve for page stack. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: I211fc9c849c9a45254e699d713475898fd20a58e
 2021-04-28 23:38:24 +03:00
Leonid Yuriev
4111d7238c mdbx: spill pages during prepare backlog for update_gc(). 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: I88463a860da0a53420904daf1b1f39e5ca4c97e5
 2021-04-28 23:38:21 +03:00
Leonid Yuriev
760cf515a7 mdbx: minor refactoring cursor_put(). 
Change-Id: Iacdc02f70278382bf0e7fb496bb57f4010403dd0
 2021-04-28 23:38:21 +03:00
Leonid Yuriev
7cbd500534 mdbx: minor simplify dirtyroom accounting. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Ia0afcffe5a245aaa9500edc70b2cbca1735070ac
 2021-04-28 23:38:19 +03:00
Leonid Yuriev
89558657ea mdbx: auto-setup MDBX_opt_txn_dp_limit. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: I8f8c4f1a2600462bc25b4fad3010a4e6ac70b386
 2021-04-28 23:38:16 +03:00
Leonid Yuriev
6134220b8a mdbx: drop P_KEEP flag and use LRU-label insted of. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Ifd3bd2533b40525dd5b0aa05df421c87462c2439
 2021-04-28 23:37:48 +03:00
Leonid Yuriev
975413b48d mdbx: more for page operation statistics. 
Change-Id: I19fc26dfc9f9627eecc8d505e95d33bad76f2b2f
 2021-04-27 18:19:13 +03:00
Leonid Yuriev
f95a277ac5 mdbx: statistics of page operations. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Ia00e6e0df9a65f880517ca33e7f444a0526b96e1
 2021-04-27 18:02:11 +03:00
Leonid Yuriev
3bbf5d03e2 mdbx: lowering the logging level for page unspill. 
Change-Id: I03f9bdaa644534d1c1e257e0394549c90a56d612
 2021-04-27 16:58:35 +03:00
Leonid Yuriev
13a784eeed mdbx: rearrange MDBX_env ro/rw fields. 
Change-Id: I055aa6ad592930be2d5d676977fbaa84149472ce
 2021-04-27 16:58:31 +03:00
Leonid Yuriev
8b95be91db mdbx: fix minor Coverity warnings. 
Change-Id: Id6ab528b597e62b05b5fc4ab87bbe93cbce6573c
 2021-04-27 00:12:09 +03:00
Leonid Yuriev
427b480f68 mdbx: add MDBX_FAKE_SPILL_WRITEMAP build option. 
Change-Id: I2c7f9267b0e725674b8bc73547bedd8478231948
 2021-04-26 18:18:51 +03:00
Leonid Yuriev
4fa5e95241 mdbx: LRU spilling feature (squashed). 
Resolves https://github.com/erthink/libmdbx/issues/186

Change-Id: Ie3318964e4e2adbeb77738b8301c45075080850b
 2021-04-26 17:51:11 +03:00
Leonid Yuriev
3e272d339a mdbx: drop mdbx_dpl_find() and always promote child LRU-serial to the parent txn. 
More for https://github.com/erthink/libmdbx/issues/186

Change-Id: Ifeac634e9fa1dee6e2b3375c868cc0fa16422099
 2021-04-26 14:08:30 +03:00
Leonid Yuriev
31cfce4ca5 mdbx: fix minor MSVC warnings. 
Change-Id: I99ab0855334c01796c57f13142127d184ab1291e
 2021-04-26 10:56:18 +03:00
Leonid Yuriev
66e5078e28 mdbx: refine page_touch() and dirtyroom checking. 
More for https://github.com/erthink/libmdbx/issues/186

Change-Id: I81a6d94e64a60ccb642d0297a9afb2abddf51c49
 2021-04-26 02:47:25 +03:00
Leonid Yuriev
f29283a620 mdbx: merge branch master into devel. 
Change-Id: I0568e92101e8e3ddf3572c6973542752cd1c235d
 2021-04-26 02:08:03 +03:00
Leonid Yuriev
4583f4bc8c mdbx-windows: minor fix assertion inside mdbx_suspend_threads_before_remap(). 
Change-Id: I2026f8400aea3d7c0763556be3925206d8485684
 2021-04-26 02:06:28 +03:00
Leonid Yuriev
55f41f40a1 mdbx: rename log2n_powerof2(). 
Change-Id: I22290630186c5041b6ae242d0db1f5c5f36da4ec
 2021-04-26 02:06:28 +03:00
Leonid Yuriev
d1fea74d0e mdbx: preparation for LRU-spilling. 
Initial for https://github.com/erthink/libmdbx/issues/186

Change-Id: I9a6d60cc142ff7490e2a23defe9654d7e3337edc
 2021-04-26 02:06:01 +03:00
Leonid Yuriev
934ca25ab2 mdbx: fix MDBX_EACCESS from mdbx_env_set_option() if env not yet opened. 
Change-Id: I0da15b71919a1787e5cc54441b5e41941afabf66
 2021-04-26 00:52:47 +03:00
Leonid Yuriev
b21ad733ea mdbx: refine cursor_copy(), add cursor_restore(). 
Change-Id: I3ac5883c95aaa0e13970a8176f0af5e374d811a1
 2021-04-23 16:06:23 +03:00
Leonid Yuriev
9b80da87f0 mdbx-docs: update bits.md 
Change-Id: Id6cfd9579f31b862f3dc0069862019d08f363f9e
 2021-04-23 16:06:23 +03:00
Leonid Yuriev
0b9272209a mdbx: minor refine spilling. 
Change-Id: I243ebff592ac7ad544bbd382c93d2e5d4444b081
 2021-04-23 16:04:35 +03:00
Leonid Yuriev
416a802789 mdbx: add missing err-log for MDBX_TXN_FULL during page allocation. 
Minor for https://github.com/erthink/libmdbx/issues/186

Change-Id: Iadb868c64fba4e9d5a4ee7df093f0cfb9288bf93
 2021-04-21 13:24:54 +03:00
Leonid Yuriev
aaf465b942 mdbx: add missing err-log for MDBX_TXN_FULL during page allocation. 
Change-Id: Iadb868c64fba4e9d5a4ee7df093f0cfb9288bf93
 2021-04-21 12:20:32 +03:00
Leonid Yuriev
c2d8c35daa mdbx: the radix-sort threshold reduced to 333. 
Change-Id: Ic69b6c3740f71997cd014a82fb32a8202ddb65e0
 2021-04-21 00:13:57 +03:00
Leonid Yuriev
eabb29d9c4 mdbx: add MDBX_xPL_PREALLOC_FOR_RADIXSORT build options. 
Change-Id: Icf15558830f86f98c5e970a7eec69cc045045467
 2021-04-21 00:13:51 +03:00
Leonid Yuriev
3b5e2680e9 mdbx: fix non-alloy build for windows. 
Change-Id: Id93eb069543f081c3b1c9734ed65f4afd80a444d
 2021-04-20 22:09:48 +03:00
Leonid Yuriev
7a0d6c6196 mdbx: use POSIX_MADV_DONTNEED. 
Change-Id: I2020eb8ec225a49331cf4dc6c5a24d6559916284
 2021-04-20 22:09:48 +03:00
Leonid Yuriev
b3b6a48797 mdbx: fix minor warning on Solaris/SunOs. 2021-04-20 22:09:48 +03:00
Leonid Yuriev
b7110eb4d4 mdbx: fix non-alloy build for windows. 
Change-Id: Id93eb069543f081c3b1c9734ed65f4afd80a444d
 2021-04-20 15:17:51 +03:00
Leonid Yuriev
c0704db03a mdbx: use POSIX_MADV_DONTNEED. 
Change-Id: I2020eb8ec225a49331cf4dc6c5a24d6559916284
 2021-04-20 15:17:51 +03:00
Leonid Yuriev
e82359bca0 mdbx: fix minor warning on Solaris/SunOs. 2021-04-20 15:17:51 +03:00
Leonid Yuriev
dc4ab96d4d mdbx-tools: fix mdbx_drop help/usage message. 
Change-Id: I100daa14641eaa351930034ba8806a7e82085289
 2021-04-20 15:17:51 +03:00
Leonid Yuriev
bedd13d411 mdbx-tools: minor fix error messages. 
Change-Id: Id2efb3db70b5df49e84bc9a8bb47645afe0eb187
 2021-04-20 15:17:51 +03:00
Leonid Yuriev
17d8ed4bad mdbx-tools: fix mdbx_drop help/usage message. 
Change-Id: I100daa14641eaa351930034ba8806a7e82085289
 2021-04-18 19:28:22 +03:00
Leonid Yuriev
aa58498d23 mdbx-tools: minor fix error messages. 
Change-Id: Id2efb3db70b5df49e84bc9a8bb47645afe0eb187
 2021-04-18 19:26:15 +03:00
Leonid Yuriev
4cf8496422 mdbx: avoid read leaf-pages during dropping trees. 
Change-Id: Ib8034a76a150c213fddcca0f7df971f63253b7a5
 2021-04-18 19:20:16 +03:00
Leonid Yuriev
d1561dc357 mdbx: refactoring page_alloc() result type. 
Change-Id: I4afc73603d610e4e28b952a38dfa9f30d56809bf
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
3f6758e18d mdbx: minor refactoring mdbx_cursor_set(). 
Change-Id: If3c984fb5ac2f77762501ac720080e2caaacac3c
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
b59836e6d9 mdbx: minor refactoring: use cursor_set_result. 
Change-Id: Ice724c541bc4127aae0e9ea85b4fc1d7c1b68471
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
3eb48340ad mdbx: minor refactoring: use node_result. 
Change-Id: I489b15c174644b2f414925f5bacbe83c091a0692
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
b3aba4691b mdbx: minor refactoring: use page_result. 
Change-Id: I7749d9463832ce9b270d06f04f43e413d5ba26b7
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
e4db019f47 mdbx: minor refine cursor_set_ex(). 
Change-Id: I0951a917c1e4c9e372f29c64fda0e1b267f48cdd
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
dc7098b3fb mdbx: minor refine page_new(). 
Change-Id: Iafdea1c592c346c7841a39cdfc96436460b27bcb
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
6d3ff10165 mdbx: simplify page_unspill(). 
Change-Id: Iaa182d1feaf2abb59015725f000a3d9e0c57de8d
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
3622433cf4 mdbx: fix page_copy() for unspilling a large/overflow page. 
Change-Id: I46c6b6bf94c877c1c0142116c404692960dc5ab7
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
9c9cdfdb6d mdbx: minor fix mdbx_is_dirty(). 
Change-Id: Ia539165ddd62a40e13fdfec48de754961681d40f
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
a0a4bbaa7c mdbx: drop goto/label in prev commit. 
More for https://github.com/erthink/libmdbx/issues/164

Change-Id: Ic708c3ba51dd22d08e96d0bef4632a0beb19999b
 2021-04-18 10:24:11 +03:00
Leonid Yuriev
3e0fad1cf6 mdbx: rework (NO)READAHEAD handling. 
Resolves https://github.com/erthink/libmdbx/issues/164

---

NOTE: Seems there is a bug in the Mach/Darwin/OSX kernel,
because MADV_WILLNEED with offset != 0 may cause SIGBUS
on following access to the hinted region.

19.6.0 Darwin Kernel Version 19.6.0: Tue Jan 12 22:13:05 PST 2021; root:xnu-6153.141.16~1/RELEASE_X86_64 x86_64

Change-Id: I11ebbf2bd35e3dba9d078be16cb5678aecf8329c
 2021-04-17 01:12:16 +03:00
Leonid Yuriev
958fd5b947 mdbx: use mp_txnid instead of P_DIRTY (squashed). 
Change-Id: I7e3174611b8819d8050b84d835fe976b87c77c0c
 2021-04-14 19:27:13 +03:00
Leonid Yuriev
f587a74597 mdbx: drop mx_dbistate. 
Change-Id: I61616ee1436ed99db93a1036c9bb7d4db7157c8f
 2021-04-14 00:26:14 +03:00
Leonid Yuriev
5e334fa830 mdbx: extract mdbx_touch_dbi(). 
Change-Id: I3b077cefe44f4c2db099b2050e1fd72a1aaa2687
 2021-04-14 00:25:15 +03:00
Leonid Yuriev
f9977975ae mdbx: remove extra TXN_DBI_CHANGED() check. 
Change-Id: If2014f7bfbfab98f3c0c357f2475da57efc50693
 2021-04-14 00:23:36 +03:00
Leonid Yuriev
e35c92eabb mdbx: refine mdbx_page_new(). 
Change-Id: Ief0cb313c39f7b11fb1cd611d9b0497c011a7a7a
 2021-04-14 00:19:34 +03:00
Leonid Yuriev
49296cad14 mdbx: refactor mdbx_drop_tree() & drop mdbx_retire pgno(). 
Change-Id: I37f33d7a11e958ccc1c48a6af31599deb677038d
 2021-04-10 22:20:31 +03:00
Leonid Yuriev
dfed1dbc17 mdbx: refactoring: merge page_retire() and page_loose(). 
Change-Id: I4b47c11eedad6ef1b83dbf3d34977cdd8fa79505
 2021-04-10 22:20:24 +03:00
Leonid Yuriev
20299f87cb mdbx: minor refine get_reasonable_db_maxsize(). 
Change-Id: I5d654940a325c0c3966a59a37615820587c52c78
 2021-04-10 17:48:31 +03:00
Leonid Yuriev
7f5cbf7dd8 mdbx: avoid float-point ops in prev commit. 
Change-Id: I7fadc5096f49502c01a60436840bb1a87dfe27bd
 2021-04-07 14:29:14 +03:00
Leonid Yuriev
c14e4235ee mdbx: guess a reasonable maximum DB size for the default upper limit of geometry. 
Fixes https://github.com/erthink/libmdbx/issues/183

Change-Id: Ic7b616e229d3008fda49e5a04121e22997ac53ea
 2021-04-07 05:23:06 +03:00