ISAR/libmdbx
3
0
Fork 0
mirror of https://github.com/isar/libmdbx.git synced 2025-01-08 08:24:14 +08:00
Code Issues Packages Projects Releases Wiki Activity
3,522 Commits 5 Branches 68 Tags 13 MiB
Commit Graph

659 Commits

Author SHA1 Message Date
Leonid Yuriev
7a06cac680 mdbx: fix unexpected EXDEV (Cross-device link) error from mdbx_env_copy() 
Fallback to `write()` if `copy_file_range()` syscall returns `EXDEV`.
 2022-01-14 12:39:38 +03:00
Leonid Yuriev
1813bf9e53 mdbx: avoid 32-bit pgno_t overflow during checking PNL on 64-bit platforms. 
Related to https://github.com/erthink/libmdbx/issues/260.
 2022-01-13 15:42:00 +03:00
Leonid Yuriev
406cafb642 mdbx: treat the UINT64_MAX value as maximum for given option inside mdbx_env_set_option(). 2022-01-06 00:37:08 +03:00
Leonid Yuriev
b19ebf0c2e mdbx: fix minor unused parameter warning. 2022-01-05 17:47:45 +03:00
Leonid Yuriev
024900ee9c mdbx: fix/refine handling options.rp_augment_limit detent while loading GC records. 2022-01-05 11:03:21 +03:00
Leonid Yuriev
ab83d173ac mdbx: minor refine logging for internal GC-related ops. 2022-01-05 10:50:39 +03:00
Leonid Yuriev
d316bde561 mdbx: minor cleanup comment. 2022-01-05 10:45:23 +03:00
Leonid Yuriev
a6340c6b03 mdbx: minor fix open-MADV logging. 2022-01-05 10:45:15 +03:00
Leonid Yuriev
6866fa3eaa mdbx: revert returning MDBX_ENODATA from cursor-next. 2021-12-30 22:12:04 +03:00
Leonid Yuriev
9511bc491a mdbx: more unlikely(). 2021-12-30 19:40:22 +03:00
Leonid Yuriev
ae5df65af0 mdbx: refine page_alloc() and update_gc() to reduce looping. 
Resolves https://github.com/erthink/libmdbx/issues/254.
 2021-12-30 17:36:33 +03:00
Leonid Yuriev
2ba90e63b1 mdbx: remove obsolete current retired-list record from gc before reserve/backlogging free-pages to reduce looping. 2021-12-28 00:10:00 +03:00
Leonid Yuriev
b30ccbde9e mdbx: don't reserve/backlog free-pages if retired-list was already put into gc. 2021-12-28 00:10:00 +03:00
Leonid Yuriev
c3c088b8c3 mdbx: minor unify gc-update debug logging (cosmetics). 2021-12-28 00:10:00 +03:00
Leonid Yuriev
c70d2d62d1 mdbx: fix mdbx_override_meta() to avoid false-positive assertions. 
Fixed https://github.com/erthink/libmdbx/issues/253.
 2021-12-13 13:03:49 +03:00
Leonid Yuriev
4e44dc69f9 mdbx: add MDBX_SET_UPPERBOUND for cursors. 
Resolves https://github.com/erthink/libmdbx/issues/250.
 2021-12-13 04:24:34 +03:00
Leonid Yuriev
bee5df20fd mdbx: fix MDBX_SHRINK_ALLOWED collision with MDBX_ACCEDE inside mdbx_env_sync_internal(). 2021-12-13 04:12:43 +03:00
Leonid Yuriev
6f2c1e52ad mdbx: add mdbx_cursor_get_batch(). 
Resolve https://github.com/erthink/libmdbx/issues/236
 2021-12-11 05:58:39 +03:00
Leonid Yuriev
f4057b2c3b mdbx: avoid extra looping inside mdbx_env_info_ex(). 2021-12-07 01:45:41 +03:00
Leonid Yuriev
839da86cac mdbx: minor fix handling MDBX_SET_LOWERBOUND to avoid MDBX_BAD_VALSIZE. 
Related to https://github.com/erthink/libmdbx/issues/248#issuecomment-986542620.
 2021-12-07 01:30:43 +03:00
Леонид Юрьев (Leonid Yuriev)
a899056fd1 mdbx: fix extra assertion. 
Fixes https://github.com/erthink/libmdbx/issues/248.
 2021-12-06 23:28:08 +03:00
Леонид Юрьев (Leonid Yuriev)
c484a92933 mdbx: minor refine prev commit. 2021-12-06 23:00:38 +03:00
Леонид Юрьев (Leonid Yuriev)
da855b13a3 mdbx: rework/simplify mdbx_env_sync_internal(). 2021-12-04 06:56:44 +03:00
Леонид Юрьев (Leonid Yuriev)
8ef8733ddc mdbx: minor refine mdbx_env_set_option(MDBX_opt_sync_bytes). 2021-12-04 06:55:41 +03:00
Leonid Yuriev
4b886e2a58 mdbx: don't return MDBX_RESULT_TRUE (-1) from mdbx_env_set_option(). 2021-12-03 15:33:22 +03:00
Леонид Юрьев (Leonid Yuriev)
d2b15b5958 mdbx: more unlikely(). 2021-12-02 20:22:37 +03:00
Леонид Юрьев (Leonid Yuriev)
d96bc98244 mdbx: minor refine/fix MDBX_ENODATA for compatibility. 
Related to https://github.com/erthink/libmdbx/issues/243
 2021-12-02 20:22:37 +03:00
Леонид Юрьев (Leonid Yuriev)
c2cab7a6a8 mdbx: using clang-format-13 (cosmetics). 2021-12-02 20:22:31 +03:00
Leonid Yuriev
76399bd643 mdbx: fix returning MDBX_NOTFOUND for non-exact seek case of MDBX_GET_BOTH. 
Fixed https://github.com/erthink/libmdbx/issues/243
 2021-11-25 19:19:04 +03:00
Leonid Yuriev
4f2aba2d22 mdbx: minor fix formatting (cosmetics). 2021-11-25 19:15:02 +03:00
Leonid Yuriev
a2141ceaac mdbx: slightly more cases to return MDBX_ENODATA. 2021-11-22 13:48:08 +03:00
Leonid Yuriev
8cb7c0f4fb mdbx: fix MDBX_PROBLEM while update an obsolete meta-page. 
Fixes https://github.com/erthink/libmdbx/issues/242
 2021-11-21 02:37:56 +03:00
Леонид Юрьев (Leonid Yuriev)
bfff6cfe85 mdbx: fix nasty sizeof(bytes) typo. 2021-11-19 16:21:44 +03:00
Leonid Yuriev
eaa77c91cd mdbx: drop obsolete internal mm_flags macro. 2021-11-10 02:43:43 +03:00
Leonid Yuriev
b46e5c4ce8 mdbx: simplify collection of page-ops statistic. 2021-11-07 22:14:33 +03:00
Leonid Yuriev
96139eef48 mdbx: a whole snapshot inside mdbx_env_info_ex(). 2021-11-07 21:18:10 +03:00
Leonid Yuriev
0c3a5da3cb mdbx: refine visibility of internal mdbx_osal_jitter(). 2021-11-07 19:48:14 +03:00
Leonid Yuriev
93a24abbab mdbx: fix minor/paranoid MSVC warning. 2021-10-24 02:28:28 +03:00
Leonid Yuriev
331232858a adds updating db-format signature during DB open. 
Fixes https://github.com/erthink/libmdbx/issues/238.
 2021-10-22 20:13:57 +03:00
Leonid Yuriev
fcb8cd2145
mdbx: alter DB-format' signature and change version to v0.11.x (not a release). 
Related to https://github.com/erthink/libmdbx/issues/238

Signed-off-by: Leonid Yuriev <leo@yuriev.ru>
 2021-10-21 15:17:18 +03:00
Leonid Yuriev
514910621e mdbx: return MDBX_CORRUPTED instead of MDBX_PAGE_NOTFOUND for invalid pages. 2021-10-15 01:11:20 +03:00
Leonid Yuriev
7251f47d5b mdbx: fix typo which lead any error conversion to the 1. 2021-10-14 20:03:37 +03:00
Leonid Yuriev
4632df5661 mdbx: cleanup non-persistent flags from meta-model. 2021-10-13 16:34:53 +03:00
Leonid Yuriev
11a5c50591 mdbx: fix turn_for_recovery() for possibility of meta-pages clashing after turn to a particular meta-page. 2021-10-13 16:34:53 +03:00
Leonid Yuriev
3092078709 mdbx: refine handling of weak or invalid meta-pages while a DB opening. 2021-10-13 16:34:53 +03:00
Leonid Yuriev
cbb71058ca mdbx: refine providing information for the @MAIN and @GC sub-databases of a last committed modification transaction's ID. 2021-10-13 16:34:25 +03:00
Leonid Yuriev
30745e0621 mdbx: refix #pragma pack for modern compilers and aligned-required arches (hotfix). 
Fix a regression after the https://github.com/erthink/libmdbx/issues/235
 2021-10-13 16:13:22 +03:00
Leonid Yuriev
fa0a38c1ac mdbx: avoid single-page (too small) shrink threshold by default when MDBX_DEBUG > 0. 2021-10-09 12:30:39 +03:00
Leonid Yuriev
fe0ec8ceca mdbx: fix and refine mdbx_update_gc() to avoid infinite looping possibility (squashed). 2021-10-09 12:29:10 +03:00
Leonid Yuriev
fe7186d549 mdbx: reflow comment (cosmetic). 2021-09-30 16:38:07 +03:00
Leonid Yuriev
d2cba98f70 mdbx: always setup madvise while opening DB. 
This partially revert 7da64b725d920b1602ec0d307581a6cdca997d05.

Hope fix https://github.com/erthink/libmdbx/issues/231
 2021-08-25 14:13:07 +03:00
Leonid Yuriev
99b75b5004 mdbx: fix/model minor Coverity issues. 2021-08-16 23:45:56 +03:00
Leonid Yuriev
42d545e579 mdbx: fix minor zero-length memcmp() UB. 2021-08-14 17:46:34 +03:00
Leonid Yuriev
b6ffec12e4 "mdbx: ignore legacy P_DIRTY flag (0x10) for nested/sub-pages. 2021-08-11 15:49:46 +03:00
Leonid Yuriev
046dc02f73 mdbx: make MDBX_STRINGIFYmacro public. 2021-07-26 03:53:11 +03:00
Leonid Yuriev
e731260056 mdbx: fix SIGSEGV regression while copying DB with enabled audit. 2021-07-26 02:24:15 +03:00
Leonid Yuriev
e7336e1d5e mdbx: add checks for unexpected LEAF2-pages. 
The fourth case of https://github.com/erthink/libmdbx/issues/217.
 2021-07-19 12:06:08 +03:00
Leonid Yuriev
28c36af67c mdbx: refine built-in ASAN option. 2021-07-16 16:04:09 +03:00
Leonid Yuriev
d67b9eaf17 mdbx: fix ASAN-regression after 1740043678951c5a1a088e6572722de438162ab2. 
Related to https://github.com/erthink/libmdbx/issues/217
 2021-07-16 16:03:49 +03:00
Leonid Yuriev
6034985686 mdbx: add MDBX_ASAN_(UN)POISON_MEMORY_REGION() macros. 2021-07-16 16:03:49 +03:00
Leonid Yuriev
7da64b725d mdbx: perform madvise only for the first process opens a DB. 2021-07-16 16:03:49 +03:00
Leonid Yuriev
8aaf5d071b mdbx: fix pagecheck(). 
Added a check that the data of the BIGDATA node (containing the target page number) is located within the boundaries of the page being checked.

The third case of https://github.com/erthink/libmdbx/issues/217.
 2021-07-16 15:55:18 +03:00
Leonid Yuriev
c8743cb9c4 mdbx: fix null-deref while override invalid meta-pages. 
Related to https://github.com/erthink/libmdbx/issues/217
 2021-07-14 03:59:52 +03:00
Leonid Yuriev
678a80dd19 mdbx: fix hang/shutdown on big-endian platforms without __cxa_thread_atexit(). 
Change-Id: I1bf706abaaf42d5b40751d85ed7c7a83d02acaf5
 2021-07-11 02:25:39 +03:00
Leonid Yuriev
c18bf4f898 mdbx: minor clarify mdbx_mapresize(). 2021-07-11 02:25:39 +03:00
Leonid Yuriev
1740043678 mdbx: minimize the size of poisoned/unpoisoned regions to avoid ASAN hangs. 
More for second case of https://github.com/erthink/libmdbx/issues/217
 2021-07-11 02:25:26 +03:00
Leonid Yuriev
108398c213 mdbx: refine rollback while opening weak/invalid DB. 
More for https://github.com/erthink/libmdbx/issues/217
 2021-07-11 02:24:51 +03:00
Leonid Yuriev
8bdee27248 mdbx: create/refactoring override_meta(). 2021-07-11 02:24:51 +03:00
Leonid Yuriev
00c6dc9788 mdbx: re-verify head and steady meta-pages while opening db by the first process. 
Basic fix for https://github.com/erthink/libmdbx/issues/217
 2021-07-11 02:24:06 +03:00
Leonid Yuriev
fb67682a79 mdbx: refine mdbx_validate_meta(). 2021-07-10 17:24:26 +03:00
Leonid Yuriev
5ed50a4739 mdbx: remove filesize arg from header/meta read functions (refactoring). 2021-07-10 16:10:30 +03:00
Леонид Юрьев (Leonid Yuriev)
2f74f405ae mdbx: avoid returning MDBX_TXN_FULL error when possible. 2021-07-09 18:29:31 +03:00
Leonid Yuriev
39c5e66ed1 mdbx: fix safe64_reset() for platforms without atomic 64-bit compare-and-swap. 2021-07-09 17:44:27 +03:00
Leonid Yuriev
cf5f31c577 mdbx: make __cold attribute first (cosmetic). 2021-07-03 01:51:04 +03:00
Leonid Yuriev
a5e10b4ea1 mdbx: minor refine mdbx_cursor_eof(). 2021-06-18 01:17:48 +03:00
Leonid Yuriev
b8e621cb2f mdbx: mdbx: avoid log a warning about empty header during DB creation. 
Resolve https://github.com/erthink/libmdbx/issues/205.
 2021-06-18 01:09:26 +03:00
Leonid Yuriev
5db855d728 mdbx: fix false-negative mdbx_cursor_eof() result. 
Fixes https://github.com/erthink/libmdbx/issues/207.
 2021-06-17 21:44:24 +03:00
Leonid Yuriev
d7c06b1337 mdbx: partial fix for recursive SRW-lock with MDBX_NOTLS on Windows. 
Here are some changes to avoid recursive acquisition of SRW-lock,
which is still in use:
 - Read transactions don't acquire the shared SRW-lock with `MDBX_NOTLS.
 - Memory-mapping of DB is always kept while DB opened,
   therefore following limitations are:
 - DB file can't be shrinked while it used,
   including auto-shrink due to auto-compactification with corresponding geometry settings.
 - The upper limit of DB size can't be changed while DB is used.
 - The DB can grow within the upper size limit defined while opening by a first process,
   but this does not work under Wine since there is no `NtExtendSection()` function.

Partially fix https://github.com/erthink/libmdbx/issues/203
 2021-06-10 13:42:30 +03:00
Leonid Yuriev
0cd1eac6a8 mdbx: don't check other meta pages if one is specified for verification/recovery. 
Fixes https://github.com/erthink/libmdbx/issues/202
 2021-06-08 20:27:18 +03:00
Leonid Yuriev
f951f246b8 mdbx: allow to predefine/override MDBX_BUILD_TIMESTAMP for builds reproducibility. 
Resolve https://github.com/erthink/libmdbx/issues/201
 2021-06-02 14:50:22 +03:00
Leonid Yuriev
ea1fcc2246 mdbx: fix insignificantly Coverity warnings. 2021-05-28 00:54:11 +03:00
Leonid Yuriev
6bedb02ac0 mdbx: fix lru-counter overflow by module 2^31. 
2 of 2 fixes for https://github.com/erthink/libmdbx/issues/195
 2021-05-20 17:42:22 +03:00
Leonid Yuriev
bc6a690733 mdbx: simplify & fix mdbx_txn_keep() and mdbx_cursor_keep(). 
1 of 2 fixes for https://github.com/erthink/libmdbx/issues/195
 2021-05-20 15:10:38 +03:00
Leonid Yuriev
84b699a47c mdbx: fix elsif typo. 2021-05-18 22:44:31 +03:00
Leonid Yuriev
841fc15dd3 mdbx: little more likely/unlikely. 2021-05-12 19:24:19 +03:00
Leonid Yuriev
0875d16656 mdbx: fix/refine mdbx_env_info_ex() to distinguish a not-available and overflow cases for returned timestamps. 2021-05-12 17:51:31 +03:00
Leonid Yuriev
16d686bc42 mdbx: account msync/fsync write in page operations. 2021-05-12 14:40:58 +03:00
Leonid Yuriev
137f80e177 mdbx: refile rollback-writes inside mdbx_setup_dxb(). 2021-05-12 14:40:58 +03:00
Leonid Yuriev
7d8099fc77 mdbx: refine mdbx_wipe_steady(). 2021-05-12 14:40:58 +03:00
Leonid Yuriev
5731ad11c8 mdbx: don't use bool flags for mdbx_msync(). 2021-05-12 14:40:58 +03:00
Leonid Yuriev
ebab75642e mdbx: add public MDBX_MAYBE_UNUSED. 
This also should fix C++ mdbx API build by MSVC 2015.
 2021-05-11 21:07:40 +03:00
Leonid Yuriev
2071958f35 mdbx: remove extra notice/debug logging. 2021-05-11 00:24:01 +03:00
Leonid Yuriev
4fee14d1cf mdbx: logging all reasons of MDBX_PROBLEM. 
Related to https://github.com/erthink/libmdbx/issues/195
 2021-05-10 16:15:58 +03:00
Leonid Yuriev
244fab6c04 mdbx: fix extra conversion of an error during nested dupsort-DB update to MDBX_PROBLEM. 
Hope this fixes https://github.com/erthink/libmdbx/issues/195
 2021-05-10 16:15:03 +03:00
Leonid Yuriev
0d100a898f mdbx: minor more const. 2021-05-10 15:53:07 +03:00
Leonid Yuriev
43f1823cdc mdbx: minor alignment fix for lck-less stub to avoid false-positive alarm from UndefinedBehaviorSanitize. 
Change-Id: I8d9bc00ccfc7228cb42f9419cf77e8eadb92f918
 2021-05-08 22:43:16 +03:00
Leonid Yuriev
e75033b1f4 mdbx: drop some unused stuff. 
Change-Id: I136dcf891757d5c1a35a8082ee90e8cf07c016c6
 2021-05-08 18:40:14 +03:00
Leonid Yuriev
ecd3c06932 mdbx-windows: remove MDBX_CONFIG_MANUAL_TLS_CALLBACK build option and add MDBX_MANUAL_MODULE_HANDLER macro. 
Briefly:
 - Now constructor/destructor of "Thread Local Storage" handled automatically when possible.
 - Otherwise the MDBX_CONFIG_MANUAL_TLS_CALLBACK macro defined to 1 to indicate that mdbx_module_handle() should be called manually.
 - Corresponding build option MDBX_CONFIG_MANUAL_TLS_CALLBACK was removed.

Related to https://github.com/erthink/libmdbx/issues/155

Change-Id: Ic4e6a34b44f874676f0ab212ff473460e3d80559
 2021-05-08 18:39:18 +03:00
Leonid Yuriev
5ae120af80 mdbx-windows: more crutches for MinGW. 
More for https://github.com/erthink/libmdbx/issues/155

Change-Id: I7de6122ff160372b2dcfd2a0a26e332cb52d0560
 2021-05-08 18:38:44 +03:00
Leonid Yuriev
c15ad25073 mdbx: add minor workaround for E2K LCC's bug. 
Related to https://bugs.mcst.ru/bugzilla/show_bug.cgi?id=6011

Change-Id: I562dbcedb3c172e0646dcb03ef6d5cd3f14e02b9
 2021-05-07 19:05:10 +03:00
Leonid Yuriev
1275bdb623 mdbx: add MDBX_opt_merge_threshold_16dot16_percent option. 
Change-Id: I416f85096e4b6fd21f2db622f07f31103cb9074a
 2021-05-06 02:05:33 +03:00
Leonid Yuriev
c914d417d2 mdbx: refine Doxygen's API description. 
Change-Id: Ifd326eded287c68c6a95b6c9be22847d6efa5678
 2021-05-03 15:00:53 +03:00
Leonid Yuriev
df387ad943 mdbx: fix minor MSVC warnings. 
Change-Id: I9a2c774f6020bbe5c77e423a1f54110b12bdcb41
 2021-05-02 17:11:50 +03:00
Leonid Yuriev
732d3cd2d4 mdbx: fix/rework mdbx_env_stat_ex() to return stat of the whole env. 
Resolves https://github.com/erthink/libmdbx/issues/190

Included fix for the first version of this change.

Change-Id: I8357de24fda2ab4c93510574055b52d6e16b0c15
 2021-05-02 17:10:37 +03:00
Leonid Yuriev
a56c72d190 mdbx: minor refine internal audit. 
Change-Id: I0e177e9c9ecf1da97084c175f80b3e987391194c
 2021-05-02 15:14:30 +03:00
Leonid Yuriev
161b00a4b6 mdbx: refine/rearrange build options. 
Change-Id: Ic27bf2b1f22e7ed9e6a1db9a1ed2496b1bb1239b
 2021-04-30 02:01:22 +03:00
Leonid Yuriev
6b6165cdeb mdbx: rename internal defines/macros to distinguish it from build options. 
Change-Id: I42fe240e6ddd5d34c8fec38633849ebdcce0e116
 2021-04-30 01:59:48 +03:00
Leonid Yuriev
d27e48ae1b mdbx-windows: rename MDBX_AVOID_CRT build option to MDBX_DISABLE_MSVC_CRT. 
Change-Id: I124c9d86d4a596718bd9671a7c1b538b84c2dd0f
 2021-04-30 01:59:48 +03:00
Leonid Yuriev
c3b2c1fdef mdbx: handling the case when all dirty pages are unspillable. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Iff7a123c1c171371624d8b7ae497941ec4622385
 2021-04-28 23:38:41 +03:00
Leonid Yuriev
aa5a39160a mdbx: internal build option for debuging dirtyroom reservation and spilling. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Ia315ae2cae5f5b5b82891843de22487db0139df3
 2021-04-28 23:38:38 +03:00
Leonid Yuriev
39230c9a22 mdbx: more debug/loggin for LRU spilling. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Ifcfabf8864830adf3c50fe8d01d24cccb950dcbb
 2021-04-28 23:38:34 +03:00
Leonid Yuriev
9fed78c92d mdbx: fix LRU spilling internal prio celling. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Ia4be12e063d1754a38889dc311eb1314b3676c21
 2021-04-28 23:38:32 +03:00
Leonid Yuriev
329a2a50e6 mdbx: account loose-pages during spilling and do purge ones if a lack of dirtyroom. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: I50c11195ad31615181f8359e123c1ffe2ebcfc30
 2021-04-28 23:38:29 +03:00
Leonid Yuriev
17116b9b46 mdbx: raise MDBX_TXN_FULL if spilled less that half of needed. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: I3c1a0e1bea6b6ad9bf41347a569e5833e82d8edc
 2021-04-28 23:38:26 +03:00
Leonid Yuriev
e57e42d0f8 mdbx: use single pointer to lck-less stub. 
Change-Id: I80542423386bd54cac574d1678af2620f147c769
 2021-04-28 23:38:26 +03:00
Leonid Yuriev
7d249c97ad mdbx: double dirtyroom reserve for page stack. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: I211fc9c849c9a45254e699d713475898fd20a58e
 2021-04-28 23:38:24 +03:00
Leonid Yuriev
4111d7238c mdbx: spill pages during prepare backlog for update_gc(). 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: I88463a860da0a53420904daf1b1f39e5ca4c97e5
 2021-04-28 23:38:21 +03:00
Leonid Yuriev
760cf515a7 mdbx: minor refactoring cursor_put(). 
Change-Id: Iacdc02f70278382bf0e7fb496bb57f4010403dd0
 2021-04-28 23:38:21 +03:00
Leonid Yuriev
7cbd500534 mdbx: minor simplify dirtyroom accounting. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Ia0afcffe5a245aaa9500edc70b2cbca1735070ac
 2021-04-28 23:38:19 +03:00
Leonid Yuriev
89558657ea mdbx: auto-setup MDBX_opt_txn_dp_limit. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: I8f8c4f1a2600462bc25b4fad3010a4e6ac70b386
 2021-04-28 23:38:16 +03:00
Leonid Yuriev
6134220b8a mdbx: drop P_KEEP flag and use LRU-label insted of. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Ifd3bd2533b40525dd5b0aa05df421c87462c2439
 2021-04-28 23:37:48 +03:00
Leonid Yuriev
975413b48d mdbx: more for page operation statistics. 
Change-Id: I19fc26dfc9f9627eecc8d505e95d33bad76f2b2f
 2021-04-27 18:19:13 +03:00
Leonid Yuriev
f95a277ac5 mdbx: statistics of page operations. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Ia00e6e0df9a65f880517ca33e7f444a0526b96e1
 2021-04-27 18:02:11 +03:00
Leonid Yuriev
3bbf5d03e2 mdbx: lowering the logging level for page unspill. 
Change-Id: I03f9bdaa644534d1c1e257e0394549c90a56d612
 2021-04-27 16:58:35 +03:00
Leonid Yuriev
8b95be91db mdbx: fix minor Coverity warnings. 
Change-Id: Id6ab528b597e62b05b5fc4ab87bbe93cbce6573c
 2021-04-27 00:12:09 +03:00
Leonid Yuriev
427b480f68 mdbx: add MDBX_FAKE_SPILL_WRITEMAP build option. 
Change-Id: I2c7f9267b0e725674b8bc73547bedd8478231948
 2021-04-26 18:18:51 +03:00
Leonid Yuriev
4fa5e95241 mdbx: LRU spilling feature (squashed). 
Resolves https://github.com/erthink/libmdbx/issues/186

Change-Id: Ie3318964e4e2adbeb77738b8301c45075080850b
 2021-04-26 17:51:11 +03:00
Leonid Yuriev
3e272d339a mdbx: drop mdbx_dpl_find() and always promote child LRU-serial to the parent txn. 
More for https://github.com/erthink/libmdbx/issues/186

Change-Id: Ifeac634e9fa1dee6e2b3375c868cc0fa16422099
 2021-04-26 14:08:30 +03:00
Leonid Yuriev
31cfce4ca5 mdbx: fix minor MSVC warnings. 
Change-Id: I99ab0855334c01796c57f13142127d184ab1291e
 2021-04-26 10:56:18 +03:00
Leonid Yuriev
66e5078e28 mdbx: refine page_touch() and dirtyroom checking. 
More for https://github.com/erthink/libmdbx/issues/186

Change-Id: I81a6d94e64a60ccb642d0297a9afb2abddf51c49
 2021-04-26 02:47:25 +03:00
Leonid Yuriev
f29283a620 mdbx: merge branch master into devel. 
Change-Id: I0568e92101e8e3ddf3572c6973542752cd1c235d
 2021-04-26 02:08:03 +03:00
Leonid Yuriev
55f41f40a1 mdbx: rename log2n_powerof2(). 
Change-Id: I22290630186c5041b6ae242d0db1f5c5f36da4ec
 2021-04-26 02:06:28 +03:00
Leonid Yuriev
d1fea74d0e mdbx: preparation for LRU-spilling. 
Initial for https://github.com/erthink/libmdbx/issues/186

Change-Id: I9a6d60cc142ff7490e2a23defe9654d7e3337edc
 2021-04-26 02:06:01 +03:00
Leonid Yuriev
934ca25ab2 mdbx: fix MDBX_EACCESS from mdbx_env_set_option() if env not yet opened. 
Change-Id: I0da15b71919a1787e5cc54441b5e41941afabf66
 2021-04-26 00:52:47 +03:00
Leonid Yuriev
b21ad733ea mdbx: refine cursor_copy(), add cursor_restore(). 
Change-Id: I3ac5883c95aaa0e13970a8176f0af5e374d811a1
 2021-04-23 16:06:23 +03:00
Leonid Yuriev
0b9272209a mdbx: minor refine spilling. 
Change-Id: I243ebff592ac7ad544bbd382c93d2e5d4444b081
 2021-04-23 16:04:35 +03:00
Leonid Yuriev
416a802789 mdbx: add missing err-log for MDBX_TXN_FULL during page allocation. 
Minor for https://github.com/erthink/libmdbx/issues/186

Change-Id: Iadb868c64fba4e9d5a4ee7df093f0cfb9288bf93
 2021-04-21 13:24:54 +03:00
Leonid Yuriev
aaf465b942 mdbx: add missing err-log for MDBX_TXN_FULL during page allocation. 
Change-Id: Iadb868c64fba4e9d5a4ee7df093f0cfb9288bf93
 2021-04-21 12:20:32 +03:00
Leonid Yuriev
c2d8c35daa mdbx: the radix-sort threshold reduced to 333. 
Change-Id: Ic69b6c3740f71997cd014a82fb32a8202ddb65e0
 2021-04-21 00:13:57 +03:00
Leonid Yuriev
eabb29d9c4 mdbx: add MDBX_xPL_PREALLOC_FOR_RADIXSORT build options. 
Change-Id: Icf15558830f86f98c5e970a7eec69cc045045467
 2021-04-21 00:13:51 +03:00
Leonid Yuriev
3b5e2680e9 mdbx: fix non-alloy build for windows. 
Change-Id: Id93eb069543f081c3b1c9734ed65f4afd80a444d
 2021-04-20 22:09:48 +03:00
Leonid Yuriev
7a0d6c6196 mdbx: use POSIX_MADV_DONTNEED. 
Change-Id: I2020eb8ec225a49331cf4dc6c5a24d6559916284
 2021-04-20 22:09:48 +03:00
Leonid Yuriev
b3b6a48797 mdbx: fix minor warning on Solaris/SunOs. 2021-04-20 22:09:48 +03:00
Leonid Yuriev
b7110eb4d4 mdbx: fix non-alloy build for windows. 
Change-Id: Id93eb069543f081c3b1c9734ed65f4afd80a444d
 2021-04-20 15:17:51 +03:00
Leonid Yuriev
c0704db03a mdbx: use POSIX_MADV_DONTNEED. 
Change-Id: I2020eb8ec225a49331cf4dc6c5a24d6559916284
 2021-04-20 15:17:51 +03:00
Leonid Yuriev
e82359bca0 mdbx: fix minor warning on Solaris/SunOs. 2021-04-20 15:17:51 +03:00
Leonid Yuriev
4cf8496422 mdbx: avoid read leaf-pages during dropping trees. 
Change-Id: Ib8034a76a150c213fddcca0f7df971f63253b7a5
 2021-04-18 19:20:16 +03:00
Leonid Yuriev
d1561dc357 mdbx: refactoring page_alloc() result type. 
Change-Id: I4afc73603d610e4e28b952a38dfa9f30d56809bf
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
3f6758e18d mdbx: minor refactoring mdbx_cursor_set(). 
Change-Id: If3c984fb5ac2f77762501ac720080e2caaacac3c
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
b59836e6d9 mdbx: minor refactoring: use cursor_set_result. 
Change-Id: Ice724c541bc4127aae0e9ea85b4fc1d7c1b68471
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
3eb48340ad mdbx: minor refactoring: use node_result. 
Change-Id: I489b15c174644b2f414925f5bacbe83c091a0692
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
b3aba4691b mdbx: minor refactoring: use page_result. 
Change-Id: I7749d9463832ce9b270d06f04f43e413d5ba26b7
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
e4db019f47 mdbx: minor refine cursor_set_ex(). 
Change-Id: I0951a917c1e4c9e372f29c64fda0e1b267f48cdd
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
dc7098b3fb mdbx: minor refine page_new(). 
Change-Id: Iafdea1c592c346c7841a39cdfc96436460b27bcb
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
6d3ff10165 mdbx: simplify page_unspill(). 
Change-Id: Iaa182d1feaf2abb59015725f000a3d9e0c57de8d
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
3622433cf4 mdbx: fix page_copy() for unspilling a large/overflow page. 
Change-Id: I46c6b6bf94c877c1c0142116c404692960dc5ab7
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
9c9cdfdb6d mdbx: minor fix mdbx_is_dirty(). 
Change-Id: Ia539165ddd62a40e13fdfec48de754961681d40f
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
a0a4bbaa7c mdbx: drop goto/label in prev commit. 
More for https://github.com/erthink/libmdbx/issues/164

Change-Id: Ic708c3ba51dd22d08e96d0bef4632a0beb19999b
 2021-04-18 10:24:11 +03:00
Leonid Yuriev
3e0fad1cf6 mdbx: rework (NO)READAHEAD handling. 
Resolves https://github.com/erthink/libmdbx/issues/164

---

NOTE: Seems there is a bug in the Mach/Darwin/OSX kernel,
because MADV_WILLNEED with offset != 0 may cause SIGBUS
on following access to the hinted region.

19.6.0 Darwin Kernel Version 19.6.0: Tue Jan 12 22:13:05 PST 2021; root:xnu-6153.141.16~1/RELEASE_X86_64 x86_64

Change-Id: I11ebbf2bd35e3dba9d078be16cb5678aecf8329c
 2021-04-17 01:12:16 +03:00
Leonid Yuriev
958fd5b947 mdbx: use mp_txnid instead of P_DIRTY (squashed). 
Change-Id: I7e3174611b8819d8050b84d835fe976b87c77c0c
 2021-04-14 19:27:13 +03:00
Leonid Yuriev
f587a74597 mdbx: drop mx_dbistate. 
Change-Id: I61616ee1436ed99db93a1036c9bb7d4db7157c8f
 2021-04-14 00:26:14 +03:00
Leonid Yuriev
5e334fa830 mdbx: extract mdbx_touch_dbi(). 
Change-Id: I3b077cefe44f4c2db099b2050e1fd72a1aaa2687
 2021-04-14 00:25:15 +03:00
Leonid Yuriev
f9977975ae mdbx: remove extra TXN_DBI_CHANGED() check. 
Change-Id: If2014f7bfbfab98f3c0c357f2475da57efc50693
 2021-04-14 00:23:36 +03:00
Leonid Yuriev
e35c92eabb mdbx: refine mdbx_page_new(). 
Change-Id: Ief0cb313c39f7b11fb1cd611d9b0497c011a7a7a
 2021-04-14 00:19:34 +03:00
Leonid Yuriev
49296cad14 mdbx: refactor mdbx_drop_tree() & drop mdbx_retire pgno(). 
Change-Id: I37f33d7a11e958ccc1c48a6af31599deb677038d
 2021-04-10 22:20:31 +03:00
Leonid Yuriev
dfed1dbc17 mdbx: refactoring: merge page_retire() and page_loose(). 
Change-Id: I4b47c11eedad6ef1b83dbf3d34977cdd8fa79505
 2021-04-10 22:20:24 +03:00
Leonid Yuriev
20299f87cb mdbx: minor refine get_reasonable_db_maxsize(). 
Change-Id: I5d654940a325c0c3966a59a37615820587c52c78
 2021-04-10 17:48:31 +03:00
Leonid Yuriev
7f5cbf7dd8 mdbx: avoid float-point ops in prev commit. 
Change-Id: I7fadc5096f49502c01a60436840bb1a87dfe27bd
 2021-04-07 14:29:14 +03:00
Leonid Yuriev
c14e4235ee mdbx: guess a reasonable maximum DB size for the default upper limit of geometry. 
Fixes https://github.com/erthink/libmdbx/issues/183

Change-Id: Ic7b616e229d3008fda49e5a04121e22997ac53ea
 2021-04-07 05:23:06 +03:00
Leonid Yuriev
8ff44026c3 mdbx: fix minor MSVC warning. 
Change-Id: Ia7ec726b8a516d2115b727960cf7fbc874882cd4
 2021-04-07 04:52:38 +03:00
Leonid Yuriev
024ccf6826 mdbx: extract mdbx_get_sysraminfo(). 
Change-Id: I8821009572d275a3b5d2d8729f605b115a80e50a
 2021-04-07 04:45:53 +03:00
Leonid Yuriev
6ee62650af mdbx: add comment for unsure_equal(). 
Change-Id: I0629b5cd4812415c3a240047ec4202f33dddd189
 2021-03-30 01:41:34 +03:00
Leonid Yuriev
66df21ba78 mdbx: add a TODO/FIXME note (first r/w opening while r/o present). 
Change-Id: I5351628aa22313fffd6c2501a2aa16f2c5f573d5
 2021-03-30 01:38:24 +03:00
Leonid Yuriev
4f62b059ef mdbx: minor clarify TODO (configurable threshold to keep reserve space for large/overflow pages). 
Change-Id: I73a3847852466eb3e34927c5648435c8cf283c0e
 2021-03-30 01:34:46 +03:00
Leonid Yuriev
af9b7b5605 mdbx: MAJOR rework page splitting (squashed). 
Basically, this (squashed) commit introduces:
 - An "auto-appending" feature upon insertion for both ascending and
   descending key sequences. As a result, the optimality of page filling
   increases significantly (more densely, less slackness) while
   inserting ordered sequences of keys,
 - A "splitting at middle" for more balanced page tree on average.

---

1. Using left/middle/right tactics for finding the split point of a page:
   - If a key is inserted close to an edge of page,
     then the page splits at that edge;
   - Otherwise a page splits at the middle,
     which leads to a more balanced tree on average;
   - So I expect a better behavior on average,
     but actually effects should be studied further practically.

2. New code for calculating the midpoint of a page split.

3. APPEND-flags no longer affect choosing the page split point.

4. Added left-side splitting by inserting a pure page with a new entry.

Change-Id: Id7441acfc8c90636e3be6bc00a0df15714690f3c
 2021-03-29 00:49:34 +03:00
Leonid Yuriev
e0795227e4 mdbx: merge branch master into devel. 
Change-Id: Ic78177f8bc5ab3dc1826e4e00599f1a7d3087dda
 2021-03-29 00:00:43 +03:00
Leonid Yuriev
3ed99f8c20 mdbx: minor refine/fix cursor_check() internals. 
Change-Id: I89ea9a5336b88e7d0201a7f59d66a8bc53849172
 2021-03-28 23:34:41 +03:00
Leonid Yuriev
636c212235 mdbx: minor refine page_check(). 
Change-Id: I91cae256c47064422d26663078c8c5a6f619a197
 2021-03-28 23:34:41 +03:00
Leonid Yuriev
83cd4f7d58 mdbx: separete DKBUF and DKBUF_DEBUG. 
Change-Id: I8d90909ce9c5faec176a12e68f7bab6c2ea1d317
 2021-03-28 23:34:41 +03:00
Leonid Yuriev
bfc9921305 mdbx: change branch-node size to ~1/2 of the page size. 
Change-Id: I0f455ec04170d792ecc933aa501b4ec1ca9b08c0
 2021-03-28 23:34:41 +03:00
Leonid Yuriev
72f3a2fc3f mdbx: check zero-size of the first key on a branch-page. 
Change-Id: I2be86ea79f04ea8baea92a6e56ef6b22691eeda0
 2021-03-28 22:25:40 +03:00
Leonid Yuriev
3c389d17e8 mdbx: minor refine page_rebalance(). 
Change-Id: I066e7a22d62c45f9e80fb4735f55f02ca34d3ee3
 2021-03-28 22:25:30 +03:00
Leonid Yuriev
2524a0e749 mdbx: output page type from bad_page(). 
Change-Id: I11e4931de26015204c26c116e91e4a0e02f909b4
 2021-03-28 22:25:24 +03:00
Leonid Yuriev
8d3b878965 mdbx: fix minor spelling. 
Change-Id: Ibab66dc18c92b0ac4fa705e10befb34e268ee1f7
 2021-03-28 22:25:17 +03:00
Leonid Yuriev
c77494e2aa mdbx: fix mdbx_env_set_option() for dp_initial & dp_limit. 
Fixes https://github.com/erthink/libmdbx/issues/179

Change-Id: I925104a45148112e0ce94a069a7468e2b4fc8209
 2021-03-24 19:27:31 +03:00
Leonid Yuriev
422d030820 mdbx: fix minor casting warning. 
Change-Id: I9615b3928bb1b88743b42be9d82a986255d1f074
 2021-03-19 03:16:47 +03:00
Leonid Yuriev
b3798a9116 mdbx: mdbx_node_search() micro-optimization. 
Change-Id: I48064ca284aa89b16e418afdc20d09c5dd6e5cba
 2021-03-19 00:17:14 +03:00
Leonid Yuriev
cb0ee2373d mdbx: add mdbx_cursor_set_ex() with non-optional exactp argument. 
Resolves https://github.com/erthink/libmdbx/pull/178

Change-Id: I24dc7d44116b5e2e02b5b75375be24c647a762f5
 2021-03-18 23:15:55 +03:00
Leonid Yuriev
d0b3c45f04 mdbx: refine check_dbi(). 
Change-Id: I6d51c27c3acdea1b93861f67a32cd3c1ee13dc0d
 2021-03-18 23:12:56 +03:00
Leonid Yuriev
7dfd3f18f8 mdbx: rework nodes/keys limitations. 
Change-Id: Id0515346d762d4554102775f26a0fc33f3c0f29e
 2021-03-17 01:18:28 +03:00
Leonid Yuriev
b164baa1f5 mdbx: add MDBX_ENABLE_MADVISE build option/macro. 
Change-Id: I6174ec62e4811e891663b8ae0f7918aa09baf5ab
 2021-03-17 01:18:18 +03:00
Leonid Yuriev
3eb343020d mdbx: simplify check_txn_rw(). 
Change-Id: Id6a24068ffcdd29fe15a888985bd4a17f1de8ab4
 2021-03-14 16:33:06 +03:00
Leonid Yuriev
4b8b7d5a77 mdbx: refine mdbx_env_set_geometry() internals and 16-bit representation of grow/shrink values. 
More for https://github.com/erthink/libmdbx/issues/166.

Change-Id: I7390f954819309ee4a01faf587aee6b5152e44bc
 2021-03-14 03:42:52 +03:00
Leonid Yuriev
55620c1d13 mdbx: fix selecting/adjustment values inside mdbx_env_set_geometry() for implicit out-of-range cases. 
Fixes https://github.com/erthink/libmdbx/issues/170.

Change-Id: Ibc8754811aa3d9077447220f9fec7a5543e40cc4
 2021-03-13 19:55:19 +03:00
Leonid Yuriev
439cccf65f mdbx: minor fix assertion in the prev commit. 
One more for https://github.com/erthink/libmdbx/issues/171.

Change-Id: I49be3df948f3dbcc838f3dc307155ac404cfb331
 2021-03-13 19:40:18 +03:00
Leonid Yuriev
f7cd08ea48 mdbx: add an explicit/strict check that the environment is open. 
More for https://github.com/erthink/libmdbx/issues/171.

Change-Id: Ifbf7f8ba143d19162bd3ed1cf29c21c31b45f0d5
 2021-03-13 18:31:45 +03:00
Leonid Yuriev
e43cf081f1 mdbx: fix mdbx_env_info_ex() for a non-open environment case. 
Fix the https://github.com/erthink/libmdbx/issues/171.

Change-Id: Iad3d0186ef40dfd40fad1c3ae0dd97f9c1161c16
 2021-03-13 17:15:17 +03:00
Leonid Yuriev
a96b6f79c6 mdbx: fix/disallowing implicit subDB deletion via operations on MAIN_DBI. 
Change-Id: I3cd786e877f42cef2c0d5556033b2633f8a8ce62
 2021-03-13 17:11:35 +03:00
Leonid Yuriev
fd021d793a mdbx: fix MSVC warning. 
More for https://github.com/erthink/libmdbx/issues/166

Change-Id: I47a2ec72831c06695e7ef4c52a77039e52a718e2
 2021-03-09 12:59:54 +03:00
Leonid Yuriev
cd90f831af mdbx: refine db growth step default. 
Change-Id: Ief80441cb578ae37d5d7cee4766e57eaacb4bbde
 2021-03-09 12:59:27 +03:00