ISAR/libmdbx
3
0
Fork 0
mirror of https://github.com/isar/libmdbx.git synced 2025-01-10 03:44:12 +08:00
Code Issues Packages Projects Releases Wiki Activity
3,139 Commits 5 Branches 68 Tags 13 MiB
Commit Graph

500 Commits

Author SHA1 Message Date
Leonid Yuriev
6034985686 mdbx: add MDBX_ASAN_(UN)POISON_MEMORY_REGION() macros. 2021-07-16 16:03:49 +03:00
Leonid Yuriev
7da64b725d mdbx: perform madvise only for the first process opens a DB. 2021-07-16 16:03:49 +03:00
Leonid Yuriev
8aaf5d071b mdbx: fix pagecheck(). 
Added a check that the data of the BIGDATA node (containing the target page number) is located within the boundaries of the page being checked.

The third case of https://github.com/erthink/libmdbx/issues/217.
 2021-07-16 15:55:18 +03:00
Leonid Yuriev
c8743cb9c4 mdbx: fix null-deref while override invalid meta-pages. 
Related to https://github.com/erthink/libmdbx/issues/217
 2021-07-14 03:59:52 +03:00
Leonid Yuriev
678a80dd19 mdbx: fix hang/shutdown on big-endian platforms without __cxa_thread_atexit(). 
Change-Id: I1bf706abaaf42d5b40751d85ed7c7a83d02acaf5
 2021-07-11 02:25:39 +03:00
Leonid Yuriev
c18bf4f898 mdbx: minor clarify mdbx_mapresize(). 2021-07-11 02:25:39 +03:00
Leonid Yuriev
1740043678 mdbx: minimize the size of poisoned/unpoisoned regions to avoid ASAN hangs. 
More for second case of https://github.com/erthink/libmdbx/issues/217
 2021-07-11 02:25:26 +03:00
Leonid Yuriev
108398c213 mdbx: refine rollback while opening weak/invalid DB. 
More for https://github.com/erthink/libmdbx/issues/217
 2021-07-11 02:24:51 +03:00
Leonid Yuriev
8bdee27248 mdbx: create/refactoring override_meta(). 2021-07-11 02:24:51 +03:00
Leonid Yuriev
00c6dc9788 mdbx: re-verify head and steady meta-pages while opening db by the first process. 
Basic fix for https://github.com/erthink/libmdbx/issues/217
 2021-07-11 02:24:06 +03:00
Leonid Yuriev
fb67682a79 mdbx: refine mdbx_validate_meta(). 2021-07-10 17:24:26 +03:00
Leonid Yuriev
5ed50a4739 mdbx: remove filesize arg from header/meta read functions (refactoring). 2021-07-10 16:10:30 +03:00
Леонид Юрьев (Leonid Yuriev)
2f74f405ae mdbx: avoid returning MDBX_TXN_FULL error when possible. 2021-07-09 18:29:31 +03:00
Leonid Yuriev
39c5e66ed1 mdbx: fix safe64_reset() for platforms without atomic 64-bit compare-and-swap. 2021-07-09 17:44:27 +03:00
Leonid Yuriev
cf5f31c577 mdbx: make __cold attribute first (cosmetic). 2021-07-03 01:51:04 +03:00
Leonid Yuriev
a5e10b4ea1 mdbx: minor refine mdbx_cursor_eof(). 2021-06-18 01:17:48 +03:00
Leonid Yuriev
b8e621cb2f mdbx: mdbx: avoid log a warning about empty header during DB creation. 
Resolve https://github.com/erthink/libmdbx/issues/205.
 2021-06-18 01:09:26 +03:00
Leonid Yuriev
5db855d728 mdbx: fix false-negative mdbx_cursor_eof() result. 
Fixes https://github.com/erthink/libmdbx/issues/207.
 2021-06-17 21:44:24 +03:00
Leonid Yuriev
d7c06b1337 mdbx: partial fix for recursive SRW-lock with MDBX_NOTLS on Windows. 
Here are some changes to avoid recursive acquisition of SRW-lock,
which is still in use:
 - Read transactions don't acquire the shared SRW-lock with `MDBX_NOTLS.
 - Memory-mapping of DB is always kept while DB opened,
   therefore following limitations are:
 - DB file can't be shrinked while it used,
   including auto-shrink due to auto-compactification with corresponding geometry settings.
 - The upper limit of DB size can't be changed while DB is used.
 - The DB can grow within the upper size limit defined while opening by a first process,
   but this does not work under Wine since there is no `NtExtendSection()` function.

Partially fix https://github.com/erthink/libmdbx/issues/203
 2021-06-10 13:42:30 +03:00
Leonid Yuriev
0cd1eac6a8 mdbx: don't check other meta pages if one is specified for verification/recovery. 
Fixes https://github.com/erthink/libmdbx/issues/202
 2021-06-08 20:27:18 +03:00
Leonid Yuriev
f951f246b8 mdbx: allow to predefine/override MDBX_BUILD_TIMESTAMP for builds reproducibility. 
Resolve https://github.com/erthink/libmdbx/issues/201
 2021-06-02 14:50:22 +03:00
Leonid Yuriev
ea1fcc2246 mdbx: fix insignificantly Coverity warnings. 2021-05-28 00:54:11 +03:00
Leonid Yuriev
6bedb02ac0 mdbx: fix lru-counter overflow by module 2^31. 
2 of 2 fixes for https://github.com/erthink/libmdbx/issues/195
 2021-05-20 17:42:22 +03:00
Leonid Yuriev
bc6a690733 mdbx: simplify & fix mdbx_txn_keep() and mdbx_cursor_keep(). 
1 of 2 fixes for https://github.com/erthink/libmdbx/issues/195
 2021-05-20 15:10:38 +03:00
Leonid Yuriev
84b699a47c mdbx: fix elsif typo. 2021-05-18 22:44:31 +03:00
Leonid Yuriev
841fc15dd3 mdbx: little more likely/unlikely. 2021-05-12 19:24:19 +03:00
Leonid Yuriev
0875d16656 mdbx: fix/refine mdbx_env_info_ex() to distinguish a not-available and overflow cases for returned timestamps. 2021-05-12 17:51:31 +03:00
Leonid Yuriev
16d686bc42 mdbx: account msync/fsync write in page operations. 2021-05-12 14:40:58 +03:00
Leonid Yuriev
137f80e177 mdbx: refile rollback-writes inside mdbx_setup_dxb(). 2021-05-12 14:40:58 +03:00
Leonid Yuriev
7d8099fc77 mdbx: refine mdbx_wipe_steady(). 2021-05-12 14:40:58 +03:00
Leonid Yuriev
5731ad11c8 mdbx: don't use bool flags for mdbx_msync(). 2021-05-12 14:40:58 +03:00
Leonid Yuriev
ebab75642e mdbx: add public MDBX_MAYBE_UNUSED. 
This also should fix C++ mdbx API build by MSVC 2015.
 2021-05-11 21:07:40 +03:00
Leonid Yuriev
2071958f35 mdbx: remove extra notice/debug logging. 2021-05-11 00:24:01 +03:00
Leonid Yuriev
4fee14d1cf mdbx: logging all reasons of MDBX_PROBLEM. 
Related to https://github.com/erthink/libmdbx/issues/195
 2021-05-10 16:15:58 +03:00
Leonid Yuriev
244fab6c04 mdbx: fix extra conversion of an error during nested dupsort-DB update to MDBX_PROBLEM. 
Hope this fixes https://github.com/erthink/libmdbx/issues/195
 2021-05-10 16:15:03 +03:00
Leonid Yuriev
0d100a898f mdbx: minor more const. 2021-05-10 15:53:07 +03:00
Leonid Yuriev
43f1823cdc mdbx: minor alignment fix for lck-less stub to avoid false-positive alarm from UndefinedBehaviorSanitize. 
Change-Id: I8d9bc00ccfc7228cb42f9419cf77e8eadb92f918
 2021-05-08 22:43:16 +03:00
Leonid Yuriev
e75033b1f4 mdbx: drop some unused stuff. 
Change-Id: I136dcf891757d5c1a35a8082ee90e8cf07c016c6
 2021-05-08 18:40:14 +03:00
Leonid Yuriev
ecd3c06932 mdbx-windows: remove MDBX_CONFIG_MANUAL_TLS_CALLBACK build option and add MDBX_MANUAL_MODULE_HANDLER macro. 
Briefly:
 - Now constructor/destructor of "Thread Local Storage" handled automatically when possible.
 - Otherwise the MDBX_CONFIG_MANUAL_TLS_CALLBACK macro defined to 1 to indicate that mdbx_module_handle() should be called manually.
 - Corresponding build option MDBX_CONFIG_MANUAL_TLS_CALLBACK was removed.

Related to https://github.com/erthink/libmdbx/issues/155

Change-Id: Ic4e6a34b44f874676f0ab212ff473460e3d80559
 2021-05-08 18:39:18 +03:00
Leonid Yuriev
5ae120af80 mdbx-windows: more crutches for MinGW. 
More for https://github.com/erthink/libmdbx/issues/155

Change-Id: I7de6122ff160372b2dcfd2a0a26e332cb52d0560
 2021-05-08 18:38:44 +03:00
Leonid Yuriev
c15ad25073 mdbx: add minor workaround for E2K LCC's bug. 
Related to https://bugs.mcst.ru/bugzilla/show_bug.cgi?id=6011

Change-Id: I562dbcedb3c172e0646dcb03ef6d5cd3f14e02b9
 2021-05-07 19:05:10 +03:00
Leonid Yuriev
1275bdb623 mdbx: add MDBX_opt_merge_threshold_16dot16_percent option. 
Change-Id: I416f85096e4b6fd21f2db622f07f31103cb9074a
 2021-05-06 02:05:33 +03:00
Leonid Yuriev
c914d417d2 mdbx: refine Doxygen's API description. 
Change-Id: Ifd326eded287c68c6a95b6c9be22847d6efa5678
 2021-05-03 15:00:53 +03:00
Leonid Yuriev
df387ad943 mdbx: fix minor MSVC warnings. 
Change-Id: I9a2c774f6020bbe5c77e423a1f54110b12bdcb41
 2021-05-02 17:11:50 +03:00
Leonid Yuriev
732d3cd2d4 mdbx: fix/rework mdbx_env_stat_ex() to return stat of the whole env. 
Resolves https://github.com/erthink/libmdbx/issues/190

Included fix for the first version of this change.

Change-Id: I8357de24fda2ab4c93510574055b52d6e16b0c15
 2021-05-02 17:10:37 +03:00
Leonid Yuriev
a56c72d190 mdbx: minor refine internal audit. 
Change-Id: I0e177e9c9ecf1da97084c175f80b3e987391194c
 2021-05-02 15:14:30 +03:00
Leonid Yuriev
161b00a4b6 mdbx: refine/rearrange build options. 
Change-Id: Ic27bf2b1f22e7ed9e6a1db9a1ed2496b1bb1239b
 2021-04-30 02:01:22 +03:00
Leonid Yuriev
6b6165cdeb mdbx: rename internal defines/macros to distinguish it from build options. 
Change-Id: I42fe240e6ddd5d34c8fec38633849ebdcce0e116
 2021-04-30 01:59:48 +03:00
Leonid Yuriev
d27e48ae1b mdbx-windows: rename MDBX_AVOID_CRT build option to MDBX_DISABLE_MSVC_CRT. 
Change-Id: I124c9d86d4a596718bd9671a7c1b538b84c2dd0f
 2021-04-30 01:59:48 +03:00
Leonid Yuriev
c3b2c1fdef mdbx: handling the case when all dirty pages are unspillable. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Iff7a123c1c171371624d8b7ae497941ec4622385
 2021-04-28 23:38:41 +03:00
Leonid Yuriev
aa5a39160a mdbx: internal build option for debuging dirtyroom reservation and spilling. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Ia315ae2cae5f5b5b82891843de22487db0139df3
 2021-04-28 23:38:38 +03:00
Leonid Yuriev
39230c9a22 mdbx: more debug/loggin for LRU spilling. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Ifcfabf8864830adf3c50fe8d01d24cccb950dcbb
 2021-04-28 23:38:34 +03:00
Leonid Yuriev
9fed78c92d mdbx: fix LRU spilling internal prio celling. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Ia4be12e063d1754a38889dc311eb1314b3676c21
 2021-04-28 23:38:32 +03:00
Leonid Yuriev
329a2a50e6 mdbx: account loose-pages during spilling and do purge ones if a lack of dirtyroom. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: I50c11195ad31615181f8359e123c1ffe2ebcfc30
 2021-04-28 23:38:29 +03:00
Leonid Yuriev
17116b9b46 mdbx: raise MDBX_TXN_FULL if spilled less that half of needed. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: I3c1a0e1bea6b6ad9bf41347a569e5833e82d8edc
 2021-04-28 23:38:26 +03:00
Leonid Yuriev
e57e42d0f8 mdbx: use single pointer to lck-less stub. 
Change-Id: I80542423386bd54cac574d1678af2620f147c769
 2021-04-28 23:38:26 +03:00
Leonid Yuriev
7d249c97ad mdbx: double dirtyroom reserve for page stack. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: I211fc9c849c9a45254e699d713475898fd20a58e
 2021-04-28 23:38:24 +03:00
Leonid Yuriev
4111d7238c mdbx: spill pages during prepare backlog for update_gc(). 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: I88463a860da0a53420904daf1b1f39e5ca4c97e5
 2021-04-28 23:38:21 +03:00
Leonid Yuriev
760cf515a7 mdbx: minor refactoring cursor_put(). 
Change-Id: Iacdc02f70278382bf0e7fb496bb57f4010403dd0
 2021-04-28 23:38:21 +03:00
Leonid Yuriev
7cbd500534 mdbx: minor simplify dirtyroom accounting. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Ia0afcffe5a245aaa9500edc70b2cbca1735070ac
 2021-04-28 23:38:19 +03:00
Leonid Yuriev
89558657ea mdbx: auto-setup MDBX_opt_txn_dp_limit. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: I8f8c4f1a2600462bc25b4fad3010a4e6ac70b386
 2021-04-28 23:38:16 +03:00
Leonid Yuriev
6134220b8a mdbx: drop P_KEEP flag and use LRU-label insted of. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Ifd3bd2533b40525dd5b0aa05df421c87462c2439
 2021-04-28 23:37:48 +03:00
Leonid Yuriev
975413b48d mdbx: more for page operation statistics. 
Change-Id: I19fc26dfc9f9627eecc8d505e95d33bad76f2b2f
 2021-04-27 18:19:13 +03:00
Leonid Yuriev
f95a277ac5 mdbx: statistics of page operations. 
Related to https://github.com/erthink/libmdbx/issues/186

Change-Id: Ia00e6e0df9a65f880517ca33e7f444a0526b96e1
 2021-04-27 18:02:11 +03:00
Leonid Yuriev
3bbf5d03e2 mdbx: lowering the logging level for page unspill. 
Change-Id: I03f9bdaa644534d1c1e257e0394549c90a56d612
 2021-04-27 16:58:35 +03:00
Leonid Yuriev
8b95be91db mdbx: fix minor Coverity warnings. 
Change-Id: Id6ab528b597e62b05b5fc4ab87bbe93cbce6573c
 2021-04-27 00:12:09 +03:00
Leonid Yuriev
427b480f68 mdbx: add MDBX_FAKE_SPILL_WRITEMAP build option. 
Change-Id: I2c7f9267b0e725674b8bc73547bedd8478231948
 2021-04-26 18:18:51 +03:00
Leonid Yuriev
4fa5e95241 mdbx: LRU spilling feature (squashed). 
Resolves https://github.com/erthink/libmdbx/issues/186

Change-Id: Ie3318964e4e2adbeb77738b8301c45075080850b
 2021-04-26 17:51:11 +03:00
Leonid Yuriev
3e272d339a mdbx: drop mdbx_dpl_find() and always promote child LRU-serial to the parent txn. 
More for https://github.com/erthink/libmdbx/issues/186

Change-Id: Ifeac634e9fa1dee6e2b3375c868cc0fa16422099
 2021-04-26 14:08:30 +03:00
Leonid Yuriev
31cfce4ca5 mdbx: fix minor MSVC warnings. 
Change-Id: I99ab0855334c01796c57f13142127d184ab1291e
 2021-04-26 10:56:18 +03:00
Leonid Yuriev
66e5078e28 mdbx: refine page_touch() and dirtyroom checking. 
More for https://github.com/erthink/libmdbx/issues/186

Change-Id: I81a6d94e64a60ccb642d0297a9afb2abddf51c49
 2021-04-26 02:47:25 +03:00
Leonid Yuriev
f29283a620 mdbx: merge branch master into devel. 
Change-Id: I0568e92101e8e3ddf3572c6973542752cd1c235d
 2021-04-26 02:08:03 +03:00
Leonid Yuriev
55f41f40a1 mdbx: rename log2n_powerof2(). 
Change-Id: I22290630186c5041b6ae242d0db1f5c5f36da4ec
 2021-04-26 02:06:28 +03:00
Leonid Yuriev
d1fea74d0e mdbx: preparation for LRU-spilling. 
Initial for https://github.com/erthink/libmdbx/issues/186

Change-Id: I9a6d60cc142ff7490e2a23defe9654d7e3337edc
 2021-04-26 02:06:01 +03:00
Leonid Yuriev
934ca25ab2 mdbx: fix MDBX_EACCESS from mdbx_env_set_option() if env not yet opened. 
Change-Id: I0da15b71919a1787e5cc54441b5e41941afabf66
 2021-04-26 00:52:47 +03:00
Leonid Yuriev
b21ad733ea mdbx: refine cursor_copy(), add cursor_restore(). 
Change-Id: I3ac5883c95aaa0e13970a8176f0af5e374d811a1
 2021-04-23 16:06:23 +03:00
Leonid Yuriev
0b9272209a mdbx: minor refine spilling. 
Change-Id: I243ebff592ac7ad544bbd382c93d2e5d4444b081
 2021-04-23 16:04:35 +03:00
Leonid Yuriev
416a802789 mdbx: add missing err-log for MDBX_TXN_FULL during page allocation. 
Minor for https://github.com/erthink/libmdbx/issues/186

Change-Id: Iadb868c64fba4e9d5a4ee7df093f0cfb9288bf93
 2021-04-21 13:24:54 +03:00
Leonid Yuriev
aaf465b942 mdbx: add missing err-log for MDBX_TXN_FULL during page allocation. 
Change-Id: Iadb868c64fba4e9d5a4ee7df093f0cfb9288bf93
 2021-04-21 12:20:32 +03:00
Leonid Yuriev
c2d8c35daa mdbx: the radix-sort threshold reduced to 333. 
Change-Id: Ic69b6c3740f71997cd014a82fb32a8202ddb65e0
 2021-04-21 00:13:57 +03:00
Leonid Yuriev
eabb29d9c4 mdbx: add MDBX_xPL_PREALLOC_FOR_RADIXSORT build options. 
Change-Id: Icf15558830f86f98c5e970a7eec69cc045045467
 2021-04-21 00:13:51 +03:00
Leonid Yuriev
3b5e2680e9 mdbx: fix non-alloy build for windows. 
Change-Id: Id93eb069543f081c3b1c9734ed65f4afd80a444d
 2021-04-20 22:09:48 +03:00
Leonid Yuriev
7a0d6c6196 mdbx: use POSIX_MADV_DONTNEED. 
Change-Id: I2020eb8ec225a49331cf4dc6c5a24d6559916284
 2021-04-20 22:09:48 +03:00
Leonid Yuriev
b3b6a48797 mdbx: fix minor warning on Solaris/SunOs. 2021-04-20 22:09:48 +03:00
Leonid Yuriev
b7110eb4d4 mdbx: fix non-alloy build for windows. 
Change-Id: Id93eb069543f081c3b1c9734ed65f4afd80a444d
 2021-04-20 15:17:51 +03:00
Leonid Yuriev
c0704db03a mdbx: use POSIX_MADV_DONTNEED. 
Change-Id: I2020eb8ec225a49331cf4dc6c5a24d6559916284
 2021-04-20 15:17:51 +03:00
Leonid Yuriev
e82359bca0 mdbx: fix minor warning on Solaris/SunOs. 2021-04-20 15:17:51 +03:00
Leonid Yuriev
4cf8496422 mdbx: avoid read leaf-pages during dropping trees. 
Change-Id: Ib8034a76a150c213fddcca0f7df971f63253b7a5
 2021-04-18 19:20:16 +03:00
Leonid Yuriev
d1561dc357 mdbx: refactoring page_alloc() result type. 
Change-Id: I4afc73603d610e4e28b952a38dfa9f30d56809bf
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
3f6758e18d mdbx: minor refactoring mdbx_cursor_set(). 
Change-Id: If3c984fb5ac2f77762501ac720080e2caaacac3c
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
b59836e6d9 mdbx: minor refactoring: use cursor_set_result. 
Change-Id: Ice724c541bc4127aae0e9ea85b4fc1d7c1b68471
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
3eb48340ad mdbx: minor refactoring: use node_result. 
Change-Id: I489b15c174644b2f414925f5bacbe83c091a0692
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
b3aba4691b mdbx: minor refactoring: use page_result. 
Change-Id: I7749d9463832ce9b270d06f04f43e413d5ba26b7
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
e4db019f47 mdbx: minor refine cursor_set_ex(). 
Change-Id: I0951a917c1e4c9e372f29c64fda0e1b267f48cdd
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
dc7098b3fb mdbx: minor refine page_new(). 
Change-Id: Iafdea1c592c346c7841a39cdfc96436460b27bcb
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
6d3ff10165 mdbx: simplify page_unspill(). 
Change-Id: Iaa182d1feaf2abb59015725f000a3d9e0c57de8d
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
3622433cf4 mdbx: fix page_copy() for unspilling a large/overflow page. 
Change-Id: I46c6b6bf94c877c1c0142116c404692960dc5ab7
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
9c9cdfdb6d mdbx: minor fix mdbx_is_dirty(). 
Change-Id: Ia539165ddd62a40e13fdfec48de754961681d40f
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
a0a4bbaa7c mdbx: drop goto/label in prev commit. 
More for https://github.com/erthink/libmdbx/issues/164

Change-Id: Ic708c3ba51dd22d08e96d0bef4632a0beb19999b
 2021-04-18 10:24:11 +03:00
Leonid Yuriev
3e0fad1cf6 mdbx: rework (NO)READAHEAD handling. 
Resolves https://github.com/erthink/libmdbx/issues/164

---

NOTE: Seems there is a bug in the Mach/Darwin/OSX kernel,
because MADV_WILLNEED with offset != 0 may cause SIGBUS
on following access to the hinted region.

19.6.0 Darwin Kernel Version 19.6.0: Tue Jan 12 22:13:05 PST 2021; root:xnu-6153.141.16~1/RELEASE_X86_64 x86_64

Change-Id: I11ebbf2bd35e3dba9d078be16cb5678aecf8329c
 2021-04-17 01:12:16 +03:00