ISAR/libmdbx
3
0
Fork 0
mirror of https://github.com/isar/libmdbx.git synced 2025-01-09 04:44:13 +08:00
Code Issues Packages Projects Releases Wiki Activity
3,522 Commits 5 Branches 68 Tags 13 MiB
Commit Graph

659 Commits

Author SHA1 Message Date
Leonid Yuriev
3eb48340ad mdbx: minor refactoring: use node_result. 
Change-Id: I489b15c174644b2f414925f5bacbe83c091a0692
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
b3aba4691b mdbx: minor refactoring: use page_result. 
Change-Id: I7749d9463832ce9b270d06f04f43e413d5ba26b7
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
e4db019f47 mdbx: minor refine cursor_set_ex(). 
Change-Id: I0951a917c1e4c9e372f29c64fda0e1b267f48cdd
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
dc7098b3fb mdbx: minor refine page_new(). 
Change-Id: Iafdea1c592c346c7841a39cdfc96436460b27bcb
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
6d3ff10165 mdbx: simplify page_unspill(). 
Change-Id: Iaa182d1feaf2abb59015725f000a3d9e0c57de8d
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
3622433cf4 mdbx: fix page_copy() for unspilling a large/overflow page. 
Change-Id: I46c6b6bf94c877c1c0142116c404692960dc5ab7
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
9c9cdfdb6d mdbx: minor fix mdbx_is_dirty(). 
Change-Id: Ia539165ddd62a40e13fdfec48de754961681d40f
 2021-04-18 10:24:22 +03:00
Leonid Yuriev
a0a4bbaa7c mdbx: drop goto/label in prev commit. 
More for https://github.com/erthink/libmdbx/issues/164

Change-Id: Ic708c3ba51dd22d08e96d0bef4632a0beb19999b
 2021-04-18 10:24:11 +03:00
Leonid Yuriev
3e0fad1cf6 mdbx: rework (NO)READAHEAD handling. 
Resolves https://github.com/erthink/libmdbx/issues/164

---

NOTE: Seems there is a bug in the Mach/Darwin/OSX kernel,
because MADV_WILLNEED with offset != 0 may cause SIGBUS
on following access to the hinted region.

19.6.0 Darwin Kernel Version 19.6.0: Tue Jan 12 22:13:05 PST 2021; root:xnu-6153.141.16~1/RELEASE_X86_64 x86_64

Change-Id: I11ebbf2bd35e3dba9d078be16cb5678aecf8329c
 2021-04-17 01:12:16 +03:00
Leonid Yuriev
958fd5b947 mdbx: use mp_txnid instead of P_DIRTY (squashed). 
Change-Id: I7e3174611b8819d8050b84d835fe976b87c77c0c
 2021-04-14 19:27:13 +03:00
Leonid Yuriev
f587a74597 mdbx: drop mx_dbistate. 
Change-Id: I61616ee1436ed99db93a1036c9bb7d4db7157c8f
 2021-04-14 00:26:14 +03:00
Leonid Yuriev
5e334fa830 mdbx: extract mdbx_touch_dbi(). 
Change-Id: I3b077cefe44f4c2db099b2050e1fd72a1aaa2687
 2021-04-14 00:25:15 +03:00
Leonid Yuriev
f9977975ae mdbx: remove extra TXN_DBI_CHANGED() check. 
Change-Id: If2014f7bfbfab98f3c0c357f2475da57efc50693
 2021-04-14 00:23:36 +03:00
Leonid Yuriev
e35c92eabb mdbx: refine mdbx_page_new(). 
Change-Id: Ief0cb313c39f7b11fb1cd611d9b0497c011a7a7a
 2021-04-14 00:19:34 +03:00
Leonid Yuriev
49296cad14 mdbx: refactor mdbx_drop_tree() & drop mdbx_retire pgno(). 
Change-Id: I37f33d7a11e958ccc1c48a6af31599deb677038d
 2021-04-10 22:20:31 +03:00
Leonid Yuriev
dfed1dbc17 mdbx: refactoring: merge page_retire() and page_loose(). 
Change-Id: I4b47c11eedad6ef1b83dbf3d34977cdd8fa79505
 2021-04-10 22:20:24 +03:00
Leonid Yuriev
20299f87cb mdbx: minor refine get_reasonable_db_maxsize(). 
Change-Id: I5d654940a325c0c3966a59a37615820587c52c78
 2021-04-10 17:48:31 +03:00
Leonid Yuriev
7f5cbf7dd8 mdbx: avoid float-point ops in prev commit. 
Change-Id: I7fadc5096f49502c01a60436840bb1a87dfe27bd
 2021-04-07 14:29:14 +03:00
Leonid Yuriev
c14e4235ee mdbx: guess a reasonable maximum DB size for the default upper limit of geometry. 
Fixes https://github.com/erthink/libmdbx/issues/183

Change-Id: Ic7b616e229d3008fda49e5a04121e22997ac53ea
 2021-04-07 05:23:06 +03:00
Leonid Yuriev
8ff44026c3 mdbx: fix minor MSVC warning. 
Change-Id: Ia7ec726b8a516d2115b727960cf7fbc874882cd4
 2021-04-07 04:52:38 +03:00
Leonid Yuriev
024ccf6826 mdbx: extract mdbx_get_sysraminfo(). 
Change-Id: I8821009572d275a3b5d2d8729f605b115a80e50a
 2021-04-07 04:45:53 +03:00
Leonid Yuriev
6ee62650af mdbx: add comment for unsure_equal(). 
Change-Id: I0629b5cd4812415c3a240047ec4202f33dddd189
 2021-03-30 01:41:34 +03:00
Leonid Yuriev
66df21ba78 mdbx: add a TODO/FIXME note (first r/w opening while r/o present). 
Change-Id: I5351628aa22313fffd6c2501a2aa16f2c5f573d5
 2021-03-30 01:38:24 +03:00
Leonid Yuriev
4f62b059ef mdbx: minor clarify TODO (configurable threshold to keep reserve space for large/overflow pages). 
Change-Id: I73a3847852466eb3e34927c5648435c8cf283c0e
 2021-03-30 01:34:46 +03:00
Leonid Yuriev
af9b7b5605 mdbx: MAJOR rework page splitting (squashed). 
Basically, this (squashed) commit introduces:
 - An "auto-appending" feature upon insertion for both ascending and
   descending key sequences. As a result, the optimality of page filling
   increases significantly (more densely, less slackness) while
   inserting ordered sequences of keys,
 - A "splitting at middle" for more balanced page tree on average.

---

1. Using left/middle/right tactics for finding the split point of a page:
   - If a key is inserted close to an edge of page,
     then the page splits at that edge;
   - Otherwise a page splits at the middle,
     which leads to a more balanced tree on average;
   - So I expect a better behavior on average,
     but actually effects should be studied further practically.

2. New code for calculating the midpoint of a page split.

3. APPEND-flags no longer affect choosing the page split point.

4. Added left-side splitting by inserting a pure page with a new entry.

Change-Id: Id7441acfc8c90636e3be6bc00a0df15714690f3c
 2021-03-29 00:49:34 +03:00
Leonid Yuriev
e0795227e4 mdbx: merge branch master into devel. 
Change-Id: Ic78177f8bc5ab3dc1826e4e00599f1a7d3087dda
 2021-03-29 00:00:43 +03:00
Leonid Yuriev
3ed99f8c20 mdbx: minor refine/fix cursor_check() internals. 
Change-Id: I89ea9a5336b88e7d0201a7f59d66a8bc53849172
 2021-03-28 23:34:41 +03:00
Leonid Yuriev
636c212235 mdbx: minor refine page_check(). 
Change-Id: I91cae256c47064422d26663078c8c5a6f619a197
 2021-03-28 23:34:41 +03:00
Leonid Yuriev
83cd4f7d58 mdbx: separete DKBUF and DKBUF_DEBUG. 
Change-Id: I8d90909ce9c5faec176a12e68f7bab6c2ea1d317
 2021-03-28 23:34:41 +03:00
Leonid Yuriev
bfc9921305 mdbx: change branch-node size to ~1/2 of the page size. 
Change-Id: I0f455ec04170d792ecc933aa501b4ec1ca9b08c0
 2021-03-28 23:34:41 +03:00
Leonid Yuriev
72f3a2fc3f mdbx: check zero-size of the first key on a branch-page. 
Change-Id: I2be86ea79f04ea8baea92a6e56ef6b22691eeda0
 2021-03-28 22:25:40 +03:00
Leonid Yuriev
3c389d17e8 mdbx: minor refine page_rebalance(). 
Change-Id: I066e7a22d62c45f9e80fb4735f55f02ca34d3ee3
 2021-03-28 22:25:30 +03:00
Leonid Yuriev
2524a0e749 mdbx: output page type from bad_page(). 
Change-Id: I11e4931de26015204c26c116e91e4a0e02f909b4
 2021-03-28 22:25:24 +03:00
Leonid Yuriev
8d3b878965 mdbx: fix minor spelling. 
Change-Id: Ibab66dc18c92b0ac4fa705e10befb34e268ee1f7
 2021-03-28 22:25:17 +03:00
Leonid Yuriev
c77494e2aa mdbx: fix mdbx_env_set_option() for dp_initial & dp_limit. 
Fixes https://github.com/erthink/libmdbx/issues/179

Change-Id: I925104a45148112e0ce94a069a7468e2b4fc8209
 2021-03-24 19:27:31 +03:00
Leonid Yuriev
422d030820 mdbx: fix minor casting warning. 
Change-Id: I9615b3928bb1b88743b42be9d82a986255d1f074
 2021-03-19 03:16:47 +03:00
Leonid Yuriev
b3798a9116 mdbx: mdbx_node_search() micro-optimization. 
Change-Id: I48064ca284aa89b16e418afdc20d09c5dd6e5cba
 2021-03-19 00:17:14 +03:00
Leonid Yuriev
cb0ee2373d mdbx: add mdbx_cursor_set_ex() with non-optional exactp argument. 
Resolves https://github.com/erthink/libmdbx/pull/178

Change-Id: I24dc7d44116b5e2e02b5b75375be24c647a762f5
 2021-03-18 23:15:55 +03:00
Leonid Yuriev
d0b3c45f04 mdbx: refine check_dbi(). 
Change-Id: I6d51c27c3acdea1b93861f67a32cd3c1ee13dc0d
 2021-03-18 23:12:56 +03:00
Leonid Yuriev
7dfd3f18f8 mdbx: rework nodes/keys limitations. 
Change-Id: Id0515346d762d4554102775f26a0fc33f3c0f29e
 2021-03-17 01:18:28 +03:00
Leonid Yuriev
b164baa1f5 mdbx: add MDBX_ENABLE_MADVISE build option/macro. 
Change-Id: I6174ec62e4811e891663b8ae0f7918aa09baf5ab
 2021-03-17 01:18:18 +03:00
Leonid Yuriev
3eb343020d mdbx: simplify check_txn_rw(). 
Change-Id: Id6a24068ffcdd29fe15a888985bd4a17f1de8ab4
 2021-03-14 16:33:06 +03:00
Leonid Yuriev
4b8b7d5a77 mdbx: refine mdbx_env_set_geometry() internals and 16-bit representation of grow/shrink values. 
More for https://github.com/erthink/libmdbx/issues/166.

Change-Id: I7390f954819309ee4a01faf587aee6b5152e44bc
 2021-03-14 03:42:52 +03:00
Leonid Yuriev
55620c1d13 mdbx: fix selecting/adjustment values inside mdbx_env_set_geometry() for implicit out-of-range cases. 
Fixes https://github.com/erthink/libmdbx/issues/170.

Change-Id: Ibc8754811aa3d9077447220f9fec7a5543e40cc4
 2021-03-13 19:55:19 +03:00
Leonid Yuriev
439cccf65f mdbx: minor fix assertion in the prev commit. 
One more for https://github.com/erthink/libmdbx/issues/171.

Change-Id: I49be3df948f3dbcc838f3dc307155ac404cfb331
 2021-03-13 19:40:18 +03:00
Leonid Yuriev
f7cd08ea48 mdbx: add an explicit/strict check that the environment is open. 
More for https://github.com/erthink/libmdbx/issues/171.

Change-Id: Ifbf7f8ba143d19162bd3ed1cf29c21c31b45f0d5
 2021-03-13 18:31:45 +03:00
Leonid Yuriev
e43cf081f1 mdbx: fix mdbx_env_info_ex() for a non-open environment case. 
Fix the https://github.com/erthink/libmdbx/issues/171.

Change-Id: Iad3d0186ef40dfd40fad1c3ae0dd97f9c1161c16
 2021-03-13 17:15:17 +03:00
Leonid Yuriev
a96b6f79c6 mdbx: fix/disallowing implicit subDB deletion via operations on MAIN_DBI. 
Change-Id: I3cd786e877f42cef2c0d5556033b2633f8a8ce62
 2021-03-13 17:11:35 +03:00
Leonid Yuriev
fd021d793a mdbx: fix MSVC warning. 
More for https://github.com/erthink/libmdbx/issues/166

Change-Id: I47a2ec72831c06695e7ef4c52a77039e52a718e2
 2021-03-09 12:59:54 +03:00
Leonid Yuriev
cd90f831af mdbx: refine db growth step default. 
Change-Id: Ief80441cb578ae37d5d7cee4766e57eaacb4bbde
 2021-03-09 12:59:27 +03:00
Leonid Yuriev
8e51a10908 mdbx: packing the 16-bit representations of grow step and shrink threshold values. 
Using float point (exponential quantized) representation for internal 16-bit values
of grow step and shrink threshold when huge ones
.
To minimize the impact on compatibility, only the odd values inside the upper half
of the range (i.e. 32769..65533) are used for the new representation.

Resolve https://github.com/erthink/libmdbx/issues/166

Change-Id: I273127c1842deef0d7d8885b55a805b1463556eb
 2021-03-09 12:54:48 +03:00
Leonid Yuriev
ec95a50bb6 mdbx: fix not setting the dirty flag for @MAIN when dropping DBI. 
This fixes https://github.com/erthink/libmdbx/issues/168

Change-Id: Ida9f11fbcea092af17946b95bfe4ac58b822a80c
 2021-03-06 20:40:15 +03:00
Leonid Yuriev
b0d449565d mdbx: shrink dbi table immediately in the mdbx_dbi_close_locked(). 
Change-Id: Idaa5ec9791b5ffa4305e7f9f7a625fba5f956dd9
 2021-03-05 03:20:21 +03:00
Leonid Yuriev
de13d6c823 mdbx: merge branch 'master' into devel. 
Change-Id: Icec2e7548923ef7b3e33bf50fc20842938d5eec5
 2021-03-04 19:23:09 +03:00
Erk
b2a0279253
Fix compilation on build targets listed below MIPS 2021-03-04 10:33:12 +01:00
Leonid Yuriev
dec11e639a mdbx: fix mdbx_dpl_alloc(). 
Change-Id: Ia7d4b1866ca3623668edffd56ab83e845f16b2a5
 2021-03-03 16:24:32 +03:00
Leonid Yuriev
0cebc50291 mdbx: refine page-rebalance. 
Change-Id: Ia94379fa51eb5da1e1ec1a52cc1dd8b67ceb150a
 2021-03-03 16:05:52 +03:00
Leonid Yuriev
46dcd6e7ca mdbx: (re)alloc dpl-list at txn start. 
Change-Id: Ie35221666f09a189f5513e77d3ecb4056968334f
 2021-03-02 03:58:27 +03:00
Leonid Yuriev
c8dccc9bc4 mdbx: limits the initial size of dpl-list to the current db-size. 
Change-Id: I5f575fc6168f50786b6f8a82ae020d323530a12e
 2021-02-28 01:02:16 +03:00
Leonid Yuriev
72d978ee48 mdbx: weakens checks during set the MDBX_opt_txn_dp_limit to avoid MDBX_EINVAL. 
Change-Id: I4852261d0c45b726c60792463ab698538fa447e5
 2021-02-28 01:02:16 +03:00
Leonid Yuriev
7fcf11013e mdbx: minor fix likely/unlikely inside mdbx_cursor_del(). 
Change-Id: I86cfc755eef7371ea96c0feb39bffd3ec5298b71
 2021-02-28 01:02:16 +03:00
Leonid Yuriev
3fd739ea2c mdbx: add MDBX_DISABLE_PAGECHECKS build option. 
Change-Id: Ia5e2727b74961e7f829f273225c5c1a6cc764b8e
 2021-02-07 02:53:54 +03:00
Leonid Yuriev
c89f30e485 mdbx: fix/rework C11 atomics usage to avoid performance regression. 
Resolve https://github.com/erthink/libmdbx/issues/160

Change-Id: Ic14ef8c9f1e4fb844952f51e81b58268d7939cfe
 2021-02-06 21:16:56 +03:00
Leonid Yuriev
7e7d526ed5 mdbx: minor fix txn_spill(). 
Change-Id: Ic624681d3a9e262b77f761796aabf0231019db72
 2021-02-06 19:34:26 +03:00
Leonid Yuriev
0e8c913c57 mdbx: minor refine node_search(). 
Change-Id: Ib9aaca3e7853b9986b6d70bc43a88e90d0a46c0c
 2021-02-06 19:33:07 +03:00
Leonid Yuriev
892402a5d8 mdbx: add missing unlikely() hints. 
Change-Id: I0baae944b9d2a994ad65fbb57591c2cc8d35f0d3
 2021-02-06 19:31:04 +03:00
Leonid Yuriev
0cc695e22e mdbx: minor simplify page_get(). 
Change-Id: I2a548ea008fad29da8382946dff3c7d9bffa1e85
 2021-02-06 19:30:05 +03:00
Leonid Yuriev
8133d93678 mdbx: add LIBMDBX_NO_EXPORTS_LEGACY_API build option. 
Change-Id: I9d9dcffe63fce843e22132c3829adca57714d818
 2021-02-02 00:25:48 +03:00
Leonid Yuriev
38a559b93e mdbx: Crutches for C11-atomics compiler's bugs. 
Change-Id: I5a857aa56c831f00c4bc67c95d7aecabc29264da
 2021-02-01 20:55:41 +03:00
Leonid Yuriev
25c4df0d3e mdbx: radix sort for large chunks of PNL and DPL. 
More for https://github.com/erthink/libmdbx/issues/132

Change-Id: I19b253f78069d4ecd4ec360a12121c78f182fc09
 2021-02-01 02:14:13 +03:00
Leonid Yuriev
88bdf4b96f mdbx: avoid branches inside bsearch() loop. 
Related for https://github.com/erthink/libmdbx/issues/132

Change-Id: Ia843556cc7052e5081a98f56b43fd80e2d0a40c7
 2021-02-01 00:39:52 +03:00
Leonid Yuriev
28bd5d81d2 mdbx: incremental lazy merge sort for dirty-page list. 
The main change for https://github.com/erthink/libmdbx/issues/132

Change-Id: I9907f6abfcf77bd40c7a263f5adb5274d5445864
 2021-02-01 00:39:19 +03:00
Leonid Yuriev
0620ec2f2e mdbx: merge branch master into the devel. 
Change-Id: Iab3a0502577f6557e3f8fccc89df49aee372416e
 2021-01-31 23:21:16 +03:00
Leonid Yuriev
077989bfed mdbx: adds pre- and post- gaps to dirty page list to avoid some comparisons and conditional branches. 
More for https://github.com/erthink/libmdbx/issues/132

Change-Id: I6562c5ff6c559341bb7bb64222b126f06cc13427
 2021-01-31 23:20:49 +03:00
Leonid Yuriev
edb6d2d661 mdbx: add audit after nested txn start. 
Change-Id: I7601994d24e7d2a5b8e9b113f697e4a14b1d2e6a
 2021-01-31 23:20:49 +03:00
Leonid Yuriev
df180d1d36 mdbx: provide the gap/reserve for ability of merge sort dirty-page list. 
Preparation for fix https://github.com/erthink/libmdbx/issues/132

Change-Id: I4aac6927878eb0361588fbfcf9698ba6d2d5fc34
 2021-01-31 23:16:10 +03:00
Leonid Yuriev
8b6608a9ca mdbx: fix dirtyroom inheritance for nested transactions. 
Change-Id: If2036ab3b7909c51e54526a535d65cdee1e0b287
 2021-01-31 21:49:20 +03:00
Leonid Yuriev
9f0ff865e8 mdbx: use C11 atomics if available instead of legacy memory barriers. 
This done better support architectures with a weak/relaxed memory consistency model (ARM, AARCH64, PPC, MIPS, RISC-V, etc).

Change-Id: Iee831c8dc564f1d027ff84b0d6daa559325d5a9b
 2021-01-30 02:28:12 +03:00
Leonid Yuriev
f698f07ff9 mdbx: fix operation for unusual small/large system page size. 
More for https://github.com/erthink/libmdbx/issues/157

Change-Id: I4f2ed54b50653d0375538b82c48590d1037cd93b
 2021-01-29 21:19:58 +03:00
Leonid Yuriev
6cfac546a1 mdbx: fix mdbx_env_get_option(MDBX_opt_sync_bytes) to return number of bytes instead of pages. 
Change-Id: I815887e82946d9644b05549ec253a52b566c8565
 2021-01-29 19:55:40 +03:00
Leonid Yuriev
9c9f6faf38 mdbx: fix madvise_threshold for large system pages. 
Fixed https://github.com/erthink/libmdbx/issues/157

Change-Id: I0e880591d236b6ba96a38d2a3fa953695d9c4df0
 2021-01-29 04:58:32 +03:00
Leonid Yuriev
0a2f2e28b4 mdbx: fix minor signed/unsigned comparison warnings. 
Change-Id: Ice6f9759560d5eda514112e838d57e63207a7559
 2021-01-27 19:23:07 +03:00
Leonid Yuriev
62b2e31bf4 mdbx: fix auto-recovery weak->steady with the same boot-id (don't truncate DB to size from steady-meta). 
Change-Id: Ib1c7f845bdf49fb9101b01aa868f8567172e5590
 2021-01-27 16:48:03 +03:00
Leonid Yuriev
943bb552a2 mdbx: fix GC-slot cleanup inside mdbx_update_gc() in extra rare complex cases. 
More for 5f1b684719 and https://github.com/erthink/libmdbx/issues/123

Change-Id: I1b78d6be2babf9ffdd0b77e7b1ee53d1313977bb
 2021-01-26 10:28:39 +03:00
Leonid Yuriev
251eda6fb8 mdbx: HNY! 
Change-Id: I86c7028001b349e3d06b837dc8e2f7e60fb3b1a2
 2021-01-26 07:27:13 +03:00
Leonid Yuriev
4fd23c4716 mdbx: fix minor debug typo. 
Change-Id: Ieffd3cea351681c1d5ccf93b12953eb0d8abb6b9
 2021-01-25 23:26:53 +03:00
Leonid Yuriev
5f1b684719 mdbx: fix regression MDBX_KEYEXISTS during mdbx_commit_ex() and overwriting GC records. 
Fix regression related to https://github.com/erthink/libmdbx/issues/123 and https://github.com/erthink/libmdbx/issues/128.
Related to https://github.com/erthink/libmdbx/issues/131.

В lifo-режиме при фиксации транзакции, записи в GC могли быть перезаписаны (с утечкой страниц БД), либо могла возникать ошибка MDBX_KEYEXISTS, по следующему сценарию:
- В истории БД были две транзакции с огромным кол-вом retired pages, после которых в GC остались две соответствующие записи.
- В ходе очередной транзакции первая из огромных GC-записей попадает в переработку и образует огромный reclaimed list.
- При фиксации транзакции производится попытка разбить огромный reclaimed list на чанки размером в одну страницу. Для этого требуется много id для записей, которые в соответствии с lifo должны быть максимально близки к голове GC, т. е. получены путем переработки последних записей GC.
- В ходе переработки последних записей очередь доходит до второй огромной записи, при этом переработка прерывается, ибо иначе reclaimed list переполнится.
- Однако прерывание переработки внутри mdbx_update_gc() трактовалось как отсутствие записей в GC, поэтому список доступных просто добавлялись соответствующие id-шники.
- Если в списке доступных id-шников для помещения в GC были переработанные, то записи с id по всему списку удалялись - тогда вторая большая запись (и возможно предыдущие) удалялись, а содержащиеся в них номера страниц выпадали из оборота.
- Если же в списке доступных id-шников не было переработанных, то чистка не проводилась - тогда при последующая попытка помещения чанков reclaimed list в GC завершалась ошибкой MDBX_KEYEXISTS, которая и возвращалась из mdbx_commit_ex().

Change-Id: I3e5d40ef7950b7476da0513c6836fcba1de74879
 2021-01-25 23:23:08 +03:00
Leonid Yuriev
a9cb6a6a90 mdbx: minor fix ASAN controls. 2021-01-25 21:35:59 +03:00
Leonid Yuriev
95ae324580 mdbx: fix reclaimed-list allocation during nested txn start. 
Change-Id: Ia70e01ed69a001249ce14b4930452a469d04b824
 2021-01-24 00:49:46 +03:00
Leonid Yuriev
4ae2a107bf mdbx: add MDBX_opt_loose_limit runtime option. 
More for https://github.com/erthink/libmdbx/issues/128

Change-Id: I298bf1eef24575df72615ee07e93f5073405e8e0
 2021-01-22 23:52:03 +03:00
Leonid Yuriev
dc34041600 mdbx: alter defaults for MDBX_opt_rp_augment_limit and MDBX_opt_txn_dp_limit runtime options. 
Change-Id: I08cdbfb18089a3fed4b32d931ca4423027a9939f
 2021-01-22 23:16:21 +03:00
Leonid Yuriev
b1a2892038 mdbx: fix minor Coverity warnings. 
Change-Id: I0c5d23a36120b8d697887462a6ae86d66a435c3d
 2021-01-22 20:13:31 +03:00
Leonid Yuriev
b82d1d5063 mdbx: add MDBX_opt_spill_parent4child_denominator runtime option. 
More for https://github.com/erthink/libmdbx/issues/128

Change-Id: If1549c3855eb59c107eb115e6dbca515e81644f9
 2021-01-22 18:18:52 +03:00
Leonid Yuriev
56aaad03bc mdbx: add MDBX_opt_spill_max_denominator and MDBX_opt_spill_min_denominator runtime options. 
More for https://github.com/erthink/libmdbx/issues/128

Change-Id: I6b1b00e30ac11e9c2e3d3d3a29cc73079b41d539
 2021-01-22 17:25:44 +03:00
Leonid Yuriev
e0d4eaf819 mdbx: minor refine mdbx_txn_merge(). 2021-01-21 22:45:34 +03:00
Leonid Yuriev
1d58ad6d9b mdbx: fix Valgrind/ASAN controls. 
Change-Id: Idf526cbdb8ce8e43600efebbd1d12a8306a58184
 2021-01-21 22:45:28 +03:00
Leonid Yuriev
58e4eda1c6 mdbx: fix spilled page list memleak. 
Change-Id: I8b3b3664711943b0809be7bcc6580aafd4e4c286
 2021-01-21 22:45:28 +03:00
Leonid Yuriev
d26ae6875b mdbx: fix assertion to avoid false-positive UB detection by CLANG/LLVM. 
Assume this resolve https://github.com/erthink/libmdbx/issues/153

Change-Id: I02ceba1063a338b6dedf17629b51b3466358b0b8
 2021-01-21 22:45:28 +03:00
Leonid Yuriev
afe93137fc mdbx: fix minor assertions. 
Change-Id: Ib3e2dcb2cb419f41717351b2ac2264ffed906014
 2021-01-21 22:45:28 +03:00
Leonid Yuriev
6bd0277020 mdbx: minor fix mdbx_cursor_close(). 
Avoids reference to txn which could be already freed.

Change-Id: I4e1fadd9786ced1533bb8468d30e013455601228
 2021-01-20 09:55:11 +03:00
Leonid Yuriev
7d09de489d mdbx: fix minor MSVC warnings. 
Change-Id: I633b8847698dca25b6c395216744e157f6fb240d
 2021-01-20 09:55:11 +03:00
Leonid Yuriev
0d09c2cda9 mdbx: avoid overflow for dirty & leftover space inside mdbx_txn_info(). 
Change-Id: I36f4d62881e68f1bfe0d11fde2e849cdf1408ed2
 2021-01-20 02:07:44 +03:00
Leonid Yuriev
42670aa64a mdbx: fix MSVC warnings aroung size_t and unsigned conversion. 
Change-Id: I32bd3ee4fe02ac62cc121f33b2f84bf63c81cb75
 2021-01-20 01:39:21 +03:00
Leonid Yuriev
6d7daf6401 mdbx: fix audit for complex nested txn cases. 
Change-Id: I2419cbe0e8d3167e77ac99f397b97edb22717833
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
f28da7c9e3 mdbx: rework spilling, page-lists, commit & abort for nested transactions, etc (squashed). 
Change-Id: I09e4744586342aeae4efcfb71f41a3248ee47758
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
7bae2a7cd2 mdbx: provide MDBX_ENABLE_REFUND build option. 
Change-Id: I0e4c94a22f3b0957176431709604a7595f85252a
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
7183f62e13 mdbx: rework DBI_DIRTY and MDBX_TXN_DIRTY propagation. 
Change-Id: Ief12ee320ce7c7142632bd1142d847aac314db0b
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
4afc30c79f mdbx: rework mdbx_page_alloc(). 
Change-Id: I5c6e2ef23abf79096dd699f9dfbecf97bfc4fe37
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
966d7027a3 mdbx: minor fix bad_page() debug output. 
Change-Id: I14531fdceaad054ad8c45bf9042fee73de1f9a36
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
dfc92d9f96 mdbx: minor fix mdbx_page_new() debug output. 
Change-Id: If60f972cc3271ca91feea923fb4f4be48d488b22
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
ac0f4e14a8 mdbx: minor fix mdbx_page_new() debug output. 
Change-Id: I2f386ec6276cab404607b5c3a79d8028d73dad62
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
ca597f4314 mdbx: fix mdbx_page_flush() for possibility of memleack. 
Change-Id: Ieab0c98fcd1020d49cce73234125ceb11a8f0b64
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
c1fef9b51d mdbx: fix mdbx_update_gc() for retired page list change case. 
Change-Id: Ic454262edcda8881182a15d6a521b90990e5551d
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
b8bb4b4cc4 mdbx: fix propagation child txn DBI-state to parent. 
Change-Id: I7d92fa99131a937af1a1a75b38af57bc6787ab37
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
4425fb0b83 mdbx: minor refine flags-reset inside mdbx_page_flush(). 
This produces little bit less code for most architectures.

Change-Id: I0440ec7b465ae2f12a8b09b0226f2c42bc52f7de
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
5bb254e629 mdbx: allow fastpath-commit for pure nested txns with parent-spilled pages. 
Change-Id: I471fd5e057c323e3701158761631e0a9999bf54a
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
8fce97dca9 mdbx: fix mdbx_cursors_eot(). 
Change-Id: I6021ff019f25110db96f9ef351d31a1f48b76c18
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
f6ff075335 mdbx: rework/speedup spill-pages list. 
Change-Id: I97e63f8d4c82ae682e898e9b04edceb0b01d82a5
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
d64b81c673 mdbx: remove senseless assertions from mdbx_refund_loose(). 
Change-Id: I028ae6bafb9358b61287fc0201c246baf160ca03
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
0ab263b329 mdbx: purge deleted spilled pagenums on refund. 
Change-Id: I325fbad82cbecb71c35dd1edd1a2abccb8108541
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
7bf147d8c2 mdbx: fix assertion inside mdbx_txn_commit_ex() for nested spilled txn. 
Change-Id: I54b37ea1b9591bdec9d8bd16d0f57280a1f9ee77
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
6ef7b2f588 mdbx: fix assertion inside mdbx_txn_begin_ex() for nested txn with spilled parent. 
Change-Id: Ie1f3cd3df6beb296a7bbdc72b39c1c4cb9105083
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
f0d79df9b9 mdbx: fix mdbx_page_flush(). 
Change-Id: I467c2367f2e99ae50996d0859ae22f70c9649a2d
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
05958a708e mdbx: fix minor/potential mem-page leak inside mdbx_page_unspill(). 
Change-Id: I4398a4d81eda494e4de9d62d81e3936c6357b744
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
21fbc36311 mdbx: minor refine mdbx_page_retire(). 
Change-Id: I29d20ed47f56e17dde8c9417bfa1ec95302c1966
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
cea29fe485 mdbx: fix assertion inside mdbx_page_retire(). 
Change-Id: I5d8f92b9fde1c961252757ea508403b5f51e7646
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
fe386a66df mdbx: minor refine mdbx_page_unspill(). 
Change-Id: I20bbd4c502a2258b945996fa1cf761eab7f88a80
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
92527a5206 mdbx: rename mdbx_page_spill to mdbx_cursor_spill(). 
Change-Id: I71628951d29ea39055b920a76521a19cb10ac8da
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
266bb70b7d mdbx: add debug to mdbx_kill_page(). 
Change-Id: Ifbb3164989a19d8ea48288b4eda1a8bcf31790ff
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
56758372cf mdbx: drop unused mdbx_audit(). 
Change-Id: I6d31c6ad83c058892319da6bbf38506b3f55085f
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
1314b29557 mdbx: add const for bsearch functions. 
Change-Id: Ie63eafa145bc8a63ae56b2693aaaf69bd21893aa
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
ef7814c018 mdbx: clean zero item of DPL-list. 
Change-Id: I7c60f3996783d97cee7cf2f0f9a7b856f4272641
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
d50fff8410 mdbx: minor refine mdbx_page_check(). 
Change-Id: I325c20fe042635a0641cc24f5661408f3476139b
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
44089b03df mdbx: minor refine mdbx_page_get(). 
Change-Id: Ie0ca17883ed88e4d9addf8b90061d25a80d77022
 2021-01-19 23:53:16 +03:00
Leonid Yuriev
a906569c58 mdbx: merge branch 'master' into devel-rebase. 
Change-Id: I96d944f283029b9c60e45683ee66b4564273b407
 2021-01-19 23:51:52 +03:00
Leonid Yuriev
b98895b8c7 mdbx: fix unaligned access to bootid on the meta-pages. 
Assume this fixes https://github.com/erthink/libmdbx/issues/153

Change-Id: I062c1bd99795a7b875d1cf60658790c1071a6775
 2021-01-18 14:27:58 +03:00
Leonid Yuriev
e766df658c mdbx-backport: avoid 4-byte aligned (i.e. unaligned) access to 64-bit integers. 
Historically, the page header provides 4-byte data alignment.
Therefore, unfortunately, the meta page data is also aligned on a 4-byte boundary, but contains 64-bit values.

This commit eliminates potentially unsafe access (SPARC, MIPS, etc) to these 64-bit values aligned on a 4-byte boundary.
Thus, a build with the `-fsanitize=undefined` now passes the tests both with CLANG 11 and GCC 10.

Change-Id: Ie441103e53ed96fd40507d8c0be0689e3fee69f5
 2021-01-18 14:08:49 +03:00
Leonid Yuriev
082df3a573 mdbx: refine MDBX_opt_* descriptions and defaults. 
Change-Id: I99ddf530d5683b755bc8bab1ea1098b0cc00b181
 2020-12-17 22:03:14 +03:00
Leonid Yuriev
697fce7ebc mdbx: rename MDBX_END_PURE_COMMIT (cosmetics). 
Change-Id: Ia3bdeca5989c841c0dc969e22981044fb334bf32
 2020-12-17 22:03:14 +03:00
Leonid Yuriev
70e76bcb4d mdbx: merge branch 'master' into devel. 
Change-Id: Idc8838922081cbc8ebe6c564555e4d304b529588
 2020-12-17 21:52:35 +03:00
Leonid Yuriev
d77af0bc1f mdbx: more checks against the use renewed (re-created or re-opened) DBI-handles. 
More for https://github.com/erthink/libmdbx/issues/146

Change-Id: I09e40598aca18e7ebd9798dc3be8675de3f8d976
 2020-12-17 10:36:05 +03:00
Leonid Yuriev
735da5fedd mdbx: auto-shrink env's DBI table. 
Change-Id: I9f423dab41863119a4491491e0ecd0a4aee42a82
 2020-12-17 01:57:06 +03:00
Leonid Yuriev
cda64ca663 mdbx: rework open/import/export of DBI-handles for robustness 
Resolve https://github.com/erthink/libmdbx/issues/146

Change-Id: Idd18dc0d038eeba47668983ecf4ff46eabd16de5
 2020-12-17 01:57:02 +03:00
Leonid Yuriev
166ed1c7d4 mdbx: refine prev commit (avoids SIGSEGV but assertion failure). 
More related to https://github.com/erthink/libmdbx/issues/146

Change-Id: Ie5277a8cc56421d20a7c9aad83724991b2efdc2d
 2020-12-15 15:43:19 +03:00
Leonid Yuriev
3758e7697e mdbx: more checks inside mdbx_cursor_close(). 
Related to https://github.com/erthink/libmdbx/issues/146

Change-Id: I7b90a0e515aa6320b0e89ec52fe01bc0be126071
 2020-12-15 15:07:42 +03:00
Leonid Yuriev
315ef41455 mdbx: use MDBX_PGL_LIMIT for DPL as a soft-limit and options.dp_limit as spilling-pressure to avoid false-positive MDBX_TXN_FULL. 
Change-Id: I0caae71d044cefa2f36286c2ae9fe14e356a6b51
 2020-12-06 14:42:13 +03:00
Leonid Yuriev
c1210d7c73 mdbx: refine mdbx_page_alloc() to avoid MDBX_TXN_FULL. 
More for https://github.com/erthink/libmdbx/issues/123

Change-Id: I527b7e9507f4c2aa565f1567924cfb1c477966dd
 2020-12-06 14:39:19 +03:00
Leonid Yuriev
338de2e1fb mdbx: refine mdbx_page_spill(). 
More for https://github.com/erthink/libmdbx/issues/132.

Change-Id: I40788c13b54453c17d5e5dae6c3d2f93226f8e00
 2020-12-04 15:38:26 +03:00
Leonid Yuriev
12ee5e6cac mdbx: minor refine mdbx_page_flush(). 
More for https://github.com/erthink/libmdbx/issues/132.

Change-Id: I703e1359d859b8901fcbdf57211f7805cd6d5ff1
 2020-12-04 15:38:26 +03:00
Leonid Yuriev
262e4981db mdbx: refactor mdbx_pages_xkeep(). 
More for https://github.com/erthink/libmdbx/issues/132.

Change-Id: Ie81b5c9efafa06e61d6d983769ff1f11a0e59a1e
 2020-12-04 15:38:15 +03:00
Leonid Yuriev
df8260e29a mdbx: add mdbx_pnl_purge_odd(). 
More for https://github.com/erthink/libmdbx/issues/132.

Change-Id: I1926ee48785f30b82e827a8e1e69be1e7a0a209f
 2020-12-04 15:38:15 +03:00
Leonid Yuriev
87c765c7be mdbx: fix/check lower border for dirty-pages list's limits/options. 
More for https://github.com/erthink/libmdbx/issues/128.

Change-Id: Ie9dccee5cf90d26968db04f4f6d9afa3018fcf0e
 2020-12-04 15:38:15 +03:00
Leonid Yuriev
0b02e080c4 mdbx: use pnl_xmerge() to merge spill-pages lists. 
More for https://github.com/erthink/libmdbx/issues/132.

Change-Id: I1f84a2be1fa5da6cf0e2783ff56f7a4c20782a7f
 2020-12-04 15:38:15 +03:00
Leonid Yuriev
76716c23b0 mdbx: add MDBX_opt_txn_dp_initial & MDBX_opt_txn_dp_limit. 
More for https://github.com/erthink/libmdbx/issues/128

Change-Id: I4be3c7476e9ce16ed4f27691d0df653552930e3c
 2020-12-03 17:17:30 +03:00
Leonid Yuriev
84235c7903 mdbx: drop unnecessary MDBX_HUGE_TRANSACTIONS option. 
Change-Id: I86116c2bada268d6e0a5c97deff808aaffe3e599
 2020-12-03 17:17:28 +03:00
Leonid Yuriev
c5061f9289 mdbx: dynamic dirty-page-list size. 
Resolve	https://github.com/erthink/libmdbx/issues/123

Change-Id: I5d9b2d8c2259132617fd2b226b9d9a6790fcad88
 2020-12-03 17:17:07 +03:00
Leonid Yuriev
6a5cec1f87 mdbx: refine mdbx_dpl_search(). 
Change-Id: I8acc87d9c2f306c981ad42cc859d32b7cfd7330e
 2020-12-03 10:25:36 +03:00
Leonid Yuriev
d8d89cca7d mdbx: refactor dirty-page-list (preparation to rework). 
Change-Id: Ib52bb52f73ef1d31f55838d879de081fc0a140c2
 2020-12-03 10:25:36 +03:00
Leonid Yuriev
5636dbf12b mdbx: add mdbx_dpl_alloc() & mdbx_dpl_free(). 
Change-Id: I0f05ff02d4a45d02faa1076cbb1a2a8e17b7e2b9
 2020-12-02 20:07:59 +03:00
Leonid Yuriev
c530c83337 mdbx: drop me_dirtylist field. 
Change-Id: I93ee802c1b6ef5c6a2443e19754e1a20dbe0aea9
 2020-12-02 20:07:59 +03:00
Leonid Yuriev
0b62453f6d mdbx: introduce MDBX_PGL_LIMIT. 
Change-Id: Ifab1156ec22fb0908284298393f52c09c0731829
 2020-12-02 20:07:59 +03:00
Leonid Yuriev
4cd08f0ef9 mdbx: add MDBX_opt_rp_augment_limit. 
More for https://github.com/erthink/libmdbx/issues/128

Change-Id: I94f3350b72ad11d0b586aaef1574f7654223bc40
 2020-12-02 20:07:55 +03:00
Leonid Yuriev
461ba000e3 mdbx: add MDBX_opt_dp_reserve_limit. 
More for https://github.com/erthink/libmdbx/issues/128

Change-Id: I620b6dcd833fcff19680770974f32431ab91c0ab
 2020-12-02 12:42:16 +03:00
Leonid Yuriev
42019e0b8d mdbx: add dp_reserve_limit with 1024 default. 
More for  More for https://github.com/erthink/libmdbx/issues/128

Change-Id: Ia92b1aea58640f183202495e3f6e2d531057afd2
 2020-12-02 12:42:16 +03:00
Leonid Yuriev
8f60050991 mdbx: initial support for set/get_options. 
Initial for https://github.com/erthink/libmdbx/issues/128

Change-Id: I540784c8b54d7443748e3c95f89a7390b818c811
 2020-12-02 12:42:13 +03:00
Leonid Yuriev
a8ed534cd3 mdbx: avoid 4-byte aligned (i.e. unaligned) access to 64-bit integers. 
Historically, the page header provides 4-byte data alignment.
Therefore, unfortunately, the meta page data is also aligned on a 4-byte boundary, but contains 64-bit values.

This commit eliminates potentially unsafe access (SPARC, MIPS, etc) to these 64-bit values aligned on a 4-byte boundary.
Thus, a build with the `-fsanitize=undefined` now passes the tests both with CLANG 11 and GCC 10.

Change-Id: Ie441103e53ed96fd40507d8c0be0689e3fee69f5
 2020-12-01 20:23:23 +03:00
Leonid Yuriev
adcb052915 mdbx: fix minor MSVC warnings. 2020-11-29 06:04:55 +03:00
Leonid Yuriev
6d2914c99b mdbx: minor/insignificant fix checking mdbx_rdt_lock() result for cases mutex-recovery after EOWNERDEAD. 
Change-Id: Ia5e13ea0d72afc97f7d678832a765a192dfacdff
 2020-11-25 17:55:29 +03:00
Leonid Yuriev
2a740d3807 mdbx-windows: fix ERROR_NOT_SUPPORTED while opening UNC pathnames. 
Resolve https://github.com/miranda-ng/miranda-ng/issues/2627

Change-Id: Iec06554dfcb9e0db215662d93fb5824cfe9ac7ef
 2020-11-24 15:53:14 +03:00
Leonid Yuriev
ddee04d991 mdbx: set DXB/LCK files position outside of a data. 
Set the position in files outside of the data to avoid corruption
due to erroneous use of file descriptors in the application code.

Change-Id: I8750f6e726edaddee254885756d5f91576a6bcb1
 2020-11-23 09:25:24 +03:00
Leonid Yuriev
537e3d18ae mdbx: minor clarify mmap-resize logging. 
Change-Id: Ibb6e4b8ee84d03a22ad0e0d1b437989739d43bb9
 2020-11-20 10:38:04 +03:00
Leonid Yuriev
1bbf20bf39 mdbx: refine multi-thread flipping of buffers during env-copy. 
Change-Id: Id132c1af0e1131da70ab1b35bce9f6a6548edbe3
 2020-11-19 11:59:42 +03:00
Leonid Yuriev
9054b25441 mdbx: fix fast completion of nested txn for case ones was open present DBIs. 
Change-Id: I87d28d1da5f72cc7ad77f8dd3f9a181f7eaa0024
 2020-11-19 09:26:33 +03:00
Leonid Yuriev
26767a5e06 mdbx: relax big-page size checking for compatibility. 
i.e. allow an overflow page be larger than required by placed data.
 2020-11-18 03:42:05 +03:00
Leonid Yuriev
96c2a56aa1 mdbx: use MDBX_SET_LOWERBOUND in mdbx_get_equal_or_great(). 
Change-Id: I5dd72fe82bd15938afc60cbc0f92e23c00f0d344
 2020-11-17 08:13:46 +03:00
Leonid Yuriev
5e02e7fb56 mdbx: add MDBX_SET_LOWERBOUND for mdbx_cursor_get(). 
Change-Id: I3638fdd10be8dfe128c43b465e9ca71f89175b3e
 2020-11-17 08:13:46 +03:00
Leonid Yuriev
9ea6922a2f mdbx: minor simplify mdbx_cursor_set(). 
Change-Id: I034f396368024af21e8ee741c13a28c9bc277121
 2020-11-17 07:59:51 +03:00
Leonid Yuriev
149b3d09e6 mdbx: don't export DBIs from nested txn. 
Change-Id: I8c354ac7f889debe2fcf830263cb060253887652
 2020-11-16 07:07:39 +03:00
Leonid Yuriev
1c925a0f2e mdbx: minor fix/avoid assertion inside mdbx_cursor_set(). 
Change-Id: I2cc38da698765b3eaa2bb575e16505d5d9438431
 2020-11-16 07:07:39 +03:00
Leonid Yuriev
0c3deac9db mdbx: fix cursors-EOF after search. 
Change-Id: Ie578611b64cca8dbcc00f958510143e8d1dc262a
 2020-11-16 07:07:39 +03:00
Leonid Yuriev
21bbba82fb mdbx: minor fix cursor_on_first/last for empty sub-db. 
Change-Id: I68c2e1bd28e62c9512bf8ef711fe21573e3450b9
 2020-11-16 07:07:39 +03:00
Leonid Yuriev
4ea2bea22e mdbx: fix save/restore/commit cursors for nested txn. 
Change-Id: Ibaf356bbc631ca2e41058787bd3e4aaaa8bc085a
 2020-11-16 07:07:38 +03:00
Leonid Yuriev
e328c1f829 mdbx: refactor/move mdbx_cursor_count(). 
Change-Id: I5487342013dfc3a9cf12d9081471ace0c37de23d
 2020-11-16 07:07:38 +03:00
Leonid Yuriev
ca115dd6a4 mdbx: refactor/rename/refine tw.cursors internals. 
Change-Id: Ie46d15f52a9d7365b52534a630754a31d3005a69
 2020-11-16 07:07:38 +03:00
Leonid Yuriev
9a2dbb845c mdbx: refactor/refine cursor next/prev. 
Change-Id: I0bfe0fed62e137e933027ba90cec23dbf1a6310b
 2020-11-16 07:07:38 +03:00
Leonid Yuriev
5f09ec73c9 mdbx: return MDBX_ENODATA instead of MDBX_EINVAL for non-positioned/EOF cursor and MDBX_GET_CURRENT. 
Change-Id: I2adf76f8f662e77e5a6aa077344fab6b430975c4
 2020-11-16 07:07:38 +03:00
Leonid Yuriev
c55f99073b mdbx: fix MDBX_NEXT for EOF case. 
Change-Id: I84c033a24d2215a4867affab5b148bf6ba5d57dc
 2020-11-16 07:07:38 +03:00
Leonid Yuriev
b1446b7752 mdbx: minor fix/distinction MDBX_EBADSIGN and MDBX_EINVAL errors for cursors. 
Change-Id: I6d72638a69ff0f793156fe5e0e7ca5531a97c7cf
 2020-11-16 07:07:38 +03:00
Leonid Yuriev
b6f0070f85 mdbx: add mdbx_cursor_copy(). 
Change-Id: I476f1230beec6a550897bef236745530154c1079
 2020-11-16 07:07:38 +03:00
Leonid Yuriev
2489e0ba6e mdbx: refactor/rename cursor_copy_internal(). 
Change-Id: I334e68d3d424ef15cafcaafd23d61f80611fce01
 2020-11-16 07:07:38 +03:00
Leonid Yuriev
459e769844 mdbx: engage cursor tracking for all merge/rebalance operations. 
Assume this resolves https://github.com/erthink/libmdbx/issues/131.

Change-Id: I5111618a34d91d083b81245ac959b744a86c4495
 2020-11-16 07:07:27 +03:00
Leonid Yuriev
37a1d546b7 mdbx: minor refine mdbx_rebalance(). 
Change-Id: I10a6a208b173bdee1d325aa7b2624de879814647
 2020-11-16 06:28:11 +03:00
Leonid Yuriev
b209e91ad4 mdbx: zeroed mc_top while tracking nested cursors. 
This avoids scanning dummy cursors during tracking.

Change-Id: Id4edc665010215b4a946f9438475691f1387313c
 2020-11-16 05:48:44 +03:00
Leonid Yuriev
0c78da99b0 mdbx: add cursor_is_tracked() and assertions. 
Change-Id: I83db1a714597b134aa30e4ca7019bb7f541a2428
 2020-11-16 05:48:23 +03:00
Leonid Yuriev
10681a53ae mdbx: cleanup/refine GC-flags inside mdbx_page_alloc(). 
Change-Id: Ie81c703815a7b3fbe59ad9b406811d5ed2913d02
 2020-11-16 05:43:53 +03:00
Leonid Yuriev
5ad167410c mdbx: fix handling MDBX_GET_BOTH_RANGE. 
Partially revert 77e84ccca8d634db087fc710b1c4aea69ee5a6a6 and 7522246ccdb3c1b0928aa3e0f593133db0fadae5.

Resolves https://github.com/erthink/libmdbx/issues/130

Change-Id: Ib9f49a3e6a108a0d914ccde7ccb4dc0ef6056ff7
 2020-10-31 20:54:53 +03:00
Leonid Yuriev
70241e25db mdbx: don't limit reclaimed-pglist if DB full. 
More for 55d190bad9723fd9766466005b1a14e70779fc89 and https://github.com/erthink/libmdbx/issues/123

Change-Id: Iecd19fe8f159d29a1434d03f43f0b277f2d6ab86
 2020-10-30 19:22:37 +03:00
Leonid Yuriev
d5658c496f mdbx: fix mp_txnid check for spilled pages. 
This is OMG stupid bug.

Тем не менее, ошибка не была замечена по трём причинам:
- some TODOs should be resolved, like [this](faddc71eac/src/core.c (4014)) and some others;
- test cases should be extended to triggering the page spilling threshold;
- mdbx_env_set_option() should be provide to tunning such threshold(s) (https://github.com/erthink/libmdbx/issues/128).

Resolves https://github.com/erthink/libmdbx/issues/126

Change-Id: If76336620aa83e6916e3aeaa838e0b6e68c34fd7
 2020-10-30 17:44:31 +03:00
Leonid Yuriev
16c900b0a1 mdbx: fix assertion inside mdbx_cursor_del0(). 
Change-Id: I87ad3777215bee8a2b19f53ec92299db89431fa4
 2020-10-29 00:55:42 +03:00
Leonid Yuriev
3fd079262c mdbx: fix extra-rare MDBX_KEY_EXIST during mdbx_commit(). 
The MDX_KEYEXISTS error could occur inside mdbx_update_gc() in the extremely rare case:

- no GC records was reclaimed before mdbx_txn_commit() called;

- there were few loose pages during the transaction;

- some reader prohibit reclaiming, therefore mdbx_page_alloc(MDBX_ALLOC_GC),
  which called for obtain present GC-record's Id, returns MDBX_NOTFOUND;

- immediately then the reader completes its transaction and unlocks reclaiming;

- mdbx_update_gc() decide that no reclaimable GC entries,
  i.e. no GC-entries with ID < find_oldest(),
  and it is safe to use find_oldest() - 1 to store loose page list;

- but find_oldest() actually returns new/larger ID than expected,
  So KEYEXISTS will returned if using this ID.

Change-Id: I9726217d6b5983f1e31a211c0eeb3edc8ff94282
 2020-10-26 14:37:16 +03:00
Leonid Yuriev
77e84ccca8 mdbx: refine mdbx_cursor_set() around the exact-flag. 
Change-Id: Ifbad6bb4a3a4eabaf85c7986f9566705d53c45b9
 2020-10-26 03:52:07 +03:00
Leonid Yuriev
7522246ccd mdbx: fix intenals for mdbx_get_equal_or_great(). 
Change-Id: I7b44df14c6f9ede4844d53e28c1b34d84fc664fa
 2020-10-26 00:42:21 +03:00
Leonid Yuriev
005517539b mdbx: adds SIGPIPE suppression inside the env_copy-thread. 
Change-Id: Ib7d22b8db7435b396bd997b65ae1d6d5d3431ba9
 2020-10-23 03:22:22 +03:00
Leonid Yuriev
fed14c8f4c mdbx: minor tweak mdbx_pages_xkeep(). 
Change-Id: I6df699aeb49640b0365918c33cff05dcbcef2b71
 2020-10-23 03:18:00 +03:00
Leonid Yuriev
55d190bad9 mdbx: avoid MDBX_TXN_FULL while searching for a large multi-page region. 
Resolves https://github.com/erthink/libmdbx/issues/123
but with TODO:
 - provide a user-configurable threshold instead of currently hard-coded default (MDBX_PNL_MAX/2).

Change-Id: Ia7bfd5f8d36e027444d234e3c3aabe4832313466
 2020-10-22 18:04:02 +03:00
Leonid Yuriev
8a7caec54a mdbx: cleanup deletion internals. 
Change-Id: Ibcc5e02d5274c4eac5d6f269bc2b029f6b30c5ab
 2020-10-22 16:28:44 +03:00
Leonid Yuriev
b0928219c3 mdbx: fast completion pure nested transactions. 
Change-Id: I467cc0f2f3e781bf23bf8c45eb021987ae7c123b
 2020-10-21 02:24:39 +03:00
Leonid Yuriev
daf37363b4 mdbx: fix cursor state after delete current entry. 
The fix affect enough of code sections,
so there is a chance that some more changes will be required.

Resolves https://github.com/erthink/libmdbx/issues/121
Related to https://github.com/ledgerwatch/turbo-geth/issues/1147

Change-Id: I45b7637516de410923a0d723dbda8d818662ba15
 2020-10-21 02:24:33 +03:00
Leonid Yuriev
b964b2abf5 mdbx: more spelling. 
Change-Id: I45e42c1d8ef51f910b8e41279b92e54a6b2ce772
 2020-10-21 01:00:48 +03:00
Leonid Yuriev
7001d971e1 mdbx: clarify/refine mdbx_flush_iov(). 
Change-Id: Ib9462efdbf97e42b1e80bf130f8150072102d9ed
 2020-10-21 01:00:48 +03:00
Leonid Yuriev
f73a8a8680 mdbx: add latency gathering for commit stages. 
Change-Id: If68ceb6e69e5e565ce9de0fd9a80424b6da280c5
 2020-10-21 01:00:48 +03:00
Leonid Yuriev
7cf92b66cf mdbx: add LIBMDBX_INLINE_API (both inline and non-inline of some API functions). 
Change-Id: I00c2b6d3d2a0467080791ea0c1c2242742a20c78
 2020-10-17 01:00:36 +03:00
Leonid Yuriev
38485c9f30 mdbx: minor refine cursor_sibling(). 
Change-Id: I2c92ef2c3081dfa0a9bdcd47de0f912a9927519e
 2020-10-16 18:57:07 +03:00
Leonid Yuriev
112ce742f5 mdbx: refine update_gc(). 
Change-Id: I877cdf2efb623c61dc810ec1cebb985fe925a120
 2020-10-12 00:09:06 +03:00
Leonid Yuriev
62da4db09a mdbx: fix/refine the use of C11 atomics. 
Change-Id: I5d925d4625b06296fd82f4b35ee36be682e7b2d3
 2020-10-11 18:54:07 +03:00
Leonid Yuriev
041a4c0aa5 mdbx: make enabling of read-ahead strategy more aggressive. 
Change-Id: I7765c1a1ac27db86ce9676846ec5a723860dc934
 2020-10-11 01:14:30 +03:00
Leonid Yuriev
071ad525c8 mdbx: refine handle_env_pathname() for direct pathname of data-file inside sudir-mode. 
Change-Id: I5b7e7c7ea5c17e00c344fedb5c96f8d94fc04fc8
 2020-10-11 00:14:53 +03:00
Leonid Yuriev
0627d902dd mdbx: add mdbx_env_delete(). 
Resolves https://github.com/erthink/libmdbx/issues/119
Related to https://github.com/Kerollmops/heed/issues/58

Change-Id: Iec5bf5978e45bb6843f3ed8dd06ea4d34f2895cb
 2020-10-10 00:58:53 +03:00
Leonid Yuriev
cd0c727880 mdbx: internally split-out mdbx_handle_env_pathname(). 
Change-Id: Ibe387662d737d4d1e274ac0e126053872be503f9
 2020-10-10 00:52:41 +03:00
George Hazan
e8fecd1eec mdbx: fix opening DB on a network shares (pass MDBX_EXCLUSIVE from env). 
Change-Id: Ic5c68033ba81043ac7800f99aae364bf082c868f
 2020-10-08 01:50:18 +03:00
Leonid Yuriev
0f3b82f661 mdbx: minor refine WITH_CURSOR_TRACKING macro. 
Change-Id: I61a4782ae76aed624fe37f7dde2b0ef7eb5dfb64
 2020-10-08 01:50:18 +03:00
Leonid Yuriev
b274fdd142 mdbx: clarify page-merge strategy. 
Change-Id: I1bb843ab6bdfcd53deae6d219a5b164bd3a79fd3
 2020-10-08 01:49:56 +03:00
Leonid Yuriev
f5ce471ebb mdbx: fix minor copy&paste typo. 
Change-Id: I5a1be230d0ca92123006e709aaaf78496918c91a
 2020-10-07 12:18:28 +03:00
Leonid Yuriev
fe65c122d2 mdbx: fix pthread_yield() for non-GLIBC. 
Change-Id: I080e37a42b62e524896dea8747e9f23e2fcd584f
 2020-10-06 00:28:12 +03:00
Leonid Yuriev
70b615e8d4 mdbx: don't use yield instruction on ARM if unsupported. 
Change-Id: I0b01d783fe4336b089f4b051fb61c203b5879aa5
 2020-10-05 19:15:07 +03:00
Leonid Yuriev
fc965c57ac mdbx: fix merge_sync_flags() to preserve MDBX_UTTERLY_NOSYNC. 
Change-Id: I59dbd1ef8ac1747dd20565ff444e4feb5d0e0e71
 2020-09-30 04:01:36 +03:00
Leonid Yuriev
f6850f5367 mdbx: Support for user-settable cursor context. 
Change-Id: I9bd60c432924e39020b2d3af3280f13c44d6cd91
 2020-09-29 21:15:25 +03:00
Leonid Yuriev
c8a0951566 mdbx: rework API and Docs around Handle-Slow-Readers (no algorithmic changes). 
Change-Id: I5b76a8400ce6f5f241f8e4a7f53d746fe39f8e1e
 2020-09-29 21:15:25 +03:00
Leonid Yuriev
6294e1710a mdbx: support for user-settable transaction context. 
Change-Id: Ib4a345628e2c1ca2f95ac7615ea53d94911e5198
 2020-09-29 20:59:55 +03:00
Leonid Yuriev
5374d06d92 mdbx: fix minor MSVC warning. 
Change-Id: Ie3c9fa7ba21cd88f7a0c08a1529542ba394fe9c8
 2020-09-29 20:59:55 +03:00
Leonid Yuriev
5627521f16 mdbx++: more spelling. 
Change-Id: I3f33e695fb918c6a57c39047bba54a72ecb9bc5b
 2020-09-27 17:01:59 +03:00
Leonid Yuriev
6eaa838e3e mdbx: clarify logging for MDBX_WANNA_RECOVERY in case sync-needed in rdonly-mode. 
Change-Id: I7032cfbce857e2acfb93de383272a4f41ab6a7fb
 2020-09-27 17:01:58 +03:00
Leonid Yuriev
12339d4e7c mdbx: rename bootid' fields to be neutral for endianess. 
Change-Id: I7d52d3c2c93dcc6e886d336885070eb1aee1c284
 2020-09-26 02:23:09 +03:00
Leonid Yuriev
75a4463811 mdbx: minor misc cosmetics & refines. 
Change-Id: I84b70aeded047ab3786a57c864dbc89364969afa
 2020-09-25 01:21:45 +03:00
Leonid Yuriev
860aa017db mdbx: Merge branch 'master' into devel. 
Change-Id: Ic130cd181097332aa2f49019d75403e18d8cba0d
 2020-09-25 01:14:57 +03:00
Josh Soref
448728f584 mdbx: a lot of spelling (squashed). 
Many Thanks to Josh Soref for these fixes.
https://github.com/jsoref

Resolves https://github.com/erthink/libmdbx/pull/118.

Change-Id: I4e09347da5c9d7a77cdd918a3b15284371440076
 2020-09-25 01:01:25 +03:00
Leonid Yuriev
58708f7162 mdbx: add mdbx_env_turn_for_recovery(). 
Change-Id: Ic02a5adf9ad398ae5b499e8808157e41ec42457e
 2020-09-23 10:05:38 +03:00
Leonid Yuriev
9b5097fe9d mdbx: add paranoid checks for txnid overflow. 
Change-Id: I0036a080d40a0da9ffc55ff838d91799ff0ceadb
 2020-09-23 10:05:38 +03:00
Leonid Yuriev
d4b16c981d mdbx: fix minor Coverity warning. 
Change-Id: Id92d85a7032a20a4b06d87878e32d8997b6558e3
 2020-09-23 10:05:37 +03:00
Leonid Yuriev
7fe4fccda8 mdbx: acquire exclusive lock for the destination file during mdbx_env_copy(). 
Change-Id: Ic75796025dd2a649840b96b1f71aecac9360d010
 2020-09-23 10:05:37 +03:00
Leonid Yuriev
a7a8631bc3 mdbx: allow sync_file_range() to fail for Valgrind/QEMU cases. 
Change-Id: I9aa77d1debfbd0cb18e940946533e4ed758d08e8
 2020-09-21 17:18:07 +03:00
Leonid Yuriev
c01750be2e mdbx: provide MDBX_USE_SYNCFILERANGE option. 
Change-Id: Icf1f27d6203653b4e2f2180a59a65e958c4d5e2e
 2020-09-21 17:18:07 +03:00
Leonid Yuriev
a2e2e5c8a0 mdbx: allow sendfile() and copy_file_range() to fail for Valgrind/QEMU cases. 
Change-Id: I37ee8d652d91a8d2106c782beceaacb13e9f667f
 2020-09-21 03:29:38 +03:00
Leonid Yuriev
25e3968199 mdbx: provide MDBX_USE_SENDFILE and MDBX_USE_SENDFILE options. 
Change-Id: Icd9a6487ae6e398276a9e720926ff24de5897c1a
 2020-09-20 20:16:26 +03:00
Leonid Yuriev
d9daf2944d mdbx: add env_open_for_recovery() (squashed). 
Change-Id: I0151b21610def433745c33d1f6e0b66ce655d1a9
 2020-09-19 01:48:24 +03:00
Leonid Yuriev
b321f67ed2 mdbx: Merge branch 'master' into devel. 
Change-Id: I4f180753fe52b54f37f4fd12af2ba1b3ca541726
 2020-09-19 01:47:12 +03:00
Leonid Yuriev
73647a5e46 mdbx: fix/refine mdbx_walk_tree() for working with invalid/corrupted DB. 
Change-Id: I2594b785f675319027ff129d12683c4392e6e2f4
 2020-09-19 01:44:21 +03:00
Leonid Yuriev
6a99303ec1 mdbx: fix mdbx_strerror() for MDBX_BUSY error. 
Change-Id: Iea02f1daa3f52a9b737366e9af3c7def748ab523
 2020-09-18 23:57:35 +03:00
Leonid Yuriev
e762442917 mdbx: don't resize mapping beyond used part of DB for read-only txn. 
This allow reading extra large DB when used part is acceptable for current platform, i.e. reading DB >4G on a 32-bit platform.
 2020-09-18 15:02:36 +03:00
Leonid Yuriev
488a272f8b mdbx: more for opening large DB from 32-bit env. 
- return `MDBX_TOO_LARGE` only when used part of DB is too large for current platform;
- auto lowering both of `mm_geo.lower` and `mm_geo.upper` if the used part of DB is acceptable for current platform;

Change-Id: If67109ebb96063451c50f279c473ed38355a098a
 2020-09-18 13:34:34 +03:00
Burak Arslan
efe400264f
mdbx: minor fix msvc build (#117). 2020-09-15 20:03:42 +03:00
Leonid Yuriev
10b170c6cd mdbx: add mdbx_cursor_create() and mdbx_cursor_bind(). 
Change-Id: I223de3cca7865d58f17a59ab27ec6be730e04a90
 2020-09-15 02:05:25 +03:00
Leonid Yuriev
234d65dc9d mdbx++: update copyright & license note. 
Change-Id: If6aac3dec6d05247aec5af956385066619685c35
 2020-09-14 21:19:56 +03:00
Leonid Yuriev
cacc4aa829 mdbx++: changes after codereview-1 (part 2 of 2). 
Change-Id: I8e1ca134bb8c5d447895f116247dfd12fa6871f0
 2020-09-14 21:19:56 +03:00
Leonid Yuriev
58bcfb006e mdbx: add mdbx_printf_args() macro. 
Change-Id: I7fca72f8cc912d8644ecf149b755c78fb3cc7e23
 2020-09-10 15:35:43 +03:00
Leonid Yuriev
50c25f479d mdbx++: more Doxygen descriptions and refinement. 
Change-Id: Iec4cfa220f140c0fcc858f51283af2f9c4dd7a65
 2020-09-10 01:15:35 +03:00
Leonid Yuriev
538d2e4d62 mdbx: skip update meta-geo in read-only mode. 
Change-Id: I1c9610920ad87dc8110e8d03038ef385220000c8
 2020-09-07 12:38:08 +03:00
Leonid Yuriev
2bcd2e510e mdbx: Merge branch 'devel' into c++. 
Change-Id: Iedfe7e7ae21d08628c24b4dd5a788bc43cced2f5
 2020-09-05 13:48:40 +03:00
Leonid Yuriev
54ccb92444 mdbx: skip update meta-geo in read-only mode. 
Change-Id: I1c9610920ad87dc8110e8d03038ef385220000c8
 2020-09-05 11:57:34 +03:00
Leonid Yuriev
c24e566591 mdbx: alter notice/warning/verbose level of some log-messages. 
Change-Id: I08c3f5cf6489095f04c8becc4818d34b8db61422
 2020-09-05 11:48:29 +03:00
Leonid Yuriev
6d7ec5a257 mdbx: enable non-debug logging in non-debug builds. 
Change-Id: I295de5ef6369a55500b023abc3dcb26f5071c2da
 2020-09-05 01:57:02 +03:00
Leonid Yuriev
b1877d08ae mdbx: rework mdbx_chk & tree-traversal. 
Change-Id: Idc131539426fe0cbb97a105cff2d0a12b1496bfe
 2020-09-05 01:57:02 +03:00
Leonid Yuriev
9a5df2b284 mdbx: refine errors returning from get_page(). 
Change-Id: I2da37d3b35679f345324d6dd97d2a3f634449b56
 2020-09-05 01:57:02 +03:00
Leonid Yuriev
6867c13bd1 mdbx: more runtime checks/errors instead of assertions. 
Change-Id: I843f28b7faeb23f986154dd92a0e2b50d66a1dde
 2020-09-05 01:57:02 +03:00
Leonid Yuriev
a758e32f91 mdbx: remove assertions from page_check(). 
Change-Id: Ic157e7fc43d9c3a84e530d0001409ff2a5c1f818
 2020-09-05 01:57:02 +03:00
Leonid Yuriev
6e339fc849 mdbx: add bad_page(). 
Change-Id: I5233e4a701ee11fd59d7083576f40f7df3068ddd
 2020-09-05 01:57:02 +03:00
Leonid Yuriev
f0de3ff098 mdbx: continued page-check. 
Change-Id: Id4aa37284a12f49e359a6c9391d2fd5b6acb2ad0
 2020-09-05 01:57:02 +03:00
Leonid Yuriev
d6645f84f2 mdbx: more error logging. 
Change-Id: I3757e587514450634b5d3ebf8721c9ed4ac182f1
 2020-09-05 01:57:02 +03:00
Leonid Yuriev
cd6aa4a708 mdbx: parent-page-txnid checking. 
Change-Id: I6d37326c4ff2aa32587704b971bd650d9221b06f
 2020-09-05 01:57:02 +03:00
Leonid Yuriev
cd4f732a87 mdbx: drop internal unused fields. 
Change-Id: I634d3e0695f300df79129a15da752a23b277a0ce
 2020-09-05 00:55:00 +03:00
Leonid Yuriev
b095ad872c mdbx: add MDBX_ALLDUPS & MDBX_UPSERT, rework handling of others. 
Change-Id: I27d437540d883935d78242e4fc7e28951ab9f496
 2020-09-05 00:26:06 +03:00
Leonid Yuriev
5ec0f5b299 mdbx: refine mdbx_replace_ex(). 
We can skip update only when a data exactly the same,
but user's md_dcmp() may returns zero even data is NOT matches byte-to-byte.
So to skip update the cmp_len fast() should be used instead of md_dcmp().

Change-Id: I6ad0f7cfc6a18722b46e565deef3a544ddf9968a
 2020-09-05 00:26:06 +03:00
Leonid Yuriev
33b1cf2931 mdbx++: Initial C++ API (some extra methods are not implemented). 
Change-Id: I0478d0c94dcd12b52916e87815e5731817407c3c
 2020-09-05 00:26:06 +03:00
Leonid Yuriev
89dba59ce2 mdbx: refine/fix handling MDBX_MULTIPLE in the mdbx_cursor_put(). 
Change-Id: I06ac9d73cec01afb492b5c663acf310d19e4900b
 2020-09-05 00:26:06 +03:00
Leonid Yuriev
426d207fc4 mdbx: fix page_get() null-defer when DB corrupted. 
Change-Id: Ia4b5870e6718cd376d25893d33b9c659e3cd151a
 2020-09-04 23:56:43 +03:00
Leonid Yuriev
6b1c2ff762 mdbx: fix mdbx_env_open() for re-opening after non-fatal errors. 
Change-Id: Ibbe9a8e017837a21fe1eeebb86e1b3a7a370e77c
 2020-09-01 20:54:15 +03:00
Leonid Yuriev
12e19cedc0 mdbx: minor fix mdbx_replace_ex(). 
Change-Id: I2e1f27b1d73b450643743cf9d4ad35d5cd4547de
 2020-08-29 16:36:30 +03:00
Leonid Yuriev
317816d6c7 mdbx: refine/simplify mdbx_cursor_del(). 
Change-Id: Ibd72817a33427483a4440eecc911815420b2e13f
 2020-08-29 14:46:15 +03:00
Leonid Yuriev
1d48cb88fe mdbx: Add mdbx_txn_break(). 
Change-Id: Id38a09a0a95d092d75f2eb4fad2cbd7805cd59d0
 2020-08-28 21:20:51 +03:00
Leonid Yuriev
04a77d3bf3 mdbx: add MDBX_ALLDUPS & MDBX_UPSERT, rework handling of others. 
Change-Id: I27d437540d883935d78242e4fc7e28951ab9f496
 2020-08-28 19:08:45 +03:00
Leonid Yuriev
08faeda430 mdbx: refine mdbx_replace_ex(). 
We can skip update only when a data exactly the same,
but user's md_dcmp() may returns zero even data is NOT matches byte-to-byte.
So to skip update the cmp_len fast() should be used instead of md_dcmp().

Change-Id: I6ad0f7cfc6a18722b46e565deef3a544ddf9968a
 2020-08-28 16:45:30 +03:00
Leonid Yuriev
88a4b8cb9b mdbx++: Initial C++ API (some extra methods are not implemented). 
Change-Id: I0478d0c94dcd12b52916e87815e5731817407c3c
 2020-08-28 16:45:30 +03:00
Leonid Yuriev
e3f0db7708 mdbx: refine/fix handling MDBX_MULTIPLE in the mdbx_cursor_put(). 
Change-Id: I06ac9d73cec01afb492b5c663acf310d19e4900b
 2020-08-28 16:45:29 +03:00
Leonid Yuriev
c53627f5ac mdbx: fix null-deref during MDBX_TXN_RDONLY_PREPARE in the MDBX_EXCLUSIVE mode. 
Change-Id: I20b2bd4137293261a546910c8175be531d38b2c9
 2020-08-23 16:28:00 +03:00
Leonid Yuriev
0a75417d5f mdbx: add pure- & const-function attributes to C API. 
Change-Id: Ie4d1742f3d4717a0cd1fd5677b9b1ae641193d45
 2020-08-23 13:29:44 +03:00
Leonid Yuriev
54a09967a1 mdbx: refine mode_t for lck-file. 
Change-Id: I7674a3a1c0a5778f0f362faa1d608c25e242d92d
 2020-08-06 01:48:56 +03:00
Leonid Yuriev
17380fe079 mdbx: minor fix for MDBX_DBG_LEGACY_MULTIOPEN. 
Change-Id: I7d1c30699708c4b9e47e1b781e96c9a4b6c2b109
 2020-08-06 01:48:56 +03:00
Leonid Yuriev
ffa10a25d6 mdbx: use enums & bool in the C API. 
Change-Id: I952f578cceb7d02ade16f545d47245d4d9106441
 2020-08-06 01:48:56 +03:00
Leonid Yuriev
87de3fc25f mdbx: add MDBX_TXN_RDONLY_PREPARE. 
Change-Id: I95647d1679b69d1e97514a45f20d7373174244d5
 2020-08-06 01:48:56 +03:00
Leonid Yuriev
1e7a1da14e mdbx: separate transaction flags. 
Change-Id: Ib9a0f6946e1ecf35059e2dccc91319ae130c9f9b
 2020-08-06 01:48:56 +03:00
Leonid Yuriev
5e43ee61a2 mdbx: drop/deprecate MDBX_MAPASYNC. 
Change-Id: I472f97f568a32325eb056c8ee4d2f2350a473bda
 2020-08-06 01:48:56 +03:00
Leonid Yuriev
f4021c8028 mdbx: refine mdbx_dump_val(). 
Change-Id: I72dac7c44c842d16d6b8776248ba20014d0709a9
 2020-07-31 01:33:16 +03:00
Leonid Yuriev
0a01b46112 mdbx: add mdbx_thread_register() and mdbx_thread_unregister(). 
Change-Id: I605bc75a20631e781043fafcc26f5e59cb40adaa
 2020-07-31 01:33:16 +03:00
Leonid Yuriev
b91918b027 mdbx: add mdbx_env_get_maxdbs(). 
Change-Id: Ie4bdd2f8a46130f277ef98861a5fca98f55bad54
 2020-07-31 01:33:16 +03:00
Leonid Yuriev
9e90718bd5 mdbx: add check_env(). 
Change-Id: Iff9258d1e31349da77ef030cfe6da7bcfc5c5a19
 2020-07-31 01:33:16 +03:00
Leonid Yuriev
0387d6f6d4 mdbx: rename MDBX_ENV_CHECKPID. 
Change-Id: Id0afbea99b0747c6d7a5062c1c4a82ebc69f4f40
 2020-07-31 01:33:16 +03:00
Leonid Yuriev
1e3c4dc0ef mdbx: define & use bool type for C API. 
Change-Id: Icb6cd635a4813bfc92d31bdc527414bdddce4a63
 2020-07-31 01:33:16 +03:00
Leonid Yuriev
a902538e34 mdbx: more Doxygen tags (almost done). 
Change-Id: I696e717e37a905f68af059c51f9df327c257332e
 2020-07-24 22:49:21 +03:00
Leonid Yuriev
5f4f828bae mdbx-doc: provide non-API docs via doxygen (squashed). 
Change-Id: Ie33858517f964f794ec182a1e8bb630730a0f172
 2020-07-23 18:44:32 +03:00
Leonid Yuriev
9720ed39f5 mdbx: using enum instead of #define for flags/modes. 
Resolve https://github.com/erthink/libmdbx/issues/108

Change-Id: I45897300375d2b5b9361aaba81dadcf9801fe3cf
 2020-07-08 02:26:46 +03:00
Leonid Yuriev
2e0d2e65af mdbx: fix minor typos. 2020-07-07 23:16:43 +03:00
Leonid Yuriev
e989cb05ed mdbx: deprecate mdbx_dbi_open_ex() and custom comparators. 
Change-Id: I1d20f77b9ba5d8d5ec891df17019377afb466b06
 2020-07-07 19:33:59 +03:00
Leonid Yuriev
3351c1f869 mdbx: implements remapping of the database file when it it possible. 
Change-Id: Ida15ba1f396a33b2c6063e680dff612f39a9608f
 2020-07-06 16:23:52 +03:00
Leonid Yuriev
2d0a5c42a9 mdbx: avoids unnecessary remap in case geometry changed concurrently by another process(es). 
Change-Id: Ibf3a90bc79b1e8e0fa752dabc21b129a697c08d4
 2020-07-06 14:35:41 +03:00
Leonid Yuriev
5dbb0b4cfe mdbx: rename internal flags & fields for clarity. 
Change-Id: I79d6d8e88b0b4c30cbb8a464ca17f07dc2ab44c6
 2020-07-06 14:35:41 +03:00
Leonid Yuriev
c22749a630 mdbx: add TODO for auto-recovery. 
Change-Id: I7d5fd83a2fdeaec0501ba0339bbe865d06356bad
 2020-07-05 00:34:32 +03:00
Leonid Yuriev
e6f5998b76 mdbx: refine mode bits while auto-creating LCK-file. 
Change-Id: Ideb603cf6a1709188b3c121a9d6bf2097b9c244d
 2020-07-05 00:34:32 +03:00
Leonid Yuriev
e6eeed6cf2 mdbx: return MDBX_CORRUPTED in case all meta-pages are weak and no other error. 
Change-Id: I38c85738d8ccdb87856fa1a6e989ddaf75109722
 2020-07-05 00:34:32 +03:00
Leonid Yuriev
0afc21eed9 mdbx: add mdbx_get_keycmp() & mdbx_get_datacmp(). 
Change-Id: I11f8a54f7eb21b0bab575c8f6b89081b7eaf1bb8
 2020-06-24 17:15:56 +03:00
Leonid Yuriev
428f753c03 mdbx: add key-to-value functions. 
Change-Id: Ie0acd8c58833047931444fd9d94d5b82e6d541bb
 2020-06-24 17:15:56 +03:00
Leonid Yuriev
0fc2d39cce mdbx: allow mixind with/without MDBX_WRITEMAP for the same database. 
Change-Id: I916bb9e9b44a58cf78786e76a8c42d62e0bfc579
 2020-06-20 23:55:15 +03:00
Leonid Yuriev
e008f3132d mdbx: support for huge transactions (MDBX_HUGE_TRANSACTIONS option). 
Change-Id: I5d6cce6a7fb816add8cb4c066cc50f31cdebf9d5
 2020-06-13 03:06:20 +03:00
Leonid Yuriev
7ab9d24d3b mdbx: fix env_set_geometry() for opened-env + outside-txn case. 
Change-Id: I4526aa8011895ef7b5529995503966117c37e107
 2020-06-08 19:12:12 +03:00
Leonid Yuriev
03cc19babb mdbx: preserve error from env_set_geometry() inside setup_dxb(). 
Change-Id: I8dae0b38eb09cfaa07cdd80cb37dc414e2f99d7a
 2020-06-08 19:12:12 +03:00
Leonid Yuriev
36dc9e26b6 mdbx: avoid forward-invalidation of DBI-handle when it just closing. 
Change-Id: I379f07c2eae9ea68ddf554d10d7d1deed1a00d2b
 2020-06-04 12:24:11 +03:00
Leonid Yuriev
0146b3e2c0 mdbx: avoids using pwritev() for single-writes (up to 10% speedup for some kernels & scenarios). 
Change-Id: I71fbf0e113b1de4d05b2a43e8532a977c42b8f91
 2020-05-29 18:39:43 +03:00
Leonid Yuriev
ea353471d5 mdbx: fix minor typo. 
Change-Id: I02ab17d1867e8b1a0b9ef971c9922688eca9dc7c
 2020-05-27 21:48:14 +03:00
Leonid Yuriev
1f301e7e6f mdbx: explicit sync lck-file after re-write/re-create. 
Change-Id: I2fad76b1d04e4b0cd2ca08cb722bd819153f9824
 2020-05-27 21:48:14 +03:00
Leonid Yuriev
5cc1cb3f87 mdbx: add mdbx_dbi_dupsort_depthmask(). 
Change-Id: Ib056663f26baea517d6f2b9d58cd64c643619ef9
 2020-05-25 14:53:38 +03:00
Leonid Yuriev
51a016245a mdbx: using couple of condvars/events to avoid glitches on Windows. 
Change-Id: I3256a8dcbb95c78e8dea3eb31ca73f42c58d2f61
 2020-05-24 13:29:55 +03:00
Leonid Yuriev
d1d65f8090 mdbx: removes wrong condition from the assertion. 
Fixes https://github.com/erthink/libmdbx/issues/105

I decided to remove the extra check, since a sub-cursor cannot return data
and there is no point in adding assignments just for this check.

Change-Id: I6a983930abb37b8f6ab768602763495572adf7fd
 2020-05-22 22:12:08 +03:00
Leonid Yuriev
78e592579a mdbx: refine update_gc() for small pages with huge transactions. 
Change-Id: I66080e11a780a6a711691918179bc21d9a121347
 2020-05-17 19:16:35 +03:00
Leonid Yuriev
d3e609d7ff mdbx: simplify page-merge for LEAF2. 
Change-Id: I25ad34f02b86b47562fc3c74f5454432f687cbde
 2020-05-17 19:16:19 +03:00
Leonid Yuriev
5e2067decd mdbx-tools: use MDBX_ACCEDE to open DBI with custom comparators. 2020-05-15 22:03:29 +03:00
Leonid Yuriev
8f31aad0fb mdbx: use MAX_TXN internally (cosmetics). 
Change-Id: Ic8171dbfa8bb32272e46e939223316f6182d3a7a
 2020-05-15 18:13:10 +03:00
Leonid Yuriev
463a8af35b mdbx: copy unaligned keys/values instead of return error to avoid break compatibility. 
Change-Id: I2f95674a3836568a5c51155ca70da0683c50b424
 2020-05-15 11:30:35 +03:00
Leonid Yuriev
c2c74075e7 mdbx: fix Valgrind minor warning with MDBX_DEBUG > 1. 
Change-Id: Idb93c6a3843e76071a55cbd6c34ddd22541c77d3
 2020-05-15 11:30:35 +03:00
Leonid Yuriev
e28d223d42 mdbx: fix MDBX_NODUPDATA handling for key-value data. 
More for https://github.com/erthink/libmdbx/issues/99
in continue to 9428efd2ae9f7fe984422b5eb1fc47f732236b51

Change-Id: I56050ddf5e56807a11d8613780c3ba76cc16d324
 2020-05-15 11:30:35 +03:00
Leonid Yuriev
8c8c86038a mdbx: fix unsigned-to-ptrdiff casting inside estimate(). 
Fixup after a238179c23e7378b2019de5229b2166cdfe515c7

Change-Id: I316e4aa1f41462fd0017f8e0a7f6d6e82aa4f3db
 2020-05-15 11:30:35 +03:00
Leonid Yuriev
e6b48b613a mdbx: more checking for MDBX_DEBUG. 
Change-Id: Ie6bedda942bc5623e5d7cee5d2011e41009ea3d9
 2020-05-15 11:30:35 +03:00
Leonid Yuriev
f6fe3b44de mdbx: fix fetching of first/lower key in case starting from LEAF2-page during page merge. 
More for b3f375c57f9ff0f1f58e77227374b1cfddcd3278
 2020-05-15 11:30:35 +03:00
Leonid Yuriev
57d76d1f7a mdbx: checking the order of keys on all pages during the audit (squashed). 2020-05-15 11:29:23 +03:00
Leonid Yuriev
30bb2363e1 mdbx: use comparators instead of is_samedata(). 2020-05-15 08:59:10 +03:00
Leonid Yuriev
b7fb9bdeb1 mdbx: fix mdbx_xcursor_init1() for prev created databases. 2020-05-15 08:59:10 +03:00
Leonid Yuriev
d986d09b7b mdbx: rework min/max length checking for keys & values (squashed). 2020-05-15 08:59:03 +03:00
Leonid Yuriev
9736a36efb mdbx: add branch-less flags_db2sub(). 2020-05-11 02:03:55 +03:00
Leonid Yuriev
e4ce115876 mdbx: fix MSVC warnings (minor). 
Change-Id: I614ee9ec1c21586fbe08c6cedb395ac57a81914f
 2020-05-10 03:56:25 +03:00
Leonid Yuriev
a238179c23 mdbx: rename/reorganize internal fields for clarity. 
Change-Id: I9b23a8ca271f30ebedda9cd8d21094b4797f8bb3
 2020-05-09 20:00:31 +03:00
Leonid Yuriev
8ee3c6ddb0 mdbx: minor fix return MDBX_EBADSIGN. 
Change-Id: Ia0da8b4c2a5f0776688c92ad50433eed5e83b73c
 2020-05-09 20:00:25 +03:00
Leonid Yuriev
9428efd2ae mdbx: avoid upsertion (early exit) the same key-value data. 
Resolve https://github.com/erthink/libmdbx/issues/99

Change-Id: I6aac66d6cb6818951e29bc8a3626c342d6137151
 2020-05-09 03:25:20 +03:00
Leonid Yuriev
db635c9fbb mdbx: fix/don't clean env-signature if open failed. 2020-05-09 02:48:56 +03:00
Leonid Yuriev
83cf4cd3d5 mdbx: refine/fix mdbx_dbi_open_ex() to safe concurrently opening the same handle from difference threads. 
Change-Id: Ib01b6015bbb28eb2777f9206d6a5ac2cf9c2575f
 2020-05-09 02:46:35 +03:00
Leonid Yuriev
f11607dfa6 mdbx: minor fix/workaround to avoid UBSAN traps for memcpy(ptr, nullptr, 0). 2020-05-03 01:48:12 +03:00
Leonid Yuriev
175c361018 mdbx: avoid gcc-anylyzer false-positive warnings. 2020-05-02 22:38:53 +03:00
Leonid Yuriev
b943466727 mdbx: fix missing comma in err-msg array (Coverity). 
Change-Id: Ic441e63cf8ed14108dfe7c38e269f819879e21ce
 2020-05-01 02:42:28 +03:00
Leonid Yuriev
b3f375c57f mdbx: fix branch-page merge. 
Change-Id: I48dce339d1735cd8d270f1ba1c091572daa897a6
 2020-04-30 16:57:29 +03:00
Leonid Yuriev
eec465704d mdbx: fix div-by-zero while copy-with-compaction for non-resizeable DB. 
Change-Id: Ice7f5675c3b2864195874f2e3c4ade9148cad048
 2020-04-29 18:57:00 +03:00
Leonid Yuriev
fac0d47913 mdbx: add MDBX_CP_FORCE_RESIZEABLE option. 
Change-Id: I5fcdae7933dbbfbf8629e8a0423545a4f6b9f6b6
 2020-04-29 18:57:00 +03:00
Leonid Yuriev
b44520b7d1 mdbx: don't reset last txnid while copy with compactification. 
Change-Id: Ia7c1b968ab6a21a3c178b45408088c84647276b1
 2020-04-29 18:37:37 +03:00
Leonid Yuriev
eea1432e80 mdbx: auto-reset running transaction in mdbx_txn_renew(). 
Change-Id: If93ad13bb5a6dab8dd5fcb80882b5eb83fdf3dad
 2020-04-26 03:32:50 +03:00