ISAR/libmdbx
3
0
Fork 0
mirror of https://github.com/isar/libmdbx.git synced 2024-10-31 11:50:27 +08:00
Code Issues Packages Projects Releases Wiki Activity
360 Commits 5 Branches 68 Tags 11 MiB
9775b409bc
Commit Graph

231 Commits

Author SHA1 Message Date
Leo Yuriev
2a6bfa56f0 mdbx: backport - more for cursor tracking. 
Includes:

 - ITS#8321 deinit empty cursors
   Always unset C_INIT flag if the cursor's target DB has been deleted

 - ITS#8321 Fix mdb_cursor_set
   Always reinit mc_pg[0] if cursor is not C_INITIALIZED
   It might have a stale value when using nested txns

 - ITS#8321 mdb_put cursor needs tracking too

 - ITS#8321 page_touch - don't fixup the cursor we just touched

 - ITS#8321 More cursor fixup
   Based on page_touch fixup from ITS#7594 but expanded:
   make sure sub-cursors agree with main cursors.

 - ITS#8321 cleanup unused var

Change-Id: I4b825e20e9d42d3166052e9b3e5bd0ac33b70e85

ITS#8321 fix ambiguity in cursor_put fixup

After delete/add of a node, other nodes may no longer be
pointing at the data they intended. This can confuse subsequent
fixups.

Change-Id: Id43406a9828b440735835b024e94e9b2d5a17693
 2015-11-25 19:08:14 +03:00
Howard Chu
3eb38d2f1b mdbx: backport - ITS#8323 Fix nested commit. 
Must remove our spilled pages from parent's dirty list

Change-Id: Ic37bcea4975801e48832968ea38050064a581d21
 2015-11-25 18:39:31 +03:00
Hallvard Furuseth
3e74ab2aee mdbx: backport - CURSOR_TMP_[UN]TRACK() -> WITH_CURSOR_TRACKING(). 
Change-Id: I008b7db829765889fd63900b802f57eca0c6cd20
 2015-11-25 18:09:11 +03:00
Leo Yuriev
aeea7ebb08 mdbx: backport - ITS#8321 a lot for cursor tracking. 
Multiple bugs were fixed in the cursor fixups which
adjust other open cursors in response to various write ops.

Includes:

 - ITS#8321 Fix del/dupsort.
   When deleting a dupsort key, if other cursors pointed at that key,
   set them to uninit'd, not EOF. They no longer have anything to
   point at.

 - ITS#8321 don't skip fixups on splitting cursors.
   Adjustments can't be skipped, in recursive calls each level must
   fixup their own level.

 - ITS#8321 fix mdb_cursor_chk().
   It was reporting spurious errors due to uninit'd cursors

 - ITS#8321 fix mdb_cursor_shadow().
   Set a valid txn so that cursor fixup code works on the shadows

 - ITS#8321 fix mdb_cursor_put.
   Ignore sub-cursors that shouldn't be fixed up

 - ITS#8321 track temporary cursors.
   In rebalance/split operations, temporary cursors need to be visible
   to propagate fixups

 - ITS#8321 simplify page_split fixups.

 - ITS#8321 reorganize page_split fixups.
   DUPFIXED fixups needed to occur after separator update.
   MDB_RESERVE handling moved after split fixup.

Change-Id: I0c04acf54ebf6e84f32996b5723ec6fafb983ad9
 2015-11-23 10:09:20 +03:00
Howard Chu
0a97fbcbab mdbx: backport - ITS#8300 more for prev commit. 
Just tell explicitly which direction we moved/merged from

Change-Id: Ib1868003d30f3afe71f105c2750253bbc6059610
 2015-11-23 10:07:49 +03:00
Howard Chu
2154b585c6 mdbx: backport - Silence some valgrind uninit warnings. 
Change-Id: Ie26d7bd08a3a4d4ceb833093a9117f1ebaca6cb0
 2015-11-23 10:07:49 +03:00
Howard Chu
90fdef7a6a mdbx: backport - Refix root split check from 5da67968afb599697d7557c13b65fb961ec408dd. 
Change-Id: If012ddcf223a3312bbe13c4b24d776b488ed3772
 2015-11-23 10:07:49 +03:00
Howard Chu
6ba0b8b467 mdbx: backport - ITS#8316 cursor fixup in page_merge. 
Deleting the merged page requires bumping down other ki's
of the page's parent.

Change-Id: Ifc94e26b8755ede2400521556556a6824bc2c62b
 2015-11-23 10:07:49 +03:00
Leo Yuriev
bf81e86d93 mdbx: prevents use the env after a fork(). 
Change-Id: I9c86500ac008a7be7eb16aeff2610fa5cfa22b84
 2015-11-22 17:48:30 +03:00
Howard Chu
7e2000ef7a mdbx: backport - ITS#8315 fix ovpage_free. 
Keep dirty_room sync'd with dirty_list

Change-Id: I9e52a72df95ffb504740e8daecf65b62970e9f25
 2015-11-19 20:44:49 +03:00
Howard Chu
12cd2361e7 mdbx: backport - ITS#8313 more for ITS#8062. 
dummy flags must be init'd due to 3d46d550

Change-Id: I3d543bd3e059da9b007f1e752f20171acc183679
 2015-11-19 20:44:44 +03:00
Hallvard Furuseth
86abc397e8 mdbx: backport - ITS#8312 Fix loose pages in commit(nested txn). 
Change-Id: Ie335dcff0c87cfc13abf9b937f5d058d3ea9d841
 2015-11-19 20:44:13 +03:00
Leo Yuriev
0230e4fda2 mdbx: clarify fsync/fdatasync comment. 
Change-Id: I2209b5e65ca8c04ff2de9b1b789f1993f24ab454
 2015-11-19 19:05:02 +03:00
Howard Chu
02da85169e mdbx: backport - ITS#8311 fix page_split from update_key. 
Check for top of stack. Usually the cursor only has height 1 when
calling page_split, but not always.

Change-Id: Iad221be30edac0f82b650f787e5dbe721cc978e0
 2015-11-19 19:05:02 +03:00
Leo Yuriev
4bdeed9bd3 mdbx: fix thread-local-storage memleak. 
This fix a TLS-memleak for thread from which mdbx_env_close_ex()
has been called. Bug was added by while fixing the
https://github.com/ReOpen/ReOpenLDAP/issues/48

In general we should explicitly free(), because
pthread_key_delete() don't calls a destructor.

Change-Id: Ic55a2348caf3be34b4331d5ad101ea33dbbdfa97
 2015-11-19 19:05:02 +03:00
Leo Yuriev
cee258fe86 mdbx: fix uninit fields in page_malloc(). 
Change-Id: I35a162d6b391d33eda4d508e9c1af7238b33665d
 2015-11-19 19:05:02 +03:00
Leo Yuriev
34d95c08eb mdbx: refines robust mutex usage for glibc. 
Change-Id: I986d10999121747b05ea132dde20b6daccc2e080
 2015-11-14 13:27:15 +03:00
Howard Chu
380c61bcf7 mdbx: backport - ITS#8310 fix mdb_rebalance cursor fixup. 
When collapsing root, must adjust entire cursor stack

Change-Id: If62c42f556311e785aa414954a3d7fddc077cf62
 2015-11-14 13:07:19 +03:00
Howard Chu
ab0b0ec7ce mdbx: backport - ITS#8304 fix page_merge (mdb_del cursor tracking). 
Was using the pre-touch dst page pointer instead of the touched page

Change-Id: Ib41af356dcf883b923dbfbbc7591a02483ee54dd
 2015-11-10 09:12:33 +03:00
Howard Chu
ee26517b5d mdbx: backpost - ITS#8300 fix rebalance after node_move. 
ITS#8258, ITS#7829 fixes checked parent index to see if we were moving
from a left neighbor. Should have just checked to see if current index
was 0, meaning we added on the left. (Parent index may not tell us
anything meaningful after a nested rebalance.)

Includes:

 - ITS#8300 fix node_move
   Don't adjust other cursors when we added a node on the right.

 - ITS#8300 more for node_move fixups
   When moving a node from the right neighbor, a different
   adjustment is needed.

 - ITS#8300 simplify

 - ITS#8300 more for node_move
   When moving a node from one page to another, make sure other cursors'
   parent index gets adjusted too.
 2015-11-06 17:46:51 +03:00
Howard Chu
7890eb8a8e mdbx: backport - ITS#8238 fix DUPFIXED page_split. 
Parent mc_ki wasn't adjusted if new_indx was > split point

Change-Id: I45548e378e53bad5ce7a3a7c2b8236d592f3c412
 2015-11-06 17:46:31 +03:00
Howard Chu
f35fb03f86 mdbx: backport - ITS#8299 fix page_merge cursor fixup. 
The parent's mc_ki has changed. We need to fix that up as well
in other cursors.

Change-Id: I1f422a07ed75b99f3e36075749800c892ca6d5ec
 2015-11-06 17:46:31 +03:00
Leo Yuriev
b430c9a22f mdbx: Add MDB_USE_ROBUST to control Robust Mutexes. 
Backported from origin.

Change-Id: I416c1d09fb1f290423f29a84831accdaf4436ab0
 2015-11-06 17:26:41 +03:00
Leo Yuriev
a542ceeae5 Merge branch 'master' into devel 2015-11-06 16:27:59 +03:00
Leo Yuriev
c6e010a872 mdbx: fix segfault on close when open failed. 2015-10-31 13:56:12 +03:00
Leo Yuriev
60387be4a5 Merge branch 'master' into devel 2015-10-21 00:24:44 +03:00
Leo Yuriev
dc3256e91c mdbx: adds signatures to detect ABI mixup. 2015-10-20 23:59:06 +03:00
Howard Chu
ad808146c8 lmdb: ITS#7771 fix cursor tracking on fake pages. 
node_del shifts nodes around, cursors pointing at fake pages
need to have their mc_pg[0] corrected.

Includes ITS#7771 more for prev commit.
 2015-10-19 16:07:03 +03:00
Howard Chu
f6045ae77e lmdb: ITS#8263 fix cursor tracking in cursor_put. 
Includes ITS#8263 streamline prev patch.
 2015-10-19 16:06:28 +03:00
Leo Yuriev
68171d5f5d lmdb: provide both interfaces - advanced 'mdbx' and original 'lmdb'. 2015-10-13 23:50:25 +03:00
Leo Yuriev
09c140c1f4 lmdb: detach from ReOpenLDAP project. 2015-10-12 20:01:16 +03:00
Howard Chu
73cfae2522 lmdb: ITS#8264 fix cursor_del cursor tracking. 
Some destination fixups need to happen immediately after nodes
are moved, before rebalancing

Change-Id: Ia4258cae368ff0ef96b7835cc421f2db40f7f741
 2015-10-11 22:21:37 +03:00
Howard Chu
f2d0824bdb lmdb: ITS#8258 fix rebalance/split. 
The tree height can also increase during rebalance, not just shrink.
This can happen if update_key needs to split a parent branch page.

Change-Id: Ie232bd9f4a6225b912b912a68f0b35411a632317
 2015-10-06 10:13:27 +03:00
Hallvard Furuseth
305ba98967 lmdb: Fix typo in mdb_tassert(). 
No effect on current code.

Change-Id: Ia935ba784c33e3704ea1cfa1d59a1f28d2c527f8
 2015-10-06 10:11:18 +03:00
Howard Chu
ef72fbaea6 lmdb: ITS#8221 don't merge branch pages needlessly. 
Change-Id: I6b535f70392e081027d8803b6623b0c51f21ec8b
 2015-10-06 10:05:46 +03:00
Leo Yuriev
8e2a9a9a78 lmdb: deferred cleanup of reader's threads. 
This should fix https://github.com/ReOpen/ReOpenLDAP/issues/48

Change-Id: I67feb3a9852f183dcbe83626321896ba43a591b7
 2015-09-07 02:38:10 +03:00
Howard Chu
83ba70e3be lmdb: ITS#8237 fix ITS#7589 regression. 
Change-Id: Idd3c95f6e515f0f70c7fd33257c80354a3953f2f
 2015-09-06 22:57:01 +03:00
Leo Yuriev
4554b8f62c all: fix build by clang (defines, warnings). 
Conflicts:
	contrib/slapd-modules/usn/usn.c

Change-Id: I68572808e30a35fdd14aa0a459cfbb8da4be6dc5
 2015-09-05 18:35:32 +03:00
Leo Yuriev
16a30be8fc lmdb: minor refine mdb_chk output, 
mostly for 'zero size-of-entry' error.

Change-Id: I0f7ae514860ea05179fd929b76f78ba843902e92
 2015-09-03 17:26:23 +03:00
Leo Yuriev
07ee471b1c lmdb: me_maxkey_limit always computed and checked against MDB_MAXKEYSIZE at opening env. 
Change-Id: Ice0a425900a8c7b37d5b6cfcd1434edc8d400a74
 2015-09-02 18:08:43 +03:00
Leo Yuriev
f7d8812e6c lmdb: n-entries info from b-tree traversal in mdb_chk. 
Change-Id: Ic9343dcdba976ac51c8f5776d2f7c9fed59da8e0
 2015-09-02 17:53:48 +03:00
Leo Yuriev
15e0600b6c lmdb: major rework of traversal b-tree for mdb_chk. 
Change-Id: I9d382516f76092f44fc1a12d7554039582b87656
 2015-09-02 16:19:12 +03:00
Leo Yuriev
e7f58da2ac lmdb: support for a utterly no-sync mode, 
by combination of MDB_NOSYNC and MDB_MAPASYNC.

Change-Id: I3e2d6c8f044a7522436dab4d0818d4d59b78b78d
 2015-09-01 12:15:59 +03:00
Hallvard Furuseth
e4d9f0a905 lmdb: ITS#7377 catch calloc error in mdb_env_init_meta(). 
Change-Id: I2b1a4ec9f86a3a7844b157863d18ce81be400cef
 2015-08-30 01:02:43 +03:00
Leo Yuriev
6a3efb82eb lmdb-test: workaround for MDB_FIXEDMAP to avoid sporadic failures. 
Change-Id: I6ba993ff50956d328fd76326c67aa83dff20ce48
 2015-08-24 03:37:05 +03:00
Leo Yuriev
537fc0fe1a lmdb: simple write benchmark. 
Change-Id: Iadcbe8ad5922b2ecf1ea597b188ef368bc097185
 2015-08-24 03:36:55 +03:00
Leo Yuriev
00d2057204 lmdb: fix rare MDB_MAP_FULL when pages are available in the gc. 
Rework preparation backlog of available pages for cleanup FreeDB.
https://github.com/ReOpen/ReOpenLDAP/issues/42

Change-Id: I47592b4cabf1950e61bb87e67c08ea03e95496fa
 2015-08-20 19:26:20 +03:00
Leo Yuriev
b8c45546f6 lmdb: more likely/unlikely hints. 
Change-Id: I38f1f80243bfc5b1ffe910175516dbb3f06c27ba
 2015-08-15 13:17:46 +03:00
Leo Yuriev
ad8c4a19bb lmdb: fix segfault in mdb_chk when mdb_env_open() failed. 
Change-Id: If72d35518c2d4dd33d284049e00fc633e65ee52c
 2015-08-13 18:30:03 +03:00
Leo Yuriev
96d69e3fa1 lmdb: added mdb_env_open_ex() and exclusive/monopoly mode for mdb_chk. 
Change-Id: I867e7f17924ffecb744440e95db96c0a7411d6ef
 2015-08-13 18:30:02 +03:00
Leo Yuriev
1171e4d73a lmdb: fix build non-debug mode (MDB_DEBUG=0). 
Change-Id: I362a6bd03c73dca70bc6df5f72badf59e560389a
 2015-08-06 17:13:25 +03:00
Leo Yuriev
3fc45cbaea lmdb: fix assertion failure in case error while opening DB in debug-mode. 
Change-Id: I56a1122369640920d594416e9bd3de0f3f9cd8fd
 2015-08-05 22:59:18 +03:00
Leo Yuriev
595f9e700d lmdb: fix b-tree walking bug in case empty db. 
Change-Id: I28c0985493327babe4d2ae5c6cd56116a26e6034
 2015-08-05 22:59:18 +03:00
Leo Yuriev
a243a38aa6 lmdb: adding page-space usage info to mdb_chk. 
Change-Id: I12dcdb54894a362d41612e973f983cb4e067520c
 2015-08-05 22:59:18 +03:00
Leo Yuriev
c32cf0fea4 lmdb, back-mdb: use 'long' as result to speedup comparison. 
Change-Id: I3ade0c7525f4a461a7edf5873af196e6b33c0727
 2015-08-05 14:35:09 +03:00
Leo Yuriev
b60f510355 lmdb: rework key/data comparison to avoid alignment errors. 
Change-Id: I291b50ad6f7dd18f6cbdef34028d917e10166cf1
 2015-08-05 00:45:57 +03:00
Hallvard Furuseth
c9a8595ef2 lmdb: Kill magic numbers for NUM_METAS, databases. 
Change-Id: I7ed12f081aed7eb65087fea511053157fb8f9c1e
 2015-08-04 13:57:14 +03:00
Hallvard Furuseth
061fe51e8c lmdb: ITS#7377 Catch mdb_put() to blocked/read-only txns. 
...early enough that txn state is left unchanged.

Change-Id: Ia475660f339f7b741eaaa126978f8419a19b67a0
 2015-08-04 13:57:14 +03:00
Hallvard Furuseth
d6b3c6f710 lmdb: ITS#7377 Always notice env error on txn startup. 
Move the check to the end of txn startup.  Catches env
breakage which happens while the new txn waits for a lock.

Change-Id: I074d411cd5339b5e6caa2691078ea07b0d49828e
 2015-08-04 13:57:14 +03:00
Hallvard Furuseth
11999c8c02 lmdb: Cleanup, Drop !mt_dbxs hack, use MDB_TXN_FINISHED. 
Change-Id: I23beaaf931c938ee9f3886ca1928ecbcd2fbe5ba
 2015-08-04 13:57:14 +03:00
Hallvard Furuseth
342d56977f lmdb: Catch most uses of finished/parent txns. 
* Add MDB_TXN_FINISHED, MDB_TXN_HAS_CHILD, MDB_TXN_BLOCKED.
* Clear mt_numdbs in writers, for TXN_DBI_EXIST() to catch.
  We already do in readers.

Change-Id: I4f714c0789188dfab4ce22b7d9d3a75d8b62ef6b
 2015-08-04 13:57:14 +03:00
Hallvard Furuseth
e3c8fa8eba lmdb: Move code into mdb_txn_end(). Was mdb_txn_reset0. 
Side effects:
* Clean txn up a bit even before freeing it.
* Tweak DEBUG output at txn end. Add DEBUG after commit(writer).

Change-Id: Ia9905257893923bb14cdf33b60b6387af3121f81
 2015-08-04 13:57:14 +03:00
Hallvard Furuseth
23af429b3f lmdb: ITS#8200 Fix mdb_midl_shrink() usage, return void. 
Change-Id: I506a170d434178d43f01b3e9597c4a03b3b3e7a7
 2015-08-04 13:57:14 +03:00
Hallvard Furuseth
862cfc30c4 lmdb: Simpler mdb_node_add(). 
Change-Id: I5b1f851a098aeb4b5adcd7ed55150a055cd63102
 2015-08-04 13:57:14 +03:00
Hallvard Furuseth
be27186f76 lmdb: ITS#8181 Verify that records are/aren't DBs. 
Except we don't catch the user passing F_SUBDATA to
mdb_cursor_<put/del>, like an internal LMDB call.

Change-Id: I5ebce36694045279702c74490a5a634217d4f95d
 2015-08-04 13:57:13 +03:00
Howard Chu
91a6b6de57 lmdb: ITS#8190 fix cursor EOF bug. 
Change-Id: I97556bc521e1d4d4ea9f3e43c14a8a2fe62b8231
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
96fb8eec8a lmdb: More ESECT declarations. 
Change-Id: Iaed5c6e35b1bd6e59caf8d1f52b265ec4852c84a
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
57b263693c lmdb: Add DB_USRVALID, to avoid 'dbi == FREE_DBI' tests. 
Change-Id: Ic44ce41fd2fff1abb4cd24821ece07ef30254565
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
4cb5d637d4 lmdb: Simpler flag/DBI checks and MDB_DEBUG. 
Change-Id: I37a00fcc23f2060538ab4378f204ae0efcdb59ea
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
9e52d29fd8 lmdb: Simpler mdb_drop(). 
MDB_DBI_CHANGED(,MAIN_DBI) is never true.

Change-Id: I0d12b9965d8251908ec41d1febc6b5c354a93a01
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
6524dbf3a1 lmdb: Simpler mdb_node_shrink(). 
Change-Id: If0e4153e69c4f858ee45471c4888ec2024253b50
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
a6fa632a54 lmdb: Simpler mdb_txn_commit(). 
mt_env is always set.
Commit(mt_child) resets mt_child, so parent need not.

Change-Id: Ib8af816a964a4b70fa6fa622b3d9dd463e9460b3
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
82213d732c lmdb: cleanup mdb_txn_begin(). 
Change-Id: Ia28e255a0497d02f7d87ca301d5be8a60886d4e1
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
b3a3dc4c6b lmdb: Omit scanning DUPSORT sub-DB leaves in mdb_drop0(). 
Change-Id: Icc670a81210a9160e6cb90b9442994c008f599bd
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
b13d850bd7 lmdb: Use mdb_cmp_int_a/mdb_cmp_long() for FREE_DBI. 
Change-Id: Id66356b27e9d257fbf5bc8ce1111845be8683d62
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
84b6ecb9a8 lmdb: Hide internal flags in mdb_env_get_flags(). 
Change-Id: I9ff2888a61a53fb0c7471de980ec7b97c59c6096
 2015-08-04 13:57:13 +03:00
Leo Yuriev
295beda00c lmdb: fix ABI compatibility with original liblmdb.so 
Change-Id: I770a49b7a506f3f2a21ea3af36fd07c16a32dc30
 2015-06-04 19:12:14 +03:00
Hallvard Furuseth
f71cb6d69e lmdb: ITS#8156 Fix MDB_MAXKEYSIZE doc. 
Change-Id: Ia9dae2e68f28dcb715ac34226b358212317199c2
 2015-06-01 15:38:18 +03:00
Hallvard Furuseth
55ae3dfa8b lmdb: ITS#8157 mdb_txn_renew0(): init after error checks. 
Change-Id: I9ea344b98cb8c99a25ba7087abefbfe62b3417a0
 2015-05-28 16:00:27 +03:00
Hallvard Furuseth
bb7b923fdd lmdb: Set/clear mp_pad, md_pad (MDB_DUPFIXED data size). 
mdb_xcursor_init1(): md_pad is only used when MDB_DUPFIXED.
mdb_page_split():    Copy mp_pad too. Used by mdb_page_list().

Change-Id: Ia35cba3a367c04929a34a5879abc968202086e89
 2015-05-21 21:40:39 +03:00
Hallvard Furuseth
476c5b02ea lmdb: ITS#8117 Fix MDB_INTEGERDUP keysize doc. 
Change-Id: I7cf7c456f35fe9c5449a37ad9130864617197726
 2015-05-21 21:40:39 +03:00
Leo Yuriev
104312d15e lmdb: update mdb_chk for checkpoints, minor cosmetics. 
This is 9/9 for https://github.com/ReOpen/ReOpenLDAP/issues/1
and https://github.com/ReOpen/ReOpenLDAP/issues/2

Change-Id: I05f59313b4346c08bcd63d62e6c48a5f53c47852
 2015-05-21 21:40:39 +03:00
Leo Yuriev
21705fd9a2 lmdb: seek to steady meta-page on db-open. 
This is 7/9 for https://github.com/ReOpen/ReOpenLDAP/issues/1
and https://github.com/ReOpen/ReOpenLDAP/issues/2

Change-Id: If59a6bfc7c6198e6b1e85f4bdc5b534ecff03123
 2015-05-21 21:40:39 +03:00
Leo Yuriev
633f2a10ad lmdb: forces a steady checkpoint if OOM occurred 
because of reclaiming stops on a weak ones.

This is 6/9 for https://github.com/ReOpen/ReOpenLDAP/issues/1
and https://github.com/ReOpen/ReOpenLDAP/issues/2

Change-Id: I91d8a3dfaf06e1917326d8f73c4a65565cde0b47
 2015-05-21 21:40:39 +03:00
Leo Yuriev
312135169f lmdb: weak/steady for meta-pages. 
This is 5/9 for https://github.com/ReOpen/ReOpenLDAP/issues/1
and https://github.com/ReOpen/ReOpenLDAP/issues/2

Change-Id: Ica2dbe0bfd6ba58c00de161e2cd50594ee39c44d
 2015-05-21 21:40:39 +03:00
Leo Yuriev
a283d782f6 lmdb: bigbang for sync-to-disk path and related. 
This is 4/9 for https://github.com/ReOpen/ReOpenLDAP/issues/1
and https://github.com/ReOpen/ReOpenLDAP/issues/2

Here a lot of changes:
 - deleted a secondary DSYNC-mode file descriptor;
 - deleted a pointers to meta-pages and toggle-selection of ones;
 - removed MDB_FDATASYNC_WORKS/FDATASYNC_MAYBE_BROKEN stuff;
 - reworked use of fdatasync/fsync for safety without performance degradation;
 - txn-to-meta updating moved info mdb_txn_commit();
 - removed mdb_env_write_meta();
 - rewrited mdb_env_sync0() for clarify;

Change-Id: I985464baa3cf486a2ceccf2c9dcb7a7fea698c46
 2015-05-21 21:40:24 +03:00
Leo Yuriev
1c37139b2a lmdb: drops no-lock operation mode (now MDB_NOLOCK is UNSUPPORTED). 
This is 3/9 for https://github.com/ReOpen/ReOpenLDAP/issues/1
and https://github.com/ReOpen/ReOpenLDAP/issues/2

Change-Id: I7cd5d90c41424d6635accbb10c5801adeb1087e9
 2015-05-21 20:23:12 +03:00
Leo Yuriev
eec1ccaa77 lmdb: drops support for non-POSIX. 
This is 2/9 for https://github.com/ReOpen/ReOpenLDAP/issues/1
and https://github.com/ReOpen/ReOpenLDAP/issues/2

Change-Id: I7ab3d5416381c9477edba4ce9db2f3fdc58a2d3d
 2015-05-21 18:43:00 +03:00
Leo Yuriev
c5db4043ee lmdb: prepare to bigbang (separated minor changes). 
This is 1/9 for https://github.com/ReOpen/ReOpenLDAP/issues/1
and https://github.com/ReOpen/ReOpenLDAP/issues/2

Change-Id: I3ba943b2e4597f3e796376a5a54851aa411b1aa1
 2015-05-21 18:05:00 +03:00
Leo Yuriev
548d9d4b03 lmdb: bugfix, OOM-handler couldn't help if LIFO-reclaiming disabled. 
This is fixes https://github.com/ReOpen/ReOpenLDAP/issues/28

Change-Id: Ifc33e1202ae38f0a4fa0cb9d572394a4dac7ec6c
 2015-05-15 01:31:22 +03:00
Leo Yuriev
bd05dd24ab lmdb: fix mdb_chk for case MainDB's records isn't a sub-db. 
Change-Id: I3a4ae1f50972fde6edab828b58199dde9b84550c
 2015-05-09 01:00:27 +03:00
Leo Yuriev
f2703156f0 lmdb: mdb: b-tree walk, page-map check in mdb_chk. 
Change-Id: I6678b4d891c8fbfbc49ed600212f4ade39e25282
 2015-05-08 15:49:33 +03:00
Leo Yuriev
23720958b6 lmdb: mdb_chk tool (initial). 
Change-Id: I7d31e0fb45ab56cf8d6df56939e5b3ded6b8e3e5
 2015-05-08 03:47:17 +03:00
Leo Yuriev
4a04e59dba lmdb: refork debug. 
Change-Id: I47d3e300b47fcba5600644ff28c68a900640c39c
 2015-05-08 03:45:30 +03:00
Howard Chu
0ac9ba6472 lmdb: ITS#8117 better fix. 
LY: This is not required for ReOpenLDAP, because it
is not such issues related to int/long in cmp/dcmp.
But let be more close to origin's sources.

Origin: Don't change mc_dbx because we would need
to undo the change if the txn aborts. Make the fix
(for get) match existing code for put.

Change-Id: I36f7ff99b4adf821cace5af6b0b84f5210eb454d
 2015-05-07 17:15:14 +03:00
Leo Yuriev
59d6560fc3 lmdb: cleanup Windows support. 
Change-Id: I7bc7f3e997eeea4c5678a97e2faf10525ba7eee6
 2015-05-04 03:48:22 +03:00
Leo Yuriev
f97cc739cb lmdb: ITS#8117 - reworks of comparison for MDB_INTEGERKEY and MDB_INTEGERDUP. 
Change-Id: Icd24589c7a139c4395de1e7c40f16d2c7ea54cf7
 2015-05-01 06:50:04 +03:00
Howard Chu
d4bddde7aa lmdb: ITS#8109 fix mdb_cursor_del0 on empty DB. 
Change-Id: Idba0acfd64b597f2bc5cb847ca99fe4560eed875
 2015-04-24 22:39:21 +03:00
Hallvard Furuseth
8103d0b458 lmdb: Clear error from previous txn on renew. 
Change-Id: I45e608d4d1ad3ffa3ad20e8a6b579708f25c3966
 2015-04-19 23:44:07 +03:00
Leo Yuriev
38789b6c0a lmdb: refine sync/async writes of meta-pages. 
Change-Id: If309023214e60f349762d337f98ee6fd515faa31
 2015-04-18 17:18:50 +03:00
Howard Chu
347c1d6dfb lmdb: ITS#8106 retry writes on EINTR. 
Change-Id: Ib422f62f6e1fc81cd773ca600524a53f8f16629e
 2015-04-18 16:44:17 +03:00
Howard Chu
d0b4943352 lmdb: ITS#8062 - fix rebalance, also handle subcursors. 
(Probably fixes the ITS, definitely fixes a bug) when collapsing
the root page, fixups of other cursors was incomplete.
 2015-04-16 13:38:42 +03:00
Hallvard Furuseth
5ac6404059 lmdb: Cleanup MDB_env.me_txn0. 
More fallout from 4d02c741b120786df1b87ee9ed49c1d3f9bc7522.

Change-Id: I3bbd913c7da3e45f00f1d40c0aac6e7bf5f48c69
 2015-04-09 13:25:02 +03:00
Hallvard Furuseth
dba429d608 lmdb: Fix per-txn MDB_NOMETASYNC, MDB_NOSYNC. 
Fallout from 4d02c741b120786df1b87ee9ed49c1d3f9bc7522. The flags
were ignored: mdb_txn_renew0/begin cleared and/or did not set them.

Change-Id: Ic7dab1e9a4cf3754e05a3c3df6a10e0efa48af5c
 2015-04-09 13:23:59 +03:00
Hallvard Furuseth
c4097ec898 lmdb: Renumber MDB_TXN_* flags, simplify flag code. 
Change-Id: I91b0836a84ce80049dbe0829dfc3101fae94c7c9
 2015-04-09 13:22:49 +03:00
Hallvard Furuseth
5daf811dcb lmdb: Clear error from previous txn in mdb_txn_renew(). 
Change-Id: I72b55bfd91c87e4608fbf50f3c70951952855f8f
 2015-04-09 13:22:16 +03:00
Leo Yuriev
4a1574c998 all: refine #defines that controls memory checking. 
LDAP_DISABLE_MEMORY_CHECK removed, LDAP_MEMORY_DEBUG is used instead.
LDAP_MEMORY_DEBUG_ASSERT replaced with LDAP_MEMORY_ASSERT.

Change-Id: I0f43b9ae10b76c1af39f2c52b34b4ab804521e78
 2015-03-26 01:04:14 +03:00
Leo Yuriev
602fdd16b9 lmdb: define MDB_INLINE macro instead of __inline. 
Change-Id: I3a2f47138a80b09f8c3682a9a26dbf5aca3445f7
 2015-02-24 15:14:01 +03:00
Quanah Gibson-Mount
3766484c9e all: 2015 copyright timestamp (from OpenLDAP). 
Change-Id: I76f457e30a0478bfc0246c760a2302d380d6f181
 2015-02-18 16:04:11 +03:00
Leo Yuriev
ddbda992c7 lmdb: refines Valgrind usage. 
Change-Id: I5f30a609b9053bbf8a693b7ca64dc57dc176f6e9
 2015-01-18 06:57:49 +03:00
Leo Yuriev
b6b2609f9e lmdb: minor refine (rename md_pad => md_psize). 
Change-Id: I60f95d171abebce5967c9bb398c21dbb01ca3d36
 2015-01-18 06:57:49 +03:00
Leo Yuriev
48e85e891e lmdb: use pwritev() on non-windows systems (MDB_USE_PWRITEV). 
Change-Id: Iebd69846fc87cd6bb584002b366837ecb631b603
 2015-01-18 06:57:49 +03:00
Leo Yuriev
1ca2b54966 lmdb: more coherent-barriers where reader's pid updated. 
Change-Id: I6262021c074a99e2f548c1fda94fd53c584d2b89
 2015-01-17 17:20:33 +03:00
Hallvard Furuseth
ae747bb4ce lmdb: ITS#7971 Fix reader allocation and me_numreaders. 
Change-Id: I75c8325de342c2a28592fb0c7a718f95855e7e4d
 2015-01-17 16:58:32 +03:00
Hallvard Furuseth
62181cfe87 lmdb: Reject CHANGELESS flags in mdb_env_set_flags(). 
Reverts part of a2ac10107e2fb845c4a38a339239063ec4407d84.
MDB_ROBUST needed to be accepted, but that flag is gone.

Change-Id: I2aba1e9e02da716d4509e465767bf8be05895c29
 2015-01-16 01:26:50 +03:00
Leo Yuriev
1f58fb66ae lmdb: ITS#7969 properly compiler/memory/coherent barriers. 
Change-Id: I875d0908a2c828626df544fed48b3144e95e3982
 2015-01-16 01:26:50 +03:00
Howard Chu
e7a0c74c09 lmdb: Fix potential null deref (coverity). 
Change-Id: Icc5cadfddecbea81c835d84e4ada785e34638514
 2015-01-16 01:26:50 +03:00
Hallvard Furuseth
eb885ab910 lmdb: Make SysV semaphores robust. Cleanup MDB_ROBUST. 
Backport from master: 66e3f5139105822196aea6a6e63596d25734222a.

Change-Id: Idda165be19ee95c2a0839a180a2ece395502ce26
 2015-01-16 01:26:49 +03:00
Leo Yuriev
3015e8d03e lmdb: account pages from penging txn in mdb_txn_straggler(). 
Backport from master: f756a26a43ad933a98909a974ae84f7d43c30266.

Change-Id: I0f8af7af722977ddeff88d2e8129bcde23ffd06f
 2015-01-16 01:26:49 +03:00
Howard Chu
13b144dc5d ITS#8021 env_sync is invalid in RDONLY env 
Change-Id: I2c9c8bcf4b28ea002cb5e4caa117396a82289924
 2015-01-16 00:25:18 +03:00
Leo Yuriev
3fd4f9cce0 lmdb: rework workaround for potential ext3/ext4 corruption issue. 
Reworked from branch 'mdb.master' origin OpenLDAP:
  8b6c425 2015-01-12 More cleanup for fdatasync hack
  ea89e3d 2015-01-11 Tweak conditionals for fdatasync hack
  462dc09 2015-01-08 fdatasync hack, again
  e86072a 2015-01-08 Revert "Note MDB_SAFE_FDATASYNC"
  293d6bb 2015-01-08 Note MDB_SAFE_FDATASYNC
  9585c01 2015-01-08 Simpler fdatasync hack
  0ef1e0b 2015-01-08 Revert "Fix prev commit for env_sync0"

Imported early while forking ReOpenLDAP:
  985bbbb 2014-12-21 Fix prev commit for env_sync0
  0018eeb 2014-12-18 Hack for potential ext3/ext4 corruption issue

Change-Id: I187fd320620b9ced2e3773cac96f281ff65f97d4
 2015-01-16 00:24:31 +03:00
Leo Yuriev
f00d2cdef6 lmdb: MDB_LIFORECLAIM & MDB_COALESCE modes. 
Reclaim FreeDB in LIFO order - this is a main feature.
Also aim to coalesce small FreeDFB records.

Change-Id: I76aa062ef59359616e5b697233ec47cbd14f43cd
 2015-01-15 23:19:52 +03:00
Leo Yuriev
9eedc88441 lmdb: ITS#7974 oom-handler feature. 
Change-Id: I2c56e003fa1e4abe934288581e4c52c80db27c08
 2015-01-15 23:19:52 +03:00
Leo Yuriev
84ce8b4cd8 lmdb: properly sync meta-pages in mdb_sync_env(). 
Meta-pages may be updated during data-syncing in mdb_sync_env(),
in this case database would be inconsistent.

Check-and-retry if lead txn-id changed during flushing data in mdb_sync_env().

Change-Id: I806e6d3766d028dcd4e6b0591aabc718107056fb
 2015-01-15 23:19:52 +03:00
Leo Yuriev
fcfe2b7042 lmdb: auto-sync with kbytes threshold (aka checkpoint by kbytes). 
Change-Id: If29a3d70e775a65d832f578f0c3edd3ea41dcc67
 2015-01-15 23:19:52 +03:00
Leo Yuriev
c9489da920 lmdb: lock meta-pages in writemap-mode to avoid unordered write. 
Without locking the meta-pages may be writen by OS before data pages,
in this case database would be inconsistent.

Change-Id: I6a262709b0741d7b57cb68231c4ae1c62a10a743
 2015-01-15 23:19:51 +03:00
Leo Yuriev
2df38ad58b lmdb: ITS#7974 get a lag of the reading for dreamcatcher. 
Change-Id: I7fc93ce4fa851904521a58a225f04c65eca2d998
 2015-01-15 23:19:51 +03:00
Leo Yuriev
7984f8817f lmdb: more usefull info from mdb_stat tool. 
Change-Id: I64015952431aebabf792b912acd022a6ddc79903
 2015-01-15 23:19:51 +03:00
Leo Yuriev
1b8398b413 lmdb: Use last page (off-by-one error). 
Change-Id: Ie1fd90d1562ab8ea8e53701188e3a17495a697e1
 2015-01-15 23:19:51 +03:00
Leo Yuriev
1b5de44e23 fix: uninitialized (warnings). 
Change-Id: I0f5eab80546a602a616209c6b585605826a06371
 2015-01-15 23:19:51 +03:00
Leo Yuriev
83055127c1 OpenLDAP mdb.master at 985bbbbdd5d64e57f55249ffdeb7c08035b240b2 (2015-01-01). 2015-01-07 18:12:57 +03:00