Commit Graph

2787 Commits

Author SHA1 Message Date
Orivej Desh
5d4a23138c mdbx: backport - ITS#8320 mdb_load: fix loading data from simple text files.
mdb_load -T was supposed to read escaped text,
but 21b51cb7 "Add mdb_load" made it read hex.

Change-Id: If12a01ee897af0570d95744626815c535cc81f9f
2015-11-23 10:13:05 +03:00
Orivej Desh
a8e2288537 mdbx: backport - ITS#8319 mdb_load: explain readline and mdb_cursor_put errors.
Change-Id: I99e8ff220bb37109da83bb6088be7597b52c6f00
2015-11-23 10:13:05 +03:00
Leo Yuriev
aeea7ebb08 mdbx: backport - ITS#8321 a lot for cursor tracking.
Multiple bugs were fixed in the cursor fixups which
adjust other open cursors in response to various write ops.

Includes:

 - ITS#8321 Fix del/dupsort.
   When deleting a dupsort key, if other cursors pointed at that key,
   set them to uninit'd, not EOF. They no longer have anything to
   point at.

 - ITS#8321 don't skip fixups on splitting cursors.
   Adjustments can't be skipped, in recursive calls each level must
   fixup their own level.

 - ITS#8321 fix mdb_cursor_chk().
   It was reporting spurious errors due to uninit'd cursors

 - ITS#8321 fix mdb_cursor_shadow().
   Set a valid txn so that cursor fixup code works on the shadows

 - ITS#8321 fix mdb_cursor_put.
   Ignore sub-cursors that shouldn't be fixed up

 - ITS#8321 track temporary cursors.
   In rebalance/split operations, temporary cursors need to be visible
   to propagate fixups

 - ITS#8321 simplify page_split fixups.

 - ITS#8321 reorganize page_split fixups.
   DUPFIXED fixups needed to occur after separator update.
   MDB_RESERVE handling moved after split fixup.

Change-Id: I0c04acf54ebf6e84f32996b5723ec6fafb983ad9
2015-11-23 10:09:20 +03:00
Howard Chu
0a97fbcbab mdbx: backport - ITS#8300 more for prev commit.
Just tell explicitly which direction we moved/merged from

Change-Id: Ib1868003d30f3afe71f105c2750253bbc6059610
2015-11-23 10:07:49 +03:00
Howard Chu
2154b585c6 mdbx: backport - Silence some valgrind uninit warnings.
Change-Id: Ie26d7bd08a3a4d4ceb833093a9117f1ebaca6cb0
2015-11-23 10:07:49 +03:00
Howard Chu
90fdef7a6a mdbx: backport - Refix root split check from 5da67968afb599697d7557c13b65fb961ec408dd.
Change-Id: If012ddcf223a3312bbe13c4b24d776b488ed3772
2015-11-23 10:07:49 +03:00
Howard Chu
6ba0b8b467 mdbx: backport - ITS#8316 cursor fixup in page_merge.
Deleting the merged page requires bumping down other ki's
of the page's parent.

Change-Id: Ifc94e26b8755ede2400521556556a6824bc2c62b
2015-11-23 10:07:49 +03:00
Leo Yuriev
bf81e86d93 mdbx: prevents use the env after a fork().
Change-Id: I9c86500ac008a7be7eb16aeff2610fa5cfa22b84
2015-11-22 17:48:30 +03:00
Howard Chu
7e2000ef7a mdbx: backport - ITS#8315 fix ovpage_free.
Keep dirty_room sync'd with dirty_list

Change-Id: I9e52a72df95ffb504740e8daecf65b62970e9f25
2015-11-19 20:44:49 +03:00
Howard Chu
12cd2361e7 mdbx: backport - ITS#8313 more for ITS#8062.
dummy flags must be init'd due to 3d46d550

Change-Id: I3d543bd3e059da9b007f1e752f20171acc183679
2015-11-19 20:44:44 +03:00
Hallvard Furuseth
86abc397e8 mdbx: backport - ITS#8312 Fix loose pages in commit(nested txn).
Change-Id: Ie335dcff0c87cfc13abf9b937f5d058d3ea9d841
2015-11-19 20:44:13 +03:00
Leo Yuriev
f3043badc5 mdbx: tools/tests - fix minor memleaks (for CI with Valgrind).
Change-Id: I0e2217d568259f26cd7c4da800a13577962742f7
2015-11-19 19:05:02 +03:00
Leo Yuriev
0230e4fda2 mdbx: clarify fsync/fdatasync comment.
Change-Id: I2209b5e65ca8c04ff2de9b1b789f1993f24ab454
2015-11-19 19:05:02 +03:00
Howard Chu
02da85169e mdbx: backport - ITS#8311 fix page_split from update_key.
Check for top of stack. Usually the cursor only has height 1 when
calling page_split, but not always.

Change-Id: Iad221be30edac0f82b650f787e5dbe721cc978e0
2015-11-19 19:05:02 +03:00
Leo Yuriev
4bdeed9bd3 mdbx: fix thread-local-storage memleak.
This fix a TLS-memleak for thread from which mdbx_env_close_ex()
has been called. Bug was added by while fixing the
https://github.com/ReOpen/ReOpenLDAP/issues/48

In general we should explicitly free(), because
pthread_key_delete() don't calls a destructor.

Change-Id: Ic55a2348caf3be34b4331d5ad101ea33dbbdfa97
2015-11-19 19:05:02 +03:00
Leo Yuriev
cee258fe86 mdbx: fix uninit fields in page_malloc().
Change-Id: I35a162d6b391d33eda4d508e9c1af7238b33665d
2015-11-19 19:05:02 +03:00
Leo Yuriev
fed7922b30 mdbx: link tools librt.so with --no-as-needed.
Change-Id: I9c5096320b264aa14f1905d15e9bd495087d1bb9
2015-11-14 13:42:44 +03:00
Leo Yuriev
34d95c08eb mdbx: refines robust mutex usage for glibc.
Change-Id: I986d10999121747b05ea132dde20b6daccc2e080
2015-11-14 13:27:15 +03:00
Howard Chu
380c61bcf7 mdbx: backport - ITS#8310 fix mdb_rebalance cursor fixup.
When collapsing root, must adjust entire cursor stack

Change-Id: If62c42f556311e785aa414954a3d7fddc077cf62
2015-11-14 13:07:19 +03:00
Leo Yuriev
01f5b5e209 Merge branch 'devel' 2015-11-10 09:15:20 +03:00
Howard Chu
ab0b0ec7ce mdbx: backport - ITS#8304 fix page_merge (mdb_del cursor tracking).
Was using the pre-touch dst page pointer instead of the touched page

Change-Id: Ib41af356dcf883b923dbfbbc7591a02483ee54dd
2015-11-10 09:12:33 +03:00
Leo Yuriev
f32dab22df Merge branch 'devel' 2015-11-06 17:50:15 +03:00
Howard Chu
ee26517b5d mdbx: backpost - ITS#8300 fix rebalance after node_move.
ITS#8258, ITS#7829 fixes checked parent index to see if we were moving
from a left neighbor. Should have just checked to see if current index
was 0, meaning we added on the left. (Parent index may not tell us
anything meaningful after a nested rebalance.)

Includes:

 - ITS#8300 fix node_move
   Don't adjust other cursors when we added a node on the right.

 - ITS#8300 more for node_move fixups
   When moving a node from the right neighbor, a different
   adjustment is needed.

 - ITS#8300 simplify

 - ITS#8300 more for node_move
   When moving a node from one page to another, make sure other cursors'
   parent index gets adjusted too.
2015-11-06 17:46:51 +03:00
Howard Chu
7890eb8a8e mdbx: backport - ITS#8238 fix DUPFIXED page_split.
Parent mc_ki wasn't adjusted if new_indx was > split point

Change-Id: I45548e378e53bad5ce7a3a7c2b8236d592f3c412
2015-11-06 17:46:31 +03:00
Howard Chu
f35fb03f86 mdbx: backport - ITS#8299 fix page_merge cursor fixup.
The parent's mc_ki has changed. We need to fix that up as well
in other cursors.

Change-Id: I1f422a07ed75b99f3e36075749800c892ca6d5ec
2015-11-06 17:46:31 +03:00
Leo Yuriev
b430c9a22f mdbx: Add MDB_USE_ROBUST to control Robust Mutexes.
Backported from origin.

Change-Id: I416c1d09fb1f290423f29a84831accdaf4436ab0
2015-11-06 17:26:41 +03:00
Howard Chu
372a6d8521 lmdb: origin - CHANGES, More misc updates.
Change-Id: I1f86203b89c895ba8caf7916173b23c7e2a211d9
2015-11-06 16:58:47 +03:00
Leo Yuriev
a542ceeae5 Merge branch 'master' into devel 2015-11-06 16:27:59 +03:00
Leo Yuriev
c6e010a872 mdbx: fix segfault on close when open failed. 2015-10-31 13:56:12 +03:00
Leo Yuriev
60387be4a5 Merge branch 'master' into devel 2015-10-21 00:24:44 +03:00
Leo Yuriev
0f82e6e5fd mdbx: Makefile split 'check' and 'tests' targets. 2015-10-21 00:04:35 +03:00
Leo Yuriev
dc3256e91c mdbx: adds signatures to detect ABI mixup. 2015-10-20 23:59:06 +03:00
Howard Chu
ad808146c8 lmdb: ITS#7771 fix cursor tracking on fake pages.
node_del shifts nodes around, cursors pointing at fake pages
need to have their mc_pg[0] corrected.

Includes ITS#7771 more for prev commit.
2015-10-19 16:07:03 +03:00
Howard Chu
f6045ae77e lmdb: ITS#8263 fix cursor tracking in cursor_put.
Includes ITS#8263 streamline prev patch.
2015-10-19 16:06:28 +03:00
Leo Yuriev
68171d5f5d lmdb: provide both interfaces - advanced 'mdbx' and original 'lmdb'. 2015-10-13 23:50:25 +03:00
Leo Yuriev
09c140c1f4 lmdb: detach from ReOpenLDAP project. 2015-10-12 20:01:16 +03:00
Howard Chu
73cfae2522 lmdb: ITS#8264 fix cursor_del cursor tracking.
Some destination fixups need to happen immediately after nodes
are moved, before rebalancing

Change-Id: Ia4258cae368ff0ef96b7835cc421f2db40f7f741
2015-10-11 22:21:37 +03:00
Leo Yuriev
8473370d65 lmdb: CHANGES - ITS#7377, #8237, #8221, #8256, #8258.
Change-Id: Id825ceaf0fbb9575c0dbfd9f69997013d6f1fad0
2015-10-11 22:21:05 +03:00
Howard Chu
f2d0824bdb lmdb: ITS#8258 fix rebalance/split.
The tree height can also increase during rebalance, not just shrink.
This can happen if update_key needs to split a parent branch page.

Change-Id: Ie232bd9f4a6225b912b912a68f0b35411a632317
2015-10-06 10:13:27 +03:00
Hallvard Furuseth
305ba98967 lmdb: Fix typo in mdb_tassert().
No effect on current code.

Change-Id: Ia935ba784c33e3704ea1cfa1d59a1f28d2c527f8
2015-10-06 10:11:18 +03:00
Howard Chu
3f27fb7822 lmdb: ITS#8259 fix usage ordering.
Change-Id: I8c6f844e211b403a5652760da0e0a4836cbdf95b
2015-10-06 10:06:57 +03:00
Howard Chu
ef72fbaea6 lmdb: ITS#8221 don't merge branch pages needlessly.
Change-Id: I6b535f70392e081027d8803b6623b0c51f21ec8b
2015-10-06 10:05:46 +03:00
Luke Yeager
8e7fbfa063 lmdb: ITS#8256 Create install directories if needed.
Change-Id: Ie44f8529989fd77da0415d0c898b4af3451a851a
2015-10-06 09:59:51 +03:00
Leo Yuriev
8e2a9a9a78 lmdb: deferred cleanup of reader's threads.
This should fix https://github.com/ReOpen/ReOpenLDAP/issues/48

Change-Id: I67feb3a9852f183dcbe83626321896ba43a591b7
2015-09-07 02:38:10 +03:00
Howard Chu
83ba70e3be lmdb: ITS#8237 fix ITS#7589 regression.
Change-Id: Idd3c95f6e515f0f70c7fd33257c80354a3953f2f
2015-09-06 22:57:01 +03:00
Leo Yuriev
1b78961cf6 lmdb: fix clang warnings.
Change-Id: I6586b44c21872e7a1e128d0053837b73ed4448ec
2015-09-06 11:30:16 +03:00
Leo Yuriev
4554b8f62c all: fix build by clang (defines, warnings).
Conflicts:
	contrib/slapd-modules/usn/usn.c

Change-Id: I68572808e30a35fdd14aa0a459cfbb8da4be6dc5
2015-09-05 18:35:32 +03:00
Leo Yuriev
fa0751771e lmdb: properly link with librt.so compatibility.
Add '--no-as-needed' to LDFLAGS before -lrt.

Otherwise ld may drop dependency to librt.so,
and DSO could't be loaded when glibc < 2.17.

Change-Id: I06f34958d7efc8c56dd80ce2a55a2b8425e791cd
2015-09-05 18:30:30 +03:00
Leo Yuriev
abda67f23e lmdb: minor Makefile update, CFLAGS from environment.
Change-Id: I1c36f6bbc064136b404b5de18a88961309883018
2015-09-05 13:18:58 +03:00
Leo Yuriev
16a30be8fc lmdb: minor refine mdb_chk output,
mostly for 'zero size-of-entry' error.

Change-Id: I0f7ae514860ea05179fd929b76f78ba843902e92
2015-09-03 17:26:23 +03:00