ISAR/libmdbx
3
0
Fork 0
mirror of https://github.com/isar/libmdbx.git synced 2025-01-09 05:14:12 +08:00
Code Issues Packages Projects Releases Wiki Activity
486 Commits 5 Branches 68 Tags 13 MiB
Commit Graph

298 Commits

Author SHA1 Message Date
Leo Yuriev
f1acaf72ca mdbx: Merge branch 'master' into nexenta. 2016-05-09 22:50:27 +03:00
Leo Yuriev
7c3c28239c mdbx: more likely/unlikely for mdb_node_add. 
Change-Id: I1f241c6cea55db6fed744ba9f6f5733f6622015f
 2016-05-08 02:29:59 +03:00
Leo Yuriev
4d3e349c16 mdbx: don't memcpy when src eq dest. 
Workaround for 'destination overlaps source' from Valgrind.

Change-Id: Ib12eacbd224057334311816346ea0ccece2b42cf
 2016-05-07 23:44:41 +03:00
Leo Yuriev
8791e3243f mdbx: workaround for pthread_setspecific's memleak. 
Change-Id: I8b48548a40bb2fe58db14b8f6944f5a3802ffbb9
 2016-05-07 15:21:59 +03:00
Leo Yuriev
14b3afdcff mdbx: notify OOM-handler about end of loop. 
Change-Id: I71d66e371df869560801e12b8b06c4f4cbf90e98
 2016-05-07 03:52:04 +03:00
Leo Yuriev
f70d6638c5 mdbx: MDBX_ALLOC_KICK for freelist backlog. 
Tries to sync only when all pages was allocated,
but not every time reclaiming had stopped at weak checkpoint.

Change-Id: I0b41fd87f1eab4725379a17fc29e16820122467d
 2016-05-06 04:53:14 +03:00
Leo Yuriev
65354dcd01 mdbx: lifo-reclaimig for weak-to-steady conversion. 
Change-Id: Id078c434cf3a624331c53f6b38ce34e31dc65aa5
 2016-05-06 04:50:51 +03:00
Leo Yuriev
f335c13b45 mdbx: clarify mdbx_oomkick() for LMDB-mode. 
Change-Id: I6e38a1e2892084f8a5f98af7f947c1c9d2f18b9c
 2016-05-06 04:13:29 +03:00
Leo Yuriev
cd14429b5d mdbx: Merge branch 'master' into nexenta. 2016-05-04 23:09:01 +03:00
Leo Yuriev
599069b2c4 mdbx: remove needless assert. 
Change-Id: I0129a9e2cd8bda4c541d8b610bcf3f887ddcb75b
 2016-05-04 22:31:58 +03:00
Leo Yuriev
2dd9638321 mdbx: refine find_oldest() and oom_kick(). 
Change-Id: I758eba1a93550b687bbec6bf90e5b2d48090af25
 2016-05-04 22:31:58 +03:00
Leo Yuriev
50f5475185 mdbx: don't mimics the MDBX's features for MDB. 
Change-Id: Ie3865434c741da77b9a285e43b7d6a1d9ec0c5e2
 2016-05-04 22:30:11 +03:00
Leo Yuriev
0a65b26c58 mdbx: Merge branch 'master' into nexenta. 2016-04-28 08:21:43 +03:00
Leo Yuriev
78e6bf5e36 mdbx: refine/speedup mdb_cmp_memn(). 
Change-Id: I2c62246b26485039f29bc400a4d42f05270a7398
 2016-04-27 16:08:08 +03:00
Leo Yuriev
477932e2b3 mdbx: MADV_REMOVE for unallocated space. 
Change-Id: I14db843cf6e0dc713e2f302dc5ad86dc405c8d62
 2016-04-27 01:52:52 +03:00
Leo Yuriev
745bdd346a mdbx: extra backlog's page for MDB_LIFORECLAIM. 
Change-Id: I24d656de96a4bc3a3d4f6409db75868b818a4233
 2016-04-26 22:48:08 +03:00
Howard Chu
7ae6c0f768 mdbx: backport - ITS#8412 fix NEXT_DUP after cursor_del. 
Change-Id: If0bc63a9d398ae440a7a9ab7f0502b9c9608640e
 2016-04-26 15:19:26 +03:00
Leo Yuriev
bfd82b5830 mdbx: refine mdb_env_sync0(). 
Change-Id: If27986d02b2ca19ba547fe8fa5c8f4a8b9019900
 2016-04-20 19:51:29 +03:00
Howard Chu
78f1fbd038 mdbx: backport - ITS#8406 fix xcursors after cursor_del. 
Don't leave them uninit'd if they now point at a valid DUP node

Change-Id: Idccf9540686816e3d06e404c9a5bef1bce059bfc
 2016-04-20 19:44:01 +03:00
Leo Yuriev
ae324984cb mdbx: rework backlog for freeDB deletion. 
Change-Id: Ib535233313e8ddd21c35df46c03e5e298f3587e6
 2016-04-20 19:44:01 +03:00
Leo Yuriev
90aa4b154a mdbx: refine mdbx_cmp2int(). 
Change-Id: I5497db2e88d764e18a1aae1809dc66fd62a5adbe
 2016-04-19 11:59:36 +03:00
Leo Yuriev
026a566fc6 mdbx: minor refine madvise order. 
Change-Id: Ia6b78cad2aae68f5cb4c8475fbd27f9a9cb44cf2
 2016-04-19 11:59:36 +03:00
Leo Yuriev
80e660317f mdbx: prohibit MDB_LIFORECLAIM with just MDB_NOMETASYNC. 
Change-Id: Ib605174bf75c473aaf42b9aa9011db29343e4484
 2016-04-19 11:59:36 +03:00
Leo Yuriev
bf7f3814f9 mdbx: minor fix non-ascii comment. 
Change-Id: Ie9a38e2bef3c8ef91adc3f45818096658c84cd34
 2016-04-19 11:59:35 +03:00
Leo Yuriev
46b8915087 mdbx: rework attributes. 
Change-Id: Id9d436a54ac14ed82c593710b1d5939871c89d1a
 2016-04-13 13:40:35 +03:00
Leo Yuriev
ce06c8df9e mdbx: 'attributes' support for Nexenta. 
Change-Id: Ib7a33d6a489d3ef6cfe67349c1ae8946a6a0548a
 2016-04-13 13:40:35 +03:00
Howard Chu
ebf1ae1bdc mdbx: backport - mdb_drop optimization. 
If we know there are no sub-DBs and no overflow pages, skip leaf scan.

Change-Id: I3005aaf0f80821f87d8a88c435fda1a52ee0557a
 2016-04-13 13:26:26 +03:00
Leo Yuriev
828a5d73ca mdbx: always copy the rest of page (MDB_RESERVE case). 
Change-Id: Iebc406767bee98a85ab6efec887ed698ffe59066

> On 30. mars 2016 19:25, Леонид Юрьев wrote:
>
> Why mdb_cursor_put() doesn't copy the rest of page in case MDB_RESERVE?
>
> In other words - why we should copy or not copy an end of page in
> dependence from MDB_RESERVE?

2016-03-30 19:46 GMT+03:00 Howard Chu <hyc@symas.com>:

That logic is backwards because I was preserving existing behavior,
i.e. make the page - including unused portions - look the same with
put() in the child txn as put() in the parent.  So with MDB_RESERVE
the entire page must be copied in case the user peeks at it before
writing to it.  Without MDB_RESERVE, the initial part of the page
body need not be copied since LMDB is about to overwrite it.

I'll comment that, unless that code should just go away.  Don't know
if this is the best behavior or it was just easier to code it than
to decide whether to cater to such obscure user behavior.  But
people do come up with special things do do about overflow pages.
 2016-04-13 13:12:47 +03:00
Leo Yuriev
c17cff1d4e check MDB_RESERVE against MDB_DUPSORT. 
Change-Id: I7459a92ecea867a5fd6510c828ff65cd551a1e9a
 2016-03-30 02:32:59 +03:00
Leo Yuriev
b59f25fd46 mdbx: minor format spaces/tabs. 
Change-Id: Ic9cce2e7a2552fb062ab8709121c72a9131be8b6
 2016-03-30 02:32:50 +03:00
Howard Chu
e7859934f2 mdbx: backport - ITS#8393 fix MDB_GET_BOTH on non-dup record. 
Change-Id: I96028ff6de55915bab98a7d0e2d97072296fc769
 2016-03-29 23:03:58 +03:00
Leo Yuriev
33e2b52118 mdbx: fix nasty/stupid mistake in cmp-functions. 
Change-Id: If9aff5d9cd46dde614ddaa688113be8be12f3e5d
 2016-03-29 23:03:58 +03:00
Leo Yuriev
321bc2b901 mdbx: fix one more minor warning. 
Change-Id: I80466714d739fc46aedbedfee0919b64704ffc37
 2016-03-25 05:08:35 +03:00
Leo Yuriev
e47f52a2f3 mdbx: fix gcc warnings. 
Change-Id: Ic9014b947541a4d6c399b0d69dfc495cc98c70a2
 2016-03-25 04:56:34 +03:00
Leo Yuriev
6a0c1c47a6 mdbx: fix mdb_kill_page() for MDB_PAGEPERTURB. 
This should fix https://github.com/ReOpen/ReOpenLDAP/issues/73

Change-Id: I9514ae76ca3b56063ba345e9d06caed7ec2c1439
 2016-03-17 14:47:45 +03:00
Leo Yuriev
e90613a98c mdbx: more __inline/__hot. 
Change-Id: I9a0971a6978c958c6d5e9ed2066630f2876f988f
 2016-03-17 14:47:11 +03:00
Leo Yuriev
1b7f1bc60a mdbx: drop lead_txnid(), g.e. undo tsan-aware changes. 
Change-Id: I18a86661b0bd58bf74c644c0ed96bd836afc040c
 2016-03-13 19:31:22 +03:00
Leo Yuriev
14ddec2051 mdbx: refine mdb_env_sync(). 
Change-Id: I37e6079a94a20624e65aed3e2e599bb4490afcc0
 2016-03-13 19:18:54 +03:00
Leo Yuriev
4d9c0657d9 mdbx: refine mdb_meta_head_r(). 
Change-Id: I038862b3dada408de46f24175513d7522b1f89f5
 2016-03-13 19:18:54 +03:00
Leo Yuriev
c05fb6862d mdbx: mdbx_env_set_syncbytes() only for MDBX-mode. 
Change-Id: I7836d4b7b538fc15407122ea40c36a2f8a5ae033
 2016-03-12 20:45:16 +03:00
Leo Yuriev
eb429da210 mdbx: minors. 
Change-Id: Id1d8ed5bf6ce253dc73b2e69e8701bffa83f7e12
 2016-03-12 20:44:20 +03:00
Howard Chu
ded1ada9eb mdbx: backport - Tweak MDB_PREV_MULTIPLE for uninit'd cursor. 
Change-Id: I1f07e7c4d80631a1a0b2c70a936ef7b458baa11a
 2016-03-03 15:51:46 +03:00
Leo Yuriev
9d775b0a36 mdbx: rework support for ThreadSanitizer. 
Change-Id: I6a1cc980fa835215d2051f68bbf8580b747e6d6b
 2016-03-03 15:51:46 +03:00
Leo Yuriev
c809b2e12c mdbx: MDB_PAGEPERTURB instead of LDAP_MEMORY_DEBUG. 
Change-Id: I87d2c0518c11f8ff27e5064b4d59900dfd2131fc
 2016-03-03 14:38:44 +03:00
Leo Yuriev
c9642d1082 mdbx: MDB_PAGEPERTURB. 
Change-Id: Id78304cb1f5caa3231e69388f6ce29a431e39976
 2016-02-09 09:28:18 +03:00
Leo Yuriev
a47cdfebba mdbx: fix percent in mdbx_txn_straggler(). 
fix 75ab3716fb7cc9ffec0dd4336666bc0d2b25c60c

Change-Id: Ia63d89d189640c617da6e5e5c4567f5f8a2b84f8
 2016-02-08 13:10:17 +03:00
Leo Yuriev
b949bbd169 mdbx: cleanup inherited errno's bug. 
Change-Id: I0635f4e4a224ff4516859d12cc72b77aac41beec
 2016-02-04 06:23:55 +03:00
Leo Yuriev
425ac1cde6 mdbx: fix madvise() flags, it is not a bitmask. 
Change-Id: Ie5109779dc596190c467dcd6e84a1684e71575f5
 2016-02-04 06:23:55 +03:00
Howard Chu
77d482ef1e mdbx: backport - Happy New Year 2016. 
Change-Id: I856f00f5cc7ef9cff0c7e93257ddcbd074e02c55
 2016-02-01 21:03:53 +03:00
Howard Chu
e6ff451d84 mdbx: backport - ITS#8355 fix subcursors. 
make sure C_DEL gets reset in subcursor after it moves.

Change-Id: I334cadcd981e7578d98e326e6bd785bed709a83a
 2016-01-25 18:36:49 +03:00
Leo Yuriev
9d14f6a2ac mdbx: avoid tsan-collision by mdb_env_reader_destr(). 
Change-Id: Ic87e67db6337e2e1fe1bc21d5e2bbde64b21935b
 2016-01-25 15:21:23 +03:00
Leo Yuriev
c9c0549b4f mdbx: fix madvise() flags. 
Change-Id: If0ed3cd972fbc0caea486656ae05526aa2022574
 2016-01-14 00:27:37 +03:00
Leo Yuriev
4ad241b864 mdbx: msync only used part instead of entire db. 
Change-Id: I55611613f12d6e01599133336df183ada4d2de3d
 2016-01-14 00:27:30 +03:00
Howard Chu
ba29ae2cd2 mdbx: backport - Add MDB_PREV_MULTIPLE. 
Logical counterpart to GET_MULTIPLE, NEXT_MULTIPLE

Change-Id: I3f42cb1599997e79dbdc76bcf23b78314ededfc9
 2016-01-09 22:28:39 +03:00
Hallvard Furuseth
b067025e8b mdbx: backport - Catch strdup failure in mdb_dbi_open(). 
Change-Id: I754e8c966cc093f162cfbb33d3b179585e3ea4a2
 2015-12-15 12:45:54 +03:00
Leo Yuriev
efda814f7e mdbx: minor flags cleanup. 
Change-Id: I6ebb0d76963c0bf41051ca97fba4ec6f564519e2
 2015-12-13 19:49:53 +03:00
Howard Chu
3df7c4b1e1 mdbx: backport - ITS#8336 fix page_search_root assert on FreeDB. 
Let "illegal" branch pages thru on the FreeDB - the condition
is only temporary and will be fixed by the time rebalance finishes.

Change-Id: I5d34fd96096f88638db55166aba7194dcef7e429
 2015-12-13 19:49:22 +03:00
Hallvard Furuseth
4fcbaa44f3 mdbx: backport - ITS#8334 Fix MDB_APPENDDUP vs. rewrite(single item). 
Change-Id: I7742bd2e75ef670dac31a46d270a96ee28c20639
 2015-12-13 19:49:22 +03:00
Leo Yuriev
886fa96a5d mdbx: tsan 'edge-mutex'. 
Change-Id: If6fe58d9e1e334fbdaec529a14a78f5ebc6d8103
 2015-12-13 19:41:08 +03:00
Leo Yuriev
24449c2a07 mdbx: avoid tsan-trap in mdb_txn_renew0() by me_live_reader. 
Change-Id: I70f173d3e77c9cc84881c86e8fc64ae2a4906f0b
 2015-12-13 19:41:07 +03:00
Leo Yuriev
f38bb9621f mdbx: avoid tsan-trap in mdb_find_oldest(). 
Change-Id: Icaaf032fd3600b2ee61feb39d32af60c4e3bb0d5
 2015-12-13 19:41:07 +03:00
Leo Yuriev
75ab3716fb mdbx: avoid tsan-trap in mdbx_txn_straggler(). 
Change-Id: Ib62cdd5592e6001a1734c321f3801722e4dcf7e4
 2015-12-13 19:41:07 +03:00
Leo Yuriev
4e5d5ef935 mdbx: avoid tsan-trap in mdb_meta_head_r(). 
Change-Id: I594c13e1fe908f8afdc663d349cfc4cf634bba77
 2015-12-13 19:41:07 +03:00
Leo Yuriev
5bc0a76899 mdbx: avoid tsan-trap in mdb_txn_renew0() by mti_txnid. 
Change-Id: Ia91bb913fc99880ec34fd0bf6fc9396555c6a56f
 2015-12-13 19:41:07 +03:00
Leo Yuriev
50c480e2de mdbx: avoid tran-trap in mdb_txn_renew0() by mt_flags. 
Avoid read txn-flags from shared write TXN (e.g. env->me_tnx0->mt_flags)
without holding a write-mutex.

Change-Id: I3a3a64597f69b7df205043c567a51fe509247826
 2015-12-13 19:41:07 +03:00
Leo Yuriev
dec88f8c35 mdbx: fix double-free on thread-destroy. 
Seems this is fixes the https://github.com/ReOpen/ReOpenLDAP/issues/60

Change-Id: I29830a1b9cec0285edc976e6892678cd38e4445c
 2015-12-13 19:40:58 +03:00
Howard Chu
d0a0b096ee mdbx: backport - Cleanup C_DEL flag usage. 
Only set it if the cursor's current position was deleted

Change-Id: I4c4c2fa157b73042419cb7b7e39e280a1eca155a
 2015-11-25 19:08:26 +03:00
Leo Yuriev
2a6bfa56f0 mdbx: backport - more for cursor tracking. 
Includes:

 - ITS#8321 deinit empty cursors
   Always unset C_INIT flag if the cursor's target DB has been deleted

 - ITS#8321 Fix mdb_cursor_set
   Always reinit mc_pg[0] if cursor is not C_INITIALIZED
   It might have a stale value when using nested txns

 - ITS#8321 mdb_put cursor needs tracking too

 - ITS#8321 page_touch - don't fixup the cursor we just touched

 - ITS#8321 More cursor fixup
   Based on page_touch fixup from ITS#7594 but expanded:
   make sure sub-cursors agree with main cursors.

 - ITS#8321 cleanup unused var

Change-Id: I4b825e20e9d42d3166052e9b3e5bd0ac33b70e85

ITS#8321 fix ambiguity in cursor_put fixup

After delete/add of a node, other nodes may no longer be
pointing at the data they intended. This can confuse subsequent
fixups.

Change-Id: Id43406a9828b440735835b024e94e9b2d5a17693
 2015-11-25 19:08:14 +03:00
Howard Chu
3eb38d2f1b mdbx: backport - ITS#8323 Fix nested commit. 
Must remove our spilled pages from parent's dirty list

Change-Id: Ic37bcea4975801e48832968ea38050064a581d21
 2015-11-25 18:39:31 +03:00
Hallvard Furuseth
3e74ab2aee mdbx: backport - CURSOR_TMP_[UN]TRACK() -> WITH_CURSOR_TRACKING(). 
Change-Id: I008b7db829765889fd63900b802f57eca0c6cd20
 2015-11-25 18:09:11 +03:00
Leo Yuriev
aeea7ebb08 mdbx: backport - ITS#8321 a lot for cursor tracking. 
Multiple bugs were fixed in the cursor fixups which
adjust other open cursors in response to various write ops.

Includes:

 - ITS#8321 Fix del/dupsort.
   When deleting a dupsort key, if other cursors pointed at that key,
   set them to uninit'd, not EOF. They no longer have anything to
   point at.

 - ITS#8321 don't skip fixups on splitting cursors.
   Adjustments can't be skipped, in recursive calls each level must
   fixup their own level.

 - ITS#8321 fix mdb_cursor_chk().
   It was reporting spurious errors due to uninit'd cursors

 - ITS#8321 fix mdb_cursor_shadow().
   Set a valid txn so that cursor fixup code works on the shadows

 - ITS#8321 fix mdb_cursor_put.
   Ignore sub-cursors that shouldn't be fixed up

 - ITS#8321 track temporary cursors.
   In rebalance/split operations, temporary cursors need to be visible
   to propagate fixups

 - ITS#8321 simplify page_split fixups.

 - ITS#8321 reorganize page_split fixups.
   DUPFIXED fixups needed to occur after separator update.
   MDB_RESERVE handling moved after split fixup.

Change-Id: I0c04acf54ebf6e84f32996b5723ec6fafb983ad9
 2015-11-23 10:09:20 +03:00
Howard Chu
0a97fbcbab mdbx: backport - ITS#8300 more for prev commit. 
Just tell explicitly which direction we moved/merged from

Change-Id: Ib1868003d30f3afe71f105c2750253bbc6059610
 2015-11-23 10:07:49 +03:00
Howard Chu
2154b585c6 mdbx: backport - Silence some valgrind uninit warnings. 
Change-Id: Ie26d7bd08a3a4d4ceb833093a9117f1ebaca6cb0
 2015-11-23 10:07:49 +03:00
Howard Chu
90fdef7a6a mdbx: backport - Refix root split check from 5da67968afb599697d7557c13b65fb961ec408dd. 
Change-Id: If012ddcf223a3312bbe13c4b24d776b488ed3772
 2015-11-23 10:07:49 +03:00
Howard Chu
6ba0b8b467 mdbx: backport - ITS#8316 cursor fixup in page_merge. 
Deleting the merged page requires bumping down other ki's
of the page's parent.

Change-Id: Ifc94e26b8755ede2400521556556a6824bc2c62b
 2015-11-23 10:07:49 +03:00
Leo Yuriev
bf81e86d93 mdbx: prevents use the env after a fork(). 
Change-Id: I9c86500ac008a7be7eb16aeff2610fa5cfa22b84
 2015-11-22 17:48:30 +03:00
Howard Chu
7e2000ef7a mdbx: backport - ITS#8315 fix ovpage_free. 
Keep dirty_room sync'd with dirty_list

Change-Id: I9e52a72df95ffb504740e8daecf65b62970e9f25
 2015-11-19 20:44:49 +03:00
Howard Chu
12cd2361e7 mdbx: backport - ITS#8313 more for ITS#8062. 
dummy flags must be init'd due to 3d46d550

Change-Id: I3d543bd3e059da9b007f1e752f20171acc183679
 2015-11-19 20:44:44 +03:00
Hallvard Furuseth
86abc397e8 mdbx: backport - ITS#8312 Fix loose pages in commit(nested txn). 
Change-Id: Ie335dcff0c87cfc13abf9b937f5d058d3ea9d841
 2015-11-19 20:44:13 +03:00
Leo Yuriev
0230e4fda2 mdbx: clarify fsync/fdatasync comment. 
Change-Id: I2209b5e65ca8c04ff2de9b1b789f1993f24ab454
 2015-11-19 19:05:02 +03:00
Howard Chu
02da85169e mdbx: backport - ITS#8311 fix page_split from update_key. 
Check for top of stack. Usually the cursor only has height 1 when
calling page_split, but not always.

Change-Id: Iad221be30edac0f82b650f787e5dbe721cc978e0
 2015-11-19 19:05:02 +03:00
Leo Yuriev
4bdeed9bd3 mdbx: fix thread-local-storage memleak. 
This fix a TLS-memleak for thread from which mdbx_env_close_ex()
has been called. Bug was added by while fixing the
https://github.com/ReOpen/ReOpenLDAP/issues/48

In general we should explicitly free(), because
pthread_key_delete() don't calls a destructor.

Change-Id: Ic55a2348caf3be34b4331d5ad101ea33dbbdfa97
 2015-11-19 19:05:02 +03:00
Leo Yuriev
cee258fe86 mdbx: fix uninit fields in page_malloc(). 
Change-Id: I35a162d6b391d33eda4d508e9c1af7238b33665d
 2015-11-19 19:05:02 +03:00
Leo Yuriev
34d95c08eb mdbx: refines robust mutex usage for glibc. 
Change-Id: I986d10999121747b05ea132dde20b6daccc2e080
 2015-11-14 13:27:15 +03:00
Howard Chu
380c61bcf7 mdbx: backport - ITS#8310 fix mdb_rebalance cursor fixup. 
When collapsing root, must adjust entire cursor stack

Change-Id: If62c42f556311e785aa414954a3d7fddc077cf62
 2015-11-14 13:07:19 +03:00
Howard Chu
ab0b0ec7ce mdbx: backport - ITS#8304 fix page_merge (mdb_del cursor tracking). 
Was using the pre-touch dst page pointer instead of the touched page

Change-Id: Ib41af356dcf883b923dbfbbc7591a02483ee54dd
 2015-11-10 09:12:33 +03:00
Howard Chu
ee26517b5d mdbx: backpost - ITS#8300 fix rebalance after node_move. 
ITS#8258, ITS#7829 fixes checked parent index to see if we were moving
from a left neighbor. Should have just checked to see if current index
was 0, meaning we added on the left. (Parent index may not tell us
anything meaningful after a nested rebalance.)

Includes:

 - ITS#8300 fix node_move
   Don't adjust other cursors when we added a node on the right.

 - ITS#8300 more for node_move fixups
   When moving a node from the right neighbor, a different
   adjustment is needed.

 - ITS#8300 simplify

 - ITS#8300 more for node_move
   When moving a node from one page to another, make sure other cursors'
   parent index gets adjusted too.
 2015-11-06 17:46:51 +03:00
Howard Chu
7890eb8a8e mdbx: backport - ITS#8238 fix DUPFIXED page_split. 
Parent mc_ki wasn't adjusted if new_indx was > split point

Change-Id: I45548e378e53bad5ce7a3a7c2b8236d592f3c412
 2015-11-06 17:46:31 +03:00
Howard Chu
f35fb03f86 mdbx: backport - ITS#8299 fix page_merge cursor fixup. 
The parent's mc_ki has changed. We need to fix that up as well
in other cursors.

Change-Id: I1f422a07ed75b99f3e36075749800c892ca6d5ec
 2015-11-06 17:46:31 +03:00
Leo Yuriev
b430c9a22f mdbx: Add MDB_USE_ROBUST to control Robust Mutexes. 
Backported from origin.

Change-Id: I416c1d09fb1f290423f29a84831accdaf4436ab0
 2015-11-06 17:26:41 +03:00
Leo Yuriev
a542ceeae5 Merge branch 'master' into devel 2015-11-06 16:27:59 +03:00
Leo Yuriev
c6e010a872 mdbx: fix segfault on close when open failed. 2015-10-31 13:56:12 +03:00
Leo Yuriev
60387be4a5 Merge branch 'master' into devel 2015-10-21 00:24:44 +03:00
Leo Yuriev
dc3256e91c mdbx: adds signatures to detect ABI mixup. 2015-10-20 23:59:06 +03:00
Howard Chu
ad808146c8 lmdb: ITS#7771 fix cursor tracking on fake pages. 
node_del shifts nodes around, cursors pointing at fake pages
need to have their mc_pg[0] corrected.

Includes ITS#7771 more for prev commit.
 2015-10-19 16:07:03 +03:00
Howard Chu
f6045ae77e lmdb: ITS#8263 fix cursor tracking in cursor_put. 
Includes ITS#8263 streamline prev patch.
 2015-10-19 16:06:28 +03:00
Leo Yuriev
68171d5f5d lmdb: provide both interfaces - advanced 'mdbx' and original 'lmdb'. 2015-10-13 23:50:25 +03:00
Leo Yuriev
09c140c1f4 lmdb: detach from ReOpenLDAP project. 2015-10-12 20:01:16 +03:00
Howard Chu
73cfae2522 lmdb: ITS#8264 fix cursor_del cursor tracking. 
Some destination fixups need to happen immediately after nodes
are moved, before rebalancing

Change-Id: Ia4258cae368ff0ef96b7835cc421f2db40f7f741
 2015-10-11 22:21:37 +03:00
Howard Chu
f2d0824bdb lmdb: ITS#8258 fix rebalance/split. 
The tree height can also increase during rebalance, not just shrink.
This can happen if update_key needs to split a parent branch page.

Change-Id: Ie232bd9f4a6225b912b912a68f0b35411a632317
 2015-10-06 10:13:27 +03:00
Hallvard Furuseth
305ba98967 lmdb: Fix typo in mdb_tassert(). 
No effect on current code.

Change-Id: Ia935ba784c33e3704ea1cfa1d59a1f28d2c527f8
 2015-10-06 10:11:18 +03:00
Howard Chu
ef72fbaea6 lmdb: ITS#8221 don't merge branch pages needlessly. 
Change-Id: I6b535f70392e081027d8803b6623b0c51f21ec8b
 2015-10-06 10:05:46 +03:00
Leo Yuriev
8e2a9a9a78 lmdb: deferred cleanup of reader's threads. 
This should fix https://github.com/ReOpen/ReOpenLDAP/issues/48

Change-Id: I67feb3a9852f183dcbe83626321896ba43a591b7
 2015-09-07 02:38:10 +03:00
Howard Chu
83ba70e3be lmdb: ITS#8237 fix ITS#7589 regression. 
Change-Id: Idd3c95f6e515f0f70c7fd33257c80354a3953f2f
 2015-09-06 22:57:01 +03:00
Leo Yuriev
4554b8f62c all: fix build by clang (defines, warnings). 
Conflicts:
	contrib/slapd-modules/usn/usn.c

Change-Id: I68572808e30a35fdd14aa0a459cfbb8da4be6dc5
 2015-09-05 18:35:32 +03:00
Leo Yuriev
16a30be8fc lmdb: minor refine mdb_chk output, 
mostly for 'zero size-of-entry' error.

Change-Id: I0f7ae514860ea05179fd929b76f78ba843902e92
 2015-09-03 17:26:23 +03:00
Leo Yuriev
07ee471b1c lmdb: me_maxkey_limit always computed and checked against MDB_MAXKEYSIZE at opening env. 
Change-Id: Ice0a425900a8c7b37d5b6cfcd1434edc8d400a74
 2015-09-02 18:08:43 +03:00
Leo Yuriev
f7d8812e6c lmdb: n-entries info from b-tree traversal in mdb_chk. 
Change-Id: Ic9343dcdba976ac51c8f5776d2f7c9fed59da8e0
 2015-09-02 17:53:48 +03:00
Leo Yuriev
15e0600b6c lmdb: major rework of traversal b-tree for mdb_chk. 
Change-Id: I9d382516f76092f44fc1a12d7554039582b87656
 2015-09-02 16:19:12 +03:00
Leo Yuriev
e7f58da2ac lmdb: support for a utterly no-sync mode, 
by combination of MDB_NOSYNC and MDB_MAPASYNC.

Change-Id: I3e2d6c8f044a7522436dab4d0818d4d59b78b78d
 2015-09-01 12:15:59 +03:00
Hallvard Furuseth
e4d9f0a905 lmdb: ITS#7377 catch calloc error in mdb_env_init_meta(). 
Change-Id: I2b1a4ec9f86a3a7844b157863d18ce81be400cef
 2015-08-30 01:02:43 +03:00
Leo Yuriev
6a3efb82eb lmdb-test: workaround for MDB_FIXEDMAP to avoid sporadic failures. 
Change-Id: I6ba993ff50956d328fd76326c67aa83dff20ce48
 2015-08-24 03:37:05 +03:00
Leo Yuriev
537fc0fe1a lmdb: simple write benchmark. 
Change-Id: Iadcbe8ad5922b2ecf1ea597b188ef368bc097185
 2015-08-24 03:36:55 +03:00
Leo Yuriev
00d2057204 lmdb: fix rare MDB_MAP_FULL when pages are available in the gc. 
Rework preparation backlog of available pages for cleanup FreeDB.
https://github.com/ReOpen/ReOpenLDAP/issues/42

Change-Id: I47592b4cabf1950e61bb87e67c08ea03e95496fa
 2015-08-20 19:26:20 +03:00
Leo Yuriev
b8c45546f6 lmdb: more likely/unlikely hints. 
Change-Id: I38f1f80243bfc5b1ffe910175516dbb3f06c27ba
 2015-08-15 13:17:46 +03:00
Leo Yuriev
ad8c4a19bb lmdb: fix segfault in mdb_chk when mdb_env_open() failed. 
Change-Id: If72d35518c2d4dd33d284049e00fc633e65ee52c
 2015-08-13 18:30:03 +03:00
Leo Yuriev
96d69e3fa1 lmdb: added mdb_env_open_ex() and exclusive/monopoly mode for mdb_chk. 
Change-Id: I867e7f17924ffecb744440e95db96c0a7411d6ef
 2015-08-13 18:30:02 +03:00
Leo Yuriev
1171e4d73a lmdb: fix build non-debug mode (MDB_DEBUG=0). 
Change-Id: I362a6bd03c73dca70bc6df5f72badf59e560389a
 2015-08-06 17:13:25 +03:00
Leo Yuriev
3fc45cbaea lmdb: fix assertion failure in case error while opening DB in debug-mode. 
Change-Id: I56a1122369640920d594416e9bd3de0f3f9cd8fd
 2015-08-05 22:59:18 +03:00
Leo Yuriev
595f9e700d lmdb: fix b-tree walking bug in case empty db. 
Change-Id: I28c0985493327babe4d2ae5c6cd56116a26e6034
 2015-08-05 22:59:18 +03:00
Leo Yuriev
a243a38aa6 lmdb: adding page-space usage info to mdb_chk. 
Change-Id: I12dcdb54894a362d41612e973f983cb4e067520c
 2015-08-05 22:59:18 +03:00
Leo Yuriev
c32cf0fea4 lmdb, back-mdb: use 'long' as result to speedup comparison. 
Change-Id: I3ade0c7525f4a461a7edf5873af196e6b33c0727
 2015-08-05 14:35:09 +03:00
Leo Yuriev
b60f510355 lmdb: rework key/data comparison to avoid alignment errors. 
Change-Id: I291b50ad6f7dd18f6cbdef34028d917e10166cf1
 2015-08-05 00:45:57 +03:00
Hallvard Furuseth
c9a8595ef2 lmdb: Kill magic numbers for NUM_METAS, databases. 
Change-Id: I7ed12f081aed7eb65087fea511053157fb8f9c1e
 2015-08-04 13:57:14 +03:00
Hallvard Furuseth
061fe51e8c lmdb: ITS#7377 Catch mdb_put() to blocked/read-only txns. 
...early enough that txn state is left unchanged.

Change-Id: Ia475660f339f7b741eaaa126978f8419a19b67a0
 2015-08-04 13:57:14 +03:00
Hallvard Furuseth
d6b3c6f710 lmdb: ITS#7377 Always notice env error on txn startup. 
Move the check to the end of txn startup.  Catches env
breakage which happens while the new txn waits for a lock.

Change-Id: I074d411cd5339b5e6caa2691078ea07b0d49828e
 2015-08-04 13:57:14 +03:00
Hallvard Furuseth
11999c8c02 lmdb: Cleanup, Drop !mt_dbxs hack, use MDB_TXN_FINISHED. 
Change-Id: I23beaaf931c938ee9f3886ca1928ecbcd2fbe5ba
 2015-08-04 13:57:14 +03:00
Hallvard Furuseth
342d56977f lmdb: Catch most uses of finished/parent txns. 
* Add MDB_TXN_FINISHED, MDB_TXN_HAS_CHILD, MDB_TXN_BLOCKED.
* Clear mt_numdbs in writers, for TXN_DBI_EXIST() to catch.
  We already do in readers.

Change-Id: I4f714c0789188dfab4ce22b7d9d3a75d8b62ef6b
 2015-08-04 13:57:14 +03:00
Hallvard Furuseth
e3c8fa8eba lmdb: Move code into mdb_txn_end(). Was mdb_txn_reset0. 
Side effects:
* Clean txn up a bit even before freeing it.
* Tweak DEBUG output at txn end. Add DEBUG after commit(writer).

Change-Id: Ia9905257893923bb14cdf33b60b6387af3121f81
 2015-08-04 13:57:14 +03:00
Hallvard Furuseth
23af429b3f lmdb: ITS#8200 Fix mdb_midl_shrink() usage, return void. 
Change-Id: I506a170d434178d43f01b3e9597c4a03b3b3e7a7
 2015-08-04 13:57:14 +03:00
Hallvard Furuseth
862cfc30c4 lmdb: Simpler mdb_node_add(). 
Change-Id: I5b1f851a098aeb4b5adcd7ed55150a055cd63102
 2015-08-04 13:57:14 +03:00
Hallvard Furuseth
be27186f76 lmdb: ITS#8181 Verify that records are/aren't DBs. 
Except we don't catch the user passing F_SUBDATA to
mdb_cursor_<put/del>, like an internal LMDB call.

Change-Id: I5ebce36694045279702c74490a5a634217d4f95d
 2015-08-04 13:57:13 +03:00
Howard Chu
91a6b6de57 lmdb: ITS#8190 fix cursor EOF bug. 
Change-Id: I97556bc521e1d4d4ea9f3e43c14a8a2fe62b8231
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
96fb8eec8a lmdb: More ESECT declarations. 
Change-Id: Iaed5c6e35b1bd6e59caf8d1f52b265ec4852c84a
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
57b263693c lmdb: Add DB_USRVALID, to avoid 'dbi == FREE_DBI' tests. 
Change-Id: Ic44ce41fd2fff1abb4cd24821ece07ef30254565
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
4cb5d637d4 lmdb: Simpler flag/DBI checks and MDB_DEBUG. 
Change-Id: I37a00fcc23f2060538ab4378f204ae0efcdb59ea
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
9e52d29fd8 lmdb: Simpler mdb_drop(). 
MDB_DBI_CHANGED(,MAIN_DBI) is never true.

Change-Id: I0d12b9965d8251908ec41d1febc6b5c354a93a01
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
6524dbf3a1 lmdb: Simpler mdb_node_shrink(). 
Change-Id: If0e4153e69c4f858ee45471c4888ec2024253b50
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
a6fa632a54 lmdb: Simpler mdb_txn_commit(). 
mt_env is always set.
Commit(mt_child) resets mt_child, so parent need not.

Change-Id: Ib8af816a964a4b70fa6fa622b3d9dd463e9460b3
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
82213d732c lmdb: cleanup mdb_txn_begin(). 
Change-Id: Ia28e255a0497d02f7d87ca301d5be8a60886d4e1
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
b3a3dc4c6b lmdb: Omit scanning DUPSORT sub-DB leaves in mdb_drop0(). 
Change-Id: Icc670a81210a9160e6cb90b9442994c008f599bd
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
b13d850bd7 lmdb: Use mdb_cmp_int_a/mdb_cmp_long() for FREE_DBI. 
Change-Id: Id66356b27e9d257fbf5bc8ce1111845be8683d62
 2015-08-04 13:57:13 +03:00
Hallvard Furuseth
84b6ecb9a8 lmdb: Hide internal flags in mdb_env_get_flags(). 
Change-Id: I9ff2888a61a53fb0c7471de980ec7b97c59c6096
 2015-08-04 13:57:13 +03:00
Leo Yuriev
295beda00c lmdb: fix ABI compatibility with original liblmdb.so 
Change-Id: I770a49b7a506f3f2a21ea3af36fd07c16a32dc30
 2015-06-04 19:12:14 +03:00
Hallvard Furuseth
f71cb6d69e lmdb: ITS#8156 Fix MDB_MAXKEYSIZE doc. 
Change-Id: Ia9dae2e68f28dcb715ac34226b358212317199c2
 2015-06-01 15:38:18 +03:00
Hallvard Furuseth
55ae3dfa8b lmdb: ITS#8157 mdb_txn_renew0(): init after error checks. 
Change-Id: I9ea344b98cb8c99a25ba7087abefbfe62b3417a0
 2015-05-28 16:00:27 +03:00
Hallvard Furuseth
bb7b923fdd lmdb: Set/clear mp_pad, md_pad (MDB_DUPFIXED data size). 
mdb_xcursor_init1(): md_pad is only used when MDB_DUPFIXED.
mdb_page_split():    Copy mp_pad too. Used by mdb_page_list().

Change-Id: Ia35cba3a367c04929a34a5879abc968202086e89
 2015-05-21 21:40:39 +03:00
Hallvard Furuseth
476c5b02ea lmdb: ITS#8117 Fix MDB_INTEGERDUP keysize doc. 
Change-Id: I7cf7c456f35fe9c5449a37ad9130864617197726
 2015-05-21 21:40:39 +03:00
Leo Yuriev
104312d15e lmdb: update mdb_chk for checkpoints, minor cosmetics. 
This is 9/9 for https://github.com/ReOpen/ReOpenLDAP/issues/1
and https://github.com/ReOpen/ReOpenLDAP/issues/2

Change-Id: I05f59313b4346c08bcd63d62e6c48a5f53c47852
 2015-05-21 21:40:39 +03:00
Leo Yuriev
21705fd9a2 lmdb: seek to steady meta-page on db-open. 
This is 7/9 for https://github.com/ReOpen/ReOpenLDAP/issues/1
and https://github.com/ReOpen/ReOpenLDAP/issues/2

Change-Id: If59a6bfc7c6198e6b1e85f4bdc5b534ecff03123
 2015-05-21 21:40:39 +03:00