Commit Graph

279 Commits

Author SHA1 Message Date
Leo Yuriev
81661ff952 mdbx: add shared cache for oldest reader's txnid.
Change-Id: I48cbc778b873445dffa8ecef1fc3633e0193131f
2017-06-16 05:21:49 +03:00
Leo Yuriev
c01aeb5c68 mdbx: refine meta_txnid().
Change-Id: I4d78b2ebed8850c90dc762405ba939be7385c9a3
2017-06-16 05:21:49 +03:00
Leo Yuriev
ae71c2aa36 mdbx: simplify find-oldest inside mdbx_page_alloc().
Change-Id: I88dfc8ee268eba2452bffd3c4e6cfd066bcd8b04
2017-06-16 05:21:27 +03:00
Leo Yuriev
760f7d1331 mdbx: fix nasty 'inverted if-condition' inside mdbx_reader_check0().
Fix c2087f186e
2017-06-15 15:05:14 +03:00
Leo Yuriev
ddc3787936 mdbx: fix MDBX_RESULT_TRUE handling inside mdbx_mutex_failed()...mdbx_oomkick(). 2017-06-15 14:56:25 +03:00
Leo Yuriev
71e2fe3df0 mdbx: add MDBX_DBG_DUMP for mdbx_setup_debug().
So, meta-pages and lck-section now will be included into a coredump.
2017-06-15 13:45:16 +03:00
Leo Yuriev
0ef41bef2b mdbx: fix mdbx_txn_straggler() for write-txn. 2017-06-13 19:27:52 +03:00
Leo Yuriev
6f53dd0719 mdbx: add mdbx_dbi_flags_ex(). 2017-06-09 16:44:34 +03:00
Leo Yuriev
3ca64535b6 mdbx: add NTAPI to OSAL. 2017-06-07 18:17:21 +03:00
Leo Yuriev
d2d513014d mdbx: add mdbx_lck_upgrade() to OSAL. 2017-06-06 20:28:00 +03:00
Leo Yuriev
204904df7c mdbx: disallow begin txn if env not yet opened. 2017-06-06 20:28:00 +03:00
Leo Yuriev
fbce45cb98 mdbx: allow calling mdbx_env_sync() inside transaction. 2017-06-06 20:28:00 +03:00
Leo Yuriev
1d15ae4b13 mdbx: zeroing MDBX_page.mp_validator for Valgrind/AddressSanitizer. 2017-06-06 17:47:46 +03:00
Leo Yuriev
465459dc58 mdbx: add MDBX_txn.mt_owner and MDBX_THREAD_MISMATCH. 2017-06-06 17:11:29 +03:00
Leo Yuriev
3bf3a08f96 mdbx: fix write-txn thread cleanup. 2017-06-06 17:09:49 +03:00
Leonid Yuriev
318646794e mdbx: partial fix MSVC level-4 warnings. 2017-06-06 04:15:08 +03:00
Leonid Yuriev
bfa6dea784 mdbx: fix size_t/unsigned/int warnings.
Change-Id: Ic5a8684aed232b8b732d6e7a87a6757f3f7afcec
2017-06-06 04:15:08 +03:00
Leonid Yuriev
7e85ad82f1 mdbx: refine read_header(), add MDBX_TOO_LARGE.
Change-Id: I25b9bb2e3817993d627b2b784dfa496d9ba7e7b0
2017-06-06 04:15:08 +03:00
Leo Yuriev
61d7ea283f mdbx: size_t for sync_threshold and mapsize. 2017-06-06 04:15:08 +03:00
Leo Yuriev
802c10f13f mdbx: log2shift instead of multiplication by pagesize.
Change-Id: I72c4c7699a409795459b466ff74c388b15b13687
2017-06-06 04:15:08 +03:00
Leo Yuriev
9ef81ac16c mdbx: add 'unlikely' to IS_OVERFLOW().
Change-Id: Ia92aac948046890ca4f9d9c3fbff0b4ac5728fab
2017-06-06 04:15:08 +03:00
Leo Yuriev
439ae3983c mdbx: use 2^N constants for fill-threshold.
Change-Id: I3e190f5af77282a7995ae733c7a6865a21daa6c3
2017-06-06 04:15:08 +03:00
Leo Yuriev
59c22ab036 mdbx: use 32-bit pgno_t (page numbers).
Change-Id: Ie9f3528e12b7be27765c0225edde26e4d8282692
2017-06-06 04:15:08 +03:00
Leo Yuriev
1343b46466 mdbx: add txnid-list functions. 2017-06-06 04:15:08 +03:00
Leo Yuriev
52f52de2d5 mdbx: include meta-pages into a coredump. 2017-06-06 04:15:08 +03:00
Leo Yuriev
95ebdb7065 mdbx: MAX_PAGESIZE always 64K. 2017-06-06 04:15:08 +03:00
Leo Yuriev
19dd181b6f mdbx: fix STATIC_ASSERT for MSVC. 2017-06-06 04:15:08 +03:00
Leo Yuriev
61a3766e23 mdbx: update DB format and signatures.
Change-Id: I9c4b187e8ebc3df63fef15ae98872e27d56a01ab
2017-06-06 04:15:08 +03:00
Lorenz Bauer
d99b2a4b16 mdbx: use F_SETNOSIGPIPE to SIGPIPE husing on a OS X. 2017-06-06 04:15:08 +03:00
Leo Yuriev
98a8fbdc7a mdbx: restore SIGPIPE hushing in the db-copy-thread. 2017-06-06 04:15:08 +03:00
Leonid Yuriev
e7ee0bc762 mdbx: refine testing-jitter, add MDBX_DBG_JITTER.
Change-Id: Ibc3bd8a16626e97aabc2cc544a3803f2e2bc3a10
2017-06-06 04:15:08 +03:00
Leo Yuriev
009618560f mdbx: refine mdbx_is_dirty(). 2017-05-30 15:32:05 +03:00
Leo Yuriev
d5b0e68326 mdbx: fix merge/rebase error (remove mm_txnid field). 2017-05-29 20:44:17 +03:00
Leo Yuriev
a36b065cd9 mdbx: logging MDBX_PROBLEM reasons. 2017-05-29 20:17:02 +03:00
Leo Yuriev
11e8727483 mdbx: fix missing braces. 2017-05-29 20:16:09 +03:00
Leo Yuriev
70fe982c3a mdbx: fix MDBX_WANNA_RECOVERY description. 2017-05-29 19:38:43 +03:00
Leo Yuriev
d91785f635 mdbx: disable unnecessary assertion in mdbx_is_dirty(). 2017-05-29 19:38:43 +03:00
Leo Yuriev
0f676db491 mdbx: add locking while open/close/drop dbi-handles (avoid dbi-related races).
This avoid races and collisions between threads when opening,
closing and deleting DBI-handles.

unfortunately, this does not resolve collision in the case an one thread
closing the DBI-handle while the another thread performs transaction.

Change-Id: I48c3ffb11a8f83739fae1712db3476645f573e09
2017-05-29 19:38:43 +03:00
Leo Yuriev
bd5d092a2b mdbx: check system pagesize (paranoia).
Change-Id: I596c686a996f7d1521789f22900022c4da629e1b
2017-05-28 18:55:18 +03:00
Leo Yuriev
a0ddf167dd mdbx: common database format for 32/64 targets (remove 'size_t' structures).
Change-Id: I2d7c77603ee27c5ae4b82f762a726f71136527c8
2017-05-28 18:35:58 +03:00
Leo Yuriev
fca74ab80c mdbx: refine meta-commit in WRITEMAP mode.
Change-Id: Ieb53bf6144a104fc88b0b07b4abdde20f4b01978
2017-05-28 18:35:58 +03:00
Leo Yuriev
2347282b4f mdbx: add assertions for reclaiming-edge.
Change-Id: I87e11f69423b4e7841c4f8ec6b5ecfdff9e96b0d
2017-05-28 17:06:04 +03:00
Leo Yuriev
236ddda1ae mdbx: more assertions for mdbx_txn_renew0().
Change-Id: I3c179d105c3e6388c08b2c371ada4d457af50bed
2017-05-28 17:06:04 +03:00
Leo Yuriev
2ea97ae281 mdbx: refine find_oldest() and meta_head(). 2017-05-28 17:06:04 +03:00
Leo Yuriev
91bc3129d5 mdbx: refine mdbx_page_search().
Change-Id: I9c87ae2ffe317538813ebec971ad093936f4c504
2017-05-28 17:06:04 +03:00
Leo Yuriev
f945e8a062 mdbx: fix cache-aligned size for MDBX_reader.
Change-Id: I1e48dd279106daa629fbcd427ec841828799c8f6
2017-05-28 17:06:04 +03:00
Leo Yuriev
bb5e1b43af mdbx: relax mdbx_is_dirrty() assertion for map-range. 2017-05-28 17:06:04 +03:00
Leo Yuriev
17c8057684 mdbx: fix gcc 'uninitialized' warning with -Og. 2017-05-28 17:06:04 +03:00
Leo Yuriev
77f0e5e38e mdbx: drop unused debug features. 2017-05-28 17:06:04 +03:00
Leo Yuriev
60fed8bbca mdbx: fix first-rdonly-blocker bug. 2017-05-28 17:06:04 +03:00
Leo Yuriev
5c3691eff1 mdbx: split atomics for 32/64 uints. 2017-05-28 17:06:04 +03:00
Leo Yuriev
76f459c08e mdbx: two-phase txnid to avoid bottom-top DMA updates. 2017-05-28 17:06:04 +03:00
Leo Yuriev
dd33bea1d8 tools: mdbx_chk - don't 'rolled-back' in cooperative more. 2017-05-28 17:02:19 +03:00
Leo Yuriev
bd70135db1 mdbx: re-read and check meta-pages in mdbx_read_header() for consistency. 2017-05-28 17:02:19 +03:00
Leo Yuriev
93c5a84bcd mdbx: more checks mdbx_read_header() for metapage's content. 2017-05-28 17:02:19 +03:00
Leo Yuriev
1c170cb866 mdbx: add MAX_PAGENO, MIN_PAGENO, MAX_MAPSIZE, MIN_MAPSIZE. 2017-05-28 17:02:19 +03:00
Leo Yuriev
18cf804d0b mdbx: add STATIC_ASSERT_MSG, refine STATIC_ASSERT. 2017-05-28 17:02:19 +03:00
Leo Yuriev
4293840055 mdbx: fix meta-invalidate race in MDBX_WRITEMAP mode (when target == head). 2017-05-28 17:02:19 +03:00
Leo Yuriev
f4d3b76f81 mdbx: minor refine and rename mdbx_sync_locked().
Change-Id: Iab650b091006e5646d9d22316d19aa58a517684b
2017-05-28 17:02:19 +03:00
Leo Yuriev
9959fe8711 mdbx: reorder fields in metapages. 2017-05-28 17:02:19 +03:00
Leo Yuriev
c02a3ac687 mdbx: ignore meta-pages with wrong pagesize. 2017-05-28 17:02:19 +03:00
Leo Yuriev
0e49b1b2b4 mdbx: update 'mapsize' while opening with txn-increment. 2017-05-28 17:02:19 +03:00
Leo Yuriev
d642127d10 tools: fix 'tail' show in mdbx_chk. 2017-05-28 17:02:19 +03:00
Leo Yuriev
5d836d2b03 mdbx: clarify steady/fluid/weak names of a head. 2017-05-28 17:02:19 +03:00
Leo Yuriev
1e0654a4a8 tool: show operational txnid in mdbx_chk. 2017-05-28 17:02:19 +03:00
Leo Yuriev
47cc45dec4 mdbx: don't check present durability mode for MDBX_RDONLY. 2017-05-28 17:02:19 +03:00
Leo Yuriev
700f3e9c2a mdbx: more checks for meta-params. 2017-05-28 17:02:19 +03:00
Leo Yuriev
9eeb00f448 mdbx: trinity of meta-pages. 2017-05-28 17:02:19 +03:00
Leo Yuriev
141306644d mdbx: fix mdbx_env_copy_asis(). 2017-05-26 03:10:31 +03:00
Leo Yuriev
7315b99b9d mdbx: fix MSVC/Windows warnings. 2017-05-25 19:12:51 +03:00
Leo Yuriev
b9dbe7c577 mdbx: fix segfault and double-free (in case twice abortion of the same write-txn). 2017-05-25 18:39:45 +03:00
Leo Yuriev
96de36baef mdbx: refine mdbx_cursor_count() API. 2017-05-25 16:21:29 +03:00
Leo Yuriev
3a166e1970 test: temporary crutch. 2017-05-25 15:30:35 +03:00
Leo Yuriev
b94e761d73 mdbx: fix 'magic' bug. 2017-05-25 15:08:45 +03:00
Leo Yuriev
5519d568f0 mdbx: fix mdbx_filesize() for Windows. 2017-05-25 09:40:29 +03:00
Leo Yuriev
15e2a45425 mdbx: check size of lck-file. 2017-05-25 09:40:29 +03:00
Leo Yuriev
c335b16c81 mdbx: rework mapsize's stuff inside mdbx_setup_dxb(). 2017-05-25 09:40:29 +03:00
Leo Yuriev
af7b468e63 mdbx: use uint64_t or size_t insted of off_t (buggy on Windows). 2017-05-24 18:50:24 +03:00
Leo Yuriev
7ef7e70012 mdbx: add STATIC_ASSERT. 2017-05-24 18:45:08 +03:00
Leo Yuriev
96b9af0b4d mdbx: minor refine idl functions. 2017-05-24 17:06:18 +03:00
Leo Yuriev
924e81ed92 mdbx: refine includes, drop midl.h and mdbx_osal.h 2017-05-24 17:05:59 +03:00
Leo Yuriev
dd7855c30d mdbx: cleanup tabs (minor). 2017-05-24 13:37:06 +03:00
Leo Yuriev
06de7a5155 mdbx: rename the rest MDBX_xyz. 2017-05-24 02:24:38 +03:00
Leo Yuriev
62ebc59330 mdbx: rework version info (stub for now). 2017-05-24 02:17:10 +03:00
Leo Yuriev
88ea2768f5 mdbx: fix gcc 'comparison is always true' warning. 2017-05-23 22:35:59 +03:00
Leo Yuriev
2c43997026 mdbx: change mdbx_env_set_oomfunc() API. 2017-05-23 22:18:21 +03:00
Leo Yuriev
132c9c994e mdbx: add mdbx_fastmutex_t. 2017-05-23 22:18:21 +03:00
Leo Yuriev
8c18622592 mdbx: rework MDBX_val. 2017-05-23 22:18:21 +03:00
Leo Yuriev
ac8e987346 mdbx: rework MDBX_txn. 2017-05-23 22:18:21 +03:00
Leo Yuriev
34213c554a mdbx: rework MDBX_reader. 2017-05-23 22:18:21 +03:00
Leo Yuriev
a3ed42b999 mdbx: rework MDBX_page, drop COPY_PGNO. 2017-05-23 22:18:20 +03:00
Leo Yuriev
8f2c21e2ba mdbx: rework MDBX_node. 2017-05-23 22:18:20 +03:00
Leo Yuriev
8828e90ff9 mdbx: mdbx_condmutex_t instead of mutex/condvar pair. 2017-05-23 19:57:26 +03:00
Leo Yuriev
2f97939efd mdbx: more cleanup mdbx_midl_sort(). 2017-05-23 19:57:26 +03:00
Leo Yuriev
eff0f92fa7 mdbx-tools: replace size_t/PRIuPTR with uint64_t/PRIu64. 2017-05-23 19:57:26 +03:00
Leo Yuriev
0cd30792bb mdbx: split MDB_ID into pgno_t and txnid_t. 2017-05-23 19:57:26 +03:00
Leo Yuriev
5c2042c466 tools: internal minor renames (preparation for changes). 2017-05-23 17:45:58 +03:00
Leo Yuriev
8b9e391dd0 mdbx: cleanup internals (mostly formatting, but not only). 2017-05-23 17:45:58 +03:00
Leo Yuriev
398b839b98 mdbx: drops old/obsolete API (mdbx_env_copy and mdbx_env_copydf). 2017-05-23 17:45:58 +03:00
Leo Yuriev
0eee938c5f mdbx: add MDBX_EIO. 2017-05-23 17:45:58 +03:00
Leo Yuriev
994481e4f4 mdbx: replace size_t with uint64_t/uint32_t in API. 2017-05-23 17:45:58 +03:00
Leo Yuriev
d0793a1daf mdbx: fix MDB_DUPSORT with MDB_CURRENT update bug. 2017-05-22 15:35:33 +03:00
Leo Yuriev
82d3595b76 mdbx: add MDBX_EKEYMISMATCH. 2017-05-22 15:35:33 +03:00
Leo Yuriev
d9eeac45b2 mdbx: fix extra-logging without NDEBUG, but with MDBX_DEBUG=0. 2017-05-19 07:39:50 +03:00
Leo Yuriev
0810f4e105 mdbx: fix MDBX_CORRUPED error (nasty copy&paste bug). 2017-05-18 22:31:19 +03:00
Leo Yuriev
b4fd0500d1 mdbx: more lck-debug. 2017-05-18 16:17:24 +03:00
Leo Yuriev
d2af39fd73 mdbx: more for without-lck mode (not completed). 2017-05-18 16:17:24 +03:00
Leo Yuriev
fd8be99282 mdbx: cleanup mdbx_env_init_metas(). 2017-05-18 16:17:24 +03:00
Leo Yuriev
818e742c58 mdbx: cleanup mdbx_env_create(). 2017-05-18 16:17:24 +03:00
Leo Yuriev
6ee3aebc32 mdbx: use flock() for share/exclusive lck. 2017-05-18 16:17:24 +03:00
Leo Yuriev
4481555c90 mdbx: refine mdbx_dkey() API. 2017-05-18 16:02:29 +03:00
Leo Yuriev
c479c5ff15 mdbx: rework mdbx_env_sync().
Change-Id: I6192e8a333b607d1e46eadddb73863943635f1ec
2017-05-18 16:02:29 +03:00
Leo Yuriev
7a4541f598 mdbx: minor cleanup. 2017-05-18 15:57:06 +03:00
Leo Yuriev
00be62e235 mdbx: cleanup formating. 2017-05-18 15:57:06 +03:00
Leo Yuriev
677625ead1 mdbx: add MDBX_WANNA_RECOVERY. 2017-05-18 04:31:54 +03:00
Leo Yuriev
015a242c89 mdbx: rollback weak checkpoint or MDB_CORRUPTED. 2017-05-18 04:30:10 +03:00
Leo Yuriev
ed46246931 mdbx: rework mdbx_dbi_open(). 2017-05-16 00:50:12 +03:00
Leo Yuriev
00081298d6 mdbx: add mdbx_get_errno_checked(). 2017-05-15 12:08:04 +03:00
Leo Yuriev
533e01cc93 mdbx: change mdbx_dbi_close() API. 2017-05-11 10:08:22 +03:00
Leo Yuriev
e8a430999c mdbx: hush MSVC warnings for atomic stubs. 2017-05-11 10:08:22 +03:00
Leo Yuriev
29f01cf3f4 mdbx: use PRIuPTR/PRIiPTR/PRIxPTR instead of %z. 2017-05-10 19:16:23 +03:00
Leo Yuriev
8848df34bc mdbx: check gcc/clang version for stdatomic.h 2017-04-27 18:46:11 +03:00
Leo Yuriev
4e9b734b52 mdbx: fix mdbx_read_header(). 2017-04-27 18:15:12 +03:00
Leo Yuriev
82f053a685 mdbx: more debug for lck-seize. 2017-04-27 18:15:12 +03:00
Leo Yuriev
678e4f5738 mdbx: more check/debug around mdbx_pread() and mdbx_read_header(). 2017-04-27 18:15:12 +03:00
Leo Yuriev
f8903ca7c7 mdbx: minor refine/speedup mdbx_cursor_put(). 2017-04-27 18:07:58 +03:00
Leo Yuriev
4b2cb66453 mdbx: use mdbx_jitter4testing() for race detection. 2017-04-27 15:52:52 +03:00
Leo Yuriev
7204c46421 mdbx: add mdbx_osal_jitter() and mdbx_jitter4testing(). 2017-04-27 15:52:52 +03:00
Leo Yuriev
40dee6f05f mdbx: rework mdbx_read_header(). 2017-04-27 15:52:52 +03:00
Leo Yuriev
fd078ee163 mdbx: check for txnid overflow (paranoia).
Change-Id: Id7cd62abf8605150ff5491fb1129383e3e748603
2017-04-27 01:37:42 +03:00
Leo Yuriev
9e07b71c27 mdbx: fix snap-state bug.
Change-Id: I8700209ab012bfef21c88fe06c39b80a640291c6
2017-04-27 01:37:42 +03:00
Leo Yuriev
939285bef2 mdbx: add atomics-ops for assertions. 2017-04-27 01:37:42 +03:00
Leo Yuriev
7ec571c9a0 mdbx: partial cleanup/reformat comments.
Change-Id: Ia96a5b307dc065b4d1920234c3cce93a0e585876
2017-04-27 01:36:31 +03:00
Leo Yuriev
333e5fada3 mdbx: internal cleanup (rename, etc). 2017-04-26 19:11:34 +03:00
Leo Yuriev
99002d016f mdbx: add yielding for lck-testing for Windows if NDEBUG undefined. 2017-04-26 14:17:52 +03:00
Leo Yuriev
5ed0ccfcbb mdbx: rework mdbx_pread(). 2017-04-25 19:59:00 +03:00
Leo Yuriev
aa59522dbe mdbx: minor refine open-path. 2017-04-25 19:51:23 +03:00
Leo Yuriev
326dea8bc1 mdbx: use FormatMessage() for Windows. 2017-04-25 19:17:57 +03:00
Leo Yuriev
993730d2f1 mdbx: more cleanup for Windows. 2017-04-24 21:14:36 +03:00
Leo Yuriev
e98a1e5319 mdbx: move MDBX_LOCK_SUFFIX to API defs. 2017-04-24 17:52:56 +03:00
Leo Yuriev
0d59cd4fe2 mdbx: rework lck/body setup. 2017-04-24 16:15:01 +03:00
Leo Yuriev
19d877635c mdbx: alter db-file extensions. 2017-04-24 16:15:01 +03:00
Leo Yuriev
2e54210e79 mdbx: add mdbx_osal.h with platform-like err codes. 2017-04-24 16:15:01 +03:00
Leo Yuriev
f7507110ef mdbx: alter mdbx_txn_id() API. 2017-04-24 16:15:01 +03:00
Leo Yuriev
ce9c4acea6 mdbx: fix mdbg_canary_put().
Change-Id: I36917a686b28405ed7ecd07dbeb1b4d517720410
2017-04-23 19:16:22 +03:00
Leo Yuriev
1bddc9dcbc mdbx: change mdbx_canary_get() API.
Change-Id: Ie8831d8acb7113916b8a44b870596a9bf1ccc8b2
2017-04-23 19:16:22 +03:00
Leo Yuriev
11ac6a02ff mdbx: rework posix-lck.
Change-Id: Icdd61731424b0d591fd28da0468b9623d079c94a
2017-04-23 19:16:22 +03:00
Leo Yuriev
6b4f92b22e mdbx: rework windows-clk.
Change-Id: Ie0f8422a39879c9d5e6b4697c7e90fe67076b021
2017-04-23 19:16:22 +03:00
Leo Yuriev
1596bceff2 mdbx: rework mdbx_env_setup_locks()'s tasks.
Change-Id: I79953554a0b2cce8235ec5dbc83f183cb3fd56a0
2017-04-23 19:16:22 +03:00
Leo Yuriev
cd37b81cc5 mdbx: refine mdbx_txn_renew0(). 2017-04-23 19:16:22 +03:00