Commit Graph

2768 Commits

Author SHA1 Message Date
Leonid Yuriev
3758e7697e mdbx: more checks inside mdbx_cursor_close().
Related to https://github.com/erthink/libmdbx/issues/146

Change-Id: I7b90a0e515aa6320b0e89ec52fe01bc0be126071
2020-12-15 15:07:42 +03:00
Leonid Yuriev
315ef41455 mdbx: use MDBX_PGL_LIMIT for DPL as a soft-limit and options.dp_limit as spilling-pressure to avoid false-positive MDBX_TXN_FULL.
Change-Id: I0caae71d044cefa2f36286c2ae9fe14e356a6b51
2020-12-06 14:42:13 +03:00
Leonid Yuriev
c1210d7c73 mdbx: refine mdbx_page_alloc() to avoid MDBX_TXN_FULL.
More for https://github.com/erthink/libmdbx/issues/123

Change-Id: I527b7e9507f4c2aa565f1567924cfb1c477966dd
2020-12-06 14:39:19 +03:00
Leonid Yuriev
a4df0acb00 mdbx: more logging for FD < STDERR_FILENO.
More for https://github.com/erthink/libmdbx/issues/144

Change-Id: I8dcd3cab8a39b16a1836c88ec92b30d6b0de8b51
2020-12-06 14:35:38 +03:00
Leonid Yuriev
c4944a58d5 mdbx: avoid spelling errors (cosmetics).
Change-Id: Ibf14e003ba740ba757b5403e92fd220e68930ce4
2020-12-05 22:51:45 +03:00
Leonid Yuriev
d116e7235a mdbx-posix: safeguard for DB corruption in case FD <= STDERR_FILENO.
Resolve https://github.com/erthink/libmdbx/issues/144

Change-Id: Ic3467338d494b021fa0d1863c8227976d0a2bcd5
2020-12-05 22:51:45 +03:00
Leonid Yuriev
3e7459b428 mdbx-windows: fix mdbx_realloc() for nullptr and MDBX_AVOID_CRT=ON.
Change-Id: I129221186d65254da5b1d84747e5c59d53864b70
2020-12-05 09:45:03 +03:00
Leonid Yuriev
338de2e1fb mdbx: refine mdbx_page_spill().
More for https://github.com/erthink/libmdbx/issues/132.

Change-Id: I40788c13b54453c17d5e5dae6c3d2f93226f8e00
2020-12-04 15:38:26 +03:00
Leonid Yuriev
12ee5e6cac mdbx: minor refine mdbx_page_flush().
More for https://github.com/erthink/libmdbx/issues/132.

Change-Id: I703e1359d859b8901fcbdf57211f7805cd6d5ff1
2020-12-04 15:38:26 +03:00
Leonid Yuriev
262e4981db mdbx: refactor mdbx_pages_xkeep().
More for https://github.com/erthink/libmdbx/issues/132.

Change-Id: Ie81b5c9efafa06e61d6d983769ff1f11a0e59a1e
2020-12-04 15:38:15 +03:00
Leonid Yuriev
df8260e29a mdbx: add mdbx_pnl_purge_odd().
More for https://github.com/erthink/libmdbx/issues/132.

Change-Id: I1926ee48785f30b82e827a8e1e69be1e7a0a209f
2020-12-04 15:38:15 +03:00
Leonid Yuriev
87c765c7be mdbx: fix/check lower border for dirty-pages list's limits/options.
More for https://github.com/erthink/libmdbx/issues/128.

Change-Id: Ie9dccee5cf90d26968db04f4f6d9afa3018fcf0e
2020-12-04 15:38:15 +03:00
Leonid Yuriev
0b02e080c4 mdbx: use pnl_xmerge() to merge spill-pages lists.
More for https://github.com/erthink/libmdbx/issues/132.

Change-Id: I1f84a2be1fa5da6cf0e2783ff56f7a4c20782a7f
2020-12-04 15:38:15 +03:00
Leonid Yuriev
d9bd306da3 mdbx-load: add -p option (purge subDB).
Change-Id: I678950c99c8b4aea1add4ce548c9b4fe4ab8bfe6
2020-12-03 21:52:22 +03:00
Leonid Yuriev
90309ec0bf mdbx: merge branch 'master' into devel.
Change-Id: Ib1d07cf6eb2e4c76b3be969b28bf2317cb326835
2020-12-03 18:42:10 +03:00
Leonid Yuriev
eb31b765a5 mdbx-tools: use MDBX_SAFE_NOSYNC | MDBX_ACCEDE inside mdbx_load.
Change-Id: Id5a3a447d6b80b96a97ec76c258825583546d791
2020-12-03 17:58:21 +03:00
Leonid Yuriev
76716c23b0 mdbx: add MDBX_opt_txn_dp_initial & MDBX_opt_txn_dp_limit.
More for https://github.com/erthink/libmdbx/issues/128

Change-Id: I4be3c7476e9ce16ed4f27691d0df653552930e3c
2020-12-03 17:17:30 +03:00
Leonid Yuriev
84235c7903 mdbx: drop unnecessary MDBX_HUGE_TRANSACTIONS option.
Change-Id: I86116c2bada268d6e0a5c97deff808aaffe3e599
2020-12-03 17:17:28 +03:00
Leonid Yuriev
c5061f9289 mdbx: dynamic dirty-page-list size.
Resolve	https://github.com/erthink/libmdbx/issues/123

Change-Id: I5d9b2d8c2259132617fd2b226b9d9a6790fcad88
2020-12-03 17:17:07 +03:00
Leonid Yuriev
6a5cec1f87 mdbx: refine mdbx_dpl_search().
Change-Id: I8acc87d9c2f306c981ad42cc859d32b7cfd7330e
2020-12-03 10:25:36 +03:00
Leonid Yuriev
d8d89cca7d mdbx: refactor dirty-page-list (preparation to rework).
Change-Id: Ib52bb52f73ef1d31f55838d879de081fc0a140c2
2020-12-03 10:25:36 +03:00
Leonid Yuriev
7210f994fb mdbx-windows: fix mdbx_realloc() for nullptr and MDBX_AVOID_CRT=ON.
Change-Id: I129221186d65254da5b1d84747e5c59d53864b70
2020-12-03 10:25:24 +03:00
Leonid Yuriev
e6eeb17030 mdbx: update ChangeLog.
Change-Id: I694056afe448883b16b8ec769d8a560040044ae5
2020-12-03 09:36:45 +03:00
Leonid Yuriev
a987301204 mdbx++: refine abort/commit.
More for https://github.com/erthink/libmdbx/pull/143

Change-Id: I07de4c43d039f904495f0f4caf586d8764174dcf
2020-12-03 09:32:25 +03:00
sasgas
5c693ccd96 mdbx++: fix missing null assignment.
Resolves https://github.com/erthink/libmdbx/pull/143

Change-Id: Ifb88cebe2f07297a2f6cf70e31ae6abe61a0c0b2
2020-12-03 09:17:36 +03:00
Leonid Yuriev
5636dbf12b mdbx: add mdbx_dpl_alloc() & mdbx_dpl_free().
Change-Id: I0f05ff02d4a45d02faa1076cbb1a2a8e17b7e2b9
2020-12-02 20:07:59 +03:00
Leonid Yuriev
c530c83337 mdbx: drop me_dirtylist field.
Change-Id: I93ee802c1b6ef5c6a2443e19754e1a20dbe0aea9
2020-12-02 20:07:59 +03:00
Leonid Yuriev
0b62453f6d mdbx: introduce MDBX_PGL_LIMIT.
Change-Id: Ifab1156ec22fb0908284298393f52c09c0731829
2020-12-02 20:07:59 +03:00
Leonid Yuriev
4cd08f0ef9 mdbx: add MDBX_opt_rp_augment_limit.
More for https://github.com/erthink/libmdbx/issues/128

Change-Id: I94f3350b72ad11d0b586aaef1574f7654223bc40
2020-12-02 20:07:55 +03:00
Leonid Yuriev
461ba000e3 mdbx: add MDBX_opt_dp_reserve_limit.
More for https://github.com/erthink/libmdbx/issues/128

Change-Id: I620b6dcd833fcff19680770974f32431ab91c0ab
2020-12-02 12:42:16 +03:00
Leonid Yuriev
42019e0b8d mdbx: add dp_reserve_limit with 1024 default.
More for  More for https://github.com/erthink/libmdbx/issues/128

Change-Id: Ia92b1aea58640f183202495e3f6e2d531057afd2
2020-12-02 12:42:16 +03:00
Leonid Yuriev
8f60050991 mdbx: initial support for set/get_options.
Initial for https://github.com/erthink/libmdbx/issues/128

Change-Id: I540784c8b54d7443748e3c95f89a7390b818c811
2020-12-02 12:42:13 +03:00
Leonid Yuriev
a8ed534cd3 mdbx: avoid 4-byte aligned (i.e. unaligned) access to 64-bit integers.
Historically, the page header provides 4-byte data alignment.
Therefore, unfortunately, the meta page data is also aligned on a 4-byte boundary, but contains 64-bit values.

This commit eliminates potentially unsafe access (SPARC, MIPS, etc) to these 64-bit values aligned on a 4-byte boundary.
Thus, a build with the `-fsanitize=undefined` now passes the tests both with CLANG 11 and GCC 10.

Change-Id: Ie441103e53ed96fd40507d8c0be0689e3fee69f5
2020-12-01 20:23:23 +03:00
Leonid Yuriev
adcb052915 mdbx: fix minor MSVC warnings. 2020-11-29 06:04:55 +03:00
Leonid Yuriev
3d41abd318 mdbx: use unsigned for DEFINE_ENUM_FLAG_OPERATORS to be more consistent with C/C++standards.
Change-Id: Iadbd59694606b577602f8e86ec371c3599639b19
2020-11-29 05:49:20 +03:00
Leonid Yuriev
5282f99bd6 mdbx-cmake: allow predefine MDBX_C_STANDARD.
Change-Id: I9895c8afff7fd80bd25292faef0e88fe2002ff21
2020-11-29 05:49:20 +03:00
Leonid Yuriev
9e5ea95f0f mdbx-packages: update package/patch for buildroot.
Change-Id: I27c1ad3843d712823258da1067f9fd79fa255853
2020-11-29 05:49:20 +03:00
Leonid Yuriev
8381ed1d2e mdbx: minor fix comment copy&paste typo.
Change-Id: I34d4eaebdcf7a4e3bda7f0834e6e14d17e26f7db
2020-11-29 05:49:20 +03:00
Leonid Yuriev
28a275cba2 mdbx: minor refine MDBX_CXX#_CONSTEXPR macros for Doxygen.
Change-Id: I35d8cb6badacdee089e2382b34524e108423acfc
2020-11-29 05:49:20 +03:00
Leonid Yuriev
f74d0f5526 mdbx: minor fix README.
Change-Id: I7dbe7204b9a1561f11ff12bb165f5c6e2137d7b9
2020-11-29 05:49:20 +03:00
Leonid Yuriev
2dab009e76 mdbx-windows: handling EXCEPTION_POSSIBLE_DEADLOCK.
Change-Id: If42c7833e9c4e02fef25634e69c0bd2e926686c1
2020-11-29 05:49:15 +03:00
Leonid Yuriev
092ab094c4
mdbx: release v0.9.2
Acknowledgements:
-----------------

 - Jens Alfke (Mobile Architect at [Couchbase](https://www.couchbase.com/)) for [NimDBX](https://github.com/snej/nimdbx).
 - Clément Renault (CTO at [MeiliSearch](https://www.meilisearch.com/)) for [mdbx-rs](https://github.com/Kerollmops/mdbx-rs).
 - Alex Sharov (Go-Lang Teach Lead at [TurboGeth/Ethereum](https://ethereum.org/)) for an extreme test cases and bug reporting.
 - George Hazan (CTO at [Miranda NG](https://www.miranda-ng.org/)) for bug reporting.
 - [Positive Technologies](https://www.ptsecurity.com/) for funding and [The Standoff](https://standoff365.com/).

Added features:
---------------

 - Provided package for [buildroot](https://buildroot.org/).
 - Binding for Nim is [available](https://github.com/snej/nimdbx) now by Jens Alfke.
 - Added `mdbx_env_delete()` for deletion an environment files in a proper and multiprocess-safe way.
 - Added `mdbx_txn_commit_ex()` with collecting latency information.
 - Fast completion pure nested transactions.
 - Added `LIBMDBX_INLINE_API` macro and inline versions of some API functions.
 - Added `mdbx_cursor_copy()` function.
 - Extended tests for checking cursor tracking.
 - Added `MDBX_SET_LOWERBOUND` operation for `mdbx_cursor_get()`.

Fixes:
------

 - Fixed missing installation of `mdbx.h++`.
 - Fixed use of obsolete `__noreturn`.
 - Fixed use of `yield` instruction on ARM if unsupported.
 - Added pthread workaround for buggy toolchain/cmake/buildroot.
 - Fixed use of `pthread_yield()` for non-GLIBC.
 - Fixed use of `RegGetValueA()` on Windows 2000/XP.
 - Fixed use of `GetTickCount64()` on Windows 2000/XP.
 - Fixed opening DB on a network shares (in the exclusive mode).
 - Fixed copy&paste typos.
 - Fixed minor false-positive GCC warning.
 - Added workaround for broken `DEFINE_ENUM_FLAG_OPERATORS` from Windows SDK.
 - Fixed cursor state after multimap/dupsort repeated deletes (https://github.com/erthink/libmdbx/issues/121).
 - Added `SIGPIPE` suppression for internal thread during `mdbx_env_copy()`.
 - Fixed extra-rare `MDBX_KEY_EXIST` error during `mdbx_commit()` (https://github.com/erthink/libmdbx/issues/131).
 - Fixed spilled pages checking (https://github.com/erthink/libmdbx/issues/126).
 - Fixed `mdbx_load` for 'plain text' and without `-s name` cases (https://github.com/erthink/libmdbx/issues/136).
 - Fixed save/restore/commit of cursors for nested transactions.
 - Fixed cursors state in rare/special cases (move next beyond end-of-data, after deletion and so on).
 - Added workaround for MSVC 19.28 (Visual Studio 16.8) (but may still hang during compilation).
 - Fixed paranoidal Clang C++ UB for bitwise operations with flags defined by enums.
 - Fixed large pages checking (for compatibility and to avoid false-positive errors from `mdbx_chk`).
 - Added workaround for Wine (https://github.com/miranda-ng/miranda-ng/issues/1209).
 - Fixed `ERROR_NOT_SUPPORTED` while opening DB by UNC pathnames (https://github.com/miranda-ng/miranda-ng/issues/2627).
 - Added handling `EXCEPTION_POSSIBLE_DEADLOCK` condition for Windows.

TODO for v0.9.3:
----------------

 - Engage new terminology (https://github.com/erthink/libmdbx/issues/137).
 - Rework/speedup the implementation of the dirty page list (lazy compactification, lazy sorting via merge).
 - Resolve few TODOs (https://github.com/erthink/libmdbx/issues/123, https://github.com/erthink/libmdbx/issues/124, https://github.com/erthink/libmdbx/issues/127, https://github.com/erthink/libmdbx/issues/128, https://github.com/erthink/libmdbx/issues/132, https://github.com/erthink/libmdbx/issues/115).
 - Finalize C++ API (few typos and trivia bugs are still likely for now).
 - Packages for ROSA Linux, ALT Linux, Fedora/RHEL, Debian/Ubuntu.

Change-Id: I1f0ac4505052f169e36b326243e5d56497b9c037
Signed-off-by: Leonid Yuriev <leo@yuriev.ru>
2020-11-27 10:09:03 +03:00
Leonid Yuriev
6d2914c99b mdbx: minor/insignificant fix checking mdbx_rdt_lock() result for cases mutex-recovery after EOWNERDEAD.
Change-Id: Ia5e13ea0d72afc97f7d678832a765a192dfacdff
2020-11-25 17:55:29 +03:00
Leonid Yuriev
e14c6646f6 mdbx-ci: use ubuntu-20.04 for GitHub Actions (fix https://erthink.github.io/libmdbx content).
Change-Id: If5af0b7022c56e45a66742feca199fa798829e54
2020-11-25 12:24:54 +03:00
Leonid Yuriev
fff793be16 mdbx: update ChangeLog.
Change-Id: I842ca80064ae8d3a77dfc45ec7882f0655587ead
2020-11-24 15:53:20 +03:00
Leonid Yuriev
2a740d3807 mdbx-windows: fix ERROR_NOT_SUPPORTED while opening UNC pathnames.
Resolve https://github.com/miranda-ng/miranda-ng/issues/2627

Change-Id: Iec06554dfcb9e0db215662d93fb5824cfe9ac7ef
2020-11-24 15:53:14 +03:00
Leonid Yuriev
659fbb3df7 mdbx-doc: fix MDBX_INTEGERDUP description.
Resolve https://github.com/erthink/libmdbx/issues/140

Change-Id: Ifb24cbbc9cba7d0c09da8e99e6d87412d151d093
2020-11-24 07:50:52 +03:00
Leonid Yuriev
23fd4444b5 mdbx-windows: remap whole section for mmap-resize if NtExtendSection() not available (Wine).
Some workaround for https://github.com/miranda-ng/miranda-ng/issues/1209

Change-Id: I8ce3cb0fb8bdc9bd36a05ab53343efbe4080d1f3
2020-11-23 10:47:07 +03:00
Leonid Yuriev
ddee04d991 mdbx: set DXB/LCK files position outside of a data.
Set the position in files outside of the data to avoid corruption
due to erroneous use of file descriptors in the application code.

Change-Id: I8750f6e726edaddee254885756d5f91576a6bcb1
2020-11-23 09:25:24 +03:00
Leonid Yuriev
f76bf72021 mdbx-windows: retry resize-mmap always with reservation (less chance to failure).
Change-Id: I472651f6906b88ba674868e0cd6c81318bfc255b
2020-11-20 11:52:44 +03:00