Commit Graph

2687 Commits

Author SHA1 Message Date
Leonid Yuriev
8f60050991 mdbx: initial support for set/get_options.
Initial for https://github.com/erthink/libmdbx/issues/128

Change-Id: I540784c8b54d7443748e3c95f89a7390b818c811
2020-12-02 12:42:13 +03:00
Leonid Yuriev
a8ed534cd3 mdbx: avoid 4-byte aligned (i.e. unaligned) access to 64-bit integers.
Historically, the page header provides 4-byte data alignment.
Therefore, unfortunately, the meta page data is also aligned on a 4-byte boundary, but contains 64-bit values.

This commit eliminates potentially unsafe access (SPARC, MIPS, etc) to these 64-bit values aligned on a 4-byte boundary.
Thus, a build with the `-fsanitize=undefined` now passes the tests both with CLANG 11 and GCC 10.

Change-Id: Ie441103e53ed96fd40507d8c0be0689e3fee69f5
2020-12-01 20:23:23 +03:00
Leonid Yuriev
adcb052915 mdbx: fix minor MSVC warnings. 2020-11-29 06:04:55 +03:00
Leonid Yuriev
3d41abd318 mdbx: use unsigned for DEFINE_ENUM_FLAG_OPERATORS to be more consistent with C/C++standards.
Change-Id: Iadbd59694606b577602f8e86ec371c3599639b19
2020-11-29 05:49:20 +03:00
Leonid Yuriev
5282f99bd6 mdbx-cmake: allow predefine MDBX_C_STANDARD.
Change-Id: I9895c8afff7fd80bd25292faef0e88fe2002ff21
2020-11-29 05:49:20 +03:00
Leonid Yuriev
9e5ea95f0f mdbx-packages: update package/patch for buildroot.
Change-Id: I27c1ad3843d712823258da1067f9fd79fa255853
2020-11-29 05:49:20 +03:00
Leonid Yuriev
8381ed1d2e mdbx: minor fix comment copy&paste typo.
Change-Id: I34d4eaebdcf7a4e3bda7f0834e6e14d17e26f7db
2020-11-29 05:49:20 +03:00
Leonid Yuriev
28a275cba2 mdbx: minor refine MDBX_CXX#_CONSTEXPR macros for Doxygen.
Change-Id: I35d8cb6badacdee089e2382b34524e108423acfc
2020-11-29 05:49:20 +03:00
Leonid Yuriev
f74d0f5526 mdbx: minor fix README.
Change-Id: I7dbe7204b9a1561f11ff12bb165f5c6e2137d7b9
2020-11-29 05:49:20 +03:00
Leonid Yuriev
2dab009e76 mdbx-windows: handling EXCEPTION_POSSIBLE_DEADLOCK.
Change-Id: If42c7833e9c4e02fef25634e69c0bd2e926686c1
2020-11-29 05:49:15 +03:00
Leonid Yuriev
092ab094c4
mdbx: release v0.9.2
Acknowledgements:
-----------------

 - Jens Alfke (Mobile Architect at [Couchbase](https://www.couchbase.com/)) for [NimDBX](https://github.com/snej/nimdbx).
 - Clément Renault (CTO at [MeiliSearch](https://www.meilisearch.com/)) for [mdbx-rs](https://github.com/Kerollmops/mdbx-rs).
 - Alex Sharov (Go-Lang Teach Lead at [TurboGeth/Ethereum](https://ethereum.org/)) for an extreme test cases and bug reporting.
 - George Hazan (CTO at [Miranda NG](https://www.miranda-ng.org/)) for bug reporting.
 - [Positive Technologies](https://www.ptsecurity.com/) for funding and [The Standoff](https://standoff365.com/).

Added features:
---------------

 - Provided package for [buildroot](https://buildroot.org/).
 - Binding for Nim is [available](https://github.com/snej/nimdbx) now by Jens Alfke.
 - Added `mdbx_env_delete()` for deletion an environment files in a proper and multiprocess-safe way.
 - Added `mdbx_txn_commit_ex()` with collecting latency information.
 - Fast completion pure nested transactions.
 - Added `LIBMDBX_INLINE_API` macro and inline versions of some API functions.
 - Added `mdbx_cursor_copy()` function.
 - Extended tests for checking cursor tracking.
 - Added `MDBX_SET_LOWERBOUND` operation for `mdbx_cursor_get()`.

Fixes:
------

 - Fixed missing installation of `mdbx.h++`.
 - Fixed use of obsolete `__noreturn`.
 - Fixed use of `yield` instruction on ARM if unsupported.
 - Added pthread workaround for buggy toolchain/cmake/buildroot.
 - Fixed use of `pthread_yield()` for non-GLIBC.
 - Fixed use of `RegGetValueA()` on Windows 2000/XP.
 - Fixed use of `GetTickCount64()` on Windows 2000/XP.
 - Fixed opening DB on a network shares (in the exclusive mode).
 - Fixed copy&paste typos.
 - Fixed minor false-positive GCC warning.
 - Added workaround for broken `DEFINE_ENUM_FLAG_OPERATORS` from Windows SDK.
 - Fixed cursor state after multimap/dupsort repeated deletes (https://github.com/erthink/libmdbx/issues/121).
 - Added `SIGPIPE` suppression for internal thread during `mdbx_env_copy()`.
 - Fixed extra-rare `MDBX_KEY_EXIST` error during `mdbx_commit()` (https://github.com/erthink/libmdbx/issues/131).
 - Fixed spilled pages checking (https://github.com/erthink/libmdbx/issues/126).
 - Fixed `mdbx_load` for 'plain text' and without `-s name` cases (https://github.com/erthink/libmdbx/issues/136).
 - Fixed save/restore/commit of cursors for nested transactions.
 - Fixed cursors state in rare/special cases (move next beyond end-of-data, after deletion and so on).
 - Added workaround for MSVC 19.28 (Visual Studio 16.8) (but may still hang during compilation).
 - Fixed paranoidal Clang C++ UB for bitwise operations with flags defined by enums.
 - Fixed large pages checking (for compatibility and to avoid false-positive errors from `mdbx_chk`).
 - Added workaround for Wine (https://github.com/miranda-ng/miranda-ng/issues/1209).
 - Fixed `ERROR_NOT_SUPPORTED` while opening DB by UNC pathnames (https://github.com/miranda-ng/miranda-ng/issues/2627).
 - Added handling `EXCEPTION_POSSIBLE_DEADLOCK` condition for Windows.

TODO for v0.9.3:
----------------

 - Engage new terminology (https://github.com/erthink/libmdbx/issues/137).
 - Rework/speedup the implementation of the dirty page list (lazy compactification, lazy sorting via merge).
 - Resolve few TODOs (https://github.com/erthink/libmdbx/issues/123, https://github.com/erthink/libmdbx/issues/124, https://github.com/erthink/libmdbx/issues/127, https://github.com/erthink/libmdbx/issues/128, https://github.com/erthink/libmdbx/issues/132, https://github.com/erthink/libmdbx/issues/115).
 - Finalize C++ API (few typos and trivia bugs are still likely for now).
 - Packages for ROSA Linux, ALT Linux, Fedora/RHEL, Debian/Ubuntu.

Change-Id: I1f0ac4505052f169e36b326243e5d56497b9c037
Signed-off-by: Leonid Yuriev <leo@yuriev.ru>
2020-11-27 10:09:03 +03:00
Leonid Yuriev
6d2914c99b mdbx: minor/insignificant fix checking mdbx_rdt_lock() result for cases mutex-recovery after EOWNERDEAD.
Change-Id: Ia5e13ea0d72afc97f7d678832a765a192dfacdff
2020-11-25 17:55:29 +03:00
Leonid Yuriev
e14c6646f6 mdbx-ci: use ubuntu-20.04 for GitHub Actions (fix https://erthink.github.io/libmdbx content).
Change-Id: If5af0b7022c56e45a66742feca199fa798829e54
2020-11-25 12:24:54 +03:00
Leonid Yuriev
fff793be16 mdbx: update ChangeLog.
Change-Id: I842ca80064ae8d3a77dfc45ec7882f0655587ead
2020-11-24 15:53:20 +03:00
Leonid Yuriev
2a740d3807 mdbx-windows: fix ERROR_NOT_SUPPORTED while opening UNC pathnames.
Resolve https://github.com/miranda-ng/miranda-ng/issues/2627

Change-Id: Iec06554dfcb9e0db215662d93fb5824cfe9ac7ef
2020-11-24 15:53:14 +03:00
Leonid Yuriev
659fbb3df7 mdbx-doc: fix MDBX_INTEGERDUP description.
Resolve https://github.com/erthink/libmdbx/issues/140

Change-Id: Ifb24cbbc9cba7d0c09da8e99e6d87412d151d093
2020-11-24 07:50:52 +03:00
Leonid Yuriev
23fd4444b5 mdbx-windows: remap whole section for mmap-resize if NtExtendSection() not available (Wine).
Some workaround for https://github.com/miranda-ng/miranda-ng/issues/1209

Change-Id: I8ce3cb0fb8bdc9bd36a05ab53343efbe4080d1f3
2020-11-23 10:47:07 +03:00
Leonid Yuriev
ddee04d991 mdbx: set DXB/LCK files position outside of a data.
Set the position in files outside of the data to avoid corruption
due to erroneous use of file descriptors in the application code.

Change-Id: I8750f6e726edaddee254885756d5f91576a6bcb1
2020-11-23 09:25:24 +03:00
Leonid Yuriev
f76bf72021 mdbx-windows: retry resize-mmap always with reservation (less chance to failure).
Change-Id: I472651f6906b88ba674868e0cd6c81318bfc255b
2020-11-20 11:52:44 +03:00
Leonid Yuriev
537e3d18ae mdbx: minor clarify mmap-resize logging.
Change-Id: Ibb6e4b8ee84d03a22ad0e0d1b437989739d43bb9
2020-11-20 10:38:04 +03:00
Leonid Yuriev
75e1da1f2d mdbx-test: fix speculum-check iterator.
Change-Id: Idaeea908fdcd7b2c1248ae10eeb7f394d69bb987
2020-11-19 17:28:47 +03:00
Leonid Yuriev
1bbf20bf39 mdbx: refine multi-thread flipping of buffers during env-copy.
Change-Id: Id132c1af0e1131da70ab1b35bce9f6a6548edbe3
2020-11-19 11:59:42 +03:00
Leonid Yuriev
9054b25441 mdbx: fix fast completion of nested txn for case ones was open present DBIs.
Change-Id: I87d28d1da5f72cc7ad77f8dd3f9a181f7eaa0024
2020-11-19 09:26:33 +03:00
Leonid Yuriev
862cfb9a3b mdbx-windows: refine WSL1 detection.
Try to fix https://github.com/snej/nimdbx/issues/1

Change-Id: Iec1c77c82ced8095e3f4e03a27a58e0bba3df76b
2020-11-18 22:38:26 +03:00
Leonid Yuriev
87161f5920 mdbx-make: add stub/guard for obsolete git.
Change-Id: I8f4837ef29be37d40ae9ccf851bebe0a6734f523
2020-11-18 16:24:40 +03:00
Leonid Yuriev
9668aa58ef mdbx-make: explicitly fetch git-tags on make dist.
Change-Id: I19d96dba645af0302f2b91870e7311336c4e5083
2020-11-18 15:11:53 +03:00
Leonid Yuriev
0ca80a9188 mdbx: update README & ChangeLog, scheduled v0.9.2
Change-Id: I019d72989362951a50df0e0561713c6d0dd9e74d
2020-11-18 12:40:49 +03:00
Leonid Yuriev
26767a5e06 mdbx: relax big-page size checking for compatibility.
i.e. allow an overflow page be larger than required by placed data.
2020-11-18 03:42:05 +03:00
Leonid Yuriev
b47a44582a mdbx: merge branch 'devel'.
Change-Id: I20c41204adaac9ccb13e59e63e829246158b6d41
2020-11-17 17:11:00 +03:00
Leonid Yuriev
5619fefe0a mdbx-test: fix actor_poll() against the EBADF error from pipe. 2020-11-17 15:40:02 +03:00
Leonid Yuriev
94fae97f88 mdbx: avoid paranoid CLANG's enum ops UB. 2020-11-17 15:00:51 +03:00
Leonid Yuriev
11fde67edc mdbx-cmake: crutch for MSVC 19.28 (Visual Studio 16.8).
Change-Id: I8fdb665d103031dc5e0c4b7963bc5eb65853a82f
2020-11-17 13:59:29 +03:00
Leonid Yuriev
015ed5bc98 mdbx-cmake: minor fix MDBX_BUILD_FLAGS preparation.
Change-Id: I91eebdffd06b4239ccc2dccd0931b3f82f0b5f52
2020-11-17 13:51:10 +03:00
Leonid Yuriev
b1008b1256 mdbx-test: use MDBX_SET_LOWERBOUND.
Change-Id: I4f6efab69996d4678a78024337a6698a65c2386b
2020-11-17 08:13:46 +03:00
Leonid Yuriev
96c2a56aa1 mdbx: use MDBX_SET_LOWERBOUND in mdbx_get_equal_or_great().
Change-Id: I5dd72fe82bd15938afc60cbc0f92e23c00f0d344
2020-11-17 08:13:46 +03:00
Leonid Yuriev
5e02e7fb56 mdbx: add MDBX_SET_LOWERBOUND for mdbx_cursor_get().
Change-Id: I3638fdd10be8dfe128c43b465e9ca71f89175b3e
2020-11-17 08:13:46 +03:00
Leonid Yuriev
9ea6922a2f mdbx: minor simplify mdbx_cursor_set().
Change-Id: I034f396368024af21e8ee741c13a28c9bc277121
2020-11-17 07:59:51 +03:00
Leonid Yuriev
d9b919d5a6 mdbx-ci: update doxygen-github-pages action.
Change-Id: I8a34588413026baea429a38402013cb9756d3bb2
2020-11-17 02:08:40 +03:00
Leonid Yuriev
89fbaa4de5 mdbx-ci: update spelling wordlist.
Change-Id: I95f75ad334464f8f88226fd496354cf4633a1091
2020-11-17 01:50:04 +03:00
Leonid Yuriev
d86759da22 mdbx: add reference to Nim bindings.
Announce https://forum.nim-lang.org/t/7087

Related to https://github.com/erthink/libmdbx/issues/137

Change-Id: I46b2aa6d262c9a83518ce713997254c98c0cd915
2020-11-17 01:33:37 +03:00
Leonid Yuriev
56a52b0b01 mdbx: merge branch 'fix-131' (early part) into devel.
Change-Id: I0df5ffa9f06da1196a60d5a7c68fed59b7027e4c
2020-11-16 07:07:53 +03:00
Leonid Yuriev
7cdbe1badb mdbx-test: extending speculum mode for cursors tracking verification.
Change-Id: I44786efcee6feb1c7d414c925717d08ed9d94e20
2020-11-16 07:07:39 +03:00
Leonid Yuriev
149b3d09e6 mdbx: don't export DBIs from nested txn.
Change-Id: I8c354ac7f889debe2fcf830263cb060253887652
2020-11-16 07:07:39 +03:00
Leonid Yuriev
1c925a0f2e mdbx: minor fix/avoid assertion inside mdbx_cursor_set().
Change-Id: I2cc38da698765b3eaa2bb575e16505d5d9438431
2020-11-16 07:07:39 +03:00
Leonid Yuriev
0c3deac9db mdbx: fix cursors-EOF after search.
Change-Id: Ie578611b64cca8dbcc00f958510143e8d1dc262a
2020-11-16 07:07:39 +03:00
Leonid Yuriev
21bbba82fb mdbx: minor fix cursor_on_first/last for empty sub-db.
Change-Id: I68c2e1bd28e62c9512bf8ef711fe21573e3450b9
2020-11-16 07:07:39 +03:00
Leonid Yuriev
4ea2bea22e mdbx: fix save/restore/commit cursors for nested txn.
Change-Id: Ibaf356bbc631ca2e41058787bd3e4aaaa8bc085a
2020-11-16 07:07:38 +03:00
Leonid Yuriev
e328c1f829 mdbx: refactor/move mdbx_cursor_count().
Change-Id: I5487342013dfc3a9cf12d9081471ace0c37de23d
2020-11-16 07:07:38 +03:00
Leonid Yuriev
ca115dd6a4 mdbx: refactor/rename/refine tw.cursors internals.
Change-Id: Ie46d15f52a9d7365b52534a630754a31d3005a69
2020-11-16 07:07:38 +03:00
Leonid Yuriev
9a2dbb845c mdbx: refactor/refine cursor next/prev.
Change-Id: I0bfe0fed62e137e933027ba90cec23dbf1a6310b
2020-11-16 07:07:38 +03:00