Commit Graph

3564 Commits

Author SHA1 Message Date
Леонид Юрьев (Leonid Yuriev)
474391c83c mdbx: поддержка асинхронного ввода-вывода для Windows и подготовка к io_ring (объединённые коммиты и исправления). 2022-10-06 20:35:07 +03:00
Леонид Юрьев (Leonid Yuriev)
9f64e2a10c mdbx: правка спилинга для устранения срабатывания проверочных утверждений в отладочных сборках. 2022-10-01 01:38:08 +03:00
Леонид Юрьев (Leonid Yuriev)
41b918f1fc mdbx: исправление проверочного утверждения внутри mdbx_txn_abort() для ошибочных транзакций. 2022-10-01 01:35:08 +03:00
Леонид Юрьев (Leonid Yuriev)
00515d50a9 mdbx: исправление проверочного утверждения в page_retire_ex(). 2022-10-01 01:33:48 +03:00
Леонид Юрьев (Leonid Yuriev)
32a3674dc8 mdbx: return MDBX_PROBLEM insted of MDBX_CORRUPTED on coherence troubles. 2022-09-22 19:48:44 +03:00
Леонид Юрьев (Leonid Yuriev)
f51ace3db8 mdbx-windows: always call debugger if it present when assertion check failed. 2022-09-22 18:27:28 +03:00
Леонид Юрьев (Leonid Yuriev)
beda291692 mdbx-windows: fix nasty clz() (i.e. using _BitScanReverse() bug. 2022-09-22 18:27:28 +03:00
Леонид Юрьев (Leonid Yuriev)
fe20de136c mdbx: require linux >= 4.0 2022-09-22 18:27:23 +03:00
Леонид Юрьев (Leonid Yuriev)
cf8540d84e mdbx: minor refine mdbx_env_create(). 2022-09-22 18:26:28 +03:00
Леонид Юрьев (Leonid Yuriev)
bec9312df5 mdbx: more off/on for clang-format. 2022-09-22 17:06:32 +03:00
Леонид Юрьев (Leonid Yuriev)
a089f73002 mdbx: fix minor MinGW warning. 2022-09-13 11:39:55 +03:00
Леонид Юрьев (Leonid Yuriev)
c4efa8dce8 mdbx: update ChangeLog. 2022-09-09 19:01:59 +03:00
Леонид Юрьев (Leonid Yuriev)
2d5438d2c2 mdbx: fix regression ASAN/Valgring-enabled builds. 2022-09-06 13:03:04 +03:00
Леонид Юрьев (Leonid Yuriev)
29da09328e mdbx: removed description of deprecated usage of MDBX_NODUPDATA. 2022-09-06 13:03:04 +03:00
Леонид Юрьев (Leonid Yuriev)
52cb6b90a7 mdbx: fix extra check for MDBX_APPENDDUP. 2022-09-02 02:02:33 +03:00
Леонид Юрьев (Leonid Yuriev)
2d7c25b263 mdbx: minor fix extra ensure/assertion check of oldest_reader inside txn_end(). 2022-09-02 01:46:11 +03:00
Леонид Юрьев (Leonid Yuriev)
3230fb5788 mdbx: update ChangeLog. 2022-08-26 19:27:17 +03:00
Леонид Юрьев (Leonid Yuriev)
b73727d73e mdbx: add MDBX_HAVE_BUILTIN_CPU_SUPPORTS build option. 2022-08-26 19:17:09 +03:00
Леонид Юрьев (Leonid Yuriev)
b36a07a512
mdbx: release v0.12.1 (Positive Proxima)
The planned frontward release with new superior features on the day of 20 anniversary of [Positive Technologies](https://ptsecurty.com).

New:
----

 - The `Big Foot` feature which significantly reduces GC overhead for processing large lists of retired pages from huge transactions.
   Now _libmdbx_ avoid creating large chunks of PNLs (page number lists) which required a long sequences of free pages, aka large/overflow pages.
   Thus avoiding searching, allocating and storing such sequences inside GC.
 - Improved hot/online validation and checking of database pages both for more robustness and performance.
 - New solid and fast method to latch meta-pages called `Troika`.
   The minimum of memory barriers, reads, comparisons and conditional transitions are used.
 - New `MDBX_VALIDATION` environment options to extra validation of DB structure and pages content for carefully/safe handling damaged or untrusted DB.
 - Accelerated ×16/×8/×4 by AVX512/AVX2/SSE2/Neon implementations of search page sequences.
 - Added the `gcrtime_seconds16dot16` counter to the "Page Operation Statistics" that accumulates time spent for GC searching and reclaiming.
 - Copy-with-compactification now clears/zeroes unused gaps inside database pages.
 - The `C` and `C++` APIs has been extended and/or refined to simplify using `wchar_t` pathnames.
   On Windows the `mdbx_env_openW()`, `mdbx_env_get_pathW()`, `mdbx_env_copyW()`, `mdbx_env_open_for_recoveryW()` are available for now,
   but the `mdbx_env_get_path()` has been replaced in favor of `mdbx_env_get_pathW()`.
 - Added explicit error message for Buildroot's Microblaze toolchain maintainers.
 - Added `MDBX_MANAGE_BUILD_FLAGS` build options for CMake.
 - Speed-up internal `bsearch`/`lower_bound` implementation using branchless tactic, including workaround for CLANG x86 optimiser bug.
 - A lot internal refinement and micro-optimisations.
 - Internally counted volume of dirty pages (unused for now but for coming features).

Fixes:
------

 - Never use modern `__cxa_thread_atexit()` on Apple's OSes.
 - Don't check owner for finished transactions.
 - Fixed typo in `MDBX_EINVAL` which breaks MingGW builds with CLANG.

37 files changed, 7604 insertions(+), 7417 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2022-08-24 16:24:22 +03:00
Леонид Юрьев (Leonid Yuriev)
b60d8e78c3 mdbx: merge branch master into devel. 2022-08-22 21:46:54 +03:00
Леонид Юрьев (Leonid Yuriev)
aaa9112c83
mdbx: release v0.11.10
The stable bugfix release.
It is planned that this will be the last release of the v0.11 branch.

New:
----

 - The C++ API has been refined to simplify support for `wchar_t` in path names.
 - Added explicit error message for Buildroot's Microblaze toolchain maintainers.

Fixes:
------

 - Never use modern `__cxa_thread_atexit()` on Apple's OSes.
 - Use `MultiByteToWideChar(CP_THREAD_ACP)` instead of `mbstowcs()`.
 - Don't check owner for finished transactions.
 - Fixed typo in `MDBX_EINVAL` which breaks MingGW builds with CLANG.

Minors:
-------

 - Fixed variable name typo.
 - Using `ldd` to check used dso.
 - Added `MDBX_WEAK_IMPORT_ATTRIBUTE` macro.
 - Use current transaction geometry for untouched parameters when `env_set_geometry()` called within a write transaction.
 - Minor clarified `iov_page()` failure case.

14 files changed, 263 insertions(+), 252 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2022-08-22 13:32:24 +03:00
Леонид Юрьев (Leonid Yuriev)
26f52a19c3 mdbx: add explicit error message for Buildroot's Microblaze toolchain maintainers. 2022-08-22 12:59:42 +03:00
Леонид Юрьев (Leonid Yuriev)
5368551081 mdbx: minor clarify iov_page() failure case. 2022-08-22 12:59:42 +03:00
Леонид Юрьев (Leonid Yuriev)
0ccec20409 mdbx: don't deem meta pages with zero txnid equal.
Устранение крайне маловероятного регресса после перехода на мета-тройку:
 - процесс А открыает БД и читает мета-траницы для формирования тройки;
 - процесс Б постоянно коммитит новые транзакции;
 - есть шанс что процесс А при чтении разных мета страниц попадет на момент их обновления более одного раза,
   это может привести к ложной ошибке коллизии мета-страниц,
   так как для обновляемых мета-страниц будет виден нулевой номер транзакции.
2022-08-20 01:54:11 +03:00
Леонид Юрьев (Leonid Yuriev)
ceba040e32 mdbx: add meta_xyz_dump(). 2022-08-20 01:54:11 +03:00
Леонид Юрьев (Leonid Yuriev)
b617f25eaa mdbx: refine & rename internal xyz to troika. 2022-08-20 01:54:06 +03:00
Леонид Юрьев (Leonid Yuriev)
b759dfafd7 mdbx: counting large/overflow dirty pages (unused for now).
This is a basis for [Large/Overflow pages accounting for dirty-room](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/192).
2022-08-20 00:14:48 +03:00
Леонид Юрьев (Leonid Yuriev)
4cef1c2376 mdbx: avoid extra using F_ISSET() macro. 2022-08-18 01:39:06 +03:00
Леонид Юрьев (Leonid Yuriev)
08a8f844dc mdbx: ×4 ARM-Neon accelerated scan4seq(). 2022-08-18 01:10:27 +03:00
Леонид Юрьев (Leonid Yuriev)
8e2c276562 mdbx: merge branch 'master' into devel branch. 2022-08-18 01:04:35 +03:00
Леонид Юрьев (Leonid Yuriev)
4f02199648 mdbx: update ChangeLog. 2022-08-17 23:56:53 +03:00
Леонид Юрьев (Leonid Yuriev)
7b36f946cb mdbx: rework/speed up accessing to meta-pages, choosing and cache of ones (squashed). 2022-08-17 21:39:22 +03:00
Леонид Юрьев (Leonid Yuriev)
ef16dd2a22 mdbx: move global_ctor() to the end . 2022-08-17 21:31:11 +03:00
Леонид Юрьев (Leonid Yuriev)
f9ad835680 mdbx: drop E2K libc obsolete workarounds. 2022-08-17 21:29:51 +03:00
Леонид Юрьев (Leonid Yuriev)
9b3faee630 mdbx: drop obsolete Nexenta attributes API. 2022-08-17 21:29:51 +03:00
Леонид Юрьев (Leonid Yuriev)
316ddf9e01 mdbx: fix typo in MDBX_EINVAL which breaks MingGW builds with CLANG. 2022-08-16 11:09:52 +03:00
Леонид Юрьев (Leonid Yuriev)
3fbbe32adf mdbx: fix checking owner for finished write transactions inside txn_abort().
Fixed regression after 06734bf8ff.
2022-08-14 12:39:21 +03:00
Леонид Юрьев (Leonid Yuriev)
8467cc6d03 mdbx: use current txn geo for untouched parameters when env_set_geometry() called within a write transaction. 2022-08-13 16:56:29 +03:00
Леонид Юрьев (Leonid Yuriev)
9f0e2ecc67 mdbx: fix variable name typo. 2022-08-13 16:56:09 +03:00
Леонид Юрьев (Leonid Yuriev)
345c3d433f mdbx-make: add -DENABLE_UBSAN to ubsan-targets. 2022-08-11 19:39:14 +03:00
Леонид Юрьев (Leonid Yuriev)
1c5ef060c5 mdbx: reduce number of memory fences in the hot paths. 2022-08-11 18:45:00 +03:00
Леонид Юрьев (Leonid Yuriev)
34a4e7e102 mdbx: avoid Valgrind warnings. 2022-08-11 17:10:13 +03:00
Леонид Юрьев (Leonid Yuriev)
ae730ae2f3 mdbx: fix minor warnings for ASAN-enabled builds. 2022-08-11 12:33:56 +03:00
Леонид Юрьев (Leonid Yuriev)
18e557c6e8 mdbx: rename internal functions, types and macros (to be closer to MithrilDB). 2022-08-11 12:33:56 +03:00
Леонид Юрьев (Leonid Yuriev)
096d6a9bd6 mdbx: some micro-optimizations. 2022-08-10 22:09:42 +03:00
Леонид Юрьев (Leonid Yuriev)
d8f0c9dc44 mdbx: more __hot. 2022-08-10 15:46:45 +03:00
Леонид Юрьев (Leonid Yuriev)
78dc699709 mdbx: add ×16 accelerated scan4seq() (AVX512BW). 2022-08-10 13:23:04 +03:00
Леонид Юрьев (Leonid Yuriev)
c2bf9ebf17 mdbx: minor refine AVX2/SSE2-accelerated scan4seq(). 2022-08-10 11:43:24 +03:00
Леонид Юрьев (Leonid Yuriev)
3c28619562 mdbx: merge branch master into devel. 2022-08-09 19:04:27 +03:00
Леонид Юрьев (Leonid Yuriev)
0287a00ee3 mdbx++: refine/simplify wchar_t support for pathnames. 2022-08-09 18:54:22 +03:00