ISAR/libmdbx
3
0
Fork 0
mirror of https://github.com/isar/libmdbx.git synced 2025-01-06 23:04:13 +08:00
Code Issues Packages Projects Releases Wiki Activity
3,581 Commits 5 Branches 68 Tags 13 MiB
Commit Graph

1978 Commits

Author SHA1 Message Date
Леонид Юрьев (Leonid Yuriev)
9f0e2ecc67 mdbx: fix variable name typo. 2022-08-13 16:56:09 +03:00
Леонид Юрьев (Leonid Yuriev)
1c5ef060c5 mdbx: reduce number of memory fences in the hot paths. 2022-08-11 18:45:00 +03:00
Леонид Юрьев (Leonid Yuriev)
34a4e7e102 mdbx: avoid Valgrind warnings. 2022-08-11 17:10:13 +03:00
Леонид Юрьев (Leonid Yuriev)
ae730ae2f3 mdbx: fix minor warnings for ASAN-enabled builds. 2022-08-11 12:33:56 +03:00
Леонид Юрьев (Leonid Yuriev)
18e557c6e8 mdbx: rename internal functions, types and macros (to be closer to MithrilDB). 2022-08-11 12:33:56 +03:00
Леонид Юрьев (Leonid Yuriev)
096d6a9bd6 mdbx: some micro-optimizations. 2022-08-10 22:09:42 +03:00
Леонид Юрьев (Leonid Yuriev)
d8f0c9dc44 mdbx: more __hot. 2022-08-10 15:46:45 +03:00
Леонид Юрьев (Leonid Yuriev)
78dc699709 mdbx: add ×16 accelerated scan4seq() (AVX512BW). 2022-08-10 13:23:04 +03:00
Леонид Юрьев (Leonid Yuriev)
c2bf9ebf17 mdbx: minor refine AVX2/SSE2-accelerated scan4seq(). 2022-08-10 11:43:24 +03:00
Леонид Юрьев (Leonid Yuriev)
3c28619562 mdbx: merge branch master into devel. 2022-08-09 19:04:27 +03:00
Леонид Юрьев (Leonid Yuriev)
0287a00ee3 mdbx++: refine/simplify wchar_t support for pathnames. 2022-08-09 18:54:22 +03:00
Леонид Юрьев (Leonid Yuriev)
2ff8d3c4f2 mdbx: native wchar_t pathname for Windows. 2022-08-09 18:27:43 +03:00
Леонид Юрьев (Leonid Yuriev)
98c53555ab mdbx: using e2k-frendly/cmov/branch-less bsearch. 
https://gitflic.ru/project/erthink/bsearch-try
 2022-08-09 18:25:05 +03:00
Леонид Юрьев (Leonid Yuriev)
c8b1392cbe mdbx: use MultiByteToWideChar(CP_THREAD_ACP) instead of mbstowcs(). 2022-08-09 16:12:24 +03:00
Леонид Юрьев (Leonid Yuriev)
6d85e35876 mdbx: never use modern __cxa_thread_atexit() on Apple's OSes. 2022-08-08 15:23:39 +03:00
Леонид Юрьев (Leonid Yuriev)
dd01aabaeb mdbx: add MDBX_WEAK_IMPORT_ATTRIBUTE macro. 2022-08-08 15:18:16 +03:00
Леонид Юрьев (Leonid Yuriev)
3de759a7be mdbx: fix page-boundary checking inside accelerated scan4seq(). 2022-08-07 22:24:00 +03:00
Леонид Юрьев (Leonid Yuriev)
d6603a0c0a mdbx: add ×8 accelerated scan4seq() (AVX2). 2022-08-07 17:08:51 +03:00
Леонид Юрьев (Leonid Yuriev)
15146d3823 mdbx: fix scan4seq() selection for non-implemented cases. 2022-08-07 15:14:50 +03:00
Леонид Юрьев (Leonid Yuriev)
d62d3e2aab mdbx: merge branch master into devel. 2022-08-07 12:42:43 +03:00
Леонид Юрьев (Leonid Yuriev)
fa854e40c3 mdbx: refine checking inside page_get(). 2022-08-07 12:29:51 +03:00
Леонид Юрьев (Leonid Yuriev)
5afc5c4e8c mdbx: reorganize/move fences to reduce overhead. 2022-08-07 12:29:51 +03:00
Леонид Юрьев (Leonid Yuriev)
c05a3b7bb9 mdbx: minor refine node_add(). 2022-08-07 12:29:50 +03:00
Леонид Юрьев (Leonid Yuriev)
1215bda188 mdbx: minor refine/speedup node_del(). 2022-08-07 12:29:50 +03:00
Леонид Юрьев (Leonid Yuriev)
0dd4532473 mdbx: reduce gap/backlog of linear scan inside dpl_search(). 2022-08-07 12:29:50 +03:00
Леонид Юрьев (Leonid Yuriev)
eac3d0499f mdbx: minor refine/speedup dpl_sort_slowpath(). 2022-08-07 12:29:50 +03:00
Леонид Юрьев (Leonid Yuriev)
a11c045f1e mdbx: using expect_with_probability() macro. 2022-08-07 12:28:35 +03:00
Леонид Юрьев (Leonid Yuriev)
c0f8ecd6f2 mdbx: add expect_with_probability() macro. 2022-08-07 11:56:23 +03:00
Леонид Юрьев (Leonid Yuriev)
8404cc1fd7 mdbx: reduce sorting-network to 8. 2022-08-07 11:56:23 +03:00
Леонид Юрьев (Leonid Yuriev)
654b020bc7 mdbx: add __restrict to quicksort internal pointers. 2022-08-07 11:56:23 +03:00
Леонид Юрьев (Leonid Yuriev)
77635116c6 mdbx: enable solib profiling with -pg and gprof with GLIBC >= 2.37. 
However such profiling requires https://sourceware.org/bugzilla/show_bug.cgi?id=29438 to be fixed.
 2022-08-07 11:56:23 +03:00
Леонид Юрьев (Leonid Yuriev)
d28110373e mdbx: add simple SORT_CMP_SWAP() macro for MDBX_HAVE_CMOV=0 case. 2022-08-07 11:56:23 +03:00
Леонид Юрьев (Leonid Yuriev)
480dc2531e mdbx: ×4 accelerated scan4seq() (SSE2 only for now). 2022-08-07 11:56:23 +03:00
Леонид Юрьев (Leonid Yuriev)
06734bf8ff mdbx: don't check owner for finished transactions. 2022-08-06 13:19:49 +03:00
Леонид Юрьев (Leonid Yuriev)
c37fb50532 mdbx: more for __amd64__ macro. 2022-08-04 13:54:07 +03:00
Леонид Юрьев (Leonid Yuriev)
9eb6953778 mdbx: fix minor typo. 2022-08-04 13:54:07 +03:00
Леонид Юрьев (Leonid Yuriev)
268b33cbf7 mdbx: simplify/speedup scan4seq(). 2022-08-04 13:54:07 +03:00
Леонид Юрьев (Leonid Yuriev)
e444c70cb7
mdbx: release v0.11.9 
The stable bugfix release.
It is planned that this will be the last release of the v0.11 branch.

Acknowledgements:
-----------------

 - [Alex Sharov](https://github.com/AskAlexSharov) and Erigon team for reporting and testing.
 - [Andrew Ashikhmin](https://gitflic.ru/user/yperbasis) for contributing.

New:
----

 - Ability to customise `MDBX_LOCK_SUFFIX`, `MDBX_DATANAME`, `MDBX_LOCKNAME` just by predefine ones during build.
 - Added to [`mdbx::env_managed`](https://libmdbx.dqdkfa.ru/group__cxx__api.html#classmdbx_1_1env__managed)'s methods a few overloads with `const char* pathname` parameter (C++ API).

Fixes:
------

 - Fixed hang copy-with-compactification of a corrupted DB
   or in case the volume of output pages is a multiple of `MDBX_ENVCOPY_WRITEBUF`.
 - Fixed standalone non-CMake build on MacOS (`#include AvailabilityMacros.h>`).
 - Fixed unexpected `MDBX_PAGE_FULL` error in rare cases with large database page sizes.

Minors:
-------

 - Minor fixes Doxygen references, comments, descriptions, etc.
 - Fixed copy&paste typo inside `meta_checktxnid()`.
 - Minor fix `meta_checktxnid()` to avoid assertion in debug mode.
 - Minor fix `mdbx_env_set_geometry()` to avoid returning `EINVAL` in particular rare cases.
 - Minor refine/fix batch-get testcase for large page size.
 - Added `--pagesize NN` option to long-stotastic test script.
 - Updated Valgrind-suppressions file for modern GCC.
 - Fixed `has no symbols` warning from Apple's ranlib.

18 files changed, 318 insertions(+), 178 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
 2022-08-02 12:00:30 +03:00
Леонид Юрьев (Leonid Yuriev)
a441c9ffb1 mdbx: logging pgno of invalid root pages inside meta_checktxnid(). 2022-08-01 14:44:33 +03:00
Леонид Юрьев (Leonid Yuriev)
71c3d20c01 mdbx: skip extra cycle during search of page sequence. 2022-08-01 00:00:14 +03:00
Леонид Юрьев (Leonid Yuriev)
75d19b5806 mdbx: minor refine/speedup pnl_check(). 2022-07-27 21:08:54 +03:00
Леонид Юрьев (Leonid Yuriev)
dc39ecfb9f mdbx: auto-coalesce of GC's records with less overhead. 2022-07-27 21:08:54 +03:00
Леонид Юрьев (Leonid Yuriev)
47d5fa7fd4 mdbx: refine/speedup pnl_merge(). 2022-07-27 21:08:54 +03:00
Леонид Юрьев (Leonid Yuriev)
fe6c6b2068 mdbx: add MDBX_HAVE_CMOV macro/option. 2022-07-27 21:08:54 +03:00
Леонид Юрьев (Leonid Yuriev)
262fafd00e mdbx: fix unused warning for case MDBX_ENABLE_PGOP_STAT=0. 2022-07-27 00:03:27 +03:00
Леонид Юрьев (Leonid Yuriev)
289636834c mdbx: fix unused warning for case MDBX_DISABLE_VALIDATION. 2022-07-27 00:03:27 +03:00
Леонид Юрьев (Leonid Yuriev)
c4dd83fbdf mdbx: minor refine page_split(). 2022-07-27 00:03:22 +03:00
Леонид Юрьев (Leonid Yuriev)
cc51a7f76e mdbx: minor refine attributes-related macros for LCC. 2022-07-23 12:15:55 +03:00
Леонид Юрьев (Leonid Yuriev)
a82f59a998 mdbx: minor refine MDBX_UNALIGNED_OK. 2022-07-23 12:15:19 +03:00
Леонид Юрьев (Leonid Yuriev)
08e936a809 mdbx: re-verify atomic-ops and remove mo_SequentialConsistency. 2022-07-23 12:15:19 +03:00
Леонид Юрьев (Leonid Yuriev)
5e565433f7 mdbx: merge branch master into devel. 2022-07-23 11:56:17 +03:00
Леонид Юрьев (Leonid Yuriev)
e963375302 mdbx: fix unexpected MDBX_PAGE_FULL in rare cases with large page-size. 2022-07-21 17:52:08 +03:00
yperbasis
23e7870e81 #include <AvailabilityMacros.h> 2022-07-19 18:35:19 +02:00
Леонид Юрьев (Leonid Yuriev)
69b495d559 mdbx++: add to mdbx::env overloads with const char* pathname parameter. 2022-07-13 21:45:23 +03:00
Леонид Юрьев (Leonid Yuriev)
0018164fef mdbx: fix wrong } oops-like typo. 
This is a `devel`-only 3-days old regression since a4a35ce9cb7176b914e0606d38c6d6e47a9431f1.
 2022-07-11 20:29:33 +03:00
Леонид Юрьев (Leonid Yuriev)
5ccfb5f30a mdbx-tools: use MDBX_DBG_DUMP, MDBX_DBG_ASSERT, MDBX_DBG_AUDIT inside mdbx_chk. 2022-07-11 20:28:07 +03:00
Леонид Юрьев (Leonid Yuriev)
149e708830 mdbx: rename MDBX_CACHE_METAPTR build-time option. 2022-07-10 10:05:39 +03:00
Леонид Юрьев (Leonid Yuriev)
194f2f45d2 mdbx: refine/fix using nested cursor's db inside copy-with-compactification. 2022-07-09 18:07:54 +03:00
Леонид Юрьев (Leonid Yuriev)
e8dd208e96 mdbx: more cursor-checking for audit-without-debug. 2022-07-09 16:12:41 +03:00
Леонид Юрьев (Leonid Yuriev)
9108a241a2 mdbx: minor fix/clarify debug logging inside page_alloc_slowpath(). 2022-07-08 23:44:03 +03:00
Леонид Юрьев (Leonid Yuriev)
9aa2aae93e mdbx: rework/simplify kick_longlived_readers(). 2022-07-08 23:44:03 +03:00
Леонид Юрьев (Leonid Yuriev)
9421bb424d mdbx: refine/simplify read-latch loop inside mdbx_txn_renew0(). 
1. Explicitly check and handle a race/collision case with `find_oldest_reader()`.
2. Handle "recovery mode" (me_stuck_meta >= 0) by the same code as for regular latch.
3. Add bailout error message for buggy compiler and/or hardware (paranoid).
 2022-07-08 23:44:03 +03:00
Леонид Юрьев (Leonid Yuriev)
434ad8edc8 mdbx: refine bind_rslot(). 2022-07-08 23:44:03 +03:00
Леонид Юрьев (Leonid Yuriev)
a4a35ce9cb mdbx: rework find_oldest_reader(). 
1. Fix regression `assert: oldest >= lck->mti_oldest_reader.weak` after d4bf0a3332c7b05331ab0a87e3cd65b0903edc3c.
2. Add explicit check, kick and notice for stuck reader.
3. Made more e2k-frendly.
 2022-07-08 23:44:03 +03:00
Леонид Юрьев (Leonid Yuriev)
d572052178 mdbx: refine meta_checktxnid() and meta_waittxnid(). 2022-07-08 23:44:03 +03:00
Леонид Юрьев (Leonid Yuriev)
6f6c581c6e mdbx: minor refine safe64_write(), safe64_read() and safe64_inc(). 2022-07-08 23:44:03 +03:00
Леонид Юрьев (Leonid Yuriev)
baea4c81c9 mdbx: simplify safe64_reset(). 2022-07-08 23:44:03 +03:00
Леонид Юрьев (Leonid Yuriev)
ad5a83586b mdbx: fix insignificant uint8_t-casting warnings. 2022-07-08 23:44:03 +03:00
Леонид Юрьев (Leonid Yuriev)
b5346ee765 mdbx: use unsigned constants for page flags (to avoid MSVC warnings). 2022-07-08 21:48:08 +03:00
Леонид Юрьев (Leonid Yuriev)
db0f4e3d1e mdbx: minor fix mdbx_page_check() for case debug-audit enforced. 2022-07-08 21:45:26 +03:00
Леонид Юрьев (Leonid Yuriev)
c499f2bb36 mdbx: minor fix EINVAL from mdbx_env_set_geometry(). 
Silently growth `size_lower` to the `MIN_PAGENO` instead of returning `MDBX_EINVAL`.
 2022-07-08 01:14:29 +03:00
Леонид Юрьев (Leonid Yuriev)
6e2a1ebfbd mdbx: minor fix meta_checktxnid() to avoid assertion in debug mode. 2022-07-08 01:08:54 +03:00
Леонид Юрьев (Leonid Yuriev)
2cfcfcf91c mdbx: merge branch erigon into devel. 2022-07-07 23:24:45 +03:00
Леонид Юрьев (Leonid Yuriev)
81ea7bd41e mdbx: fix copy&paste typo inside meta_checktxnid(). 2022-07-07 21:48:34 +03:00
Леонид Юрьев (Leonid Yuriev)
f16bee8fa1 mdbx: fix/setup zero mod_txnid during copy-with-compactification. 2022-07-07 21:48:34 +03:00
Леонид Юрьев (Leonid Yuriev)
c95143f41b mdbx: add poor_page(). 2022-07-07 21:48:34 +03:00
Леонид Юрьев (Leonid Yuriev)
19c5e4d424 mdbx-tools: use MDBX_VALIDATION. 2022-07-07 21:48:34 +03:00
Леонид Юрьев (Leonid Yuriev)
6076c510f8 mdbx-tools: refine assertions logging inside mdbx_chk. 2022-07-07 21:48:34 +03:00
Леонид Юрьев (Leonid Yuriev)
bc744a843a mdbx: refine/speedup PAGETYPE(). 2022-07-07 21:48:34 +03:00
Леонид Юрьев (Leonid Yuriev)
a812198c49 mdbx: rework/clone page_get() to three for the cases: any, large, branch-leaf. 2022-07-07 21:48:30 +03:00
Леонид Юрьев (Leonid Yuriev)
fbe97a79a3 mdbx: more checking for a large/overflow nodes and pages. 2022-07-07 21:46:19 +03:00
Леонид Юрьев (Leonid Yuriev)
3cc7f105a5 mdbx: fix copy&paste typo inside meta_checktxnid(). 2022-07-07 19:11:47 +03:00
Леонид Юрьев (Leonid Yuriev)
b31b270ffd mdbx: refine copy-with-compactification to clear/zero unused gaps on a DB pages. 2022-07-07 19:09:52 +03:00
Леонид Юрьев (Leonid Yuriev)
0b5cdee6ef mdbx: fix mdbx_env_compact() hand when DB is corrupted or the volume of an output pages aliquot to MDBX_ENVCOPY_WRITEBUF. 2022-07-07 19:05:35 +03:00
Леонид Юрьев (Leonid Yuriev)
6eefa05f3d mdbx: minor refine CC_RETIRING handling. 2022-07-07 14:10:19 +03:00
Леонид Юрьев (Leonid Yuriev)
ca3f188370 mdbx: extend CHECK_LEAF_TYPE() by adding CC_OVERFLOW. 2022-07-07 14:10:19 +03:00
Леонид Юрьев (Leonid Yuriev)
498514dae1 mdbx: remove unneeded CC_COPYING. 2022-07-07 14:10:19 +03:00
Леонид Юрьев (Leonid Yuriev)
1740f8227a mdbx: rework copy-with-compactification. 
Кроме небольшого рефакторинга здесь реализуется более регулярный способ
обхода дерева при копировании с компактификаций. В частности, полная
инициализация курсоров позволяет выполнять больше проверок/контроля
структуры БД и избавиться от флажка CC_COPYING.

Beside a small refactoring, a more regular way of traversing the tree
when copying with compactification is implemented here. In particular,
full initialization of cursors allows to perform more checks/control of
the DB structure and get rid of the CC_COPYING flag.
 2022-07-07 14:10:19 +03:00
Леонид Юрьев (Leonid Yuriev)
2d300d807b mdbx: extract node_read_bigdata(). 2022-07-07 14:10:19 +03:00
Леонид Юрьев (Leonid Yuriev)
d4ef9bf233 mdbx: rework page validation/checking, add MDBX_VALIDATION option (squashed). 
Здесь основная часть изменений преобразующих отладочную проверку страниц
в регулярный и доступный пользователю осторожный/безопасный режим работы
с потенциально поврежденной БД.

Here the major part of the changes that transform a debugging check of
pages into a regular and user-accessible careful/safe mode for working
with a potentially corrupted database.
 2022-07-07 14:10:09 +03:00
Леонид Юрьев (Leonid Yuriev)
6c5ff863ff mdbx: remove pp_txnid4chk(), preparing to rework of page checking/validation. 2022-07-05 15:32:09 +03:00
Леонид Юрьев (Leonid Yuriev)
d61c096313 mdbx: drop mdbx_recent_steady_txnid() and fix extra search for steady meta-page. 2022-07-02 22:53:48 +03:00
Леонид Юрьев (Leonid Yuriev)
b9835389f4 mdbx: add cache for pointers to last/steady meta-pages (off by default). 2022-07-02 22:52:31 +03:00
Леонид Юрьев (Leonid Yuriev)
720b4d56be mdbx: bigfoot feature. 
Chunking long list of retired pages during huge transactions commit
to avoid use sequences of pages:
 - splits a long retired page-number-list into chunks
   which fits one per single overflow/large page;
 - this requires a few unique id for keys
   for create such records into GC/freeDB;
 - just use the necessary subsequent IDs following the current
   transaction ID and then take the last of ones to update a meta-page.

Thus avoids using/allocating/searching a sequence of free pages
but just increase txnid more than one during the commit
a huge write transaction with a long retired-pages-list.
 2022-07-02 22:33:25 +03:00
Леонид Юрьев (Leonid Yuriev)
4f6b92248d mdbx: add pgop_stat.gcrtime for collect the time spent loading and searching inside GC. 2022-07-01 09:27:38 +03:00
Леонид Юрьев (Leonid Yuriev)
48c6051482 mdbx: minor fix meta_checktxnid() to avoid assertion in debug mode. 2022-06-29 13:37:26 +03:00
Леонид Юрьев (Leonid Yuriev)
23bbceb367 mdbx: minor fix EINVAL from mdbx_env_set_geometry(). 
Silently growth `size_lower` to the `MIN_PAGENO` instead of returning `MDBX_EINVAL`.
 2022-06-27 13:55:44 +03:00
Леонид Юрьев (Leonid Yuriev)
065e5849da mdbx: speedup GC-related pnl-merge and sequence-search. 2022-06-27 12:37:29 +03:00
Леонид Юрьев (Leonid Yuriev)
e3a09db3da mdbx: always coalescing GC records, regardless to MDBX_COALESCE flag. 2022-06-27 12:37:29 +03:00
Леонид Юрьев (Leonid Yuriev)
ece2fe2514 mdbx: split page_alloc() and page_new() to fast- and slow/rare- parts. 2022-06-27 12:37:29 +03:00