ISAR/libmdbx
3
0
Fork 0
mirror of https://github.com/isar/libmdbx.git synced 2025-01-08 08:24:14 +08:00
Code Issues Packages Projects Releases Wiki Activity
3,522 Commits 5 Branches 68 Tags 13 MiB
Commit Graph

659 Commits

Author SHA1 Message Date
Леонид Юрьев (Leonid Yuriev)
d572052178 mdbx: refine meta_checktxnid() and meta_waittxnid(). 2022-07-08 23:44:03 +03:00
Леонид Юрьев (Leonid Yuriev)
6f6c581c6e mdbx: minor refine safe64_write(), safe64_read() and safe64_inc(). 2022-07-08 23:44:03 +03:00
Леонид Юрьев (Leonid Yuriev)
baea4c81c9 mdbx: simplify safe64_reset(). 2022-07-08 23:44:03 +03:00
Леонид Юрьев (Leonid Yuriev)
ad5a83586b mdbx: fix insignificant uint8_t-casting warnings. 2022-07-08 23:44:03 +03:00
Леонид Юрьев (Leonid Yuriev)
db0f4e3d1e mdbx: minor fix mdbx_page_check() for case debug-audit enforced. 2022-07-08 21:45:26 +03:00
Леонид Юрьев (Leonid Yuriev)
c499f2bb36 mdbx: minor fix EINVAL from mdbx_env_set_geometry(). 
Silently growth `size_lower` to the `MIN_PAGENO` instead of returning `MDBX_EINVAL`.
 2022-07-08 01:14:29 +03:00
Леонид Юрьев (Leonid Yuriev)
6e2a1ebfbd mdbx: minor fix meta_checktxnid() to avoid assertion in debug mode. 2022-07-08 01:08:54 +03:00
Леонид Юрьев (Leonid Yuriev)
2cfcfcf91c mdbx: merge branch erigon into devel. 2022-07-07 23:24:45 +03:00
Леонид Юрьев (Leonid Yuriev)
81ea7bd41e mdbx: fix copy&paste typo inside meta_checktxnid(). 2022-07-07 21:48:34 +03:00
Леонид Юрьев (Leonid Yuriev)
f16bee8fa1 mdbx: fix/setup zero mod_txnid during copy-with-compactification. 2022-07-07 21:48:34 +03:00
Леонид Юрьев (Leonid Yuriev)
c95143f41b mdbx: add poor_page(). 2022-07-07 21:48:34 +03:00
Леонид Юрьев (Leonid Yuriev)
bc744a843a mdbx: refine/speedup PAGETYPE(). 2022-07-07 21:48:34 +03:00
Леонид Юрьев (Leonid Yuriev)
a812198c49 mdbx: rework/clone page_get() to three for the cases: any, large, branch-leaf. 2022-07-07 21:48:30 +03:00
Леонид Юрьев (Leonid Yuriev)
fbe97a79a3 mdbx: more checking for a large/overflow nodes and pages. 2022-07-07 21:46:19 +03:00
Леонид Юрьев (Leonid Yuriev)
3cc7f105a5 mdbx: fix copy&paste typo inside meta_checktxnid(). 2022-07-07 19:11:47 +03:00
Леонид Юрьев (Leonid Yuriev)
b31b270ffd mdbx: refine copy-with-compactification to clear/zero unused gaps on a DB pages. 2022-07-07 19:09:52 +03:00
Леонид Юрьев (Leonid Yuriev)
0b5cdee6ef mdbx: fix mdbx_env_compact() hand when DB is corrupted or the volume of an output pages aliquot to MDBX_ENVCOPY_WRITEBUF. 2022-07-07 19:05:35 +03:00
Леонид Юрьев (Leonid Yuriev)
6eefa05f3d mdbx: minor refine CC_RETIRING handling. 2022-07-07 14:10:19 +03:00
Леонид Юрьев (Leonid Yuriev)
ca3f188370 mdbx: extend CHECK_LEAF_TYPE() by adding CC_OVERFLOW. 2022-07-07 14:10:19 +03:00
Леонид Юрьев (Leonid Yuriev)
498514dae1 mdbx: remove unneeded CC_COPYING. 2022-07-07 14:10:19 +03:00
Леонид Юрьев (Leonid Yuriev)
1740f8227a mdbx: rework copy-with-compactification. 
Кроме небольшого рефакторинга здесь реализуется более регулярный способ
обхода дерева при копировании с компактификаций. В частности, полная
инициализация курсоров позволяет выполнять больше проверок/контроля
структуры БД и избавиться от флажка CC_COPYING.

Beside a small refactoring, a more regular way of traversing the tree
when copying with compactification is implemented here. In particular,
full initialization of cursors allows to perform more checks/control of
the DB structure and get rid of the CC_COPYING flag.
 2022-07-07 14:10:19 +03:00
Леонид Юрьев (Leonid Yuriev)
2d300d807b mdbx: extract node_read_bigdata(). 2022-07-07 14:10:19 +03:00
Леонид Юрьев (Leonid Yuriev)
d4ef9bf233 mdbx: rework page validation/checking, add MDBX_VALIDATION option (squashed). 
Здесь основная часть изменений преобразующих отладочную проверку страниц
в регулярный и доступный пользователю осторожный/безопасный режим работы
с потенциально поврежденной БД.

Here the major part of the changes that transform a debugging check of
pages into a regular and user-accessible careful/safe mode for working
with a potentially corrupted database.
 2022-07-07 14:10:09 +03:00
Леонид Юрьев (Leonid Yuriev)
6c5ff863ff mdbx: remove pp_txnid4chk(), preparing to rework of page checking/validation. 2022-07-05 15:32:09 +03:00
Леонид Юрьев (Leonid Yuriev)
d61c096313 mdbx: drop mdbx_recent_steady_txnid() and fix extra search for steady meta-page. 2022-07-02 22:53:48 +03:00
Леонид Юрьев (Leonid Yuriev)
b9835389f4 mdbx: add cache for pointers to last/steady meta-pages (off by default). 2022-07-02 22:52:31 +03:00
Леонид Юрьев (Leonid Yuriev)
720b4d56be mdbx: bigfoot feature. 
Chunking long list of retired pages during huge transactions commit
to avoid use sequences of pages:
 - splits a long retired page-number-list into chunks
   which fits one per single overflow/large page;
 - this requires a few unique id for keys
   for create such records into GC/freeDB;
 - just use the necessary subsequent IDs following the current
   transaction ID and then take the last of ones to update a meta-page.

Thus avoids using/allocating/searching a sequence of free pages
but just increase txnid more than one during the commit
a huge write transaction with a long retired-pages-list.
 2022-07-02 22:33:25 +03:00
Леонид Юрьев (Leonid Yuriev)
4f6b92248d mdbx: add pgop_stat.gcrtime for collect the time spent loading and searching inside GC. 2022-07-01 09:27:38 +03:00
Леонид Юрьев (Leonid Yuriev)
48c6051482 mdbx: minor fix meta_checktxnid() to avoid assertion in debug mode. 2022-06-29 13:37:26 +03:00
Леонид Юрьев (Leonid Yuriev)
23bbceb367 mdbx: minor fix EINVAL from mdbx_env_set_geometry(). 
Silently growth `size_lower` to the `MIN_PAGENO` instead of returning `MDBX_EINVAL`.
 2022-06-27 13:55:44 +03:00
Леонид Юрьев (Leonid Yuriev)
065e5849da mdbx: speedup GC-related pnl-merge and sequence-search. 2022-06-27 12:37:29 +03:00
Леонид Юрьев (Leonid Yuriev)
e3a09db3da mdbx: always coalescing GC records, regardless to MDBX_COALESCE flag. 2022-06-27 12:37:29 +03:00
Леонид Юрьев (Leonid Yuriev)
ece2fe2514 mdbx: split page_alloc() and page_new() to fast- and slow/rare- parts. 2022-06-27 12:37:29 +03:00
Леонид Юрьев (Leonid Yuriev)
f1ccc717b4 mdbx: add update-gc context (extracted from bigfoot). 2022-06-27 11:27:05 +03:00
Леонид Юрьев (Leonid Yuriev)
3817236b68 mdbx: avoid memory-model from variables for C11 atomics (workaround for lcc 1.26). 2022-06-12 20:04:35 +03:00
Леонид Юрьев (Leonid Yuriev)
c082eb7d8a mdbx: use _tlv_atexit() on Darwin/MacOSX. 2022-06-11 21:16:29 +03:00
Леонид Юрьев (Leonid Yuriev)
abac366eac mdbx: rework/fix rthc-cleanup using pthread' tsd to avoid write-after-free (critical). 2022-06-11 21:16:29 +03:00
Леонид Юрьев (Leonid Yuriev)
63f8eb253d mdbx: do not enable ASAN.detect_leaks in macOS since unsupported. 2022-06-10 17:26:10 +03:00
Леонид Юрьев (Leonid Yuriev)
117be44c44 mdbx: fix nasty typename typo. 
Спасибо Станиславу Очеретному за сообщение о проблеме.
 2022-06-08 20:57:42 +03:00
Леонид Юрьев (Leonid Yuriev)
cdb16c9f00 mdbx: minor fix/clarify internal comment. 2022-06-07 18:13:49 +03:00
Леонид Юрьев (Leonid Yuriev)
d1e67645a2 mdbx: refine handling EACCESS while trying create LCK-file on a read-only filesystem. 2022-06-04 02:04:55 +03:00
Леонид Юрьев (Leonid Yuriev)
a4da10bc62 mdbx: minor fix LCK-initialization. 2022-06-03 22:40:18 +03:00
Леонид Юрьев (Leonid Yuriev)
e795fe7c3e mdbx: fix without-LCK (exclusive + readonly) mode. 2022-06-03 22:39:11 +03:00
Леонид Юрьев (Leonid Yuriev)
d9407ee648 mdbx: minor refine page_alloc() internal logging. 2022-05-31 17:48:23 +03:00
Леонид Юрьев (Leonid Yuriev)
c7bde8be8d mdbx: rework internal DBI-handles serials, validation and import into a txn (squashed). 2022-05-31 17:48:10 +03:00
Леонид Юрьев (Leonid Yuriev)
21e1dc3248 mdbx: fix minor warnings from modern Apple's CLANG. 2022-05-15 23:42:57 +03:00
Леонид Юрьев (Leonid Yuriev)
5cd4190f2d mdbx: fix copy&paste bug. 
Resolves https://gitflic.ru/project/erthink/libmdbx/issue/5
 2022-05-12 14:21:19 +03:00
Леонид Юрьев (Leonid Yuriev)
405de05ca9 mdbx: return MDBX_RESULT_TRUE on empty transaction commit. 2022-05-12 13:11:59 +03:00
Леонид Юрьев (Leonid Yuriev)
548d0a50b8 mdbx: minor refine logging inside page_alloc(). 2022-05-04 19:07:33 +03:00
Леонид Юрьев (Leonid Yuriev)
f57ca4fbf6 mdbx: minor refine logging inside fetch_sdb(). 2022-05-04 18:00:54 +03:00
Леонид Юрьев (Leonid Yuriev)
838f8d8fab mdbx: move most of transactions flags to public API. 2022-05-03 11:49:28 +03:00
Леонид Юрьев (Leonid Yuriev)
447d6bfca5 mdbx: add logging to mdbx_fetch_sdb() to help users debugging complex DBI use cases. 2022-05-01 12:02:40 +03:00
Леонид Юрьев (Leonid Yuriev)
bb377fd20e mdbx: merge branch master into devel. 2022-04-21 15:44:23 +03:00
Леонид Юрьев (Leonid Yuriev)
a2aa6667e1 mdbx: avoid 32-bit Bionic/Android hang within pthread_mutex_lock(). 2022-04-21 15:41:25 +03:00
Леонид Юрьев (Leonid Yuriev)
6d15836171 mdbx: use todo4recovery://erased_by_github/ for dead (or temporarily lost) resources. 2022-04-21 15:34:53 +03:00
Леонид Юрьев (Leonid Yuriev)
ed23956e11 mdbx: basic cleanup from dead github and another corrupted services. 2022-04-20 06:42:16 +03:00
Леонид Юрьев (Leonid Yuriev)
9fa4e21165 mdbx: fix assertion regression. 
Fixes https://github.com/ledgerwatch/erigon/issues/3874.

This was a minor regression after the c4a5325aafd3f03ce7520731b9da7253d7d178f0
that affects only debug builgs (with enabled assertions) and only when the added
code catch a incoherency of unified page/buffer cache.
 2022-04-11 21:52:46 +03:00
Leonid Yuriev
21da42d23d mdbx: fix built-in status of MDBX_WITHOUT_MSVC_CRT option. 2022-03-30 18:10:03 +03:00
Леонид Юрьев (Leonid Yuriev)
2497437060 mdbx: mark the parent transaction dirty in case the undo of the geometry update failed during abortion of a nested transaction. 2022-03-29 18:57:35 +03:00
Леонид Юрьев (Leonid Yuriev)
33a9395afe mdbx: use MDBX_EPERM to indicate that the geometry cannot be updated instead of `MDBX_RESULT_TRUE'. 2022-03-29 18:57:26 +03:00
Леонид Юрьев (Leonid Yuriev)
6380f2e844 mdbx: minor reformat (cosmetics). 2022-03-25 13:47:37 +03:00
Леонид Юрьев (Leonid Yuriev)
fef90d2a3c mdbx: add explicit memset() for debugging. 2022-03-25 13:47:37 +03:00
Леонид Юрьев (Leonid Yuriev)
d522069080 mdbx: fix rare SIGSEGV in cursor tracking code. 
The error was that the array of pointers in the transaction zeroed by the
value of env->me_numdbs and before txn->mt_numdbs was set to env->me_numdbs.
Thus, a cursor pointer(s) in the starting transaction could uninitialized if
another thread opened a dbi-handle(s) between the two aforementioned events.
 2022-03-25 13:05:46 +03:00
Леонид Юрьев (Leonid Yuriev)
43070c7b26 mdbx: minor fix meta_checktxnid() for Valgrind/ASAN usage. 
Related to https://github.com/erthink/libmdbx/issues/269.
 2022-03-24 12:11:50 +03:00
Леонид Юрьев (Leonid Yuriev)
45f8197635 mdbx: clarify/refine error messages of a signature/version mismatch. 2022-03-24 00:19:34 +03:00
Леонид Юрьев (Leonid Yuriev)
b79f6712e3 mdbx: refine override_meta() to preserve format-signature for legacy/zero mod_txnid. 2022-03-23 00:48:41 +03:00
Леонид Юрьев (Leonid Yuriev)
de63041b7d mdbx: add MDBX_DBG_DONT_UPGRADE flag. 2022-03-23 00:48:41 +03:00
Леонид Юрьев (Leonid Yuriev)
a5c064c33e mdbx: preserve (don't upgrade) format-signature during turn to a specified meta-page while recovery. 2022-03-23 00:48:41 +03:00
Your Name
9c832c24a6 mdbx: minor fix error-message space. 2022-03-23 00:48:41 +03:00
Леонид Юрьев (Leonid Yuriev)
c4a5325aaf mdbx: check-and-retry against page arrival as a workaround for unified page/buffer cache incoherency. 
Part 2 of 2 of the workaround for https://github.com/erthink/libmdbx/issues/269.
 2022-03-08 17:09:29 +03:00
Леонид Юрьев (Leonid Yuriev)
93cf99a07c mdbx: drop the previous temporary workaround for unified page/buffer cache incoherency. 
Related to https://github.com/erthink/libmdbx/issues/269.
 2022-03-06 13:22:57 +03:00
Леонид Юрьев (Leonid Yuriev)
00ed61c685 mdbx: check-and-retry a mvcc-snapshot for unified page/buffer cache coherency. 
Part 1 of 2 of the workaround for https://github.com/erthink/libmdbx/issues/269.
 2022-03-06 13:22:38 +03:00
Леонид Юрьев (Leonid Yuriev)
9d8fc7b984 mdbx: minor mix result caching inside the get_reasonable_db_maxsize(). 
Change-Id: I9763ebb323d4f66e702e07f708e269e2479880ac
 2022-03-05 14:31:02 +03:00
Леонид Юрьев (Leonid Yuriev)
44fb240955 mdbx: merge branch issue-269 into the devel branch. 2022-03-04 17:46:43 +03:00
Леонид Юрьев (Leonid Yuriev)
bfea3ca9fb mdbx: minor refine mdbx_page_search() internals. 2022-03-04 17:38:25 +03:00
Леонид Юрьев (Leonid Yuriev)
7ade182d64 mdbx: minor simplify/refine mdbx_env_set_geometry() internals. 2022-03-04 17:38:25 +03:00
Леонид Юрьев (Leonid Yuriev)
4adc7aa58d mdbx: add few checks to mdbx_validate_meta(). 2022-03-04 11:53:14 +03:00
Леонид Юрьев (Leonid Yuriev)
110cf09cf8 mdbx: simplify core-dbs update inside mdbx_txn_commit_ex(). 2022-03-04 11:49:54 +03:00
Леонид Юрьев (Leonid Yuriev)
fb25648b9c mdbx: force mdbx_page_get() to inline. 2022-03-04 11:49:01 +03:00
Леонид Юрьев (Leonid Yuriev)
78170a5750 mdbx: minor reduce/lowering debug "update oldest" logging. 2022-03-04 11:41:17 +03:00
Леонид Юрьев (Leonid Yuriev)
71d07b3a8e mdbx: adding cursors tracking/lists for read-only transactions. 
Briefly, this commit fixes a missed flaw:
 - Cursor tracking is required to replacing shaded pages and adjusting the positions in writing transactions;
 - Thus, historically, an internal linked list was maintained for a read-write transactions, but not for a read-only.
   For this reason, the API for using cursors should be different for writing and reading transactions;
 - However, the libmdbx's API has been significantly improved, including the ability to reuse cursors and a uniform cursors behavior for any kind of transactions.
   My mistake is that due to working with MithrilDB, I forgot to make a same changes to libmdbx.

Fixes https://github.com/erthink/libmdbx/issues/272.
 2022-03-03 15:00:28 +03:00
Леонид Юрьев (Leonid Yuriev)
6d61b18325 mdbx: fix auxiliary mdbx_txn_valgrind(). 
Fixed https://github.com/erthink/libmdbx/issues/273.
 2022-02-27 15:42:48 +03:00
Леонид Юрьев (Leonid Yuriev)
6b45498985 mdbx: temporary workaround/hotfix for a flaw of Linux 4.19 (at least) unified page/buffer cache. 
Temporary workaround for https://github.com/erthink/libmdbx/issues/269.
 2022-02-22 23:58:57 +03:00
Леонид Юрьев (Leonid Yuriev)
a0728023cd mdbx: fix typo in a retained space calculation for the hsr-callback. 2022-02-16 15:44:06 +03:00
Леонид Юрьев (Leonid Yuriev)
3705d705d3 mdbx: refine/fix pages reserving inside mdbx_updage_gc() to avoid non-reclamation in a rare cases. 2022-02-16 12:18:53 +03:00
Леонид Юрьев (Leonid Yuriev)
c5f1f73fca mdbx: avoid extra looping inside mdbx_page_alloc() for mdbx_prep_backlog(). 2022-02-15 03:17:05 +03:00
Леонид Юрьев (Leonid Yuriev)
925a673d57 mdbx: make mdbx_env_open() idempotence in failure cases. 
Fixes https://github.com/erthink/libmdbx/issues/267.
 2022-02-13 20:30:57 +03:00
Леонид Юрьев (Leonid Yuriev)
c27787eb31 mdbx: fix potential SIGSEGV for DB with non-default page size. 
Related to https://github.com/erthink/libmdbx/issues/267.
 2022-02-13 17:04:10 +03:00
Леонид Юрьев (Leonid Yuriev)
2b6fd968d2 mdbx: rework functions for meta-pages, split-off non-volatile, more const. 2022-02-09 22:56:29 +03:00
Леонид Юрьев (Leonid Yuriev)
ef7b4289c0 mdbx: rework unaligned access. 
The three points:
 - disentangle C11-atomic fences/barriers and pure-functions (with `__attribute__((__pure__))`) to avoid compiler misoptimization;
 - fix hypotetic unaligned access to 64-bit dwords on ARM with `__ARM_FEATURE_UNALIGNED` defined;
 - reasonable paranoia that makes clarity for code readers.
 2022-02-09 22:56:28 +03:00
Leonid Yuriev
5aeb7ccf25 mdbx: fix extra casting inside mdbx_limits_dbsize_max(). 2022-02-02 14:09:03 +03:00
Leonid Yuriev
bd51e181fb mdbx: refine comment for mdbx_update_gc(). 2022-01-31 21:36:57 +03:00
Leonid Yuriev
fe5199b9d1 mdbx: fix/rework PNL checking during search, refine/simplify spilled pages search. 
Fixes https://github.com/erthink/libmdbx/issues/260.

Change-Id: I8824f6af2f770c46668ee577aeeeb93b30b17a13
 2022-01-26 13:21:21 +03:00
Leonid Yuriev
9ae56332d7 mdbx: fix Coverity warnings. 2022-01-22 18:01:31 +03:00
Leonid Yuriev
cd2c5f594c mdbx: rework checking MAX_PAGENO and DB upper-size limit. 
Related to https://github.com/erthink/libmdbx/issues/260.
 2022-01-21 02:25:27 +03:00
Leonid Yuriev
ddef217047 mdbx: relax txn-check rules for auxiliary functions. 2022-01-21 02:13:48 +03:00
Leonid Yuriev
3e28cc2a25 mdbx: allow mdbx_dbi_flags_ex() for errored transactions. 2022-01-21 02:13:48 +03:00
Leonid Yuriev
8779f665dc mdbx: fix excessive PNL checking after sort for spilling. 
This bug triggered only in the DEBUG builds or when the assertion checking is forcibly enabled.
It does not affect any core logic and cannot lead to DB corruption, data loss, and so on.

Fixes https://github.com/erthink/libmdbx/issues/260.
 2022-01-20 15:25:35 +03:00
Leonid Yuriev
c588af6aca mdbx: update Copyright year. 
Change-Id: Ib3e4c0ac94882c4b2a1e167dd98e6e26dbdf48a4
 2022-01-17 11:06:14 +03:00
Leonid Yuriev
2151be6cde mdbx: merge branch 'issue-260' into devel. 2022-01-15 15:31:45 +03:00