Compare commits

..

79 Commits

Author SHA1 Message Date
Leonid Yuriev
f836c928a8 mdbx: Happy New Year! (release v0.11.3).
Acknowledgements:
-----------------

 - [gcxfd <i@rmw.link>](https://github.com/gcxfd) for reporting, contributing and testing.
 - [장세연 (Чан Се Ен)](https://github.com/sasgas) for reporting and testing.
 - [Alex Sharov](https://github.com/AskAlexSharov) for reporting, testing and provide resources for remote debugging/investigation.

New features, extensions and improvements:
------------------------------------------

 - [Added](https://github.com/erthink/libmdbx/issues/236) `mdbx_cursor_get_batch()`.
 - [Added](https://github.com/erthink/libmdbx/issues/250) `MDBX_SET_UPPERBOUND`.
 - C++ API is finalized now.
 - The GC update stage has been [significantly speeded](https://github.com/erthink/libmdbx/issues/254) when fixing huge Erigon's transactions (Ethereum ecosystem).

Fixes:
------

 - Disabled C++20 concepts for stupid AppleClang 13.x
 - Fixed internal collision of `MDBX_SHRINK_ALLOWED` with `MDBX_ACCEDE`.

Minors:
-------

 - Fixed returning `MDBX_RESULT_TRUE` (unexpected -1) from `mdbx_env_set_option()`.
 - Added `mdbx_env_get_syncbytes()` and `mdbx_env_get_syncperiod()`.
 - [Clarified](https://github.com/erthink/libmdbx/pull/249) description of `MDBX_INTEGERKEY`.
 - Reworked/simplified `mdbx_env_sync_internal()`.
 - [Fixed](https://github.com/erthink/libmdbx/issues/248) extra assertion inside `mdbx_cursor_put()` for `MDBX_DUPFIXED` cases.
 - Avoiding extra looping inside `mdbx_env_info_ex()`.
 - Explicitly enabled core dumps from stochastic tests scripts on Linux.
 - [Fixed](https://github.com/erthink/libmdbx/issues/253) `mdbx_override_meta()` to avoid false-positive assertions.
 - For compatibility reverted returning `MDBX_ENODATA`for some cases.

Signed-off-by: Leonid Yuriev <leo@yuriev.ru>
2021-12-31 11:52:40 +03:00
Leonid Yuriev
1786374021 mdbx: update ChangeLog. 2021-12-31 10:29:54 +03:00
Leonid Yuriev
6866fa3eaa mdbx: revert returning MDBX_ENODATA from cursor-next. 2021-12-30 22:12:04 +03:00
Leonid Yuriev
9511bc491a mdbx: more unlikely(). 2021-12-30 19:40:22 +03:00
Leonid Yuriev
ae5df65af0 mdbx: refine page_alloc() and update_gc() to reduce looping.
Resolves https://github.com/erthink/libmdbx/issues/254.
2021-12-30 17:36:33 +03:00
Leonid Yuriev
2ba90e63b1 mdbx: remove obsolete current retired-list record from gc before reserve/backlogging free-pages to reduce looping. 2021-12-28 00:10:00 +03:00
Leonid Yuriev
b30ccbde9e mdbx: don't reserve/backlog free-pages if retired-list was already put into gc. 2021-12-28 00:10:00 +03:00
Leonid Yuriev
c3c088b8c3 mdbx: minor unify gc-update debug logging (cosmetics). 2021-12-28 00:10:00 +03:00
gcxfd
3c82ced097 mdbx: add link to gcxfd's rust wrapper.
Resolve https://github.com/erthink/libmdbx/pull/255
2021-12-17 14:32:20 +03:00
Leonid Yuriev
d2377f11d3 mdbx: minor cleanup spaces in the ChangeLog (cosmetics). 2021-12-13 20:22:59 +03:00
Leonid Yuriev
c70d2d62d1 mdbx: fix mdbx_override_meta() to avoid false-positive assertions.
Fixed https://github.com/erthink/libmdbx/issues/253.
2021-12-13 13:03:49 +03:00
Leonid Yuriev
29eb4c2bed mdbx: update ChangeLog. 2021-12-13 04:27:48 +03:00
Leonid Yuriev
4e44dc69f9 mdbx: add MDBX_SET_UPPERBOUND for cursors.
Resolves https://github.com/erthink/libmdbx/issues/250.
2021-12-13 04:24:34 +03:00
Leonid Yuriev
bee5df20fd mdbx: fix MDBX_SHRINK_ALLOWED collision with MDBX_ACCEDE inside mdbx_env_sync_internal(). 2021-12-13 04:12:43 +03:00
Leonid Yuriev
2f5606702e mdbx-tests: explicit enable coredumps on Linux. 2021-12-13 04:12:43 +03:00
Leonid Yuriev
6f2c1e52ad mdbx: add mdbx_cursor_get_batch().
Resolve https://github.com/erthink/libmdbx/issues/236
2021-12-11 05:58:39 +03:00
Leonid Yuriev
32e495021f mdbx-cmake: reporting the build options (cosmetics). 2021-12-10 17:17:34 +03:00
Леонид Юрьев (Leonid Yuriev)
ca19796514 mdbx-cmake: disable C++20 for CLANG < 10. 2021-12-09 15:54:16 +03:00
Леонид Юрьев (Leonid Yuriev)
3e3560753b mdbx-cmake: minor fix detecting Elbrus/LCC. 2021-12-09 15:52:12 +03:00
Leonid Yuriev
0265c847b8 mdbx++: remove preliminary label from C++ API. 2021-12-09 02:10:08 +03:00
Leonid Yuriev
739e02655e mdbx++: disable C++20 concepts for stupid AppleClang 13.x (hotfix). 2021-12-08 05:10:47 +03:00
Leonid Yuriev
df6b9028ec mdbx: update ChangeLog. 2021-12-08 03:25:46 +03:00
Leonid Yuriev
f4057b2c3b mdbx: avoid extra looping inside mdbx_env_info_ex(). 2021-12-07 01:45:41 +03:00
Leonid Yuriev
839da86cac mdbx: minor fix handling MDBX_SET_LOWERBOUND to avoid MDBX_BAD_VALSIZE.
Related to https://github.com/erthink/libmdbx/issues/248#issuecomment-986542620.
2021-12-07 01:30:43 +03:00
Леонид Юрьев (Leonid Yuriev)
a899056fd1 mdbx: fix extra assertion.
Fixes https://github.com/erthink/libmdbx/issues/248.
2021-12-06 23:28:08 +03:00
Леонид Юрьев (Leonid Yuriev)
c484a92933 mdbx: minor refine prev commit. 2021-12-06 23:00:38 +03:00
Леонид Юрьев (Leonid Yuriev)
da855b13a3 mdbx: rework/simplify mdbx_env_sync_internal(). 2021-12-04 06:56:44 +03:00
Леонид Юрьев (Leonid Yuriev)
8ef8733ddc mdbx: minor refine mdbx_env_set_option(MDBX_opt_sync_bytes). 2021-12-04 06:55:41 +03:00
gcxfd
45a11f3dc2 mdbx: Make the documentation of MDBX_INTEGERKEY clearer.
Resolves https://github.com/erthink/libmdbx/pull/249.
2021-12-03 22:29:28 +03:00
Leonid Yuriev
3fdd810653 mdbx-test: engage mdbx_env_set_syncperiod() & mdbx_env_set_syncbytes().
Related to https://github.com/erthink/libmdbx/issues/248.
2021-12-03 21:55:40 +03:00
Leonid Yuriev
f355f247c3 mdbx: add inlined mdbx_env_get_syncbytes() & mdbx_env_get_syncperiod(). 2021-12-03 21:55:40 +03:00
Leonid Yuriev
4b886e2a58 mdbx: don't return MDBX_RESULT_TRUE (-1) from mdbx_env_set_option(). 2021-12-03 15:33:22 +03:00
Леонид Юрьев (Leonid Yuriev)
d47eed079e mdbx: release v0.11.2
Acknowledgements:
-----------------

 - [장세연 (Чан Се Ен)](https://github.com/sasgas) for contributing to C++ API.
 - [Alain Picard](https://github.com/castortech) for [Java bindings](https://github.com/castortech/mdbxjni).
 - [Alex Sharov](https://github.com/AskAlexSharov) for reporting and testing.
 - [Kris Zyp](https://github.com/kriszyp) for reporting and testing.
 - [Artem Vorotnikov](https://github.com/vorot93) for support [Rust wrapper](https://github.com/vorot93/libmdbx-rs).

Fixes:
------

 - [Fixed compilation](https://github.com/erthink/libmdbx/pull/239) with `devtoolset-9` on CentOS/RHEL 7.
 - [Fixed unexpected `MDBX_PROBLEM` error](https://github.com/erthink/libmdbx/issues/242) because of update an obsolete meta-page.
 - [Fixed returning `MDBX_NOTFOUND` error](https://github.com/erthink/libmdbx/issues/243) in case an inexact value found for `MDBX_GET_BOTH` operation.
 - [Fixed compilation](https://github.com/erthink/libmdbx/issues/245) without kernel/libc-devel headers.

Minors:
-------

 - Fixed `constexpr`-related macros for legacy compilers.
 - Allowed to define 'CMAKE_CXX_STANDARD` using an environment variable.
 - Simplified collection statistics of page operation .
 - Added `MDBX_FORCE_BUILD_AS_MAIN_PROJECT` cmake option.
 - Remove unneeded `#undef P_DIRTY`.

Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2021-12-02 21:55:52 +03:00
Леонид Юрьев (Leonid Yuriev)
d2b15b5958 mdbx: more unlikely(). 2021-12-02 20:22:37 +03:00
Леонид Юрьев (Leonid Yuriev)
d96bc98244 mdbx: minor refine/fix MDBX_ENODATA for compatibility.
Related to https://github.com/erthink/libmdbx/issues/243
2021-12-02 20:22:37 +03:00
Леонид Юрьев (Leonid Yuriev)
c2cab7a6a8 mdbx: using clang-format-13 (cosmetics). 2021-12-02 20:22:31 +03:00
Леонид Юрьев (Leonid Yuriev)
38df3ca5ad mdbx: update/fix new libmdbx-rs library name. 2021-12-02 15:10:06 +03:00
Leonid Yuriev
5e4b2c9ddf mdbx: add BTC address for donations/sponsorship. 2021-11-27 17:02:27 +03:00
Leonid Yuriev
d777f5bb38 mdbx: update ChangeLog. 2021-11-27 00:29:36 +03:00
Leonid Yuriev
e912f87b2a mdbx: clarify notes about custom comparators usage. 2021-11-26 23:49:38 +03:00
Leonid Yuriev
0cc3aa3af8 mdbx: remove unneeded #undef P_DIRTY. 2021-11-26 17:51:37 +03:00
Leonid Yuriev
adac03729d mdbx: remove unneeded usage of <linux/sysctl.h>.
Fixed https://github.com/erthink/libmdbx/issues/245
2021-11-26 17:34:04 +03:00
Leonid Yuriev
17a14ec25f mdbx: update/fix new lmdbx-js library name. 2021-11-26 00:31:00 +03:00
Leonid Yuriev
4e73cdf9c6 mdbx: update ChangeLog. 2021-11-26 00:24:14 +03:00
Leonid Yuriev
66c354baff mdbx-test: add seek-test for MDBX_GET_BOTH.
Related to https://github.com/erthink/libmdbx/issues/243
2021-11-25 19:19:15 +03:00
Leonid Yuriev
76399bd643 mdbx: fix returning MDBX_NOTFOUND for non-exact seek case of MDBX_GET_BOTH.
Fixed https://github.com/erthink/libmdbx/issues/243
2021-11-25 19:19:04 +03:00
Leonid Yuriev
4f2aba2d22 mdbx: minor fix formatting (cosmetics). 2021-11-25 19:15:02 +03:00
Leonid Yuriev
085a97f835 mdbx: define MDBX_ENODATA == 9919 on systems without ENODATA.
As workaround for incompatibility C and C++ code using LLVM's C++ libraries/headers on on systems without native `ENODATA`.
2021-11-22 16:09:08 +03:00
Leonid Yuriev
a2141ceaac mdbx: slightly more cases to return MDBX_ENODATA. 2021-11-22 13:48:08 +03:00
Leonid Yuriev
9d55d06a20 mdbx-test: add check for MDBX_ENODATA condition. 2021-11-22 13:43:51 +03:00
Leonid Yuriev
e93d53ed92 mdbx: update ChangeLog. 2021-11-21 02:38:14 +03:00
Leonid Yuriev
8cb7c0f4fb mdbx: fix MDBX_PROBLEM while update an obsolete meta-page.
Fixes https://github.com/erthink/libmdbx/issues/242
2021-11-21 02:37:56 +03:00
Leonid Yuriev
773172cc99 mdbx: minor clarify descriptions of the MDBX_commit_latency fields. 2021-11-19 18:36:14 +03:00
Leonid Yuriev
937b38371f mdbx: remove obsolete remark from the reference to Java bindings. 2021-11-19 17:00:29 +03:00
Леонид Юрьев (Leonid Yuriev)
c312fead97 mdbx: update ChangeLog. 2021-11-19 16:21:44 +03:00
Леонид Юрьев (Leonid Yuriev)
bfff6cfe85 mdbx: fix nasty sizeof(bytes) typo. 2021-11-19 16:21:44 +03:00
Leonid Yuriev
86c735637e mdbx-cmake: add MDBX_FORCE_BUILD_AS_MAIN_PROJECT. 2021-11-19 16:21:40 +03:00
Leonid Yuriev
ff26d30362 mdbx: minor clarity enum MDBX_db_flags_t (database flags) descriptions.
Related to https://github.com/erthink/libmdbx/issues/241
2021-11-12 19:38:48 +03:00
Leonid Yuriev
79a5802ad4 mdbx: add the note about "Visual Studio 2015 Update 3". 2021-11-11 20:38:39 +03:00
Leonid Yuriev
0e2ca3eb51 mdbx: more parallelable CMP2INT for E2K. 2021-11-10 03:17:59 +03:00
Leonid Yuriev
e488604448 mdbx: minor fix mdbx_jitter4testing() for case MDBX_DEBUG >= 2. 2021-11-10 03:05:51 +03:00
Leonid Yuriev
eaa77c91cd mdbx: drop obsolete internal mm_flags macro. 2021-11-10 02:43:43 +03:00
Leonid Yuriev
8ed0a5946b mdbx: update ChangeLog. 2021-11-10 00:34:45 +03:00
Leonid Yuriev
0cd7dfb465 mdbx-ci: add dll-path-workaround to MinGW scenario with minor cleanup. 2021-11-10 00:23:01 +03:00
Leonid Yuriev
f38b1dab13 mdbx-ci: avoid spelling warnings. 2021-11-09 13:31:23 +03:00
sasgas
74d5a42578 mdbx: fix compilation with devtoolset-9 on CentOS/RHEL 7.
devtoolset is always using the old ABI
https://bugzilla.redhat.com/show_bug.cgi?id=1546704
https://stackoverflow.com/questions/49393888/how-can-i-use-the-new-c-11-abi-with-devtoolset-7-on-centos-rhel
2021-11-09 13:29:23 +03:00
Leonid Yuriev
6ecadba69a mdbx: update ChangeLog. 2021-11-08 18:37:21 +03:00
Leonid Yuriev
b46e5c4ce8 mdbx: simplify collection of page-ops statistic. 2021-11-07 22:14:33 +03:00
Leonid Yuriev
96139eef48 mdbx: a whole snapshot inside mdbx_env_info_ex(). 2021-11-07 21:18:10 +03:00
Leonid Yuriev
0c3a5da3cb mdbx: refine visibility of internal mdbx_osal_jitter(). 2021-11-07 19:48:14 +03:00
Leonid Yuriev
c456625ef2 mdbx-cmake: initial support for C++23. 2021-11-07 02:18:56 +03:00
Leonid Yuriev
630ef98951 mdbx-cmake: allow to define CMAKE_CXX_STANDARD by the environment variable. 2021-11-07 02:18:56 +03:00
Leonid Yuriev
7179823d56 mdbx-make: extpanded list of c++std for probing. 2021-11-07 02:18:56 +03:00
Leonid Yuriev
8870d33fcd mdbx++: refine MDBX_CXX01_CONSTEXPR for legacy compilers.
Enable `constexpr` via `MDBX_CXX01_CONSTEXPR` if __cplusplus == 201103L but __cpp_constexpr is undefined.
2021-11-07 02:18:56 +03:00
Leonid Yuriev
710fc95d9a mdbx: update patch for old buildroot versions. 2021-10-24 20:43:37 +03:00
Leonid Yuriev
93a24abbab mdbx: fix minor/paranoid MSVC warning. 2021-10-24 02:28:28 +03:00
Leonid Yuriev
113162b651 mdbx: release v0.11.1
Backward compatibility break:
-----------------------------

The database format signature has been changed to prevent
forward-interoperability with an previous releases, which may lead to a
[false positive diagnosis of database corruption](https://github.com/erthink/libmdbx/issues/238)
due to flaws of an old library versions.

This change is mostly invisible:

 - previously versions are unable to read/write a new DBs;
 - but the new release is able to handle an old DBs and will silently upgrade ones.

Acknowledgements:
-----------------
 - [Alex Sharov](https://github.com/AskAlexSharov) for reporting and testing.

Signed-off-by: Leonid Yuriev <leo@yuriev.ru>
2021-10-23 20:15:50 +03:00
Leonid Yuriev
5babf0872e mdbx++: add ifndef-guard for _CRT_SECURE_NO_WARNINGS. 2021-10-22 20:14:12 +03:00
Leonid Yuriev
331232858a adds updating db-format signature during DB open.
Fixes https://github.com/erthink/libmdbx/issues/238.
2021-10-22 20:13:57 +03:00
24 changed files with 1179 additions and 533 deletions

2
.github/FUNDING.yml vendored
View File

@@ -9,4 +9,4 @@ community_bridge: # Replace with a single Community Bridge project-name e.g., cl
liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username
otechie: # Replace with a single Otechie username
custom: ['https://sobe.ru/na/libmdbx', 'https://paypal.me/erthink', 'https://etherscan.io/address/0x19291d8658f762f3baceae1700c0b9466572ceab']
custom: ['https://sobe.ru/na/libmdbx', 'https://paypal.me/erthink', 'https://www.blockchain.com/en/eth/address/0x19291d8658f762f3baceae1700c0b9466572ceab', 'https://www.blockchain.com/en/btc/address/152u2KXNWWGHQS3qiBEoQaveWyPvaSWYGC']

View File

@@ -68,6 +68,7 @@ asprintf
aspx
assection
AstraLinux
astralinux
atal
atexit
atfork
@@ -99,6 +100,7 @@ bfin
biarch
bibtex
BIGDATA
bigdata
bindir
binfmt
binutils
@@ -458,6 +460,7 @@ EPERM
EPIPE
erasevolume
EREMOTE
Erigon
EROFS
errcode
errno
@@ -574,6 +577,7 @@ gcda
GCFREEZE
gcno
gcov
gcxfd
gdiplus
getenv
gethostid
@@ -600,6 +604,7 @@ github
githubusercontent
gitignore
glibc
GLIBCXX
globals
gmail
gmake
@@ -709,6 +714,7 @@ inprocess
INSTEADOF
integerdup
integerkey
interoperability
interprocedural
intlimits
intptr
@@ -978,6 +984,7 @@ microsoft
Mikkelson
minflt
MINGW
mingw
minimalistic
minkeys
minlen
@@ -1276,6 +1283,7 @@ pmax
pmccntr
pmcntenset
pmedvedev
pmeta
pmr
pmuseren
pmwkaa
@@ -1315,6 +1323,7 @@ proba
proces
procfs
progname
programdata
PROGRAMLISTING
projectbrief
projectlogo
@@ -1403,11 +1412,13 @@ REALMEM
REALPATH
realtime
Rebuffo
rebuffo
RECO
redis
reedom
reefont
refname
refreshenv
REGEX
regexp
reinited
@@ -1436,6 +1447,7 @@ rlock
rlt
rmdir
RMID
rmw
rocksdb
rolledback
roolback
@@ -1812,6 +1824,7 @@ unspilling
unsync
UNTRACK
updation
UPPERBOUND
upsert
UPSERTING
upsertion

View File

@@ -27,38 +27,39 @@ jobs:
- uses: actions/checkout@v2
- name: fetch tags
run: git fetch --unshallow --tags --prune --force
- name: append PATH
run: echo 'C:/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/bin' >> $GITHUB_PATH
- name: update mingw64
# wanna version >= 10.2
run: choco upgrade mingw -y --no-progress
run: choco upgrade mingw -y --no-progress && refreshenv
- name: configure-dll
run: mkdir build-dll && cd build-dll && cmake -G "MinGW Makefiles" -DMDBX_BUILD_SHARED_LIBRARY:BOOL=ON -DMDBX_INSTALL_STATIC:BOOL=OFF ..
shell: bash
run: |
mkdir build-dll && cd build-dll && \
cmake -G "MinGW Makefiles" -DMDBX_BUILD_SHARED_LIBRARY:BOOL=ON -DMDBX_INSTALL_STATIC:BOOL=OFF -DMDBX_ENABLE_TESTS:BOOL=ON ..
- name: build-dll
shell: bash
run: cd build-dll && cmake --build .
- name: test-dll
shell: pwsh
shell: bash
run: |
cd build-dll
ls
./mdbx_test --progress --console=no --pathname=test.db --dont-cleanup-after basic > test.log
Get-Content test.log | Select-Object -last 42
if ($LastExitCode -ne 0) {
throw "Exec: $ErrorMessage"
} else {
./mdbx_chk -nvv test.db | Tee-Object -file chk.log | Select-Object -last 42
}
export "PATH=/c/programdata/chocolatey/lib/mingw/tools/install/mingw64/bin:$PATH" && \
cd build-dll && \
./mdbx_test.exe --progress --console=no --pathname=test.db --dont-cleanup-after basic && \
./mdbx_chk.exe -nvv test.db
- name: configure-static
run: mkdir build-static && cd build-static && cmake -G "MinGW Makefiles" -DMDBX_BUILD_SHARED_LIBRARY:BOOL=OFF -DMDBX_INSTALL_STATIC:BOOL=ON ..
- name: build-static
run: cd build-static && cmake --build .
- name: test-static
shell: pwsh
shell: bash
run: |
cd build-static
ls
./mdbx_test --progress --console=no --pathname=test.db --dont-cleanup-after basic > test.log
Get-Content test.log | Select-Object -last 42
if ($LastExitCode -ne 0) {
throw "Exec: $ErrorMessage"
} else {
./mdbx_chk -nvv test.db | Tee-Object -file chk.log | Select-Object -last 42
}
mkdir build-static && cd build-static && \
cmake -G "MinGW Makefiles" -DMDBX_BUILD_SHARED_LIBRARY:BOOL=OFF -DMDBX_INSTALL_STATIC:BOOL=ON -DMDBX_ENABLE_TESTS:BOOL=ON ..
- name: build-static
shell: bash
run: |
cd build-static && cmake --build .
- name: test-static
shell: bash
run: |
export "PATH=/c/programdata/chocolatey/lib/mingw/tools/install/mingw64/bin:$PATH" && \
cd build-static && \
./mdbx_test.exe --progress --console=no --pathname=test.db --dont-cleanup-after basic && \
./mdbx_chk.exe -nvv test.db

View File

@@ -93,6 +93,10 @@ else()
endif()
if(DEFINED PROJECT_NAME)
option(MDBX_FORCE_BUILD_AS_MAIN_PROJECT "Force libmdbx to full control build options even it added as a subdirectory to your project." OFF)
endif()
if(DEFINED PROJECT_NAME AND NOT MDBX_FORCE_BUILD_AS_MAIN_PROJECT)
set(SUBPROJECT ON)
set(NOT_SUBPROJECT OFF)
if(NOT MDBX_AMALGAMATED_SOURCE AND NOT DEFINED BUILD_TESTING)
@@ -294,11 +298,11 @@ else()
if(UNIX)
find_program(CLANG_FORMAT
NAMES clang-format-12 clang-format-11 clang-format-10 clang-format)
NAMES clang-format-13 clang-format)
if(CLANG_FORMAT)
execute_process(COMMAND ${CLANG_FORMAT} "--version" OUTPUT_VARIABLE clang_format_version_info)
string(REGEX MATCH "version ([0-9]+)\\.([0-9]+)\\.([0-9]+)(.*)?" clang_format_version_info CLANG_FORMAT_VERSION)
if(clang_format_version_info AND NOT CLANG_FORMAT_VERSION VERSION_LESS 10.0)
if(clang_format_version_info AND NOT CLANG_FORMAT_VERSION VERSION_LESS 13.0)
# Enable 'make reformat' target.
add_custom_target(reformat
VERBATIM
@@ -334,11 +338,18 @@ list(FIND CMAKE_CXX_COMPILE_FEATURES cxx_std_11 HAS_CXX11)
list(FIND CMAKE_CXX_COMPILE_FEATURES cxx_std_14 HAS_CXX14)
list(FIND CMAKE_CXX_COMPILE_FEATURES cxx_std_17 HAS_CXX17)
list(FIND CMAKE_CXX_COMPILE_FEATURES cxx_std_20 HAS_CXX20)
list(FIND CMAKE_CXX_COMPILE_FEATURES cxx_std_23 HAS_CXX23)
if(NOT DEFINED MDBX_CXX_STANDARD)
if(DEFINED ENV{CMAKE_CXX_STANDARD})
set(CMAKE_CXX_STANDARD $ENV{CMAKE_CXX_STANDARD})
endif()
if(DEFINED CMAKE_CXX_STANDARD)
set(MDBX_CXX_STANDARD ${CMAKE_CXX_STANDARD})
elseif(NOT HAS_CXX23 LESS 0
AND NOT (CMAKE_COMPILER_IS_CLANG AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 12))
set(MDBX_CXX_STANDARD 23)
elseif(NOT HAS_CXX20 LESS 0
AND NOT (CMAKE_COMPILER_IS_CLANG AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 9))
AND NOT (CMAKE_COMPILER_IS_CLANG AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 10))
set(MDBX_CXX_STANDARD 20)
elseif(NOT HAS_CXX17 LESS 0
AND NOT (CMAKE_COMPILER_IS_CLANG AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 5))
@@ -423,7 +434,7 @@ endif()
# #### # # # #### # # ####
#
set(MDBX_BUILD_OPTIONS ENABLE_ASAN MDBX_USE_VALGRIND ENABLE_GPROF ENABLE_GCOV)
set(MDBX_BUILD_OPTIONS ENABLE_UBSAN ENABLE_ASAN MDBX_USE_VALGRIND ENABLE_GPROF ENABLE_GCOV)
macro(add_mdbx_option NAME DESCRIPTION DEFAULT)
list(APPEND MDBX_BUILD_OPTIONS ${NAME})
if(NOT ${DEFAULT} STREQUAL "AUTO")
@@ -852,7 +863,7 @@ else()
endif()
# options
set(options VERSION C_COMPILER CXX_COMPILER MDBX_BUILD_TARGET MDBX_BUILD_TYPE)
set(options VERSION C_COMPILER CXX_COMPILER MDBX_BUILD_TARGET MDBX_BUILD_TYPE ${MDBX_BUILD_OPTIONS})
foreach(item IN LISTS options)
if(DEFINED ${item})
set(value "${${item}}")
@@ -863,7 +874,7 @@ foreach(item IN LISTS options)
set(item CMAKE_${item})
set(value "${${item}}")
else()
set(value "undefined")
set(value "AUTO (not pre-defined explicitly)")
endif()
message(STATUS "${item}: ${value}")
endforeach(item)

View File

@@ -1,8 +1,6 @@
ChangeLog
---------
## v0.11.x (in development)
### TODO
- [Engage an "overlapped I/O" on Windows](https://github.com/erthink/libmdbx/issues/224).
@@ -16,11 +14,96 @@ ChangeLog
- [Support for RAW devices](https://github.com/erthink/libmdbx/issues/124).
- [Support MessagePack for Keys & Values](https://github.com/erthink/libmdbx/issues/115).
- [Engage new terminology](https://github.com/erthink/libmdbx/issues/137).
- Finalize C++ API (few typos and bugs are still maybe for now).
- Packages for [Astra Linux](https://astralinux.ru/), [ALT Linux](https://www.altlinux.org/), [ROSA Linux](https://www.rosalinux.ru/), etc.
## v0.10.5 at 2021-10-13
## v0.11.3 at 2021-12-31
Acknowledgements:
- [gcxfd <i@rmw.link>](https://github.com/gcxfd) for reporting, contributing and testing.
- [장세연 (Чан Се Ен)](https://github.com/sasgas) for reporting and testing.
- [Alex Sharov](https://github.com/AskAlexSharov) for reporting, testing and provide resources for remote debugging/investigation.
New features, extensions and improvements:
- [Added](https://github.com/erthink/libmdbx/issues/236) `mdbx_cursor_get_batch()`.
- [Added](https://github.com/erthink/libmdbx/issues/250) `MDBX_SET_UPPERBOUND`.
- C++ API is finalized now.
- The GC update stage has been [significantly speeded](https://github.com/erthink/libmdbx/issues/254) when fixing huge Erigon's transactions (Ethereum ecosystem).
Fixes:
- Disabled C++20 concepts for stupid AppleClang 13.x
- Fixed internal collision of `MDBX_SHRINK_ALLOWED` with `MDBX_ACCEDE`.
Minors:
- Fixed returning `MDBX_RESULT_TRUE` (unexpected -1) from `mdbx_env_set_option()`.
- Added `mdbx_env_get_syncbytes()` and `mdbx_env_get_syncperiod()`.
- [Clarified](https://github.com/erthink/libmdbx/pull/249) description of `MDBX_INTEGERKEY`.
- Reworked/simplified `mdbx_env_sync_internal()`.
- [Fixed](https://github.com/erthink/libmdbx/issues/248) extra assertion inside `mdbx_cursor_put()` for `MDBX_DUPFIXED` cases.
- Avoiding extra looping inside `mdbx_env_info_ex()`.
- Explicitly enabled core dumps from stochastic tests scripts on Linux.
- [Fixed](https://github.com/erthink/libmdbx/issues/253) `mdbx_override_meta()` to avoid false-positive assertions.
- For compatibility reverted returning `MDBX_ENODATA`for some cases.
## v0.11.2 at 2021-12-02
Acknowledgements:
- [장세연 (Чан Се Ен)](https://github.com/sasgas) for contributing to C++ API.
- [Alain Picard](https://github.com/castortech) for [Java bindings](https://github.com/castortech/mdbxjni).
- [Alex Sharov](https://github.com/AskAlexSharov) for reporting and testing.
- [Kris Zyp](https://github.com/kriszyp) for reporting and testing.
- [Artem Vorotnikov](https://github.com/vorot93) for support [Rust wrapper](https://github.com/vorot93/libmdbx-rs).
Fixes:
- [Fixed compilation](https://github.com/erthink/libmdbx/pull/239) with `devtoolset-9` on CentOS/RHEL 7.
- [Fixed unexpected `MDBX_PROBLEM` error](https://github.com/erthink/libmdbx/issues/242) because of update an obsolete meta-page.
- [Fixed returning `MDBX_NOTFOUND` error](https://github.com/erthink/libmdbx/issues/243) in case an inexact value found for `MDBX_GET_BOTH` operation.
- [Fixed compilation](https://github.com/erthink/libmdbx/issues/245) without kernel/libc-devel headers.
Minors:
- Fixed `constexpr`-related macros for legacy compilers.
- Allowed to define 'CMAKE_CXX_STANDARD` using an environment variable.
- Simplified collection statistics of page operation .
- Added `MDBX_FORCE_BUILD_AS_MAIN_PROJECT` cmake option.
- Remove unneeded `#undef P_DIRTY`.
## v0.11.1 at 2021-10-23
### Backward compatibility break:
The database format signature has been changed to prevent
forward-interoperability with an previous releases, which may lead to a
[false positive diagnosis of database corruption](https://github.com/erthink/libmdbx/issues/238)
due to flaws of an old library versions.
This change is mostly invisible:
- previously versions are unable to read/write a new DBs;
- but the new release is able to handle an old DBs and will silently upgrade ones.
Acknowledgements:
- [Alex Sharov](https://github.com/AskAlexSharov) for reporting and testing.
## v0.10.5 at 2021-10-13 (obsolete, please use v0.11.1)
Unfortunately, the `v0.10.5` accidentally comes not full-compatible with previous releases:
- `v0.10.5` can read/processing DBs created by previous releases, i.e. the backward-compatibility is provided;
- however, previous releases may lead to false-corrupted state with DB that was touched by `v0.10.5`, i.e. the forward-compatibility is broken for `v0.10.4` and earlier.
This cannot be fixed, as it requires fixing past versions, which as a result we will just get a current version.
Therefore, it is recommended to use `v0.11.1` instead of `v0.10.5`.
Acknowledgements:
@@ -35,14 +118,14 @@ Fixes:
Minors:
- Refined handling of weak or invalid meta-pages while a DB opening.
- Refined providing information for the @MAIN and @GC sub-databases of a last committed modification transaction's ID.
- Refined providing information for the `@MAIN` and `@GC` sub-databases of a last committed modification transaction's ID.
## v0.10.4 at 2021-10-10
Acknowledgements:
- [Artem Vorotnikov](https://github.com/vorot93) for support [Rust wrapper](https://github.com/vorot93/mdbx-rs).
- [Artem Vorotnikov](https://github.com/vorot93) for support [Rust wrapper](https://github.com/vorot93/libmdbx-rs).
- [Andrew Ashikhmin](https://github.com/yperbasis) for contributing to C++ API.
Fixes:

View File

@@ -35,7 +35,7 @@ CFLAGS ?= -std=gnu11 -O2 -g -Wall -Werror -Wextra -Wpedantic -ffunction-section
# -Wno-tautological-compare
CXX ?= g++
# Choosing C++ standard with deferred simple variable expansion trick
CXXSTD ?= $(eval CXXSTD := $$(shell PROBE=$$$$([ -f mdbx.c++ ] && echo mdbx.c++ || echo src/mdbx.c++); for std in gnu++20 c++20 gnu++2a c++2a gnu++17 c++17 gnu++14 c++14 gnu+11 c++11; do $(CXX) -std=$$$${std} -c $$$${PROBE} -o /dev/null 2>/dev/null >/dev/null && echo "-std=$$$${std}" && exit; done))$(CXXSTD)
CXXSTD ?= $(eval CXXSTD := $$(shell PROBE=$$$$([ -f mdbx.c++ ] && echo mdbx.c++ || echo src/mdbx.c++); for std in gnu++23 c++23 gnu++2b c++2b gnu++20 c++20 gnu++2a c++2a gnu++17 c++17 gnu++1z c++1z gnu++14 c++14 gnu++1y c++1y gnu+11 c++11 gnu++0x c++0x; do $(CXX) -std=$$$${std} -c $$$${PROBE} -o /dev/null 2>std-$$$${std}.err >/dev/null && echo "-std=$$$${std}" && exit; done))$(CXXSTD)
CXXFLAGS = $(CXXSTD) $(filter-out -std=gnu11,$(CFLAGS))
# TIP: Try append '--no-as-needed,-lrt' for ability to built with modern glibc, but then use with the old.

View File

@@ -2,7 +2,7 @@
> Please refer to the online [documentation](https://erthink.github.io/libmdbx/)
> with [`C` API description](https://erthink.github.io/libmdbx/group__c__api.html)
> and pay attention to the preliminary [`C++` API](https://github.com/erthink/libmdbx/blob/devel/mdbx.h%2B%2B).
> and pay attention to the [`C++` API](https://github.com/erthink/libmdbx/blob/devel/mdbx.h%2B%2B).
>
> Questions, feedback and suggestions are welcome to the [Telegram' group](https://t.me/libmdbx).
>
@@ -565,15 +565,16 @@ Bindings
| Runtime | Repo | Author |
| ------- | ------ | ------ |
| Haskell | [libmdbx-hs](https://hackage.haskell.org/package/libmdbx) | [Francisco Vallarino](https://github.com/fjvallarino) |
| Python (draft) | [python-bindings](https://github.com/erthink/libmdbx/commits/python-bindings) branch | [Noel Kuntze](https://github.com/Thermi)
| NodeJS | [lmdbx-store](https://github.com/kriszyp/lmdbx-store) | [Kris Zyp](https://github.com/kriszyp/)
| NodeJS | [lmdbx-js](https://github.com/kriszyp/lmdbx-js) | [Kris Zyp](https://github.com/kriszyp/)
| NodeJS | [node-mdbx](https://www.npmjs.com/package/node-mdbx/) | [Сергей Федотов](mailto:sergey.fedotov@corp.mail.ru) |
| Ruby | [ruby-mdbx](https://rubygems.org/gems/mdbx/) | [Mahlon E. Smith](https://github.com/mahlonsmith) |
| Go | [mdbx-go](https://github.com/torquem-ch/mdbx-go) | [Alex Sharov](https://github.com/AskAlexSharov) |
| Ruby | [ruby-mdbx](https://rubygems.org/gems/mdbx/) | [Mahlon E. Smith](https://github.com/mahlonsmith) |
| Go | [mdbx-go](https://github.com/torquem-ch/mdbx-go) | [Alex Sharov](https://github.com/AskAlexSharov) |
| [Nim](https://en.wikipedia.org/wiki/Nim_(programming_language)) | [NimDBX](https://github.com/snej/nimdbx) | [Jens Alfke](https://github.com/snej)
| Rust | [mdbx-rs](https://github.com/vorot93/mdbx-rs) | [Artem Vorotnikov](https://github.com/vorot93) |
| Java (obsolete) | [mdbxjni](https://github.com/castortech/mdbxjni) | [Castor Technologies](https://castortech.com/) |
| .NET (obsolete) | [mdbx.NET](https://github.com/wangjia184/mdbx.NET) | [Jerry Wang](https://github.com/wangjia184) |
| Rust | [libmdbx-rs](https://github.com/vorot93/libmdbx-rs) | [Artem Vorotnikov](https://github.com/vorot93) |
| Rust | [mdbx](https://crates.io/crates/mdbx) | [gcxfd](https://github.com/gcxfd) |
| Java | [mdbxjni](https://github.com/castortech/mdbxjni) | [Castor Technologies](https://castortech.com/) |
| Python (draft) | [python-bindings](https://github.com/erthink/libmdbx/commits/python-bindings) branch | [Noel Kuntze](https://github.com/Thermi)
| .NET (obsolete) | [mdbx.NET](https://github.com/wangjia184/mdbx.NET) | [Jerry Wang](https://github.com/wangjia184) |
<!-- section-end -->

View File

@@ -1,4 +1,4 @@
version: 0.11.0.{build}
version: 0.11.3.{build}
environment:
matrix:

View File

@@ -178,7 +178,7 @@ if(NOT DEFINED CMAKE_TARGET_BITNESS)
endif()
endif()
if(CMAKE_COMPILER_IS_ELBRUSC OR CMAKE_SYSTEM_PROCESSOR MATCHES "e2k.*|E2K.*|elbrus.*|ELBRUS.*")
if(CMAKE_COMPILER_IS_ELBRUSC OR CMAKE_COMPILER_IS_ELBRUSCXX OR CMAKE_SYSTEM_PROCESSOR MATCHES "e2k.*|E2K.*|elbrus.*|ELBRUS.*")
set(E2K TRUE)
set(CMAKE_SYSTEM_ARCH "Elbrus")
elseif((MSVC64 OR MINGW64) AND CMAKE_TARGET_BITNESS EQUAL 64)
@@ -271,7 +271,7 @@ if(CMAKE_CXX_COMPILER_LOADED)
endif()
# Check for LTO support by GCC
if(CMAKE_COMPILER_IS_GNU${CMAKE_PRIMARY_LANG})
if(CMAKE_COMPILER_IS_GNU${CMAKE_PRIMARY_LANG} AND NOT CMAKE_COMPILER_IS_ELBRUSC AND NOT CMAKE_COMPILER_IS_ELBRUSCXX)
unset(gcc_collect)
unset(gcc_lto_wrapper)

170
mdbx.h
View File

@@ -172,6 +172,9 @@ as a duplicates or as like a multiple values corresponds to keys.
#include <stdarg.h>
#include <stddef.h>
#include <stdint.h>
#if !defined(NDEBUG) && !defined(assert)
#include <assert.h>
#endif /* NDEBUG */
#if defined(_WIN32) || defined(_WIN64)
#include <windows.h>
@@ -393,7 +396,8 @@ typedef mode_t mdbx_mode_t;
#define MDBX_CXX01_CONSTEXPR __inline
#define MDBX_CXX01_CONSTEXPR_VAR const
#elif !defined(DOXYGEN) && \
(!defined(__cpp_constexpr) || __cpp_constexpr < 200704L || \
((__cplusplus < 201103L && defined(__cpp_constexpr) && \
__cpp_constexpr < 200704L) || \
(defined(__LCC__) && __LCC__ < 124) || \
(defined(__GNUC__) && (__GNUC__ * 100 + __GNUC_MINOR__ < 407) && \
!defined(__clang__) && !defined(__LCC__)) || \
@@ -410,7 +414,7 @@ typedef mode_t mdbx_mode_t;
#define MDBX_CXX11_CONSTEXPR __inline
#define MDBX_CXX11_CONSTEXPR_VAR const
#elif !defined(DOXYGEN) && \
(!defined(__cpp_constexpr) || __cpp_constexpr < 201304 || \
(!defined(__cpp_constexpr) || __cpp_constexpr < 201304L || \
(defined(__LCC__) && __LCC__ < 124) || \
(defined(__GNUC__) && __GNUC__ < 6 && !defined(__clang__) && \
!defined(__LCC__)) || \
@@ -1382,29 +1386,32 @@ DEFINE_ENUM_FLAG_OPERATORS(MDBX_txn_flags_t)
enum MDBX_db_flags_t {
MDBX_DB_DEFAULTS = 0,
/** Use reverse string keys */
/** Use reverse string comparison for keys. */
MDBX_REVERSEKEY = UINT32_C(0x02),
/** Use sorted duplicates, i.e. allow multi-values */
/** Use sorted duplicates, i.e. allow multi-values for a keys. */
MDBX_DUPSORT = UINT32_C(0x04),
/** Numeric keys in native byte order either uint32_t or uint64_t. The keys
* must all be of the same size and must be aligned while passing as
/** Numeric keys in native byte order either uint32_t or uint64_t
* (must be one of uint32_t or uint64_t, other integer types, for example,
* signed integer or uint16_t will not work).
* The keys must all be of the same size and must be aligned while passing as
* arguments. */
MDBX_INTEGERKEY = UINT32_C(0x08),
/** With \ref MDBX_DUPSORT; sorted dup items have fixed size */
/** With \ref MDBX_DUPSORT; sorted dup items have fixed size. The data values
* must all be of the same size. */
MDBX_DUPFIXED = UINT32_C(0x10),
/** With \ref MDBX_DUPSORT and with \ref MDBX_DUPFIXED; dups are fixed size
* \ref MDBX_INTEGERKEY -style integers. The data values must all be of the
* same size and must be aligned while passing as arguments. */
* like \ref MDBX_INTEGERKEY -style integers. The data values must all be of
* the same size and must be aligned while passing as arguments. */
MDBX_INTEGERDUP = UINT32_C(0x20),
/** With \ref MDBX_DUPSORT; use reverse string comparison */
/** With \ref MDBX_DUPSORT; use reverse string comparison for data values. */
MDBX_REVERSEDUP = UINT32_C(0x40),
/** Create DB if not already existing */
/** Create DB if not already existing. */
MDBX_CREATE = UINT32_C(0x40000),
/** Opens an existing sub-database created with unknown flags.
@@ -1565,18 +1572,33 @@ enum MDBX_cursor_op {
* a page of duplicate data items. */
MDBX_PREV_MULTIPLE,
/** Position at first key-value pair greater than or equal to specified,
* return both key and data, and the return code depends on a exact match.
/** Positions cursor at first key-value pair greater than or equal to
* specified, return both key and data, and the return code depends on whether
* a exact match.
*
* For non DUPSORT-ed collections this work the same to \ref MDBX_SET_RANGE,
* but returns \ref MDBX_SUCCESS if key found exactly and
* but returns \ref MDBX_SUCCESS if key found exactly or
* \ref MDBX_RESULT_TRUE if greater key was found.
*
* For DUPSORT-ed a data value is taken into account for duplicates,
* i.e. for a pairs/tuples of a key and an each data value of duplicates.
* Returns \ref MDBX_SUCCESS if key-value pair found exactly and
* Returns \ref MDBX_SUCCESS if key-value pair found exactly or
* \ref MDBX_RESULT_TRUE if the next pair was returned. */
MDBX_SET_LOWERBOUND
MDBX_SET_LOWERBOUND,
/** Positions cursor at first key-value pair greater than specified,
* return both key and data, and the return code depends on whether a
* upper-bound was found.
*
* For non DUPSORT-ed collections this work the same to \ref MDBX_SET_RANGE,
* but returns \ref MDBX_SUCCESS if the greater key was found or
* \ref MDBX_NOTFOUND otherwise.
*
* For DUPSORT-ed a data value is taken into account for duplicates,
* i.e. for a pairs/tuples of a key and an each data value of duplicates.
* Returns \ref MDBX_SUCCESS if the greater pair was returned or
* \ref MDBX_NOTFOUND otherwise. */
MDBX_SET_UPPERBOUND
};
#ifndef __cplusplus
/** \ingroup c_cursors */
@@ -1735,7 +1757,7 @@ enum MDBX_error_t {
#ifdef ENODATA
MDBX_ENODATA = ENODATA,
#else
MDBX_ENODATA = -1,
MDBX_ENODATA = 9919 /* for compatibility with LLVM's C++ libraries/headers */,
#endif /* ENODATA */
MDBX_EINVAL = EINVAL,
MDBX_EACCESS = EACCES,
@@ -2427,6 +2449,7 @@ LIBMDBX_INLINE_API(int, mdbx_env_sync_poll, (MDBX_env * env)) {
/** \brief Sets threshold to force flush the data buffers to disk, even any of
* \ref MDBX_SAFE_NOSYNC flag in the environment.
* \ingroup c_settings
* \see mdbx_env_get_syncbytes \see MDBX_opt_sync_bytes
*
* The threshold value affects all processes which operates with given
* environment until the last process close environment or a new value will be
@@ -2451,11 +2474,39 @@ LIBMDBX_INLINE_API(int, mdbx_env_set_syncbytes,
return mdbx_env_set_option(env, MDBX_opt_sync_bytes, threshold);
}
/** \brief Get threshold to force flush the data buffers to disk, even any of
* \ref MDBX_SAFE_NOSYNC flag in the environment.
* \ingroup c_statinfo
* \see mdbx_env_set_syncbytes() \see MDBX_opt_sync_bytes
*
* \param [in] env An environment handle returned
* by \ref mdbx_env_create().
* \param [out] threshold Address of an size_t to store
* the number of bytes of summary changes when
* a synchronous flush would be made.
*
* \returns A non-zero error value on failure and 0 on success,
* some possible errors are:
* \retval MDBX_EINVAL An invalid parameter was specified. */
LIBMDBX_INLINE_API(int, mdbx_env_get_syncbytes,
(const MDBX_env *env, size_t *threshold)) {
int rc = MDBX_EINVAL;
if (threshold) {
uint64_t proxy = 0;
rc = mdbx_env_get_option(env, MDBX_opt_sync_bytes, &proxy);
#ifdef assert
assert(proxy <= SIZE_MAX);
#endif /* assert */
*threshold = (size_t)proxy;
}
return rc;
}
/** \brief Sets relative period since the last unsteady commit to force flush
* the data buffers to disk, even of \ref MDBX_SAFE_NOSYNC flag in the
* environment.
*
* \ingroup c_settings
* \see mdbx_env_get_syncperiod \see MDBX_opt_sync_period
*
* The relative period value affects all processes which operates with given
* environment until the last process close environment or a new value will be
@@ -2486,6 +2537,36 @@ LIBMDBX_INLINE_API(int, mdbx_env_set_syncperiod,
return mdbx_env_set_option(env, MDBX_opt_sync_period, seconds_16dot16);
}
/** \brief Get relative period since the last unsteady commit to force flush
* the data buffers to disk, even of \ref MDBX_SAFE_NOSYNC flag in the
* environment.
* \ingroup c_statinfo
* \see mdbx_env_set_syncperiod() \see MDBX_opt_sync_period
*
* \param [in] env An environment handle returned
* by \ref mdbx_env_create().
* \param [out] period_seconds_16dot16 Address of an size_t to store
* the period in 1/65536 of second when
* a synchronous flush would be made since
* the last unsteady commit.
*
* \returns A non-zero error value on failure and 0 on success,
* some possible errors are:
* \retval MDBX_EINVAL An invalid parameter was specified. */
LIBMDBX_INLINE_API(int, mdbx_env_get_syncperiod,
(const MDBX_env *env, unsigned *period_seconds_16dot16)) {
int rc = MDBX_EINVAL;
if (period_seconds_16dot16) {
uint64_t proxy = 0;
rc = mdbx_env_get_option(env, MDBX_opt_sync_period, &proxy);
#ifdef assert
assert(proxy <= UINT32_MAX);
#endif /* assert */
*period_seconds_16dot16 = (unsigned)proxy;
}
return rc;
}
/** \brief Close the environment and release the memory map.
* \ingroup c_opening
*
@@ -3284,9 +3365,11 @@ struct MDBX_commit_latency {
uint32_t gc;
/** \brief Duration of internal audit if enabled. */
uint32_t audit;
/** \brief Duration of writing dirty/modified data pages. */
/** \brief Duration of writing dirty/modified data pages to a filesystem,
* i.e. the summary duration of a `write()` syscalls during commit. */
uint32_t write;
/** \brief Duration of syncing written data to the dist/storage. */
/** \brief Duration of syncing written data to the disk/storage, i.e.
* the duration of a `fdatasync()` or a `msync()` syscall during commit. */
uint32_t sync;
/** \brief Duration of transaction ending (releasing resources). */
uint32_t ending;
@@ -3509,10 +3592,14 @@ LIBMDBX_API int mdbx_canary_get(const MDBX_txn *txn, MDBX_canary *canary);
* The reasons to not using custom comparators are:
* - The order of records could not be validated without your code.
* So `mdbx_chk` utility will reports "wrong order" errors
* and the `-i` option is required to ignore ones.
* and the `-i` option is required to suppress ones.
* - A records could not be ordered or sorted without your code.
* So mdbx_load utility should be used with `-a` option to preserve
* input data order. */
* So `mdbx_load` utility should be used with `-a` option to preserve
* input data order.
* - However, the custom comparators feature will never be removed.
* You have been warned but still can use custom comparators knowing
* about the issues noted above. In this case you should ignore `deprecated`
* warnings or define `MDBX_DEPRECATED` macro to empty to avoid ones. */
typedef int(MDBX_cmp_func)(const MDBX_val *a,
const MDBX_val *b) MDBX_CXX17_NOEXCEPT;
@@ -4261,6 +4348,43 @@ LIBMDBX_API int mdbx_cursor_copy(const MDBX_cursor *src, MDBX_cursor *dest);
LIBMDBX_API int mdbx_cursor_get(MDBX_cursor *cursor, MDBX_val *key,
MDBX_val *data, MDBX_cursor_op op);
/** \brief Retrieve multiple non-dupsort key/value pairs by cursor.
* \ingroup c_crud
*
* This function retrieves multiple key/data pairs from the database without
* \ref MDBX_DUPSORT option. For `MDBX_DUPSORT` databases please
* use \ref MDBX_GET_MULTIPLE and \ref MDBX_NEXT_MULTIPLE.
*
* The number of key and value items is returned in the `size_t count`
* refers. The addresses and lengths of the keys and values are returned in the
* array to which `pairs` refers.
* \see mdbx_cursor_get()
*
* \param [in] cursor A cursor handle returned by \ref mdbx_cursor_open().
* \param [out] count The number of key and value item returned, on success
* it always be the even because the key-value
* pairs are returned.
* \param [in,out] pairs A pointer to the array of key value pairs.
* \param [in] limit The size of pairs buffer as the number of items,
* but not a pairs.
* \param [in] op A cursor operation \ref MDBX_cursor_op (only
* \ref MDBX_FIRST, \ref MDBX_NEXT, \ref MDBX_GET_CURRENT
* are supported).
*
* \returns A non-zero error value on failure and 0 on success,
* some possible errors are:
* \retval MDBX_THREAD_MISMATCH Given transaction is not owned
* by current thread.
* \retval MDBX_NOTFOUND No more key-value pairs are available.
* \retval MDBX_ENODATA The cursor is already at the end of data.
* \retval MDBX_RESULT_TRUE The specified limit is less than the available
* key-value pairs on the current page/position
* that the cursor points to.
* \retval MDBX_EINVAL An invalid parameter was specified. */
LIBMDBX_API int mdbx_cursor_get_batch(MDBX_cursor *cursor, size_t *count,
MDBX_val *pairs, size_t limit,
MDBX_cursor_op op);
/** \brief Store by cursor.
* \ingroup c_crud
*

View File

@@ -1,15 +1,16 @@
/// \file mdbx.h++
/// \brief The libmdbx C++ API header file (preliminary).
/// \brief The libmdbx C++ API header file.
///
/// \author Copyright (c) 2020-2021, Leonid Yuriev <leo@yuriev.ru>.
/// \copyright SPDX-License-Identifier: Apache-2.0
///
/// Tested with:
/// - LCC >= 1.23 (http://www.mcst.ru/lcc),
/// - GNU C++ >= 4.8,
/// - clang >= 4.0,
/// - MSVC >= 19.0 (Visual Studio 2015),
/// but 19.2x could hang due optimizer bug.
/// - Elbrus LCC >= 1.23 (http://www.mcst.ru/lcc);
/// - GNU C++ >= 4.8;
/// - clang >= 3.9;
/// - MSVC >= 14.0 (Visual Studio 2015),
/// but 19.2x could hang due optimizer bug;
/// - AppleClang, but without C++20 concepts.
///
#pragma once
@@ -19,7 +20,7 @@
#error "C++11 compiler or better is required"
#elif _MSC_VER >= 1910
#error \
"Please add ` /Zc:__cplusplus` to MSVC compiler options to enforce it conform ISO C++"
"Please add `/Zc:__cplusplus` to MSVC compiler options to enforce it conform ISO C++"
#endif /* MSVC is mad and don't define __cplusplus properly */
#endif /* __cplusplus < 201103L */
@@ -194,7 +195,9 @@
#ifndef MDBX_CXX20_CONCEPT
#if defined(DOXYGEN) || \
(defined(__cpp_concepts) && __cpp_concepts >= 201907L && \
(!defined(__clang__) || __clang_major__ >= 12))
(!defined(__clang__) || (__clang_major__ >= 12 && !defined(__APPLE__)) || \
__clang_major__ >= \
/* Hope Apple will fix concepts in AppleClang 14 */ 14))
#define MDBX_CXX20_CONCEPT(CONCEPT, NAME) CONCEPT NAME
#else
#define MDBX_CXX20_CONCEPT(CONCEPT, NAME) typename NAME
@@ -204,7 +207,9 @@
#ifndef MDBX_ASSERT_CXX20_CONCEPT_SATISFIED
#if defined(DOXYGEN) || \
(defined(__cpp_concepts) && __cpp_concepts >= 201907L && \
(!defined(__clang__) || __clang_major__ >= 12))
(!defined(__clang__) || (__clang_major__ >= 12 && !defined(__APPLE__)) || \
__clang_major__ >= \
/* Hope Apple will fix concepts in AppleClang 14 */ 14))
#define MDBX_ASSERT_CXX20_CONCEPT_SATISFIED(CONCEPT, TYPE) \
static_assert(CONCEPT<TYPE>)
#else
@@ -286,7 +291,8 @@ class cursor;
class cursor_managed;
#if defined(DOXYGEN) || \
defined(__cpp_lib_memory_resource) && __cpp_lib_memory_resource >= 201603L
(defined(__cpp_lib_memory_resource) && \
__cpp_lib_memory_resource >= 201603L && _GLIBCXX_USE_CXX11_ABI)
/// \brief Default polymorphic allocator for modern code.
using polymorphic_allocator = ::std::pmr::string::allocator_type;
#endif /* __cpp_lib_memory_resource >= 201603L */
@@ -472,34 +478,29 @@ static MDBX_CXX20_CONSTEXPR void *memcpy(void *dest, const void *src,
#if defined(DOXYGEN) || \
(defined(__cpp_concepts) && __cpp_concepts >= 201907L && \
(!defined(__clang__) || __clang_major__ >= 12))
(!defined(__clang__) || (__clang_major__ >= 12 && !defined(__APPLE__)) || \
__clang_major__ >= \
/* Hope Apple will fix concepts in AppleClang 14 */ 14))
template <typename T>
concept MutableByteProducer = requires(T a, char array[42]) {
{ a.is_empty() }
->std::same_as<bool>;
{ a.envisage_result_length() }
->std::same_as<size_t>;
{ a.write_bytes(&array[0], size_t(42)) }
->std::same_as<char *>;
{ a.is_empty() } -> std::same_as<bool>;
{ a.envisage_result_length() } -> std::same_as<size_t>;
{ a.write_bytes(&array[0], size_t(42)) } -> std::same_as<char *>;
};
template <typename T>
concept ImmutableByteProducer = requires(const T &a, char array[42]) {
{ a.is_empty() }
->std::same_as<bool>;
{ a.envisage_result_length() }
->std::same_as<size_t>;
{ a.write_bytes(&array[0], size_t(42)) }
->std::same_as<char *>;
{ a.is_empty() } -> std::same_as<bool>;
{ a.envisage_result_length() } -> std::same_as<size_t>;
{ a.write_bytes(&array[0], size_t(42)) } -> std::same_as<char *>;
};
template <typename T>
concept SliceTranscoder =
ImmutableByteProducer<T> &&requires(const slice &source, const T &a) {
concept SliceTranscoder = ImmutableByteProducer<T> &&
requires(const slice &source, const T &a) {
T(source);
{ a.is_erroneous() }
->std::same_as<bool>;
{ a.is_erroneous() } -> std::same_as<bool>;
};
#endif /* __cpp_concepts >= 201907L*/
@@ -3949,7 +3950,7 @@ inline ::std::ostream &operator<<(::std::ostream &out,
//==============================================================================
//
// Inline body of the libmdbx C++ API (preliminary draft)
// Inline body of the libmdbx C++ API
//
MDBX_CXX11_CONSTEXPR const version_info &get_version() noexcept {
@@ -4404,9 +4405,9 @@ MDBX_CXX14_CONSTEXPR intptr_t slice::compare_fast(const slice &a,
const slice &b) noexcept {
const intptr_t diff = intptr_t(a.length()) - intptr_t(b.length());
return diff ? diff
: MDBX_UNLIKELY(a.length() == 0 || a.data() == b.data())
? 0
: memcmp(a.data(), b.data(), a.length());
: MDBX_UNLIKELY(a.length() == 0 || a.data() == b.data())
? 0
: memcmp(a.data(), b.data(), a.length());
}
MDBX_CXX14_CONSTEXPR intptr_t

View File

@@ -1,9 +1,9 @@
From ce7b70a572cf77d8f37e5047d5d9c8361b2675ae Mon Sep 17 00:00:00 2001
From f732fc79456f3b296543ab2625d35eeef2655618 Mon Sep 17 00:00:00 2001
From: Leonid Yuriev <leo@yuriev.ru>
Date: Sun, 10 Oct 2021 15:31:40 +0300
Date: Sun, 24 Oct 2021 20:13:33 +0300
Subject: [PATCH] package/libmdbx: new package (library/database).
This patch adds libmdbx v0.10.4:
This patch adds libmdbx v0.11.1:
- libmdbx is one of the fastest compact embeddable key-value ACID database.
- libmdbx has a specific set of properties and capabilities,
focused on creating unique lightweight solutions.
@@ -103,18 +103,18 @@ index 0000000000..d13f73938f
+ !BR2_TOOLCHAIN_GCC_AT_LEAST_4_4
diff --git a/package/libmdbx/libmdbx.hash b/package/libmdbx/libmdbx.hash
new file mode 100644
index 0000000000..326cf57bb6
index 0000000000..c8b50f9ac3
--- /dev/null
+++ b/package/libmdbx/libmdbx.hash
@@ -0,0 +1,5 @@
+# Hashes from: https://github.com/erthink/libmdbx/releases/
+sha256 e11d5339a1e1cc34407898933b62a208936fd761a2cc31e11244d581d1d2b5d0 libmdbx-amalgamated-0.10.4.tar.gz
+sha256 f954ba8c9768914a92c2b46aac0d66bec674dbb4d7b0f01e362ea2921746ddaa libmdbx-amalgamated-0.11.1.tar.gz
+
+# Locally calculated
+sha256 310fe25c858a9515fc8c8d7d1f24a67c9496f84a91e0a0e41ea9975b1371e569 LICENSE
diff --git a/package/libmdbx/libmdbx.mk b/package/libmdbx/libmdbx.mk
new file mode 100644
index 0000000000..f38e4a533b
index 0000000000..02d00b1a5a
--- /dev/null
+++ b/package/libmdbx/libmdbx.mk
@@ -0,0 +1,42 @@
@@ -124,7 +124,7 @@ index 0000000000..f38e4a533b
+#
+################################################################################
+
+LIBMDBX_VERSION = 0.10.4
+LIBMDBX_VERSION = 0.11.1
+LIBMDBX_SOURCE = libmdbx-amalgamated-$(LIBMDBX_VERSION).tar.gz
+LIBMDBX_SITE = https://github.com/erthink/libmdbx/releases/download/v$(LIBMDBX_VERSION)
+LIBMDBX_SUPPORTS_IN_SOURCE_BUILD = NO
@@ -161,5 +161,5 @@ index 0000000000..f38e4a533b
+
+$(eval $(cmake-package))
--
2.33.0
2.33.1

File diff suppressed because it is too large Load Diff

View File

@@ -61,13 +61,26 @@
* Studio 2015 Update 3). But you could remove this #error and try to continue
* at your own risk. In such case please don't rise up an issues related ONLY to
* old compilers.
*
* NOTE:
* Unfortunately, there are several different builds of "Visual Studio" that
* are called "Visual Studio 2015 Update 3".
*
* The 190024234 is used here because it is minimal version of Visual Studio
* that was used for build and testing libmdbx in recent years. Soon this
* value will be increased to 19.0.24241.7, since build and testing using
* "Visual Studio 2015" will be performed only at https://ci.appveyor.com.
*
* Please ask Microsoft (but not us) for information about version differences
* and how to and where you can obtain the latest "Visual Studio 2015" build
* with all fixes.
*/
#error \
"At least \"Microsoft C/C++ Compiler\" version 19.00.24234 (Visual Studio 2015 Update 3) is required."
#endif
#ifndef _CRT_SECURE_NO_WARNINGS
#define _CRT_SECURE_NO_WARNINGS
#endif
#endif /* _CRT_SECURE_NO_WARNINGS */
#if _MSC_VER > 1800
#pragma warning(disable : 4464) /* relative include path contains '..' */
#endif
@@ -264,8 +277,6 @@ static __always_inline memory_order mo_c11_load(enum MDBX_memory_order fence) {
#ifndef __cplusplus
static __inline void mdbx_jitter4testing(bool tiny);
MDBX_MAYBE_UNUSED static __always_inline void
mdbx_memory_fence(enum MDBX_memory_order order, bool write) {
#ifdef MDBX_HAVE_C11ATOMICS
@@ -309,70 +320,6 @@ atomic_load32(const MDBX_atomic_uint32_t *p, enum MDBX_memory_order order) {
#endif /* MDBX_HAVE_C11ATOMICS */
}
MDBX_MAYBE_UNUSED static __always_inline uint64_t
atomic_store64(MDBX_atomic_uint64_t *p, const uint64_t value,
enum MDBX_memory_order order) {
STATIC_ASSERT(sizeof(MDBX_atomic_uint64_t) == 8);
#if MDBX_64BIT_ATOMIC
#ifdef MDBX_HAVE_C11ATOMICS
assert(atomic_is_lock_free(MDBX_c11a_rw(uint64_t, p)));
atomic_store_explicit(MDBX_c11a_rw(uint64_t, p), value, mo_c11_store(order));
#else /* MDBX_HAVE_C11ATOMICS */
if (order != mo_Relaxed)
mdbx_compiler_barrier();
p->weak = value;
mdbx_memory_fence(order, true);
#endif /* MDBX_HAVE_C11ATOMICS */
#else /* !MDBX_64BIT_ATOMIC */
mdbx_compiler_barrier();
atomic_store32(&p->low, (uint32_t)value, mo_Relaxed);
mdbx_jitter4testing(true);
atomic_store32(&p->high, (uint32_t)(value >> 32), order);
mdbx_jitter4testing(true);
#endif /* !MDBX_64BIT_ATOMIC */
return value;
}
MDBX_MAYBE_UNUSED static
#if MDBX_64BIT_ATOMIC
__always_inline
#endif /* MDBX_64BIT_ATOMIC */
uint64_t
atomic_load64(const MDBX_atomic_uint64_t *p,
enum MDBX_memory_order order) {
STATIC_ASSERT(sizeof(MDBX_atomic_uint64_t) == 8);
#if MDBX_64BIT_ATOMIC
#ifdef MDBX_HAVE_C11ATOMICS
assert(atomic_is_lock_free(MDBX_c11a_ro(uint64_t, p)));
return atomic_load_explicit(MDBX_c11a_ro(uint64_t, p), mo_c11_load(order));
#else /* MDBX_HAVE_C11ATOMICS */
mdbx_memory_fence(order, false);
const uint64_t value = p->weak;
if (order != mo_Relaxed)
mdbx_compiler_barrier();
return value;
#endif /* MDBX_HAVE_C11ATOMICS */
#else /* !MDBX_64BIT_ATOMIC */
mdbx_compiler_barrier();
uint64_t value = (uint64_t)atomic_load32(&p->high, order) << 32;
mdbx_jitter4testing(true);
value |= atomic_load32(&p->low, (order == mo_Relaxed) ? mo_Relaxed
: mo_AcquireRelease);
mdbx_jitter4testing(true);
for (;;) {
mdbx_compiler_barrier();
uint64_t again = (uint64_t)atomic_load32(&p->high, order) << 32;
mdbx_jitter4testing(true);
again |= atomic_load32(&p->low, (order == mo_Relaxed) ? mo_Relaxed
: mo_AcquireRelease);
mdbx_jitter4testing(true);
if (likely(value == again))
return value;
value = again;
}
#endif /* !MDBX_64BIT_ATOMIC */
}
#endif /* !__cplusplus */
/*----------------------------------------------------------------------------*/
@@ -488,8 +435,6 @@ typedef struct MDBX_meta {
MDBX_db mm_dbs[CORE_DBS]; /* first is free space, 2nd is main db */
/* The size of pages used in this DB */
#define mm_psize mm_dbs[FREE_DBI].md_xsize
/* Any persistent environment flags, see mdbx_env */
#define mm_flags mm_dbs[FREE_DBI].md_flags
MDBX_canary mm_canary;
#define MDBX_DATASIGN_NONE 0u
@@ -1257,6 +1202,15 @@ extern uint8_t mdbx_runtime_flags;
extern uint8_t mdbx_loglevel;
extern MDBX_debug_func *mdbx_debug_logger;
MDBX_MAYBE_UNUSED static __inline void mdbx_jitter4testing(bool tiny) {
#if MDBX_DEBUG
if (MDBX_DBG_JITTER & mdbx_runtime_flags)
mdbx_osal_jitter(tiny);
#else
(void)tiny;
#endif
}
MDBX_INTERNAL_FUNC void MDBX_PRINTF_ARGS(4, 5)
mdbx_debug_log(int level, const char *function, int line, const char *fmt,
...) MDBX_PRINTF_ARGS(4, 5);
@@ -1426,15 +1380,6 @@ MDBX_INTERNAL_FUNC void mdbx_rthc_global_init(void);
MDBX_INTERNAL_FUNC void mdbx_rthc_global_dtor(void);
MDBX_INTERNAL_FUNC void mdbx_rthc_thread_dtor(void *ptr);
MDBX_MAYBE_UNUSED static __inline void mdbx_jitter4testing(bool tiny) {
#if MDBX_DEBUG
if (MDBX_DBG_JITTER & mdbx_runtime_flags)
mdbx_osal_jitter(tiny);
#else
(void)tiny;
#endif
}
#endif /* !__cplusplus */
#define MDBX_IS_ERROR(rc) \
@@ -1569,10 +1514,11 @@ typedef struct MDBX_node {
* | 1, a > b
* \
*/
#if 1
#ifndef __e2k__
/* LY: fast enough on most systems */
#define CMP2INT(a, b) (((b) > (a)) ? -1 : (a) > (b))
#else
/* LY: more parallelable on VLIW Elbrus */
#define CMP2INT(a, b) (((a) > (b)) - ((b) > (a)))
#endif

View File

@@ -2,12 +2,12 @@
// Copyright (c) 2020-2021, Leonid Yuriev <leo@yuriev.ru>.
// SPDX-License-Identifier: Apache-2.0
//
// Non-inline part of the libmdbx C++ API (preliminary)
// Non-inline part of the libmdbx C++ API
//
#ifdef _MSC_VER
#if defined(_MSC_VER) && !defined(_CRT_SECURE_NO_WARNINGS)
#define _CRT_SECURE_NO_WARNINGS
#endif
#endif /* _CRT_SECURE_NO_WARNINGS */
#include "../mdbx.h++"
@@ -992,7 +992,8 @@ bool from_base64::is_erroneous() const noexcept {
template class LIBMDBX_API_TYPE buffer<legacy_allocator>;
#if defined(__cpp_lib_memory_resource) && __cpp_lib_memory_resource >= 201603L
#if defined(__cpp_lib_memory_resource) && \
__cpp_lib_memory_resource >= 201603L && _GLIBCXX_USE_CXX11_ABI
template class LIBMDBX_API_TYPE buffer<polymorphic_allocator>;
#endif /* __cpp_lib_memory_resource >= 201603L */
@@ -1428,7 +1429,7 @@ __cold ::std::ostream &operator<<(::std::ostream &out,
}
const auto bytes = (it.bytes < 0) ? out << "-",
size_t(-it.bytes) : size_t(it.bytes);
size_t(-it.bytes) : size_t(it.bytes);
struct {
size_t one;
const char *suffix;

View File

@@ -1461,8 +1461,9 @@ MDBX_INTERNAL_FUNC int mdbx_mmap(const int flags, mdbx_mmap_t *map,
if (!NT_SUCCESS(err))
return ntstatus2errcode(err);
SIZE_T ViewSize =
(flags & MDBX_RDONLY) ? 0 : mdbx_RunningUnderWine() ? size : limit;
SIZE_T ViewSize = (flags & MDBX_RDONLY) ? 0
: mdbx_RunningUnderWine() ? size
: limit;
err = NtMapViewOfSection(
map->section, GetCurrentProcess(), &map->address,
/* ZeroBits */ 0,

View File

@@ -33,7 +33,7 @@
#if defined(_WIN32) || defined(_WIN64)
#if !defined(_CRT_SECURE_NO_WARNINGS)
#define _CRT_SECURE_NO_WARNINGS
#endif
#endif /* _CRT_SECURE_NO_WARNINGS */
#if !defined(_NO_CRT_STDIO_INLINE) && MDBX_BUILD_SHARED_LIBRARY && \
!defined(xMDBX_TOOLS) && MDBX_WITHOUT_MSVC_CRT
#define _NO_CRT_STDIO_INLINE
@@ -114,11 +114,9 @@
#include <mach/mach_host.h>
#include <mach/mach_port.h>
#include <uuid/uuid.h>
#undef P_DIRTY
#endif
#if defined(__linux__) || defined(__gnu_linux__)
#include <linux/sysctl.h>
#include <sched.h>
#include <sys/sendfile.h>
#include <sys/statfs.h>
@@ -481,9 +479,10 @@ typedef union MDBX_srwlock {
} MDBX_srwlock;
#endif /* Windows */
#ifdef __cplusplus
extern void mdbx_osal_jitter(bool tiny);
#else
#ifndef __cplusplus
MDBX_MAYBE_UNUSED MDBX_INTERNAL_FUNC void mdbx_osal_jitter(bool tiny);
MDBX_MAYBE_UNUSED static __inline void mdbx_jitter4testing(bool tiny);
/*----------------------------------------------------------------------------*/
/* Atomics */
@@ -735,7 +734,6 @@ MDBX_MAYBE_UNUSED static __inline uintptr_t mdbx_thread_self(void) {
return (uintptr_t)thunk;
}
MDBX_MAYBE_UNUSED MDBX_INTERNAL_FUNC void mdbx_osal_jitter(bool tiny);
MDBX_INTERNAL_FUNC uint64_t mdbx_osal_monotime(void);
MDBX_INTERNAL_FUNC uint64_t
mdbx_osal_16dot16_to_monotime(uint32_t seconds_16dot16);

View File

@@ -23,7 +23,9 @@
#define _WIN32_WINNT 0x0601 /* Windows 7 */
#endif
#ifdef _MSC_VER
#ifndef _CRT_SECURE_NO_WARNINGS
#define _CRT_SECURE_NO_WARNINGS
#endif /* _CRT_SECURE_NO_WARNINGS */
#pragma warning(push, 1)
#pragma warning(disable : 4548) /* expression before comma has no effect; \
expected expression with side - effect */

View File

@@ -73,7 +73,12 @@ bool testcase_hill::run() {
uint64_t committed_serial = serial_count;
unsigned txn_nops = 0;
bool rc = false;
bool rc = speculum_verify();
if (!rc) {
log_notice("uphill: bailout before main loop");
goto bailout;
}
while (should_continue()) {
const keygen::serial_t a_serial = serial_count;
if (unlikely(!keyvalue_maker.increment(serial_count, 1))) {
@@ -280,6 +285,11 @@ bool testcase_hill::run() {
log_notice("hill: reached %d tree depth & %s sub-tree depth(s)",
stat.ms_depth, str.c_str());
}
if ((config.params.table_flags & MDBX_DUPSORT) == 0) {
if (!check_batch_get())
failure("batch-get verification failed");
}
}
while (serial_count > 1) {

View File

@@ -210,6 +210,11 @@ echo "=== use ${db_size_mb}M for DB"
# 3. Create test-directory in ramfs/tmpfs, i.e. create/format/mount if required
case ${UNAME} in
Linux)
ulimit -c unlimited
if [ "$(cat /proc/sys/kernel/core_pattern)" != "core.%p" ]; then
echo "core.%p > /proc/sys/kernel/core_pattern" >&2
echo "core.%p" | sudo tee /proc/sys/kernel/core_pattern || true
fi
;;
FreeBSD)

View File

@@ -189,6 +189,26 @@ bool testcase_nested::stochastic_breakable_restart_with_nested(
(flipcoin() || txn_underutilization_x256(txn_guard.get()) < 42))
should_continue &= pop_txn();
if (flipcoin_x3()) {
unsigned period;
int err = mdbx_env_get_syncperiod(db_guard.get(), &period);
if (unlikely(err != MDBX_SUCCESS))
failure_perror("mdbx_env_get_syncperiod()", err);
size_t bytes;
err = mdbx_env_get_syncbytes(db_guard.get(), &bytes);
if (unlikely(err != MDBX_SUCCESS))
failure_perror("mdbx_env_get_syncbytes()", err);
err = mdbx_env_set_syncperiod(db_guard.get(), period ^ 42);
if (unlikely(err != MDBX_SUCCESS) && err != MDBX_BUSY)
failure_perror("mdbx_env_set_syncperiod()", err);
err = mdbx_env_set_syncbytes(db_guard.get(), bytes ^ 42000);
if (unlikely(err != MDBX_SUCCESS) && err != MDBX_BUSY)
failure_perror("mdbx_env_set_syncbytes()", err);
}
if (should_continue)
while (stack.empty() ||
(is_nested_txn_available() && flipcoin() && stack.size() < 5))

View File

@@ -210,6 +210,11 @@ echo "=== use ${db_size_mb}M for DB"
# 3. Create test-directory in ramfs/tmpfs, i.e. create/format/mount if required
case ${UNAME} in
Linux)
ulimit -c unlimited
if [ "$(cat /proc/sys/kernel/core_pattern)" != "core.%p" ]; then
echo "core.%p > /proc/sys/kernel/core_pattern" >&2
echo "core.%p" | sudo tee /proc/sys/kernel/core_pattern || true
fi
;;
FreeBSD)

View File

@@ -164,6 +164,14 @@ void testcase::db_open() {
if (unlikely(rc != MDBX_SUCCESS))
failure_perror("mdbx_env_open()", rc);
rc = mdbx_env_set_syncperiod(db_guard.get(), unsigned(0.042 * 65536));
if (unlikely(rc != MDBX_SUCCESS) && rc != MDBX_BUSY)
failure_perror("mdbx_env_set_syncperiod()", rc);
rc = mdbx_env_set_syncbytes(db_guard.get(), INT_MAX / 421);
if (unlikely(rc != MDBX_SUCCESS) && rc != MDBX_BUSY)
failure_perror("mdbx_env_set_syncbytes()", rc);
log_trace("<< db_open");
}
@@ -641,7 +649,8 @@ enum speculum_cursors : int {
prev = 1,
prev_prev = 2,
next = 3,
next_next = 4
next_next = 4,
seek_check = 5
};
bool testcase::is_same(const Item &a, const Item &b) const {
@@ -715,7 +724,7 @@ void testcase::speculum_check_cursor(const char *where, const char *stage,
// verbose(where, stage, cursor_key, cursor_data, cursor_err);
// verbose(where, stage, it);
if (cursor_err != MDBX_SUCCESS && cursor_err != MDBX_NOTFOUND &&
cursor_err != MDBX_RESULT_TRUE)
cursor_err != MDBX_RESULT_TRUE && cursor_err != MDBX_ENODATA)
failure("speculum-%s: %s %s %d %s", where, stage, "cursor-get", cursor_err,
mdbx_strerror(cursor_err));
@@ -852,11 +861,26 @@ int testcase::insert(const keygen::buffer &akey, const keygen::buffer &adata,
int err;
bool rc = true;
Item item;
#if SPECULUM_CURSORS
MDBX_cursor *check_seek_cursor = nullptr;
MDBX_val seek_check_key, seek_check_data;
int seek_check_err = 42;
#endif /* SPECULUM_CURSORS */
if (config.params.speculum) {
item.first = iov2dataview(akey);
item.second = iov2dataview(adata);
#if SPECULUM_CURSORS
speculum_prepare_cursors(item);
check_seek_cursor = speculum_cursors[seek_check].get();
seek_check_key = akey->value;
seek_check_data = adata->value;
seek_check_err = mdbx_cursor_get(
check_seek_cursor, &seek_check_key, &seek_check_data,
(config.params.table_flags & MDBX_DUPSORT) ? MDBX_GET_BOTH
: MDBX_SET_KEY);
if (seek_check_err != MDBX_SUCCESS && seek_check_err != MDBX_NOTFOUND)
failure("speculum-%s: %s pre-insert %d %s", "insert", "seek",
seek_check_err, mdbx_strerror(seek_check_err));
#endif /* SPECULUM_CURSORS */
}
@@ -881,6 +905,26 @@ int testcase::insert(const keygen::buffer &akey, const keygen::buffer &adata,
}
#if SPECULUM_CURSORS
if (insertion_result.second) {
if (seek_check_err != MDBX_NOTFOUND) {
log_error(
"speculum.pre-insert-seek: unexpected %d {%s, %s}", seek_check_err,
mdbx_dump_val(&seek_check_key, dump_key, sizeof(dump_key)),
mdbx_dump_val(&seek_check_data, dump_value, sizeof(dump_value)));
rc = false;
}
} else {
if (seek_check_err != MDBX_SUCCESS) {
log_error(
"speculum.pre-insert-seek: unexpected %d {%s, %s}", seek_check_err,
mdbx_dump_val(&seek_check_key, dump_key, sizeof(dump_key)),
mdbx_dump_val(&seek_check_data, dump_value, sizeof(dump_value)));
speculum_check_iterator("insert", "pre-seek", insertion_result.first,
seek_check_key, seek_check_data);
rc = false;
}
}
if (insertion_result.first != speculum.begin()) {
const auto cursor_prev = speculum_cursors[prev].get();
auto it_prev = insertion_result.first;
@@ -1050,6 +1094,15 @@ bool testcase::speculum_verify() {
MDBX_val akey, avalue;
MDBX_val mkey, mvalue;
err = mdbx_cursor_get(cursor, &akey, &avalue, MDBX_FIRST);
if (err == MDBX_NOTFOUND) {
err = mdbx_cursor_get(cursor, &akey, &avalue, MDBX_GET_CURRENT);
if (err == MDBX_ENODATA)
err = MDBX_NOTFOUND;
else {
log_error("unexpected %d for MDBX_GET_CURRENT on empty DB", err);
rc = false;
}
}
unsigned extra = 0, lost = 0, n = 0;
assert(std::is_sorted(speculum.cbegin(), speculum.cend(), ItemCompare(this)));
@@ -1132,3 +1185,69 @@ bool testcase::speculum_verify() {
mdbx_cursor_close(cursor);
return rc;
}
bool testcase::check_batch_get() {
char dump_key[128], dump_value[128];
char dump_key_batch[128], dump_value_batch[128];
MDBX_cursor *cursor;
int err = mdbx_cursor_open(txn_guard.get(), dbi, &cursor);
if (err != MDBX_SUCCESS)
failure_perror("mdbx_cursor_open()", err);
MDBX_cursor *batch_cursor;
err = mdbx_cursor_open(txn_guard.get(), dbi, &batch_cursor);
if (err != MDBX_SUCCESS)
failure_perror("mdbx_cursor_open()", err);
MDBX_val pairs[42];
size_t count = 0xDeadBeef;
err = mdbx_cursor_get_batch(batch_cursor, &count, pairs, ARRAY_LENGTH(pairs),
MDBX_FIRST);
bool rc = true;
size_t i, n = 0;
while (err == MDBX_SUCCESS) {
for (i = 0; i < count; i += 2) {
mdbx::slice key, value;
int err2 = mdbx_cursor_get(cursor, &key, &value, MDBX_NEXT);
if (err2 != MDBX_SUCCESS)
failure_perror("mdbx_cursor_open()", err2);
if (key != pairs[i] || value != pairs[i + 1]) {
log_error(
"batch-get pair mismatch %zu/%zu: sequential{%s, %s} != "
"batch{%s, %s}",
n + i / 2, i, mdbx_dump_val(&key, dump_key, sizeof(dump_key)),
mdbx_dump_val(&value, dump_value, sizeof(dump_value)),
mdbx_dump_val(&pairs[i], dump_key_batch, sizeof(dump_key_batch)),
mdbx_dump_val(&pairs[i + 1], dump_value_batch,
sizeof(dump_value_batch)));
rc = false;
}
}
n += i / 2;
err = mdbx_cursor_get_batch(batch_cursor, &count, pairs,
ARRAY_LENGTH(pairs), MDBX_NEXT);
}
if (err != MDBX_NOTFOUND)
failure_perror("mdbx_cursor_get_batch()", err);
err = mdbx_cursor_eof(batch_cursor);
if (err != MDBX_RESULT_TRUE) {
log_error("batch-get %s cursor not-eof %d", "batch", err);
rc = false;
}
err = mdbx_cursor_on_last(batch_cursor);
if (err != MDBX_RESULT_TRUE) {
log_error("batch-get %s cursor not-on-last %d", "batch", err);
rc = false;
}
err = mdbx_cursor_on_last(cursor);
if (err != MDBX_RESULT_TRUE) {
log_error("batch-get %s cursor not-on-last %d", "checked", err);
rc = false;
}
mdbx_cursor_close(cursor);
mdbx_cursor_close(batch_cursor);
return rc;
}

View File

@@ -202,7 +202,7 @@ protected:
#define SPECULUM_CURSORS 1
#endif /* SPECULUM_CURSORS */
#if SPECULUM_CURSORS
scoped_cursor_guard speculum_cursors[5];
scoped_cursor_guard speculum_cursors[5 + 1];
void speculum_prepare_cursors(const Item &item);
void speculum_check_cursor(const char *where, const char *stage,
const testcase::SET::const_iterator &it,
@@ -227,6 +227,7 @@ protected:
const MDBX_val &v) const;
bool speculum_verify();
bool check_batch_get();
int insert(const keygen::buffer &akey, const keygen::buffer &adata,
MDBX_put_flags_t flags);
int replace(const keygen::buffer &akey, const keygen::buffer &new_value,