Compare commits

...

39 Commits

Author SHA1 Message Date
Леонид Юрьев (Leonid Yuriev)
aaa9112c83 mdbx: release v0.11.10
The stable bugfix release.
It is planned that this will be the last release of the v0.11 branch.

New:
----

 - The C++ API has been refined to simplify support for `wchar_t` in path names.
 - Added explicit error message for Buildroot's Microblaze toolchain maintainers.

Fixes:
------

 - Never use modern `__cxa_thread_atexit()` on Apple's OSes.
 - Use `MultiByteToWideChar(CP_THREAD_ACP)` instead of `mbstowcs()`.
 - Don't check owner for finished transactions.
 - Fixed typo in `MDBX_EINVAL` which breaks MingGW builds with CLANG.

Minors:
-------

 - Fixed variable name typo.
 - Using `ldd` to check used dso.
 - Added `MDBX_WEAK_IMPORT_ATTRIBUTE` macro.
 - Use current transaction geometry for untouched parameters when `env_set_geometry()` called within a write transaction.
 - Minor clarified `iov_page()` failure case.

14 files changed, 263 insertions(+), 252 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2022-08-22 13:32:24 +03:00
Леонид Юрьев (Leonid Yuriev)
26f52a19c3 mdbx: add explicit error message for Buildroot's Microblaze toolchain maintainers. 2022-08-22 12:59:42 +03:00
Леонид Юрьев (Leonid Yuriev)
5368551081 mdbx: minor clarify iov_page() failure case. 2022-08-22 12:59:42 +03:00
Леонид Юрьев (Leonid Yuriev)
4f02199648 mdbx: update ChangeLog. 2022-08-17 23:56:53 +03:00
Леонид Юрьев (Leonid Yuriev)
316ddf9e01 mdbx: fix typo in MDBX_EINVAL which breaks MingGW builds with CLANG. 2022-08-16 11:09:52 +03:00
Леонид Юрьев (Leonid Yuriev)
3fbbe32adf mdbx: fix checking owner for finished write transactions inside txn_abort().
Fixed regression after 06734bf8ff.
2022-08-14 12:39:21 +03:00
Леонид Юрьев (Leonid Yuriev)
8467cc6d03 mdbx: use current txn geo for untouched parameters when env_set_geometry() called within a write transaction. 2022-08-13 16:56:29 +03:00
Леонид Юрьев (Leonid Yuriev)
9f0e2ecc67 mdbx: fix variable name typo. 2022-08-13 16:56:09 +03:00
Леонид Юрьев (Leonid Yuriev)
0287a00ee3 mdbx++: refine/simplify wchar_t support for pathnames. 2022-08-09 18:54:22 +03:00
Леонид Юрьев (Leonid Yuriev)
c8b1392cbe mdbx: use MultiByteToWideChar(CP_THREAD_ACP) instead of mbstowcs(). 2022-08-09 16:12:24 +03:00
Леонид Юрьев (Leonid Yuriev)
6d85e35876 mdbx: never use modern __cxa_thread_atexit() on Apple's OSes. 2022-08-08 15:23:39 +03:00
Леонид Юрьев (Leonid Yuriev)
dd01aabaeb mdbx: add MDBX_WEAK_IMPORT_ATTRIBUTE macro. 2022-08-08 15:18:16 +03:00
Леонид Юрьев (Leonid Yuriev)
f0a46da6a5 mdbx-make: using ldd to check used dso. 2022-08-06 19:42:38 +03:00
Леонид Юрьев (Leonid Yuriev)
06734bf8ff mdbx: don't check owner for finished transactions. 2022-08-06 13:19:49 +03:00
Леонид Юрьев (Leonid Yuriev)
e444c70cb7 mdbx: release v0.11.9
The stable bugfix release.
It is planned that this will be the last release of the v0.11 branch.

Acknowledgements:
-----------------

 - [Alex Sharov](https://github.com/AskAlexSharov) and Erigon team for reporting and testing.
 - [Andrew Ashikhmin](https://gitflic.ru/user/yperbasis) for contributing.

New:
----

 - Ability to customise `MDBX_LOCK_SUFFIX`, `MDBX_DATANAME`, `MDBX_LOCKNAME` just by predefine ones during build.
 - Added to [`mdbx::env_managed`](https://libmdbx.dqdkfa.ru/group__cxx__api.html#classmdbx_1_1env__managed)'s methods a few overloads with `const char* pathname` parameter (C++ API).

Fixes:
------

 - Fixed hang copy-with-compactification of a corrupted DB
   or in case the volume of output pages is a multiple of `MDBX_ENVCOPY_WRITEBUF`.
 - Fixed standalone non-CMake build on MacOS (`#include AvailabilityMacros.h>`).
 - Fixed unexpected `MDBX_PAGE_FULL` error in rare cases with large database page sizes.

Minors:
-------

 - Minor fixes Doxygen references, comments, descriptions, etc.
 - Fixed copy&paste typo inside `meta_checktxnid()`.
 - Minor fix `meta_checktxnid()` to avoid assertion in debug mode.
 - Minor fix `mdbx_env_set_geometry()` to avoid returning `EINVAL` in particular rare cases.
 - Minor refine/fix batch-get testcase for large page size.
 - Added `--pagesize NN` option to long-stotastic test script.
 - Updated Valgrind-suppressions file for modern GCC.
 - Fixed `has no symbols` warning from Apple's ranlib.

18 files changed, 318 insertions(+), 178 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2022-08-02 12:00:30 +03:00
Леонид Юрьев (Leonid Yuriev)
a441c9ffb1 mdbx: logging pgno of invalid root pages inside meta_checktxnid(). 2022-08-01 14:44:33 +03:00
Леонид Юрьев (Leonid Yuriev)
71c3d20c01 mdbx: skip extra cycle during search of page sequence. 2022-08-01 00:00:14 +03:00
Леонид Юрьев (Leonid Yuriev)
6a1bf6035f mdbx: update ChangeLog. 2022-07-23 11:03:27 +03:00
Леонид Юрьев (Leonid Yuriev)
e963375302 mdbx: fix unexpected MDBX_PAGE_FULL in rare cases with large page-size. 2022-07-21 17:52:08 +03:00
yperbasis
0af84be269 Merged with fix_apple 2022-07-20 11:22:26 +00:00
yperbasis
23e7870e81 #include <AvailabilityMacros.h> 2022-07-19 18:35:19 +02:00
Леонид Юрьев (Leonid Yuriev)
fc53e57a64 mdbx-cmake: avoid has no symbols warning from Apple's ranlib. 2022-07-13 22:11:28 +03:00
Леонид Юрьев (Leonid Yuriev)
69b495d559 mdbx++: add to mdbx::env overloads with const char* pathname parameter. 2022-07-13 21:45:23 +03:00
Леонид Юрьев (Leonid Yuriev)
9bbf09b5c4 mdbx: minor update README for the locations of a source code. 2022-07-09 14:54:25 +03:00
Леонид Юрьев (Leonid Yuriev)
9d9df11509 mdbx: update ChangeLog. 2022-07-09 14:11:36 +03:00
Леонид Юрьев (Leonid Yuriev)
61d0d63ac2 mdbx-test: refine/fix batch-get testcase for large page size (i.e. when more pairs per page). 2022-07-09 13:30:40 +03:00
Леонид Юрьев (Leonid Yuriev)
e9a1042cc2 mdbx-test: add --pagesize NN option to long-stotastic script. 2022-07-09 13:30:40 +03:00
Леонид Юрьев (Leonid Yuriev)
f7f9eaff95 mdbx-test: update Valgrind-suppressions for modern GCC. 2022-07-09 13:30:13 +03:00
Леонид Юрьев (Leonid Yuriev)
61825e9bc8 mdbx: update ChangeLog for v0.11.9 2022-07-08 01:17:21 +03:00
Леонид Юрьев (Leonid Yuriev)
c499f2bb36 mdbx: minor fix EINVAL from mdbx_env_set_geometry().
Silently growth `size_lower` to the `MIN_PAGENO` instead of returning `MDBX_EINVAL`.
2022-07-08 01:14:29 +03:00
Леонид Юрьев (Leonid Yuriev)
6e2a1ebfbd mdbx: minor fix meta_checktxnid() to avoid assertion in debug mode. 2022-07-08 01:08:54 +03:00
Леонид Юрьев (Leonid Yuriev)
f17bd06116 mdbx: update ChangeLog. 2022-07-08 00:27:19 +03:00
Леонид Юрьев (Leonid Yuriev)
3cc7f105a5 mdbx: fix copy&paste typo inside meta_checktxnid(). 2022-07-07 19:11:47 +03:00
Леонид Юрьев (Leonid Yuriev)
0b5cdee6ef mdbx: fix mdbx_env_compact() hand when DB is corrupted or the volume of an output pages aliquot to MDBX_ENVCOPY_WRITEBUF. 2022-07-07 19:05:35 +03:00
Леонид Юрьев (Leonid Yuriev)
7b735c272d mdbx: allow predefine/customise MDBX_LOCK_SUFFIX, MDBX_DATANAME, MDBX_LOCKNAME. 2022-06-25 22:14:06 +03:00
Леонид Юрьев (Leonid Yuriev)
55d3783699 mdbx: update patch for old buildroot versions. 2022-06-25 16:24:40 +03:00
Леонид Юрьев (Leonid Yuriev)
5d933d09d3 mdbx: minor refine/clarify descriptions of MDBX_LOCKNAME and MDBX_DATANAME. 2022-06-25 15:27:33 +03:00
Леонид Юрьев (Leonid Yuriev)
8ef3bfcc95 mdbx++: refine descriptions for compare_fast() and compare_lexicographically(). 2022-06-18 12:08:34 +03:00
Леонид Юрьев (Leonid Yuriev)
51f8407a08 mdbx: add objdump -f -h -j .text to README. 2022-06-13 17:45:15 +03:00
22 changed files with 572 additions and 416 deletions

View File

@@ -569,7 +569,10 @@ else()
"${MDBX_SOURCE_DIR}/options.h" "${MDBX_SOURCE_DIR}/base.h"
"${MDBX_SOURCE_DIR}/internals.h" "${MDBX_SOURCE_DIR}/osal.h"
"${MDBX_SOURCE_DIR}/core.c" "${MDBX_SOURCE_DIR}/osal.c"
"${MDBX_SOURCE_DIR}/lck-posix.c" "${MDBX_SOURCE_DIR}/lck-windows.c")
"${MDBX_SOURCE_DIR}/lck-posix.c")
if(NOT APPLE)
list(APPEND LIBMDBX_SOURCES "${MDBX_SOURCE_DIR}/lck-windows.c")
endif()
include_directories("${MDBX_SOURCE_DIR}")
endif()
endif(MDBX_AMALGAMATED_SOURCE)

View File

@@ -1,7 +1,79 @@
ChangeLog
---------
## v0.11.8 at 2022-06-12
## v0.11.10 (the TriColor) at 2022-08-22
The stable bugfix release.
It is planned that this will be the last release of the v0.11 branch.
New:
- The C++ API has been refined to simplify support for `wchar_t` in path names.
- Added explicit error message for Buildroot's Microblaze toolchain maintainers.
Fixes:
- Never use modern `__cxa_thread_atexit()` on Apple's OSes.
- Use `MultiByteToWideChar(CP_THREAD_ACP)` instead of `mbstowcs()`.
- Don't check owner for finished transactions.
- Fixed typo in `MDBX_EINVAL` which breaks MingGW builds with CLANG.
Minors:
- Fixed variable name typo.
- Using `ldd` to check used dso.
- Added `MDBX_WEAK_IMPORT_ATTRIBUTE` macro.
- Use current transaction geometry for untouched parameters when `env_set_geometry()` called within a write transaction.
- Minor clarified `iov_page()` failure case.
-------------------------------------------------------------------------------
## v0.11.9 (Чирчик-1992) at 2022-08-02
The stable bugfix release.
```
18 files changed, 318 insertions(+), 178 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
```
Acknowledgements:
- [Alex Sharov](https://github.com/AskAlexSharov) and Erigon team for reporting and testing.
- [Andrew Ashikhmin](https://gitflic.ru/user/yperbasis) for contributing.
New:
- Ability to customise `MDBX_LOCK_SUFFIX`, `MDBX_DATANAME`, `MDBX_LOCKNAME` just by predefine ones during build.
- Added to [`mdbx::env_managed`](https://libmdbx.dqdkfa.ru/group__cxx__api.html#classmdbx_1_1env__managed)'s methods a few overloads with `const char* pathname` parameter (C++ API).
Fixes:
- Fixed hang copy-with-compactification of a corrupted DB
or in case the volume of output pages is a multiple of `MDBX_ENVCOPY_WRITEBUF`.
- Fixed standalone non-CMake build on MacOS (`#include AvailabilityMacros.h>`).
- Fixed unexpected `MDBX_PAGE_FULL` error in rare cases with large database page sizes.
Minors:
- Minor fixes Doxygen references, comments, descriptions, etc.
- Fixed copy&paste typo inside `meta_checktxnid()`.
- Minor fix `meta_checktxnid()` to avoid assertion in debug mode.
- Minor fix `mdbx_env_set_geometry()` to avoid returning `EINVAL` in particular rare cases.
- Minor refine/fix batch-get testcase for large page size.
- Added `--pagesize NN` option to long-stotastic test script.
- Updated Valgrind-suppressions file for modern GCC.
- Fixed `has no symbols` warning from Apple's ranlib.
-------------------------------------------------------------------------------
## v0.11.8 (Baked Apple) at 2022-06-12
The stable release with an important fixes and workaround for the critical macOS thread-local-storage issue.
Acknowledgements:
@@ -25,6 +97,7 @@ Fixes:
- Fixed `mdbx_check_fs_local()` for CDROM case on Windows.
- Fixed nasty typo of typename which caused false `MDBX_CORRUPTED` error in a rare execution path,
when the size of the thread-ID type not equal to 8.
- Fixed Elbrus/E2K LCC 1.26 compiler warnings (memory model for atomic operations, etc).
- Fixed write-after-free memory corruption on latest `macOS` during finalization/cleanup of thread(s) that executed read transaction(s).
> The issue was suddenly discovered by a [CI](https://en.wikipedia.org/wiki/Continuous_integration)
> after adding an iteration with macOS 11 "Big Sur", and then reproduced on recent release of macOS 12 "Monterey".
@@ -36,7 +109,6 @@ Fixes:
> This is unexpected crazy-like behavior since the order of resources releasing/destroying
> is not the reverse of ones acquiring/construction order. Nonetheless such surprise
> is now workarounded by using atomic compare-and-swap operations on a 64-bit signatures/cookies.
- Fixed Elbrus/E2K LCC 1.26 compiler warnings (memory model for atomic operations, etc).
Minors:
@@ -51,7 +123,8 @@ Minors:
-------------------------------------------------------------------------------
## v0.11.7 at 2022-04-22
## v0.11.7 (Resurrected Sarmat) at 2022-04-22
The stable risen release after the Github's intentional malicious disaster.

View File

@@ -808,13 +808,13 @@ else
define bench-rule
bench-$(1)_$(2).txt: $(3) $(IOARENA) $(lastword $(MAKEFILE_LIST))
@echo ' RUNNING ioarena for $1/$2...'
$(QUIET)LD_LIBRARY_PATH="./:$$$${LD_LIBRARY_PATH}" \
$(QUIET)(export LD_LIBRARY_PATH="./:$$$${LD_LIBRARY_PATH}"; \
ldd $(IOARENA) && \
$(IOARENA) -D $(1) -B crud -m $(BENCH_CRUD_MODE) -n $(2) \
| tee $$@ | grep throughput && \
LD_LIBRARY_PATH="./:$$$${LD_LIBRARY_PATH}" \
$(IOARENA) -D $(1) -B get,iterate -m $(BENCH_CRUD_MODE) -r 4 -n $(2) \
| tee -a $$@ | grep throughput \
|| mv -f $$@ $$@.error
| tee $$@ | grep throughput && \
$(IOARENA) -D $(1) -B iterate,get,iterate,get,iterate -m $(BENCH_CRUD_MODE) -r 4 -n $(2) \
| tee -a $$@ | grep throughput \
) || mv -f $$@ $$@.error
endef

View File

@@ -97,6 +97,24 @@ _MithrilDB_ is a rightly relevant name.
<!-- section-end -->
```
$ objdump -f -h -j .text libmdbx.so
libmdbx.so: формат файла elf64-e2k
архитектура: elbrus-v6:64, флаги 0x00000150:
HAS_SYMS, DYNAMIC, D_PAGED
начальный адрес 0x0000000000021680
Разделы:
Idx Name Разм VMA LMA Фа смещ. Выр.
10 .text 000ddd28 0000000000021680 0000000000021680 00021680 2**3
CONTENTS, ALLOC, LOAD, READONLY, CODE
$ cc --version
lcc:1.26.12:Jun-05-2022:e2k-v6-linux
gcc (GCC) 9.3.0 compatible
```
-----
## Table of Contents
@@ -376,20 +394,20 @@ since release the version 1.0.
_libmdbx_ provides two official ways for integration in source code form:
1. Using the amalgamated source code.
> The amalgamated source code includes all files required to build and
1. Using an amalgamated source code which available in the [releases section](https://gitflic.ru/project/erthink/libmdbx/release) on GitFlic.
> An amalgamated source code includes all files required to build and
> use _libmdbx_, but not for testing _libmdbx_ itself.
> Beside the releases an amalgamated sources could be created any time from the original clone of git
> repository on Linux by executing `make dist`. As a result, the desired
> set of files will be formed in the `dist` subdirectory.
2. Adding the complete original source code as a `git submodule`.
2. Adding the complete source code as a `git submodule` from the [origin git repository](https://gitflic.ru/project/erthink/libmdbx) on GitFlic.
> This allows you to build as _libmdbx_ and testing tool.
> On the other hand, this way requires you to pull git tags, and use C++11 compiler for test tool.
_**Please, avoid using any other techniques.**_ Otherwise, at least
don't ask for support and don't name such chimeras `libmdbx`.
The amalgamated source code could be created from the original clone of git
repository on Linux by executing `make dist`. As a result, the desired
set of files will be formed in the `dist` subdirectory.
## Building and Testing

12
mdbx.h
View File

@@ -824,13 +824,21 @@ enum MDBX_constants {
/* THE FILES *******************************************************************
* At the file system level, the environment corresponds to a pair of files. */
/** \brief The name of the lock file in the environment */
#ifndef MDBX_LOCKNAME
/** \brief The name of the lock file in the environment
* without using \ref MDBX_NOSUBDIR */
#define MDBX_LOCKNAME "/mdbx.lck"
/** \brief The name of the data file in the environment */
#endif
#ifndef MDBX_DATANAME
/** \brief The name of the data file in the environment
* without using \ref MDBX_NOSUBDIR */
#define MDBX_DATANAME "/mdbx.dat"
#endif
#ifndef MDBX_LOCK_SUFFIX
/** \brief The suffix of the lock file when \ref MDBX_NOSUBDIR is used */
#define MDBX_LOCK_SUFFIX "-lck"
#endif
/* DEBUG & LOGGING ************************************************************/

View File

@@ -965,20 +965,21 @@ struct LIBMDBX_API_TYPE slice : public ::MDBX_val {
hash_value() const noexcept;
/// \brief Three-way fast non-lexicographically length-based comparison.
/// \return value:
/// == 0 if "a" == "b",
/// < 0 if "a" shorter than "b",
/// > 0 if "a" longer than "b",
/// < 0 if "a" length-equal and lexicographically less than "b",
/// > 0 if "a" length-equal and lexicographically great than "b".
/// \details Firstly compares length and if it equal then compare content
/// lexicographically. \return value:
/// `== 0` if `a` the same as `b`;
/// `< 0` if `a` shorter than `b`,
/// or the same length and lexicographically less than `b`;
/// `> 0` if `a` longer than `b`,
/// or the same length and lexicographically great than `b`.
MDBX_NOTHROW_PURE_FUNCTION static MDBX_CXX14_CONSTEXPR intptr_t
compare_fast(const slice &a, const slice &b) noexcept;
/// \brief Three-way lexicographically comparison.
/// \return value:
/// < 0 if "a" < "b",
/// == 0 if "a" == "b",
/// > 0 if "a" > "b".
/// `== 0` if `a` lexicographically equal `b`;
/// `< 0` if `a` lexicographically less than `b`;
/// `> 0` if `a` lexicographically great than `b`.
MDBX_NOTHROW_PURE_FUNCTION static MDBX_CXX14_CONSTEXPR intptr_t
compare_lexicographically(const slice &a, const slice &b) noexcept;
friend MDBX_CXX14_CONSTEXPR bool operator==(const slice &a,
@@ -3227,6 +3228,8 @@ public:
#endif /* Windows */
env &copy(const ::std::string &destination, bool compactify,
bool force_dynamic_size = false);
env &copy(const char *destination, bool compactify,
bool force_dynamic_size = false);
/// \brief Copy an environment to the specified file descriptor.
env &copy(filehandle fd, bool compactify, bool force_dynamic_size = false);
@@ -3251,14 +3254,16 @@ public:
/// \brief Removes the environment's files in a proper and multiprocess-safe
/// way.
#ifdef MDBX_STD_FILESYSTEM_PATH
static bool remove(const MDBX_STD_FILESYSTEM_PATH &,
static bool remove(const MDBX_STD_FILESYSTEM_PATH &pathname,
const remove_mode mode = just_remove);
#endif /* MDBX_STD_FILESYSTEM_PATH */
#if defined(_WIN32) || defined(_WIN64) || defined(DOXYGEN)
static bool remove(const ::std::wstring &,
static bool remove(const ::std::wstring &pathname,
const remove_mode mode = just_remove);
#endif /* Windows */
static bool remove(const ::std::string &,
static bool remove(const ::std::string &pathname,
const remove_mode mode = just_remove);
static bool remove(const char *pathname,
const remove_mode mode = just_remove);
/// \brief Statistics for a database in the MDBX environment.
@@ -3496,15 +3501,17 @@ public:
/// \brief Open existing database.
#ifdef MDBX_STD_FILESYSTEM_PATH
env_managed(const MDBX_STD_FILESYSTEM_PATH &, const operate_parameters &,
bool accede = true);
env_managed(const MDBX_STD_FILESYSTEM_PATH &pathname,
const operate_parameters &, bool accede = true);
#endif /* MDBX_STD_FILESYSTEM_PATH */
#if defined(_WIN32) || defined(_WIN64) || defined(DOXYGEN)
env_managed(const ::std::wstring &, const operate_parameters &,
env_managed(const ::std::wstring &pathname, const operate_parameters &,
bool accede = true);
#endif /* Windows */
env_managed(const ::std::string &, const operate_parameters &,
env_managed(const ::std::string &pathname, const operate_parameters &,
bool accede = true);
explicit env_managed(const char *pathname, const operate_parameters &,
bool accede = true);
/// \brief Additional parameters for creating a new database.
struct create_parameters {
@@ -3517,15 +3524,18 @@ public:
/// \brief Create new or open existing database.
#ifdef MDBX_STD_FILESYSTEM_PATH
env_managed(const MDBX_STD_FILESYSTEM_PATH &, const create_parameters &,
const operate_parameters &, bool accede = true);
env_managed(const MDBX_STD_FILESYSTEM_PATH &pathname,
const create_parameters &, const operate_parameters &,
bool accede = true);
#endif /* MDBX_STD_FILESYSTEM_PATH */
#if defined(_WIN32) || defined(_WIN64) || defined(DOXYGEN)
env_managed(const ::std::wstring &, const create_parameters &,
env_managed(const ::std::wstring &pathname, const create_parameters &,
const operate_parameters &, bool accede = true);
#endif /* Windows */
env_managed(const ::std::string &, const create_parameters &,
env_managed(const ::std::string &pathname, const create_parameters &,
const operate_parameters &, bool accede = true);
explicit env_managed(const char *pathname, const create_parameters &,
const operate_parameters &, bool accede = true);
/// \brief Explicitly closes the environment and release the memory map.
///

View File

@@ -1,25 +1,26 @@
From 4c309b57bca0526b0c110990c315682f027ccfad Mon Sep 17 00:00:00 2001
From dd398c9a92b87f7c65798545d776735d27f2a4f9 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?=
=?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= <leo@yuriev.ru>
Date: Thu, 28 Apr 2022 11:02:59 +0300
Date: Sat, 25 Jun 2022 16:22:22 +0300
Subject: [PATCH] package/libmdbx: new package (library/database).
This patch adds libmdbx v0.11.7:
This patch adds libmdbx v0.11.8:
- libmdbx is one of the fastest compact embeddable key-value ACID database.
- libmdbx has a specific set of properties and capabilities,
focused on creating unique lightweight solutions.
- libmdbx surpasses the legendary LMDB (Lightning Memory-Mapped Database)
in terms of reliability, features and performance.
- https://libmdbx.dqdkfa.ru/
- https://github.com/erthink/libmdbx
Signed-off-by: Leonid Yuriev <leo@yuriev.ru>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
---
DEVELOPERS | 3 +++
package/Config.in | 1 +
package/libmdbx/Config.in | 45 ++++++++++++++++++++++++++++++++++++
package/libmdbx/libmdbx.hash | 5 ++++
package/libmdbx/libmdbx.hash | 6 +++++
package/libmdbx/libmdbx.mk | 42 +++++++++++++++++++++++++++++++++
5 files changed, 96 insertions(+)
5 files changed, 97 insertions(+)
create mode 100644 package/libmdbx/Config.in
create mode 100644 package/libmdbx/libmdbx.hash
create mode 100644 package/libmdbx/libmdbx.mk
@@ -52,7 +53,7 @@ index 016a99ed1a..a6f95bfaa9 100644
source "package/libodb-mysql/Config.in"
diff --git a/package/libmdbx/Config.in b/package/libmdbx/Config.in
new file mode 100644
index 0000000000..ef44a955fe
index 0000000000..a9a4ac45c5
--- /dev/null
+++ b/package/libmdbx/Config.in
@@ -0,0 +1,45 @@
@@ -71,7 +72,7 @@ index 0000000000..ef44a955fe
+ libmdbx surpasses the legendary LMDB in terms of
+ reliability, features and performance.
+
+ https://libmdbx.dqdkfa.ru/
+ https://libmdbx.dqdkfa.ru
+
+if BR2_PACKAGE_LIBMDBX
+
@@ -103,18 +104,19 @@ index 0000000000..ef44a955fe
+ !BR2_TOOLCHAIN_GCC_AT_LEAST_4_4
diff --git a/package/libmdbx/libmdbx.hash b/package/libmdbx/libmdbx.hash
new file mode 100644
index 0000000000..61f800d433
index 0000000000..3f2be134c3
--- /dev/null
+++ b/package/libmdbx/libmdbx.hash
@@ -0,0 +1,5 @@
+# Hashes from https://libmdbx.dqdkfa.ru//release/SHA256SUMS
@@ -0,0 +1,6 @@
+# Hashes from: https://libmdbx.dqdkfa.ru/release/SHA256SUMS
+sha256 06011f361481ee7adc2111e48b7b121384294e0b6b8f10c75e7886629297b279 libmdbx-amalgamated-0.11.8.tar.xz
+sha256 3a9fb6a4cd941e646597235518714373fda1ca6d4c5e23669afe70ea87c20940 libmdbx-amalgamated-0.11.7.tar.xz
+
+# Locally calculated
+sha256 310fe25c858a9515fc8c8d7d1f24a67c9496f84a91e0a0e41ea9975b1371e569 LICENSE
diff --git a/package/libmdbx/libmdbx.mk b/package/libmdbx/libmdbx.mk
new file mode 100644
index 0000000000..ab279614ab
index 0000000000..130fe96793
--- /dev/null
+++ b/package/libmdbx/libmdbx.mk
@@ -0,0 +1,42 @@
@@ -124,7 +126,7 @@ index 0000000000..ab279614ab
+#
+################################################################################
+
+LIBMDBX_VERSION = 0.11.7
+LIBMDBX_VERSION = 0.11.8
+LIBMDBX_SOURCE = libmdbx-amalgamated-$(LIBMDBX_VERSION).tar.xz
+LIBMDBX_SITE = https://libmdbx.dqdkfa.ru/release
+LIBMDBX_SUPPORTS_IN_SOURCE_BUILD = NO
@@ -161,5 +163,5 @@ index 0000000000..ab279614ab
+
+$(eval $(cmake-package))
--
2.32.0
2.36.1

View File

@@ -157,11 +157,12 @@
#define nullptr NULL
#endif
#ifdef __APPLE__
#if defined(__APPLE__) || defined(_DARWIN_C_SOURCE)
#include <AvailabilityMacros.h>
#include <TargetConditionals.h>
#ifndef MAC_OS_X_VERSION_MIN_REQUIRED
#define MAC_OS_X_VERSION_MIN_REQUIRED 1070 /* Mac OS X 10.7, 2011 */
#endif
#include <TargetConditionals.h>
#endif /* Apple OSX & iOS */
#if defined(__FreeBSD__) || defined(__NetBSD__) || defined(__OpenBSD__) || \
@@ -610,6 +611,19 @@ __extern_C key_t ftok(const char *, int);
#endif
#endif /* __anonymous_struct_extension__ */
#ifndef MDBX_WEAK_IMPORT_ATTRIBUTE
#ifdef WEAK_IMPORT_ATTRIBUTE
#define MDBX_WEAK_IMPORT_ATTRIBUTE WEAK_IMPORT_ATTRIBUTE
#elif __has_attribute(__weak__) && __has_attribute(__weak_import__)
#define MDBX_WEAK_IMPORT_ATTRIBUTE __attribute__((__weak__, __weak_import__))
#elif __has_attribute(__weak__) || \
(defined(__GNUC__) && __GNUC__ >= 4 && defined(__ELF__))
#define MDBX_WEAK_IMPORT_ATTRIBUTE __attribute__((__weak__))
#else
#define MDBX_WEAK_IMPORT_ATTRIBUTE
#endif
#endif /* MDBX_WEAK_IMPORT_ATTRIBUTE */
/*----------------------------------------------------------------------------*/
#if defined(MDBX_USE_VALGRIND)

View File

@@ -660,6 +660,7 @@ page_room(const MDBX_page *mp) {
return mp->mp_upper - mp->mp_lower;
}
/* Maximum free space in an empty page */
MDBX_NOTHROW_PURE_FUNCTION static __always_inline unsigned
page_space(const MDBX_env *env) {
STATIC_ASSERT(PAGEHDRSZ % 2 == 0);
@@ -1268,55 +1269,50 @@ rthc_compare_and_clean(const void *rthc, const uint64_t signature) {
static __inline int rthc_atexit(void (*dtor)(void *), void *obj,
void *dso_symbol) {
int rc = MDBX_ENOSYS;
#ifndef MDBX_HAVE_CXA_THREAD_ATEXIT_IMPL
#if defined(LIBCXXABI_HAS_CXA_THREAD_ATEXIT_IMPL) || \
defined(HAVE___CXA_THREAD_ATEXIT_IMPL) || __GLIBC_PREREQ(2, 18) || \
defined(ANDROID)
#define MDBX_HAVE_CXA_THREAD_ATEXIT_IMPL 1
#else
#define MDBX_HAVE_CXA_THREAD_ATEXIT_IMPL 0
#endif
#endif /* MDBX_HAVE_CXA_THREAD_ATEXIT_IMPL */
#if defined(__APPLE__) || defined(_DARWIN_C_SOURCE)
#if !defined(MAC_OS_X_VERSION_MIN_REQUIRED) || !defined(MAC_OS_X_VERSION_10_7)
#error \
"The <AvailabilityMacros.h> should be included and MAC_OS_X_VERSION_MIN_REQUIRED must be defined"
#elif MAC_OS_X_VERSION_MIN_REQUIRED >= MAC_OS_X_VERSION_10_7
#ifndef MDBX_HAVE_CXA_THREAD_ATEXIT
#if defined(LIBCXXABI_HAS_CXA_THREAD_ATEXIT) || \
defined(HAVE___CXA_THREAD_ATEXIT)
#define MDBX_HAVE_CXA_THREAD_ATEXIT 1
#elif !MDBX_HAVE_CXA_THREAD_ATEXIT_IMPL && \
(defined(__linux__) || defined(__gnu_linux__))
#define MDBX_HAVE_CXA_THREAD_ATEXIT 1
#else
#define MDBX_HAVE_CXA_THREAD_ATEXIT 0
#endif
#endif /* MDBX_HAVE_CXA_THREAD_ATEXIT */
int rc = MDBX_ENOSYS;
#if MDBX_HAVE_CXA_THREAD_ATEXIT_IMPL && !MDBX_HAVE_CXA_THREAD_ATEXIT
#define __cxa_thread_atexit __cxa_thread_atexit_impl
#endif
#if MDBX_HAVE_CXA_THREAD_ATEXIT || defined(__cxa_thread_atexit)
extern int __cxa_thread_atexit(void (*dtor)(void *), void *obj,
void *dso_symbol) MDBX_WEAK_IMPORT_ATTRIBUTE;
if (&__cxa_thread_atexit)
rc = __cxa_thread_atexit(dtor, obj, dso_symbol);
#elif defined(__APPLE__) || defined(_DARWIN_C_SOURCE)
extern void _tlv_atexit(void (*termfunc)(void *objAddr), void *objAddr)
__attribute__((__weak__, __weak_import__));
if (rc && &_tlv_atexit) {
MDBX_WEAK_IMPORT_ATTRIBUTE;
if (&_tlv_atexit) {
(void)dso_symbol;
_tlv_atexit(dtor, obj);
rc = 0;
}
#elif !defined(MDBX_HAVE_CXA_THREAD_ATEXIT)
#define MDBX_HAVE_CXA_THREAD_ATEXIT 1
#endif /* MAC_OS_X_VERSION_MIN_REQUIRED */
#endif /* Apple */
#if defined(MDBX_HAVE_CXA_THREAD_ATEXIT) && MDBX_HAVE_CXA_THREAD_ATEXIT
extern int __cxa_thread_atexit(void (*dtor)(void *), void *obj,
void *dso_symbol)
#ifdef WEAK_IMPORT_ATTRIBUTE
WEAK_IMPORT_ATTRIBUTE
#elif defined(MAC_OS_X_VERSION_MIN_REQUIRED) && \
MAC_OS_X_VERSION_MIN_REQUIRED >= 1020 && \
((__has_attribute(__weak__) && __has_attribute(__weak_import__)) || \
(defined(__GNUC__) && __GNUC__ >= 4))
__attribute__((__weak__, __weak_import__))
#elif (__has_attribute(__weak__) || (defined(__GNUC__) && __GNUC__ >= 4)) && \
!defined(MAC_OS_X_VERSION_MIN_REQUIRED)
__attribute__((__weak__))
#endif
;
if (rc && &__cxa_thread_atexit)
rc = __cxa_thread_atexit(dtor, obj, dso_symbol);
#elif __GLIBC_PREREQ(2, 18) || defined(ANDROID) || defined(__linux__) || \
defined(__gnu_linux__)
extern int __cxa_thread_atexit_impl(void (*dtor)(void *), void *obj,
void *dso_symbol)
__attribute__((__weak__));
if (rc && &__cxa_thread_atexit_impl)
rc = __cxa_thread_atexit_impl(dtor, obj, dso_symbol);
#else
(void)dtor;
(void)obj;
(void)dso_symbol;
#endif
return rc;
}
@@ -5615,8 +5611,11 @@ static int mdbx_txn_spill(MDBX_txn *const txn, MDBX_cursor *const m0,
txn->tw.dirtyroom += spilled;
mdbx_tassert(txn, mdbx_dirtylist_check(txn));
if (ctx.iov_items)
if (ctx.iov_items) {
/* iov_page() frees dirty-pages and reset iov_items in case of failure. */
mdbx_tassert(txn, rc == MDBX_SUCCESS);
rc = mdbx_iov_write(txn, &ctx);
}
if (unlikely(rc != MDBX_SUCCESS))
goto bailout;
@@ -6607,9 +6606,9 @@ no_loose:
while (true) {
if (re_list[range_begin - wanna_range] - pgno == wanna_range)
goto done;
if (range_begin == wanna_range)
if (--range_begin == wanna_range)
break;
pgno = re_list[--range_begin];
pgno = re_list[range_begin];
}
#endif /* MDBX_PNL sort-order */
}
@@ -7724,7 +7723,7 @@ __cold int mdbx_thread_unregister(const MDBX_env *env) {
/* check against todo4recovery://erased_by_github/libmdbx/issues/269 */
static bool meta_checktxnid(const MDBX_env *env, const MDBX_meta *meta,
bool report) {
const txnid_t meta_txnid = constmeta_txnid(env, meta);
const txnid_t head_txnid = meta_txnid(env, meta);
const txnid_t freedb_mod_txnid = meta->mm_dbs[FREE_DBI].md_mod_txnid;
const txnid_t maindb_mod_txnid = meta->mm_dbs[MAIN_DBI].md_mod_txnid;
@@ -7741,25 +7740,25 @@ static bool meta_checktxnid(const MDBX_env *env, const MDBX_meta *meta,
const uint64_t magic_and_version =
unaligned_peek_u64(4, &meta->mm_magic_and_version);
bool ok = true;
if (unlikely(meta_txnid < freedb_mod_txnid ||
if (unlikely(!head_txnid || head_txnid < freedb_mod_txnid ||
(!freedb_mod_txnid && freedb_root &&
likely(magic_and_version == MDBX_DATA_MAGIC)))) {
if (report)
mdbx_warning(
"catch invalid %sdb_mod_txnid %" PRIaTXN " for meta_txnid %" PRIaTXN
"catch invalid %sdb.mod_txnid %" PRIaTXN " for meta_txnid %" PRIaTXN
" %s",
"free", freedb_mod_txnid, meta_txnid,
"free", freedb_mod_txnid, head_txnid,
"(workaround for incoherent flaw of unified page/buffer cache)");
ok = false;
}
if (unlikely(meta_txnid < maindb_mod_txnid ||
if (unlikely(head_txnid < maindb_mod_txnid ||
(!maindb_mod_txnid && maindb_root &&
likely(magic_and_version == MDBX_DATA_MAGIC)))) {
if (report)
mdbx_warning(
"catch invalid %sdb_mod_txnid %" PRIaTXN " for meta_txnid %" PRIaTXN
"catch invalid %sdb.mod_txnid %" PRIaTXN " for meta_txnid %" PRIaTXN
" %s",
"main", maindb_mod_txnid, meta_txnid,
"main", maindb_mod_txnid, head_txnid,
"(workaround for incoherent flaw of unified page/buffer cache)");
ok = false;
}
@@ -7771,9 +7770,9 @@ static bool meta_checktxnid(const MDBX_env *env, const MDBX_meta *meta,
if (unlikely(root_txnid != freedb_mod_txnid)) {
if (report)
mdbx_warning(
"catch invalid root_page_txnid %" PRIaTXN
" for %sdb_mod_txnid %" PRIaTXN " %s",
root_txnid, "free", maindb_mod_txnid,
"catch invalid root_page %" PRIaPGNO " mod_txnid %" PRIaTXN
" for %sdb.mod_txnid %" PRIaTXN " %s",
freedb_root_pgno, root_txnid, "free", freedb_mod_txnid,
"(workaround for incoherent flaw of unified page/buffer cache)");
ok = false;
}
@@ -7786,9 +7785,9 @@ static bool meta_checktxnid(const MDBX_env *env, const MDBX_meta *meta,
if (unlikely(root_txnid != maindb_mod_txnid)) {
if (report)
mdbx_warning(
"catch invalid root_page_txnid %" PRIaTXN
" for %sdb_mod_txnid %" PRIaTXN " %s",
root_txnid, "main", maindb_mod_txnid,
"catch invalid root_page %" PRIaPGNO " mod_txnid %" PRIaTXN
" for %sdb.mod_txnid %" PRIaTXN " %s",
maindb_root_pgno, root_txnid, "main", maindb_mod_txnid,
"(workaround for incoherent flaw of unified page/buffer cache)");
ok = false;
}
@@ -7800,7 +7799,7 @@ static bool meta_checktxnid(const MDBX_env *env, const MDBX_meta *meta,
* for todo4recovery://erased_by_github/libmdbx/issues/269 */
static int meta_waittxnid(const MDBX_env *env, const MDBX_meta *meta,
uint64_t *timestamp) {
if (likely(meta_checktxnid(env, (const MDBX_meta *)meta, !*timestamp)))
if (likely(meta_checktxnid(env, meta, !*timestamp)))
return MDBX_SUCCESS;
if (!*timestamp)
@@ -8151,13 +8150,19 @@ static __always_inline int check_txn(const MDBX_txn *txn, int bad_bits) {
if (unlikely(txn->mt_flags & bad_bits))
return MDBX_BAD_TXN;
mdbx_tassert(txn, (txn->mt_flags & MDBX_NOTLS) ==
((txn->mt_flags & MDBX_TXN_RDONLY)
? txn->mt_env->me_flags & MDBX_NOTLS
: 0));
#if MDBX_TXN_CHECKOWNER
if ((txn->mt_flags & MDBX_NOTLS) == 0 &&
unlikely(txn->mt_owner != mdbx_thread_self()))
STATIC_ASSERT(MDBX_NOTLS > MDBX_TXN_FINISHED + MDBX_TXN_RDONLY);
if (unlikely(txn->mt_owner != mdbx_thread_self()) &&
(txn->mt_flags & (MDBX_NOTLS | MDBX_TXN_FINISHED | MDBX_TXN_RDONLY)) <
(MDBX_TXN_FINISHED | MDBX_TXN_RDONLY))
return txn->mt_owner ? MDBX_THREAD_MISMATCH : MDBX_BAD_TXN;
#endif /* MDBX_TXN_CHECKOWNER */
if (unlikely(!txn->mt_env->me_map))
if (bad_bits && unlikely(!txn->mt_env->me_map))
return MDBX_EPERM;
return MDBX_SUCCESS;
@@ -8958,6 +8963,9 @@ int mdbx_txn_abort(MDBX_txn *txn) {
return mdbx_txn_end(txn, MDBX_END_ABORT | MDBX_END_UPDATE | MDBX_END_SLOT |
MDBX_END_FREE);
if (unlikely(txn->mt_flags & MDBX_TXN_FINISHED))
return MDBX_BAD_TXN;
if (txn->mt_child)
mdbx_txn_abort(txn->mt_child);
@@ -9974,8 +9982,11 @@ static int mdbx_txn_write(MDBX_txn *txn, struct mdbx_iov_ctx *ctx) {
break;
}
if (ctx->iov_items)
if (ctx->iov_items) {
/* iov_page() frees dirty-pages and reset iov_items in case of failure. */
mdbx_tassert(txn, rc == MDBX_SUCCESS);
rc = mdbx_iov_write(txn, ctx);
}
while (r <= dl->length)
dl->items[++w] = dl->items[r++];
@@ -11671,38 +11682,40 @@ mdbx_env_set_geometry(MDBX_env *env, intptr_t size_lower, intptr_t size_now,
if (unlikely(env->me_flags & MDBX_RDONLY))
return MDBX_EACCESS;
if (!inside_txn) {
const MDBX_geo *geo = nullptr;
if (inside_txn)
geo = &env->me_txn->mt_geo;
else {
int err = mdbx_txn_lock(env, false);
if (unlikely(err != MDBX_SUCCESS))
return err;
need_unlock = true;
}
const MDBX_meta *head = constmeta_prefer_last(env);
if (!inside_txn) {
const MDBX_meta *head = constmeta_prefer_last(env);
geo = &head->mm_geo;
env->me_txn0->mt_txnid = constmeta_txnid(env, head);
mdbx_find_oldest(env->me_txn0);
}
/* get untouched params from DB */
/* get untouched params from current write-txn or DB */
if (pagesize <= 0 || pagesize >= INT_MAX)
pagesize = env->me_psize;
if (size_lower < 0)
size_lower = pgno2bytes(env, head->mm_geo.lower);
size_lower = pgno2bytes(env, geo->lower);
if (size_now < 0)
size_now = pgno2bytes(env, head->mm_geo.now);
size_now = pgno2bytes(env, geo->now);
if (size_upper < 0)
size_upper = pgno2bytes(env, head->mm_geo.upper);
size_upper = pgno2bytes(env, geo->upper);
if (growth_step < 0)
growth_step = pgno2bytes(env, pv2pages(head->mm_geo.grow_pv));
growth_step = pgno2bytes(env, pv2pages(geo->grow_pv));
if (shrink_threshold < 0)
shrink_threshold = pgno2bytes(env, pv2pages(head->mm_geo.shrink_pv));
shrink_threshold = pgno2bytes(env, pv2pages(geo->shrink_pv));
if (pagesize != (intptr_t)env->me_psize) {
rc = MDBX_EINVAL;
goto bailout;
}
const size_t usedbytes =
pgno2bytes(env, mdbx_find_largest(env, head->mm_geo.next));
const size_t usedbytes = pgno2bytes(env, mdbx_find_largest(env, geo->next));
if ((size_t)size_upper < usedbytes) {
rc = MDBX_MAP_FULL;
goto bailout;
@@ -11796,8 +11809,13 @@ mdbx_env_set_geometry(MDBX_env *env, intptr_t size_lower, intptr_t size_now,
}
if ((uint64_t)size_lower / pagesize < MIN_PAGENO) {
rc = MDBX_EINVAL;
goto bailout;
size_lower = pagesize * MIN_PAGENO;
if (unlikely(size_lower > size_upper)) {
rc = MDBX_EINVAL;
goto bailout;
}
if (size_now < size_lower)
size_now = size_lower;
}
if (unlikely((size_t)size_upper > MAX_MAPSIZE ||
@@ -12433,7 +12451,7 @@ __cold static int mdbx_setup_dxb(MDBX_env *env, const int lck_rc,
} else {
const txnid_t txnid = constmeta_txnid(env, head);
const txnid_t next_txnid = safe64_txnid_next(txnid);
if (unlikely(txnid > MAX_TXNID)) {
if (unlikely(next_txnid > MAX_TXNID)) {
mdbx_error("txnid overflow, raise %d", MDBX_TXN_FULL);
return MDBX_TXN_FULL;
}
@@ -12966,12 +12984,8 @@ __cold static int mdbx_handle_env_pathname(MDBX_handle_env_pathname *ctx,
return MDBX_EINVAL;
#if defined(_WIN32) || defined(_WIN64)
const size_t wlen = mbstowcs(nullptr, pathname, INT_MAX);
if (wlen < 1 || wlen > /* MAX_PATH */ INT16_MAX)
return ERROR_INVALID_NAME;
wchar_t *const pathnameW = _alloca((wlen + 1) * sizeof(wchar_t));
if (wlen != mbstowcs(pathnameW, pathname, wlen + 1))
return ERROR_INVALID_NAME;
const wchar_t *pathnameW = nullptr;
MUSTDIE_MB2WIDE(pathname, pathnameW);
const DWORD dwAttrib = GetFileAttributesW(pathnameW);
if (dwAttrib == INVALID_FILE_ATTRIBUTES) {
@@ -15245,7 +15259,7 @@ int mdbx_cursor_get_batch(MDBX_cursor *mc, size_t *count, MDBX_val *pairs,
break;
default:
mdbx_debug("unhandled/unimplemented cursor operation %u", op);
rc = EINVAL;
rc = MDBX_EINVAL;
break;
}
@@ -17304,6 +17318,7 @@ static int mdbx_node_move(MDBX_cursor *csrc, MDBX_cursor *cdst, bool fromleft) {
} break;
default:
assert(false);
goto bailout;
}
@@ -18622,6 +18637,7 @@ static int mdbx_page_split(MDBX_cursor *mc, const MDBX_val *const newkey,
(newindx < nkeys)
? /* split at the middle */ (nkeys + 1) / 2
: /* split at the end (i.e. like append-mode ) */ nkeys - minkeys + 1;
mdbx_assert(env, split_indx >= minkeys && split_indx <= nkeys - minkeys + 1);
mdbx_cassert(mc, !IS_BRANCH(mp) || newindx > 0);
/* It is reasonable and possible to split the page at the begin */
@@ -18719,11 +18735,6 @@ static int mdbx_page_split(MDBX_cursor *mc, const MDBX_val *const newkey,
goto done;
}
} else {
/* Maximum free space in an empty page */
const unsigned max_space = page_space(env);
const size_t new_size = IS_LEAF(mp) ? leaf_size(env, newkey, newdata)
: branch_size(env, newkey);
/* grab a page to hold a temporary copy */
tmp_ki_copy = mdbx_page_malloc(mc->mc_txn, 1);
if (unlikely(tmp_ki_copy == NULL)) {
@@ -18731,6 +18742,10 @@ static int mdbx_page_split(MDBX_cursor *mc, const MDBX_val *const newkey,
goto done;
}
const unsigned max_space = page_space(env);
const size_t new_size = IS_LEAF(mp) ? leaf_size(env, newkey, newdata)
: branch_size(env, newkey);
/* prepare to insert */
for (unsigned j = i = 0; i < nkeys; ++i, ++j) {
tmp_ki_copy->mp_ptrs[j] = 0;
@@ -18743,34 +18758,35 @@ static int mdbx_page_split(MDBX_cursor *mc, const MDBX_val *const newkey,
tmp_ki_copy->mp_lower = 0;
tmp_ki_copy->mp_upper = (indx_t)max_space;
/* When items are relatively large the split point needs
* to be checked, because being off-by-one will make the
* difference between success or failure in mdbx_node_add.
/* Добавляемый узел может не поместиться в страницу-половину вместе
* с количественной половиной узлов из исходной страницы. В худшем случае,
* в страницу-половину с добавляемым узлом могут попасть самые больше узлы
* из исходной страницы, а другую половину только узлы с самыми короткими
* ключами и с пустыми данными. Поэтому, чтобы найти подходящую границу
* разреза требуется итерировать узлы и считая их объем.
*
* It's also relevant if a page happens to be laid out
* such that one half of its nodes are all "small" and
* the other half of its nodes are "large". If the new
* item is also "large" and falls on the half with
* "large" nodes, it also may not fit.
*
* As a final tweak, if the new item goes on the last
* spot on the page (and thus, onto the new page), bias
* the split so the new page is emptier than the old page.
* This yields better packing during sequential inserts. */
* Однако, при простом количественном делении (без учета размера ключей
* и данных) на страницах-половинах будет примерно вдвое меньше узлов.
* Поэтому добавляемый узел точно поместится, если его размер не больше
* чем место "освобождающееся" от заголовков узлов, которые переедут
* в другую страницу-половину. Кроме этого, как минимум по одному байту
* будет в каждом ключе, в худшем случае кроме одного, который может быть
* нулевого размера. */
if (nkeys < 32 || new_size > max_space / 16) {
if (newindx == split_indx && split_indx + minkeys <= nkeys)
split_indx += 1;
mdbx_assert(env,
split_indx >= minkeys && split_indx <= nkeys - minkeys + 1);
const unsigned dim_nodes =
(newindx >= split_indx) ? split_indx : nkeys - split_indx;
const unsigned dim_used = (sizeof(indx_t) + NODESIZE + 1) * dim_nodes;
if (new_size >= dim_used) {
/* Find split point */
int dir;
if (newindx <= split_indx) {
i = 0;
dir = 1;
} else {
i = nkeys;
dir = -1;
}
i = (newindx < split_indx) ? 0 : nkeys;
int dir = (newindx < split_indx) ? 1 : -1;
size_t before = 0, after = new_size + page_used(env, mp);
int best = split_indx;
int best_offset = nkeys + 1;
unsigned best_split = split_indx;
unsigned best_offset = INT_MAX;
mdbx_trace("seek separator from %u, step %i, default %u, new-idx %u, "
"new-size %zu",
@@ -18783,8 +18799,8 @@ static int mdbx_page_split(MDBX_cursor *mc, const MDBX_val *const newkey,
(MDBX_node *)((char *)mp + tmp_ki_copy->mp_ptrs[i] + PAGEHDRSZ);
size = NODESIZE + node_ks(node) + sizeof(indx_t);
if (IS_LEAF(mp))
size += F_ISSET(node_flags(node), F_BIGDATA) ? sizeof(pgno_t)
: node_ds(node);
size += (node_flags(node) & F_BIGDATA) ? sizeof(pgno_t)
: node_ds(node);
size = EVEN(size);
}
@@ -18794,21 +18810,23 @@ static int mdbx_page_split(MDBX_cursor *mc, const MDBX_val *const newkey,
size, before, after, max_space);
if (before <= max_space && after <= max_space) {
int offset = branchless_abs(split_indx - i);
if (offset >= best_offset)
break;
best_offset = offset;
best = i;
const unsigned split = i + (dir > 0);
if (split >= minkeys && split <= nkeys + 1 - minkeys) {
const unsigned offset = branchless_abs(split_indx - split);
if (offset >= best_offset)
break;
best_offset = offset;
best_split = split;
}
}
i += dir;
} while (i < nkeys);
split_indx = best + (dir > 0);
split_indx = (split_indx <= nkeys - minkeys + 1) ? split_indx
: nkeys - minkeys + 1;
split_indx = (split_indx >= minkeys) ? split_indx : minkeys;
split_indx = best_split;
mdbx_trace("chosen %u", split_indx);
}
mdbx_assert(env,
split_indx >= minkeys && split_indx <= nkeys - minkeys + 1);
sepkey.iov_len = newkey->iov_len;
sepkey.iov_base = newkey->iov_base;
@@ -19557,7 +19575,8 @@ __cold static int mdbx_env_compact(MDBX_env *env, MDBX_txn *read_txn,
}
if (rc == MDBX_SUCCESS)
rc = mdbx_env_cwalk(&ctx, &root, 0);
mdbx_env_cthr_toggle(&ctx);
if (ctx.mc_wlen[ctx.mc_head & 1])
mdbx_env_cthr_toggle(&ctx);
mdbx_env_cthr_toggle(&ctx);
thread_err = mdbx_thread_join(thread);
mdbx_assert(env, (ctx.mc_tail == ctx.mc_head &&

View File

@@ -201,9 +201,12 @@ static int lck_op(const mdbx_filehandle_t fd, int cmd, const int lck,
((uint64_t)offset + (uint64_t)len));
for (;;) {
struct flock lock_op;
STATIC_ASSERT(sizeof(off_t) <= sizeof(lock_op.l_start) &&
sizeof(off_t) <= sizeof(lock_op.l_len) &&
OFF_T_MAX == (off_t)OFF_T_MAX);
STATIC_ASSERT_MSG(sizeof(off_t) <= sizeof(lock_op.l_start) &&
sizeof(off_t) <= sizeof(lock_op.l_len) &&
OFF_T_MAX == (off_t)OFF_T_MAX,
"Support for large/64-bit-sized files is misconfigured "
"for the target system and/or toolchain. "
"Please fix it or at least disable it completely.");
memset(&lock_op, 0, sizeof(lock_op));
lock_op.l_type = lck;
lock_op.l_whence = SEEK_SET;

View File

@@ -1,6 +1,6 @@
.\" Copyright 2015-2022 Leonid Yuriev <leo@yuriev.ru>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_CHK 1 "2022-04-22" "MDBX 0.11.8"
.TH MDBX_CHK 1 "2022-08-22" "MDBX 0.11.10"
.SH NAME
mdbx_chk \- MDBX checking tool
.SH SYNOPSIS

View File

@@ -2,7 +2,7 @@
.\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_COPY 1 "2022-04-22" "MDBX 0.11.8"
.TH MDBX_COPY 1 "2022-08-22" "MDBX 0.11.10"
.SH NAME
mdbx_copy \- MDBX environment copy tool
.SH SYNOPSIS

View File

@@ -1,7 +1,7 @@
.\" Copyright 2021-2022 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2014-2021 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_DROP 1 "2022-04-22" "MDBX 0.11.8"
.TH MDBX_DROP 1 "2022-08-22" "MDBX 0.11.10"
.SH NAME
mdbx_drop \- MDBX database delete tool
.SH SYNOPSIS

View File

@@ -2,7 +2,7 @@
.\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_DUMP 1 "2022-04-22" "MDBX 0.11.8"
.TH MDBX_DUMP 1 "2022-08-22" "MDBX 0.11.10"
.SH NAME
mdbx_dump \- MDBX environment export tool
.SH SYNOPSIS

View File

@@ -2,7 +2,7 @@
.\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_LOAD 1 "2022-04-22" "MDBX 0.11.8"
.TH MDBX_LOAD 1 "2022-08-22" "MDBX 0.11.10"
.SH NAME
mdbx_load \- MDBX environment import tool
.SH SYNOPSIS

View File

@@ -2,7 +2,7 @@
.\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_STAT 1 "2022-04-22" "MDBX 0.11.8"
.TH MDBX_STAT 1 "2022-08-22" "MDBX 0.11.10"
.SH NAME
mdbx_stat \- MDBX environment status tool
.SH SYNOPSIS

View File

@@ -203,64 +203,35 @@ __cold bug::~bug() noexcept {}
//------------------------------------------------------------------------------
template <typename PATH> struct path_to_pchar {
const std::string str;
path_to_pchar(const PATH &path) : str(path.generic_string()) {}
operator const char *() const { return str.c_str(); }
};
template <typename PATH>
MDBX_MAYBE_UNUSED PATH pchar_to_path(const char *c_str) {
return PATH(c_str);
}
template <> struct path_to_pchar<std::string> {
const char *const ptr;
path_to_pchar(const std::string &path) : ptr(path.c_str()) {}
operator const char *() const { return ptr; }
};
#if defined(_WIN32) || defined(_WIN64)
#ifndef WC_ERR_INVALID_CHARS
static const DWORD WC_ERR_INVALID_CHARS =
(6 /* Windows Vista */ <= /* MajorVersion */ LOBYTE(LOWORD(GetVersion())))
? 0x00000080
: 0;
#endif /* WC_ERR_INVALID_CHARS */
template <> struct path_to_pchar<std::wstring> {
std::string str;
path_to_pchar(const std::wstring &path) {
if (!path.empty()) {
const int chars =
WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS, path.data(),
int(path.size()), nullptr, 0, nullptr, nullptr);
if (chars == 0)
mdbx::error::throw_exception(GetLastError());
str.append(chars, '\0');
WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS, path.data(),
int(path.size()), const_cast<char *>(str.data()),
chars, nullptr, nullptr);
}
}
operator const char *() const { return str.c_str(); }
};
template <>
MDBX_MAYBE_UNUSED std::wstring pchar_to_path<std::wstring>(const char *c_str) {
std::wstring wstr;
if (c_str && *c_str) {
const int chars = MultiByteToWideChar(CP_UTF8, MB_ERR_INVALID_CHARS, c_str,
int(strlen(c_str)), nullptr, 0);
if (chars == 0)
std::string w2mb(const std::wstring &in) {
std::string out;
if (!in.empty()) {
const auto out_len = mdbx_w2mb(nullptr, 0, in.data(), in.size());
if (out_len < 1)
mdbx::error::throw_exception(GetLastError());
out.append(out_len, '\0');
if (out_len != mdbx_w2mb(const_cast<char *>(out.data()), out_len, in.data(),
in.size()))
mdbx::error::throw_exception(GetLastError());
wstr.append(chars, '\0');
MultiByteToWideChar(CP_UTF8, MB_ERR_INVALID_CHARS, c_str,
int(strlen(c_str)), const_cast<wchar_t *>(wstr.data()),
chars);
}
return wstr;
return out;
}
std::wstring mb2w(const char *in) {
std::wstring out;
if (in && *in) {
const auto in_len = strlen(in);
const auto out_len = mdbx_mb2w(nullptr, 0, in, in_len);
if (out_len < 1)
mdbx::error::throw_exception(GetLastError());
out.append(out_len, '\0');
if (out_len !=
mdbx_mb2w(const_cast<wchar_t *>(out.data()), out_len, in, in_len))
mdbx::error::throw_exception(GetLastError());
}
return out;
}
#endif /* Windows */
@@ -1245,43 +1216,6 @@ bool env::is_pristine() const {
bool env::is_empty() const { return get_stat().ms_leaf_pages == 0; }
#ifdef MDBX_STD_FILESYSTEM_PATH
env &env::copy(const MDBX_STD_FILESYSTEM_PATH &destination, bool compactify,
bool force_dynamic_size) {
const path_to_pchar<MDBX_STD_FILESYSTEM_PATH> utf8(destination);
error::success_or_throw(
::mdbx_env_copy(handle_, utf8,
(compactify ? MDBX_CP_COMPACT : MDBX_CP_DEFAULTS) |
(force_dynamic_size ? MDBX_CP_FORCE_DYNAMIC_SIZE
: MDBX_CP_DEFAULTS)));
return *this;
}
#endif /* MDBX_STD_FILESYSTEM_PATH */
#if defined(_WIN32) || defined(_WIN64)
env &env::copy(const ::std::wstring &destination, bool compactify,
bool force_dynamic_size) {
const path_to_pchar<::std::wstring> utf8(destination);
error::success_or_throw(
::mdbx_env_copy(handle_, utf8,
(compactify ? MDBX_CP_COMPACT : MDBX_CP_DEFAULTS) |
(force_dynamic_size ? MDBX_CP_FORCE_DYNAMIC_SIZE
: MDBX_CP_DEFAULTS)));
return *this;
}
#endif /* Windows */
env &env::copy(const ::std::string &destination, bool compactify,
bool force_dynamic_size) {
const path_to_pchar<::std::string> utf8(destination);
error::success_or_throw(
::mdbx_env_copy(handle_, utf8,
(compactify ? MDBX_CP_COMPACT : MDBX_CP_DEFAULTS) |
(force_dynamic_size ? MDBX_CP_FORCE_DYNAMIC_SIZE
: MDBX_CP_DEFAULTS)));
return *this;
}
env &env::copy(filehandle fd, bool compactify, bool force_dynamic_size) {
error::success_or_throw(
::mdbx_env_copy2fd(handle_, fd,
@@ -1291,35 +1225,69 @@ env &env::copy(filehandle fd, bool compactify, bool force_dynamic_size) {
return *this;
}
env &env::copy(const char *destination, bool compactify,
bool force_dynamic_size) {
error::success_or_throw(
::mdbx_env_copy(handle_, destination,
(compactify ? MDBX_CP_COMPACT : MDBX_CP_DEFAULTS) |
(force_dynamic_size ? MDBX_CP_FORCE_DYNAMIC_SIZE
: MDBX_CP_DEFAULTS)));
return *this;
}
env &env::copy(const ::std::string &destination, bool compactify,
bool force_dynamic_size) {
return copy(destination.c_str(), compactify, force_dynamic_size);
}
#ifdef MDBX_STD_FILESYSTEM_PATH
env &env::copy(const MDBX_STD_FILESYSTEM_PATH &destination, bool compactify,
bool force_dynamic_size) {
return copy(destination.native(), compactify, force_dynamic_size);
}
#endif /* MDBX_STD_FILESYSTEM_PATH */
#if defined(_WIN32) || defined(_WIN64)
env &env::copy(const ::std::wstring &destination, bool compactify,
bool force_dynamic_size) {
return copy(w2mb(destination), compactify, force_dynamic_size);
}
#endif /* Windows */
path env::get_path() const {
const char *c_str;
error::success_or_throw(::mdbx_env_get_path(handle_, &c_str));
return pchar_to_path<path>(c_str);
#if defined(_WIN32) || defined(_WIN64)
static_assert(sizeof(path::value_type) == sizeof(wchar_t), "Oops");
return path(mb2w(c_str));
#else
static_assert(sizeof(path::value_type) == sizeof(char), "Oops");
return path(c_str);
#endif
}
bool env::remove(const char *pathname, const remove_mode mode) {
return error::boolean_or_throw(
::mdbx_env_delete(pathname, MDBX_env_delete_mode_t(mode)));
}
bool env::remove(const ::std::string &pathname, const remove_mode mode) {
return remove(pathname.c_str(), mode);
}
#ifdef MDBX_STD_FILESYSTEM_PATH
bool env::remove(const MDBX_STD_FILESYSTEM_PATH &pathname,
const remove_mode mode) {
const path_to_pchar<MDBX_STD_FILESYSTEM_PATH> utf8(pathname);
return error::boolean_or_throw(
::mdbx_env_delete(utf8, MDBX_env_delete_mode_t(mode)));
return remove(pathname.native(), mode);
}
#endif /* MDBX_STD_FILESYSTEM_PATH */
#if defined(_WIN32) || defined(_WIN64)
bool env::remove(const ::std::wstring &pathname, const remove_mode mode) {
const path_to_pchar<::std::wstring> utf8(pathname);
return error::boolean_or_throw(
::mdbx_env_delete(utf8, MDBX_env_delete_mode_t(mode)));
return remove(w2mb(pathname), mode);
}
#endif /* Windows */
bool env::remove(const ::std::string &pathname, const remove_mode mode) {
const path_to_pchar<::std::string> utf8(pathname);
return error::boolean_or_throw(
::mdbx_env_delete(utf8, MDBX_env_delete_mode_t(mode)));
}
//------------------------------------------------------------------------------
static inline MDBX_env *create_env() {
@@ -1356,97 +1324,64 @@ __cold void env_managed::setup(unsigned max_maps, unsigned max_readers) {
error::success_or_throw(::mdbx_env_set_maxdbs(handle_, max_maps));
}
#ifdef MDBX_STD_FILESYSTEM_PATH
__cold env_managed::env_managed(const MDBX_STD_FILESYSTEM_PATH &pathname,
__cold env_managed::env_managed(const char *pathname,
const operate_parameters &op, bool accede)
: env_managed(create_env()) {
setup(op.max_maps, op.max_readers);
const path_to_pchar<MDBX_STD_FILESYSTEM_PATH> utf8(pathname);
error::success_or_throw(
::mdbx_env_open(handle_, utf8, op.make_flags(accede), 0));
::mdbx_env_open(handle_, pathname, op.make_flags(accede), 0));
if (op.options.nested_write_transactions &&
!get_options().nested_write_transactions)
MDBX_CXX20_UNLIKELY error::throw_exception(MDBX_INCOMPATIBLE);
}
__cold env_managed::env_managed(const MDBX_STD_FILESYSTEM_PATH &pathname,
__cold env_managed::env_managed(const char *pathname,
const env_managed::create_parameters &cp,
const env::operate_parameters &op, bool accede)
: env_managed(create_env()) {
setup(op.max_maps, op.max_readers);
const path_to_pchar<MDBX_STD_FILESYSTEM_PATH> utf8(pathname);
set_geometry(cp.geometry);
error::success_or_throw(
::mdbx_env_open(handle_, utf8, op.make_flags(accede, cp.use_subdirectory),
cp.file_mode_bits));
error::success_or_throw(::mdbx_env_open(
handle_, pathname, op.make_flags(accede, cp.use_subdirectory),
cp.file_mode_bits));
if (op.options.nested_write_transactions &&
!get_options().nested_write_transactions)
MDBX_CXX20_UNLIKELY error::throw_exception(MDBX_INCOMPATIBLE);
}
__cold env_managed::env_managed(const ::std::string &pathname,
const operate_parameters &op, bool accede)
: env_managed(pathname.c_str(), op, accede) {}
__cold env_managed::env_managed(const ::std::string &pathname,
const env_managed::create_parameters &cp,
const env::operate_parameters &op, bool accede)
: env_managed(pathname.c_str(), cp, op, accede) {}
#ifdef MDBX_STD_FILESYSTEM_PATH
__cold env_managed::env_managed(const MDBX_STD_FILESYSTEM_PATH &pathname,
const operate_parameters &op, bool accede)
: env_managed(pathname.native(), op, accede) {}
__cold env_managed::env_managed(const MDBX_STD_FILESYSTEM_PATH &pathname,
const env_managed::create_parameters &cp,
const env::operate_parameters &op, bool accede)
: env_managed(pathname.native(), cp, op, accede) {}
#endif /* MDBX_STD_FILESYSTEM_PATH */
#if defined(_WIN32) || defined(_WIN64)
__cold env_managed::env_managed(const ::std::wstring &pathname,
const operate_parameters &op, bool accede)
: env_managed(create_env()) {
setup(op.max_maps, op.max_readers);
const path_to_pchar<::std::wstring> utf8(pathname);
error::success_or_throw(
::mdbx_env_open(handle_, utf8, op.make_flags(accede), 0));
if (op.options.nested_write_transactions &&
!get_options().nested_write_transactions)
MDBX_CXX20_UNLIKELY error::throw_exception(MDBX_INCOMPATIBLE);
}
: env_managed(w2mb(pathname), op, accede) {}
__cold env_managed::env_managed(const ::std::wstring &pathname,
const env_managed::create_parameters &cp,
const env::operate_parameters &op, bool accede)
: env_managed(create_env()) {
setup(op.max_maps, op.max_readers);
const path_to_pchar<::std::wstring> utf8(pathname);
set_geometry(cp.geometry);
error::success_or_throw(
::mdbx_env_open(handle_, utf8, op.make_flags(accede, cp.use_subdirectory),
cp.file_mode_bits));
if (op.options.nested_write_transactions &&
!get_options().nested_write_transactions)
MDBX_CXX20_UNLIKELY error::throw_exception(MDBX_INCOMPATIBLE);
}
: env_managed(w2mb(pathname), cp, op, accede) {}
#endif /* Windows */
__cold env_managed::env_managed(const ::std::string &pathname,
const operate_parameters &op, bool accede)
: env_managed(create_env()) {
setup(op.max_maps, op.max_readers);
const path_to_pchar<::std::string> utf8(pathname);
error::success_or_throw(
::mdbx_env_open(handle_, utf8, op.make_flags(accede), 0));
if (op.options.nested_write_transactions &&
!get_options().nested_write_transactions)
MDBX_CXX20_UNLIKELY error::throw_exception(MDBX_INCOMPATIBLE);
}
__cold env_managed::env_managed(const ::std::string &pathname,
const env_managed::create_parameters &cp,
const env::operate_parameters &op, bool accede)
: env_managed(create_env()) {
setup(op.max_maps, op.max_readers);
const path_to_pchar<::std::string> utf8(pathname);
set_geometry(cp.geometry);
error::success_or_throw(
::mdbx_env_open(handle_, utf8, op.make_flags(accede, cp.use_subdirectory),
cp.file_mode_bits));
if (op.options.nested_write_transactions &&
!get_options().nested_write_transactions)
MDBX_CXX20_UNLIKELY error::throw_exception(MDBX_INCOMPATIBLE);
}
//------------------------------------------------------------------------------
txn_managed txn::start_nested() {

View File

@@ -518,14 +518,35 @@ MDBX_INTERNAL_FUNC int mdbx_fastmutex_release(mdbx_fastmutex_t *fastmutex) {
/*----------------------------------------------------------------------------*/
#if defined(_WIN32) || defined(_WIN64)
#ifndef WC_ERR_INVALID_CHARS
static const DWORD WC_ERR_INVALID_CHARS =
(6 /* Windows Vista */ <= /* MajorVersion */ LOBYTE(LOWORD(GetVersion())))
? 0x00000080
: 0;
#endif /* WC_ERR_INVALID_CHARS */
MDBX_INTERNAL_FUNC size_t mdbx_mb2w(wchar_t *dst, size_t dst_n, const char *src,
size_t src_n) {
return MultiByteToWideChar(CP_THREAD_ACP, MB_ERR_INVALID_CHARS, src,
(int)src_n, dst, (int)dst_n);
}
MDBX_INTERNAL_FUNC size_t mdbx_w2mb(char *dst, size_t dst_n, const wchar_t *src,
size_t src_n) {
return WideCharToMultiByte(CP_THREAD_ACP, WC_ERR_INVALID_CHARS, src,
(int)src_n, dst, (int)dst_n, nullptr, nullptr);
}
#endif /* Windows */
/*----------------------------------------------------------------------------*/
MDBX_INTERNAL_FUNC int mdbx_removefile(const char *pathname) {
#if defined(_WIN32) || defined(_WIN64)
const size_t wlen = mbstowcs(nullptr, pathname, INT_MAX);
if (wlen < 1 || wlen > /* MAX_PATH */ INT16_MAX)
return ERROR_INVALID_NAME;
wchar_t *const pathnameW = _alloca((wlen + 1) * sizeof(wchar_t));
if (wlen != mbstowcs(pathnameW, pathname, wlen + 1))
return ERROR_INVALID_NAME;
const wchar_t *pathnameW = nullptr;
MUSTDIE_MB2WIDE(pathname, pathnameW);
return DeleteFileW(pathnameW) ? MDBX_SUCCESS : (int)GetLastError();
#else
return unlink(pathname) ? errno : MDBX_SUCCESS;
@@ -538,12 +559,8 @@ static bool is_valid_fd(int fd) { return !(isatty(fd) < 0 && errno == EBADF); }
MDBX_INTERNAL_FUNC int mdbx_removedirectory(const char *pathname) {
#if defined(_WIN32) || defined(_WIN64)
const size_t wlen = mbstowcs(nullptr, pathname, INT_MAX);
if (wlen < 1 || wlen > /* MAX_PATH */ INT16_MAX)
return ERROR_INVALID_NAME;
wchar_t *const pathnameW = _alloca((wlen + 1) * sizeof(wchar_t));
if (wlen != mbstowcs(pathnameW, pathname, wlen + 1))
return ERROR_INVALID_NAME;
const wchar_t *pathnameW = nullptr;
MUSTDIE_MB2WIDE(pathname, pathnameW);
return RemoveDirectoryW(pathnameW) ? MDBX_SUCCESS : (int)GetLastError();
#else
return rmdir(pathname) ? errno : MDBX_SUCCESS;
@@ -557,12 +574,8 @@ MDBX_INTERNAL_FUNC int mdbx_openfile(const enum mdbx_openfile_purpose purpose,
*fd = INVALID_HANDLE_VALUE;
#if defined(_WIN32) || defined(_WIN64)
const size_t wlen = mbstowcs(nullptr, pathname, INT_MAX);
if (wlen < 1 || wlen > /* MAX_PATH */ INT16_MAX)
return ERROR_INVALID_NAME;
wchar_t *const pathnameW = _alloca((wlen + 1) * sizeof(wchar_t));
if (wlen != mbstowcs(pathnameW, pathname, wlen + 1))
return ERROR_INVALID_NAME;
const wchar_t *pathnameW = nullptr;
MUSTDIE_MB2WIDE(pathname, pathnameW);
DWORD CreationDisposition = unix_mode_bits ? OPEN_ALWAYS : OPEN_EXISTING;
DWORD FlagsAndAttributes =

View File

@@ -180,6 +180,11 @@ static inline void mdbx_free(void *ptr) { HeapFree(GetProcessHeap(), 0, ptr); }
#define vsnprintf _vsnprintf /* ntdll */
#endif
MDBX_INTERNAL_FUNC size_t mdbx_mb2w(wchar_t *dst, size_t dst_n, const char *src,
size_t src_n);
MDBX_INTERNAL_FUNC size_t mdbx_w2mb(char *dst, size_t dst_n, const wchar_t *src,
size_t src_n);
#else /*----------------------------------------------------------------------*/
typedef pthread_t mdbx_thread_t;
@@ -549,6 +554,20 @@ MDBX_INTERNAL_FUNC int mdbx_rpid_check(MDBX_env *env, uint32_t pid);
#if defined(_WIN32) || defined(_WIN64)
#define MUSTDIE_MB2WIDE(FROM, TO) \
do { \
const char *const from_tmp = (FROM); \
const size_t from_mblen = strlen(from_tmp); \
const size_t to_wlen = mdbx_mb2w(nullptr, 0, from_tmp, from_mblen); \
if (to_wlen < 1 || to_wlen > /* MAX_PATH */ INT16_MAX) \
return ERROR_INVALID_NAME; \
wchar_t *const to_tmp = _alloca((to_wlen + 1) * sizeof(wchar_t)); \
if (to_wlen + 1 != \
mdbx_mb2w(to_tmp, to_wlen + 1, from_tmp, from_mblen + 1)) \
return ERROR_INVALID_NAME; \
(TO) = to_tmp; \
} while (0)
typedef void(WINAPI *MDBX_srwlock_function)(MDBX_srwlock *);
MDBX_INTERNAL_VAR MDBX_srwlock_function mdbx_srwlock_Init,
mdbx_srwlock_AcquireShared, mdbx_srwlock_ReleaseShared,

View File

@@ -14,6 +14,7 @@ GEOMETRY_JITTER=yes
BANNER="$(which banner 2>/dev/null | echo echo)"
UNAME="$(uname -s 2>/dev/null || echo Unknown)"
DB_UPTO_MB=17408
PAGESIZE=min
while [ -n "$1" ]
do
@@ -33,6 +34,7 @@ do
echo "--dir PATH Specifies directory for test DB and other files (it will be cleared)"
echo "--db-upto-mb NN Limits upper size of test DB to the NN megabytes"
echo "--no-geometry-jitter Disable jitter for geometry upper-size"
echo "--pagesize NN Use specified page size (256 is minimal and used by default) "
echo "--help Print this usage help and exit"
exit -2
;;
@@ -109,6 +111,39 @@ do
--no-geometry-jitter)
GEOMETRY_JITTER=no
;;
--pagesize)
case "$2" in
min|max|256|512|1024|2048|4096|8192|16386|32768|65536)
PAGESIZE=$2
;;
1|1k|1K|k|K)
PAGESIZE=$((1024*1))
;;
2|2k|2K)
PAGESIZE=$((1024*2))
;;
4|4k|4K)
PAGESIZE=$((1024*4))
;;
8|8k|8K)
PAGESIZE=$((1024*8))
;;
16|16k|16K)
PAGESIZE=$((1024*16))
;;
32|32k|32K)
PAGESIZE=$((1024*32))
;;
64|64k|64K)
PAGESIZE=$((1024*64))
;;
*)
echo "Invalig page size '$2'"
exit -2
;;
esac
shift
;;
*)
echo "Unknown option '$1'"
exit -2
@@ -346,65 +381,65 @@ for nops in 10 33 100 333 1000 3333 10000 33333 100000 333333 1000000 3333333 10
split=30
caption="Probe #$((++count)) int-key,with-dups, split=${split}, case $((++subcase)) of ${cases}" probe \
--pagesize=min --size-upper-upto=${db_size_mb}M --table=+key.integer,+data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+key.integer,+data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%3]} \
--keygen.seed=${seed}
caption="Probe #$((++count)) int-key,int-data, split=${split}, case $((++subcase)) of ${cases}" probe \
--pagesize=min --size-upper-upto=${db_size_mb}M --table=+key.integer,+data.integer --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+key.integer,+data.integer --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%3]} \
--keygen.seed=${seed}
caption="Probe #$((++count)) with-dups, split=${split}, case $((++subcase)) of ${cases}" probe \
--pagesize=min --size-upper-upto=${db_size_mb}M --table=+data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%3]} \
--keygen.seed=${seed}
split=24
caption="Probe #$((++count)) int-key,with-dups, split=${split}, case $((++subcase)) of ${cases}" probe \
--pagesize=min --size-upper-upto=${db_size_mb}M --table=+key.integer,+data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+key.integer,+data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%3]} \
--keygen.seed=${seed}
caption="Probe #$((++count)) int-key,int-data, split=${split}, case $((++subcase)) of ${cases}" probe \
--pagesize=min --size-upper-upto=${db_size_mb}M --table=+key.integer,+data.integer --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+key.integer,+data.integer --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%3]} \
--keygen.seed=${seed}
caption="Probe #$((++count)) with-dups, split=${split}, case $((++subcase)) of ${cases}" probe \
--pagesize=min --size-upper-upto=${db_size_mb}M --table=+data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%3]} \
--keygen.seed=${seed}
split=16
caption="Probe #$((++count)) int-key,w/o-dups, split=${split}, case $((++subcase)) of ${cases}" probe \
--pagesize=min --size-upper-upto=${db_size_mb}M --table=+key.integer,-data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=1111 \
--pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+key.integer,-data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=1111 \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%3]} \
--keygen.seed=${seed}
caption="Probe #$((++count)) int-key,with-dups, split=${split}, case $((++subcase)) of ${cases}" probe \
--pagesize=min --size-upper-upto=${db_size_mb}M --table=+key.integer,+data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+key.integer,+data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%3]} \
--keygen.seed=${seed}
caption="Probe #$((++count)) int-key,int-data, split=${split}, case $((++subcase)) of ${cases}" probe \
--pagesize=min --size-upper-upto=${db_size_mb}M --table=+key.integer,+data.integer --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+key.integer,+data.integer --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%3]} \
--keygen.seed=${seed}
caption="Probe #$((++count)) w/o-dups, split=${split}, case $((++subcase)) of ${cases}" probe \
--pagesize=min --size-upper-upto=${db_size_mb}M --table=-data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=1111 \
--pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=-data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=1111 \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%3]} \
--keygen.seed=${seed}
caption="Probe #$((++count)) with-dups, split=${split}, case $((++subcase)) of ${cases}" probe \
--pagesize=min --size-upper-upto=${db_size_mb}M --table=+data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%3]} \
--keygen.seed=${seed}
split=4
caption="Probe #$((++count)) int-key,w/o-dups, split=${split}, case $((++subcase)) of ${cases}" probe \
--pagesize=min --size-upper-upto=${db_size_mb}M --table=+key.integer,-data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=1111 \
--pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+key.integer,-data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=1111 \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%3]} \
--keygen.seed=${seed}
caption="Probe #$((++count)) int-key,int-data, split=${split}, case $((++subcase)) of ${cases}" probe \
--pagesize=min --size-upper-upto=${db_size_mb}M --table=+key.integer,+data.integer --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+key.integer,+data.integer --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%3]} \
--keygen.seed=${seed}
caption="Probe #$((++count)) w/o-dups, split=${split}, case $((++subcase)) of ${cases}" probe \
--pagesize=min --size-upper-upto=${db_size_mb}M --table=-data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=1111 \
--pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=-data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=1111 \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%3]} \
--keygen.seed=${seed}
done # options

View File

@@ -1189,28 +1189,29 @@ bool testcase::check_batch_get() {
char dump_key[128], dump_value[128];
char dump_key_batch[128], dump_value_batch[128];
MDBX_cursor *cursor;
int err = mdbx_cursor_open(txn_guard.get(), dbi, &cursor);
if (err != MDBX_SUCCESS)
failure_perror("mdbx_cursor_open()", err);
MDBX_cursor *check_cursor;
int check_err = mdbx_cursor_open(txn_guard.get(), dbi, &check_cursor);
if (check_err != MDBX_SUCCESS)
failure_perror("mdbx_cursor_open()", check_err);
MDBX_cursor *batch_cursor;
err = mdbx_cursor_open(txn_guard.get(), dbi, &batch_cursor);
if (err != MDBX_SUCCESS)
failure_perror("mdbx_cursor_open()", err);
int batch_err = mdbx_cursor_open(txn_guard.get(), dbi, &batch_cursor);
if (batch_err != MDBX_SUCCESS)
failure_perror("mdbx_cursor_open()", batch_err);
bool rc = true;
MDBX_val pairs[42];
size_t count = 0xDeadBeef;
err = mdbx_cursor_get_batch(batch_cursor, &count, pairs, ARRAY_LENGTH(pairs),
MDBX_FIRST);
bool rc = true;
MDBX_cursor_op batch_op;
batch_err = mdbx_cursor_get_batch(batch_cursor, &count, pairs,
ARRAY_LENGTH(pairs), batch_op = MDBX_FIRST);
size_t i, n = 0;
while (err == MDBX_SUCCESS) {
while (batch_err == MDBX_SUCCESS || batch_err == MDBX_RESULT_TRUE) {
for (i = 0; i < count; i += 2) {
mdbx::slice k, v;
int err2 = mdbx_cursor_get(cursor, &k, &v, MDBX_NEXT);
if (err2 != MDBX_SUCCESS)
failure_perror("mdbx_cursor_open()", err2);
check_err = mdbx_cursor_get(check_cursor, &k, &v, MDBX_NEXT);
if (check_err != MDBX_SUCCESS)
failure_perror("batch-verify: mdbx_cursor_get(MDBX_NEXT)", check_err);
if (k != pairs[i] || v != pairs[i + 1]) {
log_error(
"batch-get pair mismatch %zu/%zu: sequential{%s, %s} != "
@@ -1224,29 +1225,32 @@ bool testcase::check_batch_get() {
}
}
n += i / 2;
err = mdbx_cursor_get_batch(batch_cursor, &count, pairs,
ARRAY_LENGTH(pairs), MDBX_NEXT);
batch_op = (batch_err == MDBX_RESULT_TRUE) ? MDBX_GET_CURRENT : MDBX_NEXT;
batch_err = mdbx_cursor_get_batch(batch_cursor, &count, pairs,
ARRAY_LENGTH(pairs), batch_op);
}
if (err != MDBX_NOTFOUND)
failure_perror("mdbx_cursor_get_batch()", err);
err = mdbx_cursor_eof(batch_cursor);
if (err != MDBX_RESULT_TRUE) {
log_error("batch-get %s cursor not-eof %d", "batch", err);
rc = false;
}
err = mdbx_cursor_on_last(batch_cursor);
if (err != MDBX_RESULT_TRUE) {
log_error("batch-get %s cursor not-on-last %d", "batch", err);
if (batch_err != MDBX_NOTFOUND) {
log_error("mdbx_cursor_get_batch(), op %u, err %d", batch_op, batch_err);
rc = false;
}
err = mdbx_cursor_on_last(cursor);
if (err != MDBX_RESULT_TRUE) {
log_error("batch-get %s cursor not-on-last %d", "checked", err);
batch_err = mdbx_cursor_eof(batch_cursor);
if (batch_err != MDBX_RESULT_TRUE) {
log_error("batch-get %s-cursor not-eof %d", "batch", batch_err);
rc = false;
}
mdbx_cursor_close(cursor);
batch_err = mdbx_cursor_on_last(batch_cursor);
if (batch_err != MDBX_RESULT_TRUE) {
log_error("batch-get %s-cursor not-on-last %d", "batch", batch_err);
rc = false;
}
check_err = mdbx_cursor_on_last(check_cursor);
if (check_err != MDBX_RESULT_TRUE) {
log_error("batch-get %s-cursor not-on-last %d", "checked", check_err);
rc = false;
}
mdbx_cursor_close(check_cursor);
mdbx_cursor_close(batch_cursor);
return rc;
}

View File

@@ -4,7 +4,7 @@
msync(start)
fun:msync
...
fun:mdbx_sync_locked
fun:mdbx_sync_locked*
}
{
msync-whole-mmap-2
@@ -12,7 +12,7 @@
msync(start)
fun:msync
...
fun:mdbx_env_sync_internal
fun:mdbx_env_sync_internal*
}
{
msync-whole-mmap-3
@@ -20,7 +20,7 @@
msync(start)
fun:msync
...
fun:mdbx_mapresize
fun:mdbx_mapresize*
}
{
msync-wipe-steady
@@ -28,7 +28,7 @@
msync(start)
fun:msync
...
fun:mdbx_wipe_steady
fun:mdbx_wipe_steady*
}
# memcmp() inside mdbx_iov_write() as workaround for todo4recovery://erased_by_github/libmdbx/issues/269