Compare commits

...

45 Commits

Author SHA1 Message Date
Леонид Юрьев (Leonid Yuriev)
b415265d16 mdbx: release v0.11.14 (Sergey Kapitsa)
The stable bugfix release in memory of [Sergey Kapitsa](https://en.wikipedia.org/wiki/Sergey_Kapitsa) on his 95th birthday.

Fixes:
------

 - backport: Fixed insignificant typo of `||` inside `#if` byte-order condition.

 - backport: Fixed `SIGSEGV` or an erroneous call to `free()` in situations where
   errors occur when reopening by `mdbx_env_open()` of a previously used
   environment.

 - backport: Fixed `cursor_put_nochecklen()` internals for case when dupsort'ed named subDb
   contains a single key with multiple values (aka duplicates), which are replaced
   with a single value by put-operation with the `MDBX_UPSERT+MDBX_ALLDUPS` flags.
   In this case, the database becomes completely empty, without any pages.
   However exactly this condition was not considered and thus wasn't handled correctly.
   See [issue#8](https://gitflic.ru/project/erthink/libmdbx/issue/8) for more information.

 - backport: Fixed extra assertion inside `override_meta()`, which could
   lead to false-positive failing of the assertion in a debug builds during
   DB recovery and auto-rollback.

 - backport: Refined the `__cold`/`__hot` macros to avoid the
   `error: inlining failed in call to ‘always_inline FOO(...)’: target specific option mismatch`
   issue during build using GCC >10.x for SH4 arch.

Minors:
-------

 - backport: Using the https://libmdbx.dqdkfa.ru/dead-github
   for resources deleted by the Github' administration.
 - backport: Fixed English typos.
 - backport: Fixed proto of `__asan_default_options()`.
 - backport: Fixed doxygen-description of C++ API, especially of C++20 concepts.
 - backport: Refined `const` and `noexcept` for few C++ API methods.
 - backport: Fixed copy&paste typo of "Getting started".
 - backport: Update MithrilDB status.
 - backport: Resolve false-posirive `used uninitialized` warning from GCC >10.x
   while build for SH4 arch.

22 files changed, 250 insertions(+), 174 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2023-02-14 15:10:41 +03:00
Леонид Юрьев (Leonid Yuriev)
257a534fbe mdbx: update ChangeLog. 2023-02-13 21:35:20 +03:00
Леонид Юрьев (Leonid Yuriev)
33b5aeb768 mdbx: refine __cold/__hot macros (backport).
В том числе для устранения проблемы
`error: inlining failed in call to ‘always_inline FOO(...)’: target specific option mismatch`
при сборке посредством GCC >10.x для SH4.
2023-02-13 21:35:20 +03:00
Леонид Юрьев (Leonid Yuriev)
f532e907e9 mdbx: fix false-positive SH4 GCC warning (backport). 2023-02-13 21:35:20 +03:00
Леонид Юрьев (Leonid Yuriev)
60736dbabb mdbx: fix English typos (backport).
Thanks to Dimitris Apostolou <dimitris.apostolou@icloud.com>
2023-02-13 21:35:20 +03:00
Леонид Юрьев (Leonid Yuriev)
24df8073ac mdbx: fix extra assertion inside override_meta() (backport). 2023-02-13 21:35:20 +03:00
Леонид Юрьев (Leonid Yuriev)
d504ca1747 mdbx: fix proto of __asan_default_options() (backport). 2023-02-13 21:35:14 +03:00
Леонид Юрьев (Leonid Yuriev)
b7ace5b216 mdbx-doc: fix doxygen-description of C++ API, especially of C++20 concepts (backport). 2023-02-13 21:28:33 +03:00
Леонид Юрьев (Leonid Yuriev)
2fabac18c0 mdbx++: refine const and noexcept for few C++ API methods (backport). 2023-02-13 21:27:55 +03:00
Леонид Юрьев (Leonid Yuriev)
51789f3605 mdbx: fix put(MDBX_UPSERT+MDBX_ALLDUPS) for case of replacement all values of a single key inside dupsorted subDb (backport).
Fixed cursor_put_nochecklen() internals for case when dupsort'ed named subDb
contains a single key with multiple values (aka duplicates), which are replaced
with a single value by put-operation with the `MDBX_UPSERT+MDBX_ALLDUPS` flags.

In this case, the database becomes completely empty, without any pages.
However exactly this condition was not considered and
thus wasn't handled correctly.

Fixes https://gitflic.ru/project/erthink/libmdbx/issue/8

Thanks Masatoshi Fukunaga <https://gitflic.ru/user/mah0x211> for reporting.
2023-02-13 21:27:55 +03:00
Леонид Юрьев (Leonid Yuriev)
6899142872 mdbx: remove extra assertion (backport).
The removed assertion could be triggered in debug builds when a reading
and writing transactions are overlapped simultaneously with a change of DB
size.

There were no other negative consequences.
2023-02-13 21:27:44 +03:00
Леонид Юрьев (Leonid Yuriev)
c44c8132e4 mdbx-doc: fix copy&paste typo of "Getting started" (backport). 2023-02-13 21:27:10 +03:00
Леонид Юрьев (Leonid Yuriev)
d376feb7bc mdbx: update MithrilDB status (backport). 2023-02-13 21:26:49 +03:00
Леонид Юрьев (Leonid Yuriev)
47851135f3 mdbx-doc: using the https://libmdbx.dqdkfa.ru/dead-github for resources deleted by the Github' administration (backport). 2022-12-03 06:23:31 +03:00
Леонид Юрьев (Leonid Yuriev)
6139443ef1 mdbx: fix SIGSEGV/invalid-deref/invalid-free inside env_close() when mdbx_env_open() failed in re-open case (backport).
Thanks to [@leisim](https://t.me/leisim) for [reporting](https://t.me/libmdbx/3946) this issue.
2022-12-03 05:48:30 +03:00
Jan Biedermann
30f292d496 mdbx: fix typo of || inside #if byte-order condition (backport).
https://gitflic.ru/project/erthink/libmdbx/merge-request/4
2022-12-03 05:47:36 +03:00
Леонид Юрьев (Leonid Yuriev)
d6b47c7bd1 mdbx: release v0.11.13 (Swashplate)
The stable bugfix release in memory of [Boris Yuryev](https://ru.wikipedia.org/wiki/Юрьев,_Борис_Николаевич) on his 133rd birthday.

Fixes:
------

 - Fixed builds with older libc versions after using `fcntl64()` (backport).
 - Fixed builds with  older `stdatomic.h` versions,
   where the `ATOMIC_*_LOCK_FREE` macros mistakenly redefined using functions (backport).
 - Added workaround for `mremap()` defect to avoid assertion failure (backport).
 - Workaround for `encryptfs` bug(s) in the `copy_file_range` implementation  (backport).
 - Fixed unexpected `MDBX_BUSY` from `mdbx_env_set_option()`, `mdbx_env_set_syncbytes()`
   and `mdbx_env_set_syncperiod()` (backport).
 - CMake requirements lowered to version 3.0.2 (backport).
 - Added admonition of insecure for RISC-V (backport).

Minors:
-------

 - Minor clarification output of `--help` for `mdbx_test` (backport).
 - Added admonition of insecure for RISC-V (backport).
 - Stochastic scripts and CMake files synchronized with the `devel` branch.
 - Use `--dont-check-ram-size` for small-tests make-targets (backport).

30 files changed, 405 insertions(+), 136 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2022-11-10 13:07:28 +03:00
Леонид Юрьев (Leonid Yuriev)
8f0c5bc7c7 mdbx: update ChangeLog. 2022-11-10 12:11:11 +03:00
Леонид Юрьев (Leonid Yuriev)
29eab4afdd mdbx-make: use --dont-check-ram-size for small-tests targets (backport). 2022-11-10 12:09:34 +03:00
Леонид Юрьев (Leonid Yuriev)
12717aac8c mdbx-test: sync stochastic scripts with devel branch. 2022-11-10 10:57:16 +03:00
Леонид Юрьев (Leonid Yuriev)
e992da9efe mdbx-cmake: синхронизация CMake-файлов. 2022-11-10 01:10:39 +03:00
Леонид Юрьев (Leonid Yuriev)
d7e4cb2e22 mdbx: add admonition of insecure for RISC-V (backport). 2022-11-10 01:10:39 +03:00
Леонид Юрьев (Leonid Yuriev)
af1d01ffb3 mdbx-windows: уточнение проверок макросов MinGW и устранение предупреждений (backport). 2022-11-10 01:10:39 +03:00
Леонид Юрьев (Leonid Yuriev)
cce052e869 mdbx-cmake: совместимость с CMake 3.0.2 для CI на старых системах (backport). 2022-11-10 00:48:54 +03:00
Леонид Юрьев (Leonid Yuriev)
094c2f345d mdbx: переименование packages/rpm (backport). 2022-11-10 00:48:54 +03:00
Леонид Юрьев (Leonid Yuriev)
7b2eee91af mdbx: добавлено описание параметров MDBX_debug_func и MDBX_debug_func (backport). 2022-11-10 00:48:54 +03:00
Леонид Юрьев (Leonid Yuriev)
d863629387 mdbx-test: минорное уточнение подсказки --help (backport). 2022-11-10 00:48:54 +03:00
Леонид Юрьев (Leonid Yuriev)
1b3b6e4479 mdbx: исправление неожиданного MDBX_BUSY из mdbx_env_set_option() (backport). 2022-11-10 00:48:54 +03:00
Леонид Юрьев (Leonid Yuriev)
5dcc0171fa mdbx-windows: попытка борьбы с ложно-положительными конфликтами LockFileEx() (backport). 2022-11-10 00:48:54 +03:00
Леонид Юрьев (Leonid Yuriev)
425730c2b3 mdbx: чуток апостофов для имен в doxygen-комментариях (backport). 2022-11-07 16:44:00 +03:00
Леонид Юрьев (Leonid Yuriev)
471e854551 mdbx: workaround for encryptfs's copy_file_range() bug(s) (backport).
Выяснилось что утилита `mdbx_copy` и функции `mdbx_env_copy()` могут
создавать ПРОБЛЕМЫ если целевой файл расположен в encryptfs (такая
файловая система в Linux).

При этом может быть четыре исхода в зависимости от версии ядра и
положения звезд на небе:
 - всё хорошо;
 - плохие данные в копии без возврата ошибок;
 - ошибка EINVAL(22) при копировании;
 - oops или зависание ядра, отвал смонтированной encryptfs и т.п.

В текущем понимании, причина обусловлена ошибой в коде fs, которая
проявляется при использовании системного вызова `copy_file_range`.
2022-11-07 16:44:00 +03:00
Леонид Юрьев (Leonid Yuriev)
1bd0eb35bc mdbx-cmake: исправление для совместимости с CMake 3.8 (backport). 2022-11-07 16:43:32 +03:00
Леонид Юрьев (Leonid Yuriev)
6fdae12996 mdbx: workaround for mremap() defect (backport).
Есть основание полагать, что mremap() может возвращать MAP_FAILED, но НЕ
устанавливать errno в некоторых пограничных ситуациях. Например, когда
системных ресурсов не хватает на актуализацию/копирование/клонирование
состояния отображения на финальной стадии, в том числе из-за раскраски
исходного отображения разными флагами через madvise().
2022-10-14 14:05:44 +03:00
Леонид Юрьев (Leonid Yuriev)
2684c89d91 mdbx: workaround for older stdatomic.h versions, where the ATOMIC_*_LOCK_FREE macros mistakenly redefined using functions (backport). 2022-10-14 14:05:16 +03:00
Леонид Юрьев (Leonid Yuriev)
7b60363a31 mdbx: fix builds with older libc versions after using fcntl64() (backport). 2022-10-14 14:01:02 +03:00
Леонид Юрьев (Leonid Yuriev)
290630f118 mdbx: release v0.11.12 (Эребуни)
The stable bugfix release.

Fixes:
------

 - Fixed static assertion failure on platforms where the `off_t` type is wider
   than corresponding fields of `struct flock` used for file locking (backport).
   Now _libmdbx_ will use `fcntl64(F_GETLK64/F_SETLK64/F_SETLKW64)` if available.
 - Fixed assertion check inside `page_retire_ex()` (backport).

Minors:
-------

 - Fixed `-Wint-to-pointer-cast` warnings while casting to `mdbx_tid_t` (backport).
 - Removed needless `LockFileEx()` inside `mdbx_env_copy()` (backport).

11 files changed, 96 insertions(+), 49 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2022-10-12 21:11:47 +03:00
Леонид Юрьев (Leonid Yuriev)
bdcc345455 mdbx: minor fix assertion check inside page_retire_ex() (backport). 2022-10-12 21:00:35 +03:00
Леонид Юрьев (Leonid Yuriev)
db9e2c6f07 mdbx: using fcntl64(F_GETLK64/F_SETLK64/F_SETLKW64) when available (backport).
This fixes issues (static assertion failure, etc) on platforms where the
`off_t` type is wider than corresponding fields of `struct flock`.
2022-10-12 20:59:05 +03:00
Леонид Юрьев (Leonid Yuriev)
50e9e0e561 mdbx-windows: drop needless LockFileEx() inside mdbx_env_copy() (backport). 2022-10-12 18:32:56 +03:00
Леонид Юрьев (Leonid Yuriev)
8505203080 mdbx: avoid -Wint-to-pointer-cast warnings while casting to mdbx_tid_t (backport). 2022-10-12 18:32:49 +03:00
Леонид Юрьев (Leonid Yuriev)
50ba4bc2f2 mdbx: release v0.11.11 (Тендра-1790)
The stable bugfix release.
It is planned that this will be the last release of the v0.11 branch.

Fixes:
------

 - Fixed an extra check for `MDBX_APPENDDUP` inside `mdbx_cursor_put()` which could result in returning `MDBX_EKEYMISMATCH` for valid cases.
 - Fixed an extra ensure/assertion check of `oldest_reader` inside `mdbx_txn_end()`.
 - Fixed derived C++ builds by removing `MDBX_INTERNAL_FUNC` for `mdbx_w2mb()` and `mdbx_mb2w()`.

10 files changed, 38 insertions(+), 21 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2022-09-11 14:33:56 +03:00
Леонид Юрьев (Leonid Yuriev)
cf6576984d mdbx: update ChangeLog. 2022-09-11 13:08:21 +03:00
Леонид Юрьев (Leonid Yuriev)
4f3e1a60f1 mdbx: remove MDBX_INTERNAL_FUNC from mdbx_w2mb() and mdbx_mb2w() for C++ API. 2022-09-11 13:04:49 +03:00
Леонид Юрьев (Leonid Yuriev)
04f60af669 mdbx: minor fix extra ensure/assertion check of oldest_reader inside txn_end(). 2022-09-09 19:56:48 +03:00
Леонид Юрьев (Leonid Yuriev)
c6cd642ff1 mdbx: fix extra check for MDBX_APPENDDUP. 2022-09-09 19:48:24 +03:00
38 changed files with 772 additions and 361 deletions

View File

@@ -34,7 +34,9 @@
## The Future will (be) Positive. Всё будет хорошо.
##
if(CMAKE_VERSION VERSION_LESS 3.12)
if(CMAKE_VERSION VERSION_LESS 3.8.2)
cmake_minimum_required(VERSION 3.0.2)
elseif(CMAKE_VERSION VERSION_LESS 3.12)
cmake_minimum_required(VERSION 3.8.2)
else()
cmake_minimum_required(VERSION 3.12)
@@ -240,34 +242,42 @@ else()
option(BUILD_FOR_NATIVE_CPU "Generate code for the compiling machine CPU" OFF)
endif()
if(CMAKE_CONFIGURATION_TYPES OR NOT CMAKE_BUILD_TYPE_UPPERCASE STREQUAL "DEBUG")
set(INTERPROCEDURAL_OPTIMIZATION_DEFAULT ON)
else()
set(INTERPROCEDURAL_OPTIMIZATION_DEFAULT OFF)
endif()
if(CMAKE_INTERPROCEDURAL_OPTIMIZATION_AVAILABLE
OR GCC_LTO_AVAILABLE OR MSVC_LTO_AVAILABLE OR
(CLANG_LTO_AVAILABLE AND
((DEFINED MDBX_ENABLE_TESTS AND NOT MDBX_ENABLE_TESTS)
OR NOT CMAKE_CXX_COMPILER_VERSION VERSION_LESS 4.0)))
option(INTERPROCEDURAL_OPTIMIZATION "Enable interprocedural/LTO optimization" ${INTERPROCEDURAL_OPTIMIZATION_DEFAULT})
else()
set(INTERPROCEDURAL_OPTIMIZATION OFF)
OR GCC_LTO_AVAILABLE OR MSVC_LTO_AVAILABLE OR CLANG_LTO_AVAILABLE)
if((CMAKE_CONFIGURATION_TYPES OR NOT CMAKE_BUILD_TYPE_UPPERCASE STREQUAL "DEBUG") AND
((MSVC_LTO_AVAILABLE AND NOT CMAKE_C_COMPILER_VERSION VERSION_LESS 19) OR
(GCC_LTO_AVAILABLE AND NOT CMAKE_C_COMPILER_VERSION VERSION_LESS 7) OR
(CLANG_LTO_AVAILABLE AND NOT CMAKE_C_COMPILER_VERSION VERSION_LESS 5)))
set(INTERPROCEDURAL_OPTIMIZATION_DEFAULT ON)
else()
set(INTERPROCEDURAL_OPTIMIZATION_DEFAULT OFF)
endif()
option(INTERPROCEDURAL_OPTIMIZATION "Enable interprocedural/LTO optimization." ${INTERPROCEDURAL_OPTIMIZATION_DEFAULT})
endif()
if(INTERPROCEDURAL_OPTIMIZATION)
if(GCC_LTO_AVAILABLE)
set(LTO_ENABLED TRUE)
set(CMAKE_AR ${CMAKE_GCC_AR} CACHE PATH "Path to ar program with LTO-plugin" FORCE)
set(CMAKE_C_COMPILER_AR ${CMAKE_AR} CACHE PATH "Path to ar program with LTO-plugin" FORCE)
set(CMAKE_CXX_COMPILER_AR ${CMAKE_AR} CACHE PATH "Path to ar program with LTO-plugin" FORCE)
set(CMAKE_NM ${CMAKE_GCC_NM} CACHE PATH "Path to nm program with LTO-plugin" FORCE)
set(CMAKE_RANLIB ${CMAKE_GCC_RANLIB} CACHE PATH "Path to ranlib program with LTO-plugin" FORCE)
set(CMAKE_C_COMPILER_RANLIB ${CMAKE_RANLIB} CACHE PATH "Path to ranlib program with LTO-plugin" FORCE)
set(CMAKE_CXX_COMPILER_RANLIB ${CMAKE_RANLIB} CACHE PATH "Path to ranlib program with LTO-plugin" FORCE)
message(STATUS "MDBX indulge Link-Time Optimization by GCC")
elseif(CLANG_LTO_AVAILABLE)
set(LTO_ENABLED TRUE)
if(CMAKE_CLANG_LD)
set(CMAKE_LINKER ${CMAKE_CLANG_LD} CACHE PATH "Path to lld or ld program with LTO-plugin" FORCE)
endif()
set(CMAKE_AR ${CMAKE_CLANG_AR} CACHE PATH "Path to ar program with LTO-plugin" FORCE)
set(CMAKE_C_COMPILER_AR ${CMAKE_AR} CACHE PATH "Path to ar program with LTO-plugin" FORCE)
set(CMAKE_CXX_COMPILER_AR ${CMAKE_AR} CACHE PATH "Path to ar program with LTO-plugin" FORCE)
set(CMAKE_NM ${CMAKE_CLANG_NM} CACHE PATH "Path to nm program with LTO-plugin" FORCE)
set(CMAKE_RANLIB ${CMAKE_CLANG_RANLIB} CACHE PATH "Path to ranlib program with LTO-plugin" FORCE)
set(CMAKE_C_COMPILER_RANLIB ${CMAKE_RANLIB} CACHE PATH "Path to ranlib program with LTO-plugin" FORCE)
set(CMAKE_CXX_COMPILER_RANLIB ${CMAKE_RANLIB} CACHE PATH "Path to ranlib program with LTO-plugin" FORCE)
message(STATUS "MDBX indulge Link-Time Optimization by CLANG")
elseif(MSVC_LTO_AVAILABLE)
set(LTO_ENABLED TRUE)
@@ -369,6 +379,8 @@ if(NOT DEFINED MDBX_CXX_STANDARD)
set(MDBX_CXX_STANDARD 14)
elseif(NOT HAS_CXX11 LESS 0)
set(MDBX_CXX_STANDARD 11)
elseif(CXX_FALLBACK_GNU11 OR CXX_FALLBACK_11)
set(MDBX_CXX_STANDARD 11)
else()
set(MDBX_CXX_STANDARD 98)
endif()
@@ -514,7 +526,7 @@ else()
unset(MDBX_LINK_TOOLS_NONSTATIC CACHE)
endif()
if(CMAKE_CXX_COMPILER_LOADED AND MDBX_CXX_STANDARD GREATER_EQUAL 11 AND MDBX_CXX_STANDARD LESS 83)
if(CMAKE_CXX_COMPILER_LOADED AND MDBX_CXX_STANDARD LESS 83 AND NOT MDBX_CXX_STANDARD LESS 11)
if(NOT MDBX_AMALGAMATED_SOURCE)
option(MDBX_ENABLE_TESTS "Build MDBX tests" ${BUILD_TESTING})
endif()
@@ -620,9 +632,13 @@ macro(target_setup_options TARGET)
endmacro()
macro(libmdbx_setup_libs TARGET MODE)
target_link_libraries(${TARGET} ${MODE} Threads::Threads)
if(CMAKE_VERSION VERSION_LESS 3.1)
target_link_libraries(${TARGET} ${MODE} ${CMAKE_THREAD_LIBS_INIT})
else()
target_link_libraries(${TARGET} ${MODE} Threads::Threads)
endif()
if(${CMAKE_SYSTEM_NAME} STREQUAL "Windows")
target_link_libraries(${TARGET} ${MODE} ntdll advapi32)
target_link_libraries(${TARGET} ${MODE} ntdll user32 kernel32 advapi32)
if(MDBX_NTDLL_EXTRA_IMPLIB AND MDBX_WITHOUT_MSVC_CRT)
target_link_libraries(${TARGET} ${MODE} ntdll_extra)
endif()
@@ -743,7 +759,6 @@ if(MDBX_BUILD_SHARED_LIBRARY)
if(CMAKE_VERSION VERSION_LESS 3.12)
install(TARGETS mdbx EXPORT libmdbx
LIBRARY DESTINATION ${MDBX_DLL_INSTALL_DESTINATION} COMPONENT runtime
OBJECTS DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT devel
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT devel
PUBLIC_HEADER DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} COMPONENT devel
INCLUDES DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} COMPONENT devel)
@@ -888,16 +903,16 @@ else()
endif()
if(CMAKE_C_COMPILER_ABI
AND NOT (CMAKE_C_COMPILER_ABI MATCHES ".*${MDBX_BUILD_TARGET}.*" OR MDBX_BUILD_TARGET MATCHES ".*${CMAKE_C_COMPILER_ABI}.*"))
string(APPEND MDBX_BUILD_TARGET "-${CMAKE_C_COMPILER_ABI}")
string(CONCAT MDBX_BUILD_TARGET "${MDBX_BUILD_TARGET}-${CMAKE_C_COMPILER_ABI}")
endif()
if(CMAKE_C_PLATFORM_ID
AND NOT (CMAKE_SYSTEM_NAME
AND (CMAKE_C_PLATFORM_ID MATCHES ".*${CMAKE_SYSTEM_NAME}.*" OR CMAKE_SYSTEM_NAME MATCHES ".*${CMAKE_C_PLATFORM_ID}.*"))
AND NOT (CMAKE_C_PLATFORM_ID MATCHES ".*${CMAKE_C_PLATFORM_ID}.*" OR MDBX_BUILD_TARGET MATCHES ".*${CMAKE_C_PLATFORM_ID}.*"))
string(APPEND MDBX_BUILD_TARGET "-${CMAKE_C_COMPILER_ABI}")
string(CONCAT MDBX_BUILD_TARGET "${MDBX_BUILD_TARGET}-${CMAKE_C_COMPILER_ABI}")
endif()
if(CMAKE_SYSTEM_NAME)
string(APPEND MDBX_BUILD_TARGET "-${CMAKE_SYSTEM_NAME}")
string(CONCAT MDBX_BUILD_TARGET "${MDBX_BUILD_TARGET}-${CMAKE_SYSTEM_NAME}")
endif()
endif()

View File

@@ -1,10 +1,124 @@
ChangeLog
---------
## v0.11.14 (Sergey Kapitsa) at 2023-02-14
The stable bugfix release in memory of [Sergey Kapitsa](https://en.wikipedia.org/wiki/Sergey_Kapitsa) on his 95th birthday.
```
22 files changed, 250 insertions(+), 174 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
```
Fixes:
- backport: Fixed insignificant typo of `||` inside `#if` byte-order condition.
- backport: Fixed `SIGSEGV` or an erroneous call to `free()` in situations where
errors occur when reopening by `mdbx_env_open()` of a previously used
environment.
- backport: Fixed `cursor_put_nochecklen()` internals for case when dupsort'ed named subDb
contains a single key with multiple values (aka duplicates), which are replaced
with a single value by put-operation with the `MDBX_UPSERT+MDBX_ALLDUPS` flags.
In this case, the database becomes completely empty, without any pages.
However exactly this condition was not considered and thus wasn't handled correctly.
See [issue#8](https://gitflic.ru/project/erthink/libmdbx/issue/8) for more information.
- backport: Fixed extra assertion inside `override_meta()`, which could
lead to false-positive failing of the assertion in a debug builds during
DB recovery and auto-rollback.
- backport: Refined the `__cold`/`__hot` macros to avoid the
`error: inlining failed in call to always_inline FOO(...): target specific option mismatch`
issue during build using GCC >10.x for SH4 arch.
Minors:
- backport: Using the https://libmdbx.dqdkfa.ru/dead-github
for resources deleted by the Github' administration.
- backport: Fixed English typos.
- backport: Fixed proto of `__asan_default_options()`.
- backport: Fixed doxygen-description of C++ API, especially of C++20 concepts.
- backport: Refined `const` and `noexcept` for few C++ API methods.
- backport: Fixed copy&paste typo of "Getting started".
- backport: Update MithrilDB status.
- backport: Resolve false-posirive `used uninitialized` warning from GCC >10.x
while build for SH4 arch.
## v0.11.13 (Swashplate) at 2022-11-10
The stable bugfix release in memory of [Boris Yuryev](https://ru.wikipedia.org/wiki/Юрьев,_Борис_Николаевич) on his 133rd birthday.
```
30 files changed, 405 insertions(+), 136 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
```
Fixes:
- Fixed builds with older libc versions after using `fcntl64()` (backport).
- Fixed builds with older `stdatomic.h` versions,
where the `ATOMIC_*_LOCK_FREE` macros mistakenly redefined using functions (backport).
- Added workaround for `mremap()` defect to avoid assertion failure (backport).
- Workaround for `encryptfs` bug(s) in the `copy_file_range` implementation (backport).
- Fixed unexpected `MDBX_BUSY` from `mdbx_env_set_option()`, `mdbx_env_set_syncbytes()`
and `mdbx_env_set_syncperiod()` (backport).
- CMake requirements lowered to version 3.0.2 (backport).
Minors:
- Minor clarification output of `--help` for `mdbx_test` (backport).
- Added admonition of insecure for RISC-V (backport).
- Stochastic scripts and CMake files synchronized with the `devel` branch.
- Use `--dont-check-ram-size` for small-tests make-targets (backport).
## v0.11.12 (Эребуни) at 2022-10-12
The stable bugfix release.
```
11 files changed, 96 insertions(+), 49 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
```
Fixes:
- Fixed static assertion failure on platforms where the `off_t` type is wider
than corresponding fields of `struct flock` used for file locking (backport).
Now _libmdbx_ will use `fcntl64(F_GETLK64/F_SETLK64/F_SETLKW64)` if available.
- Fixed assertion check inside `page_retire_ex()` (backport).
Minors:
- Fixed `-Wint-to-pointer-cast` warnings while casting to `mdbx_tid_t` (backport).
- Removed needless `LockFileEx()` inside `mdbx_env_copy()` (backport).
## v0.11.11 (Тендра-1790) at 2022-09-11
The stable bugfix release.
```
10 files changed, 38 insertions(+), 21 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
```
Fixes:
- Fixed an extra check for `MDBX_APPENDDUP` inside `mdbx_cursor_put()` which could result in returning `MDBX_EKEYMISMATCH` for valid cases.
- Fixed an extra ensure/assertion check of `oldest_reader` inside `mdbx_txn_end()`.
- Fixed derived C++ builds by removing `MDBX_INTERNAL_FUNC` for `mdbx_w2mb()` and `mdbx_mb2w()`.
-------------------------------------------------------------------------------
## v0.11.10 (the TriColor) at 2022-08-22
The stable bugfix release.
It is planned that this will be the last release of the v0.11 branch.
New:
@@ -185,7 +299,7 @@ Minors:
The stable release with the complete workaround for an incoherence flaw of Linux unified page/buffer cache.
Nonetheless the cause for this trouble may be an issue of Intel CPU cache/MESI.
See [issue#269](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/269) for more information.
See [issue#269](https://libmdbx.dqdkfa.ru/dead-github/issues/269) for more information.
Acknowledgements:
@@ -194,8 +308,8 @@ Acknowledgements:
Fixes:
- [Added complete workaround](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/269) for an incoherence flaw of Linux unified page/buffer cache.
- [Fixed](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/272) cursor reusing for read-only transactions.
- [Added complete workaround](https://libmdbx.dqdkfa.ru/dead-github/issues/269) for an incoherence flaw of Linux unified page/buffer cache.
- [Fixed](https://libmdbx.dqdkfa.ru/dead-github/issues/272) cursor reusing for read-only transactions.
- Fixed copy&paste typo inside `mdbx::cursor::find_multivalue()`.
Minors:
@@ -210,7 +324,7 @@ Minors:
## v0.11.5 at 2022-02-23
The release with the temporary hotfix for a flaw of Linux unified page/buffer cache.
See [issue#269](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/269) for more information.
See [issue#269](https://libmdbx.dqdkfa.ru/dead-github/issues/269) for more information.
Acknowledgements:
@@ -220,10 +334,10 @@ Acknowledgements:
Fixes:
- [Added hotfix](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/269) for a flaw of Linux unified page/buffer cache.
- [Fixed/Reworked](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/pull/270) move-assignment operators for "managed" classes of C++ API.
- [Added hotfix](https://libmdbx.dqdkfa.ru/dead-github/issues/269) for a flaw of Linux unified page/buffer cache.
- [Fixed/Reworked](https://libmdbx.dqdkfa.ru/dead-github/pull/270) move-assignment operators for "managed" classes of C++ API.
- Fixed potential `SIGSEGV` while open DB with overrided non-default page size.
- [Made](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/267) `mdbx_env_open()` idempotence in failure cases.
- [Made](https://libmdbx.dqdkfa.ru/dead-github/issues/267) `mdbx_env_open()` idempotence in failure cases.
- Refined/Fixed pages reservation inside `mdbx_update_gc()` to avoid non-reclamation in a rare cases.
- Fixed typo in a retained space calculation for the hsr-callback.
@@ -256,15 +370,15 @@ New features, extensions and improvements:
Fixes:
- Fixed handling `MDBX_opt_rp_augment_limit` for GC's records from huge transactions (Erigon/Akula/Ethereum).
- [Fixed](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/258) build on Android (avoid including `sys/sem.h`).
- [Fixed](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/pull/261) missing copy assignment operator for `mdbx::move_result`.
- [Fixed](https://libmdbx.dqdkfa.ru/dead-github/issues/258) build on Android (avoid including `sys/sem.h`).
- [Fixed](https://libmdbx.dqdkfa.ru/dead-github/pull/261) missing copy assignment operator for `mdbx::move_result`.
- Fixed missing `&` for `std::ostream &operator<<()` overloads.
- Fixed unexpected `EXDEV` (Cross-device link) error from `mdbx_env_copy()`.
- Fixed base64 encoding/decoding bugs in auxillary C++ API.
- Fixed overflow of `pgno_t` during checking PNL on 64-bit platforms.
- [Fixed](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/260) excessive PNL checking after sort for spilling.
- [Fixed](https://libmdbx.dqdkfa.ru/dead-github/issues/260) excessive PNL checking after sort for spilling.
- Reworked checking `MAX_PAGENO` and DB upper-size geometry limit.
- [Fixed](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/265) build for some combinations of versions of MSVC and Windows SDK.
- [Fixed](https://libmdbx.dqdkfa.ru/dead-github/issues/265) build for some combinations of versions of MSVC and Windows SDK.
Minors:
@@ -291,10 +405,10 @@ Acknowledgements:
New features, extensions and improvements:
- [Added](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/236) `mdbx_cursor_get_batch()`.
- [Added](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/250) `MDBX_SET_UPPERBOUND`.
- [Added](https://libmdbx.dqdkfa.ru/dead-github/issues/236) `mdbx_cursor_get_batch()`.
- [Added](https://libmdbx.dqdkfa.ru/dead-github/issues/250) `MDBX_SET_UPPERBOUND`.
- C++ API is finalized now.
- The GC update stage has been [significantly speeded](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/254) when fixing huge Erigon's transactions (Ethereum ecosystem).
- The GC update stage has been [significantly speeded](https://libmdbx.dqdkfa.ru/dead-github/issues/254) when fixing huge Erigon's transactions (Ethereum ecosystem).
Fixes:
@@ -305,12 +419,12 @@ Minors:
- Fixed returning `MDBX_RESULT_TRUE` (unexpected -1) from `mdbx_env_set_option()`.
- Added `mdbx_env_get_syncbytes()` and `mdbx_env_get_syncperiod()`.
- [Clarified](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/pull/249) description of `MDBX_INTEGERKEY`.
- [Clarified](https://libmdbx.dqdkfa.ru/dead-github/pull/249) description of `MDBX_INTEGERKEY`.
- Reworked/simplified `mdbx_env_sync_internal()`.
- [Fixed](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/248) extra assertion inside `mdbx_cursor_put()` for `MDBX_DUPFIXED` cases.
- [Fixed](https://libmdbx.dqdkfa.ru/dead-github/issues/248) extra assertion inside `mdbx_cursor_put()` for `MDBX_DUPFIXED` cases.
- Avoiding extra looping inside `mdbx_env_info_ex()`.
- Explicitly enabled core dumps from stochastic tests scripts on Linux.
- [Fixed](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/253) `mdbx_override_meta()` to avoid false-positive assertions.
- [Fixed](https://libmdbx.dqdkfa.ru/dead-github/issues/253) `mdbx_override_meta()` to avoid false-positive assertions.
- For compatibility reverted returning `MDBX_ENODATA`for some cases.
@@ -326,10 +440,10 @@ Acknowledgements:
Fixes:
- [Fixed compilation](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/pull/239) with `devtoolset-9` on CentOS/RHEL 7.
- [Fixed unexpected `MDBX_PROBLEM` error](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/242) because of update an obsolete meta-page.
- [Fixed returning `MDBX_NOTFOUND` error](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/243) in case an inexact value found for `MDBX_GET_BOTH` operation.
- [Fixed compilation](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/245) without kernel/libc-devel headers.
- [Fixed compilation](https://libmdbx.dqdkfa.ru/dead-github/pull/239) with `devtoolset-9` on CentOS/RHEL 7.
- [Fixed unexpected `MDBX_PROBLEM` error](https://libmdbx.dqdkfa.ru/dead-github/issues/242) because of update an obsolete meta-page.
- [Fixed returning `MDBX_NOTFOUND` error](https://libmdbx.dqdkfa.ru/dead-github/issues/243) in case an inexact value found for `MDBX_GET_BOTH` operation.
- [Fixed compilation](https://libmdbx.dqdkfa.ru/dead-github/issues/245) without kernel/libc-devel headers.
Minors:
@@ -346,7 +460,7 @@ Minors:
The database format signature has been changed to prevent
forward-interoperability with an previous releases, which may lead to a
[false positive diagnosis of database corruption](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/238)
[false positive diagnosis of database corruption](https://libmdbx.dqdkfa.ru/dead-github/issues/238)
due to flaws of an old library versions.
This change is mostly invisible:
@@ -398,7 +512,7 @@ Acknowledgements:
Fixes:
- Fixed possibility of looping update GC during transaction commit (no public issue since the problem was discovered inside [Positive Technologies](https://www.ptsecurity.ru)).
- Fixed `#pragma pack` to avoid provoking some compilers to generate code with [unaligned access](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/235).
- Fixed `#pragma pack` to avoid provoking some compilers to generate code with [unaligned access](https://libmdbx.dqdkfa.ru/dead-github/issues/235).
- Fixed `noexcept` for potentially throwing `txn::put()` of C++ API.
Minors:
@@ -424,7 +538,7 @@ Extensions and improvements:
Fixes:
- Always setup `madvise` while opening DB (fixes https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/231).
- Always setup `madvise` while opening DB (fixes https://libmdbx.dqdkfa.ru/dead-github/issues/231).
- Fixed checking legacy `P_DIRTY` flag (`0x10`) for nested/sub-pages.
Minors:
@@ -445,11 +559,11 @@ Acknowledgements:
- [Lionel Debroux](https://github.com/debrouxl) for fuzzing tests and reporting bugs.
- [Sergey Fedotov](https://github.com/SergeyFromHell/) for [`node-mdbx` NodeJS bindings](https://www.npmjs.com/package/node-mdbx).
- [Kris Zyp](https://github.com/kriszyp) for [`lmdbx-store` NodeJS bindings](https://github.com/kriszyp/lmdbx-store).
- [Noel Kuntze](https://github.com/Thermi) for [draft Python bindings](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/commits/python-bindings).
- [Noel Kuntze](https://github.com/Thermi) for [draft Python bindings](https://libmdbx.dqdkfa.ru/dead-github/commits/python-bindings).
New features, extensions and improvements:
- [Allow to predefine/override `MDBX_BUILD_TIMESTAMP` for builds reproducibility](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/201).
- [Allow to predefine/override `MDBX_BUILD_TIMESTAMP` for builds reproducibility](https://libmdbx.dqdkfa.ru/dead-github/issues/201).
- Added options support for `long-stochastic` script.
- Avoided `MDBX_TXN_FULL` error for large transactions when possible.
- The `MDBX_READERS_LIMIT` increased to `32767`.
@@ -457,7 +571,7 @@ New features, extensions and improvements:
- Minimized the size of poisoned/unpoisoned regions to avoid Valgrind/ASAN stuck.
- Added more workarounds for QEMU for testing builds for 32-bit platforms, Alpha and Sparc architectures.
- `mdbx_chk` now skips iteration & checking of DB' records if corresponding page-tree is corrupted (to avoid `SIGSEGV`, ASAN failures, etc).
- Added more checks for [rare/fuzzing corruption cases](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/217).
- Added more checks for [rare/fuzzing corruption cases](https://libmdbx.dqdkfa.ru/dead-github/issues/217).
Backward compatibility break:
@@ -469,18 +583,18 @@ Backward compatibility break:
Fixes:
- Fixed excess meta-pages checks in case `mdbx_chk` is called to check the DB for a specific meta page and thus could prevent switching to the selected meta page, even if the check passed without errors.
- Fixed [recursive use of SRW-lock on Windows cause by `MDBX_NOTLS` option](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/203).
- Fixed [log a warning during a new DB creation](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/205).
- Fixed [false-negative `mdbx_cursor_eof()` result](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/207).
- Fixed [`make install` with non-GNU `install` utility (OSX, BSD)](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/208).
- Fixed [installation by `CMake` in special cases by complete use `GNUInstallDirs`'s variables](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/209).
- Fixed [C++ Buffer issue with `std::string` and alignment](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/191).
- Fixed [recursive use of SRW-lock on Windows cause by `MDBX_NOTLS` option](https://libmdbx.dqdkfa.ru/dead-github/issues/203).
- Fixed [log a warning during a new DB creation](https://libmdbx.dqdkfa.ru/dead-github/issues/205).
- Fixed [false-negative `mdbx_cursor_eof()` result](https://libmdbx.dqdkfa.ru/dead-github/issues/207).
- Fixed [`make install` with non-GNU `install` utility (OSX, BSD)](https://libmdbx.dqdkfa.ru/dead-github/issues/208).
- Fixed [installation by `CMake` in special cases by complete use `GNUInstallDirs`'s variables](https://libmdbx.dqdkfa.ru/dead-github/issues/209).
- Fixed [C++ Buffer issue with `std::string` and alignment](https://libmdbx.dqdkfa.ru/dead-github/issues/191).
- Fixed `safe64_reset()` for platforms without atomic 64-bit compare-and-swap.
- Fixed hang/shutdown on big-endian platforms without `__cxa_thread_atexit()`.
- Fixed [using bad meta-pages if DB was partially/recoverable corrupted](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/217).
- Fixed [using bad meta-pages if DB was partially/recoverable corrupted](https://libmdbx.dqdkfa.ru/dead-github/issues/217).
- Fixed extra `noexcept` for `buffer::&assign_reference()`.
- Fixed `bootid` generation on Windows for case of change system' time.
- Fixed [test framework keygen-related issue](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/127).
- Fixed [test framework keygen-related issue](https://libmdbx.dqdkfa.ru/dead-github/issues/127).
## v0.10.1 at 2021-06-01
@@ -501,10 +615,10 @@ New features:
Fixes:
- Fixed minor "foo not used" warnings from modern C++ compilers when building the C++ part of the library.
- Fixed confusing/messy errors when build library from unfit github's archives (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/197).
- Fixed confusing/messy errors when build library from unfit github's archives (https://libmdbx.dqdkfa.ru/dead-github/issues/197).
- Fixed `#elsif` typo.
- Fixed rare unexpected `MDBX_PROBLEM` error during altering data in huge transactions due to wrong spilling/oust of dirty pages (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/195).
- Re-Fixed WSL1/WSL2 detection with distinguishing (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/97).
- Fixed rare unexpected `MDBX_PROBLEM` error during altering data in huge transactions due to wrong spilling/oust of dirty pages (https://libmdbx.dqdkfa.ru/dead-github/issues/195).
- Re-Fixed WSL1/WSL2 detection with distinguishing (https://libmdbx.dqdkfa.ru/dead-github/issues/97).
## v0.10.0 at 2021-05-09
@@ -527,7 +641,7 @@ New features:
and conjointly with the `MDBX_ENV_CHECKPID=0` and `MDBX_TXN_CHECKOWNER=0` options can yield
up to 30% more performance compared to LMDB.
- Using float point (exponential quantized) representation for internal 16-bit values
of grow step and shrink threshold when huge ones (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/166).
of grow step and shrink threshold when huge ones (https://libmdbx.dqdkfa.ru/dead-github/issues/166).
To minimize the impact on compatibility, only the odd values inside the upper half
of the range (i.e. 32769..65533) are used for the new representation.
- Added the `mdbx_drop` similar to LMDB command-line tool to purge or delete (sub)database(s).
@@ -536,7 +650,7 @@ New features:
- The internal node sizes were refined, resulting in a reduction in large/overflow pages in some use cases
and a slight increase in limits for a keys size to ≈½ of page size.
- Added to `mdbx_chk` output number of keys/items on pages.
- Added explicit `install-strip` and `install-no-strip` targets to the `Makefile` (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/pull/180).
- Added explicit `install-strip` and `install-no-strip` targets to the `Makefile` (https://libmdbx.dqdkfa.ru/dead-github/pull/180).
- Major rework page splitting (af9b7b560505684249b76730997f9e00614b8113) for
- An "auto-appending" feature upon insertion for both ascending and
descending key sequences. As a result, the optimality of page filling
@@ -544,7 +658,7 @@ New features:
inserting ordered sequences of keys,
- A "splitting at middle" to make page tree more balanced on average.
- Added `mdbx_get_sysraminfo()` to the API.
- Added guessing a reasonable maximum DB size for the default upper limit of geometry (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/183).
- Added guessing a reasonable maximum DB size for the default upper limit of geometry (https://libmdbx.dqdkfa.ru/dead-github/issues/183).
- Major rework internal labeling of a dirty pages (958fd5b9479f52f2124ab7e83c6b18b04b0e7dda) for
a "transparent spilling" feature with the gist to make a dirty pages
be ready to spilling (writing to a disk) without further altering ones.
@@ -560,7 +674,7 @@ New features:
- Support `make help` to list available make targets.
- Silently `make`'s build by default.
- Preliminary [Python bindings](https://github.com/Thermi/libmdbx/tree/python-bindings) is available now
by [Noel Kuntze](https://github.com/Thermi) (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/147).
by [Noel Kuntze](https://github.com/Thermi) (https://libmdbx.dqdkfa.ru/dead-github/issues/147).
Backward compatibility break:
@@ -575,22 +689,22 @@ Backward compatibility break:
Fixes:
- Fixed performance regression due non-optimal C11 atomics usage (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/160).
- Fixed "reincarnation" of subDB after it deletion (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/168).
- Fixed performance regression due non-optimal C11 atomics usage (https://libmdbx.dqdkfa.ru/dead-github/issues/160).
- Fixed "reincarnation" of subDB after it deletion (https://libmdbx.dqdkfa.ru/dead-github/issues/168).
- Fixed (disallowing) implicit subDB deletion via operations on `@MAIN`'s DBI-handle.
- Fixed a crash of `mdbx_env_info_ex()` in case of a call for a non-open environment (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/171).
- Fixed the selecting/adjustment values inside `mdbx_env_set_geometry()` for implicit out-of-range cases (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/170).
- Fixed `mdbx_env_set_option()` for set initial and limit size of dirty page list ((https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/179).
- Fixed an unreasonably huge default upper limit for DB geometry (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/183).
- Fixed a crash of `mdbx_env_info_ex()` in case of a call for a non-open environment (https://libmdbx.dqdkfa.ru/dead-github/issues/171).
- Fixed the selecting/adjustment values inside `mdbx_env_set_geometry()` for implicit out-of-range cases (https://libmdbx.dqdkfa.ru/dead-github/issues/170).
- Fixed `mdbx_env_set_option()` for set initial and limit size of dirty page list ((https://libmdbx.dqdkfa.ru/dead-github/issues/179).
- Fixed an unreasonably huge default upper limit for DB geometry (https://libmdbx.dqdkfa.ru/dead-github/issues/183).
- Fixed `constexpr` specifier for the `slice::invalid()`.
- Fixed (no)readahead auto-handling (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/164).
- Fixed (no)readahead auto-handling (https://libmdbx.dqdkfa.ru/dead-github/issues/164).
- Fixed non-alloy build for Windows.
- Switched to using Heap-functions instead of LocalAlloc/LocalFree on Windows.
- Fixed `mdbx_env_stat_ex()` to returning statistics of the whole environment instead of MainDB only (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/190).
- Fixed `mdbx_env_stat_ex()` to returning statistics of the whole environment instead of MainDB only (https://libmdbx.dqdkfa.ru/dead-github/issues/190).
- Fixed building by GCC 4.8.5 (added workaround for a preprocessor's bug).
- Fixed building C++ part for iOS <= 13.0 (unavailability of `std::filesystem::path`).
- Fixed building for Windows target versions prior to Windows Vista (`WIN32_WINNT < 0x0600`).
- Fixed building by MinGW for Windows (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/155).
- Fixed building by MinGW for Windows (https://libmdbx.dqdkfa.ru/dead-github/issues/155).
-------------------------------------------------------------------------------
@@ -613,7 +727,7 @@ Removed options and features:
New features:
- Package for FreeBSD is available now by Mahlon E. Smith.
- New API functions to get/set various options (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/128):
- New API functions to get/set various options (https://libmdbx.dqdkfa.ru/dead-github/issues/128):
- the maximum number of named databases for the environment;
- the maximum number of threads/reader slots;
- threshold (since the last unsteady commit) to force flush the data buffers to disk;
@@ -626,7 +740,7 @@ New features:
- maximal part of the dirty pages may be spilled when necessary;
- minimal part of the dirty pages should be spilled when necessary;
- how much of the parent transaction dirty pages will be spilled while start each child transaction;
- Unlimited/Dynamic size of retired and dirty page lists (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/123).
- Unlimited/Dynamic size of retired and dirty page lists (https://libmdbx.dqdkfa.ru/dead-github/issues/123).
- Added `-p` option (purge subDB before loading) to `mdbx_load` tool.
- Reworked spilling of large transaction and committing of nested transactions:
- page spilling code reworked to avoid the flaws and bugs inherited from LMDB;
@@ -636,22 +750,22 @@ New features:
- Added `MDBX_ENABLE_REFUND` and `MDBX_PNL_ASCENDING` internal/advanced build options.
- Added `mdbx_default_pagesize()` function.
- Better support architectures with a weak/relaxed memory consistency model (ARM, AARCH64, PPC, MIPS, RISC-V, etc) by means [C11 atomics](https://en.cppreference.com/w/c/atomic).
- Speed up page number lists and dirty page lists (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/132).
- Speed up page number lists and dirty page lists (https://libmdbx.dqdkfa.ru/dead-github/issues/132).
- Added `LIBMDBX_NO_EXPORTS_LEGACY_API` build option.
Fixes:
- Fixed missing cleanup (null assigned) in the C++ commit/abort (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/pull/143).
- Fixed missing cleanup (null assigned) in the C++ commit/abort (https://libmdbx.dqdkfa.ru/dead-github/pull/143).
- Fixed `mdbx_realloc()` for case of nullptr and `MDBX_WITHOUT_MSVC_CRT=ON` for Windows.
- Fixed the possibility to use invalid and renewed (closed & re-opened, dropped & re-created) DBI-handles (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/146).
- Fixed 4-byte aligned access to 64-bit integers, including access to the `bootid` meta-page's field (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/153).
- Fixed the possibility to use invalid and renewed (closed & re-opened, dropped & re-created) DBI-handles (https://libmdbx.dqdkfa.ru/dead-github/issues/146).
- Fixed 4-byte aligned access to 64-bit integers, including access to the `bootid` meta-page's field (https://libmdbx.dqdkfa.ru/dead-github/issues/153).
- Fixed minor/potential memory leak during page flushing and unspilling.
- Fixed handling states of cursors's and subDBs's for nested transactions.
- Fixed page leak in extra rare case the list of retired pages changed during update GC on transaction commit.
- Fixed assertions to avoid false-positive UB detection by CLANG/LLVM (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/153).
- Fixed `MDBX_TXN_FULL` and regressive `MDBX_KEYEXIST` during large transaction commit with `MDBX_LIFORECLAIM` (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/123).
- Fixed assertions to avoid false-positive UB detection by CLANG/LLVM (https://libmdbx.dqdkfa.ru/dead-github/issues/153).
- Fixed `MDBX_TXN_FULL` and regressive `MDBX_KEYEXIST` during large transaction commit with `MDBX_LIFORECLAIM` (https://libmdbx.dqdkfa.ru/dead-github/issues/123).
- Fixed auto-recovery (`weak->steady` with the same boot-id) when Database size at last weak checkpoint is large than at last steady checkpoint.
- Fixed operation on systems with unusual small/large page size, including PowerPC (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/157).
- Fixed operation on systems with unusual small/large page size, including PowerPC (https://libmdbx.dqdkfa.ru/dead-github/issues/157).
## v0.9.2 at 2020-11-27
@@ -689,11 +803,11 @@ Fixes:
- Fixed copy&paste typos.
- Fixed minor false-positive GCC warning.
- Added workaround for broken `DEFINE_ENUM_FLAG_OPERATORS` from Windows SDK.
- Fixed cursor state after multimap/dupsort repeated deletes (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/121).
- Fixed cursor state after multimap/dupsort repeated deletes (https://libmdbx.dqdkfa.ru/dead-github/issues/121).
- Added `SIGPIPE` suppression for internal thread during `mdbx_env_copy()`.
- Fixed extra-rare `MDBX_KEY_EXIST` error during `mdbx_commit()` (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/131).
- Fixed spilled pages checking (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/126).
- Fixed `mdbx_load` for 'plain text' and without `-s name` cases (https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/136).
- Fixed extra-rare `MDBX_KEY_EXIST` error during `mdbx_commit()` (https://libmdbx.dqdkfa.ru/dead-github/issues/131).
- Fixed spilled pages checking (https://libmdbx.dqdkfa.ru/dead-github/issues/126).
- Fixed `mdbx_load` for 'plain text' and without `-s name` cases (https://libmdbx.dqdkfa.ru/dead-github/issues/136).
- Fixed save/restore/commit of cursors for nested transactions.
- Fixed cursors state in rare/special cases (move next beyond end-of-data, after deletion and so on).
- Added workaround for MSVC 19.28 (Visual Studio 16.8) (but may still hang during compilation).
@@ -746,7 +860,7 @@ Fixes:
- Fix a lot of typos & spelling (Thanks to Josh Soref for PR).
- Fix `getopt()` messages for Windows (Thanks to Andrey Sporaw for reporting).
- Fix MSVC compiler version requirements (Thanks to Andrey Sporaw for reporting).
- Workarounds for QEMU's bugs to run tests for cross-builded library under QEMU.
- Workarounds for QEMU's bugs to run tests for cross-built[A library under QEMU.
- Now C++ compiler optional for building by CMake.
@@ -815,7 +929,7 @@ Deprecated functions and flags:
- Avoid using `pwritev()` for single-writes (up to 10% speedup for some kernels & scenarios).
- Avoiding `MDBX_UTTERLY_NOSYNC` as result of flags merge.
- Add `mdbx_dbi_dupsort_depthmask()` function.
- Add `MDBX_CP_FORCE_RESIZEABLE` option.
- Add `MDBX_CP_FORCE_RESIZABLE` option.
- Add deprecated `MDBX_MAP_RESIZED` for compatibility.
- Add `MDBX_BUILD_TOOLS` option (default `ON`).
- Refine `mdbx_dbi_open_ex()` to safe concurrently opening the same handle from different threads.

View File

@@ -6,7 +6,7 @@
#
################################################################################
#
# Basic internal definitios. For a customizable variables and options see below.
# Basic internal definitions. For a customizable variables and options see below.
#
$(info // The GNU Make $(MAKE_VERSION))
SHELL := $(shell env bash -c 'echo $$BASH')
@@ -414,7 +414,7 @@ smoke-fault: build-test
test: build-test
@echo ' RUNNING `test/long_stochastic.sh --loops 2`...'
$(QUIET)test/long_stochastic.sh --loops 2 --db-upto-mb 256 --skip-make >$(TEST_LOG)
$(QUIET)test/long_stochastic.sh --dont-check-ram-size --loops 2 --db-upto-mb 256 --skip-make >$(TEST_LOG)
long-test: build-test
@echo ' RUNNING `test/long_stochastic.sh --loops 42`...'
@@ -422,7 +422,7 @@ long-test: build-test
test-singleprocess: build-test
@echo ' RUNNING `test/long_stochastic.sh --single --loops 2`...'
$(QUIET)test/long_stochastic.sh --single --loops 2 --db-upto-mb 256 --skip-make >$(TEST_LOG)
$(QUIET)test/long_stochastic.sh --dont-check-ram-size --single --loops 2 --db-upto-mb 256 --skip-make >$(TEST_LOG)
test-valgrind: CFLAGS_EXTRA=-Ofast -DMDBX_USE_VALGRIND
test-valgrind: build-test

View File

@@ -81,19 +81,48 @@ Historically, _libmdbx_ is a deeply revised and extended descendant of the amazi
[Lightning Memory-Mapped Database](https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database).
_libmdbx_ inherits all benefits from _LMDB_, but resolves some issues and adds [a set of improvements](#improvements-beyond-lmdb).
### MithrilDB and Future
<!-- section-begin mithril -->
The next version is under active non-public development from scratch and will be
The next version is under non-public development from scratch and will be
released as **MithrilDB** and `libmithrildb` for libraries & packages.
Admittedly mythical [Mithril](https://en.wikipedia.org/wiki/Mithril) is
resembling silver but being stronger and lighter than steel. Therefore
_MithrilDB_ is a rightly relevant name.
> _MithrilDB_ will be radically different from _libmdbx_ by the new
> database format and API based on C++17, as well as the [Apache 2.0
> License](https://www.apache.org/licenses/LICENSE-2.0). The goal of this
> revolution is to provide a clearer and robust API, add more features and
> new valuable properties of the database.
_MithrilDB_ is radically different from _libmdbx_ by the new database
format and API based on C++20. The goal of this revolution is to provide
a clearer and robust API, add more features and new valuable properties
of the database. All fundamental architectural problems of libmdbx/LMDB
have been solved there, but now the active development has been
suspended for top-three reasons:
1. For now _libmdbx_ «mostly» enough for all [our products](https://www.ptsecurity.com/ww-en/products/),
and Im busy in development of replication for scalability.
2. Waiting for fresh [Elbrus CPU](https://wiki.elbrus.ru/) of [e2k architecture](https://en.wikipedia.org/wiki/Elbrus_2000),
especially with hardware acceleration of [Streebog](https://en.wikipedia.org/wiki/Streebog) and
[Kuznyechik](https://en.wikipedia.org/wiki/Kuznyechik), which are required for Merkle tree, etc.
3. The expectation of needs and opportunities due to the wide use of NVDIMM (aka persistent memory),
modern NVMe and [Ангара](https://ru.wikipedia.org/wiki/Ангара_(интерконнект)).
However, _MithrilDB_ will not be available for countries unfriendly to
Russia (i.e. acceded the sanctions, devil adepts and/or NATO). But it is
not yet known whether such restriction will be implemented only through
a license and support, either the source code will not be open at all.
Basically we are not inclined to allow our work to contribute to the
profit that goes to weapons that kill our relatives and friends.
NO OPTIONS.
Nonetheless, I try not to make any promises regarding _MithrilDB_ until release.
Contrary to _MithrilDB_, _libmdbx_ will forever free and open source.
Moreover with high-quality support whenever possible. Tu deviens
responsable pour toujours de ce que tu as apprivois. So we will continue
to comply with the original open license and the principles of
constructive cooperation, in spite of outright Github sabotage and
sanctions. I will also try to keep (not drop) Windows support, despite
it is an unused obsolete technology for us.
<!-- section-end -->
@@ -435,7 +464,7 @@ unexpected or broken down.
### Testing
The amalgamated source code does not contain any tests for or several reasons.
Please read [the explanation](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/214#issuecomment-870717981) and don't ask to alter this.
Please read [the explanation](https://libmdbx.dqdkfa.ru/dead-github/issues/214#issuecomment-870717981) and don't ask to alter this.
So for testing _libmdbx_ itself you need a full source code, i.e. the clone of a git repository, there is no option.
The full source code of _libmdbx_ has a [`test` subdirectory](https://gitflic.ru/project/erthink/libmdbx/tree/master/test) with minimalistic test "framework".
@@ -618,7 +647,7 @@ Bindings
| Rust | [libmdbx-rs](https://github.com/vorot93/libmdbx-rs) | [Artem Vorotnikov](https://github.com/vorot93) |
| Rust | [mdbx](https://crates.io/crates/mdbx) | [gcxfd](https://github.com/gcxfd) |
| Java | [mdbxjni](https://github.com/castortech/mdbxjni) | [Castor Technologies](https://castortech.com/) |
| Python (draft) | [python-bindings](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/commits/python-bindings) branch | [Noel Kuntze](https://github.com/Thermi)
| Python (draft) | [python-bindings](https://libmdbx.dqdkfa.ru/dead-github/commits/python-bindings) branch | [Noel Kuntze](https://github.com/Thermi)
| .NET (obsolete) | [mdbx.NET](https://github.com/wangjia184/mdbx.NET) | [Jerry Wang](https://github.com/wangjia184) |
<!-- section-end -->

22
TODO.md
View File

@@ -11,15 +11,15 @@ For the same reason ~~Github~~ is blacklisted forever.
So currently most of the links are broken due to noted malicious ~~Github~~ sabotage.
- [Engage an "overlapped I/O" on Windows](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/224).
- [Simple careful mode for working with corrupted DB](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/223).
- [Move most of `mdbx_chk` functional to the library API](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/204).
- [Replace SRW-lock on Windows to allow shrink DB with `MDBX_NOTLS` option](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/210).
- [More flexible support of asynchronous runtime/framework(s)](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/200).
- [Migration guide from LMDB to MDBX](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/199).
- [Get rid of dirty-pages list in MDBX_WRITEMAP mode](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/193).
- [Large/Overflow pages accounting for dirty-room](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/192).
- [Support for RAW devices](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/124).
- [Support MessagePack for Keys & Values](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/115).
- [Engage new terminology](https://web.archive.org/web/20220414235959/https://github.com/erthink/libmdbx/issues/137).
- [Engage an "overlapped I/O" on Windows](https://libmdbx.dqdkfa.ru/dead-github/issues/224).
- [Simple careful mode for working with corrupted DB](https://libmdbx.dqdkfa.ru/dead-github/issues/223).
- [Move most of `mdbx_chk` functional to the library API](https://libmdbx.dqdkfa.ru/dead-github/issues/204).
- [Replace SRW-lock on Windows to allow shrink DB with `MDBX_NOTLS` option](https://libmdbx.dqdkfa.ru/dead-github/issues/210).
- [More flexible support of asynchronous runtime/framework(s)](https://libmdbx.dqdkfa.ru/dead-github/issues/200).
- [Migration guide from LMDB to MDBX](https://libmdbx.dqdkfa.ru/dead-github/issues/199).
- [Get rid of dirty-pages list in MDBX_WRITEMAP mode](https://libmdbx.dqdkfa.ru/dead-github/issues/193).
- [Large/Overflow pages accounting for dirty-room](https://libmdbx.dqdkfa.ru/dead-github/issues/192).
- [Support for RAW devices](https://libmdbx.dqdkfa.ru/dead-github/issues/124).
- [Support MessagePack for Keys & Values](https://libmdbx.dqdkfa.ru/dead-github/issues/115).
- [Engage new terminology](https://libmdbx.dqdkfa.ru/dead-github/issues/137).
- Packages for [Astra Linux](https://astralinux.ru/), [ALT Linux](https://www.altlinux.org/), [ROSA Linux](https://www.rosalinux.ru/), etc.

View File

@@ -13,7 +13,9 @@
## limitations under the License.
##
if(CMAKE_VERSION VERSION_LESS 3.12)
if(CMAKE_VERSION VERSION_LESS 3.8.2)
cmake_minimum_required(VERSION 3.0.2)
elseif(CMAKE_VERSION VERSION_LESS 3.12)
cmake_minimum_required(VERSION 3.8.2)
else()
cmake_minimum_required(VERSION 3.12)
@@ -203,9 +205,38 @@ endif()
if(NOT CMAKE_SYSTEM_ARCH)
if(CMAKE_${CMAKE_PRIMARY_LANG}_COMPILER_ARCHITECTURE_ID)
set(CMAKE_SYSTEM_ARCH "${CMAKE_${CMAKE_PRIMARY_LANG}_COMPILER_ARCHITECTURE_ID}")
string(TOLOWER "${CMAKE_${CMAKE_PRIMARY_LANG}_COMPILER_ARCHITECTURE_ID}" CMAKE_SYSTEM_ARCH)
if(CMAKE_SYSTEM_ARCH STREQUAL "x86")
set(X86_32 TRUE)
elseif(CMAKE_SYSTEM_ARCH STREQUAL "x86_64" OR CMAKE_SYSTEM_ARCH STREQUAL "x64")
set(X86_64 TRUE)
set(CMAKE_SYSTEM_ARCH "x86_64")
elseif(CMAKE_SYSTEM_ARCH MATCHES "^(aarch.*|arm.*)")
if(CMAKE_TARGET_BITNESS EQUAL 64)
set(AARCH64 TRUE)
else()
set(ARM32 TRUE)
endif()
endif()
elseif(CMAKE_ANDROID_ARCH_ABI)
set(CMAKE_SYSTEM_ARCH "${CMAKE_ANDROID_ARCH_ABI}")
if(CMAKE_SYSTEM_ARCH STREQUAL "x86")
set(X86_32 TRUE)
elseif(CMAKE_SYSTEM_ARCH STREQUAL "x86_64")
set(X86_64 TRUE)
elseif(CMAKE_SYSTEM_ARCH MATCHES "^(aarch.*|AARCH.*|arm.*|ARM.*)")
if(CMAKE_TARGET_BITNESS EQUAL 64)
set(AARCH64 TRUE)
else()
set(ARM32 TRUE)
endif()
elseif(CMAKE_SYSTEM_ARCH MATCHES "^(mips|MIPS).*")
if(CMAKE_TARGET_BITNESS EQUAL 64)
set(MIPS64 TRUE)
else()
set(MIPS32 TRUE)
endif()
endif()
elseif(CMAKE_COMPILER_IS_ELBRUSC OR CMAKE_COMPILER_IS_ELBRUSCXX
OR CMAKE_${CMAKE_PRIMARY_LANG}_COMPILER_ID STREQUAL "LCC"
OR CMAKE_SYSTEM_PROCESSOR MATCHES "e2k.*|E2K.*|elbrus.*|ELBRUS.*")
@@ -929,12 +960,13 @@ endmacro(setup_compile_flags)
macro(probe_libcxx_filesystem)
if(CMAKE_CXX_COMPILER_LOADED AND NOT DEFINED LIBCXX_FILESYSTEM)
list(FIND CMAKE_CXX_COMPILE_FEATURES cxx_std_11 HAS_CXX11)
if(NOT HAS_CXX11 LESS 0)
if(NOT HAS_CXX11 LESS 0 OR CXX_FALLBACK_GNU11 OR CXX_FALLBACK_11)
include(CMakePushCheckState)
include(CheckCXXSourceCompiles)
cmake_push_check_state()
set(stdfs_probe_save_libraries ${CMAKE_REQUIRED_LIBRARIES})
set(stdfs_probe_save_flags ${CMAKE_REQUIRED_FLAGS})
set(stdfs_probe_flags ${CMAKE_REQUIRED_FLAGS})
set(stdfs_probe_save_link_options ${CMAKE_REQUIRED_LINK_OPTIONS})
unset(stdfs_probe_clear_cxx_standard)
if(NOT DEFINED CMAKE_CXX_STANDARD)
@@ -945,18 +977,23 @@ macro(probe_libcxx_filesystem)
set(CMAKE_CXX_STANDARD 17)
elseif(NOT HAS_CXX14 LESS 0)
set(CMAKE_CXX_STANDARD 14)
else()
elseif(NOT HAS_CXX11 LESS 0)
set(CMAKE_CXX_STANDARD 11)
elseif(CXX_FALLBACK_GNU11)
set(stdfs_probe_flags ${stdfs_probe_flags} "-std=gnu++11")
else()
set(stdfs_probe_flags ${stdfs_probe_flags} "-std=c++11")
endif()
set(stdfs_probe_clear_cxx_standard ON)
endif()
if(CMAKE_COMPILER_IS_ELBRUSCXX AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 1.25.23)
if(CMAKE_VERSION VERSION_LESS 3.14)
set(CMAKE_REQUIRED_FLAGS ${stdfs_probe_save_flags} "-Wl,--allow-multiple-definition")
set(stdfs_probe_flags ${stdfs_probe_flags} "-Wl,--allow-multiple-definition")
else()
set(CMAKE_REQUIRED_LINK_OPTIONS ${stdfs_probe_save_link_options} "-Wl,--allow-multiple-definition")
endif()
endif()
set(CMAKE_REQUIRED_FLAGS ${stdfs_probe_flags})
set(stdfs_probe_code [[
#if defined(__SIZEOF_INT128__) && !defined(__GLIBCXX_TYPE_INT_N_0) && defined(__clang__) && __clang_major__ < 4
@@ -1037,6 +1074,7 @@ macro(probe_libcxx_filesystem)
unset(stdfs_probe_clear_cxx_standard)
unset(stdfs_probe_save_link_options)
unset(stdfs_probe_save_flags)
unset(stdfs_probe_flags)
unset(stdfs_probe_save_libraries)
unset(stdfs_probe_code)
unset(stdfs_probe_rc)

View File

@@ -13,7 +13,9 @@
## limitations under the License.
##
if(CMAKE_VERSION VERSION_LESS 3.12)
if(CMAKE_VERSION VERSION_LESS 3.8.2)
cmake_minimum_required(VERSION 3.0.2)
elseif(CMAKE_VERSION VERSION_LESS 3.12)
cmake_minimum_required(VERSION 3.8.2)
else()
cmake_minimum_required(VERSION 3.12)

View File

@@ -13,7 +13,9 @@
## limitations under the License.
##
if(CMAKE_VERSION VERSION_LESS 3.12)
if(CMAKE_VERSION VERSION_LESS 3.8.2)
cmake_minimum_required(VERSION 3.0.2)
elseif(CMAKE_VERSION VERSION_LESS 3.12)
cmake_minimum_required(VERSION 3.8.2)
else()
cmake_minimum_required(VERSION 3.12)

View File

@@ -67,7 +67,7 @@ the end is hit.
To retrieve all keys starting from a specified key value, use \ref MDBX_SET. For
more cursor operations, see the \ref c_api reference.
When using \ref mdbx_cursor_put()\ref , either the function will position the cursor
When using \ref mdbx_cursor_put(), either the function will position the cursor
for you based on the key, or you can use operation \ref MDBX_CURRENT to use the
current position of the cursor. \note Note that key must then match the current
position's key.

View File

@@ -38,4 +38,4 @@ including creating [merge-request](https://gitflic.ru/project/erthink/libmdbx/me
---
\section MithrilDB MithrilDB
\section MithrilDB MithrilDB and Future

View File

@@ -18,7 +18,13 @@
* <http://www.OpenLDAP.org/license.html>.
*/
#if (defined(__MINGW__) || defined(__MINGW32__) || defined(__MINGW64__)) && \
!defined(__USE_MINGW_ANSI_STDIO)
#define __USE_MINGW_ANSI_STDIO 1
#endif /* MinGW */
#include "mdbx.h"
#include <limits.h>
#include <stdint.h>
#include <stdio.h>

48
mdbx.h
View File

@@ -75,6 +75,14 @@ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
#ifndef LIBMDBX_H
#define LIBMDBX_H
#if defined(__riscv) || defined(__riscv__) || defined(__RISCV) || \
defined(__RISCV__)
#warning The RISC-V architecture is intentionally insecure by design. \
Please delete this admonition at your own risk, \
if you make such decision informed and consciously. \
Refer to https://clck.ru/32d9xH for more information.
#endif /* RISC-V */
#ifdef _MSC_VER
#pragma warning(push, 1)
#pragma warning(disable : 4548) /* expression before comma has no effect; \
@@ -687,11 +695,11 @@ extern LIBMDBX_VERINFO_API const struct MDBX_build_info {
* automatically (de)initialization, releasing reader lock table slots
* and so on.
*
* If MDBX builded as a DLL this is done out-of-the-box by DllEntry() function,
* If MDBX built as a DLL this is done out-of-the-box by DllEntry() function,
* which called automatically by Windows core with passing corresponding reason
* argument.
*
* Otherwise, if MDBX was builded not as a DLL, some black magic
* Otherwise, if MDBX was built not as a DLL, some black magic
* may be required depending of Windows version:
*
* - Modern Windows versions, including Windows Vista and later, provides
@@ -843,7 +851,7 @@ enum MDBX_constants {
/* DEBUG & LOGGING ************************************************************/
/** \addtogroup c_debug
* \note Most of debug feature enabled only when libmdbx builded with
* \note Most of debug feature enabled only when libmdbx built with
* \ref MDBX_DEBUG build option. @{ */
/** Log level
@@ -908,7 +916,7 @@ typedef enum MDBX_log_level_t MDBX_log_level_t;
*
* \details `MDBX_DBG_DUMP` and `MDBX_DBG_LEGACY_MULTIOPEN` always have an
* effect, but `MDBX_DBG_ASSERT`, `MDBX_DBG_AUDIT` and `MDBX_DBG_JITTER` only if
* libmdbx builded with \ref MDBX_DEBUG. */
* libmdbx built with \ref MDBX_DEBUG. */
enum MDBX_debug_flags_t {
MDBX_DBG_NONE = 0,
@@ -958,8 +966,16 @@ DEFINE_ENUM_FLAG_OPERATORS(MDBX_debug_flags_t)
* called before printing the message and aborting.
* \see mdbx_setup_debug()
*
* \param [in] env An environment handle returned by \ref mdbx_env_create().
* \param [in] msg The assertion message, not including newline. */
* \param [in] loglevel The severity of message.
* \param [in] function The function name which emits message,
* may be NULL.
* \param [in] line The source code line number which emits message,
* may be zero.
* \param [in] fmt The printf-like format string with message.
* \param [in] args The variable argument list respectively for the
* format-message string passed by `fmt` argument.
* Maybe NULL or invalid if the format-message string
* don't contain `%`-specification of arguments. */
typedef void MDBX_debug_func(MDBX_log_level_t loglevel, const char *function,
int line, const char *fmt,
va_list args) MDBX_CXX17_NOEXCEPT;
@@ -978,8 +994,12 @@ LIBMDBX_API int mdbx_setup_debug(MDBX_log_level_t log_level,
* called before printing the message and aborting.
* \see mdbx_env_set_assert()
*
* \param [in] env An environment handle returned by mdbx_env_create().
* \param [in] msg The assertion message, not including newline. */
* \param [in] env An environment handle.
* \param [in] msg The assertion message, not including newline.
* \param [in] function The function name where the assertion check failed,
* may be NULL.
* \param [in] line The line number in the source file
* where the assertion check failed, may be zero. */
typedef void MDBX_assert_func(const MDBX_env *env, const char *msg,
const char *function,
unsigned line) MDBX_CXX17_NOEXCEPT;
@@ -1622,7 +1642,7 @@ enum MDBX_copy_flags_t {
* pages sequentially */
MDBX_CP_COMPACT = 1u,
/** Force to make resizeable copy, i.e. dynamic size instead of fixed */
/** Force to make resizable copy, i.e. dynamic size instead of fixed */
MDBX_CP_FORCE_DYNAMIC_SIZE = 2u
};
#ifndef __cplusplus
@@ -2338,7 +2358,7 @@ LIBMDBX_API int mdbx_env_delete(const char *pathname,
* account skipping free pages.
*
* - \ref MDBX_CP_FORCE_DYNAMIC_SIZE
* Force to make resizeable copy, i.e. dynamic size instead of fixed.
* Force to make resizable copy, i.e. dynamic size instead of fixed.
*
* \returns A non-zero error value on failure and 0 on success. */
LIBMDBX_API int mdbx_env_copy(MDBX_env *env, const char *dest,
@@ -2391,7 +2411,7 @@ typedef struct MDBX_stat MDBX_stat;
/** \brief Return statistics about the MDBX environment.
* \ingroup c_statinfo
*
* At least one of env or txn argument must be non-null. If txn is passed
* At least one of `env` or `txn` argument must be non-null. If txn is passed
* non-null then stat will be filled accordingly to the given transaction.
* Otherwise, if txn is null, then stat will be populated by a snapshot from
* the last committed write transaction, and at next time, other information
@@ -2500,7 +2520,7 @@ typedef struct MDBX_envinfo MDBX_envinfo;
/** \brief Return information about the MDBX environment.
* \ingroup c_statinfo
*
* At least one of env or txn argument must be non-null. If txn is passed
* At least one of `env` or `txn` argument must be non-null. If txn is passed
* non-null then stat will be filled accordingly to the given transaction.
* Otherwise, if txn is null, then stat will be populated by a snapshot from
* the last committed write transaction, and at next time, other information
@@ -2856,7 +2876,7 @@ LIBMDBX_API int mdbx_env_get_fd(const MDBX_env *env, mdbx_filehandle_t *fd);
* was called after \ref mdbx_env_open() but OUTSIDE a write transaction,
* then MDBX will execute internal pseudo-transaction to apply new parameters
* (but only if anything has been changed), and changes be visible to any
* others processes immediately after succesful completion of function.
* others processes immediately after successful completion of function.
*
* Essentially a concept of "automatic size management" is simple and useful:
* - There are the lower and upper bounds of the database file size;
@@ -4936,7 +4956,7 @@ mdbx_get_datacmp(MDBX_db_flags_t flags);
* \param [in] thread The reader thread ID.
* \param [in] bytes_used The number of last used page
* in the MVCC-snapshot which being read,
* i.e. database file can't shrinked beyond this.
* i.e. database file can't be shrunk beyond this.
* \param [in] bytes_retained The total size of the database pages that were
* retired by committed write transactions after
* the reader's MVCC-snapshot,

View File

@@ -223,17 +223,18 @@
#endif /* MDBX_CXX20_UNLIKELY */
#ifndef MDBX_HAVE_CXX20_CONCEPTS
#if defined(DOXYGEN) || \
(defined(__cpp_lib_concepts) && __cpp_lib_concepts >= 202002L)
#if defined(__cpp_lib_concepts) && __cpp_lib_concepts >= 202002L
#include <concepts>
#define MDBX_HAVE_CXX20_CONCEPTS 1
#elif defined(DOXYGEN)
#define MDBX_HAVE_CXX20_CONCEPTS 1
#else
#define MDBX_HAVE_CXX20_CONCEPTS 0
#endif /* <concepts> */
#endif /* MDBX_HAVE_CXX20_CONCEPTS */
#ifndef MDBX_CXX20_CONCEPT
#if MDBX_HAVE_CXX20_CONCEPTS
#if MDBX_HAVE_CXX20_CONCEPTS || defined(DOXYGEN)
#define MDBX_CXX20_CONCEPT(CONCEPT, NAME) CONCEPT NAME
#else
#define MDBX_CXX20_CONCEPT(CONCEPT, NAME) typename NAME
@@ -241,7 +242,7 @@
#endif /* MDBX_CXX20_CONCEPT */
#ifndef MDBX_ASSERT_CXX20_CONCEPT_SATISFIED
#if MDBX_HAVE_CXX20_CONCEPTS
#if MDBX_HAVE_CXX20_CONCEPTS || defined(DOXYGEN)
#define MDBX_ASSERT_CXX20_CONCEPT_SATISFIED(CONCEPT, TYPE) \
static_assert(CONCEPT<TYPE>)
#else
@@ -287,7 +288,7 @@ namespace mdbx {
// To enable all kinds of an compiler optimizations we use a byte-like type
// that don't presumes aliases for pointers as does the `char` type and its
// derivatives/typedefs.
// Please see todo4recovery://erased_by_github/libmdbx/issues/263
// Please see https://libmdbx.dqdkfa.ru/dead-github/issues/263
// for reasoning of the use of `char8_t` type and switching to `__restrict__`.
using byte = char8_t;
#else
@@ -551,8 +552,11 @@ static MDBX_CXX14_CONSTEXPR size_t check_length(size_t headroom, size_t payload,
/// \defgroup cxx_data slices and buffers
/// @{
#if MDBX_HAVE_CXX20_CONCEPTS
#if MDBX_HAVE_CXX20_CONCEPTS || defined(DOXYGEN)
/** \concept MutableByteProducer
* \interface MutableByteProducer
* \brief MutableByteProducer C++20 concept */
template <typename T>
concept MutableByteProducer = requires(T a, char array[42]) {
{ a.is_empty() } -> std::same_as<bool>;
@@ -560,6 +564,9 @@ concept MutableByteProducer = requires(T a, char array[42]) {
{ a.write_bytes(&array[0], size_t(42)) } -> std::same_as<char *>;
};
/** \concept ImmutableByteProducer
* \interface ImmutableByteProducer
* \brief ImmutableByteProducer C++20 concept */
template <typename T>
concept ImmutableByteProducer = requires(const T &a, char array[42]) {
{ a.is_empty() } -> std::same_as<bool>;
@@ -567,6 +574,9 @@ concept ImmutableByteProducer = requires(const T &a, char array[42]) {
{ a.write_bytes(&array[0], size_t(42)) } -> std::same_as<char *>;
};
/** \concept SliceTranscoder
* \interface SliceTranscoder
* \brief SliceTranscoder C++20 concept */
template <typename T>
concept SliceTranscoder = ImmutableByteProducer<T> &&
requires(const slice &source, const T &a) {
@@ -2639,7 +2649,7 @@ public:
return buffer(src, make_reference);
}
static buffer key_from(const silo &&src) noexcept {
static buffer key_from(silo &&src) noexcept {
return buffer(::std::move(src));
}
@@ -3106,10 +3116,12 @@ public:
operate_parameters(const operate_parameters &) noexcept = default;
MDBX_CXX14_CONSTEXPR operate_parameters &
operator=(const operate_parameters &) noexcept = default;
MDBX_env_flags_t
make_flags(bool accede = true, ///< \copydoc MDBX_ACCEDE
bool use_subdirectory =
false ///< use subdirectory to place the DB files
MDBX_env_flags_t make_flags(
bool accede = true, ///< Allows accepting incompatible operating options
///< in case the database is already being used by
///< another process(es) \see MDBX_ACCEDE
bool use_subdirectory =
false ///< use subdirectory to place the DB files
) const;
static env::mode mode_from_flags(MDBX_env_flags_t) noexcept;
static env::durability durability_from_flags(MDBX_env_flags_t) noexcept;
@@ -3416,7 +3428,7 @@ public:
/// transactions since the current read
/// transaction started.
size_t bytes_used; ///< The number of last used page in the MVCC-snapshot
///< which being read, i.e. database file can't shrinked
///< which being read, i.e. database file can't be shrunk
///< beyond this.
size_t bytes_retained; ///< The total size of the database pages that
///< were retired by committed write transactions
@@ -3553,7 +3565,7 @@ public:
void close(bool dont_sync = false);
env_managed(env_managed &&) = default;
env_managed &operator=(env_managed &&other) {
env_managed &operator=(env_managed &&other) noexcept {
if (MDBX_UNLIKELY(handle_))
MDBX_CXX20_UNLIKELY {
assert(handle_ != other.handle_);
@@ -3852,7 +3864,7 @@ class LIBMDBX_API_TYPE txn_managed : public txn {
public:
MDBX_CXX11_CONSTEXPR txn_managed() noexcept = default;
txn_managed(txn_managed &&) = default;
txn_managed &operator=(txn_managed &&other) {
txn_managed &operator=(txn_managed &&other) noexcept {
if (MDBX_UNLIKELY(handle_))
MDBX_CXX20_UNLIKELY {
assert(handle_ != other.handle_);
@@ -4055,7 +4067,7 @@ public:
void close();
cursor_managed(cursor_managed &&) = default;
cursor_managed &operator=(cursor_managed &&other) {
cursor_managed &operator=(cursor_managed &&other) noexcept {
if (MDBX_UNLIKELY(handle_))
MDBX_CXX20_UNLIKELY {
assert(handle_ != other.handle_);

View File

@@ -236,10 +236,6 @@ __extern_C key_t ftok(const char *, int);
#elif _WIN32_WINNT < 0x0500
#error At least 'Windows 2000' API is required for libmdbx.
#endif /* _WIN32_WINNT */
#if (defined(__MINGW32__) || defined(__MINGW64__)) && \
!defined(__USE_MINGW_ANSI_STDIO)
#define __USE_MINGW_ANSI_STDIO 1
#endif /* MinGW */
#ifndef WIN32_LEAN_AND_MEAN
#define WIN32_LEAN_AND_MEAN
#endif /* WIN32_LEAN_AND_MEAN */
@@ -295,8 +291,8 @@ __extern_C key_t ftok(const char *, int);
/* Byteorder */
#if defined(i386) || defined(__386) || defined(__i386) || defined(__i386__) || \
defined(i486) || defined(__i486) || defined(__i486__) || \
defined(i586) | defined(__i586) || defined(__i586__) || defined(i686) || \
defined(i486) || defined(__i486) || defined(__i486__) || defined(i586) || \
defined(__i586) || defined(__i586__) || defined(i686) || \
defined(__i686) || defined(__i686__) || defined(_M_IX86) || \
defined(_X86_) || defined(__THW_INTEL__) || defined(__I86__) || \
defined(__INTEL__) || defined(__x86_64) || defined(__x86_64__) || \
@@ -541,15 +537,13 @@ __extern_C key_t ftok(const char *, int);
#ifndef __hot
#if defined(__OPTIMIZE__)
#if defined(__e2k__)
#define __hot __attribute__((__hot__)) __optimize(3)
#elif defined(__clang__) && !__has_attribute(__hot_) && \
#if defined(__clang__) && !__has_attribute(__hot__) && \
__has_attribute(__section__) && \
(defined(__linux__) || defined(__gnu_linux__))
/* just put frequently used functions in separate section */
#define __hot __attribute__((__section__("text.hot"))) __optimize("O3")
#elif defined(__GNUC__) || __has_attribute(__hot__)
#define __hot __attribute__((__hot__)) __optimize("O3")
#define __hot __attribute__((__hot__))
#else
#define __hot __optimize("O3")
#endif
@@ -560,15 +554,13 @@ __extern_C key_t ftok(const char *, int);
#ifndef __cold
#if defined(__OPTIMIZE__)
#if defined(__e2k__)
#define __cold __attribute__((__cold__)) __optimize(1)
#elif defined(__clang__) && !__has_attribute(cold) && \
#if defined(__clang__) && !__has_attribute(__cold__) && \
__has_attribute(__section__) && \
(defined(__linux__) || defined(__gnu_linux__))
/* just put infrequently used functions in separate section */
#define __cold __attribute__((__section__("text.unlikely"))) __optimize("Os")
#elif defined(__GNUC__) || __has_attribute(cold)
#define __cold __attribute__((__cold__)) __optimize("Os")
#elif defined(__GNUC__) || __has_attribute(__cold__)
#define __cold __attribute__((__cold__))
#else
#define __cold __optimize("Os")
#endif

View File

@@ -397,7 +397,7 @@ node_largedata_pgno(const MDBX_node *const __restrict node) {
*
* BRANCH_NODE_MAX
* Branch-page must contain at least two nodes, within each a key and a child
* page number. But page can't be splitted if it contains less that 4 keys,
* page number. But page can't be split if it contains less that 4 keys,
* i.e. a page should not overflow before adding the fourth key. Therefore,
* at least 3 branch-node should fit in the single branch-page. Further, the
* first node of a branch-page doesn't contain a key, i.e. the first node
@@ -411,8 +411,8 @@ node_largedata_pgno(const MDBX_node *const __restrict node) {
* Leaf-node must fit into single leaf-page, where a value could be placed on
* a large/overflow page. However, may require to insert a nearly page-sized
* node between two large nodes are already fill-up a page. In this case the
* page must be splitted to two if some pair of nodes fits on one page, or
* otherwise the page should be splitted to the THREE with a single node
* page must be split to two if some pair of nodes fits on one page, or
* otherwise the page should be split to the THREE with a single node
* per each of ones. Such 1-into-3 page splitting is costly and complex since
* requires TWO insertion into the parent page, that could lead to split it
* and so on up to the root. Therefore double-splitting is avoided here and
@@ -983,14 +983,7 @@ static __always_inline bool atomic_cas64(MDBX_atomic_uint64_t *p, uint64_t c,
uint64_t v) {
#ifdef MDBX_HAVE_C11ATOMICS
STATIC_ASSERT(sizeof(long long) >= sizeof(uint64_t));
#ifdef ATOMIC_LLONG_LOCK_FREE
STATIC_ASSERT(ATOMIC_LLONG_LOCK_FREE > 0);
#if ATOMIC_LLONG_LOCK_FREE < 2
assert(atomic_is_lock_free(MDBX_c11a_rw(uint64_t, p)));
#endif /* ATOMIC_LLONG_LOCK_FREE < 2 */
#else /* defined(ATOMIC_LLONG_LOCK_FREE) */
assert(atomic_is_lock_free(MDBX_c11a_rw(uint64_t, p)));
#endif
return atomic_compare_exchange_strong(MDBX_c11a_rw(uint64_t, p), &c, v);
#elif defined(__GNUC__) || defined(__clang__)
return __sync_bool_compare_and_swap(&p->weak, c, v);
@@ -1009,14 +1002,7 @@ static __always_inline bool atomic_cas32(MDBX_atomic_uint32_t *p, uint32_t c,
uint32_t v) {
#ifdef MDBX_HAVE_C11ATOMICS
STATIC_ASSERT(sizeof(int) >= sizeof(uint32_t));
#ifdef ATOMIC_INT_LOCK_FREE
STATIC_ASSERT(ATOMIC_INT_LOCK_FREE > 0);
#if ATOMIC_INT_LOCK_FREE < 2
assert(atomic_is_lock_free(MDBX_c11a_rw(uint32_t, p)));
#endif
#else
assert(atomic_is_lock_free(MDBX_c11a_rw(uint32_t, p)));
#endif
return atomic_compare_exchange_strong(MDBX_c11a_rw(uint32_t, p), &c, v);
#elif defined(__GNUC__) || defined(__clang__)
return __sync_bool_compare_and_swap(&p->weak, c, v);
@@ -1035,14 +1021,7 @@ static __always_inline uint32_t atomic_add32(MDBX_atomic_uint32_t *p,
uint32_t v) {
#ifdef MDBX_HAVE_C11ATOMICS
STATIC_ASSERT(sizeof(int) >= sizeof(uint32_t));
#ifdef ATOMIC_INT_LOCK_FREE
STATIC_ASSERT(ATOMIC_INT_LOCK_FREE > 0);
#if ATOMIC_INT_LOCK_FREE < 2
assert(atomic_is_lock_free(MDBX_c11a_rw(uint32_t, p)));
#endif
#else
assert(atomic_is_lock_free(MDBX_c11a_rw(uint32_t, p)));
#endif
return atomic_fetch_add(MDBX_c11a_rw(uint32_t, p), v);
#elif defined(__GNUC__) || defined(__clang__)
return __sync_fetch_and_add(&p->weak, v);
@@ -1108,7 +1087,7 @@ static __always_inline bool safe64_reset_compare(MDBX_atomic_uint64_t *p,
* the asynchronously cancellation of read transaction. Therefore,
* there may be a collision between the cleanup performed here and
* asynchronous termination and restarting of the read transaction
* in another proces/thread. In general we MUST NOT reset the `mr_txnid`
* in another process/thread. In general we MUST NOT reset the `mr_txnid`
* if a new transaction was started (i.e. if `mr_txnid` was changed). */
#if MDBX_64BIT_CAS
bool rc = atomic_cas64(p, compare, UINT64_MAX);
@@ -1789,7 +1768,7 @@ __cold static int uniq_check(const mdbx_mmap_t *pending, MDBX_env **found) {
? uniq_peek(pending, &scan->me_lck_mmap)
: uniq_poke(pending, &scan->me_lck_mmap, &salt);
if (err == MDBX_ENODATA) {
uint64_t length;
uint64_t length = 0;
if (likely(mdbx_filesize(pending->fd, &length) == MDBX_SUCCESS &&
length == 0)) {
/* LY: skip checking since LCK-file is empty, i.e. just created. */
@@ -4835,7 +4814,8 @@ static int mdbx_page_retire_ex(MDBX_cursor *mc, const pgno_t pgno,
rc = mdbx_page_get(mc, pgno, &check, txn->mt_front);
if (unlikely(rc != MDBX_SUCCESS))
return rc;
mdbx_tassert(txn, (PAGETYPE(check) & ~P_LEAF2) == (pagetype & ~P_FROZEN));
mdbx_tassert(txn, (PAGETYPE(check) & ~(P_LEAF2 | P_SPILLED)) ==
(pagetype & ~(P_LEAF2 | P_FROZEN)));
mdbx_tassert(txn, !(pagetype & P_FROZEN) || IS_FROZEN(txn, check));
}
if (pagetype & P_FROZEN) {
@@ -5175,7 +5155,7 @@ static int mdbx_iov_write(MDBX_txn *const txn, struct mdbx_iov_ctx *ctx) {
MDBX_page *wp = (MDBX_page *)ctx->iov[i].iov_base;
const MDBX_page *rp = pgno2page(txn->mt_env, wp->mp_pgno);
/* check with timeout as the workaround
* for todo4recovery://erased_by_github/libmdbx/issues/269 */
* for https://libmdbx.dqdkfa.ru/dead-github/issues/269 */
while (likely(rc == MDBX_SUCCESS) &&
unlikely(memcmp(wp, rp, ctx->iov[i].iov_len) != 0)) {
if (!timestamp) {
@@ -6348,7 +6328,6 @@ __cold static int mdbx_mapresize(MDBX_env *env, const pgno_t used_pgno,
bailout:
if (rc == MDBX_SUCCESS) {
mdbx_assert(env, size_bytes == env->me_dxb_mmap.current);
mdbx_assert(env, size_bytes <= env->me_dxb_mmap.filesize);
mdbx_assert(env, limit_bytes == env->me_dxb_mmap.limit);
#ifdef MDBX_USE_VALGRIND
@@ -6526,7 +6505,7 @@ __hot static struct page_result mdbx_page_alloc(MDBX_cursor *mc,
catch-up with itself by growing while trying to save it. */
(mc->mc_flags & C_RECLAIMING) ||
/* avoid (recursive) search inside empty tree and while tree is
updating, todo4recovery://erased_by_github/libmdbx/issues/31 */
updating, https://libmdbx.dqdkfa.ru/dead-github/issues/31 */
txn->mt_dbs[FREE_DBI].md_entries == 0 ||
/* If our dirty list is already full, we can't touch GC */
(txn->tw.dirtyroom < txn->mt_dbs[FREE_DBI].md_depth &&
@@ -6740,7 +6719,7 @@ no_loose:
/* Stop reclaiming to avoid overflow the page list.
* This is a rare case while search for a continuously multi-page region
* in a large database.
* todo4recovery://erased_by_github/libmdbx/issues/123 */
* https://libmdbx.dqdkfa.ru/dead-github/issues/123 */
mdbx_notice("stop reclaiming to avoid PNL overflow: %u (current) + %u "
"(chunk) -> %u",
MDBX_PNL_SIZE(txn->tw.reclaimed_pglist), gc_len,
@@ -7720,7 +7699,7 @@ __cold int mdbx_thread_unregister(const MDBX_env *env) {
return MDBX_SUCCESS;
}
/* check against todo4recovery://erased_by_github/libmdbx/issues/269 */
/* check against https://libmdbx.dqdkfa.ru/dead-github/issues/269 */
static bool meta_checktxnid(const MDBX_env *env, const MDBX_meta *meta,
bool report) {
const txnid_t head_txnid = meta_txnid(env, meta);
@@ -7796,7 +7775,7 @@ static bool meta_checktxnid(const MDBX_env *env, const MDBX_meta *meta,
}
/* check with timeout as the workaround
* for todo4recovery://erased_by_github/libmdbx/issues/269 */
* for https://libmdbx.dqdkfa.ru/dead-github/issues/269 */
static int meta_waittxnid(const MDBX_env *env, const MDBX_meta *meta,
uint64_t *timestamp) {
if (likely(meta_checktxnid(env, meta, !*timestamp)))
@@ -7941,8 +7920,7 @@ static int mdbx_txn_renew0(MDBX_txn *txn, const unsigned flags) {
snap == meta_txnid(env, meta) &&
snap >= atomic_load64(&env->me_lck->mti_oldest_reader,
mo_AcquireRelease))) {
/* workaround for todo4recovery://erased_by_github/libmdbx/issues/269
*/
/* workaround for https://libmdbx.dqdkfa.ru/dead-github/issues/269 */
rc = meta_waittxnid(env, (const MDBX_meta *)meta, &timestamp);
mdbx_jitter4testing(false);
if (likely(rc == MDBX_SUCCESS))
@@ -8030,8 +8008,7 @@ static int mdbx_txn_renew0(MDBX_txn *txn, const unsigned flags) {
mdbx_jitter4testing(false);
const MDBX_meta *meta = constmeta_prefer_last(env);
uint64_t timestamp = 0;
while (
"workaround for todo4recovery://erased_by_github/libmdbx/issues/269") {
while ("workaround for https://libmdbx.dqdkfa.ru/dead-github/issues/269") {
rc = meta_waittxnid(env, (const MDBX_meta *)meta, &timestamp);
if (likely(rc == MDBX_SUCCESS))
break;
@@ -8788,10 +8765,6 @@ static int mdbx_txn_end(MDBX_txn *txn, const unsigned mode) {
(void *)env, txn->mt_dbs[MAIN_DBI].md_root,
txn->mt_dbs[FREE_DBI].md_root);
mdbx_ensure(env, txn->mt_txnid >=
/* paranoia is appropriate here */ env->me_lck
->mti_oldest_reader.weak);
if (!(mode & MDBX_END_EOTDONE)) /* !(already closed cursors) */
mdbx_cursors_eot(txn, false);
@@ -8801,6 +8774,9 @@ static int mdbx_txn_end(MDBX_txn *txn, const unsigned mode) {
MDBX_reader *slot = txn->to.reader;
mdbx_assert(env, slot->mr_pid.weak == env->me_pid);
if (likely(!F_ISSET(txn->mt_flags, MDBX_TXN_FINISHED))) {
mdbx_ensure(env, txn->mt_txnid >=
/* paranoia is appropriate here */ env->me_lck
->mti_oldest_reader.weak);
mdbx_assert(env, txn->mt_txnid == slot->mr_txnid.weak &&
slot->mr_txnid.weak >=
env->me_lck->mti_oldest_reader.weak);
@@ -8828,7 +8804,10 @@ static int mdbx_txn_end(MDBX_txn *txn, const unsigned mode) {
txn->mt_numdbs = 0; /* prevent further DBI activity */
txn->mt_flags = MDBX_TXN_RDONLY | MDBX_TXN_FINISHED;
txn->mt_owner = 0;
} else if (!F_ISSET(txn->mt_flags, MDBX_TXN_FINISHED)) {
} else if (!(txn->mt_flags & MDBX_TXN_FINISHED)) {
mdbx_ensure(env, txn->mt_txnid >=
/* paranoia is appropriate here */ env->me_lck
->mti_oldest_reader.weak);
#if defined(MDBX_USE_VALGRIND) || defined(__SANITIZE_ADDRESS__)
if (txn == env->me_txn0)
mdbx_txn_valgrind(env, nullptr);
@@ -11917,7 +11896,7 @@ mdbx_env_set_geometry(MDBX_env *env, intptr_t size_lower, intptr_t size_now,
uint64_t timestamp = 0;
while ("workaround for "
"todo4recovery://erased_by_github/libmdbx/issues/269") {
"https://libmdbx.dqdkfa.ru/dead-github/issues/269") {
meta = *head;
rc = meta_waittxnid(env, &meta, &timestamp);
if (likely(rc == MDBX_SUCCESS))
@@ -12845,7 +12824,8 @@ __cold static int __must_check_result mdbx_override_meta(
mdbx_meta_model(env, page, target);
MDBX_meta *const model = page_meta(page);
meta_set_txnid(env, model, txnid);
mdbx_assert(env, meta_checktxnid(env, model, true));
if (txnid)
mdbx_assert(env, meta_checktxnid(env, model, true));
if (shape) {
if (txnid && unlikely(!meta_checktxnid(env, shape, false))) {
mdbx_error("bailout overriding meta-%u since model failed "
@@ -13188,7 +13168,7 @@ __cold int mdbx_env_open(MDBX_env *env, const char *pathname,
} else {
#if MDBX_MMAP_INCOHERENT_FILE_WRITE
/* Temporary `workaround` for OpenBSD kernel's flaw.
* See todo4recovery://erased_by_github/libmdbx/issues/67 */
* See https://libmdbx.dqdkfa.ru/dead-github/issues/67 */
if ((flags & MDBX_WRITEMAP) == 0) {
if (flags & MDBX_ACCEDE)
flags |= MDBX_WRITEMAP;
@@ -13465,9 +13445,10 @@ __cold static int mdbx_env_close0(MDBX_env *env) {
}
if (env->me_dbxs) {
for (unsigned i = env->me_numdbs; --i >= CORE_DBS;)
for (unsigned i = CORE_DBS; i < env->me_numdbs; ++i)
mdbx_free(env->me_dbxs[i].md_name.iov_base);
mdbx_free(env->me_dbxs);
env->me_numdbs = CORE_DBS;
env->me_dbxs = nullptr;
}
if (env->me_pbuf) {
@@ -15542,7 +15523,7 @@ int mdbx_cursor_put(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data,
mc->mc_ki[mc->mc_top]++; /* step forward for appending */
rc = MDBX_NOTFOUND;
} else {
if (unlikely(rc != MDBX_SUCCESS || !(flags & MDBX_APPENDDUP)))
if (unlikely(rc != MDBX_SUCCESS))
/* new-key < last-key
* or new-key == last-key without MDBX_APPENDDUP */
return MDBX_EKEYMISMATCH;
@@ -15579,7 +15560,7 @@ int mdbx_cursor_put(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data,
if (unlikely(rc != MDBX_SUCCESS))
return rc;
flags -= MDBX_ALLDUPS;
rc = MDBX_NOTFOUND;
rc = mc->mc_snum ? MDBX_NOTFOUND : MDBX_NO_ROOT;
exact = false;
} else /* checking for early exit without dirtying pages */
if (!(flags & (MDBX_RESERVE | MDBX_MULTIPLE)) &&
@@ -19484,8 +19465,8 @@ __cold static void compact_fixup_meta(MDBX_env *env, MDBX_meta *meta) {
unaligned_poke_u64(4, meta->mm_datasync_sign, meta_sign(meta));
}
/* Make resizeable */
__cold static void make_sizeable(MDBX_meta *meta) {
/* Make resizable */
__cold static void make_resizable(MDBX_meta *meta) {
meta->mm_geo.lower = MIN_PAGENO;
if (meta->mm_geo.grow_pv == 0) {
const pgno_t step = 1 + (meta->mm_geo.upper - meta->mm_geo.lower) / 42;
@@ -19508,7 +19489,7 @@ __cold static int mdbx_env_compact(MDBX_env *env, MDBX_txn *read_txn,
meta_set_txnid(env, meta, read_txn->mt_txnid);
if (flags & MDBX_CP_FORCE_DYNAMIC_SIZE)
make_sizeable(meta);
make_resizable(meta);
/* copy canary sequences if present */
if (read_txn->mt_canary.v) {
@@ -19671,7 +19652,7 @@ __cold static int mdbx_env_copy_asis(MDBX_env *env, MDBX_txn *read_txn,
mdbx_txn_unlock(env);
if (flags & MDBX_CP_FORCE_DYNAMIC_SIZE)
make_sizeable(headcopy);
make_resizable(headcopy);
/* Update signature to steady */
unaligned_poke_u64(4, headcopy->mm_datasync_sign, meta_sign(headcopy));
@@ -19688,6 +19669,11 @@ __cold static int mdbx_env_copy_asis(MDBX_env *env, MDBX_txn *read_txn,
#if MDBX_USE_COPYFILERANGE
static bool copyfilerange_unavailable;
bool not_the_same_filesystem = false;
struct statfs statfs_info;
if (fstatfs(fd, &statfs_info) ||
statfs_info.f_type == /* ECRYPTFS_SUPER_MAGIC */ 0xf15f)
/* avoid use copyfilerange_unavailable() to ecryptfs due bugs */
not_the_same_filesystem = true;
#endif /* MDBX_USE_COPYFILERANGE */
for (size_t offset = meta_bytes; rc == MDBX_SUCCESS && offset < used_size;) {
#if MDBX_USE_SENDFILE
@@ -19721,7 +19707,9 @@ __cold static int mdbx_env_copy_asis(MDBX_env *env, MDBX_txn *read_txn,
if (bytes_copied == 0)
break;
rc = errno;
if (rc == EXDEV)
if (rc == EXDEV || rc == /* workaround for ecryptfs bug(s),
maybe useful for others FS */
EINVAL)
not_the_same_filesystem = true;
else if (ignore_enosys(rc) == MDBX_RESULT_TRUE)
copyfilerange_unavailable = true;
@@ -19851,30 +19839,25 @@ __cold int mdbx_env_copy(MDBX_env *env, const char *dest_path,
#endif
);
if (rc == MDBX_SUCCESS) {
#if defined(_WIN32) || defined(_WIN64)
OVERLAPPED ov;
memset(&ov, 0, sizeof(ov));
if (!LockFileEx(newfd, LOCKFILE_EXCLUSIVE_LOCK | LOCKFILE_FAIL_IMMEDIATELY,
0, 0, INT32_MAX, &ov))
rc = GetLastError();
/* no locking required since the file opened with ShareMode == 0 */
#else
struct flock lock_op;
if (rc == MDBX_SUCCESS) {
MDBX_STRUCT_FLOCK lock_op;
memset(&lock_op, 0, sizeof(lock_op));
lock_op.l_type = F_WRLCK;
lock_op.l_whence = SEEK_SET;
lock_op.l_start = 0;
lock_op.l_len =
(sizeof(lock_op.l_len) > 4 ? INT64_MAX : INT32_MAX) & ~(size_t)0xffff;
if (fcntl(newfd, F_SETLK, &lock_op)
lock_op.l_len = OFF_T_MAX;
if (MDBX_FCNTL(newfd, MDBX_F_SETLK, &lock_op)
#if (defined(__linux__) || defined(__gnu_linux__)) && defined(LOCK_EX) && \
(!defined(__ANDROID_API__) || __ANDROID_API__ >= 24)
|| flock(newfd, LOCK_EX | LOCK_NB)
#endif /* Linux */
)
rc = errno;
#endif /* Windows / POSIX */
}
#endif /* Windows / POSIX */
if (rc == MDBX_SUCCESS)
rc = mdbx_env_copy2fd(env, newfd, flags);
@@ -20184,7 +20167,7 @@ __cold static int fetch_envinfo_ex(const MDBX_env *env, const MDBX_txn *txn,
const size_t size_before_pgop_stat = offsetof(MDBX_envinfo, mi_pgop_stat);
/* is the environment open?
* (todo4recovery://erased_by_github/libmdbx/issues/171) */
* (https://libmdbx.dqdkfa.ru/dead-github/issues/171) */
if (unlikely(!env->me_map)) {
/* environment not yet opened */
#if 1
@@ -20995,8 +20978,8 @@ __cold int mdbx_reader_list(const MDBX_env *env, MDBX_reader_list_func *func,
reader_pages_retired))
: 0;
}
rc = func(ctx, ++serial, i, pid, (mdbx_tid_t)tid, txnid, lag, bytes_used,
bytes_retained);
rc = func(ctx, ++serial, i, pid, (mdbx_tid_t)((intptr_t)tid), txnid, lag,
bytes_used, bytes_retained);
if (unlikely(rc != MDBX_SUCCESS))
break;
}
@@ -21248,7 +21231,7 @@ __cold static txnid_t mdbx_kick_longlived_readers(MDBX_env *env,
? pgno2bytes(env, (pgno_t)(head_retired - oldest_retired))
: 0;
int rc = env->me_hsr_callback(
env, env->me_txn, pid, (mdbx_tid_t)tid, laggard,
env, env->me_txn, pid, (mdbx_tid_t)((intptr_t)tid), laggard,
(gap < UINT_MAX) ? (unsigned)gap : UINT_MAX, space, retry);
if (rc < 0)
break;
@@ -22686,14 +22669,17 @@ __cold int mdbx_env_set_option(MDBX_env *env, const MDBX_option_t option,
return MDBX_EPERM;
if (unlikely(value > SIZE_MAX - 65536))
return MDBX_TOO_LARGE;
if (atomic_store32(&env->me_lck->mti_autosync_threshold,
bytes2pgno(env, (size_t)value + env->me_psize - 1),
mo_Relaxed) != 0 &&
(env->me_flags & MDBX_ENV_ACTIVE)) {
err = mdbx_env_sync_poll(env);
if (unlikely(MDBX_IS_ERROR(err)))
return err;
err = MDBX_SUCCESS;
value = bytes2pgno(env, (size_t)value + env->me_psize - 1);
if ((uint32_t)value != atomic_load32(&env->me_lck->mti_autosync_threshold,
mo_AcquireRelease) &&
atomic_store32(&env->me_lck->mti_autosync_threshold, (uint32_t)value,
mo_Relaxed)
/* Дергаем sync(force=off) только если задано новое не-нулевое значение
* и мы вне транзакции */
&& lock_needed) {
err = mdbx_env_sync_internal(env, false, false);
if (err == /* нечего сбрасывать на диск */ MDBX_RESULT_TRUE)
err = MDBX_SUCCESS;
}
break;
@@ -22706,14 +22692,16 @@ __cold int mdbx_env_set_option(MDBX_env *env, const MDBX_option_t option,
return MDBX_EPERM;
if (unlikely(value > UINT32_MAX))
return MDBX_TOO_LARGE;
if (atomic_store64(&env->me_lck->mti_autosync_period,
mdbx_osal_16dot16_to_monotime((uint32_t)value),
mo_Relaxed) != 0 &&
(env->me_flags & MDBX_ENV_ACTIVE)) {
err = mdbx_env_sync_poll(env);
if (unlikely(MDBX_IS_ERROR(err)))
return err;
err = MDBX_SUCCESS;
value = mdbx_osal_16dot16_to_monotime((uint32_t)value);
if (value != atomic_load64(&env->me_lck->mti_autosync_period,
mo_AcquireRelease) &&
atomic_store64(&env->me_lck->mti_autosync_period, value, mo_Relaxed)
/* Дергаем sync(force=off) только если задано новое не-нулевое значение
* и мы вне транзакции */
&& lock_needed) {
err = mdbx_env_sync_internal(env, false, false);
if (err == /* нечего сбрасывать на диск */ MDBX_RESULT_TRUE)
err = MDBX_SUCCESS;
}
break;
@@ -23244,6 +23232,8 @@ __dll_export
"MINGW-64 " MDBX_STRINGIFY(__MINGW64_MAJOR_VERSION) "." MDBX_STRINGIFY(__MINGW64_MINOR_VERSION)
#elif defined(__MINGW32__)
"MINGW-32 " MDBX_STRINGIFY(__MINGW32_MAJOR_VERSION) "." MDBX_STRINGIFY(__MINGW32_MINOR_VERSION)
#elif defined(__MINGW__)
"MINGW " MDBX_STRINGIFY(__MINGW_MAJOR_VERSION) "." MDBX_STRINGIFY(__MINGW_MINOR_VERSION)
#elif defined(__IBMC__)
"IBM C " MDBX_STRINGIFY(__IBMC__)
#elif defined(__GNUC__)
@@ -23277,7 +23267,7 @@ __dll_export
};
#ifdef __SANITIZE_ADDRESS__
LIBMDBX_API __attribute__((__weak__)) const char *__asan_default_options() {
LIBMDBX_API __attribute__((__weak__)) const char *__asan_default_options(void) {
return "symbolize=1:allow_addr2line=1:"
#if MDBX_DEBUG
"debug=1:"

View File

@@ -35,7 +35,7 @@
/** Disables using GNU/Linux libc extensions.
* \ingroup build_option
* \note This option couldn't be moved to the options.h since dependant
* \note This option couldn't be moved to the options.h since dependent
* control macros/defined should be prepared before include the options.h */
#ifndef MDBX_DISABLE_GNU_SOURCE
#define MDBX_DISABLE_GNU_SOURCE 0
@@ -126,7 +126,7 @@
#if (defined(__MINGW__) || defined(__MINGW32__) || defined(__MINGW64__)) && \
!defined(__USE_MINGW_ANSI_STDIO)
#define __USE_MINGW_ANSI_STDIO 1
#endif /* __USE_MINGW_ANSI_STDIO */
#endif /* MinGW */
#include "../mdbx.h"
#include "base.h"

View File

@@ -158,26 +158,21 @@ __cold static void choice_fcntl(void) {
of reliability reasons */
#endif /* linux */
) {
op_setlk = F_OFD_SETLK;
op_setlkw = F_OFD_SETLKW;
op_getlk = F_OFD_GETLK;
op_setlk = MDBX_F_OFD_SETLK;
op_setlkw = MDBX_F_OFD_SETLKW;
op_getlk = MDBX_F_OFD_GETLK;
return;
}
op_setlk = F_SETLK;
op_setlkw = F_SETLKW;
op_getlk = F_GETLK;
op_setlk = MDBX_F_SETLK;
op_setlkw = MDBX_F_SETLKW;
op_getlk = MDBX_F_GETLK;
}
#else
#define op_setlk F_SETLK
#define op_setlkw F_SETLKW
#define op_getlk F_GETLK
#define op_setlk MDBX_F_SETLK
#define op_setlkw MDBX_F_SETLKW
#define op_getlk MDBX_F_GETLK
#endif /* MDBX_USE_OFDLOCKS */
#ifndef OFF_T_MAX
#define OFF_T_MAX \
(((sizeof(off_t) > 4) ? INT64_MAX : INT32_MAX) & ~(size_t)0xffff)
#endif
static int lck_op(const mdbx_filehandle_t fd, int cmd, const int lck,
const off_t offset, off_t len) {
STATIC_ASSERT(sizeof(off_t) >= sizeof(void *) &&
@@ -200,7 +195,7 @@ static int lck_op(const mdbx_filehandle_t fd, int cmd, const int lck,
assert((uint64_t)((off_t)((uint64_t)offset + (uint64_t)len)) ==
((uint64_t)offset + (uint64_t)len));
for (;;) {
struct flock lock_op;
MDBX_STRUCT_FLOCK lock_op;
STATIC_ASSERT_MSG(sizeof(off_t) <= sizeof(lock_op.l_start) &&
sizeof(off_t) <= sizeof(lock_op.l_len) &&
OFF_T_MAX == (off_t)OFF_T_MAX,
@@ -212,7 +207,7 @@ static int lck_op(const mdbx_filehandle_t fd, int cmd, const int lck,
lock_op.l_whence = SEEK_SET;
lock_op.l_start = offset;
lock_op.l_len = len;
int rc = fcntl(fd, cmd, &lock_op);
int rc = MDBX_FCNTL(fd, cmd, &lock_op);
mdbx_jitter4testing(true);
if (rc != -1) {
if (cmd == op_getlk) {
@@ -226,18 +221,18 @@ static int lck_op(const mdbx_filehandle_t fd, int cmd, const int lck,
}
rc = errno;
#if MDBX_USE_OFDLOCKS
if (rc == EINVAL &&
(cmd == F_OFD_SETLK || cmd == F_OFD_SETLKW || cmd == F_OFD_GETLK)) {
if (rc == EINVAL && (cmd == MDBX_F_OFD_SETLK || cmd == MDBX_F_OFD_SETLKW ||
cmd == MDBX_F_OFD_GETLK)) {
/* fallback to non-OFD locks */
if (cmd == F_OFD_SETLK)
cmd = F_SETLK;
else if (cmd == F_OFD_SETLKW)
cmd = F_SETLKW;
if (cmd == MDBX_F_OFD_SETLK)
cmd = MDBX_F_SETLK;
else if (cmd == MDBX_F_OFD_SETLKW)
cmd = MDBX_F_SETLKW;
else
cmd = F_GETLK;
op_setlk = F_SETLK;
op_setlkw = F_SETLKW;
op_getlk = F_GETLK;
cmd = MDBX_F_GETLK;
op_setlk = MDBX_F_SETLK;
op_setlkw = MDBX_F_SETLKW;
op_getlk = MDBX_F_GETLK;
continue;
}
#endif /* MDBX_USE_OFDLOCKS */

View File

@@ -161,7 +161,21 @@ int mdbx_txn_lock(MDBX_env *env, bool dontwait) {
: (LCK_EXCLUSIVE | LCK_WAITFOR),
LCK_BODY))
return MDBX_SUCCESS;
int rc = (int)GetLastError();
if (rc == ERROR_LOCK_VIOLATION && dontwait) {
SleepEx(0, true);
if (flock(env->me_lazy_fd, LCK_EXCLUSIVE | LCK_DONTWAIT, LCK_BODY))
return MDBX_SUCCESS;
rc = (int)GetLastError();
if (rc == ERROR_LOCK_VIOLATION) {
SleepEx(0, true);
if (flock(env->me_lazy_fd, LCK_EXCLUSIVE | LCK_DONTWAIT, LCK_BODY))
return MDBX_SUCCESS;
rc = (int)GetLastError();
}
}
LeaveCriticalSection(&env->me_windowsbug_lock);
return (!dontwait || rc != ERROR_LOCK_VIOLATION) ? rc : MDBX_BUSY;
}

View File

@@ -1,6 +1,6 @@
.\" Copyright 2015-2022 Leonid Yuriev <leo@yuriev.ru>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_CHK 1 "2022-08-22" "MDBX 0.11.10"
.TH MDBX_CHK 1 "2023-02-14" "MDBX 0.11.14"
.SH NAME
mdbx_chk \- MDBX checking tool
.SH SYNOPSIS

View File

@@ -2,7 +2,7 @@
.\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_COPY 1 "2022-08-22" "MDBX 0.11.10"
.TH MDBX_COPY 1 "2023-02-14" "MDBX 0.11.14"
.SH NAME
mdbx_copy \- MDBX environment copy tool
.SH SYNOPSIS

View File

@@ -1,7 +1,7 @@
.\" Copyright 2021-2022 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2014-2021 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_DROP 1 "2022-08-22" "MDBX 0.11.10"
.TH MDBX_DROP 1 "2023-02-14" "MDBX 0.11.14"
.SH NAME
mdbx_drop \- MDBX database delete tool
.SH SYNOPSIS

View File

@@ -2,7 +2,7 @@
.\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_DUMP 1 "2022-08-22" "MDBX 0.11.10"
.TH MDBX_DUMP 1 "2023-02-14" "MDBX 0.11.14"
.SH NAME
mdbx_dump \- MDBX environment export tool
.SH SYNOPSIS

View File

@@ -2,7 +2,7 @@
.\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_LOAD 1 "2022-08-22" "MDBX 0.11.10"
.TH MDBX_LOAD 1 "2023-02-14" "MDBX 0.11.14"
.SH NAME
mdbx_load \- MDBX environment import tool
.SH SYNOPSIS

View File

@@ -2,7 +2,7 @@
.\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_STAT 1 "2022-08-22" "MDBX 0.11.10"
.TH MDBX_STAT 1 "2023-02-14" "MDBX 0.11.14"
.SH NAME
mdbx_stat \- MDBX environment status tool
.SH SYNOPSIS

View File

@@ -12,7 +12,7 @@
#if (defined(__MINGW__) || defined(__MINGW32__) || defined(__MINGW64__)) && \
!defined(__USE_MINGW_ANSI_STDIO)
#define __USE_MINGW_ANSI_STDIO 1
#endif /* __USE_MINGW_ANSI_STDIO */
#endif /* MinGW */
#include "../mdbx.h++"

View File

@@ -1157,7 +1157,7 @@ int main(int argc, char *argv[]) {
envflags &= ~MDBX_RDONLY;
#if MDBX_MMAP_INCOHERENT_FILE_WRITE
/* Temporary `workaround` for OpenBSD kernel's flaw.
* See todo4recovery://erased_by_github/libmdbx/issues/67 */
* See https://libmdbx.dqdkfa.ru/dead-github/issues/67 */
envflags |= MDBX_WRITEMAP;
#endif /* MDBX_MMAP_INCOHERENT_FILE_WRITE */
break;
@@ -1370,7 +1370,7 @@ int main(int argc, char *argv[]) {
alloc_pages = backed_pages;
}
} else {
/* LY: DB may be shrinked by writer down to the allocated pages. */
/* LY: DB may be shrunk by writer down to the allocated pages. */
if (alloc_pages > backed_pages) {
print(" ! alloc-pages %" PRIu64 " > backed-pages %" PRIu64 "\n",
alloc_pages, backed_pages);

View File

@@ -129,7 +129,7 @@
/** Controls sort order of internal page number lists.
* This mostly experimental/advanced option with not for regular MDBX users.
* \warning The database format depend on this option and libmdbx builded with
* \warning The database format depend on this option and libmdbx built with
* different option value are incompatible. */
#ifndef MDBX_PNL_ASCENDING
#define MDBX_PNL_ASCENDING 0
@@ -237,7 +237,10 @@
/** Advanced: Using POSIX OFD-locks (autodetection by default). */
#ifndef MDBX_USE_OFDLOCKS
#if defined(F_OFD_SETLK) && defined(F_OFD_SETLKW) && defined(F_OFD_GETLK) && \
#if ((defined(F_OFD_SETLK) && defined(F_OFD_SETLKW) && \
defined(F_OFD_GETLK)) || \
(defined(F_OFD_SETLK64) && defined(F_OFD_SETLKW64) && \
defined(F_OFD_GETLK64))) && \
!defined(MDBX_SAFE4QEMU) && \
!defined(__sun) /* OFD-lock are broken on Solaris */
#define MDBX_USE_OFDLOCKS 1
@@ -323,13 +326,7 @@
#endif /* MDBX_64BIT_ATOMIC */
#ifndef MDBX_64BIT_CAS
#if defined(ATOMIC_LLONG_LOCK_FREE)
#if ATOMIC_LLONG_LOCK_FREE > 1
#define MDBX_64BIT_CAS 1
#else
#define MDBX_64BIT_CAS 0
#endif
#elif defined(__GCC_ATOMIC_LLONG_LOCK_FREE)
#if defined(__GCC_ATOMIC_LLONG_LOCK_FREE)
#if __GCC_ATOMIC_LLONG_LOCK_FREE > 1
#define MDBX_64BIT_CAS 1
#else
@@ -341,6 +338,12 @@
#else
#define MDBX_64BIT_CAS 0
#endif
#elif defined(ATOMIC_LLONG_LOCK_FREE)
#if ATOMIC_LLONG_LOCK_FREE > 1
#define MDBX_64BIT_CAS 1
#else
#define MDBX_64BIT_CAS 0
#endif
#elif defined(_MSC_VER) || defined(__APPLE__) || defined(DOXYGEN)
#define MDBX_64BIT_CAS 1
#else

View File

@@ -527,14 +527,12 @@ static const DWORD WC_ERR_INVALID_CHARS =
: 0;
#endif /* WC_ERR_INVALID_CHARS */
MDBX_INTERNAL_FUNC size_t mdbx_mb2w(wchar_t *dst, size_t dst_n, const char *src,
size_t src_n) {
size_t mdbx_mb2w(wchar_t *dst, size_t dst_n, const char *src, size_t src_n) {
return MultiByteToWideChar(CP_THREAD_ACP, MB_ERR_INVALID_CHARS, src,
(int)src_n, dst, (int)dst_n);
}
MDBX_INTERNAL_FUNC size_t mdbx_w2mb(char *dst, size_t dst_n, const wchar_t *src,
size_t src_n) {
size_t mdbx_w2mb(char *dst, size_t dst_n, const wchar_t *src, size_t src_n) {
return WideCharToMultiByte(CP_THREAD_ACP, WC_ERR_INVALID_CHARS, src,
(int)src_n, dst, (int)dst_n, nullptr, nullptr);
}
@@ -694,7 +692,7 @@ MDBX_INTERNAL_FUNC int mdbx_openfile(const enum mdbx_openfile_purpose purpose,
flags |= O_CLOEXEC;
#endif /* O_CLOEXEC */
/* Safeguard for todo4recovery://erased_by_github/libmdbx/issues/144 */
/* Safeguard for https://libmdbx.dqdkfa.ru/dead-github/issues/144 */
#if STDIN_FILENO == 0 && STDOUT_FILENO == 1 && STDERR_FILENO == 2
int stub_fd0 = -1, stub_fd1 = -1, stub_fd2 = -1;
static const char dev_null[] = "/dev/null";
@@ -732,7 +730,7 @@ MDBX_INTERNAL_FUNC int mdbx_openfile(const enum mdbx_openfile_purpose purpose,
errno = EACCES /* restore errno if file exists */;
}
/* Safeguard for todo4recovery://erased_by_github/libmdbx/issues/144 */
/* Safeguard for https://libmdbx.dqdkfa.ru/dead-github/issues/144 */
#if STDIN_FILENO == 0 && STDOUT_FILENO == 1 && STDERR_FILENO == 2
if (*fd == STDIN_FILENO) {
mdbx_warning("Got STD%s_FILENO/%d, avoid using it by dup(fd)", "IN",
@@ -1549,6 +1547,7 @@ MDBX_INTERNAL_FUNC int mdbx_mmap(const int flags, mdbx_mmap_t *map,
map->limit = 0;
map->current = 0;
map->address = nullptr;
assert(errno != 0);
return errno;
}
map->limit = limit;
@@ -1574,8 +1573,7 @@ MDBX_INTERNAL_FUNC int mdbx_munmap(mdbx_mmap_t *map) {
VALGRIND_MAKE_MEM_NOACCESS(map->address, map->current);
/* Unpoisoning is required for ASAN to avoid false-positive diagnostic
* when this memory will re-used by malloc or another mmapping.
* See todo4recovery://erased_by_github/libmdbx/pull/93#issuecomment-613687203
*/
* See https://libmdbx.dqdkfa.ru/dead-github/pull/93#issuecomment-613687203 */
MDBX_ASAN_UNPOISON_MEMORY_REGION(map->address,
(map->filesize && map->filesize < map->limit)
? map->filesize
@@ -1587,8 +1585,10 @@ MDBX_INTERNAL_FUNC int mdbx_munmap(mdbx_mmap_t *map) {
if (!NT_SUCCESS(rc))
ntstatus2errcode(rc);
#else
if (unlikely(munmap(map->address, map->limit)))
if (unlikely(munmap(map->address, map->limit))) {
assert(errno != 0);
return errno;
}
#endif /* ! Windows */
map->limit = 0;
@@ -1651,8 +1651,7 @@ MDBX_INTERNAL_FUNC int mdbx_mresize(const int flags, mdbx_mmap_t *map,
/* Unpoisoning is required for ASAN to avoid false-positive diagnostic
* when this memory will re-used by malloc or another mmapping.
* See todo4recovery://erased_by_github/libmdbx/pull/93#issuecomment-613687203
*/
* See https://libmdbx.dqdkfa.ru/dead-github/pull/93#issuecomment-613687203 */
MDBX_ASAN_UNPOISON_MEMORY_REGION(map->address, map->limit);
status = NtUnmapViewOfSection(GetCurrentProcess(), map->address);
if (!NT_SUCCESS(status))
@@ -1811,8 +1810,10 @@ retry_mapview:;
if (limit < map->limit) {
/* unmap an excess at end of mapping. */
// coverity[offset_free : FALSE]
if (unlikely(munmap(map->dxb + limit, map->limit - limit)))
if (unlikely(munmap(map->dxb + limit, map->limit - limit))) {
assert(errno != 0);
return errno;
}
map->limit = limit;
return rc;
}
@@ -1824,14 +1825,19 @@ retry_mapview:;
assert(limit > map->limit);
uint8_t *ptr = MAP_FAILED;
#if defined(MREMAP_MAYMOVE)
#if (defined(__linux__) || defined(__gnu_linux__)) && defined(_GNU_SOURCE)
ptr = mremap(map->address, map->limit, limit,
(flags & MDBX_MRESIZE_MAY_MOVE) ? MREMAP_MAYMOVE : 0);
#if defined(MREMAP_MAYMOVE)
(flags & MDBX_MRESIZE_MAY_MOVE) ? MREMAP_MAYMOVE :
#endif /* MREMAP_MAYMOVE */
0);
if (ptr == MAP_FAILED) {
err = errno;
assert(err != 0);
switch (err) {
default:
return err;
case 0 /* paranoia */:
case EAGAIN:
case ENOMEM:
return MDBX_UNABLE_EXTEND_MAPSIZE;
@@ -1839,7 +1845,7 @@ retry_mapview:;
break;
}
}
#endif /* MREMAP_MAYMOVE */
#endif /* Linux & _GNU_SOURCE */
const unsigned mmap_flags =
MAP_CONCEAL | MAP_SHARED | MAP_FILE | MAP_NORESERVE |
@@ -1852,17 +1858,22 @@ retry_mapview:;
ptr = mmap(map->dxb + map->limit, limit - map->limit, mmap_prot,
mmap_flags | MAP_FIXED_NOREPLACE, map->fd, map->limit);
if (ptr == map->dxb + map->limit)
/* успешно прилепили отображение в конец */
ptr = map->dxb;
else if (ptr != MAP_FAILED) {
/* the desired address is busy, unmap unsuitable one */
if (unlikely(munmap(ptr, limit - map->limit)))
if (unlikely(munmap(ptr, limit - map->limit))) {
assert(errno != 0);
return errno;
}
ptr = MAP_FAILED;
} else {
err = errno;
assert(err != 0);
switch (err) {
default:
return err;
case 0 /* paranoia */:
case EAGAIN:
case ENOMEM:
return MDBX_UNABLE_EXTEND_MAPSIZE;
@@ -1881,8 +1892,10 @@ retry_mapview:;
return MDBX_UNABLE_EXTEND_MAPSIZE;
}
if (unlikely(munmap(map->address, map->limit)))
if (unlikely(munmap(map->address, map->limit))) {
assert(errno != 0);
return errno;
}
// coverity[pass_freed_arg : FALSE]
ptr = mmap(map->address, limit, mmap_prot,
@@ -1917,15 +1930,14 @@ retry_mapview:;
VALGRIND_MAKE_MEM_NOACCESS(map->address, map->current);
/* Unpoisoning is required for ASAN to avoid false-positive diagnostic
* when this memory will re-used by malloc or another mmapping.
* See
* todo4recovery://erased_by_github/libmdbx/pull/93#issuecomment-613687203
*/
* See https://libmdbx.dqdkfa.ru/dead-github/pull/93#issuecomment-613687203 */
MDBX_ASAN_UNPOISON_MEMORY_REGION(
map->address,
(map->current < map->limit) ? map->current : map->limit);
map->limit = 0;
map->current = 0;
map->address = nullptr;
assert(errno != 0);
return errno;
}
rc = MDBX_UNABLE_EXTEND_MAPSIZE;
@@ -1938,9 +1950,7 @@ retry_mapview:;
VALGRIND_MAKE_MEM_NOACCESS(map->address, map->current);
/* Unpoisoning is required for ASAN to avoid false-positive diagnostic
* when this memory will re-used by malloc or another mmapping.
* See
* todo4recovery://erased_by_github/libmdbx/pull/93#issuecomment-613687203
*/
* See https://libmdbx.dqdkfa.ru/dead-github/pull/93#issuecomment-613687203 */
MDBX_ASAN_UNPOISON_MEMORY_REGION(
map->address, (map->current < map->limit) ? map->current : map->limit);
@@ -1952,8 +1962,10 @@ retry_mapview:;
#if MDBX_ENABLE_MADVISE
#ifdef MADV_DONTFORK
if (unlikely(madvise(map->address, map->limit, MADV_DONTFORK) != 0))
if (unlikely(madvise(map->address, map->limit, MADV_DONTFORK) != 0)) {
assert(errno != 0);
return errno;
}
#endif /* MADV_DONTFORK */
#ifdef MADV_NOHUGEPAGE
(void)madvise(map->address, map->limit, MADV_NOHUGEPAGE);
@@ -1962,6 +1974,9 @@ retry_mapview:;
#endif /* POSIX / Windows */
assert(rc != MDBX_SUCCESS ||
(map->address != nullptr && map->address != MAP_FAILED &&
map->current == size && map->limit == limit));
return rc;
}

View File

@@ -180,10 +180,8 @@ static inline void mdbx_free(void *ptr) { HeapFree(GetProcessHeap(), 0, ptr); }
#define vsnprintf _vsnprintf /* ntdll */
#endif
MDBX_INTERNAL_FUNC size_t mdbx_mb2w(wchar_t *dst, size_t dst_n, const char *src,
size_t src_n);
MDBX_INTERNAL_FUNC size_t mdbx_w2mb(char *dst, size_t dst_n, const wchar_t *src,
size_t src_n);
size_t mdbx_mb2w(wchar_t *dst, size_t dst_n, const char *src, size_t src_n);
size_t mdbx_w2mb(char *dst, size_t dst_n, const wchar_t *src, size_t src_n);
#else /*----------------------------------------------------------------------*/
@@ -289,7 +287,48 @@ MDBX_MAYBE_UNUSED static __inline void mdbx_jitter4testing(bool tiny);
#if defined(_WIN32) || defined(_WIN64)
#define MAX_WRITE UINT32_C(0x01000000)
#else
#define MAX_WRITE UINT32_C(0x3fff0000)
#define MAX_WRITE UINT32_C(0x3f000000)
#if defined(F_GETLK64) && defined(F_SETLK64) && defined(F_SETLKW64) && \
!defined(__ANDROID_API__)
#define MDBX_F_SETLK F_SETLK64
#define MDBX_F_SETLKW F_SETLKW64
#define MDBX_F_GETLK F_GETLK64
#if (__GLIBC_PREREQ(2, 28) && \
(defined(__USE_LARGEFILE64) || defined(__LARGEFILE64_SOURCE) || \
defined(_USE_LARGEFILE64) || defined(_LARGEFILE64_SOURCE))) || \
defined(fcntl64)
#define MDBX_FCNTL fcntl64
#else
#define MDBX_FCNTL fcntl
#endif
#define MDBX_STRUCT_FLOCK struct flock64
#ifndef OFF_T_MAX
#define OFF_T_MAX UINT64_C(0x7fffFFFFfff00000)
#endif /* OFF_T_MAX */
#else
#define MDBX_F_SETLK F_SETLK
#define MDBX_F_SETLKW F_SETLKW
#define MDBX_F_GETLK F_GETLK
#define MDBX_FCNTL fcntl
#define MDBX_STRUCT_FLOCK struct flock
#endif /* MDBX_F_SETLK, MDBX_F_SETLKW, MDBX_F_GETLK */
#if defined(F_OFD_SETLK64) && defined(F_OFD_SETLKW64) && \
defined(F_OFD_GETLK64) && !defined(__ANDROID_API__)
#define MDBX_F_OFD_SETLK F_OFD_SETLK64
#define MDBX_F_OFD_SETLKW F_OFD_SETLKW64
#define MDBX_F_OFD_GETLK F_OFD_GETLK64
#else
#define MDBX_F_OFD_SETLK F_OFD_SETLK
#define MDBX_F_OFD_SETLKW F_OFD_SETLKW
#define MDBX_F_OFD_GETLK F_OFD_GETLK
#ifndef OFF_T_MAX
#define OFF_T_MAX \
(((sizeof(off_t) > 4) ? INT64_MAX : INT32_MAX) & ~(size_t)0xFffff)
#endif /* OFF_T_MAX */
#endif /* MDBX_F_OFD_SETLK64, MDBX_F_OFD_SETLKW64, MDBX_F_OFD_GETLK64 */
#endif
#if defined(__linux__) || defined(__gnu_linux__)
@@ -500,7 +539,7 @@ MDBX_INTERNAL_FUNC int mdbx_lck_destroy(MDBX_env *env,
MDBX_INTERNAL_FUNC int mdbx_lck_seize(MDBX_env *env);
/// \brief Downgrades the level of initially acquired lock to
/// operational level specified by argument. The reson for such downgrade:
/// operational level specified by argument. The reason for such downgrade:
/// - unblocking of other processes that are waiting for access, i.e.
/// if (env->me_flags & MDBX_EXCLUSIVE) != 0, then other processes
/// should be made aware that access is unavailable rather than

View File

@@ -54,7 +54,11 @@ if(NOT MDBX_BUILD_CXX AND LIBCXX_FILESYSTEM)
target_link_libraries(mdbx_test ${LIBCXX_FILESYSTEM})
endif()
target_link_libraries(mdbx_test ${TOOL_MDBX_LIB} ${LIB_MATH} ${CMAKE_THREAD_LIBS_INIT})
if(CMAKE_VERSION VERSION_LESS 3.1)
target_link_libraries(mdbx_test ${TOOL_MDBX_LIB} ${LIB_MATH} ${CMAKE_THREAD_LIBS_INIT})
else()
target_link_libraries(mdbx_test ${TOOL_MDBX_LIB} ${LIB_MATH} Threads::Threads)
endif()
if(${CMAKE_SYSTEM_NAME} STREQUAL "Windows")
target_link_libraries(mdbx_test winmm.lib)
endif()

View File

@@ -1,8 +1,4 @@
#!/usr/bin/env bash
if ! which make cc c++ tee >/dev/null; then
echo "Please install the following prerequisites: make cc c++ tee banner" >&2
exit 1
fi
LIST=basic
FROM=1
@@ -15,6 +11,7 @@ BANNER="$(which banner 2>/dev/null | echo echo)"
UNAME="$(uname -s 2>/dev/null || echo Unknown)"
DB_UPTO_MB=17408
PAGESIZE=min
DONT_CHECK_RAM=no
while [ -n "$1" ]
do
@@ -35,6 +32,7 @@ do
echo "--db-upto-mb NN Limits upper size of test DB to the NN megabytes"
echo "--no-geometry-jitter Disable jitter for geometry upper-size"
echo "--pagesize NN Use specified page size (256 is minimal and used by default) "
echo "--dont-check-ram-size Don't check available RAM "
echo "--help Print this usage help and exit"
exit -2
;;
@@ -111,7 +109,7 @@ do
--no-geometry-jitter)
GEOMETRY_JITTER=no
;;
--pagesize)
--pagesize|--page-size)
case "$2" in
min|max|256|512|1024|2048|4096|8192|16386|32768|65536)
PAGESIZE=$2
@@ -144,6 +142,9 @@ do
esac
shift
;;
--dont-check-ram-size)
DONT_CHECK_RAM=yes
;;
*)
echo "Unknown option '$1'"
exit -2
@@ -163,6 +164,11 @@ if [ -z "$MONITOR" ]; then
export MALLOC_CHECK_=7 MALLOC_PERTURB_=42
fi
if ! which $([ "$SKIP_MAKE" == "no" ] && echo make cc c++) tee >/dev/null; then
echo "Please install the following prerequisites: make cc c++ tee banner" >&2
exit 1
fi
###############################################################################
# 1. clean data from prev runs and examine available RAM
@@ -220,6 +226,19 @@ case ${UNAME} in
echo "pagesize ${pagesize}K, freepages ${freepages}, ram_avail_mb ${ram_avail_mb}"
;;
MSYS*|MINGW*)
if [ -z "${TESTDB_DIR:-}" ]; then
for old_test_dir in $(ls -d /tmp/mdbx-test.[0-9]* 2>/dev/null); do
rm -rf $old_test_dir
done
TESTDB_DIR="/tmp/mdbx-test.$$"
fi
mkdir -p $TESTDB_DIR && rm -f $TESTDB_DIR/*
echo "FIXME: Fake support for ${UNAME}"
ram_avail_mb=32768
;;
*)
echo "FIXME: ${UNAME} not supported by this script"
exit 2
@@ -232,11 +251,15 @@ rm -f ${TESTDB_DIR}/*
# 2. estimate reasonable RAM space for test-db
echo "=== ${ram_avail_mb}M RAM available"
ram_reserve4logs_mb=1234
if [ $ram_avail_mb -lt $ram_reserve4logs_mb ]; then
echo "=== At least ${ram_reserve4logs_mb}Mb RAM required"
exit 3
fi
if [ $DONT_CHECK_RAM = yes ]; then
db_size_mb=$DB_UPTO_MB
ram_reserve4logs_mb=64
else
ram_reserve4logs_mb=1234
if [ $ram_avail_mb -lt $ram_reserve4logs_mb ]; then
echo "=== At least ${ram_reserve4logs_mb}Mb RAM required"
exit 3
fi
#
# В режимах отличных от MDBX_WRITEMAP изменения до записи в файл
@@ -256,9 +279,10 @@ fi
# that malloc() will not return the allocated memory to the
# system immediately, as well some space is required for logs.
#
db_size_mb=$(((ram_avail_mb - ram_reserve4logs_mb) / 4))
if [ $db_size_mb -gt $DB_UPTO_MB ]; then
db_size_mb=$DB_UPTO_MB
db_size_mb=$(((ram_avail_mb - ram_reserve4logs_mb) / 4))
if [ $db_size_mb -gt $DB_UPTO_MB ]; then
db_size_mb=$DB_UPTO_MB
fi
fi
echo "=== use ${db_size_mb}M for DB"
@@ -269,7 +293,11 @@ case ${UNAME} in
ulimit -c unlimited
if [ "$(cat /proc/sys/kernel/core_pattern)" != "core.%p" ]; then
echo "core.%p > /proc/sys/kernel/core_pattern" >&2
echo "core.%p" | sudo tee /proc/sys/kernel/core_pattern || true
if [ $(id -u) -ne 0 -a -n "$(which sudo 2>/dev/null)" ]; then
echo "core.%p" | sudo tee /proc/sys/kernel/core_pattern || true
else
(echo "core.%p" > /proc/sys/kernel/core_pattern) || true
fi
fi
;;
@@ -288,6 +316,10 @@ case ${UNAME} in
fi
;;
MSYS*|MINGW*)
echo "FIXME: Fake support for ${UNAME}"
;;
*)
echo "FIXME: ${UNAME} not supported by this script"
exit 2
@@ -319,7 +351,7 @@ else
fi
syncmodes=("" ,+nosync-safe ,+nosync-utterly)
options=(writemap coalesce lifo notls perturb)
options=(writemap lifo notls perturb)
function join { local IFS="$1"; shift; echo "$*"; }

View File

@@ -24,7 +24,7 @@ MDBX_NORETURN void usage(void) {
"usage:\n"
" --help or -h Show this text\n"
"Common parameters:\n"
" --loglevel=[0-7]|[extra..fatal]"
" --loglevel=[0-7]|[fatal..extra]s"
" --pathname=... Path and/or name of database files\n"
" --repeat=N Set repeat counter\n"
" --threads=N Number of thread (unsupported for now)\n"

View File

@@ -360,6 +360,7 @@ actor_status osal_actor_info(const mdbx_pid_t pid) {
status = as_coredump;
break;
default:
log_error("pid %zu, exit code %u", (intptr_t)pid, (unsigned)ExitCode);
status = as_failed;
break;
}

View File

@@ -1,18 +1,17 @@
#!/usr/bin/env bash
if ! which make cc c++ tee >/dev/null; then
echo "Please install the following prerequisites: make cc c++ tee banner" >&2
exit 1
fi
LIST=--hill
FROM=1
UPTO=9999
UPTO=9999999
MONITOR=
LOOPS=
SKIP_MAKE=no
GEOMETRY_JITTER=yes
BANNER="$(which banner 2>/dev/null | echo echo)"
UNAME="$(uname -s 2>/dev/null || echo Unknown)"
DB_UPTO_MB=17408
PAGESIZE=min
DONT_CHECK_RAM=no
while [ -n "$1" ]
do
@@ -20,6 +19,10 @@ do
--help)
echo "--multi Engage multi-process test scenario (default)"
echo "--single Execute series of single-process tests (for QEMU, etc)"
echo "--nested Execute only 'nested' testcase"
echo "--hill Execute only 'hill' testcase"
echo "--append Execute only 'append' testcase"
echo "--ttl Execute only 'ttl' testcase"
echo "--with-valgrind Run tests under Valgrind's memcheck tool"
echo "--skip-make Don't (re)build libmdbx and test's executable"
echo "--from NN Start iterating from the NN ops per test case"
@@ -27,6 +30,9 @@ do
echo "--loops NN Stop after the NN loops"
echo "--dir PATH Specifies directory for test DB and other files (it will be cleared)"
echo "--db-upto-mb NN Limits upper size of test DB to the NN megabytes"
echo "--no-geometry-jitter Disable jitter for geometry upper-size"
echo "--pagesize NN Use specified page size (256 is minimal and used by default) "
echo "--dont-check-ram-size Don't check available RAM "
echo "--help Print this usage help and exit"
exit -2
;;
@@ -36,6 +42,18 @@ do
--single)
LIST="--nested --hill --append --ttl --copy"
;;
--nested)
LIST="--nested"
;;
--hill)
LIST="--hill"
;;
--append)
LIST="--append"
;;
--ttl)
LIST="--ttl"
;;
--with-valgrind)
echo " NOTE: Valgrind could produce some false-positive warnings"
echo " in multi-process environment with shared memory."
@@ -88,6 +106,45 @@ do
fi
shift
;;
--no-geometry-jitter)
GEOMETRY_JITTER=no
;;
--pagesize|--page-size)
case "$2" in
min|max|256|512|1024|2048|4096|8192|16386|32768|65536)
PAGESIZE=$2
;;
1|1k|1K|k|K)
PAGESIZE=$((1024*1))
;;
2|2k|2K)
PAGESIZE=$((1024*2))
;;
4|4k|4K)
PAGESIZE=$((1024*4))
;;
8|8k|8K)
PAGESIZE=$((1024*8))
;;
16|16k|16K)
PAGESIZE=$((1024*16))
;;
32|32k|32K)
PAGESIZE=$((1024*32))
;;
64|64k|64K)
PAGESIZE=$((1024*64))
;;
*)
echo "Invalig page size '$2'"
exit -2
;;
esac
shift
;;
--dont-check-ram-size)
DONT_CHECK_RAM=yes
;;
*)
echo "Unknown option '$1'"
exit -2
@@ -107,6 +164,11 @@ if [ -z "$MONITOR" ]; then
export MALLOC_CHECK_=7 MALLOC_PERTURB_=42
fi
if ! which $([ "$SKIP_MAKE" == "no" ] && echo make cc c++) tee >/dev/null; then
echo "Please install the following prerequisites: make cc c++ tee banner" >&2
exit 1
fi
###############################################################################
# 1. clean data from prev runs and examine available RAM
@@ -123,9 +185,9 @@ case ${UNAME} in
mkdir -p $TESTDB_DIR && rm -f $TESTDB_DIR/*
if LC_ALL=C free | grep -q -i available; then
ram_avail_mb=$(($(LC_ALL=C free | grep -i Mem: | tr -s [:blank:] ' ' | cut -d ' ' -f 7) / 1024))
ram_avail_mb=$(($(LC_ALL=C free | grep -i Mem: | tr -s '[:blank:]' ' ' | cut -d ' ' -f 7) / 1024))
else
ram_avail_mb=$(($(LC_ALL=C free | grep -i Mem: | tr -s [:blank:] ' ' | cut -d ' ' -f 4) / 1024))
ram_avail_mb=$(($(LC_ALL=C free | grep -i Mem: | tr -s '[:blank:]' ' ' | cut -d ' ' -f 4) / 1024))
fi
;;
@@ -164,6 +226,19 @@ case ${UNAME} in
echo "pagesize ${pagesize}K, freepages ${freepages}, ram_avail_mb ${ram_avail_mb}"
;;
MSYS*|MINGW*)
if [ -z "${TESTDB_DIR:-}" ]; then
for old_test_dir in $(ls -d /tmp/mdbx-test.[0-9]* 2>/dev/null); do
rm -rf $old_test_dir
done
TESTDB_DIR="/tmp/mdbx-test.$$"
fi
mkdir -p $TESTDB_DIR && rm -f $TESTDB_DIR/*
echo "FIXME: Fake support for ${UNAME}"
ram_avail_mb=32768
;;
*)
echo "FIXME: ${UNAME} not supported by this script"
exit 2
@@ -176,11 +251,15 @@ rm -f ${TESTDB_DIR}/*
# 2. estimate reasonable RAM space for test-db
echo "=== ${ram_avail_mb}M RAM available"
ram_reserve4logs_mb=1234
if [ $ram_avail_mb -lt $ram_reserve4logs_mb ]; then
echo "=== At least ${ram_reserve4logs_mb}Mb RAM required"
exit 3
fi
if [ $DONT_CHECK_RAM = yes ]; then
db_size_mb=$DB_UPTO_MB
ram_reserve4logs_mb=64
else
ram_reserve4logs_mb=1234
if [ $ram_avail_mb -lt $ram_reserve4logs_mb ]; then
echo "=== At least ${ram_reserve4logs_mb}Mb RAM required"
exit 3
fi
#
# В режимах отличных от MDBX_WRITEMAP изменения до записи в файл
@@ -200,9 +279,10 @@ fi
# that malloc() will not return the allocated memory to the
# system immediately, as well some space is required for logs.
#
db_size_mb=$(((ram_avail_mb - ram_reserve4logs_mb) / 4))
if [ $db_size_mb -gt $DB_UPTO_MB ]; then
db_size_mb=$DB_UPTO_MB
db_size_mb=$(((ram_avail_mb - ram_reserve4logs_mb) / 4))
if [ $db_size_mb -gt $DB_UPTO_MB ]; then
db_size_mb=$DB_UPTO_MB
fi
fi
echo "=== use ${db_size_mb}M for DB"
@@ -213,7 +293,11 @@ case ${UNAME} in
ulimit -c unlimited
if [ "$(cat /proc/sys/kernel/core_pattern)" != "core.%p" ]; then
echo "core.%p > /proc/sys/kernel/core_pattern" >&2
echo "core.%p" | sudo tee /proc/sys/kernel/core_pattern || true
if [ $(id -u) -ne 0 -a -n "$(which sudo 2>/dev/null)" ]; then
echo "core.%p" | sudo tee /proc/sys/kernel/core_pattern || true
else
(echo "core.%p" > /proc/sys/kernel/core_pattern) || true
fi
fi
;;
@@ -232,6 +316,10 @@ case ${UNAME} in
fi
;;
MSYS*|MINGW*)
echo "FIXME: Fake support for ${UNAME}"
;;
*)
echo "FIXME: ${UNAME} not supported by this script"
exit 2
@@ -263,7 +351,7 @@ else
fi
syncmodes=("" ,+nosync-safe ,+nosync-utterly)
options=(writemap coalesce lifo notls perturb)
options=(writemap lifo notls perturb)
function join { local IFS="$1"; shift; echo "$*"; }
@@ -296,8 +384,8 @@ function probe {
rm -f ${TESTDB_DIR}/* || failed
for case in $LIST
do
echo "Run ./mdbx_test ${speculum} --random-writemap=no --ignore-dbfull --repeat=1 --pathname=${TESTDB_DIR}/long.db --cleanup-after=no $@ $case"
${MONITOR} ./mdbx_test ${speculum} --random-writemap=no --ignore-dbfull --repeat=1 --pathname=${TESTDB_DIR}/long.db --cleanup-before=yes --cleanup-after=no "$@" $case | check_deep \
echo "Run ./mdbx_test ${speculum} --random-writemap=no --ignore-dbfull --repeat=11 --pathname=${TESTDB_DIR}/long.db --cleanup-after=no --geometry-jitter=${GEOMETRY_JITTER} $@ $case"
${MONITOR} ./mdbx_test ${speculum} --random-writemap=no --ignore-dbfull --repeat=11 --pathname=${TESTDB_DIR}/long.db --cleanup-after=no --geometry-jitter=${GEOMETRY_JITTER} "$@" $case | check_deep \
&& ${MONITOR} ./mdbx_chk ${TESTDB_DIR}/long.db | tee ${TESTDB_DIR}/long-chk.log \
&& ([ ! -e ${TESTDB_DIR}/long.db-copy ] || ${MONITOR} ./mdbx_chk ${TESTDB_DIR}/long.db-copy | tee ${TESTDB_DIR}/long-chk-copy.log) \
|| failed

View File

@@ -31,7 +31,7 @@
fun:mdbx_wipe_steady*
}
# memcmp() inside mdbx_iov_write() as workaround for todo4recovery://erased_by_github/libmdbx/issues/269
# memcmp() inside mdbx_iov_write() as workaround for https://libmdbx.dqdkfa.ru/dead-github/issues/269
{
write-page-check-bcmp
Memcheck:Cond