Compare commits

...

29 Commits

Author SHA1 Message Date
Леонид Юрьев (Leonid Yuriev)
d6b47c7bd1 mdbx: release v0.11.13 (Swashplate)
The stable bugfix release in memory of [Boris Yuryev](https://ru.wikipedia.org/wiki/Юрьев,_Борис_Николаевич) on his 133rd birthday.

Fixes:
------

 - Fixed builds with older libc versions after using `fcntl64()` (backport).
 - Fixed builds with  older `stdatomic.h` versions,
   where the `ATOMIC_*_LOCK_FREE` macros mistakenly redefined using functions (backport).
 - Added workaround for `mremap()` defect to avoid assertion failure (backport).
 - Workaround for `encryptfs` bug(s) in the `copy_file_range` implementation  (backport).
 - Fixed unexpected `MDBX_BUSY` from `mdbx_env_set_option()`, `mdbx_env_set_syncbytes()`
   and `mdbx_env_set_syncperiod()` (backport).
 - CMake requirements lowered to version 3.0.2 (backport).
 - Added admonition of insecure for RISC-V (backport).

Minors:
-------

 - Minor clarification output of `--help` for `mdbx_test` (backport).
 - Added admonition of insecure for RISC-V (backport).
 - Stochastic scripts and CMake files synchronized with the `devel` branch.
 - Use `--dont-check-ram-size` for small-tests make-targets (backport).

30 files changed, 405 insertions(+), 136 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2022-11-10 13:07:28 +03:00
Леонид Юрьев (Leonid Yuriev)
8f0c5bc7c7 mdbx: update ChangeLog. 2022-11-10 12:11:11 +03:00
Леонид Юрьев (Leonid Yuriev)
29eab4afdd mdbx-make: use --dont-check-ram-size for small-tests targets (backport). 2022-11-10 12:09:34 +03:00
Леонид Юрьев (Leonid Yuriev)
12717aac8c mdbx-test: sync stochastic scripts with devel branch. 2022-11-10 10:57:16 +03:00
Леонид Юрьев (Leonid Yuriev)
e992da9efe mdbx-cmake: синхронизация CMake-файлов. 2022-11-10 01:10:39 +03:00
Леонид Юрьев (Leonid Yuriev)
d7e4cb2e22 mdbx: add admonition of insecure for RISC-V (backport). 2022-11-10 01:10:39 +03:00
Леонид Юрьев (Leonid Yuriev)
af1d01ffb3 mdbx-windows: уточнение проверок макросов MinGW и устранение предупреждений (backport). 2022-11-10 01:10:39 +03:00
Леонид Юрьев (Leonid Yuriev)
cce052e869 mdbx-cmake: совместимость с CMake 3.0.2 для CI на старых системах (backport). 2022-11-10 00:48:54 +03:00
Леонид Юрьев (Leonid Yuriev)
094c2f345d mdbx: переименование packages/rpm (backport). 2022-11-10 00:48:54 +03:00
Леонид Юрьев (Leonid Yuriev)
7b2eee91af mdbx: добавлено описание параметров MDBX_debug_func и MDBX_debug_func (backport). 2022-11-10 00:48:54 +03:00
Леонид Юрьев (Leonid Yuriev)
d863629387 mdbx-test: минорное уточнение подсказки --help (backport). 2022-11-10 00:48:54 +03:00
Леонид Юрьев (Leonid Yuriev)
1b3b6e4479 mdbx: исправление неожиданного MDBX_BUSY из mdbx_env_set_option() (backport). 2022-11-10 00:48:54 +03:00
Леонид Юрьев (Leonid Yuriev)
5dcc0171fa mdbx-windows: попытка борьбы с ложно-положительными конфликтами LockFileEx() (backport). 2022-11-10 00:48:54 +03:00
Леонид Юрьев (Leonid Yuriev)
425730c2b3 mdbx: чуток апостофов для имен в doxygen-комментариях (backport). 2022-11-07 16:44:00 +03:00
Леонид Юрьев (Leonid Yuriev)
471e854551 mdbx: workaround for encryptfs's copy_file_range() bug(s) (backport).
Выяснилось что утилита `mdbx_copy` и функции `mdbx_env_copy()` могут
создавать ПРОБЛЕМЫ если целевой файл расположен в encryptfs (такая
файловая система в Linux).

При этом может быть четыре исхода в зависимости от версии ядра и
положения звезд на небе:
 - всё хорошо;
 - плохие данные в копии без возврата ошибок;
 - ошибка EINVAL(22) при копировании;
 - oops или зависание ядра, отвал смонтированной encryptfs и т.п.

В текущем понимании, причина обусловлена ошибой в коде fs, которая
проявляется при использовании системного вызова `copy_file_range`.
2022-11-07 16:44:00 +03:00
Леонид Юрьев (Leonid Yuriev)
1bd0eb35bc mdbx-cmake: исправление для совместимости с CMake 3.8 (backport). 2022-11-07 16:43:32 +03:00
Леонид Юрьев (Leonid Yuriev)
6fdae12996 mdbx: workaround for mremap() defect (backport).
Есть основание полагать, что mremap() может возвращать MAP_FAILED, но НЕ
устанавливать errno в некоторых пограничных ситуациях. Например, когда
системных ресурсов не хватает на актуализацию/копирование/клонирование
состояния отображения на финальной стадии, в том числе из-за раскраски
исходного отображения разными флагами через madvise().
2022-10-14 14:05:44 +03:00
Леонид Юрьев (Leonid Yuriev)
2684c89d91 mdbx: workaround for older stdatomic.h versions, where the ATOMIC_*_LOCK_FREE macros mistakenly redefined using functions (backport). 2022-10-14 14:05:16 +03:00
Леонид Юрьев (Leonid Yuriev)
7b60363a31 mdbx: fix builds with older libc versions after using fcntl64() (backport). 2022-10-14 14:01:02 +03:00
Леонид Юрьев (Leonid Yuriev)
290630f118 mdbx: release v0.11.12 (Эребуни)
The stable bugfix release.

Fixes:
------

 - Fixed static assertion failure on platforms where the `off_t` type is wider
   than corresponding fields of `struct flock` used for file locking (backport).
   Now _libmdbx_ will use `fcntl64(F_GETLK64/F_SETLK64/F_SETLKW64)` if available.
 - Fixed assertion check inside `page_retire_ex()` (backport).

Minors:
-------

 - Fixed `-Wint-to-pointer-cast` warnings while casting to `mdbx_tid_t` (backport).
 - Removed needless `LockFileEx()` inside `mdbx_env_copy()` (backport).

11 files changed, 96 insertions(+), 49 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2022-10-12 21:11:47 +03:00
Леонид Юрьев (Leonid Yuriev)
bdcc345455 mdbx: minor fix assertion check inside page_retire_ex() (backport). 2022-10-12 21:00:35 +03:00
Леонид Юрьев (Leonid Yuriev)
db9e2c6f07 mdbx: using fcntl64(F_GETLK64/F_SETLK64/F_SETLKW64) when available (backport).
This fixes issues (static assertion failure, etc) on platforms where the
`off_t` type is wider than corresponding fields of `struct flock`.
2022-10-12 20:59:05 +03:00
Леонид Юрьев (Leonid Yuriev)
50e9e0e561 mdbx-windows: drop needless LockFileEx() inside mdbx_env_copy() (backport). 2022-10-12 18:32:56 +03:00
Леонид Юрьев (Leonid Yuriev)
8505203080 mdbx: avoid -Wint-to-pointer-cast warnings while casting to mdbx_tid_t (backport). 2022-10-12 18:32:49 +03:00
Леонид Юрьев (Leonid Yuriev)
50ba4bc2f2 mdbx: release v0.11.11 (Тендра-1790)
The stable bugfix release.
It is planned that this will be the last release of the v0.11 branch.

Fixes:
------

 - Fixed an extra check for `MDBX_APPENDDUP` inside `mdbx_cursor_put()` which could result in returning `MDBX_EKEYMISMATCH` for valid cases.
 - Fixed an extra ensure/assertion check of `oldest_reader` inside `mdbx_txn_end()`.
 - Fixed derived C++ builds by removing `MDBX_INTERNAL_FUNC` for `mdbx_w2mb()` and `mdbx_mb2w()`.

10 files changed, 38 insertions(+), 21 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2022-09-11 14:33:56 +03:00
Леонид Юрьев (Leonid Yuriev)
cf6576984d mdbx: update ChangeLog. 2022-09-11 13:08:21 +03:00
Леонид Юрьев (Leonid Yuriev)
4f3e1a60f1 mdbx: remove MDBX_INTERNAL_FUNC from mdbx_w2mb() and mdbx_mb2w() for C++ API. 2022-09-11 13:04:49 +03:00
Леонид Юрьев (Leonid Yuriev)
04f60af669 mdbx: minor fix extra ensure/assertion check of oldest_reader inside txn_end(). 2022-09-09 19:56:48 +03:00
Леонид Юрьев (Leonid Yuriev)
c6cd642ff1 mdbx: fix extra check for MDBX_APPENDDUP. 2022-09-09 19:48:24 +03:00
31 changed files with 530 additions and 195 deletions

View File

@@ -34,7 +34,9 @@
## The Future will (be) Positive. Всё будет хорошо.
##
if(CMAKE_VERSION VERSION_LESS 3.12)
if(CMAKE_VERSION VERSION_LESS 3.8.2)
cmake_minimum_required(VERSION 3.0.2)
elseif(CMAKE_VERSION VERSION_LESS 3.12)
cmake_minimum_required(VERSION 3.8.2)
else()
cmake_minimum_required(VERSION 3.12)
@@ -240,34 +242,42 @@ else()
option(BUILD_FOR_NATIVE_CPU "Generate code for the compiling machine CPU" OFF)
endif()
if(CMAKE_CONFIGURATION_TYPES OR NOT CMAKE_BUILD_TYPE_UPPERCASE STREQUAL "DEBUG")
set(INTERPROCEDURAL_OPTIMIZATION_DEFAULT ON)
else()
set(INTERPROCEDURAL_OPTIMIZATION_DEFAULT OFF)
endif()
if(CMAKE_INTERPROCEDURAL_OPTIMIZATION_AVAILABLE
OR GCC_LTO_AVAILABLE OR MSVC_LTO_AVAILABLE OR
(CLANG_LTO_AVAILABLE AND
((DEFINED MDBX_ENABLE_TESTS AND NOT MDBX_ENABLE_TESTS)
OR NOT CMAKE_CXX_COMPILER_VERSION VERSION_LESS 4.0)))
option(INTERPROCEDURAL_OPTIMIZATION "Enable interprocedural/LTO optimization" ${INTERPROCEDURAL_OPTIMIZATION_DEFAULT})
else()
set(INTERPROCEDURAL_OPTIMIZATION OFF)
OR GCC_LTO_AVAILABLE OR MSVC_LTO_AVAILABLE OR CLANG_LTO_AVAILABLE)
if((CMAKE_CONFIGURATION_TYPES OR NOT CMAKE_BUILD_TYPE_UPPERCASE STREQUAL "DEBUG") AND
((MSVC_LTO_AVAILABLE AND NOT CMAKE_C_COMPILER_VERSION VERSION_LESS 19) OR
(GCC_LTO_AVAILABLE AND NOT CMAKE_C_COMPILER_VERSION VERSION_LESS 7) OR
(CLANG_LTO_AVAILABLE AND NOT CMAKE_C_COMPILER_VERSION VERSION_LESS 5)))
set(INTERPROCEDURAL_OPTIMIZATION_DEFAULT ON)
else()
set(INTERPROCEDURAL_OPTIMIZATION_DEFAULT OFF)
endif()
option(INTERPROCEDURAL_OPTIMIZATION "Enable interprocedural/LTO optimization." ${INTERPROCEDURAL_OPTIMIZATION_DEFAULT})
endif()
if(INTERPROCEDURAL_OPTIMIZATION)
if(GCC_LTO_AVAILABLE)
set(LTO_ENABLED TRUE)
set(CMAKE_AR ${CMAKE_GCC_AR} CACHE PATH "Path to ar program with LTO-plugin" FORCE)
set(CMAKE_C_COMPILER_AR ${CMAKE_AR} CACHE PATH "Path to ar program with LTO-plugin" FORCE)
set(CMAKE_CXX_COMPILER_AR ${CMAKE_AR} CACHE PATH "Path to ar program with LTO-plugin" FORCE)
set(CMAKE_NM ${CMAKE_GCC_NM} CACHE PATH "Path to nm program with LTO-plugin" FORCE)
set(CMAKE_RANLIB ${CMAKE_GCC_RANLIB} CACHE PATH "Path to ranlib program with LTO-plugin" FORCE)
set(CMAKE_C_COMPILER_RANLIB ${CMAKE_RANLIB} CACHE PATH "Path to ranlib program with LTO-plugin" FORCE)
set(CMAKE_CXX_COMPILER_RANLIB ${CMAKE_RANLIB} CACHE PATH "Path to ranlib program with LTO-plugin" FORCE)
message(STATUS "MDBX indulge Link-Time Optimization by GCC")
elseif(CLANG_LTO_AVAILABLE)
set(LTO_ENABLED TRUE)
if(CMAKE_CLANG_LD)
set(CMAKE_LINKER ${CMAKE_CLANG_LD} CACHE PATH "Path to lld or ld program with LTO-plugin" FORCE)
endif()
set(CMAKE_AR ${CMAKE_CLANG_AR} CACHE PATH "Path to ar program with LTO-plugin" FORCE)
set(CMAKE_C_COMPILER_AR ${CMAKE_AR} CACHE PATH "Path to ar program with LTO-plugin" FORCE)
set(CMAKE_CXX_COMPILER_AR ${CMAKE_AR} CACHE PATH "Path to ar program with LTO-plugin" FORCE)
set(CMAKE_NM ${CMAKE_CLANG_NM} CACHE PATH "Path to nm program with LTO-plugin" FORCE)
set(CMAKE_RANLIB ${CMAKE_CLANG_RANLIB} CACHE PATH "Path to ranlib program with LTO-plugin" FORCE)
set(CMAKE_C_COMPILER_RANLIB ${CMAKE_RANLIB} CACHE PATH "Path to ranlib program with LTO-plugin" FORCE)
set(CMAKE_CXX_COMPILER_RANLIB ${CMAKE_RANLIB} CACHE PATH "Path to ranlib program with LTO-plugin" FORCE)
message(STATUS "MDBX indulge Link-Time Optimization by CLANG")
elseif(MSVC_LTO_AVAILABLE)
set(LTO_ENABLED TRUE)
@@ -369,6 +379,8 @@ if(NOT DEFINED MDBX_CXX_STANDARD)
set(MDBX_CXX_STANDARD 14)
elseif(NOT HAS_CXX11 LESS 0)
set(MDBX_CXX_STANDARD 11)
elseif(CXX_FALLBACK_GNU11 OR CXX_FALLBACK_11)
set(MDBX_CXX_STANDARD 11)
else()
set(MDBX_CXX_STANDARD 98)
endif()
@@ -514,7 +526,7 @@ else()
unset(MDBX_LINK_TOOLS_NONSTATIC CACHE)
endif()
if(CMAKE_CXX_COMPILER_LOADED AND MDBX_CXX_STANDARD GREATER_EQUAL 11 AND MDBX_CXX_STANDARD LESS 83)
if(CMAKE_CXX_COMPILER_LOADED AND MDBX_CXX_STANDARD LESS 83 AND NOT MDBX_CXX_STANDARD LESS 11)
if(NOT MDBX_AMALGAMATED_SOURCE)
option(MDBX_ENABLE_TESTS "Build MDBX tests" ${BUILD_TESTING})
endif()
@@ -620,9 +632,13 @@ macro(target_setup_options TARGET)
endmacro()
macro(libmdbx_setup_libs TARGET MODE)
target_link_libraries(${TARGET} ${MODE} Threads::Threads)
if(CMAKE_VERSION VERSION_LESS 3.1)
target_link_libraries(${TARGET} ${MODE} ${CMAKE_THREAD_LIBS_INIT})
else()
target_link_libraries(${TARGET} ${MODE} Threads::Threads)
endif()
if(${CMAKE_SYSTEM_NAME} STREQUAL "Windows")
target_link_libraries(${TARGET} ${MODE} ntdll advapi32)
target_link_libraries(${TARGET} ${MODE} ntdll user32 kernel32 advapi32)
if(MDBX_NTDLL_EXTRA_IMPLIB AND MDBX_WITHOUT_MSVC_CRT)
target_link_libraries(${TARGET} ${MODE} ntdll_extra)
endif()
@@ -743,7 +759,6 @@ if(MDBX_BUILD_SHARED_LIBRARY)
if(CMAKE_VERSION VERSION_LESS 3.12)
install(TARGETS mdbx EXPORT libmdbx
LIBRARY DESTINATION ${MDBX_DLL_INSTALL_DESTINATION} COMPONENT runtime
OBJECTS DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT devel
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT devel
PUBLIC_HEADER DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} COMPONENT devel
INCLUDES DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} COMPONENT devel)
@@ -888,16 +903,16 @@ else()
endif()
if(CMAKE_C_COMPILER_ABI
AND NOT (CMAKE_C_COMPILER_ABI MATCHES ".*${MDBX_BUILD_TARGET}.*" OR MDBX_BUILD_TARGET MATCHES ".*${CMAKE_C_COMPILER_ABI}.*"))
string(APPEND MDBX_BUILD_TARGET "-${CMAKE_C_COMPILER_ABI}")
string(CONCAT MDBX_BUILD_TARGET "${MDBX_BUILD_TARGET}-${CMAKE_C_COMPILER_ABI}")
endif()
if(CMAKE_C_PLATFORM_ID
AND NOT (CMAKE_SYSTEM_NAME
AND (CMAKE_C_PLATFORM_ID MATCHES ".*${CMAKE_SYSTEM_NAME}.*" OR CMAKE_SYSTEM_NAME MATCHES ".*${CMAKE_C_PLATFORM_ID}.*"))
AND NOT (CMAKE_C_PLATFORM_ID MATCHES ".*${CMAKE_C_PLATFORM_ID}.*" OR MDBX_BUILD_TARGET MATCHES ".*${CMAKE_C_PLATFORM_ID}.*"))
string(APPEND MDBX_BUILD_TARGET "-${CMAKE_C_COMPILER_ABI}")
string(CONCAT MDBX_BUILD_TARGET "${MDBX_BUILD_TARGET}-${CMAKE_C_COMPILER_ABI}")
endif()
if(CMAKE_SYSTEM_NAME)
string(APPEND MDBX_BUILD_TARGET "-${CMAKE_SYSTEM_NAME}")
string(CONCAT MDBX_BUILD_TARGET "${MDBX_BUILD_TARGET}-${CMAKE_SYSTEM_NAME}")
endif()
endif()

View File

@@ -1,10 +1,78 @@
ChangeLog
---------
## v0.11.13 at (Swashplate) 2022-11-10
The stable bugfix release in memory of [Boris Yuryev](https://ru.wikipedia.org/wiki/Юрьев,_Борис_Николаевич) on his 133rd birthday.
```
30 files changed, 405 insertions(+), 136 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
```
Fixes:
- Fixed builds with older libc versions after using `fcntl64()` (backport).
- Fixed builds with older `stdatomic.h` versions,
where the `ATOMIC_*_LOCK_FREE` macros mistakenly redefined using functions (backport).
- Added workaround for `mremap()` defect to avoid assertion failure (backport).
- Workaround for `encryptfs` bug(s) in the `copy_file_range` implementation (backport).
- Fixed unexpected `MDBX_BUSY` from `mdbx_env_set_option()`, `mdbx_env_set_syncbytes()`
and `mdbx_env_set_syncperiod()` (backport).
- CMake requirements lowered to version 3.0.2 (backport).
Minors:
- Minor clarification output of `--help` for `mdbx_test` (backport).
- Added admonition of insecure for RISC-V (backport).
- Stochastic scripts and CMake files synchronized with the `devel` branch.
- Use `--dont-check-ram-size` for small-tests make-targets (backport).
## v0.11.12 (Эребуни) at 2022-10-12
The stable bugfix release.
```
11 files changed, 96 insertions(+), 49 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
```
Fixes:
- Fixed static assertion failure on platforms where the `off_t` type is wider
than corresponding fields of `struct flock` used for file locking (backport).
Now _libmdbx_ will use `fcntl64(F_GETLK64/F_SETLK64/F_SETLKW64)` if available.
- Fixed assertion check inside `page_retire_ex()` (backport).
Minors:
- Fixed `-Wint-to-pointer-cast` warnings while casting to `mdbx_tid_t` (backport).
- Removed needless `LockFileEx()` inside `mdbx_env_copy()` (backport).
## v0.11.11 (Тендра-1790) at 2022-09-11
The stable bugfix release.
```
10 files changed, 38 insertions(+), 21 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
```
Fixes:
- Fixed an extra check for `MDBX_APPENDDUP` inside `mdbx_cursor_put()` which could result in returning `MDBX_EKEYMISMATCH` for valid cases.
- Fixed an extra ensure/assertion check of `oldest_reader` inside `mdbx_txn_end()`.
- Fixed derived C++ builds by removing `MDBX_INTERNAL_FUNC` for `mdbx_w2mb()` and `mdbx_mb2w()`.
-------------------------------------------------------------------------------
## v0.11.10 (the TriColor) at 2022-08-22
The stable bugfix release.
It is planned that this will be the last release of the v0.11 branch.
New:

View File

@@ -414,7 +414,7 @@ smoke-fault: build-test
test: build-test
@echo ' RUNNING `test/long_stochastic.sh --loops 2`...'
$(QUIET)test/long_stochastic.sh --loops 2 --db-upto-mb 256 --skip-make >$(TEST_LOG)
$(QUIET)test/long_stochastic.sh --dont-check-ram-size --loops 2 --db-upto-mb 256 --skip-make >$(TEST_LOG)
long-test: build-test
@echo ' RUNNING `test/long_stochastic.sh --loops 42`...'
@@ -422,7 +422,7 @@ long-test: build-test
test-singleprocess: build-test
@echo ' RUNNING `test/long_stochastic.sh --single --loops 2`...'
$(QUIET)test/long_stochastic.sh --single --loops 2 --db-upto-mb 256 --skip-make >$(TEST_LOG)
$(QUIET)test/long_stochastic.sh --dont-check-ram-size --single --loops 2 --db-upto-mb 256 --skip-make >$(TEST_LOG)
test-valgrind: CFLAGS_EXTRA=-Ofast -DMDBX_USE_VALGRIND
test-valgrind: build-test

View File

@@ -13,7 +13,9 @@
## limitations under the License.
##
if(CMAKE_VERSION VERSION_LESS 3.12)
if(CMAKE_VERSION VERSION_LESS 3.8.2)
cmake_minimum_required(VERSION 3.0.2)
elseif(CMAKE_VERSION VERSION_LESS 3.12)
cmake_minimum_required(VERSION 3.8.2)
else()
cmake_minimum_required(VERSION 3.12)
@@ -203,9 +205,38 @@ endif()
if(NOT CMAKE_SYSTEM_ARCH)
if(CMAKE_${CMAKE_PRIMARY_LANG}_COMPILER_ARCHITECTURE_ID)
set(CMAKE_SYSTEM_ARCH "${CMAKE_${CMAKE_PRIMARY_LANG}_COMPILER_ARCHITECTURE_ID}")
string(TOLOWER "${CMAKE_${CMAKE_PRIMARY_LANG}_COMPILER_ARCHITECTURE_ID}" CMAKE_SYSTEM_ARCH)
if(CMAKE_SYSTEM_ARCH STREQUAL "x86")
set(X86_32 TRUE)
elseif(CMAKE_SYSTEM_ARCH STREQUAL "x86_64" OR CMAKE_SYSTEM_ARCH STREQUAL "x64")
set(X86_64 TRUE)
set(CMAKE_SYSTEM_ARCH "x86_64")
elseif(CMAKE_SYSTEM_ARCH MATCHES "^(aarch.*|arm.*)")
if(CMAKE_TARGET_BITNESS EQUAL 64)
set(AARCH64 TRUE)
else()
set(ARM32 TRUE)
endif()
endif()
elseif(CMAKE_ANDROID_ARCH_ABI)
set(CMAKE_SYSTEM_ARCH "${CMAKE_ANDROID_ARCH_ABI}")
if(CMAKE_SYSTEM_ARCH STREQUAL "x86")
set(X86_32 TRUE)
elseif(CMAKE_SYSTEM_ARCH STREQUAL "x86_64")
set(X86_64 TRUE)
elseif(CMAKE_SYSTEM_ARCH MATCHES "^(aarch.*|AARCH.*|arm.*|ARM.*)")
if(CMAKE_TARGET_BITNESS EQUAL 64)
set(AARCH64 TRUE)
else()
set(ARM32 TRUE)
endif()
elseif(CMAKE_SYSTEM_ARCH MATCHES "^(mips|MIPS).*")
if(CMAKE_TARGET_BITNESS EQUAL 64)
set(MIPS64 TRUE)
else()
set(MIPS32 TRUE)
endif()
endif()
elseif(CMAKE_COMPILER_IS_ELBRUSC OR CMAKE_COMPILER_IS_ELBRUSCXX
OR CMAKE_${CMAKE_PRIMARY_LANG}_COMPILER_ID STREQUAL "LCC"
OR CMAKE_SYSTEM_PROCESSOR MATCHES "e2k.*|E2K.*|elbrus.*|ELBRUS.*")
@@ -929,12 +960,13 @@ endmacro(setup_compile_flags)
macro(probe_libcxx_filesystem)
if(CMAKE_CXX_COMPILER_LOADED AND NOT DEFINED LIBCXX_FILESYSTEM)
list(FIND CMAKE_CXX_COMPILE_FEATURES cxx_std_11 HAS_CXX11)
if(NOT HAS_CXX11 LESS 0)
if(NOT HAS_CXX11 LESS 0 OR CXX_FALLBACK_GNU11 OR CXX_FALLBACK_11)
include(CMakePushCheckState)
include(CheckCXXSourceCompiles)
cmake_push_check_state()
set(stdfs_probe_save_libraries ${CMAKE_REQUIRED_LIBRARIES})
set(stdfs_probe_save_flags ${CMAKE_REQUIRED_FLAGS})
set(stdfs_probe_flags ${CMAKE_REQUIRED_FLAGS})
set(stdfs_probe_save_link_options ${CMAKE_REQUIRED_LINK_OPTIONS})
unset(stdfs_probe_clear_cxx_standard)
if(NOT DEFINED CMAKE_CXX_STANDARD)
@@ -945,18 +977,23 @@ macro(probe_libcxx_filesystem)
set(CMAKE_CXX_STANDARD 17)
elseif(NOT HAS_CXX14 LESS 0)
set(CMAKE_CXX_STANDARD 14)
else()
elseif(NOT HAS_CXX11 LESS 0)
set(CMAKE_CXX_STANDARD 11)
elseif(CXX_FALLBACK_GNU11)
set(stdfs_probe_flags ${stdfs_probe_flags} "-std=gnu++11")
else()
set(stdfs_probe_flags ${stdfs_probe_flags} "-std=c++11")
endif()
set(stdfs_probe_clear_cxx_standard ON)
endif()
if(CMAKE_COMPILER_IS_ELBRUSCXX AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 1.25.23)
if(CMAKE_VERSION VERSION_LESS 3.14)
set(CMAKE_REQUIRED_FLAGS ${stdfs_probe_save_flags} "-Wl,--allow-multiple-definition")
set(stdfs_probe_flags ${stdfs_probe_flags} "-Wl,--allow-multiple-definition")
else()
set(CMAKE_REQUIRED_LINK_OPTIONS ${stdfs_probe_save_link_options} "-Wl,--allow-multiple-definition")
endif()
endif()
set(CMAKE_REQUIRED_FLAGS ${stdfs_probe_flags})
set(stdfs_probe_code [[
#if defined(__SIZEOF_INT128__) && !defined(__GLIBCXX_TYPE_INT_N_0) && defined(__clang__) && __clang_major__ < 4
@@ -1037,6 +1074,7 @@ macro(probe_libcxx_filesystem)
unset(stdfs_probe_clear_cxx_standard)
unset(stdfs_probe_save_link_options)
unset(stdfs_probe_save_flags)
unset(stdfs_probe_flags)
unset(stdfs_probe_save_libraries)
unset(stdfs_probe_code)
unset(stdfs_probe_rc)

View File

@@ -13,7 +13,9 @@
## limitations under the License.
##
if(CMAKE_VERSION VERSION_LESS 3.12)
if(CMAKE_VERSION VERSION_LESS 3.8.2)
cmake_minimum_required(VERSION 3.0.2)
elseif(CMAKE_VERSION VERSION_LESS 3.12)
cmake_minimum_required(VERSION 3.8.2)
else()
cmake_minimum_required(VERSION 3.12)

View File

@@ -13,7 +13,9 @@
## limitations under the License.
##
if(CMAKE_VERSION VERSION_LESS 3.12)
if(CMAKE_VERSION VERSION_LESS 3.8.2)
cmake_minimum_required(VERSION 3.0.2)
elseif(CMAKE_VERSION VERSION_LESS 3.12)
cmake_minimum_required(VERSION 3.8.2)
else()
cmake_minimum_required(VERSION 3.12)

View File

@@ -18,7 +18,13 @@
* <http://www.OpenLDAP.org/license.html>.
*/
#if (defined(__MINGW__) || defined(__MINGW32__) || defined(__MINGW64__)) && \
!defined(__USE_MINGW_ANSI_STDIO)
#define __USE_MINGW_ANSI_STDIO 1
#endif /* MinGW */
#include "mdbx.h"
#include <limits.h>
#include <stdint.h>
#include <stdio.h>

32
mdbx.h
View File

@@ -75,6 +75,14 @@ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
#ifndef LIBMDBX_H
#define LIBMDBX_H
#if defined(__riscv) || defined(__riscv__) || defined(__RISCV) || \
defined(__RISCV__)
#warning The RISC-V architecture is intentionally insecure by design. \
Please delete this admonition at your own risk, \
if you make such decision informed and consciously. \
Refer to https://clck.ru/32d9xH for more information.
#endif /* RISC-V */
#ifdef _MSC_VER
#pragma warning(push, 1)
#pragma warning(disable : 4548) /* expression before comma has no effect; \
@@ -958,8 +966,16 @@ DEFINE_ENUM_FLAG_OPERATORS(MDBX_debug_flags_t)
* called before printing the message and aborting.
* \see mdbx_setup_debug()
*
* \param [in] env An environment handle returned by \ref mdbx_env_create().
* \param [in] msg The assertion message, not including newline. */
* \param [in] loglevel The severity of message.
* \param [in] function The function name which emits message,
* may be NULL.
* \param [in] line The source code line number which emits message,
* may be zero.
* \param [in] fmt The printf-like format string with message.
* \param [in] args The variable argument list respectively for the
* format-message string passed by `fmt` argument.
* Maybe NULL or invalid if the format-message string
* don't contain `%`-specification of arguments. */
typedef void MDBX_debug_func(MDBX_log_level_t loglevel, const char *function,
int line, const char *fmt,
va_list args) MDBX_CXX17_NOEXCEPT;
@@ -978,8 +994,12 @@ LIBMDBX_API int mdbx_setup_debug(MDBX_log_level_t log_level,
* called before printing the message and aborting.
* \see mdbx_env_set_assert()
*
* \param [in] env An environment handle returned by mdbx_env_create().
* \param [in] msg The assertion message, not including newline. */
* \param [in] env An environment handle.
* \param [in] msg The assertion message, not including newline.
* \param [in] function The function name where the assertion check failed,
* may be NULL.
* \param [in] line The line number in the source file
* where the assertion check failed, may be zero. */
typedef void MDBX_assert_func(const MDBX_env *env, const char *msg,
const char *function,
unsigned line) MDBX_CXX17_NOEXCEPT;
@@ -2391,7 +2411,7 @@ typedef struct MDBX_stat MDBX_stat;
/** \brief Return statistics about the MDBX environment.
* \ingroup c_statinfo
*
* At least one of env or txn argument must be non-null. If txn is passed
* At least one of `env` or `txn` argument must be non-null. If txn is passed
* non-null then stat will be filled accordingly to the given transaction.
* Otherwise, if txn is null, then stat will be populated by a snapshot from
* the last committed write transaction, and at next time, other information
@@ -2500,7 +2520,7 @@ typedef struct MDBX_envinfo MDBX_envinfo;
/** \brief Return information about the MDBX environment.
* \ingroup c_statinfo
*
* At least one of env or txn argument must be non-null. If txn is passed
* At least one of `env` or `txn` argument must be non-null. If txn is passed
* non-null then stat will be filled accordingly to the given transaction.
* Otherwise, if txn is null, then stat will be populated by a snapshot from
* the last committed write transaction, and at next time, other information

View File

@@ -236,10 +236,6 @@ __extern_C key_t ftok(const char *, int);
#elif _WIN32_WINNT < 0x0500
#error At least 'Windows 2000' API is required for libmdbx.
#endif /* _WIN32_WINNT */
#if (defined(__MINGW32__) || defined(__MINGW64__)) && \
!defined(__USE_MINGW_ANSI_STDIO)
#define __USE_MINGW_ANSI_STDIO 1
#endif /* MinGW */
#ifndef WIN32_LEAN_AND_MEAN
#define WIN32_LEAN_AND_MEAN
#endif /* WIN32_LEAN_AND_MEAN */

View File

@@ -983,14 +983,7 @@ static __always_inline bool atomic_cas64(MDBX_atomic_uint64_t *p, uint64_t c,
uint64_t v) {
#ifdef MDBX_HAVE_C11ATOMICS
STATIC_ASSERT(sizeof(long long) >= sizeof(uint64_t));
#ifdef ATOMIC_LLONG_LOCK_FREE
STATIC_ASSERT(ATOMIC_LLONG_LOCK_FREE > 0);
#if ATOMIC_LLONG_LOCK_FREE < 2
assert(atomic_is_lock_free(MDBX_c11a_rw(uint64_t, p)));
#endif /* ATOMIC_LLONG_LOCK_FREE < 2 */
#else /* defined(ATOMIC_LLONG_LOCK_FREE) */
assert(atomic_is_lock_free(MDBX_c11a_rw(uint64_t, p)));
#endif
return atomic_compare_exchange_strong(MDBX_c11a_rw(uint64_t, p), &c, v);
#elif defined(__GNUC__) || defined(__clang__)
return __sync_bool_compare_and_swap(&p->weak, c, v);
@@ -1009,14 +1002,7 @@ static __always_inline bool atomic_cas32(MDBX_atomic_uint32_t *p, uint32_t c,
uint32_t v) {
#ifdef MDBX_HAVE_C11ATOMICS
STATIC_ASSERT(sizeof(int) >= sizeof(uint32_t));
#ifdef ATOMIC_INT_LOCK_FREE
STATIC_ASSERT(ATOMIC_INT_LOCK_FREE > 0);
#if ATOMIC_INT_LOCK_FREE < 2
assert(atomic_is_lock_free(MDBX_c11a_rw(uint32_t, p)));
#endif
#else
assert(atomic_is_lock_free(MDBX_c11a_rw(uint32_t, p)));
#endif
return atomic_compare_exchange_strong(MDBX_c11a_rw(uint32_t, p), &c, v);
#elif defined(__GNUC__) || defined(__clang__)
return __sync_bool_compare_and_swap(&p->weak, c, v);
@@ -1035,14 +1021,7 @@ static __always_inline uint32_t atomic_add32(MDBX_atomic_uint32_t *p,
uint32_t v) {
#ifdef MDBX_HAVE_C11ATOMICS
STATIC_ASSERT(sizeof(int) >= sizeof(uint32_t));
#ifdef ATOMIC_INT_LOCK_FREE
STATIC_ASSERT(ATOMIC_INT_LOCK_FREE > 0);
#if ATOMIC_INT_LOCK_FREE < 2
assert(atomic_is_lock_free(MDBX_c11a_rw(uint32_t, p)));
#endif
#else
assert(atomic_is_lock_free(MDBX_c11a_rw(uint32_t, p)));
#endif
return atomic_fetch_add(MDBX_c11a_rw(uint32_t, p), v);
#elif defined(__GNUC__) || defined(__clang__)
return __sync_fetch_and_add(&p->weak, v);
@@ -4835,7 +4814,8 @@ static int mdbx_page_retire_ex(MDBX_cursor *mc, const pgno_t pgno,
rc = mdbx_page_get(mc, pgno, &check, txn->mt_front);
if (unlikely(rc != MDBX_SUCCESS))
return rc;
mdbx_tassert(txn, (PAGETYPE(check) & ~P_LEAF2) == (pagetype & ~P_FROZEN));
mdbx_tassert(txn, (PAGETYPE(check) & ~(P_LEAF2 | P_SPILLED)) ==
(pagetype & ~(P_LEAF2 | P_FROZEN)));
mdbx_tassert(txn, !(pagetype & P_FROZEN) || IS_FROZEN(txn, check));
}
if (pagetype & P_FROZEN) {
@@ -8788,10 +8768,6 @@ static int mdbx_txn_end(MDBX_txn *txn, const unsigned mode) {
(void *)env, txn->mt_dbs[MAIN_DBI].md_root,
txn->mt_dbs[FREE_DBI].md_root);
mdbx_ensure(env, txn->mt_txnid >=
/* paranoia is appropriate here */ env->me_lck
->mti_oldest_reader.weak);
if (!(mode & MDBX_END_EOTDONE)) /* !(already closed cursors) */
mdbx_cursors_eot(txn, false);
@@ -8801,6 +8777,9 @@ static int mdbx_txn_end(MDBX_txn *txn, const unsigned mode) {
MDBX_reader *slot = txn->to.reader;
mdbx_assert(env, slot->mr_pid.weak == env->me_pid);
if (likely(!F_ISSET(txn->mt_flags, MDBX_TXN_FINISHED))) {
mdbx_ensure(env, txn->mt_txnid >=
/* paranoia is appropriate here */ env->me_lck
->mti_oldest_reader.weak);
mdbx_assert(env, txn->mt_txnid == slot->mr_txnid.weak &&
slot->mr_txnid.weak >=
env->me_lck->mti_oldest_reader.weak);
@@ -8828,7 +8807,10 @@ static int mdbx_txn_end(MDBX_txn *txn, const unsigned mode) {
txn->mt_numdbs = 0; /* prevent further DBI activity */
txn->mt_flags = MDBX_TXN_RDONLY | MDBX_TXN_FINISHED;
txn->mt_owner = 0;
} else if (!F_ISSET(txn->mt_flags, MDBX_TXN_FINISHED)) {
} else if (!(txn->mt_flags & MDBX_TXN_FINISHED)) {
mdbx_ensure(env, txn->mt_txnid >=
/* paranoia is appropriate here */ env->me_lck
->mti_oldest_reader.weak);
#if defined(MDBX_USE_VALGRIND) || defined(__SANITIZE_ADDRESS__)
if (txn == env->me_txn0)
mdbx_txn_valgrind(env, nullptr);
@@ -15542,7 +15524,7 @@ int mdbx_cursor_put(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data,
mc->mc_ki[mc->mc_top]++; /* step forward for appending */
rc = MDBX_NOTFOUND;
} else {
if (unlikely(rc != MDBX_SUCCESS || !(flags & MDBX_APPENDDUP)))
if (unlikely(rc != MDBX_SUCCESS))
/* new-key < last-key
* or new-key == last-key without MDBX_APPENDDUP */
return MDBX_EKEYMISMATCH;
@@ -19688,6 +19670,11 @@ __cold static int mdbx_env_copy_asis(MDBX_env *env, MDBX_txn *read_txn,
#if MDBX_USE_COPYFILERANGE
static bool copyfilerange_unavailable;
bool not_the_same_filesystem = false;
struct statfs statfs_info;
if (fstatfs(fd, &statfs_info) ||
statfs_info.f_type == /* ECRYPTFS_SUPER_MAGIC */ 0xf15f)
/* avoid use copyfilerange_unavailable() to ecryptfs due bugs */
not_the_same_filesystem = true;
#endif /* MDBX_USE_COPYFILERANGE */
for (size_t offset = meta_bytes; rc == MDBX_SUCCESS && offset < used_size;) {
#if MDBX_USE_SENDFILE
@@ -19721,7 +19708,9 @@ __cold static int mdbx_env_copy_asis(MDBX_env *env, MDBX_txn *read_txn,
if (bytes_copied == 0)
break;
rc = errno;
if (rc == EXDEV)
if (rc == EXDEV || rc == /* workaround for ecryptfs bug(s),
maybe usefull for others fs */
EINVAL)
not_the_same_filesystem = true;
else if (ignore_enosys(rc) == MDBX_RESULT_TRUE)
copyfilerange_unavailable = true;
@@ -19851,30 +19840,25 @@ __cold int mdbx_env_copy(MDBX_env *env, const char *dest_path,
#endif
);
if (rc == MDBX_SUCCESS) {
#if defined(_WIN32) || defined(_WIN64)
OVERLAPPED ov;
memset(&ov, 0, sizeof(ov));
if (!LockFileEx(newfd, LOCKFILE_EXCLUSIVE_LOCK | LOCKFILE_FAIL_IMMEDIATELY,
0, 0, INT32_MAX, &ov))
rc = GetLastError();
/* no locking required since the file opened with ShareMode == 0 */
#else
struct flock lock_op;
if (rc == MDBX_SUCCESS) {
MDBX_STRUCT_FLOCK lock_op;
memset(&lock_op, 0, sizeof(lock_op));
lock_op.l_type = F_WRLCK;
lock_op.l_whence = SEEK_SET;
lock_op.l_start = 0;
lock_op.l_len =
(sizeof(lock_op.l_len) > 4 ? INT64_MAX : INT32_MAX) & ~(size_t)0xffff;
if (fcntl(newfd, F_SETLK, &lock_op)
lock_op.l_len = OFF_T_MAX;
if (MDBX_FCNTL(newfd, MDBX_F_SETLK, &lock_op)
#if (defined(__linux__) || defined(__gnu_linux__)) && defined(LOCK_EX) && \
(!defined(__ANDROID_API__) || __ANDROID_API__ >= 24)
|| flock(newfd, LOCK_EX | LOCK_NB)
#endif /* Linux */
)
rc = errno;
#endif /* Windows / POSIX */
}
#endif /* Windows / POSIX */
if (rc == MDBX_SUCCESS)
rc = mdbx_env_copy2fd(env, newfd, flags);
@@ -20995,8 +20979,8 @@ __cold int mdbx_reader_list(const MDBX_env *env, MDBX_reader_list_func *func,
reader_pages_retired))
: 0;
}
rc = func(ctx, ++serial, i, pid, (mdbx_tid_t)tid, txnid, lag, bytes_used,
bytes_retained);
rc = func(ctx, ++serial, i, pid, (mdbx_tid_t)((intptr_t)tid), txnid, lag,
bytes_used, bytes_retained);
if (unlikely(rc != MDBX_SUCCESS))
break;
}
@@ -21248,7 +21232,7 @@ __cold static txnid_t mdbx_kick_longlived_readers(MDBX_env *env,
? pgno2bytes(env, (pgno_t)(head_retired - oldest_retired))
: 0;
int rc = env->me_hsr_callback(
env, env->me_txn, pid, (mdbx_tid_t)tid, laggard,
env, env->me_txn, pid, (mdbx_tid_t)((intptr_t)tid), laggard,
(gap < UINT_MAX) ? (unsigned)gap : UINT_MAX, space, retry);
if (rc < 0)
break;
@@ -22686,14 +22670,17 @@ __cold int mdbx_env_set_option(MDBX_env *env, const MDBX_option_t option,
return MDBX_EPERM;
if (unlikely(value > SIZE_MAX - 65536))
return MDBX_TOO_LARGE;
if (atomic_store32(&env->me_lck->mti_autosync_threshold,
bytes2pgno(env, (size_t)value + env->me_psize - 1),
mo_Relaxed) != 0 &&
(env->me_flags & MDBX_ENV_ACTIVE)) {
err = mdbx_env_sync_poll(env);
if (unlikely(MDBX_IS_ERROR(err)))
return err;
err = MDBX_SUCCESS;
value = bytes2pgno(env, (size_t)value + env->me_psize - 1);
if ((uint32_t)value != atomic_load32(&env->me_lck->mti_autosync_threshold,
mo_AcquireRelease) &&
atomic_store32(&env->me_lck->mti_autosync_threshold, (uint32_t)value,
mo_Relaxed)
/* Дергаем sync(force=off) только если задано новое не-нулевое значение
* и мы вне транзакции */
&& lock_needed) {
err = mdbx_env_sync_internal(env, false, false);
if (err == /* нечего сбрасывать на диск */ MDBX_RESULT_TRUE)
err = MDBX_SUCCESS;
}
break;
@@ -22706,14 +22693,16 @@ __cold int mdbx_env_set_option(MDBX_env *env, const MDBX_option_t option,
return MDBX_EPERM;
if (unlikely(value > UINT32_MAX))
return MDBX_TOO_LARGE;
if (atomic_store64(&env->me_lck->mti_autosync_period,
mdbx_osal_16dot16_to_monotime((uint32_t)value),
mo_Relaxed) != 0 &&
(env->me_flags & MDBX_ENV_ACTIVE)) {
err = mdbx_env_sync_poll(env);
if (unlikely(MDBX_IS_ERROR(err)))
return err;
err = MDBX_SUCCESS;
value = mdbx_osal_16dot16_to_monotime((uint32_t)value);
if (value != atomic_load64(&env->me_lck->mti_autosync_period,
mo_AcquireRelease) &&
atomic_store64(&env->me_lck->mti_autosync_period, value, mo_Relaxed)
/* Дергаем sync(force=off) только если задано новое не-нулевое значение
* и мы вне транзакции */
&& lock_needed) {
err = mdbx_env_sync_internal(env, false, false);
if (err == /* нечего сбрасывать на диск */ MDBX_RESULT_TRUE)
err = MDBX_SUCCESS;
}
break;
@@ -23244,6 +23233,8 @@ __dll_export
"MINGW-64 " MDBX_STRINGIFY(__MINGW64_MAJOR_VERSION) "." MDBX_STRINGIFY(__MINGW64_MINOR_VERSION)
#elif defined(__MINGW32__)
"MINGW-32 " MDBX_STRINGIFY(__MINGW32_MAJOR_VERSION) "." MDBX_STRINGIFY(__MINGW32_MINOR_VERSION)
#elif defined(__MINGW__)
"MINGW " MDBX_STRINGIFY(__MINGW_MAJOR_VERSION) "." MDBX_STRINGIFY(__MINGW_MINOR_VERSION)
#elif defined(__IBMC__)
"IBM C " MDBX_STRINGIFY(__IBMC__)
#elif defined(__GNUC__)

View File

@@ -126,7 +126,7 @@
#if (defined(__MINGW__) || defined(__MINGW32__) || defined(__MINGW64__)) && \
!defined(__USE_MINGW_ANSI_STDIO)
#define __USE_MINGW_ANSI_STDIO 1
#endif /* __USE_MINGW_ANSI_STDIO */
#endif /* MinGW */
#include "../mdbx.h"
#include "base.h"

View File

@@ -158,26 +158,21 @@ __cold static void choice_fcntl(void) {
of reliability reasons */
#endif /* linux */
) {
op_setlk = F_OFD_SETLK;
op_setlkw = F_OFD_SETLKW;
op_getlk = F_OFD_GETLK;
op_setlk = MDBX_F_OFD_SETLK;
op_setlkw = MDBX_F_OFD_SETLKW;
op_getlk = MDBX_F_OFD_GETLK;
return;
}
op_setlk = F_SETLK;
op_setlkw = F_SETLKW;
op_getlk = F_GETLK;
op_setlk = MDBX_F_SETLK;
op_setlkw = MDBX_F_SETLKW;
op_getlk = MDBX_F_GETLK;
}
#else
#define op_setlk F_SETLK
#define op_setlkw F_SETLKW
#define op_getlk F_GETLK
#define op_setlk MDBX_F_SETLK
#define op_setlkw MDBX_F_SETLKW
#define op_getlk MDBX_F_GETLK
#endif /* MDBX_USE_OFDLOCKS */
#ifndef OFF_T_MAX
#define OFF_T_MAX \
(((sizeof(off_t) > 4) ? INT64_MAX : INT32_MAX) & ~(size_t)0xffff)
#endif
static int lck_op(const mdbx_filehandle_t fd, int cmd, const int lck,
const off_t offset, off_t len) {
STATIC_ASSERT(sizeof(off_t) >= sizeof(void *) &&
@@ -200,7 +195,7 @@ static int lck_op(const mdbx_filehandle_t fd, int cmd, const int lck,
assert((uint64_t)((off_t)((uint64_t)offset + (uint64_t)len)) ==
((uint64_t)offset + (uint64_t)len));
for (;;) {
struct flock lock_op;
MDBX_STRUCT_FLOCK lock_op;
STATIC_ASSERT_MSG(sizeof(off_t) <= sizeof(lock_op.l_start) &&
sizeof(off_t) <= sizeof(lock_op.l_len) &&
OFF_T_MAX == (off_t)OFF_T_MAX,
@@ -212,7 +207,7 @@ static int lck_op(const mdbx_filehandle_t fd, int cmd, const int lck,
lock_op.l_whence = SEEK_SET;
lock_op.l_start = offset;
lock_op.l_len = len;
int rc = fcntl(fd, cmd, &lock_op);
int rc = MDBX_FCNTL(fd, cmd, &lock_op);
mdbx_jitter4testing(true);
if (rc != -1) {
if (cmd == op_getlk) {
@@ -226,18 +221,18 @@ static int lck_op(const mdbx_filehandle_t fd, int cmd, const int lck,
}
rc = errno;
#if MDBX_USE_OFDLOCKS
if (rc == EINVAL &&
(cmd == F_OFD_SETLK || cmd == F_OFD_SETLKW || cmd == F_OFD_GETLK)) {
if (rc == EINVAL && (cmd == MDBX_F_OFD_SETLK || cmd == MDBX_F_OFD_SETLKW ||
cmd == MDBX_F_OFD_GETLK)) {
/* fallback to non-OFD locks */
if (cmd == F_OFD_SETLK)
cmd = F_SETLK;
else if (cmd == F_OFD_SETLKW)
cmd = F_SETLKW;
if (cmd == MDBX_F_OFD_SETLK)
cmd = MDBX_F_SETLK;
else if (cmd == MDBX_F_OFD_SETLKW)
cmd = MDBX_F_SETLKW;
else
cmd = F_GETLK;
op_setlk = F_SETLK;
op_setlkw = F_SETLKW;
op_getlk = F_GETLK;
cmd = MDBX_F_GETLK;
op_setlk = MDBX_F_SETLK;
op_setlkw = MDBX_F_SETLKW;
op_getlk = MDBX_F_GETLK;
continue;
}
#endif /* MDBX_USE_OFDLOCKS */

View File

@@ -161,7 +161,21 @@ int mdbx_txn_lock(MDBX_env *env, bool dontwait) {
: (LCK_EXCLUSIVE | LCK_WAITFOR),
LCK_BODY))
return MDBX_SUCCESS;
int rc = (int)GetLastError();
if (rc == ERROR_LOCK_VIOLATION && dontwait) {
SleepEx(0, true);
if (flock(env->me_lazy_fd, LCK_EXCLUSIVE | LCK_DONTWAIT, LCK_BODY))
return MDBX_SUCCESS;
rc = (int)GetLastError();
if (rc == ERROR_LOCK_VIOLATION) {
SleepEx(0, true);
if (flock(env->me_lazy_fd, LCK_EXCLUSIVE | LCK_DONTWAIT, LCK_BODY))
return MDBX_SUCCESS;
rc = (int)GetLastError();
}
}
LeaveCriticalSection(&env->me_windowsbug_lock);
return (!dontwait || rc != ERROR_LOCK_VIOLATION) ? rc : MDBX_BUSY;
}

View File

@@ -1,6 +1,6 @@
.\" Copyright 2015-2022 Leonid Yuriev <leo@yuriev.ru>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_CHK 1 "2022-08-22" "MDBX 0.11.10"
.TH MDBX_CHK 1 "2022-11-10" "MDBX 0.11.13"
.SH NAME
mdbx_chk \- MDBX checking tool
.SH SYNOPSIS

View File

@@ -2,7 +2,7 @@
.\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_COPY 1 "2022-08-22" "MDBX 0.11.10"
.TH MDBX_COPY 1 "2022-11-10" "MDBX 0.11.13"
.SH NAME
mdbx_copy \- MDBX environment copy tool
.SH SYNOPSIS

View File

@@ -1,7 +1,7 @@
.\" Copyright 2021-2022 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2014-2021 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_DROP 1 "2022-08-22" "MDBX 0.11.10"
.TH MDBX_DROP 1 "2022-11-10" "MDBX 0.11.13"
.SH NAME
mdbx_drop \- MDBX database delete tool
.SH SYNOPSIS

View File

@@ -2,7 +2,7 @@
.\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_DUMP 1 "2022-08-22" "MDBX 0.11.10"
.TH MDBX_DUMP 1 "2022-11-10" "MDBX 0.11.13"
.SH NAME
mdbx_dump \- MDBX environment export tool
.SH SYNOPSIS

View File

@@ -2,7 +2,7 @@
.\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_LOAD 1 "2022-08-22" "MDBX 0.11.10"
.TH MDBX_LOAD 1 "2022-11-10" "MDBX 0.11.13"
.SH NAME
mdbx_load \- MDBX environment import tool
.SH SYNOPSIS

View File

@@ -2,7 +2,7 @@
.\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_STAT 1 "2022-08-22" "MDBX 0.11.10"
.TH MDBX_STAT 1 "2022-11-10" "MDBX 0.11.13"
.SH NAME
mdbx_stat \- MDBX environment status tool
.SH SYNOPSIS

View File

@@ -12,7 +12,7 @@
#if (defined(__MINGW__) || defined(__MINGW32__) || defined(__MINGW64__)) && \
!defined(__USE_MINGW_ANSI_STDIO)
#define __USE_MINGW_ANSI_STDIO 1
#endif /* __USE_MINGW_ANSI_STDIO */
#endif /* MinGW */
#include "../mdbx.h++"

View File

@@ -237,7 +237,10 @@
/** Advanced: Using POSIX OFD-locks (autodetection by default). */
#ifndef MDBX_USE_OFDLOCKS
#if defined(F_OFD_SETLK) && defined(F_OFD_SETLKW) && defined(F_OFD_GETLK) && \
#if ((defined(F_OFD_SETLK) && defined(F_OFD_SETLKW) && \
defined(F_OFD_GETLK)) || \
(defined(F_OFD_SETLK64) && defined(F_OFD_SETLKW64) && \
defined(F_OFD_GETLK64))) && \
!defined(MDBX_SAFE4QEMU) && \
!defined(__sun) /* OFD-lock are broken on Solaris */
#define MDBX_USE_OFDLOCKS 1
@@ -323,13 +326,7 @@
#endif /* MDBX_64BIT_ATOMIC */
#ifndef MDBX_64BIT_CAS
#if defined(ATOMIC_LLONG_LOCK_FREE)
#if ATOMIC_LLONG_LOCK_FREE > 1
#define MDBX_64BIT_CAS 1
#else
#define MDBX_64BIT_CAS 0
#endif
#elif defined(__GCC_ATOMIC_LLONG_LOCK_FREE)
#if defined(__GCC_ATOMIC_LLONG_LOCK_FREE)
#if __GCC_ATOMIC_LLONG_LOCK_FREE > 1
#define MDBX_64BIT_CAS 1
#else
@@ -341,6 +338,12 @@
#else
#define MDBX_64BIT_CAS 0
#endif
#elif defined(ATOMIC_LLONG_LOCK_FREE)
#if ATOMIC_LLONG_LOCK_FREE > 1
#define MDBX_64BIT_CAS 1
#else
#define MDBX_64BIT_CAS 0
#endif
#elif defined(_MSC_VER) || defined(__APPLE__) || defined(DOXYGEN)
#define MDBX_64BIT_CAS 1
#else

View File

@@ -527,14 +527,12 @@ static const DWORD WC_ERR_INVALID_CHARS =
: 0;
#endif /* WC_ERR_INVALID_CHARS */
MDBX_INTERNAL_FUNC size_t mdbx_mb2w(wchar_t *dst, size_t dst_n, const char *src,
size_t src_n) {
size_t mdbx_mb2w(wchar_t *dst, size_t dst_n, const char *src, size_t src_n) {
return MultiByteToWideChar(CP_THREAD_ACP, MB_ERR_INVALID_CHARS, src,
(int)src_n, dst, (int)dst_n);
}
MDBX_INTERNAL_FUNC size_t mdbx_w2mb(char *dst, size_t dst_n, const wchar_t *src,
size_t src_n) {
size_t mdbx_w2mb(char *dst, size_t dst_n, const wchar_t *src, size_t src_n) {
return WideCharToMultiByte(CP_THREAD_ACP, WC_ERR_INVALID_CHARS, src,
(int)src_n, dst, (int)dst_n, nullptr, nullptr);
}
@@ -1549,6 +1547,7 @@ MDBX_INTERNAL_FUNC int mdbx_mmap(const int flags, mdbx_mmap_t *map,
map->limit = 0;
map->current = 0;
map->address = nullptr;
assert(errno != 0);
return errno;
}
map->limit = limit;
@@ -1587,8 +1586,10 @@ MDBX_INTERNAL_FUNC int mdbx_munmap(mdbx_mmap_t *map) {
if (!NT_SUCCESS(rc))
ntstatus2errcode(rc);
#else
if (unlikely(munmap(map->address, map->limit)))
if (unlikely(munmap(map->address, map->limit))) {
assert(errno != 0);
return errno;
}
#endif /* ! Windows */
map->limit = 0;
@@ -1811,8 +1812,10 @@ retry_mapview:;
if (limit < map->limit) {
/* unmap an excess at end of mapping. */
// coverity[offset_free : FALSE]
if (unlikely(munmap(map->dxb + limit, map->limit - limit)))
if (unlikely(munmap(map->dxb + limit, map->limit - limit))) {
assert(errno != 0);
return errno;
}
map->limit = limit;
return rc;
}
@@ -1824,14 +1827,19 @@ retry_mapview:;
assert(limit > map->limit);
uint8_t *ptr = MAP_FAILED;
#if defined(MREMAP_MAYMOVE)
#if (defined(__linux__) || defined(__gnu_linux__)) && defined(_GNU_SOURCE)
ptr = mremap(map->address, map->limit, limit,
(flags & MDBX_MRESIZE_MAY_MOVE) ? MREMAP_MAYMOVE : 0);
#if defined(MREMAP_MAYMOVE)
(flags & MDBX_MRESIZE_MAY_MOVE) ? MREMAP_MAYMOVE :
#endif /* MREMAP_MAYMOVE */
0);
if (ptr == MAP_FAILED) {
err = errno;
assert(err != 0);
switch (err) {
default:
return err;
case 0 /* paranoia */:
case EAGAIN:
case ENOMEM:
return MDBX_UNABLE_EXTEND_MAPSIZE;
@@ -1839,7 +1847,7 @@ retry_mapview:;
break;
}
}
#endif /* MREMAP_MAYMOVE */
#endif /* Linux & _GNU_SOURCE */
const unsigned mmap_flags =
MAP_CONCEAL | MAP_SHARED | MAP_FILE | MAP_NORESERVE |
@@ -1852,17 +1860,22 @@ retry_mapview:;
ptr = mmap(map->dxb + map->limit, limit - map->limit, mmap_prot,
mmap_flags | MAP_FIXED_NOREPLACE, map->fd, map->limit);
if (ptr == map->dxb + map->limit)
/* успешно прилепили отображение в конец */
ptr = map->dxb;
else if (ptr != MAP_FAILED) {
/* the desired address is busy, unmap unsuitable one */
if (unlikely(munmap(ptr, limit - map->limit)))
if (unlikely(munmap(ptr, limit - map->limit))) {
assert(errno != 0);
return errno;
}
ptr = MAP_FAILED;
} else {
err = errno;
assert(err != 0);
switch (err) {
default:
return err;
case 0 /* paranoia */:
case EAGAIN:
case ENOMEM:
return MDBX_UNABLE_EXTEND_MAPSIZE;
@@ -1881,8 +1894,10 @@ retry_mapview:;
return MDBX_UNABLE_EXTEND_MAPSIZE;
}
if (unlikely(munmap(map->address, map->limit)))
if (unlikely(munmap(map->address, map->limit))) {
assert(errno != 0);
return errno;
}
// coverity[pass_freed_arg : FALSE]
ptr = mmap(map->address, limit, mmap_prot,
@@ -1926,6 +1941,7 @@ retry_mapview:;
map->limit = 0;
map->current = 0;
map->address = nullptr;
assert(errno != 0);
return errno;
}
rc = MDBX_UNABLE_EXTEND_MAPSIZE;
@@ -1952,8 +1968,10 @@ retry_mapview:;
#if MDBX_ENABLE_MADVISE
#ifdef MADV_DONTFORK
if (unlikely(madvise(map->address, map->limit, MADV_DONTFORK) != 0))
if (unlikely(madvise(map->address, map->limit, MADV_DONTFORK) != 0)) {
assert(errno != 0);
return errno;
}
#endif /* MADV_DONTFORK */
#ifdef MADV_NOHUGEPAGE
(void)madvise(map->address, map->limit, MADV_NOHUGEPAGE);
@@ -1962,6 +1980,9 @@ retry_mapview:;
#endif /* POSIX / Windows */
assert(rc != MDBX_SUCCESS ||
(map->address != nullptr && map->address != MAP_FAILED &&
map->current == size && map->limit == limit));
return rc;
}

View File

@@ -180,10 +180,8 @@ static inline void mdbx_free(void *ptr) { HeapFree(GetProcessHeap(), 0, ptr); }
#define vsnprintf _vsnprintf /* ntdll */
#endif
MDBX_INTERNAL_FUNC size_t mdbx_mb2w(wchar_t *dst, size_t dst_n, const char *src,
size_t src_n);
MDBX_INTERNAL_FUNC size_t mdbx_w2mb(char *dst, size_t dst_n, const wchar_t *src,
size_t src_n);
size_t mdbx_mb2w(wchar_t *dst, size_t dst_n, const char *src, size_t src_n);
size_t mdbx_w2mb(char *dst, size_t dst_n, const wchar_t *src, size_t src_n);
#else /*----------------------------------------------------------------------*/
@@ -289,7 +287,48 @@ MDBX_MAYBE_UNUSED static __inline void mdbx_jitter4testing(bool tiny);
#if defined(_WIN32) || defined(_WIN64)
#define MAX_WRITE UINT32_C(0x01000000)
#else
#define MAX_WRITE UINT32_C(0x3fff0000)
#define MAX_WRITE UINT32_C(0x3f000000)
#if defined(F_GETLK64) && defined(F_SETLK64) && defined(F_SETLKW64) && \
!defined(__ANDROID_API__)
#define MDBX_F_SETLK F_SETLK64
#define MDBX_F_SETLKW F_SETLKW64
#define MDBX_F_GETLK F_GETLK64
#if (__GLIBC_PREREQ(2, 28) && \
(defined(__USE_LARGEFILE64) || defined(__LARGEFILE64_SOURCE) || \
defined(_USE_LARGEFILE64) || defined(_LARGEFILE64_SOURCE))) || \
defined(fcntl64)
#define MDBX_FCNTL fcntl64
#else
#define MDBX_FCNTL fcntl
#endif
#define MDBX_STRUCT_FLOCK struct flock64
#ifndef OFF_T_MAX
#define OFF_T_MAX UINT64_C(0x7fffFFFFfff00000)
#endif /* OFF_T_MAX */
#else
#define MDBX_F_SETLK F_SETLK
#define MDBX_F_SETLKW F_SETLKW
#define MDBX_F_GETLK F_GETLK
#define MDBX_FCNTL fcntl
#define MDBX_STRUCT_FLOCK struct flock
#endif /* MDBX_F_SETLK, MDBX_F_SETLKW, MDBX_F_GETLK */
#if defined(F_OFD_SETLK64) && defined(F_OFD_SETLKW64) && \
defined(F_OFD_GETLK64) && !defined(__ANDROID_API__)
#define MDBX_F_OFD_SETLK F_OFD_SETLK64
#define MDBX_F_OFD_SETLKW F_OFD_SETLKW64
#define MDBX_F_OFD_GETLK F_OFD_GETLK64
#else
#define MDBX_F_OFD_SETLK F_OFD_SETLK
#define MDBX_F_OFD_SETLKW F_OFD_SETLKW
#define MDBX_F_OFD_GETLK F_OFD_GETLK
#ifndef OFF_T_MAX
#define OFF_T_MAX \
(((sizeof(off_t) > 4) ? INT64_MAX : INT32_MAX) & ~(size_t)0xFffff)
#endif /* OFF_T_MAX */
#endif /* MDBX_F_OFD_SETLK64, MDBX_F_OFD_SETLKW64, MDBX_F_OFD_GETLK64 */
#endif
#if defined(__linux__) || defined(__gnu_linux__)

View File

@@ -54,7 +54,11 @@ if(NOT MDBX_BUILD_CXX AND LIBCXX_FILESYSTEM)
target_link_libraries(mdbx_test ${LIBCXX_FILESYSTEM})
endif()
target_link_libraries(mdbx_test ${TOOL_MDBX_LIB} ${LIB_MATH} ${CMAKE_THREAD_LIBS_INIT})
if(CMAKE_VERSION VERSION_LESS 3.1)
target_link_libraries(mdbx_test ${TOOL_MDBX_LIB} ${LIB_MATH} ${CMAKE_THREAD_LIBS_INIT})
else()
target_link_libraries(mdbx_test ${TOOL_MDBX_LIB} ${LIB_MATH} Threads::Threads)
endif()
if(${CMAKE_SYSTEM_NAME} STREQUAL "Windows")
target_link_libraries(mdbx_test winmm.lib)
endif()

View File

@@ -1,8 +1,4 @@
#!/usr/bin/env bash
if ! which make cc c++ tee >/dev/null; then
echo "Please install the following prerequisites: make cc c++ tee banner" >&2
exit 1
fi
LIST=basic
FROM=1
@@ -15,6 +11,7 @@ BANNER="$(which banner 2>/dev/null | echo echo)"
UNAME="$(uname -s 2>/dev/null || echo Unknown)"
DB_UPTO_MB=17408
PAGESIZE=min
DONT_CHECK_RAM=no
while [ -n "$1" ]
do
@@ -35,6 +32,7 @@ do
echo "--db-upto-mb NN Limits upper size of test DB to the NN megabytes"
echo "--no-geometry-jitter Disable jitter for geometry upper-size"
echo "--pagesize NN Use specified page size (256 is minimal and used by default) "
echo "--dont-check-ram-size Don't check available RAM "
echo "--help Print this usage help and exit"
exit -2
;;
@@ -111,7 +109,7 @@ do
--no-geometry-jitter)
GEOMETRY_JITTER=no
;;
--pagesize)
--pagesize|--page-size)
case "$2" in
min|max|256|512|1024|2048|4096|8192|16386|32768|65536)
PAGESIZE=$2
@@ -144,6 +142,9 @@ do
esac
shift
;;
--dont-check-ram-size)
DONT_CHECK_RAM=yes
;;
*)
echo "Unknown option '$1'"
exit -2
@@ -163,6 +164,11 @@ if [ -z "$MONITOR" ]; then
export MALLOC_CHECK_=7 MALLOC_PERTURB_=42
fi
if ! which $([ "$SKIP_MAKE" == "no" ] && echo make cc c++) tee >/dev/null; then
echo "Please install the following prerequisites: make cc c++ tee banner" >&2
exit 1
fi
###############################################################################
# 1. clean data from prev runs and examine available RAM
@@ -220,6 +226,19 @@ case ${UNAME} in
echo "pagesize ${pagesize}K, freepages ${freepages}, ram_avail_mb ${ram_avail_mb}"
;;
MSYS*|MINGW*)
if [ -z "${TESTDB_DIR:-}" ]; then
for old_test_dir in $(ls -d /tmp/mdbx-test.[0-9]* 2>/dev/null); do
rm -rf $old_test_dir
done
TESTDB_DIR="/tmp/mdbx-test.$$"
fi
mkdir -p $TESTDB_DIR && rm -f $TESTDB_DIR/*
echo "FIXME: Fake support for ${UNAME}"
ram_avail_mb=32768
;;
*)
echo "FIXME: ${UNAME} not supported by this script"
exit 2
@@ -232,11 +251,15 @@ rm -f ${TESTDB_DIR}/*
# 2. estimate reasonable RAM space for test-db
echo "=== ${ram_avail_mb}M RAM available"
ram_reserve4logs_mb=1234
if [ $ram_avail_mb -lt $ram_reserve4logs_mb ]; then
echo "=== At least ${ram_reserve4logs_mb}Mb RAM required"
exit 3
fi
if [ $DONT_CHECK_RAM = yes ]; then
db_size_mb=$DB_UPTO_MB
ram_reserve4logs_mb=64
else
ram_reserve4logs_mb=1234
if [ $ram_avail_mb -lt $ram_reserve4logs_mb ]; then
echo "=== At least ${ram_reserve4logs_mb}Mb RAM required"
exit 3
fi
#
# В режимах отличных от MDBX_WRITEMAP изменения до записи в файл
@@ -256,9 +279,10 @@ fi
# that malloc() will not return the allocated memory to the
# system immediately, as well some space is required for logs.
#
db_size_mb=$(((ram_avail_mb - ram_reserve4logs_mb) / 4))
if [ $db_size_mb -gt $DB_UPTO_MB ]; then
db_size_mb=$DB_UPTO_MB
db_size_mb=$(((ram_avail_mb - ram_reserve4logs_mb) / 4))
if [ $db_size_mb -gt $DB_UPTO_MB ]; then
db_size_mb=$DB_UPTO_MB
fi
fi
echo "=== use ${db_size_mb}M for DB"
@@ -269,7 +293,11 @@ case ${UNAME} in
ulimit -c unlimited
if [ "$(cat /proc/sys/kernel/core_pattern)" != "core.%p" ]; then
echo "core.%p > /proc/sys/kernel/core_pattern" >&2
echo "core.%p" | sudo tee /proc/sys/kernel/core_pattern || true
if [ $(id -u) -ne 0 -a -n "$(which sudo 2>/dev/null)" ]; then
echo "core.%p" | sudo tee /proc/sys/kernel/core_pattern || true
else
(echo "core.%p" > /proc/sys/kernel/core_pattern) || true
fi
fi
;;
@@ -288,6 +316,10 @@ case ${UNAME} in
fi
;;
MSYS*|MINGW*)
echo "FIXME: Fake support for ${UNAME}"
;;
*)
echo "FIXME: ${UNAME} not supported by this script"
exit 2
@@ -319,7 +351,7 @@ else
fi
syncmodes=("" ,+nosync-safe ,+nosync-utterly)
options=(writemap coalesce lifo notls perturb)
options=(writemap lifo notls perturb)
function join { local IFS="$1"; shift; echo "$*"; }

View File

@@ -24,7 +24,7 @@ MDBX_NORETURN void usage(void) {
"usage:\n"
" --help or -h Show this text\n"
"Common parameters:\n"
" --loglevel=[0-7]|[extra..fatal]"
" --loglevel=[0-7]|[fatal..extra]s"
" --pathname=... Path and/or name of database files\n"
" --repeat=N Set repeat counter\n"
" --threads=N Number of thread (unsupported for now)\n"

View File

@@ -360,6 +360,7 @@ actor_status osal_actor_info(const mdbx_pid_t pid) {
status = as_coredump;
break;
default:
log_error("pid %zu, exit code %u", (intptr_t)pid, (unsigned)ExitCode);
status = as_failed;
break;
}

View File

@@ -1,18 +1,17 @@
#!/usr/bin/env bash
if ! which make cc c++ tee >/dev/null; then
echo "Please install the following prerequisites: make cc c++ tee banner" >&2
exit 1
fi
LIST=--hill
FROM=1
UPTO=9999
UPTO=9999999
MONITOR=
LOOPS=
SKIP_MAKE=no
GEOMETRY_JITTER=yes
BANNER="$(which banner 2>/dev/null | echo echo)"
UNAME="$(uname -s 2>/dev/null || echo Unknown)"
DB_UPTO_MB=17408
PAGESIZE=min
DONT_CHECK_RAM=no
while [ -n "$1" ]
do
@@ -20,6 +19,10 @@ do
--help)
echo "--multi Engage multi-process test scenario (default)"
echo "--single Execute series of single-process tests (for QEMU, etc)"
echo "--nested Execute only 'nested' testcase"
echo "--hill Execute only 'hill' testcase"
echo "--append Execute only 'append' testcase"
echo "--ttl Execute only 'ttl' testcase"
echo "--with-valgrind Run tests under Valgrind's memcheck tool"
echo "--skip-make Don't (re)build libmdbx and test's executable"
echo "--from NN Start iterating from the NN ops per test case"
@@ -27,6 +30,9 @@ do
echo "--loops NN Stop after the NN loops"
echo "--dir PATH Specifies directory for test DB and other files (it will be cleared)"
echo "--db-upto-mb NN Limits upper size of test DB to the NN megabytes"
echo "--no-geometry-jitter Disable jitter for geometry upper-size"
echo "--pagesize NN Use specified page size (256 is minimal and used by default) "
echo "--dont-check-ram-size Don't check available RAM "
echo "--help Print this usage help and exit"
exit -2
;;
@@ -36,6 +42,18 @@ do
--single)
LIST="--nested --hill --append --ttl --copy"
;;
--nested)
LIST="--nested"
;;
--hill)
LIST="--hill"
;;
--append)
LIST="--append"
;;
--ttl)
LIST="--ttl"
;;
--with-valgrind)
echo " NOTE: Valgrind could produce some false-positive warnings"
echo " in multi-process environment with shared memory."
@@ -88,6 +106,45 @@ do
fi
shift
;;
--no-geometry-jitter)
GEOMETRY_JITTER=no
;;
--pagesize|--page-size)
case "$2" in
min|max|256|512|1024|2048|4096|8192|16386|32768|65536)
PAGESIZE=$2
;;
1|1k|1K|k|K)
PAGESIZE=$((1024*1))
;;
2|2k|2K)
PAGESIZE=$((1024*2))
;;
4|4k|4K)
PAGESIZE=$((1024*4))
;;
8|8k|8K)
PAGESIZE=$((1024*8))
;;
16|16k|16K)
PAGESIZE=$((1024*16))
;;
32|32k|32K)
PAGESIZE=$((1024*32))
;;
64|64k|64K)
PAGESIZE=$((1024*64))
;;
*)
echo "Invalig page size '$2'"
exit -2
;;
esac
shift
;;
--dont-check-ram-size)
DONT_CHECK_RAM=yes
;;
*)
echo "Unknown option '$1'"
exit -2
@@ -107,6 +164,11 @@ if [ -z "$MONITOR" ]; then
export MALLOC_CHECK_=7 MALLOC_PERTURB_=42
fi
if ! which $([ "$SKIP_MAKE" == "no" ] && echo make cc c++) tee >/dev/null; then
echo "Please install the following prerequisites: make cc c++ tee banner" >&2
exit 1
fi
###############################################################################
# 1. clean data from prev runs and examine available RAM
@@ -123,9 +185,9 @@ case ${UNAME} in
mkdir -p $TESTDB_DIR && rm -f $TESTDB_DIR/*
if LC_ALL=C free | grep -q -i available; then
ram_avail_mb=$(($(LC_ALL=C free | grep -i Mem: | tr -s [:blank:] ' ' | cut -d ' ' -f 7) / 1024))
ram_avail_mb=$(($(LC_ALL=C free | grep -i Mem: | tr -s '[:blank:]' ' ' | cut -d ' ' -f 7) / 1024))
else
ram_avail_mb=$(($(LC_ALL=C free | grep -i Mem: | tr -s [:blank:] ' ' | cut -d ' ' -f 4) / 1024))
ram_avail_mb=$(($(LC_ALL=C free | grep -i Mem: | tr -s '[:blank:]' ' ' | cut -d ' ' -f 4) / 1024))
fi
;;
@@ -164,6 +226,19 @@ case ${UNAME} in
echo "pagesize ${pagesize}K, freepages ${freepages}, ram_avail_mb ${ram_avail_mb}"
;;
MSYS*|MINGW*)
if [ -z "${TESTDB_DIR:-}" ]; then
for old_test_dir in $(ls -d /tmp/mdbx-test.[0-9]* 2>/dev/null); do
rm -rf $old_test_dir
done
TESTDB_DIR="/tmp/mdbx-test.$$"
fi
mkdir -p $TESTDB_DIR && rm -f $TESTDB_DIR/*
echo "FIXME: Fake support for ${UNAME}"
ram_avail_mb=32768
;;
*)
echo "FIXME: ${UNAME} not supported by this script"
exit 2
@@ -176,11 +251,15 @@ rm -f ${TESTDB_DIR}/*
# 2. estimate reasonable RAM space for test-db
echo "=== ${ram_avail_mb}M RAM available"
ram_reserve4logs_mb=1234
if [ $ram_avail_mb -lt $ram_reserve4logs_mb ]; then
echo "=== At least ${ram_reserve4logs_mb}Mb RAM required"
exit 3
fi
if [ $DONT_CHECK_RAM = yes ]; then
db_size_mb=$DB_UPTO_MB
ram_reserve4logs_mb=64
else
ram_reserve4logs_mb=1234
if [ $ram_avail_mb -lt $ram_reserve4logs_mb ]; then
echo "=== At least ${ram_reserve4logs_mb}Mb RAM required"
exit 3
fi
#
# В режимах отличных от MDBX_WRITEMAP изменения до записи в файл
@@ -200,9 +279,10 @@ fi
# that malloc() will not return the allocated memory to the
# system immediately, as well some space is required for logs.
#
db_size_mb=$(((ram_avail_mb - ram_reserve4logs_mb) / 4))
if [ $db_size_mb -gt $DB_UPTO_MB ]; then
db_size_mb=$DB_UPTO_MB
db_size_mb=$(((ram_avail_mb - ram_reserve4logs_mb) / 4))
if [ $db_size_mb -gt $DB_UPTO_MB ]; then
db_size_mb=$DB_UPTO_MB
fi
fi
echo "=== use ${db_size_mb}M for DB"
@@ -213,7 +293,11 @@ case ${UNAME} in
ulimit -c unlimited
if [ "$(cat /proc/sys/kernel/core_pattern)" != "core.%p" ]; then
echo "core.%p > /proc/sys/kernel/core_pattern" >&2
echo "core.%p" | sudo tee /proc/sys/kernel/core_pattern || true
if [ $(id -u) -ne 0 -a -n "$(which sudo 2>/dev/null)" ]; then
echo "core.%p" | sudo tee /proc/sys/kernel/core_pattern || true
else
(echo "core.%p" > /proc/sys/kernel/core_pattern) || true
fi
fi
;;
@@ -232,6 +316,10 @@ case ${UNAME} in
fi
;;
MSYS*|MINGW*)
echo "FIXME: Fake support for ${UNAME}"
;;
*)
echo "FIXME: ${UNAME} not supported by this script"
exit 2
@@ -263,7 +351,7 @@ else
fi
syncmodes=("" ,+nosync-safe ,+nosync-utterly)
options=(writemap coalesce lifo notls perturb)
options=(writemap lifo notls perturb)
function join { local IFS="$1"; shift; echo "$*"; }
@@ -296,8 +384,8 @@ function probe {
rm -f ${TESTDB_DIR}/* || failed
for case in $LIST
do
echo "Run ./mdbx_test ${speculum} --random-writemap=no --ignore-dbfull --repeat=1 --pathname=${TESTDB_DIR}/long.db --cleanup-after=no $@ $case"
${MONITOR} ./mdbx_test ${speculum} --random-writemap=no --ignore-dbfull --repeat=1 --pathname=${TESTDB_DIR}/long.db --cleanup-before=yes --cleanup-after=no "$@" $case | check_deep \
echo "Run ./mdbx_test ${speculum} --random-writemap=no --ignore-dbfull --repeat=11 --pathname=${TESTDB_DIR}/long.db --cleanup-after=no --geometry-jitter=${GEOMETRY_JITTER} $@ $case"
${MONITOR} ./mdbx_test ${speculum} --random-writemap=no --ignore-dbfull --repeat=11 --pathname=${TESTDB_DIR}/long.db --cleanup-after=no --geometry-jitter=${GEOMETRY_JITTER} "$@" $case | check_deep \
&& ${MONITOR} ./mdbx_chk ${TESTDB_DIR}/long.db | tee ${TESTDB_DIR}/long-chk.log \
&& ([ ! -e ${TESTDB_DIR}/long.db-copy ] || ${MONITOR} ./mdbx_chk ${TESTDB_DIR}/long.db-copy | tee ${TESTDB_DIR}/long-chk-copy.log) \
|| failed