Compare commits

...

34 Commits

Author SHA1 Message Date
Леонид Юрьев (Leonid Yuriev)
82a1c2d893 mdbx: update ChangeLog. 2025-11-10 22:05:01 +03:00
Leonid Yuriev
4899b155e3 mdbx: fix tls-dtor in case library unloading with an incomplete initialized env instance (backport). 2025-11-10 21:57:17 +03:00
Леонид Юрьев (Leonid Yuriev)
8cab3a522a mdbx: add workaround for ext4 fast-commit bug (backport). 2025-11-08 09:43:50 +03:00
Леонид Юрьев (Leonid Yuriev)
5e43fe33c1 mdbx-test: minor fix jitter test internals (backport). 2025-11-08 09:43:22 +03:00
Леонид Юрьев (Leonid Yuriev)
60ad6cf030 mdbx: update ChangeLog. 2025-10-31 23:44:07 +03:00
Леонид Юрьев (Leonid Yuriev)
4c20a61363 mdbx: workaroud/fix for unused function 'osal_yield' from modern clang. 2025-10-31 19:03:09 +03:00
Леонид Юрьев (Leonid Yuriev)
11052d8af0 mdbx: fix minor typos in the ChangeLog. 2025-10-31 17:29:53 +03:00
Леонид Юрьев (Leonid Yuriev)
cad99d7a09 mdbx: patch update for older versions of buildroot (backport). 2025-10-31 17:29:35 +03:00
Леонид Юрьев (Leonid Yuriev)
926e90ac9a mdbx: release v0.13.9 "ИС-2" (IS-2).
The supporting release of a stable branch with bug fixes,
in memory of the most powerful Soviet IS-2 tank of the Great Patriotic War.
 - Support of Harmony OS;
 - fixed assertion (BMI > 0) in debugging builds for 32-bit platforms;
 - fixed regression of DB growth (no shrinking) after using fallocate() for `SIGBUS` prevention;
 - workaround to avoid `EAGAIN` on Android after an application restart;

For more information please see [ChangeLog in the `stable` branch](https://gitflic.ru/project/erthink/libmdbx/blob?file=ChangeLog.md&branch=stable).

git diff' stat: 24 files changed, 262 insertions(+), 226 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2025-10-31 12:33:51 +03:00
Леонид Юрьев (Leonid Yuriev)
0a168bee9f mdbx: update ChangeLog. 2025-10-29 01:12:34 +03:00
Леонид Юрьев (Leonid Yuriev)
a51bec9582 mdbx: minor workaround for HarmonyOS's bug (backport).
The libc from HarmonyOS SDK erroneously defines `EOWNERDEAD`,
`_POSIX_THREAD_PROCESS_SHARED >= 200809` and `PTHREAD_MUTEX_ROBUST` but
the same time doesn't provide `pthread_mutexattr_setrobust()` and
`pthread_mutex_consistent()`.

This commit add a minor workaround for such bug for successful building
without explicitly defining `MDBX_LOCKING=2001`.

Related to https://github.com/erthink/libmdbx/issues/285, https://github.com/vorot93/libmdbx-rs/issues/41, https://github.com/isar-community/isar-community/issues/28.
2025-10-29 01:10:02 +03:00
Leonid Yuriev
3135dfb5df mdbx: minor clean-up tautology in assertion (backport). 2025-10-29 00:26:59 +03:00
Леонид Юрьев (Leonid Yuriev)
55f2ffda3b mdbx-cmake: support of MDBX_USE_FALLOCATE for CMake and Conan (backport). 2025-10-29 00:26:47 +03:00
Леонид Юрьев (Leonid Yuriev)
7a6a4eae8d mdbx: update ChangeLog. 2025-10-18 14:38:06 +03:00
Леонид Юрьев (Leonid Yuriev)
b9591e2cfe mdbx: introduce internal lck_setlk_with3retries() as workaround EAGAIN on Android (backport). 2025-10-18 14:38:06 +03:00
Леонид Юрьев (Leonid Yuriev)
e63f55c717 mdbx: extract internal osal_yield() (backport). 2025-10-18 14:38:06 +03:00
Леонид Юрьев (Leonid Yuriev)
eccb777bd7 mdbx: update ChangeLog. 2025-10-18 14:22:50 +03:00
Леонид Юрьев (Leonid Yuriev)
2172c4f60a mdbx: fix regression related to fallocate() and introduce osal_fsetsize() (backport).
This fixes regression after the 2930b304dc as when
a DXB file remains longer than necessary on Mac or Linux when building without `_GNU_SOURCE`.
2025-10-18 14:12:25 +03:00
Леонид Юрьев (Leonid Yuriev)
e09d835766 mdbx-tests: fix minor typo in the battery-tmux script (backport). 2025-10-18 14:12:17 +03:00
Леонид Юрьев (Leonid Yuriev)
339be13778 mdbx: cleanup redundant MDBX_INTERNAL (backport). 2025-10-18 14:10:18 +03:00
Леонид Юрьев (Leonid Yuriev)
f4dafd62d1 mdbx: update ChangeLog. 2025-09-11 22:34:37 +03:00
Леонид Юрьев (Leonid Yuriev)
57e28198ce mdbx: fix rare/specific unexpected assertion failure bmi > 0 on 32-bit debug builds (backport).
Fixes https://github.com/isar-community/isar-community/issues/33
2025-09-11 22:34:29 +03:00
Леонид Юрьев (Leonid Yuriev)
ba1bb34cbe mdbx-cmake: fix/refactoring search for lib.exe/dlltool.exe (backport). 2025-09-11 18:07:24 +03:00
Леонид Юрьев (Leonid Yuriev)
15fa152097 mdbx: remove TODO file which is irrelevent for the stable branch. 2025-09-01 11:04:28 +03:00
Леонид Юрьев (Leonid Yuriev)
af8dea2e60 mdbx: begin v0.13.9 release engineering cycle. 2025-08-31 18:33:05 +03:00
Леонид Юрьев (Leonid Yuriev)
4d58857f8f mdbx: release v0.13.8 "Всеобуч" (v`seabooch).
The supporting release of a stable branch with bug fixes,
on the day of the 100th anniversary of the Resolution of Russian Central Executive Committee on common free primary education.

 - enabled old Linux kernels starting from 3.16;
 - fixed unexpected `SIGBUS` is not enough space in a filesystem;
 - fixed inappropriate/irrelevant `MDBX_WANNA_RECOVERY` when the DB size is not rounded to sys-allocation-granularity.

For more information please see [ChangeLog](https://libmdbx.dqdkfa.ru/md__change_log.html).

git diff' stat: 9 files changed, 101 insertions(+), 36 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
2025-08-31 14:56:32 +03:00
Леонид Юрьев (Leonid Yuriev)
db389cde2a mdbx: update ChangeLog. 2025-08-29 11:08:35 +03:00
Леонид Юрьев (Leonid Yuriev)
43cb9133ee mdbx: fix inappropriate/irrelevant MDBX_WANNA_RECOVERY when the DB size is not rounded to sys-allocation-granularity.
The check corrected by this commit was never fundamentally important,
but it was added to verify the logic of rounding the database size in
various cases.
However, after the commit `2930b304dc674bbccd188b7ce7c3f83755ef706e`
(using `fallocate()` to prevent `SIGBUS`), the behavior change led to
the returning error `MDBX_WANNA_RECOVERY` in fairly harmless conditions.

This fix is not perfect, but it is just a patch that eliminates
regression by making the least risky/invasive changes.

A complete fix involves refining the rounding of a database size, which
is not difficult, but it is likely to lead to a few minor unexpected
regressions. So for the `stable` branch, I preferred minimal
conservative patch.
2025-08-28 10:50:00 +03:00
Леонид Юрьев (Leonid Yuriev)
71df2ec129 mdbx-tests: fix silently/unclear failures of stochastic test due an errors from mdbx_chk. 2025-08-28 10:30:55 +03:00
Леонид Юрьев (Leonid Yuriev)
e2fb593504 mdbx: update ChangeLog. 2025-08-26 23:32:29 +03:00
Леонид Юрьев (Leonid Yuriev)
2930b304dc mdbx: fix unexpected SIGBUS is not enough space in a filesystem (backport, squashed).
On a modern Linux the allocation of space for a file can be deferred
and/or lazy, rather than when setting its length using `ftruncate()`.
The actual allocation of space occurs when writing to the corresponding
areas of the file, or when reading ones (in this case, the file system
fills these areas with zeros).

The specific behavior depends on the type of file system and the kernel
version, but the main thing is that possibilities currently are, when
setting the file size, just the instantaneous ability to allocate space
is checked, without any booking.

If the file system is running out of space, an `ENOSPC` error may occur
when processing (inside a OS kernel) a page fault when accessing one of
the added pages after the database has been enlarged. In this case, the
OS kernel has no other alternative but to send a `SIGBUS` signal to the
process.

This commit fixes the problem by adding the use of system calls to
explicitly allocate space for a given file size.
Related-to https://github.com/erigontech/erigon/issues/16709

This is a simple improvement, however which is complicated by the need
to take into account the availability of the appropriate system API and
handle non-fatal errors from file systems that do not support the
appropriate operations. Therefore, there is a risk of regressions in
unusual/rare situations, including when hosting databases on network
media.
2025-08-24 10:30:17 +03:00
Леонид Юрьев (Leonid Yuriev)
a39dfd99a7 mdbx: update ChangeLog. 2025-08-05 13:12:22 +03:00
Леонид Юрьев (Leonid Yuriev)
66d116c301 mdbx: enabling older Linux kernels starting from 3.16 (backport). 2025-08-05 13:03:30 +03:00
Леонид Юрьев (Leonid Yuriev)
f71b729759 mdbx: patch update for older versions of buildroot (backport). 2025-08-04 00:20:02 +03:00
30 changed files with 397 additions and 369 deletions

View File

@@ -574,9 +574,8 @@ if(WIN32 AND EXISTS "${MDBX_SOURCE_DIR}/ntdll.def")
if(MSVC)
if(NOT MSVC_LIB_EXE)
# Find lib.exe
get_filename_component(CL_NAME ${CMAKE_C_COMPILER} NAME)
string(REPLACE cl.exe lib.exe MSVC_LIB_EXE ${CL_NAME})
find_program(MSVC_LIB_EXE ${MSVC_LIB_EXE})
get_filename_component(CC_DIR ${CMAKE_C_COMPILER} DIRECTORY)
find_program(MSVC_LIB_EXE "lib.exe" HINTS ${CC_DIR})
endif()
if(MSVC_LIB_EXE)
message(STATUS "Found MSVC's lib tool: ${MSVC_LIB_EXE}")
@@ -588,25 +587,24 @@ if(WIN32 AND EXISTS "${MDBX_SOURCE_DIR}/ntdll.def")
COMMAND ${MSVC_LIB_EXE} /def:"${MDBX_SOURCE_DIR}/ntdll.def" /out:"${MDBX_NTDLL_EXTRA_IMPLIB}"
${INITIAL_CMAKE_STATIC_LINKER_FLAGS})
else()
message(WARNING "MSVC's lib tool not found")
message(WARNING "MSVC's lib.exe not found")
endif()
elseif(MINGW OR MINGW64)
if(NOT DLLTOOL)
# Find dlltool
get_filename_component(GCC_NAME ${CMAKE_C_COMPILER} NAME)
string(REPLACE gcc dlltool DLLTOOL_NAME ${GCC_NAME})
find_program(DLLTOOL NAMES ${DLLTOOL_NAME})
if(NOT MINGW_DLLTOOL_EXE)
# Find dlltool.exe
get_filename_component(CC_DIR ${CMAKE_C_COMPILER} DIRECTORY)
find_program(MINGW_DLLTOOL_EXE "dlltool.exe" HINTS ${CC_DIR})
endif()
if(DLLTOOL)
message(STATUS "Found dlltool: ${DLLTOOL}")
if(MINGW_DLLTOOL_EXE)
message(STATUS "Found MINGW's dlltool: ${MINGW_DLLTOOL_EXE}")
set(MDBX_NTDLL_EXTRA_IMPLIB "${CMAKE_CURRENT_BINARY_DIR}/mdbx_ntdll_extra.a")
add_custom_command(
OUTPUT "${MDBX_NTDLL_EXTRA_IMPLIB}"
COMMENT "Create extra-import-library for ntdll.dll"
MAIN_DEPENDENCY "${MDBX_SOURCE_DIR}/ntdll.def"
COMMAND ${DLLTOOL} -d "${MDBX_SOURCE_DIR}/ntdll.def" -l "${MDBX_NTDLL_EXTRA_IMPLIB}")
COMMAND ${MINGW_DLLTOOL_EXE} -d "${MDBX_SOURCE_DIR}/ntdll.def" -l "${MDBX_NTDLL_EXTRA_IMPLIB}")
else()
message(WARNING "dlltool not found")
message(WARNING "MINGW's dlltool.exe not found")
endif()
endif()
@@ -702,6 +700,8 @@ mark_as_advanced(MDBX_ENABLE_PROFGC)
add_option(MDBX ENABLE_DBI_SPARSE
"Support for sparse sets of DBI handles to reduce overhead when starting and processing transactions" ON)
add_option(MDBX ENABLE_DBI_LOCKFREE "Support for deferred releasing and a lockfree path to quickly open DBI handles" ON)
add_option(MDBX USE_FALLOCATE "Using posix_fallocate() or fcntl(F_PREALLOCATE) on OSX" AUTO)
mark_as_advanced(MDBX_USE_FALLOCATE)
if(NOT MDBX_AMALGAMATED_SOURCE)
if(CMAKE_CONFIGURATION_TYPES OR CMAKE_BUILD_TYPE_UPPERCASE STREQUAL "DEBUG")

View File

@@ -10,6 +10,89 @@ Donations are welcome to ETH `0xD104d8f8B2dC312aaD74899F83EBf3EEBDC1EA3A`.
Всё будет хорошо!
## v0.13.10 в процессе накопления изменений
Поддерживающий выпуск стабильной ветки с исправлением обнаруженных ошибок и устранением недочётов.
Благодарности:
- [Erigon](https://erigon.tech/) за спонсорство.
Исправления:
- Устранение предупреждения "unused function 'osal_yield()'" от новых версий CLANG при сборке амальгамированного исходного кода и при компиляции в режиме одного объектного модуля.
- Для Linux добавлено предотвращение проявления ошибки в реализации fast_commit файловой системы Ext4.
- Устранено падение при выполнении Thread-Local-Storage конструкторов при выгрузке библиотеки и наличия экземпляров env, инициализация которых не была завершена.
--------------------------------------------------------------------------------
## v0.13.9 "ИС-2" (IS-2) от 2025-10-31
Поддерживающий выпуск стабильной ветки с исправлением обнаруженных ошибок и устранением недочётов.
Выпуск назван в память о cамом мощном тяжелом советском танке ["ИС-2"](https://ru.ruwiki.ru/wiki/ИС-2), который был принят на вооружение
31 октября 1943 года в разгар Великой Отечественной Войны и долгое время оставался одной из сильнейших машин мира в категории по массе 40—50 тонн.
Благодарности:
- [Erigon](https://erigon.tech/) за спонсорство.
Исправления:
- Исправлена assert-проверка в пути сканирования битовой карты DBI-дескрипторов приводившая к редким падениям 32-битных отладочных сборок.
- Переделан поиск утилит `lib.exe` и `dlltool.exe` при сборке посредством CMake на Windows.
- Устранён регресс проявлявшийся увеличением (не-уменьшением) размера БД, после добавления использования `fallocate()`
ради предотвращения SIGBUS при нехватке места в файловой системе где расположена БД.
- Устранена опечатка в тестовом скрипте `test/battery-tmux.sh` приводящая к созданию мусорного файла с именем `-`.
- Удалено лишнее/ненужное использование макроса `MDBX_INTERNAL` оставшееся после рефакторинга.
- Для Android добавлен обход (workaround) для уменьшения вероятности системной ошибки `EAGAIN` возникающей
из-за нехватки системных ресурсов и переходных процессов при закрытии и быстром повторном открытии БД.
Прочие доработки:
- Поддержка Harmony OS (OHOS).
--------------------------------------------------------------------------------
## v0.13.8 "Всеобуч" (v`seabooch) от 2025-08-31
Поддерживающий выпуск стабильной ветки с исправлением обнаруженных ошибок и устранением недочётов,
в день 100 летнего юбилея Постановления Всероссийского центрального исполнительного комитета о всеобщем бесплатном начальном образовании.
Благодарности:
- [Erigon](https://erigon.tech/) за спонсорство.
Исправления:
- Устранена возможность получения неожиданного `SIGBUS` из-за отложенного/ленивого выделение места в заполненной файловой системе после приращения файла БД.
Более подробное пояснение в комментарии коммита [`2930b304dc674bbccd188b7ce7c3f83755ef706e`](https://gitflic.ru/project/erthink/libmdbx/commit/2930b304dc674bbccd188b7ce7c3f83755ef706e).
Изменение поведения:
- Вновь включена/разрешена на старых ядрах Linux, начиная с версии 3.16, так как
сейчас уже нет причин отказываться от работы на 3.16 поддерживая при этом ядра 4.x,
и еще есть проекты (Isar, Isar-Community, Hive) которым требуется такая поддержка.
- Ошибка `MDBX_WANNA_RECOVERY` при открытии БД в режиме только-чтение теперь возвращается если размер БД не кратен размеру системной страницы,
но игнорируется не кратность размеру блока выделения виртуальной памяти.
Этим устраняется регресс, проявившейся вследствие изменения поведения после задействования
системного вызова `fallocate()` для предотвращения `SIGBUS` после приращения файла БД в заполненной файловой системе.
--------------------------------------------------------------------------------
## v0.13.7 "Дружба" (Friendship) от 2025-07-30.
Поддерживающий выпуск стабильной ветки с исправлением обнаруженных ошибок и устранением недочётов,

43
TODO.md
View File

@@ -1,43 +0,0 @@
TODO
----
Unfortunately, on 2022-04-15 the Github administration, without any
warning nor explanation, deleted _libmdbx_ along with a lot of other
projects, simultaneously blocking access for many developers. Therefore
on 2022-04-21 we have migrated to a reliable trusted infrastructure.
The origin for now is at[GitFlic](https://gitflic.ru/project/erthink/libmdbx)
with backup at [ABF by ROSA Лаб](https://abf.rosalinux.ru/erthink/libmdbx).
For the same reason ~~Github~~ is blacklisted forever.
So currently most of the links are broken due to noted malicious ~~Github~~ sabotage.
- Внутри `txn_renew()` вынести проверку когерентности mmap за/после изменение размера.
- [Migration guide from LMDB to MDBX](https://libmdbx.dqdkfa.ru/dead-github/issues/199).
- [Support for RAW devices](https://libmdbx.dqdkfa.ru/dead-github/issues/124).
- [Support MessagePack for Keys & Values](https://libmdbx.dqdkfa.ru/dead-github/issues/115).
- Packages for [Astra Linux](https://astralinux.ru/), [ALT Linux](https://www.altlinux.org/), [ROSA Linux](https://www.rosalinux.ru/), etc.
Done
----
- [Engage new terminology](https://libmdbx.dqdkfa.ru/dead-github/issues/137).
- [More flexible support of asynchronous runtime/framework(s)](https://libmdbx.dqdkfa.ru/dead-github/issues/200).
- [Move most of `mdbx_chk` functional to the library API](https://libmdbx.dqdkfa.ru/dead-github/issues/204).
- [Simple careful mode for working with corrupted DB](https://libmdbx.dqdkfa.ru/dead-github/issues/223).
- [Engage an "overlapped I/O" on Windows](https://libmdbx.dqdkfa.ru/dead-github/issues/224).
- [Large/Overflow pages accounting for dirty-room](https://libmdbx.dqdkfa.ru/dead-github/issues/192).
- [Get rid of dirty-pages list in MDBX_WRITEMAP mode](https://libmdbx.dqdkfa.ru/dead-github/issues/193).
Cancelled
--------
- [Replace SRW-lock on Windows to allow shrink DB with `MDBX_NOSTICKYTHREADS` option](https://libmdbx.dqdkfa.ru/dead-github/issues/210).
Доработка не может быть реализована, так как замена SRW-блокировки
лишает лишь предварительную проблему, но не главную. На Windows
уменьшение размера отображенного в память файла не поддерживается ядром
ОС. Для этого необходимо снять отображение, изменить размер файла и
затем отобразить обратно. В свою очередь, для это необходимо
приостановить работающие с БД потоки выполняющие транзакции чтения, либо
готовые к такому выполнению. Но в режиме MDBX_NOSTICKYTHREADS нет
возможности отслеживать работающие с БД потоки, а приостановка всех
потоков неприемлема для большинства приложений.

View File

@@ -82,6 +82,7 @@ class libmdbx(ConanFile):
'mdbx.use_mincore': ['Auto', True, False],
'mdbx.use_ofdlocks': ['Auto', True, False],
'mdbx.use_sendfile': ['Auto', True, False],
'mdbx.use_fallocate': ['Auto', True, False],
'mdbx.without_msvc_crt': ['Default', True, False],
'shared': [True, False],
}
@@ -113,6 +114,7 @@ class libmdbx(ConanFile):
'mdbx.use_mincore': 'Auto',
'mdbx.use_ofdlocks': 'Auto',
'mdbx.use_sendfile': 'Auto',
'mdbx.use_fallocate': 'Auto',
'mdbx.without_msvc_crt': 'Default',
'shared': True,
}
@@ -143,7 +145,8 @@ class libmdbx(ConanFile):
'mdbx.use_copyfilerange': 'Advanced: Use `copy_file_range()` syscall. ',
'mdbx.use_mincore': "Use Unix' `mincore()` to determine whether database pages are resident in memory. ",
'mdbx.use_ofdlocks': 'Advanced: Use POSIX OFD-locks. ',
'mdbx.use_sendfile': 'Advancedc: Use `sendfile()` syscall. ',
'mdbx.use_sendfile': 'Advanced: Use `sendfile()` syscall. ',
'mdbx.use_fallocate': 'Advanced: Use posix_fallocate() or fcntl(F_PREALLOCATE) on OSX. ',
'mdbx.without_msvc_crt': 'Avoid dependence from MSVC CRT and use ntdll.dll instead. ',
}
@@ -160,6 +163,7 @@ class libmdbx(ConanFile):
self.options.rm_safe('mdbx.mmap_incoherent_file_write')
self.options.rm_safe('mdbx.use_mincore')
self.options.rm_safe('mdbx.use_ofdlocks')
self.options.rm_safe('mdbx.use_fallocate')
else:
self.options.rm_safe('mdbx.without_msvc_crt')
if is_apple_os(self):

View File

@@ -1,13 +1,13 @@
From 349c08cf21b66ecea851340133a1b845c25675f7 Mon Sep 17 00:00:00 2001
From b2f1297dd2cd42cc0e04f1900fbf6da6c2694b7b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?=
=?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= <leo@yuriev.ru>
Date: Tue, 22 Apr 2025 14:38:49 +0300
Date: Fri, 31 Oct 2025 16:58:31 +0300
Subject: [PATCH] package/libmdbx: new package (library/database).
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This patch adds libmdbx:
This patch adds libmdbx 0.13.9:
- libmdbx is one of the fastest compact embeddable key-value ACID database.
- libmdbx has a specific set of properties and capabilities,
focused on creating unique lightweight solutions.
@@ -15,159 +15,55 @@ This patch adds libmdbx:
in terms of reliability, features and performance.
- more information at https://libmdbx.dqdkfa.ru
The 0.13.6 "Бузина" (Elderberry) is stable release of _libmdbx_ branch with new superior features.
The complete ChangeLog: https://gitflic.ru/project/erthink/libmdbx/blob?file=ChangeLog.md
The 0.13.9 "ИС-2" (IS-2) is bugfix release of the stable branch.
For more information please see [ChangeLog](https://github.com/erthink/libmdbx/blob/stable/ChangeLog.md).
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
---
DEVELOPERS | 3 +++
package/Config.in | 1 +
package/libmdbx/Config.in | 45 ++++++++++++++++++++++++++++++++++++
package/libmdbx/libmdbx.hash | 6 +++++
package/libmdbx/libmdbx.mk | 42 +++++++++++++++++++++++++++++++++
5 files changed, 97 insertions(+)
create mode 100644 package/libmdbx/Config.in
create mode 100644 package/libmdbx/libmdbx.hash
create mode 100644 package/libmdbx/libmdbx.mk
package/libmdbx/Config.in | 4 +++-
package/libmdbx/libmdbx.hash | 2 +-
package/libmdbx/libmdbx.mk | 2 +-
3 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/DEVELOPERS b/DEVELOPERS
index 9ab1e125f4..758ff6a2d5 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -1482,6 +1482,9 @@ N: Leon Anavi <leon.anavi@konsulko.com>
F: board/olimex/a10_olinuxino
F: configs/olimex_a10_olinuxino_lime_defconfig
+N: Leonid Yuriev <leo@yuriev.ru>
+F: package/libmdbx/
+
N: Lionel Flandrin <lionel@svkt.org>
F: package/python-babel/
F: package/python-daemonize/
diff --git a/package/Config.in b/package/Config.in
index 016a99ed1a..a6f95bfaa9 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1372,6 +1372,7 @@ menu "Database"
source "package/kompexsqlite/Config.in"
source "package/leveldb/Config.in"
source "package/libgit2/Config.in"
+ source "package/libmdbx/Config.in"
source "package/libodb/Config.in"
source "package/libodb-boost/Config.in"
source "package/libodb-mysql/Config.in"
diff --git a/package/libmdbx/Config.in b/package/libmdbx/Config.in
new file mode 100644
index 0000000000..a9a4ac45c5
--- /dev/null
index a9a4ac45c5..1640dbd9de 100644
--- a/package/libmdbx/Config.in
+++ b/package/libmdbx/Config.in
@@ -0,0 +1,45 @@
+config BR2_PACKAGE_LIBMDBX
+ bool "libmdbx"
+ depends on BR2_USE_MMU
+ depends on BR2_TOOLCHAIN_HAS_SYNC_4
+ depends on BR2_TOOLCHAIN_HAS_THREADS
+ depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_4
+ help
+ One of the fastest compact key-value ACID database
+ without WAL. libmdbx has a specific set of properties
+ and capabilities, focused on creating unique lightweight
+ solutions.
+
+ libmdbx surpasses the legendary LMDB in terms of
+ reliability, features and performance.
+
+ https://libmdbx.dqdkfa.ru
+
+if BR2_PACKAGE_LIBMDBX
+
+config BR2_PACKAGE_LIBMDBX_TOOLS
+ bool "install tools"
+ help
+ Install libmdbx tools for checking, dump, restore
+ and show statistics of databases.
+
+config BR2_PACKAGE_LIBMDBX_CXX
+ bool "C++ API"
+ depends on BR2_INSTALL_LIBSTDCPP
+ depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8
+ depends on !BR2_TOOLCHAIN_HAS_GCC_BUG_64735
+ help
+ Enable modern C++11/14/17/20 API for libmdbx.
+
+comment "libmdbx C++ support needs a toolchain w/ C++11, gcc >= 4.8 w/o bug#64735"
+ depends on !BR2_INSTALL_LIBSTDCPP || \
+ !BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 || \
+ BR2_TOOLCHAIN_HAS_GCC_BUG_64735
+
+endif
+
+comment "libmdbx needs MMU, a toolchain w/ threads, gcc >= 4.4 w/ 4-byte atomics"
+ depends on BR2_USE_MMU
+ depends on !BR2_TOOLCHAIN_HAS_THREADS || \
+ !BR2_TOOLCHAIN_HAS_SYNC_4 || \
+ !BR2_TOOLCHAIN_GCC_AT_LEAST_4_4
@@ -11,7 +11,9 @@ config BR2_PACKAGE_LIBMDBX
solutions.
libmdbx surpasses the legendary LMDB in terms of
- reliability, features and performance.
+ reliability, features and performance. At the end of 2024
+ libmdbx was chosen by all modern Ethereum frontiers/nodes
+ as a storage engine.
https://libmdbx.dqdkfa.ru
diff --git a/package/libmdbx/libmdbx.hash b/package/libmdbx/libmdbx.hash
new file mode 100644
index 0000000000..ae5266716b
--- /dev/null
index ae5266716b..4a4f302015 100644
--- a/package/libmdbx/libmdbx.hash
+++ b/package/libmdbx/libmdbx.hash
@@ -0,0 +1,6 @@
+# Hashes from: https://libmdbx.dqdkfa.ru/release/SHA256SUMS
+sha256 57db987de6f7ccc66a66ae28a7bda9f9fbb48ac5fb9279bcca92fd5de13075d1 libmdbx-amalgamated-0.13.6.tar.xz
+
+# Locally calculated
+sha256 0d542e0c8804e39aa7f37eb00da5a762149dc682d7829451287e11b938e94594 LICENSE
+sha256 651f71b46c6bb0046d2122df7f9def9cb24f4dc28c5b11cef059f66565cda30f NOTICE
@@ -1,5 +1,5 @@
# Hashes from: https://libmdbx.dqdkfa.ru/release/SHA256SUMS
-sha256 57db987de6f7ccc66a66ae28a7bda9f9fbb48ac5fb9279bcca92fd5de13075d1 libmdbx-amalgamated-0.13.6.tar.xz
+sha256 63d2608c8f7c23185c0d27d817d42dd720e84973224ffc584c7f7b522f5f06fe libmdbx-amalgamated-0.13.9.tar.xz
# Locally calculated
sha256 0d542e0c8804e39aa7f37eb00da5a762149dc682d7829451287e11b938e94594 LICENSE
diff --git a/package/libmdbx/libmdbx.mk b/package/libmdbx/libmdbx.mk
new file mode 100644
index 0000000000..571757262e
--- /dev/null
index f461d98397..62817a98f8 100644
--- a/package/libmdbx/libmdbx.mk
+++ b/package/libmdbx/libmdbx.mk
@@ -0,0 +1,42 @@
+################################################################################
+#
+# libmdbx
+#
+################################################################################
+
+LIBMDBX_VERSION = 0.13.6
+LIBMDBX_SOURCE = libmdbx-amalgamated-$(LIBMDBX_VERSION).tar.xz
+LIBMDBX_SITE = https://libmdbx.dqdkfa.ru/release
+LIBMDBX_SUPPORTS_IN_SOURCE_BUILD = NO
+LIBMDBX_LICENSE = Apache-2.0
+LIBMDBX_LICENSE_FILES = LICENSE NOTICE
+LIBMDBX_REDISTRIBUTE = YES
+LIBMDBX_STRIP_COMPONENTS = 0
+LIBMDBX_INSTALL_STAGING = YES
+
+# Set CMAKE_BUILD_TYPE to Release to remove -Werror and avoid a build failure
+# with glibc < 2.12
+LIBMDBX_CONF_OPTS = \
+ -DCMAKE_BUILD_TYPE=Release \
+ -DMDBX_INSTALL_MANPAGES=OFF \
+ -DBUILD_FOR_NATIVE_CPU=OFF \
+ -DMDBX_BUILD_CXX=$(if $(BR2_PACKAGE_LIBMDBX_CXX),ON,OFF) \
+ -DMDBX_BUILD_TOOLS=$(if $(BR2_PACKAGE_LIBMDBX_TOOLS),ON,OFF)
+
+ifeq ($(BR2_STATIC_LIBS)$(BR2_SHARED_STATIC_LIBS),y)
+LIBMDBX_CONF_OPTS += -DMDBX_INSTALL_STATIC=ON
+else
+LIBMDBX_CONF_OPTS += -DMDBX_INSTALL_STATIC=OFF
+endif
+
+ifeq ($(BR2_SHARED_LIBS)$(BR2_SHARED_STATIC_LIBS),y)
+LIBMDBX_CONF_OPTS += \
+ -DMDBX_BUILD_SHARED_LIBRARY=ON \
+ -DMDBX_LINK_TOOLS_NONSTATIC=ON
+else
+LIBMDBX_CONF_OPTS += \
+ -DMDBX_BUILD_SHARED_LIBRARY=OFF \
+ -DMDBX_LINK_TOOLS_NONSTATIC=OFF
+endif
+
+$(eval $(cmake-package))
@@ -4,7 +4,7 @@
#
################################################################################
-LIBMDBX_VERSION = 0.13.6
+LIBMDBX_VERSION = 0.13.9
LIBMDBX_SOURCE = libmdbx-amalgamated-$(LIBMDBX_VERSION).tar.xz
LIBMDBX_SITE = https://libmdbx.dqdkfa.ru/release
LIBMDBX_SUPPORTS_IN_SOURCE_BUILD = NO
--
2.49.0
2.51.2

View File

@@ -479,7 +479,7 @@ __cold static int copy_with_compacting(MDBX_env *env, MDBX_txn *txn, mdbx_fileha
if (meta->geometry.now != meta->geometry.first_unallocated) {
const size_t whole_size = pgno2bytes(env, meta->geometry.now);
if (!dest_is_pipe)
return osal_ftruncate(fd, whole_size);
return osal_fsetsize(fd, whole_size);
const size_t used_size = pgno2bytes(env, meta->geometry.first_unallocated);
memset(data_buffer, 0, (size_t)MDBX_ENVCOPY_WRITEBUF);
@@ -648,7 +648,7 @@ retry_snap_meta:
/* Extend file if required */
if (likely(rc == MDBX_SUCCESS) && whole_size != used_size) {
if (!dest_is_pipe)
rc = osal_ftruncate(fd, whole_size);
rc = osal_fsetsize(fd, whole_size);
else {
memset(data_buffer, 0, (size_t)MDBX_ENVCOPY_WRITEBUF);
for (size_t offset = used_size; rc == MDBX_SUCCESS && offset < whole_size;) {

View File

@@ -208,10 +208,19 @@ __cold int mdbx_env_create(MDBX_env **penv) {
}
#if defined(__linux__) || defined(__gnu_linux__)
if (unlikely(globals.linux_kernel_version < 0x04000000)) {
/* 2022-09-01: Прошло уже более двух лет после окончания какой-либо
* поддержки самого "долгоиграющего" ядра 3.16.85 ветки 3.x */
ERROR("too old linux kernel %u.%u.%u.%u, the >= 4.0.0 is required", globals.linux_kernel_version >> 24,
if (unlikely(globals.linux_kernel_version < 0x03100000)) {
/* 2025-08-05: Ядро 3.16 выпущено 11 лет назад и было самым долго поддерживаемым из 3.x до июля 2020.
* Три года назад (в 2022) здесь была заблокирована работа на ядрах меньше 4.x, как устаревших и для которых
* крайне затруднительно обеспечить какое-либо тестирование. Теперь же я решил изменить решение и разрешить
* работу на старых ядрах начиная с 3.16, логика тут такая:
* - поведение старых ядер уже точно не будет меняться,
* а в текущем коде libmdbx есть всё необходимое для работы начиная с 3.16;
* - есть широко-используемые проекты (Isar), которым требуется поддержка старых ядер;
* - сейчас тестирование для 4.x также затруднено, как и для 3.16, уже не приносит какого-либо облегчения
* с тестированием и мне приходится полагаться на гарантии совместимости API ядра и glibc/musl;
* - использование возможностей из новых ядер всё равно требует проверок/ветвлений;
* = поэтому сейчас нет причин отказываться от работы на 3.16 поддерживая ядра 4.0 */
ERROR("too old linux kernel %u.%u.%u.%u, the >= 3.16 is required", globals.linux_kernel_version >> 24,
(globals.linux_kernel_version >> 16) & 255, (globals.linux_kernel_version >> 8) & 255,
globals.linux_kernel_version & 255);
return LOG_IFERR(MDBX_INCOMPATIBLE);

View File

@@ -171,10 +171,8 @@ MDBX_MAYBE_UNUSED static __always_inline void atomic_yield(void) {
#elif defined(__mips) || defined(__mips__) || defined(__mips64) || defined(__mips64__) || defined(_M_MRX000) || \
defined(_MIPS_) || defined(__MWERKS__) || defined(__sgi)
__asm__ __volatile__(".word 0x00000140");
#elif defined(__linux__) || defined(__gnu_linux__) || defined(_UNIX03_SOURCE)
sched_yield();
#elif (defined(_GNU_SOURCE) && __GLIBC_PREREQ(2, 1)) || defined(_OPEN_THREADS)
pthread_yield();
#else
osal_yield();
#endif
}

View File

@@ -103,15 +103,7 @@ __cold int coherency_timeout(uint64_t *timestamp, intptr_t pgno, const MDBX_env
}
osal_memory_fence(mo_AcquireRelease, true);
#if defined(_WIN32) || defined(_WIN64)
SwitchToThread();
#elif defined(__linux__) || defined(__gnu_linux__) || defined(_UNIX03_SOURCE)
sched_yield();
#elif (defined(_GNU_SOURCE) && __GLIBC_PREREQ(2, 1)) || defined(_OPEN_THREADS)
pthread_yield();
#else
usleep(42);
#endif
osal_yield();
return MDBX_RESULT_TRUE;
}

View File

@@ -63,6 +63,11 @@
#cmakedefine01 MDBX_USE_MINCORE
#cmakedefine MDBX_USE_FALLOCATE_AUTO
#ifndef MDBX_USE_FALLOCATE_AUTO
#cmakedefine01 MDBX_USE_FALLOCATE
#endif /* MDBX_USE_FALLOCATE */
/* Build Info */
#ifndef MDBX_BUILD_TIMESTAMP
#cmakedefine MDBX_BUILD_TIMESTAMP "@MDBX_BUILD_TIMESTAMP@"

View File

@@ -236,7 +236,7 @@ enum cursor_checking {
MDBX_INTERNAL int __must_check_result cursor_validate(const MDBX_cursor *mc);
MDBX_MAYBE_UNUSED MDBX_NOTHROW_PURE_FUNCTION static inline size_t cursor_dbi(const MDBX_cursor *mc) {
cASSERT(mc, mc->txn && mc->txn->signature == txn_signature);
cASSERT(mc, mc->txn->signature == txn_signature);
size_t dbi = mc->dbi_state - mc->txn->dbi_state;
cASSERT(mc, dbi < mc->txn->env->n_dbi);
return dbi;

View File

@@ -5,7 +5,7 @@
#if MDBX_ENABLE_DBI_SPARSE
size_t dbi_bitmap_ctz_fallback(const MDBX_txn *txn, intptr_t bmi) {
tASSERT(txn, bmi > 0);
tASSERT(txn, bmi != 0);
bmi &= -bmi;
if (sizeof(txn->dbi_sparse[0]) > 4) {
static const uint8_t debruijn_ctz64[64] = {0, 1, 2, 53, 3, 7, 54, 27, 4, 38, 41, 8, 34, 55, 48, 28,
@@ -172,13 +172,8 @@ int dbi_defer_release(MDBX_env *const env, defer_free_item_t *const chain) {
#endif /* MDBX_ENABLE_DBI_LOCKFREE */
ENSURE(env, osal_fastmutex_release(&env->dbi_lock) == MDBX_SUCCESS);
if (length > 42) {
#if defined(_WIN32) || defined(_WIN64)
SwitchToThread();
#else
sched_yield();
#endif /* Windows */
}
if (length > 42)
osal_yield();
while (obsolete_chain) {
defer_free_item_t *item = obsolete_chain;
obsolete_chain = obsolete_chain->next;

View File

@@ -11,7 +11,7 @@ MDBX_NOTHROW_CONST_FUNCTION MDBX_MAYBE_UNUSED MDBX_INTERNAL size_t dbi_bitmap_ct
intptr_t bmi);
static inline size_t dbi_bitmap_ctz(const MDBX_txn *txn, intptr_t bmi) {
tASSERT(txn, bmi > 0);
tASSERT(txn, bmi != 0);
STATIC_ASSERT(sizeof(bmi) >= sizeof(txn->dbi_sparse[0]));
#if __GNUC_PREREQ(4, 1) || __has_builtin(__builtin_ctzl)
if (sizeof(txn->dbi_sparse[0]) <= sizeof(int))

View File

@@ -532,7 +532,7 @@ __cold int dxb_setup(MDBX_env *env, const int lck_rc, const mdbx_mode_t mode_bit
if (unlikely(err != MDBX_SUCCESS))
return err;
err = osal_ftruncate(env->lazy_fd, env->dxb_mmap.filesize = env->dxb_mmap.current = env->geo_in_bytes.now);
err = osal_fsetsize(env->lazy_fd, env->dxb_mmap.filesize = env->dxb_mmap.current = env->geo_in_bytes.now);
if (unlikely(err != MDBX_SUCCESS))
return err;
@@ -661,9 +661,8 @@ __cold int dxb_setup(MDBX_env *env, const int lck_rc, const mdbx_mode_t mode_bit
}
if (env->flags & MDBX_RDONLY) {
if (filesize_before & (globals.sys_allocation_granularity - 1)) {
ERROR("filesize should be rounded-up to system allocation granularity %u",
globals.sys_allocation_granularity);
if (filesize_before & (globals.sys_pagesize - 1)) {
ERROR("filesize should be rounded-up to system page size %u", globals.sys_pagesize);
return MDBX_WANNA_RECOVERY;
}
WARNING("%s", "ignore filesize mismatch in readonly-mode");
@@ -682,7 +681,7 @@ __cold int dxb_setup(MDBX_env *env, const int lck_rc, const mdbx_mode_t mode_bit
!(env->flags & MDBX_NORDAHEAD) && mdbx_is_readahead_reasonable(used_bytes, 0) == MDBX_RESULT_TRUE;
err = osal_mmap(env->flags, &env->dxb_mmap, env->geo_in_bytes.now, env->geo_in_bytes.upper,
(lck_rc && env->stuck_meta < 0) ? MMAP_OPTION_TRUNCATE : 0, env->pathname.dxb);
(lck_rc && env->stuck_meta < 0) ? MMAP_OPTION_SETLENGTH : 0, env->pathname.dxb);
if (unlikely(err != MDBX_SUCCESS))
return err;

View File

@@ -379,6 +379,7 @@ __dll_export
#else /* Windows */
" MDBX_LOCKING=" MDBX_LOCKING_CONFIG
" MDBX_USE_OFDLOCKS=" MDBX_USE_OFDLOCKS_CONFIG
" MDBX_USE_FALLOCATE=" MDBX_USE_FALLOCATE_CONFIG
#endif /* !Windows */
" MDBX_CACHELINE_SIZE=" MDBX_STRINGIFY(MDBX_CACHELINE_SIZE)
" MDBX_CPU_WRITEBACK_INCOHERENT=" MDBX_STRINGIFY(MDBX_CPU_WRITEBACK_INCOHERENT)

View File

@@ -155,7 +155,28 @@ static int lck_op(const mdbx_filehandle_t fd, int cmd, const int lck, const off_
}
}
MDBX_INTERNAL int osal_lockfile(mdbx_filehandle_t fd, bool wait) {
static int lck_setlk_with3retries(const mdbx_filehandle_t fd, const int lck, const off_t offset, off_t len) {
assert(lck != F_UNLCK);
int retry_left = 3;
#if defined(__ANDROID_API__)
retry_left *= 3;
#endif /* Android */
while (true) {
int rc = lck_op(fd, op_setlk, lck, offset, len);
if (!(rc == EAGAIN || rc == EACCES || rc == EBUSY || rc == EWOULDBLOCK || rc == EDEADLK) || --retry_left < 1)
return rc;
#if defined(__ANDROID_API__)
if (retry_left == 5 || retry_left == 3) {
usleep(1000 * 42 / 2);
continue;
}
#endif /* Android */
if (osal_yield())
return rc;
}
}
int osal_lockfile(mdbx_filehandle_t fd, bool wait) {
#if MDBX_USE_OFDLOCKS
if (unlikely(op_setlk == 0))
choice_fcntl();
@@ -163,7 +184,7 @@ MDBX_INTERNAL int osal_lockfile(mdbx_filehandle_t fd, bool wait) {
return lck_op(fd, wait ? op_setlkw : op_setlk, F_WRLCK, 0, OFF_T_MAX);
}
MDBX_INTERNAL int lck_rpid_set(MDBX_env *env) {
int lck_rpid_set(MDBX_env *env) {
assert(env->lck_mmap.fd != INVALID_HANDLE_VALUE);
assert(env->pid > 0);
if (unlikely(osal_getpid() != env->pid))
@@ -171,13 +192,13 @@ MDBX_INTERNAL int lck_rpid_set(MDBX_env *env) {
return lck_op(env->lck_mmap.fd, op_setlk, F_WRLCK, env->pid, 1);
}
MDBX_INTERNAL int lck_rpid_clear(MDBX_env *env) {
int lck_rpid_clear(MDBX_env *env) {
assert(env->lck_mmap.fd != INVALID_HANDLE_VALUE);
assert(env->pid > 0);
return lck_op(env->lck_mmap.fd, op_setlk, F_UNLCK, env->pid, 1);
}
MDBX_INTERNAL int lck_rpid_check(MDBX_env *env, uint32_t pid) {
int lck_rpid_check(MDBX_env *env, uint32_t pid) {
assert(env->lck_mmap.fd != INVALID_HANDLE_VALUE);
assert(pid > 0);
return lck_op(env->lck_mmap.fd, op_getlk, F_WRLCK, pid, 1);
@@ -186,7 +207,7 @@ MDBX_INTERNAL int lck_rpid_check(MDBX_env *env, uint32_t pid) {
/*---------------------------------------------------------------------------*/
#if MDBX_LOCKING > MDBX_LOCKING_SYSV
MDBX_INTERNAL int lck_ipclock_stubinit(osal_ipclock_t *ipc) {
int lck_ipclock_stubinit(osal_ipclock_t *ipc) {
#if MDBX_LOCKING == MDBX_LOCKING_POSIX1988
return sem_init(ipc, false, 1) ? errno : 0;
#elif MDBX_LOCKING == MDBX_LOCKING_POSIX2001 || MDBX_LOCKING == MDBX_LOCKING_POSIX2008
@@ -196,7 +217,7 @@ MDBX_INTERNAL int lck_ipclock_stubinit(osal_ipclock_t *ipc) {
#endif
}
MDBX_INTERNAL int lck_ipclock_destroy(osal_ipclock_t *ipc) {
int lck_ipclock_destroy(osal_ipclock_t *ipc) {
#if MDBX_LOCKING == MDBX_LOCKING_POSIX1988
return sem_destroy(ipc) ? errno : 0;
#elif MDBX_LOCKING == MDBX_LOCKING_POSIX2001 || MDBX_LOCKING == MDBX_LOCKING_POSIX2008
@@ -260,7 +281,7 @@ static int check_fstat(MDBX_env *env) {
return rc;
}
__cold MDBX_INTERNAL int lck_seize(MDBX_env *env) {
__cold int lck_seize(MDBX_env *env) {
assert(env->lazy_fd != INVALID_HANDLE_VALUE);
if (unlikely(osal_getpid() != env->pid))
return MDBX_PANIC;
@@ -284,7 +305,7 @@ __cold MDBX_INTERNAL int lck_seize(MDBX_env *env) {
if (env->lck_mmap.fd == INVALID_HANDLE_VALUE) {
/* LY: without-lck mode (e.g. exclusive or on read-only filesystem) */
rc = lck_op(env->lazy_fd, op_setlk, (env->flags & MDBX_RDONLY) ? F_RDLCK : F_WRLCK, 0, OFF_T_MAX);
rc = lck_setlk_with3retries(env->lazy_fd, (env->flags & MDBX_RDONLY) ? F_RDLCK : F_WRLCK, 0, OFF_T_MAX);
if (rc != MDBX_SUCCESS) {
ERROR("%s, err %u", "without-lck", rc);
eASSERT(env, MDBX_IS_ERROR(rc));
@@ -292,9 +313,6 @@ __cold MDBX_INTERNAL int lck_seize(MDBX_env *env) {
}
return MDBX_RESULT_TRUE /* Done: return with exclusive locking. */;
}
#if defined(_POSIX_PRIORITY_SCHEDULING) && _POSIX_PRIORITY_SCHEDULING > 0
sched_yield();
#endif
retry:
if (rc == MDBX_RESULT_TRUE) {
@@ -307,14 +325,14 @@ retry:
}
/* Firstly try to get exclusive locking. */
rc = lck_op(env->lck_mmap.fd, op_setlk, F_WRLCK, 0, 1);
rc = lck_setlk_with3retries(env->lck_mmap.fd, F_WRLCK, 0, 1);
if (rc == MDBX_SUCCESS) {
rc = check_fstat(env);
if (MDBX_IS_ERROR(rc))
return rc;
continue_dxb_exclusive:
rc = lck_op(env->lazy_fd, op_setlk, (env->flags & MDBX_RDONLY) ? F_RDLCK : F_WRLCK, 0, OFF_T_MAX);
rc = lck_setlk_with3retries(env->lazy_fd, (env->flags & MDBX_RDONLY) ? F_RDLCK : F_WRLCK, 0, OFF_T_MAX);
if (rc == MDBX_SUCCESS)
return MDBX_RESULT_TRUE /* Done: return with exclusive locking. */;
@@ -362,7 +380,7 @@ retry:
}
/* got shared, retry exclusive */
rc = lck_op(env->lck_mmap.fd, op_setlk, F_WRLCK, 0, 1);
rc = lck_setlk_with3retries(env->lck_mmap.fd, F_WRLCK, 0, 1);
if (rc == MDBX_SUCCESS)
goto continue_dxb_exclusive;
@@ -373,7 +391,7 @@ retry:
}
/* Lock against another process operating in without-lck or exclusive mode. */
rc = lck_op(env->lazy_fd, op_setlk, (env->flags & MDBX_RDONLY) ? F_RDLCK : F_WRLCK, env->pid, 1);
rc = lck_setlk_with3retries(env->lazy_fd, (env->flags & MDBX_RDONLY) ? F_RDLCK : F_WRLCK, env->pid, 1);
if (rc != MDBX_SUCCESS) {
ERROR("%s, err %u", "lock-against-without-lck", rc);
eASSERT(env, MDBX_IS_ERROR(rc));
@@ -384,7 +402,7 @@ retry:
return MDBX_RESULT_FALSE;
}
MDBX_INTERNAL int lck_downgrade(MDBX_env *env) {
int lck_downgrade(MDBX_env *env) {
assert(env->lck_mmap.fd != INVALID_HANDLE_VALUE);
if (unlikely(osal_getpid() != env->pid))
return MDBX_PANIC;
@@ -396,7 +414,7 @@ MDBX_INTERNAL int lck_downgrade(MDBX_env *env) {
rc = lck_op(env->lazy_fd, op_setlk, F_UNLCK, env->pid + 1, OFF_T_MAX - env->pid - 1);
}
if (rc == MDBX_SUCCESS)
rc = lck_op(env->lck_mmap.fd, op_setlk, F_RDLCK, 0, 1);
rc = lck_setlk_with3retries(env->lck_mmap.fd, F_RDLCK, 0, 1);
if (unlikely(rc != 0)) {
ERROR("%s, err %u", "lck", rc);
assert(MDBX_IS_ERROR(rc));
@@ -404,7 +422,7 @@ MDBX_INTERNAL int lck_downgrade(MDBX_env *env) {
return rc;
}
MDBX_INTERNAL int lck_upgrade(MDBX_env *env, bool dont_wait) {
int lck_upgrade(MDBX_env *env, bool dont_wait) {
assert(env->lck_mmap.fd != INVALID_HANDLE_VALUE);
if (unlikely(osal_getpid() != env->pid))
return MDBX_PANIC;
@@ -415,10 +433,10 @@ MDBX_INTERNAL int lck_upgrade(MDBX_env *env, bool dont_wait) {
rc = (env->pid > 1) ? lck_op(env->lazy_fd, cmd, F_WRLCK, 0, env->pid - 1) : MDBX_SUCCESS;
if (rc == MDBX_SUCCESS) {
rc = lck_op(env->lazy_fd, cmd, F_WRLCK, env->pid + 1, OFF_T_MAX - env->pid - 1);
if (rc != MDBX_SUCCESS && env->pid > 1 && lck_op(env->lazy_fd, op_setlk, F_UNLCK, 0, env->pid - 1))
if (rc != MDBX_SUCCESS && env->pid > 1 && lck_setlk_with3retries(env->lazy_fd, F_UNLCK, 0, env->pid - 1))
rc = MDBX_PANIC;
}
if (rc != MDBX_SUCCESS && lck_op(env->lck_mmap.fd, op_setlk, F_RDLCK, 0, 1))
if (rc != MDBX_SUCCESS && lck_setlk_with3retries(env->lck_mmap.fd, F_RDLCK, 0, 1))
rc = MDBX_PANIC;
}
if (unlikely(rc != 0)) {
@@ -428,7 +446,7 @@ MDBX_INTERNAL int lck_upgrade(MDBX_env *env, bool dont_wait) {
return rc;
}
__cold MDBX_INTERNAL int lck_destroy(MDBX_env *env, MDBX_env *inprocess_neighbor, const uint32_t current_pid) {
__cold int lck_destroy(MDBX_env *env, MDBX_env *inprocess_neighbor, const uint32_t current_pid) {
eASSERT(env, osal_getpid() == current_pid);
int rc = MDBX_SUCCESS;
struct stat lck_info;
@@ -517,7 +535,7 @@ __cold MDBX_INTERNAL int lck_destroy(MDBX_env *env, MDBX_env *inprocess_neighbor
/*---------------------------------------------------------------------------*/
__cold MDBX_INTERNAL int lck_init(MDBX_env *env, MDBX_env *inprocess_neighbor, int global_uniqueness_flag) {
__cold int lck_init(MDBX_env *env, MDBX_env *inprocess_neighbor, int global_uniqueness_flag) {
#if MDBX_LOCKING == MDBX_LOCKING_SYSV
int semid = -1;
/* don't initialize semaphores twice */
@@ -727,7 +745,7 @@ __cold static int osal_ipclock_failed(MDBX_env *env, osal_ipclock_t *ipc, const
}
#if defined(__ANDROID_API__) || defined(ANDROID) || defined(BIONIC)
MDBX_INTERNAL int osal_check_tid4bionic(void) {
int osal_check_tid4bionic(void) {
/* avoid 32-bit Bionic bug/hang with 32-pit TID */
if (sizeof(pthread_mutex_t) < sizeof(pid_t) + sizeof(unsigned)) {
pid_t tid = gettid();
@@ -810,7 +828,7 @@ static int osal_ipclock_unlock(MDBX_env *env, osal_ipclock_t *ipc) {
return rc;
}
MDBX_INTERNAL int lck_rdt_lock(MDBX_env *env) {
int lck_rdt_lock(MDBX_env *env) {
TRACE("%s", ">>");
jitter4testing(true);
int rc = osal_ipclock_lock(env, &env->lck->rdt_lock, false);
@@ -818,7 +836,7 @@ MDBX_INTERNAL int lck_rdt_lock(MDBX_env *env) {
return rc;
}
MDBX_INTERNAL void lck_rdt_unlock(MDBX_env *env) {
void lck_rdt_unlock(MDBX_env *env) {
TRACE("%s", ">>");
int err = osal_ipclock_unlock(env, &env->lck->rdt_lock);
TRACE("<< err %d", err);

View File

@@ -140,7 +140,7 @@ void lck_txn_unlock(MDBX_env *env) {
#define LCK_LOWER LCK_LO_OFFSET, LCK_LO_LEN
#define LCK_UPPER LCK_UP_OFFSET, LCK_UP_LEN
MDBX_INTERNAL int lck_rdt_lock(MDBX_env *env) {
int lck_rdt_lock(MDBX_env *env) {
imports.srwl_AcquireShared(&env->remap_guard);
if (env->lck_mmap.fd == INVALID_HANDLE_VALUE)
return MDBX_SUCCESS; /* readonly database in readonly filesystem */
@@ -158,7 +158,7 @@ MDBX_INTERNAL int lck_rdt_lock(MDBX_env *env) {
return rc;
}
MDBX_INTERNAL void lck_rdt_unlock(MDBX_env *env) {
void lck_rdt_unlock(MDBX_env *env) {
if (env->lck_mmap.fd != INVALID_HANDLE_VALUE && (env->flags & MDBX_EXCLUSIVE) == 0) {
/* transition from S-E (locked) to S-? (used), e.g. unlock upper-part */
int err = funlock(env->lck_mmap.fd, LCK_UPPER);
@@ -168,7 +168,7 @@ MDBX_INTERNAL void lck_rdt_unlock(MDBX_env *env) {
imports.srwl_ReleaseShared(&env->remap_guard);
}
MDBX_INTERNAL int osal_lockfile(mdbx_filehandle_t fd, bool wait) {
int osal_lockfile(mdbx_filehandle_t fd, bool wait) {
return flock(fd, wait ? LCK_EXCLUSIVE | LCK_WAITFOR : LCK_EXCLUSIVE | LCK_DONTWAIT, 0, DXB_MAXLEN);
}
@@ -204,7 +204,7 @@ static int suspend_and_append(mdbx_handle_array_t **array, const DWORD ThreadId)
return MDBX_SUCCESS;
}
MDBX_INTERNAL int osal_suspend_threads_before_remap(MDBX_env *env, mdbx_handle_array_t **array) {
int osal_suspend_threads_before_remap(MDBX_env *env, mdbx_handle_array_t **array) {
eASSERT(env, (env->flags & MDBX_NOSTICKYTHREADS) == 0);
const uintptr_t CurrentTid = GetCurrentThreadId();
int rc;
@@ -271,7 +271,7 @@ MDBX_INTERNAL int osal_suspend_threads_before_remap(MDBX_env *env, mdbx_handle_a
return MDBX_SUCCESS;
}
MDBX_INTERNAL int osal_resume_threads_after_remap(mdbx_handle_array_t *array) {
int osal_resume_threads_after_remap(mdbx_handle_array_t *array) {
int rc = MDBX_SUCCESS;
for (unsigned i = 0; i < array->count; ++i) {
const HANDLE hThread = array->handles[i];
@@ -407,7 +407,7 @@ static int internal_seize_lck(HANDLE lfd) {
return rc;
}
MDBX_INTERNAL int lck_seize(MDBX_env *env) {
int lck_seize(MDBX_env *env) {
const HANDLE fd4data = env->ioring.overlapped_fd ? env->ioring.overlapped_fd : env->lazy_fd;
assert(fd4data != INVALID_HANDLE_VALUE);
if (env->flags & MDBX_EXCLUSIVE)
@@ -449,7 +449,7 @@ MDBX_INTERNAL int lck_seize(MDBX_env *env) {
return rc;
}
MDBX_INTERNAL int lck_downgrade(MDBX_env *env) {
int lck_downgrade(MDBX_env *env) {
const HANDLE fd4data = env->ioring.overlapped_fd ? env->ioring.overlapped_fd : env->lazy_fd;
/* Transite from exclusive-write state (E-E) to used (S-?) */
assert(fd4data != INVALID_HANDLE_VALUE);
@@ -479,7 +479,7 @@ MDBX_INTERNAL int lck_downgrade(MDBX_env *env) {
return MDBX_SUCCESS /* 5) now at S-? (used), done */;
}
MDBX_INTERNAL int lck_upgrade(MDBX_env *env, bool dont_wait) {
int lck_upgrade(MDBX_env *env, bool dont_wait) {
/* Transite from used state (S-?) to exclusive-write (E-E) */
assert(env->lck_mmap.fd != INVALID_HANDLE_VALUE);
@@ -513,7 +513,7 @@ MDBX_INTERNAL int lck_upgrade(MDBX_env *env, bool dont_wait) {
return MDBX_SUCCESS /* 6) now at E-E (exclusive-write), done */;
}
MDBX_INTERNAL int lck_init(MDBX_env *env, MDBX_env *inprocess_neighbor, int global_uniqueness_flag) {
int lck_init(MDBX_env *env, MDBX_env *inprocess_neighbor, int global_uniqueness_flag) {
(void)env;
(void)inprocess_neighbor;
(void)global_uniqueness_flag;
@@ -534,7 +534,7 @@ MDBX_INTERNAL int lck_init(MDBX_env *env, MDBX_env *inprocess_neighbor, int glob
return MDBX_SUCCESS;
}
MDBX_INTERNAL int lck_destroy(MDBX_env *env, MDBX_env *inprocess_neighbor, const uint32_t current_pid) {
int lck_destroy(MDBX_env *env, MDBX_env *inprocess_neighbor, const uint32_t current_pid) {
(void)current_pid;
/* LY: should unmap before releasing the locks to avoid race condition and
* STATUS_USER_MAPPED_FILE/ERROR_USER_MAPPED_FILE */
@@ -546,7 +546,7 @@ MDBX_INTERNAL int lck_destroy(MDBX_env *env, MDBX_env *inprocess_neighbor, const
if (synced && !inprocess_neighbor && env->lck_mmap.fd != INVALID_HANDLE_VALUE &&
lck_upgrade(env, true) == MDBX_SUCCESS)
/* this will fail if LCK is used/mmapped by other process(es) */
osal_ftruncate(env->lck_mmap.fd, 0);
osal_fsetsize(env->lck_mmap.fd, 0);
}
lck_unlock(env);
return MDBX_SUCCESS;
@@ -555,12 +555,12 @@ MDBX_INTERNAL int lck_destroy(MDBX_env *env, MDBX_env *inprocess_neighbor, const
/*----------------------------------------------------------------------------*/
/* reader checking (by pid) */
MDBX_INTERNAL int lck_rpid_set(MDBX_env *env) {
int lck_rpid_set(MDBX_env *env) {
(void)env;
return MDBX_SUCCESS;
}
MDBX_INTERNAL int lck_rpid_clear(MDBX_env *env) {
int lck_rpid_clear(MDBX_env *env) {
(void)env;
return MDBX_SUCCESS;
}
@@ -571,7 +571,7 @@ MDBX_INTERNAL int lck_rpid_clear(MDBX_env *env) {
* MDBX_RESULT_TRUE, if pid is live (unable to acquire lock)
* MDBX_RESULT_FALSE, if pid is dead (lock acquired)
* or otherwise the errcode. */
MDBX_INTERNAL int lck_rpid_check(MDBX_env *env, uint32_t pid) {
int lck_rpid_check(MDBX_env *env, uint32_t pid) {
(void)env;
HANDLE hProcess = OpenProcess(SYNCHRONIZE, FALSE, pid);
int rc;

View File

@@ -62,9 +62,9 @@ __cold static int lck_setup_locked(MDBX_env *env) {
}
env->max_readers = (maxreaders <= MDBX_READERS_LIMIT) ? (unsigned)maxreaders : (unsigned)MDBX_READERS_LIMIT;
err =
osal_mmap((env->flags & MDBX_EXCLUSIVE) | MDBX_WRITEMAP, &env->lck_mmap, (size_t)size, (size_t)size,
lck_seize_rc ? MMAP_OPTION_TRUNCATE | MMAP_OPTION_SEMAPHORE : MMAP_OPTION_SEMAPHORE, env->pathname.lck);
err = osal_mmap((env->flags & MDBX_EXCLUSIVE) | MDBX_WRITEMAP, &env->lck_mmap, (size_t)size, (size_t)size,
lck_seize_rc ? MMAP_OPTION_SETLENGTH | MMAP_OPTION_SEMAPHORE : MMAP_OPTION_SEMAPHORE,
env->pathname.lck);
if (unlikely(err != MDBX_SUCCESS))
return err;

View File

@@ -204,7 +204,7 @@ static bool pid_insert(uint32_t *list, uint32_t pid) {
return true;
}
__cold MDBX_INTERNAL int mvcc_cleanup_dead(MDBX_env *env, int rdt_locked, int *dead) {
__cold int mvcc_cleanup_dead(MDBX_env *env, int rdt_locked, int *dead) {
int rc = check_env(env, true);
if (unlikely(rc != MDBX_SUCCESS))
return rc;

View File

@@ -295,7 +295,8 @@
((defined(_POSIX_THREAD_ROBUST_PRIO_INHERIT) && _POSIX_THREAD_ROBUST_PRIO_INHERIT > 0) || \
(defined(_POSIX_THREAD_ROBUST_PRIO_PROTECT) && _POSIX_THREAD_ROBUST_PRIO_PROTECT > 0) || \
defined(PTHREAD_MUTEX_ROBUST) || defined(PTHREAD_MUTEX_ROBUST_NP)) && \
(!defined(__GLIBC__) || __GLIBC_PREREQ(2, 10) /* troubles with Robust mutexes before 2.10 */)
(!defined(__GLIBC__) || __GLIBC_PREREQ(2, 10) /* troubles with Robust mutexes before 2.10 */) && \
!defined(__OHOS__) /* Harmony OS doesn't support robust mutexes at the end of 2025 */
#define MDBX_LOCKING MDBX_LOCKING_POSIX2008
#else
#define MDBX_LOCKING MDBX_LOCKING_POSIX2001
@@ -350,6 +351,22 @@
#error MDBX_USE_COPYFILERANGE must be defined as 0 or 1
#endif /* MDBX_USE_COPYFILERANGE */
/** Advanced: Using posix_fallocate() or fcntl(F_PREALLOCATE) on OSX (autodetection by default). */
#ifndef MDBX_USE_FALLOCATE
#if defined(__APPLE__)
#define MDBX_USE_FALLOCATE 0 /* Too slow and unclean, but not required to prevent SIGBUS */
#elif (defined(_POSIX_C_SOURCE) && _POSIX_C_SOURCE >= 200112L) || (__GLIBC_PREREQ(2, 10) && defined(_GNU_SOURCE))
#define MDBX_USE_FALLOCATE 1
#else
#define MDBX_USE_FALLOCATE 0
#endif
#define MDBX_USE_FALLOCATE_CONFIG "AUTO=" MDBX_STRINGIFY(MDBX_USE_FALLOCATE)
#elif !(MDBX_USE_FALLOCATE == 0 || MDBX_USE_FALLOCATE == 1)
#error MDBX_USE_FALLOCATE must be defined as 0 or 1
#else
#define MDBX_USE_FALLOCATE_CONFIG MDBX_STRINGIFY(MDBX_USE_FALLOCATE)
#endif /* MDBX_USE_FALLOCATE */
//------------------------------------------------------------------------------
#ifndef MDBX_CPU_WRITEBACK_INCOHERENT

View File

@@ -271,7 +271,7 @@ __cold void mdbx_panic(const char *fmt, ...) {
/*----------------------------------------------------------------------------*/
#ifndef osal_vasprintf
MDBX_INTERNAL int osal_vasprintf(char **strp, const char *fmt, va_list ap) {
int osal_vasprintf(char **strp, const char *fmt, va_list ap) {
va_list ones;
va_copy(ones, ap);
const int needed = vsnprintf(nullptr, 0, fmt, ones);
@@ -303,7 +303,7 @@ MDBX_INTERNAL int osal_vasprintf(char **strp, const char *fmt, va_list ap) {
#endif /* osal_vasprintf */
#ifndef osal_asprintf
MDBX_INTERNAL int osal_asprintf(char **strp, const char *fmt, ...) {
int osal_asprintf(char **strp, const char *fmt, ...) {
va_list ap;
va_start(ap, fmt);
const int rc = osal_vasprintf(strp, fmt, ap);
@@ -313,7 +313,7 @@ MDBX_INTERNAL int osal_asprintf(char **strp, const char *fmt, ...) {
#endif /* osal_asprintf */
#ifndef osal_memalign_alloc
MDBX_INTERNAL int osal_memalign_alloc(size_t alignment, size_t bytes, void **result) {
int osal_memalign_alloc(size_t alignment, size_t bytes, void **result) {
assert(is_powerof2(alignment) && alignment >= sizeof(void *));
#if defined(_WIN32) || defined(_WIN64)
(void)alignment;
@@ -335,7 +335,7 @@ MDBX_INTERNAL int osal_memalign_alloc(size_t alignment, size_t bytes, void **res
#endif /* osal_memalign_alloc */
#ifndef osal_memalign_free
MDBX_INTERNAL void osal_memalign_free(void *ptr) {
void osal_memalign_free(void *ptr) {
#if defined(_WIN32) || defined(_WIN64)
VirtualFree(ptr, 0, MEM_RELEASE);
#else
@@ -358,7 +358,7 @@ char *osal_strdup(const char *str) {
/*----------------------------------------------------------------------------*/
MDBX_INTERNAL int osal_condpair_init(osal_condpair_t *condpair) {
int osal_condpair_init(osal_condpair_t *condpair) {
int rc;
memset(condpair, 0, sizeof(osal_condpair_t));
#if defined(_WIN32) || defined(_WIN64)
@@ -397,7 +397,7 @@ bailout_mutex:
return rc;
}
MDBX_INTERNAL int osal_condpair_destroy(osal_condpair_t *condpair) {
int osal_condpair_destroy(osal_condpair_t *condpair) {
#if defined(_WIN32) || defined(_WIN64)
int rc = CloseHandle(condpair->mutex) ? MDBX_SUCCESS : (int)GetLastError();
rc = CloseHandle(condpair->event[0]) ? rc : (int)GetLastError();
@@ -411,7 +411,7 @@ MDBX_INTERNAL int osal_condpair_destroy(osal_condpair_t *condpair) {
return rc;
}
MDBX_INTERNAL int osal_condpair_lock(osal_condpair_t *condpair) {
int osal_condpair_lock(osal_condpair_t *condpair) {
#if defined(_WIN32) || defined(_WIN64)
DWORD code = WaitForSingleObject(condpair->mutex, INFINITE);
return waitstatus2errcode(code);
@@ -420,7 +420,7 @@ MDBX_INTERNAL int osal_condpair_lock(osal_condpair_t *condpair) {
#endif
}
MDBX_INTERNAL int osal_condpair_unlock(osal_condpair_t *condpair) {
int osal_condpair_unlock(osal_condpair_t *condpair) {
#if defined(_WIN32) || defined(_WIN64)
return ReleaseMutex(condpair->mutex) ? MDBX_SUCCESS : (int)GetLastError();
#else
@@ -428,7 +428,7 @@ MDBX_INTERNAL int osal_condpair_unlock(osal_condpair_t *condpair) {
#endif
}
MDBX_INTERNAL int osal_condpair_signal(osal_condpair_t *condpair, bool part) {
int osal_condpair_signal(osal_condpair_t *condpair, bool part) {
#if defined(_WIN32) || defined(_WIN64)
return SetEvent(condpair->event[part]) ? MDBX_SUCCESS : (int)GetLastError();
#else
@@ -436,7 +436,7 @@ MDBX_INTERNAL int osal_condpair_signal(osal_condpair_t *condpair, bool part) {
#endif
}
MDBX_INTERNAL int osal_condpair_wait(osal_condpair_t *condpair, bool part) {
int osal_condpair_wait(osal_condpair_t *condpair, bool part) {
#if defined(_WIN32) || defined(_WIN64)
DWORD code = SignalObjectAndWait(condpair->mutex, condpair->event[part], INFINITE, FALSE);
if (code == WAIT_OBJECT_0) {
@@ -452,7 +452,7 @@ MDBX_INTERNAL int osal_condpair_wait(osal_condpair_t *condpair, bool part) {
/*----------------------------------------------------------------------------*/
MDBX_INTERNAL int osal_fastmutex_init(osal_fastmutex_t *fastmutex) {
int osal_fastmutex_init(osal_fastmutex_t *fastmutex) {
#if defined(_WIN32) || defined(_WIN64)
InitializeCriticalSection(fastmutex);
return MDBX_SUCCESS;
@@ -471,7 +471,7 @@ MDBX_INTERNAL int osal_fastmutex_init(osal_fastmutex_t *fastmutex) {
#endif
}
MDBX_INTERNAL int osal_fastmutex_destroy(osal_fastmutex_t *fastmutex) {
int osal_fastmutex_destroy(osal_fastmutex_t *fastmutex) {
#if defined(_WIN32) || defined(_WIN64)
DeleteCriticalSection(fastmutex);
return MDBX_SUCCESS;
@@ -480,7 +480,7 @@ MDBX_INTERNAL int osal_fastmutex_destroy(osal_fastmutex_t *fastmutex) {
#endif
}
MDBX_INTERNAL int osal_fastmutex_acquire(osal_fastmutex_t *fastmutex) {
int osal_fastmutex_acquire(osal_fastmutex_t *fastmutex) {
#if defined(_WIN32) || defined(_WIN64)
__try {
EnterCriticalSection(fastmutex);
@@ -495,7 +495,7 @@ MDBX_INTERNAL int osal_fastmutex_acquire(osal_fastmutex_t *fastmutex) {
#endif
}
MDBX_INTERNAL int osal_fastmutex_release(osal_fastmutex_t *fastmutex) {
int osal_fastmutex_release(osal_fastmutex_t *fastmutex) {
#if defined(_WIN32) || defined(_WIN64)
LeaveCriticalSection(fastmutex);
return MDBX_SUCCESS;
@@ -508,7 +508,7 @@ MDBX_INTERNAL int osal_fastmutex_release(osal_fastmutex_t *fastmutex) {
#if defined(_WIN32) || defined(_WIN64)
MDBX_INTERNAL int osal_mb2w(const char *const src, wchar_t **const pdst) {
int osal_mb2w(const char *const src, wchar_t **const pdst) {
const size_t dst_wlen = MultiByteToWideChar(CP_THREAD_ACP, MB_ERR_INVALID_CHARS, src, -1, nullptr, 0);
wchar_t *dst = *pdst;
int rc = ERROR_INVALID_NAME;
@@ -578,10 +578,10 @@ static size_t osal_iov_max;
#undef OSAL_IOV_MAX
#endif /* OSAL_IOV_MAX */
MDBX_INTERNAL int osal_ioring_create(osal_ioring_t *ior
int osal_ioring_create(osal_ioring_t *ior
#if defined(_WIN32) || defined(_WIN64)
,
bool enable_direct, mdbx_filehandle_t overlapped_fd
,
bool enable_direct, mdbx_filehandle_t overlapped_fd
#endif /* Windows */
) {
memset(ior, 0, sizeof(osal_ioring_t));
@@ -624,7 +624,7 @@ static inline ior_item_t *ior_next(ior_item_t *item, size_t sgvcnt) {
#endif
}
MDBX_INTERNAL int osal_ioring_add(osal_ioring_t *ior, const size_t offset, void *data, const size_t bytes) {
int osal_ioring_add(osal_ioring_t *ior, const size_t offset, void *data, const size_t bytes) {
assert(bytes && data);
assert(bytes % MDBX_MIN_PAGESIZE == 0 && bytes <= MAX_WRITE);
assert(offset % MDBX_MIN_PAGESIZE == 0 && offset + (uint64_t)bytes <= MAX_MAPSIZE);
@@ -736,8 +736,8 @@ MDBX_INTERNAL int osal_ioring_add(osal_ioring_t *ior, const size_t offset, void
return MDBX_SUCCESS;
}
MDBX_INTERNAL void osal_ioring_walk(osal_ioring_t *ior, iov_ctx_t *ctx,
void (*callback)(iov_ctx_t *ctx, size_t offset, void *data, size_t bytes)) {
void osal_ioring_walk(osal_ioring_t *ior, iov_ctx_t *ctx,
void (*callback)(iov_ctx_t *ctx, size_t offset, void *data, size_t bytes)) {
for (ior_item_t *item = ior->pool; item <= ior->last;) {
#if defined(_WIN32) || defined(_WIN64)
size_t offset = ior_offset(item);
@@ -778,7 +778,7 @@ MDBX_INTERNAL void osal_ioring_walk(osal_ioring_t *ior, iov_ctx_t *ctx,
}
}
MDBX_INTERNAL osal_ioring_write_result_t osal_ioring_write(osal_ioring_t *ior, mdbx_filehandle_t fd) {
osal_ioring_write_result_t osal_ioring_write(osal_ioring_t *ior, mdbx_filehandle_t fd) {
osal_ioring_write_result_t r = {MDBX_SUCCESS, 0};
#if defined(_WIN32) || defined(_WIN64)
@@ -999,7 +999,7 @@ MDBX_INTERNAL osal_ioring_write_result_t osal_ioring_write(osal_ioring_t *ior, m
return r;
}
MDBX_INTERNAL void osal_ioring_reset(osal_ioring_t *ior) {
void osal_ioring_reset(osal_ioring_t *ior) {
#if defined(_WIN32) || defined(_WIN64)
if (ior->last) {
for (ior_item_t *item = ior->pool; item <= ior->last;) {
@@ -1041,7 +1041,7 @@ static void ior_cleanup(osal_ioring_t *ior, const size_t since) {
#endif /* Windows */
}
MDBX_INTERNAL int osal_ioring_resize(osal_ioring_t *ior, size_t items) {
int osal_ioring_resize(osal_ioring_t *ior, size_t items) {
assert(items > 0 && items < INT_MAX / sizeof(ior_item_t));
#if defined(_WIN32) || defined(_WIN64)
if (ior->state & IOR_STATE_LOCKED)
@@ -1093,7 +1093,7 @@ MDBX_INTERNAL int osal_ioring_resize(osal_ioring_t *ior, size_t items) {
return MDBX_SUCCESS;
}
MDBX_INTERNAL void osal_ioring_destroy(osal_ioring_t *ior) {
void osal_ioring_destroy(osal_ioring_t *ior) {
if (ior->allocated)
ior_cleanup(ior, 0);
#if defined(_WIN32) || defined(_WIN64)
@@ -1110,7 +1110,7 @@ MDBX_INTERNAL void osal_ioring_destroy(osal_ioring_t *ior) {
/*----------------------------------------------------------------------------*/
MDBX_INTERNAL int osal_removefile(const pathchar_t *pathname) {
int osal_removefile(const pathchar_t *pathname) {
#if defined(_WIN32) || defined(_WIN64)
return DeleteFileW(pathname) ? MDBX_SUCCESS : (int)GetLastError();
#else
@@ -1122,7 +1122,7 @@ MDBX_INTERNAL int osal_removefile(const pathchar_t *pathname) {
static bool is_valid_fd(int fd) { return !(isatty(fd) < 0 && errno == EBADF); }
#endif /*! Windows */
MDBX_INTERNAL int osal_removedirectory(const pathchar_t *pathname) {
int osal_removedirectory(const pathchar_t *pathname) {
#if defined(_WIN32) || defined(_WIN64)
return RemoveDirectoryW(pathname) ? MDBX_SUCCESS : (int)GetLastError();
#else
@@ -1130,7 +1130,7 @@ MDBX_INTERNAL int osal_removedirectory(const pathchar_t *pathname) {
#endif
}
MDBX_INTERNAL int osal_fileexists(const pathchar_t *pathname) {
int osal_fileexists(const pathchar_t *pathname) {
#if defined(_WIN32) || defined(_WIN64)
if (GetFileAttributesW(pathname) != INVALID_FILE_ATTRIBUTES)
return MDBX_RESULT_TRUE;
@@ -1144,7 +1144,7 @@ MDBX_INTERNAL int osal_fileexists(const pathchar_t *pathname) {
#endif
}
MDBX_INTERNAL pathchar_t *osal_fileext(const pathchar_t *pathname, size_t len) {
pathchar_t *osal_fileext(const pathchar_t *pathname, size_t len) {
const pathchar_t *ext = nullptr;
for (size_t i = 0; i < len && pathname[i]; i++)
if (pathname[i] == '.')
@@ -1154,7 +1154,7 @@ MDBX_INTERNAL pathchar_t *osal_fileext(const pathchar_t *pathname, size_t len) {
return (pathchar_t *)ext;
}
MDBX_INTERNAL bool osal_pathequal(const pathchar_t *l, const pathchar_t *r, size_t len) {
bool osal_pathequal(const pathchar_t *l, const pathchar_t *r, size_t len) {
#if defined(_WIN32) || defined(_WIN64)
for (size_t i = 0; i < len; ++i) {
pathchar_t a = l[i];
@@ -1170,8 +1170,8 @@ MDBX_INTERNAL bool osal_pathequal(const pathchar_t *l, const pathchar_t *r, size
#endif
}
MDBX_INTERNAL int osal_openfile(const enum osal_openfile_purpose purpose, const MDBX_env *env,
const pathchar_t *pathname, mdbx_filehandle_t *fd, mdbx_mode_t unix_mode_bits) {
int osal_openfile(const enum osal_openfile_purpose purpose, const MDBX_env *env, const pathchar_t *pathname,
mdbx_filehandle_t *fd, mdbx_mode_t unix_mode_bits) {
*fd = INVALID_HANDLE_VALUE;
#if defined(_WIN32) || defined(_WIN64)
@@ -1380,7 +1380,7 @@ MDBX_INTERNAL int osal_openfile(const enum osal_openfile_purpose purpose, const
return MDBX_SUCCESS;
}
MDBX_INTERNAL int osal_closefile(mdbx_filehandle_t fd) {
int osal_closefile(mdbx_filehandle_t fd) {
#if defined(_WIN32) || defined(_WIN64)
return CloseHandle(fd) ? MDBX_SUCCESS : (int)GetLastError();
#else
@@ -1389,7 +1389,7 @@ MDBX_INTERNAL int osal_closefile(mdbx_filehandle_t fd) {
#endif
}
MDBX_INTERNAL int osal_pread(mdbx_filehandle_t fd, void *buf, size_t bytes, uint64_t offset) {
int osal_pread(mdbx_filehandle_t fd, void *buf, size_t bytes, uint64_t offset) {
if (bytes > MAX_WRITE)
return MDBX_EINVAL;
#if defined(_WIN32) || defined(_WIN64)
@@ -1414,7 +1414,7 @@ MDBX_INTERNAL int osal_pread(mdbx_filehandle_t fd, void *buf, size_t bytes, uint
return (bytes == (size_t)read) ? MDBX_SUCCESS : MDBX_ENODATA;
}
MDBX_INTERNAL int osal_pwrite(mdbx_filehandle_t fd, const void *buf, size_t bytes, uint64_t offset) {
int osal_pwrite(mdbx_filehandle_t fd, const void *buf, size_t bytes, uint64_t offset) {
while (true) {
#if defined(_WIN32) || defined(_WIN64)
OVERLAPPED ov;
@@ -1445,7 +1445,7 @@ MDBX_INTERNAL int osal_pwrite(mdbx_filehandle_t fd, const void *buf, size_t byte
}
}
MDBX_INTERNAL int osal_write(mdbx_filehandle_t fd, const void *buf, size_t bytes) {
int osal_write(mdbx_filehandle_t fd, const void *buf, size_t bytes) {
while (true) {
#if defined(_WIN32) || defined(_WIN64)
DWORD written;
@@ -1498,7 +1498,7 @@ int osal_pwritev(mdbx_filehandle_t fd, struct iovec *iov, size_t sgvcnt, uint64_
#endif
}
MDBX_INTERNAL int osal_fsync(mdbx_filehandle_t fd, enum osal_syncmode_bits mode_bits) {
int osal_fsync(mdbx_filehandle_t fd, enum osal_syncmode_bits mode_bits) {
#if defined(_WIN32) || defined(_WIN64)
if ((mode_bits & (MDBX_SYNC_DATA | MDBX_SYNC_IODQ)) && !FlushFileBuffers(fd))
return (int)GetLastError();
@@ -1561,7 +1561,7 @@ int osal_filesize(mdbx_filehandle_t fd, uint64_t *length) {
return MDBX_SUCCESS;
}
MDBX_INTERNAL int osal_is_pipe(mdbx_filehandle_t fd) {
int osal_is_pipe(mdbx_filehandle_t fd) {
#if defined(_WIN32) || defined(_WIN64)
switch (GetFileType(fd)) {
case FILE_TYPE_DISK:
@@ -1592,7 +1592,7 @@ MDBX_INTERNAL int osal_is_pipe(mdbx_filehandle_t fd) {
#endif
}
MDBX_INTERNAL int osal_ftruncate(mdbx_filehandle_t fd, uint64_t length) {
int osal_fsetsize(mdbx_filehandle_t fd, const uint64_t length) {
#if defined(_WIN32) || defined(_WIN64)
if (imports.SetFileInformationByHandle) {
FILE_END_OF_FILE_INFO EndOfFileInfo;
@@ -1607,11 +1607,61 @@ MDBX_INTERNAL int osal_ftruncate(mdbx_filehandle_t fd, uint64_t length) {
}
#else
STATIC_ASSERT_MSG(sizeof(off_t) >= sizeof(size_t), "libmdbx requires 64-bit file I/O on 64-bit systems");
return ftruncate(fd, length) == 0 ? MDBX_SUCCESS : errno;
#if MDBX_USE_FALLOCATE
struct stat info;
if (unlikely(fstat(fd, &info)))
return errno;
const uint64_t allocated = UINT64_C(512) * info.st_blocks;
if (length > allocated) {
#if defined(__APPLE__)
fstore_t store = {
.fst_flags = F_ALLOCATECONTIG, .fst_posmode = F_PEOFPOSMODE, .fst_offset = 0, .fst_length = length};
int err = MDBX_SUCCESS;
if (fcntl(fd, F_PREALLOCATE, &store)) {
/* TODO: implement step-by-step allocation in chunks of 16384, 8192, 4094, 2048, 1024 Kb */
store.fst_flags = F_ALLOCATEALL;
if (fcntl(fd, F_PREALLOCATE, &store))
err = errno;
}
#elif defined(_POSIX_C_SOURCE) && _POSIX_C_SOURCE >= 200112L
const int err = posix_fallocate(fd, 0, length);
if (!err && length > (uint64_t)info.st_size)
info.st_size = length /* posix_fallocate() extends the file */;
#else
const int err = fallocate(fd, 0, 0, length) ? errno : MDBX_SUCCESS;
if (!err && length > (uint64_t)info.st_size)
info.st_size = length /* fallocate() extends the file */;
#endif
if (unlikely(err) && ignore_enosys_and_eremote(err) != MDBX_RESULT_TRUE) {
/* Workaround for testing: ignore ENOSPC for TMPFS/RAMFS.
* This is insignificant for production, but it helps in some tests using /dev/shm inside docker/containers. */
if (err != ENOSPC || osal_check_fs_incore(fd) != MDBX_RESULT_TRUE)
return err;
}
}
if (length == (uint64_t)info.st_size)
return MDBX_SUCCESS;
#endif
#if defined(__linux__) || defined(__gnu_linux__)
if (globals.linux_kernel_version < 0x05110000 && globals.linux_kernel_version >= 0x050a0000) {
struct statfs statfs_info;
if (fstatfs(fd, &statfs_info))
return errno;
if (statfs_info.f_type == 0xEF53 /* EXT4_SUPER_MAGIC */ && unlikely(fdatasync(fd)))
return errno;
}
#endif /* Linux */
return unlikely(ftruncate(fd, length)) ? errno : MDBX_SUCCESS;
#endif /* !Windows */
}
MDBX_INTERNAL int osal_fseek(mdbx_filehandle_t fd, uint64_t pos) {
int osal_fseek(mdbx_filehandle_t fd, uint64_t pos) {
#if defined(_WIN32) || defined(_WIN64)
LARGE_INTEGER li;
li.QuadPart = pos;
@@ -1624,8 +1674,7 @@ MDBX_INTERNAL int osal_fseek(mdbx_filehandle_t fd, uint64_t pos) {
/*----------------------------------------------------------------------------*/
MDBX_INTERNAL int osal_thread_create(osal_thread_t *thread, THREAD_RESULT(THREAD_CALL *start_routine)(void *),
void *arg) {
int osal_thread_create(osal_thread_t *thread, THREAD_RESULT(THREAD_CALL *start_routine)(void *), void *arg) {
#if defined(_WIN32) || defined(_WIN64)
*thread = CreateThread(nullptr, 0, start_routine, arg, 0, nullptr);
return *thread ? MDBX_SUCCESS : (int)GetLastError();
@@ -1634,7 +1683,7 @@ MDBX_INTERNAL int osal_thread_create(osal_thread_t *thread, THREAD_RESULT(THREAD
#endif
}
MDBX_INTERNAL int osal_thread_join(osal_thread_t thread) {
int osal_thread_join(osal_thread_t thread) {
#if defined(_WIN32) || defined(_WIN64)
DWORD code = WaitForSingleObject(thread, INFINITE);
return waitstatus2errcode(code);
@@ -1646,7 +1695,7 @@ MDBX_INTERNAL int osal_thread_join(osal_thread_t thread) {
/*----------------------------------------------------------------------------*/
MDBX_INTERNAL int osal_msync(const osal_mmap_t *map, size_t offset, size_t length, enum osal_syncmode_bits mode_bits) {
int osal_msync(const osal_mmap_t *map, size_t offset, size_t length, enum osal_syncmode_bits mode_bits) {
if (!MDBX_MMAP_NEEDS_JOLT && mode_bits == MDBX_SYNC_NONE)
return MDBX_SUCCESS;
@@ -1677,7 +1726,7 @@ MDBX_INTERNAL int osal_msync(const osal_mmap_t *map, size_t offset, size_t lengt
return MDBX_SUCCESS;
}
MDBX_INTERNAL int osal_check_fs_rdonly(mdbx_filehandle_t handle, const pathchar_t *pathname, int err) {
int osal_check_fs_rdonly(mdbx_filehandle_t handle, const pathchar_t *pathname, int err) {
#if defined(_WIN32) || defined(_WIN64)
(void)pathname;
(void)err;
@@ -1704,7 +1753,7 @@ MDBX_INTERNAL int osal_check_fs_rdonly(mdbx_filehandle_t handle, const pathchar_
return MDBX_SUCCESS;
}
MDBX_INTERNAL int osal_check_fs_incore(mdbx_filehandle_t handle) {
int osal_check_fs_incore(mdbx_filehandle_t handle) {
#if defined(_WIN32) || defined(_WIN64)
(void)handle;
#else
@@ -1745,7 +1794,7 @@ MDBX_INTERNAL int osal_check_fs_incore(mdbx_filehandle_t handle) {
return MDBX_RESULT_FALSE;
}
MDBX_INTERNAL int osal_check_fs_local(mdbx_filehandle_t handle, int flags) {
int osal_check_fs_local(mdbx_filehandle_t handle, int flags) {
#if defined(_WIN32) || defined(_WIN64)
if (globals.running_under_Wine && !(flags & MDBX_EXCLUSIVE))
return ERROR_NOT_CAPABLE /* workaround for Wine */;
@@ -2023,8 +2072,8 @@ static int check_mmap_limit(const size_t limit) {
return MDBX_SUCCESS;
}
MDBX_INTERNAL int osal_mmap(const int flags, osal_mmap_t *map, size_t size, const size_t limit, const unsigned options,
const pathchar_t *pathname4logging) {
int osal_mmap(const int flags, osal_mmap_t *map, size_t size, const size_t limit, const unsigned options,
const pathchar_t *pathname4logging) {
assert(size <= limit);
map->limit = 0;
map->current = 0;
@@ -2061,9 +2110,9 @@ MDBX_INTERNAL int osal_mmap(const int flags, osal_mmap_t *map, size_t size, cons
if (unlikely(err != MDBX_SUCCESS))
return err;
if ((flags & MDBX_RDONLY) == 0 && (options & MMAP_OPTION_TRUNCATE) != 0) {
err = osal_ftruncate(map->fd, size);
VERBOSE("ftruncate %zu, err %d", size, err);
if ((flags & MDBX_RDONLY) == 0 && (options & MMAP_OPTION_SETLENGTH) != 0) {
err = osal_fsetsize(map->fd, size);
VERBOSE("osal_fsetsize %zu, err %d", size, err);
if (err != MDBX_SUCCESS)
return err;
map->filesize = size;
@@ -2176,7 +2225,7 @@ MDBX_INTERNAL int osal_mmap(const int flags, osal_mmap_t *map, size_t size, cons
return MDBX_SUCCESS;
}
MDBX_INTERNAL int osal_munmap(osal_mmap_t *map) {
int osal_munmap(osal_mmap_t *map) {
VALGRIND_MAKE_MEM_NOACCESS(map->base, map->current);
/* Unpoisoning is required for ASAN to avoid false-positive diagnostic
* when this memory will re-used by malloc or another mmapping.
@@ -2202,7 +2251,7 @@ MDBX_INTERNAL int osal_munmap(osal_mmap_t *map) {
return MDBX_SUCCESS;
}
MDBX_INTERNAL int osal_mresize(const int flags, osal_mmap_t *map, size_t size, size_t limit) {
int osal_mresize(const int flags, osal_mmap_t *map, size_t size, size_t limit) {
int rc = osal_filesize(map->fd, &map->filesize);
VERBOSE("flags 0x%x, size %zu, limit %zu, filesize %" PRIu64, flags, size, limit, map->filesize);
assert(size <= limit);
@@ -2308,7 +2357,7 @@ retry_file_and_section:
}
if ((flags & MDBX_RDONLY) == 0 && map->filesize != size) {
err = osal_ftruncate(map->fd, size);
err = osal_fsetsize(map->fd, size);
if (err == MDBX_SUCCESS)
map->filesize = size;
/* ignore error, because Windows unable shrink file
@@ -2386,10 +2435,15 @@ retry_mapview:;
rc = MDBX_EPERM;
map->current = (map->filesize > limit) ? limit : (size_t)map->filesize;
} else {
if (size > map->filesize || (size < map->filesize && (flags & txn_shrink_allowed))) {
rc = osal_ftruncate(map->fd, size);
VERBOSE("ftruncate %zu, err %d", size, rc);
if (rc != MDBX_SUCCESS)
if (map->filesize != size) {
if (size > map->filesize) {
rc = osal_fsetsize(map->fd, size);
VERBOSE("osal_fsetsize-%s %zu, err %d", "extend", size, rc);
} else if (flags & txn_shrink_allowed) {
rc = osal_fsetsize(map->fd, size);
VERBOSE("osal_fsetsize-%s %zu, err %d", "shrink", size, rc);
}
if (unlikely(rc != MDBX_SUCCESS))
return rc;
map->filesize = size;
}
@@ -2576,7 +2630,7 @@ retry_mapview:;
/*----------------------------------------------------------------------------*/
__cold MDBX_INTERNAL void osal_jitter(bool tiny) {
__cold void osal_jitter(bool tiny) {
for (;;) {
#if defined(_M_IX86) || defined(_M_X64) || defined(__i386__) || defined(__x86_64__)
unsigned salt = 5296013u * (unsigned)__rdtsc();
@@ -2594,7 +2648,7 @@ __cold MDBX_INTERNAL void osal_jitter(bool tiny) {
break;
#if defined(_WIN32) || defined(_WIN64)
if (coin < 43 * 2 / 3)
SwitchToThread();
osal_yield();
else {
static HANDLE timer;
if (!timer)
@@ -2609,7 +2663,7 @@ __cold MDBX_INTERNAL void osal_jitter(bool tiny) {
break;
}
#else
sched_yield();
osal_yield();
if (coin > 43 * 2 / 3)
usleep(coin);
#endif
@@ -2645,7 +2699,7 @@ __cold static clockid_t choice_monoclock(void) {
#define posix_clockid CLOCK_REALTIME
#endif
MDBX_INTERNAL uint64_t osal_16dot16_to_monotime(uint32_t seconds_16dot16) {
uint64_t osal_16dot16_to_monotime(uint32_t seconds_16dot16) {
#if defined(_WIN32) || defined(_WIN64)
const uint64_t ratio = performance_frequency.QuadPart;
#elif defined(__APPLE__) || defined(__MACH__)
@@ -2658,7 +2712,7 @@ MDBX_INTERNAL uint64_t osal_16dot16_to_monotime(uint32_t seconds_16dot16) {
}
static uint64_t monotime_limit;
MDBX_INTERNAL uint32_t osal_monotime_to_16dot16(uint64_t monotime) {
uint32_t osal_monotime_to_16dot16(uint64_t monotime) {
if (unlikely(monotime > monotime_limit))
return UINT32_MAX;
@@ -2673,7 +2727,7 @@ MDBX_INTERNAL uint32_t osal_monotime_to_16dot16(uint64_t monotime) {
return ret;
}
MDBX_INTERNAL uint64_t osal_monotime(void) {
uint64_t osal_monotime(void) {
#if defined(_WIN32) || defined(_WIN64)
LARGE_INTEGER counter;
if (QueryPerformanceCounter(&counter))
@@ -2688,7 +2742,7 @@ MDBX_INTERNAL uint64_t osal_monotime(void) {
return 0;
}
MDBX_INTERNAL uint64_t osal_cputime(size_t *optional_page_faults) {
uint64_t osal_cputime(size_t *optional_page_faults) {
#if defined(_WIN32) || defined(_WIN64)
if (optional_page_faults) {
PROCESS_MEMORY_COUNTERS pmc;
@@ -3353,7 +3407,7 @@ __cold int mdbx_get_sysraminfo(intptr_t *page_size, intptr_t *total_pages, intpt
#include <wincrypt.h>
#endif /* Windows */
MDBX_INTERNAL bin128_t osal_guid(const MDBX_env *env) {
bin128_t osal_guid(const MDBX_env *env) {
struct {
uint64_t begin, end, cputime;
uintptr_t thread, pid;

View File

@@ -171,6 +171,14 @@ typedef char pathchar_t;
#define MDBX_PRIsPATH "s"
#endif
MDBX_MAYBE_UNUSED static inline bool osal_yield(void) {
#if defined(_WIN32) || defined(_WIN64)
return SleepEx(0, true) == WAIT_IO_COMPLETION;
#else
return sched_yield() != 0;
#endif
}
typedef struct osal_mmap {
union {
void *base;
@@ -433,7 +441,7 @@ enum osal_syncmode_bits {
};
MDBX_INTERNAL int osal_fsync(mdbx_filehandle_t fd, const enum osal_syncmode_bits mode_bits);
MDBX_INTERNAL int osal_ftruncate(mdbx_filehandle_t fd, uint64_t length);
MDBX_INTERNAL int osal_fsetsize(mdbx_filehandle_t fd, const uint64_t length);
MDBX_INTERNAL int osal_fseek(mdbx_filehandle_t fd, uint64_t pos);
MDBX_INTERNAL int osal_filesize(mdbx_filehandle_t fd, uint64_t *length);
@@ -469,7 +477,7 @@ MDBX_INTERNAL int osal_removedirectory(const pathchar_t *pathname);
MDBX_INTERNAL int osal_is_pipe(mdbx_filehandle_t fd);
MDBX_INTERNAL int osal_lockfile(mdbx_filehandle_t fd, bool wait);
#define MMAP_OPTION_TRUNCATE 1
#define MMAP_OPTION_SETLENGTH 1
#define MMAP_OPTION_SEMAPHORE 2
MDBX_INTERNAL int osal_mmap(const int flags, osal_mmap_t *map, size_t size, const size_t limit, const unsigned options,
const pathchar_t *pathname4logging);

View File

@@ -513,7 +513,9 @@ __cold void rthc_dtor(const uint32_t current_pid) {
MDBX_env *const env = rthc_table[i].env;
if (env->pid != current_pid)
continue;
if (!(env->flags & ENV_TXKEY))
if (!env->lck_mmap.lck || env->lck_mmap.base == MAP_FAILED)
continue;
if (!(env->flags & ENV_TXKEY) || !env->lck_mmap.lck)
continue;
env->flags -= ENV_TXKEY;
reader_slot_t *const begin = &env->lck_mmap.lck->rdt[0];

View File

@@ -3,7 +3,7 @@
#include "internals.h"
MDBX_MAYBE_UNUSED MDBX_NOTHROW_CONST_FUNCTION MDBX_INTERNAL unsigned log2n_powerof2(size_t value_uintptr) {
MDBX_MAYBE_UNUSED MDBX_NOTHROW_CONST_FUNCTION unsigned log2n_powerof2(size_t value_uintptr) {
assert(value_uintptr > 0 && value_uintptr < INT32_MAX && is_powerof2(value_uintptr));
assert((value_uintptr & -(intptr_t)value_uintptr) == value_uintptr);
const uint32_t value_uint32 = (uint32_t)value_uintptr;
@@ -22,7 +22,7 @@ MDBX_MAYBE_UNUSED MDBX_NOTHROW_CONST_FUNCTION MDBX_INTERNAL unsigned log2n_power
#endif
}
MDBX_NOTHROW_CONST_FUNCTION MDBX_INTERNAL uint64_t rrxmrrxmsx_0(uint64_t v) {
MDBX_NOTHROW_CONST_FUNCTION uint64_t rrxmrrxmsx_0(uint64_t v) {
/* Pelle Evensen's mixer, https://bit.ly/2HOfynt */
v ^= (v << 39 | v >> 25) ^ (v << 14 | v >> 50);
v *= UINT64_C(0xA24BAED4963EE407);

View File

@@ -8,7 +8,7 @@ DIR="$(dirname ${BASH_SOURCE[0]})"
TEST="${DIR}/stochastic.sh --skip-make --db-upto-gb 32"
PREFIX="/dev/shm/mdbxtest-"
NUMACTL="$(which numactl 2>-)"
NUMACTL="$(which numactl 2>&-)"
NUMALIST=()
NUMAIDX=0
if [ -n "${NUMACTL}" -a $(${NUMACTL} --hardware | grep 'node [0-9]\+ cpus' | wc -l) -gt 1 ]; then

View File

@@ -22,9 +22,7 @@ void testcase_jitter::check_dbi_error(int expect, const char *stage) {
bool testcase_jitter::run() {
int err;
size_t upper_limit = config.params.size_upper;
if (upper_limit < 1)
upper_limit = config.params.size_now * 2;
size_t upper_limit = (config.params.size_upper < 1) ? config.params.size_now * 2 : config.params.size_upper;
tablename_buf buffer;
const char *const tablename = db_tablename(buffer);

View File

@@ -572,11 +572,6 @@ int osal_actor_poll(mdbx_pid_t &pid, unsigned timeout) {
return sigbreak ? EINTR : 0 /* timeout */;
}
void osal_yield(void) {
if (sched_yield())
failure_perror("sched_yield()", errno);
}
void osal_udelay(size_t us) {
chrono::time until, now = chrono::now_monotonic();
until.fixedpoint = now.fixedpoint + chrono::from_us(us).fixedpoint;

View File

@@ -401,8 +401,6 @@ int osal_actor_poll(mdbx_pid_t &pid, unsigned timeout) {
}
}
void osal_yield(void) { SwitchToThread(); }
void osal_udelay(size_t us) {
chrono::time until, now = chrono::now_monotonic();
until.fixedpoint = now.fixedpoint + chrono::from_us(us).fixedpoint;

View File

@@ -20,7 +20,6 @@ bool osal_multiactor_mode(void);
int osal_delay(unsigned seconds);
void osal_udelay(size_t us);
void osal_yield(void);
bool osal_istty(int fd);
std::string osal_tempdir(void);

View File

@@ -516,8 +516,8 @@ function probe {
exec {LFD}> >(logger)
fi
${NUMABIND} ${MONITOR} ./mdbx_test ${speculum} --random-writemap=no --ignore-dbfull --repeat=${REPEAT} --pathname=${TESTDB_DIR}/long.db --cleanup-after=no --geometry-jitter=${GEOMETRY_JITTER} "$@" $case >&${LFD} \
&& ${NUMABIND} ${MONITOR} ./mdbx_chk -q ${TESTDB_DIR}/long.db | tee ${TESTDB_DIR}/long-chk.log \
&& ([ ! -e ${TESTDB_DIR}/long.db-copy ] || ${NUMABIND} ${MONITOR} ./mdbx_chk -q ${TESTDB_DIR}/long.db-copy | tee ${TESTDB_DIR}/long-chk-copy.log) \
&& ${NUMABIND} ${MONITOR} ./mdbx_chk ${TESTDB_DIR}/long.db | tee ${TESTDB_DIR}/long-chk.log \
&& ([ ! -e ${TESTDB_DIR}/long.db-copy ] || ${NUMABIND} ${MONITOR} ./mdbx_chk ${TESTDB_DIR}/long.db-copy | tee ${TESTDB_DIR}/long-chk-copy.log) \
|| failed
if [ ${LFD} -ne 0 ]; then
echo "@@@ END-OF-LOG/ITERATION" >&${LFD}