mdbx: merge branch master into stable.

This commit is contained in:
Леонид Юрьев (Leonid Yuriev) 2023-06-18 17:18:16 +03:00
commit 3d187abc1b
19 changed files with 333 additions and 133 deletions

View File

@ -5,6 +5,107 @@ English version [by Google](https://gitflic-ru.translate.goog/project/erthink/li
and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/gitflic.ru/project/erthink/libmdbx/blob?file=ChangeLog.md). and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/gitflic.ru/project/erthink/libmdbx/blob?file=ChangeLog.md).
## v0.12.7 "Артек" от 2023-06-16
Стабилизирующий выпуск с исправлением обнаруженных ошибок и устранением
недочетов, в день основания международного детского центра [«Арте́к»](https://ru.wikipedia.org/wiki/Артек).
```
14 files changed, 222 insertions(+), 56 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
```
Исправления и доработки:
- Исправление опечатки в имени переменной внутри `mdbx_env_turn_for_recovery()`,
что приводило к неверному поведению в некоторых ситуациях.
С точки зрения пользователя, с учетом актуальных сценариев использования
утилиты `mdbx_chk`, был только один специфический/редкий сценарий
проявления ошибки/проблемы - когда выполнялась проверка и активация
слабой/weak мета-страницы с НЕ-последней транзакцией после системной
аварии машины, где БД использовалась в хрупком/небезопасном режиме.
В сценарии, при успешной проверке целевой страницы и её последующей
активации выводилось сообщение об ошибке, связанной со срабатыванием
механизма контроля не-когерентности кэша файловой системы и отображенных
в ОЗУ данных БД. При этом БД успешно восстанавливалось и не было
каких-либо негативных последствия, кроме самого сообщения об ошибке.
Технически же ошибка проявлялась при "переключении" на мета-страницу,
когда у хотя-бы одной из двух других мета-страниц номер транзакции был
больше:
* Если содержимое других мета-страниц было корректным, а номера
связанных транзакций были больше, то результирующий номер транзакции в
целевой/активируемой мета-страницы устанавливается без учета этих
мета-страниц и мог быть меньше-или-равным.
* В результате, если такие мета-страницы были в статусе слабых/weak, то
при закрытии БД после переключения могла срабатывать защита от
не-когерентности unified buffer/page cache, а в отладочных сборках могла
срабатывать assert-проверка.
* Если же такие мета-страницы были в статусе сильных/steady, то
переключение на новую мета-страницу могло не давать эффекта либо
приводить к появлению двух мета-страниц с одинаковым номером транзакции,
что является ошибочной ситуацией.
- Обходное решение проблем сборки посредством GCC с использование опций `-m32 -arch=i686 -Ofast`.
Проблема обусловлена ошибкой GCC, из-за которой конструкция `__attribute__((__target__("sse2")))`
не включает полноценное использование инструкций SSE и SSE2, если это не было сделано посредством
опций командной строки, но была использована опция `-Ofast`. В результате сборка заканчивалась
сообщением об ошибке:
`error: inlining failed in call to 'always_inline' '_mm_movemask_ps': target specific option mismatch`
- Доработка режима "восстановления" БД и переключения на заданную мета-страницу:
* Устранение обновления без необходимости мета-страницы с увеличением номера транзакции;
* Устранение вывода (логирования) бессмысленного/лишнего предупреждения о пропуске обновления геометрии БД;
* Более ожидаемое и безопасное поведение при проверке БД с указанием целевой мета-страницы в режиме чтения-записи.
Теперь при открытии БД посредством `mdbx_env_open_for_recovery()` не
выполняется неявное изменение/обновление БД, в том числе при закрытии
БД. Это позволяет обезопасить БД (снизить шанс её разрушения) если
пользователь при попытке восстановления, либо просто в качестве
эксперимента, задал утилите `mdbx_chk` неверную или опасную комбинацию
параметров. При этом обычная проверка, как и явное переключение
мета-страниц, работают по-прежнему.
Мелочи:
- Незначительное уточнение CMake-пробника для `std::filesystem`,
проверяющего необходимость линковки с дополнительными библиотеками C++.
- Устранение минорных предупреждений старых компиляторов в тестах.
- Устранение причины ложно-позитивного предупреждения новых версий GCC в C++ API.
- Исправление ссылки на репозиторий бенчмарка ioarena.
- Добавление перекрестных ссылок в doxygen-документацию по C++ API.
- Уточнение ограничений в разделе [Restrictions & Caveats](https://libmdbx.dqdkfa.ru/intro.html#restrictions).
- Исправление ссылок на описание `mdbx_canary_put()`.
--------------------------------------------------------------------------------
## v0.12.6 "ЦСКА" от 2023-04-29
Стабилизирующий выпуск с исправлением обнаруженных ошибок и устранением
недочетов, в день 100-летнего юбилея спортивного клуба [«ЦСКА»](https://ru.wikipedia.org/wiki/Центральный_спортивный_клуб_Армии).
```
14 files changed, 117 insertions(+), 83 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
```
Мелочи:
- Обновление патча для старых версий buildroot.
- Использование clang-format-16.
- Использование `enum`-типов вместо `int` для устранения предупреждений GCC 13,
что могло ломать сборку в Fedora 38.
--------------------------------------------------------------------------------
## v0.12.5 "Динамо" от 2023-04-18 ## v0.12.5 "Динамо" от 2023-04-18
Стабилизирующий выпуск с исправлением обнаруженных ошибок и устранением Стабилизирующий выпуск с исправлением обнаруженных ошибок и устранением
@ -45,7 +146,7 @@ Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
- Корректировка `osal_vasprintf()` для устранения предупреждений статических анализаторов. - Корректировка `osal_vasprintf()` для устранения предупреждений статических анализаторов.
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.12.4 "Арта-333" от 2023-03-03 ## v0.12.4 "Арта-333" от 2023-03-03
@ -124,7 +225,7 @@ Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
- Вывод всех счетчиков page-operations в `mdbx_stat`. - Вывод всех счетчиков page-operations в `mdbx_stat`.
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.12.3 "Акула" от 2023-01-07 ## v0.12.3 "Акула" от 2023-01-07
@ -267,7 +368,7 @@ Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
- Рефакторинг и микрооптимизация. - Рефакторинг и микрооптимизация.
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.12.2 "Иван Ярыгин" от 2022-11-11 ## v0.12.2 "Иван Ярыгин" от 2022-11-11
@ -333,7 +434,7 @@ Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
суммарный размер пары ключ-значение. суммарный размер пары ключ-значение.
- Реализовано использование асинхронной (overlapped) записи в Windows, - Реализовано использование асинхронной (overlapped) записи в Windows,
включая использования небуфферизированного ввода-вывода и `WriteGather()`. включая использования небуферизированного ввода-вывода и `WriteGather()`.
Это позволяет сократить накладные расходы и частично обойти проблемы Это позволяет сократить накладные расходы и частично обойти проблемы
Windows с низкой производительностью ввода-вывода, включая большие Windows с низкой производительностью ввода-вывода, включая большие
задержки `FlushFileBuffers()`. Новый код также обеспечивает консолидацию задержки `FlushFileBuffers()`. Новый код также обеспечивает консолидацию
@ -437,7 +538,7 @@ Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
- Fixed minor MingGW warning. - Fixed minor MingGW warning.
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.12.1 "Positive Proxima" at 2022-08-24 ## v0.12.1 "Positive Proxima" at 2022-08-24
@ -462,7 +563,7 @@ New:
- Added the `gcrtime_seconds16dot16` counter to the "Page Operation Statistics" that accumulates time spent for GC searching and reclaiming. - Added the `gcrtime_seconds16dot16` counter to the "Page Operation Statistics" that accumulates time spent for GC searching and reclaiming.
- Copy-with-compactification now clears/zeroes unused gaps inside database pages. - Copy-with-compactification now clears/zeroes unused gaps inside database pages.
- The `C` and `C++` APIs has been extended and/or refined to simplify using `wchar_t` pathnames. - The `C` and `C++` APIs has been extended and/or refined to simplify using `wchar_t` pathnames.
On Windows the `mdbx_env_openW()`, ``mdbx_env_get_pathW()`()`, `mdbx_env_copyW()`, `mdbx_env_open_for_recoveryW()` are available for now, On Windows the `mdbx_env_openW()`, `mdbx_env_get_pathW()`, `mdbx_env_copyW()`, `mdbx_env_open_for_recoveryW()` are available for now,
but the `mdbx_env_get_path()` has been replaced in favor of `mdbx_env_get_pathW()`. but the `mdbx_env_get_path()` has been replaced in favor of `mdbx_env_get_pathW()`.
- Added explicit error message for Buildroot's Microblaze toolchain maintainers. - Added explicit error message for Buildroot's Microblaze toolchain maintainers.
- Added `MDBX_MANAGE_BUILD_FLAGS` build options for CMake. - Added `MDBX_MANAGE_BUILD_FLAGS` build options for CMake.
@ -482,7 +583,7 @@ Fixes:
Not a release but preparation for changing feature set and API. Not a release but preparation for changing feature set and API.
=============================================================================== ********************************************************************************
## v0.11.14 "Sergey Kapitsa" at 2023-02-14 ## v0.11.14 "Sergey Kapitsa" at 2023-02-14
@ -526,7 +627,7 @@ Minors:
while build for SH4 arch. while build for SH4 arch.
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.11.13 at "Swashplate" 2022-11-10 ## v0.11.13 at "Swashplate" 2022-11-10
@ -557,7 +658,7 @@ Minors:
- Use `--dont-check-ram-size` for small-tests make-targets (backport). - Use `--dont-check-ram-size` for small-tests make-targets (backport).
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.11.12 "Эребуни" at 2022-10-12 ## v0.11.12 "Эребуни" at 2022-10-12
@ -582,7 +683,7 @@ Minors:
- Removed needless `LockFileEx()` inside `mdbx_env_copy()` (backport). - Removed needless `LockFileEx()` inside `mdbx_env_copy()` (backport).
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.11.11 "Тендра-1790" at 2022-09-11 ## v0.11.11 "Тендра-1790" at 2022-09-11
@ -601,7 +702,7 @@ Fixes:
- Fixed derived C++ builds by removing `MDBX_INTERNAL_FUNC` for `mdbx_w2mb()` and `mdbx_mb2w()`. - Fixed derived C++ builds by removing `MDBX_INTERNAL_FUNC` for `mdbx_w2mb()` and `mdbx_mb2w()`.
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.11.10 "the TriColor" at 2022-08-22 ## v0.11.10 "the TriColor" at 2022-08-22
@ -634,7 +735,7 @@ Minors:
- Minor clarified `iov_page()` failure case. - Minor clarified `iov_page()` failure case.
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.11.9 "Чирчик-1992" at 2022-08-02 ## v0.11.9 "Чирчик-1992" at 2022-08-02
@ -675,7 +776,7 @@ Minors:
- Fixed `has no symbols` warning from Apple's ranlib. - Fixed `has no symbols` warning from Apple's ranlib.
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.11.8 "Baked Apple" at 2022-06-12 ## v0.11.8 "Baked Apple" at 2022-06-12
@ -728,7 +829,7 @@ Minors:
- Don't provide nor report package information if used as a CMake subproject. - Don't provide nor report package information if used as a CMake subproject.
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.11.7 "Resurrected Sarmat" at 2022-04-22 ## v0.11.7 "Resurrected Sarmat" at 2022-04-22
@ -785,7 +886,7 @@ Minors:
- Added `CMAKE_HOST_ARCH` and `CMAKE_HOST_CAN_RUN_EXECUTABLES_BUILT_FOR_TARGET`. - Added `CMAKE_HOST_ARCH` and `CMAKE_HOST_CAN_RUN_EXECUTABLES_BUILT_FOR_TARGET`.
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.11.6 at 2022-03-24 ## v0.11.6 at 2022-03-24
@ -814,7 +915,7 @@ Minors:
- Clarified error messages of a signature/version mismatch. - Clarified error messages of a signature/version mismatch.
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.11.5 at 2022-02-23 ## v0.11.5 at 2022-02-23
@ -846,7 +947,7 @@ Minors:
- Minor fixes Doxygen references, comments, descriptions, etc. - Minor fixes Doxygen references, comments, descriptions, etc.
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.11.4 at 2022-02-02 ## v0.11.4 at 2022-02-02
@ -894,7 +995,7 @@ Minors:
- Using the `-fno-semantic interposition` option to reduce the overhead to calling self own public functions. - Using the `-fno-semantic interposition` option to reduce the overhead to calling self own public functions.
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.11.3 at 2021-12-31 ## v0.11.3 at 2021-12-31
@ -930,7 +1031,7 @@ Minors:
- For compatibility reverted returning `MDBX_ENODATA`for some cases. - For compatibility reverted returning `MDBX_ENODATA`for some cases.
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.11.2 at 2021-12-02 ## v0.11.2 at 2021-12-02
@ -959,7 +1060,7 @@ Minors:
- Remove unneeded `#undef P_DIRTY`. - Remove unneeded `#undef P_DIRTY`.
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.11.1 at 2021-10-23 ## v0.11.1 at 2021-10-23
@ -981,7 +1082,7 @@ Acknowledgments:
- [Alex Sharov](https://github.com/AskAlexSharov) for reporting and testing. - [Alex Sharov](https://github.com/AskAlexSharov) for reporting and testing.
=============================================================================== ********************************************************************************
## v0.10.5 at 2021-10-13 (obsolete, please use v0.11.1) ## v0.10.5 at 2021-10-13 (obsolete, please use v0.11.1)
@ -1010,7 +1111,7 @@ Minors:
- Refined providing information for the `@MAIN` and `@GC` sub-databases of a last committed modification transaction's ID. - Refined providing information for the `@MAIN` and `@GC` sub-databases of a last committed modification transaction's ID.
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.10.4 at 2021-10-10 ## v0.10.4 at 2021-10-10
@ -1033,7 +1134,7 @@ Minors:
- In debugging builds fixed a too small (single page) by default DB shrink threshold. - In debugging builds fixed a too small (single page) by default DB shrink threshold.
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.10.3 at 2021-08-27 ## v0.10.3 at 2021-08-27
@ -1064,7 +1165,7 @@ Minors:
- Fixed CMake warning about compatibility with 3.8.2 - Fixed CMake warning about compatibility with 3.8.2
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.10.2 at 2021-07-26 ## v0.10.2 at 2021-07-26
@ -1114,7 +1215,7 @@ Fixes:
- Fixed [test framework keygen-related issue](https://libmdbx.dqdkfa.ru/dead-github/issues/127). - Fixed [test framework keygen-related issue](https://libmdbx.dqdkfa.ru/dead-github/issues/127).
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.10.1 at 2021-06-01 ## v0.10.1 at 2021-06-01
@ -1141,7 +1242,7 @@ Fixes:
- Re-Fixed WSL1/WSL2 detection with distinguishing (https://libmdbx.dqdkfa.ru/dead-github/issues/97). - Re-Fixed WSL1/WSL2 detection with distinguishing (https://libmdbx.dqdkfa.ru/dead-github/issues/97).
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.10.0 at 2021-05-09 ## v0.10.0 at 2021-05-09
@ -1230,7 +1331,7 @@ Fixes:
- Fixed building by MinGW for Windows (https://libmdbx.dqdkfa.ru/dead-github/issues/155). - Fixed building by MinGW for Windows (https://libmdbx.dqdkfa.ru/dead-github/issues/155).
=============================================================================== ********************************************************************************
## v0.9.3 at 2021-02-02 ## v0.9.3 at 2021-02-02
@ -1291,7 +1392,7 @@ Fixes:
- Fixed operation on systems with unusual small/large page size, including PowerPC (https://libmdbx.dqdkfa.ru/dead-github/issues/157). - Fixed operation on systems with unusual small/large page size, including PowerPC (https://libmdbx.dqdkfa.ru/dead-github/issues/157).
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.9.2 at 2020-11-27 ## v0.9.2 at 2020-11-27
@ -1344,7 +1445,7 @@ Fixes:
- Added handling `EXCEPTION_POSSIBLE_DEADLOCK` condition for Windows. - Added handling `EXCEPTION_POSSIBLE_DEADLOCK` condition for Windows.
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.9.1 2020-09-30 ## v0.9.1 2020-09-30
@ -1393,7 +1494,7 @@ Fixes:
- Now C++ compiler optional for building by CMake. - Now C++ compiler optional for building by CMake.
------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## v0.9.0 2020-07-31 (not a release, but API changes) ## v0.9.0 2020-07-31 (not a release, but API changes)
@ -1409,7 +1510,7 @@ Deprecated functions and flags:
Please use the value-to-key functions to provide keys that are compatible with the built-in libmdbx comparators. Please use the value-to-key functions to provide keys that are compatible with the built-in libmdbx comparators.
=============================================================================== ********************************************************************************
## 2020-07-06 ## 2020-07-06
@ -1550,7 +1651,7 @@ Deprecated functions and flags:
- Checking for non-local filesystems to avoid DB corruption. - Checking for non-local filesystems to avoid DB corruption.
=============================================================================== ********************************************************************************
For early changes see the git commit history. For early changes see the git commit history.

View File

@ -277,7 +277,7 @@ the user's point of view.
> and up to 30% faster when _libmdbx_ compiled with specific build options > and up to 30% faster when _libmdbx_ compiled with specific build options
> which downgrades several runtime checks to be match with LMDB behaviour. > which downgrades several runtime checks to be match with LMDB behaviour.
> >
> These and other results could be easily reproduced with [ioArena](https://abf.io/erthink/ioarena.git) just by `make bench-quartet` command, > These and other results could be easily reproduced with [ioArena](https://abf.io/erthink/ioarena) just by `make bench-quartet` command,
> including comparisons with [RockDB](https://en.wikipedia.org/wiki/RocksDB) > including comparisons with [RockDB](https://en.wikipedia.org/wiki/RocksDB)
> and [WiredTiger](https://en.wikipedia.org/wiki/WiredTiger). > and [WiredTiger](https://en.wikipedia.org/wiki/WiredTiger).
@ -659,7 +659,7 @@ Bindings
Performance comparison Performance comparison
====================== ======================
All benchmarks were done in 2015 by [IOArena](https://abf.io/erthink/ioarena.git) All benchmarks were done in 2015 by [IOArena](https://abf.io/erthink/ioarena)
and multiple [scripts](https://github.com/pmwkaa/ioarena/tree/HL%2B%2B2015) and multiple [scripts](https://github.com/pmwkaa/ioarena/tree/HL%2B%2B2015)
runs on Lenovo Carbon-2 laptop, i7-4600U 2.1 GHz (2 physical cores, 4 HyperThreading cores), 8 Gb RAM, runs on Lenovo Carbon-2 laptop, i7-4600U 2.1 GHz (2 physical cores, 4 HyperThreading cores), 8 Gb RAM,
SSD SAMSUNG MZNTD512HAGL-000L1 (DXT23L0Q) 512 Gb. SSD SAMSUNG MZNTD512HAGL-000L1 (DXT23L0Q) 512 Gb.

View File

@ -1039,9 +1039,15 @@ macro(probe_libcxx_filesystem)
#endif #endif
int main(int argc, const char*argv[]) { int main(int argc, const char*argv[]) {
fs::path probe(argv[0]); std::string str(argv[0]);
fs::path probe(str);
if (argc != 1) throw fs::filesystem_error(std::string("fake"), std::error_code()); if (argc != 1) throw fs::filesystem_error(std::string("fake"), std::error_code());
return fs::is_directory(probe.relative_path()); int r = fs::is_directory(probe.relative_path());
for (auto const& i : fs::directory_iterator(probe)) {
++r;
(void)i;
}
return r;
} }
]]) ]])
set(LIBCXX_FILESYSTEM "") set(LIBCXX_FILESYSTEM "")

View File

@ -35,10 +35,11 @@ or debugging of a client application while retaining an active read
transaction. LMDB this results in `MDB_MAP_FULL` error and subsequent write transaction. LMDB this results in `MDB_MAP_FULL` error and subsequent write
performance degradation. performance degradation.
MDBX mostly solve "long-lived" readers issue by using the Handle-Slow-Readers MDBX mostly solve "long-lived" readers issue by using the
\ref MDBX_hsr_func callback which allows to abort long-lived read transactions, Handle-Slow-Readers \ref MDBX_hsr_func callback which allows to abort
and using the \ref MDBX_LIFORECLAIM mode which addresses subsequent performance degradation. long-lived read transactions, and using the \ref MDBX_LIFORECLAIM mode
The "next" version of libmdbx (\ref MithrilDB) will completely solve this. which addresses subsequent performance degradation. The "next" version
of libmdbx (\ref MithrilDB) will completely solve this.
- Avoid suspending a process with active transactions. These would then be - Avoid suspending a process with active transactions. These would then be
"long-lived" as above. "long-lived" as above.
@ -80,6 +81,19 @@ list of pages to be retired.
Both of these issues will be addressed in MithrilDB. Both of these issues will be addressed in MithrilDB.
#### Since v0.12.1 and later
Some aspects related to GC have been refined and improved, in particular:
- The search for free consecutive/adjacent pages through GC has been
significantly speeded, including acceleration using
NOEN/SSE2/AVX2/AVX512 instructions.
- The `Big Foot` feature which significantly reduces GC overhead for
processing large lists of retired pages from huge transactions. Now
libmdbx avoid creating large chunks of PNLs (page number lists) which
required a long sequences of free pages, aka large/overflow pages. Thus
avoiding searching, allocating and storing such sequences inside GC.
## Space reservation ## Space reservation
An MDBX database configuration will often reserve considerable unused An MDBX database configuration will often reserve considerable unused
@ -87,6 +101,10 @@ memory address space and maybe file size for future growth. This does
not use actual memory or disk space, but users may need to understand not use actual memory or disk space, but users may need to understand
the difference so they won't be scared off. the difference so they won't be scared off.
However, on 64-bit systems with a relative small amount of RAM, such
reservation can deplete system resources (trigger ENOMEM error, etc)
when setting an inadequately large upper DB size using \ref
mdbx_env_set_geometry() or \ref mdbx::env::geometry. So just avoid this.
## Remote filesystems ## Remote filesystems
Do not use MDBX databases on remote filesystems, even between processes Do not use MDBX databases on remote filesystems, even between processes

6
mdbx.h
View File

@ -4031,7 +4031,7 @@ LIBMDBX_API int mdbx_txn_renew(MDBX_txn *txn);
/** \brief The fours integers markers (aka "canary") associated with the /** \brief The fours integers markers (aka "canary") associated with the
* environment. * environment.
* \ingroup c_crud * \ingroup c_crud
* \see mdbx_canary_set() * \see mdbx_canary_put()
* \see mdbx_canary_get() * \see mdbx_canary_get()
* *
* The `x`, `y` and `z` values could be set by \ref mdbx_canary_put(), while the * The `x`, `y` and `z` values could be set by \ref mdbx_canary_put(), while the
@ -4069,10 +4069,10 @@ LIBMDBX_API int mdbx_canary_put(MDBX_txn *txn, const MDBX_canary *canary);
/** \brief Returns fours integers markers (aka "canary") associated with the /** \brief Returns fours integers markers (aka "canary") associated with the
* environment. * environment.
* \ingroup c_crud * \ingroup c_crud
* \see mdbx_canary_set() * \see mdbx_canary_put()
* *
* \param [in] txn A transaction handle returned by \ref mdbx_txn_begin(). * \param [in] txn A transaction handle returned by \ref mdbx_txn_begin().
* \param [in] canary The address of an MDBX_canary structure where the * \param [in] canary The address of an \ref MDBX_canary structure where the
* information will be copied. * information will be copied.
* *
* \returns A non-zero error value on failure and 0 on success. */ * \returns A non-zero error value on failure and 0 on success. */

View File

@ -591,11 +591,11 @@ concept ImmutableByteProducer = requires(const T &a, char array[42]) {
* \interface SliceTranscoder * \interface SliceTranscoder
* \brief SliceTranscoder C++20 concept */ * \brief SliceTranscoder C++20 concept */
template <typename T> template <typename T>
concept SliceTranscoder = ImmutableByteProducer<T> && concept SliceTranscoder =
requires(const slice &source, const T &a) { ImmutableByteProducer<T> && requires(const slice &source, const T &a) {
T(source); T(source);
{ a.is_erroneous() } -> std::same_as<bool>; { a.is_erroneous() } -> std::same_as<bool>;
}; };
#endif /* MDBX_HAVE_CXX20_CONCEPTS */ #endif /* MDBX_HAVE_CXX20_CONCEPTS */
@ -3000,7 +3000,11 @@ public:
//---------------------------------------------------------------------------- //----------------------------------------------------------------------------
/// Database geometry for size management. /// \brief Database geometry for size management.
/// \see env_managed::create_parameters
/// \see env_managed::env_managed(const ::std::string &pathname, const
/// create_parameters &, const operate_parameters &, bool accede)
struct LIBMDBX_API_TYPE geometry { struct LIBMDBX_API_TYPE geometry {
enum : int64_t { enum : int64_t {
default_value = -1, ///< Means "keep current or use default" default_value = -1, ///< Means "keep current or use default"
@ -3659,6 +3663,8 @@ public:
bool accede = true); bool accede = true);
/// \brief Additional parameters for creating a new database. /// \brief Additional parameters for creating a new database.
/// \see env_managed(const ::std::string &pathname, const create_parameters &,
/// const operate_parameters &, bool accede)
struct create_parameters { struct create_parameters {
env::geometry geometry; env::geometry geometry;
mdbx_mode_t file_mode_bits{0640}; mdbx_mode_t file_mode_bits{0640};
@ -5134,7 +5140,7 @@ inline filehandle env::get_filehandle() const {
} }
inline MDBX_env_flags_t env::get_flags() const { inline MDBX_env_flags_t env::get_flags() const {
unsigned bits; unsigned bits = 0;
error::success_or_throw(::mdbx_env_get_flags(handle_, &bits)); error::success_or_throw(::mdbx_env_get_flags(handle_, &bits));
return MDBX_env_flags_t(bits); return MDBX_env_flags_t(bits);
} }

View File

@ -1,26 +1,33 @@
From dd398c9a92b87f7c65798545d776735d27f2a4f9 Mon Sep 17 00:00:00 2001 From 790cbdc02c2597650964a564e05fbb5af503adc9 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?=
=?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= <leo@yuriev.ru> =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= <leo@yuriev.ru>
Date: Sat, 25 Jun 2022 16:22:22 +0300 Date: Wed, 19 Apr 2023 13:34:37 +0300
Subject: [PATCH] package/libmdbx: new package (library/database). Subject: [PATCH] package/libmdbx: new package (library/database).
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This patch adds libmdbx v0.11.8: This patch adds libmdbx:
- libmdbx is one of the fastest compact embeddable key-value ACID database. - libmdbx is one of the fastest compact embeddable key-value ACID database.
- libmdbx has a specific set of properties and capabilities, - libmdbx has a specific set of properties and capabilities,
focused on creating unique lightweight solutions. focused on creating unique lightweight solutions.
- libmdbx surpasses the legendary LMDB (Lightning Memory-Mapped Database) - libmdbx surpasses the legendary LMDB (Lightning Memory-Mapped Database)
in terms of reliability, features and performance. in terms of reliability, features and performance.
- https://github.com/erthink/libmdbx - https://gitflic.ru/project/erthink/libmdbx
Signed-off-by: Leonid Yuriev <leo@yuriev.ru> The v0.12.5 "Dynamo" is stable release of frontward _libmdbx_ branch with new superior features
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr> on the day of 100 anniversary of USSR' «Dynamo» sports and fitness society.
The complete ChangeLog: https://gitflic.ru/project/erthink/libmdbx/blob?file=ChangeLog.md
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
--- ---
DEVELOPERS | 3 +++ DEVELOPERS | 3 +++
package/Config.in | 1 + package/Config.in | 1 +
package/libmdbx/Config.in | 45 ++++++++++++++++++++++++++++++++++++ package/libmdbx/Config.in | 45 ++++++++++++++++++++++++++++++++++++
package/libmdbx/libmdbx.hash | 6 +++++ package/libmdbx/libmdbx.hash | 5 ++++
package/libmdbx/libmdbx.mk | 42 +++++++++++++++++++++++++++++++++ package/libmdbx/libmdbx.mk | 42 +++++++++++++++++++++++++++++++++
5 files changed, 97 insertions(+) 5 files changed, 96 insertions(+)
create mode 100644 package/libmdbx/Config.in create mode 100644 package/libmdbx/Config.in
create mode 100644 package/libmdbx/libmdbx.hash create mode 100644 package/libmdbx/libmdbx.hash
create mode 100644 package/libmdbx/libmdbx.mk create mode 100644 package/libmdbx/libmdbx.mk
@ -104,19 +111,18 @@ index 0000000000..a9a4ac45c5
+ !BR2_TOOLCHAIN_GCC_AT_LEAST_4_4 + !BR2_TOOLCHAIN_GCC_AT_LEAST_4_4
diff --git a/package/libmdbx/libmdbx.hash b/package/libmdbx/libmdbx.hash diff --git a/package/libmdbx/libmdbx.hash b/package/libmdbx/libmdbx.hash
new file mode 100644 new file mode 100644
index 0000000000..3f2be134c3 index 0000000000..7a5b19952e
--- /dev/null --- /dev/null
+++ b/package/libmdbx/libmdbx.hash +++ b/package/libmdbx/libmdbx.hash
@@ -0,0 +1,6 @@ @@ -0,0 +1,5 @@
+# Hashes from: https://libmdbx.dqdkfa.ru/release/SHA256SUMS +# Hashes from: https://libmdbx.dqdkfa.ru/release/SHA256SUMS
+sha256 06011f361481ee7adc2111e48b7b121384294e0b6b8f10c75e7886629297b279 libmdbx-amalgamated-0.11.8.tar.xz +sha256 9c3abaaf9079a9518bb7155734817a2e286fffea46f7cc0825dfbd1cf9174075 libmdbx-amalgamated-0.12.5.tar.xz
+sha256 3a9fb6a4cd941e646597235518714373fda1ca6d4c5e23669afe70ea87c20940 libmdbx-amalgamated-0.11.7.tar.xz
+ +
+# Locally calculated +# Locally calculated
+sha256 310fe25c858a9515fc8c8d7d1f24a67c9496f84a91e0a0e41ea9975b1371e569 LICENSE +sha256 310fe25c858a9515fc8c8d7d1f24a67c9496f84a91e0a0e41ea9975b1371e569 LICENSE
diff --git a/package/libmdbx/libmdbx.mk b/package/libmdbx/libmdbx.mk diff --git a/package/libmdbx/libmdbx.mk b/package/libmdbx/libmdbx.mk
new file mode 100644 new file mode 100644
index 0000000000..130fe96793 index 0000000000..9a196eda60
--- /dev/null --- /dev/null
+++ b/package/libmdbx/libmdbx.mk +++ b/package/libmdbx/libmdbx.mk
@@ -0,0 +1,42 @@ @@ -0,0 +1,42 @@
@ -126,7 +132,7 @@ index 0000000000..130fe96793
+# +#
+################################################################################ +################################################################################
+ +
+LIBMDBX_VERSION = 0.11.8 +LIBMDBX_VERSION = 0.12.5
+LIBMDBX_SOURCE = libmdbx-amalgamated-$(LIBMDBX_VERSION).tar.xz +LIBMDBX_SOURCE = libmdbx-amalgamated-$(LIBMDBX_VERSION).tar.xz
+LIBMDBX_SITE = https://libmdbx.dqdkfa.ru/release +LIBMDBX_SITE = https://libmdbx.dqdkfa.ru/release
+LIBMDBX_SUPPORTS_IN_SOURCE_BUILD = NO +LIBMDBX_SUPPORTS_IN_SOURCE_BUILD = NO
@ -163,5 +169,5 @@ index 0000000000..130fe96793
+ +
+$(eval $(cmake-package)) +$(eval $(cmake-package))
-- --
2.36.1 2.34.1

View File

@ -835,6 +835,9 @@ atomic_store64(MDBX_atomic_uint64_t *p, const uint64_t value,
enum MDBX_memory_order order) { enum MDBX_memory_order order) {
STATIC_ASSERT(sizeof(MDBX_atomic_uint64_t) == 8); STATIC_ASSERT(sizeof(MDBX_atomic_uint64_t) == 8);
#if MDBX_64BIT_ATOMIC #if MDBX_64BIT_ATOMIC
#if __GNUC_PREREQ(11, 0)
STATIC_ASSERT(__alignof__(MDBX_atomic_uint64_t) >= sizeof(uint64_t));
#endif /* GNU C >= 11 */
#ifdef MDBX_HAVE_C11ATOMICS #ifdef MDBX_HAVE_C11ATOMICS
assert(atomic_is_lock_free(MDBX_c11a_rw(uint64_t, p))); assert(atomic_is_lock_free(MDBX_c11a_rw(uint64_t, p)));
atomic_store_explicit(MDBX_c11a_rw(uint64_t, p), value, mo_c11_store(order)); atomic_store_explicit(MDBX_c11a_rw(uint64_t, p), value, mo_c11_store(order));
@ -3400,8 +3403,8 @@ static int __must_check_result setup_dbx(MDBX_dbx *const dbx,
const MDBX_db *const db, const MDBX_db *const db,
const unsigned pagesize); const unsigned pagesize);
static __inline MDBX_cmp_func *get_default_keycmp(unsigned flags); static __inline MDBX_cmp_func *get_default_keycmp(MDBX_db_flags_t flags);
static __inline MDBX_cmp_func *get_default_datacmp(unsigned flags); static __inline MDBX_cmp_func *get_default_datacmp(MDBX_db_flags_t flags);
__cold const char *mdbx_liberr2str(int errnum) { __cold const char *mdbx_liberr2str(int errnum) {
/* Table of descriptions for MDBX errors */ /* Table of descriptions for MDBX errors */
@ -6469,27 +6472,47 @@ MDBX_MAYBE_UNUSED static __always_inline size_t __builtin_clzl(size_t value) {
#define MDBX_ATTRIBUTE_TARGET(target) __attribute__((__target__(target))) #define MDBX_ATTRIBUTE_TARGET(target) __attribute__((__target__(target)))
#endif /* MDBX_ATTRIBUTE_TARGET */ #endif /* MDBX_ATTRIBUTE_TARGET */
#if defined(__SSE2__) #ifndef MDBX_GCC_FASTMATH_i686_SIMD_WORKAROUND
/* Workaround for GCC's bug with `-m32 -march=i686 -Ofast`
* gcc/i686-buildroot-linux-gnu/12.2.0/include/xmmintrin.h:814:1:
* error: inlining failed in call to 'always_inline' '_mm_movemask_ps':
* target specific option mismatch */
#if !defined(__FAST_MATH__) || !__FAST_MATH__ || !defined(__GNUC__) || \
defined(__e2k__) || defined(__clang__) || defined(__amd64__) || \
defined(__SSE2__)
#define MDBX_GCC_FASTMATH_i686_SIMD_WORKAROUND 0
#else
#define MDBX_GCC_FASTMATH_i686_SIMD_WORKAROUND 1
#endif
#endif /* MDBX_GCC_FASTMATH_i686_SIMD_WORKAROUND */
#if defined(__SSE2__) && defined(__SSE__)
#define MDBX_ATTRIBUTE_TARGET_SSE2 /* nope */ #define MDBX_ATTRIBUTE_TARGET_SSE2 /* nope */
#elif (defined(_M_IX86_FP) && _M_IX86_FP >= 2) || defined(__amd64__) #elif (defined(_M_IX86_FP) && _M_IX86_FP >= 2) || defined(__amd64__)
#define __SSE2__ #define __SSE2__
#define MDBX_ATTRIBUTE_TARGET_SSE2 /* nope */ #define MDBX_ATTRIBUTE_TARGET_SSE2 /* nope */
#elif defined(MDBX_ATTRIBUTE_TARGET) && defined(__ia32__) #elif defined(MDBX_ATTRIBUTE_TARGET) && defined(__ia32__) && \
#define MDBX_ATTRIBUTE_TARGET_SSE2 MDBX_ATTRIBUTE_TARGET("sse2") !MDBX_GCC_FASTMATH_i686_SIMD_WORKAROUND
#define MDBX_ATTRIBUTE_TARGET_SSE2 MDBX_ATTRIBUTE_TARGET("sse,sse2")
#endif /* __SSE2__ */ #endif /* __SSE2__ */
#if defined(__AVX2__) #if defined(__AVX2__)
#define MDBX_ATTRIBUTE_TARGET_AVX2 /* nope */ #define MDBX_ATTRIBUTE_TARGET_AVX2 /* nope */
#elif defined(MDBX_ATTRIBUTE_TARGET) && defined(__ia32__) #elif defined(MDBX_ATTRIBUTE_TARGET) && defined(__ia32__) && \
#define MDBX_ATTRIBUTE_TARGET_AVX2 MDBX_ATTRIBUTE_TARGET("avx2") !MDBX_GCC_FASTMATH_i686_SIMD_WORKAROUND
#define MDBX_ATTRIBUTE_TARGET_AVX2 MDBX_ATTRIBUTE_TARGET("sse,sse2,avx,avx2")
#endif /* __AVX2__ */ #endif /* __AVX2__ */
#if defined(MDBX_ATTRIBUTE_TARGET_AVX2)
#if defined(__AVX512BW__) #if defined(__AVX512BW__)
#define MDBX_ATTRIBUTE_TARGET_AVX512BW /* nope */ #define MDBX_ATTRIBUTE_TARGET_AVX512BW /* nope */
#elif defined(MDBX_ATTRIBUTE_TARGET) && defined(__ia32__) && \ #elif defined(MDBX_ATTRIBUTE_TARGET) && defined(__ia32__) && \
!MDBX_GCC_FASTMATH_i686_SIMD_WORKAROUND && \
(__GNUC_PREREQ(6, 0) || __CLANG_PREREQ(5, 0)) (__GNUC_PREREQ(6, 0) || __CLANG_PREREQ(5, 0))
#define MDBX_ATTRIBUTE_TARGET_AVX512BW MDBX_ATTRIBUTE_TARGET("avx512bw") #define MDBX_ATTRIBUTE_TARGET_AVX512BW \
MDBX_ATTRIBUTE_TARGET("sse,sse2,avx,avx2,avx512bw")
#endif /* __AVX512BW__ */ #endif /* __AVX512BW__ */
#endif /* MDBX_ATTRIBUTE_TARGET_AVX2 for MDBX_ATTRIBUTE_TARGET_AVX512BW */
#ifdef MDBX_ATTRIBUTE_TARGET_SSE2 #ifdef MDBX_ATTRIBUTE_TARGET_SSE2
MDBX_ATTRIBUTE_TARGET_SSE2 static __always_inline unsigned MDBX_ATTRIBUTE_TARGET_SSE2 static __always_inline unsigned
@ -6563,6 +6586,15 @@ diffcmp2mask_avx2(const pgno_t *const ptr, const ptrdiff_t offset,
return _mm256_movemask_ps(*(const __m256 *)&cmp); return _mm256_movemask_ps(*(const __m256 *)&cmp);
} }
MDBX_ATTRIBUTE_TARGET_AVX2 static __always_inline unsigned
diffcmp2mask_sse2avx(const pgno_t *const ptr, const ptrdiff_t offset,
const __m128i pattern) {
const __m128i f = _mm_loadu_si128((const __m128i *)ptr);
const __m128i l = _mm_loadu_si128((const __m128i *)(ptr + offset));
const __m128i cmp = _mm_cmpeq_epi32(_mm_sub_epi32(f, l), pattern);
return _mm_movemask_ps(*(const __m128 *)&cmp);
}
MDBX_MAYBE_UNUSED __hot MDBX_ATTRIBUTE_TARGET_AVX2 static pgno_t * MDBX_MAYBE_UNUSED __hot MDBX_ATTRIBUTE_TARGET_AVX2 static pgno_t *
scan4seq_avx2(pgno_t *range, const size_t len, const size_t seq) { scan4seq_avx2(pgno_t *range, const size_t len, const size_t seq) {
assert(seq > 0 && len > seq); assert(seq > 0 && len > seq);
@ -6608,7 +6640,7 @@ scan4seq_avx2(pgno_t *range, const size_t len, const size_t seq) {
} }
#endif /* __SANITIZE_ADDRESS__ */ #endif /* __SANITIZE_ADDRESS__ */
if (range - 3 > detent) { if (range - 3 > detent) {
mask = diffcmp2mask_sse2(range - 3, offset, *(const __m128i *)&pattern); mask = diffcmp2mask_sse2avx(range - 3, offset, *(const __m128i *)&pattern);
if (mask) if (mask)
return range + 28 - __builtin_clz(mask); return range + 28 - __builtin_clz(mask);
range -= 4; range -= 4;
@ -6682,7 +6714,7 @@ scan4seq_avx512bw(pgno_t *range, const size_t len, const size_t seq) {
range -= 8; range -= 8;
} }
if (range - 3 > detent) { if (range - 3 > detent) {
mask = diffcmp2mask_sse2(range - 3, offset, *(const __m128i *)&pattern); mask = diffcmp2mask_sse2avx(range - 3, offset, *(const __m128i *)&pattern);
if (mask) if (mask)
return range + 28 - __builtin_clz(mask); return range + 28 - __builtin_clz(mask);
range -= 4; range -= 4;
@ -6799,7 +6831,7 @@ static pgno_t *(*scan4seq_impl)(pgno_t *range, const size_t len,
static pgno_t *scan4seq_resolver(pgno_t *range, const size_t len, static pgno_t *scan4seq_resolver(pgno_t *range, const size_t len,
const size_t seq) { const size_t seq) {
pgno_t *(*choice)(pgno_t * range, const size_t len, const size_t seq) = pgno_t *(*choice)(pgno_t *range, const size_t len, const size_t seq) =
nullptr; nullptr;
#if __has_builtin(__builtin_cpu_init) || defined(__BUILTIN_CPU_INIT__) || \ #if __has_builtin(__builtin_cpu_init) || defined(__BUILTIN_CPU_INIT__) || \
__GNUC_PREREQ(4, 8) __GNUC_PREREQ(4, 8)
@ -8086,13 +8118,9 @@ retry:;
} }
const bool inside_txn = (env->me_txn0->mt_owner == osal_thread_self()); const bool inside_txn = (env->me_txn0->mt_owner == osal_thread_self());
meta_ptr_t head; const meta_troika_t troika =
if (inside_txn | locked) (inside_txn | locked) ? env->me_txn0->tw.troika : meta_tap(env);
head = meta_recent(env, &env->me_txn0->tw.troika); const meta_ptr_t head = meta_recent(env, &troika);
else {
const meta_troika_t troika = meta_tap(env);
head = meta_recent(env, &troika);
}
const uint64_t unsynced_pages = const uint64_t unsynced_pages =
atomic_load64(&env->me_lck->mti_unsynced_pages, mo_Relaxed); atomic_load64(&env->me_lck->mti_unsynced_pages, mo_Relaxed);
if (unsynced_pages == 0) { if (unsynced_pages == 0) {
@ -8105,10 +8133,19 @@ retry:;
if (!inside_txn && locked && (env->me_flags & MDBX_WRITEMAP) && if (!inside_txn && locked && (env->me_flags & MDBX_WRITEMAP) &&
unlikely(head.ptr_c->mm_geo.next > unlikely(head.ptr_c->mm_geo.next >
bytes2pgno(env, env->me_dxb_mmap.current))) { bytes2pgno(env, env->me_dxb_mmap.current))) {
rc = dxb_resize(env, head.ptr_c->mm_geo.next, head.ptr_c->mm_geo.now,
head.ptr_c->mm_geo.upper, implicit_grow); if (unlikely(env->me_stuck_meta >= 0) &&
if (unlikely(rc != MDBX_SUCCESS)) troika.recent != (uint8_t)env->me_stuck_meta) {
goto bailout; NOTICE("skip %s since wagering meta-page (%u) is mispatch the recent "
"meta-page (%u)",
"sync datafile", env->me_stuck_meta, troika.recent);
rc = MDBX_RESULT_TRUE;
} else {
rc = dxb_resize(env, head.ptr_c->mm_geo.next, head.ptr_c->mm_geo.now,
head.ptr_c->mm_geo.upper, implicit_grow);
if (unlikely(rc != MDBX_SUCCESS))
goto bailout;
}
} }
const size_t autosync_threshold = const size_t autosync_threshold =
@ -8187,6 +8224,14 @@ retry:;
eASSERT(env, inside_txn || locked); eASSERT(env, inside_txn || locked);
eASSERT(env, !inside_txn || (flags & MDBX_SHRINK_ALLOWED) == 0); eASSERT(env, !inside_txn || (flags & MDBX_SHRINK_ALLOWED) == 0);
if (!head.is_steady && unlikely(env->me_stuck_meta >= 0) &&
troika.recent != (uint8_t)env->me_stuck_meta) {
NOTICE("skip %s since wagering meta-page (%u) is mispatch the recent "
"meta-page (%u)",
"sync datafile", env->me_stuck_meta, troika.recent);
rc = MDBX_RESULT_TRUE;
goto bailout;
}
if (!head.is_steady || ((flags & MDBX_SAFE_NOSYNC) == 0 && unsynced_pages)) { if (!head.is_steady || ((flags & MDBX_SAFE_NOSYNC) == 0 && unsynced_pages)) {
DEBUG("meta-head %" PRIaPGNO ", %s, sync_pending %" PRIu64, DEBUG("meta-head %" PRIaPGNO ", %s, sync_pending %" PRIu64,
data_page(head.ptr_c)->mp_pgno, durable_caption(head.ptr_c), data_page(head.ptr_c)->mp_pgno, durable_caption(head.ptr_c),
@ -13732,8 +13777,9 @@ __cold static int setup_dxb(MDBX_env *env, const int lck_rc,
mdbx_is_readahead_reasonable(used_bytes, 0) == MDBX_RESULT_TRUE; mdbx_is_readahead_reasonable(used_bytes, 0) == MDBX_RESULT_TRUE;
#endif /* MDBX_ENABLE_MADVISE */ #endif /* MDBX_ENABLE_MADVISE */
err = osal_mmap(env->me_flags, &env->me_dxb_mmap, env->me_dbgeo.now, err = osal_mmap(
env->me_dbgeo.upper, lck_rc ? MMAP_OPTION_TRUNCATE : 0); env->me_flags, &env->me_dxb_mmap, env->me_dbgeo.now, env->me_dbgeo.upper,
(lck_rc && env->me_stuck_meta < 0) ? MMAP_OPTION_TRUNCATE : 0);
if (unlikely(err != MDBX_SUCCESS)) if (unlikely(err != MDBX_SUCCESS))
return err; return err;
@ -13933,7 +13979,12 @@ __cold static int setup_dxb(MDBX_env *env, const int lck_rc,
} }
const meta_ptr_t recent = meta_recent(env, &troika); const meta_ptr_t recent = meta_recent(env, &troika);
if (memcmp(&header.mm_geo, &recent.ptr_c->mm_geo, sizeof(header.mm_geo))) { if (/* не учитываем различия в geo.next */
header.mm_geo.grow_pv != recent.ptr_c->mm_geo.grow_pv ||
header.mm_geo.shrink_pv != recent.ptr_c->mm_geo.shrink_pv ||
header.mm_geo.lower != recent.ptr_c->mm_geo.lower ||
header.mm_geo.upper != recent.ptr_c->mm_geo.upper ||
header.mm_geo.now != recent.ptr_c->mm_geo.now) {
if ((env->me_flags & MDBX_RDONLY) != 0 || if ((env->me_flags & MDBX_RDONLY) != 0 ||
/* recovery mode */ env->me_stuck_meta >= 0) { /* recovery mode */ env->me_stuck_meta >= 0) {
WARNING("skipped update meta.geo in %s mode: from l%" PRIaPGNO WARNING("skipped update meta.geo in %s mode: from l%" PRIaPGNO
@ -14383,8 +14434,12 @@ __cold static int __must_check_result override_meta(MDBX_env *env,
if (unlikely(MDBX_IS_ERROR(rc))) if (unlikely(MDBX_IS_ERROR(rc)))
return MDBX_PROBLEM; return MDBX_PROBLEM;
if (shape && memcmp(model, shape, sizeof(MDBX_meta)) == 0) if (shape && memcmp(model, shape, sizeof(MDBX_meta)) == 0) {
NOTICE("skip overriding meta-%zu since no changes "
"for txnid #%" PRIaTXN,
target, txnid);
return MDBX_SUCCESS; return MDBX_SUCCESS;
}
if (env->me_flags & MDBX_WRITEMAP) { if (env->me_flags & MDBX_WRITEMAP) {
#if MDBX_ENABLE_PGOP_STAT #if MDBX_ENABLE_PGOP_STAT
@ -14438,14 +14493,16 @@ __cold int mdbx_env_turn_for_recovery(MDBX_env *env, unsigned target) {
MDBX_EXCLUSIVE)) MDBX_EXCLUSIVE))
return MDBX_EPERM; return MDBX_EPERM;
const MDBX_meta *target_meta = METAPAGE(env, target); const MDBX_meta *const target_meta = METAPAGE(env, target);
txnid_t new_txnid = safe64_txnid_next(constmeta_txnid(target_meta)); txnid_t new_txnid = constmeta_txnid(target_meta);
for (size_t n = 0; n < NUM_METAS; ++n) { if (new_txnid < MIN_TXNID)
new_txnid = MIN_TXNID;
for (unsigned n = 0; n < NUM_METAS; ++n) {
if (n == target) if (n == target)
continue; continue;
MDBX_meta meta = *METAPAGE(env, target); MDBX_page *const page = pgno2page(env, n);
if (validate_meta(env, &meta, pgno2page(env, n), (pgno_t)n, nullptr) != MDBX_meta meta = *page_meta(page);
MDBX_SUCCESS) { if (validate_meta(env, &meta, page, n, nullptr) != MDBX_SUCCESS) {
int err = override_meta(env, n, 0, nullptr); int err = override_meta(env, n, 0, nullptr);
if (unlikely(err != MDBX_SUCCESS)) if (unlikely(err != MDBX_SUCCESS))
return err; return err;
@ -16719,7 +16776,7 @@ static int cursor_last(MDBX_cursor *mc, MDBX_val *key, MDBX_val *data) {
static __hot int cursor_get(MDBX_cursor *mc, MDBX_val *key, MDBX_val *data, static __hot int cursor_get(MDBX_cursor *mc, MDBX_val *key, MDBX_val *data,
MDBX_cursor_op op) { MDBX_cursor_op op) {
int (*mfunc)(MDBX_cursor * mc, MDBX_val * key, MDBX_val * data); int (*mfunc)(MDBX_cursor *mc, MDBX_val *key, MDBX_val *data);
int rc; int rc;
switch (op) { switch (op) {
@ -17889,7 +17946,7 @@ static __hot int cursor_put_checklen(MDBX_cursor *mc, const MDBX_val *key,
} }
int mdbx_cursor_put(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data, int mdbx_cursor_put(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data,
unsigned flags) { MDBX_put_flags_t flags) {
if (unlikely(mc == NULL || key == NULL || data == NULL)) if (unlikely(mc == NULL || key == NULL || data == NULL))
return MDBX_EINVAL; return MDBX_EINVAL;
@ -21016,7 +21073,7 @@ done:
} }
int mdbx_put(MDBX_txn *txn, MDBX_dbi dbi, const MDBX_val *key, MDBX_val *data, int mdbx_put(MDBX_txn *txn, MDBX_dbi dbi, const MDBX_val *key, MDBX_val *data,
unsigned flags) { MDBX_put_flags_t flags) {
int rc = check_txn_rw(txn, MDBX_TXN_BLOCKED); int rc = check_txn_rw(txn, MDBX_TXN_BLOCKED);
if (unlikely(rc != MDBX_SUCCESS)) if (unlikely(rc != MDBX_SUCCESS))
return rc; return rc;
@ -21431,7 +21488,8 @@ __cold static void meta_make_sizeable(MDBX_meta *meta) {
/* Copy environment with compaction. */ /* Copy environment with compaction. */
__cold static int env_compact(MDBX_env *env, MDBX_txn *read_txn, __cold static int env_compact(MDBX_env *env, MDBX_txn *read_txn,
mdbx_filehandle_t fd, uint8_t *buffer, mdbx_filehandle_t fd, uint8_t *buffer,
const bool dest_is_pipe, const int flags) { const bool dest_is_pipe,
const MDBX_copy_flags_t flags) {
const size_t meta_bytes = pgno2bytes(env, NUM_METAS); const size_t meta_bytes = pgno2bytes(env, NUM_METAS);
uint8_t *const data_buffer = uint8_t *const data_buffer =
buffer + ceil_powerof2(meta_bytes, env->me_os_psize); buffer + ceil_powerof2(meta_bytes, env->me_os_psize);
@ -21578,7 +21636,8 @@ __cold static int env_compact(MDBX_env *env, MDBX_txn *read_txn,
/* Copy environment as-is. */ /* Copy environment as-is. */
__cold static int env_copy_asis(MDBX_env *env, MDBX_txn *read_txn, __cold static int env_copy_asis(MDBX_env *env, MDBX_txn *read_txn,
mdbx_filehandle_t fd, uint8_t *buffer, mdbx_filehandle_t fd, uint8_t *buffer,
const bool dest_is_pipe, const int flags) { const bool dest_is_pipe,
const MDBX_copy_flags_t flags) {
/* We must start the actual read txn after blocking writers */ /* We must start the actual read txn after blocking writers */
int rc = txn_end(read_txn, MDBX_END_RESET_TMP); int rc = txn_end(read_txn, MDBX_END_RESET_TMP);
if (unlikely(rc != MDBX_SUCCESS)) if (unlikely(rc != MDBX_SUCCESS))
@ -21704,7 +21763,7 @@ __cold static int env_copy_asis(MDBX_env *env, MDBX_txn *read_txn,
} }
__cold int mdbx_env_copy2fd(MDBX_env *env, mdbx_filehandle_t fd, __cold int mdbx_env_copy2fd(MDBX_env *env, mdbx_filehandle_t fd,
unsigned flags) { MDBX_copy_flags_t flags) {
int rc = check_env(env, true); int rc = check_env(env, true);
if (unlikely(rc != MDBX_SUCCESS)) if (unlikely(rc != MDBX_SUCCESS))
return rc; return rc;
@ -22367,13 +22426,13 @@ __cold int mdbx_env_info_ex(const MDBX_env *env, const MDBX_txn *txn,
} }
} }
static __inline MDBX_cmp_func *get_default_keycmp(unsigned flags) { static __inline MDBX_cmp_func *get_default_keycmp(MDBX_db_flags_t flags) {
return (flags & MDBX_REVERSEKEY) ? cmp_reverse return (flags & MDBX_REVERSEKEY) ? cmp_reverse
: (flags & MDBX_INTEGERKEY) ? cmp_int_align2 : (flags & MDBX_INTEGERKEY) ? cmp_int_align2
: cmp_lexical; : cmp_lexical;
} }
static __inline MDBX_cmp_func *get_default_datacmp(unsigned flags) { static __inline MDBX_cmp_func *get_default_datacmp(MDBX_db_flags_t flags) {
return !(flags & MDBX_DUPSORT) return !(flags & MDBX_DUPSORT)
? cmp_lenfast ? cmp_lenfast
: ((flags & MDBX_INTEGERDUP) : ((flags & MDBX_INTEGERDUP)
@ -23220,7 +23279,8 @@ __cold MDBX_INTERNAL_FUNC int cleanup_dead_readers(MDBX_env *env,
return rc; return rc;
} }
__cold int mdbx_setup_debug(int level, int flags, MDBX_debug_func *logger) { __cold int mdbx_setup_debug(MDBX_log_level_t level, MDBX_debug_flags_t flags,
MDBX_debug_func *logger) {
const int rc = runtime_flags | (loglevel << 16); const int rc = runtime_flags | (loglevel << 16);
if (level != MDBX_LOG_DONTCHANGE) if (level != MDBX_LOG_DONTCHANGE)
@ -24671,11 +24731,11 @@ int64_t mdbx_int64_from_key(const MDBX_val v) {
UINT64_C(0x8000000000000000)); UINT64_C(0x8000000000000000));
} }
__cold MDBX_cmp_func *mdbx_get_keycmp(unsigned flags) { __cold MDBX_cmp_func *mdbx_get_keycmp(MDBX_db_flags_t flags) {
return get_default_keycmp(flags); return get_default_keycmp(flags);
} }
__cold MDBX_cmp_func *mdbx_get_datacmp(unsigned flags) { __cold MDBX_cmp_func *mdbx_get_datacmp(MDBX_db_flags_t flags) {
return get_default_datacmp(flags); return get_default_datacmp(flags);
} }

View File

@ -90,8 +90,8 @@
#endif #endif
#if _MSC_VER > 1914 #if _MSC_VER > 1914
#pragma warning( \ #pragma warning( \
disable : 5105) /* winbase.h(9531): warning C5105: macro expansion \ disable : 5105) /* winbase.h(9531): warning C5105: macro expansion \
producing 'defined' has undefined behavior */ producing 'defined' has undefined behavior */
#endif #endif
#if _MSC_VER > 1930 #if _MSC_VER > 1930
#pragma warning(disable : 6235) /* <expression> is always a constant */ #pragma warning(disable : 6235) /* <expression> is always a constant */
@ -101,16 +101,16 @@
#pragma warning(disable : 4710) /* 'xyz': function not inlined */ #pragma warning(disable : 4710) /* 'xyz': function not inlined */
#pragma warning(disable : 4711) /* function 'xyz' selected for automatic \ #pragma warning(disable : 4711) /* function 'xyz' selected for automatic \
inline expansion */ inline expansion */
#pragma warning( \ #pragma warning(disable : 4201) /* nonstandard extension used: nameless \
disable : 4201) /* nonstandard extension used : nameless struct / union */ struct/union */
#pragma warning(disable : 4702) /* unreachable code */ #pragma warning(disable : 4702) /* unreachable code */
#pragma warning(disable : 4706) /* assignment within conditional expression */ #pragma warning(disable : 4706) /* assignment within conditional expression */
#pragma warning(disable : 4127) /* conditional expression is constant */ #pragma warning(disable : 4127) /* conditional expression is constant */
#pragma warning(disable : 4324) /* 'xyz': structure was padded due to \ #pragma warning(disable : 4324) /* 'xyz': structure was padded due to \
alignment specifier */ alignment specifier */
#pragma warning(disable : 4310) /* cast truncates constant value */ #pragma warning(disable : 4310) /* cast truncates constant value */
#pragma warning( \ #pragma warning(disable : 4820) /* bytes padding added after data member for \
disable : 4820) /* bytes padding added after data member for alignment */ alignment */
#pragma warning(disable : 4548) /* expression before comma has no effect; \ #pragma warning(disable : 4548) /* expression before comma has no effect; \
expected expression with side - effect */ expected expression with side - effect */
#pragma warning(disable : 4366) /* the result of the unary '&' operator may be \ #pragma warning(disable : 4366) /* the result of the unary '&' operator may be \
@ -120,8 +120,8 @@
#pragma warning(disable : 4204) /* nonstandard extension used: non-constant \ #pragma warning(disable : 4204) /* nonstandard extension used: non-constant \
aggregate initializer */ aggregate initializer */
#pragma warning( \ #pragma warning( \
disable : 4505) /* unreferenced local function has been removed */ disable : 4505) /* unreferenced local function has been removed */
#endif /* _MSC_VER (warnings) */ #endif /* _MSC_VER (warnings) */
#if defined(__GNUC__) && __GNUC__ < 9 #if defined(__GNUC__) && __GNUC__ < 9
#pragma GCC diagnostic ignored "-Wattributes" #pragma GCC diagnostic ignored "-Wattributes"

View File

@ -1,6 +1,6 @@
.\" Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>. .\" Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE. .\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_CHK 1 "2023-04-18" "MDBX 0.12.5" .TH MDBX_CHK 1 "2023-06-16" "MDBX 0.12.7"
.SH NAME .SH NAME
mdbx_chk \- MDBX checking tool mdbx_chk \- MDBX checking tool
.SH SYNOPSIS .SH SYNOPSIS

View File

@ -2,7 +2,7 @@
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>. .\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved. .\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE. .\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_COPY 1 "2023-04-18" "MDBX 0.12.5" .TH MDBX_COPY 1 "2023-06-16" "MDBX 0.12.7"
.SH NAME .SH NAME
mdbx_copy \- MDBX environment copy tool mdbx_copy \- MDBX environment copy tool
.SH SYNOPSIS .SH SYNOPSIS

View File

@ -1,7 +1,7 @@
.\" Copyright 2021-2023 Leonid Yuriev <leo@yuriev.ru>. .\" Copyright 2021-2023 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2014-2021 Howard Chu, Symas Corp. All Rights Reserved. .\" Copyright 2014-2021 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE. .\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_DROP 1 "2023-04-18" "MDBX 0.12.5" .TH MDBX_DROP 1 "2023-06-16" "MDBX 0.12.7"
.SH NAME .SH NAME
mdbx_drop \- MDBX database delete tool mdbx_drop \- MDBX database delete tool
.SH SYNOPSIS .SH SYNOPSIS

View File

@ -2,7 +2,7 @@
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>. .\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved. .\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE. .\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_DUMP 1 "2023-04-18" "MDBX 0.12.5" .TH MDBX_DUMP 1 "2023-06-16" "MDBX 0.12.7"
.SH NAME .SH NAME
mdbx_dump \- MDBX environment export tool mdbx_dump \- MDBX environment export tool
.SH SYNOPSIS .SH SYNOPSIS

View File

@ -2,7 +2,7 @@
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>. .\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved. .\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE. .\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_LOAD 1 "2023-04-18" "MDBX 0.12.5" .TH MDBX_LOAD 1 "2023-06-16" "MDBX 0.12.7"
.SH NAME .SH NAME
mdbx_load \- MDBX environment import tool mdbx_load \- MDBX environment import tool
.SH SYNOPSIS .SH SYNOPSIS

View File

@ -2,7 +2,7 @@
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>. .\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved. .\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE. .\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_STAT 1 "2023-04-18" "MDBX 0.12.5" .TH MDBX_STAT 1 "2023-06-16" "MDBX 0.12.7"
.SH NAME .SH NAME
mdbx_stat \- MDBX environment status tool mdbx_stat \- MDBX environment status tool
.SH SYNOPSIS .SH SYNOPSIS

View File

@ -246,8 +246,12 @@ typedef struct osal_mmap {
} osal_mmap_t; } osal_mmap_t;
typedef union bin128 { typedef union bin128 {
__anonymous_struct_extension__ struct { uint64_t x, y; }; __anonymous_struct_extension__ struct {
__anonymous_struct_extension__ struct { uint32_t a, b, c, d; }; uint64_t x, y;
};
__anonymous_struct_extension__ struct {
uint32_t a, b, c, d;
};
} bin128_t; } bin128_t;
#if defined(_WIN32) || defined(_WIN64) #if defined(_WIN32) || defined(_WIN64)

View File

@ -111,8 +111,8 @@
#ifdef _MSC_VER #ifdef _MSC_VER
#pragma warning(pop) #pragma warning(pop)
#pragma warning(disable : 4201) /* nonstandard extension used : \ #pragma warning(disable : 4201) /* nonstandard extension used: nameless \
nameless struct / union */ struct/union */
#pragma warning(disable : 4127) /* conditional expression is constant */ #pragma warning(disable : 4127) /* conditional expression is constant */
#if _MSC_VER < 1900 #if _MSC_VER < 1900
#pragma warning(disable : 4510) /* default constructor could \ #pragma warning(disable : 4510) /* default constructor could \

View File

@ -12,8 +12,8 @@
#include <unistd.h> #include <unistd.h>
static int dump(MDBX_cursor *cur) { static int dump(MDBX_cursor *cur) {
MDBX_val key = {}; MDBX_val key = {NULL, 0};
MDBX_val data = {}; MDBX_val data = {NULL, 0};
int rc = mdbx_cursor_get(cur, &key, &data, MDBX_FIRST); int rc = mdbx_cursor_get(cur, &key, &data, MDBX_FIRST);
while (rc == 0) { while (rc == 0) {
@ -25,15 +25,14 @@ static int dump(MDBX_cursor *cur) {
} }
static int clear(MDBX_cursor *cur) { static int clear(MDBX_cursor *cur) {
MDBX_val key = {}; MDBX_val key = {NULL, 0};
MDBX_val data = {}; MDBX_val data = {NULL, 0};
int rc = mdbx_cursor_get(cur, &key, &data, MDBX_FIRST); int rc = mdbx_cursor_get(cur, &key, &data, MDBX_FIRST);
while (rc == 0) { while (rc == 0) {
rc = mdbx_cursor_del(cur, MDBX_ALLDUPS); rc = mdbx_cursor_del(cur, MDBX_ALLDUPS);
if (rc) { if (rc)
return rc; return rc;
}
rc = mdbx_cursor_get(cur, &key, &data, MDBX_NEXT); rc = mdbx_cursor_get(cur, &key, &data, MDBX_NEXT);
} }
return (rc == MDBX_NOTFOUND) ? 0 : rc; return (rc == MDBX_NOTFOUND) ? 0 : rc;

View File

@ -41,7 +41,7 @@
#if _MSC_FULL_VER < 190024215 #if _MSC_FULL_VER < 190024215
#pragma message( \ #pragma message( \
"It is recommended to use Visual Studio 2015 (MSC 19.0) or newer.") "It is recommended to use Visual Studio 2015 (MSC 19.0) or newer.")
#endif #endif
#define bswap64(v) _byteswap_uint64(v) #define bswap64(v) _byteswap_uint64(v)