From d3daa23c632f15d20db3ee4b87e16734958813bc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Mon, 13 Jan 2025 13:13:26 +0300 Subject: [PATCH 01/90] =?UTF-8?q?mdbx:=20=D0=BE=D0=B1=D0=BD=D0=BE=D0=B2?= =?UTF-8?q?=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20ChangeLog=20(=D0=BF=D0=BE?= =?UTF-8?q?=D0=B4=D0=B3=D0=BE=D1=82=D0=BE=D0=B2=D0=BA=D0=B0=20=D0=B2=D1=8B?= =?UTF-8?q?=D0=BF=D1=83=D1=81=D0=BA=D0=B0=200.13.4).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ChangeLog.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/ChangeLog.md b/ChangeLog.md index 1a57e791..bc7ef66c 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -5,6 +5,16 @@ English version [by liar Google](https://libmdbx-dqdkfa-ru.translate.goog/md__ch and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/libmdbx.dqdkfa.ru/md__change_log.html). +## v0.13.4 в процессе накопления изменений + +Поддерживающий выпуск стабильной ветки с исправлением обнаруженных ошибок и устранением недочётов. + +Предполагаемая дата выпуска между 2025-02-11 и 2025-03-13. + + +-------------------------------------------------------------------------------- + + ## v0.13.3 "Королёв" от 2025-01-12 Поддерживающий выпуск с исправлением обнаруженных ошибок и устранением недочетов From 88d782e5eb5f61ccf1684e934388c1e61f85334f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Tue, 14 Jan 2025 13:00:53 +0300 Subject: [PATCH 02/90] =?UTF-8?q?mdbx:=20=D0=BE=D0=B1=D0=BD=D0=BE=D0=B2?= =?UTF-8?q?=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=BF=D0=B0=D1=82=D1=87=D0=B0?= =?UTF-8?q?=20=D0=B4=D0=BB=D1=8F=20=D1=81=D1=82=D0=B0=D1=80=D1=8B=D1=85=20?= =?UTF-8?q?=D0=B2=D0=B5=D1=80=D1=81=D0=B8=D0=B9=20buildroot=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...libmdbx-new-package-library-database.patch | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/packages/buildroot/0001-package-libmdbx-new-package-library-database.patch b/packages/buildroot/0001-package-libmdbx-new-package-library-database.patch index 98ffaa79..9c7afff4 100644 --- a/packages/buildroot/0001-package-libmdbx-new-package-library-database.patch +++ b/packages/buildroot/0001-package-libmdbx-new-package-library-database.patch @@ -1,8 +1,8 @@ -From 40efe497b511c322470aa9c084fe4c1759788c57 Mon Sep 17 00:00:00 2001 +From 0ba6ba5e6d6311213a21f033729e18826729230a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= -Date: Sun, 27 Oct 2024 22:34:19 +0300 -Subject: [PATCH 1/1] package/libmdbx: new package (library/database). +Date: Tue, 14 Jan 2025 12:57:03 +0300 +Subject: [PATCH] package/libmdbx: new package (library/database). MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @@ -15,9 +15,9 @@ This patch adds libmdbx: in terms of reliability, features and performance. - more information at https://gitflic.ru/project/erthink/libmdbx -The 0.12.12 "Dollezhal" is stable release of _libmdbx_ branch in memory -of the Soviet energy scientist Nikolai Antonovich Dollezhal on the 125th -anniversary of his birth. +The 0.13.13 "Korolev" is stable release of _libmdbx_ branch with new superior features +on the birthday and in memory of Sergei Korolev who was the lead Soviet rocket +engineer and spacecraft designer. The complete ChangeLog: https://gitflic.ru/project/erthink/libmdbx/blob?file=ChangeLog.md @@ -112,18 +112,18 @@ index 0000000000..a9a4ac45c5 + !BR2_TOOLCHAIN_GCC_AT_LEAST_4_4 diff --git a/package/libmdbx/libmdbx.hash b/package/libmdbx/libmdbx.hash new file mode 100644 -index 0000000000..222e7caf5c +index 0000000000..1c91aa2f90 --- /dev/null +++ b/package/libmdbx/libmdbx.hash @@ -0,0 +1,5 @@ +# Hashes from: https://libmdbx.dqdkfa.ru/release/SHA256SUMS -+sha256 19c0eb33e1ed43ca2a94dceb06dd31946432d16f30a9751d3701c67efa22eb1a libmdbx-amalgamated-0.12.12.tar.xz ++sha256 2e42505f1ceb57945569db3c1a5db9b5216d8f72da7c75c240ff81196f8e9a0b libmdbx-amalgamated-0.13.3.tar.xz + +# Locally calculated -+sha256 310fe25c858a9515fc8c8d7d1f24a67c9496f84a91e0a0e41ea9975b1371e569 LICENSE ++sha256 0d542e0c8804e39aa7f37eb00da5a762149dc682d7829451287e11b938e94594 LICENSE diff --git a/package/libmdbx/libmdbx.mk b/package/libmdbx/libmdbx.mk new file mode 100644 -index 0000000000..da4a53913f +index 0000000000..b42ab629fe --- /dev/null +++ b/package/libmdbx/libmdbx.mk @@ -0,0 +1,42 @@ @@ -133,7 +133,7 @@ index 0000000000..da4a53913f +# +################################################################################ + -+LIBMDBX_VERSION = 0.12.12 ++LIBMDBX_VERSION = 0.13.3 +LIBMDBX_SOURCE = libmdbx-amalgamated-$(LIBMDBX_VERSION).tar.xz +LIBMDBX_SITE = https://libmdbx.dqdkfa.ru/release +LIBMDBX_SUPPORTS_IN_SOURCE_BUILD = NO @@ -170,5 +170,5 @@ index 0000000000..da4a53913f + +$(eval $(cmake-package)) -- -2.47.0 +2.48.0 From 36abcc57f0501835a92747c3710caacfb33f1598 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Wed, 15 Jan 2025 19:30:00 +0300 Subject: [PATCH 03/90] =?UTF-8?q?mdbx:=20=D0=BE=D0=B1=D0=BD=D0=BE=D0=B2?= =?UTF-8?q?=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=B3=D0=BE=D0=B4=D0=B0=20?= =?UTF-8?q?=D0=B2=20=C2=A9=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- CMakeLists.txt | 2 +- COPYRIGHT | 2 +- NOTICE | 2 +- cmake/compiler.cmake | 2 +- cmake/profile.cmake | 2 +- cmake/utils.cmake | 2 +- example/example-mdbx.c | 2 +- example/sample-bdb.txt | 2 +- mdbx.h | 2 +- mdbx.h++ | 2 +- src/alloy.c | 2 +- src/api-cold.c | 2 +- src/api-copy.c | 2 +- src/api-cursor.c | 2 +- src/api-dbi.c | 2 +- src/api-env.c | 2 +- src/api-extra.c | 2 +- src/api-key-transform.c | 2 +- src/api-misc.c | 2 +- src/api-opts.c | 2 +- src/api-range-estimate.c | 2 +- src/api-txn-data.c | 2 +- src/api-txn.c | 2 +- src/atomics-ops.h | 2 +- src/atomics-types.h | 2 +- src/audit.c | 2 +- src/chk.c | 2 +- src/cogs.c | 2 +- src/cogs.h | 2 +- src/coherency.c | 2 +- src/cursor.c | 2 +- src/cursor.h | 2 +- src/dbi.c | 2 +- src/dbi.h | 2 +- src/dpl.c | 2 +- src/dpl.h | 2 +- src/dxb.c | 2 +- src/env.c | 2 +- src/essentials.h | 2 +- src/gc-get.c | 2 +- src/gc-put.c | 2 +- src/gc.h | 2 +- src/global.c | 2 +- src/internals.h | 2 +- src/layout-dxb.h | 2 +- src/layout-lck.h | 2 +- src/lck-posix.c | 2 +- src/lck-windows.c | 2 +- src/lck.c | 2 +- src/lck.h | 2 +- src/logging_and_debug.c | 2 +- src/logging_and_debug.h | 2 +- src/man1/mdbx_chk.1 | 2 +- src/man1/mdbx_copy.1 | 2 +- src/man1/mdbx_drop.1 | 2 +- src/man1/mdbx_dump.1 | 2 +- src/man1/mdbx_load.1 | 2 +- src/man1/mdbx_stat.1 | 2 +- src/mdbx.c++ | 2 +- src/meta.c | 2 +- src/meta.h | 2 +- src/mvcc-readers.c | 2 +- src/node.c | 2 +- src/node.h | 2 +- src/options.h | 2 +- src/osal.c | 2 +- src/osal.h | 2 +- src/page-get.c | 2 +- src/page-iov.c | 2 +- src/page-iov.h | 2 +- src/page-ops.c | 2 +- src/page-ops.h | 2 +- src/pnl.c | 2 +- src/pnl.h | 2 +- src/preface.h | 2 +- src/proto.h | 2 +- src/refund.c | 2 +- src/sort.h | 2 +- src/spill.c | 2 +- src/spill.h | 2 +- src/table.c | 2 +- src/tls.c | 2 +- src/tls.h | 2 +- src/tools/chk.c | 2 +- src/tools/copy.c | 2 +- src/tools/drop.c | 2 +- src/tools/dump.c | 2 +- src/tools/load.c | 2 +- src/tools/stat.c | 2 +- src/tree-ops.c | 2 +- src/tree-search.c | 2 +- src/txl.c | 2 +- src/txl.h | 2 +- src/txn.c | 2 +- src/unaligned.h | 2 +- src/utils.c | 2 +- src/utils.h | 2 +- src/walk.c | 2 +- src/walk.h | 2 +- src/windows-import.c | 2 +- src/windows-import.h | 2 +- test/CMakeLists.txt | 2 +- test/append.c++ | 2 +- test/base.h++ | 2 +- test/cases.c++ | 2 +- test/chrono.c++ | 2 +- test/chrono.h++ | 2 +- test/config.c++ | 2 +- test/config.h++ | 2 +- test/copy.c++ | 2 +- test/dead.c++ | 2 +- test/extra/doubtless_positioning.c++ | 2 +- test/extra/dupfix_multiple.c++ | 2 +- test/extra/hex_base64_base58.c++ | 2 +- test/extra/maindb_ordinal.c++ | 2 +- test/extra/pcrf/pcrf_test.c | 2 +- test/fork.c++ | 2 +- test/hill.c++ | 2 +- test/jitter.c++ | 2 +- test/keygen.c++ | 2 +- test/keygen.h++ | 2 +- test/log.c++ | 2 +- test/log.h++ | 2 +- test/main.c++ | 2 +- test/nested.c++ | 2 +- test/osal-unix.c++ | 2 +- test/osal-windows.c++ | 2 +- test/osal.h++ | 2 +- test/test.c++ | 2 +- test/test.h++ | 2 +- test/try.c++ | 2 +- test/ttl.c++ | 2 +- test/utils.c++ | 2 +- test/utils.h++ | 2 +- 134 files changed, 134 insertions(+), 134 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index 8f8d1b02..0996291b 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -1,4 +1,4 @@ -# Copyright (c) 2020-2024 Леонид Юрьев aka Leonid Yuriev ############################################### +# Copyright (c) 2020-2025 Леонид Юрьев aka Leonid Yuriev ############################################### # SPDX-License-Identifier: Apache-2.0 # # Donations are welcome to ETH `0xD104d8f8B2dC312aaD74899F83EBf3EEBDC1EA3A`. Всё будет хорошо! diff --git a/COPYRIGHT b/COPYRIGHT index 81fadf1b..fb29a937 100644 --- a/COPYRIGHT +++ b/COPYRIGHT @@ -1,4 +1,4 @@ -Copyright (c) 2015-2024 Леонид Юрьев aka Leonid Yuriev +Copyright (c) 2015-2025 Леонид Юрьев aka Leonid Yuriev Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. diff --git a/NOTICE b/NOTICE index 639c628c..4df84dba 100644 --- a/NOTICE +++ b/NOTICE @@ -11,7 +11,7 @@ to the Telegram' group https://t.me/libmdbx. Donations are welcome to ETH `0xD104d8f8B2dC312aaD74899F83EBf3EEBDC1EA3A`. Всё будет хорошо! -Copyright 2015-2024 Леонид Юрьев aka Leonid Yuriev +Copyright 2015-2025 Леонид Юрьев aka Leonid Yuriev SPDX-License-Identifier: Apache-2.0 For notes about the license change, credits and acknowledgments, please refer to the COPYRIGHT file within original libmdbx source code diff --git a/cmake/compiler.cmake b/cmake/compiler.cmake index 05c067f9..5fefdbae 100644 --- a/cmake/compiler.cmake +++ b/cmake/compiler.cmake @@ -1,4 +1,4 @@ -# Copyright (c) 2010-2024 Леонид Юрьев aka Leonid Yuriev ############################################### +# Copyright (c) 2010-2025 Леонид Юрьев aka Leonid Yuriev ############################################### # SPDX-License-Identifier: Apache-2.0 if(CMAKE_VERSION VERSION_LESS 3.8.2) diff --git a/cmake/profile.cmake b/cmake/profile.cmake index dc930f3a..6a8a466e 100644 --- a/cmake/profile.cmake +++ b/cmake/profile.cmake @@ -1,4 +1,4 @@ -# Copyright (c) 2012-2024 Леонид Юрьев aka Leonid Yuriev ############################################### +# Copyright (c) 2012-2025 Леонид Юрьев aka Leonid Yuriev ############################################### # SPDX-License-Identifier: Apache-2.0 if(CMAKE_VERSION VERSION_LESS 3.8.2) diff --git a/cmake/utils.cmake b/cmake/utils.cmake index adb8004d..abb4cd30 100644 --- a/cmake/utils.cmake +++ b/cmake/utils.cmake @@ -1,4 +1,4 @@ -# Copyright (c) 2012-2024 Леонид Юрьев aka Leonid Yuriev ############################################### +# Copyright (c) 2012-2025 Леонид Юрьев aka Leonid Yuriev ############################################### # SPDX-License-Identifier: Apache-2.0 if(CMAKE_VERSION VERSION_LESS 3.8.2) diff --git a/example/example-mdbx.c b/example/example-mdbx.c index beae17f5..a0eeb121 100644 --- a/example/example-mdbx.c +++ b/example/example-mdbx.c @@ -4,7 +4,7 @@ */ /* - * Copyright 2015-2023 Leonid Yuriev . + * Copyright 2015-2025 Leonid Yuriev . * Copyright 2017 Ilya Shipitsin . * Copyright 2012-2015 Howard Chu, Symas Corp. * All rights reserved. diff --git a/example/sample-bdb.txt b/example/sample-bdb.txt index d3478a16..5dc9a78c 100644 --- a/example/sample-bdb.txt +++ b/example/sample-bdb.txt @@ -4,7 +4,7 @@ */ /* - * Copyright 2015-2023 Leonid Yuriev . + * Copyright 2015-2025 Leonid Yuriev . * Copyright 2012-2015 Howard Chu, Symas Corp. * Copyright 2015,2016 Peter-Service R&D LLC. * All rights reserved. diff --git a/mdbx.h b/mdbx.h index af01e309..a0886799 100644 --- a/mdbx.h +++ b/mdbx.h @@ -31,7 +31,7 @@ developers. For the same reason ~~Github~~ is blacklisted forever. \copyright SPDX-License-Identifier: Apache-2.0 \note Please refer to the COPYRIGHT file for explanations license change, credits and acknowledgments. -\author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +\author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 *******************************************************************************/ diff --git a/mdbx.h++ b/mdbx.h++ index e96c2012..62315f61 100644 --- a/mdbx.h++ +++ b/mdbx.h++ @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2020-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2020-2025 /// /// Donations are welcome to ETH `0xD104d8f8B2dC312aaD74899F83EBf3EEBDC1EA3A`. /// Всё будет хорошо! diff --git a/src/alloy.c b/src/alloy.c index 076d39da..ef96cf94 100644 --- a/src/alloy.c +++ b/src/alloy.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #define xMDBX_ALLOY 1 /* alloyed build */ #include "internals.h" /* must be included first */ diff --git a/src/api-cold.c b/src/api-cold.c index 7498d8ab..1d124852 100644 --- a/src/api-cold.c +++ b/src/api-cold.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/api-copy.c b/src/api-copy.c index 3c7a4f2c..08c01192 100644 --- a/src/api-copy.c +++ b/src/api-copy.c @@ -1,7 +1,7 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 /// \note Please refer to the COPYRIGHT file for explanations license change, /// credits and acknowledgments. -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/api-cursor.c b/src/api-cursor.c index e52f5817..a10e09ef 100644 --- a/src/api-cursor.c +++ b/src/api-cursor.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/api-dbi.c b/src/api-dbi.c index 93bbdf21..e7f6f27d 100644 --- a/src/api-dbi.c +++ b/src/api-dbi.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/api-env.c b/src/api-env.c index dfd7ca6f..9a6be7b3 100644 --- a/src/api-env.c +++ b/src/api-env.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/api-extra.c b/src/api-extra.c index dd394f7b..61b0fac9 100644 --- a/src/api-extra.c +++ b/src/api-extra.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/api-key-transform.c b/src/api-key-transform.c index 6143a62e..6c42260d 100644 --- a/src/api-key-transform.c +++ b/src/api-key-transform.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/api-misc.c b/src/api-misc.c index c34c9df7..7e54c51d 100644 --- a/src/api-misc.c +++ b/src/api-misc.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/api-opts.c b/src/api-opts.c index d470bf25..2465864a 100644 --- a/src/api-opts.c +++ b/src/api-opts.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/api-range-estimate.c b/src/api-range-estimate.c index de3e1d12..5356d4da 100644 --- a/src/api-range-estimate.c +++ b/src/api-range-estimate.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/api-txn-data.c b/src/api-txn-data.c index b3cee3c3..76bdf32a 100644 --- a/src/api-txn-data.c +++ b/src/api-txn-data.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/api-txn.c b/src/api-txn.c index 6831fd5d..61b994bd 100644 --- a/src/api-txn.c +++ b/src/api-txn.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/atomics-ops.h b/src/atomics-ops.h index c61c055d..7dae2f34 100644 --- a/src/atomics-ops.h +++ b/src/atomics-ops.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/atomics-types.h b/src/atomics-types.h index fe977b94..0dbf747d 100644 --- a/src/atomics-types.h +++ b/src/atomics-types.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/audit.c b/src/audit.c index 91e18873..094722ff 100644 --- a/src/audit.c +++ b/src/audit.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/chk.c b/src/chk.c index 4197b29c..d848b014 100644 --- a/src/chk.c +++ b/src/chk.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/cogs.c b/src/cogs.c index 2e7d18c6..7fc924ca 100644 --- a/src/cogs.c +++ b/src/cogs.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/cogs.h b/src/cogs.h index 5b8f20c7..11d7fe1d 100644 --- a/src/cogs.h +++ b/src/cogs.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/coherency.c b/src/coherency.c index 6b9df51d..af1cc6fd 100644 --- a/src/coherency.c +++ b/src/coherency.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/cursor.c b/src/cursor.c index 8e92297e..43cf8e48 100644 --- a/src/cursor.c +++ b/src/cursor.c @@ -1,7 +1,7 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 /// \note Please refer to the COPYRIGHT file for explanations license change, /// credits and acknowledgments. -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/cursor.h b/src/cursor.h index 0ea13903..c2de019b 100644 --- a/src/cursor.h +++ b/src/cursor.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/dbi.c b/src/dbi.c index 7716e6d2..13d5a3eb 100644 --- a/src/dbi.c +++ b/src/dbi.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/dbi.h b/src/dbi.h index c654a0fa..e9b16efe 100644 --- a/src/dbi.h +++ b/src/dbi.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/dpl.c b/src/dpl.c index 1d2bef79..d1a1ef52 100644 --- a/src/dpl.c +++ b/src/dpl.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/dpl.h b/src/dpl.h index 0059dc58..ea66d214 100644 --- a/src/dpl.h +++ b/src/dpl.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/dxb.c b/src/dxb.c index b85258bb..aa222025 100644 --- a/src/dxb.c +++ b/src/dxb.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/env.c b/src/env.c index 58ee0346..525ed8ec 100644 --- a/src/env.c +++ b/src/env.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/essentials.h b/src/essentials.h index 3331e524..f46aec2e 100644 --- a/src/essentials.h +++ b/src/essentials.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/gc-get.c b/src/gc-get.c index c6d2c62b..bb1c9680 100644 --- a/src/gc-get.c +++ b/src/gc-get.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/gc-put.c b/src/gc-put.c index 4c6fd070..b5231773 100644 --- a/src/gc-put.c +++ b/src/gc-put.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/gc.h b/src/gc.h index b3ec23cf..171037f4 100644 --- a/src/gc.h +++ b/src/gc.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/global.c b/src/global.c index e8d13cf4..e3435f72 100644 --- a/src/global.c +++ b/src/global.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/internals.h b/src/internals.h index bb7fabfa..57b91f8b 100644 --- a/src/internals.h +++ b/src/internals.h @@ -1,7 +1,7 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 /// \note Please refer to the COPYRIGHT file for explanations license change, /// credits and acknowledgments. -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/layout-dxb.h b/src/layout-dxb.h index 3d9514c8..001bc59d 100644 --- a/src/layout-dxb.h +++ b/src/layout-dxb.h @@ -1,7 +1,7 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 /// \note Please refer to the COPYRIGHT file for explanations license change, /// credits and acknowledgments. -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/layout-lck.h b/src/layout-lck.h index b635c595..accf79a9 100644 --- a/src/layout-lck.h +++ b/src/layout-lck.h @@ -1,7 +1,7 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 /// \note Please refer to the COPYRIGHT file for explanations license change, /// credits and acknowledgments. -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/lck-posix.c b/src/lck-posix.c index 41f49f48..99a01f91 100644 --- a/src/lck-posix.c +++ b/src/lck-posix.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #if !(defined(_WIN32) || defined(_WIN64)) /*----------------------------------------------------------------------------* diff --git a/src/lck-windows.c b/src/lck-windows.c index 10c362a6..296679d1 100644 --- a/src/lck-windows.c +++ b/src/lck-windows.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #if defined(_WIN32) || defined(_WIN64) diff --git a/src/lck.c b/src/lck.c index 097aeb69..0bb3205c 100644 --- a/src/lck.c +++ b/src/lck.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/lck.h b/src/lck.h index eee38e15..ba1ebc88 100644 --- a/src/lck.h +++ b/src/lck.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/logging_and_debug.c b/src/logging_and_debug.c index f796db35..e8538934 100644 --- a/src/logging_and_debug.c +++ b/src/logging_and_debug.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/logging_and_debug.h b/src/logging_and_debug.h index e99ccea5..3141f3cd 100644 --- a/src/logging_and_debug.h +++ b/src/logging_and_debug.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/man1/mdbx_chk.1 b/src/man1/mdbx_chk.1 index b53422c7..bc6de4b7 100644 --- a/src/man1/mdbx_chk.1 +++ b/src/man1/mdbx_chk.1 @@ -1,4 +1,4 @@ -.\" Copyright 2015-2024 Leonid Yuriev . +.\" Copyright 2015-2025 Leonid Yuriev . .\" Copying restrictions apply. See COPYRIGHT/LICENSE. .TH MDBX_CHK 1 "2024-08-29" "MDBX 0.13" .SH NAME diff --git a/src/man1/mdbx_copy.1 b/src/man1/mdbx_copy.1 index 16c0fced..83ec8553 100644 --- a/src/man1/mdbx_copy.1 +++ b/src/man1/mdbx_copy.1 @@ -1,4 +1,4 @@ -.\" Copyright 2015-2024 Leonid Yuriev . +.\" Copyright 2015-2025 Leonid Yuriev . .\" Copyright 2015,2016 Peter-Service R&D LLC . .\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved. .\" Copying restrictions apply. See COPYRIGHT/LICENSE. diff --git a/src/man1/mdbx_drop.1 b/src/man1/mdbx_drop.1 index 3bf5683b..a6abbe41 100644 --- a/src/man1/mdbx_drop.1 +++ b/src/man1/mdbx_drop.1 @@ -1,4 +1,4 @@ -.\" Copyright 2021-2024 Leonid Yuriev . +.\" Copyright 2021-2025 Leonid Yuriev . .\" Copyright 2014-2021 Howard Chu, Symas Corp. All Rights Reserved. .\" Copying restrictions apply. See COPYRIGHT/LICENSE. .TH MDBX_DROP 1 "2024-08-29" "MDBX 0.13" diff --git a/src/man1/mdbx_dump.1 b/src/man1/mdbx_dump.1 index 96342054..030617c3 100644 --- a/src/man1/mdbx_dump.1 +++ b/src/man1/mdbx_dump.1 @@ -1,4 +1,4 @@ -.\" Copyright 2015-2024 Leonid Yuriev . +.\" Copyright 2015-2025 Leonid Yuriev . .\" Copyright 2015,2016 Peter-Service R&D LLC . .\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved. .\" Copying restrictions apply. See COPYRIGHT/LICENSE. diff --git a/src/man1/mdbx_load.1 b/src/man1/mdbx_load.1 index 5e790876..27b533e3 100644 --- a/src/man1/mdbx_load.1 +++ b/src/man1/mdbx_load.1 @@ -1,4 +1,4 @@ -.\" Copyright 2015-2024 Leonid Yuriev . +.\" Copyright 2015-2025 Leonid Yuriev . .\" Copyright 2015,2016 Peter-Service R&D LLC . .\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved. .\" Copying restrictions apply. See COPYRIGHT/LICENSE. diff --git a/src/man1/mdbx_stat.1 b/src/man1/mdbx_stat.1 index 935a541f..68a698c1 100644 --- a/src/man1/mdbx_stat.1 +++ b/src/man1/mdbx_stat.1 @@ -1,4 +1,4 @@ -.\" Copyright 2015-2024 Leonid Yuriev . +.\" Copyright 2015-2025 Leonid Yuriev . .\" Copyright 2015,2016 Peter-Service R&D LLC . .\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved. .\" Copying restrictions apply. See COPYRIGHT/LICENSE. diff --git a/src/mdbx.c++ b/src/mdbx.c++ index 17ca4718..d891b654 100644 --- a/src/mdbx.c++ +++ b/src/mdbx.c++ @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2020-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2020-2025 /// /// \brief Non-inline part of the libmdbx C++ API /// diff --git a/src/meta.c b/src/meta.c index cbdea2d0..562c81db 100644 --- a/src/meta.c +++ b/src/meta.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/meta.h b/src/meta.h index 899f1d5a..64ef59cb 100644 --- a/src/meta.h +++ b/src/meta.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/mvcc-readers.c b/src/mvcc-readers.c index be0220a9..e97eaaa5 100644 --- a/src/mvcc-readers.c +++ b/src/mvcc-readers.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/node.c b/src/node.c index 934bad65..f4b264fc 100644 --- a/src/node.c +++ b/src/node.c @@ -1,7 +1,7 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 /// \note Please refer to the COPYRIGHT file for explanations license change, /// credits and acknowledgments. -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/node.h b/src/node.h index 50de16e1..03069021 100644 --- a/src/node.h +++ b/src/node.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/options.h b/src/options.h index 1a80cd20..9449fed9 100644 --- a/src/options.h +++ b/src/options.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /******************************************************************************* ******************************************************************************* diff --git a/src/osal.c b/src/osal.c index dd1c7b4b..d4f4b03a 100644 --- a/src/osal.c +++ b/src/osal.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// /// https://en.wikipedia.org/wiki/Operating_system_abstraction_layer diff --git a/src/osal.h b/src/osal.h index c82f8af1..8f0d020c 100644 --- a/src/osal.h +++ b/src/osal.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// /// https://en.wikipedia.org/wiki/Operating_system_abstraction_layer diff --git a/src/page-get.c b/src/page-get.c index a07768a7..9ce3ecac 100644 --- a/src/page-get.c +++ b/src/page-get.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/page-iov.c b/src/page-iov.c index b6686017..0ec83807 100644 --- a/src/page-iov.c +++ b/src/page-iov.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/page-iov.h b/src/page-iov.h index bf367fef..1b298588 100644 --- a/src/page-iov.h +++ b/src/page-iov.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/page-ops.c b/src/page-ops.c index 6cecd119..43e6a79a 100644 --- a/src/page-ops.c +++ b/src/page-ops.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/page-ops.h b/src/page-ops.h index bb4ebd16..6874be60 100644 --- a/src/page-ops.h +++ b/src/page-ops.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/pnl.c b/src/pnl.c index 6a7f18ad..88f14c7f 100644 --- a/src/pnl.c +++ b/src/pnl.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/pnl.h b/src/pnl.h index ba033775..c0b7d542 100644 --- a/src/pnl.h +++ b/src/pnl.h @@ -1,7 +1,7 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 /// \note Please refer to the COPYRIGHT file for explanations license change, /// credits and acknowledgments. -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/preface.h b/src/preface.h index 962c7ae7..5f43b67e 100644 --- a/src/preface.h +++ b/src/preface.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/proto.h b/src/proto.h index 857a6794..842cbf62 100644 --- a/src/proto.h +++ b/src/proto.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/refund.c b/src/refund.c index 3d2fc368..68710614 100644 --- a/src/refund.c +++ b/src/refund.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/sort.h b/src/sort.h index 824c6bf5..bc676271 100644 --- a/src/sort.h +++ b/src/sort.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// /// \file sort.h /// \brief Маркосы реализующие сортировку и двоичный поиск diff --git a/src/spill.c b/src/spill.c index 261adb78..bded62c7 100644 --- a/src/spill.c +++ b/src/spill.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/spill.h b/src/spill.h index 32a8c9b2..5e7b7dad 100644 --- a/src/spill.h +++ b/src/spill.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/table.c b/src/table.c index c9a1e980..690ab9ff 100644 --- a/src/table.c +++ b/src/table.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/tls.c b/src/tls.c index 79f228f2..9d8fab2f 100644 --- a/src/tls.c +++ b/src/tls.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/tls.h b/src/tls.h index 5bcbfa23..5f3fd2b8 100644 --- a/src/tls.h +++ b/src/tls.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/tools/chk.c b/src/tools/chk.c index a22fd57b..6399ea60 100644 --- a/src/tools/chk.c +++ b/src/tools/chk.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// /// diff --git a/src/tools/copy.c b/src/tools/copy.c index 6b781025..2a55339b 100644 --- a/src/tools/copy.c +++ b/src/tools/copy.c @@ -1,7 +1,7 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 /// \note Please refer to the COPYRIGHT file for explanations license change, /// credits and acknowledgments. -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// /// mdbx_copy.c - memory-mapped database backup tool /// diff --git a/src/tools/drop.c b/src/tools/drop.c index 3c6e0f10..e74dcfe4 100644 --- a/src/tools/drop.c +++ b/src/tools/drop.c @@ -1,7 +1,7 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 /// \note Please refer to the COPYRIGHT file for explanations license change, /// credits and acknowledgments. -/// \author Леонид Юрьев aka Leonid Yuriev \date 2021-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2021-2025 /// /// mdbx_drop.c - memory-mapped database delete tool /// diff --git a/src/tools/dump.c b/src/tools/dump.c index 12e1f88f..c09b2b17 100644 --- a/src/tools/dump.c +++ b/src/tools/dump.c @@ -1,7 +1,7 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 /// \note Please refer to the COPYRIGHT file for explanations license change, /// credits and acknowledgments. -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// /// mdbx_dump.c - memory-mapped database dump tool /// diff --git a/src/tools/load.c b/src/tools/load.c index 7bb8c822..0220ce22 100644 --- a/src/tools/load.c +++ b/src/tools/load.c @@ -1,7 +1,7 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 /// \note Please refer to the COPYRIGHT file for explanations license change, /// credits and acknowledgments. -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// /// mdbx_load.c - memory-mapped database load tool /// diff --git a/src/tools/stat.c b/src/tools/stat.c index b3dd6811..c65d892a 100644 --- a/src/tools/stat.c +++ b/src/tools/stat.c @@ -1,7 +1,7 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 /// \note Please refer to the COPYRIGHT file for explanations license change, /// credits and acknowledgments. -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// /// mdbx_stat.c - memory-mapped database status tool /// diff --git a/src/tree-ops.c b/src/tree-ops.c index 3c51a65c..d0456792 100644 --- a/src/tree-ops.c +++ b/src/tree-ops.c @@ -1,7 +1,7 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 /// \note Please refer to the COPYRIGHT file for explanations license change, /// credits and acknowledgments. -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/tree-search.c b/src/tree-search.c index c19a9274..9cc68d7d 100644 --- a/src/tree-search.c +++ b/src/tree-search.c @@ -1,7 +1,7 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 /// \note Please refer to the COPYRIGHT file for explanations license change, /// credits and acknowledgments. -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/txl.c b/src/txl.c index c9830f81..d369f3bd 100644 --- a/src/txl.c +++ b/src/txl.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/txl.h b/src/txl.h index d8d67e05..79cb5524 100644 --- a/src/txl.h +++ b/src/txl.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/txn.c b/src/txn.c index 478f4d07..eb198965 100644 --- a/src/txn.c +++ b/src/txn.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/unaligned.h b/src/unaligned.h index 4085b51d..b119004f 100644 --- a/src/unaligned.h +++ b/src/unaligned.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/utils.c b/src/utils.c index 05b7a21e..ead1e4d3 100644 --- a/src/utils.c +++ b/src/utils.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/utils.h b/src/utils.h index 85563b09..9043a760 100644 --- a/src/utils.h +++ b/src/utils.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/walk.c b/src/walk.c index 585c350d..4a8c6e8c 100644 --- a/src/walk.c +++ b/src/walk.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #include "internals.h" diff --git a/src/walk.h b/src/walk.h index be6dc507..c87b8479 100644 --- a/src/walk.h +++ b/src/walk.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/src/windows-import.c b/src/windows-import.c index ee0ea9c8..d11cee29 100644 --- a/src/windows-import.c +++ b/src/windows-import.c @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #if defined(_WIN32) || defined(_WIN64) diff --git a/src/windows-import.h b/src/windows-import.h index 0362e9b6..d35fccdf 100644 --- a/src/windows-import.h +++ b/src/windows-import.h @@ -1,5 +1,5 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 #pragma once diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt index 28c4e74e..f8a72d68 100644 --- a/test/CMakeLists.txt +++ b/test/CMakeLists.txt @@ -1,4 +1,4 @@ -# Copyright (c) 2020-2024 Леонид Юрьев aka Leonid Yuriev ############################################### +# Copyright (c) 2020-2025 Леонид Юрьев aka Leonid Yuriev ############################################### # SPDX-License-Identifier: Apache-2.0 enable_language(CXX) diff --git a/test/append.c++ b/test/append.c++ index 59f6d1ca..91bbf33f 100644 --- a/test/append.c++ +++ b/test/append.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "test.h++" diff --git a/test/base.h++ b/test/base.h++ index 21af13a2..c3394f31 100644 --- a/test/base.h++ +++ b/test/base.h++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #pragma once diff --git a/test/cases.c++ b/test/cases.c++ index fdb99485..41461051 100644 --- a/test/cases.c++ +++ b/test/cases.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "test.h++" diff --git a/test/chrono.c++ b/test/chrono.c++ index daf285b1..04353037 100644 --- a/test/chrono.c++ +++ b/test/chrono.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "test.h++" diff --git a/test/chrono.h++ b/test/chrono.h++ index 0749db8a..12868b5e 100644 --- a/test/chrono.h++ +++ b/test/chrono.h++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #pragma once diff --git a/test/config.c++ b/test/config.c++ index 514f9ae4..4188e3c0 100644 --- a/test/config.c++ +++ b/test/config.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "test.h++" diff --git a/test/config.h++ b/test/config.h++ index 71ff5c97..7c5d0ee6 100644 --- a/test/config.h++ +++ b/test/config.h++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #pragma once diff --git a/test/copy.c++ b/test/copy.c++ index b5d068d8..777ad4d9 100644 --- a/test/copy.c++ +++ b/test/copy.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "test.h++" diff --git a/test/dead.c++ b/test/dead.c++ index c34443eb..2cfe818f 100644 --- a/test/dead.c++ +++ b/test/dead.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "test.h++" diff --git a/test/extra/doubtless_positioning.c++ b/test/extra/doubtless_positioning.c++ index 28c37492..ecc24daa 100644 --- a/test/extra/doubtless_positioning.c++ +++ b/test/extra/doubtless_positioning.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "mdbx.h++" diff --git a/test/extra/dupfix_multiple.c++ b/test/extra/dupfix_multiple.c++ index a8897314..a212984b 100644 --- a/test/extra/dupfix_multiple.c++ +++ b/test/extra/dupfix_multiple.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "mdbx.h++" diff --git a/test/extra/hex_base64_base58.c++ b/test/extra/hex_base64_base58.c++ index 7ae1fac4..972673e9 100644 --- a/test/extra/hex_base64_base58.c++ +++ b/test/extra/hex_base64_base58.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "mdbx.h++" diff --git a/test/extra/maindb_ordinal.c++ b/test/extra/maindb_ordinal.c++ index c766049f..9e6bd7ef 100644 --- a/test/extra/maindb_ordinal.c++ +++ b/test/extra/maindb_ordinal.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "mdbx.h++" diff --git a/test/extra/pcrf/pcrf_test.c b/test/extra/pcrf/pcrf_test.c index 1d023fcf..1a9d59e2 100644 --- a/test/extra/pcrf/pcrf_test.c +++ b/test/extra/pcrf/pcrf_test.c @@ -1,5 +1,5 @@ /* - * Copyright 2016-2024 Leonid Yuriev . + * Copyright 2016-2025 Leonid Yuriev . * Copyright 2015 Vladimir Romanov * , Yota Lab. * diff --git a/test/fork.c++ b/test/fork.c++ index d546cf89..c8d4558f 100644 --- a/test/fork.c++ +++ b/test/fork.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "test.h++" diff --git a/test/hill.c++ b/test/hill.c++ index ac6254e9..24ac25ce 100644 --- a/test/hill.c++ +++ b/test/hill.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "test.h++" diff --git a/test/jitter.c++ b/test/jitter.c++ index 3bf75100..fa190132 100644 --- a/test/jitter.c++ +++ b/test/jitter.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "test.h++" diff --git a/test/keygen.c++ b/test/keygen.c++ index 8ac4987d..f73ae55d 100644 --- a/test/keygen.c++ +++ b/test/keygen.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "test.h++" diff --git a/test/keygen.h++ b/test/keygen.h++ index 389cab4d..80739fb5 100644 --- a/test/keygen.h++ +++ b/test/keygen.h++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #pragma once diff --git a/test/log.c++ b/test/log.c++ index 9b952785..a1144d5c 100644 --- a/test/log.c++ +++ b/test/log.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "test.h++" diff --git a/test/log.h++ b/test/log.h++ index bb92783e..147d2869 100644 --- a/test/log.h++ +++ b/test/log.h++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #pragma once diff --git a/test/main.c++ b/test/main.c++ index 59c58729..bc2bee70 100644 --- a/test/main.c++ +++ b/test/main.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "test.h++" diff --git a/test/nested.c++ b/test/nested.c++ index dda73f92..e44b5413 100644 --- a/test/nested.c++ +++ b/test/nested.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "test.h++" diff --git a/test/osal-unix.c++ b/test/osal-unix.c++ index 3a6bf215..f440aeca 100644 --- a/test/osal-unix.c++ +++ b/test/osal-unix.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "test.h++" diff --git a/test/osal-windows.c++ b/test/osal-windows.c++ index 94f52b4a..b03d73ea 100644 --- a/test/osal-windows.c++ +++ b/test/osal-windows.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "test.h++" diff --git a/test/osal.h++ b/test/osal.h++ index 058f7078..85c5f0f3 100644 --- a/test/osal.h++ +++ b/test/osal.h++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #pragma once diff --git a/test/test.c++ b/test/test.c++ index 1d8db898..cf178f95 100644 --- a/test/test.c++ +++ b/test/test.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "test.h++" diff --git a/test/test.h++ b/test/test.h++ index 861429ff..3b0afa20 100644 --- a/test/test.h++ +++ b/test/test.h++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #pragma once diff --git a/test/try.c++ b/test/try.c++ index 4333a996..395e641e 100644 --- a/test/try.c++ +++ b/test/try.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "test.h++" diff --git a/test/ttl.c++ b/test/ttl.c++ index a4482e1f..9d23c0c6 100644 --- a/test/ttl.c++ +++ b/test/ttl.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "test.h++" diff --git a/test/utils.c++ b/test/utils.c++ index 442e4a21..e6da160b 100644 --- a/test/utils.c++ +++ b/test/utils.c++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #include "test.h++" diff --git a/test/utils.h++ b/test/utils.h++ index caa34049..131c75de 100644 --- a/test/utils.h++ +++ b/test/utils.h++ @@ -1,4 +1,4 @@ -/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2024 +/// \author Леонид Юрьев aka Leonid Yuriev \date 2015-2025 /// \copyright SPDX-License-Identifier: Apache-2.0 #pragma once From 4a0a32a54bbe06016c4e118afde3b2b5f6f97046 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Fri, 17 Jan 2025 22:41:26 +0300 Subject: [PATCH 04/90] =?UTF-8?q?mdbx-doc:=20=D0=B4=D0=BE=D0=B1=D0=B0?= =?UTF-8?q?=D0=B2=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=B2=20README=20=D1=81?= =?UTF-8?q?=D1=81=D1=8B=D0=BB=D0=BA=D0=B8=20=D0=BD=D0=B0=20=D0=B0=D1=80?= =?UTF-8?q?=D1=85=D0=B8=D0=B2=20=D1=81=D0=BE=D0=BE=D0=B1=D1=89=D0=B5=D0=BD?= =?UTF-8?q?=D0=B8=D0=B9=20=D1=82=D0=B5=D0=BB=D0=B5=D0=B3=D1=80=D0=B0=D0=BC?= =?UTF-8?q?=D0=BC-=D0=B3=D1=80=D1=83=D0=BF=D0=BF=D1=8B=202020-2024=20?= =?UTF-8?q?=D0=B3=D0=BE=D0=B4=D0=BE=D0=B2=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index df7e67c8..22ea871c 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,9 @@ > with [`C` API description](https://libmdbx.dqdkfa.ru/group__c__api.html) > and pay attention to the [`C++` API](https://gitflic.ru/project/erthink/libmdbx/blob?file=mdbx.h%2B%2B#line-num-1). -> Questions, feedback and suggestions are welcome to the [Telegram' group](https://t.me/libmdbx). +> Questions, feedback and suggestions are welcome to the [Telegram' group](https://t.me/libmdbx) (archive [1](https://libmdbx.dqdkfa.ru/tg-archive/messages1.html), +> [2](https://libmdbx.dqdkfa.ru/tg-archive/messages2.html), [3](https://libmdbx.dqdkfa.ru/tg-archive/messages3.html), [4](https://libmdbx.dqdkfa.ru/tg-archive/messages4.html), +> [5](https://libmdbx.dqdkfa.ru/tg-archive/messages5.html), [6](https://libmdbx.dqdkfa.ru/tg-archive/messages6.html), [7](https://libmdbx.dqdkfa.ru/tg-archive/messages7.html)). > See the [ChangeLog](https://gitflic.ru/project/erthink/libmdbx/blob?file=ChangeLog.md) for `NEWS` and latest updates. > Donations are welcome to ETH `0xD104d8f8B2dC312aaD74899F83EBf3EEBDC1EA3A`. From 03685aba5a45896864b1720007dacac70988dc6b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Sat, 18 Jan 2025 19:03:14 +0300 Subject: [PATCH 05/90] =?UTF-8?q?mdbx-doc:=20=D1=80=D0=B0=D0=B7=D0=B4?= =?UTF-8?q?=D0=B5=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=B0=D0=BA=D1=82=D1=83?= =?UTF-8?q?=D0=B0=D0=BB=D1=8C=D0=BD=D1=8B=D1=85=20=D0=B8=20=D1=83=D1=81?= =?UTF-8?q?=D1=82=D0=B0=D1=80=D0=B5=D0=B2=D1=88=D0=B8=D1=85/=D0=BD=D0=B5?= =?UTF-8?q?=D0=BF=D0=BE=D0=B4=D0=B4=D0=B5=D1=80=D0=B6=D0=B8=D0=B2=D0=B0?= =?UTF-8?q?=D0=B5=D0=BC=D1=8B=D1=85=20=D0=BF=D1=80=D0=B8=D0=B2=D1=8F=D0=B7?= =?UTF-8?q?=D0=BE=D0=BA=20=D0=B2=20README=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 22ea871c..ef65d7e4 100644 --- a/README.md +++ b/README.md @@ -641,19 +641,24 @@ Bindings | Runtime | Repo | Author | | ------- | ------ | ------ | +| Rust | [libmdbx-rs](https://github.com/vorot93/libmdbx-rs) | [Artem Vorotnikov](https://github.com/vorot93) | +| Python | [PyPi/libmdbx](https://pypi.org/project/libmdbx/) | [Lazymio](https://github.com/wtdcode) | +| Java | [mdbxjni](https://github.com/castortech/mdbxjni) | [Castor Technologies](https://castortech.com/) | +| Go | [mdbx-go](https://github.com/torquem-ch/mdbx-go) | [Alex Sharov](https://github.com/AskAlexSharov) | +| Ruby | [ruby-mdbx](https://rubygems.org/gems/mdbx/) | [Mahlon E. Smith](https://github.com/mahlonsmith) | + +##### Obsolete/Outdated/Unsupported: + +| Runtime | Repo | Author | +| ------- | ------ | ------ | +| .NET | [mdbx.NET](https://github.com/wangjia184/mdbx.NET) | [Jerry Wang](https://github.com/wangjia184) | | Scala | [mdbx4s](https://github.com/david-bouyssie/mdbx4s) | [David Bouyssié](https://github.com/david-bouyssie) | +| Rust | [mdbx](https://crates.io/crates/mdbx) | [gcxfd](https://github.com/gcxfd) | | Haskell | [libmdbx-hs](https://hackage.haskell.org/package/libmdbx) | [Francisco Vallarino](https://github.com/fjvallarino) | +| Lua | [lua-libmdbx](https://github.com/mah0x211/lua-libmdbx) | [Masatoshi Fukunaga](https://github.com/mah0x211) | | NodeJS, [Deno](https://deno.land/) | [lmdbx-js](https://github.com/kriszyp/lmdbx-js) | [Kris Zyp](https://github.com/kriszyp/) | NodeJS | [node-mdbx](https://www.npmjs.com/package/node-mdbx/) | [Сергей Федотов](mailto:sergey.fedotov@corp.mail.ru) | -| Ruby | [ruby-mdbx](https://rubygems.org/gems/mdbx/) | [Mahlon E. Smith](https://github.com/mahlonsmith) | -| Go | [mdbx-go](https://github.com/torquem-ch/mdbx-go) | [Alex Sharov](https://github.com/AskAlexSharov) | -| [Nim](https://en.wikipedia.org/wiki/Nim_(programming_language)) | [NimDBX](https://github.com/snej/nimdbx) | [Jens Alfke](https://github.com/snej) -| Lua | [lua-libmdbx](https://github.com/mah0x211/lua-libmdbx) | [Masatoshi Fukunaga](https://github.com/mah0x211) | -| Rust | [libmdbx-rs](https://github.com/vorot93/libmdbx-rs) | [Artem Vorotnikov](https://github.com/vorot93) | -| Rust | [mdbx](https://crates.io/crates/mdbx) | [gcxfd](https://github.com/gcxfd) | -| Java | [mdbxjni](https://github.com/castortech/mdbxjni) | [Castor Technologies](https://castortech.com/) | -| Python | [PyPi/libmdbx](https://pypi.org/project/libmdbx/) | [Lazymio](https://github.com/wtdcode) | -| .NET (obsolete) | [mdbx.NET](https://github.com/wangjia184/mdbx.NET) | [Jerry Wang](https://github.com/wangjia184) | +| Nim | [NimDBX](https://github.com/snej/nimdbx) | [Jens Alfke](https://github.com/snej) From c615e4d0a6b7ae3f333dbd1694086861666fcf3a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Sun, 19 Jan 2025 02:14:19 +0300 Subject: [PATCH 06/90] =?UTF-8?q?mdbx-doc:=20=D0=B4=D0=BE=D1=80=D0=B0?= =?UTF-8?q?=D0=B1=D0=BE=D1=82=D0=BA=D0=B0/=D0=B0=D0=BA=D1=82=D1=83=D0=B0?= =?UTF-8?q?=D0=BB=D0=B8=D0=B7=D0=B0=D1=86=D0=B8=D1=8F=20=D1=80=D0=B0=D0=B7?= =?UTF-8?q?=D0=B4=D0=B5=D0=BB=D0=B0=20"Restrictions=20&=20Caveats"=20(back?= =?UTF-8?q?port).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/_restrictions.md | 79 +++++++++++++++++++++++++++---------------- 1 file changed, 49 insertions(+), 30 deletions(-) diff --git a/docs/_restrictions.md b/docs/_restrictions.md index 7e905b5e..eb488a43 100644 --- a/docs/_restrictions.md +++ b/docs/_restrictions.md @@ -42,18 +42,33 @@ long-lived read transactions, and using the \ref MDBX_LIFORECLAIM mode which addresses subsequent performance degradation. The "next" version of libmdbx (aka \ref MithrilDB) will completely solve this. -- Avoid suspending a process with active transactions. These would then be - "long-lived" as above. +Nonetheless, situations that encourage lengthy read transactions while +intensively updating data should be avoided. For example, you should +avoid suspending/blocking processes/threads performing read +transactions, including during debugging, and use transaction parking if +necessary. -- Avoid aborting a process with an active read-only transaction in scenarios - with high rate of write transactions. The transaction becomes "long-lived" - as above until a check for stale readers is performed or the LCK-file is - reset, since the process may not remove it from the lockfile. This does - not apply to write transactions if the system clears stale writers, see - above. +You should also beware of aborting processes that perform reading +transactions. Despite the fact that libmdbx automatically checks and +cleans readers, as an a process aborting (especially with core dump) can +take a long time, and checking readers cannot be performed too often due +to performance degradation. + +This issue will be addressed in MithrlDB and one of libmdbx releases, +presumably in 2025. To do this, nonlinear GC recycling will be +implemented, without stopping garbage recycling on the old MVCC snapshot +used by a long read transaction. + +After the planned implementation, any long-term reading transaction will +still keep the used MVCC-snapshot (all the database pages forming it) +from being recycled, but it will allow all unused MVCC snapshots to be +recycled, both before and after the readable one. This will eliminate +one of the main architectural flaws inherited from LMDB and caused the +growth of a database in proportion to a volume of data changes made +concurrently with a long-running read transaction. -## Large data items and huge transactions +## Large data items MDBX allows you to store values up to 1 gigabyte in size, but this is not the main functionality for a key-value storage, but an additional @@ -71,6 +86,18 @@ and in the absence of a sufficient sequence of free pages, increase the DB file. Thus, for long values, MDBX provides maximum read performance at the expense of write performance. +Some aspects related to GC have been refined and improved in 2022 within +the first releases of the 0.12.x series. In particular the search for +free consecutive/adjacent pages through GC has been significantly +speeded, including acceleration using NOEN/SSE2/AVX2/AVX512 +instructions. + +This issue will be addressed in MithrlDB and refined within one of +0.15.x libmdbx releases, presumably at end of 2025. + + +### Huge transactions + A similar situation can be with huge transactions, in which a lot of database pages are retired. The retired pages should be put into GC as a list of page numbers for future reuse. But in huge transactions, such a @@ -80,20 +107,15 @@ you delete large amounts of information from the database in a single transaction, MDBX may need to increase the database file to save the list of pages to be retired. -Both of these issues will be addressed in MithrilDB. +This issue was fixed in 2022 within the first releases of the 0.12.x +series by `Big Foot` feature, which now is enabled by default. +See \ref MDBX_ENABLE_BIGFOOT build-time option. -#### Since v0.12.1 and later -Some aspects related to GC have been refined and improved, in particular: - - - The search for free consecutive/adjacent pages through GC has been - significantly speeded, including acceleration using - NOEN/SSE2/AVX2/AVX512 instructions. - - - The `Big Foot` feature which significantly reduces GC overhead for - processing large lists of retired pages from huge transactions. Now - libmdbx avoid creating large chunks of PNLs (page number lists) which - required a long sequences of free pages, aka large/overflow pages. Thus - avoiding searching, allocating and storing such sequences inside GC. +The `Big Foot` feature which significantly reduces GC overhead for +processing large lists of retired pages from huge transactions. Now +libmdbx avoid creating large chunks of PNLs (page number lists) which +required a long sequences of free pages, aka large/overflow pages. Thus +avoiding searching, allocating and storing such sequences inside GC. ## Space reservation @@ -135,9 +157,11 @@ On the other hand, MDBX allow calling \ref mdbx_env_close() in such cases to release resources, but no more and in general this is a wrong way. #### Since v0.13.1 and later -Начиная с версии 0.13.1 в API доступна функция \ref mdbx_env_resurrect_after_fork(), -которая позволяет пере-использовать в дочерних процессах уже открытую среду БД, -но строго без наследования транзакций от родительского процесса. + +Starting from the v0.13.1 release, the \ref mdbx_env_resurrect_after_work() +is available, which allows you to reuse an already open database +environment in child processes, but strictly without inheriting any +transactions from a parent process. ## Read-only mode @@ -149,9 +173,6 @@ to without-LCK mode on appropriate errors (`EROFS`, `EACCESS`, `EPERM`) if the read-only mode was requested by the \ref MDBX_RDONLY flag which is described below. -The "next" version of libmdbx (\ref MithrilDB) will solve this issue for the "many -readers without writer" case. - ## Troubleshooting the LCK-file 1. A broken LCK-file can cause sync issues, including appearance of @@ -166,8 +187,6 @@ readers without writer" case. multiple processes in a lock-free manner and any locking is unwise due to a large overhead. - The "next" version of libmdbx (\ref MithrilDB) will solve this issue. - \note Workaround: Just make all programs using the database close it; the LCK-file is always reset on first open. From 27a2166be7a202b53b061f4e7af38879b7e17df2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Tue, 21 Jan 2025 15:38:42 +0300 Subject: [PATCH 07/90] =?UTF-8?q?mdbx-doc:=20=D0=B8=D1=81=D0=BF=D1=80?= =?UTF-8?q?=D0=B0=D0=B2=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=BE=D1=80=D1=84?= =?UTF-8?q?=D0=BE=D0=B3=D1=80=D0=B0=D1=84=D0=B8=D0=B8/=D0=BE=D0=BF=D0=B5?= =?UTF-8?q?=D1=87=D0=B0=D1=82=D0=BA=D0=B8=20=D0=B2=20ChangeLog=20(backport?= =?UTF-8?q?).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ChangeLog.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ChangeLog.md b/ChangeLog.md index bc7ef66c..7b46fddc 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -2459,7 +2459,7 @@ Deprecated functions and flags: - Rework `MADV_DONTNEED` threshold. - Fix `mdbx_chk` utility for don't checking some numbers if walking on the B-tree was disabled. - Use page's mp_txnid for basic integrity checking. - - Add `MDBX_FORCE_ASSERTIONS` built-time option. + - Add `MDBX_FORCE_ASSERTIONS` build-time option. - Rework `MDBX_DBG_DUMP` to avoid performance degradation. - Rename `MDBX_NOSYNC` to `MDBX_SAFE_NOSYNC` for clarity. - Interpret `ERROR_ACCESS_DENIED` from `OpenProcess()` as 'process exists'. From a59c5f9316d1df8b344766b552ef607ada930d16 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Sat, 25 Jan 2025 19:18:18 +0300 Subject: [PATCH 08/90] =?UTF-8?q?mdbx:=20=D1=83=D0=BF=D1=80=D0=BE=D1=89?= =?UTF-8?q?=D0=B5=D0=BD=D0=B8=D0=B5=20`gcu=5Floose()`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/gc-put.c | 19 +++++-------------- 1 file changed, 5 insertions(+), 14 deletions(-) diff --git a/src/gc-put.c b/src/gc-put.c index b5231773..12bd22da 100644 --- a/src/gc-put.c +++ b/src/gc-put.c @@ -146,20 +146,12 @@ static inline void zeroize_reserved(const MDBX_env *env, MDBX_val pnl) { static int gcu_loose(MDBX_txn *txn, gcu_t *ctx) { tASSERT(txn, txn->tw.loose_count > 0); - /* Return loose page numbers to tw.repnl, - * though usually none are left at this point. + /* Return loose page numbers to tw.repnl, though usually none are left at this point. * The pages themselves remain in dirtylist. */ if (unlikely(!txn->tw.gc.retxl && txn->tw.gc.last_reclaimed < 1)) { - TRACE("%s: try allocate gc-slot for %zu loose-pages", dbg_prefix(ctx), txn->tw.loose_count); - int err = gc_alloc_ex(&ctx->cursor, 0, ALLOC_RESERVE).err; - if (err == MDBX_SUCCESS) { - TRACE("%s: retry since gc-slot for %zu loose-pages available", dbg_prefix(ctx), txn->tw.loose_count); - return MDBX_RESULT_TRUE; - } - - /* Put loose page numbers in tw.retired_pages, - * since unable to return ones to tw.repnl. */ - err = pnl_need(&txn->tw.retired_pages, txn->tw.loose_count); + /* Put loose page numbers in tw.retired_pages, since unable to return ones to tw.repnl. */ + TRACE("%s: merge %zu loose-pages into %s-pages", dbg_prefix(ctx), txn->tw.loose_count, "retired"); + int err = pnl_need(&txn->tw.retired_pages, txn->tw.loose_count); if (unlikely(err != MDBX_SUCCESS)) return err; for (page_t *lp = txn->tw.loose_pages; lp; lp = page_next(lp)) { @@ -167,9 +159,9 @@ static int gcu_loose(MDBX_txn *txn, gcu_t *ctx) { MDBX_ASAN_UNPOISON_MEMORY_REGION(&page_next(lp), sizeof(page_t *)); VALGRIND_MAKE_MEM_DEFINED(&page_next(lp), sizeof(page_t *)); } - TRACE("%s: append %zu loose-pages to retired-pages", dbg_prefix(ctx), txn->tw.loose_count); } else { /* Room for loose pages + temp PNL with same */ + TRACE("%s: merge %zu loose-pages into %s-pages", dbg_prefix(ctx), txn->tw.loose_count, "reclaimed"); int err = pnl_need(&txn->tw.repnl, 2 * txn->tw.loose_count + 2); if (unlikely(err != MDBX_SUCCESS)) return err; @@ -185,7 +177,6 @@ static int gcu_loose(MDBX_txn *txn, gcu_t *ctx) { MDBX_PNL_SETSIZE(loose, count); pnl_sort(loose, txn->geo.first_unallocated); pnl_merge(txn->tw.repnl, loose); - TRACE("%s: append %zu loose-pages to reclaimed-pages", dbg_prefix(ctx), txn->tw.loose_count); } /* filter-out list of dirty-pages from loose-pages */ From 11e1346f9df6f91b302cd8c34ac2e71f2c7bc594 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Sun, 26 Jan 2025 12:41:19 +0300 Subject: [PATCH 09/90] =?UTF-8?q?mdbx:=20=D0=B8=D1=81=D0=BF=D1=80=D0=B0?= =?UTF-8?q?=D0=B2=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=BE=D0=BF=D0=B5=D1=87?= =?UTF-8?q?=D0=B0=D1=82=D0=BA=D0=B8=20=D0=B2=20`cursor=5Ftouch()`=20(backp?= =?UTF-8?q?ort).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit При переделке курсоров было пропущено отрицание в условии, при оценке кол-ва страниц, которые могут потребоваться для выполнения операции. В текущем понимании ошибка не приводила к каким-либо проблемам, ибо оценка делает по верхней границе с существенным запасом, а в худшем случае это могло приводить к прерыванию транзакции из-за достижения ограничения на кол-во грязных страниц. --- src/cursor.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/cursor.c b/src/cursor.c index 43cf8e48..36ef13cf 100644 --- a/src/cursor.c +++ b/src/cursor.c @@ -145,7 +145,7 @@ __hot int cursor_touch(MDBX_cursor *const mc, const MDBX_val *key, const MDBX_va if (!cursor_is_gc(mc)) { need += txn->dbs[FREE_DBI].height + (size_t)3; /* 3) Named DBs also dirty the main DB */ - if (cursor_is_main(mc)) + if (!cursor_is_main(mc)) need += txn->dbs[MAIN_DBI].height + (size_t)3; } #if xMDBX_DEBUG_SPILLING != 2 From b59937adb8a76a65f0a373c50808ad3738d699dd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Sun, 26 Jan 2025 17:36:40 +0300 Subject: [PATCH 10/90] =?UTF-8?q?mdbx-doc:=20=D0=B8=D1=81=D0=BF=D1=80?= =?UTF-8?q?=D0=B0=D0=B2=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=BE=D0=BF=D0=B5?= =?UTF-8?q?=D1=87=D0=B0=D1=82=D0=BA=D0=B8=20=D0=B2=20=D1=83=D0=BF=D0=BE?= =?UTF-8?q?=D0=BC=D0=B8=D0=BD=D0=B0=D0=BD=D0=B8=D0=B8=20`mdbx=5Fenv=5Fresu?= =?UTF-8?q?rrect=5Fafter=5Ffork()`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/_restrictions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_restrictions.md b/docs/_restrictions.md index eb488a43..631f625f 100644 --- a/docs/_restrictions.md +++ b/docs/_restrictions.md @@ -158,7 +158,7 @@ release resources, but no more and in general this is a wrong way. #### Since v0.13.1 and later -Starting from the v0.13.1 release, the \ref mdbx_env_resurrect_after_work() +Starting from the v0.13.1 release, the \ref mdbx_env_resurrect_after_fork() is available, which allows you to reuse an already open database environment in child processes, but strictly without inheriting any transactions from a parent process. From 6d346d8630d1cbbcdaf07ea39d80af1c891b17d0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Sun, 26 Jan 2025 16:18:08 +0300 Subject: [PATCH 11/90] =?UTF-8?q?mdbx-cmake:=20=D0=BF=D0=BE=D0=B4=D0=B4?= =?UTF-8?q?=D0=B5=D1=80=D0=B6=D0=BA=D0=B0=20MacOS=20universal=20binaries?= =?UTF-8?q?=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Thank Alain Picard (Castor Technologies) so much for this patch and supporting the Java bindings! --- CMakeLists.txt | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/CMakeLists.txt b/CMakeLists.txt index 0996291b..7d0e3f43 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -213,6 +213,20 @@ if(DEFINED PROJECT_NAME AND NOT MDBX_FORCE_BUILD_AS_MAIN_PROJECT) else() set(SUBPROJECT OFF) set(NOT_SUBPROJECT ON) + + # Setup Apple stuff which should be set prior to the first project() or enable_language() + if(APPLE) + # Enable universal binaries for macOS (target arm64 and x86_64) + if(NOT DEFINED CMAKE_OSX_ARCHITECTURES) + set(CMAKE_OSX_ARCHITECTURES "arm64;x86_64") + endif() + + # Set the minimum macOS deployment target if not already defined + if(NOT DEFINED CMAKE_OSX_DEPLOYMENT_TARGET) + set(CMAKE_OSX_DEPLOYMENT_TARGET "13.0") + endif() + endif() + project(libmdbx C) if(NOT MDBX_AMALGAMATED_SOURCE AND NOT DEFINED BUILD_TESTING) set(BUILD_TESTING ON) From 21630ea115690a5cb39cfa921f9d199271a08102 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Mon, 27 Jan 2025 11:04:51 +0300 Subject: [PATCH 12/90] =?UTF-8?q?mdbx:=20=D1=83=D1=81=D1=82=D1=80=D0=B0?= =?UTF-8?q?=D0=BD=D0=B5=D0=BD=D0=B8=D0=B5=20=D1=80=D0=B5=D0=B3=D1=80=D0=B5?= =?UTF-8?q?=D1=81=D1=81=D0=B0=20=D0=B2=D0=B5=D1=80=D0=BE=D1=8F=D1=82=D0=BD?= =?UTF-8?q?=D0=BE=D1=81=D1=82=D0=B8=20SIGSEGV=20=D0=BF=D1=80=D0=B8=20?= =?UTF-8?q?=D0=B2=D1=8B=D1=82=D0=B5=D1=81=D0=BD=D0=B5=D0=BD=D0=B8=D0=B8/sp?= =?UTF-8?q?illing=20=D1=81=D1=82=D1=80=D0=B0=D0=BD=D0=B8=D1=86=20(backport?= =?UTF-8?q?).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Ошибка внесена коммитом `a6f7d74a32a3cbcc310916a624a31302dbebfa07` от 2024-03-07 и присутствует в выпусках v0.13.1, v0.13.2, v0.13.3. Проблема оставалась незамеченной из-за специфических условий и низкой вероятности проявления. Суть ошибки: - функция cursor_touch() подготавливает стек страниц курсора к внесению изменений, при этом все страницы в стеке (от корневой до листовой в текущей позиции курсора) должны стать доступными для модификации. - микрооптимизация добавленная коммитом пропускала обход стека, если корневая страница уже доступна для модификации, но это допустимо/корректно только при отсутствии в стеке вытесненных/spilled страниц. - если же складывалась ситуация когда в стека была вытесненная некорневая страница, то она так и оставалась недоступной для записи и при попытке её изменения возникал SIGSEGV. --- src/cursor.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/cursor.c b/src/cursor.c index 36ef13cf..f24677f2 100644 --- a/src/cursor.c +++ b/src/cursor.c @@ -168,7 +168,7 @@ __hot int cursor_touch(MDBX_cursor *const mc, const MDBX_val *key, const MDBX_va return err; } - if (likely(mc->top >= 0) && !is_modifable(mc->txn, mc->pg[mc->top])) { + if (likely(is_pointed(mc)) && ((mc->txn->flags & MDBX_TXN_SPILLS) || !is_modifable(mc->txn, mc->pg[mc->top]))) { const int8_t top = mc->top; mc->top = 0; do { From b46d2def80e0fc8924b280c7d1e1fe00de25b391 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Mon, 27 Jan 2025 12:12:38 +0300 Subject: [PATCH 13/90] =?UTF-8?q?mdbx:=20=D0=B4=D0=BE=D0=BF=D0=BE=D0=BB?= =?UTF-8?q?=D0=BD=D0=B5=D0=BD=D0=B8=D0=B5=20ChangeLog.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ChangeLog.md | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-) diff --git a/ChangeLog.md b/ChangeLog.md index 7b46fddc..8b436622 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -5,11 +5,34 @@ English version [by liar Google](https://libmdbx-dqdkfa-ru.translate.goog/md__ch and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/libmdbx.dqdkfa.ru/md__change_log.html). -## v0.13.4 в процессе накопления изменений +## v0.13.4 "Sigma Boy" запланирован на 2025-02-14 Поддерживающий выпуск стабильной ветки с исправлением обнаруженных ошибок и устранением недочётов. -Предполагаемая дата выпуска между 2025-02-11 и 2025-03-13. +Благодарности: + + - [Алексею Костюку (aka Keller)](https://t.me/keller18306) за сообщения об ошибках и недочетах. + - [Erigon](https://docs.erigon.tech/) за спонсорство. + - [Alain Picard](https://github.com/castortech) for support [Java bindings](https://github.com/castortech/mdbxjni) and MacOS universal binaries patch for CMake build scenario. + - [Alex Sharov](https://github.com/AskAlexSharov) за сообщение об ошибках и тестирование. + +Новое: + + - Поддержка MacOS universal binaries при сборке посредством CMake. + +Исправления: + + - Устранён регресс допускающий SIGSEGV в операциях обновления после вытеснения/spilling страниц в больших транзакциях. + Ошибка присутствует в выпусках v0.13.1, v0.13.2, v0.13.3 и оставалась незамеченной из-за специфических условий и низкой вероятности проявления. + Более подробная информация в описании коммита `a060057e480aa5196c7d5c00ea441c962e3b952d`. + + - Исправлена опечатка в документации в упоминании `mdbx_env_resurrect_after_fork()`. + + - Исправлена опечатка в условном операторе внутри `cursor_touch()`. + При переделке курсоров было пропущено отрицание в условии, при оценке кол-ва страниц, которые могут потребоваться для выполнения операции. + В текущем понимании ошибка не приводила к каким-либо проблемам, ибо оценка делает по верхней границе с существенным запасом, а в худшем + случае это могло приводить к прерыванию транзакции из-за достижения ограничения на кол-во грязных страниц. + -------------------------------------------------------------------------------- From 24f2b9b0992b4273989438f5841c147b1030e547 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Mon, 3 Feb 2025 18:46:01 +0300 Subject: [PATCH 14/90] =?UTF-8?q?mdbx-conan:=20=D0=B8=D1=81=D0=BF=D1=80?= =?UTF-8?q?=D0=B0=D0=B2=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=BE=D0=BF=D0=B5?= =?UTF-8?q?=D1=87=D0=B0=D1=82=D0=BA=D0=B8=20=D0=B2=20=D0=B8=D0=BC=D0=B5?= =?UTF-8?q?=D0=BD=D0=B8=20=D0=BF=D0=B5=D1=80=D0=B5=D0=BC=D0=B5=D0=BD=D0=BD?= =?UTF-8?q?=D0=BE=D0=B9=20`version=5Fjson=5Fpathname`=20=D0=B2=20verbose-?= =?UTF-8?q?=D0=B2=D1=8B=D0=B2=D0=BE=D0=B4=D0=B5.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Спасибо Виктору Логунову (https://t.me/vl_username) за сообщение о проблеме. --- ChangeLog.md | 1 + conanfile.py | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/ChangeLog.md b/ChangeLog.md index 8b436622..6b23a9df 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -15,6 +15,7 @@ and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/libmdbx - [Erigon](https://docs.erigon.tech/) за спонсорство. - [Alain Picard](https://github.com/castortech) for support [Java bindings](https://github.com/castortech/mdbxjni) and MacOS universal binaries patch for CMake build scenario. - [Alex Sharov](https://github.com/AskAlexSharov) за сообщение об ошибках и тестирование. + - [Виктору Логунову](https://t.me/vl_username) за сообщение об опечатки в имени переменной в Conan-рецепте. Новое: diff --git a/conanfile.py b/conanfile.py index 79241260..b36e7b44 100644 --- a/conanfile.py +++ b/conanfile.py @@ -228,7 +228,7 @@ class libmdbx(ConanFile): if os.path.exists(version_json_pathname): self.version = json.load( open(version_json_pathname, encoding='utf-8'))['semver'] - version_from = "'" + version_jsonpath_name + "'" + version_from = "'" + version_json_pathname + "'" else: self.version = self.fetch_versioninfo_from_git()['semver'] version_from = 'Git' From 79572b485064c5ec7528b15ffd44773c21796f5e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Tue, 11 Feb 2025 14:03:13 +0300 Subject: [PATCH 15/90] =?UTF-8?q?mdbx:=20=D0=BA=D0=BE=D1=80=D1=80=D0=B5?= =?UTF-8?q?=D0=BA=D1=82=D0=B8=D1=80=D0=BE=D0=B2=D0=BA=D0=B0=20=D0=B8=D0=B7?= =?UTF-8?q?=D0=BB=D0=B8=D1=88=D0=BD=D0=B5=20=D1=81=D1=82=D1=80=D0=BE=D0=B3?= =?UTF-8?q?=D0=BE=D0=B3=D0=BE=20=D1=83=D1=81=D0=BB=D0=BE=D0=B2=D0=B8=D1=8F?= =?UTF-8?q?=20=D0=B2=20assert-=D0=BF=D1=80=D0=BE=D0=B2=D0=B5=D1=80=D0=BA?= =?UTF-8?q?=D0=B5=20=D0=B2=D0=BD=D1=83=D1=82=D1=80=D0=B8=20`recalculate=5F?= =?UTF-8?q?subpage=5Fthresholds()`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/page-ops.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/page-ops.c b/src/page-ops.c index 43e6a79a..11916749 100644 --- a/src/page-ops.c +++ b/src/page-ops.c @@ -726,7 +726,7 @@ void recalculate_subpage_thresholds(MDBX_env *env) { env->subpage_reserve_prereq = page_space(env); else if (env->subpage_reserve_prereq < env->subpage_room_threshold + env->subpage_reserve_limit) env->subpage_reserve_prereq = env->subpage_room_threshold + env->subpage_reserve_limit; - eASSERT(env, env->subpage_reserve_prereq > env->subpage_room_threshold + env->subpage_reserve_limit); + eASSERT(env, env->subpage_reserve_prereq >= env->subpage_room_threshold + env->subpage_reserve_limit); } size_t page_subleaf2_reserve(const MDBX_env *env, size_t host_page_room, size_t subpage_len, size_t item_len) { From 75122b311d3672409823d2477564689ec0be082c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Fri, 14 Feb 2025 12:56:12 +0300 Subject: [PATCH 16/90] =?UTF-8?q?mdbx:=20=D0=B2=D1=8B=D0=BF=D1=83=D1=81?= =?UTF-8?q?=D0=BA=200.13.4=20"Sigma=20Boy".?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Поддерживающий выпуск стабильной ветки с исправлением обнаруженных ошибок и устранением недочётов. За перечнем доработок и изменений обращайтесь к [ChangeLog](https://libmdbx.dqdkfa.ru/md__change_log.html). git diff' stat: 139 files changed, 391 insertions(+), 208 deletions(-) Signed-off-by: Леонид Юрьев (Leonid Yuriev) --- ChangeLog.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/ChangeLog.md b/ChangeLog.md index 6b23a9df..f1e01426 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -5,7 +5,7 @@ English version [by liar Google](https://libmdbx-dqdkfa-ru.translate.goog/md__ch and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/libmdbx.dqdkfa.ru/md__change_log.html). -## v0.13.4 "Sigma Boy" запланирован на 2025-02-14 +## v0.13.4 "Sigma Boy" от 2025-02-14 Поддерживающий выпуск стабильной ветки с исправлением обнаруженных ошибок и устранением недочётов. @@ -30,10 +30,15 @@ and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/libmdbx - Исправлена опечатка в документации в упоминании `mdbx_env_resurrect_after_fork()`. - Исправлена опечатка в условном операторе внутри `cursor_touch()`. - При переделке курсоров было пропущено отрицание в условии, при оценке кол-ва страниц, которые могут потребоваться для выполнения операции. + При переделке курсоров было пропущено отрицание в условии, при оценке количества страниц, которые могут потребоваться для выполнения операции. В текущем понимании ошибка не приводила к каким-либо проблемам, ибо оценка делает по верхней границе с существенным запасом, а в худшем случае это могло приводить к прерыванию транзакции из-за достижения ограничения на кол-во грязных страниц. + - Корректировка излишне строгого условия в assert-проверке внутри `recalculate_subpage_thresholds()`. + Ошибка могла проявляться только в отладочных сборках при выставлении определенной комбинации предельных значений опций `MDBX_opt_subpage_limit`, + `MDBX_opt_subpage_room_threshold`, `MDBX_opt_subpage_reserve_prereq`, `MDBX_opt_subpage_reserve_limit`. + + - Исправление опечатки в Conan-рецепте в коде протокольно-отладочного вывода в имени переменной `version_json_pathname`. -------------------------------------------------------------------------------- From fcdd2e2db3cf2a511547abd200118e5099d1f660 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Fri, 14 Feb 2025 15:23:28 +0300 Subject: [PATCH 17/90] =?UTF-8?q?mdbx:=20=D0=BE=D0=B1=D0=BD=D0=BE=D0=B2?= =?UTF-8?q?=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20ChangeLog.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ChangeLog.md | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/ChangeLog.md b/ChangeLog.md index f1e01426..f9df0108 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -4,6 +4,17 @@ ChangeLog English version [by liar Google](https://libmdbx-dqdkfa-ru.translate.goog/md__change_log.html?_x_tr_sl=ru&_x_tr_tl=en) and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/libmdbx.dqdkfa.ru/md__change_log.html). +## v0.13.5 в процессе накопления изменений + +Поддерживающий выпуск стабильной ветки с исправлением обнаруженных ошибок и устранением недочётов. + +Благодарности: + + - [Erigon](https://docs.erigon.tech/) за спонсорство. + + +-------------------------------------------------------------------------------- + ## v0.13.4 "Sigma Boy" от 2025-02-14 @@ -11,8 +22,8 @@ and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/libmdbx Благодарности: - - [Алексею Костюку (aka Keller)](https://t.me/keller18306) за сообщения об ошибках и недочетах. - [Erigon](https://docs.erigon.tech/) за спонсорство. + - [Алексею Костюку (aka Keller)](https://t.me/keller18306) за сообщения об ошибках и недочетах. - [Alain Picard](https://github.com/castortech) for support [Java bindings](https://github.com/castortech/mdbxjni) and MacOS universal binaries patch for CMake build scenario. - [Alex Sharov](https://github.com/AskAlexSharov) за сообщение об ошибках и тестирование. - [Виктору Логунову](https://t.me/vl_username) за сообщение об опечатки в имени переменной в Conan-рецепте. @@ -25,7 +36,7 @@ and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/libmdbx - Устранён регресс допускающий SIGSEGV в операциях обновления после вытеснения/spilling страниц в больших транзакциях. Ошибка присутствует в выпусках v0.13.1, v0.13.2, v0.13.3 и оставалась незамеченной из-за специфических условий и низкой вероятности проявления. - Более подробная информация в описании коммита `a060057e480aa5196c7d5c00ea441c962e3b952d`. + Более подробная информация в описании коммита `21630ea115690a5cb39cfa921f9d199271a08102`. - Исправлена опечатка в документации в упоминании `mdbx_env_resurrect_after_fork()`. From aa2ff20fafd8318762d98c9997e4a11b5f2f1ce8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Fri, 14 Feb 2025 21:36:38 +0300 Subject: [PATCH 18/90] =?UTF-8?q?mdbx:=20=D0=BE=D0=B1=D0=BD=D0=BE=D0=B2?= =?UTF-8?q?=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=BF=D0=B0=D1=82=D1=87=D0=B0?= =?UTF-8?q?=20=D0=B4=D0=BB=D1=8F=20=D1=81=D1=82=D0=B0=D1=80=D1=8B=D1=85=20?= =?UTF-8?q?=D0=B2=D0=B5=D1=80=D1=81=D0=B8=D0=B9=20buildroot=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...libmdbx-new-package-library-database.patch | 31 +++++++++---------- 1 file changed, 15 insertions(+), 16 deletions(-) diff --git a/packages/buildroot/0001-package-libmdbx-new-package-library-database.patch b/packages/buildroot/0001-package-libmdbx-new-package-library-database.patch index 9c7afff4..2ccb2846 100644 --- a/packages/buildroot/0001-package-libmdbx-new-package-library-database.patch +++ b/packages/buildroot/0001-package-libmdbx-new-package-library-database.patch @@ -1,7 +1,7 @@ -From 0ba6ba5e6d6311213a21f033729e18826729230a Mon Sep 17 00:00:00 2001 +From 49256dcd050fd0ee67860b7bc544dabe088d08e9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= -Date: Tue, 14 Jan 2025 12:57:03 +0300 +Date: Fri, 14 Feb 2025 21:34:25 +0300 Subject: [PATCH] package/libmdbx: new package (library/database). MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 @@ -13,11 +13,9 @@ This patch adds libmdbx: focused on creating unique lightweight solutions. - libmdbx surpasses the legendary LMDB (Lightning Memory-Mapped Database) in terms of reliability, features and performance. - - more information at https://gitflic.ru/project/erthink/libmdbx + - more information at https://libmdbx.dqdkfa.ru -The 0.13.13 "Korolev" is stable release of _libmdbx_ branch with new superior features -on the birthday and in memory of Sergei Korolev who was the lead Soviet rocket -engineer and spacecraft designer. +The 0.13.4 "Sigma Boy" is stable release of _libmdbx_ branch with new superior features. The complete ChangeLog: https://gitflic.ru/project/erthink/libmdbx/blob?file=ChangeLog.md @@ -26,9 +24,9 @@ Signed-off-by: Леонид Юрьев (Leonid Yuriev) DEVELOPERS | 3 +++ package/Config.in | 1 + package/libmdbx/Config.in | 45 ++++++++++++++++++++++++++++++++++++ - package/libmdbx/libmdbx.hash | 5 ++++ + package/libmdbx/libmdbx.hash | 6 +++++ package/libmdbx/libmdbx.mk | 42 +++++++++++++++++++++++++++++++++ - 5 files changed, 96 insertions(+) + 5 files changed, 97 insertions(+) create mode 100644 package/libmdbx/Config.in create mode 100644 package/libmdbx/libmdbx.hash create mode 100644 package/libmdbx/libmdbx.mk @@ -112,18 +110,19 @@ index 0000000000..a9a4ac45c5 + !BR2_TOOLCHAIN_GCC_AT_LEAST_4_4 diff --git a/package/libmdbx/libmdbx.hash b/package/libmdbx/libmdbx.hash new file mode 100644 -index 0000000000..1c91aa2f90 +index 0000000000..202937e7be --- /dev/null +++ b/package/libmdbx/libmdbx.hash -@@ -0,0 +1,5 @@ +@@ -0,0 +1,6 @@ +# Hashes from: https://libmdbx.dqdkfa.ru/release/SHA256SUMS -+sha256 2e42505f1ceb57945569db3c1a5db9b5216d8f72da7c75c240ff81196f8e9a0b libmdbx-amalgamated-0.13.3.tar.xz ++sha256 86df30ca2231c9b3ad71424bb829dca9041947f5539d4295030c653d4982c1be libmdbx-amalgamated-0.13.4.tar.xz + +# Locally calculated +sha256 0d542e0c8804e39aa7f37eb00da5a762149dc682d7829451287e11b938e94594 LICENSE ++sha256 699a62986b6c8d31124646dffd4b15872c7d3bc5eecea5994edb1f5195df49d1 NOTICE diff --git a/package/libmdbx/libmdbx.mk b/package/libmdbx/libmdbx.mk new file mode 100644 -index 0000000000..b42ab629fe +index 0000000000..a8a6f3dbdf --- /dev/null +++ b/package/libmdbx/libmdbx.mk @@ -0,0 +1,42 @@ @@ -133,12 +132,12 @@ index 0000000000..b42ab629fe +# +################################################################################ + -+LIBMDBX_VERSION = 0.13.3 ++LIBMDBX_VERSION = 0.13.4 +LIBMDBX_SOURCE = libmdbx-amalgamated-$(LIBMDBX_VERSION).tar.xz +LIBMDBX_SITE = https://libmdbx.dqdkfa.ru/release +LIBMDBX_SUPPORTS_IN_SOURCE_BUILD = NO -+LIBMDBX_LICENSE = OLDAP-2.8 -+LIBMDBX_LICENSE_FILES = LICENSE ++LIBMDBX_LICENSE = Apache-2.0 ++LIBMDBX_LICENSE_FILES = LICENSE NOTICE +LIBMDBX_REDISTRIBUTE = YES +LIBMDBX_STRIP_COMPONENTS = 0 +LIBMDBX_INSTALL_STAGING = YES @@ -170,5 +169,5 @@ index 0000000000..b42ab629fe + +$(eval $(cmake-package)) -- -2.48.0 +2.48.1 From 822213f75d8ce977f4c0ca6bae69786fb22c4d4e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Sat, 15 Feb 2025 15:46:44 +0300 Subject: [PATCH 19/90] =?UTF-8?q?mdbx:=20=D0=B8=D0=BD=D1=84=D0=BE=D1=80?= =?UTF-8?q?=D0=BC=D0=B0=D1=86=D0=B8=D1=8F=20=D0=BE=20=D1=81=D1=82=D0=B0?= =?UTF-8?q?=D1=82=D1=83=D1=81=D0=B5=20Github=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 46 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index ef65d7e4..f37d56f9 100644 --- a/README.md +++ b/README.md @@ -65,7 +65,51 @@ Historically, _libmdbx_ is a deeply revised and extended descendant of the amazi [Lightning Memory-Mapped Database](https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database). _libmdbx_ inherits all benefits from _LMDB_, but resolves some issues and adds [a set of improvements](#improvements-beyond-lmdb). -### MithrilDB and Future +## Github + +### на Русском (мой родной язык) + +Весной 2022, без каких-либо предупреждений или пояснений, администрация +Github удалила мой аккаунт и все проекты. Через несколько месяцев, без +какого-либо моего участия или уведомления, проекты были +восстановлены/открыты в статусе "public read-only archive" из какой-то +неполноценной резервной копии. Эти действия Github я расцениваю как +злонамеренный саботаж, а сам сервис Github считаю навсегда утратившим +какое-либо доверие. + +Вследствие произошедшего, никогда и ни при каких условиях, я не буду +размещать на Github первоисточники (aka origins) моих проектов, либо +как-либо полагаться на инфраструктуру Github. + +Тем не менее, понимая что пользователям моих проектов удобнее получать к +ним доступ именно на Github, я не хочу ограничивать их свободу или +создавать неудобство, и поэтому размещаю на Github зеркала (aka mirrors) +репозиториев моих проектов. При этом ещё раз акцентирую внимание, что +это только зеркала, которые могут быть заморожены, заблокированы или +удалены в любой момент, как это уже было в 2022. + +### in English + +In the spring of 2022, without any warnings or explanations, the Github +administration deleted my account and all projects. A few months later, +without any involvement or notification from me, the projects were +restored/opened in the "public read-only archive" status from some kind +of incomplete backup. I regard these actions of Github as malicious +sabotage, and I consider the Github service itself to have lost any +trust forever. + +As a result of what has happened, I will never, under any circumstances, +post the primary sources (aka origins) of my projects on Github, or rely +in any way on the Github infrastructure. + +Nevertheless, realizing that it is more convenient for users of my +projects to access them on Github, I do not want to restrict their +freedom or create inconvenience, and therefore I place mirrors of my +project repositories on Github. At the same time, I would like to +emphasize once again that these are only mirrors that can be frozen, +blocked or deleted at any time, as was the case in 2022. + +## MithrilDB and Future From 80de77b1eec98ec92ccaef2ed6d3cb059b751f90 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Sun, 16 Feb 2025 16:52:53 +0300 Subject: [PATCH 20/90] =?UTF-8?q?mdbx-doc:=20=D0=BE=D0=BF=D0=B5=D1=87?= =?UTF-8?q?=D0=B0=D1=82=D0=BA=D0=B8=20=D0=B2=20README=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index f37d56f9..1bad0676 100644 --- a/README.md +++ b/README.md @@ -279,7 +279,7 @@ which is also (mostly) applicable to _libmdbx_ with minor clarification: - a database could shared by multiple processes, i.e. no multi-process issues; - no issues with moving a cursor(s) after the deletion; - _libmdbx_ provides zero-overhead database compactification, so a database file could be shrinked/truncated in particular cases; - - excluding dist I/O time _libmdbx_ could be -3 times faster than BoltDB and up to 10-100K times faster than both BoltDB and LMDB in particular extreme cases; + - excluding disk I/O time _libmdbx_ could be ≈3 times faster than BoltDB and up to 10-100K times faster than both BoltDB and LMDB in particular extreme cases; - _libmdbx_ provides more features compared to BoltDB and/or LMDB. From 0ef0f49e2e3b33c5b89e5834ed0e12ce3edd6629 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Wed, 19 Feb 2025 23:38:15 +0300 Subject: [PATCH 21/90] =?UTF-8?q?mdbx:=20=D1=83=D1=81=D1=82=D1=80=D0=B0?= =?UTF-8?q?=D0=BD=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=B8=D0=B7=D0=BB=D0=B8=D1=88?= =?UTF-8?q?=D0=BD=D0=B5=D0=B3=D0=BE=20=D0=BF=D1=80=D0=B5=D0=B4=D1=83=D0=BF?= =?UTF-8?q?=D1=80=D0=B5=D0=B6=D0=B4=D0=B5=D0=BD=D0=B8=D1=8F=20=D0=BF=D1=80?= =?UTF-8?q?=D0=B8=20=D1=81=D0=BC=D0=B5=D0=BD=D0=B5=20=D1=80=D0=B0=D0=B7?= =?UTF-8?q?=D0=BC=D0=B5=D1=80=D0=B0=20=D0=91=D0=94=20=D0=B2=D0=BE=20=D0=B2?= =?UTF-8?q?=D1=80=D0=B5=D0=BC=D1=8F=20=D0=BE=D1=82=D0=BA=D1=80=D1=8B=D1=82?= =?UTF-8?q?=D0=B8=D1=8F=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Изменение геометрии (увеличение размера) больших БД может быть не возможно после их открытия вследствие системных ограничений (отсутствия свободного адресного пространства). Поэтому API предусматривает возможность запросить изменение геометрии/размера БД перед её открытием. В этом сценарии ранее могло выдаваться лишнее/ненужное предупреждение о несоответствии файла БД новому размеру. Теперь этот недостаток исправлен. Спасибо Илье Михееву (Erigon) за сообщение об этом недочете. --- ChangeLog.md | 7 +++++++ src/dxb.c | 16 +++++++++++----- 2 files changed, 18 insertions(+), 5 deletions(-) diff --git a/ChangeLog.md b/ChangeLog.md index f9df0108..a1859394 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -11,7 +11,14 @@ and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/libmdbx Благодарности: - [Erigon](https://docs.erigon.tech/) за спонсорство. + - [Илье Михееву](https://t.me/IlyaMkhv) за сообщение о лишнем/ненужном предупреждении несоответствия файла БД новому размеру. +Исправления: + + - Устранение лишнего/ненужного предупреждения в сценарии изменения размера БД посредством вызова `mdbx_env_set_geometry()` до её открытия. + API предусматривает возможность запросить изменение геометрии/размера БД перед её открытием, чтобы избежать как лишних накладных расходов, + так и потенциальных ошибок из-за нехватки адресного пространства. В этом сценарии ранее могло выдаваться лишнее/ненужное предупреждение + о несоответствии файла БД новому размеру. Теперь этот недостаток исправлен. -------------------------------------------------------------------------------- diff --git a/src/dxb.c b/src/dxb.c index aa222025..22379d33 100644 --- a/src/dxb.c +++ b/src/dxb.c @@ -567,6 +567,7 @@ __cold int dxb_setup(MDBX_env *env, const int lck_rc, const mdbx_mode_t mode_bit return err; } + bool new_size_from_user = false; const size_t used_bytes = pgno2bytes(env, header.geometry.first_unallocated); const size_t used_aligned2os_bytes = ceil_powerof2(used_bytes, globals.sys_pagesize); if ((env->flags & MDBX_RDONLY) /* readonly */ @@ -601,6 +602,8 @@ __cold int dxb_setup(MDBX_env *env, const int lck_rc, const mdbx_mode_t mode_bit /* pre-shrink if enabled */ env->geo_in_bytes.now = used_bytes + env->geo_in_bytes.shrink - used_bytes % env->geo_in_bytes.shrink; + /* сейчас БД еще не открыта, поэтому этот вызов не изменит геометрию, но проверит и скорректирует параметры + * с учетом реального размера страницы. */ err = mdbx_env_set_geometry(env, env->geo_in_bytes.lower, env->geo_in_bytes.now, env->geo_in_bytes.upper, env->geo_in_bytes.grow, env->geo_in_bytes.shrink, header.pagesize); if (unlikely(err != MDBX_SUCCESS)) { @@ -608,8 +611,10 @@ __cold int dxb_setup(MDBX_env *env, const int lck_rc, const mdbx_mode_t mode_bit return (err == MDBX_EINVAL) ? MDBX_INCOMPATIBLE : err; } - /* update meta fields */ - header.geometry.now = bytes2pgno(env, env->geo_in_bytes.now); + /* altering fields to match geometry given from user */ + new_size_from_user = header.geometry.now != bytes2pgno(env, env->geo_in_bytes.now); + if (new_size_from_user) + header.geometry.now = bytes2pgno(env, env->geo_in_bytes.now); header.geometry.lower = bytes2pgno(env, env->geo_in_bytes.lower); header.geometry.upper = bytes2pgno(env, env->geo_in_bytes.upper); header.geometry.grow_pv = pages2pv(bytes2pgno(env, env->geo_in_bytes.grow)); @@ -646,9 +651,10 @@ __cold int dxb_setup(MDBX_env *env, const int lck_rc, const mdbx_mode_t mode_bit env->geo_in_bytes.now, bytes2pgno(env, env->geo_in_bytes.now), filesize_before, bytes2pgno(env, (size_t)filesize_before)); } else { - WARNING("filesize mismatch (expect %" PRIuSIZE "b/%" PRIaPGNO "p, have %" PRIu64 "b/%" PRIaPGNO "p)", - env->geo_in_bytes.now, bytes2pgno(env, env->geo_in_bytes.now), filesize_before, - bytes2pgno(env, (size_t)filesize_before)); + if (!new_size_from_user) + WARNING("filesize mismatch (expect %" PRIuSIZE "b/%" PRIaPGNO "p, have %" PRIu64 "b/%" PRIaPGNO "p)", + env->geo_in_bytes.now, bytes2pgno(env, env->geo_in_bytes.now), filesize_before, + bytes2pgno(env, (size_t)filesize_before)); if (filesize_before < used_bytes) { ERROR("last-page beyond end-of-file (last %" PRIaPGNO ", have %" PRIaPGNO ")", header.geometry.first_unallocated, bytes2pgno(env, (size_t)filesize_before)); From c585fcd61356d0669c72c589d84f3929c7c4ab0a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Feb 2025 23:48:48 +0300 Subject: [PATCH 22/90] =?UTF-8?q?mdbx-tests:=20=D1=80=D0=B0=D1=81=D1=88?= =?UTF-8?q?=D0=B8=D1=80=D0=B5=D0=BD=D0=B8=D0=B5=20`extra/open`=20(backport?= =?UTF-8?q?).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- test/extra/open.c++ | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/test/extra/open.c++ b/test/extra/open.c++ index 55b58c8c..459ca708 100644 --- a/test/extra/open.c++ +++ b/test/extra/open.c++ @@ -44,6 +44,13 @@ int main(int argc, const char *argv[]) { txn2.commit(); } + { + mdbx::env::operate_parameters operateParameters(100, 10); + mdbx::env_managed::create_parameters createParameters; + createParameters.geometry.make_dynamic(21 * mdbx::env::geometry::MiB, mdbx::env::geometry::GiB / 2); + mdbx::env_managed env(path, createParameters, operateParameters); + } + mdbx::env::operate_parameters operateParameters(100, 10); mdbx::env_managed::create_parameters createParameters; createParameters.geometry.make_dynamic(21 * mdbx::env::geometry::MiB, 84 * mdbx::env::geometry::MiB); From 22c6763d57b1f045b224a2a9a7083d0db5d52f19 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Sat, 1 Mar 2025 22:52:19 +0300 Subject: [PATCH 23/90] =?UTF-8?q?mdbx-tests:=20=D1=83=D0=B4=D0=B0=D0=BB?= =?UTF-8?q?=D0=B5=D0=BD=D0=B8=D0=B5=20=D1=82=D0=B5=D1=81=D1=82=D0=BE=D0=B2?= =?UTF-8?q?=D0=BE=D0=B9=20=D0=91=D0=94=20=D0=BF=D0=B5=D1=80=D0=B5=D0=B4=20?= =?UTF-8?q?=D0=BD=D0=B0=D1=87=D0=B0=D0=BB=D0=BE=D0=BC=20=D1=82=D0=B5=D1=81?= =?UTF-8?q?=D1=82=D0=B0=20=D0=B2=20`extra/dupfix=5Faddodd`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- test/extra/dupfix_addodd.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/test/extra/dupfix_addodd.c b/test/extra/dupfix_addodd.c index b2e6aeea..7d976f9b 100644 --- a/test/extra/dupfix_addodd.c +++ b/test/extra/dupfix_addodd.c @@ -25,6 +25,9 @@ int main() { MDBX_val key, data; MDBX_txn *txn = NULL; + const char *db_filename = "./example-db"; + mdbx_env_delete(db_filename, MDBX_ENV_JUST_DELETE); + rc = mdbx_env_create(&env); if (rc != MDBX_SUCCESS) { fprintf(stderr, "mdbx_env_create: (%d) %s\n", rc, mdbx_strerror(rc)); @@ -37,7 +40,7 @@ int main() { exit(EXIT_FAILURE); } - rc = mdbx_env_open(env, "./example-db", MDBX_NOSUBDIR | MDBX_LIFORECLAIM, 0664); + rc = mdbx_env_open(env, db_filename, MDBX_NOSUBDIR | MDBX_LIFORECLAIM, 0664); if (rc != MDBX_SUCCESS) { fprintf(stderr, "mdbx_env_open: (%d) %s\n", rc, mdbx_strerror(rc)); exit(EXIT_FAILURE); From 23600241e18a8bbe202d6e93cc486925f6aebc9b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Sat, 1 Mar 2025 22:52:44 +0300 Subject: [PATCH 24/90] =?UTF-8?q?mdbx:=20=D1=83=D0=BC=D0=B5=D0=BD=D1=8C?= =?UTF-8?q?=D1=88=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=B2=2016=20=D1=80=D0=B0?= =?UTF-8?q?=D0=B7=20=D0=BF=D1=80=D0=B5=D0=B4=D0=BB=D0=B0=D0=B3=D0=B0=D0=B5?= =?UTF-8?q?=D0=BC=D0=BE=D0=B3=D0=BE=20=D1=80=D0=B0=D0=B7=D0=BC=D0=B5=D1=80?= =?UTF-8?q?=D0=B0=20=D0=91=D0=94=20=D0=B4=D0=BB=D1=8F=20=D1=83=D1=81=D1=82?= =?UTF-8?q?=D1=80=D0=B0=D0=BD=D0=B5=D0=BD=D0=B8=D1=8F=20=D0=BF=D1=80=D0=BE?= =?UTF-8?q?=D0=B1=D0=BB=D0=B5=D0=BC=20Valgrind/ASAN=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/api-env.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/api-env.c b/src/api-env.c index 9a6be7b3..6a9a719b 100644 --- a/src/api-env.c +++ b/src/api-env.c @@ -11,6 +11,12 @@ __cold static intptr_t reasonable_db_maxsize(void) { /* the 32-bit limit is good enough for fallback */ return cached_result = MAX_MAPSIZE32; +#if defined(__SANITIZE_ADDRESS__) + total_ram_pages >>= 4; +#endif /* __SANITIZE_ADDRESS__ */ + if (RUNNING_ON_VALGRIND) + total_ram_pages >>= 4; + if (unlikely((size_t)total_ram_pages * 2 > MAX_MAPSIZE / (size_t)pagesize)) return cached_result = MAX_MAPSIZE; assert(MAX_MAPSIZE >= (size_t)(total_ram_pages * pagesize * 2)); From c712147eeb9c6c0017bcdac294d091f037877c23 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Sat, 1 Mar 2025 22:58:12 +0300 Subject: [PATCH 25/90] =?UTF-8?q?mdbx:=20=D0=B8=D1=81=D0=BF=D1=80=D0=B0?= =?UTF-8?q?=D0=B2=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=BE=D0=BF=D0=BB=D0=BE?= =?UTF-8?q?=D1=88=D0=BD=D0=BE=D1=81=D1=82=D0=B8=20=D0=B2=20=D1=81=D0=BF?= =?UTF-8?q?=D0=B5=D1=86=D0=B8=D1=84=D0=B8=D0=BA=D0=B0=D1=86=D0=B8=D0=B8=20?= =?UTF-8?q?=D1=84=D0=BE=D1=80=D0=BC=D0=B0=D1=82=D0=B0=20=D0=BF=D1=80=D0=B8?= =?UTF-8?q?=20=D0=BB=D0=BE=D0=B3=D0=B8=D1=80=D0=BE=D0=B2=D0=B0=D0=BD=D0=B8?= =?UTF-8?q?=D0=B8=20=D0=B8=D0=BC=D0=B5=D0=BD=20=D1=82=D0=B0=D0=B1=D0=BB?= =?UTF-8?q?=D0=B8=D1=86=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/audit.c | 2 +- src/table.c | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/src/audit.c b/src/audit.c index 094722ff..0fa86625 100644 --- a/src/audit.c +++ b/src/audit.c @@ -78,7 +78,7 @@ __cold static int audit_ex_locked(MDBX_txn *txn, size_t retired_stored, bool don if (db) ctx.used += audit_db_used(db); else if (dbi_state(txn, dbi)) - WARNING("audit %s@%" PRIaTXN ": unable account dbi %zd / \"%*s\", state 0x%02x", txn->parent ? "nested-" : "", + WARNING("audit %s@%" PRIaTXN ": unable account dbi %zd / \"%.*s\", state 0x%02x", txn->parent ? "nested-" : "", txn->txnid, dbi, (int)env->kvs[dbi].name.iov_len, (const char *)env->kvs[dbi].name.iov_base, dbi_state(txn, dbi)); } diff --git a/src/table.c b/src/table.c index 690ab9ff..faa44da1 100644 --- a/src/table.c +++ b/src/table.c @@ -38,7 +38,7 @@ int tbl_fetch(MDBX_txn *txn, size_t dbi) { rc = tree_search(&couple.outer, &kvx->name, 0); if (unlikely(rc != MDBX_SUCCESS)) { bailout: - NOTICE("dbi %zu refs to inaccessible table `%*s` for txn %" PRIaTXN " (err %d)", dbi, (int)kvx->name.iov_len, + NOTICE("dbi %zu refs to inaccessible table `%.*s` for txn %" PRIaTXN " (err %d)", dbi, (int)kvx->name.iov_len, (const char *)kvx->name.iov_base, txn->txnid, rc); return (rc == MDBX_NOTFOUND) ? MDBX_BAD_DBI : rc; } @@ -50,7 +50,7 @@ int tbl_fetch(MDBX_txn *txn, size_t dbi) { goto bailout; } if (unlikely((node_flags(nsr.node) & (N_DUP | N_TREE)) != N_TREE)) { - NOTICE("dbi %zu refs to not a named table `%*s` for txn %" PRIaTXN " (%s)", dbi, (int)kvx->name.iov_len, + NOTICE("dbi %zu refs to not a named table `%.*s` for txn %" PRIaTXN " (%s)", dbi, (int)kvx->name.iov_len, (const char *)kvx->name.iov_base, txn->txnid, "wrong flags"); return MDBX_INCOMPATIBLE; /* not a named DB */ } @@ -60,7 +60,7 @@ int tbl_fetch(MDBX_txn *txn, size_t dbi) { return rc; if (unlikely(data.iov_len != sizeof(tree_t))) { - NOTICE("dbi %zu refs to not a named table `%*s` for txn %" PRIaTXN " (%s)", dbi, (int)kvx->name.iov_len, + NOTICE("dbi %zu refs to not a named table `%.*s` for txn %" PRIaTXN " (%s)", dbi, (int)kvx->name.iov_len, (const char *)kvx->name.iov_base, txn->txnid, "wrong rec-size"); return MDBX_INCOMPATIBLE; /* not a named DB */ } @@ -70,7 +70,7 @@ int tbl_fetch(MDBX_txn *txn, size_t dbi) { * have dropped and recreated the DB with other flags. */ tree_t *const db = &txn->dbs[dbi]; if (unlikely((db->flags & DB_PERSISTENT_FLAGS) != flags)) { - NOTICE("dbi %zu refs to the re-created table `%*s` for txn %" PRIaTXN + NOTICE("dbi %zu refs to the re-created table `%.*s` for txn %" PRIaTXN " with different flags (present 0x%X != wanna 0x%X)", dbi, (int)kvx->name.iov_len, (const char *)kvx->name.iov_base, txn->txnid, db->flags & DB_PERSISTENT_FLAGS, flags); From 1ec13c63abd418f20c1584911fb7e2558caf5a9d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Sat, 1 Mar 2025 23:11:44 +0300 Subject: [PATCH 26/90] =?UTF-8?q?mdbx:=20=D1=83=D1=81=D1=82=D1=80=D0=B0?= =?UTF-8?q?=D0=BD=D0=B5=D0=BD=D0=B8=D0=B5=20=D1=81=D0=B1=D0=BE=D1=8F=20?= =?UTF-8?q?=D0=B0=D1=83=D0=B4=D0=B8=D1=82=D0=B0=20=D1=82=D0=B0=D0=B1=D0=BB?= =?UTF-8?q?=D0=B8=D1=86=20=D0=BF=D1=80=D0=B8=20=D0=B8=D0=BD=D0=B2=D0=B0?= =?UTF-8?q?=D0=BB=D0=B8=D0=B4=D0=B0=D1=86=D0=B8=D0=B8=20dbi-=D1=85=D0=B5?= =?UTF-8?q?=D0=BD=D0=B4=D0=BB=D0=B0=20=D0=B2=D1=81=D0=BB=D0=B5=D0=B4=D1=81?= =?UTF-8?q?=D1=82=D0=B2=D0=B8=D0=B5=20=D0=BE=D1=82=D0=BC=D0=B5=D0=BD=D1=8B?= =?UTF-8?q?=20=D0=B2=D0=BB=D0=BE=D0=B6=D0=B5=D0=BD=D0=BD=D0=BE=D0=B9=20?= =?UTF-8?q?=D1=82=D1=80=D0=B0=D0=BD=D0=B7=D0=B0=D0=BA=D1=86=D0=B8=D0=B8=20?= =?UTF-8?q?(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/dbi.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/dbi.c b/src/dbi.c index 13d5a3eb..691911e9 100644 --- a/src/dbi.c +++ b/src/dbi.c @@ -683,7 +683,7 @@ __cold const tree_t *dbi_dig(const MDBX_txn *txn, const size_t dbi, tree_t *fall case DBI_OLDEN: return dig->dbs + dbi; case 0: - return nullptr; + return fallback; case DBI_VALID | DBI_STALE: case DBI_OLDEN | DBI_STALE: break; From 90635e72481ff0ecbee44ec6e68fcd0af42f5127 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Sun, 2 Mar 2025 01:05:32 +0300 Subject: [PATCH 27/90] =?UTF-8?q?mdbx:=20=D0=B8=D1=81=D0=BF=D1=80=D0=B0?= =?UTF-8?q?=D0=B2=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=BD=D0=B0=D1=81=D0=BB?= =?UTF-8?q?=D0=B5=D0=B4=D0=BE=D0=B2=D0=B0=D0=BD=D0=B8=D1=8F=20dbi-=D1=85?= =?UTF-8?q?=D0=B5=D0=BD=D0=B4=D0=BB=D0=B0=20=D0=BE=D1=82=D0=BA=D1=80=D1=8B?= =?UTF-8?q?=D1=82=D0=BE=D0=B3=D0=BE=20=D0=B2=20=D0=B4=D0=BE=D1=87=D0=B5?= =?UTF-8?q?=D1=80=D0=BD=D0=B5=D0=B9=20=D1=82=D1=80=D0=B0=D0=BD=D0=B7=D0=B0?= =?UTF-8?q?=D0=BA=D1=86=D0=B8=D0=B8=20=D0=B1=D0=B5=D0=B7=20=D0=B8=D0=B7?= =?UTF-8?q?=D0=BC=D0=B5=D0=BD=D0=B5=D0=BD=D0=B8=D1=8F=20=D0=B4=D0=B0=D0=BD?= =?UTF-8?q?=D0=BD=D1=8B=D1=85.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/api-txn.c | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/src/api-txn.c b/src/api-txn.c index 61b994bd..1f24c930 100644 --- a/src/api-txn.c +++ b/src/api-txn.c @@ -453,20 +453,29 @@ int mdbx_txn_commit_ex(MDBX_txn *txn, MDBX_commit_latency *latency) { eASSERT(env, dpl_check(txn)); if (txn->tw.dirtylist->length == 0 && !(txn->flags & MDBX_TXN_DIRTY) && parent->n_dbi == txn->n_dbi) { - TXN_FOREACH_DBI_ALL(txn, i) { - tASSERT(txn, (txn->dbi_state[i] & DBI_DIRTY) == 0); - if ((txn->dbi_state[i] & DBI_STALE) && !(parent->dbi_state[i] & DBI_STALE)) - tASSERT(txn, memcmp(&parent->dbs[i], &txn->dbs[i], sizeof(tree_t)) == 0); - } + /* fast completion of pure nested transaction */ + VERBOSE("fast-complete pure nested txn %" PRIaTXN, txn->txnid); tASSERT(txn, memcmp(&parent->geo, &txn->geo, sizeof(parent->geo)) == 0); tASSERT(txn, memcmp(&parent->canary, &txn->canary, sizeof(parent->canary)) == 0); tASSERT(txn, !txn->tw.spilled.list || MDBX_PNL_GETSIZE(txn->tw.spilled.list) == 0); tASSERT(txn, txn->tw.loose_count == 0); - /* fast completion of pure nested transaction */ - VERBOSE("fast-complete pure nested txn %" PRIaTXN, txn->txnid); - end_mode = TXN_END_PURE_COMMIT | TXN_END_SLOT | TXN_END_FREE; + /* Update parent's DBs array */ + eASSERT(env, parent->n_dbi == txn->n_dbi); + TXN_FOREACH_DBI_ALL(txn, dbi) { + tASSERT(txn, (txn->dbi_state[dbi] & (DBI_CREAT | DBI_DIRTY)) == 0); + if (txn->dbi_state[dbi] & DBI_FRESH) { + parent->dbs[dbi] = txn->dbs[dbi]; + /* preserve parent's status */ + const uint8_t state = txn->dbi_state[dbi] | DBI_FRESH; + DEBUG("dbi %zu dbi-state %s 0x%02x -> 0x%02x", dbi, (parent->dbi_state[dbi] != state) ? "update" : "still", + parent->dbi_state[dbi], state); + parent->dbi_state[dbi] = state; + } + } + txn_done_cursors(txn, true); + end_mode = TXN_END_PURE_COMMIT | TXN_END_SLOT | TXN_END_FREE | TXN_END_EOTDONE; goto done; } From 1792bdc7635116f6539cd445c5848fa5d0ca6048 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Sun, 2 Mar 2025 00:42:55 +0300 Subject: [PATCH 28/90] =?UTF-8?q?mdbx-tests:=20=D1=80=D0=B0=D1=81=D1=88?= =?UTF-8?q?=D0=B8=D1=80=D0=B5=D0=BD=D0=B8=D0=B5=20`extra/dbi`=20(backport)?= =?UTF-8?q?.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- test/extra/dbi.c++ | 51 ++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 8 deletions(-) diff --git a/test/extra/dbi.c++ b/test/extra/dbi.c++ index 9ed37c45..fe238415 100644 --- a/test/extra/dbi.c++ +++ b/test/extra/dbi.c++ @@ -20,18 +20,53 @@ int main(int argc, const char *argv[]) { mdbx::path db_filename = "test-dbi"; mdbx::env::remove(db_filename); - mdbx::env::operate_parameters operateParameters(100, 10); + mdbx::env::operate_parameters operateParameters(100, 10, mdbx::env::nested_transactions); mdbx::env_managed::create_parameters createParameters; { - mdbx::env_managed env2(db_filename, createParameters, operateParameters); - mdbx::txn_managed txn2 = env2.start_write(false); - /* mdbx::map_handle testHandle2 = */ txn2.create_map("fap1", mdbx::key_mode::reverse, mdbx::value_mode::single); - txn2.commit(); + mdbx::env_managed env(db_filename, createParameters, operateParameters); + mdbx::txn_managed txn = env.start_write(); + /* mdbx::map_handle dbi = */ txn.create_map("fap1", mdbx::key_mode::reverse, mdbx::value_mode::single); + txn.commit(); } + mdbx::env_managed env(db_filename, createParameters, operateParameters); - mdbx::txn_managed txn = env.start_write(false); - /* mdbx::map_handle testHandle = */ txn.create_map("fap1", mdbx::key_mode::usual, mdbx::value_mode::single); - txn.commit(); + { + // проверяем доступность в родительской транзакции хендла открытого в дочерней транзакции после коммита + mdbx::txn_managed txn = env.start_write(); + mdbx::txn_managed nested = txn.start_nested(); + mdbx::map_handle dbi = nested.open_map_accede("fap1"); + nested.commit(); + MDBX_MAYBE_UNUSED auto stat = txn.get_map_stat(dbi); + txn.commit(); + env.close_map(dbi); + } + + { + // проверяем НЕ доступность в родительской транзакции хендла открытого в дочерней транзакции после прерывания + mdbx::txn_managed txn = env.start_write(); + mdbx::txn_managed nested = txn.start_nested(); + mdbx::map_handle dbi = nested.open_map_accede("fap1"); + nested.abort(); + MDBX_stat stat; + int err = mdbx_dbi_stat(txn, dbi, &stat, sizeof(stat)); + if (err != MDBX_BAD_DBI) { + std::cerr << "unexpected result err-code " << err; + return EXIT_FAILURE; + } + txn.commit(); + } + + { + // снова проверяем что таблица открывается и хендл доступень в родительской транзакции после коммита открывшей его + // дочерней + mdbx::txn_managed txn = env.start_write(); + mdbx::txn_managed nested = txn.start_nested(); + mdbx::map_handle dbi = nested.open_map_accede("fap1"); + nested.commit(); + MDBX_MAYBE_UNUSED auto stat = txn.get_map_stat(dbi); + txn.commit(); + env.close_map(dbi); + } std::cout << "OK\n"; return EXIT_SUCCESS; From 9277daa1850ab22d933be080cf9d171617becd3a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Sun, 2 Mar 2025 11:44:10 +0300 Subject: [PATCH 29/90] =?UTF-8?q?mdbx:=20=D0=B1=D0=BE=D0=BB=D0=B5=D0=B5=20?= =?UTF-8?q?=D0=BF=D0=BE=D0=BB=D0=BD=D0=B0=D1=8F=20=D0=BE=D1=87=D0=B8=D1=81?= =?UTF-8?q?=D1=82=D0=BA=D0=B0=20=D0=BA=D1=83=D1=80=D1=81=D0=BE=D1=80=D0=BE?= =?UTF-8?q?=D0=B2=20=D0=BF=D1=80=D0=B8=20=D0=B7=D0=B0=D0=BA=D1=80=D1=8B?= =?UTF-8?q?=D1=82=D0=B8=D0=B8/=D0=BE=D1=82=D0=BA=D0=BB=D1=8E=D1=87=D0=B5?= =?UTF-8?q?=D0=BD=D0=B8=D0=B8=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/api-cursor.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/api-cursor.c b/src/api-cursor.c index a10e09ef..f58d37f5 100644 --- a/src/api-cursor.c +++ b/src/api-cursor.c @@ -121,8 +121,8 @@ int mdbx_cursor_unbind(MDBX_cursor *mc) { } mc->next = mc; } + be_poor(mc); mc->signature = cur_signature_ready4dispose; - mc->flags = 0; return MDBX_SUCCESS; } @@ -171,6 +171,7 @@ void mdbx_cursor_close(MDBX_cursor *mc) { /* Cursor closed before nested txn ends */ tASSERT(txn, mc->signature == cur_signature_live); ENSURE(txn->env, check_txn_rw(txn, 0) == MDBX_SUCCESS); + be_poor(mc); mc->signature = cur_signature_wait4eot; } } @@ -218,8 +219,8 @@ int mdbx_txn_release_all_cursors(const MDBX_txn *txn, bool unbind) { txn->cursors[i] = mc->next; mc->next = mc; if (unbind) { + be_poor(mc); mc->signature = cur_signature_ready4dispose; - mc->flags = 0; } else { mc->signature = 0; osal_free(mc); From d313008d8260302854da5c94c9ffea11a93e5e58 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Sun, 2 Mar 2025 11:45:28 +0300 Subject: [PATCH 30/90] =?UTF-8?q?mdbx:=20=D0=B4=D0=BE=D0=BF=D0=BE=D0=BB?= =?UTF-8?q?=D0=BD=D0=B8=D1=82=D0=B5=D0=BB=D1=8C=D0=BD=D1=8B=D0=B5=20=D0=BF?= =?UTF-8?q?=D1=80=D0=BE=D0=B2=D0=B5=D1=80=D0=BA=D0=B8=20=D1=81=D0=B8=D0=B3?= =?UTF-8?q?=D0=BD=D0=B0=D1=82=D1=83=D1=80=20=D0=BA=D1=83=D1=80=D1=81=D0=BE?= =?UTF-8?q?=D1=80=D0=BE=D0=B2=20=D0=BF=D1=80=D0=B8=20=D0=B8=D1=82=D0=B5?= =?UTF-8?q?=D1=80=D0=B0=D1=86=D0=B8=D0=B8=20=D1=81=D0=B2=D1=8F=D0=B7=D0=B0?= =?UTF-8?q?=D0=BD=D0=BD=D1=8B=D1=85=20=D1=81=D0=BF=D0=B8=D1=81=D0=BA=D0=BE?= =?UTF-8?q?=D0=B2=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/api-cursor.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/src/api-cursor.c b/src/api-cursor.c index f58d37f5..b925ac8b 100644 --- a/src/api-cursor.c +++ b/src/api-cursor.c @@ -114,8 +114,12 @@ int mdbx_cursor_unbind(MDBX_cursor *mc) { cASSERT(mc, dbi < mc->txn->n_dbi); if (dbi < mc->txn->n_dbi) { MDBX_cursor **prev = &mc->txn->cursors[dbi]; - while (*prev && *prev != mc) + while (*prev) { + ENSURE(mc->txn->env, (*prev)->signature == cur_signature_live || (*prev)->signature == cur_signature_wait4eot); + if (*prev == mc) + break; prev = &(*prev)->next; + } cASSERT(mc, *prev == mc); *prev = mc->next; } @@ -158,8 +162,12 @@ void mdbx_cursor_close(MDBX_cursor *mc) { tASSERT(txn, dbi < txn->n_dbi); if (dbi < txn->n_dbi) { MDBX_cursor **prev = &txn->cursors[dbi]; - while (*prev && *prev != mc) + while (*prev) { + ENSURE(txn->env, (*prev)->signature == cur_signature_live || (*prev)->signature == cur_signature_wait4eot); + if (*prev == mc) + break; prev = &(*prev)->next; + } tASSERT(txn, *prev == mc); *prev = mc->next; } From 5e714ed946b42be7d61d847cfde3e2cf7db70164 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Mon, 3 Mar 2025 01:48:36 +0300 Subject: [PATCH 31/90] =?UTF-8?q?mdbx:=20=D0=BF=D0=B5=D1=80=D0=B5=D0=B4?= =?UTF-8?q?=D0=B5=D0=BB=D0=BA=D0=B0=20`env=5Fowned=5Fwrtxn()`=20=D0=B8=20?= =?UTF-8?q?=D0=BC=D0=B5=D1=81=D1=82=20=D0=B5=D1=91=20=D0=B2=D1=8B=D0=B7?= =?UTF-8?q?=D0=BE=D0=B2=D0=B0=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Цель в том чтобы избавить от коллизии блокировки возникающей внутри dxb_sanitize_tail() при использовании Valgrind/ASAN, а также упросить код. --- src/api-cold.c | 2 +- src/api-env.c | 27 ++++++++++++--------------- src/api-opts.c | 2 +- src/dxb.c | 2 +- src/env.c | 21 +++++++++++++-------- src/proto.h | 2 +- 6 files changed, 29 insertions(+), 27 deletions(-) diff --git a/src/api-cold.c b/src/api-cold.c index 1d124852..8a8c8588 100644 --- a/src/api-cold.c +++ b/src/api-cold.c @@ -342,7 +342,7 @@ __cold int mdbx_env_set_flags(MDBX_env *env, MDBX_env_flags_t flags, bool onoff) if (unlikely(env->flags & MDBX_RDONLY)) return LOG_IFERR(MDBX_EACCESS); - const bool lock_needed = (env->flags & ENV_ACTIVE) && !env_txn0_owned(env); + const bool lock_needed = (env->flags & ENV_ACTIVE) && !env_owned_wrtxn(env); bool should_unlock = false; if (lock_needed) { rc = lck_txn_lock(env, false); diff --git a/src/api-env.c b/src/api-env.c index 6a9a719b..2bfbdeb2 100644 --- a/src/api-env.c +++ b/src/api-env.c @@ -958,8 +958,7 @@ __cold int mdbx_env_set_geometry(MDBX_env *env, intptr_t size_lower, intptr_t si if (unlikely(rc != MDBX_SUCCESS)) return LOG_IFERR(rc); - const bool txn0_owned = env->basal_txn && env_txn0_owned(env); - const bool inside_txn = txn0_owned && env->txn; + MDBX_txn *const txn_owned = env_owned_wrtxn(env); bool should_unlock = false; #if MDBX_DEBUG && 0 /* минимальные шаги для проверки/отладки уже не нужны */ @@ -975,7 +974,7 @@ __cold int mdbx_env_set_geometry(MDBX_env *env, intptr_t size_lower, intptr_t si if (unlikely(env->flags & MDBX_RDONLY)) return LOG_IFERR(MDBX_EACCESS); - if (!txn0_owned) { + if (!txn_owned) { int err = lck_txn_lock(env, false); if (unlikely(err != MDBX_SUCCESS)) return LOG_IFERR(err); @@ -989,8 +988,7 @@ __cold int mdbx_env_set_geometry(MDBX_env *env, intptr_t size_lower, intptr_t si /* get untouched params from current TXN or DB */ if (pagesize <= 0 || pagesize >= INT_MAX) pagesize = env->ps; - const geo_t *const geo = - inside_txn ? &env->txn->geo : &meta_recent(env, &env->basal_txn->tw.troika).ptr_c->geometry; + const geo_t *const geo = env->txn ? &env->txn->geo : &meta_recent(env, &env->basal_txn->tw.troika).ptr_c->geometry; if (size_lower < 0) size_lower = pgno2bytes(env, geo->lower); if (size_now < 0) @@ -1015,7 +1013,7 @@ __cold int mdbx_env_set_geometry(MDBX_env *env, intptr_t size_lower, intptr_t si size_now = usedbytes; } else { /* env NOT yet mapped */ - if (unlikely(inside_txn)) + if (unlikely(env->txn)) return LOG_IFERR(MDBX_PANIC); /* is requested some auto-value for pagesize ? */ @@ -1204,8 +1202,7 @@ __cold int mdbx_env_set_geometry(MDBX_env *env, intptr_t size_lower, intptr_t si ENSURE(env, pagesize == (intptr_t)env->ps); meta_t meta; memset(&meta, 0, sizeof(meta)); - if (!inside_txn) { - eASSERT(env, should_unlock); + if (!env->txn) { const meta_ptr_t head = meta_recent(env, &env->basal_txn->tw.troika); uint64_t timestamp = 0; @@ -1295,7 +1292,7 @@ __cold int mdbx_env_set_geometry(MDBX_env *env, intptr_t size_lower, intptr_t si if (unlikely(rc != MDBX_SUCCESS)) goto bailout; } - if (inside_txn) { + if (env->txn) { env->txn->geo = new_geo; env->txn->flags |= MDBX_TXN_DIRTY; } else { @@ -1420,17 +1417,17 @@ __cold int mdbx_env_stat_ex(const MDBX_env *env, const MDBX_txn *txn, MDBX_stat if (unlikely(err != MDBX_SUCCESS)) return LOG_IFERR(err); - if (env->txn && env_txn0_owned(env)) + MDBX_txn *txn_owned = env_owned_wrtxn(env); + if (txn_owned) /* inside write-txn */ - return LOG_IFERR(stat_acc(env->txn, dest, bytes)); + return LOG_IFERR(stat_acc(txn_owned, dest, bytes)); - MDBX_txn *tmp_txn; - err = mdbx_txn_begin((MDBX_env *)env, nullptr, MDBX_TXN_RDONLY, &tmp_txn); + err = mdbx_txn_begin((MDBX_env *)env, nullptr, MDBX_TXN_RDONLY, &txn_owned); if (unlikely(err != MDBX_SUCCESS)) return LOG_IFERR(err); - const int rc = stat_acc(tmp_txn, dest, bytes); - err = mdbx_txn_abort(tmp_txn); + const int rc = stat_acc(txn_owned, dest, bytes); + err = mdbx_txn_abort(txn_owned); if (unlikely(err != MDBX_SUCCESS)) return LOG_IFERR(err); return LOG_IFERR(rc); diff --git a/src/api-opts.c b/src/api-opts.c index 2465864a..4f13d875 100644 --- a/src/api-opts.c +++ b/src/api-opts.c @@ -180,7 +180,7 @@ __cold int mdbx_env_set_option(MDBX_env *env, const MDBX_option_t option, uint64 if (unlikely(err != MDBX_SUCCESS)) return LOG_IFERR(err); - const bool lock_needed = ((env->flags & ENV_ACTIVE) && env->basal_txn && !env_txn0_owned(env)); + const bool lock_needed = ((env->flags & ENV_ACTIVE) && env->basal_txn && !env_owned_wrtxn(env)); bool should_unlock = false; switch (option) { case MDBX_opt_sync_bytes: diff --git a/src/dxb.c b/src/dxb.c index 22379d33..2b3cc03a 100644 --- a/src/dxb.c +++ b/src/dxb.c @@ -368,7 +368,7 @@ void dxb_sanitize_tail(MDBX_env *env, MDBX_txn *txn) { if (env->pid != osal_getpid()) { /* resurrect after fork */ return; - } else if (env->txn && env_txn0_owned(env)) { + } else if (env_owned_wrtxn(env)) { /* inside write-txn */ last = meta_recent(env, &env->basal_txn->tw.troika).ptr_v->geometry.first_unallocated; } else if (env->flags & MDBX_RDONLY) { diff --git a/src/env.c b/src/env.c index 525ed8ec..e9b33646 100644 --- a/src/env.c +++ b/src/env.c @@ -3,9 +3,14 @@ #include "internals.h" -bool env_txn0_owned(const MDBX_env *env) { - return (env->flags & MDBX_NOSTICKYTHREADS) ? (env->basal_txn->owner != 0) - : (env->basal_txn->owner == osal_thread_self()); +MDBX_txn *env_owned_wrtxn(const MDBX_env *env) { + if (likely(env->basal_txn)) { + const bool is_owned = (env->flags & MDBX_NOSTICKYTHREADS) ? (env->basal_txn->owner != 0) + : (env->basal_txn->owner == osal_thread_self()); + if (is_owned) + return env->txn ? env->txn : env->basal_txn; + } + return nullptr; } int env_page_auxbuffer(MDBX_env *env) { @@ -60,7 +65,7 @@ __cold int env_sync(MDBX_env *env, bool force, bool nonblock) { if (unlikely(env->flags & MDBX_RDONLY)) return MDBX_EACCESS; - const bool txn0_owned = env_txn0_owned(env); + MDBX_txn *const txn_owned = env_owned_wrtxn(env); bool should_unlock = false; int rc = MDBX_RESULT_TRUE /* means "nothing to sync" */; @@ -71,7 +76,7 @@ retry:; goto bailout; } - const troika_t troika = (txn0_owned | should_unlock) ? env->basal_txn->tw.troika : meta_tap(env); + const troika_t troika = (txn_owned || should_unlock) ? env->basal_txn->tw.troika : meta_tap(env); const meta_ptr_t head = meta_recent(env, &troika); const uint64_t unsynced_pages = atomic_load64(&env->lck->unsynced_pages, mo_Relaxed); if (unsynced_pages == 0) { @@ -104,7 +109,7 @@ retry:; osal_monotime() - eoos_timestamp >= autosync_period)) flags &= MDBX_WRITEMAP /* clear flags for full steady sync */; - if (!txn0_owned) { + if (!txn_owned) { if (!should_unlock) { #if MDBX_ENABLE_PGOP_STAT unsigned wops = 0; @@ -163,8 +168,8 @@ retry:; flags |= txn_shrink_allowed; } - eASSERT(env, txn0_owned || should_unlock); - eASSERT(env, !txn0_owned || (flags & txn_shrink_allowed) == 0); + eASSERT(env, txn_owned || should_unlock); + eASSERT(env, !txn_owned || (flags & txn_shrink_allowed) == 0); if (!head.is_steady && unlikely(env->stuck_meta >= 0) && troika.recent != (uint8_t)env->stuck_meta) { NOTICE("skip %s since wagering meta-page (%u) is mispatch the recent " diff --git a/src/proto.h b/src/proto.h index 842cbf62..f0a6657f 100644 --- a/src/proto.h +++ b/src/proto.h @@ -76,7 +76,7 @@ MDBX_INTERNAL int env_open(MDBX_env *env, mdbx_mode_t mode); MDBX_INTERNAL int env_info(const MDBX_env *env, const MDBX_txn *txn, MDBX_envinfo *out, size_t bytes, troika_t *troika); MDBX_INTERNAL int env_sync(MDBX_env *env, bool force, bool nonblock); MDBX_INTERNAL int env_close(MDBX_env *env, bool resurrect_after_fork); -MDBX_INTERNAL bool env_txn0_owned(const MDBX_env *env); +MDBX_INTERNAL MDBX_txn *env_owned_wrtxn(const MDBX_env *env); MDBX_INTERNAL int __must_check_result env_page_auxbuffer(MDBX_env *env); MDBX_INTERNAL unsigned env_setup_pagesize(MDBX_env *env, const size_t pagesize); From 40f655e2da0fe30c88cea04ec3c9e9fa252808fc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Mon, 3 Mar 2025 01:49:32 +0300 Subject: [PATCH 32/90] =?UTF-8?q?mdbx:=20=D0=BA=D0=BE=D1=80=D1=80=D0=B5?= =?UTF-8?q?=D0=BA=D1=82=D0=B8=D1=80=D0=BE=D0=B2=D0=BA=D0=B0=20`log=5Ferror?= =?UTF-8?q?()`=20=D0=B4=D0=BB=D1=8F=20=D1=83=D1=81=D1=82=D1=80=D0=B0=D0=BD?= =?UTF-8?q?=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=BB=D0=BE=D0=B6=D0=BD=D1=8B=D1=85?= =?UTF-8?q?=20=D0=BE=D1=88=D0=B8=D0=B1=D0=BE=D0=BA=20=D0=BF=D1=80=D0=B8=20?= =?UTF-8?q?=D1=80=D0=B0=D0=B1=D0=BE=D1=82=D0=B5=20mdbx=5Fchk=20=D1=81=20?= =?UTF-8?q?=D0=B2=D1=8B=D1=81=D0=BE=D0=BA=D0=B8=D0=BC=20=D1=83=D1=80=D0=BE?= =?UTF-8?q?=D0=B2=D0=BD=D0=B5=D0=BC=20=D0=BB=D0=BE=D0=B3=D0=B8=D1=80=D0=BE?= =?UTF-8?q?=D0=B2=D0=B0=D0=BD=D0=B8=D1=8F=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Некая проблема была в том, что при высоком уровне логирования в логгер также отправлялись неизбежные MDBX_NOTFOND при достижении конца интегрируемых данных. В свою очередь, chk-логика формирования отчета подсчитывала эти сообщения как ошибки при проверке БД... --- src/logging_and_debug.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/logging_and_debug.c b/src/logging_and_debug.c index e8538934..71153f1f 100644 --- a/src/logging_and_debug.c +++ b/src/logging_and_debug.c @@ -58,10 +58,11 @@ __cold void debug_log(int level, const char *function, int line, const char *fmt __cold void log_error(const int err, const char *func, unsigned line) { assert(err != MDBX_SUCCESS); - if (unlikely(globals.loglevel >= MDBX_LOG_DEBUG) && - (globals.loglevel >= MDBX_LOG_TRACE || !(err == MDBX_RESULT_TRUE || err == MDBX_NOTFOUND))) { + if (unlikely(globals.loglevel >= MDBX_LOG_DEBUG)) { + const bool is_error = err != MDBX_RESULT_TRUE && err != MDBX_NOTFOUND; char buf[256]; - debug_log(MDBX_LOG_ERROR, func, line, "error %d (%s)\n", err, mdbx_strerror_r(err, buf, sizeof(buf))); + debug_log(is_error ? MDBX_LOG_ERROR : MDBX_LOG_VERBOSE, func, line, "%s %d (%s)\n", + is_error ? "error" : "condition", err, mdbx_strerror_r(err, buf, sizeof(buf))); } } From bc2f1c59cb449040157b29ae94ff146ec3ac478f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Tue, 4 Mar 2025 01:14:03 +0300 Subject: [PATCH 33/90] =?UTF-8?q?mdbx:=20=D0=BE=D0=B1=D0=BD=D0=BE=D0=B2?= =?UTF-8?q?=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20ChangeLog.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ChangeLog.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/ChangeLog.md b/ChangeLog.md index a1859394..ebc5aac8 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -12,6 +12,7 @@ and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/libmdbx - [Erigon](https://docs.erigon.tech/) за спонсорство. - [Илье Михееву](https://t.me/IlyaMkhv) за сообщение о лишнем/ненужном предупреждении несоответствия файла БД новому размеру. + - [maxc0d3r](https://gitflic.ru/user/maxc0d3r) for bug reporting and testing. Исправления: @@ -20,6 +21,25 @@ and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/libmdbx так и потенциальных ошибок из-за нехватки адресного пространства. В этом сценарии ранее могло выдаваться лишнее/ненужное предупреждение о несоответствии файла БД новому размеру. Теперь этот недостаток исправлен. + - Восстановлена доступность дескрипторов таблиц, открытых в дочерней транзакции, после её фиксации, в случае отсутствия изменений в данных. + Проблема не была замечена ранее из-за специфического сценария проявления. + Ошибка присутствует в версиях 0.13.x и последующих, начиная с коммита `e6af7d7c53428ca2892bcbf7eec1c2acee06fd44` от 2023-11-05. + + - Устранён сбой аудита таблиц при инвалидации дескрипторов таблиц вследствие отмены вложенной транзакции. + Проблема не была замечена ранее из-за специфического сценария проявления. + Ошибка присутствует в версиях 0.13.x и последующих, начиная с коммита `e6af7d7c53428ca2892bcbf7eec1c2acee06fd44` от 2023-11-05. + + - Устранена причина потенциальных сбоев и/и деградации производительности в сценарии закрытия курсора до завершения вложенной транзакции, + с последующим изменением данных той-же таблицы в текущей вложенной транзакции, либо её дочерних транзакциях. + Проблема обнаружена при ручном анализе кода, сценарии воспроизведения/проявления проблемы пока не известны. + Ошибка присутствует в версиях 0.13.x и последующих, начиная с коммита `3de3d425a128a3c6f7866503f5f93b80c09dbe41` от 2024-05-19. + + - Устранена причина ложных ошибок при работе `mdbx_chk` с высоким уровнем логирования. + Проблема возникала из-за неверной трактовки `MDBX_NOTFOUND` при штатном окончании итерируемых данных. + + - Устранена причина попыток рекурсивного захвата мьютекса при работе `mdbx_chk -w` в сборах с поддержкой Valring/ASAN и под управлением этих инструментов. + + -------------------------------------------------------------------------------- From 0604accecf43ef696d5b7c7dbd2cca00304c14b7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Tue, 4 Mar 2025 10:44:42 +0300 Subject: [PATCH 34/90] =?UTF-8?q?mdbx:=20=D0=BF=D1=80=D0=BE=D0=B2=D0=B5?= =?UTF-8?q?=D1=80=D0=BA=D0=B0=20=D0=B2=D0=BB=D0=B0=D0=B4=D0=B5=D0=BB=D1=8C?= =?UTF-8?q?=D1=86=D0=B0=20=D0=BF=D0=BE=D1=82=D0=BE=D0=BA=D0=B0=20=D0=B2?= =?UTF-8?q?=D0=BB=D0=B0=D0=B4=D0=B5=D1=8E=D1=89=D0=B5=D0=B3=D0=BE=20=D1=82?= =?UTF-8?q?=D1=80=D0=B0=D0=BD=D0=B7=D0=B0=D0=BA=D1=86=D0=B8=D0=B5=D0=B9=20?= =?UTF-8?q?=D1=82=D0=BE=D0=BB=D1=8C=D0=BA=D0=BE=20=D0=BF=D1=80=D0=B8=20`MD?= =?UTF-8?q?BX=5FTXN=5FCHECKOWNER=3DON`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/api-env.c | 2 +- src/api-extra.c | 3 ++- src/api-txn.c | 4 ++++ test/extra/txn.c++ | 2 ++ 4 files changed, 9 insertions(+), 2 deletions(-) diff --git a/src/api-env.c b/src/api-env.c index 2bfbdeb2..bd3702e9 100644 --- a/src/api-env.c +++ b/src/api-env.c @@ -648,7 +648,7 @@ __cold int mdbx_env_close_ex(MDBX_env *env, bool dont_sync) { #endif /* Windows */ } - if (env->basal_txn && env->basal_txn->owner == osal_thread_self()) + if (env->basal_txn && (MDBX_TXN_CHECKOWNER ? env->basal_txn->owner == osal_thread_self() : !!env->basal_txn->owner)) lck_txn_unlock(env); eASSERT(env, env->signature.weak == 0); diff --git a/src/api-extra.c b/src/api-extra.c index 61b0fac9..32ad4bfd 100644 --- a/src/api-extra.c +++ b/src/api-extra.c @@ -117,7 +117,6 @@ __cold int mdbx_thread_unregister(const MDBX_env *env) { return MDBX_RESULT_TRUE /* not registered */; eASSERT(env, r->pid.weak == env->pid); - eASSERT(env, r->tid.weak == osal_thread_self()); if (unlikely(r->pid.weak != env->pid || r->tid.weak != osal_thread_self())) return LOG_IFERR(MDBX_BAD_RSLOT); @@ -154,8 +153,10 @@ int mdbx_txn_unlock(MDBX_env *env) { if (unlikely(env->flags & MDBX_RDONLY)) return LOG_IFERR(MDBX_EACCESS); +#if MDBX_TXN_CHECKOWNER if (unlikely(env->basal_txn->owner != osal_thread_self())) return LOG_IFERR(MDBX_THREAD_MISMATCH); +#endif /* MDBX_TXN_CHECKOWNER */ if (unlikely((env->basal_txn->flags & MDBX_TXN_FINISHED) == 0)) return LOG_IFERR(MDBX_BUSY); diff --git a/src/api-txn.c b/src/api-txn.c index 1f24c930..2fe01a4e 100644 --- a/src/api-txn.c +++ b/src/api-txn.c @@ -101,11 +101,13 @@ int mdbx_txn_abort(MDBX_txn *txn) { if (unlikely(rc != MDBX_SUCCESS)) return LOG_IFERR(rc); +#if MDBX_TXN_CHECKOWNER if ((txn->flags & (MDBX_TXN_RDONLY | MDBX_NOSTICKYTHREADS)) == MDBX_NOSTICKYTHREADS && unlikely(txn->owner != osal_thread_self())) { mdbx_txn_break(txn); return LOG_IFERR(MDBX_THREAD_MISMATCH); } +#endif /* MDBX_TXN_CHECKOWNER */ return LOG_IFERR(txn_abort(txn)); } @@ -420,11 +422,13 @@ int mdbx_txn_commit_ex(MDBX_txn *txn, MDBX_commit_latency *latency) { goto done; } +#if MDBX_TXN_CHECKOWNER if (!txn->parent && (txn->flags & MDBX_NOSTICKYTHREADS) && unlikely(txn->owner != osal_thread_self())) { txn->flags |= MDBX_TXN_ERROR; rc = MDBX_THREAD_MISMATCH; return LOG_IFERR(rc); } +#endif /* MDBX_TXN_CHECKOWNER */ if (unlikely(txn->flags & MDBX_TXN_ERROR)) { rc = MDBX_RESULT_TRUE; diff --git a/test/extra/txn.c++ b/test/extra/txn.c++ index d817cc7e..3a616015 100644 --- a/test/extra/txn.c++ +++ b/test/extra/txn.c++ @@ -131,6 +131,7 @@ int main(int argc, const char *argv[]) { std::thread t([&]() { s.wait(); +#if MDBX_TXN_CHECKOWNER err = mdbx_txn_reset(c_txn); assert(err == MDBX_THREAD_MISMATCH); ok = ok && err == MDBX_THREAD_MISMATCH; @@ -143,6 +144,7 @@ int main(int argc, const char *argv[]) { err = mdbx_txn_abort(c_txn); assert(err == MDBX_THREAD_MISMATCH); ok = ok && err == MDBX_THREAD_MISMATCH; +#endif /* MDBX_TXN_CHECKOWNER */ err = mdbx_txn_begin(env, txn, MDBX_TXN_READWRITE, &c_txn); assert(err == MDBX_BAD_TXN); ok = ok && err == MDBX_BAD_TXN; From 94a2abaf31a293fb3eddc7fe3b61a1d846a60a0d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Tue, 4 Mar 2025 14:45:13 +0300 Subject: [PATCH 35/90] =?UTF-8?q?mdbx:=20=D0=B4=D0=BE=D0=B1=D0=B0=D0=B2?= =?UTF-8?q?=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=B2=20API=20`mdbx=5Ftxn=5Fr?= =?UTF-8?q?elease=5Fall=5Fcursors=5Fex()`=20=D0=B8=20=D0=B8=D0=B7=D0=BC?= =?UTF-8?q?=D0=B5=D0=BD=D0=B5=D0=BD=D0=B8=D0=B5=20=D1=81=D0=B5=D0=BC=D0=B0?= =?UTF-8?q?=D0=BD=D1=82=D0=B8=D0=BA=D0=B8=20=D1=80=D0=B5=D0=B7=D1=83=D0=BB?= =?UTF-8?q?=D1=8C=D1=82=D0=B0=D1=82=D0=B0=20`mdbx=5Ftxn=5Frelease=5Fall=5F?= =?UTF-8?q?cursors()`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit По недосмотру в выпусках остался предварительный/черновой вариант функции mdbx_txn_release_all_cursors(), который смешивает в возвращаемом значении информацию об ошибке/успехе и количество обработанных курсоров. За-за чего невозможно отличить одно от другого, например ошибку EPERM на Linux от одного успешно закрытого курсора. Теперь mdbx_txn_release_all_cursors() возвращает только код ошибки, а для получения кол-ва закрытых курсоров в API добавлена функция mdbx_txn_release_all_cursors_ex(). --- mdbx.h | 37 +++++++++++++++++++++++++++++++------ src/api-cursor.c | 8 +++++--- 2 files changed, 36 insertions(+), 9 deletions(-) diff --git a/mdbx.h b/mdbx.h index a0886799..99a34f49 100644 --- a/mdbx.h +++ b/mdbx.h @@ -5227,17 +5227,42 @@ LIBMDBX_API void mdbx_cursor_close(MDBX_cursor *cursor); * \see mdbx_cursor_unbind() * \see mdbx_cursor_close() * - * \param [in] txn A transaction handle returned by \ref mdbx_txn_begin(). - * \param [in] unbind If non-zero, unbinds cursors and leaves ones reusable. - * Otherwise close and dispose cursors. + * \param [in] txn A transaction handle returned by \ref mdbx_txn_begin(). + * \param [in] unbind If non-zero, unbinds cursors and leaves ones reusable. + * Otherwise close and dispose cursors. + * \param [in,out] count An optional pointer to return the number of cursors + * processed by the requested operation. * - * \returns A negative error value on failure or the number of closed cursors - * on success, some possible errors are: + * \returns A non-zero error value on failure and 0 on success, + * some possible errors are: * \retval MDBX_THREAD_MISMATCH Given transaction is not owned * by current thread. * \retval MDBX_BAD_TXN Given transaction is invalid or has * a child/nested transaction transaction. */ -LIBMDBX_API int mdbx_txn_release_all_cursors(const MDBX_txn *txn, bool unbind); +LIBMDBX_API int mdbx_txn_release_all_cursors_ex(const MDBX_txn *txn, bool unbind, size_t *count); + +/** \brief Unbind or closes all cursors of a given transaction. + * \ingroup c_cursors + * + * Unbinds either closes all cursors associated (opened or renewed) with + * a given transaction in a bulk with minimal overhead. + * + * \see mdbx_cursor_unbind() + * \see mdbx_cursor_close() + * + * \param [in] txn A transaction handle returned by \ref mdbx_txn_begin(). + * \param [in] unbind If non-zero, unbinds cursors and leaves ones reusable. + * Otherwise close and dispose cursors. + * + * \returns A non-zero error value on failure and 0 on success, + * some possible errors are: + * \retval MDBX_THREAD_MISMATCH Given transaction is not owned + * by current thread. + * \retval MDBX_BAD_TXN Given transaction is invalid or has + * a child/nested transaction transaction. */ +LIBMDBX_INLINE_API(int, mdbx_txn_release_all_cursors, (const MDBX_txn *txn, bool unbind)) { + return mdbx_txn_release_all_cursors_ex(txn, unbind, NULL); +} /** \brief Renew a cursor handle for use within the given transaction. * \ingroup c_cursors diff --git a/src/api-cursor.c b/src/api-cursor.c index b925ac8b..1fefaf2a 100644 --- a/src/api-cursor.c +++ b/src/api-cursor.c @@ -216,14 +216,15 @@ again: return MDBX_SUCCESS; } -int mdbx_txn_release_all_cursors(const MDBX_txn *txn, bool unbind) { +int mdbx_txn_release_all_cursors_ex(const MDBX_txn *txn, bool unbind, size_t *count) { int rc = check_txn(txn, MDBX_TXN_FINISHED | MDBX_TXN_HAS_CHILD); + size_t n = 0; if (likely(rc == MDBX_SUCCESS)) { TXN_FOREACH_DBI_FROM(txn, i, MAIN_DBI) { while (txn->cursors[i]) { + ++n; MDBX_cursor *mc = txn->cursors[i]; ENSURE(nullptr, mc->signature == cur_signature_live && (mc->next != mc) && !mc->backup); - rc = likely(rc < INT_MAX) ? rc + 1 : rc; txn->cursors[i] = mc->next; mc->next = mc; if (unbind) { @@ -236,9 +237,10 @@ int mdbx_txn_release_all_cursors(const MDBX_txn *txn, bool unbind) { } } } else { - eASSERT(nullptr, rc < 0); LOG_IFERR(rc); } + if (count) + *count = n; return rc; } From 7504a8f8f2179a4065c040a1f523de58d0ea8152 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Tue, 4 Mar 2025 14:47:13 +0300 Subject: [PATCH 36/90] =?UTF-8?q?mdbx:=20=D0=BE=D0=B1=D0=BD=D0=BE=D0=B2?= =?UTF-8?q?=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20ChangeLog.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ChangeLog.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/ChangeLog.md b/ChangeLog.md index ebc5aac8..96ae55e4 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -11,7 +11,7 @@ and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/libmdbx Благодарности: - [Erigon](https://docs.erigon.tech/) за спонсорство. - - [Илье Михееву](https://t.me/IlyaMkhv) за сообщение о лишнем/ненужном предупреждении несоответствия файла БД новому размеру. + - [Илье Михееву](https://t.me/IlyaMkhv) за сообщения о недочетах и тестирование. - [maxc0d3r](https://gitflic.ru/user/maxc0d3r) for bug reporting and testing. Исправления: @@ -39,6 +39,13 @@ and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/libmdbx - Устранена причина попыток рекурсивного захвата мьютекса при работе `mdbx_chk -w` в сборах с поддержкой Valring/ASAN и под управлением этих инструментов. + - Проверка владельца потока владеющего транзакцией только при `MDBX_TXN_CHECKOWNER=ON`. + +Изменение поведения: + + - Функция `dbx_txn_release_all_cursors()` возвращает только код ошибки, не смешивая его с количеством обработанных/закрытых курсоров. + Для аналогичных действий с получением количества закрытых курсоров в API добавлена функция `mdbx_txn_release_all_cursors_ex()`. + -------------------------------------------------------------------------------- From 805d84480d67285a3c6e5795d13cf950e122122b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Wed, 5 Mar 2025 01:44:21 +0300 Subject: [PATCH 37/90] =?UTF-8?q?mdbx:=20=D0=B8=D1=81=D0=BF=D1=80=D0=B0?= =?UTF-8?q?=D0=B2=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=BE=D0=BF=D0=B5=D1=87?= =?UTF-8?q?=D0=B0=D1=82=D0=BA=D0=B8=20=D0=B2=20ChangeLog.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ChangeLog.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ChangeLog.md b/ChangeLog.md index 96ae55e4..3d4634e3 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -43,7 +43,7 @@ and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/libmdbx Изменение поведения: - - Функция `dbx_txn_release_all_cursors()` возвращает только код ошибки, не смешивая его с количеством обработанных/закрытых курсоров. + - Функция `mdbx_txn_release_all_cursors()` возвращает только код ошибки, не смешивая его с количеством обработанных/закрытых курсоров. Для аналогичных действий с получением количества закрытых курсоров в API добавлена функция `mdbx_txn_release_all_cursors_ex()`. From 49e6bd9296e0396427cc47e013fc4a49e8971078 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 00:29:20 +0300 Subject: [PATCH 38/90] =?UTF-8?q?mdbx++:=20=D0=B8=D1=81=D0=BF=D0=BE=D0=BB?= =?UTF-8?q?=D1=8C=D0=B7=D0=BE=D0=B2=D0=B0=D0=BD=D0=B8=D0=B5=20`mdbx=5Ftxn?= =?UTF-8?q?=5Frelease=5Fall=5Fcursors=5Fex()`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- mdbx.h++ | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/mdbx.h++ b/mdbx.h++ index 62315f61..c0226157 100644 --- a/mdbx.h++ +++ b/mdbx.h++ @@ -5604,10 +5604,9 @@ inline cursor_managed txn::open_cursor(map_handle map) const { } inline size_t txn::release_all_cursors(bool unbind) const { - int err = ::mdbx_txn_release_all_cursors(handle_, unbind); - if (MDBX_UNLIKELY(err < 0)) - MDBX_CXX20_UNLIKELY error::throw_exception(err); - return size_t(err); + size_t count; + error::success_or_throw(::mdbx_txn_release_all_cursors_ex(handle_, unbind, &count)); + return count; } inline ::mdbx::map_handle txn::open_map(const ::mdbx::slice &name, const ::mdbx::key_mode key_mode, From 44467d088320a91a147f2dc4bf3b70a4e1ba638d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 00:30:01 +0300 Subject: [PATCH 39/90] =?UTF-8?q?mdbx-make:=20=D0=B4=D0=BE=D0=B1=D0=B0?= =?UTF-8?q?=D0=B2=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D1=86=D0=B5=D0=BB=D0=B8?= =?UTF-8?q?=20`ninja-assertions`=20=D0=B8=20=D0=B5=D1=91=20=D0=B8=D1=81?= =?UTF-8?q?=D0=BF=D0=BE=D0=BB=D1=8C=D0=B7=D0=BE=D0=B2=D0=B0=D0=BD=D0=B8?= =?UTF-8?q?=D0=B5=20=D0=BF=D1=80=D0=B8=20`make=20check`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- GNUmakefile | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/GNUmakefile b/GNUmakefile index 355ce295..2d26a381 100644 --- a/GNUmakefile +++ b/GNUmakefile @@ -295,6 +295,10 @@ lib-shared libmdbx.$(SO_SUFFIX): mdbx-dylib.o $(call select_by,MDBX_BUILD_CXX,md @echo ' LD $@' $(QUIET)$(call select_by,MDBX_BUILD_CXX,$(CXX) $(CXXFLAGS),$(CC) $(CFLAGS)) $^ -pthread -shared $(LDFLAGS) $(call select_by,MDBX_BUILD_CXX,$(LIB_STDCXXFS)) $(LIBS) -o $@ +ninja-assertions: CMAKE_OPT += -DMDBX_FORCE_ASSERTIONS=ON +ninja-assertions: cmake-build +ninja-debug: CMAKE_OPT += -DCMAKE_BUILD_TYPE=Debug +ninja-debug: cmake-build ninja: cmake-build cmake-build: @echo " RUN: cmake -G Ninja && cmake --build" @@ -424,7 +428,7 @@ MDBX_SMOKE_EXTRA ?= check: DESTDIR = $(shell pwd)/@check-install check: CMAKE_OPT = -Werror=dev -check: smoke-assertion ninja dist install test ctest +check: smoke-assertion ninja-assertions dist install test ctest smoke-assertion: MDBX_BUILD_OPTIONS:=$(strip $(MDBX_BUILD_OPTIONS) -DMDBX_FORCE_ASSERTIONS=1 -UNDEBUG -DMDBX_DEBUG=0) smoke-assertion: smoke @@ -660,7 +664,7 @@ release-assets: libmdbx-amalgamated-$(MDBX_GIT_3DOT).zpaq \ @echo -n ' VERIFY amalgamated sources...' $(QUIET)rm -rf $@ $(DIST_DIR)/@tmp-essentials.inc $(DIST_DIR)/@tmp-internals.inc \ && if grep -R "define xMDBX_ALLOY" dist | grep -q MDBX_BUILD_SOURCERY; then echo "sed output is WRONG!" >&2; exit 2; fi \ - && rm -rf @dist-check && cp -r -p $(DIST_DIR) @dist-check && ($(MAKE) -j IOARENA=false CXXSTD=$(CXXSTD) -C @dist-check all ninja >@dist-check.log 2>@dist-check.err || (cat @dist-check.err && exit 1)) \ + && rm -rf @dist-check && cp -r -p $(DIST_DIR) @dist-check && ($(MAKE) -j IOARENA=false CXXSTD=$(CXXSTD) -C @dist-check all ninja-assertions >@dist-check.log 2>@dist-check.err || (cat @dist-check.err && exit 1)) \ && touch $@ || (echo " FAILED! See @dist-check.log and @dist-check.err" >&2; exit 2) && echo " Ok" %.tar.gz: @dist-checked.tag From 4d454d6e804c8ed24295c472e3122f927af591e3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 00:35:47 +0300 Subject: [PATCH 40/90] =?UTF-8?q?mdbx-cmake:=20=D1=80=D0=B0=D1=81=D1=88?= =?UTF-8?q?=D0=B8=D1=80=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=BF=D0=BE=D0=B8=D1=81?= =?UTF-8?q?=D0=BA=D0=B0=20`LLVMgold.so`=20=D0=B2=20=D0=BE=D1=82=D0=BD?= =?UTF-8?q?=D0=BE=D1=81=D0=B8=D1=82=D0=B5=D0=BB=D1=8C=D0=BD=D1=8B=D1=85=20?= =?UTF-8?q?lib-=D0=B4=D0=B8=D1=80=D0=B5=D0=BA=D1=82=D0=BE=D1=80=D0=B8?= =?UTF-8?q?=D1=8F=D1=85=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- cmake/compiler.cmake | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/cmake/compiler.cmake b/cmake/compiler.cmake index 5fefdbae..d9197e66 100644 --- a/cmake/compiler.cmake +++ b/cmake/compiler.cmake @@ -541,13 +541,21 @@ if(CMAKE_COMPILER_IS_CLANG) if(regexp_valid) string(REGEX REPLACE "(^|\n.*)(.*programs: =)([^\n]+)((\n.*)|$)" "\\3" list ${clang_search_dirs}) string(REPLACE ":" ";" list "${list}") + set(libs_extra_subdirs "lib;../lib;lib64;../lib64;lib32;../lib32") foreach(dir IN LISTS list) get_filename_component(dir "${dir}" REALPATH) if(dir MATCHES ".*llvm.*" OR dir MATCHES ".*clang.*") - list(APPEND clang_bindirs "${dir}") + set(list_suffix "") else() - list(APPEND clang_bindirs_x "${dir}") + set(list_suffix "_x") endif() + list(APPEND clang_bindirs${list_suffix} "${dir}") + foreach(subdir IN LISTS libs_extra_subdirs) + get_filename_component(subdir "${dir}/${subdir}" REALPATH) + if(EXISTS "${subdir}") + list(APPEND clang_libdirs${list_suffix} "${subdir}") + endif() + endforeach() endforeach() list(APPEND clang_bindirs "${clang_bindirs_x}") list(REMOVE_DUPLICATES clang_bindirs) @@ -559,10 +567,11 @@ if(CMAKE_COMPILER_IS_CLANG) foreach(dir IN LISTS list) get_filename_component(dir "${dir}" REALPATH) if(dir MATCHES ".*llvm.*" OR dir MATCHES ".*clang.*") - list(APPEND clang_libdirs "${dir}") + set(list_suffix "") else() - list(APPEND clang_libdirs_x "${dir}") + set(list_suffix "_x") endif() + list(APPEND clang_libdirs${list_suffix} "${dir}") endforeach() list(APPEND clang_libdirs "${clang_libdirs_x}") list(REMOVE_DUPLICATES clang_libdirs) From d6b359756c603b2ba18d32d3d1822b5064e7ac1d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 00:40:02 +0300 Subject: [PATCH 41/90] =?UTF-8?q?mdbx-cmake:=20=D1=80=D0=B0=D1=81=D1=81?= =?UTF-8?q?=D0=BB=D0=B0=D0=B1=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D1=83=D1=81?= =?UTF-8?q?=D0=BB=D0=BE=D0=B2=D0=B8=D0=B9=20=D0=B4=D0=BB=D1=8F=20=D0=B8?= =?UTF-8?q?=D1=81=D0=BF=D0=BE=D0=BB=D1=8C=D0=B7=D0=BE=D0=B2=D0=B0=D0=BD?= =?UTF-8?q?=D0=B8=D1=8F=20LTO=20=D1=81=20CLANG=20=D0=BD=D0=B0=20Linux=20(b?= =?UTF-8?q?ackport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- cmake/compiler.cmake | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cmake/compiler.cmake b/cmake/compiler.cmake index d9197e66..8cee26d4 100644 --- a/cmake/compiler.cmake +++ b/cmake/compiler.cmake @@ -665,7 +665,7 @@ if(CMAKE_COMPILER_IS_CLANG) AND CMAKE_CLANG_NM AND CMAKE_CLANG_RANLIB AND ((CLANG_LTO_PLUGIN AND CMAKE_LD_GOLD) - OR (CMAKE_CLANG_LD AND NOT (CMAKE_HOST_SYSTEM_NAME STREQUAL "Linux" AND CMAKE_SYSTEM_NAME STREQUAL "Linux")) + OR CMAKE_CLANG_LD OR APPLE)) if(ANDROID AND CMAKE_${CMAKE_PRIMARY_LANG}_COMPILER_VERSION VERSION_LESS 12) set(CLANG_LTO_AVAILABLE FALSE) From f82b760b6e7ad7aed067f899d34265cad0f19b6d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 00:37:13 +0300 Subject: [PATCH 42/90] =?UTF-8?q?mdbx-cmake:=20=D0=B8=D0=B7=D0=B1=D0=B5?= =?UTF-8?q?=D0=B3=D0=B0=D0=B5=D0=BC=20=D0=B4=D0=B2=D0=BE=D0=B9=D0=BD=D0=BE?= =?UTF-8?q?=D0=B9=20=D1=80=D0=B0=D0=B1=D0=BE=D1=82=D1=8B=20`compiler.cmake?= =?UTF-8?q?`=20=D0=B1=D0=B5=D0=B7=20=D0=BD=D0=B5=D0=BE=D0=B1=D1=85=D0=BE?= =?UTF-8?q?=D0=B4=D0=B8=D0=BC=D0=BE=D1=81=D1=82=D0=B8=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- test/CMakeLists.txt | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt index f8a72d68..d4dfc086 100644 --- a/test/CMakeLists.txt +++ b/test/CMakeLists.txt @@ -1,8 +1,10 @@ # Copyright (c) 2020-2025 Леонид Юрьев aka Leonid Yuriev ############################################### # SPDX-License-Identifier: Apache-2.0 -enable_language(CXX) -include(../cmake/compiler.cmake) +if(NOT CMAKE_CXX_COMPILER_LOADED) + enable_language(CXX) + include(../cmake/compiler.cmake) +endif() function(add_extra_test name) set(options DISABLED) From 4f59864ef51345c455345aef70d63bc4c7b7f01d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 00:41:13 +0300 Subject: [PATCH 43/90] =?UTF-8?q?mdbx-cmake:=20=D0=B8=D1=81=D0=BF=D0=BE?= =?UTF-8?q?=D0=BB=D1=8C=D0=B7=D1=83=D0=B5=D0=BC=20`-flto=3Dauto`=20=D0=B4?= =?UTF-8?q?=D0=BB=D1=8F=20GCC=20>=3D=2011.4=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit При сборке посредством GCC >= 11.4 больше не возникает предупреждений: lto-wrapper: warning: using serial compilation of # LTRANS jobs lto-wrapper: note: see the ‘-flto’ option documentation for more information Однако, использование auto-режима не является оптимальным решением, так как при параллельной сборке посредством make или ninja, каждая уже запущенная ветвь компиляции породит потоки ещё для каждого ядра ЦПУ. Таким образом реальная нагрузка может расти квадратично, т.е. чем больше у вас ядер -- тем хуже и при 96 ядрах может работать 9216 потоков сборки. Тем не менее, использование `job-server` в CMake пока не возможно, а при сборке libmdbx не так много работы чтобы чтобы обрушить систему нагрузкой. --- cmake/compiler.cmake | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/cmake/compiler.cmake b/cmake/compiler.cmake index 8cee26d4..a3d789ce 100644 --- a/cmake/compiler.cmake +++ b/cmake/compiler.cmake @@ -502,7 +502,11 @@ if(CMAKE_COMPILER_IS_GNU${CMAKE_PRIMARY_LANG} AND CMAKE_GCC_RANLIB AND gcc_lto_wrapper) message(STATUS "Found GCC's LTO toolset: ${gcc_lto_wrapper}, ${CMAKE_GCC_AR}, ${CMAKE_GCC_RANLIB}") - set(GCC_LTO_CFLAGS "-flto -fno-fat-lto-objects -fuse-linker-plugin") + if(CMAKE_${CMAKE_PRIMARY_LANG}_COMPILER_VERSION VERSION_LESS 11.4) + set(GCC_LTO_CFLAGS "-flto -fno-fat-lto-objects -fuse-linker-plugin") + else() + set(GCC_LTO_CFLAGS "-flto=auto -fno-fat-lto-objects -fuse-linker-plugin") + endif() set(GCC_LTO_AVAILABLE TRUE) message(STATUS "Link-Time Optimization by GCC is available") else() From 32ca9691c391c2decdc3d638bbce22c67d4ccf11 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 00:44:54 +0300 Subject: [PATCH 44/90] =?UTF-8?q?mdbx-doc:=20=D0=B4=D0=BE=D0=B1=D0=B0?= =?UTF-8?q?=D0=B2=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D1=81=D1=81=D1=8B=D0=BB?= =?UTF-8?q?=D0=BA=D0=B8=20=D0=BD=D0=B0=20=D0=BF=D1=80=D0=B8=D0=B2=D1=8F?= =?UTF-8?q?=D0=B7=D0=BA=D1=83=20=D0=BA=20Zig=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 1bad0676..69f67b9b 100644 --- a/README.md +++ b/README.md @@ -690,6 +690,7 @@ Bindings | Java | [mdbxjni](https://github.com/castortech/mdbxjni) | [Castor Technologies](https://castortech.com/) | | Go | [mdbx-go](https://github.com/torquem-ch/mdbx-go) | [Alex Sharov](https://github.com/AskAlexSharov) | | Ruby | [ruby-mdbx](https://rubygems.org/gems/mdbx/) | [Mahlon E. Smith](https://github.com/mahlonsmith) | +| Zig | [mdbx-zig](https://github.com/theseyan/lmdbx-zig) | [Sayan J. Das](https://github.com/theseyan) | ##### Obsolete/Outdated/Unsupported: From 4150f411dc8162765776683535f081525d6f873b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 00:45:57 +0300 Subject: [PATCH 45/90] =?UTF-8?q?mdbx:=20=D0=BF=D0=B5=D1=80=D0=B5=D1=80?= =?UTF-8?q?=D0=B0=D0=B1=D0=BE=D1=82=D0=BA=D0=B0=20=D0=BF=D1=80=D0=BE=D0=B2?= =?UTF-8?q?=D0=B5=D1=80=D0=BA=D0=B0=20=D1=80=D0=B0=D0=B7=D0=BC=D0=B5=D1=80?= =?UTF-8?q?=D0=B0=20=D1=84=D0=B0=D0=B9=D0=BB=D0=B0=20=D0=91=D0=94=20=D0=BF?= =?UTF-8?q?=D1=80=D0=B8=20=D0=BE=D1=82=D0=BA=D1=80=D1=8B=D1=82=D0=B8=D0=B8?= =?UTF-8?q?=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Переработка 05cdf9d202b14ac09c801c7893e65271fa27f378. У предыдущего варианта был недостаток, при необходимости выдачи предупреждения и открытии БД с изменением геометрии, предупреждение не выдавалось, что может затруднять анализ/разбор проблемных ситуаций. --- src/dxb.c | 30 ++++++++++++++---------------- 1 file changed, 14 insertions(+), 16 deletions(-) diff --git a/src/dxb.c b/src/dxb.c index 2b3cc03a..c0d34719 100644 --- a/src/dxb.c +++ b/src/dxb.c @@ -567,7 +567,7 @@ __cold int dxb_setup(MDBX_env *env, const int lck_rc, const mdbx_mode_t mode_bit return err; } - bool new_size_from_user = false; + size_t expected_filesize = 0; const size_t used_bytes = pgno2bytes(env, header.geometry.first_unallocated); const size_t used_aligned2os_bytes = ceil_powerof2(used_bytes, globals.sys_pagesize); if ((env->flags & MDBX_RDONLY) /* readonly */ @@ -612,28 +612,25 @@ __cold int dxb_setup(MDBX_env *env, const int lck_rc, const mdbx_mode_t mode_bit } /* altering fields to match geometry given from user */ - new_size_from_user = header.geometry.now != bytes2pgno(env, env->geo_in_bytes.now); - if (new_size_from_user) - header.geometry.now = bytes2pgno(env, env->geo_in_bytes.now); + expected_filesize = pgno_align2os_bytes(env, header.geometry.now); + header.geometry.now = bytes2pgno(env, env->geo_in_bytes.now); header.geometry.lower = bytes2pgno(env, env->geo_in_bytes.lower); header.geometry.upper = bytes2pgno(env, env->geo_in_bytes.upper); header.geometry.grow_pv = pages2pv(bytes2pgno(env, env->geo_in_bytes.grow)); header.geometry.shrink_pv = pages2pv(bytes2pgno(env, env->geo_in_bytes.shrink)); - VERBOSE("amended: root %" PRIaPGNO "/%" PRIaPGNO ", geo %" PRIaPGNO "/%" PRIaPGNO "-%" PRIaPGNO "/%" PRIaPGNO + VERBOSE("amending: root %" PRIaPGNO "/%" PRIaPGNO ", geo %" PRIaPGNO "/%" PRIaPGNO "-%" PRIaPGNO "/%" PRIaPGNO " +%u -%u, txn_id %" PRIaTXN ", %s", header.trees.main.root, header.trees.gc.root, header.geometry.lower, header.geometry.first_unallocated, header.geometry.now, header.geometry.upper, pv2pages(header.geometry.grow_pv), pv2pages(header.geometry.shrink_pv), unaligned_peek_u64(4, header.txnid_a), durable_caption(&header)); } else { - /* fetch back 'now/current' size, since it was ignored during comparison - * and may differ. */ + /* fetch back 'now/current' size, since it was ignored during comparison and may differ. */ env->geo_in_bytes.now = pgno_align2os_bytes(env, header.geometry.now); } ENSURE(env, header.geometry.now >= header.geometry.first_unallocated); } else { - /* geo-params are not pre-configured by user, - * get current values from the meta. */ + /* geo-params are not pre-configured by user, get current values from the meta. */ env->geo_in_bytes.now = pgno2bytes(env, header.geometry.now); env->geo_in_bytes.lower = pgno2bytes(env, header.geometry.lower); env->geo_in_bytes.upper = pgno2bytes(env, header.geometry.upper); @@ -643,18 +640,19 @@ __cold int dxb_setup(MDBX_env *env, const int lck_rc, const mdbx_mode_t mode_bit ENSURE(env, pgno_align2os_bytes(env, header.geometry.now) == env->geo_in_bytes.now); ENSURE(env, env->geo_in_bytes.now >= used_bytes); + if (!expected_filesize) + expected_filesize = env->geo_in_bytes.now; const uint64_t filesize_before = env->dxb_mmap.filesize; if (unlikely(filesize_before != env->geo_in_bytes.now)) { if (lck_rc != /* lck exclusive */ MDBX_RESULT_TRUE) { - VERBOSE("filesize mismatch (expect %" PRIuPTR "b/%" PRIaPGNO "p, have %" PRIu64 "b/%" PRIaPGNO "p), " - "assume other process working", + VERBOSE("filesize mismatch (expect %" PRIuPTR "b/%" PRIaPGNO "p, have %" PRIu64 "b/%" PRIu64 + "p), assume other process working", env->geo_in_bytes.now, bytes2pgno(env, env->geo_in_bytes.now), filesize_before, - bytes2pgno(env, (size_t)filesize_before)); + filesize_before >> env->ps2ln); } else { - if (!new_size_from_user) - WARNING("filesize mismatch (expect %" PRIuSIZE "b/%" PRIaPGNO "p, have %" PRIu64 "b/%" PRIaPGNO "p)", - env->geo_in_bytes.now, bytes2pgno(env, env->geo_in_bytes.now), filesize_before, - bytes2pgno(env, (size_t)filesize_before)); + if (filesize_before != expected_filesize) + WARNING("filesize mismatch (expect %" PRIuSIZE "b/%" PRIaPGNO "p, have %" PRIu64 "b/%" PRIu64 "p)", + expected_filesize, bytes2pgno(env, expected_filesize), filesize_before, filesize_before >> env->ps2ln); if (filesize_before < used_bytes) { ERROR("last-page beyond end-of-file (last %" PRIaPGNO ", have %" PRIaPGNO ")", header.geometry.first_unallocated, bytes2pgno(env, (size_t)filesize_before)); From d8f9f3ba5842f3be288049e2af0ac8a1be965125 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 00:46:45 +0300 Subject: [PATCH 46/90] =?UTF-8?q?mdbx:=20=D0=BF=D1=80=D0=BE=D0=B2=D0=B5?= =?UTF-8?q?=D1=80=D1=8F=D0=B5=D0=BC=20=D0=B2=D1=8B=D1=80=D0=B0=D0=B2=D0=BD?= =?UTF-8?q?=D0=B8=D0=B2=D0=B0=D0=BD=D0=B8=D0=B5=20=D1=80=D0=B0=D0=B7=D0=BC?= =?UTF-8?q?=D0=B5=D1=80=D0=B0=20=D0=91=D0=94=20=D0=BD=D0=B0=20=D1=8E=D0=BD?= =?UTF-8?q?=D0=B8=D1=82=20=D0=B2=D1=8B=D0=B4=D0=B5=D0=BB=D0=B5=D0=BD=D0=B8?= =?UTF-8?q?=D1=8F=20=D0=BF=D0=B0=D0=BC=D1=8F=D1=82=D0=B8,=20=D0=B0=20?= =?UTF-8?q?=D0=BD=D0=B5=20=D0=BD=D0=B0=20=D1=80=D0=B0=D0=B7=D0=BC=D0=B5?= =?UTF-8?q?=D1=80=20=D1=81=D1=82=D1=80=D0=B0=D0=BD=D0=B8=D1=86=D1=8B=20(ba?= =?UTF-8?q?ckport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Теоретически до этого коммита могла быть некоторая неувязка: - при открытии БД с размером страницы 4K на Windows (где размер секции кратен 64K) в режиме read-only, - после того как БД использовалась на POSIX (где размер отображения кратен размеру системной страницы). Ранее ошибка могла возвращаться со стороны системы (например INVALID_PARAMETER) и по ней крайне сложно было понять в чем дело. Теперь же будет логирование ошибки и возврат MDBX_WANNA_RECOVERY. --- src/dxb.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/dxb.c b/src/dxb.c index c0d34719..fa3b8f6e 100644 --- a/src/dxb.c +++ b/src/dxb.c @@ -660,8 +660,9 @@ __cold int dxb_setup(MDBX_env *env, const int lck_rc, const mdbx_mode_t mode_bit } if (env->flags & MDBX_RDONLY) { - if (filesize_before & (globals.sys_pagesize - 1)) { - ERROR("%s", "filesize should be rounded-up to system page"); + if (filesize_before & (globals.sys_allocation_granularity - 1)) { + ERROR("filesize should be rounded-up to system allocation granularity %u", + globals.sys_allocation_granularity); return MDBX_WANNA_RECOVERY; } WARNING("%s", "ignore filesize mismatch in readonly-mode"); From 15bd9cfc89e2a0875342030d4e335f4dce3a2c96 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 00:52:16 +0300 Subject: [PATCH 47/90] =?UTF-8?q?mdbx:=20=D1=83=D0=B4=D0=B0=D0=BB=D0=B5?= =?UTF-8?q?=D0=BD=D0=B8=D0=B5=20`const`=20=D1=83=20=D1=82=D1=80=D0=B0?= =?UTF-8?q?=D0=BD=D0=B7=D0=B0=D0=BA=D1=86=D0=B8=D0=B8=20=D0=B2=20`cursor?= =?UTF-8?q?=5Fbind()`=20=D0=B8=20`cursor=5Frenew()`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- mdbx.h | 6 +++--- mdbx.h++ | 8 ++++---- src/api-cursor.c | 6 +++--- 3 files changed, 10 insertions(+), 10 deletions(-) diff --git a/mdbx.h b/mdbx.h index 99a34f49..728e6eb0 100644 --- a/mdbx.h +++ b/mdbx.h @@ -5131,7 +5131,7 @@ MDBX_NOTHROW_PURE_FUNCTION LIBMDBX_API void *mdbx_cursor_get_userctx(const MDBX_ * \retval MDBX_THREAD_MISMATCH Given transaction is not owned * by current thread. * \retval MDBX_EINVAL An invalid parameter was specified. */ -LIBMDBX_API int mdbx_cursor_bind(const MDBX_txn *txn, MDBX_cursor *cursor, MDBX_dbi dbi); +LIBMDBX_API int mdbx_cursor_bind(MDBX_txn *txn, MDBX_cursor *cursor, MDBX_dbi dbi); /** \brief Unbind cursor from a transaction. * \ingroup c_cursors @@ -5200,7 +5200,7 @@ LIBMDBX_API int mdbx_cursor_reset(MDBX_cursor *cursor); * \retval MDBX_THREAD_MISMATCH Given transaction is not owned * by current thread. * \retval MDBX_EINVAL An invalid parameter was specified. */ -LIBMDBX_API int mdbx_cursor_open(const MDBX_txn *txn, MDBX_dbi dbi, MDBX_cursor **cursor); +LIBMDBX_API int mdbx_cursor_open(MDBX_txn *txn, MDBX_dbi dbi, MDBX_cursor **cursor); /** \brief Close a cursor handle. * \ingroup c_cursors @@ -5288,7 +5288,7 @@ LIBMDBX_INLINE_API(int, mdbx_txn_release_all_cursors, (const MDBX_txn *txn, bool * \retval MDBX_EINVAL An invalid parameter was specified. * \retval MDBX_BAD_DBI The cursor was not bound to a DBI-handle * or such a handle became invalid. */ -LIBMDBX_API int mdbx_cursor_renew(const MDBX_txn *txn, MDBX_cursor *cursor); +LIBMDBX_API int mdbx_cursor_renew(MDBX_txn *txn, MDBX_cursor *cursor); /** \brief Return the cursor's transaction handle. * \ingroup c_cursors diff --git a/mdbx.h++ b/mdbx.h++ index c0226157..e993f17f 100644 --- a/mdbx.h++ +++ b/mdbx.h++ @@ -4474,11 +4474,11 @@ public: /// \brief Renew/bind a cursor with a new transaction and previously used /// key-value map handle. - inline void renew(const ::mdbx::txn &txn); + inline void renew(::mdbx::txn &txn); /// \brief Bind/renew a cursor with a new transaction and specified key-value /// map handle. - inline void bind(const ::mdbx::txn &txn, ::mdbx::map_handle map_handle); + inline void bind(::mdbx::txn &txn, ::mdbx::map_handle map_handle); /// \brief Unbind cursor from a transaction. inline void unbind(); @@ -6166,9 +6166,9 @@ inline cursor::estimate_result cursor::estimate(move_operation operation) const return estimate_result(*this, operation); } -inline void cursor::renew(const ::mdbx::txn &txn) { error::success_or_throw(::mdbx_cursor_renew(txn, handle_)); } +inline void cursor::renew(::mdbx::txn &txn) { error::success_or_throw(::mdbx_cursor_renew(txn, handle_)); } -inline void cursor::bind(const ::mdbx::txn &txn, ::mdbx::map_handle map_handle) { +inline void cursor::bind(::mdbx::txn &txn, ::mdbx::map_handle map_handle) { error::success_or_throw(::mdbx_cursor_bind(txn, handle_, map_handle.dbi)); } diff --git a/src/api-cursor.c b/src/api-cursor.c index 1fefaf2a..9176b42c 100644 --- a/src/api-cursor.c +++ b/src/api-cursor.c @@ -23,7 +23,7 @@ MDBX_cursor *mdbx_cursor_create(void *context) { return &couple->outer; } -int mdbx_cursor_renew(const MDBX_txn *txn, MDBX_cursor *mc) { +int mdbx_cursor_renew(MDBX_txn *txn, MDBX_cursor *mc) { return likely(mc) ? mdbx_cursor_bind(txn, mc, (kvx_t *)mc->clc - txn->env->kvs) : LOG_IFERR(MDBX_EINVAL); } @@ -40,7 +40,7 @@ int mdbx_cursor_reset(MDBX_cursor *mc) { return MDBX_SUCCESS; } -int mdbx_cursor_bind(const MDBX_txn *txn, MDBX_cursor *mc, MDBX_dbi dbi) { +int mdbx_cursor_bind(MDBX_txn *txn, MDBX_cursor *mc, MDBX_dbi dbi) { if (unlikely(!mc)) return LOG_IFERR(MDBX_EINVAL); @@ -130,7 +130,7 @@ int mdbx_cursor_unbind(MDBX_cursor *mc) { return MDBX_SUCCESS; } -int mdbx_cursor_open(const MDBX_txn *txn, MDBX_dbi dbi, MDBX_cursor **ret) { +int mdbx_cursor_open(MDBX_txn *txn, MDBX_dbi dbi, MDBX_cursor **ret) { if (unlikely(!ret)) return LOG_IFERR(MDBX_EINVAL); *ret = nullptr; From 69895e2b552699c58cccf62d9db5c0e910e93378 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 00:57:17 +0300 Subject: [PATCH 48/90] =?UTF-8?q?mdbx-make:=20=D0=BF=D0=BE=D0=B8=D1=81?= =?UTF-8?q?=D0=BA=20`gnu-sed`=20=D0=BD=D0=B0=20Darwin/MacOS=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- GNUmakefile | 57 ++++++++++++++++++++++++++++++----------------------- 1 file changed, 32 insertions(+), 25 deletions(-) diff --git a/GNUmakefile b/GNUmakefile index 2d26a381..4b7aa689 100644 --- a/GNUmakefile +++ b/GNUmakefile @@ -164,6 +164,10 @@ else $(info $(TIP) Use `make V=1` for verbose.) endif +ifeq ($(UNAME),Darwin) + $(info $(TIP) Use `brew install gnu-sed gnu-tar` and add ones to the beginning of the PATH.) +endif + all: show-options $(LIBRARIES) $(MDBX_TOOLS) help: @@ -396,6 +400,9 @@ TEST_ITER := $(shell $(uname2titer)) TEST_SRC := test/osal-$(TEST_OSAL).c++ $(filter-out $(wildcard test/osal-*.c++),$(wildcard test/*.c++)) $(call select_by,MDBX_BUILD_CXX,,src/mdbx.c++) TEST_INC := $(wildcard test/*.h++) TEST_OBJ := $(patsubst %.c++,%.o,$(TEST_SRC)) +ifndef SED +SED := $(shell which gnu-sed 2>&- || echo sed) +endif TAR ?= $(shell which gnu-tar 2>&- || echo tar) ZIP ?= $(shell which zip || echo "echo 'Please install zip'") CLANG_FORMAT ?= $(shell (which clang-format-19 || which clang-format) 2>/dev/null) @@ -412,11 +419,11 @@ MAN_SRCDIR := src/man1/ ALLOY_DEPS := $(shell git ls-files src/ | grep -e /tools -e /man -v) MDBX_GIT_DIR := $(shell if [ -d .git ]; then echo .git; elif [ -s .git -a -f .git ]; then grep '^gitdir: ' .git | cut -d ':' -f 2; else echo git_directory_is_absent; fi) MDBX_GIT_LASTVTAG := $(shell git describe --tags --dirty=-DIRTY --abbrev=0 '--match=v[0-9]*' 2>&- || echo 'Please fetch tags and/or install non-obsolete git version') -MDBX_GIT_3DOT := $(shell set -o pipefail; echo "$(MDBX_GIT_LASTVTAG)" | sed -n 's|^v*\([0-9]\{1,\}\.[0-9]\{1,\}\.[0-9]\{1,\}\)\(.*\)|\1|p' || echo 'Please fetch tags and/or use non-obsolete git version') +MDBX_GIT_3DOT := $(shell set -o pipefail; echo "$(MDBX_GIT_LASTVTAG)" | $(SED) -n 's|^v*\([0-9]\{1,\}\.[0-9]\{1,\}\.[0-9]\{1,\}\)\(.*\)|\1|p' || echo 'Please fetch tags and/or use non-obsolete git version') MDBX_GIT_TWEAK := $(shell set -o pipefail; git rev-list $(shell git describe --tags --abbrev=0 '--match=v[0-9]*')..HEAD --count 2>&- || echo 'Please fetch tags and/or use non-obsolete git version') MDBX_GIT_TIMESTAMP := $(shell git show --no-patch --format=%cI HEAD 2>&- || echo 'Please install latest get version') MDBX_GIT_DESCRIBE := $(shell git describe --tags --long --dirty '--match=v[0-9]*' 2>&- || echo 'Please fetch tags and/or install non-obsolete git version') -MDBX_GIT_PRERELEASE := $(shell echo "$(MDBX_GIT_LASTVTAG)" | sed -n 's|^v*\([0-9]\{1,\}\.[0-9]\{1,\}\.[0-9]\{1,\}\)\(.*\)-\([-.0-1a-zA-Z]\+\)|\3|p') +MDBX_GIT_PRERELEASE := $(shell echo "$(MDBX_GIT_LASTVTAG)" | $(SED) -n 's|^v*\([0-9]\{1,\}\.[0-9]\{1,\}\.[0-9]\{1,\}\)\(.*\)-\([-.0-1a-zA-Z]\+\)|\3|p') MDBX_VERSION_PURE = $(MDBX_GIT_3DOT)$(if $(filter-out 0,$(MDBX_GIT_TWEAK)),.$(MDBX_GIT_TWEAK),)$(if $(MDBX_GIT_PRERELEASE),-$(MDBX_GIT_PRERELEASE),) MDBX_VERSION_IDENT = $(shell set -o pipefail; echo -n '$(MDBX_GIT_DESCRIBE)' | tr -c -s '[a-zA-Z0-9.]' _) MDBX_VERSION_NODOT = $(subst .,_,$(MDBX_VERSION_IDENT)) @@ -556,7 +563,7 @@ $(MDBX_GIT_DIR)/HEAD $(MDBX_GIT_DIR)/index $(MDBX_GIT_DIR)/refs/tags: src/version.c: src/version.c.in $(lastword $(MAKEFILE_LIST)) $(MDBX_GIT_DIR)/HEAD $(MDBX_GIT_DIR)/index $(MDBX_GIT_DIR)/refs/tags LICENSE NOTICE @echo ' MAKE $@' - $(QUIET)sed \ + $(QUIET)$(SED) \ -e "s|@MDBX_GIT_TIMESTAMP@|$(MDBX_GIT_TIMESTAMP)|" \ -e "s|@MDBX_GIT_TREE@|$(shell git show --no-patch --format=%T HEAD || echo 'Please install latest get version')|" \ -e "s|@MDBX_GIT_COMMIT@|$(shell git show --no-patch --format=%H HEAD || echo 'Please install latest get version')|" \ @@ -590,7 +597,7 @@ mdbx-static.o: src/config.h src/version.c src/alloy.c $(ALLOY_DEPS) $(lastword $ docs/Doxyfile: docs/Doxyfile.in src/version.c $(lastword $(MAKEFILE_LIST)) @echo ' MAKE $@' - $(QUIET)sed \ + $(QUIET)$(SED) \ -e "s|@MDBX_GIT_TIMESTAMP@|$(MDBX_GIT_TIMESTAMP)|" \ -e "s|@MDBX_GIT_TREE@|$(shell git show --no-patch --format=%T HEAD || echo 'Please install latest get version')|" \ -e "s|@MDBX_GIT_COMMIT@|$(shell git show --no-patch --format=%H HEAD || echo 'Please install latest get version')|" \ @@ -606,7 +613,7 @@ docs/Doxyfile: docs/Doxyfile.in src/version.c $(lastword $(MAKEFILE_LIST)) define md-extract-section docs/__$(1).md: $(2) $(lastword $(MAKEFILE_LIST)) @echo ' EXTRACT $1' - $(QUIET)sed -n '//,//p' $(2) >$$@ && test -s $$@ + $(QUIET)$(SED) -n '//,//p' $(2) >$$@ && test -s $$@ endef $(foreach section,overview mithril characteristics improvements history usage performance bindings,$(eval $(call md-extract-section,$(section),README.md))) @@ -621,16 +628,16 @@ docs/overall.md: docs/__overview.md docs/_toc.md docs/__mithril.md docs/__histor docs/intro.md: docs/_preface.md docs/__characteristics.md docs/__improvements.md docs/_restrictions.md docs/__performance.md @echo ' MAKE $@' - $(QUIET)cat $^ | sed 's/^Performance comparison$$/Performance comparison {#performance}/;s/^Improvements beyond LMDB$$/Improvements beyond LMDB {#improvements}/' >$@ + $(QUIET)cat $^ | $(SED) 's/^Performance comparison$$/Performance comparison {#performance}/;s/^Improvements beyond LMDB$$/Improvements beyond LMDB {#improvements}/' >$@ docs/usage.md: docs/__usage.md docs/_starting.md docs/__bindings.md @echo ' MAKE $@' - $(QUIET)echo -e "\\page usage Usage\n\\section getting Building & Embedding" | cat - $^ | sed 's/^Bindings$$/Bindings {#bindings}/' >$@ + $(QUIET)echo -e "\\page usage Usage\n\\section getting Building & Embedding" | cat - $^ | $(SED) 's/^Bindings$$/Bindings {#bindings}/' >$@ doxygen: docs/Doxyfile docs/overall.md docs/intro.md docs/usage.md mdbx.h mdbx.h++ src/options.h ChangeLog.md COPYRIGHT LICENSE NOTICE $(lastword $(MAKEFILE_LIST)) @echo ' RUNNING doxygen...' $(QUIET)rm -rf docs/html && \ - cat mdbx.h | tr '\n' '\r' | sed -e 's/LIBMDBX_INLINE_API\s*(\s*\([^,]\+\),\s*\([^,]\+\),\s*(\s*\([^)]\+\)\s*)\s*)\s*{/inline \1 \2(\3) {/g' | tr '\r' '\n' >docs/mdbx.h && \ + cat mdbx.h | tr '\n' '\r' | $(SED) -e 's/LIBMDBX_INLINE_API\s*(\s*\([^,]\+\),\s*\([^,]\+\),\s*(\s*\([^)]\+\)\s*)\s*)\s*{/inline \1 \2(\3) {/g' | tr '\r' '\n' >docs/mdbx.h && \ cp mdbx.h++ src/options.h ChangeLog.md docs/ && (cd docs && doxygen Doxyfile $(HUSH)) && cp COPYRIGHT LICENSE NOTICE docs/html/ mdbx++-dylib.o: src/config.h src/mdbx.c++ mdbx.h mdbx.h++ $(lastword $(MAKEFILE_LIST)) @@ -654,7 +661,7 @@ release-assets: libmdbx-amalgamated-$(MDBX_GIT_3DOT).zpaq \ libmdbx-amalgamated-$(MDBX_GIT_3DOT).tar.gz \ libmdbx-amalgamated-$(subst .,_,$(MDBX_GIT_3DOT)).zip $(QUIET)([ \ - "$$(set -o pipefail; git describe | sed -n '/^v[0-9]\{1,\}\.[0-9]\{1,\}\.[0-9]\{1,\}$$/p' || echo fail-left)" \ + "$$(set -o pipefail; git describe | $(SED) -n '/^v[0-9]\{1,\}\.[0-9]\{1,\}\.[0-9]\{1,\}$$/p' || echo fail-left)" \ == \ "$$(git describe --tags --dirty=-dirty || echo fail-right)" ] \ || (echo 'ERROR: Is not a valid release because not in the clean state with a suitable annotated tag!!!' >&2 && false)) \ @@ -691,7 +698,7 @@ $(DIST_DIR)/@tmp-essentials.inc: src/version.c $(ALLOY_DEPS) $(lastword $(MAKEFI @echo ' ALLOYING...' $(QUIET)mkdir -p dist \ && (grep -v '#include ' src/alloy.c && echo '#define MDBX_BUILD_SOURCERY $(MDBX_BUILD_SOURCERY)' \ - && sed \ + && $(SED) \ -e 's|#include "../mdbx.h"|@INCLUDE "mdbx.h"|' \ -e '/#include "preface.h"/r src/preface.h' \ -e '/#include "osal.h"/r src/osal.h' \ @@ -703,14 +710,14 @@ $(DIST_DIR)/@tmp-essentials.inc: src/version.c $(ALLOY_DEPS) $(lastword $(MAKEFI -e '/#include "utils.h"/r src/utils.h' \ -e '/#include "pnl.h"/r src/pnl.h' \ src/essentials.h \ - | sed \ + | $(SED) \ -e '/#pragma once/d' -e '/#include "/d' \ -e '/ clang-format o/d' -e '/ \*INDENT-O/d' \ | grep -v '^/// ') >$@ $(DIST_DIR)/@tmp-internals.inc: $(DIST_DIR)/@tmp-essentials.inc src/version.c $(ALLOY_DEPS) $(lastword $(MAKEFILE_LIST)) $(QUIET)(cat $(DIST_DIR)/@tmp-essentials.inc \ - && sed \ + && $(SED) \ -e '/#include "essentials.h"/d' \ -e '/#include "atomics-ops.h"/r src/atomics-ops.h' \ -e '/#include "proto.h"/r src/proto.h' \ @@ -732,22 +739,22 @@ $(DIST_DIR)/@tmp-internals.inc: $(DIST_DIR)/@tmp-essentials.inc src/version.c $( -e '/#include "walk.h"/r src/walk.h' \ -e '/#include "windows-import.h"/r src/windows-import.h' \ src/internals.h \ - | sed \ + | $(SED) \ -e '/#pragma once/d' -e '/#include "/d' \ -e '/ clang-format o/d' -e '/ \*INDENT-O/d' \ | grep -v '^/// ') >$@ $(DIST_DIR)/mdbx.c: $(DIST_DIR)/@tmp-internals.inc $(lastword $(MAKEFILE_LIST)) @echo ' MAKE $@' - $(QUIET)(cat $(DIST_DIR)/@tmp-internals.inc $(shell git ls-files src/*.c | grep -v alloy) src/version.c | sed \ + $(QUIET)(cat $(DIST_DIR)/@tmp-internals.inc $(shell git ls-files src/*.c | grep -v alloy) src/version.c | $(SED) \ -e '/#include "debug_begin.h"/r src/debug_begin.h' \ -e '/#include "debug_end.h"/r src/debug_end.h' \ - ) | sed -e '/#include "/d;/#pragma once/d' -e 's|@INCLUDE|#include|' \ + ) | $(SED) -e '/#include "/d;/#pragma once/d' -e 's|@INCLUDE|#include|' \ -e '/ clang-format o/d;/ \*INDENT-O/d' -e '3i /* clang-format off */' | cat -s >$@ $(DIST_DIR)/mdbx.c++: $(DIST_DIR)/@tmp-essentials.inc src/mdbx.c++ $(lastword $(MAKEFILE_LIST)) @echo ' MAKE $@' - $(QUIET)cat $(DIST_DIR)/@tmp-essentials.inc src/mdbx.c++ | sed \ + $(QUIET)cat $(DIST_DIR)/@tmp-essentials.inc src/mdbx.c++ | $(SED) \ -e '/#define xMDBX_ALLOY/d' \ -e '/#include "/d;/#pragma once/d' \ -e 's|@INCLUDE|#include|;s|"mdbx.h"|"mdbx.h++"|' \ @@ -757,12 +764,12 @@ define dist-tool-rule $(DIST_DIR)/mdbx_$(1).c: src/tools/$(1).c src/tools/wingetopt.h src/tools/wingetopt.c \ $(DIST_DIR)/@tmp-internals.inc $(lastword $(MAKEFILE_LIST)) @echo ' MAKE $$@' - $(QUIET)mkdir -p dist && sed \ + $(QUIET)mkdir -p dist && $(SED) \ -e '/#include "essentials.h"/r $(DIST_DIR)/@tmp-essentials.inc' \ -e '/#include "wingetopt.h"/r src/tools/wingetopt.c' \ -e '/ clang-format o/d' -e '/ \*INDENT-O/d' \ src/tools/$(1).c \ - | sed -e '/#include "/d;/#pragma once/d;/#define xMDBX_ALLOY/d' -e 's|@INCLUDE|#include|' \ + | $(SED) -e '/#include "/d;/#pragma once/d;/#define xMDBX_ALLOY/d' -e 's|@INCLUDE|#include|' \ -e '/ clang-format o/d;/ \*INDENT-O/d' -e '9i /* clang-format off */' | cat -s >$$@ endef @@ -771,7 +778,7 @@ $(foreach file,$(TOOLS),$(eval $(call dist-tool-rule,$(file)))) define dist-extra-rule $(DIST_DIR)/$(1): $(1) src/version.c $(lastword $(MAKEFILE_LIST)) @echo ' REFINE $$@' - $(QUIET)mkdir -p $$(dir $$@) && sed -e '/^#> dist-cutoff-begin/,/^#< dist-cutoff-end/d' $$< | cat -s >$$@ + $(QUIET)mkdir -p $$(dir $$@) && $(SED) -e '/^#> dist-cutoff-begin/,/^#< dist-cutoff-end/d' $$< | cat -s >$$@ endef $(foreach file,mdbx.h mdbx.h++ $(filter-out man1/% VERSION.json .clang-format-ignore %.in ntdll.def,$(DIST_EXTRA)),$(eval $(call dist-extra-rule,$(file)))) @@ -820,7 +827,7 @@ cross-gcc: @echo "FOR INSTANCE: sudo apt install \$$(apt list 'g++-*' | grep 'g++-[a-z0-9]\+-linux-gnu/' | cut -f 1 -d / | sort -u)" $(QUIET)for CC in $(CROSS_LIST_NOQEMU) $(CROSS_LIST); do \ echo "===================== $$CC"; \ - $(MAKE) IOARENA=false CXXSTD= clean && CC=$$CC CXX=$$(echo $$CC | sed 's/-gcc/-g++/') EXE_LDFLAGS=-static $(MAKE) IOARENA=false all || exit $$?; \ + $(MAKE) IOARENA=false CXXSTD= clean && CC=$$CC CXX=$$(echo $$CC | $(SED) 's/-gcc/-g++/') EXE_LDFLAGS=-static $(MAKE) IOARENA=false all || exit $$?; \ done # Unfortunately qemu don't provide robust support for futexes. @@ -834,7 +841,7 @@ cross-qemu: $(QUIET)for CC in $(CROSS_LIST); do \ echo "===================== $$CC + qemu"; \ $(MAKE) IOARENA=false CXXSTD= clean && \ - CC=$$CC CXX=$$(echo $$CC | sed 's/-gcc/-g++/') EXE_LDFLAGS=-static MDBX_BUILD_OPTIONS="-DMDBX_SAFE4QEMU $(MDBX_BUILD_OPTIONS)" \ + CC=$$CC CXX=$$(echo $$CC | $(SED) 's/-gcc/-g++/') EXE_LDFLAGS=-static MDBX_BUILD_OPTIONS="-DMDBX_SAFE4QEMU $(MDBX_BUILD_OPTIONS)" \ $(MAKE) IOARENA=false smoke-singleprocess test-singleprocess || exit $$?; \ done @@ -901,13 +908,13 @@ bench-$(1)_$(2).txt: $(3) $(IOARENA) $(lastword $(MAKEFILE_LIST)) $(QUIET)(export LD_LIBRARY_PATH="./:$$$${LD_LIBRARY_PATH}"; \ ldd $(IOARENA) | grep -i $(1) && \ $(IOARENA) -D $(1) -B batch -m $(BENCH_CRUD_MODE) -n $(2) \ - | tee $$@ | grep throughput | sed 's/throughput/batch×N/' && \ + | tee $$@ | grep throughput | $(SED) 's/throughput/batch×N/' && \ $(IOARENA) -D $(1) -B crud -m $(BENCH_CRUD_MODE) -n $(2) \ - | tee -a $$@ | grep throughput | sed 's/throughput/ crud/' && \ + | tee -a $$@ | grep throughput | $(SED) 's/throughput/ crud/' && \ $(IOARENA) -D $(1) -B iterate,get,iterate,get,iterate -m $(BENCH_CRUD_MODE) -r 4 -n $(2) \ - | tee -a $$@ | grep throughput | sed '0,/throughput/{s/throughput/iterate/};s/throughput/ get/' && \ + | tee -a $$@ | grep throughput | $(SED) '0,/throughput/{s/throughput/iterate/};s/throughput/ get/' && \ $(IOARENA) -D $(1) -B delete -m $(BENCH_CRUD_MODE) -n $(2) \ - | tee -a $$@ | grep throughput | sed 's/throughput/ delete/' && \ + | tee -a $$@ | grep throughput | $(SED) 's/throughput/ delete/' && \ true) || mv -f $$@ $$@.error endef From f9d7eb552547948342657e706d7f9e27306ebc12 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:01:05 +0300 Subject: [PATCH 49/90] =?UTF-8?q?mdbx-doc:=20=D0=B0=D0=BA=D1=82=D1=83?= =?UTF-8?q?=D0=B0=D0=BB=D0=B8=D0=B7=D0=B0=D1=86=D0=B8=D1=8F=20=D1=80=D0=B0?= =?UTF-8?q?=D0=B7=D0=B4=D0=B5=D0=BB=D0=B0=20MacOS=20=D0=B2=20README=20(bac?= =?UTF-8?q?kport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 69f67b9b..d3cd1c1b 100644 --- a/README.md +++ b/README.md @@ -651,16 +651,19 @@ error when opening the database in a _WSL1_ environment. ### MacOS Current [native build tools](https://en.wikipedia.org/wiki/Xcode) for MacOS include GNU Make, CLANG and an outdated version of Bash. -Therefore, to build the library, it is enough to run `make all` in the +However, the build script uses GNU-kind of `sed` and `tar`. +So the easiest way to install all prerequirements is to use [Homebrew](https://brew.sh/), +just by `brew install bash make cmake ninja gnu-sed gnu-tar --with-default-names`. + +Next, to build the library, it is enough to run `make all` in the directory with source code, and run `make check` to execute the base tests. If something goes wrong, it is recommended to install [Homebrew](https://brew.sh/) and try again. To run the [long stochastic test scenario](test/stochastic.sh), you will need to install the current (not outdated) version of -[Bash](https://en.wikipedia.org/wiki/Bash_(Unix_shell)). To do this, I -recommend that you install [Homebrew](https://brew.sh/) and then execute -`brew install bash`. +[Bash](https://en.wikipedia.org/wiki/Bash_(Unix_shell)). +Just install it as noted above. ### Android I recommend using CMake to build _libmdbx_ for Android. From 175e4a2e1bc567c0d4d613298f979ba2befa919a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:06:56 +0300 Subject: [PATCH 50/90] =?UTF-8?q?mdbx:=20=D0=BF=D1=80=D0=BE=D0=B2=D0=B5?= =?UTF-8?q?=D1=80=D0=BA=D0=B0=20=D0=B2=D0=BB=D0=B0=D0=B4=D0=B5=D0=BB=D1=8C?= =?UTF-8?q?=D1=86=D0=B0=20=D0=BF=D0=BE=D1=82=D0=BE=D0=BA=D0=B0=20=D0=B2?= =?UTF-8?q?=D0=BB=D0=B0=D0=B4=D0=B5=D1=8E=D1=89=D0=B5=D0=B3=D0=BE=20=D1=82?= =?UTF-8?q?=D1=80=D0=B0=D0=BD=D0=B7=D0=B0=D0=BA=D1=86=D0=B8=D0=B5=D0=B9=20?= =?UTF-8?q?=D1=82=D0=BE=D0=BB=D1=8C=D0=BA=D0=BE=20=D0=BF=D1=80=D0=B8=20`MD?= =?UTF-8?q?BX=5FTXN=5FCHECKOWNER=3DON`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/api-txn.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api-txn.c b/src/api-txn.c index 2fe01a4e..786e1e42 100644 --- a/src/api-txn.c +++ b/src/api-txn.c @@ -423,7 +423,7 @@ int mdbx_txn_commit_ex(MDBX_txn *txn, MDBX_commit_latency *latency) { } #if MDBX_TXN_CHECKOWNER - if (!txn->parent && (txn->flags & MDBX_NOSTICKYTHREADS) && unlikely(txn->owner != osal_thread_self())) { + if ((txn->flags & MDBX_NOSTICKYTHREADS) && txn == env->basal_txn && unlikely(txn->owner != osal_thread_self())) { txn->flags |= MDBX_TXN_ERROR; rc = MDBX_THREAD_MISMATCH; return LOG_IFERR(rc); From b6dcdcf2dc908852a819f9abc36c456d92057412 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:10:11 +0300 Subject: [PATCH 51/90] =?UTF-8?q?mdbx:=20=D0=B7=D0=B0=D0=BF=D1=80=D0=B5?= =?UTF-8?q?=D1=89=D0=B5=D0=BD=D0=B8=D0=B5=20unbind/close=20=D0=BA=D1=83?= =?UTF-8?q?=D1=80=D1=81=D0=BE=D1=80=D0=BE=D0=B2=20=D0=B4=D0=BB=D1=8F=20?= =?UTF-8?q?=D0=B2=D0=BB=D0=BE=D0=B6=D0=B5=D0=BD=D0=BD=D1=8B=D1=85=20=D1=82?= =?UTF-8?q?=D1=80=D0=B0=D0=BD=D0=B7=D0=B0=D0=BA=D1=86=D0=B8=D0=B9=20(backp?= =?UTF-8?q?ort).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- mdbx.h | 20 ++++++++++-- src/api-cursor.c | 79 +++++++++++++++++++++--------------------------- 2 files changed, 53 insertions(+), 46 deletions(-) diff --git a/mdbx.h b/mdbx.h index 728e6eb0..559c664b 100644 --- a/mdbx.h +++ b/mdbx.h @@ -5116,6 +5116,10 @@ MDBX_NOTHROW_PURE_FUNCTION LIBMDBX_API void *mdbx_cursor_get_userctx(const MDBX_ * the same table handle as it was created with. This may be done whether the * previous transaction is live or dead. * + * If the transaction is nested, then the cursor should not be used in its parent transaction. + * Otherwise it is no way to restore state if this nested transaction will be aborted, + * nor impossible to define the expected behavior. + * * \note In contrast to LMDB, the MDBX required that any opened cursors can be * reused and must be freed explicitly, regardless ones was opened in a * read-only or write transaction. The REASON for this is eliminates ambiguity @@ -5140,6 +5144,10 @@ LIBMDBX_API int mdbx_cursor_bind(MDBX_txn *txn, MDBX_cursor *cursor, MDBX_dbi db * the original DBI-handle internally. Thus it could be renewed with any running * transaction or closed. * + * If the transaction is nested, then the cursor should not be used in its parent transaction. + * Otherwise it is no way to restore state if this nested transaction will be aborted, + * nor impossible to define the expected behavior. + * * \see mdbx_cursor_renew() * \see mdbx_cursor_bind() * \see mdbx_cursor_close() @@ -5224,6 +5232,10 @@ LIBMDBX_API void mdbx_cursor_close(MDBX_cursor *cursor); * Unbinds either closes all cursors associated (opened or renewed) with * a given transaction in a bulk with minimal overhead. * + * A transaction should not be nested, since in this case no way to restore + * state if this nested transaction will be aborted, nor impossible to define + * the expected behavior. + * * \see mdbx_cursor_unbind() * \see mdbx_cursor_close() * @@ -5237,7 +5249,7 @@ LIBMDBX_API void mdbx_cursor_close(MDBX_cursor *cursor); * some possible errors are: * \retval MDBX_THREAD_MISMATCH Given transaction is not owned * by current thread. - * \retval MDBX_BAD_TXN Given transaction is invalid or has + * \retval MDBX_BAD_TXN Given transaction is invalid, nested or has * a child/nested transaction transaction. */ LIBMDBX_API int mdbx_txn_release_all_cursors_ex(const MDBX_txn *txn, bool unbind, size_t *count); @@ -5247,6 +5259,10 @@ LIBMDBX_API int mdbx_txn_release_all_cursors_ex(const MDBX_txn *txn, bool unbind * Unbinds either closes all cursors associated (opened or renewed) with * a given transaction in a bulk with minimal overhead. * + * A transaction should not be nested, since in this case no way to restore + * state if this nested transaction will be aborted, nor impossible to define + * the expected behavior. + * * \see mdbx_cursor_unbind() * \see mdbx_cursor_close() * @@ -5258,7 +5274,7 @@ LIBMDBX_API int mdbx_txn_release_all_cursors_ex(const MDBX_txn *txn, bool unbind * some possible errors are: * \retval MDBX_THREAD_MISMATCH Given transaction is not owned * by current thread. - * \retval MDBX_BAD_TXN Given transaction is invalid or has + * \retval MDBX_BAD_TXN Given transaction is invalid, nested or has * a child/nested transaction transaction. */ LIBMDBX_INLINE_API(int, mdbx_txn_release_all_cursors, (const MDBX_txn *txn, bool unbind)) { return mdbx_txn_release_all_cursors_ex(txn, unbind, NULL); diff --git a/src/api-cursor.c b/src/api-cursor.c index 9176b42c..ec87e4bd 100644 --- a/src/api-cursor.c +++ b/src/api-cursor.c @@ -44,8 +44,10 @@ int mdbx_cursor_bind(MDBX_txn *txn, MDBX_cursor *mc, MDBX_dbi dbi) { if (unlikely(!mc)) return LOG_IFERR(MDBX_EINVAL); - if (unlikely(mc->signature != cur_signature_ready4dispose && mc->signature != cur_signature_live)) - return LOG_IFERR(MDBX_EBADSIGN); + if (unlikely(mc->signature != cur_signature_ready4dispose && mc->signature != cur_signature_live)) { + int rc = (mc->signature == cur_signature_wait4eot) ? MDBX_EINVAL : MDBX_EBADSIGN; + return LOG_IFERR(rc); + } int rc = check_txn(txn, MDBX_TXN_BLOCKED); if (unlikely(rc != MDBX_SUCCESS)) @@ -58,24 +60,12 @@ int mdbx_cursor_bind(MDBX_txn *txn, MDBX_cursor *mc, MDBX_dbi dbi) { if (unlikely(dbi == FREE_DBI && !(txn->flags & MDBX_TXN_RDONLY))) return LOG_IFERR(MDBX_EACCESS); - if (unlikely(mc->backup)) /* Cursor from parent transaction */ { - cASSERT(mc, mc->signature == cur_signature_live); - if (unlikely(cursor_dbi(mc) != dbi || - /* paranoia */ mc->signature != cur_signature_live || mc->txn != txn)) - return LOG_IFERR(MDBX_EINVAL); - - cASSERT(mc, mc->tree == &txn->dbs[dbi]); - cASSERT(mc, mc->clc == &txn->env->kvs[dbi].clc); - cASSERT(mc, cursor_dbi(mc) == dbi); - return likely(cursor_dbi(mc) == dbi && - /* paranoia */ mc->signature == cur_signature_live && mc->txn == txn) - ? MDBX_SUCCESS - : LOG_IFERR(MDBX_EINVAL) /* Disallow change DBI in nested - transactions */ - ; - } + if (unlikely(mc->backup)) /* Cursor from parent transaction */ + LOG_IFERR(MDBX_EINVAL); if (mc->signature == cur_signature_live) { + if (mc->txn == txn && cursor_dbi(mc) == dbi) + return MDBX_SUCCESS; rc = mdbx_cursor_unbind(mc); if (unlikely(rc != MDBX_SUCCESS)) return rc; @@ -99,25 +89,22 @@ int mdbx_cursor_unbind(MDBX_cursor *mc) { return (mc->signature == cur_signature_ready4dispose) ? MDBX_SUCCESS : LOG_IFERR(MDBX_EBADSIGN); if (unlikely(mc->backup)) /* Cursor from parent transaction */ + /* TODO: реализовать при переходе на двусвязный список курсоров */ return LOG_IFERR(MDBX_EINVAL); - eASSERT(nullptr, mc->txn && mc->txn->signature == txn_signature); - cASSERT(mc, mc->signature == cur_signature_live); - cASSERT(mc, !mc->backup); if (unlikely(!mc->txn || mc->txn->signature != txn_signature)) { ERROR("Wrong cursor's transaction %p 0x%x", __Wpedantic_format_voidptr(mc->txn), mc->txn ? mc->txn->signature : 0); return LOG_IFERR(MDBX_PROBLEM); } + if (mc->next != mc) { - const size_t dbi = (kvx_t *)mc->clc - mc->txn->env->kvs; - cASSERT(mc, cursor_dbi(mc) == dbi); + const size_t dbi = cursor_dbi(mc); cASSERT(mc, dbi < mc->txn->n_dbi); + cASSERT(mc, &mc->txn->env->kvs[dbi].clc == mc->clc); if (dbi < mc->txn->n_dbi) { MDBX_cursor **prev = &mc->txn->cursors[dbi]; - while (*prev) { + while (/* *prev && */ *prev != mc) { ENSURE(mc->txn->env, (*prev)->signature == cur_signature_live || (*prev)->signature == cur_signature_wait4eot); - if (*prev == mc) - break; prev = &(*prev)->next; } cASSERT(mc, *prev == mc); @@ -218,30 +205,34 @@ again: int mdbx_txn_release_all_cursors_ex(const MDBX_txn *txn, bool unbind, size_t *count) { int rc = check_txn(txn, MDBX_TXN_FINISHED | MDBX_TXN_HAS_CHILD); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); + if (unlikely(txn->parent)) { + rc = MDBX_BAD_TXN; + ERROR("%s, err %d", "must not unbind or close cursors for a nested txn", rc); + return rc; + } + size_t n = 0; - if (likely(rc == MDBX_SUCCESS)) { - TXN_FOREACH_DBI_FROM(txn, i, MAIN_DBI) { - while (txn->cursors[i]) { - ++n; - MDBX_cursor *mc = txn->cursors[i]; - ENSURE(nullptr, mc->signature == cur_signature_live && (mc->next != mc) && !mc->backup); - txn->cursors[i] = mc->next; - mc->next = mc; - if (unbind) { - be_poor(mc); - mc->signature = cur_signature_ready4dispose; - } else { - mc->signature = 0; - osal_free(mc); - } + TXN_FOREACH_DBI_FROM(txn, i, MAIN_DBI) { + while (txn->cursors[i]) { + ++n; + MDBX_cursor *mc = txn->cursors[i]; + ENSURE(nullptr, mc->signature == cur_signature_live && (mc->next != mc) && !mc->backup); + txn->cursors[i] = mc->next; + mc->next = mc; + if (unbind) { + be_poor(mc); + mc->signature = cur_signature_ready4dispose; + } else { + mc->signature = 0; + osal_free(mc); } } - } else { - LOG_IFERR(rc); } if (count) *count = n; - return rc; + return MDBX_SUCCESS; } int mdbx_cursor_compare(const MDBX_cursor *l, const MDBX_cursor *r, bool ignore_multival) { From 2b6a7687508e10cdf345e51df7f70661067a0d69 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:12:15 +0300 Subject: [PATCH 52/90] =?UTF-8?q?mdbx:=20=D0=BA=D0=BE=D1=81=D0=BC=D0=B5?= =?UTF-8?q?=D1=82=D0=B8=D1=87=D0=B5=D1=81=D0=BA=D0=B8=D0=B9=20=D1=80=D0=B5?= =?UTF-8?q?=D1=84=D0=B0=D0=BA=D1=82=D0=BE=D1=80=D0=B8=D0=BD=D0=B3=20`curso?= =?UTF-8?q?r=5Fshadow()`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/cursor.c | 40 ++++++++++++++++++---------------------- src/cursor.h | 2 +- 2 files changed, 19 insertions(+), 23 deletions(-) diff --git a/src/cursor.c b/src/cursor.c index f24677f2..4667652e 100644 --- a/src/cursor.c +++ b/src/cursor.c @@ -184,14 +184,13 @@ __hot int cursor_touch(MDBX_cursor *const mc, const MDBX_val *key, const MDBX_va /*----------------------------------------------------------------------------*/ -int cursor_shadow(MDBX_cursor *parent_cursor, MDBX_txn *nested_txn, const size_t dbi) { - - tASSERT(nested_txn, dbi > FREE_DBI && dbi < nested_txn->n_dbi); - const size_t size = parent_cursor->subcur ? sizeof(MDBX_cursor) + sizeof(subcur_t) : sizeof(MDBX_cursor); - for (MDBX_cursor *bk; parent_cursor; parent_cursor = bk->next) { - cASSERT(parent_cursor, parent_cursor != parent_cursor->next); - bk = parent_cursor; - if (parent_cursor->signature != cur_signature_live) +int cursor_shadow(MDBX_cursor *mc, MDBX_txn *nested, const size_t dbi) { + tASSERT(nested, dbi > FREE_DBI && dbi < nested->n_dbi); + const size_t size = mc->subcur ? sizeof(MDBX_cursor) + sizeof(subcur_t) : sizeof(MDBX_cursor); + for (MDBX_cursor *bk; mc; mc = bk->next) { + cASSERT(mc, mc != mc->next); + bk = mc; + if (mc->signature != cur_signature_live) continue; bk = osal_malloc(size); if (unlikely(!bk)) @@ -200,22 +199,19 @@ int cursor_shadow(MDBX_cursor *parent_cursor, MDBX_txn *nested_txn, const size_t memset(bk, 0xCD, size); VALGRIND_MAKE_MEM_UNDEFINED(bk, size); #endif /* MDBX_DEBUG */ - *bk = *parent_cursor; - parent_cursor->backup = bk; - /* Kill pointers into src to reduce abuse: The - * user may not use mc until dst ends. But we need a valid - * txn pointer here for cursor fixups to keep working. */ - parent_cursor->txn = nested_txn; - parent_cursor->tree = &nested_txn->dbs[dbi]; - parent_cursor->dbi_state = &nested_txn->dbi_state[dbi]; - subcur_t *mx = parent_cursor->subcur; - if (mx != nullptr) { + *bk = *mc; + mc->backup = bk; + mc->txn = nested; + mc->tree = &nested->dbs[dbi]; + mc->dbi_state = &nested->dbi_state[dbi]; + subcur_t *mx = mc->subcur; + if (mx) { *(subcur_t *)(bk + 1) = *mx; - mx->cursor.txn = nested_txn; - mx->cursor.dbi_state = parent_cursor->dbi_state; + mx->cursor.txn = nested; + mx->cursor.dbi_state = &nested->dbi_state[dbi]; } - parent_cursor->next = nested_txn->cursors[dbi]; - nested_txn->cursors[dbi] = parent_cursor; + mc->next = nested->cursors[dbi]; + nested->cursors[dbi] = mc; } return MDBX_SUCCESS; } diff --git a/src/cursor.h b/src/cursor.h index c2de019b..6d984084 100644 --- a/src/cursor.h +++ b/src/cursor.h @@ -293,7 +293,7 @@ MDBX_NOTHROW_PURE_FUNCTION static inline bool check_leaf_type(const MDBX_cursor } MDBX_INTERNAL void cursor_eot(MDBX_cursor *mc, const bool merge); -MDBX_INTERNAL int cursor_shadow(MDBX_cursor *parent_cursor, MDBX_txn *nested_txn, const size_t dbi); +MDBX_INTERNAL int cursor_shadow(MDBX_cursor *mc, MDBX_txn *nested, const size_t dbi); MDBX_INTERNAL MDBX_cursor *cursor_cpstk(const MDBX_cursor *csrc, MDBX_cursor *cdst); From 2476fba287793c49cbdd6963cb88b9cf9d6fc5bf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:12:59 +0300 Subject: [PATCH 53/90] =?UTF-8?q?mdbx:=20=D1=80=D0=B5=D1=84=D0=B0=D0=BA?= =?UTF-8?q?=D1=82=D0=BE=D1=80=D0=B8=D0=BD=D0=B3=20`cursor=5Feot()`=20?= =?UTF-8?q?=D0=B4=D0=BB=D1=8F=20=D1=83=D0=BF=D1=80=D0=BE=D1=89=D0=B5=D0=BD?= =?UTF-8?q?=D0=B8=D1=8F=20`txn=5Fdone=5Fcursors()`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/cursor.c | 21 ++++++++++++--------- src/cursor.h | 2 +- src/dbi.h | 2 +- src/txn.c | 8 +++----- 4 files changed, 17 insertions(+), 16 deletions(-) diff --git a/src/cursor.c b/src/cursor.c index 4667652e..bf12cd66 100644 --- a/src/cursor.c +++ b/src/cursor.c @@ -216,17 +216,21 @@ int cursor_shadow(MDBX_cursor *mc, MDBX_txn *nested, const size_t dbi) { return MDBX_SUCCESS; } -void cursor_eot(MDBX_cursor *mc, const bool merge) { +MDBX_cursor *cursor_eot(MDBX_cursor *mc, MDBX_txn *txn, const bool merge) { + MDBX_cursor *const next = mc->next; + mc->next = mc; const unsigned stage = mc->signature; MDBX_cursor *const bk = mc->backup; - ENSURE(mc->txn->env, stage == cur_signature_live || (stage == cur_signature_wait4eot && bk)); + ENSURE(txn->env, stage == cur_signature_live || (stage == cur_signature_wait4eot && bk)); + tASSERT(txn, mc->txn == txn); if (bk) { subcur_t *mx = mc->subcur; - cASSERT(mc, mc->txn->parent != nullptr); + tASSERT(txn, mc->txn->parent != nullptr); + tASSERT(txn, bk->txn == txn->parent); /* Zap: Using uninitialized memory '*mc->backup'. */ MDBX_SUPPRESS_GOOFY_MSVC_ANALYZER(6001); ENSURE(mc->txn->env, bk->signature == cur_signature_live); - cASSERT(mc, mx == bk->subcur); + tASSERT(txn, mx == bk->subcur); if (merge) { /* Update pointers to parent txn */ mc->next = bk->next; @@ -235,24 +239,23 @@ void cursor_eot(MDBX_cursor *mc, const bool merge) { mc->tree = bk->tree; mc->dbi_state = bk->dbi_state; if (mx) { - mx->cursor.txn = mc->txn; - mx->cursor.dbi_state = mc->dbi_state; + mx->cursor.txn = bk->txn; + mx->cursor.dbi_state = bk->dbi_state; } } else { /* Restore from backup, i.e. rollback/abort nested txn */ *mc = *bk; + mc->signature = stage /* Promote (cur_signature_wait4eot) state to parent txn */; if (mx) *mx = *(subcur_t *)(bk + 1); } - if (stage == cur_signature_wait4eot /* Cursor was closed by user */) - mc->signature = stage /* Promote closed state to parent txn */; bk->signature = 0; osal_free(bk); } else { ENSURE(mc->txn->env, stage == cur_signature_live); mc->signature = cur_signature_ready4dispose /* Cursor may be reused */; - mc->next = mc; } + return next; } /*----------------------------------------------------------------------------*/ diff --git a/src/cursor.h b/src/cursor.h index 6d984084..cfaca52e 100644 --- a/src/cursor.h +++ b/src/cursor.h @@ -292,7 +292,7 @@ MDBX_NOTHROW_PURE_FUNCTION static inline bool check_leaf_type(const MDBX_cursor return (((page_type(mp) ^ mc->checking) & (z_branch | z_leaf | z_largepage | z_dupfix)) == 0); } -MDBX_INTERNAL void cursor_eot(MDBX_cursor *mc, const bool merge); +MDBX_INTERNAL MDBX_cursor *cursor_eot(MDBX_cursor *mc, MDBX_txn *txn, const bool merge); MDBX_INTERNAL int cursor_shadow(MDBX_cursor *mc, MDBX_txn *nested, const size_t dbi); MDBX_INTERNAL MDBX_cursor *cursor_cpstk(const MDBX_cursor *csrc, MDBX_cursor *cdst); diff --git a/src/dbi.h b/src/dbi.h index e9b16efe..2dffd5f7 100644 --- a/src/dbi.h +++ b/src/dbi.h @@ -101,7 +101,7 @@ static inline bool dbi_changed(const MDBX_txn *txn, const size_t dbi) { const MDBX_env *const env = txn->env; eASSERT(env, dbi_state(txn, dbi) & DBI_LINDO); const uint32_t snap_seq = atomic_load32(&env->dbi_seqs[dbi], mo_AcquireRelease); - return snap_seq != txn->dbi_seqs[dbi]; + return unlikely(snap_seq != txn->dbi_seqs[dbi]); } static inline int dbi_check(const MDBX_txn *txn, const size_t dbi) { diff --git a/src/txn.c b/src/txn.c index eb198965..a83cd9c8 100644 --- a/src/txn.c +++ b/src/txn.c @@ -13,11 +13,9 @@ void txn_done_cursors(MDBX_txn *txn, const bool merge) { MDBX_cursor *mc = txn->cursors[i]; if (mc) { txn->cursors[i] = nullptr; - do { - MDBX_cursor *const next = mc->next; - cursor_eot(mc, merge); - mc = next; - } while (mc); + do + mc = cursor_eot(mc, txn, merge); + while (mc); } } } From 33ceba0a5afb679bce2b671a0a86c2d61b97d353 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:13:44 +0300 Subject: [PATCH 54/90] =?UTF-8?q?mdbx:=20=D0=B4=D0=BE=D0=B1=D0=B0=D0=B2?= =?UTF-8?q?=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20`cursor=5Freset()`=20=D0=B8=20?= =?UTF-8?q?`cursor=5Fdrown()`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/api-cursor.c | 26 +++++++++++++------------- src/cursor.c | 3 ++- src/cursor.h | 16 ++++++++++++++++ 3 files changed, 31 insertions(+), 14 deletions(-) diff --git a/src/api-cursor.c b/src/api-cursor.c index ec87e4bd..cfe935f1 100644 --- a/src/api-cursor.c +++ b/src/api-cursor.c @@ -12,8 +12,7 @@ MDBX_cursor *mdbx_cursor_create(void *context) { couple->outer.signature = cur_signature_ready4dispose; couple->outer.next = &couple->outer; couple->userctx = context; - couple->outer.top_and_flags = z_poor_mark; - couple->inner.cursor.top_and_flags = z_poor_mark | z_inner; + cursor_reset(couple); VALGRIND_MAKE_MEM_DEFINED(&couple->outer.backup, sizeof(couple->outer.backup)); VALGRIND_MAKE_MEM_DEFINED(&couple->outer.tree, sizeof(couple->outer.tree)); VALGRIND_MAKE_MEM_DEFINED(&couple->outer.clc, sizeof(couple->outer.clc)); @@ -31,13 +30,15 @@ int mdbx_cursor_reset(MDBX_cursor *mc) { if (unlikely(!mc)) return LOG_IFERR(MDBX_EINVAL); - if (unlikely(mc->signature != cur_signature_ready4dispose && mc->signature != cur_signature_live)) - return LOG_IFERR(MDBX_EBADSIGN); + if (likely(mc->signature == cur_signature_live)) { + cursor_reset((cursor_couple_t *)mc); + return MDBX_SUCCESS; + } - cursor_couple_t *couple = (cursor_couple_t *)mc; - couple->outer.top_and_flags = z_poor_mark; - couple->inner.cursor.top_and_flags = z_poor_mark | z_inner; - return MDBX_SUCCESS; + if (likely(mc->signature == cur_signature_ready4dispose)) + return MDBX_SUCCESS; + + return LOG_IFERR(MDBX_EBADSIGN); } int mdbx_cursor_bind(MDBX_txn *txn, MDBX_cursor *mc, MDBX_dbi dbi) { @@ -112,7 +113,7 @@ int mdbx_cursor_unbind(MDBX_cursor *mc) { } mc->next = mc; } - be_poor(mc); + cursor_drown((cursor_couple_t *)mc); mc->signature = cur_signature_ready4dispose; return MDBX_SUCCESS; } @@ -221,10 +222,9 @@ int mdbx_txn_release_all_cursors_ex(const MDBX_txn *txn, bool unbind, size_t *co ENSURE(nullptr, mc->signature == cur_signature_live && (mc->next != mc) && !mc->backup); txn->cursors[i] = mc->next; mc->next = mc; - if (unbind) { - be_poor(mc); - mc->signature = cur_signature_ready4dispose; - } else { + mc->signature = cur_signature_ready4dispose; + cursor_drown((cursor_couple_t *)mc); + if (!unbind) { mc->signature = 0; osal_free(mc); } diff --git a/src/cursor.c b/src/cursor.c index bf12cd66..73da6a8b 100644 --- a/src/cursor.c +++ b/src/cursor.c @@ -218,7 +218,6 @@ int cursor_shadow(MDBX_cursor *mc, MDBX_txn *nested, const size_t dbi) { MDBX_cursor *cursor_eot(MDBX_cursor *mc, MDBX_txn *txn, const bool merge) { MDBX_cursor *const next = mc->next; - mc->next = mc; const unsigned stage = mc->signature; MDBX_cursor *const bk = mc->backup; ENSURE(txn->env, stage == cur_signature_live || (stage == cur_signature_wait4eot && bk)); @@ -253,6 +252,8 @@ MDBX_cursor *cursor_eot(MDBX_cursor *mc, MDBX_txn *txn, const bool merge) { osal_free(bk); } else { ENSURE(mc->txn->env, stage == cur_signature_live); + cursor_drown((cursor_couple_t *)mc); + mc->next = mc; mc->signature = cur_signature_ready4dispose /* Cursor may be reused */; } return next; diff --git a/src/cursor.h b/src/cursor.h index cfaca52e..d06bad3a 100644 --- a/src/cursor.h +++ b/src/cursor.h @@ -355,3 +355,19 @@ MDBX_MAYBE_UNUSED static inline void cursor_inner_refresh(const MDBX_cursor *mc, } MDBX_MAYBE_UNUSED MDBX_INTERNAL bool cursor_is_tracked(const MDBX_cursor *mc); + +static inline void cursor_reset(cursor_couple_t *couple) { + couple->outer.top_and_flags = z_fresh_mark; + couple->inner.cursor.top_and_flags = z_fresh_mark | z_inner; +} + +static inline void cursor_drown(cursor_couple_t *couple) { + couple->outer.top_and_flags = z_poor_mark; + couple->inner.cursor.top_and_flags = z_poor_mark | z_inner; + couple->outer.txn = nullptr; + couple->inner.cursor.txn = nullptr; + couple->outer.tree = nullptr; + /* сохраняем clc-указатель, так он используется для вычисления dbi в mdbx_cursor_renew(). */ + couple->outer.dbi_state = nullptr; + couple->inner.cursor.dbi_state = nullptr; +} From 753b2270fdfc8df36a23c82e2e2707c7357de955 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:14:34 +0300 Subject: [PATCH 55/90] =?UTF-8?q?mdbx:=20=D0=B4=D0=BE=D0=B1=D0=B0=D0=B2?= =?UTF-8?q?=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20`mdbx=5Fcursor=5Fclose2()`=20?= =?UTF-8?q?=D0=B2=20API=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- mdbx.h | 30 +++++++++++++++- mdbx.h++ | 5 ++- src/api-cursor.c | 91 ++++++++++++++++++++++++++++++------------------ src/cursor.c | 4 +-- src/mdbx.c++ | 9 ----- 5 files changed, 93 insertions(+), 46 deletions(-) diff --git a/mdbx.h b/mdbx.h index 559c664b..e8e5748b 100644 --- a/mdbx.h +++ b/mdbx.h @@ -5210,12 +5210,17 @@ LIBMDBX_API int mdbx_cursor_reset(MDBX_cursor *cursor); * \retval MDBX_EINVAL An invalid parameter was specified. */ LIBMDBX_API int mdbx_cursor_open(MDBX_txn *txn, MDBX_dbi dbi, MDBX_cursor **cursor); -/** \brief Close a cursor handle. +/** \brief Closes a cursor handle without returning error code. * \ingroup c_cursors * * The cursor handle will be freed and must not be used again after this call, * but its transaction may still be live. * + * This function returns `void` but panic in case of error. Use \ref mdbx_cursor_close2() + * if you need to receive an error code instead of an app crash. + * + * \see mdbx_cursor_close2 + * * \note In contrast to LMDB, the MDBX required that any opened cursors can be * reused and must be freed explicitly, regardless ones was opened in a * read-only or write transaction. The REASON for this is eliminates ambiguity @@ -5226,6 +5231,29 @@ LIBMDBX_API int mdbx_cursor_open(MDBX_txn *txn, MDBX_dbi dbi, MDBX_cursor **curs * or \ref mdbx_cursor_create(). */ LIBMDBX_API void mdbx_cursor_close(MDBX_cursor *cursor); +/** \brief Closes a cursor handle with returning error code. + * \ingroup c_cursors + * + * The cursor handle will be freed and must not be used again after this call, + * but its transaction may still be live. + * + * \see mdbx_cursor_close + * + * \note In contrast to LMDB, the MDBX required that any opened cursors can be + * reused and must be freed explicitly, regardless ones was opened in a + * read-only or write transaction. The REASON for this is eliminates ambiguity + * which helps to avoid errors such as: use-after-free, double-free, i.e. + * memory corruption and segfaults. + * + * \param [in] cursor A cursor handle returned by \ref mdbx_cursor_open() + * or \ref mdbx_cursor_create(). + * \returns A non-zero error value on failure and 0 on success, + * some possible errors are: + * \retval MDBX_THREAD_MISMATCH Given transaction is not owned + * by current thread. + * \retval MDBX_EINVAL An invalid parameter was specified. */ +LIBMDBX_API int mdbx_cursor_close2(MDBX_cursor *cursor); + /** \brief Unbind or closes all cursors of a given transaction. * \ingroup c_cursors * diff --git a/mdbx.h++ b/mdbx.h++ index e993f17f..3b6e6239 100644 --- a/mdbx.h++ +++ b/mdbx.h++ @@ -4545,7 +4545,10 @@ public: } /// \brief Explicitly closes the cursor. - void close(); + inline void close() { + error::success_or_throw(::mdbx_cursor_close2(handle_)); + handle_ = nullptr; + } cursor_managed(cursor_managed &&) = default; cursor_managed &operator=(cursor_managed &&other) noexcept { diff --git a/src/api-cursor.c b/src/api-cursor.c index cfe935f1..d86bc34e 100644 --- a/src/api-cursor.c +++ b/src/api-cursor.c @@ -93,6 +93,10 @@ int mdbx_cursor_unbind(MDBX_cursor *mc) { /* TODO: реализовать при переходе на двусвязный список курсоров */ return LOG_IFERR(MDBX_EINVAL); + int rc = check_txn(mc->txn, MDBX_TXN_FINISHED | MDBX_TXN_HAS_CHILD); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); + if (unlikely(!mc->txn || mc->txn->signature != txn_signature)) { ERROR("Wrong cursor's transaction %p 0x%x", __Wpedantic_format_voidptr(mc->txn), mc->txn ? mc->txn->signature : 0); return LOG_IFERR(MDBX_PROBLEM); @@ -137,42 +141,63 @@ int mdbx_cursor_open(MDBX_txn *txn, MDBX_dbi dbi, MDBX_cursor **ret) { return MDBX_SUCCESS; } -void mdbx_cursor_close(MDBX_cursor *mc) { - if (likely(mc)) { - ENSURE(nullptr, mc->signature == cur_signature_live || mc->signature == cur_signature_ready4dispose); - MDBX_txn *const txn = mc->txn; - if (!mc->backup) { - mc->txn = nullptr; - /* Unlink from txn, if tracked. */ - if (mc->next != mc) { - ENSURE(txn->env, check_txn(txn, 0) == MDBX_SUCCESS); - const size_t dbi = (kvx_t *)mc->clc - txn->env->kvs; - tASSERT(txn, dbi < txn->n_dbi); - if (dbi < txn->n_dbi) { - MDBX_cursor **prev = &txn->cursors[dbi]; - while (*prev) { - ENSURE(txn->env, (*prev)->signature == cur_signature_live || (*prev)->signature == cur_signature_wait4eot); - if (*prev == mc) - break; - prev = &(*prev)->next; - } - tASSERT(txn, *prev == mc); - *prev = mc->next; - } - mc->next = mc; - } - mc->signature = 0; - osal_free(mc); - } else { - /* Cursor closed before nested txn ends */ - tASSERT(txn, mc->signature == cur_signature_live); - ENSURE(txn->env, check_txn_rw(txn, 0) == MDBX_SUCCESS); - be_poor(mc); - mc->signature = cur_signature_wait4eot; - } +void mdbx_cursor_close(MDBX_cursor *cursor) { + if (likely(cursor)) { + int err = mdbx_cursor_close2(cursor); + if (unlikely(err != MDBX_SUCCESS)) + mdbx_panic("%s:%d error %d (%s) while closing cursor", __func__, __LINE__, err, mdbx_liberr2str(err)); } } +int mdbx_cursor_close2(MDBX_cursor *mc) { + if (unlikely(!mc)) + return LOG_IFERR(MDBX_EINVAL); + + if (mc->signature == cur_signature_ready4dispose) { + if (unlikely(mc->txn || mc->backup)) + return LOG_IFERR(MDBX_PANIC); + cursor_drown((cursor_couple_t *)mc); + mc->signature = 0; + osal_free(mc); + return MDBX_SUCCESS; + } + + if (unlikely(mc->signature != cur_signature_live)) + return LOG_IFERR(MDBX_EBADSIGN); + + MDBX_txn *const txn = mc->txn; + int rc = check_txn(txn, MDBX_TXN_FINISHED); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); + + if (mc->backup) { + /* Cursor closed before nested txn ends */ + cursor_reset((cursor_couple_t *)mc); + mc->signature = cur_signature_wait4eot; + return MDBX_SUCCESS; + } + + if (mc->next != mc) { + const size_t dbi = cursor_dbi(mc); + cASSERT(mc, dbi < mc->txn->n_dbi); + cASSERT(mc, &mc->txn->env->kvs[dbi].clc == mc->clc); + if (likely(dbi < txn->n_dbi)) { + MDBX_cursor **prev = &txn->cursors[dbi]; + while (/* *prev && */ *prev != mc) { + ENSURE(txn->env, (*prev)->signature == cur_signature_live || (*prev)->signature == cur_signature_wait4eot); + prev = &(*prev)->next; + } + tASSERT(txn, *prev == mc); + *prev = mc->next; + } + mc->next = mc; + } + cursor_drown((cursor_couple_t *)mc); + mc->signature = 0; + osal_free(mc); + return MDBX_SUCCESS; +} + int mdbx_cursor_copy(const MDBX_cursor *src, MDBX_cursor *dest) { if (unlikely(!src)) return LOG_IFERR(MDBX_EINVAL); diff --git a/src/cursor.c b/src/cursor.c index 73da6a8b..bb6f4e61 100644 --- a/src/cursor.c +++ b/src/cursor.c @@ -252,9 +252,9 @@ MDBX_cursor *cursor_eot(MDBX_cursor *mc, MDBX_txn *txn, const bool merge) { osal_free(bk); } else { ENSURE(mc->txn->env, stage == cur_signature_live); - cursor_drown((cursor_couple_t *)mc); - mc->next = mc; mc->signature = cur_signature_ready4dispose /* Cursor may be reused */; + mc->next = mc; + cursor_drown((cursor_couple_t *)mc); } return next; } diff --git a/src/mdbx.c++ b/src/mdbx.c++ index d891b654..63f56190 100644 --- a/src/mdbx.c++ +++ b/src/mdbx.c++ @@ -1590,15 +1590,6 @@ __cold bool txn::rename_map(const ::std::string &old_name, const ::std::string & //------------------------------------------------------------------------------ -void cursor_managed::close() { - if (MDBX_UNLIKELY(!handle_)) - MDBX_CXX20_UNLIKELY error::throw_exception(MDBX_EINVAL); - ::mdbx_cursor_close(handle_); - handle_ = nullptr; -} - -//------------------------------------------------------------------------------ - __cold ::std::ostream &operator<<(::std::ostream &out, const slice &it) { out << "{"; if (!it.is_valid()) From 2fbdaccf6038abb31ace63b8d5ebe0344e981a81 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:15:53 +0300 Subject: [PATCH 56/90] =?UTF-8?q?mdbx-tests:=20=D0=BF=D0=BE=D0=B4=D0=B4?= =?UTF-8?q?=D0=B5=D1=80=D0=B6=D0=BA=D0=B0=20=D0=B7=D0=BD=D0=B0=D1=87=D0=B5?= =?UTF-8?q?=D0=BD=D0=B8=D0=B9=20`on`/`off`=20=D0=B4=D0=BB=D1=8F=20=D0=BE?= =?UTF-8?q?=D0=BF=D1=86=D0=B8=D0=B9=20=D0=BA=D0=BE=D0=BC=D0=B0=D0=BD=D0=B4?= =?UTF-8?q?=D0=BD=D0=BE=D0=B9=20=D1=81=D1=82=D1=80=D0=BE=D0=BA=D0=B8=20(ba?= =?UTF-8?q?ckport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- test/config.c++ | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/test/config.c++ b/test/config.c++ index 4188e3c0..f9e75dac 100644 --- a/test/config.c++ +++ b/test/config.c++ @@ -313,12 +313,12 @@ bool parse_option(int argc, char *const argv[], int &narg, const char *option, b return true; } - if (strcasecmp(value_cstr, "yes") == 0 || strcasecmp(value_cstr, "1") == 0) { + if (strcasecmp(value_cstr, "yes") == 0 || strcasecmp(value_cstr, "1") == 0 || strcasecmp(value_cstr, "on") == 0) { value = true; return true; } - if (strcasecmp(value_cstr, "no") == 0 || strcasecmp(value_cstr, "0") == 0) { + if (strcasecmp(value_cstr, "no") == 0 || strcasecmp(value_cstr, "0") == 0 || strcasecmp(value_cstr, "off") == 0) { value = false; return true; } From 484b488f92a1e61be9398bf5c284f2c523518662 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:16:48 +0300 Subject: [PATCH 57/90] =?UTF-8?q?mdbx-tests:=20=D0=B4=D0=BE=D0=B1=D0=B0?= =?UTF-8?q?=D0=B2=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=BF=D0=BE=D0=B4=D0=B4?= =?UTF-8?q?=D0=B5=D1=80=D0=B6=D0=BA=D0=B8=20=D0=BE=D0=BF=D1=86=D0=B8=D0=B8?= =?UTF-8?q?=20`MDBX=5FVALIDATION`=20=D0=B8=20=D0=B8=D1=81=D0=BF=D0=BE?= =?UTF-8?q?=D0=BB=D1=8C=D0=B7=D0=BE=D0=B2=D0=B0=D0=BD=D0=B8=D0=B5=20=D0=B2?= =?UTF-8?q?=20=D1=81=D1=82=D0=BE=D1=85=D0=B0=D1=81=D1=82=D0=B8=D1=87=D0=B5?= =?UTF-8?q?=D1=81=D0=BA=D0=BE=D0=BC=20=D1=82=D0=B5=D1=81=D1=82=D0=B5=20(ba?= =?UTF-8?q?ckport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- test/config.c++ | 1 + test/stochastic.sh | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/test/config.c++ b/test/config.c++ index f9e75dac..4bc20b32 100644 --- a/test/config.c++ +++ b/test/config.c++ @@ -342,6 +342,7 @@ const struct option_verb mode_bits[] = {{"rdonly", unsigned(MDBX_RDONLY)}, {"perturb", unsigned(MDBX_PAGEPERTURB)}, {"accede", unsigned(MDBX_ACCEDE)}, {"exclusive", unsigned(MDBX_EXCLUSIVE)}, + {"validation", unsigned(MDBX_VALIDATION)}, {nullptr, 0}}; const struct option_verb table_bits[] = {{"key.reverse", unsigned(MDBX_REVERSEKEY)}, diff --git a/test/stochastic.sh b/test/stochastic.sh index 18ea6737..d7abcdf5 100755 --- a/test/stochastic.sh +++ b/test/stochastic.sh @@ -440,7 +440,7 @@ else fi if [ "$EXTRA" != "no" ]; then - options=(perturb nomeminit nordahead writemap lifo nostickythreads) + options=(perturb nomeminit nordahead writemap lifo nostickythreads validation) else options=(writemap lifo nostickythreads) fi From 0fff8d07042c75930749a2708f21c60245783ced Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:17:11 +0300 Subject: [PATCH 58/90] =?UTF-8?q?mdbx-doc:=20doxygen-=D0=BE=D0=BF=D0=B8?= =?UTF-8?q?=D1=81=D0=B0=D0=BD=D0=B8=D1=8F=20=D0=B4=D0=BB=D1=8F=20doubtless?= =?UTF-8?q?-positioning=20=D0=BA=D0=BE=D0=BD=D1=81=D1=82=D0=B0=D0=BD=D1=82?= =?UTF-8?q?=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- mdbx.h | 30 ++++++++++++++++-------------- 1 file changed, 16 insertions(+), 14 deletions(-) diff --git a/mdbx.h b/mdbx.h index e8e5748b..fb7d1400 100644 --- a/mdbx.h +++ b/mdbx.h @@ -1791,26 +1791,28 @@ typedef enum MDBX_cursor_op { * \ref MDBX_NOTFOUND otherwise. */ MDBX_SET_UPPERBOUND, - /* Doubtless cursor positioning at a specified key. */ + /** Doubtless cursor positioning at a specified key. */ MDBX_TO_KEY_LESSER_THAN, - MDBX_TO_KEY_LESSER_OR_EQUAL, - MDBX_TO_KEY_EQUAL, - MDBX_TO_KEY_GREATER_OR_EQUAL, - MDBX_TO_KEY_GREATER_THAN, + MDBX_TO_KEY_LESSER_OR_EQUAL /** \copydoc MDBX_TO_KEY_LESSER_THAN */, + MDBX_TO_KEY_EQUAL /** \copydoc MDBX_TO_KEY_LESSER_THAN */, + MDBX_TO_KEY_GREATER_OR_EQUAL /** \copydoc MDBX_TO_KEY_LESSER_THAN */, + MDBX_TO_KEY_GREATER_THAN /** \copydoc MDBX_TO_KEY_LESSER_THAN */, - /* Doubtless cursor positioning at a specified key-value pair + /** Doubtless cursor positioning at a specified key-value pair * for dupsort/multi-value hives. */ MDBX_TO_EXACT_KEY_VALUE_LESSER_THAN, - MDBX_TO_EXACT_KEY_VALUE_LESSER_OR_EQUAL, - MDBX_TO_EXACT_KEY_VALUE_EQUAL, - MDBX_TO_EXACT_KEY_VALUE_GREATER_OR_EQUAL, - MDBX_TO_EXACT_KEY_VALUE_GREATER_THAN, + MDBX_TO_EXACT_KEY_VALUE_LESSER_OR_EQUAL /** \copydoc MDBX_TO_EXACT_KEY_VALUE_LESSER_THAN */, + MDBX_TO_EXACT_KEY_VALUE_EQUAL /** \copydoc MDBX_TO_EXACT_KEY_VALUE_LESSER_THAN */, + MDBX_TO_EXACT_KEY_VALUE_GREATER_OR_EQUAL /** \copydoc MDBX_TO_EXACT_KEY_VALUE_LESSER_THAN */, + MDBX_TO_EXACT_KEY_VALUE_GREATER_THAN /** \copydoc MDBX_TO_EXACT_KEY_VALUE_LESSER_THAN */, + /** Doubtless cursor positioning at a specified key-value pair + * for dupsort/multi-value hives. */ MDBX_TO_PAIR_LESSER_THAN, - MDBX_TO_PAIR_LESSER_OR_EQUAL, - MDBX_TO_PAIR_EQUAL, - MDBX_TO_PAIR_GREATER_OR_EQUAL, - MDBX_TO_PAIR_GREATER_THAN + MDBX_TO_PAIR_LESSER_OR_EQUAL /** \copydoc MDBX_TO_PAIR_LESSER_THAN */, + MDBX_TO_PAIR_EQUAL /** \copydoc MDBX_TO_PAIR_LESSER_THAN */, + MDBX_TO_PAIR_GREATER_OR_EQUAL /** \copydoc MDBX_TO_PAIR_LESSER_THAN */, + MDBX_TO_PAIR_GREATER_THAN /** \copydoc MDBX_TO_PAIR_LESSER_THAN */ } MDBX_cursor_op; /** \brief Errors and return codes From 91570a084f1f633138208c8c7928544fdcd058d5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:17:47 +0300 Subject: [PATCH 59/90] =?UTF-8?q?mdbx:=20=D0=B4=D0=BE=D0=B1=D0=B0=D0=B2?= =?UTF-8?q?=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20`MDBX=5FSEEK=5FAND=5FGET=5FMUL?= =?UTF-8?q?TIPLE`=20=D0=B2=20API=20=D0=BE=D0=BF=D0=B5=D1=80=D0=B0=D1=86?= =?UTF-8?q?=D0=B8=D0=B9=20=D0=BA=D1=83=D1=80=D1=81=D0=BE=D1=80=D0=B0=20(ba?= =?UTF-8?q?ckport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- mdbx.h | 16 +++++++++++----- src/cursor.c | 38 +++++++++++++------------------------- 2 files changed, 24 insertions(+), 30 deletions(-) diff --git a/mdbx.h b/mdbx.h index fb7d1400..cce6c6a5 100644 --- a/mdbx.h +++ b/mdbx.h @@ -1718,7 +1718,7 @@ typedef enum MDBX_cursor_op { /** \ref MDBX_DUPFIXED -only: Return up to a page of duplicate data items * from current cursor position. Move cursor to prepare - * for \ref MDBX_NEXT_MULTIPLE. */ + * for \ref MDBX_NEXT_MULTIPLE. \see MDBX_SEEK_AND_GET_MULTIPLE */ MDBX_GET_MULTIPLE, /** Position at last key/data item */ @@ -1734,8 +1734,8 @@ typedef enum MDBX_cursor_op { MDBX_NEXT_DUP, /** \ref MDBX_DUPFIXED -only: Return up to a page of duplicate data items - * from next cursor position. Move cursor to prepare - * for `MDBX_NEXT_MULTIPLE`. */ + * from next cursor position. Move cursor to prepare for `MDBX_NEXT_MULTIPLE`. + * \see MDBX_SEEK_AND_GET_MULTIPLE \see MDBX_GET_MULTIPLE */ MDBX_NEXT_MULTIPLE, /** Position at first data item of next key */ @@ -1760,7 +1760,8 @@ typedef enum MDBX_cursor_op { MDBX_SET_RANGE, /** \ref MDBX_DUPFIXED -only: Position at previous page and return up to - * a page of duplicate data items. */ + * a page of duplicate data items. + * \see MDBX_SEEK_AND_GET_MULTIPLE \see MDBX_GET_MULTIPLE */ MDBX_PREV_MULTIPLE, /** Positions cursor at first key-value pair greater than or equal to @@ -1812,7 +1813,12 @@ typedef enum MDBX_cursor_op { MDBX_TO_PAIR_LESSER_OR_EQUAL /** \copydoc MDBX_TO_PAIR_LESSER_THAN */, MDBX_TO_PAIR_EQUAL /** \copydoc MDBX_TO_PAIR_LESSER_THAN */, MDBX_TO_PAIR_GREATER_OR_EQUAL /** \copydoc MDBX_TO_PAIR_LESSER_THAN */, - MDBX_TO_PAIR_GREATER_THAN /** \copydoc MDBX_TO_PAIR_LESSER_THAN */ + MDBX_TO_PAIR_GREATER_THAN /** \copydoc MDBX_TO_PAIR_LESSER_THAN */, + + /** \ref MDBX_DUPFIXED -only: Seek to given key and return up to a page of + * duplicate data items from current cursor position. Move cursor to prepare + * for \ref MDBX_NEXT_MULTIPLE. \see MDBX_GET_MULTIPLE */ + MDBX_SEEK_AND_GET_MULTIPLE } MDBX_cursor_op; /** \brief Errors and return codes diff --git a/src/cursor.c b/src/cursor.c index bb6f4e61..9c1192d9 100644 --- a/src/cursor.c +++ b/src/cursor.c @@ -2047,27 +2047,24 @@ __hot int cursor_ops(MDBX_cursor *mc, MDBX_val *key, MDBX_val *data, const MDBX_ cASSERT(mc, is_poor(mc) && !is_filled(mc)); return rc; + case MDBX_SEEK_AND_GET_MULTIPLE: + if (unlikely(!key)) + return MDBX_EINVAL; + rc = cursor_seek(mc, key, data, MDBX_SET).err; + if (unlikely(rc != MDBX_SUCCESS)) + return rc; + __fallthrough /* fall through */; case MDBX_GET_MULTIPLE: if (unlikely(!data)) return MDBX_EINVAL; if (unlikely((mc->tree->flags & MDBX_DUPFIXED) == 0)) return MDBX_INCOMPATIBLE; - if (unlikely(!is_pointed(mc))) { - if (unlikely(!key)) - return MDBX_EINVAL; - if (unlikely((mc->flags & z_fresh) == 0)) - return MDBX_ENODATA; - rc = cursor_seek(mc, key, data, MDBX_SET).err; - if (unlikely(rc != MDBX_SUCCESS)) - return rc; - } else { - if (unlikely(!is_filled(mc))) - return MDBX_ENODATA; - if (key) { - const page_t *mp = mc->pg[mc->top]; - const node_t *node = page_node(mp, mc->ki[mc->top]); - *key = get_key(node); - } + if (unlikely(!is_filled(mc))) + return MDBX_ENODATA; + if (key) { + const page_t *mp = mc->pg[mc->top]; + const node_t *node = page_node(mp, mc->ki[mc->top]); + *key = get_key(node); } cASSERT(mc, is_filled(mc)); if (unlikely(!inner_filled(mc))) { @@ -2102,15 +2099,6 @@ __hot int cursor_ops(MDBX_cursor *mc, MDBX_val *key, MDBX_val *data, const MDBX_ return MDBX_EINVAL; if (unlikely(mc->subcur == nullptr)) return MDBX_INCOMPATIBLE; - if (unlikely(!is_pointed(mc))) { - if (unlikely((mc->flags & z_fresh) == 0)) - return MDBX_ENODATA; - rc = outer_last(mc, key, data); - if (unlikely(rc != MDBX_SUCCESS)) - return rc; - mc->subcur->cursor.ki[mc->subcur->cursor.top] = 0; - goto fetch_multiple; - } if (unlikely(!is_filled(mc) || !inner_filled(mc))) return MDBX_ENODATA; rc = cursor_sibling_left(&mc->subcur->cursor); From 17207620800c285200901bb99d24fb12070bb013 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:19:31 +0300 Subject: [PATCH 60/90] =?UTF-8?q?mdbx:=20=D0=BF=D0=B5=D1=80=D0=B5=D0=B8?= =?UTF-8?q?=D0=BC=D0=B5=D0=BD=D0=BE=D0=B2=D0=B0=D0=BD=D0=B8=D0=B5=20`curso?= =?UTF-8?q?r=5Fvalidate()`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/cursor.c | 20 ++++++++++---------- src/cursor.h | 4 ++-- src/tree-ops.c | 16 ++++++++-------- 3 files changed, 20 insertions(+), 20 deletions(-) diff --git a/src/cursor.c b/src/cursor.c index 9c1192d9..ff2c67e8 100644 --- a/src/cursor.c +++ b/src/cursor.c @@ -5,7 +5,7 @@ #include "internals.h" -__cold int cursor_check(const MDBX_cursor *mc) { +__cold int cursor_validate(const MDBX_cursor *mc) { if (!mc->txn->tw.dirtylist) { cASSERT(mc, (mc->txn->flags & MDBX_WRITEMAP) != 0 && !MDBX_AVOID_MSYNC); } else { @@ -81,10 +81,10 @@ __cold int cursor_check(const MDBX_cursor *mc) { return MDBX_SUCCESS; } -__cold int cursor_check_updating(MDBX_cursor *mc) { +__cold int cursor_validate_updating(MDBX_cursor *mc) { const uint8_t checking = mc->checking; mc->checking |= z_updating; - const int rc = cursor_check(mc); + const int rc = cursor_validate(mc); mc->checking = checking; return rc; } @@ -938,7 +938,7 @@ __hot int cursor_put(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data, unsig } if (AUDIT_ENABLED()) { - err = cursor_check(mc); + err = cursor_validate(mc); if (unlikely(err != MDBX_SUCCESS)) return err; } @@ -947,7 +947,7 @@ __hot int cursor_put(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data, unsig more: if (AUDIT_ENABLED()) { - err = cursor_check(mc); + err = cursor_validate(mc); if (unlikely(err != MDBX_SUCCESS)) return err; } @@ -1008,7 +1008,7 @@ __hot int cursor_put(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data, unsig memcpy(page_data(lp.page), data->iov_base, data->iov_len); if (AUDIT_ENABLED()) { - err = cursor_check(mc); + err = cursor_validate(mc); if (unlikely(err != MDBX_SUCCESS)) return err; } @@ -1274,7 +1274,7 @@ __hot int cursor_put(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data, unsig } if (AUDIT_ENABLED()) { - err = cursor_check(mc); + err = cursor_validate(mc); if (unlikely(err != MDBX_SUCCESS)) return err; } @@ -1292,7 +1292,7 @@ insert_node:; if (page_room(mc->pg[mc->top]) < nsize) { rc = page_split(mc, key, ref_data, P_INVALID, insert_key ? naf : naf | MDBX_SPLIT_REPLACE); if (rc == MDBX_SUCCESS && AUDIT_ENABLED()) - rc = insert_key ? cursor_check(mc) : cursor_check_updating(mc); + rc = insert_key ? cursor_validate(mc) : cursor_validate_updating(mc); } else { /* There is room already in this leaf page. */ if (is_dupfix_leaf(mc->pg[mc->top])) { @@ -1414,7 +1414,7 @@ insert_node:; } } if (AUDIT_ENABLED()) - rc = cursor_check(mc); + rc = cursor_validate(mc); } return rc; @@ -1686,7 +1686,7 @@ del_key: cASSERT(mc, rc == MDBX_SUCCESS); if (AUDIT_ENABLED()) - rc = cursor_check(mc); + rc = cursor_validate(mc); return rc; fail: diff --git a/src/cursor.h b/src/cursor.h index d06bad3a..4fa0e585 100644 --- a/src/cursor.h +++ b/src/cursor.h @@ -233,7 +233,7 @@ enum cursor_checking { z_pagecheck = 0x80 /* perform page checking, see MDBX_VALIDATION */ }; -MDBX_INTERNAL int __must_check_result cursor_check(const MDBX_cursor *mc); +MDBX_INTERNAL int __must_check_result cursor_validate(const MDBX_cursor *mc); MDBX_MAYBE_UNUSED MDBX_NOTHROW_PURE_FUNCTION static inline size_t cursor_dbi(const MDBX_cursor *mc) { cASSERT(mc, mc->txn && mc->txn->signature == txn_signature); @@ -305,7 +305,7 @@ MDBX_INTERNAL int __must_check_result cursor_put_checklen(MDBX_cursor *mc, const MDBX_INTERNAL int __must_check_result cursor_put(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data, unsigned flags); -MDBX_INTERNAL int __must_check_result cursor_check_updating(MDBX_cursor *mc); +MDBX_INTERNAL int __must_check_result cursor_validate_updating(MDBX_cursor *mc); MDBX_INTERNAL int __must_check_result cursor_del(MDBX_cursor *mc, unsigned flags); diff --git a/src/tree-ops.c b/src/tree-ops.c index d0456792..85131c42 100644 --- a/src/tree-ops.c +++ b/src/tree-ops.c @@ -880,7 +880,7 @@ retry: if (nkeys >= minkeys) { mc->ki[mc->top] = (indx_t)ki_top; if (AUDIT_ENABLED()) - return cursor_check_updating(mc); + return cursor_validate_updating(mc); return MDBX_SUCCESS; } @@ -920,7 +920,7 @@ int page_split(MDBX_cursor *mc, const MDBX_val *const newkey, MDBX_val *const ne const size_t newindx = mc->ki[mc->top]; size_t nkeys = page_numkeys(mp); if (AUDIT_ENABLED()) { - rc = cursor_check_updating(mc); + rc = cursor_validate_updating(mc); if (unlikely(rc != MDBX_SUCCESS)) return rc; } @@ -979,7 +979,7 @@ int page_split(MDBX_cursor *mc, const MDBX_val *const newkey, MDBX_val *const ne mc->top = 1; prev_top = 0; if (AUDIT_ENABLED()) { - rc = cursor_check_updating(mc); + rc = cursor_validate_updating(mc); if (unlikely(rc != MDBX_SUCCESS)) goto done; } @@ -1092,10 +1092,10 @@ int page_split(MDBX_cursor *mc, const MDBX_val *const newkey, MDBX_val *const ne } if (AUDIT_ENABLED()) { - rc = cursor_check_updating(mc); + rc = cursor_validate_updating(mc); if (unlikely(rc != MDBX_SUCCESS)) goto done; - rc = cursor_check_updating(mn); + rc = cursor_validate_updating(mn); if (unlikely(rc != MDBX_SUCCESS)) goto done; } @@ -1221,7 +1221,7 @@ int page_split(MDBX_cursor *mc, const MDBX_val *const newkey, MDBX_val *const ne goto done; cASSERT(mc, mc->top - top == mc->tree->height - height); if (AUDIT_ENABLED()) { - rc = cursor_check_updating(mc); + rc = cursor_validate_updating(mc); if (unlikely(rc != MDBX_SUCCESS)) goto done; } @@ -1474,7 +1474,7 @@ done: mc->txn->flags |= MDBX_TXN_ERROR; else { if (AUDIT_ENABLED()) - rc = cursor_check_updating(mc); + rc = cursor_validate_updating(mc); if (unlikely(naf & MDBX_RESERVE)) { node_t *node = page_node(mc->pg[mc->top], mc->ki[mc->top]); if (!(node_flags(node) & N_BIG)) @@ -1524,7 +1524,7 @@ int tree_propagate_key(MDBX_cursor *mc, const MDBX_val *key) { node_del(mc, 0); int err = page_split(mc, key, nullptr, pgno, MDBX_SPLIT_REPLACE); if (err == MDBX_SUCCESS && AUDIT_ENABLED()) - err = cursor_check_updating(mc); + err = cursor_validate_updating(mc); return err; } From 5dfe3433a8f446d5c63a032bcccff011c6cd492a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:21:02 +0300 Subject: [PATCH 61/90] =?UTF-8?q?mdbx:=20=D1=83=D1=81=D1=82=D1=80=D0=B0?= =?UTF-8?q?=D0=BD=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=B3=D0=BE=D0=BD=D0=BA=D0=B8?= =?UTF-8?q?=20=D0=B2=20`tbl=5Fsetup(MDBX=5FDUPFIXED=20|=20MDBX=5FINTEGERDU?= =?UTF-8?q?P)`=20=D0=BF=D1=80=D0=B8=20=D1=80=D0=B0=D0=B1=D0=BE=D1=82=D0=B5?= =?UTF-8?q?=20=D0=B2=20=D1=80=D0=B0=D0=B7=D0=BD=D1=8B=D1=85=20=D0=BF=D0=BE?= =?UTF-8?q?=D1=82=D0=BE=D0=BA=D0=B0=D1=85=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Проблема была в том, что в случаях фиксированного размера значений clc.lmin/clc.lmax устанавливались в env->kvs[], а затем корректировались по актуальному размеру данных в БД. Поэтому при конкурентном вызове из разных потоков, один поток мог выполнять инициализацию, а второй прочитать временные/промежуточные значения lmin/lmax. В результате, при конкурентном старте транзакций в разных потоках при использовании только-что открытого dbi-хендла, проверка допустимости длины значения могла заканчиваться ложной ошибкой MDBX_BAD_VALSIZE. --- src/cogs.h | 4 ++++ src/cursor.c | 6 ++---- src/proto.h | 2 +- src/table.c | 40 ++++++++++++++++++++++++++-------------- 4 files changed, 33 insertions(+), 19 deletions(-) diff --git a/src/cogs.h b/src/cogs.h index 11d7fe1d..7e814e7b 100644 --- a/src/cogs.h +++ b/src/cogs.h @@ -200,6 +200,10 @@ static inline bool check_table_flags(unsigned flags) { } } +static inline int tbl_setup_ifneed(const MDBX_env *env, volatile kvx_t *const kvx, const tree_t *const db) { + return likely(kvx->clc.v.lmax) ? MDBX_SUCCESS : tbl_setup(env, kvx, db); +} + /*----------------------------------------------------------------------------*/ MDBX_NOTHROW_PURE_FUNCTION static inline size_t pgno2bytes(const MDBX_env *env, size_t pgno) { diff --git a/src/cursor.c b/src/cursor.c index ff2c67e8..01fc8a56 100644 --- a/src/cursor.c +++ b/src/cursor.c @@ -298,10 +298,7 @@ static __always_inline int couple_init(cursor_couple_t *couple, const MDBX_txn * if (unlikely(*dbi_state & DBI_STALE)) return tbl_fetch(couple->outer.txn, cursor_dbi(&couple->outer)); - if (unlikely(kvx->clc.k.lmax == 0)) - return tbl_setup(txn->env, kvx, tree); - - return MDBX_SUCCESS; + return tbl_setup_ifneed(txn->env, kvx, tree); } __cold int cursor_init4walk(cursor_couple_t *couple, const MDBX_txn *const txn, tree_t *const tree, kvx_t *const kvx) { @@ -387,6 +384,7 @@ int cursor_dupsort_setup(MDBX_cursor *mc, const node_t *node, const page_t *mp) } mc->tree->dupfix_size = mx->nested_tree.dupfix_size; mc->clc->v.lmin = mc->clc->v.lmax = mx->nested_tree.dupfix_size; + cASSERT(mc, mc->clc->v.lmax >= mc->clc->v.lmin); } DEBUG("Sub-db dbi -%zu root page %" PRIaPGNO, cursor_dbi(&mx->cursor), mx->nested_tree.root); diff --git a/src/proto.h b/src/proto.h index f0a6657f..92bbce6d 100644 --- a/src/proto.h +++ b/src/proto.h @@ -95,7 +95,7 @@ MDBX_INTERNAL void recalculate_subpage_thresholds(MDBX_env *env); /* table.c */ MDBX_INTERNAL int __must_check_result tbl_fetch(MDBX_txn *txn, size_t dbi); -MDBX_INTERNAL int __must_check_result tbl_setup(const MDBX_env *env, kvx_t *const kvx, const tree_t *const db); +MDBX_INTERNAL int __must_check_result tbl_setup(const MDBX_env *env, volatile kvx_t *const kvx, const tree_t *const db); /* coherency.c */ MDBX_INTERNAL bool coherency_check_meta(const MDBX_env *env, const volatile meta_t *meta, bool report); diff --git a/src/table.c b/src/table.c index faa44da1..5b3e441a 100644 --- a/src/table.c +++ b/src/table.c @@ -3,28 +3,37 @@ #include "internals.h" -int tbl_setup(const MDBX_env *env, kvx_t *const kvx, const tree_t *const db) { +int tbl_setup(const MDBX_env *env, volatile kvx_t *const kvx, const tree_t *const db) { + osal_memory_fence(mo_AcquireRelease, false); + if (unlikely(!check_table_flags(db->flags))) { ERROR("incompatible or invalid db.flags (0x%x) ", db->flags); return MDBX_INCOMPATIBLE; } - if (unlikely(!kvx->clc.k.cmp)) { - kvx->clc.k.cmp = builtin_keycmp(db->flags); - kvx->clc.v.cmp = builtin_datacmp(db->flags); + + size_t v_lmin = valsize_min(db->flags); + size_t v_lmax = env_valsize_max(env, db->flags); + if ((db->flags & (MDBX_DUPFIXED | MDBX_INTEGERDUP)) != 0 && db->dupfix_size) { + if (!MDBX_DISABLE_VALIDATION && unlikely(db->dupfix_size < v_lmin || db->dupfix_size > v_lmax)) { + ERROR("db.dupfix_size (%u) <> min/max value-length (%zu/%zu)", db->dupfix_size, v_lmin, v_lmax); + return MDBX_CORRUPTED; + } + v_lmin = v_lmax = db->dupfix_size; } kvx->clc.k.lmin = keysize_min(db->flags); kvx->clc.k.lmax = env_keysize_max(env, db->flags); - kvx->clc.v.lmin = valsize_min(db->flags); - kvx->clc.v.lmax = env_valsize_max(env, db->flags); - - if ((db->flags & (MDBX_DUPFIXED | MDBX_INTEGERDUP)) != 0 && db->dupfix_size) { - if (!MDBX_DISABLE_VALIDATION && unlikely(db->dupfix_size < kvx->clc.v.lmin || db->dupfix_size > kvx->clc.v.lmax)) { - ERROR("db.dupfix_size (%u) <> min/max value-length (%zu/%zu)", db->dupfix_size, kvx->clc.v.lmin, kvx->clc.v.lmax); - return MDBX_CORRUPTED; - } - kvx->clc.v.lmin = kvx->clc.v.lmax = db->dupfix_size; + if (unlikely(!kvx->clc.k.cmp)) { + kvx->clc.v.cmp = builtin_datacmp(db->flags); + kvx->clc.k.cmp = builtin_keycmp(db->flags); } + kvx->clc.v.lmin = v_lmin; + osal_memory_fence(mo_Relaxed, true); + kvx->clc.v.lmax = v_lmax; + osal_memory_fence(mo_AcquireRelease, true); + + eASSERT(env, kvx->clc.k.lmax >= kvx->clc.k.lmin); + eASSERT(env, kvx->clc.v.lmax >= kvx->clc.v.lmin); return MDBX_SUCCESS; } @@ -86,10 +95,13 @@ int tbl_fetch(MDBX_txn *txn, size_t dbi) { return MDBX_CORRUPTED; } #endif /* !MDBX_DISABLE_VALIDATION */ - rc = tbl_setup(txn->env, kvx, db); + rc = tbl_setup_ifneed(txn->env, kvx, db); if (unlikely(rc != MDBX_SUCCESS)) return rc; + if (unlikely(dbi_changed(txn, dbi))) + return MDBX_BAD_DBI; + txn->dbi_state[dbi] &= ~DBI_STALE; return MDBX_SUCCESS; } From 58729a2fbd4a2cd6b9cbe80be993149c2ee3b946 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:40:11 +0300 Subject: [PATCH 62/90] =?UTF-8?q?mdbx:=20=D0=BA=D0=BE=D1=80=D1=80=D0=B5?= =?UTF-8?q?=D0=BA=D1=82=D0=B8=D1=80=D0=BE=D0=B2=D0=BA=D0=B0=20=D0=BE=D0=BF?= =?UTF-8?q?=D0=B8=D1=81=D0=B0=D0=BD=D0=B8=D1=8F=20`MDBX=5FMVCC=5FRETARDED`?= =?UTF-8?q?=20=D0=B8=20=D1=81=D0=BE=D0=BE=D1=82=D0=B2=D0=B5=D1=82=D1=81?= =?UTF-8?q?=D1=82=D0=B2=D1=83=D1=8E=D1=89=D0=B5=D0=B3=D0=BE=20=D1=81=D0=BE?= =?UTF-8?q?=D0=BE=D0=B1=D1=89=D0=B5=D0=BD=D0=B8=D1=8F=20=D0=BE=D0=B1=20?= =?UTF-8?q?=D0=BE=D1=88=D0=B8=D0=B1=D0=BA=D0=B5=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- mdbx.h | 3 +-- src/api-misc.c | 4 +--- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/mdbx.h b/mdbx.h index cce6c6a5..0ae5921a 100644 --- a/mdbx.h +++ b/mdbx.h @@ -1972,8 +1972,7 @@ typedef enum MDBX_error { * recycling old MVCC snapshots. */ MDBX_OUSTED = -30411, - /** MVCC snapshot used by read transaction is outdated and could not be - * copied since corresponding meta-pages was overwritten. */ + /** MVCC snapshot used by parked transaction was bygone. */ MDBX_MVCC_RETARDED = -30410, /* The last of MDBX-added error codes */ diff --git a/src/api-misc.c b/src/api-misc.c index 7e54c51d..27e82a90 100644 --- a/src/api-misc.c +++ b/src/api-misc.c @@ -199,9 +199,7 @@ __cold const char *mdbx_liberr2str(int errnum) { return "MDBX_OUSTED: The parked read transaction was outed for the sake" " of recycling old MVCC snapshots"; case MDBX_MVCC_RETARDED: - return "MDBX_MVCC_RETARDED: MVCC snapshot used by read transaction" - " is outdated and could not be copied" - " since corresponding meta-pages was overwritten"; + return "MDBX_MVCC_RETARDED: MVCC snapshot used by parked transaction was bygone"; default: return nullptr; } From 73d52c19638c52ae8ab4de8a377b93b187cd2337 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:40:41 +0300 Subject: [PATCH 63/90] =?UTF-8?q?mdbx++:=20=D0=B4=D0=BE=D0=B1=D0=B0=D0=B2?= =?UTF-8?q?=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20`mdbx::cursor=5Fmanaged::withd?= =?UTF-8?q?raw=5Fhandle()`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- mdbx.h++ | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/mdbx.h++ b/mdbx.h++ index 3b6e6239..8ba9f102 100644 --- a/mdbx.h++ +++ b/mdbx.h++ @@ -4155,9 +4155,9 @@ public: class LIBMDBX_API_TYPE cursor { protected: MDBX_cursor *handle_{nullptr}; - MDBX_CXX11_CONSTEXPR cursor(MDBX_cursor *ptr) noexcept; public: + MDBX_CXX11_CONSTEXPR cursor(MDBX_cursor *ptr) noexcept; MDBX_CXX11_CONSTEXPR cursor() noexcept = default; cursor(const cursor &) noexcept = default; inline cursor &operator=(cursor &&other) noexcept; @@ -4561,6 +4561,12 @@ public: return *this; } + inline MDBX_cursor *withdraw_handle() noexcept { + MDBX_cursor *handle = handle_; + handle_ = nullptr; + return handle; + } + cursor_managed(const cursor_managed &) = delete; cursor_managed &operator=(const cursor_managed &) = delete; ~cursor_managed() noexcept { ::mdbx_cursor_close(handle_); } From d8890bc1695e69cc748ac2e78fe7426d358a8214 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:41:05 +0300 Subject: [PATCH 64/90] =?UTF-8?q?mdbx++:=20=D0=B4=D0=BE=D0=B1=D0=B0=D0=B2?= =?UTF-8?q?=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20`inplace=5Fstorage=5Fsize=5Fro?= =?UTF-8?q?unding`=20=D0=B2=20`capacity=5Fpolicy`=20=D0=B4=D0=BB=D1=8F=20?= =?UTF-8?q?=D0=B1=D1=83=D1=84=D0=B5=D1=80=D0=BE=D0=B2=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- mdbx.h++ | 111 +++++++++++++++++++++++++++++++++---------------------- 1 file changed, 66 insertions(+), 45 deletions(-) diff --git a/mdbx.h++ b/mdbx.h++ index 8ba9f102..97120b3f 100644 --- a/mdbx.h++ +++ b/mdbx.h++ @@ -1162,7 +1162,12 @@ template struct swap_alloc { } // namespace allocation_aware_details struct default_capacity_policy { - enum : size_t { extra_inplace_storage = 0, pettiness_threshold = 64, max_reserve = 65536 }; + enum : size_t { + extra_inplace_storage = 0, + inplace_storage_size_rounding = 16, + pettiness_threshold = 64, + max_reserve = 65536 + }; static MDBX_CXX11_CONSTEXPR size_t round(const size_t value) { static_assert((pettiness_threshold & (pettiness_threshold - 1)) == 0, "pettiness_threshold must be a power of 2"); @@ -1486,6 +1491,10 @@ public: max_length = MDBX_MAXDATASIZE, max_capacity = (max_length / 3u * 4u + 1023u) & ~size_t(1023), extra_inplace_storage = reservation_policy::extra_inplace_storage, + inplace_storage_size_rounding = + (alignof(max_align_t) * 2 > size_t(reservation_policy::inplace_storage_size_rounding)) + ? alignof(max_align_t) * 2 + : size_t(reservation_policy::inplace_storage_size_rounding), pettiness_threshold = reservation_policy::pettiness_threshold }; @@ -1529,41 +1538,51 @@ private: #endif /* __cpp_lib_to_address */ } - union bin { - struct allocated { + union alignas(max_align_t) bin { + struct stub_allocated_holder /* используется только для вычисления (минимального необходимого) размера, + с учетом выравнивания */ + { allocator_pointer ptr_; - size_t capacity_bytes_; - constexpr allocated(allocator_pointer ptr, size_t bytes) noexcept : ptr_(ptr), capacity_bytes_(bytes) {} - constexpr allocated(const allocated &) noexcept = default; - constexpr allocated(allocated &&) noexcept = default; - MDBX_CXX17_CONSTEXPR allocated &operator=(const allocated &) noexcept = default; - MDBX_CXX17_CONSTEXPR allocated &operator=(allocated &&) noexcept = default; + size_t stub_capacity_bytes_; }; - allocated allocated_; - uint64_t align_hint_; - byte inplace_[(sizeof(allocated) + extra_inplace_storage + 7u) & ~size_t(7)]; - - static constexpr bool is_suitable_for_inplace(size_t capacity_bytes) noexcept { - static_assert(sizeof(bin) == sizeof(inplace_), "WTF?"); - return capacity_bytes < sizeof(bin); - } - - enum : byte { lastbyte_inplace_signature = byte(~byte(0)) }; + enum : byte { lastbyte_poison = 0, lastbyte_inplace_signature = byte(~byte(lastbyte_poison)) }; enum : size_t { inplace_signature_limit = size_t(lastbyte_inplace_signature) - << (sizeof(size_t /* allocated::capacity_bytes_ */) - 1) * CHAR_BIT + << (sizeof(size_t /* allocated::capacity_bytes_ */) - 1) * CHAR_BIT, + inplace_size_rounding = size_t(inplace_storage_size_rounding) - 1, + inplace_size = + (sizeof(stub_allocated_holder) + extra_inplace_storage + inplace_size_rounding) & ~inplace_size_rounding }; - constexpr byte inplace_lastbyte() const noexcept { return inplace_[sizeof(bin) - 1]; } - MDBX_CXX17_CONSTEXPR byte &inplace_lastbyte() noexcept { return inplace_[sizeof(bin) - 1]; } + struct capacity_holder { + byte pad_[inplace_size - sizeof(allocator_pointer)]; + size_t bytes_; + }; + + struct inplace_flag_holder { + byte buffer_[inplace_size - sizeof(byte)]; + byte lastbyte_; + }; + + allocator_pointer allocated_ptr_; + capacity_holder capacity_; + inplace_flag_holder inplace_; + + static constexpr bool is_suitable_for_inplace(size_t capacity_bytes) noexcept { + static_assert((size_t(reservation_policy::inplace_storage_size_rounding) & + (size_t(reservation_policy::inplace_storage_size_rounding) - 1)) == 0, + "CAPACITY_POLICY::inplace_storage_size_rounding must be power of 2"); + static_assert(sizeof(bin) == sizeof(inplace_) && sizeof(bin) == sizeof(capacity_), "WTF?"); + return capacity_bytes < sizeof(bin); + } constexpr bool is_inplace() const noexcept { static_assert(size_t(inplace_signature_limit) > size_t(max_capacity), "WTF?"); static_assert(std::numeric_limits::max() - (std::numeric_limits::max() >> CHAR_BIT) == inplace_signature_limit, "WTF?"); - return inplace_lastbyte() == lastbyte_inplace_signature; + return inplace_.lastbyte_ == lastbyte_inplace_signature; } constexpr bool is_allocated() const noexcept { return !is_inplace(); } @@ -1571,26 +1590,27 @@ private: if (destroy_ptr) { MDBX_CONSTEXPR_ASSERT(is_allocated()); /* properly destroy allocator::pointer */ - allocated_.~allocated(); + allocated_ptr_.~allocator_pointer(); } if (::std::is_trivial::value) /* workaround for "uninitialized" warning from some compilers */ - memset(&allocated_.ptr_, 0, sizeof(allocated_.ptr_)); - inplace_lastbyte() = lastbyte_inplace_signature; - MDBX_CONSTEXPR_ASSERT(is_inplace() && address() == inplace_ && is_suitable_for_inplace(capacity())); + memset(&allocated_ptr_, 0, sizeof(allocated_ptr_)); + inplace_.lastbyte_ = lastbyte_inplace_signature; + MDBX_CONSTEXPR_ASSERT(is_inplace() && address() == inplace_.buffer_ && is_suitable_for_inplace(capacity())); return address(); } template MDBX_CXX17_CONSTEXPR byte *make_allocated(allocator_pointer ptr, size_t capacity_bytes) noexcept { MDBX_CONSTEXPR_ASSERT(inplace_signature_limit > capacity_bytes); - if (construct_ptr) + if (construct_ptr) { /* properly construct allocator::pointer */ - new (&allocated_) allocated(ptr, capacity_bytes); - else { + new (&allocated_ptr_) allocator_pointer(ptr); + capacity_.bytes_ = capacity_bytes; + } else { MDBX_CONSTEXPR_ASSERT(is_allocated()); - allocated_.ptr_ = ptr; - allocated_.capacity_bytes_ = capacity_bytes; + allocated_ptr_ = ptr; + capacity_.bytes_ = capacity_bytes; } MDBX_CONSTEXPR_ASSERT(is_allocated() && address() == to_address(ptr) && capacity() == capacity_bytes); return address(); @@ -1608,16 +1628,17 @@ private: MDBX_CXX20_CONSTEXPR ~bin() { if (is_allocated()) /* properly destroy allocator::pointer */ - allocated_.~allocated(); + allocated_ptr_.~allocator_pointer(); } MDBX_CXX20_CONSTEXPR bin(bin &&ditto) noexcept { if (ditto.is_inplace()) { // micro-optimization: don't use make_inplace<> here // since memcpy() will copy the flag. - memcpy(inplace_, ditto.inplace_, sizeof(inplace_)); + memcpy(&inplace_, &ditto.inplace_, sizeof(inplace_)); MDBX_CONSTEXPR_ASSERT(is_inplace()); } else { - new (&allocated_) allocated(::std::move(ditto.allocated_)); + new (&allocated_ptr_) allocator_pointer(::std::move(ditto.allocated_ptr_)); + capacity_.bytes_ = ditto.capacity_.bytes_; ditto.make_inplace(); MDBX_CONSTEXPR_ASSERT(is_allocated()); } @@ -1629,13 +1650,13 @@ private: // since memcpy() will copy the flag. if (is_allocated()) /* properly destroy allocator::pointer */ - allocated_.~allocated(); - memcpy(inplace_, ditto.inplace_, sizeof(inplace_)); + allocated_ptr_.~allocator_pointer(); + memcpy(&inplace_, &ditto.inplace_, sizeof(inplace_)); MDBX_CONSTEXPR_ASSERT(is_inplace()); } else if (is_inplace()) - make_allocated(ditto.allocated_.ptr_, ditto.allocated_.capacity_bytes_); + make_allocated(ditto.allocated_ptr_, ditto.capacity_.bytes_); else - make_allocated(ditto.allocated_.ptr_, ditto.allocated_.capacity_bytes_); + make_allocated(ditto.allocated_ptr_, ditto.capacity_.bytes_); return *this; } @@ -1656,12 +1677,12 @@ private: } constexpr const byte *address() const noexcept { - return is_inplace() ? inplace_ : static_cast(to_address(allocated_.ptr_)); + return is_inplace() ? inplace_.buffer_ : static_cast(to_address(allocated_ptr_)); } MDBX_CXX17_CONSTEXPR byte *address() noexcept { - return is_inplace() ? inplace_ : static_cast(to_address(allocated_.ptr_)); + return is_inplace() ? inplace_.buffer_ : static_cast(to_address(allocated_ptr_)); } - constexpr size_t capacity() const noexcept { return is_inplace() ? sizeof(bin) - 1 : allocated_.capacity_bytes_; } + constexpr size_t capacity() const noexcept { return is_inplace() ? sizeof(bin) - 1 : capacity_.bytes_; } } bin_; MDBX_CXX20_CONSTEXPR void *init(size_t capacity) { @@ -1678,7 +1699,7 @@ private: MDBX_CXX20_CONSTEXPR void release() noexcept { if (bin_.is_allocated()) { - deallocate_storage(bin_.allocated_.ptr_, bin_.allocated_.capacity_bytes_); + deallocate_storage(bin_.allocated_ptr_, bin_.capacity_.bytes_); bin_.template make_inplace(); } } @@ -1709,7 +1730,7 @@ private: if (bin::is_suitable_for_inplace(new_capacity)) { assert(bin_.is_allocated()); - const auto old_allocated = ::std::move(bin_.allocated_.ptr_); + const auto old_allocated = ::std::move(bin_.allocated_ptr_); byte *const new_place = bin_.template make_inplace() + wanna_headroom; if (MDBX_LIKELY(length)) MDBX_CXX20_LIKELY memcpy(new_place, content, length); @@ -1727,7 +1748,7 @@ private: return new_place; } - const auto old_allocated = ::std::move(bin_.allocated_.ptr_); + const auto old_allocated = ::std::move(bin_.allocated_ptr_); if (external_content) deallocate_storage(old_allocated, old_capacity); const auto pair = allocate_storage(new_capacity); From c5936eb5da73f1c70091707c895a4fd3b4805253 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:41:32 +0300 Subject: [PATCH 65/90] =?UTF-8?q?mdbx++:=20=D1=83=D0=B4=D0=B0=D0=BB=D0=B5?= =?UTF-8?q?=D0=BD=D0=B8=D0=B5=20=D0=B8=D1=81=D0=BA=D0=BB=D1=8E=D1=87=D0=B5?= =?UTF-8?q?=D0=BD=D0=B8=D1=8F=20=D0=BF=D1=80=D0=B8=20=D0=B7=D0=B0=D0=BF?= =?UTF-8?q?=D1=80=D0=BE=D1=81=D0=B5=20=D1=82=D1=80=D0=B0=D0=BD=D0=B7=D0=B0?= =?UTF-8?q?=D0=BA=D1=86=D0=B8=D0=B8=20=D1=83=20=D0=BE=D1=82=D1=81=D0=BE?= =?UTF-8?q?=D0=B5=D0=B4=D0=B8=D0=BD=D1=91=D0=BD=D0=BD=D0=BE=D0=B3=D0=BE=20?= =?UTF-8?q?=D0=BA=D1=83=D1=80=D1=81=D0=BE=D1=80=D0=B0=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- mdbx.h++ | 1 - 1 file changed, 1 deletion(-) diff --git a/mdbx.h++ b/mdbx.h++ index 97120b3f..e6186b1e 100644 --- a/mdbx.h++ +++ b/mdbx.h++ @@ -6206,7 +6206,6 @@ inline void cursor::unbind() { error::success_or_throw(::mdbx_cursor_unbind(hand inline txn cursor::txn() const { MDBX_txn *txn = ::mdbx_cursor_txn(handle_); - error::throw_on_nullptr(txn, MDBX_EINVAL); return ::mdbx::txn(txn); } From 682233ba28488827c3eb15291ed1c9cbdb218f73 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:41:58 +0300 Subject: [PATCH 66/90] =?UTF-8?q?mdbx++:=20=D0=BF=D0=B5=D1=80=D0=B5=D1=84?= =?UTF-8?q?=D0=BE=D1=80=D0=BC=D0=B0=D1=82=D0=B8=D1=80=D0=BE=D0=B2=D0=B0?= =?UTF-8?q?=D0=BD=D0=B8=D0=B5=20(=D0=B2=D1=80=D0=B5=D0=BC=D0=B5=D0=BD?= =?UTF-8?q?=D0=BD=D0=BE)=20=D0=BD=D0=B5=D0=B8=D1=81=D0=BF=D0=BE=D0=BB?= =?UTF-8?q?=D1=8C=D0=B7=D1=83=D0=B5=D0=BC=D0=BE=D0=B3=D0=BE=20=D0=BA=D0=BE?= =?UTF-8?q?=D0=B4=D0=B0=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/mdbx.c++ | 25 +++++++++++-------------- 1 file changed, 11 insertions(+), 14 deletions(-) diff --git a/src/mdbx.c++ b/src/mdbx.c++ index 63f56190..7b4d8aa1 100644 --- a/src/mdbx.c++ +++ b/src/mdbx.c++ @@ -63,8 +63,8 @@ class trouble_location { #endif public: - MDBX_CXX11_CONSTEXPR trouble_location(unsigned line, const char *condition, - const char *function, const char *filename) + MDBX_CXX11_CONSTEXPR trouble_location(unsigned line, const char *condition, const char *function, + const char *filename) : #if TROUBLE_PROVIDE_LINENO line_(line) @@ -133,7 +133,7 @@ public: //------------------------------------------------------------------------------ -__cold std::string format_va(const char *fmt, va_list ap) { +__cold std::string format_va(const char *fmt, va_list ap) { va_list ones; va_copy(ones, ap); #ifdef _MSC_VER @@ -146,15 +146,14 @@ __cold std::string format_va(const char *fmt, va_list ap) { result.reserve(size_t(needed + 1)); result.resize(size_t(needed), '\0'); assert(int(result.capacity()) > needed); - int actual = vsnprintf(const_cast(result.data()), result.capacity(), - fmt, ones); + int actual = vsnprintf(const_cast(result.data()), result.capacity(), fmt, ones); assert(actual == needed); (void)actual; va_end(ones); return result; } -__cold std::string format(const char *fmt, ...) { +__cold std::string format(const char *fmt, ...) { va_list ap; va_start(ap, fmt); std::string result = format_va(fmt, ap); @@ -175,17 +174,14 @@ public: virtual ~bug() noexcept; }; -__cold bug::bug(const trouble_location &location) noexcept - : std::runtime_error(format("mdbx.bug: %s.%s at %s:%u", location.function(), - location.condition(), location.filename(), - location.line())), +__cold bug::bug(const trouble_location &location) noexcept + : std::runtime_error(format("mdbx.bug: %s.%s at %s:%u", location.function(), location.condition(), + location.filename(), location.line())), location_(location) {} -__cold bug::~bug() noexcept {} +__cold bug::~bug() noexcept {} -[[noreturn]] __cold void raise_bug(const trouble_location &what_and_where) { - throw bug(what_and_where); -} +[[maybe_unused, noreturn]] __cold void raise_bug(const trouble_location &what_and_where) { throw bug(what_and_where); } #define RAISE_BUG(line, condition, function, file) \ do { \ @@ -193,6 +189,7 @@ __cold bug::~bug() noexcept {} raise_bug(bug); \ } while (0) +#undef ENSURE #define ENSURE(condition) \ do \ if (MDBX_UNLIKELY(!(condition))) \ From 5fd319bbc2913b452b48ce5dc06958694044e46c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:42:50 +0300 Subject: [PATCH 67/90] =?UTF-8?q?mdbx:=20=D0=BF=D0=B5=D1=80=D0=B5=D1=80?= =?UTF-8?q?=D0=B0=D0=B1=D0=BE=D1=82=D0=BA=D0=B0=20`mdbx=5Ftxn=5Frelease=5F?= =?UTF-8?q?all=5Fcursors=5Fex()`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- mdbx.h | 26 ++++++++++-------------- src/api-cursor.c | 51 +++++++++++++++++++++++++++++++----------------- 2 files changed, 43 insertions(+), 34 deletions(-) diff --git a/mdbx.h b/mdbx.h index 0ae5921a..ae2af5ad 100644 --- a/mdbx.h +++ b/mdbx.h @@ -5261,15 +5261,12 @@ LIBMDBX_API void mdbx_cursor_close(MDBX_cursor *cursor); * \retval MDBX_EINVAL An invalid parameter was specified. */ LIBMDBX_API int mdbx_cursor_close2(MDBX_cursor *cursor); -/** \brief Unbind or closes all cursors of a given transaction. +/** \brief Unbind or closes all cursors of a given transaction and of all + * its parent transactions if ones are. * \ingroup c_cursors * - * Unbinds either closes all cursors associated (opened or renewed) with - * a given transaction in a bulk with minimal overhead. - * - * A transaction should not be nested, since in this case no way to restore - * state if this nested transaction will be aborted, nor impossible to define - * the expected behavior. + * Unbinds either closes all cursors associated (opened, renewed or binded) with + * the given transaction in a bulk with minimal overhead. * * \see mdbx_cursor_unbind() * \see mdbx_cursor_close() @@ -5284,19 +5281,16 @@ LIBMDBX_API int mdbx_cursor_close2(MDBX_cursor *cursor); * some possible errors are: * \retval MDBX_THREAD_MISMATCH Given transaction is not owned * by current thread. - * \retval MDBX_BAD_TXN Given transaction is invalid, nested or has + * \retval MDBX_BAD_TXN Given transaction is invalid or has * a child/nested transaction transaction. */ LIBMDBX_API int mdbx_txn_release_all_cursors_ex(const MDBX_txn *txn, bool unbind, size_t *count); -/** \brief Unbind or closes all cursors of a given transaction. +/** \brief Unbind or closes all cursors of a given transaction and of all + * its parent transactions if ones are. * \ingroup c_cursors * - * Unbinds either closes all cursors associated (opened or renewed) with - * a given transaction in a bulk with minimal overhead. - * - * A transaction should not be nested, since in this case no way to restore - * state if this nested transaction will be aborted, nor impossible to define - * the expected behavior. + * Unbinds either closes all cursors associated (opened, renewed or binded) with + * the given transaction in a bulk with minimal overhead. * * \see mdbx_cursor_unbind() * \see mdbx_cursor_close() @@ -5309,7 +5303,7 @@ LIBMDBX_API int mdbx_txn_release_all_cursors_ex(const MDBX_txn *txn, bool unbind * some possible errors are: * \retval MDBX_THREAD_MISMATCH Given transaction is not owned * by current thread. - * \retval MDBX_BAD_TXN Given transaction is invalid, nested or has + * \retval MDBX_BAD_TXN Given transaction is invalid or has * a child/nested transaction transaction. */ LIBMDBX_INLINE_API(int, mdbx_txn_release_all_cursors, (const MDBX_txn *txn, bool unbind)) { return mdbx_txn_release_all_cursors_ex(txn, unbind, NULL); diff --git a/src/api-cursor.c b/src/api-cursor.c index d86bc34e..24288fb5 100644 --- a/src/api-cursor.c +++ b/src/api-cursor.c @@ -233,28 +233,43 @@ int mdbx_txn_release_all_cursors_ex(const MDBX_txn *txn, bool unbind, size_t *co int rc = check_txn(txn, MDBX_TXN_FINISHED | MDBX_TXN_HAS_CHILD); if (unlikely(rc != MDBX_SUCCESS)) return LOG_IFERR(rc); - if (unlikely(txn->parent)) { - rc = MDBX_BAD_TXN; - ERROR("%s, err %d", "must not unbind or close cursors for a nested txn", rc); - return rc; - } size_t n = 0; - TXN_FOREACH_DBI_FROM(txn, i, MAIN_DBI) { - while (txn->cursors[i]) { - ++n; - MDBX_cursor *mc = txn->cursors[i]; - ENSURE(nullptr, mc->signature == cur_signature_live && (mc->next != mc) && !mc->backup); - txn->cursors[i] = mc->next; - mc->next = mc; - mc->signature = cur_signature_ready4dispose; - cursor_drown((cursor_couple_t *)mc); - if (!unbind) { - mc->signature = 0; - osal_free(mc); + do { + TXN_FOREACH_DBI_FROM(txn, i, MAIN_DBI) { + MDBX_cursor *mc = txn->cursors[i], *next = nullptr; + if (mc) { + txn->cursors[i] = nullptr; + do { + next = mc->next; + if (mc->signature == cur_signature_live) { + mc->signature = cur_signature_wait4eot; + cursor_drown((cursor_couple_t *)mc); + } else + ENSURE(nullptr, mc->signature == cur_signature_wait4eot); + if (mc->backup) { + MDBX_cursor *bk = mc->backup; + mc->next = bk->next; + mc->backup = bk->backup; + mc->backup = nullptr; + bk->signature = 0; + bk = bk->next; + osal_free(bk); + } else { + mc->signature = cur_signature_ready4dispose; + mc->next = mc; + ++n; + if (!unbind) { + mc->signature = 0; + osal_free(mc); + } + } + } while ((mc = next) != nullptr); } } - } + txn = txn->parent; + } while (txn); + if (count) *count = n; return MDBX_SUCCESS; From 6c036add8bc0eec3461e6a2c93df85c29c59daf8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:44:20 +0300 Subject: [PATCH 68/90] =?UTF-8?q?mdbx:=20=D0=BF=D0=B5=D1=80=D0=B5=D1=80?= =?UTF-8?q?=D0=B0=D0=B1=D0=BE=D1=82=D0=BA=D0=B0=20=D0=BF=D1=80=D0=BE=D0=B2?= =?UTF-8?q?=D0=B5=D1=80=D0=BA=D0=B8=20=D0=BA=D1=83=D1=80=D1=81=D0=BE=D1=80?= =?UTF-8?q?=D0=BE=D0=B2=20=D0=BD=D0=B0=20=D0=B2=D1=85=D0=BE=D0=B4=D0=B5=20?= =?UTF-8?q?API-=D1=84=D1=83=D0=BD=D0=BA=D1=86=D0=B8=D0=B9=20=D1=81=20?= =?UTF-8?q?=D0=B4=D0=BE=D0=B1=D0=B0=D0=B2=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5?= =?UTF-8?q?=D0=BC=20`cursor=5Fcheck()`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/api-cold.c | 2 +- src/api-cursor.c | 155 ++++++++++++--------------------------- src/api-range-estimate.c | 34 ++++----- src/api-txn-data.c | 57 ++++++-------- src/api-txn.c | 12 ++- src/cogs.h | 14 +--- src/cursor.c | 30 ++++++++ src/cursor.h | 15 ++++ src/txn.c | 5 +- 9 files changed, 149 insertions(+), 175 deletions(-) diff --git a/src/api-cold.c b/src/api-cold.c index 8a8c8588..57e1d667 100644 --- a/src/api-cold.c +++ b/src/api-cold.c @@ -141,7 +141,7 @@ __cold int mdbx_env_warmup(const MDBX_env *env, const MDBX_txn *txn, MDBX_warmup return LOG_IFERR(MDBX_EINVAL); if (txn) { - int err = check_txn(txn, MDBX_TXN_BLOCKED - MDBX_TXN_ERROR); + int err = check_txn(txn, MDBX_TXN_FINISHED | MDBX_TXN_ERROR); if (unlikely(err != MDBX_SUCCESS)) return LOG_IFERR(err); } diff --git a/src/api-cursor.c b/src/api-cursor.c index 24288fb5..18396299 100644 --- a/src/api-cursor.c +++ b/src/api-cursor.c @@ -27,18 +27,12 @@ int mdbx_cursor_renew(MDBX_txn *txn, MDBX_cursor *mc) { } int mdbx_cursor_reset(MDBX_cursor *mc) { - if (unlikely(!mc)) - return LOG_IFERR(MDBX_EINVAL); + int rc = cursor_check(mc, MDBX_TXN_FINISHED); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); - if (likely(mc->signature == cur_signature_live)) { - cursor_reset((cursor_couple_t *)mc); - return MDBX_SUCCESS; - } - - if (likely(mc->signature == cur_signature_ready4dispose)) - return MDBX_SUCCESS; - - return LOG_IFERR(MDBX_EBADSIGN); + cursor_reset((cursor_couple_t *)mc); + return MDBX_SUCCESS; } int mdbx_cursor_bind(MDBX_txn *txn, MDBX_cursor *mc, MDBX_dbi dbi) { @@ -50,17 +44,17 @@ int mdbx_cursor_bind(MDBX_txn *txn, MDBX_cursor *mc, MDBX_dbi dbi) { return LOG_IFERR(rc); } - int rc = check_txn(txn, MDBX_TXN_BLOCKED); - if (unlikely(rc != MDBX_SUCCESS)) - return LOG_IFERR(rc); - - rc = dbi_check(txn, dbi); + int rc = check_txn(txn, MDBX_TXN_FINISHED | MDBX_TXN_HAS_CHILD); if (unlikely(rc != MDBX_SUCCESS)) return LOG_IFERR(rc); if (unlikely(dbi == FREE_DBI && !(txn->flags & MDBX_TXN_RDONLY))) return LOG_IFERR(MDBX_EACCESS); + rc = dbi_check(txn, dbi); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); + if (unlikely(mc->backup)) /* Cursor from parent transaction */ LOG_IFERR(MDBX_EINVAL); @@ -199,12 +193,11 @@ int mdbx_cursor_close2(MDBX_cursor *mc) { } int mdbx_cursor_copy(const MDBX_cursor *src, MDBX_cursor *dest) { - if (unlikely(!src)) - return LOG_IFERR(MDBX_EINVAL); - if (unlikely(src->signature != cur_signature_live)) - return LOG_IFERR((src->signature == cur_signature_ready4dispose) ? MDBX_EINVAL : MDBX_EBADSIGN); + int rc = cursor_check(src, MDBX_TXN_FINISHED | MDBX_TXN_HAS_CHILD); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); - int rc = mdbx_cursor_bind(src->txn, dest, cursor_dbi(src)); + rc = mdbx_cursor_bind(src->txn, dest, cursor_dbi(src)); if (unlikely(rc != MDBX_SUCCESS)) return rc; @@ -277,15 +270,16 @@ int mdbx_txn_release_all_cursors_ex(const MDBX_txn *txn, bool unbind, size_t *co int mdbx_cursor_compare(const MDBX_cursor *l, const MDBX_cursor *r, bool ignore_multival) { const int incomparable = INT16_MAX + 1; + if (unlikely(!l)) return r ? -incomparable * 9 : 0; else if (unlikely(!r)) return incomparable * 9; - if (unlikely(l->signature != cur_signature_live)) - return (r->signature == cur_signature_live) ? -incomparable * 8 : 0; - if (unlikely(r->signature != cur_signature_live)) - return (l->signature == cur_signature_live) ? incomparable * 8 : 0; + if (unlikely(cursor_check_pure(l) != MDBX_SUCCESS)) + return (cursor_check_pure(r) == MDBX_SUCCESS) ? -incomparable * 8 : 0; + if (unlikely(cursor_check_pure(r) != MDBX_SUCCESS)) + return (cursor_check_pure(l) == MDBX_SUCCESS) ? incomparable * 8 : 0; if (unlikely(l->clc != r->clc)) { if (l->txn->env != r->txn->env) @@ -351,13 +345,7 @@ int mdbx_cursor_compare(const MDBX_cursor *l, const MDBX_cursor *r, bool ignore_ } int mdbx_cursor_count_ex(const MDBX_cursor *mc, size_t *count, MDBX_stat *ns, size_t bytes) { - if (unlikely(mc == nullptr)) - return LOG_IFERR(MDBX_EINVAL); - - if (unlikely(mc->signature != cur_signature_live)) - return LOG_IFERR((mc->signature == cur_signature_ready4dispose) ? MDBX_EINVAL : MDBX_EBADSIGN); - - int rc = check_txn(mc->txn, MDBX_TXN_BLOCKED); + int rc = cursor_check_ro(mc); if (unlikely(rc != MDBX_SUCCESS)) return LOG_IFERR(rc); @@ -407,11 +395,9 @@ int mdbx_cursor_count(const MDBX_cursor *mc, size_t *count) { } int mdbx_cursor_on_first(const MDBX_cursor *mc) { - if (unlikely(mc == nullptr)) - return LOG_IFERR(MDBX_EINVAL); - - if (unlikely(mc->signature != cur_signature_live)) - return LOG_IFERR((mc->signature == cur_signature_ready4dispose) ? MDBX_EINVAL : MDBX_EBADSIGN); + int rc = cursor_check_pure(mc); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); for (intptr_t i = 0; i <= mc->top; ++i) { if (mc->ki[i]) @@ -422,11 +408,9 @@ int mdbx_cursor_on_first(const MDBX_cursor *mc) { } int mdbx_cursor_on_first_dup(const MDBX_cursor *mc) { - if (unlikely(mc == nullptr)) - return LOG_IFERR(MDBX_EINVAL); - - if (unlikely(mc->signature != cur_signature_live)) - return LOG_IFERR((mc->signature == cur_signature_ready4dispose) ? MDBX_EINVAL : MDBX_EBADSIGN); + int rc = cursor_check_pure(mc); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); if (is_filled(mc) && mc->subcur) { mc = &mc->subcur->cursor; @@ -440,11 +424,9 @@ int mdbx_cursor_on_first_dup(const MDBX_cursor *mc) { } int mdbx_cursor_on_last(const MDBX_cursor *mc) { - if (unlikely(mc == nullptr)) - return LOG_IFERR(MDBX_EINVAL); - - if (unlikely(mc->signature != cur_signature_live)) - return LOG_IFERR((mc->signature == cur_signature_ready4dispose) ? MDBX_EINVAL : MDBX_EBADSIGN); + int rc = cursor_check_pure(mc); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); for (intptr_t i = 0; i <= mc->top; ++i) { size_t nkeys = page_numkeys(mc->pg[i]); @@ -456,11 +438,9 @@ int mdbx_cursor_on_last(const MDBX_cursor *mc) { } int mdbx_cursor_on_last_dup(const MDBX_cursor *mc) { - if (unlikely(mc == nullptr)) - return LOG_IFERR(MDBX_EINVAL); - - if (unlikely(mc->signature != cur_signature_live)) - return LOG_IFERR((mc->signature == cur_signature_ready4dispose) ? MDBX_EINVAL : MDBX_EBADSIGN); + int rc = cursor_check_pure(mc); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); if (is_filled(mc) && mc->subcur) { mc = &mc->subcur->cursor; @@ -475,29 +455,18 @@ int mdbx_cursor_on_last_dup(const MDBX_cursor *mc) { } int mdbx_cursor_eof(const MDBX_cursor *mc) { - if (unlikely(mc == nullptr)) - return LOG_IFERR(MDBX_EINVAL); - - if (unlikely(mc->signature != cur_signature_live)) - return LOG_IFERR((mc->signature == cur_signature_ready4dispose) ? MDBX_EINVAL : MDBX_EBADSIGN); + int rc = cursor_check_pure(mc); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); return is_eof(mc) ? MDBX_RESULT_TRUE : MDBX_RESULT_FALSE; } int mdbx_cursor_get(MDBX_cursor *mc, MDBX_val *key, MDBX_val *data, MDBX_cursor_op op) { - if (unlikely(mc == nullptr)) - return LOG_IFERR(MDBX_EINVAL); - - if (unlikely(mc->signature != cur_signature_live)) - return LOG_IFERR((mc->signature == cur_signature_ready4dispose) ? MDBX_EINVAL : MDBX_EBADSIGN); - - int rc = check_txn(mc->txn, MDBX_TXN_BLOCKED); + int rc = cursor_check_ro(mc); if (unlikely(rc != MDBX_SUCCESS)) return LOG_IFERR(rc); - if (unlikely(cursor_dbi_changed(mc))) - return LOG_IFERR(MDBX_BAD_DBI); - return LOG_IFERR(cursor_ops(mc, key, data, op)); } @@ -622,19 +591,13 @@ int mdbx_cursor_get_batch(MDBX_cursor *mc, size_t *count, MDBX_val *pairs, size_ return LOG_IFERR(MDBX_EINVAL); *count = 0; - if (unlikely(mc == nullptr || limit < 4 || limit > INTPTR_MAX - 2)) + if (unlikely(limit < 4 || limit > INTPTR_MAX - 2)) return LOG_IFERR(MDBX_EINVAL); - if (unlikely(mc->signature != cur_signature_live)) - return LOG_IFERR((mc->signature == cur_signature_ready4dispose) ? MDBX_EINVAL : MDBX_EBADSIGN); - - int rc = check_txn(mc->txn, MDBX_TXN_BLOCKED); + int rc = cursor_check_ro(mc); if (unlikely(rc != MDBX_SUCCESS)) return LOG_IFERR(rc); - if (unlikely(cursor_dbi_changed(mc))) - return LOG_IFERR(MDBX_BAD_DBI); - if (unlikely(mc->subcur)) return LOG_IFERR(MDBX_INCOMPATIBLE) /* must be a non-dupsort table */; @@ -703,11 +666,9 @@ bailout: /*----------------------------------------------------------------------------*/ int mdbx_cursor_set_userctx(MDBX_cursor *mc, void *ctx) { - if (unlikely(!mc)) - return LOG_IFERR(MDBX_EINVAL); - - if (unlikely(mc->signature != cur_signature_ready4dispose && mc->signature != cur_signature_live)) - return LOG_IFERR(MDBX_EBADSIGN); + int rc = cursor_check(mc, 0); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); cursor_couple_t *couple = container_of(mc, cursor_couple_t, outer); couple->userctx = ctx; @@ -745,21 +706,13 @@ MDBX_dbi mdbx_cursor_dbi(const MDBX_cursor *mc) { /*----------------------------------------------------------------------------*/ int mdbx_cursor_put(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data, MDBX_put_flags_t flags) { - if (unlikely(mc == nullptr || key == nullptr || data == nullptr)) + if (unlikely(key == nullptr || data == nullptr)) return LOG_IFERR(MDBX_EINVAL); - if (unlikely(mc->signature != cur_signature_live)) - return LOG_IFERR((mc->signature == cur_signature_ready4dispose) ? MDBX_EINVAL : MDBX_EBADSIGN); - - int rc = check_txn_rw(mc->txn, MDBX_TXN_BLOCKED); + int rc = cursor_check_rw(mc); if (unlikely(rc != MDBX_SUCCESS)) return LOG_IFERR(rc); - if (unlikely(cursor_dbi_changed(mc))) - return LOG_IFERR(MDBX_BAD_DBI); - - cASSERT(mc, cursor_is_tracked(mc)); - /* Check this first so counter will always be zero on any early failures. */ if (unlikely(flags & MDBX_MULTIPLE)) { if (unlikely(flags & MDBX_RESERVE)) @@ -784,35 +737,21 @@ int mdbx_cursor_put(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data, MDBX_p data->iov_base = nullptr; } - if (unlikely(mc->txn->flags & (MDBX_TXN_RDONLY | MDBX_TXN_BLOCKED))) - return LOG_IFERR((mc->txn->flags & MDBX_TXN_RDONLY) ? MDBX_EACCESS : MDBX_BAD_TXN); - return LOG_IFERR(cursor_put_checklen(mc, key, data, flags)); } int mdbx_cursor_del(MDBX_cursor *mc, MDBX_put_flags_t flags) { - if (unlikely(!mc)) - return LOG_IFERR(MDBX_EINVAL); - - if (unlikely(mc->signature != cur_signature_live)) - return LOG_IFERR((mc->signature == cur_signature_ready4dispose) ? MDBX_EINVAL : MDBX_EBADSIGN); - - int rc = check_txn_rw(mc->txn, MDBX_TXN_BLOCKED); + int rc = cursor_check_rw(mc); if (unlikely(rc != MDBX_SUCCESS)) return LOG_IFERR(rc); - if (unlikely(cursor_dbi_changed(mc))) - return LOG_IFERR(MDBX_BAD_DBI); - return LOG_IFERR(cursor_del(mc, flags)); } __cold int mdbx_cursor_ignord(MDBX_cursor *mc) { - if (unlikely(!mc)) - return LOG_IFERR(MDBX_EINVAL); - - if (unlikely(mc->signature != cur_signature_live)) - return LOG_IFERR((mc->signature == cur_signature_ready4dispose) ? MDBX_EINVAL : MDBX_EBADSIGN); + int rc = cursor_check(mc, 0); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); mc->checking |= z_ignord; if (mc->subcur) diff --git a/src/api-range-estimate.c b/src/api-range-estimate.c index 5356d4da..56564da0 100644 --- a/src/api-range-estimate.c +++ b/src/api-range-estimate.c @@ -16,12 +16,6 @@ __hot static int cursor_diff(const MDBX_cursor *const __restrict x, const MDBX_c r->level = 0; r->root_nkeys = 0; - if (unlikely(x->signature != cur_signature_live)) - return (x->signature == cur_signature_ready4dispose) ? MDBX_EINVAL : MDBX_EBADSIGN; - - if (unlikely(y->signature != cur_signature_live)) - return (y->signature == cur_signature_ready4dispose) ? MDBX_EINVAL : MDBX_EBADSIGN; - int rc = check_txn(x->txn, MDBX_TXN_BLOCKED); if (unlikely(rc != MDBX_SUCCESS)) return rc; @@ -146,12 +140,20 @@ __hot static ptrdiff_t estimate(const tree_t *tree, diff_t *const __restrict dr) * Range-Estimation API */ __hot int mdbx_estimate_distance(const MDBX_cursor *first, const MDBX_cursor *last, ptrdiff_t *distance_items) { - if (unlikely(first == nullptr || last == nullptr || distance_items == nullptr)) + if (unlikely(!distance_items)) return LOG_IFERR(MDBX_EINVAL); + int rc = cursor_check_pure(first); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); + + rc = cursor_check_pure(last); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); + *distance_items = 0; diff_t dr; - int rc = cursor_diff(last, first, &dr); + rc = cursor_diff(last, first, &dr); if (unlikely(rc != MDBX_SUCCESS)) return LOG_IFERR(rc); @@ -172,14 +174,10 @@ __hot int mdbx_estimate_distance(const MDBX_cursor *first, const MDBX_cursor *la __hot int mdbx_estimate_move(const MDBX_cursor *cursor, MDBX_val *key, MDBX_val *data, MDBX_cursor_op move_op, ptrdiff_t *distance_items) { - if (unlikely(cursor == nullptr || distance_items == nullptr || move_op == MDBX_GET_CURRENT || - move_op == MDBX_GET_MULTIPLE)) + if (unlikely(!distance_items || move_op == MDBX_GET_CURRENT || move_op == MDBX_GET_MULTIPLE)) return LOG_IFERR(MDBX_EINVAL); - if (unlikely(cursor->signature != cur_signature_live)) - return LOG_IFERR((cursor->signature == cur_signature_ready4dispose) ? MDBX_EINVAL : MDBX_EBADSIGN); - - int rc = check_txn(cursor->txn, MDBX_TXN_BLOCKED); + int rc = cursor_check_ro(cursor); if (unlikely(rc != MDBX_SUCCESS)) return LOG_IFERR(rc); @@ -232,10 +230,6 @@ __hot int mdbx_estimate_move(const MDBX_cursor *cursor, MDBX_val *key, MDBX_val __hot int mdbx_estimate_range(const MDBX_txn *txn, MDBX_dbi dbi, const MDBX_val *begin_key, const MDBX_val *begin_data, const MDBX_val *end_key, const MDBX_val *end_data, ptrdiff_t *size_items) { - int rc = check_txn(txn, MDBX_TXN_BLOCKED); - if (unlikely(rc != MDBX_SUCCESS)) - return LOG_IFERR(rc); - if (unlikely(!size_items)) return LOG_IFERR(MDBX_EINVAL); @@ -248,6 +242,10 @@ __hot int mdbx_estimate_range(const MDBX_txn *txn, MDBX_dbi dbi, const MDBX_val if (unlikely(begin_key == MDBX_EPSILON && end_key == MDBX_EPSILON)) return LOG_IFERR(MDBX_EINVAL); + int rc = check_txn(txn, MDBX_TXN_BLOCKED); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); + cursor_couple_t begin; /* LY: first, initialize cursor to refresh a DB in case it have DB_STALE */ rc = cursor_init(&begin.outer, txn, dbi); diff --git a/src/api-txn-data.c b/src/api-txn-data.c index 76bdf32a..2be54df8 100644 --- a/src/api-txn-data.c +++ b/src/api-txn-data.c @@ -51,15 +51,15 @@ __cold int mdbx_dbi_dupsort_depthmask(const MDBX_txn *txn, MDBX_dbi dbi, uint32_ } int mdbx_canary_get(const MDBX_txn *txn, MDBX_canary *canary) { - int rc = check_txn(txn, MDBX_TXN_BLOCKED); + if (unlikely(canary == nullptr)) + return LOG_IFERR(MDBX_EINVAL); + + int rc = check_txn(txn, MDBX_TXN_BLOCKED - MDBX_TXN_PARKED); if (unlikely(rc != MDBX_SUCCESS)) { memset(canary, 0, sizeof(*canary)); return LOG_IFERR(rc); } - if (unlikely(canary == nullptr)) - return LOG_IFERR(MDBX_EINVAL); - *canary = txn->canary; return MDBX_SUCCESS; } @@ -68,13 +68,13 @@ int mdbx_get(const MDBX_txn *txn, MDBX_dbi dbi, const MDBX_val *key, MDBX_val *d DKBUF_DEBUG; DEBUG("===> get db %u key [%s]", dbi, DKEY_DEBUG(key)); + if (unlikely(!key || !data)) + return LOG_IFERR(MDBX_EINVAL); + int rc = check_txn(txn, MDBX_TXN_BLOCKED); if (unlikely(rc != MDBX_SUCCESS)) return LOG_IFERR(rc); - if (unlikely(!key || !data)) - return LOG_IFERR(MDBX_EINVAL); - cursor_couple_t cx; rc = cursor_init(&cx.outer, txn, dbi); if (unlikely(rc != MDBX_SUCCESS)) @@ -84,15 +84,12 @@ int mdbx_get(const MDBX_txn *txn, MDBX_dbi dbi, const MDBX_val *key, MDBX_val *d } int mdbx_get_equal_or_great(const MDBX_txn *txn, MDBX_dbi dbi, MDBX_val *key, MDBX_val *data) { - int rc = check_txn(txn, MDBX_TXN_BLOCKED); - if (unlikely(rc != MDBX_SUCCESS)) - return LOG_IFERR(rc); - if (unlikely(!key || !data)) return LOG_IFERR(MDBX_EINVAL); - if (unlikely(txn->flags & MDBX_TXN_BLOCKED)) - return LOG_IFERR(MDBX_BAD_TXN); + int rc = check_txn(txn, MDBX_TXN_BLOCKED); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); cursor_couple_t cx; rc = cursor_init(&cx.outer, txn, dbi); @@ -106,13 +103,13 @@ int mdbx_get_ex(const MDBX_txn *txn, MDBX_dbi dbi, MDBX_val *key, MDBX_val *data DKBUF_DEBUG; DEBUG("===> get db %u key [%s]", dbi, DKEY_DEBUG(key)); + if (unlikely(!key || !data)) + return LOG_IFERR(MDBX_EINVAL); + int rc = check_txn(txn, MDBX_TXN_BLOCKED); if (unlikely(rc != MDBX_SUCCESS)) return LOG_IFERR(rc); - if (unlikely(!key || !data)) - return LOG_IFERR(MDBX_EINVAL); - cursor_couple_t cx; rc = cursor_init(&cx.outer, txn, dbi); if (unlikely(rc != MDBX_SUCCESS)) @@ -179,7 +176,7 @@ int mdbx_canary_put(MDBX_txn *txn, const MDBX_canary *canary) { * расположен в той-же странице памяти, в том числе для многостраничных * P_LARGE страниц с длинными данными. */ int mdbx_is_dirty(const MDBX_txn *txn, const void *ptr) { - int rc = check_txn(txn, MDBX_TXN_BLOCKED); + int rc = check_txn(txn, MDBX_TXN_BLOCKED - MDBX_TXN_PARKED); if (unlikely(rc != MDBX_SUCCESS)) return LOG_IFERR(rc); @@ -215,18 +212,15 @@ int mdbx_is_dirty(const MDBX_txn *txn, const void *ptr) { } int mdbx_del(MDBX_txn *txn, MDBX_dbi dbi, const MDBX_val *key, const MDBX_val *data) { - int rc = check_txn_rw(txn, MDBX_TXN_BLOCKED); - if (unlikely(rc != MDBX_SUCCESS)) - return LOG_IFERR(rc); - if (unlikely(!key)) return LOG_IFERR(MDBX_EINVAL); if (unlikely(dbi <= FREE_DBI)) return LOG_IFERR(MDBX_BAD_DBI); - if (unlikely(txn->flags & (MDBX_TXN_RDONLY | MDBX_TXN_BLOCKED))) - return LOG_IFERR((txn->flags & MDBX_TXN_RDONLY) ? MDBX_EACCESS : MDBX_BAD_TXN); + int rc = check_txn_rw(txn, MDBX_TXN_BLOCKED); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); cursor_couple_t cx; rc = cursor_init(&cx.outer, txn, dbi); @@ -254,10 +248,6 @@ int mdbx_del(MDBX_txn *txn, MDBX_dbi dbi, const MDBX_val *key, const MDBX_val *d } int mdbx_put(MDBX_txn *txn, MDBX_dbi dbi, const MDBX_val *key, MDBX_val *data, MDBX_put_flags_t flags) { - int rc = check_txn_rw(txn, MDBX_TXN_BLOCKED); - if (unlikely(rc != MDBX_SUCCESS)) - return LOG_IFERR(rc); - if (unlikely(!key || !data)) return LOG_IFERR(MDBX_EINVAL); @@ -268,8 +258,9 @@ int mdbx_put(MDBX_txn *txn, MDBX_dbi dbi, const MDBX_val *key, MDBX_val *data, M MDBX_APPENDDUP | MDBX_CURRENT | MDBX_MULTIPLE))) return LOG_IFERR(MDBX_EINVAL); - if (unlikely(txn->flags & (MDBX_TXN_RDONLY | MDBX_TXN_BLOCKED))) - return LOG_IFERR((txn->flags & MDBX_TXN_RDONLY) ? MDBX_EACCESS : MDBX_BAD_TXN); + int rc = check_txn_rw(txn, MDBX_TXN_BLOCKED); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); cursor_couple_t cx; rc = cursor_init(&cx.outer, txn, dbi); @@ -330,10 +321,6 @@ int mdbx_put(MDBX_txn *txn, MDBX_dbi dbi, const MDBX_val *key, MDBX_val *data, M int mdbx_replace_ex(MDBX_txn *txn, MDBX_dbi dbi, const MDBX_val *key, MDBX_val *new_data, MDBX_val *old_data, MDBX_put_flags_t flags, MDBX_preserve_func preserver, void *preserver_context) { - int rc = check_txn_rw(txn, MDBX_TXN_BLOCKED); - if (unlikely(rc != MDBX_SUCCESS)) - return LOG_IFERR(rc); - if (unlikely(!key || !old_data || old_data == new_data)) return LOG_IFERR(MDBX_EINVAL); @@ -350,6 +337,10 @@ int mdbx_replace_ex(MDBX_txn *txn, MDBX_dbi dbi, const MDBX_val *key, MDBX_val * MDBX_APPENDDUP | MDBX_CURRENT))) return LOG_IFERR(MDBX_EINVAL); + int rc = check_txn_rw(txn, MDBX_TXN_BLOCKED); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); + cursor_couple_t cx; rc = cursor_init(&cx.outer, txn, dbi); if (unlikely(rc != MDBX_SUCCESS)) diff --git a/src/api-txn.c b/src/api-txn.c index 786e1e42..474f32fd 100644 --- a/src/api-txn.c +++ b/src/api-txn.c @@ -9,7 +9,7 @@ __attribute__((__no_sanitize_thread__, __noinline__)) #endif int mdbx_txn_straggler(const MDBX_txn *txn, int *percent) { - int rc = check_txn(txn, MDBX_TXN_BLOCKED); + int rc = check_txn(txn, MDBX_TXN_BLOCKED - MDBX_TXN_PARKED); if (unlikely(rc != MDBX_SUCCESS)) return LOG_IFERR((rc > 0) ? -rc : rc); @@ -200,9 +200,13 @@ int mdbx_txn_begin_ex(MDBX_env *env, MDBX_txn *parent, MDBX_txn_flags_t flags, M MDBX_txn *txn = nullptr; if (parent) { /* Nested transactions: Max 1 child, write txns only, no writemap */ - rc = check_txn_rw(parent, MDBX_TXN_RDONLY | MDBX_WRITEMAP | MDBX_TXN_BLOCKED); - if (unlikely(rc != MDBX_SUCCESS)) { - if (rc == MDBX_BAD_TXN && (parent->flags & (MDBX_TXN_RDONLY | MDBX_TXN_BLOCKED)) == 0) { + rc = check_txn(parent, MDBX_TXN_BLOCKED - MDBX_TXN_PARKED); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); + + if (unlikely(parent->flags & (MDBX_TXN_RDONLY | MDBX_WRITEMAP))) { + rc = MDBX_BAD_TXN; + if ((parent->flags & MDBX_TXN_RDONLY) == 0) { ERROR("%s mode is incompatible with nested transactions", "MDBX_WRITEMAP"); rc = MDBX_INCOMPATIBLE; } diff --git a/src/cogs.h b/src/cogs.h index 7e814e7b..498e92b7 100644 --- a/src/cogs.h +++ b/src/cogs.h @@ -416,10 +416,11 @@ static __always_inline int check_txn(const MDBX_txn *txn, int bad_bits) { return MDBX_EPERM; if (unlikely(txn->flags & bad_bits)) { + if ((bad_bits & MDBX_TXN_RDONLY) && unlikely(txn->flags & MDBX_TXN_RDONLY)) + return MDBX_EACCESS; if ((bad_bits & MDBX_TXN_PARKED) == 0) return MDBX_BAD_TXN; - else - return txn_check_badbits_parked(txn, bad_bits); + return txn_check_badbits_parked(txn, bad_bits); } } @@ -437,14 +438,7 @@ static __always_inline int check_txn(const MDBX_txn *txn, int bad_bits) { } static inline int check_txn_rw(const MDBX_txn *txn, int bad_bits) { - int err = check_txn(txn, bad_bits & ~MDBX_TXN_PARKED); - if (unlikely(err)) - return err; - - if (unlikely(txn->flags & MDBX_TXN_RDONLY)) - return MDBX_EACCESS; - - return MDBX_SUCCESS; + return check_txn(txn, (bad_bits | MDBX_TXN_RDONLY) & ~MDBX_TXN_PARKED); } /*----------------------------------------------------------------------------*/ diff --git a/src/cursor.c b/src/cursor.c index 01fc8a56..8a96387d 100644 --- a/src/cursor.c +++ b/src/cursor.c @@ -2329,3 +2329,33 @@ __hot int cursor_ops(MDBX_cursor *mc, MDBX_val *key, MDBX_val *data, const MDBX_ return MDBX_EINVAL; } } + +int cursor_check(const MDBX_cursor *mc, int txn_bad_bits) { + if (unlikely(mc == nullptr)) + return MDBX_EINVAL; + + if (unlikely(mc->signature != cur_signature_live)) { + if (mc->signature != cur_signature_ready4dispose) + return MDBX_EBADSIGN; + return (txn_bad_bits > MDBX_TXN_FINISHED) ? MDBX_EINVAL : MDBX_SUCCESS; + } + + /* проверяем что курсор в связном списке для отслеживания, исключение допускается только для read-only операций для + * служебных/временных курсоров на стеке. */ + MDBX_MAYBE_UNUSED char stack_top[sizeof(void *)]; + cASSERT(mc, cursor_is_tracked(mc) || (!(txn_bad_bits & MDBX_TXN_RDONLY) && stack_top < (char *)mc && + (char *)mc - stack_top < (ptrdiff_t)globals.sys_pagesize * 4)); + + if (txn_bad_bits) { + int rc = check_txn(mc->txn, txn_bad_bits); + if (unlikely(rc != MDBX_SUCCESS)) { + cASSERT(mc, rc != MDBX_RESULT_TRUE); + return rc; + } + + if (unlikely(cursor_dbi_changed(mc))) + return MDBX_BAD_DBI; + } + + return MDBX_SUCCESS; +} diff --git a/src/cursor.h b/src/cursor.h index 4fa0e585..7f109bcc 100644 --- a/src/cursor.h +++ b/src/cursor.h @@ -292,6 +292,21 @@ MDBX_NOTHROW_PURE_FUNCTION static inline bool check_leaf_type(const MDBX_cursor return (((page_type(mp) ^ mc->checking) & (z_branch | z_leaf | z_largepage | z_dupfix)) == 0); } +MDBX_INTERNAL int cursor_check(const MDBX_cursor *mc, int txn_bad_bits); + +/* без необходимости доступа к данным, без активации припаркованных транзакций. */ +static inline int cursor_check_pure(const MDBX_cursor *mc) { + return cursor_check(mc, MDBX_TXN_BLOCKED - MDBX_TXN_PARKED); +} + +/* для чтения данных, с активацией припаркованных транзакций. */ +static inline int cursor_check_ro(const MDBX_cursor *mc) { return cursor_check(mc, MDBX_TXN_BLOCKED); } + +/* для записи данных. */ +static inline int cursor_check_rw(const MDBX_cursor *mc) { + return cursor_check(mc, (MDBX_TXN_BLOCKED - MDBX_TXN_PARKED) | MDBX_TXN_RDONLY); +} + MDBX_INTERNAL MDBX_cursor *cursor_eot(MDBX_cursor *mc, MDBX_txn *txn, const bool merge); MDBX_INTERNAL int cursor_shadow(MDBX_cursor *mc, MDBX_txn *nested, const size_t dbi); diff --git a/src/txn.c b/src/txn.c index a83cd9c8..36ddaf64 100644 --- a/src/txn.c +++ b/src/txn.c @@ -995,7 +995,10 @@ int txn_check_badbits_parked(const MDBX_txn *txn, int bad_bits) { * - но при распарковке поломанные транзакции завершаются. * - получается что транзакцию можно припарковать, потом поломать вызвав * mdbx_txn_break(), но далее любое её использование приведет к завершению - * при распарковке. */ + * при распарковке. + * + * Поэтому для припаркованных транзакций возвращается ошибка если не-включена + * авто-распарковка, либо есть другие плохие биты. */ if ((txn->flags & (bad_bits | MDBX_TXN_AUTOUNPARK)) != (MDBX_TXN_PARKED | MDBX_TXN_AUTOUNPARK)) return LOG_IFERR(MDBX_BAD_TXN); From c457804fad3732f827506b530b00d8c796b9d916 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:46:13 +0300 Subject: [PATCH 69/90] =?UTF-8?q?mdbx:=20=D0=B8=D1=81=D0=BF=D1=80=D0=B0?= =?UTF-8?q?=D0=B2=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=B7=D0=B0=D1=82=D0=B5?= =?UTF-8?q?=D0=BD=D0=B5=D0=BD=D0=B8=D1=8F=20=D0=BA=D1=83=D1=80=D1=81=D0=BE?= =?UTF-8?q?=D1=80=D0=BE=D0=B2=20=D0=B2=D0=BE=20=D0=B2=D0=BB=D0=BE=D0=B6?= =?UTF-8?q?=D0=B5=D0=BD=D0=BD=D1=8B=D1=85=20=D1=82=D1=80=D0=B0=D0=BD=D0=B7?= =?UTF-8?q?=D0=B0=D0=BA=D1=86=D0=B8=D1=8F=D1=85=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/api-cursor.c | 6 ++---- src/cursor.c | 13 ++++++++++--- 2 files changed, 12 insertions(+), 7 deletions(-) diff --git a/src/api-cursor.c b/src/api-cursor.c index 18396299..f5f09c20 100644 --- a/src/api-cursor.c +++ b/src/api-cursor.c @@ -690,11 +690,9 @@ MDBX_txn *mdbx_cursor_txn(const MDBX_cursor *mc) { if (unlikely(!mc || mc->signature != cur_signature_live)) return nullptr; MDBX_txn *txn = mc->txn; - if (unlikely(!txn || txn->signature != txn_signature)) + if (unlikely(!txn || txn->signature != txn_signature || (txn->flags & MDBX_TXN_FINISHED))) return nullptr; - if (unlikely(txn->flags & MDBX_TXN_FINISHED)) - return nullptr; - return txn; + return (txn->flags & MDBX_TXN_HAS_CHILD) ? txn->env->txn : txn; } MDBX_dbi mdbx_cursor_dbi(const MDBX_cursor *mc) { diff --git a/src/cursor.c b/src/cursor.c index 8a96387d..b313a78f 100644 --- a/src/cursor.c +++ b/src/cursor.c @@ -2347,14 +2347,21 @@ int cursor_check(const MDBX_cursor *mc, int txn_bad_bits) { (char *)mc - stack_top < (ptrdiff_t)globals.sys_pagesize * 4)); if (txn_bad_bits) { - int rc = check_txn(mc->txn, txn_bad_bits); + int rc = check_txn(mc->txn, txn_bad_bits & ~MDBX_TXN_HAS_CHILD); if (unlikely(rc != MDBX_SUCCESS)) { cASSERT(mc, rc != MDBX_RESULT_TRUE); return rc; } - if (unlikely(cursor_dbi_changed(mc))) - return MDBX_BAD_DBI; + if (likely((mc->txn->flags & MDBX_TXN_HAS_CHILD) == 0)) + return likely(!cursor_dbi_changed(mc)) ? MDBX_SUCCESS : MDBX_BAD_DBI; + + cASSERT(mc, (mc->txn->flags & MDBX_TXN_RDONLY) == 0 && mc->txn != mc->txn->env->txn && mc->txn->env->txn); + rc = dbi_check(mc->txn->env->txn, cursor_dbi(mc)); + if (unlikely(rc != MDBX_SUCCESS)) + return rc; + + cASSERT(mc, (mc->txn->flags & MDBX_TXN_RDONLY) == 0 && mc->txn == mc->txn->env->txn); } return MDBX_SUCCESS; From 5f37ea60d2b45c531a1cd92df6d65d165964ca51 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:46:43 +0300 Subject: [PATCH 70/90] =?UTF-8?q?mdbx++:=20=D0=BF=D1=80=D0=BE=D0=B2=D0=B5?= =?UTF-8?q?=D1=80=D0=BA=D0=B0=20`=5F=5Fcpp=5Fconcepts=20>=3D=20202002`=20?= =?UTF-8?q?=D0=B4=D0=BB=D1=8F=20=D0=B8=D1=81=D0=BF=D0=BE=D0=BB=D1=8C=D0=B7?= =?UTF-8?q?=D0=BE=D0=B2=D0=B0=D0=BD=D0=B8=D1=8F=20=D0=BA=D0=BE=D0=BD=D1=86?= =?UTF-8?q?=D0=B5=D0=BF=D1=82=D0=BE=D0=B2=20C++=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- mdbx.h++ | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mdbx.h++ b/mdbx.h++ index e6186b1e..55bfc8e9 100644 --- a/mdbx.h++ +++ b/mdbx.h++ @@ -243,7 +243,7 @@ #endif /* MDBX_CXX20_UNLIKELY */ #ifndef MDBX_HAVE_CXX20_CONCEPTS -#if defined(__cpp_lib_concepts) && __cpp_lib_concepts >= 202002L +#if defined(__cpp_concepts) && __cpp_concepts >= 202002L && defined(__cpp_lib_concepts) && __cpp_lib_concepts >= 202002L #include #define MDBX_HAVE_CXX20_CONCEPTS 1 #elif defined(DOXYGEN) From 9c177de034386a2414af2eda1172e3891a44d0f5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:47:24 +0300 Subject: [PATCH 71/90] =?UTF-8?q?mdbx-tests:=20=D0=B4=D0=BE=D0=BF=D0=BE?= =?UTF-8?q?=D0=BB=D0=BD=D0=B5=D0=BD=D0=B8=D0=B5=20extra/txn=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- test/extra/txn.c++ | 539 ++++++++++++++++++++++++--------------------- 1 file changed, 287 insertions(+), 252 deletions(-) diff --git a/test/extra/txn.c++ b/test/extra/txn.c++ index 3a616015..e1abb1e6 100644 --- a/test/extra/txn.c++ +++ b/test/extra/txn.c++ @@ -1,4 +1,5 @@ #include "mdbx.h++" +#include MDBX_CONFIG_H #include @@ -25,267 +26,301 @@ static void logger_nofmt(MDBX_log_level_t loglevel, const char *function, int li fprintf(stdout, "%s:%u %s", function, line, msg); } +bool case0(const mdbx::path &path) { + mdbx::env_managed::create_parameters createParameters; + createParameters.geometry.make_dynamic(21 * mdbx::env::geometry::MiB, 84 * mdbx::env::geometry::MiB); + + mdbx::env::operate_parameters operateParameters(100, 10); + operateParameters.options.no_sticky_threads = false; + mdbx::env_managed env(path, createParameters, operateParameters); + auto txn = env.start_write(false); + /* mdbx::map_handle testHandle = */ txn.create_map("xyz", mdbx::key_mode::usual, mdbx::value_mode::single); + txn.commit(); + + //------------------------------------- + txn = env.start_write(); + MDBX_txn *c_txn = txn; + int err = mdbx_txn_reset(txn); + assert(err == MDBX_EINVAL); + bool ok = err == MDBX_EINVAL; + + err = mdbx_txn_break(txn); + assert(err == MDBX_SUCCESS); + ok = ok && err == MDBX_SUCCESS; + + err = mdbx_txn_commit(txn); + assert(err == MDBX_RESULT_TRUE); + ok = ok && err == MDBX_RESULT_TRUE; + + //------------------------------------- + err = mdbx_txn_begin(env, nullptr, MDBX_TXN_READWRITE, &c_txn); + assert(err == MDBX_SUCCESS); + ok = ok && err == MDBX_SUCCESS; + assert(c_txn == (const MDBX_txn *)txn); + + err = mdbx_txn_break(txn); + assert(err == MDBX_SUCCESS); + ok = ok && err == MDBX_SUCCESS; + + err = mdbx_txn_reset(txn); + assert(err == MDBX_EINVAL); + ok = ok && err == MDBX_EINVAL; + + err = mdbx_txn_commit(txn); + assert(err == MDBX_RESULT_TRUE); + ok = ok && err == MDBX_RESULT_TRUE; + + err = mdbx_txn_abort(c_txn); + assert(err == MDBX_BAD_TXN); + ok = ok && err == MDBX_BAD_TXN; + //------------------------------------- + err = mdbx_txn_begin(env, nullptr, MDBX_TXN_READWRITE, &c_txn); + assert(err == MDBX_SUCCESS); + ok = ok && err == MDBX_SUCCESS; + assert(c_txn == (const MDBX_txn *)txn); + txn.commit(); + + err = mdbx_txn_reset(c_txn); + assert(err == MDBX_BAD_TXN); + ok = ok && err == MDBX_BAD_TXN; + err = mdbx_txn_break(c_txn); + assert(err == MDBX_BAD_TXN); + ok = ok && err == MDBX_BAD_TXN; + err = mdbx_txn_abort(c_txn); + assert(err == MDBX_BAD_TXN); + ok = ok && err == MDBX_BAD_TXN; + + //===================================== + + txn = env.start_read(); + err = mdbx_txn_begin(env, txn, MDBX_TXN_READWRITE, &c_txn); + assert(err == MDBX_BAD_TXN); + ok = ok && err == MDBX_BAD_TXN; + txn.make_broken(); + err = mdbx_txn_begin(env, txn, MDBX_TXN_READWRITE, &c_txn); + assert(err == MDBX_BAD_TXN); + ok = ok && err == MDBX_BAD_TXN; + txn.reset_reading(); + err = mdbx_txn_begin(env, txn, MDBX_TXN_READWRITE, &c_txn); + assert(err == MDBX_BAD_TXN); + ok = ok && err == MDBX_BAD_TXN; + txn.abort(); + + //------------------------------------- + + txn = env.start_read(); + txn.reset_reading(); + txn.make_broken(); + txn.abort(); + + //===================================== + + std::latch s(1); + txn = env.start_read(); + c_txn = txn; + + std::thread t([&]() { + s.wait(); +#if MDBX_TXN_CHECKOWNER + err = mdbx_txn_reset(c_txn); + assert(err == MDBX_THREAD_MISMATCH); + ok = ok && err == MDBX_THREAD_MISMATCH; + err = mdbx_txn_break(c_txn); + assert(err == MDBX_THREAD_MISMATCH); + ok = ok && err == MDBX_THREAD_MISMATCH; + err = mdbx_txn_commit(c_txn); + assert(err == MDBX_THREAD_MISMATCH); + ok = ok && err == MDBX_THREAD_MISMATCH; + err = mdbx_txn_abort(c_txn); + assert(err == MDBX_THREAD_MISMATCH); + ok = ok && err == MDBX_THREAD_MISMATCH; +#endif /* MDBX_TXN_CHECKOWNER */ + + err = mdbx_txn_begin(env, txn, MDBX_TXN_READWRITE, &c_txn); +#if MDBX_TXN_CHECKOWNER + assert(err == MDBX_THREAD_MISMATCH); + ok = ok && err == MDBX_THREAD_MISMATCH; +#else + assert(err == MDBX_BAD_TXN); + ok = ok && err == MDBX_BAD_TXN; +#endif /* MDBX_TXN_CHECKOWNER */ + }); + + s.count_down(); + t.join(); + + return ok; +} + +bool case1(const mdbx::path &path) { + mdbx::env::operate_parameters operateParameters(100, 10); + operateParameters.options.no_sticky_threads = true; + operateParameters.options.nested_write_transactions = true; + mdbx::env_managed env(path, operateParameters); + + //------------------------------------- + auto txn = env.start_write(); + MDBX_txn *c_txn = txn; + int err = mdbx_txn_reset(txn); + assert(err == MDBX_EINVAL); + bool ok = err == MDBX_EINVAL; + + err = mdbx_txn_break(txn); + assert(err == MDBX_SUCCESS); + ok = ok && err == MDBX_SUCCESS; + + err = mdbx_txn_commit(txn); + assert(err == MDBX_RESULT_TRUE); + ok = ok && err == MDBX_RESULT_TRUE; + + //------------------------------------- + err = mdbx_txn_begin(env, nullptr, MDBX_TXN_READWRITE, &c_txn); + assert(err == MDBX_SUCCESS); + ok = ok && err == MDBX_SUCCESS; + assert(c_txn == (const MDBX_txn *)txn); + + err = mdbx_txn_break(txn); + assert(err == MDBX_SUCCESS); + ok = ok && err == MDBX_SUCCESS; + + err = mdbx_txn_reset(txn); + assert(err == MDBX_EINVAL); + ok = ok && err == MDBX_EINVAL; + + err = mdbx_txn_commit(txn); + assert(err == MDBX_RESULT_TRUE); + ok = ok && err == MDBX_RESULT_TRUE; + + err = mdbx_txn_abort(c_txn); + assert(err == MDBX_BAD_TXN); + ok = ok && err == MDBX_BAD_TXN; + //------------------------------------- + err = mdbx_txn_begin(env, nullptr, MDBX_TXN_READWRITE, &c_txn); + assert(err == MDBX_SUCCESS); + ok = ok && err == MDBX_SUCCESS; + assert(c_txn == (const MDBX_txn *)txn); + txn.commit(); + + err = mdbx_txn_reset(c_txn); + assert(err == MDBX_BAD_TXN); + ok = ok && err == MDBX_BAD_TXN; + err = mdbx_txn_break(c_txn); + assert(err == MDBX_BAD_TXN); + ok = ok && err == MDBX_BAD_TXN; + err = mdbx_txn_abort(c_txn); + assert(err == MDBX_BAD_TXN); + ok = ok && err == MDBX_BAD_TXN; + + //===================================== + + txn = env.start_read(); + err = mdbx_txn_begin(env, txn, MDBX_TXN_READWRITE, &c_txn); + assert(err == MDBX_BAD_TXN); + ok = ok && err == MDBX_BAD_TXN; + txn.make_broken(); + err = mdbx_txn_begin(env, txn, MDBX_TXN_READWRITE, &c_txn); + assert(err == MDBX_BAD_TXN); + ok = ok && err == MDBX_BAD_TXN; + txn.reset_reading(); + err = mdbx_txn_begin(env, txn, MDBX_TXN_READWRITE, &c_txn); + assert(err == MDBX_BAD_TXN); + ok = ok && err == MDBX_BAD_TXN; + txn.abort(); + + //------------------------------------- + + txn = env.start_read(); + txn.reset_reading(); + txn.make_broken(); + txn.abort(); + + //===================================== + + std::latch s1(1), s2(1), s3(1); + txn = env.start_read(); + c_txn = txn; + + std::thread t([&]() { + s1.wait(); + err = mdbx_txn_break(c_txn); + assert(err == MDBX_SUCCESS); + ok = ok && err == MDBX_SUCCESS; + err = mdbx_txn_reset(c_txn); + assert(err == MDBX_SUCCESS); + ok = ok && err == MDBX_SUCCESS; + txn.renew_reading(); + s2.count_down(); + + s3.wait(); + err = mdbx_txn_begin(env, txn, MDBX_TXN_READWRITE, &c_txn); + assert(err == MDBX_SUCCESS); + ok = ok && err == MDBX_SUCCESS; + err = mdbx_txn_commit(c_txn); + assert(err == MDBX_SUCCESS); + ok = ok && err == MDBX_SUCCESS; + c_txn = txn; + err = mdbx_txn_commit(c_txn); + assert(err == MDBX_THREAD_MISMATCH); + ok = ok && err == MDBX_THREAD_MISMATCH; + err = mdbx_txn_abort(c_txn); + assert(err == MDBX_THREAD_MISMATCH); + ok = ok && err == MDBX_THREAD_MISMATCH; + err = mdbx_txn_break(c_txn); + assert(err == MDBX_SUCCESS); + ok = ok && err == MDBX_SUCCESS; + err = mdbx_txn_reset(c_txn); + assert(err == MDBX_EINVAL); + ok = ok && err == MDBX_EINVAL; + }); + + s1.count_down(); + s2.wait(); + txn.commit(); + txn = env.start_write(); + s3.count_down(); + + t.join(); + txn.abort(); + + return ok; +} + +bool case2(const mdbx::path &path, bool no_sticky_threads) { + mdbx::env::operate_parameters operateParameters(100, 10); + operateParameters.options.no_sticky_threads = no_sticky_threads; + mdbx::env_managed env(path, operateParameters); + + std::latch s(1); + std::vector l; + for (size_t n = 0; n < 8; ++n) + l.push_back(std::thread([&]() { + s.wait(); + for (size_t i = 0; i < 1000000; ++i) { + auto txn = env.start_read(); + txn.abort(); + } + })); + + s.count_down(); + for (auto &t : l) + t.join(); + + return true; +} + int main(int argc, const char *argv[]) { (void)argc; (void)argv; - bool ok = true; - int err; mdbx_setup_debug_nofmt(MDBX_LOG_VERBOSE, MDBX_DBG_ASSERT, logger_nofmt, log_buffer, sizeof(log_buffer)); mdbx::path path = "test-txn"; mdbx::env::remove(path); - mdbx::env::operate_parameters operateParameters(100, 10); - { - mdbx::env_managed::create_parameters createParameters; - createParameters.geometry.make_dynamic(21 * mdbx::env::geometry::MiB, 84 * mdbx::env::geometry::MiB); - - operateParameters.options.no_sticky_threads = false; - mdbx::env_managed env(path, createParameters, operateParameters); - auto txn = env.start_write(false); - /* mdbx::map_handle testHandle = */ txn.create_map("xyz", mdbx::key_mode::usual, mdbx::value_mode::single); - txn.commit(); - - //------------------------------------- - txn = env.start_write(); - MDBX_txn *c_txn = txn; - err = mdbx_txn_reset(txn); - assert(err == MDBX_EINVAL); - ok = ok && err == MDBX_EINVAL; - - err = mdbx_txn_break(txn); - assert(err == MDBX_SUCCESS); - ok = ok && err == MDBX_SUCCESS; - - err = mdbx_txn_commit(txn); - assert(err == MDBX_RESULT_TRUE); - ok = ok && err == MDBX_RESULT_TRUE; - - //------------------------------------- - err = mdbx_txn_begin(env, nullptr, MDBX_TXN_READWRITE, &c_txn); - assert(err == MDBX_SUCCESS); - ok = ok && err == MDBX_SUCCESS; - assert(c_txn == (const MDBX_txn *)txn); - - err = mdbx_txn_break(txn); - assert(err == MDBX_SUCCESS); - ok = ok && err == MDBX_SUCCESS; - - err = mdbx_txn_reset(txn); - assert(err == MDBX_EINVAL); - ok = ok && err == MDBX_EINVAL; - - err = mdbx_txn_commit(txn); - assert(err == MDBX_RESULT_TRUE); - ok = ok && err == MDBX_RESULT_TRUE; - - err = mdbx_txn_abort(c_txn); - assert(err == MDBX_BAD_TXN); - ok = ok && err == MDBX_BAD_TXN; - //------------------------------------- - err = mdbx_txn_begin(env, nullptr, MDBX_TXN_READWRITE, &c_txn); - assert(err == MDBX_SUCCESS); - ok = ok && err == MDBX_SUCCESS; - assert(c_txn == (const MDBX_txn *)txn); - txn.commit(); - - err = mdbx_txn_reset(c_txn); - assert(err == MDBX_BAD_TXN); - ok = ok && err == MDBX_BAD_TXN; - err = mdbx_txn_break(c_txn); - assert(err == MDBX_BAD_TXN); - ok = ok && err == MDBX_BAD_TXN; - err = mdbx_txn_abort(c_txn); - assert(err == MDBX_BAD_TXN); - ok = ok && err == MDBX_BAD_TXN; - - //===================================== - - txn = env.start_read(); - err = mdbx_txn_begin(env, txn, MDBX_TXN_READWRITE, &c_txn); - assert(err == MDBX_BAD_TXN); - ok = ok && err == MDBX_BAD_TXN; - txn.make_broken(); - err = mdbx_txn_begin(env, txn, MDBX_TXN_READWRITE, &c_txn); - assert(err == MDBX_BAD_TXN); - ok = ok && err == MDBX_BAD_TXN; - txn.reset_reading(); - err = mdbx_txn_begin(env, txn, MDBX_TXN_READWRITE, &c_txn); - assert(err == MDBX_BAD_TXN); - ok = ok && err == MDBX_BAD_TXN; - txn.abort(); - - //------------------------------------- - - txn = env.start_read(); - txn.reset_reading(); - txn.make_broken(); - txn.abort(); - - //===================================== - - std::latch s(1); - txn = env.start_read(); - c_txn = txn; - - std::thread t([&]() { - s.wait(); -#if MDBX_TXN_CHECKOWNER - err = mdbx_txn_reset(c_txn); - assert(err == MDBX_THREAD_MISMATCH); - ok = ok && err == MDBX_THREAD_MISMATCH; - err = mdbx_txn_break(c_txn); - assert(err == MDBX_THREAD_MISMATCH); - ok = ok && err == MDBX_THREAD_MISMATCH; - err = mdbx_txn_commit(c_txn); - assert(err == MDBX_THREAD_MISMATCH); - ok = ok && err == MDBX_THREAD_MISMATCH; - err = mdbx_txn_abort(c_txn); - assert(err == MDBX_THREAD_MISMATCH); - ok = ok && err == MDBX_THREAD_MISMATCH; -#endif /* MDBX_TXN_CHECKOWNER */ - err = mdbx_txn_begin(env, txn, MDBX_TXN_READWRITE, &c_txn); - assert(err == MDBX_BAD_TXN); - ok = ok && err == MDBX_BAD_TXN; - }); - - s.count_down(); - t.join(); - } - - //===================================== - //===================================== - - { - operateParameters.options.no_sticky_threads = true; - operateParameters.options.nested_write_transactions = true; - mdbx::env_managed env(path, operateParameters); - - //------------------------------------- - auto txn = env.start_write(); - MDBX_txn *c_txn = txn; - err = mdbx_txn_reset(txn); - assert(err == MDBX_EINVAL); - ok = ok && err == MDBX_EINVAL; - - err = mdbx_txn_break(txn); - assert(err == MDBX_SUCCESS); - ok = ok && err == MDBX_SUCCESS; - - err = mdbx_txn_commit(txn); - assert(err == MDBX_RESULT_TRUE); - ok = ok && err == MDBX_RESULT_TRUE; - - //------------------------------------- - err = mdbx_txn_begin(env, nullptr, MDBX_TXN_READWRITE, &c_txn); - assert(err == MDBX_SUCCESS); - ok = ok && err == MDBX_SUCCESS; - assert(c_txn == (const MDBX_txn *)txn); - - err = mdbx_txn_break(txn); - assert(err == MDBX_SUCCESS); - ok = ok && err == MDBX_SUCCESS; - - err = mdbx_txn_reset(txn); - assert(err == MDBX_EINVAL); - ok = ok && err == MDBX_EINVAL; - - err = mdbx_txn_commit(txn); - assert(err == MDBX_RESULT_TRUE); - ok = ok && err == MDBX_RESULT_TRUE; - - err = mdbx_txn_abort(c_txn); - assert(err == MDBX_BAD_TXN); - ok = ok && err == MDBX_BAD_TXN; - //------------------------------------- - err = mdbx_txn_begin(env, nullptr, MDBX_TXN_READWRITE, &c_txn); - assert(err == MDBX_SUCCESS); - ok = ok && err == MDBX_SUCCESS; - assert(c_txn == (const MDBX_txn *)txn); - txn.commit(); - - err = mdbx_txn_reset(c_txn); - assert(err == MDBX_BAD_TXN); - ok = ok && err == MDBX_BAD_TXN; - err = mdbx_txn_break(c_txn); - assert(err == MDBX_BAD_TXN); - ok = ok && err == MDBX_BAD_TXN; - err = mdbx_txn_abort(c_txn); - assert(err == MDBX_BAD_TXN); - ok = ok && err == MDBX_BAD_TXN; - - //===================================== - - txn = env.start_read(); - err = mdbx_txn_begin(env, txn, MDBX_TXN_READWRITE, &c_txn); - assert(err == MDBX_BAD_TXN); - ok = ok && err == MDBX_BAD_TXN; - txn.make_broken(); - err = mdbx_txn_begin(env, txn, MDBX_TXN_READWRITE, &c_txn); - assert(err == MDBX_BAD_TXN); - ok = ok && err == MDBX_BAD_TXN; - txn.reset_reading(); - err = mdbx_txn_begin(env, txn, MDBX_TXN_READWRITE, &c_txn); - assert(err == MDBX_BAD_TXN); - ok = ok && err == MDBX_BAD_TXN; - txn.abort(); - - //------------------------------------- - - txn = env.start_read(); - txn.reset_reading(); - txn.make_broken(); - txn.abort(); - - //===================================== - - std::latch s1(1), s2(1), s3(1); - txn = env.start_read(); - c_txn = txn; - - std::thread t([&]() { - s1.wait(); - err = mdbx_txn_break(c_txn); - assert(err == MDBX_SUCCESS); - ok = ok && err == MDBX_SUCCESS; - err = mdbx_txn_reset(c_txn); - assert(err == MDBX_SUCCESS); - ok = ok && err == MDBX_SUCCESS; - txn.renew_reading(); - s2.count_down(); - - s3.wait(); - err = mdbx_txn_begin(env, txn, MDBX_TXN_READWRITE, &c_txn); - assert(err == MDBX_SUCCESS); - ok = ok && err == MDBX_SUCCESS; - err = mdbx_txn_commit(c_txn); - assert(err == MDBX_SUCCESS); - ok = ok && err == MDBX_SUCCESS; - c_txn = txn; - err = mdbx_txn_commit(c_txn); - assert(err == MDBX_THREAD_MISMATCH); - ok = ok && err == MDBX_THREAD_MISMATCH; - err = mdbx_txn_abort(c_txn); - assert(err == MDBX_THREAD_MISMATCH); - ok = ok && err == MDBX_THREAD_MISMATCH; - err = mdbx_txn_break(c_txn); - assert(err == MDBX_SUCCESS); - ok = ok && err == MDBX_SUCCESS; - err = mdbx_txn_reset(c_txn); - assert(err == MDBX_EINVAL); - ok = ok && err == MDBX_EINVAL; - }); - - s1.count_down(); - s2.wait(); - txn.commit(); - txn = env.start_write(); - s3.count_down(); - - t.join(); - txn.abort(); - } + bool ok = case0(path); + ok = case1(path) && ok; + ok = case2(path, false) && ok; + ok = case2(path, true) && ok; std::cout << (ok ? "OK\n" : "FAIL\n"); return ok ? EXIT_SUCCESS : EXIT_FAILURE; From 0a9c9840daa77c7eb0995e9accae7f48bb9587d6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:47:56 +0300 Subject: [PATCH 72/90] =?UTF-8?q?mdbx-tests:=20=D1=81=D1=83=D1=89=D0=B5?= =?UTF-8?q?=D1=81=D1=82=D0=B2=D0=B5=D0=BD=D0=BD=D0=BE=D0=B5=20=D1=80=D0=B0?= =?UTF-8?q?=D1=81=D1=88=D0=B8=D1=80=D0=B5=D0=BD=D0=B8=D0=B5=20`extra/curso?= =?UTF-8?q?r-closing`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- test/extra/cursor_closing.c++ | 341 +++++++++++++++++++++++++++++++--- 1 file changed, 313 insertions(+), 28 deletions(-) diff --git a/test/extra/cursor_closing.c++ b/test/extra/cursor_closing.c++ index 045b7677..60749dbd 100644 --- a/test/extra/cursor_closing.c++ +++ b/test/extra/cursor_closing.c++ @@ -1,6 +1,13 @@ #include "mdbx.h++" +#include +#include #include +#include +#if defined(__cpp_lib_latch) && __cpp_lib_latch >= 201907L +#include +#include +#endif static void logger_nofmt(MDBX_log_level_t loglevel, const char *function, int line, const char *msg, unsigned length) noexcept { @@ -11,6 +18,304 @@ static void logger_nofmt(MDBX_log_level_t loglevel, const char *function, int li static char log_buffer[1024]; +//-------------------------------------------------------------------------------------------- + +bool case0(mdbx::env env) { + auto txn = env.start_write(); + auto table = txn.create_map("case0", mdbx::key_mode::usual, mdbx::value_mode::single); + auto cursor_1 = txn.open_cursor(table); + auto cursor_2 = cursor_1.clone(); + + auto nested = env.start_write(txn); + auto nested_cursor_1 = nested.open_cursor(table); + auto nested_cursor_2 = nested_cursor_1.clone(); + auto nested_cursor_3 = cursor_1.clone(); + + auto deep = env.start_write(nested); + auto deep_cursor_1 = deep.open_cursor(table); + auto deep_cursor_2 = nested_cursor_1.clone(); + auto deep_cursor_3 = cursor_1.clone(); + deep_cursor_1.close(); + deep.commit(); + deep_cursor_2.close(); + + nested_cursor_1.close(); + nested.abort(); + nested_cursor_2.close(); + + cursor_1.close(); + txn.commit(); + cursor_2.close(); + return true; +} + +//-------------------------------------------------------------------------------------------- + +/* Сценарий: + * + * 0. Создаём N таблиц, курсор для каждой таблицы и заполняем (1000 ключей, от 1 до 1000 значений в каждом ключе). + * 1. Запускаем N-1 фоновых потоков и используем текущий/основной. + * 2. В каждом потоке 100500 раз повторяем последовательность действий: + * - 100500 раз запускаем читающую транзакцию и выполняем "читающий цикл": + * - в читающей транзакции создаем 0..3 курсоров, потом подключаем заранее созданный курсор, + * потом еще 0..3 курсоров; + * - выполняем по паре поисков через каждый курсор; + * - отключаем заранее созданный курсор; + * - снова выполняем несколько поисков по каждому курсору; + * - псевдослучайно закрываем один из курсоров и один отключаем; + * - псевдослучайно выполняем один из путей: + * - закрываем все курсоры посредством mdbx_txn_release_all_cursors(); + * - отсоединяем все курсоры посредством mdbx_txn_release_all_cursors(); + * - псевдослучайно закрываем один из курсоров и один отключаем; + * - ничего не делаем; + * - завершаем читающую транзакцию псевдослучайно выбирая между commit и abort; + * - закрываем оставшиеся курсоры. + * 3. Выполняем "пишущий цикл": + * - запускаем пишущую или вложенную транзакцию; + * - из оставшихся с предыдущих итераций курсоров половину закрываем, + * половину подключаем к транзакции; + * - для каждой таблицы с вероятностью 1/2 выполняем "читающий цикл"; + * - для каждой таблицы с вероятностью 1/2 выполняем "модифицирующий" цикл: + * - подключаем курсор, либо создаем при отсутствии подходящих; + * - 100 раз выполняем поиск случайных пар ключ/значение; + * - при успешном поиске удаляем значение, иначе вставляем; + * - с вероятностью 1/2 повторяем "читающий цикл"; + * - с вероятностью 7/16 запускаем вложенную транзакцию: + * - действуем рекурсивно как с пишущей транзакцией; + * - в "читающих циклах" немного меняем поведение: + * - игнорируем ожидаемые ошибки mdbx_cursor_unbind(); + * - в 2-3 раза уменьшаем вероятность использования mdbx_txn_release_all_cursors(); + * - завершаем вложенную транзакцию псевдослучайно выбирая между commit и abort; + * - для каждой таблицы с вероятностью 1/2 выполняем "читающий цикл"; + * - завершаем транзакцию псевдослучайно выбирая между commit и abort; + * 4. Ждем завершения фоновых потоков. + * 5. Закрываем оставшиеся курсоры и закрываем БД. */ + +thread_local size_t salt; + +static size_t prng() { + salt = salt * 134775813 + 1; + return salt ^ ((salt >> 11) * 1822226723); +} + +static inline bool flipcoin() { return prng() & 1; } + +static inline size_t prng(size_t range) { return prng() % range; } + +void case1_shuffle_pool(std::vector &pool) { + for (size_t n = 1; n < pool.size(); ++n) { + const auto i = prng(n); + if (i != n) + std::swap(pool[n], pool[i]); + } +} + +void case1_read_pool(std::vector &pool) { + for (auto c : pool) + if (flipcoin()) + mdbx::cursor(c).find_multivalue(mdbx::slice::wrap(prng(1000)), mdbx::slice::wrap(prng(1000)), false); + for (auto c : pool) + if (flipcoin()) + mdbx::cursor(c).find_multivalue(mdbx::slice::wrap(prng(1000)), mdbx::slice::wrap(prng(1000)), false); +} + +MDBX_cursor *case1_try_unbind(MDBX_cursor *cursor) { + if (cursor) { + auto err = mdbx::error(static_cast(mdbx_cursor_unbind(cursor))); + if (err.code() != MDBX_EINVAL) + err.success_or_throw(); + } + return cursor; +} + +MDBX_cursor *case1_pool_remove(std::vector &pool) { + switch (pool.size()) { + case 0: + return nullptr; + case 1: + if (flipcoin()) { + const auto c = pool[0]; + pool.pop_back(); + return c; + } + return nullptr; + default: + const auto i = prng(pool.size()); + const auto c = pool[i]; + pool.erase(pool.begin() + i); + return c; + } +} + +mdbx::map_handle case1_cycle_dbi(std::deque &dbi) { + const auto h = dbi.front(); + dbi.pop_front(); + dbi.push_back(h); + return h; +} + +void case1_read_cycle(mdbx::txn txn, std::deque &dbi, std::vector &pool, + mdbx::cursor pre, bool nested = false) { + for (auto c : pool) + mdbx::cursor(c).bind(txn, case1_cycle_dbi(dbi)); + pre.bind(txn, case1_cycle_dbi(dbi)); + + for (auto n = prng(3 + dbi.size()); n > 0; --n) { + auto c = txn.open_cursor(dbi[prng(dbi.size())]); + pool.push_back(c.withdraw_handle()); + } + case1_shuffle_pool(pool); + case1_read_pool(pool); + + pool.push_back(pre); + case1_read_pool(pool); + pool.pop_back(); + + for (auto n = prng(3 + dbi.size()); n > 0; --n) { + auto c = txn.open_cursor(dbi[prng(dbi.size())]); + pool.push_back(c.withdraw_handle()); + } + pool.push_back(pre); + case1_read_pool(pool); + pool.pop_back(); + + case1_try_unbind(pre); + case1_shuffle_pool(pool); + case1_read_pool(pool); + + if (flipcoin()) { + mdbx_cursor_close(case1_pool_remove(pool)); + auto u = case1_try_unbind(case1_pool_remove(pool)); + case1_read_pool(pool); + if (u) + pool.push_back(u); + } else { + auto u = case1_try_unbind(case1_pool_remove(pool)); + mdbx_cursor_close(case1_pool_remove(pool)); + case1_read_pool(pool); + if (u) + pool.push_back(u); + } + + switch (prng(nested ? 7 : 3)) { + case 0: + for (auto i = pool.begin(); i != pool.end();) + if (mdbx_cursor_txn(*i)) + i = pool.erase(i); + else + ++i; + txn.close_all_cursors(); + break; + case 1: + txn.unbind_all_cursors(); + break; + } +} + +void case1_write_cycle(mdbx::txn_managed txn, std::deque &dbi, std::vector &pool, + mdbx::cursor pre, bool nested = false) { + if (flipcoin()) + case1_cycle_dbi(dbi); + if (flipcoin()) + case1_shuffle_pool(pool); + + for (auto n = prng(dbi.size() + 1); n > 1; n -= 2) { + if (!nested) + pre.unbind(); + if (!pre.txn()) + pre.bind(txn, dbi[prng(dbi.size())]); + for (auto i = 0; i < 1000; ++i) { + auto k = mdbx::default_buffer::wrap(prng(1000)); + auto v = mdbx::default_buffer::wrap(prng(1000)); + if (pre.find_multivalue(k, v, false)) + pre.erase(); + else + pre.upsert(k, v); + } + } + + if (prng(16) > 8) + case1_write_cycle(txn.start_nested(), dbi, pool, pre, true); + + if (flipcoin()) + txn.commit(); + else + txn.abort(); +} + +bool case1_thread(mdbx::env env, std::deque dbi, mdbx::cursor pre) { + salt = size_t(std::chrono::high_resolution_clock::now().time_since_epoch().count()); + std::vector pool; + for (auto loop = 0; loop < 333; ++loop) { + for (auto read = 0; read < 333; ++read) { + auto txn = env.start_read(); + case1_read_cycle(txn, dbi, pool, pre); + if (flipcoin()) + txn.commit(); + else + txn.abort(); + } + + case1_write_cycle(env.start_write(), dbi, pool, pre); + + for (auto c : pool) + mdbx_cursor_close(c); + pool.clear(); + } + + pre.unbind(); + return true; +} + +bool case1(mdbx::env env) { + bool ok = true; + std::deque dbi; + std::vector cursors; +#if defined(__cpp_lib_latch) && __cpp_lib_latch >= 201907L + static const auto N = 10; +#else + static const auto N = 3; +#endif + for (auto t = 0; t < N; ++t) { + auto txn = env.start_write(); + auto table = txn.create_map(std::to_string(t), mdbx::key_mode::ordinal, mdbx::value_mode::multi_samelength); + auto cursor = txn.open_cursor(table); + for (size_t i = 0; i < 10000; ++i) + cursor.upsert(mdbx::default_buffer::wrap(prng(1000)), mdbx::default_buffer::wrap(prng(1000))); + txn.commit(); + + cursors.push_back(std::move(cursor)); + dbi.push_back(table); + } + +#if defined(__cpp_lib_latch) && __cpp_lib_latch >= 201907L + std::latch s(1); + std::vector threads; + for (auto t = 1; t < N; ++t) { + case1_cycle_dbi(dbi); + threads.push_back(std::thread([&, t]() { + s.wait(); + if (!case1_thread(env, dbi, cursors[t])) + ok = false; + })); + } + case1_cycle_dbi(dbi); + s.count_down(); +#endif + + if (!case1_thread(env, dbi, cursors[0])) + ok = false; + +#if defined(__cpp_lib_latch) && __cpp_lib_latch >= 201907L + for (auto &t : threads) + t.join(); +#endif + + return ok; +} + +//-------------------------------------------------------------------------------------------- + int main(int argc, const char *argv[]) { (void)argc; (void)argv; @@ -23,34 +328,14 @@ int main(int argc, const char *argv[]) { mdbx::env_managed env(db_filename, mdbx::env_managed::create_parameters(), mdbx::env::operate_parameters(42, 0, mdbx::env::nested_transactions)); - { - auto txn = env.start_write(); - auto table = txn.create_map("dummy", mdbx::key_mode::usual, mdbx::value_mode::single); - auto cursor_1 = txn.open_cursor(table); - auto cursor_2 = cursor_1.clone(); + bool ok = case0(env); + ok = case1(env) && ok; - auto nested = env.start_write(txn); - auto nested_cursor_1 = nested.open_cursor(table); - auto nested_cursor_2 = nested_cursor_1.clone(); - auto nested_cursor_3 = cursor_1.clone(); - - auto deep = env.start_write(nested); - auto deep_cursor_1 = deep.open_cursor(table); - auto deep_cursor_2 = nested_cursor_1.clone(); - auto deep_cursor_3 = cursor_1.clone(); - deep_cursor_1.close(); - deep.commit(); - deep_cursor_2.close(); - - nested_cursor_1.close(); - nested.abort(); - nested_cursor_2.close(); - - cursor_1.close(); - txn.commit(); - cursor_2.close(); + if (ok) { + std::cout << "OK\n"; + return EXIT_SUCCESS; + } else { + std::cout << "FAIL!\n"; + return EXIT_FAILURE; } - - std::cout << "OK\n"; - return EXIT_SUCCESS; } From 767ba21977535819ddc24ae41a3feaee1e8211dc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:48:22 +0300 Subject: [PATCH 73/90] =?UTF-8?q?mdbx:=20=D0=BA=D0=BE=D1=81=D1=82=D1=8B?= =?UTF-8?q?=D0=BB=D0=B8=20=D0=B4=D0=BB=D1=8F=20CLANG=20<=2020=20=D0=BF?= =?UTF-8?q?=D1=80=D0=B8=20=D0=B8=D1=81=D0=BF=D0=BE=D0=BB=D1=8C=D0=B7=D0=BE?= =?UTF-8?q?=D0=B2=D0=B0=D0=BD=D0=B8=D0=B8=20`[[=D0=B0=D1=82=D1=80=D0=B8?= =?UTF-8?q?=D0=B1=D1=83=D1=82=D0=BE=D0=B2]]`=20C23=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- mdbx.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mdbx.h b/mdbx.h index ae2af5ad..a6cdb7de 100644 --- a/mdbx.h +++ b/mdbx.h @@ -204,7 +204,7 @@ typedef mode_t mdbx_mode_t; #ifndef __has_cpp_attribute #define __has_cpp_attribute(x) 0 #define __has_cpp_attribute_qualified(x) 0 -#elif defined(_MSC_VER) +#elif defined(_MSC_VER) || (__clang__ && __clang__ < 14) /* MSVC don't support `namespace::attr` syntax */ #define __has_cpp_attribute_qualified(x) 0 #else @@ -318,7 +318,7 @@ typedef mode_t mdbx_mode_t; #ifndef MDBX_DEPRECATED #ifdef __deprecated #define MDBX_DEPRECATED __deprecated -#elif defined(DOXYGEN) || ((!defined(__GNUC__) || defined(__clang__) || __GNUC__ > 5) && \ +#elif defined(DOXYGEN) || ((!defined(__GNUC__) || (defined(__clang__) && __clang__ > 19) || __GNUC__ > 5) && \ ((defined(__cplusplus) && __cplusplus >= 201403L && __has_cpp_attribute(deprecated) && \ __has_cpp_attribute(deprecated) >= 201309L) || \ (!defined(__cplusplus) && defined(__STDC_VERSION__) && __STDC_VERSION__ >= 202304L))) @@ -504,7 +504,7 @@ typedef mode_t mdbx_mode_t; #if defined(DOXYGEN) || \ (defined(__cplusplus) && __cplusplus >= 201603L && __has_cpp_attribute(maybe_unused) && \ - __has_cpp_attribute(maybe_unused) >= 201603L) || \ + __has_cpp_attribute(maybe_unused) >= 201603L && (!defined(__clang__) || __clang__ > 19)) || \ (!defined(__cplusplus) && defined(__STDC_VERSION__) && __STDC_VERSION__ > 202005L) #define MDBX_MAYBE_UNUSED [[maybe_unused]] #elif defined(__GNUC__) || __has_attribute(__unused__) From 1dfe1e872e993ff5cf9fa59aa7920cc91c121f0e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:48:58 +0300 Subject: [PATCH 74/90] =?UTF-8?q?mdbx++:=20=D0=B4=D0=BE=D0=B1=D0=B0=D0=B2?= =?UTF-8?q?=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20`mdbx::cursor::seek=5Fmultiple?= =?UTF-8?q?=5Fsamelength()`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- mdbx.h++ | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/mdbx.h++ b/mdbx.h++ index 55bfc8e9..ba403bca 100644 --- a/mdbx.h++ +++ b/mdbx.h++ @@ -4265,9 +4265,14 @@ public: batch_samelength = MDBX_GET_MULTIPLE, batch_samelength_next = MDBX_NEXT_MULTIPLE, - batch_samelength_previous = MDBX_PREV_MULTIPLE + batch_samelength_previous = MDBX_PREV_MULTIPLE, + seek_and_batch_samelength = MDBX_SEEK_AND_GET_MULTIPLE }; + // TODO: добавить легковесный proxy-класс для замещения параметра throw_notfound более сложным набором опций, + // в том числе с explicit-конструктором из bool, чтобы защититься от неявной конвертации ключей поиска + // и других параметров в bool-throw_notfound. + struct move_result : public pair_result { inline move_result(const cursor &cursor, bool throw_notfound); move_result(cursor &cursor, move_operation operation, bool throw_notfound) @@ -4465,8 +4470,8 @@ public: inline move_result lower_bound_multivalue(const slice &key, const slice &value, bool throw_notfound = false); inline move_result upper_bound_multivalue(const slice &key, const slice &value, bool throw_notfound = false); - inline move_result get_multiple_samelength(const slice &key, bool throw_notfound = true) { - return move(batch_samelength, key, throw_notfound); + inline move_result seek_multiple_samelength(const slice &key, bool throw_notfound = true) { + return move(seek_and_batch_samelength, key, throw_notfound); } inline move_result get_multiple_samelength(bool throw_notfound = false) { From 529f2c23809014fdf1fca590afb20f2a2913c230 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:49:17 +0300 Subject: [PATCH 75/90] =?UTF-8?q?mdbx-tests:=20=D1=83=D0=BC=D0=B5=D0=BD?= =?UTF-8?q?=D1=8C=D1=88=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=BA=D0=BE=D0=BB-?= =?UTF-8?q?=D0=B2=D0=B0=20=D0=B8=D1=82=D0=B5=D1=80=D0=B0=D1=86=D0=B8=D0=B9?= =?UTF-8?q?=20=D0=B2=20extra/crunched-delete=20=D0=B4=D0=BB=D1=8F=2032-?= =?UTF-8?q?=D0=B1=D0=B8=D1=82=D0=BD=D1=8B=D1=85=20=D1=81=D0=B1=D0=BE=D1=80?= =?UTF-8?q?=D0=BE=D0=BA=20=D0=B2=D0=BE=20=D0=B8=D0=B7=D0=B1=D0=B5=D0=B6?= =?UTF-8?q?=D0=B0=D0=BD=D0=B8=D1=8F=20`MDBX=5FMAP=5FFULL`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- test/extra/crunched_delete.c++ | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/test/extra/crunched_delete.c++ b/test/extra/crunched_delete.c++ index 5655fd8a..134dbafe 100644 --- a/test/extra/crunched_delete.c++ +++ b/test/extra/crunched_delete.c++ @@ -7,8 +7,10 @@ #if MDBX_DEBUG || !defined(NDEBUG) || defined(__APPLE__) || defined(_WIN32) #define NN 1024 -#else +#elif UINTPTR_MAX > 0xffffFFFFul || ULONG_MAX > 0xffffFFFFul #define NN 4096 +#else +#define NN 2048 #endif std::string format_va(const char *fmt, va_list ap) { From 3282adf8bd53811b198ef09633b58f87fdbcc4e6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:49:42 +0300 Subject: [PATCH 76/90] =?UTF-8?q?mdbx:=20=D0=B8=D1=81=D0=BF=D1=80=D0=B0?= =?UTF-8?q?=D0=B2=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D1=80=D0=B5=D0=B3=D1=80?= =?UTF-8?q?=D0=B5=D1=81=D1=81=D0=B0=20=D0=B2=20=D0=BF=D1=83=D1=82=D0=B8=20?= =?UTF-8?q?=D0=BE=D0=B1=D1=80=D0=B0=D0=B1=D0=BE=D1=82=D0=BA=D0=B8=20`MDBX?= =?UTF-8?q?=5FMULTIPLE`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Пакетная вставка значений посредством операции `MDBX_MULTIPLE` могла приводить к падениям и повреждению структуры БД. Ошибка оставалось не замеченной из-за специфических условий проявления, которые не реализовались в тестах. Проблема присутствовала во всех выпусках начиная с v0.13.1, но соответствующая ошибка не связана с конкретным коммита в истории, а является следствием нескольких доработок (шагов рефакторинга), которые суммарно привели к регрессу. Технически ошибка обусловлена не-обнулением переменной, которая не обнулялась в некотором пути выполнения и исходно не требовала обнуления, но такое обнуление потребовалось после ряда этапов оптимизации кода и рефакторинга. Основным условием проявления является пакетная вставка multi-значений в dupsort-таблицу с фиксированным размером значений, при котором набор значений соответствующий обновляемом ключу, перестаёт помещаться на вложенной странице и преобразуется/выносится во вложенное дерево страниц. Если такой вынос/преобразование происходило до исчерпания переданного набора значений, то при следующей итерации повторно производились действия соответствующие выносу данных в отдельное дерево страниц. Что могла приводить как к разыменованию неверных указателей (повреждению содержимого памяти) и/или к повреждению содержимого страниц образующих структуру БД. Исправление свелось к добавлению одной строчки кода, но также были расширены тесты для покрытия соответствующих сценариев. --- src/cursor.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/cursor.c b/src/cursor.c index b313a78f..036ab123 100644 --- a/src/cursor.c +++ b/src/cursor.c @@ -1408,6 +1408,7 @@ insert_node:; data[0].iov_base = ptr_disp(data[0].iov_base, data[0].iov_len); insert_key = insert_data = false; old_singledup.iov_base = nullptr; + sub_root = nullptr; goto more; } } From 52a19feccaf11dc39f1770f8ae6984f66a5fad15 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:50:27 +0300 Subject: [PATCH 77/90] =?UTF-8?q?mdbx++:=20=D1=8F=D0=B2=D0=BD=D0=BE=D0=B5?= =?UTF-8?q?=20=D0=BE=D0=BF=D1=80=D0=B5=D0=B4=D0=B5=D0=BB=D0=B5=D0=BD=D0=B8?= =?UTF-8?q?=D0=B5=20external-=D0=B8=D0=BD=D1=81=D1=82=D0=B0=D0=BD=D1=86?= =?UTF-8?q?=D0=B8=D1=80=D0=BE=D0=B2=D0=B0=D0=BD=D0=B8=D1=8F=20`mdbx::buffe?= =?UTF-8?q?r<>`=20c=20API-=D0=B0=D1=82=D1=80=D0=B8=D0=B1=D1=83=D1=82=D0=B0?= =?UTF-8?q?=D0=BC=D0=B8=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- mdbx.h++ | 23 +++++++++++++++++++++-- src/mdbx.c++ | 24 ++++++++++++++++++++++-- 2 files changed, 43 insertions(+), 4 deletions(-) diff --git a/mdbx.h++ b/mdbx.h++ index ba403bca..6426f3fa 100644 --- a/mdbx.h++ +++ b/mdbx.h++ @@ -107,6 +107,16 @@ #include #endif +#if !defined(_MSC_VER) || defined(__clang__) +/* adequate compilers */ +#define MDBX_EXTERN_API_TEMPLATE(API_ATTRIBUTES, API_TYPENAME) extern template class API_ATTRIBUTES API_TYPENAME +#define MDBX_INSTALL_API_TEMPLATE(API_ATTRIBUTES, API_TYPENAME) template class API_TYPENAME +#else +/* stupid microsoft showing off */ +#define MDBX_EXTERN_API_TEMPLATE(API_ATTRIBUTES, API_TYPENAME) extern template class API_TYPENAME +#define MDBX_INSTALL_API_TEMPLATE(API_ATTRIBUTES, API_TYPENAME) template class API_ATTRIBUTES API_TYPENAME +#endif + #if __cplusplus >= 201103L #include #include @@ -1500,8 +1510,7 @@ public: private: friend class txn; - struct silo; - using swap_alloc = allocation_aware_details::swap_alloc; + using swap_alloc = allocation_aware_details::swap_alloc; struct silo /* Empty Base Class Optimization */ : public allocator_type { MDBX_CXX20_CONSTEXPR const allocator_type &get_allocator() const noexcept { return *this; } MDBX_CXX20_CONSTEXPR allocator_type &get_allocator() noexcept { return *this; } @@ -2729,6 +2738,12 @@ inline string make_string(const PRODUCER &producer, const ALLOCATOR & return result; } +MDBX_EXTERN_API_TEMPLATE(LIBMDBX_API_TYPE, buffer); + +#if defined(__cpp_lib_memory_resource) && __cpp_lib_memory_resource >= 201603L && _GLIBCXX_USE_CXX11_ABI +MDBX_EXTERN_API_TEMPLATE(LIBMDBX_API_TYPE, buffer); +#endif /* __cpp_lib_memory_resource >= 201603L */ + /// \brief Combines data slice with boolean flag to represent result of certain /// operations. struct value_result { @@ -2854,9 +2869,13 @@ template struct buffer_pair_spec operator pair() const noexcept { return pair(key, value); } }; +/// \brief Combines pair of buffers for key and value to hold an operands for certain operations. template using buffer_pair = buffer_pair_spec; +/// \brief Default pair of buffers. +using default_buffer_pair = buffer_pair; + /// end of cxx_data @} //------------------------------------------------------------------------------ diff --git a/src/mdbx.c++ b/src/mdbx.c++ index 7b4d8aa1..565c79e9 100644 --- a/src/mdbx.c++ +++ b/src/mdbx.c++ @@ -1162,12 +1162,32 @@ bool from_base64::is_erroneous() const noexcept { //------------------------------------------------------------------------------ -template class LIBMDBX_API_TYPE buffer; +#if defined(_MSC_VER) +#pragma warning(push) +/* warning C4251: 'mdbx::buffer<...>::silo_': + * struct 'mdbx::buffer<..>::silo' needs to have dll-interface to be used by clients of class 'mdbx::buffer<...>' + * + * Microsoft не хочет признавать ошибки и пересматривать приятные решения, поэтому MSVC продолжает кошмарить + * и стращать разработчиков предупреждениями, тем самым перекладывая ответственность на их плечи. + * + * В данном случае предупреждение выдаётся из-за инстанцирования std::string::allocator_type::pointer и + * std::pmr::string::allocator_type::pointer внутри mdbx::buffer<..>::silo. А так как эти типы являются частью + * стандартной библиотеки C++ они всегда будут доступны и без необходимости их инстанцирования и экспорта из libmdbx. + * + * Поэтому нет других вариантов как заглушить это предупреждение и еще раз плюнуть в сторону microsoft. */ +#pragma warning(disable : 4251) +#endif /* MSVC */ + +MDBX_INSTALL_API_TEMPLATE(LIBMDBX_API_TYPE, buffer); #if defined(__cpp_lib_memory_resource) && __cpp_lib_memory_resource >= 201603L && _GLIBCXX_USE_CXX11_ABI -template class LIBMDBX_API_TYPE buffer; +MDBX_INSTALL_API_TEMPLATE(LIBMDBX_API_TYPE, buffer); #endif /* __cpp_lib_memory_resource >= 201603L */ +#if defined(_MSC_VER) +#pragma warning(pop) +#endif /* MSVC */ + //------------------------------------------------------------------------------ static inline MDBX_env_flags_t mode2flags(env::mode mode) { From 7ed769e9c6d12d643816424b5d576f8bdb2239de Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:51:37 +0300 Subject: [PATCH 78/90] =?UTF-8?q?mdbx:=20=D0=BF=D0=BE=D0=B4=D0=B4=D0=B5?= =?UTF-8?q?=D1=80=D0=B6=D0=BA=D0=B0=20`MDBX=5FMULTIPLE`=20=D1=81=20=D0=BD?= =?UTF-8?q?=D1=83=D0=BB=D0=B5=D0=B2=D1=8B=D0=BC=20=D1=80=D0=B0=D0=B7=D0=BC?= =?UTF-8?q?=D0=B5=D1=80=D0=BE=D0=BC=20=D0=B4=D0=B0=D0=BD=D0=BD=D1=8B=D1=85?= =?UTF-8?q?=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/cursor.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/src/cursor.c b/src/cursor.c index 036ab123..632aecc7 100644 --- a/src/cursor.c +++ b/src/cursor.c @@ -731,8 +731,17 @@ __hot int cursor_put(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data, unsig if (mc->clc->k.cmp(key, ¤t_key) != 0) return MDBX_EKEYMISMATCH; - if (unlikely((flags & MDBX_MULTIPLE))) - goto drop_current; + if (unlikely((flags & MDBX_MULTIPLE))) { + if (unlikely(!mc->subcur)) + return MDBX_EINVAL; + err = cursor_del(mc, flags & MDBX_ALLDUPS); + if (unlikely(err != MDBX_SUCCESS)) + return err; + if (unlikely(data[1].iov_len == 0)) + return MDBX_SUCCESS; + flags -= MDBX_CURRENT; + goto skip_check_samedata; + } if (mc->subcur) { node_t *node = page_node(mc->pg[mc->top], mc->ki[mc->top]); @@ -742,7 +751,6 @@ __hot int cursor_put(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data, unsig * отличается, то вместо обновления требуется удаление и * последующая вставка. */ if (mc->subcur->nested_tree.items > 1 || current_data.iov_len != data->iov_len) { - drop_current: err = cursor_del(mc, flags & MDBX_ALLDUPS); if (unlikely(err != MDBX_SUCCESS)) return err; @@ -847,6 +855,8 @@ __hot int cursor_put(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data, unsig size_t *batch_dupfix_done = nullptr, batch_dupfix_given = 0; if (unlikely(flags & MDBX_MULTIPLE)) { batch_dupfix_given = data[1].iov_len; + if (unlikely(data[1].iov_len == 0)) + return /* nothing todo */ MDBX_SUCCESS; batch_dupfix_done = &data[1].iov_len; *batch_dupfix_done = 0; } From 9653c8f45bb0dba18ac73c9d8052cafdcc8701e6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:52:21 +0300 Subject: [PATCH 79/90] =?UTF-8?q?mdbx:=20=D1=80=D0=B5=D1=84=D0=B0=D0=BA?= =?UTF-8?q?=D1=82=D0=BE=D1=80=D0=B8=D0=BD=D0=B3=20=D0=BF=D1=80=D0=BE=D0=B2?= =?UTF-8?q?=D0=B5=D1=80=D0=BE=D0=BA=20=D1=81=20=D0=B2=D1=8B=D0=BD=D0=BE?= =?UTF-8?q?=D1=81=D0=BE=D0=BC=20=D0=B2=20`cursor=5Fcheck=5Fmultiple()`=20(?= =?UTF-8?q?backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/api-cursor.c | 18 +++--------------- src/api-txn-data.c | 13 +++++++++++++ src/cursor.c | 15 +++++++++++++++ src/cursor.h | 3 +++ 4 files changed, 34 insertions(+), 15 deletions(-) diff --git a/src/api-cursor.c b/src/api-cursor.c index f5f09c20..ef905024 100644 --- a/src/api-cursor.c +++ b/src/api-cursor.c @@ -711,22 +711,10 @@ int mdbx_cursor_put(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data, MDBX_p if (unlikely(rc != MDBX_SUCCESS)) return LOG_IFERR(rc); - /* Check this first so counter will always be zero on any early failures. */ if (unlikely(flags & MDBX_MULTIPLE)) { - if (unlikely(flags & MDBX_RESERVE)) - return LOG_IFERR(MDBX_EINVAL); - if (unlikely(!(mc->tree->flags & MDBX_DUPFIXED))) - return LOG_IFERR(MDBX_INCOMPATIBLE); - const size_t dcount = data[1].iov_len; - if (unlikely(dcount < 2 || data->iov_len == 0)) - return LOG_IFERR(MDBX_BAD_VALSIZE); - if (unlikely(mc->tree->dupfix_size != data->iov_len) && mc->tree->dupfix_size) - return LOG_IFERR(MDBX_BAD_VALSIZE); - if (unlikely(dcount > MAX_MAPSIZE / 2 / (BRANCH_NODE_MAX(MDBX_MAX_PAGESIZE) - NODESIZE))) { - /* checking for multiplication overflow */ - if (unlikely(dcount > MAX_MAPSIZE / 2 / data->iov_len)) - return LOG_IFERR(MDBX_TOO_LARGE); - } + rc = cursor_check_multiple(mc, key, data, flags); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); } if (flags & MDBX_RESERVE) { diff --git a/src/api-txn-data.c b/src/api-txn-data.c index 2be54df8..cc0891cb 100644 --- a/src/api-txn-data.c +++ b/src/api-txn-data.c @@ -266,6 +266,19 @@ int mdbx_put(MDBX_txn *txn, MDBX_dbi dbi, const MDBX_val *key, MDBX_val *data, M rc = cursor_init(&cx.outer, txn, dbi); if (unlikely(rc != MDBX_SUCCESS)) return LOG_IFERR(rc); + + if (unlikely(flags & MDBX_MULTIPLE)) { + rc = cursor_check_multiple(&cx.outer, key, data, flags); + if (unlikely(rc != MDBX_SUCCESS)) + return LOG_IFERR(rc); + } + + if (flags & MDBX_RESERVE) { + if (unlikely(cx.outer.tree->flags & (MDBX_DUPSORT | MDBX_REVERSEDUP | MDBX_INTEGERDUP | MDBX_DUPFIXED))) + return LOG_IFERR(MDBX_INCOMPATIBLE); + data->iov_base = nullptr; + } + cx.outer.next = txn->cursors[dbi]; txn->cursors[dbi] = &cx.outer; diff --git a/src/cursor.c b/src/cursor.c index 632aecc7..be82940f 100644 --- a/src/cursor.c +++ b/src/cursor.c @@ -1438,6 +1438,21 @@ insert_node:; return rc; } +int cursor_check_multiple(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data, unsigned flags) { + (void)key; + if (unlikely(flags & MDBX_RESERVE)) + return MDBX_EINVAL; + if (unlikely(!(mc->tree->flags & MDBX_DUPFIXED))) + return MDBX_INCOMPATIBLE; + const size_t number = data[1].iov_len; + if (unlikely(number > MAX_MAPSIZE / 2 / (BRANCH_NODE_MAX(MDBX_MAX_PAGESIZE) - NODESIZE))) { + /* checking for multiplication overflow */ + if (unlikely(number > MAX_MAPSIZE / 2 / data->iov_len)) + return MDBX_TOO_LARGE; + } + return MDBX_SUCCESS; +} + __hot int cursor_put_checklen(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data, unsigned flags) { cASSERT(mc, (mc->flags & z_inner) == 0); if (unlikely(key->iov_len > mc->clc->k.lmax || key->iov_len < mc->clc->k.lmin)) { diff --git a/src/cursor.h b/src/cursor.h index 7f109bcc..cf5c8548 100644 --- a/src/cursor.h +++ b/src/cursor.h @@ -315,6 +315,9 @@ MDBX_INTERNAL MDBX_cursor *cursor_cpstk(const MDBX_cursor *csrc, MDBX_cursor *cd MDBX_INTERNAL int __must_check_result cursor_ops(MDBX_cursor *mc, MDBX_val *key, MDBX_val *data, const MDBX_cursor_op op); +MDBX_INTERNAL int __must_check_result cursor_check_multiple(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data, + unsigned flags); + MDBX_INTERNAL int __must_check_result cursor_put_checklen(MDBX_cursor *mc, const MDBX_val *key, MDBX_val *data, unsigned flags); From 065aef35eac679e7e95b7485b4597166294317f2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:55:10 +0300 Subject: [PATCH 80/90] =?UTF-8?q?mdbx++:=20=D0=B4=D0=BE=D0=B1=D0=B0=D0=B2?= =?UTF-8?q?=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20`mdbx::cursor::put=5Fmultiple?= =?UTF-8?q?=5Fsamelength()`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- mdbx.h++ | 38 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 36 insertions(+), 2 deletions(-) diff --git a/mdbx.h++ b/mdbx.h++ index 6426f3fa..ff4d0dba 100644 --- a/mdbx.h++ +++ b/mdbx.h++ @@ -4100,8 +4100,9 @@ public: return append(map, kv.key, kv.value, multivalue_order_preserved); } - size_t put_multiple_samelength(map_handle map, const slice &key, const size_t value_length, const void *values_array, - size_t values_count, put_mode mode, bool allow_partial = false); + inline size_t put_multiple_samelength(map_handle map, const slice &key, const size_t value_length, + const void *values_array, size_t values_count, put_mode mode, + bool allow_partial = false); template size_t put_multiple_samelength(map_handle map, const slice &key, const VALUE *values_array, size_t values_count, put_mode mode, bool allow_partial = false) { @@ -4567,6 +4568,21 @@ public: /// \brief Seeks and removes the particular multi-value entry of the key. /// \return `True` if the given key-value pair is found and removed. inline bool erase(const slice &key, const slice &value); + + inline size_t put_multiple_samelength(const slice &key, const size_t value_length, const void *values_array, + size_t values_count, put_mode mode, bool allow_partial = false); + template + size_t put_multiple_samelength(const slice &key, const VALUE *values_array, size_t values_count, put_mode mode, + bool allow_partial = false) { + static_assert(::std::is_standard_layout::value && !::std::is_pointer::value && + !::std::is_array::value, + "Must be a standard layout type!"); + return put_multiple_samelength(key, sizeof(VALUE), values_array, values_count, mode, allow_partial); + } + template + void put_multiple_samelength(const slice &key, const ::std::vector &vector, put_mode mode) { + put_multiple_samelength(key, vector.data(), vector.size(), mode); + } }; /// \brief Managed cursor. @@ -6354,6 +6370,24 @@ inline bool cursor::erase(const slice &key, const slice &value) { return data.done && erase(); } +inline size_t cursor::put_multiple_samelength(const slice &key, const size_t value_length, const void *values_array, + size_t values_count, put_mode mode, bool allow_partial) { + MDBX_val args[2] = {{const_cast(values_array), value_length}, {nullptr, values_count}}; + const int err = ::mdbx_cursor_put(handle_, const_cast(&key), args, MDBX_put_flags_t(mode) | MDBX_MULTIPLE); + switch (err) { + case MDBX_SUCCESS: + MDBX_CXX20_LIKELY break; + case MDBX_KEYEXIST: + if (allow_partial) + break; + mdbx_txn_break(txn()); + MDBX_CXX17_FALLTHROUGH /* fallthrough */; + default: + MDBX_CXX20_UNLIKELY error::throw_exception(err); + } + return args[1].iov_len /* done item count */; +} + /// end cxx_api @} } // namespace mdbx From 2127d3b7d725349560b1bcea9f5e7da6848a7204 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:55:41 +0300 Subject: [PATCH 81/90] =?UTF-8?q?mdbx-tests:=20=D1=80=D0=B0=D1=81=D1=88?= =?UTF-8?q?=D0=B8=D1=80=D0=B5=D0=BD=D0=B8=D0=B5=20extra/dupfix-multiple=20?= =?UTF-8?q?(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- test/extra/dupfix_multiple.c++ | 168 ++++++++++++++++++++++----------- 1 file changed, 111 insertions(+), 57 deletions(-) diff --git a/test/extra/dupfix_multiple.c++ b/test/extra/dupfix_multiple.c++ index a212984b..ecc02bc7 100644 --- a/test/extra/dupfix_multiple.c++ +++ b/test/extra/dupfix_multiple.c++ @@ -2,15 +2,12 @@ /// \copyright SPDX-License-Identifier: Apache-2.0 #include "mdbx.h++" -#include +#include #include -int doit() { - mdbx::path db_filename = "test-dupfix-multiple"; - mdbx::env_managed::remove(db_filename); - mdbx::env_managed env(db_filename, mdbx::env_managed::create_parameters(), mdbx::env::operate_parameters()); +using buffer = mdbx::default_buffer; - using buffer = mdbx::buffer; +bool case1_ordering(mdbx::env env) { auto txn = env.start_write(); auto map = txn.create_map(nullptr, mdbx::key_mode::ordinal, mdbx::value_mode::multi_ordinal); @@ -30,10 +27,9 @@ int doit() { cursor.to_next().value.as_uint64() != 17 || cursor.to_next().value.as_uint64() != 16 || cursor.to_next().value.as_uint64() != 15 || cursor.to_next().value.as_uint64() != 14 || cursor.to_next().value.as_uint64() != 13 || cursor.to_next().value.as_uint64() != 12 || - cursor.to_next(false).done || !cursor.eof()) { - std::cerr << "Fail\n"; - return EXIT_FAILURE; - } + cursor.to_next(false).done || !cursor.eof()) + return false; + txn.abort(); const uint64_t array[] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 42, 17, 99, 0, 33, 333}; @@ -87,10 +83,9 @@ int doit() { /* key = 24 */ cursor.to_next().value.as_uint64() != 15 || /* key = 25 */ cursor.to_next().value.as_uint64() != 14 || /* key = 26 */ cursor.to_next().value.as_uint64() != 13 || - /* key = 27 */ cursor.to_next().value.as_uint64() != 12 || cursor.to_next(false).done || !cursor.eof()) { - std::cerr << "Fail\n"; - return EXIT_FAILURE; - } + /* key = 27 */ cursor.to_next().value.as_uint64() != 12 || cursor.to_next(false).done || !cursor.eof()) + return false; + txn.abort(); txn = env.start_write(); @@ -163,40 +158,24 @@ int doit() { cursor.to_next().value.as_uint64() != 0 || cursor.to_next().value.as_uint64() != 33 || cursor.to_next().value.as_uint64() != 333 || - cursor.to_next(false).done || !cursor.eof()) { - std::cerr << "Fail\n"; - return EXIT_FAILURE; - } + cursor.to_next(false).done || !cursor.eof()) + return false; + txn.abort(); - //---------------------------------------------------------------------------- - - // let dir = tempdir().unwrap(); - // let db = Database::open(&dir).unwrap(); - - // let txn = db.begin_rw_txn().unwrap(); - // let table = txn - // .create_table(None, TableFlags::DUP_SORT | TableFlags::DUP_FIXED) - // .unwrap(); - // for (k, v) in [ - // (b"key1", b"val1"), - // (b"key1", b"val2"), - // (b"key1", b"val3"), - // (b"key2", b"val1"), - // (b"key2", b"val2"), - // (b"key2", b"val3"), - // ] { - // txn.put(&table, k, v, WriteFlags::empty()).unwrap(); - // } - - // let mut cursor = txn.cursor(&table).unwrap(); - // assert_eq!(cursor.first().unwrap(), Some((*b"key1", *b"val1"))); - // assert_eq!(cursor.get_multiple().unwrap(), Some(*b"val1val2val3")); - // assert_eq!(cursor.next_multiple::<(), ()>().unwrap(), None); - txn = env.start_write(); txn.clear_map(map); - map = txn.create_map(nullptr, mdbx::key_mode::usual, mdbx::value_mode::multi_samelength); + txn.commit(); + + return true; +} + +//-------------------------------------------------------------------------------------------- + +bool case2_batch_read(mdbx::env env) { + + auto txn = env.start_write(); + auto map = txn.create_map(nullptr, mdbx::key_mode::usual, mdbx::value_mode::multi_samelength); txn.upsert(map, mdbx::slice("key1"), mdbx::slice("val1")); txn.upsert(map, mdbx::pair("key1", "val2")); txn.upsert(map, mdbx::pair("key1", "val3")); @@ -205,30 +184,105 @@ int doit() { txn.upsert(map, mdbx::pair("key2", "val3")); // cursor.close(); - cursor = txn.open_cursor(map); + auto cursor = txn.open_cursor(map); const auto t1 = cursor.to_first(); if (!t1 || t1.key != "key1" || t1.value != "val1") { std::cerr << "Fail-t1\n"; - return EXIT_FAILURE; + return false; } const auto t2 = cursor.get_multiple_samelength(); if (!t2 || t2.key != "key1" || t2.value != "val1val2val3") { std::cerr << "Fail-t2\n"; - return EXIT_FAILURE; + return false; } - // const auto t3 = cursor.get_multiple_samelength("key2"); - // if (!t3 || t3.key != "key2" || t3.value != "val1val2val3") { - // std::cerr << "Fail-t3\n"; - // return EXIT_FAILURE; - // } - const auto t4 = cursor.next_multiple_samelength(); - if (t4) { + const auto t3 = cursor.next_multiple_samelength(); + if (t3) { + std::cerr << "Fail-t3\n"; + return false; + } + const auto t4 = cursor.seek_multiple_samelength("key2"); + if (!t4 || t4.key != "key2" || t4.value != "val1val2val3") { std::cerr << "Fail-t4\n"; - return EXIT_FAILURE; + return false; } - std::cout << "OK\n"; - return EXIT_SUCCESS; + txn.clear_map(map); + txn.commit(); + + return true; +} + +//-------------------------------------------------------------------------------------------- + +size_t salt; + +static size_t prng() { + salt = salt * 134775813 + 1; + return salt ^ ((salt >> 11) * 1822226723); +} + +static inline size_t prng(size_t range) { return prng() % range; } + +static mdbx::default_buffer_pair prng_kv(size_t n, size_t space) { + space = (space + !space) * 1024 * 32; + const size_t w = (n ^ 1455614549) * 1664525 + 1013904223; + const size_t k = (prng(42 + w % space) ^ 1725278851) * 433750991; + const size_t v = prng(); + return mdbx::default_buffer_pair(mdbx::slice::wrap(k), mdbx::slice::wrap(v)); +} + +bool case3_put_a_lot(mdbx::env env) { + salt = size_t(std::chrono::high_resolution_clock::now().time_since_epoch().count()); + auto txn = env.start_write(); + auto map = txn.create_map("case3", mdbx::key_mode::ordinal, mdbx::value_mode::multi_ordinal); + for (size_t n = 0; n < 5555555; ++n) + txn.upsert(map, prng_kv(n, 1)); + txn.commit(); + + for (size_t t = 0; t < 555; ++t) { + txn = env.start_write(); + auto cursor = txn.open_cursor(map); + for (size_t n = 0; n < 111; ++n) { + auto v = std::vector(); + const auto r = 1 + prng(3); + if (r & 1) { + const auto k = prng_kv(n + t, 2).key; + for (size_t i = prng(42 + prng(111) * prng(111)); i > 0; --i) + v.push_back(prng()); + txn.put_multiple_samelength(map, k, v, mdbx::upsert); + } + if (r & 2) { + const auto k = prng_kv(n + t, 2).key; + if (cursor.seek(k)) { + v.clear(); + for (size_t i = prng(42 + prng(111) * prng(111)); i > 0; --i) + v.push_back(prng()); + cursor.put_multiple_samelength(k, v, mdbx::upsert); + } + } + } + txn.commit(); + } + + return true; +} + +int doit() { + mdbx::path db_filename = "test-dupfix-multiple"; + mdbx::env_managed::remove(db_filename); + mdbx::env_managed env(db_filename, mdbx::env_managed::create_parameters(), mdbx::env::operate_parameters(1)); + + bool ok = case1_ordering(env); + ok = case2_batch_read(env) && ok; + ok = case3_put_a_lot(env) && ok; + + if (ok) { + std::cout << "OK\n"; + return EXIT_SUCCESS; + } else { + std::cerr << "Fail\n"; + return EXIT_FAILURE; + } } int main(int argc, const char *argv[]) { From 59343d9106ec257d8ecf1ff44b7819d0c36c1d89 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:56:07 +0300 Subject: [PATCH 82/90] mdbx++: minor reflow Doxygen comments (backport). --- mdbx.h++ | 205 ++++++++++++++++++------------------------------------- 1 file changed, 68 insertions(+), 137 deletions(-) diff --git a/mdbx.h++ b/mdbx.h++ index ff4d0dba..2d5f62b1 100644 --- a/mdbx.h++ +++ b/mdbx.h++ @@ -158,8 +158,7 @@ #endif #endif /* Byte Order */ -/** Workaround for old compilers without properly support for `C++17 constexpr`. - */ +/** Workaround for old compilers without properly support for `C++17 constexpr` */ #if defined(DOXYGEN) #define MDBX_CXX17_CONSTEXPR constexpr #elif defined(__cpp_constexpr) && __cpp_constexpr >= 201603L && \ @@ -170,8 +169,7 @@ #define MDBX_CXX17_CONSTEXPR inline #endif /* MDBX_CXX17_CONSTEXPR */ -/** Workaround for old compilers without properly support for C++20 `constexpr`. - */ +/** Workaround for old compilers without properly support for C++20 `constexpr`. */ #if defined(DOXYGEN) #define MDBX_CXX20_CONSTEXPR constexpr #elif defined(__cpp_lib_is_constant_evaluated) && __cpp_lib_is_constant_evaluated >= 201811L && \ @@ -195,8 +193,7 @@ #define MDBX_CXX20_CONSTEXPR_ENUM inline #endif /* CONSTEXPR_ENUM_FLAGS_OPERATIONS */ -/** Workaround for old compilers without support assertion inside `constexpr` - * functions. */ +/** Workaround for old compilers without support assertion inside `constexpr` functions. */ #if defined(CONSTEXPR_ASSERT) #define MDBX_CONSTEXPR_ASSERT(expr) CONSTEXPR_ASSERT(expr) #elif defined NDEBUG @@ -221,8 +218,7 @@ #endif #endif /* MDBX_UNLIKELY */ -/** Workaround for old compilers without properly support for C++20 `if - * constexpr`. */ +/** Workaround for old compilers without properly support for C++20 `if constexpr`. */ #if defined(DOXYGEN) #define MDBX_IF_CONSTEXPR constexpr #elif defined(__cpp_if_constexpr) && __cpp_if_constexpr >= 201606L @@ -398,8 +394,7 @@ namespace filesystem = ::std::experimental::filesystem; namespace filesystem = ::std::filesystem; /// \brief Defined if `mdbx::filesystem::path` is available. /// \details If defined, it is always `mdbx::filesystem::path`, -/// which in turn can be refs to `std::filesystem::path` -/// or `std::experimental::filesystem::path`. +/// which in turn can be refs to `std::filesystem::path` or `std::experimental::filesystem::path`. /// Nonetheless `MDBX_STD_FILESYSTEM_PATH` not defined if the `::mdbx::path` /// is fallbacked to c `std::string` or `std::wstring`. #define MDBX_STD_FILESYSTEM_PATH ::mdbx::filesystem::path @@ -499,9 +494,7 @@ public: static inline void success_or_panic(int error_code, const char *context_where, const char *func_who) noexcept; }; -/// \brief Base class for all libmdbx's exceptions that are corresponds -/// to libmdbx errors. -/// +/// \brief Base class for all libmdbx's exceptions that are corresponds to libmdbx errors. /// \see MDBX_error_t class LIBMDBX_API_TYPE exception : public ::std::runtime_error { using base = ::std::runtime_error; @@ -517,8 +510,7 @@ public: const ::mdbx::error error() const noexcept { return error_; } }; -/// \brief Fatal exception that lead termination anyway -/// in dangerous unrecoverable cases. +/// \brief Fatal exception that lead termination anyway in dangerous unrecoverable cases. class LIBMDBX_API_TYPE fatal : public exception { using base = exception; @@ -654,8 +646,7 @@ struct LIBMDBX_API_TYPE slice : public ::MDBX_val { /// \brief Create an empty slice. MDBX_CXX11_CONSTEXPR slice() noexcept; - /// \brief Create a slice that refers to [0,bytes-1] of memory bytes pointed - /// by ptr. + /// \brief Create a slice that refers to [0,bytes-1] of memory bytes pointed by ptr. MDBX_CXX14_CONSTEXPR slice(const void *ptr, size_t bytes); /// \brief Create a slice that refers to [begin,end] of memory bytes. @@ -1010,8 +1001,7 @@ struct LIBMDBX_API_TYPE slice : public ::MDBX_val { /// \brief Checks the slice is not refers to null address or has zero length. MDBX_CXX11_CONSTEXPR bool is_valid() const noexcept { return !(iov_base == nullptr && iov_len != 0); } - /// \brief Build an invalid slice which non-zero length and refers to null - /// address. + /// \brief Build an invalid slice which non-zero length and refers to null address. MDBX_CXX14_CONSTEXPR static slice invalid() noexcept { return slice(size_t(-1)); } template MDBX_CXX14_CONSTEXPR POD as_pod() const { @@ -1239,8 +1229,7 @@ struct LIBMDBX_API to_hex { char *write_bytes(char *dest, size_t dest_size) const; /// \brief Output hexadecimal dump of passed slice to the std::ostream. - /// \throws std::ios_base::failure corresponding to std::ostream::write() - /// behaviour. + /// \throws std::ios_base::failure corresponding to std::ostream::write() behaviour. ::std::ostream &output(::std::ostream &out) const; /// \brief Checks whether a passed slice is empty, @@ -1283,23 +1272,18 @@ struct LIBMDBX_API to_base58 { return wrap_width ? bytes + bytes / wrap_width : bytes; } - /// \brief Fills the buffer by [Base58](https://en.wikipedia.org/wiki/Base58) - /// dump of passed slice. + /// \brief Fills the buffer by [Base58](https://en.wikipedia.org/wiki/Base58) dump of passed slice. /// \throws std::length_error if given buffer is too small. char *write_bytes(char *dest, size_t dest_size) const; - /// \brief Output [Base58](https://en.wikipedia.org/wiki/Base58) - /// dump of passed slice to the std::ostream. - /// \throws std::ios_base::failure corresponding to std::ostream::write() - /// behaviour. + /// \brief Output [Base58](https://en.wikipedia.org/wiki/Base58) dump of passed slice to the std::ostream. + /// \throws std::ios_base::failure corresponding to std::ostream::write() behaviour. ::std::ostream &output(::std::ostream &out) const; - /// \brief Checks whether a passed slice is empty, - /// and therefore there will be no output bytes. + /// \brief Checks whether a passed slice is empty, and therefore there will be no output bytes. bool is_empty() const noexcept { return source.empty(); } - /// \brief Checks whether the content of a passed slice is a valid data - /// and could be encoded or unexpectedly not. + /// \brief Checks whether the content of a passed slice is a valid data and could be encoded or unexpectedly not. bool is_erroneous() const noexcept { return false; } }; @@ -1341,8 +1325,7 @@ struct LIBMDBX_API to_base64 { /// \brief Output [Base64](https://en.wikipedia.org/wiki/Base64) /// dump of passed slice to the std::ostream. - /// \throws std::ios_base::failure corresponding to std::ostream::write() - /// behaviour. + /// \throws std::ios_base::failure corresponding to std::ostream::write() behaviour. ::std::ostream &output(::std::ostream &out) const; /// \brief Checks whether a passed slice is empty, @@ -1383,13 +1366,11 @@ struct LIBMDBX_API from_hex { /// hexadecimal dump from a passed slice to decoded data. MDBX_CXX11_CONSTEXPR size_t envisage_result_length() const noexcept { return source.length() >> 1; } - /// \brief Fills the destination with data decoded from hexadecimal dump - /// from a passed slice. + /// \brief Fills the destination with data decoded from hexadecimal dump from a passed slice. /// \throws std::length_error if given buffer is too small. char *write_bytes(char *dest, size_t dest_size) const; - /// \brief Checks whether a passed slice is empty, - /// and therefore there will be no output bytes. + /// \brief Checks whether a passed slice is empty, and therefore there will be no output bytes. bool is_empty() const noexcept { return source.empty(); } /// \brief Checks whether the content of a passed slice is a valid hexadecimal @@ -1422,8 +1403,7 @@ struct LIBMDBX_API from_base58 { } /// \brief Returns the number of bytes needed for conversion - /// [Base58](https://en.wikipedia.org/wiki/Base58) dump from a passed slice to - /// decoded data. + /// [Base58](https://en.wikipedia.org/wiki/Base58) dump from a passed slice to decoded data. MDBX_CXX11_CONSTEXPR size_t envisage_result_length() const noexcept { return source.length() /* могут быть все нули кодируемые один-к-одному */; } @@ -1433,13 +1413,11 @@ struct LIBMDBX_API from_base58 { /// \throws std::length_error if given buffer is too small. char *write_bytes(char *dest, size_t dest_size) const; - /// \brief Checks whether a passed slice is empty, - /// and therefore there will be no output bytes. + /// \brief Checks whether a passed slice is empty, and therefore there will be no output bytes. bool is_empty() const noexcept { return source.empty(); } /// \brief Checks whether the content of a passed slice is a valid - /// [Base58](https://en.wikipedia.org/wiki/Base58) dump, and therefore there - /// could be decoded or not. + /// [Base58](https://en.wikipedia.org/wiki/Base58) dump, and therefore there could be decoded or not. bool is_erroneous() const noexcept; }; @@ -1468,8 +1446,7 @@ struct LIBMDBX_API from_base64 { } /// \brief Returns the number of bytes needed for conversion - /// [Base64](https://en.wikipedia.org/wiki/Base64) dump from a passed slice to - /// decoded data. + /// [Base64](https://en.wikipedia.org/wiki/Base64) dump from a passed slice to decoded data. MDBX_CXX11_CONSTEXPR size_t envisage_result_length() const noexcept { return (source.length() + 3) / 4 * 3; } /// \brief Fills the destination with data decoded from @@ -1936,8 +1913,7 @@ public: /// the buffer, rather than stores it. MDBX_NOTHROW_PURE_FUNCTION MDBX_CXX20_CONSTEXPR bool is_reference() const noexcept { return !is_freestanding(); } - /// \brief Returns the number of bytes that can be held in currently allocated - /// storage. + /// \brief Returns the number of bytes that can be held in currently allocated storage. MDBX_NOTHROW_PURE_FUNCTION MDBX_CXX20_CONSTEXPR size_t capacity() const noexcept { return is_freestanding() ? silo_.capacity() : 0; } @@ -1961,16 +1937,14 @@ public: MDBX_CXX11_CONSTEXPR const byte *end_byte_ptr() const noexcept { return slice_.end_byte_ptr(); } /// \brief Returns casted to pointer to byte an address of data. - /// \pre REQUIRES: The buffer should store data chunk, but not referenced to - /// an external one. + /// \pre REQUIRES: The buffer should store data chunk, but not referenced to an external one. MDBX_CXX11_CONSTEXPR byte *byte_ptr() noexcept { MDBX_CONSTEXPR_ASSERT(is_freestanding()); return const_cast(slice_.byte_ptr()); } /// \brief Returns casted to pointer to byte an end of data. - /// \pre REQUIRES: The buffer should store data chunk, but not referenced to - /// an external one. + /// \pre REQUIRES: The buffer should store data chunk, but not referenced to an external one. MDBX_CXX11_CONSTEXPR byte *end_byte_ptr() noexcept { MDBX_CONSTEXPR_ASSERT(is_freestanding()); return const_cast(slice_.end_byte_ptr()); @@ -1983,16 +1957,14 @@ public: MDBX_CXX11_CONSTEXPR const char *end_char_ptr() const noexcept { return slice_.end_char_ptr(); } /// \brief Returns casted to pointer to char an address of data. - /// \pre REQUIRES: The buffer should store data chunk, but not referenced to - /// an external one. + /// \pre REQUIRES: The buffer should store data chunk, but not referenced to an external one. MDBX_CXX11_CONSTEXPR char *char_ptr() noexcept { MDBX_CONSTEXPR_ASSERT(is_freestanding()); return const_cast(slice_.char_ptr()); } /// \brief Returns casted to pointer to char an end of data. - /// \pre REQUIRES: The buffer should store data chunk, but not referenced to - /// an external one. + /// \pre REQUIRES: The buffer should store data chunk, but not referenced to an external one. MDBX_CXX11_CONSTEXPR char *end_char_ptr() noexcept { MDBX_CONSTEXPR_ASSERT(is_freestanding()); return const_cast(slice_.end_char_ptr()); @@ -2005,16 +1977,14 @@ public: MDBX_CXX11_CONSTEXPR const void *end() const noexcept { return slice_.end(); } /// \brief Return a pointer to the beginning of the referenced data. - /// \pre REQUIRES: The buffer should store data chunk, but not referenced to - /// an external one. + /// \pre REQUIRES: The buffer should store data chunk, but not referenced to an external one. MDBX_CXX11_CONSTEXPR void *data() noexcept { MDBX_CONSTEXPR_ASSERT(is_freestanding()); return const_cast(slice_.data()); } /// \brief Return a pointer to the end of the referenced data. - /// \pre REQUIRES: The buffer should store data chunk, but not referenced to - /// an external one. + /// \pre REQUIRES: The buffer should store data chunk, but not referenced to an external one. MDBX_CXX11_CONSTEXPR void *end() noexcept { MDBX_CONSTEXPR_ASSERT(is_freestanding()); return const_cast(slice_.end()); @@ -2744,8 +2714,7 @@ MDBX_EXTERN_API_TEMPLATE(LIBMDBX_API_TYPE, buffer); MDBX_EXTERN_API_TEMPLATE(LIBMDBX_API_TYPE, buffer); #endif /* __cpp_lib_memory_resource >= 201603L */ -/// \brief Combines data slice with boolean flag to represent result of certain -/// operations. +/// \brief Combines data slice with boolean flag to represent result of certain operations. struct value_result { slice value; bool done; @@ -2758,8 +2727,7 @@ struct value_result { } }; -/// \brief Combines pair of slices for key and value to represent result of -/// certain operations. +/// \brief Combines pair of slices for key and value to represent result of certain operations. struct pair { using stl_pair = std::pair; slice key, value; @@ -2919,8 +2887,7 @@ MDBX_CXX01_CONSTEXPR_ENUM bool is_reverse(key_mode mode) noexcept { MDBX_CXX01_CONSTEXPR_ENUM bool is_msgpack(key_mode mode) noexcept { return mode == key_mode::msgpack; } -/// \brief Kind of the values and sorted multi-values with corresponding -/// comparison. +/// \brief Kind of the values and sorted multi-values with corresponding comparison. enum class value_mode { single = MDBX_DB_DEFAULTS, ///< Usual single value for each key. In terms of ///< keys, they are unique. @@ -3000,8 +2967,7 @@ MDBX_CXX01_CONSTEXPR_ENUM bool is_reverse(value_mode mode) noexcept { MDBX_CXX01_CONSTEXPR_ENUM bool is_msgpack(value_mode mode) noexcept { return mode == value_mode::msgpack; } -/// \brief A handle for an individual table (aka key-value space, maps or -/// sub-database) in the environment. +/// \brief A handle for an individual table (aka key-value space, maps or sub-database) in the environment. /// \see txn::open_map() \see txn::create_map() /// \see txn::clear_map() \see txn::drop_map() /// \see txn::get_handle_info() \see txn::get_map_stat() @@ -3130,12 +3096,10 @@ public: /// environment). intptr_t size_upper{default_value}; - /// \brief The growth step in bytes, must be greater than zero to allow the - /// database to grow. + /// \brief The growth step in bytes, must be greater than zero to allow the database to grow. intptr_t growth_step{default_value}; - /// \brief The shrink threshold in bytes, must be greater than zero to allow - /// the database to shrink. + /// \brief The shrink threshold in bytes, must be greater than zero to allow the database to shrink. intptr_t shrink_threshold{default_value}; /// \brief The database page size for new database creation @@ -3270,47 +3234,34 @@ public: static inline size_t pagesize_min() noexcept; /// \brief Returns the maximal database page size in bytes. static inline size_t pagesize_max() noexcept; - /// \brief Returns the minimal database size in bytes for specified page - /// size. + /// \brief Returns the minimal database size in bytes for specified page size. static inline size_t dbsize_min(intptr_t pagesize); - /// \brief Returns the maximal database size in bytes for specified page - /// size. + /// \brief Returns the maximal database size in bytes for specified page size. static inline size_t dbsize_max(intptr_t pagesize); - /// \brief Returns the minimal key size in bytes for specified table - /// flags. + /// \brief Returns the minimal key size in bytes for specified table flags. static inline size_t key_min(MDBX_db_flags_t flags) noexcept; /// \brief Returns the minimal key size in bytes for specified keys mode. static inline size_t key_min(key_mode mode) noexcept; - /// \brief Returns the maximal key size in bytes for specified page size and - /// table flags. + /// \brief Returns the maximal key size in bytes for specified page size and table flags. static inline size_t key_max(intptr_t pagesize, MDBX_db_flags_t flags); - /// \brief Returns the maximal key size in bytes for specified page size and - /// keys mode. + /// \brief Returns the maximal key size in bytes for specified page size and keys mode. static inline size_t key_max(intptr_t pagesize, key_mode mode); - /// \brief Returns the maximal key size in bytes for given environment and - /// table flags. + /// \brief Returns the maximal key size in bytes for given environment and table flags. static inline size_t key_max(const env &, MDBX_db_flags_t flags); - /// \brief Returns the maximal key size in bytes for given environment and - /// keys mode. + /// \brief Returns the maximal key size in bytes for given environment and keys mode. static inline size_t key_max(const env &, key_mode mode); - /// \brief Returns the minimal values size in bytes for specified table - /// flags. + /// \brief Returns the minimal values size in bytes for specified table flags. static inline size_t value_min(MDBX_db_flags_t flags) noexcept; - /// \brief Returns the minimal values size in bytes for specified values - /// mode. + /// \brief Returns the minimal values size in bytes for specified values mode. static inline size_t value_min(value_mode) noexcept; - /// \brief Returns the maximal value size in bytes for specified page size - /// and table flags. + /// \brief Returns the maximal value size in bytes for specified page size and table flags. static inline size_t value_max(intptr_t pagesize, MDBX_db_flags_t flags); - /// \brief Returns the maximal value size in bytes for specified page size - /// and values mode. + /// \brief Returns the maximal value size in bytes for specified page size and values mode. static inline size_t value_max(intptr_t pagesize, value_mode); - /// \brief Returns the maximal value size in bytes for given environment and - /// table flags. + /// \brief Returns the maximal value size in bytes for given environment and table flags. static inline size_t value_max(const env &, MDBX_db_flags_t flags); - /// \brief Returns the maximal value size in bytes for specified page size - /// and values mode. + /// \brief Returns the maximal value size in bytes for specified page size and values mode. static inline size_t value_max(const env &, value_mode); /// \brief Returns maximal size of key-value pair to fit in a single page @@ -3390,13 +3341,11 @@ public: /// \brief Make sure that the environment is not being used by other /// processes, or return an error otherwise. ensure_unused = MDBX_ENV_ENSURE_UNUSED, - /// \brief Wait until other processes closes the environment before - /// deletion. + /// \brief Wait until other processes closes the environment before deletion. wait_for_unused = MDBX_ENV_WAIT_FOR_UNUSED }; - /// \brief Removes the environment's files in a proper and multiprocess-safe - /// way. + /// \brief Removes the environment's files in a proper and multiprocess-safe way. #ifdef MDBX_STD_FILESYSTEM_PATH static bool remove(const MDBX_STD_FILESYSTEM_PATH &pathname, const remove_mode mode = just_remove); #endif /* MDBX_STD_FILESYSTEM_PATH */ @@ -3422,12 +3371,10 @@ public: /// \brief Return snapshot information about the MDBX environment. inline info get_info() const; - /// \brief Return statistics about the MDBX environment accordingly to the - /// specified transaction. + /// \brief Return statistics about the MDBX environment accordingly to the specified transaction. inline stat get_stat(const txn &) const; - /// \brief Return information about the MDBX environment accordingly to the - /// specified transaction. + /// \brief Return information about the MDBX environment accordingly to the specified transaction. inline info get_info(const txn &) const; /// \brief Returns the file descriptor for the DXB file of MDBX environment. @@ -3439,8 +3386,7 @@ public: /// Returns environment flags. inline MDBX_env_flags_t get_flags() const; - /// \brief Returns the maximum number of threads/reader slots for the - /// environment. + /// \brief Returns the maximum number of threads/reader slots for the environment. /// \see extra_runtime_option::max_readers inline unsigned max_readers() const; @@ -3824,8 +3770,7 @@ public: /// volume of dirty pages) in bytes. size_t size_max() const { return env().transaction_size_max(); } - /// \brief Returns current write transaction size (i.e.summary volume of dirty - /// pages) in bytes. + /// \brief Returns current write transaction size (i.e.summary volume of dirty pages) in bytes. size_t size_current() const { assert(is_readwrite()); return size_t(get_info().txn_space_dirty); @@ -3984,42 +3929,32 @@ public: inline map_handle::info get_handle_info(map_handle map) const; using canary = ::MDBX_canary; - /// \brief Set integers markers (aka "canary") associated with the - /// environment. + /// \brief Set integers markers (aka "canary") associated with the environment. inline txn &put_canary(const canary &); - /// \brief Returns fours integers markers (aka "canary") associated with the - /// environment. + /// \brief Returns fours integers markers (aka "canary") associated with the environment. inline canary get_canary() const; - /// Reads sequence generator associated with a key-value map (aka - /// table). + /// Reads sequence generator associated with a key-value map (aka table). inline uint64_t sequence(map_handle map) const; - /// \brief Reads and increment sequence generator associated with a key-value - /// map (aka table). + /// \brief Reads and increment sequence generator associated with a key-value map (aka table). inline uint64_t sequence(map_handle map, uint64_t increment); - /// \brief Compare two keys according to a particular key-value map (aka - /// table). + /// \brief Compare two keys according to a particular key-value map (aka table). inline int compare_keys(map_handle map, const slice &a, const slice &b) const noexcept; - /// \brief Compare two values according to a particular key-value map (aka - /// table). + /// \brief Compare two values according to a particular key-value map (aka table). inline int compare_values(map_handle map, const slice &a, const slice &b) const noexcept; - /// \brief Compare keys of two pairs according to a particular key-value map - /// (aka table). + /// \brief Compare keys of two pairs according to a particular key-value map (aka table). inline int compare_keys(map_handle map, const pair &a, const pair &b) const noexcept; - /// \brief Compare values of two pairs according to a particular key-value map - /// (aka table). + /// \brief Compare values of two pairs according to a particular key-value map(aka table). inline int compare_values(map_handle map, const pair &a, const pair &b) const noexcept; /// \brief Get value by key from a key-value map (aka table). inline slice get(map_handle map, const slice &key) const; - /// \brief Get first of multi-value and values count by key from a key-value - /// multimap (aka table). + /// \brief Get first of multi-value and values count by key from a key-value multimap (aka table). inline slice get(map_handle map, slice key, size_t &values_count) const; /// \brief Get value by key from a key-value map (aka table). inline slice get(map_handle map, const slice &key, const slice &value_at_absence) const; - /// \brief Get first of multi-value and values count by key from a key-value - /// multimap (aka table). + /// \brief Get first of multi-value and values count by key from a key-value multimap (aka table). inline slice get(map_handle map, slice key, size_t &values_count, const slice &value_at_absence) const; /// \brief Get value for equal or great key from a table. /// \return Bundle of key-value pair and boolean flag, @@ -4518,12 +4453,10 @@ public: //---------------------------------------------------------------------------- - /// \brief Renew/bind a cursor with a new transaction and previously used - /// key-value map handle. + /// \brief Renew/bind a cursor with a new transaction and previously used key-value map handle. inline void renew(::mdbx::txn &txn); - /// \brief Bind/renew a cursor with a new transaction and specified key-value - /// map handle. + /// \brief Bind/renew a cursor with a new transaction and specified key-value map handle. inline void bind(::mdbx::txn &txn, ::mdbx::map_handle map_handle); /// \brief Unbind cursor from a transaction. @@ -4556,12 +4489,10 @@ public: value_result try_insert(const pair &kv) { return try_insert(kv.key, kv.value); } void upsert(const pair &kv) { return upsert(kv.key, kv.value); } - /// \brief Removes single key-value pair or all multi-values at the current - /// cursor position. + /// \brief Removes single key-value pair or all multi-values at the current cursor position. inline bool erase(bool whole_multivalue = false); - /// \brief Seeks and removes first value or whole multi-value of the given - /// key. + /// \brief Seeks and removes first value or whole multi-value of the given key. /// \return `True` if the key is found and a value(s) is removed. inline bool erase(const slice &key, bool whole_multivalue = true); From e11d419d205732c2abe698c647883f2e7a51329c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:56:55 +0300 Subject: [PATCH 83/90] =?UTF-8?q?mdbx-tests:=20=D0=BF=D0=B5=D1=80=D0=B5?= =?UTF-8?q?=D1=85=D0=B2=D0=B0=D1=82=20=D0=B8=20=D0=BB=D0=BE=D0=B3=D0=B8?= =?UTF-8?q?=D1=80=D0=BE=D0=B2=D0=B0=D0=BD=D0=B8=D0=B5=20=D0=B8=D1=81=D0=BA?= =?UTF-8?q?=D0=BB=D1=8E=D1=87=D0=B5=D0=BD=D0=B8=D0=B9=20=D0=B2=20extra-C++?= =?UTF-8?q?=20=D1=82=D0=B5=D1=81=D1=82=D0=B0=D1=85=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- test/extra/crunched_delete.c++ | 19 +++++++++++++------ test/extra/cursor_closing.c++ | 19 +++++++++++++------ test/extra/dbi.c++ | 19 +++++++++++++------ test/extra/doubtless_positioning.c++ | 16 ++++++++++++---- test/extra/dupfix_addodd.c | 2 +- test/extra/dupfix_multiple.c++ | 7 +++---- test/extra/early_close_dbi.c++ | 17 +++++++++++++---- test/extra/hex_base64_base58.c++ | 16 ++++++++++++---- test/extra/maindb_ordinal.c++ | 18 +++++++++++++----- test/extra/open.c++ | 19 +++++++++++++------ test/extra/txn.c++ | 19 +++++++++++++------ 11 files changed, 119 insertions(+), 52 deletions(-) diff --git a/test/extra/crunched_delete.c++ b/test/extra/crunched_delete.c++ index 134dbafe..af2e6fad 100644 --- a/test/extra/crunched_delete.c++ +++ b/test/extra/crunched_delete.c++ @@ -349,12 +349,7 @@ bool simple(mdbx::env env) { return true; } -int main(int argc, const char *argv[]) { - (void)argc; - (void)argv; - - mdbx_setup_debug_nofmt(MDBX_LOG_NOTICE, MDBX_DBG_ASSERT, logger_nofmt, log_buffer, sizeof(log_buffer)); - +int doit() { mdbx::path db_filename = "test-crunched-del"; mdbx::env::remove(db_filename); @@ -392,3 +387,15 @@ int main(int argc, const char *argv[]) { std::cout << "OK\n"; return EXIT_SUCCESS; } + +int main(int argc, char *argv[]) { + (void)argc; + (void)argv; + mdbx_setup_debug_nofmt(MDBX_LOG_NOTICE, MDBX_DBG_ASSERT, logger_nofmt, log_buffer, sizeof(log_buffer)); + try { + return doit(); + } catch (const std::exception &ex) { + std::cerr << "Exception: " << ex.what() << "\n"; + return EXIT_FAILURE; + } +} diff --git a/test/extra/cursor_closing.c++ b/test/extra/cursor_closing.c++ index 60749dbd..5f575c89 100644 --- a/test/extra/cursor_closing.c++ +++ b/test/extra/cursor_closing.c++ @@ -316,12 +316,7 @@ bool case1(mdbx::env env) { //-------------------------------------------------------------------------------------------- -int main(int argc, const char *argv[]) { - (void)argc; - (void)argv; - - mdbx_setup_debug_nofmt(MDBX_LOG_NOTICE, MDBX_DBG_ASSERT, logger_nofmt, log_buffer, sizeof(log_buffer)); - +int doit() { mdbx::path db_filename = "test-cursor-closing"; mdbx::env::remove(db_filename); @@ -339,3 +334,15 @@ int main(int argc, const char *argv[]) { return EXIT_FAILURE; } } + +int main(int argc, char *argv[]) { + (void)argc; + (void)argv; + mdbx_setup_debug_nofmt(MDBX_LOG_NOTICE, MDBX_DBG_ASSERT, logger_nofmt, log_buffer, sizeof(log_buffer)); + try { + return doit(); + } catch (const std::exception &ex) { + std::cerr << "Exception: " << ex.what() << "\n"; + return EXIT_FAILURE; + } +} diff --git a/test/extra/dbi.c++ b/test/extra/dbi.c++ index fe238415..9aca25a5 100644 --- a/test/extra/dbi.c++ +++ b/test/extra/dbi.c++ @@ -11,12 +11,7 @@ static void logger_nofmt(MDBX_log_level_t loglevel, const char *function, int li fprintf(stdout, "%s:%u %s", function, line, msg); } -int main(int argc, const char *argv[]) { - (void)argc; - (void)argv; - - mdbx_setup_debug_nofmt(MDBX_LOG_NOTICE, MDBX_DBG_ASSERT, logger_nofmt, log_buffer, sizeof(log_buffer)); - +int doit() { mdbx::path db_filename = "test-dbi"; mdbx::env::remove(db_filename); @@ -71,3 +66,15 @@ int main(int argc, const char *argv[]) { std::cout << "OK\n"; return EXIT_SUCCESS; } + +int main(int argc, char *argv[]) { + (void)argc; + (void)argv; + mdbx_setup_debug_nofmt(MDBX_LOG_NOTICE, MDBX_DBG_ASSERT, logger_nofmt, log_buffer, sizeof(log_buffer)); + try { + return doit(); + } catch (const std::exception &ex) { + std::cerr << "Exception: " << ex.what() << "\n"; + return EXIT_FAILURE; + } +} diff --git a/test/extra/doubtless_positioning.c++ b/test/extra/doubtless_positioning.c++ index ecc24daa..a7a50e41 100644 --- a/test/extra/doubtless_positioning.c++ +++ b/test/extra/doubtless_positioning.c++ @@ -211,10 +211,7 @@ static bool test(mdbx::txn txn, mdbx::map_handle dbi) { return ok; } -int main(int argc, const char *argv[]) { - (void)argc; - (void)argv; - +int doit() { mdbx::path db_filename = "test-posi"; mdbx::env_managed::remove(db_filename); mdbx::env_managed env(db_filename, mdbx::env_managed::create_parameters(), mdbx::env::operate_parameters(3)); @@ -243,3 +240,14 @@ int main(int argc, const char *argv[]) { std::cout << "OK\n"; return EXIT_SUCCESS; } + +int main(int argc, char *argv[]) { + (void)argc; + (void)argv; + try { + return doit(); + } catch (const std::exception &ex) { + std::cerr << "Exception: " << ex.what() << "\n"; + return EXIT_FAILURE; + } +} diff --git a/test/extra/dupfix_addodd.c b/test/extra/dupfix_addodd.c index 7d976f9b..4d79b27e 100644 --- a/test/extra/dupfix_addodd.c +++ b/test/extra/dupfix_addodd.c @@ -25,7 +25,7 @@ int main() { MDBX_val key, data; MDBX_txn *txn = NULL; - const char *db_filename = "./example-db"; + const char *db_filename = "./test-dupfix-addodd"; mdbx_env_delete(db_filename, MDBX_ENV_JUST_DELETE); rc = mdbx_env_create(&env); diff --git a/test/extra/dupfix_multiple.c++ b/test/extra/dupfix_multiple.c++ index ecc02bc7..f4aa94be 100644 --- a/test/extra/dupfix_multiple.c++ +++ b/test/extra/dupfix_multiple.c++ @@ -9,7 +9,7 @@ using buffer = mdbx::default_buffer; bool case1_ordering(mdbx::env env) { auto txn = env.start_write(); - auto map = txn.create_map(nullptr, mdbx::key_mode::ordinal, mdbx::value_mode::multi_ordinal); + auto map = txn.create_map("case1", mdbx::key_mode::ordinal, mdbx::value_mode::multi_ordinal); txn.insert(map, buffer::key_from_u64(21), buffer::key_from_u64(18)); txn.insert(map, buffer::key_from_u64(7), buffer::key_from_u64(19)); @@ -175,7 +175,7 @@ bool case1_ordering(mdbx::env env) { bool case2_batch_read(mdbx::env env) { auto txn = env.start_write(); - auto map = txn.create_map(nullptr, mdbx::key_mode::usual, mdbx::value_mode::multi_samelength); + auto map = txn.create_map("case2", mdbx::key_mode::usual, mdbx::value_mode::multi_samelength); txn.upsert(map, mdbx::slice("key1"), mdbx::slice("val1")); txn.upsert(map, mdbx::pair("key1", "val2")); txn.upsert(map, mdbx::pair("key1", "val3")); @@ -270,7 +270,7 @@ bool case3_put_a_lot(mdbx::env env) { int doit() { mdbx::path db_filename = "test-dupfix-multiple"; mdbx::env_managed::remove(db_filename); - mdbx::env_managed env(db_filename, mdbx::env_managed::create_parameters(), mdbx::env::operate_parameters(1)); + mdbx::env_managed env(db_filename, mdbx::env_managed::create_parameters(), mdbx::env::operate_parameters(3)); bool ok = case1_ordering(env); ok = case2_batch_read(env) && ok; @@ -288,7 +288,6 @@ int doit() { int main(int argc, const char *argv[]) { (void)argc; (void)argv; - try { return doit(); } catch (const std::exception &ex) { diff --git a/test/extra/early_close_dbi.c++ b/test/extra/early_close_dbi.c++ index 3eeea42c..cb5d1b6d 100644 --- a/test/extra/early_close_dbi.c++ +++ b/test/extra/early_close_dbi.c++ @@ -1,13 +1,11 @@ #include "mdbx.h++" #include +#include static const char *const testkey = "testkey"; static uint64_t testval = 11; -int main(int argc, char *argv[]) { - (void)argc; - (void)argv; - +int doit() { mdbx::path db_filename = "test-early_close_dbi"; mdbx::env_managed::remove(db_filename); @@ -126,3 +124,14 @@ int main(int argc, char *argv[]) { return 0; } + +int main(int argc, char *argv[]) { + (void)argc; + (void)argv; + try { + return doit(); + } catch (const std::exception &ex) { + std::cerr << "Exception: " << ex.what() << "\n"; + return EXIT_FAILURE; + } +} diff --git a/test/extra/hex_base64_base58.c++ b/test/extra/hex_base64_base58.c++ index 972673e9..6196d348 100644 --- a/test/extra/hex_base64_base58.c++ +++ b/test/extra/hex_base64_base58.c++ @@ -75,10 +75,7 @@ static bool basic() { return ok; } -int main(int argc, const char *argv[]) { - (void)argc; - (void)argv; - +int doit() { auto ok = basic(); for (size_t n = 0; n < 1000; ++n) { for (size_t length = 0; ok && length < 111; ++length) { @@ -108,3 +105,14 @@ int main(int argc, const char *argv[]) { std::cout << "OK\n"; return EXIT_SUCCESS; } + +int main(int argc, char *argv[]) { + (void)argc; + (void)argv; + try { + return doit(); + } catch (const std::exception &ex) { + std::cerr << "Exception: " << ex.what() << "\n"; + return EXIT_FAILURE; + } +} diff --git a/test/extra/maindb_ordinal.c++ b/test/extra/maindb_ordinal.c++ index 9e6bd7ef..359cff89 100644 --- a/test/extra/maindb_ordinal.c++ +++ b/test/extra/maindb_ordinal.c++ @@ -4,11 +4,8 @@ #include "mdbx.h++" #include -int main(int argc, const char *argv[]) { - (void)argc; - (void)argv; - - mdbx::path db_filename = "test-dupfix-multiple"; +static int doit() { + mdbx::path db_filename = "test-maindb-ordinal"; mdbx::env_managed::remove(db_filename); mdbx::env_managed env(db_filename, mdbx::env_managed::create_parameters(), mdbx::env::operate_parameters()); @@ -51,3 +48,14 @@ int main(int argc, const char *argv[]) { return EXIT_SUCCESS; #endif /* __cpp_lib_string_view >= 201606L */ } + +int main(int argc, char *argv[]) { + (void)argc; + (void)argv; + try { + return doit(); + } catch (const std::exception &ex) { + std::cerr << "Exception: " << ex.what() << "\n"; + return EXIT_FAILURE; + } +} diff --git a/test/extra/open.c++ b/test/extra/open.c++ index 459ca708..353b848e 100644 --- a/test/extra/open.c++ +++ b/test/extra/open.c++ @@ -25,12 +25,7 @@ static void logger_nofmt(MDBX_log_level_t loglevel, const char *function, int li fprintf(stdout, "%s:%u %s", function, line, msg); } -int main(int argc, const char *argv[]) { - (void)argc; - (void)argv; - - mdbx_setup_debug_nofmt(MDBX_LOG_VERBOSE, MDBX_DBG_ASSERT, logger_nofmt, log_buffer, sizeof(log_buffer)); - +int doit() { mdbx::path path = "test-open"; mdbx::env::remove(path); @@ -86,4 +81,16 @@ int main(int argc, const char *argv[]) { return EXIT_SUCCESS; } +int main(int argc, char *argv[]) { + (void)argc; + (void)argv; + mdbx_setup_debug_nofmt(MDBX_LOG_VERBOSE, MDBX_DBG_ASSERT, logger_nofmt, log_buffer, sizeof(log_buffer)); + try { + return doit(); + } catch (const std::exception &ex) { + std::cerr << "Exception: " << ex.what() << "\n"; + return EXIT_FAILURE; + } +} + #endif /* __cpp_lib_latch */ diff --git a/test/extra/txn.c++ b/test/extra/txn.c++ index e1abb1e6..583328cf 100644 --- a/test/extra/txn.c++ +++ b/test/extra/txn.c++ @@ -308,12 +308,7 @@ bool case2(const mdbx::path &path, bool no_sticky_threads) { return true; } -int main(int argc, const char *argv[]) { - (void)argc; - (void)argv; - - mdbx_setup_debug_nofmt(MDBX_LOG_VERBOSE, MDBX_DBG_ASSERT, logger_nofmt, log_buffer, sizeof(log_buffer)); - +int doit() { mdbx::path path = "test-txn"; mdbx::env::remove(path); @@ -326,4 +321,16 @@ int main(int argc, const char *argv[]) { return ok ? EXIT_SUCCESS : EXIT_FAILURE; } +int main(int argc, char *argv[]) { + (void)argc; + (void)argv; + mdbx_setup_debug_nofmt(MDBX_LOG_VERBOSE, MDBX_DBG_ASSERT, logger_nofmt, log_buffer, sizeof(log_buffer)); + try { + return doit(); + } catch (const std::exception &ex) { + std::cerr << "Exception: " << ex.what() << "\n"; + return EXIT_FAILURE; + } +} + #endif /* __cpp_lib_latch */ From 3a0dbee58ca54dbdb008c132df0e9a12c1f03d29 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:57:38 +0300 Subject: [PATCH 84/90] =?UTF-8?q?mdbx-tests:=20=D1=83=D1=81=D1=82=D1=80?= =?UTF-8?q?=D0=B0=D0=BD=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=BD=D0=B5=D0=B2=D1=8B?= =?UTF-8?q?=D1=80=D0=B0=D0=B2=D0=BD=D0=B5=D0=BD=D0=BD=D0=BE=D0=B3=D0=BE=20?= =?UTF-8?q?=D0=B4=D0=BE=D1=81=D1=82=D1=83=D0=BF=D0=B0=20=D0=B2=20extra/clo?= =?UTF-8?q?se-dbi=20=D0=B4=D0=BB=D1=8F=20UBSAN=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- test/extra/early_close_dbi.c++ | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/test/extra/early_close_dbi.c++ b/test/extra/early_close_dbi.c++ index cb5d1b6d..0ed94fb6 100644 --- a/test/extra/early_close_dbi.c++ +++ b/test/extra/early_close_dbi.c++ @@ -65,13 +65,13 @@ int doit() { assert(err == MDBX_SUCCESS); err = mdbx_get(transaction, textindex, &mdbxkey, &mdbxval); assert(err == MDBX_SUCCESS); - assert(testval == *reinterpret_cast(mdbxval.iov_base)); + assert(testval == mdbx::slice(mdbxval).as_uint64()); err = mdbx_put(transaction, textindex, &mdbxkey, &mdbxput, MDBX_NOOVERWRITE); assert(err == MDBX_KEYEXIST); err = mdbx_get(transaction, textindex, &mdbxkey, &mdbxval); assert(err == MDBX_SUCCESS); - assert(testval == *reinterpret_cast(mdbxval.iov_base)); + assert(testval == mdbx::slice(mdbxval).as_uint64()); err = mdbx_dbi_flags_ex(transaction, textindex, &dbi_flags, &dbi_state); assert(err == MDBX_SUCCESS); @@ -87,7 +87,7 @@ int doit() { assert(err == MDBX_SUCCESS); err = mdbx_get(transaction, textindex, &mdbxkey, &mdbxval); assert(err == MDBX_SUCCESS); - assert(testval == *reinterpret_cast(mdbxval.iov_base)); + assert(testval == mdbx::slice(mdbxval).as_uint64()); err = mdbx_dbi_close(environment, textindex); assert(err == MDBX_SUCCESS); From 3c3628c7989f75a343e48d50c868e2f60c3220d1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 01:58:02 +0300 Subject: [PATCH 85/90] =?UTF-8?q?mdbx-tests:=20=D0=BA=D1=80=D0=B0=D1=82?= =?UTF-8?q?=D0=BD=D0=BE=D0=B5=20=D1=81=D0=BE=D0=BA=D1=80=D0=B0=D1=89=D0=B5?= =?UTF-8?q?=D0=BD=D0=B8=D0=B5=20=D0=B8=D1=82=D0=B5=D1=80=D0=B0=D1=86=D0=B8?= =?UTF-8?q?=D0=B9=20=D1=82=D0=B5=D1=81=D1=82=D0=BE=D0=B2=20=D0=B2=20=D0=B7?= =?UTF-8?q?=D0=B0=D0=B2=D0=B8=D1=81=D0=B8=D0=BC=D0=BE=D1=81=D1=82=D0=B8=20?= =?UTF-8?q?=D0=BE=D1=82=20=D0=BA=D0=BE=D0=BD=D1=84=D0=B8=D0=B3=D1=83=D1=80?= =?UTF-8?q?=D0=B0=D1=86=D0=B8=D0=B8=20Valgrind/Debug/CI=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- test/CMakeLists.txt | 4 ++-- test/extra/crunched_delete.c++ | 18 +++++++++++++----- test/extra/cursor_closing.c++ | 34 +++++++++++++++++++++++++--------- test/extra/dupfix_multiple.c++ | 24 +++++++++++++++++++----- test/extra/txn.c++ | 16 +++++++++++++++- 5 files changed, 74 insertions(+), 22 deletions(-) diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt index d4dfc086..dda4a909 100644 --- a/test/CMakeLists.txt +++ b/test/CMakeLists.txt @@ -300,10 +300,10 @@ else() endif() if(MDBX_BUILD_CXX) if(NOT WIN32 OR NOT MDBX_CXX_STANDARD LESS 17) - add_extra_test(cursor_closing) + add_extra_test(cursor_closing TIMEOUT 10800) add_extra_test(early_close_dbi) add_extra_test(maindb_ordinal) - add_extra_test(dupfix_multiple) + add_extra_test(dupfix_multiple TIMEOUT 10800) add_extra_test(doubtless_positioning TIMEOUT 10800) add_extra_test(crunched_delete TIMEOUT 10800) add_extra_test(dbi) diff --git a/test/extra/crunched_delete.c++ b/test/extra/crunched_delete.c++ index af2e6fad..4b740664 100644 --- a/test/extra/crunched_delete.c++ +++ b/test/extra/crunched_delete.c++ @@ -5,13 +5,21 @@ #include #include -#if MDBX_DEBUG || !defined(NDEBUG) || defined(__APPLE__) || defined(_WIN32) -#define NN 1024 -#elif UINTPTR_MAX > 0xffffFFFFul || ULONG_MAX > 0xffffFFFFul -#define NN 4096 +#if defined(ENABLE_MEMCHECK) || defined(MDBX_CI) +#if MDBX_DEBUG || !defined(NDEBUG) +#define RELIEF_FACTOR 16 #else -#define NN 2048 +#define RELIEF_FACTOR 8 #endif +#elif MDBX_DEBUG || !defined(NDEBUG) || defined(__APPLE__) || defined(_WIN32) +#define RELIEF_FACTOR 4 +#elif UINTPTR_MAX > 0xffffFFFFul || ULONG_MAX > 0xffffFFFFul +#define RELIEF_FACTOR 2 +#else +#define RELIEF_FACTOR 1 +#endif + +#define NN (2048 / RELIEF_FACTOR) std::string format_va(const char *fmt, va_list ap) { va_list ones; diff --git a/test/extra/cursor_closing.c++ b/test/extra/cursor_closing.c++ index 5f575c89..24633560 100644 --- a/test/extra/cursor_closing.c++ +++ b/test/extra/cursor_closing.c++ @@ -9,6 +9,22 @@ #include #endif +#if defined(ENABLE_MEMCHECK) || defined(MDBX_CI) +#if MDBX_DEBUG || !defined(NDEBUG) +#define RELIEF_FACTOR 16 +#else +#define RELIEF_FACTOR 8 +#endif +#elif MDBX_DEBUG || !defined(NDEBUG) || defined(__APPLE__) || defined(_WIN32) +#define RELIEF_FACTOR 4 +#elif UINTPTR_MAX > 0xffffFFFFul || ULONG_MAX > 0xffffFFFFul +#define RELIEF_FACTOR 2 +#else +#define RELIEF_FACTOR 1 +#endif + +#define NN (1000 / RELIEF_FACTOR) + static void logger_nofmt(MDBX_log_level_t loglevel, const char *function, int line, const char *msg, unsigned length) noexcept { (void)length; @@ -113,10 +129,10 @@ void case1_shuffle_pool(std::vector &pool) { void case1_read_pool(std::vector &pool) { for (auto c : pool) if (flipcoin()) - mdbx::cursor(c).find_multivalue(mdbx::slice::wrap(prng(1000)), mdbx::slice::wrap(prng(1000)), false); + mdbx::cursor(c).find_multivalue(mdbx::slice::wrap(prng(NN)), mdbx::slice::wrap(prng(NN)), false); for (auto c : pool) if (flipcoin()) - mdbx::cursor(c).find_multivalue(mdbx::slice::wrap(prng(1000)), mdbx::slice::wrap(prng(1000)), false); + mdbx::cursor(c).find_multivalue(mdbx::slice::wrap(prng(NN)), mdbx::slice::wrap(prng(NN)), false); } MDBX_cursor *case1_try_unbind(MDBX_cursor *cursor) { @@ -224,9 +240,9 @@ void case1_write_cycle(mdbx::txn_managed txn, std::deque &dbi, pre.unbind(); if (!pre.txn()) pre.bind(txn, dbi[prng(dbi.size())]); - for (auto i = 0; i < 1000; ++i) { - auto k = mdbx::default_buffer::wrap(prng(1000)); - auto v = mdbx::default_buffer::wrap(prng(1000)); + for (auto i = 0; i < NN; ++i) { + auto k = mdbx::default_buffer::wrap(prng(NN)); + auto v = mdbx::default_buffer::wrap(prng(NN)); if (pre.find_multivalue(k, v, false)) pre.erase(); else @@ -246,8 +262,8 @@ void case1_write_cycle(mdbx::txn_managed txn, std::deque &dbi, bool case1_thread(mdbx::env env, std::deque dbi, mdbx::cursor pre) { salt = size_t(std::chrono::high_resolution_clock::now().time_since_epoch().count()); std::vector pool; - for (auto loop = 0; loop < 333; ++loop) { - for (auto read = 0; read < 333; ++read) { + for (auto loop = 0; loop < 333 / RELIEF_FACTOR; ++loop) { + for (auto read = 0; read < 333 / RELIEF_FACTOR; ++read) { auto txn = env.start_read(); case1_read_cycle(txn, dbi, pool, pre); if (flipcoin()) @@ -280,8 +296,8 @@ bool case1(mdbx::env env) { auto txn = env.start_write(); auto table = txn.create_map(std::to_string(t), mdbx::key_mode::ordinal, mdbx::value_mode::multi_samelength); auto cursor = txn.open_cursor(table); - for (size_t i = 0; i < 10000; ++i) - cursor.upsert(mdbx::default_buffer::wrap(prng(1000)), mdbx::default_buffer::wrap(prng(1000))); + for (size_t i = 0; i < NN * 11; ++i) + cursor.upsert(mdbx::default_buffer::wrap(prng(NN)), mdbx::default_buffer::wrap(prng(NN))); txn.commit(); cursors.push_back(std::move(cursor)); diff --git a/test/extra/dupfix_multiple.c++ b/test/extra/dupfix_multiple.c++ index f4aa94be..a18ebfc8 100644 --- a/test/extra/dupfix_multiple.c++ +++ b/test/extra/dupfix_multiple.c++ @@ -5,6 +5,20 @@ #include #include +#if defined(ENABLE_MEMCHECK) || defined(MDBX_CI) +#if MDBX_DEBUG || !defined(NDEBUG) +#define RELIEF_FACTOR 16 +#else +#define RELIEF_FACTOR 8 +#endif +#elif MDBX_DEBUG || !defined(NDEBUG) || defined(__APPLE__) || defined(_WIN32) +#define RELIEF_FACTOR 4 +#elif UINTPTR_MAX > 0xffffFFFFul || ULONG_MAX > 0xffffFFFFul +#define RELIEF_FACTOR 2 +#else +#define RELIEF_FACTOR 1 +#endif + using buffer = mdbx::default_buffer; bool case1_ordering(mdbx::env env) { @@ -224,7 +238,7 @@ static size_t prng() { static inline size_t prng(size_t range) { return prng() % range; } static mdbx::default_buffer_pair prng_kv(size_t n, size_t space) { - space = (space + !space) * 1024 * 32; + space = (space + !space) * 1024 * 32 / RELIEF_FACTOR; const size_t w = (n ^ 1455614549) * 1664525 + 1013904223; const size_t k = (prng(42 + w % space) ^ 1725278851) * 433750991; const size_t v = prng(); @@ -235,11 +249,11 @@ bool case3_put_a_lot(mdbx::env env) { salt = size_t(std::chrono::high_resolution_clock::now().time_since_epoch().count()); auto txn = env.start_write(); auto map = txn.create_map("case3", mdbx::key_mode::ordinal, mdbx::value_mode::multi_ordinal); - for (size_t n = 0; n < 5555555; ++n) + for (size_t n = 0; n < 5555555 / RELIEF_FACTOR; ++n) txn.upsert(map, prng_kv(n, 1)); txn.commit(); - for (size_t t = 0; t < 555; ++t) { + for (size_t t = 0; t < 555 / RELIEF_FACTOR; ++t) { txn = env.start_write(); auto cursor = txn.open_cursor(map); for (size_t n = 0; n < 111; ++n) { @@ -247,7 +261,7 @@ bool case3_put_a_lot(mdbx::env env) { const auto r = 1 + prng(3); if (r & 1) { const auto k = prng_kv(n + t, 2).key; - for (size_t i = prng(42 + prng(111) * prng(111)); i > 0; --i) + for (size_t i = prng(42 + prng(111) * prng(111 / RELIEF_FACTOR)); i > 0; --i) v.push_back(prng()); txn.put_multiple_samelength(map, k, v, mdbx::upsert); } @@ -255,7 +269,7 @@ bool case3_put_a_lot(mdbx::env env) { const auto k = prng_kv(n + t, 2).key; if (cursor.seek(k)) { v.clear(); - for (size_t i = prng(42 + prng(111) * prng(111)); i > 0; --i) + for (size_t i = prng(42 + prng(111) * prng(111 / RELIEF_FACTOR)); i > 0; --i) v.push_back(prng()); cursor.put_multiple_samelength(k, v, mdbx::upsert); } diff --git a/test/extra/txn.c++ b/test/extra/txn.c++ index 583328cf..40f9ab0b 100644 --- a/test/extra/txn.c++ +++ b/test/extra/txn.c++ @@ -3,6 +3,20 @@ #include +#if defined(ENABLE_MEMCHECK) || defined(MDBX_CI) +#if MDBX_DEBUG || !defined(NDEBUG) +#define RELIEF_FACTOR 16 +#else +#define RELIEF_FACTOR 8 +#endif +#elif MDBX_DEBUG || !defined(NDEBUG) || defined(__APPLE__) || defined(_WIN32) +#define RELIEF_FACTOR 4 +#elif UINTPTR_MAX > 0xffffFFFFul || ULONG_MAX > 0xffffFFFFul +#define RELIEF_FACTOR 2 +#else +#define RELIEF_FACTOR 1 +#endif + #if !defined(__cpp_lib_latch) && __cpp_lib_latch < 201907L int main(int argc, const char *argv[]) { @@ -295,7 +309,7 @@ bool case2(const mdbx::path &path, bool no_sticky_threads) { for (size_t n = 0; n < 8; ++n) l.push_back(std::thread([&]() { s.wait(); - for (size_t i = 0; i < 1000000; ++i) { + for (size_t i = 0; i < 1000000 / RELIEF_FACTOR; ++i) { auto txn = env.start_read(); txn.abort(); } From 8157d07b009650e18e45b32d1e4d60ac30bbf3c4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 13:53:47 +0300 Subject: [PATCH 86/90] =?UTF-8?q?mdbx:=20=D0=B8=D1=81=D0=BF=D0=BE=D0=BB?= =?UTF-8?q?=D1=8C=D0=B7=D0=BE=D0=B2=D0=B0=D0=BD=D0=B8=D0=B5=20=D0=BF=D1=80?= =?UTF-8?q?=D0=B8=20=D0=BD=D0=B0=D0=BB=D0=B8=D1=87=D0=B8=D0=B8=20`EREMOTEI?= =?UTF-8?q?O`=20=D0=B2=D0=BC=D0=B5=D1=81=D1=82=D0=BE=20`ENOTBLK`=20=D0=B2?= =?UTF-8?q?=20=D0=BA=D0=B0=D1=87=D0=B5=D1=81=D1=82=D0=B2=D0=B5=20`MDBX=5FE?= =?UTF-8?q?REMOTE`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- mdbx.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/mdbx.h b/mdbx.h index a6cdb7de..f515eae1 100644 --- a/mdbx.h +++ b/mdbx.h @@ -2006,7 +2006,12 @@ typedef enum MDBX_error { MDBX_EPERM = EPERM, MDBX_EINTR = EINTR, MDBX_ENOFILE = ENOENT, +#if defined(EREMOTEIO) || defined(DOXYGEN) + /** Cannot use the database on a network file system or when exporting it via NFS. */ + MDBX_EREMOTE = EREMOTEIO, +#else MDBX_EREMOTE = ENOTBLK, +#endif /* EREMOTEIO */ MDBX_EDEADLK = EDEADLK #endif /* !Windows */ } MDBX_error_t; From 35349cf5388217ccdbd6c81eb6bcdffa83c8d02b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 13:55:07 +0300 Subject: [PATCH 87/90] =?UTF-8?q?mdbx:=20=D0=B4=D0=BE=D0=B1=D0=B0=D0=B2?= =?UTF-8?q?=D0=BB=D0=B5=D0=BD=D0=B8=D0=B5=20=D0=BE=D0=BF=D1=86=D0=B8=D0=B8?= =?UTF-8?q?=20=D1=81=D0=B1=D0=BE=D1=80=D0=BA=D0=B8=20`MDBX=5FENABLE=5FNON?= =?UTF-8?q?=5FREADONLY=5FEXPORT`=20=D0=B8=20=D0=BB=D0=BE=D0=B3=D0=B8=D1=80?= =?UTF-8?q?=D0=BE=D0=B2=D0=B0=D0=BD=D0=B8=D0=B5=20=D1=81=D0=BE=D0=BE=D1=82?= =?UTF-8?q?=D0=B2=D0=B5=D1=82=D1=81=D1=82=D0=B2=D1=83=D1=8E=D1=89=D0=B8?= =?UTF-8?q?=D1=85=20=D1=81=D0=B8=D1=82=D1=83=D0=B0=D1=86=D0=B8=D0=B9=20(ba?= =?UTF-8?q?ckport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Закрывает [запрос](https://gitflic.ru/project/erthink/libmdbx/issue/16). --- src/dxb.c | 2 +- src/lck.c | 5 +++-- src/options.h | 8 ++++++++ src/osal.c | 45 ++++++++++++++++++++++++++++++++------------- src/osal.h | 3 ++- 5 files changed, 46 insertions(+), 17 deletions(-) diff --git a/src/dxb.c b/src/dxb.c index fa3b8f6e..29df1ee6 100644 --- a/src/dxb.c +++ b/src/dxb.c @@ -681,7 +681,7 @@ __cold int dxb_setup(MDBX_env *env, const int lck_rc, const mdbx_mode_t mode_bit !(env->flags & MDBX_NORDAHEAD) && mdbx_is_readahead_reasonable(used_bytes, 0) == MDBX_RESULT_TRUE; err = osal_mmap(env->flags, &env->dxb_mmap, env->geo_in_bytes.now, env->geo_in_bytes.upper, - (lck_rc && env->stuck_meta < 0) ? MMAP_OPTION_TRUNCATE : 0); + (lck_rc && env->stuck_meta < 0) ? MMAP_OPTION_TRUNCATE : 0, env->pathname.dxb); if (unlikely(err != MDBX_SUCCESS)) return err; diff --git a/src/lck.c b/src/lck.c index 0bb3205c..0e19b29c 100644 --- a/src/lck.c +++ b/src/lck.c @@ -62,8 +62,9 @@ __cold static int lck_setup_locked(MDBX_env *env) { } env->max_readers = (maxreaders <= MDBX_READERS_LIMIT) ? (unsigned)maxreaders : (unsigned)MDBX_READERS_LIMIT; - err = osal_mmap((env->flags & MDBX_EXCLUSIVE) | MDBX_WRITEMAP, &env->lck_mmap, (size_t)size, (size_t)size, - lck_seize_rc ? MMAP_OPTION_TRUNCATE | MMAP_OPTION_SEMAPHORE : MMAP_OPTION_SEMAPHORE); + err = + osal_mmap((env->flags & MDBX_EXCLUSIVE) | MDBX_WRITEMAP, &env->lck_mmap, (size_t)size, (size_t)size, + lck_seize_rc ? MMAP_OPTION_TRUNCATE | MMAP_OPTION_SEMAPHORE : MMAP_OPTION_SEMAPHORE, env->pathname.lck); if (unlikely(err != MDBX_SUCCESS)) return err; diff --git a/src/options.h b/src/options.h index 9449fed9..7ea12ac8 100644 --- a/src/options.h +++ b/src/options.h @@ -257,6 +257,14 @@ #error MDBX_HAVE_BUILTIN_CPU_SUPPORTS must be defined as 0 or 1 #endif /* MDBX_HAVE_BUILTIN_CPU_SUPPORTS */ +/** if enabled then instead of the returned error `MDBX_REMOTE`, only a warning is issued, when + * the database being opened in non-read-only mode is located in a file system exported via NFS. */ +#ifndef MDBX_ENABLE_NON_READONLY_EXPORT +#define MDBX_ENABLE_NON_READONLY_EXPORT 0 +#elif !(MDBX_ENABLE_NON_READONLY_EXPORT == 0 || MDBX_ENABLE_NON_READONLY_EXPORT == 1) +#error MDBX_ENABLE_NON_READONLY_EXPORT must be defined as 0 or 1 +#endif /* MDBX_ENABLE_NON_READONLY_EXPORT */ + //------------------------------------------------------------------------------ /** Win32 File Locking API for \ref MDBX_LOCKING */ diff --git a/src/osal.c b/src/osal.c index d4f4b03a..ec2e0d7c 100644 --- a/src/osal.c +++ b/src/osal.c @@ -1760,7 +1760,7 @@ static int osal_check_fs_local(mdbx_filehandle_t handle, int flags) { if ((RemoteProtocolInfo.Flags & REMOTE_PROTOCOL_INFO_FLAG_OFFLINE) && !(flags & MDBX_RDONLY)) return ERROR_FILE_OFFLINE; if (!(RemoteProtocolInfo.Flags & REMOTE_PROTOCOL_INFO_FLAG_LOOPBACK) && !(flags & MDBX_EXCLUSIVE)) - return ERROR_REMOTE_STORAGE_MEDIA_ERROR; + return MDBX_EREMOTE; } } @@ -1779,7 +1779,7 @@ static int osal_check_fs_local(mdbx_filehandle_t handle, int flags) { 0, &GetExternalBacking_OutputBuffer, sizeof(GetExternalBacking_OutputBuffer)); if (NT_SUCCESS(rc)) { if (!(flags & MDBX_EXCLUSIVE)) - return ERROR_REMOTE_STORAGE_MEDIA_ERROR; + return MDBX_EREMOTE; } else if (rc != STATUS_OBJECT_NOT_EXTERNALLY_BACKED && rc != STATUS_INVALID_DEVICE_REQUEST && rc != STATUS_NOT_SUPPORTED) return ntstatus2errcode(rc); @@ -1800,7 +1800,7 @@ static int osal_check_fs_local(mdbx_filehandle_t handle, int flags) { if ((flags & MDBX_RDONLY) == 0) { if (FileSystemFlags & (FILE_SEQUENTIAL_WRITE_ONCE | FILE_READ_ONLY_VOLUME | FILE_VOLUME_IS_COMPRESSED)) { - rc = ERROR_REMOTE_STORAGE_MEDIA_ERROR; + rc = MDBX_EREMOTE; goto bailout; } } @@ -1808,7 +1808,7 @@ static int osal_check_fs_local(mdbx_filehandle_t handle, int flags) { if (imports.GetFinalPathNameByHandleW(handle, PathBuffer, INT16_MAX, FILE_NAME_NORMALIZED | VOLUME_NAME_NT)) { if (_wcsnicmp(PathBuffer, L"\\Device\\Mup\\", 12) == 0) { if (!(flags & MDBX_EXCLUSIVE)) { - rc = ERROR_REMOTE_STORAGE_MEDIA_ERROR; + rc = MDBX_EREMOTE; goto bailout; } } @@ -1836,7 +1836,7 @@ static int osal_check_fs_local(mdbx_filehandle_t handle, int flags) { case DRIVE_REMOTE: default: if (!(flags & MDBX_EXCLUSIVE)) - rc = ERROR_REMOTE_STORAGE_MEDIA_ERROR; + rc = MDBX_EREMOTE; // fall through case DRIVE_REMOVABLE: case DRIVE_FIXED: @@ -1968,11 +1968,11 @@ static int osal_check_fs_local(mdbx_filehandle_t handle, int flags) { #endif /* ST/MNT_LOCAL */ #ifdef ST_EXPORTED - if ((st_flags & ST_EXPORTED) != 0 && !(flags & MDBX_RDONLY)) - return MDBX_EREMOTE; + if ((st_flags & ST_EXPORTED) != 0 && !(flags & (MDBX_RDONLY | MDBX_EXCLUSIVE)))) + return MDBX_RESULT_TRUE; #elif defined(MNT_EXPORTED) - if ((mnt_flags & MNT_EXPORTED) != 0 && !(flags & MDBX_RDONLY)) - return MDBX_EREMOTE; + if ((mnt_flags & MNT_EXPORTED) != 0 && !(flags & (MDBX_RDONLY | MDBX_EXCLUSIVE))) + return MDBX_RESULT_TRUE; #endif /* ST/MNT_EXPORTED */ switch (type) { @@ -2023,8 +2023,8 @@ static int check_mmap_limit(const size_t limit) { return MDBX_SUCCESS; } -MDBX_INTERNAL int osal_mmap(const int flags, osal_mmap_t *map, size_t size, const size_t limit, - const unsigned options) { +MDBX_INTERNAL int osal_mmap(const int flags, osal_mmap_t *map, size_t size, const size_t limit, const unsigned options, + const pathchar_t *pathname4logging) { assert(size <= limit); map->limit = 0; map->current = 0; @@ -2035,8 +2035,27 @@ MDBX_INTERNAL int osal_mmap(const int flags, osal_mmap_t *map, size_t size, cons #endif /* Windows */ int err = osal_check_fs_local(map->fd, flags); - if (unlikely(err != MDBX_SUCCESS)) - return err; + if (unlikely(err != MDBX_SUCCESS)) { +#if defined(_WIN32) || defined(_WIN64) + if (globals.running_under_Wine) + NOTICE("%s", "Please use native Linux application or WSL at least, instead of trouble-full Wine!"); +#endif /* Windows */ + switch (err) { + case MDBX_RESULT_TRUE: +#if MDBX_ENABLE_NON_READONLY_EXPORT + WARNING("%" MDBX_PRIsPATH " is exported via NFS, avoid using the file on a remote side!", pathname4logging); + break; +#else + ERROR("%" MDBX_PRIsPATH " is exported via NFS", pathname4logging); + return MDBX_EREMOTE; +#endif /* MDBX_PROHIBIT_NON_READONLY_EXPORT */ + case MDBX_EREMOTE: + ERROR("%" MDBX_PRIsPATH " is on a remote file system, the %s is required", pathname4logging, "MDBX_EXCLUSIVE"); + __fallthrough /* fall through */; + default: + return err; + } + } err = check_mmap_limit(limit); if (unlikely(err != MDBX_SUCCESS)) diff --git a/src/osal.h b/src/osal.h index 8f0d020c..484bbab8 100644 --- a/src/osal.h +++ b/src/osal.h @@ -464,7 +464,8 @@ MDBX_INTERNAL int osal_lockfile(mdbx_filehandle_t fd, bool wait); #define MMAP_OPTION_TRUNCATE 1 #define MMAP_OPTION_SEMAPHORE 2 -MDBX_INTERNAL int osal_mmap(const int flags, osal_mmap_t *map, size_t size, const size_t limit, const unsigned options); +MDBX_INTERNAL int osal_mmap(const int flags, osal_mmap_t *map, size_t size, const size_t limit, const unsigned options, + const pathchar_t *pathname4logging); MDBX_INTERNAL int osal_munmap(osal_mmap_t *map); #define MDBX_MRESIZE_MAY_MOVE 0x00000100 #define MDBX_MRESIZE_MAY_UNMAP 0x00000200 From 1e0a1014a4527998f72f4193b77e78592d483412 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 14:23:19 +0300 Subject: [PATCH 88/90] =?UTF-8?q?mdbx-tests:=20=D0=BA=D0=BE=D1=80=D1=80?= =?UTF-8?q?=D0=B5=D0=BA=D1=82=D0=B8=D1=80=D0=BE=D0=B2=D0=BA=D0=B0=20=D0=BE?= =?UTF-8?q?=D0=B1=D1=80=D0=B0=D0=B1=D0=BE=D1=82=D0=BA=D0=B8=20=D0=BF=D1=80?= =?UTF-8?q?=D0=B5=D1=80=D1=8B=D0=B2=D0=B0=D0=BD=D0=B8=D1=8F=20=D1=82=D0=B5?= =?UTF-8?q?=D1=81=D1=82=D0=B0=20=D0=BF=D0=BE=D1=81=D1=80=D0=B5=D0=B4=D1=81?= =?UTF-8?q?=D1=82=D0=B2=D0=BE=D0=BC=20`SIGTERM`/`SIGINT`=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- test/osal-unix.c++ | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/osal-unix.c++ b/test/osal-unix.c++ index f440aeca..b7233d03 100644 --- a/test/osal-unix.c++ +++ b/test/osal-unix.c++ @@ -562,7 +562,7 @@ int osal_actor_poll(mdbx_pid_t &pid, unsigned timeout) { continue; } - if (pid == 0) + if (pid == 0 || sigbreak) break; if (err != EINTR) From 5d9fb63fb8f512915e2000929c594c9abc86b896 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 18:00:44 +0300 Subject: [PATCH 89/90] =?UTF-8?q?mdbx:=20=D1=83=D1=81=D0=B8=D0=BB=D0=B5?= =?UTF-8?q?=D0=BD=D0=B8=D0=B5=20=D0=BA=D0=BE=D0=BD=D1=82=D1=80=D0=BE=D0=BB?= =?UTF-8?q?=D1=8F=20=D1=81=D0=B8=D0=B3=D0=BD=D0=B0=D1=82=D1=83=D1=80=20?= =?UTF-8?q?=D0=BA=D1=83=D1=80=D1=81=D0=BE=D1=80=D0=BE=D0=B2=20(backport).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/cursor.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/src/cursor.c b/src/cursor.c index be82940f..8b6541dd 100644 --- a/src/cursor.c +++ b/src/cursor.c @@ -189,9 +189,11 @@ int cursor_shadow(MDBX_cursor *mc, MDBX_txn *nested, const size_t dbi) { const size_t size = mc->subcur ? sizeof(MDBX_cursor) + sizeof(subcur_t) : sizeof(MDBX_cursor); for (MDBX_cursor *bk; mc; mc = bk->next) { cASSERT(mc, mc != mc->next); - bk = mc; - if (mc->signature != cur_signature_live) + if (mc->signature != cur_signature_live) { + ENSURE(nested->env, mc->signature == cur_signature_wait4eot); + bk = mc; continue; + } bk = osal_malloc(size); if (unlikely(!bk)) return MDBX_ENOMEM; @@ -228,7 +230,7 @@ MDBX_cursor *cursor_eot(MDBX_cursor *mc, MDBX_txn *txn, const bool merge) { tASSERT(txn, bk->txn == txn->parent); /* Zap: Using uninitialized memory '*mc->backup'. */ MDBX_SUPPRESS_GOOFY_MSVC_ANALYZER(6001); - ENSURE(mc->txn->env, bk->signature == cur_signature_live); + ENSURE(txn->env, bk->signature == cur_signature_live); tASSERT(txn, mx == bk->subcur); if (merge) { /* Update pointers to parent txn */ From 999f8644f67df16cf9314633ecb54124afdad74a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?= =?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= Date: Thu, 20 Mar 2025 18:02:00 +0300 Subject: [PATCH 90/90] =?UTF-8?q?mdbx:=20=D0=B4=D0=BE=D0=BF=D0=BE=D0=BB?= =?UTF-8?q?=D0=BD=D0=B5=D0=BD=D0=B8=D0=B5=20ChangeLog.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ChangeLog.md | 90 ++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 88 insertions(+), 2 deletions(-) diff --git a/ChangeLog.md b/ChangeLog.md index 3d4634e3..a8e7c46f 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -10,9 +10,33 @@ and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/libmdbx Благодарности: - - [Erigon](https://docs.erigon.tech/) за спонсорство. + - [Erigon](https://erigon.tech/) за спонсорство. - [Илье Михееву](https://t.me/IlyaMkhv) за сообщения о недочетах и тестирование. + - [Alex Sharov](https://github.com/AskAlexSharov) за сообщение об ошибках и тестирование. - [maxc0d3r](https://gitflic.ru/user/maxc0d3r) for bug reporting and testing. + - [Alain Picard](https://github.com/castortech) for support [Java bindings](https://github.com/castortech/mdbxjni) and MacOS universal binaries patch for CMake build scenario, + also for bug reporting (put-`MDBX_MULTIPLE` regression). Big thank for assistance with debugging and testing. + + +Новое: + + - Добавлена опция сборки `MDBX_ENABLE_NON_READONLY_EXPORT` позволяющая использовать в режиме чтения-записи БД расположенных в файловых системах экспортированных через NFS. + По-умолчанию опция выключена и при открытии в неэксклюзивном режиме чтения-записи БД расположенных файловых системах доступных извне по NFS будет возвращаться ошибка `MDBX_EREMOTE`. + Включение опции позволяет открывать БД в описанных выше ситуациях, но риск чтения неверных данных на удалённой стороне ложится на пользователя. + + - Поддержка MacOS universal binaries при сборке посредством CMake. + + - Для закрытия или отсоединения всех курсоров с получением их количества в API добавлена функция `mdbx_txn_release_all_cursors_ex()`. + + - Добавлена операция `MDBX_SEEK_AND_GET_MULTIPLE` в API курсора, позволяющая за одну операцию выполнить позиционирование + курсора на конкретное значение и начать чтение multi-значений в пакетном режиме. + + - Добавлены методы `mdbx::cursor::put_multiple_samelength()`, `mdbx::cursor::seek_multiple_samelength()`, `mdbx::cursor_managed::withdraw_handle()`. + + - В политику управления выделением для `mdbx::buffer` добавлен параметр `inplace_storage_size_rounding`. + Одновременно с этим переработан внутренний union-тип `mdbx::buffer::silo::bin` для возможности увеличения без пенальти встроенного в экземпляр буфера места под данные. + + - В API добавлена функция `mdbx_cursor_close2()` возвращающая код ошибки. Исправления: @@ -39,13 +63,75 @@ and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/libmdbx - Устранена причина попыток рекурсивного захвата мьютекса при работе `mdbx_chk -w` в сборах с поддержкой Valring/ASAN и под управлением этих инструментов. - - Проверка владельца потока владеющего транзакцией только при `MDBX_TXN_CHECKOWNER=ON`. + - Устранены проверки потока владеющего транзакцией при сборке с опцией `MDBX_TXN_CHECKOWNER=OFF`. + + - Устранена вероятность ситуации гонки в `tbl_setup(MDBX_DUPFIXED | MDBX_INTEGERDUP)` при работе в разных потоках. + В реальных сценариях вероятность проявления проблемы была близка к нулю. + Для подробностей смотрите комментарий коммита `3e91500fac475947f5b58268d5edd3c9cc4f77f6`. + + - Устранён регресс затенения курсоров во вложенных транзакциях. + При реализации отложенной/ленивой инициализации dbi-дескрипторов также было реализовано отложенное затенение курсоров (создание копии состояния для отката при прерывании транзакции), + что существенно уменьшало накладные расходы при старте и завершении вложенных транзакций в сценариях с большим количеством курсоров. + Однако, была допущена логическая ошибка, вследствие которой отложенная инициализация и затенение выполнялись при использовании dbi-дескрипторов, но не курсора открытого в родительской транзакции. + В результате, родительские курсоры во вложенных транзакциях могли не затеняться, что приводило к неконсистентному состоянию в случае + прерывания/откате вложенной транзакции и в соответствующей таблицы были изменения в рамках прерванной вложенной транзакции. + Проблема не реализовывалась в тестовых сценариях и не была замечена при эксплуатации, но была обнаружена при расширении тестов. + Ошибка присутствует в версиях 0.13.x и последующих, начиная с коммита `e6af7d7c53428ca2892bcbf7eec1c2acee06fd44` от 2023-11-05. + + - Устранён регресс в пути обработки операции `MDBX_MULTIPLE`. + Пакетная вставка значений посредством `MDBX_MULTIPLE` могла приводить к падениям и повреждению структуры БД. Ошибка оставалось не + замеченной из-за специфических условий проявления, которые не реализовались в тестах. + Проблема присутствовала во всех выпусках начиная с v0.13.1, но соответствующая ошибка не связана с конкретным коммита в истории, а + является следствием нескольких доработок (шагов рефакторинга), которые суммарно привели к регрессу. + Технически ошибка обусловлена не-обнулением переменной, чего не происходило в некотором пути выполнения, так как исходно не требовалось. + Однако, такое обнуление потребовалось после ряда этапов оптимизации и рефакторинга смежных участков кода. + Для подробностей смотрите комментарий коммита `23a417fe19614481c6546845995d6dc845baf797`. + + - Скорректировано описание ошибки `MDBX_MVCC_RETARDED` и текста соответствующего сообщения. + + - В C++ API добавлена упущенная проверка `__cpp_concepts >= 202002` для использования концептов C++. Изменение поведения: - Функция `mdbx_txn_release_all_cursors()` возвращает только код ошибки, не смешивая его с количеством обработанных/закрытых курсоров. Для аналогичных действий с получением количества закрытых курсоров в API добавлена функция `mdbx_txn_release_all_cursors_ex()`. + - Использование системного кода ошибки `EREMOTEIO` ("Remote I/O error") вместо `ENOTBLK` ("Block device required") в качестве `MDBX_EREMOTE` для индикации ошибочной ситуации открытия БД расположенной на сетевом носителе. + + - Для основных вариантов использования шаблона `mdbx::buffer<>` теперь явно инстанцируются внутри библиотеки, + одновременно соответствующие специализации шаблона помечены как `external` для предотвращения повторного инстанцирования в пользовательском коде. + + - Запрещена отвязка/открепление курсоров во вложенных транзакциях, т.е. вызовы `mdbx_cursor_unbind()` и + `mdbx_txn_release_all_cursors(unbind=true)` для курсоров открытых в одной из родительских транзакций. + Причина в том, что в случае отмены вложенной транзакции возникает неконструктивная неопределенность + — следует ли восстанавливать состояние курсоров. Если не восстанавливать, то получается что вложенная транзакция может + поломать родительскую, сделав её продолжение невозможным. Если восстанавливать, то также следует «воскрешать» закрытые + курсоры, что неизбежно приведет к путанице, утечкам памяти и использованию после освобождения. + + - В C++ API отменён вброс исключения при запросе транзакции у отсоединённого курсора посредством вывоза `mdbx::cursor::txn()`. + +Прочие доработки: + + - Доработка использования LTO в CMake-сценариях: использование `-flto=auto` для GCC >= 11.4, + расслабление условий для включения LTO для CLANG на Linux, расширение поиска `LLVMgold.so` в относительных lib-директориях. + + - Добавлены дополнительные проверки сигнатур курсоров при итерации связанных списков. + + - Кратное сокращение итераций тестов в зависимости от конфигурации Valgrind/Debug/CI. + + - Устранены предупреждения UBASN о невыравненном доступе в тесте extra/close-dbi. + + - Добавлен перехват и логирование исключений в extra-тестах на C++. + + - Расширены тесты extra/dupfix-multiple, extra/cursor-closing и extra/txn. + + - В утилиту тестирования добавлена поддержка режима/опции `MDBX_VALIDATION` и поддержка значений `on`/`off` для опций командной строки. + + - Добавлены doxygen-описания для doubtless-positioning констант. + + - Переработана проверка курсоров на входе в API-функций с добавлением `cursor_check()`, `cursor_reset()` и `cursor_drown()`. + + - Отключено использование C23 `[[атрибутов]]` для версий CLANG меньше 20. --------------------------------------------------------------------------------