mdbx: merge branch master into stable.

This commit is contained in:
Леонид Юрьев (Leonid Yuriev) 2024-03-14 19:48:12 +03:00
commit 8cc3dba7ae
No known key found for this signature in database
GPG Key ID: 518BD10B927E8686
58 changed files with 760 additions and 341 deletions

View File

@ -1,5 +1,5 @@
##
## Copyright 2020-2023 Leonid Yuriev <leo@yuriev.ru>
## Copyright 2020-2024 Leonid Yuriev <leo@yuriev.ru>
## and other libmdbx authors: please see AUTHORS file.
## All rights reserved.
##

View File

@ -1,4 +1,4 @@
Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>.
Copyright 2015-2024 Leonid Yuriev <leo@yuriev.ru>.
Copyright 2011-2015 Howard Chu, Symas Corp.
Copyright 2015,2016 Peter-Service R&D LLC.
All rights reserved.

View File

@ -4,6 +4,86 @@ ChangeLog
English version [by Google](https://gitflic-ru.translate.goog/project/erthink/libmdbx/blob?file=ChangeLog.md&_x_tr_sl=ru&_x_tr_tl=en)
and [by Yandex](https://translated.turbopages.org/proxy_u/ru-en.en/https/gitflic.ru/project/erthink/libmdbx/blob?file=ChangeLog.md).
## v0.12.10 "СЭМ" от 2024-03-12
Поддерживающий выпуск с исправлением обнаруженных ошибок и устранением недочетов
в память Героя России гвардии майора Дмитрия Семёнова с позывным "СЭМ".
```
git diff' stat: 19 commits, 57 files changed, 751 insertions(+), 331 deletions(-)
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
```
Благодарности:
- [Dvir H](https://t.me/Dvirsw) за [сообщение](https://t.me/libmdbx/5368)
об ошибке `MDBX_CORRUPTED` в сценарии работы в режиме `MDBX_DUPFIXED` и нечетной длинной
мульти-значений, с предоставлением точного минимального сценария воспроизведения.
Значимые исправления и доработки:
- Устранение унаследованной от LMDB ошибки приводящей к повреждению БД при использовании `MDBX_DUPFIXED`.
Проблема была выявлена при расширении тестов сценариями с длинными мульти-значениями в режиме `MDBX_DUPFIXED`.
Сообщений о проявлении этой ошибки в эксплуатационных/продуктовых средах не поступало.
На LEAF2-страниц, используемых для компактного хранения мульти-значений фиксированной длины,
выполнялось резервирование места без учета возможности превышения размера страниц,
с последующим переполнением, повреждением структуры БД и/или повреждением содержимого ОЗУ.
Вероятность проявления ошибки существенно увеличивалась с увеличением размера/длины
мульти-значений/дубликатов и уменьшением размера страницы БД. Поэтому при использовании `MDBX_INTEGERDUP`
вероятность проявления близка к нулю, а сценарий такого проявления найти не удалось.
В MDBX ошибка присутствовала с момента отделения проекта от LMDB,
где эта ошибка присутствует более 11 лети, по настоящее время.
- Исправление ложной ошибки `MDBX_CORRUPTED (-30796)` в сценарии работы
в режиме `MDBX_DUPFIXED` и нечетной длинной мульти-значений.
- Исправление недочета корректировки сопутствующих курсоров при разделении страницы
по сценарию добавления пустой страницы слева.
- Доработка `rebalance()` ради уменьшения WAF. Новый функционал, включая
контролируемую пользователем опцию `enum MDBX_option_t`, будет доступен
в выпусках ветки `0.13.x`, а в этом выпуске доработка сводится к тактике
не-вовленичения чистой страницы при нехватке запаса страниц в ходе обновления GC,
за счет ухудшения баланса дерева страниц.
- Устранение упущения приводящего к нелогичной ситуации
`me_dxb_mmap.curren > me_dxb_mmap.limit` при "дребезге" размера БД.
В текущем понимании, последствий кроме срабатывания assert-проверки нет, а
вероятность проявления близка к нулю.
- Исправление в функционале обхода дерева, используемого утилитой
`mdbx_chk`, подсчета места затраченного на выравнивание на вложенной
под-странице, в случае нечетного количества dupfixed-элементов нечетного
размера.
Сообщений о проявлении этой ошибки в эксплуатационных/продуктовых средах не поступало.
- Исправление assert-проверки внутри `check_txn()` для случая завершенных транзакций в режиме `MDBX_NO_TLS`.
Последствий ошибки, кроме срабатывания assert-проверки в отладочных сборках, нет.
- Устранение ошибки при открытии БД на файловой системе только-для-чтения.
- Удалены излишне строгие проверки в утилите `mdbx_chk`, которые
приводили к ложно-позитивным ошибкам при проверке БД после серии
последних доработок. Подробности см в комментариях к коммитам [781b3f64d52b73cbaeb00a55811d1247c25624a8](https://gitflic.ru/project/erthink/libmdbx/commit/781b3f64d52b73cbaeb00a55811d1247c25624a8)
и [0741c81cfd8dc0864fcf55e04192b2207c8f68f7](https://gitflic.ru/project/erthink/libmdbx/commit/0741c81cfd8dc0864fcf55e04192b2207c8f68f7).
Прочее:
- Расширение стохастического теста dupfixed-сценариями.
- Корректировка условия в assert-проверке для `MDBX_TXN_DRAINED_GC`.
- Добавление в jitter-сценарий простого теста сброса и перезапуска читающих транзакций.
- Вынужденное продолжение очистки/рефакторинга унаследованных ребусов в `cursor_put_nochecklen()`.
- Фиксация транзакции при ошибках теста для последующего анализа содержимого БД.
- Сопутствующий рефакторинг `node_shrink()` для ясности исходного кода.
- Приведение в соответствие протоколируемых имен тестов опциям командной строки.
- Добавление cmoke-теста `extra/dupfixed_addodd`.
--------------------------------------------------------------------------------
## v0.12.9 "Ясень-4" от 2023-12-11

View File

@ -1,4 +1,4 @@
## Copyright (c) 2012-2023 Leonid Yuriev <leo@yuriev.ru>.
## Copyright (c) 2012-2024 Leonid Yuriev <leo@yuriev.ru>.
##
## Licensed under the Apache License, Version 2.0 (the "License");
## you may not use this file except in compliance with the License.

View File

@ -1,4 +1,4 @@
## Copyright (c) 2012-2023 Leonid Yuriev <leo@yuriev.ru>.
## Copyright (c) 2012-2024 Leonid Yuriev <leo@yuriev.ru>.
##
## Licensed under the Apache License, Version 2.0 (the "License");
## you may not use this file except in compliance with the License.

View File

@ -1,4 +1,4 @@
## Copyright (c) 2012-2023 Leonid Yuriev <leo@yuriev.ru>.
## Copyright (c) 2012-2024 Leonid Yuriev <leo@yuriev.ru>.
##
## Licensed under the Apache License, Version 2.0 (the "License");
## you may not use this file except in compliance with the License.

2
mdbx.h
View File

@ -25,7 +25,7 @@ _The Future will (be) [Positive](https://www.ptsecurity.com). Всё будет
\section copyright LICENSE & COPYRIGHT
\authors Copyright (c) 2015-2023, Leonid Yuriev <leo@yuriev.ru>
\authors Copyright (c) 2015-2024, Leonid Yuriev <leo@yuriev.ru>
and other _libmdbx_ authors: please see [AUTHORS](./AUTHORS) file.
\copyright Redistribution and use in source and binary forms, with or without

View File

@ -1,7 +1,7 @@
/// \file mdbx.h++
/// \brief The libmdbx C++ API header file.
///
/// \author Copyright (c) 2020-2023, Leonid Yuriev <leo@yuriev.ru>.
/// \author Copyright (c) 2020-2024, Leonid Yuriev <leo@yuriev.ru>.
/// \copyright SPDX-License-Identifier: Apache-2.0
///
/// Tested with:

View File

@ -1,4 +1,4 @@
From 3efdf07a80f750c23de126ac80e78fb0545a1b63 Mon Sep 17 00:00:00 2001
From 22732cf074188912caea6da6795c36098efb7eb5 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4=20=D0=AE=D1=80=D1=8C?=
=?UTF-8?q?=D0=B5=D0=B2=20=28Leonid=20Yuriev=29?= <leo@yuriev.ru>
Date: Mon, 23 Oct 2023 18:07:13 +0300
@ -13,11 +13,10 @@ This patch adds libmdbx:
focused on creating unique lightweight solutions.
- libmdbx surpasses the legendary LMDB (Lightning Memory-Mapped Database)
in terms of reliability, features and performance.
- https://gitflic.ru/project/erthink/libmdbx
- more information at https://gitflic.ru/project/erthink/libmdbx
The v0.12.8 "Vladimir Utkin" is stable release of frontward _libmdbx_
branch with new superior features on the day of 100 anniversary of the birth
of the outstanding Soviet and Russian scientist and engineer Vladimir Fedorovich Utkin.
The v0.12.10 "SEM" is stable release of _libmdbx_ branch with new superior features,
in memory of the Hero of Russia Guard Major Dmitry Semenov with the call sign "SEM".
The complete ChangeLog: https://gitflic.ru/project/erthink/libmdbx/blob?file=ChangeLog.md
@ -112,18 +111,18 @@ index 0000000000..a9a4ac45c5
+ !BR2_TOOLCHAIN_GCC_AT_LEAST_4_4
diff --git a/package/libmdbx/libmdbx.hash b/package/libmdbx/libmdbx.hash
new file mode 100644
index 0000000000..82cf28d6a9
index 0000000000..1dbdef7c69
--- /dev/null
+++ b/package/libmdbx/libmdbx.hash
@@ -0,0 +1,5 @@
+# Hashes from: https://libmdbx.dqdkfa.ru/release/SHA256SUMS
+sha256 c78c56c53708bbfc519bf53ebf520d1f09d30ee6427a4bedf713316696e671d0 libmdbx-amalgamated-0.12.8.tar.xz
+sha256 4637e06768a9a8fc7577e6e458e045ad1f9a8baee74996a3c88bc0ad64cbcf67 libmdbx-amalgamated-0.12.10.tar.xz
+
+# Locally calculated
+sha256 310fe25c858a9515fc8c8d7d1f24a67c9496f84a91e0a0e41ea9975b1371e569 LICENSE
diff --git a/package/libmdbx/libmdbx.mk b/package/libmdbx/libmdbx.mk
new file mode 100644
index 0000000000..d198fe5b22
index 0000000000..36df45b64f
--- /dev/null
+++ b/package/libmdbx/libmdbx.mk
@@ -0,0 +1,42 @@
@ -133,7 +132,7 @@ index 0000000000..d198fe5b22
+#
+################################################################################
+
+LIBMDBX_VERSION = 0.12.8
+LIBMDBX_VERSION = 0.12.10
+LIBMDBX_SOURCE = libmdbx-amalgamated-$(LIBMDBX_VERSION).tar.xz
+LIBMDBX_SITE = https://libmdbx.dqdkfa.ru/release
+LIBMDBX_SUPPORTS_IN_SOURCE_BUILD = NO
@ -170,5 +169,5 @@ index 0000000000..d198fe5b22
+
+$(eval $(cmake-package))
--
2.42.0
2.43.2

View File

@ -1,5 +1,5 @@
/*
* Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

File diff suppressed because it is too large Load Diff

View File

@ -1,5 +1,5 @@
/*
* Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@ -1376,8 +1376,12 @@ struct MDBX_env {
struct MDBX_lockinfo *me_lck;
unsigned me_psize; /* DB page size, initialized from me_os_psize */
unsigned me_leaf_nodemax; /* max size of a leaf-node */
unsigned me_branch_nodemax; /* max size of a branch-node */
uint16_t me_leaf_nodemax; /* max size of a leaf-node */
uint16_t me_branch_nodemax; /* max size of a branch-node */
uint16_t me_subpage_limit;
uint16_t me_subpage_room_threshold;
uint16_t me_subpage_reserve_prereq;
uint16_t me_subpage_reserve_limit;
atomic_pgno_t me_mlocked_pgno;
uint8_t me_psize2log; /* log2 of DB page size */
int8_t me_stuck_meta; /* recovery-only: target meta page or less that zero */

View File

@ -1,5 +1,5 @@
/*
* Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,6 +1,6 @@
.\" Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2015-2024 Leonid Yuriev <leo@yuriev.ru>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_CHK 1 "2023-10-17" "MDBX 0.12.8"
.TH MDBX_CHK 1 "2024-03-13" "MDBX 0.12.10"
.SH NAME
mdbx_chk \- MDBX checking tool
.SH SYNOPSIS

View File

@ -1,8 +1,8 @@
.\" Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2015-2024 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_COPY 1 "2023-10-17" "MDBX 0.12.8"
.TH MDBX_COPY 1 "2024-03-13" "MDBX 0.12.10"
.SH NAME
mdbx_copy \- MDBX environment copy tool
.SH SYNOPSIS

View File

@ -1,7 +1,7 @@
.\" Copyright 2021-2023 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2021-2024 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2014-2021 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_DROP 1 "2023-10-17" "MDBX 0.12.8"
.TH MDBX_DROP 1 "2024-03-13" "MDBX 0.12.10"
.SH NAME
mdbx_drop \- MDBX database delete tool
.SH SYNOPSIS

View File

@ -1,8 +1,8 @@
.\" Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2015-2024 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_DUMP 1 "2023-10-17" "MDBX 0.12.8"
.TH MDBX_DUMP 1 "2024-03-13" "MDBX 0.12.10"
.SH NAME
mdbx_dump \- MDBX environment export tool
.SH SYNOPSIS

View File

@ -1,8 +1,8 @@
.\" Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2015-2024 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_LOAD 1 "2023-10-17" "MDBX 0.12.8"
.TH MDBX_LOAD 1 "2024-03-13" "MDBX 0.12.10"
.SH NAME
mdbx_load \- MDBX environment import tool
.SH SYNOPSIS

View File

@ -1,8 +1,8 @@
.\" Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2015-2024 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.
.TH MDBX_STAT 1 "2023-10-17" "MDBX 0.12.8"
.TH MDBX_STAT 1 "2024-03-13" "MDBX 0.12.10"
.SH NAME
mdbx_stat \- MDBX environment status tool
.SH SYNOPSIS

View File

@ -1,5 +1,5 @@
//
// Copyright (c) 2020-2023, Leonid Yuriev <leo@yuriev.ru>.
// Copyright (c) 2020-2024, Leonid Yuriev <leo@yuriev.ru>.
// SPDX-License-Identifier: Apache-2.0
//
// Non-inline part of the libmdbx C++ API

View File

@ -1,7 +1,7 @@
/* mdbx_chk.c - memory-mapped database check tool */
/*
* Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@ -537,26 +537,14 @@ static int pgvisitor(const uint64_t pgno, const unsigned pgnumber,
data_tree_problems += !is_gc_tree;
gc_tree_problems += is_gc_tree;
}
if (payload_bytes < 1) {
if (nentries > 1) {
problem_add("page", pgno, "zero size-of-entry",
"%s-page: payload %" PRIuPTR " bytes, %" PRIuPTR " entries",
pagetype_caption, payload_bytes, nentries);
/* if ((size_t)header_bytes + unused_bytes < page_size) {
// LY: hush a misuse error
page_bytes = page_size;
} */
data_tree_problems += !is_gc_tree;
gc_tree_problems += is_gc_tree;
} else {
problem_add("page", pgno, "empty",
"%s-page: payload %" PRIuPTR " bytes, %" PRIuPTR
" entries, deep %i",
pagetype_caption, payload_bytes, nentries, deep);
dbi->pages.empty += 1;
data_tree_problems += !is_gc_tree;
gc_tree_problems += is_gc_tree;
}
if (nentries < 1 || (pagetype == MDBX_page_branch && nentries < 2)) {
problem_add("page", pgno, nentries ? "half-empty" : "empty",
"%s-page: payload %" PRIuPTR " bytes, %" PRIuPTR
" entries, deep %i",
pagetype_caption, payload_bytes, nentries, deep);
dbi->pages.empty += 1;
data_tree_problems += !is_gc_tree;
gc_tree_problems += is_gc_tree;
}
if (pgnumber) {
@ -611,7 +599,7 @@ static int handle_freedb(const uint64_t record_number, const MDBX_val *key,
problem_add("entry", txnid, "wrong idl size", "%" PRIuPTR,
data->iov_len);
size_t number = (data->iov_len >= sizeof(pgno_t)) ? *iptr++ : 0;
if (number < 1 || number > MDBX_PGL_LIMIT)
if (number > MDBX_PGL_LIMIT)
problem_add("entry", txnid, "wrong idl length", "%" PRIuPTR, number);
else if ((number + 1) * sizeof(pgno_t) > data->iov_len) {
problem_add("entry", txnid, "trimmed idl",

View File

@ -1,7 +1,7 @@
/* mdbx_copy.c - memory-mapped database backup tool */
/*
* Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,7 +1,7 @@
/* mdbx_drop.c - memory-mapped database delete tool */
/*
* Copyright 2021-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2021-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
*
* Copyright 2016-2021 Howard Chu, Symas Corp.

View File

@ -1,7 +1,7 @@
/* mdbx_dump.c - memory-mapped database dump tool */
/*
* Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,7 +1,7 @@
/* mdbx_load.c - memory-mapped database load tool */
/*
* Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,7 +1,7 @@
/* mdbx_stat.c - memory-mapped database status tool */
/*
* Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,7 +1,7 @@
/* https://en.wikipedia.org/wiki/Operating_system_abstraction_layer */
/*
* Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@ -1826,8 +1826,8 @@ MDBX_INTERNAL_FUNC int osal_check_fs_rdonly(mdbx_filehandle_t handle,
#else
struct statvfs info;
if (err != MDBX_ENOFILE) {
if (statvfs(pathname, &info) == 0 && (info.f_flag & ST_RDONLY) == 0)
return err;
if (statvfs(pathname, &info) == 0)
return (info.f_flag & ST_RDONLY) ? MDBX_SUCCESS : err;
if (errno != MDBX_ENOFILE)
return errno;
}
@ -2571,7 +2571,7 @@ retry_mapview:;
ptr_disp(map->base, size),
((map->current < map->limit) ? map->current : map->limit) - size);
}
map->current = size;
map->current = (size < map->limit) ? size : map->limit;
}
if (limit == map->limit)
@ -2732,6 +2732,7 @@ retry_mapview:;
map->base = ptr;
}
map->limit = limit;
map->current = size;
#if MDBX_ENABLE_MADVISE
#ifdef MADV_DONTFORK

View File

@ -1,7 +1,7 @@
/* https://en.wikipedia.org/wiki/Operating_system_abstraction_layer */
/*
* Copyright 2015-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -72,6 +72,10 @@ if(UNIX AND NOT SUBPROJECT)
target_include_directories(test_extra_upsert_alldups PRIVATE "${PROJECT_SOURCE_DIR}")
target_link_libraries(test_extra_upsert_alldups ${TOOL_MDBX_LIB})
add_executable(test_extra_dupfixed_addodd extra/dupfixed_addodd.c)
target_include_directories(test_extra_dupfixed_addodd PRIVATE "${PROJECT_SOURCE_DIR}")
target_link_libraries(test_extra_dupfixed_addodd ${TOOL_MDBX_LIB})
if(MDBX_BUILD_CXX)
add_executable(test_extra_maindb_ordinal extra/maindb_ordinal.c++)
target_include_directories(test_extra_maindb_ordinal PRIVATE "${PROJECT_SOURCE_DIR}")
@ -168,6 +172,7 @@ else()
if(UNIX AND NOT SUBPROJECT)
add_test(NAME extra_upsert_alldups COMMAND test_extra_upsert_alldups)
add_test(NAME extra_dupfixed_addodd COMMAND test_extra_dupfixed_addodd)
if(MDBX_BUILD_CXX)
add_test(NAME extra_maindb_ordinal COMMAND test_extra_maindb_ordinal)
add_test(NAME extra_dupfixed_multiple COMMAND test_extra_dupfixed_multiple)

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -0,0 +1,93 @@
/*
* @Dvirsw (https://t.me/Dvirsw)
* I think there is a bug with DUPFIXED. The following code fails.
*
* https://t.me/libmdbx/5368
*/
#include <sys/stat.h>
#include <sys/time.h>
#include "mdbx.h"
#include <assert.h>
#include <inttypes.h>
#include <limits.h>
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main() {
int rc;
MDBX_env *env = NULL;
MDBX_dbi dbi = 0;
MDBX_val key, data;
MDBX_txn *txn = NULL;
rc = mdbx_env_create(&env);
if (rc != MDBX_SUCCESS) {
fprintf(stderr, "mdbx_env_create: (%d) %s\n", rc, mdbx_strerror(rc));
exit(EXIT_FAILURE);
}
rc = mdbx_env_set_maxdbs(env, 1);
if (rc != MDBX_SUCCESS) {
fprintf(stderr, "mdbx_env_create: (%d) %s\n", rc, mdbx_strerror(rc));
exit(EXIT_FAILURE);
}
rc = mdbx_env_open(env, "./example-db",
MDBX_NOSUBDIR | MDBX_COALESCE | MDBX_LIFORECLAIM, 0664);
if (rc != MDBX_SUCCESS) {
fprintf(stderr, "mdbx_env_open: (%d) %s\n", rc, mdbx_strerror(rc));
exit(EXIT_FAILURE);
}
rc = mdbx_txn_begin(env, NULL, 0, &txn);
if (rc != MDBX_SUCCESS) {
fprintf(stderr, "mdbx_txn_begin: (%d) %s\n", rc, mdbx_strerror(rc));
exit(EXIT_FAILURE);
}
rc = mdbx_dbi_open(txn, "test", MDBX_DUPSORT | MDBX_DUPFIXED | MDBX_CREATE,
&dbi);
if (rc != MDBX_SUCCESS) {
fprintf(stderr, "mdbx_dbi_open: (%d) %s\n", rc, mdbx_strerror(rc));
exit(EXIT_FAILURE);
}
char key_bytes[32] = {0};
key.iov_len = 32;
key.iov_base = key_bytes;
// Another put after this will fail.
unsigned char idx;
for (idx = 0; idx < 129; idx++) {
char data_bytes[15] = {idx};
data.iov_len = 15;
data.iov_base = data_bytes;
rc = mdbx_put(txn, dbi, &key, &data, 0);
if (rc != MDBX_SUCCESS) {
fprintf(stderr, "mdbx_put: (%d) %s\n", rc, mdbx_strerror(rc));
exit(EXIT_FAILURE);
}
}
// This will fail and exit.
char data_bytes[15] = {idx};
data.iov_len = 15;
data.iov_base = data_bytes;
rc = mdbx_put(txn, dbi, &key, &data, 0);
if (rc != MDBX_SUCCESS) {
fprintf(stderr, "mdbx_put: (%d) %s\n", rc, mdbx_strerror(rc));
fprintf(stderr, "expected failure\n");
exit(EXIT_FAILURE);
}
rc = mdbx_txn_commit(txn);
if (rc) {
fprintf(stderr, "mdbx_txn_commit: (%d) %s\n", rc, mdbx_strerror(rc));
exit(EXIT_FAILURE);
}
}

View File

@ -1,5 +1,5 @@
/*
* Copyright 2016-2023 Leonid Yuriev <leo@yuriev.ru>.
* Copyright 2016-2024 Leonid Yuriev <leo@yuriev.ru>.
* Copyright 2015 Vladimir Romanov
* <https://www.linkedin.com/in/vladimirromanov>, Yota Lab.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@ -99,6 +99,37 @@ bool testcase_jitter::run() {
jitter_delay();
txn_begin(true);
fetch_canary();
if (flipcoin()) {
MDBX_txn_info info;
err = mdbx_txn_reset(txn_guard.get());
if (err)
failure_perror("mdbx_txn_reset()", err);
err = mdbx_txn_info(txn_guard.get(), &info, false);
if (err != MDBX_BAD_TXN)
failure_perror("mdbx_txn_info(MDBX_BAD_TXN)", err);
err = mdbx_txn_reset(txn_guard.get());
if (err)
failure_perror("mdbx_txn_reset(again)", err);
err = mdbx_txn_break(txn_guard.get());
if (err)
failure_perror("mdbx_txn_break()", err);
err = mdbx_txn_abort(txn_guard.get());
if (err)
failure_perror("mdbx_txn_abort()", err);
txn_guard.release();
txn_begin(true);
err = mdbx_txn_reset(txn_guard.get());
if (err)
failure_perror("mdbx_txn_reset()", err);
err = mdbx_txn_renew(txn_guard.get());
if (err)
failure_perror("mdbx_txn_renew()", err);
err = mdbx_txn_info(txn_guard.get(), &info, false);
if (err)
failure_perror("mdbx_txn_info()", err);
}
jitter_delay();
txn_end(flipcoin());
}

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -432,6 +432,12 @@ for nops in 10 33 100 333 1000 3333 10000 33333 100000 333333 1000000 3333333 10
--pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%4]} \
--keygen.seed=${seed}
caption="Probe #$((++count)) int-key,fixdups, split=${split}, case $((++subcase)) of ${cases}" probe \
--keygen.seed=${seed} --pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+key.integer,+data.fixed --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen=9 \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%4]}
caption="Probe #$((++count)) fixdups, split=${split}, case $((++subcase)) of ${cases}" probe \
--keygen.seed=${seed} --pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+data.fixed --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen=11 \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%4]}
split=24
caption="Probe #$((++count)) int-key,with-dups, split=${split}, case $((++subcase)) of ${cases}" probe \
@ -446,6 +452,12 @@ for nops in 10 33 100 333 1000 3333 10000 33333 100000 333333 1000000 3333333 10
--pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%4]} \
--keygen.seed=${seed}
caption="Probe #$((++count)) int-key,fixdups, split=${split}, case $((++subcase)) of ${cases}" probe \
--keygen.seed=${seed} --pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+key.integer,+data.fixed --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen=10 \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%4]}
caption="Probe #$((++count)) fixdups, split=${split}, case $((++subcase)) of ${cases}" probe \
--keygen.seed=${seed} --pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+data.fixed --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen=12 \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%4]}
split=16
caption="Probe #$((++count)) int-key,w/o-dups, split=${split}, case $((++subcase)) of ${cases}" probe \
@ -468,6 +480,12 @@ for nops in 10 33 100 333 1000 3333 10000 33333 100000 333333 1000000 3333333 10
--pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%4]} \
--keygen.seed=${seed}
caption="Probe #$((++count)) int-key,fixdups, split=${split}, case $((++subcase)) of ${cases}" probe \
--keygen.seed=${seed} --pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+key.integer,+data.fixed --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen=9 \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%4]}
caption="Probe #$((++count)) fixdups, split=${split}, case $((++subcase)) of ${cases}" probe \
--keygen.seed=${seed} --pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+data.fixed --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen=11 \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%4]}
if [ "$EXTRA" != "no" ]; then
split=10
@ -491,6 +509,12 @@ for nops in 10 33 100 333 1000 3333 10000 33333 100000 333333 1000000 3333333 10
--pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%4]} \
--keygen.seed=${seed}
caption="Probe #$((++count)) int-key,fixdups, split=${split}, case $((++subcase)) of ${cases}" probe \
--keygen.seed=${seed} --pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+key.integer,+data.fixed --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen=13 \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%4]}
caption="Probe #$((++count)) fixdups, split=${split}, case $((++subcase)) of ${cases}" probe \
--keygen.seed=${seed} --pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+data.fixed --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen=16 \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%4]}
fi
split=4
@ -506,6 +530,12 @@ for nops in 10 33 100 333 1000 3333 10000 33333 100000 333333 1000000 3333333 10
--pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=-data.dups --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=1111 \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%4]} \
--keygen.seed=${seed}
caption="Probe #$((++count)) int-key,fixdups, split=${split}, case $((++subcase)) of ${cases}" probe \
--keygen.seed=${seed} --pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+key.integer,+data.fixed --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen=21 \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%4]}
caption="Probe #$((++count)) fixdups, split=${split}, case $((++subcase)) of ${cases}" probe \
--keygen.seed=${seed} --pagesize=$PAGESIZE --size-upper-upto=${db_size_mb}M --table=+data.fixed --keygen.split=${split} --keylen.min=min --keylen.max=max --datalen=32 \
--nops=$nops --batch.write=$wbatch --mode=$(bits2options $bits)${syncmodes[count%4]}
done # options
loop=$((loop + 1))
if [ -n "$LOOPS" ] && [ $loop -ge "$LOOPS" ]; then break; fi

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@ -24,9 +24,9 @@ const char *testcase2str(const actor_testcase testcase) {
case ac_hill:
return "hill";
case ac_deadread:
return "deadread";
return "dead.reader";
case ac_deadwrite:
return "deadwrite";
return "dead.writer";
case ac_jitter:
return "jitter";
case ac_try:
@ -733,6 +733,18 @@ void testcase::speculum_check_iterator(const char *where, const char *stage,
mdbx_dump_val(&v, dump_value, sizeof(dump_value)));
}
void testcase::failure(const char *fmt, ...) const {
va_list ap;
va_start(ap, fmt);
fflush(nullptr);
logging::output_nocheckloglevel_ap(logging::failure, fmt, ap);
va_end(ap);
fflush(nullptr);
if (txn_guard)
mdbx_txn_commit(const_cast<testcase *>(this)->txn_guard.release());
exit(EXIT_FAILURE);
}
#if SPECULUM_CURSORS
void testcase::speculum_check_cursor(const char *where, const char *stage,
const testcase::SET::const_iterator &it,

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@ -277,6 +277,7 @@ protected:
void signal();
bool should_continue(bool check_timeout_only = false) const;
void failure(const char *fmt, ...) const;
void generate_pair(const keygen::serial_t serial, keygen::buffer &out_key,
keygen::buffer &out_value, keygen::serial_t data_age) {
keyvalue_maker.pair(serial, out_key, out_value, data_age, false);

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2023 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2024 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*