Compare commits

...

27 Commits

Author SHA1 Message Date
Leonid Yuriev
5c488d7033 mdbx: backport - fix pwrite() for WRITE_MAX.
Change-Id: If4924d20c1e267c2d3a190c860b89fc2fda0d517
2019-06-24 02:45:05 +03:00
Leonid Yuriev
5413407f23 mdbx-test: backport - fix dbsize-options handling.
Change-Id: Ia51f802ac1ad4e8b1b059a3f3b38214bda6b43fc
2019-06-24 02:45:05 +03:00
Leonid Yuriev
ebc8e9935e mdbx: bump version to v0.1.7
Change-Id: I0f72ed31fbd1ed74a875c2aa2023521855e72894
2019-06-22 23:31:10 +03:00
Leonid Yuriev
26838a2164 mdbx: rework RECLAIMING inside update_gc().
Change-Id: I9cf592476780bfdb346472baa12497d68a3d5aad
2019-06-22 23:31:05 +03:00
Howard Chu
828889de5c mdbx: import - tweak mdb_page_split (ITS#8969).
Bump up number of keys for which we use fine-grained splitpoint search

Change-Id: Icca2e1953cbcd6898b790f657636c2195b397790
2019-06-22 15:50:28 +03:00
Leonid Yuriev
6c160d02af mdbx: backport - fix TAGRET typo (minor).
Change-Id: Iffafbed7fdad3492aeb51f17caf8109a5b3e35c0
2019-06-22 01:48:12 +03:00
Leonid Yuriev
fead1c3853 mdbx: backport - fix handling MDBX_APPENDDUP mode.
Change-Id: I36de2a8dcab5126dab3857a7840ab3904a1d19c8
2019-06-22 01:48:12 +03:00
Leo Yuriev
06c35dd59c mdbx: backport - fix __ANDROID__ typo.
Thank to Howard Chu <hyc@openldap.org>.

Change-Id: Ibcbe2e4790a5df5758d9fd6c621793ea42a94682
2019-06-22 01:48:12 +03:00
Leonid Yuriev
46eb178f07 mdbx: backport - fix GC corruption due deep recursive rebalance from update_gc().
Change-Id: I810250deb25cd625e737000282b434e3158ef8cc
2019-06-22 01:32:47 +03:00
Leo Yuriev
b1ffe87556 mdbx: fix one more comment typo (minor). 2019-02-13 20:27:20 +03:00
Leo Yuriev
131485e516 mdbx: fix comment typo (minor). 2019-02-13 20:27:14 +03:00
Leo Yuriev
64f6648d0c mdbx: make API compatible to the master branch.
Change-Id: I95c5db639cebe4bba9c600f97c9966082bc9bc09
2019-02-03 13:17:43 +03:00
Leonid Yuriev
ba00b597a7 mdbx-windows: backport - fix returning negative value on failure.
Change-Id: Iaf5fb1f0cbcc3c14e2d3edf1f57538ecc0dfdf00
2019-02-03 13:17:43 +03:00
Leo Yuriev
4f79e3756c mdbx: 2019 HNY.
Change-Id: Iec6b7affedef0ea89fad917eb17af7e8201a7482
2019-02-03 13:17:43 +03:00
Leo Yuriev
f183cef7d7 mdbx-windows: backport - always susppend local threads while resize DB (workaround for Windows kernel bug).
We should not concern about performance on Windows platform,
it just unreasonable. Therefore just always suspend the local
threads to avoid this issue.

This resolves https://github.com/leo-yuriev/libmdbx/issues/48

Change-Id: I6e652692794b8c4c0d41625be62f2051b63c033a
2018-11-22 22:39:04 +03:00
Leonid Yuriev
850fe8408e mdbx: backport - relax DBI-sequences for concurrent open DBI-handles for present tables.
Change-Id: I7f07d2e716074bd9c2847aeb062e366f46cca214
2018-11-20 14:29:20 +03:00
Leonid Yuriev
0043f62a43 mdbx: backport - silently put mm_geo.now into [geo.lower...geo.upper] boundaries.
Copy-with-compaction by previous version of libmfbx could produce
DB-file less than meta.geo.lower bound, in case actual filling
is low or no data at all.

This is not a problem as there is no damage or loss of data.
Therefore it is better not to consider such situation as an
error, but silently correct it.

Change-Id: Ia662656cc3584c07efcfbdfc80f80e3c76e6dd59
2018-11-05 14:35:24 +03:00
Leonid Yuriev
de8d0479ab mdbx: backport - fix typo in mdbx_limits_dbsize_max().
Change-Id: Ie55e3ca108ac6aab9a41d65f316a3d5ff5ff6f1f
2018-11-05 02:16:29 +03:00
Leo Yuriev
feab109c61 mdbx-test: backport - fix osal_actor_poll() for 32-bit builds.
Change-Id: I36b2f955295d8ca5435a68737c0c2e7f069bfe34
2018-11-05 02:16:24 +03:00
Leonid Yuriev
6120c2be0a mdbx-test: backport - add checks for db-copy after the basic testcase.
Change-Id: I5e7d343266c66418a8798d272e697e1c3d5c775b
2018-11-05 02:16:19 +03:00
Leonid Yuriev
ee0c8bb249 mdbx: backport - add db-copy testcase.
Change-Id: Ib554880ebbabcb5dfc55bdb3c71767d0fa1630fd
2018-11-05 02:16:13 +03:00
Leonid Yuriev
3d59c9f9e7 mdbx: backport - take in account shrink/growing thresholds while copy-with-compactification.
Change-Id: Id93e62089819dfcc8cbc83620e0bdd806d8c1950
2018-11-05 02:16:07 +03:00
Leonid Yuriev
86e63f0b6b mdbx: backport - refine mdbx_env_copy() internals (required for next patch).
Change-Id: I9e8f0dc87398564524a5ec98eda2cb9bde100909
2018-11-05 02:16:07 +03:00
Leonid Yuriev
affd28654c mdbx: backport - fix mdbx_txn_abort().
This resolves https://github.com/leo-yuriev/libfpta/issues/20

Change-Id: I43c0c960d5c871d837b307cd370ee7327db01ff6
2018-10-22 01:45:45 +03:00
Leonid Yuriev
08130df595 mdbx-windows: backport - workaround for Windows10 bugs.
This resolves https://github.com/leo-yuriev/libmdbx/issues/47

Change-Id: I6e0d6dfbfec15b68200438b68a2996c357d46b77
2018-10-22 01:45:28 +03:00
Leonid Yuriev
5acf2b126f mdbx: backport - fix mdbx_dbi_sequence().
Change-Id: Ic620896ef42c1c2d85c07c146b72e773ab43a67d
2018-10-22 01:40:59 +03:00
Leonid Yuriev
cc84f85722 mdbx-ci: backport - disable CI for old MSVC compilers.
Change-Id: Ia1072745664d9a97d4114149a305a6399bde71aa
2018-10-22 01:37:29 +03:00
50 changed files with 546 additions and 302 deletions

View File

@@ -1,4 +1,4 @@
Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>.
Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>.
Copyright 2011-2015 Howard Chu, Symas Corp.
Copyright 2015,2016 Peter-Service R&D LLC.
All rights reserved.

View File

@@ -82,13 +82,16 @@ clean:
rm -rf $(TOOLS) mdbx_test @* *.[ao] *.[ls]o *~ tmp.db/* *.gcov *.log *.err src/*.o test/*.o
check: all
rm -f $(TESTDB) $(TESTLOG) && (set -o pipefail; ./mdbx_test --pathname=$(TESTDB) --dont-cleanup-after basic | tee -a $(TESTLOG) | tail -n 42) && ./mdbx_chk -vvn $(TESTDB)
rm -f $(TESTDB) $(TESTLOG) && (set -o pipefail; ./mdbx_test --pathname=$(TESTDB) --dont-cleanup-after basic | tee -a $(TESTLOG) | tail -n 42) \
&& ./mdbx_chk -vvn $(TESTDB) && ./mdbx_chk -vvn $(TESTDB)-copy
check-singleprocess: all
rm -f $(TESTDB) $(TESTLOG) && (set -o pipefail; ./mdbx_test --pathname=$(TESTDB) --dont-cleanup-after --hill | tee -a $(TESTLOG) | tail -n 42) && ./mdbx_chk -vvn $(TESTDB)
rm -f $(TESTDB) $(TESTLOG) && (set -o pipefail; ./mdbx_test --pathname=$(TESTDB) --dont-cleanup-after --hill | tee -a $(TESTLOG) | tail -n 42) \
&& ./mdbx_chk -vvn $(TESTDB) && ./mdbx_chk -vvn $(TESTDB)-copy
check-fault: all
rm -f $(TESTDB) $(TESTLOG) && (set -o pipefail; ./mdbx_test --pathname=$(TESTDB) --inject-writefault=42 --dump-config --dont-cleanup-after basic | tee -a $(TESTLOG) | tail -n 42) && ./mdbx_chk -vvn $(TESTDB)
rm -f $(TESTDB) $(TESTLOG) && (set -o pipefail; ./mdbx_test --pathname=$(TESTDB) --inject-writefault=42 --dump-config --dont-cleanup-after basic | tee -a $(TESTLOG) | tail -n 42) \
&& ./mdbx_chk -vvn $(TESTDB) && ./mdbx_chk -vvn $(TESTDB)-copy
define core-rule
$(patsubst %.c,%.o,$(1)): $(1) $(CORE_INC) mdbx.h Makefile

View File

@@ -12,24 +12,21 @@ and [by Yandex](https://translate.yandex.ru/translate?url=https%3A%2F%2Fgithub.c
### Project Status
**Сейчас MDBX _активно перерабатывается_** предстоит
большое изменение как API, так и формата базы данных. К сожалению,
обновление приведет к потере совместимости с предыдущими версиями.
**Сейчас MDBX _активно перерабатывается_** и к середине 2018
ожидается большое изменение как API, так и формата базы данных.
К сожалению, обновление приведет к потере совместимости с
предыдущими версиями.
Цель этой революции - обеспечение более четкого надежного API и
добавление новых функции, а также наделение базы данных новыми
свойствами.
Цель этой революции - обеспечение более четкого надежного
API и добавление новых функции, а также наделение базы данных
новыми свойствами.
В настоящее время MDBX предназначена для Linux, а также
поддерживает Windows (начиная с Windows Server 2008) в качестве
дополнительной платформы. Поддержка других ОС может быть
обеспечена на коммерческой основе. Однако такие
усовершенствования (т. е. pull-requests) могут быть приняты в
мейнстрим только в том случае, если будет доступен
соответствующий публичный и бесплатный сервис непрерывной
интеграции (aka Continuous Integration).
В настоящее время MDBX предназначена для Linux, а также поддерживает
Windows (начиная с Windows Server 2008) в качестве дополнительной
платформы. Поддержка других ОС может быть обеспечена на коммерческой
основе. Однако такие усовершенствования (т. е. pull-requests) могут быть
приняты в мейнстрим только в том случае, если будет доступен
соответствующий публичный и бесплатный сервис непрерывной интеграции
(aka Continuous Integration).
## Содержание

View File

@@ -4,10 +4,10 @@ environment:
matrix:
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
TOOLSET: v141
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015
TOOLSET: v140
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2013
TOOLSET: v120
# - APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015
# TOOLSET: v140
# - APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2013
# TOOLSET: v120
branches:
except:

View File

@@ -3,6 +3,7 @@ README-RU.md
pcrf_test/CMakeLists.txt
src/tools/CMakeLists.txt
test/CMakeLists.txt
test/copy.cc
tutorial/CMakeLists.txt
tutorial/sample-mdbx.c
AUTHORS

13
mdbx.h
View File

@@ -1,6 +1,6 @@
/* LICENSE AND COPYRUSTING *****************************************************
*
* Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@@ -60,8 +60,8 @@
/* IMPENDING CHANGES WARNING ***************************************************
*
* MDBX is under active development, database format and API aren't stable
* at least until 2018Q2. New version won't be backwards compatible. Main focus
* MDBX is under active non-public development, database format and API
* will be refined. New version won't be backwards compatible. Main focus
* of the rework is to provide clear and robust API and new features.
*
******************************************************************************/
@@ -100,6 +100,7 @@ typedef DWORD mdbx_tid_t;
#define MDBX_EIO ERROR_WRITE_FAULT
#define MDBX_EPERM ERROR_INVALID_FUNCTION
#define MDBX_EINTR ERROR_CANCELLED
#define MDBX_ENOFILE ERROR_FILE_NOT_FOUND
#else
@@ -120,6 +121,8 @@ typedef pthread_t mdbx_tid_t;
#define MDBX_EIO EIO
#define MDBX_EPERM EPERM
#define MDBX_EINTR EINTR
#define MDBX_ENOFILE ENOENT
#endif
#ifdef _MSC_VER
@@ -588,7 +591,7 @@ LIBMDBX_API int mdbx_env_create(MDBX_env **penv);
* - MDBX_NOTLS
* Don't use Thread-Local Storage. Tie reader locktable slots to
* MDBX_txn objects instead of to threads. I.e. mdbx_txn_reset() keeps
* the slot reseved for the MDBX_txn object. A thread may use parallel
* the slot reserved for the MDBX_txn object. A thread may use parallel
* read-only transactions. A read-only transaction may span threads if
* the user synchronizes its use. Applications that multiplex many
* user threads over individual OS threads need this option. Such an
@@ -1680,6 +1683,8 @@ LIBMDBX_API int mdbx_limits_pgsize_min(void);
LIBMDBX_API int mdbx_limits_pgsize_max(void);
LIBMDBX_API intptr_t mdbx_limits_dbsize_min(intptr_t pagesize);
LIBMDBX_API intptr_t mdbx_limits_dbsize_max(intptr_t pagesize);
LIBMDBX_API intptr_t mdbx_limits_keysize_max(intptr_t pagesize);
LIBMDBX_API intptr_t mdbx_limits_txnsize_max(intptr_t pagesize);
/*----------------------------------------------------------------------------*/
/* attribute support functions for Nexenta */

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@@ -20,8 +20,8 @@
#ifndef MDBX_USE_ROBUST
/* Howard Chu: Android currently lacks Robust Mutex support */
#if defined(EOWNERDEAD) && \
!defined(ANDROID) /* LY: glibc before 2.10 has a troubles with Robust \
Mutex too. */ \
!defined(__ANDROID__) /* LY: glibc before 2.10 has a troubles \
with Robust Mutex too. */ \
&& __GLIBC_PREREQ(2, 10)
#define MDBX_USE_ROBUST 1
#else

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@@ -196,12 +196,19 @@ static int suspend_and_append(mdbx_handle_array_t **array,
(*array)->limit = limit * 2;
}
HANDLE hThread = OpenThread(THREAD_SUSPEND_RESUME, FALSE, ThreadId);
HANDLE hThread = OpenThread(THREAD_SUSPEND_RESUME | THREAD_QUERY_INFORMATION,
FALSE, ThreadId);
if (hThread == NULL)
return GetLastError();
if (SuspendThread(hThread) == -1) {
int err = GetLastError();
DWORD ExitCode;
if (err == /* workaround for Win10 UCRT bug */ ERROR_ACCESS_DENIED ||
!GetExitCodeThread(hThread, &ExitCode) || ExitCode != STILL_ACTIVE)
err = MDBX_SUCCESS;
CloseHandle(hThread);
return GetLastError();
return err;
}
(*array)->handles[(*array)->count++] = hThread;
@@ -285,9 +292,15 @@ int mdbx_suspend_threads_before_remap(MDBX_env *env,
int mdbx_resume_threads_after_remap(mdbx_handle_array_t *array) {
int rc = MDBX_SUCCESS;
for (unsigned i = 0; i < array->count; ++i) {
if (ResumeThread(array->handles[i]) == -1)
rc = GetLastError();
CloseHandle(array->handles[i]);
const HANDLE hThread = array->handles[i];
if (ResumeThread(hThread) == -1) {
const int err = GetLastError();
DWORD ExitCode;
if (err != /* workaround for Win10 UCRT bug */ ERROR_ACCESS_DENIED &&
GetExitCodeThread(hThread, &ExitCode) && ExitCode == STILL_ACTIVE)
rc = err;
}
CloseHandle(hThread);
}
return rc;
}

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>
/*
* Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@@ -1983,19 +1983,21 @@ static int mdbx_mapresize(MDBX_env *env, const pgno_t size_pgno,
env->me_dxb_mmap.current == env->me_dxb_mmap.filesize)
goto bailout;
if ((env->me_flags & MDBX_RDONLY) || limit_bytes != env->me_dxb_mmap.length ||
size_bytes < env->me_dxb_mmap.current) {
/* Windows allows only extending a read-write section, but not a
* corresponing mapped view. Therefore in other cases we must suspend
* the local threads for safe remap. */
array_onstack.limit = ARRAY_LENGTH(array_onstack.handles);
array_onstack.count = 0;
suspended = &array_onstack;
rc = mdbx_suspend_threads_before_remap(env, &suspended);
if (rc != MDBX_SUCCESS) {
mdbx_error("failed suspend-for-remap: errcode %d", rc);
goto bailout;
}
/* 1) Windows allows only extending a read-write section, but not a
* corresponing mapped view. Therefore in other cases we must suspend
* the local threads for safe remap.
* 2) At least on Windows 10 1803 the entire mapped section is unavailable
* for short time during NtExtendSection() or VirtualAlloc() execution.
*
* THEREFORE LOCAL THREADS SUSPENDING IS ALWAYS REQUIRED!
*/
array_onstack.limit = ARRAY_LENGTH(array_onstack.handles);
array_onstack.count = 0;
suspended = &array_onstack;
rc = mdbx_suspend_threads_before_remap(env, &suspended);
if (rc != MDBX_SUCCESS) {
mdbx_error("failed suspend-for-remap: errcode %d", rc);
goto bailout;
}
#else
/* Acquire guard to avoid collision between read and write txns
@@ -2032,18 +2034,20 @@ bailout:
VALGRIND_CREATE_BLOCK(env->me_map, env->me_mapsize, "mdbx");
}
#endif
} else if (rc != MDBX_RESULT_TRUE) {
mdbx_error("failed resize datafile/mapping: "
"present %" PRIuPTR " -> %" PRIuPTR ", "
"limit %" PRIuPTR " -> %" PRIuPTR ", errcode %d",
env->me_dbgeo.now, size_bytes, env->me_dbgeo.upper, limit_bytes,
rc);
} else {
mdbx_notice("unable resize datafile/mapping: "
"present %" PRIuPTR " -> %" PRIuPTR ", "
"limit %" PRIuPTR " -> %" PRIuPTR ", errcode %d",
env->me_dbgeo.now, size_bytes, env->me_dbgeo.upper, limit_bytes,
rc);
if (rc != MDBX_RESULT_TRUE) {
mdbx_error("failed resize datafile/mapping: "
"present %" PRIuPTR " -> %" PRIuPTR ", "
"limit %" PRIuPTR " -> %" PRIuPTR ", errcode %d",
env->me_dbgeo.now, size_bytes, env->me_dbgeo.upper,
limit_bytes, rc);
} else {
mdbx_notice("unable resize datafile/mapping: "
"present %" PRIuPTR " -> %" PRIuPTR ", "
"limit %" PRIuPTR " -> %" PRIuPTR ", errcode %d",
env->me_dbgeo.now, size_bytes, env->me_dbgeo.upper,
limit_bytes, rc);
}
if (!env->me_dxb_mmap.address) {
env->me_flags |= MDBX_FATAL_ERROR;
if (env->me_txn)
@@ -2892,6 +2896,16 @@ static int mdbx_txn_renew0(MDBX_txn *txn, unsigned flags) {
rc = mdbx_rdt_lock(env);
if (unlikely(MDBX_IS_ERROR(rc)))
return rc;
if (unlikely(env->me_flags & MDBX_FATAL_ERROR)) {
mdbx_rdt_unlock(env);
return MDBX_PANIC;
}
#if defined(_WIN32) || defined(_WIN64)
if (unlikely(!env->me_map)) {
mdbx_rdt_unlock(env);
return MDBX_EPERM;
}
#endif /* Windows */
rc = MDBX_SUCCESS;
if (unlikely(env->me_live_reader != pid)) {
@@ -2990,6 +3004,16 @@ static int mdbx_txn_renew0(MDBX_txn *txn, unsigned flags) {
rc = mdbx_txn_lock(env, F_ISSET(flags, MDBX_TRYTXN));
if (unlikely(rc))
return rc;
if (unlikely(env->me_flags & MDBX_FATAL_ERROR)) {
mdbx_txn_unlock(env);
return MDBX_PANIC;
}
#if defined(_WIN32) || defined(_WIN64)
if (unlikely(!env->me_map)) {
mdbx_txn_unlock(env);
return MDBX_EPERM;
}
#endif /* Windows */
mdbx_jitter4testing(false);
MDBX_meta *meta = mdbx_meta_head(env);
@@ -3049,8 +3073,11 @@ static int mdbx_txn_renew0(MDBX_txn *txn, unsigned flags) {
goto bailout;
}
rc = mdbx_mapresize(env, txn->mt_end_pgno, upper_pgno);
if (rc != MDBX_SUCCESS)
if (rc != MDBX_SUCCESS) {
if (rc == MDBX_RESULT_TRUE)
rc = MDBX_MAP_RESIZED;
goto bailout;
}
}
txn->mt_owner = mdbx_thread_self();
return MDBX_SUCCESS;
@@ -3098,6 +3125,7 @@ int mdbx_txn_begin(MDBX_env *env, MDBX_txn *parent, unsigned flags,
if (unlikely(!env || !ret))
return MDBX_EINVAL;
*ret = NULL;
if (unlikely(env->me_signature != MDBX_ME_SIGNATURE))
return MDBX_EBADSIGN;
@@ -3107,8 +3135,12 @@ int mdbx_txn_begin(MDBX_env *env, MDBX_txn *parent, unsigned flags,
if (unlikely(env->me_flags & MDBX_FATAL_ERROR))
return MDBX_PANIC;
#if !defined(_WIN32) && !defined(_WIN64)
/* Don't check env->me_map until lock to avoid race with re-mapping for
* shrinking */
if (unlikely(!env->me_map))
return MDBX_EPERM;
#endif /* Windows */
flags &= MDBX_TXN_BEGIN_FLAGS;
flags |= env->me_flags & MDBX_WRITEMAP;
@@ -3119,16 +3151,21 @@ int mdbx_txn_begin(MDBX_env *env, MDBX_txn *parent, unsigned flags,
if (parent) {
if (unlikely(parent->mt_signature != MDBX_MT_SIGNATURE))
return MDBX_EINVAL;
return MDBX_EBADSIGN;
if (unlikely(parent->mt_owner != mdbx_thread_self()))
return MDBX_THREAD_MISMATCH;
#if defined(_WIN32) || defined(_WIN64)
if (unlikely(!env->me_map))
return MDBX_EPERM;
#endif /* Windows */
/* Nested transactions: Max 1 child, write txns only, no writemap */
flags |= parent->mt_flags;
if (unlikely(flags & (MDBX_RDONLY | MDBX_WRITEMAP | MDBX_TXN_BLOCKED))) {
if (unlikely(flags & (MDBX_RDONLY | MDBX_WRITEMAP | MDBX_TXN_BLOCKED)))
return (parent->mt_flags & MDBX_TXN_RDONLY) ? MDBX_EINVAL : MDBX_BAD_TXN;
}
/* Child txns save MDBX_pgstate and use own copy of cursors */
size = env->me_maxdbs * (sizeof(MDBX_db) + sizeof(MDBX_cursor *) + 1);
size += tsize = sizeof(MDBX_ntxn);
@@ -3204,10 +3241,12 @@ int mdbx_txn_begin(MDBX_env *env, MDBX_txn *parent, unsigned flags,
rc = mdbx_txn_renew0(txn, flags);
}
if (unlikely(rc)) {
if (unlikely(rc != MDBX_SUCCESS)) {
if (txn != env->me_txn0)
free(txn);
} else {
mdbx_assert(env,
(txn->mt_flags & ~(MDBX_TXN_RDONLY | MDBX_TXN_WRITEMAP)) == 0);
txn->mt_signature = MDBX_MT_SIGNATURE;
*ret = txn;
mdbx_debug("begin txn %" PRIaTXN "%c %p on env %p, root page %" PRIaPGNO
@@ -3398,7 +3437,7 @@ int mdbx_txn_abort(MDBX_txn *txn) {
if (unlikely(txn->mt_signature != MDBX_MT_SIGNATURE))
return MDBX_EBADSIGN;
if (unlikely(txn->mt_owner && txn->mt_owner != mdbx_thread_self()))
if (unlikely(txn->mt_owner != mdbx_thread_self()))
return MDBX_THREAD_MISMATCH;
if (F_ISSET(txn->mt_flags, MDBX_TXN_RDONLY))
@@ -3432,6 +3471,7 @@ static int mdbx_prep_backlog(MDBX_txn *txn, MDBX_cursor *mc) {
const int extra = mdbx_backlog_extragap(txn->mt_env);
if (mdbx_backlog_size(txn) < mc->mc_db->md_depth + extra) {
mc->mc_flags &= ~C_RECLAIMING;
int rc = mdbx_cursor_touch(mc);
if (unlikely(rc))
return rc;
@@ -3445,6 +3485,7 @@ static int mdbx_prep_backlog(MDBX_txn *txn, MDBX_cursor *mc) {
break;
}
}
mc->mc_flags |= C_RECLAIMING;
}
return MDBX_SUCCESS;
@@ -3465,6 +3506,7 @@ static int mdbx_update_gc(MDBX_txn *txn) {
if (unlikely(rc != MDBX_SUCCESS))
return rc;
mc.mc_flags |= C_RECLAIMING;
mc.mc_next = txn->mt_cursors[FREE_DBI];
txn->mt_cursors[FREE_DBI] = &mc;
@@ -3520,9 +3562,7 @@ retry:
mdbx_tassert(txn, cleaned_gc_id < *env->me_oldest);
mdbx_trace("%s.cleanup-reclaimed-id [%u]%" PRIaTXN, dbg_prefix_mode,
cleaned_gc_slot, cleaned_gc_id);
mc.mc_flags |= C_RECLAIMING;
rc = mdbx_cursor_del(&mc, 0);
mc.mc_flags ^= C_RECLAIMING;
if (unlikely(rc != MDBX_SUCCESS))
goto bailout;
} while (cleaned_gc_slot < txn->mt_lifo_reclaimed[0]);
@@ -3542,9 +3582,7 @@ retry:
mdbx_tassert(txn, cleaned_gc_id < *env->me_oldest);
mdbx_trace("%s.cleanup-reclaimed-id %" PRIaTXN, dbg_prefix_mode,
cleaned_gc_id);
mc.mc_flags |= C_RECLAIMING;
rc = mdbx_cursor_del(&mc, 0);
mc.mc_flags ^= C_RECLAIMING;
if (unlikely(rc != MDBX_SUCCESS))
goto bailout;
settled = 0;
@@ -3642,7 +3680,9 @@ retry:
if (befree_stored < txn->mt_befree_pages[0]) {
if (unlikely(!befree_stored)) {
/* Make sure last page of freeDB is touched and on befree-list */
mc.mc_flags &= ~C_RECLAIMING;
rc = mdbx_page_search(&mc, NULL, MDBX_PS_LAST | MDBX_PS_MODIFY);
mc.mc_flags |= C_RECLAIMING;
if (unlikely(rc != MDBX_SUCCESS && rc != MDBX_NOTFOUND))
goto bailout;
}
@@ -3727,7 +3767,9 @@ retry:
left > ((unsigned)txn->mt_lifo_reclaimed[0] - reused_gc_slot) *
env->me_maxgc_ov1page) {
/* LY: need just a txn-id for save page list. */
mc.mc_flags &= ~C_RECLAIMING;
rc = mdbx_page_alloc(&mc, 0, NULL, MDBX_ALLOC_GC | MDBX_ALLOC_KICK);
mc.mc_flags |= C_RECLAIMING;
if (likely(rc == MDBX_SUCCESS)) {
/* LY: ok, reclaimed from freedb. */
mdbx_trace("%s: took @%" PRIaTXN " from GC, continue",
@@ -3932,9 +3974,7 @@ retry:
key.iov_base = &fill_gc_id;
key.iov_len = sizeof(fill_gc_id);
mc.mc_flags |= C_RECLAIMING;
rc = mdbx_cursor_put(&mc, &key, &data, MDBX_CURRENT | MDBX_RESERVE);
mc.mc_flags ^= C_RECLAIMING;
mdbx_tassert(txn, mdbx_pnl_check(env->me_reclaimed_pglist));
if (unlikely(rc != MDBX_SUCCESS))
goto bailout;
@@ -4400,8 +4440,11 @@ int mdbx_txn_commit(MDBX_txn *txn) {
rc = mdbx_sync_locked(
env, env->me_flags | txn->mt_flags | MDBX_SHRINK_ALLOWED, &meta);
}
if (unlikely(rc != MDBX_SUCCESS))
if (unlikely(rc != MDBX_SUCCESS)) {
env->me_flags |= MDBX_FATAL_ERROR;
goto fail;
}
env->me_lck->mti_readers_refresh_flag = false;
end_mode = MDBX_END_COMMITTED | MDBX_END_UPDATE | MDBX_END_EOTDONE;
@@ -4552,15 +4595,6 @@ static int __cold mdbx_read_header(MDBX_env *env, MDBX_meta *meta,
continue;
}
/* LY: check end_pgno */
if (page.mp_meta.mm_geo.now < page.mp_meta.mm_geo.lower ||
page.mp_meta.mm_geo.now > page.mp_meta.mm_geo.upper) {
mdbx_notice("meta[%u] has invalid end-pageno (%" PRIaPGNO "), skip it",
meta_number, page.mp_meta.mm_geo.now);
rc = MDBX_CORRUPTED;
continue;
}
/* LY: check last_pgno */
if (page.mp_meta.mm_geo.next < MIN_PAGENO ||
page.mp_meta.mm_geo.next - 1 > MAX_PAGENO) {
@@ -4574,6 +4608,7 @@ static int __cold mdbx_read_header(MDBX_env *env, MDBX_meta *meta,
const uint64_t used_bytes =
page.mp_meta.mm_geo.next * (uint64_t)page.mp_meta.mm_psize;
if (used_bytes > *filesize) {
/* Here could be a race with DB-shrinking performed by other process */
rc = mdbx_filesize(env->me_fd, filesize);
if (unlikely(rc != MDBX_SUCCESS))
return rc;
@@ -4617,10 +4652,20 @@ static int __cold mdbx_read_header(MDBX_env *env, MDBX_meta *meta,
"but size of used space still acceptable (%" PRIu64 ")",
meta_number, mapsize_max, used_bytes);
page.mp_meta.mm_geo.upper = (pgno_t)(MAX_MAPSIZE / page.mp_meta.mm_psize);
if (page.mp_meta.mm_geo.now > page.mp_meta.mm_geo.upper)
page.mp_meta.mm_geo.now = page.mp_meta.mm_geo.upper;
}
/* LY: check and silently put mm_geo.now into [geo.lower...geo.upper].
*
* Copy-with-compaction by previous version of libmdbx could produce DB-file
* less than meta.geo.lower bound, in case actual filling is low or no data
* at all. This is not a problem as there is no damage or loss of data.
* Therefore it is better not to consider such situation as an error, but
* silently correct it. */
if (page.mp_meta.mm_geo.now < page.mp_meta.mm_geo.lower)
page.mp_meta.mm_geo.now = page.mp_meta.mm_geo.lower;
if (page.mp_meta.mm_geo.now > page.mp_meta.mm_geo.upper)
page.mp_meta.mm_geo.now = page.mp_meta.mm_geo.upper;
if (page.mp_meta.mm_geo.next > page.mp_meta.mm_geo.now) {
mdbx_notice("meta[%u] next-pageno (%" PRIaPGNO
") is beyond end-pgno (%" PRIaPGNO "), skip it",
@@ -4980,7 +5025,7 @@ fail:
int __cold mdbx_env_get_maxkeysize(MDBX_env *env) {
if (!env || env->me_signature != MDBX_ME_SIGNATURE || !env->me_maxkey_limit)
return -MDBX_EINVAL;
return (MDBX_EINVAL > 0) ? -MDBX_EINVAL : MDBX_EINVAL;
return env->me_maxkey_limit;
}
@@ -4993,14 +5038,7 @@ int __cold mdbx_env_get_maxkeysize(MDBX_env *env) {
(((pagesize)-PAGEHDRSZ) / sizeof(pgno_t) - 1)
int mdbx_get_maxkeysize(intptr_t pagesize) {
if (pagesize < 1)
pagesize = (intptr_t)mdbx_syspagesize();
else if (unlikely(pagesize < (intptr_t)MIN_PAGESIZE ||
pagesize > (intptr_t)MAX_PAGESIZE ||
!mdbx_is_power2((size_t)pagesize)))
return -MDBX_EINVAL;
return mdbx_maxkey(mdbx_nodemax(pagesize));
return (int)mdbx_limits_keysize_max(pagesize);
}
static void __cold mdbx_setup_pagesize(MDBX_env *env, const size_t pagesize) {
@@ -5561,6 +5599,10 @@ static int __cold mdbx_setup_dxb(MDBX_env *env, int lck_rc) {
}
if (env->me_flags & MDBX_RDONLY) {
if (filesize_before_mmap % env->me_os_psize) {
mdbx_error("filesize should be rounded-up to system page");
return MDBX_WANNA_RECOVERY;
}
mdbx_notice("ignore filesize mismatch in readonly-mode");
} else {
mdbx_info("resize datafile to %" PRIuSIZE " bytes, %" PRIaPGNO " pages",
@@ -7656,7 +7698,7 @@ int mdbx_cursor_put(MDBX_cursor *mc, MDBX_val *key, MDBX_val *data,
if (rc > 0) {
rc = MDBX_NOTFOUND;
mc->mc_ki[mc->mc_top]++;
} else {
} else if (unlikely(rc < 0 || (flags & MDBX_APPENDDUP) == 0)) {
/* new key is <= last key */
rc = MDBX_EKEYMISMATCH;
}
@@ -9965,7 +10007,7 @@ static int mdbx_page_split(MDBX_cursor *mc, MDBX_val *newkey, MDBX_val *newdata,
* This yields better packing during sequential inserts.
*/
int dir;
if (nkeys < 20 || nsize > pmax / 16 || newindx >= nkeys) {
if (nkeys < 32 || nsize > pmax / 16 || newindx >= nkeys) {
/* Find split point */
psize = 0;
if (newindx <= split_indx || newindx >= nkeys) {
@@ -10536,184 +10578,244 @@ done:
}
/* Copy environment with compaction. */
static int __cold mdbx_env_compact(MDBX_env *env, mdbx_filehandle_t fd) {
MDBX_txn *txn = NULL;
mdbx_thread_t thr;
mdbx_copy ctx;
memset(&ctx, 0, sizeof(ctx));
int rc = mdbx_condmutex_init(&ctx.mc_condmutex);
if (unlikely(rc != MDBX_SUCCESS))
return rc;
const size_t buffer_size = pgno2bytes(env, NUM_METAS) + MDBX_WBUF * 2;
uint8_t *buffer = NULL;
rc = mdbx_memalign_alloc(env->me_os_psize, buffer_size, (void **)&buffer);
if (unlikely(rc != MDBX_SUCCESS))
goto done;
ctx.mc_wbuf[0] = buffer + pgno2bytes(env, NUM_METAS);
memset(ctx.mc_wbuf[0], 0, MDBX_WBUF * 2);
ctx.mc_wbuf[1] = ctx.mc_wbuf[0] + MDBX_WBUF;
ctx.mc_next_pgno = NUM_METAS;
ctx.mc_env = env;
ctx.mc_fd = fd;
rc = mdbx_thread_create(&thr, mdbx_env_copythr, &ctx);
if (unlikely(rc != MDBX_SUCCESS))
goto done;
rc = mdbx_txn_begin(env, NULL, MDBX_RDONLY, &txn);
if (unlikely(rc != MDBX_SUCCESS))
goto finish;
static int __cold mdbx_env_compact(MDBX_env *env, MDBX_txn *read_txn,
mdbx_filehandle_t fd, uint8_t *buffer) {
MDBX_page *const meta = mdbx_init_metas(env, buffer);
/* copy canary sequenses if present */
if (read_txn->mt_canary.v) {
meta->mp_meta.mm_canary = read_txn->mt_canary;
meta->mp_meta.mm_canary.v = mdbx_meta_txnid_stable(env, &meta->mp_meta);
}
/* Set metapage 1 with current main DB */
pgno_t new_root, root = txn->mt_dbs[MAIN_DBI].md_root;
if ((new_root = root) != P_INVALID) {
pgno_t new_root, root = read_txn->mt_dbs[MAIN_DBI].md_root;
if ((new_root = root) == P_INVALID) {
/* When the DB is empty, handle it specially to
* fix any breakage like page leaks from ITS#8174. */
meta->mp_meta.mm_dbs[MAIN_DBI].md_flags =
read_txn->mt_dbs[MAIN_DBI].md_flags;
} else {
/* Count free pages + freeDB pages. Subtract from last_pg
* to find the new last_pg, which also becomes the new root. */
pgno_t freecount = 0;
MDBX_cursor mc;
MDBX_val key, data;
rc = mdbx_cursor_init(&mc, txn, FREE_DBI, NULL);
int rc = mdbx_cursor_init(&mc, read_txn, FREE_DBI, NULL);
if (unlikely(rc != MDBX_SUCCESS))
return rc;
while ((rc = mdbx_cursor_get(&mc, &key, &data, MDBX_NEXT)) == 0)
freecount += *(pgno_t *)data.iov_base;
if (unlikely(rc != MDBX_NOTFOUND))
goto finish;
return rc;
freecount += txn->mt_dbs[FREE_DBI].md_branch_pages +
txn->mt_dbs[FREE_DBI].md_leaf_pages +
txn->mt_dbs[FREE_DBI].md_overflow_pages;
freecount += read_txn->mt_dbs[FREE_DBI].md_branch_pages +
read_txn->mt_dbs[FREE_DBI].md_leaf_pages +
read_txn->mt_dbs[FREE_DBI].md_overflow_pages;
new_root = txn->mt_next_pgno - 1 - freecount;
meta->mp_meta.mm_geo.next = meta->mp_meta.mm_geo.now = new_root + 1;
meta->mp_meta.mm_dbs[MAIN_DBI] = txn->mt_dbs[MAIN_DBI];
new_root = read_txn->mt_next_pgno - 1 - freecount;
meta->mp_meta.mm_geo.next = new_root + 1;
meta->mp_meta.mm_dbs[MAIN_DBI] = read_txn->mt_dbs[MAIN_DBI];
meta->mp_meta.mm_dbs[MAIN_DBI].md_root = new_root;
} else {
/* When the DB is empty, handle it specially to
* fix any breakage like page leaks from ITS#8174. */
meta->mp_meta.mm_dbs[MAIN_DBI].md_flags = txn->mt_dbs[MAIN_DBI].md_flags;
}
/* copy canary sequenses if present */
if (txn->mt_canary.v) {
meta->mp_meta.mm_canary = txn->mt_canary;
meta->mp_meta.mm_canary.v = mdbx_meta_txnid_stable(env, &meta->mp_meta);
}
mdbx_copy ctx;
memset(&ctx, 0, sizeof(ctx));
rc = mdbx_condmutex_init(&ctx.mc_condmutex);
if (unlikely(rc != MDBX_SUCCESS))
return rc;
/* update signature */
meta->mp_meta.mm_datasync_sign = mdbx_meta_sign(&meta->mp_meta);
ctx.mc_wbuf[0] = buffer + pgno2bytes(env, NUM_METAS);
memset(ctx.mc_wbuf[0], 0, MDBX_WBUF * 2);
ctx.mc_wbuf[1] = ctx.mc_wbuf[0] + MDBX_WBUF;
ctx.mc_next_pgno = NUM_METAS;
ctx.mc_env = env;
ctx.mc_fd = fd;
ctx.mc_txn = read_txn;
mdbx_thread_t thread;
int thread_err = mdbx_thread_create(&thread, mdbx_env_copythr, &ctx);
if (likely(thread_err == MDBX_SUCCESS)) {
rc = mdbx_env_cwalk(&ctx, &root, 0);
mdbx_env_cthr_toggle(&ctx, 1 | MDBX_EOF);
thread_err = mdbx_thread_join(thread);
mdbx_condmutex_destroy(&ctx.mc_condmutex);
}
if (unlikely(thread_err != MDBX_SUCCESS))
return thread_err;
if (unlikely(rc != MDBX_SUCCESS))
return rc;
if (unlikely(ctx.mc_error != MDBX_SUCCESS))
return ctx.mc_error;
ctx.mc_wlen[0] = pgno2bytes(env, NUM_METAS);
ctx.mc_txn = txn;
rc = mdbx_env_cwalk(&ctx, &root, 0);
if (rc == MDBX_SUCCESS && root != new_root) {
if (root > new_root) {
mdbx_error("post-compactification root %" PRIaPGNO
" GT expected %" PRIaPGNO " (source DB corrupted)",
root, new_root);
rc = MDBX_CORRUPTED; /* page leak or corrupt DB */
} else {
mdbx_error("post-compactification root %" PRIaPGNO
" LT expected %" PRIaPGNO " (page leak(s) in source DB)",
root, new_root);
/* fixup and rewrite metas */
return MDBX_CORRUPTED; /* page leak or corrupt DB */
}
if (root < new_root) {
mdbx_notice("post-compactification root %" PRIaPGNO
" LT expected %" PRIaPGNO " (page leak(s) in source DB)",
root, new_root);
/* fixup meta */
meta->mp_meta.mm_dbs[MAIN_DBI].md_root = root;
meta->mp_meta.mm_geo.next = meta->mp_meta.mm_geo.now = root + 1;
meta->mp_meta.mm_datasync_sign = mdbx_meta_sign(&meta->mp_meta);
rc = mdbx_pwrite(fd, buffer, pgno2bytes(env, NUM_METAS), 0);
meta->mp_meta.mm_geo.next = root + 1;
}
}
finish:
if (rc != MDBX_SUCCESS)
ctx.mc_error = rc;
mdbx_env_cthr_toggle(&ctx, 1 | MDBX_EOF);
rc = mdbx_thread_join(thr);
mdbx_txn_abort(txn);
/* Calculate filesize taking in account shrink/growing thresholds */
if (meta->mp_meta.mm_geo.next > meta->mp_meta.mm_geo.now) {
const pgno_t aligned =
pgno_align2os_pgno(env, pgno_add(meta->mp_meta.mm_geo.next,
meta->mp_meta.mm_geo.grow -
meta->mp_meta.mm_geo.next %
meta->mp_meta.mm_geo.grow));
meta->mp_meta.mm_geo.now = aligned;
} else if (meta->mp_meta.mm_geo.next < meta->mp_meta.mm_geo.now) {
meta->mp_meta.mm_geo.now = meta->mp_meta.mm_geo.next;
const pgno_t aligner = meta->mp_meta.mm_geo.grow
? meta->mp_meta.mm_geo.grow
: meta->mp_meta.mm_geo.shrink;
const pgno_t aligned =
pgno_align2os_pgno(env, meta->mp_meta.mm_geo.next + aligner -
meta->mp_meta.mm_geo.next % aligner);
meta->mp_meta.mm_geo.now = aligned;
}
done:
mdbx_memalign_free(buffer);
mdbx_condmutex_destroy(&ctx.mc_condmutex);
return rc ? rc : ctx.mc_error;
if (meta->mp_meta.mm_geo.now < meta->mp_meta.mm_geo.lower)
meta->mp_meta.mm_geo.now = meta->mp_meta.mm_geo.lower;
if (meta->mp_meta.mm_geo.now > meta->mp_meta.mm_geo.upper)
meta->mp_meta.mm_geo.now = meta->mp_meta.mm_geo.upper;
/* Update signature */
assert(meta->mp_meta.mm_geo.now >= meta->mp_meta.mm_geo.next);
meta->mp_meta.mm_datasync_sign = mdbx_meta_sign(&meta->mp_meta);
/* Extend file if required */
return (meta->mp_meta.mm_geo.now != meta->mp_meta.mm_geo.next)
? mdbx_ftruncate(fd, pgno2bytes(env, meta->mp_meta.mm_geo.now))
: MDBX_SUCCESS;
}
/* Copy environment as-is. */
static int __cold mdbx_env_copy_asis(MDBX_env *env, mdbx_filehandle_t fd) {
MDBX_txn *txn = NULL;
/* Do the lock/unlock of the reader mutex before starting the
* write txn. Otherwise other read txns could block writers. */
int rc = mdbx_txn_begin(env, NULL, MDBX_RDONLY, &txn);
static int __cold mdbx_env_copy_asis(MDBX_env *env, MDBX_txn *read_txn,
mdbx_filehandle_t fd, uint8_t *buffer) {
/* We must start the actual read txn after blocking writers */
int rc = mdbx_txn_end(read_txn, MDBX_END_RESET_TMP);
if (unlikely(rc != MDBX_SUCCESS))
return rc;
/* We must start the actual read txn after blocking writers */
rc = mdbx_txn_end(txn, MDBX_END_RESET_TMP);
if (unlikely(rc != MDBX_SUCCESS))
goto bailout; /* FIXME: or just return? */
/* Temporarily block writers until we snapshot the meta pages */
rc = mdbx_txn_lock(env, false);
if (unlikely(rc != MDBX_SUCCESS))
goto bailout;
return rc;
rc = mdbx_txn_renew0(txn, MDBX_RDONLY);
rc = mdbx_txn_renew0(read_txn, MDBX_RDONLY);
if (unlikely(rc != MDBX_SUCCESS)) {
mdbx_txn_unlock(env);
goto bailout;
return rc;
}
rc = mdbx_write(fd, env->me_map, pgno2bytes(env, NUM_METAS));
MDBX_meta *const head = mdbx_meta_head(env);
/* Make a snapshot of meta-pages,
* but writing ones after the data was flushed */
memcpy(buffer, env->me_map, pgno2bytes(env, NUM_METAS));
MDBX_meta *const headcopy = /* LY: get pointer to the spanshot copy */
(MDBX_meta *)(buffer + ((uint8_t *)mdbx_meta_head(env) - env->me_map));
const uint64_t size =
mdbx_roundup2(pgno2bytes(env, head->mm_geo.now), env->me_os_psize);
mdbx_roundup2(pgno2bytes(env, headcopy->mm_geo.now), env->me_os_psize);
mdbx_txn_unlock(env);
if (likely(rc == MDBX_SUCCESS))
rc = mdbx_write(fd, env->me_map + pgno2bytes(env, NUM_METAS),
pgno2bytes(env, txn->mt_next_pgno - NUM_METAS));
/* Update signature to steady */
headcopy->mm_datasync_sign = mdbx_meta_sign(headcopy);
/* Copy the data */
rc = mdbx_pwrite(fd, env->me_map + pgno2bytes(env, NUM_METAS),
pgno2bytes(env, read_txn->mt_next_pgno - NUM_METAS),
pgno2bytes(env, NUM_METAS));
if (likely(rc == MDBX_SUCCESS))
rc = mdbx_ftruncate(fd, size);
bailout:
mdbx_txn_abort(txn);
return rc;
}
int __cold mdbx_env_copy2fd(MDBX_env *env, mdbx_filehandle_t fd,
unsigned flags) {
if (flags & MDBX_CP_COMPACT)
return mdbx_env_compact(env, fd);
if (unlikely(!env))
return MDBX_EINVAL;
return mdbx_env_copy_asis(env, fd);
if (unlikely(env->me_signature != MDBX_ME_SIGNATURE))
return MDBX_EBADSIGN;
int rc = mdbx_fseek(fd, 0);
if (unlikely(rc != MDBX_SUCCESS))
return rc;
const size_t buffer_size = pgno2bytes(env, NUM_METAS) +
((flags & MDBX_CP_COMPACT) ? MDBX_WBUF * 2 : 0);
uint8_t *buffer = NULL;
rc = mdbx_memalign_alloc(env->me_os_psize, buffer_size, (void **)&buffer);
if (unlikely(rc != MDBX_SUCCESS))
return rc;
MDBX_txn *read_txn = NULL;
/* Do the lock/unlock of the reader mutex before starting the
* write txn. Otherwise other read txns could block writers. */
rc = mdbx_txn_begin(env, NULL, MDBX_RDONLY, &read_txn);
if (unlikely(rc != MDBX_SUCCESS)) {
mdbx_memalign_free(buffer);
return rc;
}
/* Firstly write a stub to meta-pages.
* Now we sure to incomplete copy will not be used. */
memset(buffer, -1, pgno2bytes(env, NUM_METAS));
rc = mdbx_write(fd, buffer, pgno2bytes(env, NUM_METAS));
if (likely(rc == MDBX_SUCCESS)) {
memset(buffer, 0, pgno2bytes(env, NUM_METAS));
rc = (flags & MDBX_CP_COMPACT)
? mdbx_env_compact(env, read_txn, fd, buffer)
: mdbx_env_copy_asis(env, read_txn, fd, buffer);
}
mdbx_txn_abort(read_txn);
if (likely(rc == MDBX_SUCCESS))
rc = mdbx_filesync(fd, true);
/* Write actual meta */
if (likely(rc == MDBX_SUCCESS))
rc = mdbx_pwrite(fd, buffer, pgno2bytes(env, NUM_METAS), 0);
mdbx_memalign_free(buffer);
return rc;
}
int __cold mdbx_env_copy(MDBX_env *env, const char *path, unsigned flags) {
char *lck_pathname;
int __cold mdbx_env_copy(MDBX_env *env, const char *dest_path, unsigned flags) {
if (unlikely(!env || !dest_path))
return MDBX_EINVAL;
if (unlikely(env->me_signature != MDBX_ME_SIGNATURE))
return MDBX_EBADSIGN;
char *dxb_pathname;
mdbx_filehandle_t newfd = INVALID_HANDLE_VALUE;
if (env->me_flags & MDBX_NOSUBDIR) {
lck_pathname = (char *)path;
dxb_pathname = (char *)dest_path;
} else {
size_t len = strlen(path);
size_t len = strlen(dest_path);
len += sizeof(MDBX_DATANAME);
lck_pathname = malloc(len);
if (!lck_pathname)
dxb_pathname = malloc(len);
if (!dxb_pathname)
return MDBX_ENOMEM;
sprintf(lck_pathname, "%s" MDBX_DATANAME, path);
sprintf(dxb_pathname, "%s" MDBX_DATANAME, dest_path);
}
/* The destination path must exist, but the destination file must not.
* We don't want the OS to cache the writes, since the source data is
* already in the OS cache. */
int rc =
mdbx_openfile(lck_pathname, O_WRONLY | O_CREAT | O_EXCL, 0666, &newfd);
mdbx_openfile(dxb_pathname, O_WRONLY | O_CREAT | O_EXCL, 0640, &newfd);
if (rc == MDBX_SUCCESS) {
if (env->me_psize >= env->me_os_psize) {
#ifdef F_NOCACHE /* __APPLE__ */
@@ -10727,15 +10829,17 @@ int __cold mdbx_env_copy(MDBX_env *env, const char *path, unsigned flags) {
rc = mdbx_env_copy2fd(env, newfd, flags);
}
if (!(env->me_flags & MDBX_NOSUBDIR))
free(lck_pathname);
if (newfd != INVALID_HANDLE_VALUE) {
int err = mdbx_closefile(newfd);
if (rc == MDBX_SUCCESS && err != rc)
rc = err;
if (rc != MDBX_SUCCESS)
(void)mdbx_removefile(dxb_pathname);
}
if (dxb_pathname != dest_path)
free(dxb_pathname);
return rc;
}
@@ -11105,19 +11209,19 @@ int mdbx_dbi_open_ex(MDBX_txn *txn, const char *table_name, unsigned user_flags,
bailout:
free(namedup);
} else {
txn->mt_dbiseqs[slot] = (env->me_dbiseqs[slot] += 1);
txn->mt_dbflags[slot] = (uint8_t)dbflag;
txn->mt_dbxs[slot].md_name.iov_base = namedup;
mdbx_compiler_barrier();
txn->mt_dbxs[slot].md_name.iov_len = len;
if (slot == txn->mt_numdbs)
txn->mt_numdbs++;
txn->mt_numdbs += (slot == txn->mt_numdbs);
if ((dbflag & DB_CREAT) == 0) {
env->me_dbflags[slot] = txn->mt_dbs[slot].md_flags | MDBX_VALID;
mdbx_compiler_barrier();
if (env->me_numdbs <= slot)
env->me_numdbs = slot + 1;
} else {
env->me_dbiseqs[slot] += 1;
}
txn->mt_dbiseqs[slot] = env->me_dbiseqs[slot];
*dbi = slot;
}
@@ -11428,7 +11532,7 @@ int __cold mdbx_reader_list(MDBX_env *env, MDBX_msg_func *func, void *ctx) {
int rc = 0, first = 1;
if (unlikely(!env || !func))
return -MDBX_EINVAL;
return (MDBX_EINVAL > 0) ? -MDBX_EINVAL : MDBX_EINVAL;
if (unlikely(env->me_signature != MDBX_ME_SIGNATURE))
return MDBX_EBADSIGN;
@@ -11738,7 +11842,7 @@ __attribute__((no_sanitize_thread, noinline))
int mdbx_txn_straggler(MDBX_txn *txn, int *percent)
{
if (unlikely(!txn))
return -MDBX_EINVAL;
return (MDBX_EINVAL > 0) ? -MDBX_EINVAL : MDBX_EINVAL;
if (unlikely(txn->mt_signature != MDBX_MT_SIGNATURE))
return MDBX_EBADSIGN;
@@ -12356,6 +12460,15 @@ int mdbx_dbi_sequence(MDBX_txn *txn, MDBX_dbi dbi, uint64_t *result,
if (unlikely(TXN_DBI_CHANGED(txn, dbi)))
return MDBX_BAD_DBI;
if (unlikely(txn->mt_dbflags[dbi] & DB_STALE)) {
MDBX_cursor mc;
MDBX_xcursor mx;
/* Stale, must read the DB's root. cursor_init does it for us. */
int rc = mdbx_cursor_init(&mc, txn, dbi, &mx);
if (unlikely(rc != MDBX_SUCCESS))
return rc;
}
MDBX_db *dbs = &txn->mt_dbs[dbi];
if (likely(result))
*result = dbs->md_seq;
@@ -12382,6 +12495,17 @@ int mdbx_dbi_sequence(MDBX_txn *txn, MDBX_dbi dbi, uint64_t *result,
/*----------------------------------------------------------------------------*/
__cold intptr_t mdbx_limits_keysize_max(intptr_t pagesize) {
if (pagesize < 1)
pagesize = (intptr_t)mdbx_syspagesize();
else if (unlikely(pagesize < (intptr_t)MIN_PAGESIZE ||
pagesize > (intptr_t)MAX_PAGESIZE ||
!mdbx_is_power2((size_t)pagesize)))
return (MDBX_EINVAL > 0) ? -MDBX_EINVAL : MDBX_EINVAL;
return mdbx_maxkey(mdbx_nodemax(pagesize));
}
__cold int mdbx_limits_pgsize_min(void) { return MIN_PAGESIZE; }
__cold int mdbx_limits_pgsize_max(void) { return MAX_PAGESIZE; }
@@ -12392,7 +12516,7 @@ __cold intptr_t mdbx_limits_dbsize_min(intptr_t pagesize) {
else if (unlikely(pagesize < (intptr_t)MIN_PAGESIZE ||
pagesize > (intptr_t)MAX_PAGESIZE ||
!mdbx_is_power2((size_t)pagesize)))
return -MDBX_EINVAL;
return (MDBX_EINVAL > 0) ? -MDBX_EINVAL : MDBX_EINVAL;
return MIN_PAGENO * pagesize;
}
@@ -12403,11 +12527,22 @@ __cold intptr_t mdbx_limits_dbsize_max(intptr_t pagesize) {
else if (unlikely(pagesize < (intptr_t)MIN_PAGESIZE ||
pagesize > (intptr_t)MAX_PAGESIZE ||
!mdbx_is_power2((size_t)pagesize)))
return -MDBX_EINVAL;
return (MDBX_EINVAL > 0) ? -MDBX_EINVAL : MDBX_EINVAL;
const uint64_t limit = MAX_PAGENO * (uint64_t)pagesize;
return (limit < (intptr_t)MAX_MAPSIZE) ? (intptr_t)limit
: (intptr_t)MAX_PAGESIZE;
: (intptr_t)MAX_MAPSIZE;
}
__cold intptr_t mdbx_limits_txnsize_max(intptr_t pagesize) {
if (pagesize < 1)
pagesize = (intptr_t)mdbx_syspagesize();
else if (unlikely(pagesize < (intptr_t)MIN_PAGESIZE ||
pagesize > (intptr_t)MAX_PAGESIZE ||
!mdbx_is_power2((size_t)pagesize)))
return (MDBX_EINVAL > 0) ? -MDBX_EINVAL : MDBX_EINVAL;
return pagesize * (MDBX_PNL_UM_SIZE - 1);
}
/*----------------------------------------------------------------------------*/

View File

@@ -1,7 +1,7 @@
/* https://en.wikipedia.org/wiki/Operating_system_abstraction_layer */
/*
* Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@@ -399,6 +399,13 @@ int mdbx_fastmutex_release(mdbx_fastmutex_t *fastmutex) {
/*----------------------------------------------------------------------------*/
int mdbx_removefile(const char *pathname) {
#if defined(_WIN32) || defined(_WIN64)
return DeleteFileA(pathname) ? MDBX_SUCCESS : GetLastError();
#else
return unlink(pathname) ? errno : MDBX_SUCCESS;
#endif
}
int mdbx_openfile(const char *pathname, int flags, mode_t mode,
mdbx_filehandle_t *fd) {
*fd = INVALID_HANDLE_VALUE;
@@ -526,17 +533,25 @@ int mdbx_pwrite(mdbx_filehandle_t fd, const void *buf, size_t bytes,
return (bytes == written) ? MDBX_SUCCESS : MDBX_EIO /* ERROR_WRITE_FAULT */;
return GetLastError();
#else
int rc;
intptr_t written;
do {
while (true) {
STATIC_ASSERT_MSG(sizeof(off_t) >= sizeof(size_t),
"libmdbx requires 64-bit file I/O on 64-bit systems");
written = pwrite(fd, buf, bytes, offset);
const intptr_t written =
pwrite(fd, buf, (bytes <= MAX_WRITE) ? bytes : MAX_WRITE, offset);
if (likely(bytes == (size_t)written))
return MDBX_SUCCESS;
rc = errno;
} while (rc == EINTR);
return (written < 0) ? rc : MDBX_EIO /* Use which error code (ENOSPC)? */;
if (written < 0) {
const int rc = errno;
if (rc != EINTR)
return rc;
} else if (written > 0) {
bytes -= written;
offset += written;
buf = (char *)buf + written;
} else {
return -1;
}
}
#endif
}
@@ -698,6 +713,19 @@ int mdbx_ftruncate(mdbx_filehandle_t fd, uint64_t length) {
#endif
}
int mdbx_fseek(mdbx_filehandle_t fd, uint64_t pos) {
#if defined(_WIN32) || defined(_WIN64)
LARGE_INTEGER li;
li.QuadPart = pos;
return SetFilePointerEx(fd, li, NULL, FILE_BEGIN) ? MDBX_SUCCESS
: GetLastError();
#else
STATIC_ASSERT_MSG(sizeof(off_t) >= sizeof(size_t),
"libmdbx requires 64-bit file I/O on 64-bit systems");
return (lseek(fd, pos, SEEK_SET) < 0) ? errno : MDBX_SUCCESS;
#endif
}
/*----------------------------------------------------------------------------*/
int mdbx_thread_create(mdbx_thread_t *thread,
@@ -973,11 +1001,11 @@ int mdbx_mresize(int flags, mdbx_mmap_t *map, size_t size, size_t limit) {
&ReservedSize, MEM_RESERVE, PAGE_NOACCESS);
if (!NT_SUCCESS(status)) {
ReservedAddress = NULL;
if (status != /* STATUS_CONFLICTING_ADDRESSES */ 0xC0000018 ||
limit == map->length)
if (status != /* STATUS_CONFLICTING_ADDRESSES */ 0xC0000018)
goto bailout_ntstatus /* no way to recovery */;
/* assume we can change base address if mapping size changed */
/* assume we can change base address if mapping size changed or prev address
* couldn't be used */
map->address = NULL;
}
@@ -1034,8 +1062,8 @@ retry_mapview:;
if (!NT_SUCCESS(status)) {
if (status == /* STATUS_CONFLICTING_ADDRESSES */ 0xC0000018 &&
map->address && limit != map->length) {
/* try remap at another base address, but only if the limit is changing */
map->address) {
/* try remap at another base address */
map->address = NULL;
goto retry_mapview;
}

View File

@@ -1,7 +1,7 @@
/* https://en.wikipedia.org/wiki/Operating_system_abstraction_layer */
/*
* Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@@ -473,10 +473,12 @@ int mdbx_thread_join(mdbx_thread_t thread);
int mdbx_filesync(mdbx_filehandle_t fd, bool fullsync);
int mdbx_filesize_sync(mdbx_filehandle_t fd);
int mdbx_ftruncate(mdbx_filehandle_t fd, uint64_t length);
int mdbx_fseek(mdbx_filehandle_t fd, uint64_t pos);
int mdbx_filesize(mdbx_filehandle_t fd, uint64_t *length);
int mdbx_openfile(const char *pathname, int flags, mode_t mode,
mdbx_filehandle_t *fd);
int mdbx_closefile(mdbx_filehandle_t fd);
int mdbx_removefile(const char *pathname);
typedef struct mdbx_mmap_param {
union {

View File

@@ -1,7 +1,7 @@
/* mdbx_chk.c - memory-mapped database check tool */
/* mdbx_chk.c - memory-mapped database check tool */
/*
* Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -1,4 +1,4 @@
.\" Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.

View File

@@ -1,7 +1,7 @@
/* mdbx_copy.c - memory-mapped database backup tool */
/*
* Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -1,4 +1,4 @@
.\" Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.

View File

@@ -1,7 +1,7 @@
/* mdbx_dump.c - memory-mapped database dump tool */
/*
* Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -1,4 +1,4 @@
.\" Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2014-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.

View File

@@ -1,7 +1,7 @@
/* mdbx_load.c - memory-mapped database load tool */
/*
* Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -1,4 +1,4 @@
.\" Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>.
.\" Copyright 2012-2015 Howard Chu, Symas Corp. All Rights Reserved.
.\" Copyright 2015,2016 Peter-Service R&D LLC <http://billing.ru/>.
.\" Copying restrictions apply. See COPYRIGHT/LICENSE.

View File

@@ -1,7 +1,7 @@
/* mdbx_stat.c - memory-mapped database status tool */
/*
* Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@@ -18,8 +18,8 @@
#error "API version mismatch!"
#endif
#define MDBX_VERSION_RELEASE 6
#define MDBX_VERSION_REVISION 1
#define MDBX_VERSION_RELEASE 7
#define MDBX_VERSION_REVISION 0
/*LIBMDBX_EXPORTS*/ const mdbx_version_info mdbx_version = {
MDBX_VERSION_MAJOR,
@@ -30,5 +30,5 @@
"@MDBX_GIT_DESCRIBE@"}};
/*LIBMDBX_EXPORTS*/ const mdbx_build_info mdbx_build = {
"@MDBX_BUILD_TIMESTAMP@", "@MDBX_BUILD_TAGRET@", "@MDBX_BUILD_OPTIONS@",
"@MDBX_BUILD_TIMESTAMP@", "@MDBX_BUILD_TARGET@", "@MDBX_BUILD_OPTIONS@",
"@MDBX_BUILD_COMPILER@", "@MDBX_BUILD_FLAGS@"};

View File

@@ -5,30 +5,31 @@ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-missing-declarations")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-cast-qual")
add_executable(${TARGET}
base.h
cases.cc
chrono.cc
chrono.h
config.cc
config.h
dead.cc
hill.cc
jitter.cc
keygen.cc
keygen.h
log.cc
log.h
main.cc
osal.h
osal-unix.cc
test.cc
test.h
try.cc
utils.cc
utils.h
)
base.h
cases.cc
chrono.cc
chrono.h
config.cc
config.h
copy.cc
dead.cc
hill.cc
jitter.cc
keygen.cc
keygen.h
log.cc
log.h
main.cc
osal.h
osal-unix.cc
test.cc
test.h
try.cc
utils.cc
utils.h
)
target_link_libraries(${TARGET}
mdbx
)
mdbx
)

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@@ -68,6 +68,7 @@ void testcase_setup(const char *casename, actor_params &params,
configure_actor(last_space_id, ac_jitter, nullptr, params);
configure_actor(last_space_id, ac_hill, nullptr, params);
configure_actor(last_space_id, ac_try, nullptr, params);
configure_actor(last_space_id, ac_copy, nullptr, params);
log_notice("<<< testcase_setup(%s): done", casename);
} else {
failure("unknown testcase `%s`", casename);

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@@ -26,7 +26,8 @@ enum actor_testcase {
ac_deadread,
ac_deadwrite,
ac_jitter,
ac_try
ac_try,
ac_copy
};
enum actor_status {

26
test/copy.cc Normal file
View File

@@ -0,0 +1,26 @@
#include "test.h"
void testcase_copy::copy_db(const bool with_compaction) {
int err = osal_removefile(copy_pathname);
if (err != MDBX_SUCCESS && err != MDBX_ENOFILE)
failure_perror("mdbx_removefile()", err);
err = mdbx_env_copy(db_guard.get(), copy_pathname.c_str(),
with_compaction ? MDBX_CP_COMPACT : 0);
if (unlikely(err != MDBX_SUCCESS))
failure_perror(with_compaction ? "mdbx_env_copy(MDBX_CP_COMPACT)"
: "mdbx_env_copy(MDBX_CP_ASIS)",
err);
}
bool testcase_copy::run() {
jitter_delay();
db_open();
assert(!txn_guard);
const bool order = flipcoin();
jitter_delay();
copy_db(order);
jitter_delay();
copy_db(!order);
return true;
}

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -25,6 +25,7 @@ function probe {
rm -f ${TESTDB_PREFIX}* \
&& ./mdbx_test --pathname=${TESTDB_PREFIX}db "$@" | lz4 > ${TESTDB_PREFIX}log.lz4 \
&& ./mdbx_chk -nvv ${TESTDB_PREFIX}db | tee ${TESTDB_PREFIX}chk \
&& ./mdbx_chk -nvv ${TESTDB_PREFIX}db-copy | tee ${TESTDB_PREFIX}chk-copy \
|| (echo "FAILED"; exit 1)
}
@@ -52,11 +53,11 @@ for nops in {2..7}; do
caption="Probe #$((++count)) w/o-dups, repeat ${rep} of ${loops}" probe \
--pagesize=min --size=6G --table=-data.dups --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=1111 \
--nops=$( rep9 $nops ) --batch.write=$( rep9 $wbatch ) --mode=$(bits2list options $bits) \
--keygen.seed=${seed} --hill
--keygen.seed=${seed} basic
caption="Probe #$((++count)) with-dups, repeat ${rep} of ${loops}" probe \
--pagesize=min --size=6G --table=+data.dups --keylen.min=min --keylen.max=max --datalen.min=min --datalen.max=max \
--nops=$( rep9 $nops ) --batch.write=$( rep9 $wbatch ) --mode=$(bits2list options $bits) \
--keygen.seed=${seed} --hill
--keygen.seed=${seed} basic
done
done
done

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
/*
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@@ -182,11 +182,11 @@ int main(int argc, char *const argv[]) {
mdbx_limits_dbsize_min(params.pagesize),
mdbx_limits_dbsize_max(params.pagesize)))
continue;
if (config::parse_option(argc, argv, narg, "size", params.size_now,
if (config::parse_option(argc, argv, narg, "size-upper", params.size_upper,
mdbx_limits_dbsize_min(params.pagesize),
mdbx_limits_dbsize_max(params.pagesize)))
continue;
if (config::parse_option(argc, argv, narg, "size-upper", params.size_upper,
if (config::parse_option(argc, argv, narg, "size", params.size_now,
mdbx_limits_dbsize_min(params.pagesize),
mdbx_limits_dbsize_max(params.pagesize)))
continue;
@@ -337,6 +337,10 @@ int main(int argc, char *const argv[]) {
configure_actor(last_space_id, ac_deadwrite, value, params);
continue;
}
if (config::parse_option(argc, argv, narg, "copy", nullptr)) {
configure_actor(last_space_id, ac_copy, value, params);
continue;
}
if (config::parse_option(argc, argv, narg, "failfast",
global::config::failfast))
continue;

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@@ -184,7 +184,7 @@ void osal_killall_actors(void) {
int osal_actor_poll(mdbx_pid_t &pid, unsigned timeout) {
struct timespec ts;
ts.tv_nsec = 0;
ts.tv_sec = timeout;
ts.tv_sec = (timeout > INT_MAX) ? INT_MAX : timeout;
retry:
int status, options = WNOHANG;
#ifdef WUNTRACED
@@ -301,3 +301,7 @@ std::string osal_tempdir(void) {
return "/dev/shm/";
return "";
}
int osal_removefile(const std::string &pathname) {
return unlink(pathname.c_str()) ? errno : MDBX_SUCCESS;
}

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@@ -405,3 +405,7 @@ std::string osal_tempdir(void) {
DWORD len = GetTempPathA(sizeof(buf), buf);
return std::string(buf, len);
}
int osal_removefile(const std::string &pathname) {
return DeleteFileA(pathname.c_str()) ? MDBX_SUCCESS : GetLastError();
}

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@@ -32,6 +32,7 @@ void osal_udelay(unsigned us);
void osal_yield(void);
bool osal_istty(int fd);
std::string osal_tempdir(void);
int osal_removefile(const std::string &pathname);
#ifdef _MSC_VER
#ifndef STDIN_FILENO

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@@ -31,6 +31,8 @@ const char *testcase2str(const actor_testcase testcase) {
return "jitter";
case ac_try:
return "try";
case ac_copy:
return "copy";
}
}
@@ -443,6 +445,9 @@ bool test_execute(const actor_config &config) {
case ac_try:
test.reset(new testcase_try(config, pid));
break;
case ac_copy:
test.reset(new testcase_copy(config, pid));
break;
default:
test.reset(new testcase(config, pid));
break;

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*
@@ -203,3 +203,14 @@ public:
bool run();
bool teardown();
};
class testcase_copy : public testcase {
const std::string copy_pathname;
void copy_db(const bool with_compaction);
public:
testcase_copy(const actor_config &config, const mdbx_pid_t pid)
: testcase(config, pid),
copy_pathname(config.params.pathname_db + "-copy") {}
bool run();
};

View File

@@ -184,6 +184,7 @@
<ClCompile Include="cases.cc" />
<ClCompile Include="chrono.cc" />
<ClCompile Include="config.cc" />
<ClCompile Include="copy.cc" />
<ClCompile Include="dead.cc" />
<ClCompile Include="hill.cc" />
<ClCompile Include="try.cc" />

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2017-2018 Leonid Yuriev <leo@yuriev.ru>
* Copyright 2017-2019 Leonid Yuriev <leo@yuriev.ru>
* and other libmdbx authors: please see AUTHORS file.
* All rights reserved.
*

View File

@@ -4,7 +4,7 @@
*/
/*
* Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>.
* Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>.
* Copyright 2012-2015 Howard Chu, Symas Corp.
* Copyright 2015,2016 Peter-Service R&D LLC.
* All rights reserved.

View File

@@ -5,7 +5,7 @@
/*
* Copyright 2017 Ilya Shipitsin <chipitsine@gmail.com>.
* Copyright 2015-2018 Leonid Yuriev <leo@yuriev.ru>.
* Copyright 2015-2019 Leonid Yuriev <leo@yuriev.ru>.
* Copyright 2012-2015 Howard Chu, Symas Corp.
* All rights reserved.
*