mdbx: release v0.10.0
Acknowledgements:
-----------------
- [Mahlon E. Smith](https://github.com/mahlonsmith) for [Ruby bindings](https://rubygems.org/gems/mdbx/).
- [Alex Sharov](https://github.com/AskAlexSharov) for [mdbx-go](https://github.com/torquem-ch/mdbx-go), bug reporting and testing.
- [Artem Vorotnikov](https://github.com/vorot93) for bug reporting and PR.
- [Paolo Rebuffo](https://www.linkedin.com/in/paolo-rebuffo-8255766/), [Alexey Akhunov](https://github.com/AlexeyAkhunov) and Mark Grosberg for donations.
- [Noel Kuntze](https://github.com/Thermi) for preliminary [Python bindings](https://github.com/Thermi/libmdbx/tree/python-bindings)
New features:
-------------
- Added `mdbx_env_set_option()` and `mdbx_env_get_option()` for controls
various runtime options for an environment (announce of this feature was missed in a previous news).
- Added `MDBX_DISABLE_PAGECHECKS` build option to disable some checks to reduce an overhead
and detection probability of database corruption to a values closer to the LMDB.
The `MDBX_DISABLE_PAGECHECKS=1` provides a performance boost of about 10% in CRUD scenarios,
and conjointly with the `MDBX_ENV_CHECKPID=0` and `MDBX_TXN_CHECKOWNER=0` options can yield
up to 30% more performance compared to LMDB.
- Using float point (exponential quantized) representation for internal 16-bit values
of grow step and shrink threshold when huge ones (https://github.com/erthink/libmdbx/issues/166).
To minimize the impact on compatibility, only the odd values inside the upper half
of the range (i.e. 32769..65533) are used for the new representation.
- Added the `mdbx_drop` similar to LMDB command-line tool to purge or delete (sub)database(s).
- [Ruby bindings](https://rubygems.org/gems/mdbx/) is available now by [Mahlon E. Smith](https://github.com/mahlonsmith).
- Added `MDBX_ENABLE_MADVISE` build option which controls the use of POSIX `madvise()` hints and friends.
- The internal node sizes were refined, resulting in a reduction in large/overflow pages in some use cases
and a slight increase in limits for a keys size to ≈½ of page size.
- Added to `mdbx_chk` output number of keys/items on pages.
- Added explicit `install-strip` and `install-no-strip` targets to the `Makefile` (https://github.com/erthink/libmdbx/pull/180).
- Major rework page splitting (af9b7b560505684249b76730997f9e00614b8113) for
- An "auto-appending" feature upon insertion for both ascending and
descending key sequences. As a result, the optimality of page filling
increases significantly (more densely, less slackness) while
inserting ordered sequences of keys,
- A "splitting at middle" to make page tree more balanced on average.
- Added `mdbx_get_sysraminfo()` to the API.
- Added guessing a reasonable maximum DB size for the default upper limit of geometry (https://github.com/erthink/libmdbx/issues/183).
- Major rework internal labeling of a dirty pages (958fd5b9479f52f2124ab7e83c6b18b04b0e7dda) for
a "transparent spilling" feature with the gist to make a dirty pages
be ready to spilling (writing to a disk) without further altering ones.
Thus in the `MDBX_WRITEMAP` mode the OS kernel able to oust dirty pages
to DB file without further penalty during transaction commit.
As a result, page swapping and I/O could be significantly reduced during extra large transactions and/or lack of memory.
- Minimized reading leaf-pages during dropping subDB(s) and nested trees.
- Major rework a spilling of dirty pages to support [LRU](https://en.wikipedia.org/wiki/Cache_replacement_policies#Least_recently_used_(LRU))
policy and prioritization for a large/overflow pages.
- Statistics of page operations (split, merge, copy, spill, etc) now available through `mdbx_env_info_ex()`.
- Auto-setup limit for length of dirty pages list (`MDBX_opt_txn_dp_limit` option).
- Support `make options` to list available build options.
- Support `make help` to list available make targets.
- Silently `make`'s build by default.
- Preliminary [Python bindings](https://github.com/Thermi/libmdbx/tree/python-bindings) is available now
by [Noel Kuntze](https://github.com/Thermi) (https://github.com/erthink/libmdbx/issues/147).
Backward compatibility break:
-----------------------------
- The `MDBX_AVOID_CRT` build option was renamed to `MDBX_WITHOUT_MSVC_CRT`.
This option is only relevant when building for Windows.
- The `mdbx_env_stat()` always, and `mdbx_env_stat_ex()` when called with the zeroed transaction parameter,
now internally start temporary read transaction and thus may returns `MDBX_BAD_RSLOT` error.
So, just never use deprecated `mdbx_env_stat()' and call `mdbx_env_stat_ex()` with transaction parameter.
- The build option `MDBX_CONFIG_MANUAL_TLS_CALLBACK` was removed and now just a non-zero value of
the `MDBX_MANUAL_MODULE_HANDLER` macro indicates the requirement to manually call `mdbx_module_handler()`
when loading libraries and applications uses statically linked libmdbx on an obsolete Windows versions.
Fixes:
------
- Fixed performance regression due non-optimal C11 atomics usage (https://github.com/erthink/libmdbx/issues/160).
- Fixed "reincarnation" of subDB after it deletion (https://github.com/erthink/libmdbx/issues/168).
- Fixed (disallowing) implicit subDB deletion via operations on `@MAIN`'s DBI-handle.
- Fixed a crash of `mdbx_env_info_ex()` in case of a call for a non-open environment (https://github.com/erthink/libmdbx/issues/171).
- Fixed the selecting/adjustment values inside `mdbx_env_set_geometry()` for implicit out-of-range cases (https://github.com/erthink/libmdbx/issues/170).
- Fixed `mdbx_env_set_option()` for set initial and limit size of dirty page list ((https://github.com/erthink/libmdbx/issues/179).
- Fixed an unreasonably huge default upper limit for DB geometry (https://github.com/erthink/libmdbx/issues/183).
- Fixed `constexpr` specifier for the `slice::invalid()`.
- Fixed (no)readahead auto-handling (https://github.com/erthink/libmdbx/issues/164).
- Fixed non-alloy build for Windows.
- Switched to using Heap-functions instead of LocalAlloc/LocalFree on Windows.
- Fixed `mdbx_env_stat_ex()` to returning statistics of the whole environment instead of MainDB only (https://github.com/erthink/libmdbx/issues/190).
- Fixed building by GCC 4.8.5 (added workaround for a preprocessor's bug).
- Fixed building C++ part for iOS <= 13.0 (unavailability of `std::filesystem::path`).
- Fixed building for Windows target versions prior to Windows Vista (`WIN32_WINNT < 0x0600`).
- Fixed building by MinGW for Windows (https://github.com/erthink/libmdbx/issues/155).
TODO for a next releases:
-------------------------
- [Get rid of dirty-pages list in MDBX_WRITEMAP mode](https://github.com/erthink/libmdbx/issues/193).
- [Large/Overflow pages accounting for dirty-room](https://github.com/erthink/libmdbx/issues/192).
- [C++ Buffer issue](https://github.com/erthink/libmdbx/issues/191).
- Finalize C++ API (few typos and trivia bugs are still likely for now).
- [Support for RAW devices](https://github.com/erthink/libmdbx/issues/124).
- [Test framework issue](https://github.com/erthink/libmdbx/issues/127).
- [Support MessagePack for Keys & Values](https://github.com/erthink/libmdbx/issues/115).
- [Engage new terminology](https://github.com/erthink/libmdbx/issues/137).
- Packages for [Astra Linux](https://astralinux.ru/), [ALT Linux](https://www.altlinux.org/), [ROSA Linux](https://www.rosalinux.ru/), Fedora/RHEL, Debian/Ubuntu.
2021-05-09 02:57:12 +03:00
|
|
|
version: 0.10.0.{build}
|
2017-04-04 15:42:30 +03:00
|
|
|
|
|
|
|
environment:
|
|
|
|
matrix:
|
2020-08-22 20:19:46 +03:00
|
|
|
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015
|
|
|
|
CMAKE_GENERATOR: Visual Studio 14 2015
|
|
|
|
TOOLSET: 140
|
2019-08-31 17:13:02 +03:00
|
|
|
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
|
|
|
|
CMAKE_GENERATOR: Visual Studio 16 2019
|
|
|
|
TOOLSET: 142
|
2019-10-24 13:01:58 +03:00
|
|
|
MDBX_BUILD_SHARED_LIBRARY: OFF
|
2021-04-29 20:09:16 +03:00
|
|
|
MDBX_WITHOUT_MSVC_CRT: OFF
|
2019-10-24 13:01:58 +03:00
|
|
|
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
|
|
|
|
CMAKE_GENERATOR: Visual Studio 16 2019
|
|
|
|
TOOLSET: 142
|
|
|
|
MDBX_BUILD_SHARED_LIBRARY: ON
|
2021-04-29 20:09:16 +03:00
|
|
|
MDBX_WITHOUT_MSVC_CRT: ON
|
2019-10-24 13:01:58 +03:00
|
|
|
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
|
|
|
|
CMAKE_GENERATOR: Visual Studio 16 2019
|
|
|
|
TOOLSET: 142
|
|
|
|
MDBX_BUILD_SHARED_LIBRARY: OFF
|
2021-04-29 20:09:16 +03:00
|
|
|
MDBX_WITHOUT_MSVC_CRT: ON
|
2019-10-24 13:01:58 +03:00
|
|
|
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
|
|
|
|
CMAKE_GENERATOR: Visual Studio 16 2019
|
|
|
|
TOOLSET: 142
|
|
|
|
MDBX_BUILD_SHARED_LIBRARY: ON
|
2021-04-29 20:09:16 +03:00
|
|
|
MDBX_WITHOUT_MSVC_CRT: OFF
|
2017-07-05 21:40:18 +03:00
|
|
|
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
|
2019-08-31 17:13:02 +03:00
|
|
|
CMAKE_GENERATOR: Visual Studio 15 2017
|
|
|
|
TOOLSET: 141
|
2017-03-29 01:11:33 +03:00
|
|
|
|
2017-07-21 22:51:56 +03:00
|
|
|
branches:
|
|
|
|
except:
|
|
|
|
- coverity_scan
|
|
|
|
|
2017-03-29 01:11:33 +03:00
|
|
|
configuration:
|
2017-07-05 21:40:18 +03:00
|
|
|
- Debug
|
2020-09-14 03:48:59 +03:00
|
|
|
# MSVC-2019 hangs during code generation/optimization due to its own internal errors.
|
|
|
|
# I have found out that the problem occurs because of the /Ob2 option (see https://github.com/erthink/libmdbx/issues/116).
|
|
|
|
# So the simplest workaround is to using RelWithDebiInfo configuration for testing (cmake will uses /Ob1 option), instead of Release.
|
|
|
|
# - Release
|
|
|
|
- RelWithDebInfo
|
2017-07-05 21:40:18 +03:00
|
|
|
|
|
|
|
platform:
|
2019-08-31 17:13:02 +03:00
|
|
|
- Win32
|
2018-09-14 14:10:36 +03:00
|
|
|
- x64
|
2017-03-29 01:11:33 +03:00
|
|
|
|
2020-09-07 20:49:40 +03:00
|
|
|
# MSVC-2019 may hang up during code generation/optimization due to its own internal errors.
|
|
|
|
matrix:
|
|
|
|
allow_failures:
|
|
|
|
- image: Visual Studio 2019
|
|
|
|
configuration: Release
|
|
|
|
|
|
|
|
# Enable RDP for troubleshooting
|
|
|
|
init:
|
|
|
|
- ps: iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-rdp.ps1'))
|
|
|
|
|
2019-08-31 17:13:02 +03:00
|
|
|
before_build:
|
2019-10-24 13:01:58 +03:00
|
|
|
- git clean -x -f -d
|
2019-08-31 17:13:02 +03:00
|
|
|
- git submodule sync
|
2021-03-02 01:48:16 +03:00
|
|
|
- git fetch --tags --prune --force
|
2019-08-31 17:13:02 +03:00
|
|
|
- git submodule update --init --recursive
|
2021-03-02 01:48:16 +03:00
|
|
|
- git submodule foreach --recursive git fetch --tags --prune --force
|
2019-08-31 17:13:02 +03:00
|
|
|
- cmake --version
|
2017-03-29 01:11:33 +03:00
|
|
|
|
2019-08-31 17:13:02 +03:00
|
|
|
build_script:
|
2017-03-30 18:54:57 +03:00
|
|
|
- ps: |
|
2019-08-31 17:13:02 +03:00
|
|
|
Write-Output "*******************************************************************************"
|
|
|
|
Write-Output "Configuration: $env:CONFIGURATION"
|
|
|
|
Write-Output "Platform: $env:PLATFORM"
|
|
|
|
Write-Output "Toolchain: $env:CMAKE_GENERATOR v$env:TOOLSET"
|
2021-05-08 13:26:57 +03:00
|
|
|
Write-Output "Options: MDBX_WITHOUT_MSVC_CRT=$env:MDBX_WITHOUT_MSVC_CRT MDBX_BUILD_SHARED_LIBRARY=$env:MDBX_BUILD_SHARED_LIBRARY"
|
2019-08-31 17:13:02 +03:00
|
|
|
Write-Output "*******************************************************************************"
|
|
|
|
|
|
|
|
md _build -Force | Out-Null
|
|
|
|
cd _build
|
|
|
|
|
|
|
|
$generator = $env:CMAKE_GENERATOR
|
|
|
|
if ($env:TOOLSET -lt 142) {
|
|
|
|
if ($env:PLATFORM -eq "x64") {
|
|
|
|
$generator = "$generator Win64"
|
|
|
|
}
|
2021-05-08 13:26:57 +03:00
|
|
|
& cmake -G "$generator" -D CMAKE_CONFIGURATION_TYPES="Debug;Release;RelWithDebInfo" -D MDBX_WITHOUT_MSVC_CRT:BOOL=$env:MDBX_WITHOUT_MSVC_CRT -D MDBX_BUILD_SHARED_LIBRARY:BOOL=$env:MDBX_BUILD_SHARED_LIBRARY ..
|
2017-07-05 21:40:18 +03:00
|
|
|
} else {
|
2021-05-08 13:26:57 +03:00
|
|
|
& cmake -G "$generator" -A $env:PLATFORM -D CMAKE_CONFIGURATION_TYPES="Debug;Release;RelWithDebInfo" -DMDBX_WITHOUT_MSVC_CRT:BOOL=$env:MDBX_WITHOUT_MSVC_CRT -D MDBX_BUILD_SHARED_LIBRARY:BOOL=$env:MDBX_BUILD_SHARED_LIBRARY ..
|
2017-07-05 21:40:18 +03:00
|
|
|
}
|
2019-08-31 17:13:02 +03:00
|
|
|
if ($LastExitCode -ne 0) {
|
|
|
|
throw "Exec: $ErrorMessage"
|
|
|
|
}
|
|
|
|
Write-Output "*******************************************************************************"
|
|
|
|
|
|
|
|
& cmake --build . --config $env:CONFIGURATION
|
|
|
|
if ($LastExitCode -ne 0) {
|
|
|
|
throw "Exec: $ErrorMessage"
|
|
|
|
}
|
|
|
|
Write-Output "*******************************************************************************"
|
2017-07-05 21:40:18 +03:00
|
|
|
|
2019-08-31 17:13:02 +03:00
|
|
|
test_script:
|
|
|
|
- ps: |
|
|
|
|
if (($env:PLATFORM -ne "ARM") -and ($env:PLATFORM -ne "ARM64")) {
|
2019-10-02 01:21:41 +03:00
|
|
|
& ./$env:CONFIGURATION/mdbx_test.exe --progress --console=no --pathname=test.db --dont-cleanup-after basic > test.log
|
|
|
|
Get-Content test.log | Select-Object -last 42
|
2019-08-31 17:13:02 +03:00
|
|
|
if ($LastExitCode -ne 0) {
|
|
|
|
throw "Exec: $ErrorMessage"
|
|
|
|
} else {
|
2019-09-01 16:53:11 +03:00
|
|
|
& ./$env:CONFIGURATION/mdbx_chk.exe -nvv test.db | Tee-Object -file chk.log | Select-Object -last 42
|
2019-08-31 17:13:02 +03:00
|
|
|
}
|
2017-04-10 23:34:59 +03:00
|
|
|
}
|
2017-04-24 19:03:38 +03:00
|
|
|
|
2017-04-25 18:30:31 +03:00
|
|
|
on_failure:
|
2019-10-27 20:55:29 +03:00
|
|
|
- ps: Push-AppveyorArtifact \projects\libmdbx\_build\test.log
|
|
|
|
- ps: Push-AppveyorArtifact \projects\libmdbx\_build\test.db
|
|
|
|
- ps: Push-AppveyorArtifact \projects\libmdbx\_build\chk.log
|