Commit Graph

3055 Commits

Author SHA1 Message Date
Leonid Yuriev
16d686bc42 mdbx: account msync/fsync write in page operations. 2021-05-12 14:40:58 +03:00
Leonid Yuriev
137f80e177 mdbx: refile rollback-writes inside mdbx_setup_dxb(). 2021-05-12 14:40:58 +03:00
Leonid Yuriev
7d8099fc77 mdbx: refine mdbx_wipe_steady(). 2021-05-12 14:40:58 +03:00
Leonid Yuriev
5731ad11c8 mdbx: don't use bool flags for mdbx_msync(). 2021-05-12 14:40:58 +03:00
Leonid Yuriev
23e11e3a57 mdbx: refine/fix MDBX_MAYBE_UNUSED for modern GCC 11.x and C20. 2021-05-12 14:40:58 +03:00
Leonid Yuriev
76b9cb5dcc mdbx-tools: minor refine built-in usage/help. 2021-05-12 11:33:25 +03:00
Leonid Yuriev
9746dd20df mdbx-tools: refine/fix quiet mode. 2021-05-12 11:33:25 +03:00
Leonid Yuriev
dc9869f1a1 mdbx-ci: minor fix 'mingw' workflow (prev commit). 2021-05-11 21:25:16 +03:00
Leonid Yuriev
1d9f495c46 mdbx-ci: build both dll and static library by MinGW for testing.
Related to https://github.com/erthink/libmdbx/issues/196
2021-05-11 21:25:16 +03:00
Leonid Yuriev
ebab75642e mdbx: add public MDBX_MAYBE_UNUSED.
This also should fix C++ mdbx API build by MSVC 2015.
2021-05-11 21:07:40 +03:00
Leonid Yuriev
ed58ff9f81 mdbx++: add [[maybe_unused]] to internal functions which depends on platform and compiler features. 2021-05-11 16:53:47 +03:00
Leonid Yuriev
605326b6e8 mdbx-cmake: don't specify filename extension while linking with ntdll. 2021-05-11 08:07:42 +03:00
Leonid Yuriev
2071958f35 mdbx: remove extra notice/debug logging. 2021-05-11 00:24:01 +03:00
Leonid Yuriev
4fee14d1cf mdbx: logging all reasons of MDBX_PROBLEM.
Related to https://github.com/erthink/libmdbx/issues/195
2021-05-10 16:15:58 +03:00
Leonid Yuriev
244fab6c04 mdbx: fix extra conversion of an error during nested dupsort-DB update to MDBX_PROBLEM.
Hope this fixes https://github.com/erthink/libmdbx/issues/195
2021-05-10 16:15:03 +03:00
Leonid Yuriev
74ea79ac1b mdbx: minor reorder/shrink transaction fields. 2021-05-10 16:05:19 +03:00
Leonid Yuriev
0d100a898f mdbx: minor more const. 2021-05-10 15:53:07 +03:00
Leonid Yuriev
0a28b28da3 mdbx-doc: actualize key size limits. 2021-05-09 23:29:55 +03:00
Леонид Юрьев (Leonid Yuriev)
aa1f6fbd5f
mdbx: release v0.10.0
Acknowledgements:
-----------------
 - [Mahlon E. Smith](https://github.com/mahlonsmith) for [Ruby bindings](https://rubygems.org/gems/mdbx/).
 - [Alex Sharov](https://github.com/AskAlexSharov) for [mdbx-go](https://github.com/torquem-ch/mdbx-go), bug reporting and testing.
 - [Artem Vorotnikov](https://github.com/vorot93) for bug reporting and PR.
 - [Paolo Rebuffo](https://www.linkedin.com/in/paolo-rebuffo-8255766/), [Alexey Akhunov](https://github.com/AlexeyAkhunov) and Mark Grosberg for donations.
 - [Noel Kuntze](https://github.com/Thermi) for preliminary [Python bindings](https://github.com/Thermi/libmdbx/tree/python-bindings)

New features:
-------------
 - Added `mdbx_env_set_option()` and `mdbx_env_get_option()` for controls
   various runtime options for an environment (announce of this feature  was missed in a previous news).
 - Added `MDBX_DISABLE_PAGECHECKS` build option to disable some checks to reduce an overhead
   and detection probability of database corruption to a values closer to the LMDB.
   The `MDBX_DISABLE_PAGECHECKS=1` provides a performance boost of about 10% in CRUD scenarios,
   and conjointly with the `MDBX_ENV_CHECKPID=0` and `MDBX_TXN_CHECKOWNER=0` options can yield
   up to 30% more performance compared to LMDB.
 - Using float point (exponential quantized) representation for internal 16-bit values
   of grow step and shrink threshold when huge ones (https://github.com/erthink/libmdbx/issues/166).
   To minimize the impact on compatibility, only the odd values inside the upper half
   of the range (i.e. 32769..65533) are used for the new representation.
 - Added the `mdbx_drop` similar to LMDB command-line tool to purge or delete (sub)database(s).
 - [Ruby bindings](https://rubygems.org/gems/mdbx/) is available now by [Mahlon E. Smith](https://github.com/mahlonsmith).
 - Added `MDBX_ENABLE_MADVISE` build option which controls the use of POSIX `madvise()` hints and friends.
 - The internal node sizes were refined, resulting in a reduction in large/overflow pages in some use cases
   and a slight increase in limits for a keys size to ≈½ of page size.
 - Added to `mdbx_chk` output number of keys/items on pages.
 - Added explicit `install-strip` and `install-no-strip` targets to the `Makefile` (https://github.com/erthink/libmdbx/pull/180).
 - Major rework page splitting (af9b7b5605) for
     - An "auto-appending" feature upon insertion for both ascending and
       descending key sequences. As a result, the optimality of page filling
       increases significantly (more densely, less slackness) while
       inserting ordered sequences of keys,
     - A "splitting at middle" to make page tree more balanced on average.
 - Added `mdbx_get_sysraminfo()` to the API.
 - Added guessing a reasonable maximum DB size for the default upper limit of geometry (https://github.com/erthink/libmdbx/issues/183).
 - Major rework internal labeling of a dirty pages (958fd5b947) for
   a "transparent spilling" feature with the gist to make a dirty pages
   be ready to spilling (writing to a disk) without further altering ones.
   Thus in the `MDBX_WRITEMAP` mode the OS kernel able to oust dirty pages
   to DB file without further penalty during transaction commit.
   As a result, page swapping and I/O could be significantly reduced during extra large transactions and/or lack of memory.
 - Minimized reading leaf-pages during dropping subDB(s) and nested trees.
 - Major rework a spilling of dirty pages to support [LRU](https://en.wikipedia.org/wiki/Cache_replacement_policies#Least_recently_used_(LRU))
   policy and prioritization for a large/overflow pages.
 - Statistics of page operations (split, merge, copy, spill, etc) now available through `mdbx_env_info_ex()`.
 - Auto-setup limit for length of dirty pages list (`MDBX_opt_txn_dp_limit` option).
 - Support `make options` to list available build options.
 - Support `make help` to list available make targets.
 - Silently `make`'s build by default.
 - Preliminary [Python bindings](https://github.com/Thermi/libmdbx/tree/python-bindings) is available now
   by [Noel Kuntze](https://github.com/Thermi) (https://github.com/erthink/libmdbx/issues/147).

Backward compatibility break:
-----------------------------
 - The `MDBX_AVOID_CRT` build option was renamed to `MDBX_WITHOUT_MSVC_CRT`.
   This option is only relevant when building for Windows.
 - The `mdbx_env_stat()` always, and `mdbx_env_stat_ex()` when called with the zeroed transaction parameter,
   now internally start temporary read transaction and thus may returns `MDBX_BAD_RSLOT` error.
   So, just never use deprecated `mdbx_env_stat()' and call `mdbx_env_stat_ex()` with transaction parameter.
 - The build option `MDBX_CONFIG_MANUAL_TLS_CALLBACK` was removed and now just a non-zero value of
   the `MDBX_MANUAL_MODULE_HANDLER` macro indicates the requirement to manually call `mdbx_module_handler()`
   when loading libraries and applications uses statically linked libmdbx on an obsolete Windows versions.

Fixes:
------
 - Fixed performance regression due non-optimal C11 atomics usage (https://github.com/erthink/libmdbx/issues/160).
 - Fixed "reincarnation" of subDB after it deletion (https://github.com/erthink/libmdbx/issues/168).
 - Fixed (disallowing) implicit subDB deletion via operations on `@MAIN`'s DBI-handle.
 - Fixed a crash of `mdbx_env_info_ex()` in case of a call for a non-open environment (https://github.com/erthink/libmdbx/issues/171).
 - Fixed the selecting/adjustment values inside `mdbx_env_set_geometry()` for implicit out-of-range cases (https://github.com/erthink/libmdbx/issues/170).
 - Fixed `mdbx_env_set_option()` for set initial and limit size of dirty page list ((https://github.com/erthink/libmdbx/issues/179).
 - Fixed an unreasonably huge default upper limit for DB geometry (https://github.com/erthink/libmdbx/issues/183).
 - Fixed `constexpr` specifier for the `slice::invalid()`.
 - Fixed (no)readahead auto-handling (https://github.com/erthink/libmdbx/issues/164).
 - Fixed non-alloy build for Windows.
 - Switched to using Heap-functions instead of LocalAlloc/LocalFree on Windows.
 - Fixed `mdbx_env_stat_ex()` to returning statistics of the whole environment instead of MainDB only (https://github.com/erthink/libmdbx/issues/190).
 - Fixed building by GCC 4.8.5 (added workaround for a preprocessor's bug).
 - Fixed building C++ part for iOS <= 13.0 (unavailability of  `std::filesystem::path`).
 - Fixed building for Windows target versions prior to Windows Vista (`WIN32_WINNT < 0x0600`).
 - Fixed building by MinGW for Windows (https://github.com/erthink/libmdbx/issues/155).

TODO for a next releases:
-------------------------
 - [Get rid of dirty-pages list in MDBX_WRITEMAP mode](https://github.com/erthink/libmdbx/issues/193).
 - [Large/Overflow pages accounting for dirty-room](https://github.com/erthink/libmdbx/issues/192).
 - [C++ Buffer issue](https://github.com/erthink/libmdbx/issues/191).
 - Finalize C++ API (few typos and trivia bugs are still likely for now).
 - [Support for RAW devices](https://github.com/erthink/libmdbx/issues/124).
 - [Test framework issue](https://github.com/erthink/libmdbx/issues/127).
 - [Support MessagePack for Keys & Values](https://github.com/erthink/libmdbx/issues/115).
 - [Engage new terminology](https://github.com/erthink/libmdbx/issues/137).
 - Packages for [Astra Linux](https://astralinux.ru/), [ALT Linux](https://www.altlinux.org/), [ROSA Linux](https://www.rosalinux.ru/), Fedora/RHEL, Debian/Ubuntu.
2021-05-09 03:01:59 +03:00
Leonid Yuriev
daac459797 mdbx: update ChangeLog for last fixes.
Change-Id: I9e9a56c02212cb2bd698fb0dddb42ecd127e3936
2021-05-09 00:53:42 +03:00
Leonid Yuriev
43f1823cdc mdbx: minor alignment fix for lck-less stub to avoid false-positive alarm from UndefinedBehaviorSanitize.
Change-Id: I8d9bc00ccfc7228cb42f9419cf77e8eadb92f918
2021-05-08 22:43:16 +03:00
Leonid Yuriev
7c45f75010 mdbx-test: update Valgrind's suppressions for actual function names. 2021-05-08 20:02:09 +03:00
Leonid Yuriev
3531164fed mdbx-make: fix escaping of quotation mark character. 2021-05-08 20:02:09 +03:00
Leonid Yuriev
e75033b1f4 mdbx: drop some unused stuff.
Change-Id: I136dcf891757d5c1a35a8082ee90e8cf07c016c6
2021-05-08 18:40:14 +03:00
Leonid Yuriev
cc46f398ed mdbx-ci: add MinGW build.
Resolves Related to https://github.com/erthink/libmdbx/issues/155

Change-Id: I4419c4e494139e644ff0ed755ce4560000099d82
2021-05-08 18:39:35 +03:00
Leonid Yuriev
ecd3c06932 mdbx-windows: remove MDBX_CONFIG_MANUAL_TLS_CALLBACK build option and add MDBX_MANUAL_MODULE_HANDLER macro.
Briefly:
 - Now constructor/destructor of "Thread Local Storage" handled automatically when possible.
 - Otherwise the MDBX_CONFIG_MANUAL_TLS_CALLBACK macro defined to 1 to indicate that mdbx_module_handle() should be called manually.
 - Corresponding build option MDBX_CONFIG_MANUAL_TLS_CALLBACK was removed.

Related to https://github.com/erthink/libmdbx/issues/155

Change-Id: Ic4e6a34b44f874676f0ab212ff473460e3d80559
2021-05-08 18:39:18 +03:00
Leonid Yuriev
3aa8701c6d mdbx-ci: fix AppVeyor config after new name MDBX_WITHOUT_MSVC_CRT.
Related to https://github.com/erthink/libmdbx/issues/155

Change-Id: Ib9976ccb4bd03e52b4c8cdbbf9f6b45f367c6b3d
2021-05-08 18:39:06 +03:00
Leonid Yuriev
11a521f10a mdbx-windows: minor fix mdbx_free() for MDBX_WITHOUT_MSVC_CRT=ON.
Related to https://github.com/erthink/libmdbx/issues/155

Change-Id: I378d9c47b51c55e1e61bc7f6269cbc8182265870
2021-05-08 18:38:53 +03:00
Leonid Yuriev
5ae120af80 mdbx-windows: more crutches for MinGW.
More for https://github.com/erthink/libmdbx/issues/155

Change-Id: I7de6122ff160372b2dcfd2a0a26e332cb52d0560
2021-05-08 18:38:44 +03:00
Leonid Yuriev
33e8b19ea4 mdbx-windows: check _WIN32_WINNT definition.
Related to https://github.com/erthink/libmdbx/issues/155

Change-Id: I49c294bd0fa055026b742a12a6f6ea9cd805cf02
2021-05-08 18:38:36 +03:00
Leonid Yuriev
ab1fc94a5b mdbx-windows: fix build for Windows XP/2000 (_WIN32_WINNT < 0x0600).
Related to https://github.com/erthink/libmdbx/issues/155

Change-Id: Ibd795817e05b6da39ef270ce7b55b31d963d07b0
2021-05-08 18:38:26 +03:00
Leonid Yuriev
8e078fb708 mdbx++: fix using std::filesystem::path for iOS <= 13.0
Change-Id: I8f75e0698acfb9e0da325f5a3c1e2a513cb43e2e
2021-05-08 00:21:12 +03:00
Leonid Yuriev
db4e2cec9c mdbx: update Changelog (preparation for v0.10.0 release).
Change-Id: I1be5f6395f2f2d2ea9354e4af47de2339434364e
2021-05-07 23:10:41 +03:00
Leonid Yuriev
c15ad25073 mdbx: add minor workaround for E2K LCC's bug.
Related to https://bugs.mcst.ru/bugzilla/show_bug.cgi?id=6011

Change-Id: I562dbcedb3c172e0646dcb03ef6d5cd3f14e02b9
2021-05-07 19:05:10 +03:00
Leonid Yuriev
a14d6bcb11 mdbx: fix mis-defined MDBX_FORCE_ASSERTIONS.
Change-Id: I6f7ad4fdd6aca99a4f908d634af0015a6e95c0d1
2021-05-06 17:03:57 +03:00
Leonid Yuriev
1275bdb623 mdbx: add MDBX_opt_merge_threshold_16dot16_percent option.
Change-Id: I416f85096e4b6fd21f2db622f07f31103cb9074a
2021-05-06 02:05:33 +03:00
Leonid Yuriev
009e3d6c0f mdbx-doc: describing the caveat of large values & huge transactions with a huge number of retired pages.
Resolves https://github.com/erthink/libmdbx/issues/177

Change-Id: Id5f643f2ee78894293f73c3ad1789dc0a99ecffb
2021-05-05 01:36:21 +03:00
Leonid Yuriev
d7d8725ca9 mdbx: update Changelog.
Change-Id: If1dc3f5337ef88d55d500b21632345040289efb0
2021-05-04 23:22:20 +03:00
Leonid Yuriev
7d54518d60 mdbx: add workaround for GCC 4.8 preprocessor bug.
Change-Id: I965cfbcd43596ef896a6b849dd043804bdb3f41f
2021-05-03 15:00:53 +03:00
Leonid Yuriev
d6e67e3982 mdbx: minor retine options.h
Change-Id: Ic340e6cea25df3e5dddf920ba851e39e714033e0
2021-05-03 15:00:53 +03:00
Leonid Yuriev
c914d417d2 mdbx: refine Doxygen's API description.
Change-Id: Ifd326eded287c68c6a95b6c9be22847d6efa5678
2021-05-03 15:00:53 +03:00
Leonid Yuriev
65919abd9a mdbx: update Changelog.
Change-Id: I543c1681d1886a54d7c24ea0bca024e82039541e
2021-05-03 01:16:37 +03:00
Leonid Yuriev
df387ad943 mdbx: fix minor MSVC warnings.
Change-Id: I9a2c774f6020bbe5c77e423a1f54110b12bdcb41
2021-05-02 17:11:50 +03:00
Leonid Yuriev
732d3cd2d4 mdbx: fix/rework mdbx_env_stat_ex() to return stat of the whole env.
Resolves https://github.com/erthink/libmdbx/issues/190

Included fix for the first version of this change.

Change-Id: I8357de24fda2ab4c93510574055b52d6e16b0c15
2021-05-02 17:10:37 +03:00
Leonid Yuriev
a56c72d190 mdbx: minor refine internal audit.
Change-Id: I0e177e9c9ecf1da97084c175f80b3e987391194c
2021-05-02 15:14:30 +03:00
Leonid Yuriev
9ba8d5892a mdbx-tools: avoid use mdbx_env_stat_ex().
Change-Id: I484ead640967b3a6caa3c2536983a34956ae1827
2021-05-01 21:20:18 +03:00
Leonid Yuriev
fe04c98327 mdbx-tools: cosmetic refine mdbx_chk.
Change-Id: I815baf437d9e0af3c78f9df717b0fb669863993c
2021-05-01 20:40:52 +03:00
Leonid Yuriev
3e5dbb42f5 mdbx-make: build silently by default.
Resolves https://github.com/erthink/libmdbx/issues/165

Change-Id: I52e80f079989c2bc8d6cdfea5205d2a28a27512b
2021-05-01 19:34:46 +03:00
Leonid Yuriev
dfeb6b5acb mdbx-make: support for make help to list available make targets.
Change-Id: I72b92101a494fc7b507f54259c3db69990ce5a2d
2021-05-01 11:55:53 +03:00
Leonid Yuriev
547c8dd3f2 mdbx: remove obsolete Travis-CI badge from README.
Change-Id: I095c953e1727cfe7f26309170b1214f521c508d6
2021-04-30 20:38:49 +03:00