Commit Graph

85 Commits

Author SHA1 Message Date
Leonid Yuriev
70933d81a8
mdbx: release v0.10.2
Acknowledgements:
-----------------
 - [Alex Sharov](https://github.com/AskAlexSharov) for reporting and testing.
 - [Andrea Lanfranchi](https://github.com/AndreaLanfranchi) for reporting bugs.
 - [Lionel Debroux](https://github.com/debrouxl) for fuzzing tests and reporting bugs.
 - [Sergey Fedotov](https://github.com/SergeyFromHell/) for [`node-mdbx` NodeJS bindings](https://www.npmjs.com/package/node-mdbx).
 - [Kris Zyp](https://github.com/kriszyp) for [`lmdbx-store` NodeJS bindings](https://github.com/kriszyp/lmdbx-store).
 - [Noel Kuntze](https://github.com/Thermi) for [draft Python bindings](https://github.com/erthink/libmdbx/commits/python-bindings).

New features, extensions and improvements:
------------------------------------------
 - [Allow to predefine/override `MDBX_BUILD_TIMESTAMP` for builds reproducibility](https://github.com/erthink/libmdbx/issues/201).
 - Added options support for `long-stochastic` script.
 - Avoided `MDBX_TXN_FULL` error for large transactions when possible.
 - The `MDBX_READERS_LIMIT` increased to `32767`.
 - Raise `MDBX_TOO_LARGE` under Valgrind/ASAN if being opened DB is 100 larger than RAM (to avoid hangs and OOM).
 - Minimized the size of poisoned/unpoisoned regions to avoid Valgrind/ASAN stuck.
 - Added more workarounds for QEMU for testing builds for 32-bit platforms, Alpha and Sparc architectures.
 - `mdbx_chk` now skips iteration & checking of DB' records if corresponding page-tree is corrupted (to avoid `SIGSEGV`, ASAN failures, etc).
 - Added more checks for [rare/fuzzing corruption cases](https://github.com/erthink/libmdbx/issues/217).

Backward compatibility break:
-----------------------------
 - Use file `VERSION.txt` for version information instead of `VERSION` to avoid collision with `#include <version>`.
 - Rename `slice::from/to_FOO_bytes()` to `slice::envisage_from/to_FOO_length()'.
 - Rename `MDBX_TEST_EXTRA` make's variable to `MDBX_SMOKE_EXTRA`.
 - Some details of the C++ API have been changed for subsequent freezing.

Fixes:
------
 - Fixed excess meta-pages checks in case `mdbx_chk` is called to check the DB for a specific meta page and thus could prevent switching to the selected meta page, even if the check passed without errors.
 - Fixed [recursive use of SRW-lock on Windows cause by `MDBX_NOTLS` option](https://github.com/erthink/libmdbx/issues/203).
 - Fixed [log a warning during a new DB creation](https://github.com/erthink/libmdbx/issues/205).
 - Fixed [false-negative `mdbx_cursor_eof()` result](https://github.com/erthink/libmdbx/issues/207).
 - Fixed [`make install` with non-GNU `install` utility (OSX, BSD)](https://github.com/erthink/libmdbx/issues/208).
 - Fixed [installation by `CMake` in special cases by complete use `GNUInstallDirs`'s variables](https://github.com/erthink/libmdbx/issues/209).
 - Fixed [C++ Buffer issue with `std::string` and alignment](https://github.com/erthink/libmdbx/issues/191).
 - Fixed `safe64_reset()` for platforms without atomic 64-bit compare-and-swap.
 - Fixed hang/shutdown on big-endian platforms without `__cxa_thread_atexit()`.
 - Fixed [using bad meta-pages if DB was partially/recoverable corrupted](https://github.com/erthink/libmdbx/issues/217).
 - Fixed extra `noexcept` for `buffer::&assign_reference()`.
 - Fixed `bootid` generation on Windows for case of change system' time.
 - Fixed [test framework keygen-related issue](https://github.com/erthink/libmdbx/issues/127).
2021-07-26 05:23:52 +03:00
Leonid Yuriev
c2fa453725 mdbx-test: fix keygen-related issue.
Fixes https://github.com/erthink/libmdbx/issues/127
2021-07-26 03:52:46 +03:00
Leonid Yuriev
9a04c9a350 mdbx: update available Bindings list. 2021-07-21 17:39:34 +03:00
Leonid Yuriev
13912be35d mdbx: update ChangeLog. 2021-07-21 13:37:42 +03:00
Leonid Yuriev
9a1dffc7dc mdbx: update ChangeLog. 2021-07-19 13:21:47 +03:00
Leonid Yuriev
77a1f32e2a mdbx: use VERSION.txt instead of VERSION to avoid collision with #include <version>. 2021-06-24 14:59:29 +03:00
Leonid Yuriev
63e7276c7d mdbx: update ChangeLog. 2021-06-18 15:13:51 +03:00
Leonid Yuriev
18bc28bea2 mdbx: announce more TODO. 2021-06-10 02:47:40 +03:00
Leonid Yuriev
93e6e64eb0 mdbx: update ChangeLog. 2021-06-09 13:48:03 +03:00
Leonid Yuriev
0e83a8e5ef mdbx-doc: add note for Reproducible builds.
More for https://github.com/erthink/libmdbx/issues/201
2021-06-02 17:08:40 +03:00
Leonid Yuriev
f951f246b8 mdbx: allow to predefine/override MDBX_BUILD_TIMESTAMP for builds reproducibility.
Resolve https://github.com/erthink/libmdbx/issues/201
2021-06-02 14:50:22 +03:00
Leonid Yuriev
50328de63c
mdbx: release v0.10.1
Acknowledgements:
-----------------
 - [Alexey Akhunov](https://github.com/AlexeyAkhunov) and [Alex Sharov](https://github.com/AskAlexSharov) for bug reporting and testing.

New features:
-------------
 - Added `-p` option to `mdbx_stat` utility for printing page operations statistic.
 - Added explicit checking for and warning about using unfit github's archives.

Fixes:
------
 - Fixed minor "foo not used" warnings from modern C++ compilers when building the C++ part of the library.
 - Fixed confusing/messy errors when build library from unfit github's archives (https://github.com/erthink/libmdbx/issues/197).
 - Fixed `#​e​l​s​i​f` typo.
 - Fixed rare unexpected `MDBX_PROBLEM` error during altering data in huge transactions due to wrong spilling/oust of dirty pages (https://github.com/erthink/libmdbx/issues/195).
2021-06-01 03:21:59 +03:00
Leonid Yuriev
a6c8c20bd9 mdbx: update ChangeLog. 2021-05-28 01:46:24 +03:00
Leonid Yuriev
2b161db6d8 mdbx: update ChangeLog. 2021-05-21 00:12:55 +03:00
Leonid Yuriev
daac459797 mdbx: update ChangeLog for last fixes.
Change-Id: I9e9a56c02212cb2bd698fb0dddb42ecd127e3936
2021-05-09 00:53:42 +03:00
Leonid Yuriev
db4e2cec9c mdbx: update Changelog (preparation for v0.10.0 release).
Change-Id: I1be5f6395f2f2d2ea9354e4af47de2339434364e
2021-05-07 23:10:41 +03:00
Leonid Yuriev
d7d8725ca9 mdbx: update Changelog.
Change-Id: If1dc3f5337ef88d55d500b21632345040289efb0
2021-05-04 23:22:20 +03:00
Leonid Yuriev
65919abd9a mdbx: update Changelog.
Change-Id: I543c1681d1886a54d7c24ea0bca024e82039541e
2021-05-03 01:16:37 +03:00
Leonid Yuriev
dfeb6b5acb mdbx-make: support for make help to list available make targets.
Change-Id: I72b92101a494fc7b507f54259c3db69990ce5a2d
2021-05-01 11:55:53 +03:00
Leonid Yuriev
70b7ec0c1c mdbx-make: support for make options to list available build options.
Change-Id: Ib153834241e33c672867aa402d3fc66a06a489fa
2021-04-30 14:05:36 +03:00
Leonid Yuriev
d27e48ae1b mdbx-windows: rename MDBX_AVOID_CRT build option to MDBX_DISABLE_MSVC_CRT.
Change-Id: I124c9d86d4a596718bd9671a7c1b538b84c2dd0f
2021-04-30 01:59:48 +03:00
Leonid Yuriev
b71206d119 mdbx: update Changelog.
Change-Id: I2277465a701afebbe512ca45335ce90179b945bd
2021-03-19 19:13:22 +03:00
Leonid Yuriev
b164baa1f5 mdbx: add MDBX_ENABLE_MADVISE build option/macro.
Change-Id: I6174ec62e4811e891663b8ae0f7918aa09baf5ab
2021-03-17 01:18:18 +03:00
Leonid Yuriev
0e0682ff7a mdbx: add link to Ruby bindings.
Change-Id: Idc3f23bec7fc4673cd614eb64a6b3788ae7acd92
2021-03-15 03:36:51 +03:00
Leonid Yuriev
55620c1d13 mdbx: fix selecting/adjustment values inside mdbx_env_set_geometry() for implicit out-of-range cases.
Fixes https://github.com/erthink/libmdbx/issues/170.

Change-Id: Ibc8754811aa3d9077447220f9fec7a5543e40cc4
2021-03-13 19:55:19 +03:00
Leonid Yuriev
e43cf081f1 mdbx: fix mdbx_env_info_ex() for a non-open environment case.
Fix the https://github.com/erthink/libmdbx/issues/171.

Change-Id: Iad3d0186ef40dfd40fad1c3ae0dd97f9c1161c16
2021-03-13 17:15:17 +03:00
Leonid Yuriev
a96b6f79c6 mdbx: fix/disallowing implicit subDB deletion via operations on MAIN_DBI.
Change-Id: I3cd786e877f42cef2c0d5556033b2633f8a8ce62
2021-03-13 17:11:35 +03:00
Leonid Yuriev
f01e0efc2d mdbx-tools: add mdbx_drop tool.
Change-Id: Ib7b32668c13fcef5951ff7250df57b3263e14d69
2021-03-13 15:45:15 +03:00
Leonid Yuriev
8e51a10908 mdbx: packing the 16-bit representations of grow step and shrink threshold values.
Using float point (exponential quantized) representation for internal 16-bit values
of grow step and shrink threshold when huge ones
.
To minimize the impact on compatibility, only the odd values inside the upper half
of the range (i.e. 32769..65533) are used for the new representation.

Resolve https://github.com/erthink/libmdbx/issues/166

Change-Id: I273127c1842deef0d7d8885b55a805b1463556eb
2021-03-09 12:54:48 +03:00
Leonid Yuriev
57af1d2310 mdbx: update ChangeLog.
Change-Id: I675f6a215eb566c89dc3f0f875752999b43fd4b6
2021-03-07 19:46:56 +03:00
Leonid Yuriev
34dcb410a9 mdbx: add reference to Go bindings.
Change-Id: Ib50012e30c0ab98d97024b7268b159aeeb883363
2021-02-07 14:32:27 +03:00
Leonid Yuriev
e7dfa98030 mdbx: update Changelog.
Change-Id: Iea4a2cf27c808ed9bbb4e3359050065ac73f3eda
2021-02-07 02:53:54 +03:00
Leonid Yuriev
be220dc905 mdbx: release v0.9.3
Acknowledgements:
-----------------
 - [Mahlon E. Smith](http://www.martini.nu/) for [FreeBSD port of libmdbx](https://svnweb.freebsd.org/ports/head/databases/mdbx/).
 - [장세연](http://www.castis.com) for bug fixing and PR.
 - [Clément Renault](https://github.com/Kerollmops/heed) for [Heed](https://github.com/Kerollmops/heed) fully typed Rust wrapper.
 - [Alex Sharov](https://github.com/AskAlexSharov) for bug reporting.
 - [Noel Kuntze](https://github.com/Thermi) for bug reporting.

Removed options and features:
-----------------------------
 - Drop `MDBX_HUGE_TRANSACTIONS` build-option (now no longer required).

New features:
-------------
 - Package for FreeBSD is available now by Mahlon E. Smith.
 - New API functions to get/set various options (https://github.com/erthink/libmdbx/issues/128):
    - the maximum number of named databases for the environment;
    - the maximum number of threads/reader slots;
    - threshold (since the last unsteady commit) to force flush the data buffers to disk;
    - relative period (since the last unsteady commit) to force flush the data buffers to disk;
    - limit to grow a list of reclaimed/recycled page's numbers for finding a sequence of contiguous pages for large data items;
    - limit to grow a cache of dirty pages for reuse in the current transaction;
    - limit of a pre-allocated memory items for dirty pages;
    - limit of dirty pages for a write transaction;
    - initial allocation size for dirty pages list of a write transaction;
    - maximal part of the dirty pages may be spilled when necessary;
    - minimal part of the dirty pages should be spilled when necessary;
    - how much of the parent transaction dirty pages will be spilled while start each child transaction;
 - Unlimited/Dynamic size of retired and dirty page lists (https://github.com/erthink/libmdbx/issues/123).
 - Added `-p` option (purge subDB before loading) to `mdbx_load` tool.
 - Reworked spilling of large transaction and committing of nested transactions:
    - page spilling code reworked to avoid the flaws and bugs inherited from LMDB;
    - limit for number of dirty pages now is controllable at runtime;
    - a spilled pages, including overflow/large pages, now can be reused and refunded/compactified in nested transactions;
    - more effective refunding/compactification especially for the loosed page cache.
 - Added `MDBX_ENABLE_REFUND` and `MDBX_PNL_ASCENDING` internal/advanced build options.
 - Added `mdbx_default_pagesize()` function.
 - Better support architectures with a weak/relaxed memory consistency model (ARM, AARCH64, PPC, MIPS, RISC-V, etc) by means [C11 atomics](https://en.cppreference.com/w/c/atomic).
 - Speed up page number lists and dirty page lists (https://github.com/erthink/libmdbx/issues/132).
 - Added `LIBMDBX_NO_EXPORTS_LEGACY_API` build option.

Fixes:
------
 - Fixed missing cleanup (null assigned) in the C++ commit/abort (https://github.com/erthink/libmdbx/pull/143).
 - Fixed `mdbx_realloc()` for case of nullptr and `MDBX_AVOID_CRT=ON` for Windows.
 - Fixed the possibility to use invalid and renewed (closed & re-opened, dropped & re-created) DBI-handles (https://github.com/erthink/libmdbx/issues/146).
 - Fixed 4-byte aligned access to 64-bit integers, including access to the `bootid` meta-page's field (https://github.com/erthink/libmdbx/issues/153).
 - Fixed minor/potential memory leak during page flushing and unspilling.
 - Fixed handling states of cursors's and subDBs's for nested transactions.
 - Fixed page leak in extra rare case the list of retired pages changed during update GC on transaction commit.
 - Fixed assertions to avoid false-positive UB detection by CLANG/LLVM (https://github.com/erthink/libmdbx/issues/153).
 - Fixed `MDBX_TXN_FULL` and regressive `MDBX_KEYEXIST` during large transaction commit with `MDBX_LIFORECLAIM` (https://github.com/erthink/libmdbx/issues/123).
 - Fixed auto-recovery (`weak->steady` with the same boot-id) when Database size at last weak checkpoint is large than at last steady checkpoint.
 - Fixed operation on systems with unusual small/large page size, including PowerPC (https://github.com/erthink/libmdbx/issues/157).

TODO:
-----
 - Engage new terminology (https://github.com/erthink/libmdbx/issues/137).
 - Resolve few TODOs (https://github.com/erthink/libmdbx/issues/124, https://github.com/erthink/libmdbx/issues/127,    https://github.com/erthink/libmdbx/issues/115).
 - Finalize C++ API.
 - Packages for [ROSA Linux](https://www.rosalinux.ru/), [ALT Linux](https://www.altlinux.org/), Fedora/RHEL, Debian/Ubuntu.

Change-Id: I414b8ef2e4b90e04fb344779c0e3f1b4bd1c06be
2021-02-02 22:18:21 +03:00
Leonid Yuriev
8133d93678 mdbx: add LIBMDBX_NO_EXPORTS_LEGACY_API build option.
Change-Id: I9d9dcffe63fce843e22132c3829adca57714d818
2021-02-02 00:25:48 +03:00
Leonid Yuriev
2b290e08ea mdbx: update ChangeLog.
Change-Id: Ib7af5490a2d8138bdd65403df3f0552e68e30425
2021-02-01 02:14:28 +03:00
Leonid Yuriev
9f0ff865e8 mdbx: use C11 atomics if available instead of legacy memory barriers.
This done better support architectures with a weak/relaxed memory consistency model (ARM, AARCH64, PPC, MIPS, RISC-V, etc).

Change-Id: Iee831c8dc564f1d027ff84b0d6daa559325d5a9b
2021-01-30 02:28:12 +03:00
Leonid Yuriev
9ae054caf6 mdbx: update ChangeLog.
Change-Id: Ifb831c86e79d3180943d978d316ef0e7b691c3b8
2021-01-29 21:54:24 +03:00
Leonid Yuriev
f698f07ff9 mdbx: fix operation for unusual small/large system page size.
More for https://github.com/erthink/libmdbx/issues/157

Change-Id: I4f2ed54b50653d0375538b82c48590d1037cd93b
2021-01-29 21:19:58 +03:00
Leonid Yuriev
9b69eed5ce mdbx-doc: update ChangeLog.
Change-Id: I350611d4fc83d2cf764ee1d3bedcf9c86f1a7e26
2021-01-26 22:19:14 +03:00
Leonid Yuriev
f2b9babfd3 mdbx: update ChangeLog.
Change-Id: I429b8ca8400910ad5a25c7159d5060f0d0f1843f
2021-01-26 01:37:32 +03:00
Leonid Yuriev
4e13d1239a mdbx: 'runtime options' feature release.
Resolve https://github.com/erthink/libmdbx/issues/128

Change-Id: I67411f623ecb917d3be1cdc6aa78b879d8e68b4c
2021-01-23 01:13:20 +03:00
Leonid Yuriev
0861a0652b mdbx: update ChangeLog.
Change-Id: I22490e504e503661dff13e6ca4abc75ac5ebc7b6
2021-01-21 22:45:28 +03:00
Leonid Yuriev
a906569c58 mdbx: merge branch 'master' into devel-rebase.
Change-Id: I96d944f283029b9c60e45683ee66b4564273b407
2021-01-19 23:51:52 +03:00
Leonid Yuriev
b98895b8c7 mdbx: fix unaligned access to bootid on the meta-pages.
Assume this fixes https://github.com/erthink/libmdbx/issues/153

Change-Id: I062c1bd99795a7b875d1cf60658790c1071a6775
2021-01-18 14:27:58 +03:00
Leonid Yuriev
e766df658c mdbx-backport: avoid 4-byte aligned (i.e. unaligned) access to 64-bit integers.
Historically, the page header provides 4-byte data alignment.
Therefore, unfortunately, the meta page data is also aligned on a 4-byte boundary, but contains 64-bit values.

This commit eliminates potentially unsafe access (SPARC, MIPS, etc) to these 64-bit values aligned on a 4-byte boundary.
Thus, a build with the `-fsanitize=undefined` now passes the tests both with CLANG 11 and GCC 10.

Change-Id: Ie441103e53ed96fd40507d8c0be0689e3fee69f5
2021-01-18 14:08:49 +03:00
Leonid Yuriev
70e76bcb4d mdbx: merge branch 'master' into devel.
Change-Id: Idc8838922081cbc8ebe6c564555e4d304b529588
2020-12-17 21:52:35 +03:00
Leonid Yuriev
1ebc1e7c49 mdbx: update ChangeLog.
Change-Id: Ib53b3180b2b8e5437eec649aa010bdd6779d41c9
2020-12-17 17:42:35 +03:00
Leonid Yuriev
3e7459b428 mdbx-windows: fix mdbx_realloc() for nullptr and MDBX_AVOID_CRT=ON.
Change-Id: I129221186d65254da5b1d84747e5c59d53864b70
2020-12-05 09:45:03 +03:00
Leonid Yuriev
d9bd306da3 mdbx-load: add -p option (purge subDB).
Change-Id: I678950c99c8b4aea1add4ce548c9b4fe4ab8bfe6
2020-12-03 21:52:22 +03:00
Leonid Yuriev
90309ec0bf mdbx: merge branch 'master' into devel.
Change-Id: Ib1d07cf6eb2e4c76b3be969b28bf2317cb326835
2020-12-03 18:42:10 +03:00