Commit Graph

2697 Commits

Author SHA1 Message Date
Erk
b2a0279253
Fix compilation on build targets listed below MIPS 2021-03-04 10:33:12 +01:00
Leonid Yuriev
46dcd6e7ca mdbx: (re)alloc dpl-list at txn start.
Change-Id: Ie35221666f09a189f5513e77d3ecb4056968334f
2021-03-02 03:58:27 +03:00
Leonid Yuriev
8cabd99d24 mdbx: merge branch 'devel'.
Change-Id: I2593f0229694d0bdd310975887e31651ead6dcc0
2021-03-02 02:12:10 +03:00
Leonid Yuriev
c29d3a4ecb mdbx-ci: add --force for all git fetch --tags.
Change-Id: I016e412250dca651fdae65b08aba5d5e204e61b8
2021-03-02 02:01:35 +03:00
Leonid Yuriev
c4b24b6a4d mdbx-tools: minor fix a condition for details output about leaf pages.
Thanks to Alexander Gavrilov (https://github.com/dartraiden) for the hint.

Change-Id: I2a674b7d23310f8c776eef551353f79dfca7a3d8
2021-03-01 23:09:18 +03:00
Leonid Yuriev
c8dccc9bc4 mdbx: limits the initial size of dpl-list to the current db-size.
Change-Id: I5f575fc6168f50786b6f8a82ae020d323530a12e
2021-02-28 01:02:16 +03:00
Leonid Yuriev
72d978ee48 mdbx: weakens checks during set the MDBX_opt_txn_dp_limit to avoid MDBX_EINVAL.
Change-Id: I4852261d0c45b726c60792463ab698538fa447e5
2021-02-28 01:02:16 +03:00
Leonid Yuriev
7fcf11013e mdbx: minor fix likely/unlikely inside mdbx_cursor_del().
Change-Id: I86cfc755eef7371ea96c0feb39bffd3ec5298b71
2021-02-28 01:02:16 +03:00
Leonid Yuriev
d816e0605b mdbx-ci: fix spelling github-action error.
Change-Id: I93c6bfdc59e95fa69f1f440c7bd32f4db3fe6547
2021-02-08 20:29:12 +03:00
Leonid Yuriev
34dcb410a9 mdbx: add reference to Go bindings.
Change-Id: Ib50012e30c0ab98d97024b7268b159aeeb883363
2021-02-07 14:32:27 +03:00
Leonid Yuriev
2727072ad9 mdbx: update README, now we can say - up to 30% faster than LMDB.
Change-Id: I902282d9032ce5d7ea8a95547f02da13d593a42f
2021-02-07 03:14:49 +03:00
Leonid Yuriev
e7dfa98030 mdbx: update Changelog.
Change-Id: Iea4a2cf27c808ed9bbb4e3359050065ac73f3eda
2021-02-07 02:53:54 +03:00
Leonid Yuriev
3fd739ea2c mdbx: add MDBX_DISABLE_PAGECHECKS build option.
Change-Id: Ia5e2727b74961e7f829f273225c5c1a6cc764b8e
2021-02-07 02:53:54 +03:00
Leonid Yuriev
c89f30e485 mdbx: fix/rework C11 atomics usage to avoid performance regression.
Resolve https://github.com/erthink/libmdbx/issues/160

Change-Id: Ic14ef8c9f1e4fb844952f51e81b58268d7939cfe
2021-02-06 21:16:56 +03:00
Leonid Yuriev
016f644815 mdbx: merge branch 'master' into devel.
Change-Id: I04863923fd5a3a4294a96190477ad28562c805fc
2021-02-06 19:36:17 +03:00
Leonid Yuriev
7e7d526ed5 mdbx: minor fix txn_spill().
Change-Id: Ic624681d3a9e262b77f761796aabf0231019db72
2021-02-06 19:34:26 +03:00
Leonid Yuriev
0e8c913c57 mdbx: minor refine node_search().
Change-Id: Ib9aaca3e7853b9986b6d70bc43a88e90d0a46c0c
2021-02-06 19:33:07 +03:00
Leonid Yuriev
892402a5d8 mdbx: add missing unlikely() hints.
Change-Id: I0baae944b9d2a994ad65fbb57591c2cc8d35f0d3
2021-02-06 19:31:04 +03:00
Leonid Yuriev
0cc695e22e mdbx: minor simplify page_get().
Change-Id: I2a548ea008fad29da8382946dff3c7d9bffa1e85
2021-02-06 19:30:05 +03:00
Leonid Yuriev
cc7c41a9c0 mdbx-ci: use --force to avoid errors while fetching flags.
Change-Id: I8ba31e2f6cfd9b396d6706c71ed0ff42493f7fe9
2021-02-04 19:24:23 +03:00
Leonid Yuriev
d78150de79 mdbx-ci: hotfix for github-action+git issue.
Change-Id: I235a9d5a55c284aa545fc2a18fb7cb8411747f89
2021-02-02 22:34:42 +03:00
Leonid Yuriev
be220dc905 mdbx: release v0.9.3
Acknowledgements:
-----------------
 - [Mahlon E. Smith](http://www.martini.nu/) for [FreeBSD port of libmdbx](https://svnweb.freebsd.org/ports/head/databases/mdbx/).
 - [장세연](http://www.castis.com) for bug fixing and PR.
 - [Clément Renault](https://github.com/Kerollmops/heed) for [Heed](https://github.com/Kerollmops/heed) fully typed Rust wrapper.
 - [Alex Sharov](https://github.com/AskAlexSharov) for bug reporting.
 - [Noel Kuntze](https://github.com/Thermi) for bug reporting.

Removed options and features:
-----------------------------
 - Drop `MDBX_HUGE_TRANSACTIONS` build-option (now no longer required).

New features:
-------------
 - Package for FreeBSD is available now by Mahlon E. Smith.
 - New API functions to get/set various options (https://github.com/erthink/libmdbx/issues/128):
    - the maximum number of named databases for the environment;
    - the maximum number of threads/reader slots;
    - threshold (since the last unsteady commit) to force flush the data buffers to disk;
    - relative period (since the last unsteady commit) to force flush the data buffers to disk;
    - limit to grow a list of reclaimed/recycled page's numbers for finding a sequence of contiguous pages for large data items;
    - limit to grow a cache of dirty pages for reuse in the current transaction;
    - limit of a pre-allocated memory items for dirty pages;
    - limit of dirty pages for a write transaction;
    - initial allocation size for dirty pages list of a write transaction;
    - maximal part of the dirty pages may be spilled when necessary;
    - minimal part of the dirty pages should be spilled when necessary;
    - how much of the parent transaction dirty pages will be spilled while start each child transaction;
 - Unlimited/Dynamic size of retired and dirty page lists (https://github.com/erthink/libmdbx/issues/123).
 - Added `-p` option (purge subDB before loading) to `mdbx_load` tool.
 - Reworked spilling of large transaction and committing of nested transactions:
    - page spilling code reworked to avoid the flaws and bugs inherited from LMDB;
    - limit for number of dirty pages now is controllable at runtime;
    - a spilled pages, including overflow/large pages, now can be reused and refunded/compactified in nested transactions;
    - more effective refunding/compactification especially for the loosed page cache.
 - Added `MDBX_ENABLE_REFUND` and `MDBX_PNL_ASCENDING` internal/advanced build options.
 - Added `mdbx_default_pagesize()` function.
 - Better support architectures with a weak/relaxed memory consistency model (ARM, AARCH64, PPC, MIPS, RISC-V, etc) by means [C11 atomics](https://en.cppreference.com/w/c/atomic).
 - Speed up page number lists and dirty page lists (https://github.com/erthink/libmdbx/issues/132).
 - Added `LIBMDBX_NO_EXPORTS_LEGACY_API` build option.

Fixes:
------
 - Fixed missing cleanup (null assigned) in the C++ commit/abort (https://github.com/erthink/libmdbx/pull/143).
 - Fixed `mdbx_realloc()` for case of nullptr and `MDBX_AVOID_CRT=ON` for Windows.
 - Fixed the possibility to use invalid and renewed (closed & re-opened, dropped & re-created) DBI-handles (https://github.com/erthink/libmdbx/issues/146).
 - Fixed 4-byte aligned access to 64-bit integers, including access to the `bootid` meta-page's field (https://github.com/erthink/libmdbx/issues/153).
 - Fixed minor/potential memory leak during page flushing and unspilling.
 - Fixed handling states of cursors's and subDBs's for nested transactions.
 - Fixed page leak in extra rare case the list of retired pages changed during update GC on transaction commit.
 - Fixed assertions to avoid false-positive UB detection by CLANG/LLVM (https://github.com/erthink/libmdbx/issues/153).
 - Fixed `MDBX_TXN_FULL` and regressive `MDBX_KEYEXIST` during large transaction commit with `MDBX_LIFORECLAIM` (https://github.com/erthink/libmdbx/issues/123).
 - Fixed auto-recovery (`weak->steady` with the same boot-id) when Database size at last weak checkpoint is large than at last steady checkpoint.
 - Fixed operation on systems with unusual small/large page size, including PowerPC (https://github.com/erthink/libmdbx/issues/157).

TODO:
-----
 - Engage new terminology (https://github.com/erthink/libmdbx/issues/137).
 - Resolve few TODOs (https://github.com/erthink/libmdbx/issues/124, https://github.com/erthink/libmdbx/issues/127,    https://github.com/erthink/libmdbx/issues/115).
 - Finalize C++ API.
 - Packages for [ROSA Linux](https://www.rosalinux.ru/), [ALT Linux](https://www.altlinux.org/), Fedora/RHEL, Debian/Ubuntu.

Change-Id: I414b8ef2e4b90e04fb344779c0e3f1b4bd1c06be
2021-02-02 22:18:21 +03:00
Leonid Yuriev
8133d93678 mdbx: add LIBMDBX_NO_EXPORTS_LEGACY_API build option.
Change-Id: I9d9dcffe63fce843e22132c3829adca57714d818
2021-02-02 00:25:48 +03:00
Leonid Yuriev
38a559b93e mdbx: Crutches for C11-atomics compiler's bugs.
Change-Id: I5a857aa56c831f00c4bc67c95d7aecabc29264da
2021-02-01 20:55:41 +03:00
Leonid Yuriev
2b290e08ea mdbx: update ChangeLog.
Change-Id: Ib7af5490a2d8138bdd65403df3f0552e68e30425
2021-02-01 02:14:28 +03:00
Leonid Yuriev
25c4df0d3e mdbx: radix sort for large chunks of PNL and DPL.
More for https://github.com/erthink/libmdbx/issues/132

Change-Id: I19b253f78069d4ecd4ec360a12121c78f182fc09
2021-02-01 02:14:13 +03:00
Leonid Yuriev
88bdf4b96f mdbx: avoid branches inside bsearch() loop.
Related for https://github.com/erthink/libmdbx/issues/132

Change-Id: Ia843556cc7052e5081a98f56b43fd80e2d0a40c7
2021-02-01 00:39:52 +03:00
Leonid Yuriev
28bd5d81d2 mdbx: incremental lazy merge sort for dirty-page list.
The main change for https://github.com/erthink/libmdbx/issues/132

Change-Id: I9907f6abfcf77bd40c7a263f5adb5274d5445864
2021-02-01 00:39:19 +03:00
Leonid Yuriev
0620ec2f2e mdbx: merge branch master into the devel.
Change-Id: Iab3a0502577f6557e3f8fccc89df49aee372416e
2021-01-31 23:21:16 +03:00
Leonid Yuriev
077989bfed mdbx: adds pre- and post- gaps to dirty page list to avoid some comparisons and conditional branches.
More for https://github.com/erthink/libmdbx/issues/132

Change-Id: I6562c5ff6c559341bb7bb64222b126f06cc13427
2021-01-31 23:20:49 +03:00
Leonid Yuriev
b57a338546 mdbx-test: minor refactor mdbx_logger().
Change-Id: I058be7e01b5fc031610363bf6747ae7f62ba4c67
2021-01-31 23:20:49 +03:00
Leonid Yuriev
edb6d2d661 mdbx: add audit after nested txn start.
Change-Id: I7601994d24e7d2a5b8e9b113f697e4a14b1d2e6a
2021-01-31 23:20:49 +03:00
Leonid Yuriev
df180d1d36 mdbx: provide the gap/reserve for ability of merge sort dirty-page list.
Preparation for fix https://github.com/erthink/libmdbx/issues/132

Change-Id: I4aac6927878eb0361588fbfcf9698ba6d2d5fc34
2021-01-31 23:16:10 +03:00
Leonid Yuriev
8b6608a9ca mdbx: fix dirtyroom inheritance for nested transactions.
Change-Id: If2036ab3b7909c51e54526a535d65cdee1e0b287
2021-01-31 21:49:20 +03:00
Leonid Yuriev
9f0ff865e8 mdbx: use C11 atomics if available instead of legacy memory barriers.
This done better support architectures with a weak/relaxed memory consistency model (ARM, AARCH64, PPC, MIPS, RISC-V, etc).

Change-Id: Iee831c8dc564f1d027ff84b0d6daa559325d5a9b
2021-01-30 02:28:12 +03:00
Leonid Yuriev
bc33875a9e mdbx-test: more iterations and steps for long-stochastic scripts. 2021-01-30 02:28:04 +03:00
Leonid Yuriev
9ae054caf6 mdbx: update ChangeLog.
Change-Id: Ifb831c86e79d3180943d978d316ef0e7b691c3b8
2021-01-29 21:54:24 +03:00
Leonid Yuriev
f698f07ff9 mdbx: fix operation for unusual small/large system page size.
More for https://github.com/erthink/libmdbx/issues/157

Change-Id: I4f2ed54b50653d0375538b82c48590d1037cd93b
2021-01-29 21:19:58 +03:00
Leonid Yuriev
6cfac546a1 mdbx: fix mdbx_env_get_option(MDBX_opt_sync_bytes) to return number of bytes instead of pages.
Change-Id: I815887e82946d9644b05549ec253a52b566c8565
2021-01-29 19:55:40 +03:00
Leonid Yuriev
4704e3a966 mdbx-ci: add GithubCI badge status to the README.
Change-Id: I33b2e63e7903c7d744dd40921524c3f9ce6fb17e
2021-01-29 19:55:40 +03:00
Leonid Yuriev
9c9f6faf38 mdbx: fix madvise_threshold for large system pages.
Fixed https://github.com/erthink/libmdbx/issues/157

Change-Id: I0e880591d236b6ba96a38d2a3fa953695d9c4df0
2021-01-29 04:58:32 +03:00
Leonid Yuriev
7e2de94e1f mdbx-make: show dist-verification build error(s) on failure.
Change-Id: I78f54341b10dd605eb37ceaa19c355df2e7e1707
2021-01-29 04:38:13 +03:00
Leonid Yuriev
0a2f2e28b4 mdbx: fix minor signed/unsigned comparison warnings.
Change-Id: Ice6f9759560d5eda514112e838d57e63207a7559
2021-01-27 19:23:07 +03:00
Leonid Yuriev
62b2e31bf4 mdbx: fix auto-recovery weak->steady with the same boot-id (don't truncate DB to size from steady-meta).
Change-Id: Ib1c7f845bdf49fb9101b01aa868f8567172e5590
2021-01-27 16:48:03 +03:00
Leonid Yuriev
86db3670aa mdbx-doc: use multiple Doxygen's groups for cursor functions (haven't effect with current Doxygen).
Change-Id: I00263bfd19588e83410f449886b7eefce4bcf0e1
2021-01-26 22:19:14 +03:00
Leonid Yuriev
9b69eed5ce mdbx-doc: update ChangeLog.
Change-Id: I350611d4fc83d2cf764ee1d3bedcf9c86f1a7e26
2021-01-26 22:19:14 +03:00
Leonid Yuriev
aaaa54a397 mdbx-doc: add link to heed Rust wrapper.
Change-Id: I1201bd062941d7cbeaafe0d0fae51ef36cd73e5a
2021-01-26 22:19:14 +03:00
Leonid Yuriev
7ae5ddefe5 mdbx-doc: fix typo and add refs to c_crud_hints anchor.
Change-Id: I46b26ee6f9ca4e1f06c573b61a42d8fbedefd836
2021-01-26 22:18:28 +03:00
Leonid Yuriev
943bb552a2 mdbx: fix GC-slot cleanup inside mdbx_update_gc() in extra rare complex cases.
More for 5f1b684719 and https://github.com/erthink/libmdbx/issues/123

Change-Id: I1b78d6be2babf9ffdd0b77e7b1ee53d1313977bb
2021-01-26 10:28:39 +03:00
Leonid Yuriev
251eda6fb8 mdbx: HNY!
Change-Id: I86c7028001b349e3d06b837dc8e2f7e60fb3b1a2
2021-01-26 07:27:13 +03:00