diff --git a/README.md b/README.md index ab1ee660..90f29246 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ libmdbx _libmdbx_ is an extremely fast, compact, powerful, embedded, transactional [key-value database](https://en.wikipedia.org/wiki/Key-value_database), with [permissive license](./LICENSE). -_MDBX_ has a specific set of properties and capabilities, +_libmdbx_ has a specific set of properties and capabilities, focused on creating unique lightweight solutions. 1. Allows **a swarm of multi-threaded processes to @@ -30,9 +30,9 @@ tree](https://en.wikipedia.org/wiki/B%2B_tree). [WAL](https://en.wikipedia.org/wiki/Write-ahead_logging), but that might be a caveat for write-intensive workloads with durability requirements. -4. **Compact and friendly for fully embedding**. Only 25KLOC of `C11`, -64K x86 binary code, no internal threads neither processes, but -implements a simplified variant of the [Berkeley +4. **Compact and friendly for fully embedding**. Only ≈25KLOC of `C11`, +≈64K x86 binary code of core, no internal threads neither server process(es), +but implements a simplified variant of the [Berkeley DB](https://en.wikipedia.org/wiki/Berkeley_DB) and [dbm](https://en.wikipedia.org/wiki/DBM_(computing)) API. @@ -51,9 +51,9 @@ OpenSolaris, OpenIndiana, NetBSD, OpenBSD and other systems compliant with **POSIX.1-2008**. -Historically, _MDBX_ is a deeply revised and extended descendant of the amazing +Historically, _libmdbx_ is a deeply revised and extended descendant of the amazing [Lightning Memory-Mapped Database](https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database). -_MDBX_ inherits all benefits from _LMDB_, but resolves some issues and adds [a set of improvements](#improvements-beyond-lmdb). +_libmdbx_ inherits all benefits from _LMDB_, but resolves some issues and adds [a set of improvements](#improvements-beyond-lmdb). The next version is under active non-public development from scratch and will be @@ -161,19 +161,19 @@ transaction journal. No crash recovery needed. No maintenance is required. 1. There cannot be more than one writer at a time, i.e. no more than one write transaction at a time. -2. MDBX is based on [B+ tree](https://en.wikipedia.org/wiki/B%2B_tree), so access to database pages is mostly random. +2. _libmdbx_ is based on [B+ tree](https://en.wikipedia.org/wiki/B%2B_tree), so access to database pages is mostly random. Thus SSDs provide a significant performance boost over spinning disks for large databases. -3. MDBX uses [shadow paging](https://en.wikipedia.org/wiki/Shadow_paging) instead of [WAL](https://en.wikipedia.org/wiki/Write-ahead_logging). Thus syncing data to disk might be a bottleneck for write intensive workload. +3. _libmdbx_ uses [shadow paging](https://en.wikipedia.org/wiki/Shadow_paging) instead of [WAL](https://en.wikipedia.org/wiki/Write-ahead_logging). Thus syncing data to disk might be a bottleneck for write intensive workload. -4. MDBX uses [copy-on-write](https://en.wikipedia.org/wiki/Copy-on-write) for [snapshot isolation](https://en.wikipedia.org/wiki/Snapshot_isolation) during updates, but read transactions prevents recycling an old retired/freed pages, since it read ones. Thus altering of data during a parallel +4. _libmdbx_ uses [copy-on-write](https://en.wikipedia.org/wiki/Copy-on-write) for [snapshot isolation](https://en.wikipedia.org/wiki/Snapshot_isolation) during updates, but read transactions prevents recycling an old retired/freed pages, since it read ones. Thus altering of data during a parallel long-lived read operation will increase the process work set, may exhaust entire free database space, the database can grow quickly, and result in performance degradation. Try to avoid long running read transactions. -5. MDBX is extraordinarily fast and provides minimal overhead for data access, +5. _libmdbx_ is extraordinarily fast and provides minimal overhead for data access, so you should reconsider using brute force techniques and double check your code. -On the one hand, in the case of MDBX, a simple linear search may be more profitable than complex indexes. +On the one hand, in the case of _libmdbx_, a simple linear search may be more profitable than complex indexes. On the other hand, if you make something suboptimally, you can notice detrimentally only on sufficiently large data. ## Comparison with other databases @@ -305,7 +305,7 @@ named mutexes are used. # History -Historically, _MDBX_ is a deeply revised and extended descendant of the +Historically, _libmdbx_ is a deeply revised and extended descendant of the [Lightning Memory-Mapped Database](https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database). At first the development was carried out within the [ReOpenLDAP](https://github.com/erthink/ReOpenLDAP) project. About a @@ -318,7 +318,7 @@ and development is funded by [Positive Technologies](https://www.ptsecurity.com) ## Acknowledgments Howard Chu is the author of LMDB, from which -originated the MDBX in 2015. +originated the _libmdbx_ in 2015. Martin Hedenfalk is the author of `btree.c` code, which was used to begin development of LMDB. diff --git a/docs/Doxyfile.in b/docs/Doxyfile.in index 06829af7..9d4f2077 100644 --- a/docs/Doxyfile.in +++ b/docs/Doxyfile.in @@ -44,7 +44,7 @@ PROJECT_NUMBER = "${MDBX_VERSION_MAJOR}.${MDBX_VERSION_MINOR}.${MDBX_VER # for a project that appears at the top of each page and should give viewer a # quick idea about the purpose of the project. Keep the description short. -PROJECT_BRIEF = "One of the fastest embeddable key-value ACID database without WAL." +PROJECT_BRIEF = "One of the fastest compact embeddable key-value ACID database without WAL." # With the PROJECT_LOGO tag one can specify a logo or an icon that is included # in the documentation. The maximum height of the logo should not exceed 55 diff --git a/docs/_preface.md b/docs/_preface.md index 7f444b6f..b6361523 100644 --- a/docs/_preface.md +++ b/docs/_preface.md @@ -34,7 +34,7 @@ periodic checkpointing and/or compaction of their log or database files otherwise they grow without bound. MDBX tracks retired/freed pages within the database and re-uses them for new write operations, so the database size does not grow without bound in normal use. It is worth noting that the "next" -version libmdbx (MithrilDB) will solve this problem. +version libmdbx (\ref MithrilDB) will solve this problem. The memory map can be used as a read-only or read-write map. It is read-only by default as this provides total immunity to corruption. Using read-write diff --git a/docs/_restrictions.md b/docs/_restrictions.md index 739af82f..7bbd0765 100644 --- a/docs/_restrictions.md +++ b/docs/_restrictions.md @@ -15,10 +15,10 @@ In addition to those listed for some functions. multiple processes in a lock-free manner and any locking is unwise due to a large overhead. - The "next" version of libmdbx (MithrilDB) will solve this issue. + The "next" version of libmdbx (\ref MithrilDB) will solve this issue. \note Workaround: Just make all programs using the database close it; - the LCK-file is always reset on first open. + the LCK-file is always reset on first open. 2. Stale reader transactions left behind by an aborted program cause further writes to grow the database quickly, and stale locks can @@ -27,7 +27,7 @@ In addition to those listed for some functions. growth the database. But in some cases, this may not be enough. \note Workaround: Check for stale readers periodically, using the - `mdbx_reader_check()` function or the mdbx_stat tool. + \ref mdbx_reader_check() function or the mdbx_stat tool. 3. Stale writers will be cleared automatically by MDBX on supported platforms. But this is platform-specific, especially of @@ -36,8 +36,7 @@ In addition to those listed for some functions. Windows and FreeBSD. \note Workaround: Otherwise just make all programs using the database - close it; the LCK-file is always reset on first open - of the environment. + close it; the LCK-file is always reset on first open of the environment. ## Remote filesystems @@ -51,19 +50,19 @@ a read-only network shares. ## Child processes -Do not use opened `MDBX_env` instance(s) in a child processes after `fork()`. +Do not use opened \ref MDBX_env instance(s) in a child processes after `fork()`. It would be insane to call fork() and any MDBX-functions simultaneously from multiple threads. The best way is to prevent the presence of open MDBX-instances during `fork()`. -The `MDBX_ENV_CHECKPID` build-time option, which is ON by default on +The \ref MDBX_ENV_CHECKPID build-time option, which is ON by default on non-Windows platforms (i.e. where `fork()` is available), enables PID checking at a few critical points. But this does not give any guarantees, but only allows you to detect such errors a little sooner. Depending on the platform, you should expect an application crash and/or database corruption in such cases. -On the other hand, MDBX allow calling `mdbx_close_env()` in such cases to +On the other hand, MDBX allow calling \ref mdbx_env_close() in such cases to release resources, but no more and in general this is a wrong way. ## Read-only mode @@ -72,24 +71,24 @@ readers need write access to LCK-file to be ones visible for writer. So MDBX always tries to open/create LCK-file for read-write, but switches to without-LCK mode on appropriate errors (`EROFS`, `EACCESS`, `EPERM`) -if the read-only mode was requested by the `MDBX_RDONLY` flag which is +if the read-only mode was requested by the \ref MDBX_RDONLY flag which is described below. -The "next" version of libmdbx (MithrilDB) will solve this issue for the "many +The "next" version of libmdbx (\ref MithrilDB) will solve this issue for the "many readers without writer" case. ## One thread - One transaction A thread can only use one transaction at a time, plus any nested read-write transactions in the non-writemap mode. Each transaction - belongs to one thread. The `MDBX_NOTLS` flag changes this for read-only + belongs to one thread. The \ref MDBX_NOTLS flag changes this for read-only transactions. See below. Do not start more than one transaction for a one thread. If you think about this, it's really strange to do something with two data snapshots at once, which may be different. MDBX checks and preventing this by - returning corresponding error code (`MDBX_TXN_OVERLAPPING`, `MDBX_BAD_RSLOT`, - `MDBX_BUSY`) unless you using `MDBX_NOTLS` option on the environment. + returning corresponding error code (\ref MDBX_TXN_OVERLAPPING, \ref MDBX_BAD_RSLOT, + \ref MDBX_BUSY) unless you using \ref MDBX_NOTLS option on the environment. Nonetheless, with the `MDBX_NOTLS` option, you must know exactly what you are doing, otherwise you will get deadlocks or reading an alien data. @@ -97,7 +96,7 @@ readers without writer" case. ## Do not open twice Do not have open an MDBX database twice in the same process at the same time. By default MDBX prevent this in most cases by tracking databases -opening and return `MDBX_BUSY` if anyone LCK-file is already open. +opening and return \ref MDBX_BUSY if anyone LCK-file is already open. The reason for this is that when the "Open file description" locks (aka OFD-locks) are not available, MDBX uses POSIX locks on files, and these @@ -130,7 +129,8 @@ reasonable to simplify this as follows: discernible because of high transaction rate and intentional internals simplification in favor of performance. - 2. MDBX employs Multiversion concurrency control on the Copy-on-Write + 2. MDBX employs [Multiversion concurrency control](https://en.wikipedia.org/wiki/Multiversion_concurrency_control) + on the [Copy-on-Write](https://en.wikipedia.org/wiki/Copy-on-write) basis, that allows multiple readers runs in parallel with a write transaction without blocking. An each write transaction needs free pages to put the changed data, that pages will be placed in the new @@ -150,13 +150,11 @@ performance degradation. MDBX mostly solve "long-lived" readers issue by using the Handle-Slow-Readers \ref MDBX_hsr_func callback which allows to abort long-lived read transactions, and using the \ref MDBX_LIFORECLAIM mode which addresses subsequent performance degradation. -The "next" version of libmdbx (MithrilDB) will completely solve this. +The "next" version of libmdbx (\ref MithrilDB) will completely solve this. - Avoid suspending a process with active transactions. These would then be "long-lived" as above. - The "next" version of libmdbx (MithrilDB) will solve this issue. - - Avoid aborting a process with an active read-only transaction in scenarios with high rate of write transactions. The transaction becomes "long-lived" as above until a check for stale readers is performed or the LCK-file is diff --git a/docs/_toc.md b/docs/_toc.md index e74896c0..97c7afd8 100644 --- a/docs/_toc.md +++ b/docs/_toc.md @@ -27,4 +27,4 @@ including creating [PR](https://help.github.com/en/github/collaborating-with-iss --- -\section mithril Mithril DB +\section MithrilDB MithrilDB diff --git a/mdbx.h b/mdbx.h index e9f69839..27057342 100644 --- a/mdbx.h +++ b/mdbx.h @@ -3259,7 +3259,7 @@ LIBMDBX_API int mdbx_dbi_flags(MDBX_txn *txn, MDBX_dbi dbi, unsigned *flags); * \note Use with care. * This call is synchronized via mutex with \ref mdbx_dbi_close(), but NOT with * other transactions running by other threads. The "next" version of libmdbx - * (MithrilDB) will solve this issue. + * (\ref MithrilDB) will solve this issue. * * Handles should only be closed if no other threads are going to reference * the database handle or one of its cursors any further. Do not close a handle @@ -3562,11 +3562,6 @@ LIBMDBX_API int mdbx_del(MDBX_txn *txn, MDBX_dbi dbi, const MDBX_val *key, * which helps to avoid errors such as: use-after-free, double-free, i.e. * memory corruption and segfaults. * - * \param [in] txn A transaction handle returned by \ref mdbx_txn_begin(). - * \param [in] dbi A database handle returned by \ref mdbx_dbi_open(). - * \param [out] cursor Address where the new \ref MDBX_cursor handle will be - * stored. - * * \returns Created cursor handle or NULL in case out of memory. */ LIBMDBX_API MDBX_cursor *mdbx_cursor_create(void); @@ -3589,6 +3584,7 @@ LIBMDBX_API MDBX_cursor *mdbx_cursor_create(void); * * \param [in] txn A transaction handle returned by \ref mdbx_txn_begin(). * \param [in] dbi A database handle returned by \ref mdbx_dbi_open(). + * \param [out] cursor A cursor handle returned by \ref mdbx_cursor_create(). * * \returns A non-zero error value on failure and 0 on success, * some possible errors are: @@ -3645,7 +3641,8 @@ LIBMDBX_API int mdbx_cursor_open(MDBX_txn *txn, MDBX_dbi dbi, * which helps to avoid errors such as: use-after-free, double-free, i.e. * memory corruption and segfaults. * - * \param [in] cursor A cursor handle returned by mdbx_cursor_open(). */ + * \param [in] cursor A cursor handle returned by \ref mdbx_cursor_open() + * or \ref mdbx_cursor_create(). */ LIBMDBX_API void mdbx_cursor_close(MDBX_cursor *cursor); /** \brief Renew a cursor handle. diff --git a/mdbx.h++ b/mdbx.h++ index 31f14e2b..0d20ff5e 100644 --- a/mdbx.h++ +++ b/mdbx.h++ @@ -2051,7 +2051,7 @@ public: /// \note Use with care. /// This call is synchronized via mutex with other calls \ref close_map(), but /// NOT with other transactions running by other threads. The "next" version - /// of libmdbx (MithrilDB) will solve this issue. + /// of libmdbx (\ref MithrilDB) will solve this issue. /// /// Handles should only be closed if no other threads are going to reference /// the database handle or one of its cursors any further. Do not close a @@ -2127,10 +2127,13 @@ public: /// \brief Starts read (read-only) transaction. inline txn_managed start_read() const; + /// \brief Creates but not start read transaction. inline txn_managed prepare_read() const; + /// \brief Starts write (read-write) transaction. inline txn_managed start_write(bool dont_wait = false); + /// \brief Tries to start write (read-write) transaction without blocking. inline txn_managed try_start_write(); }; diff --git a/src/options.h b/src/options.h index c7ee36a4..6ca8a1e7 100644 --- a/src/options.h +++ b/src/options.h @@ -61,7 +61,7 @@ #ifndef MDBX_ENV_CHECKPID #if defined(MADV_DONTFORK) || defined(_WIN32) || defined(_WIN64) /* PID check could be omitted: - * - on Linux when madvise(MADV_DONTFORK) is available. i.e. after the fork() + * - on Linux when madvise(MADV_DONTFORK) is available, i.e. after the fork() * mapped pages will not be available for child process. * - in Windows where fork() not available. */ #define MDBX_ENV_CHECKPID 0