ISAR/libmdbx
3
0
Fork 0
mirror of https://github.com/isar/libmdbx.git synced 2024-12-30 03:24:13 +08:00
Code Issues Packages Projects Releases Wiki Activity

mdbx-doc: describing the caveat of large values & huge transactions with a huge number of retired pages.

Browse Source 
Resolves https://github.com/erthink/libmdbx/issues/177

Change-Id: Id5f643f2ee78894293f73c3ad1789dc0a99ecffb
This commit is contained in:
Leonid Yuriev 2021-05-05 01:29:21 +03:00
parent d7d8725ca9
commit 009e3d6c0f
1 changed files with 28 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

28
docs/_restrictions.md
View File

@ -162,6 +162,34 @@ The "next" version of libmdbx (\ref MithrilDB) will completely solve this.
not apply to write transactions if the system clears stale writers, see not apply to write transactions if the system clears stale writers, see
above. above.
## Large data items and huge transactions
MDBX allows you to store values up to 1 gigabyte in size, but this is
not the main functionality for a key-value storage, but an additional
feature that should not be abused. Such long values are stored in
consecutive/adjacent DB pages, which has both pros and cons. This allows
you to read long values directly without copying and without any
overhead from a linear section of memory.
On the other hand, when putting such values in the database, it is
required to find a sufficient number of free consecutive/adjacent
database pages, which can be very difficult and expensive, moreover
sometimes impossible since b-tree tends to fragmentation. So, when
placing very long values, the engine may need to process the entire GC,
and in the absence of a sufficient sequence of free pages, increase the
DB file. Thus, for long values, MDBX provides maximum read performance
at the expense of write performance.
A similar situation can be with huge transactions, in which a lot of
database pages are retired. The retired pages should be put into GC as a
list of page numbers for future reuse. But in huge transactions, such a
list of retired page numbers can also be huge, i.e. it is a very long
value and requires a long sequence of free pages to be saved. Thus, if
you delete large amounts of information from the database in a single
transaction, MDBX may need to increase the database file to save the
list of pages to be retired.
Both of these issues will be addressed in MithrilDB.
## Space reservation ## Space reservation
An MDBX database configuration will often reserve considerable unused An MDBX database configuration will often reserve considerable unused