diff --git a/COPYRIGHT b/COPYRIGHT index fe4825af..842d1677 100644 --- a/COPYRIGHT +++ b/COPYRIGHT @@ -1,4 +1,24 @@ -Copyright 2011-2016 Howard Chu, Symas Corp. +Copyright (c) 2015,2016 Leonid Yuriev . +Copyright (c) 2015,2016 Peter-Service R&D LLC. + +This file is part of ReOpenMDBX. + +ReOpenMDBX is free software; you can redistribute it and/or modify it under +the terms of the GNU Affero General Public License as published by +the Free Software Foundation; either version 3 of the License, or +(at your option) any later version. + +ReOpenMDBX is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU Affero General Public License for more details. + +You should have received a copy of the GNU Affero General Public License +along with this program. If not, see . + +--- + +Copyright 2011-2014 Howard Chu, Symas Corp. All rights reserved. Redistribution and use in source and binary forms, with or without diff --git a/LICENSE b/LICENSE index 05ad7571..210c8aa8 100644 --- a/LICENSE +++ b/LICENSE @@ -1,3 +1,667 @@ + GNU AFFERO GENERAL PUBLIC LICENSE + Version 3, 19 November 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU Affero General Public License is a free, copyleft license for +software and other kinds of works, specifically designed to ensure +cooperation with the community in the case of network server software. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +our General Public Licenses are intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + Developers that use our General Public Licenses protect your rights +with two steps: (1) assert copyright on the software, and (2) offer +you this License which gives you legal permission to copy, distribute +and/or modify the software. + + A secondary benefit of defending all users' freedom is that +improvements made in alternate versions of the program, if they +receive widespread use, become available for other developers to +incorporate. Many developers of free software are heartened and +encouraged by the resulting cooperation. However, in the case of +software used on network servers, this result may fail to come about. +The GNU General Public License permits making a modified version and +letting the public access it on a server without ever releasing its +source code to the public. + + The GNU Affero General Public License is designed specifically to +ensure that, in such cases, the modified source code becomes available +to the community. It requires the operator of a network server to +provide the source code of the modified version running there to the +users of that server. Therefore, public use of a modified version, on +a publicly accessible server, gives the public access to the source +code of the modified version. + + An older license, called the Affero General Public License and +published by Affero, was designed to accomplish similar goals. This is +a different license, not a version of the Affero GPL, but Affero has +released a new version of the Affero GPL which permits relicensing under +this license. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU Affero General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Remote Network Interaction; Use with the GNU General Public License. + + Notwithstanding any other provision of this License, if you modify the +Program, your modified version must prominently offer all users +interacting with it remotely through a computer network (if your version +supports such interaction) an opportunity to receive the Corresponding +Source of your version by providing access to the Corresponding Source +from a network server at no charge, through some standard or customary +means of facilitating copying of software. This Corresponding Source +shall include the Corresponding Source for any work covered by version 3 +of the GNU General Public License that is incorporated pursuant to the +following paragraph. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the work with which it is combined will remain governed by version +3 of the GNU General Public License. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU Affero General Public License from time to time. Such new versions +will be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU Affero General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU Affero General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU Affero General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU Affero General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Affero General Public License for more details. + + You should have received a copy of the GNU Affero General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If your software can interact with users remotely through a computer +network, you should also make sure that it provides a way for users to +get its source. For example, if your program is a web application, its +interface could display a "Source" link that leads users to an archive +of the code. There are many ways you could offer source, and different +solutions will be better for different programs; see section 13 for the +specific requirements. + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU AGPL, see +. + +--- + The OpenLDAP Public License Version 2.8, 17 August 2003 diff --git a/Makefile b/Makefile index 438d4f39..978cb21b 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,5 @@ -# GNU Makefile for libmdbx (lightning memory-mapped database library for Linux). +# GNU Makefile for libmdbx (reliable lightning memory-mapped DB library for Linux). +# https://github.com/ReOpen/libmdbx ######################################################################## # Configuration. The compiler options must enable threaded compilation. @@ -6,58 +7,69 @@ # Preprocessor macros (for XCFLAGS) of interest... # Note that the defaults should already be correct for most # platforms; you should not need to change any of these. -# Read their descriptions in mdb.c if you do: +# Read their descriptions in mdb.c if you do. There may be +# other macros of interest. You should read mdb.c +# before changing any of them. # -# - MDB_USE_ROBUST -# -# There may be other macros in mdb.c of interest. You should -# read mdb.c before changing any of them. -# -DESTDIR ?= + +# install sandbox +SANDBOX ?= + +# install prefixes (inside sandbox) prefix ?= /usr/local mandir ?= $(prefix)/man +# lib/bin suffix for multiarch/biarch, e.g. '.x86_64' +suffix ?= + CC ?= gcc XCFLAGS ?= -DNDEBUG=1 -DMDB_DEBUG=0 CFLAGS ?= -O2 -g3 -Wall -Werror -Wextra CFLAGS += -pthread $(XCFLAGS) +# LY: for ability to built with modern glibc, +# but then run with the old +LDOPS ?= -Wl,--no-as-needed,-lrt + +# LY: just for benchmarking IOARENA ?= ../ioarena.git/@BUILD/src/ioarena ######################################################################## -IHDRS := lmdb.h mdbx.h -ILIBS := libmdbx.a libmdbx.so -IPROGS := mdbx_stat mdbx_copy mdbx_dump mdbx_load mdbx_chk -IDOCS := mdb_stat.1 mdb_copy.1 mdb_dump.1 mdb_load.1 -PROGS := $(IPROGS) mtest0 mtest1 mtest2 mtest3 mtest4 mtest5 mtest6 mtest7 mtest8 wbench -PROGS += yota_test1 yota_test2 +HEADERS := lmdb.h mdbx.h +LIBRARIES := libmdbx.a libmdbx.so +TOOLS := mdbx_stat mdbx_copy mdbx_dump mdbx_load mdbx_chk +MANPAGES := mdb_stat.1 mdb_copy.1 mdb_dump.1 mdb_load.1 +TESTS := mtest0 mtest1 mtest2 mtest3 mtest4 mtest5 mtest6 wbench \ + yota_test1 yota_test2 mtest7 mtest8 -SRC_LMDB := mdb.c midl.c lmdb.h midl.h reopen.h barriers.h -SRC_MDBX := $(SRC_LMDB) mdbx.h +SRC_LMDB := mdb.c midl.c lmdb.h midl.h reopen.h barriers.h +SRC_MDBX := $(SRC_LMDB) mdbx.h .PHONY: mdbx lmdb all install clean check tests coverage -all: $(ILIBS) $(IPROGS) +all: $(LIBRARIES) $(TOOLS) mdbx: libmdbx.a libmdbx.so lmdb: liblmdb.a liblmdb.so -install: $(ILIBS) $(IPROGS) $(IHDRS) - mkdir -p $(DESTDIR)$(prefix)/bin \ - && cp -t $(DESTDIR)$(prefix)/bin $(IPROGS) && \ - mkdir -p $(DESTDIR)$(prefix)/lib \ - && cp -t $(DESTDIR)$(prefix)/lib $(ILIBS) && \ - mkdir -p $(DESTDIR)$(prefix)/include \ - && cp -t $(DESTDIR)$(prefix)/include $(IHDRS) && \ - mkdir -p $(DESTDIR)$(mandir)/man1 \ - && cp -t $(DESTDIR)$(mandir)/man1 $(IDOCS) +tools: $(TOOLS) + +install: $(LIBRARIES) $(TOOLS) $(HEADERS) + mkdir -p $(SANDBOX)$(prefix)/bin$(suffix) \ + && cp -t $(SANDBOX)$(prefix)/bin$(suffix) $(TOOLS) && \ + mkdir -p $(SANDBOX)$(prefix)/lib$(suffix) \ + && cp -t $(SANDBOX)$(prefix)/lib$(suffix) $(LIBRARIES) && \ + mkdir -p $(SANDBOX)$(prefix)/include \ + && cp -t $(SANDBOX)$(prefix)/include $(HEADERS) && \ + mkdir -p $(SANDBOX)$(mandir)/man1 \ + && cp -t $(SANDBOX)$(mandir)/man1 $(MANPAGES) clean: - rm -rf $(PROGS) @* *.[ao] *.[ls]o *~ testdb/* *.gcov + rm -rf $(TOOLS )$(TESTS) @* *.[ao] *.[ls]o *~ testdb/* *.gcov -tests: mdbx $(PROGS) +tests: $(TESTS) check: tests [ -d testdb ] || mkdir testdb && rm -f testdb/* \ @@ -76,28 +88,28 @@ libmdbx.a: mdbx.o $(AR) rs $@ $^ libmdbx.so: mdbx.lo - $(CC) $(CFLAGS) $(LDFLAGS) -save-temps -pthread -shared -o $@ $^ + $(CC) $(CFLAGS) $(LDFLAGS) -save-temps -pthread -shared $(LDOPS) -o $@ $^ liblmdb.a: lmdb.o $(AR) rs $@ $^ liblmdb.so: lmdb.lo - $(CC) $(CFLAGS) $(LDFLAGS) -pthread -shared -o $@ $^ + $(CC) $(CFLAGS) $(LDFLAGS) -pthread -shared $(LDOPS) -o $@ $^ mdbx_stat: mdb_stat.o mdbx.o - $(CC) $(CFLAGS) $(LDFLAGS) -o $@ $^ + $(CC) $(CFLAGS) $(LDFLAGS) $(LDOPS) -o $@ $^ mdbx_copy: mdb_copy.o mdbx.o - $(CC) $(CFLAGS) $(LDFLAGS) -o $@ $^ + $(CC) $(CFLAGS) $(LDFLAGS) $(LDOPS) -o $@ $^ mdbx_dump: mdb_dump.o mdbx.o - $(CC) $(CFLAGS) $(LDFLAGS) -o $@ $^ + $(CC) $(CFLAGS) $(LDFLAGS) $(LDOPS) -o $@ $^ mdbx_load: mdb_load.o mdbx.o - $(CC) $(CFLAGS) $(LDFLAGS) -o $@ $^ + $(CC) $(CFLAGS) $(LDFLAGS) $(LDOPS) -o $@ $^ mdbx_chk: mdb_chk.o mdbx.o - $(CC) $(CFLAGS) $(LDFLAGS) -o $@ $^ -Wl,--no-as-needed,-lrt + $(CC) $(CFLAGS) $(LDFLAGS) $(LDOPS) -o $@ $^ mtest0: mtest0.o mdbx.o $(CC) $(CFLAGS) $(LDFLAGS) -o $@ $^ @@ -133,13 +145,13 @@ yota_test2: yota_test2.o mdbx.o $(CC) $(CFLAGS) $(LDFLAGS) -o $@ $^ wbench: wbench.o mdbx.o - $(CC) $(CFLAGS) $(LDFLAGS) -o $@ $^ -Wl,--no-as-needed,-lrt + $(CC) $(CFLAGS) $(LDFLAGS) -o $@ $^ mdbx.o: $(SRC_MDBX) - $(CC) $(CFLAGS) -include mdbx.h -c mdb.c -o $@ + $(CC) $(CFLAGS) -c mdbx.c -o $@ mdbx.lo: $(SRC_MDBX) - $(CC) $(CFLAGS) -include mdbx.h -fPIC -c mdb.c -o $@ + $(CC) $(CFLAGS) -fPIC -c mdbx.c -o $@ lmdb.o: $(SRC_LMDB) $(CC) $(CFLAGS) -c mdb.c -o $@ @@ -153,14 +165,14 @@ lmdb.lo: $(SRC_LMDB) %.o: %.c lmdb.h mdbx.h $(CC) $(CFLAGS) -c $< -COV_FLAGS=-fprofile-arcs -ftest-coverage +COFLAGS = -fprofile-arcs -ftest-coverage @gcov-mdb.o: $(SRC_MDBX) - $(CC) $(CFLAGS) $(COV_FLAGS) -O0 -include mdbx.h -c mdb.c -o $@ + $(CC) $(CFLAGS) $(COFLAGS) -O0 -c mdbx.c -o $@ coverage: @gcov-mdb.o for t in mtest*.c; do x=`basename \$$t .c`; $(MAKE) $$x.o; \ - gcc -o @gcov-$$x $$x.o $^ -pthread $(COV_FLAGS); \ + gcc -o @gcov-$$x $$x.o $^ -pthread $(COFLAGS); \ rm -rf testdb; mkdir testdb; ./@gcov-$$x; done gcov @gcov-mdb @@ -193,3 +205,14 @@ $(eval $(call bench-rule,debug,10)) bench: bench-lmdb.txt bench-mdbx.txt endif + +ci-rule = @( CC=$$(which $1); if [ -n "$$CC" ]; then \ + CC=$$(readlink -f $$CC); echo -n "probe by $2 ($$CC): " && \ + $(MAKE) clean >$1.log 2>$1.err && $(MAKE) all check 1>$1.log 2>$1.err && echo "OK" \ + || ( echo "FAILED"; cat $1.err >&2; exit 1 ); \ + else echo "no $2 ($1) for probe"; fi; ) +ci: + $(call ci-rule,cc,default C compiler) + $(call ci-rule,gcc,GCC) + $(call ci-rule,clang,clang LLVM) + $(call ci-rule,icc,Intel C) diff --git a/barriers.h b/barriers.h index a040a8d0..15b1e2cf 100644 --- a/barriers.h +++ b/barriers.h @@ -1,6 +1,27 @@ +/* + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + */ + /***************************************************************************** * Properly compiler/memory/coherence barriers - * in the most portable way for ReOpenLDAP project. + * in the most portable way for ReOpenMDBX project. * * Feedback and comments are welcome. * https://gist.github.com/leo-yuriev/ba186a6bf5cf3a27bae7 */ @@ -136,4 +157,3 @@ static MDBX_INLINE void mdb_invalidate_cache(void *addr, int nbytes) { (void) nbytes; #endif } - diff --git a/intro.doc b/intro.doc index 2886c20a..aeaeb21d 100644 --- a/intro.doc +++ b/intro.doc @@ -1,5 +1,25 @@ /* - * Copyright 2015-2016 Howard Chu, Symas Corp. + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * --- + * + * Copyright 2015-2014 Howard Chu, Symas Corp. * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -10,6 +30,7 @@ * top-level directory of the distribution or, alternatively, at * . */ + /** @page starting Getting Started LMDB is compact, fast, powerful, and robust and implements a simplified diff --git a/lmdb.h b/lmdb.h index be22d931..479d2da3 100644 --- a/lmdb.h +++ b/lmdb.h @@ -1,10 +1,10 @@ /** @file lmdb.h - * @brief Lightning memory-mapped database library + * @brief Reliable Lightning memory-mapped database library * - * @mainpage Lightning Memory-Mapped Database Manager (LMDB) + * @mainpage Reliable Lightning Memory-Mapped Database Manager (MDBX) * * @section intro_sec Introduction - * LMDB is a Btree-based database management library modeled loosely on the + * MDBX is a Btree-based database management library modeled loosely on the * BerkeleyDB API, but much simplified. The entire database is exposed * in a memory map, and all data fetches return data directly * from the mapped memory, so no malloc's or memcpy's occur during @@ -26,10 +26,10 @@ * readers, and readers don't block writers. * * Unlike other well-known database mechanisms which use either write-ahead - * transaction logs or append-only data writes, LMDB requires no maintenance + * transaction logs or append-only data writes, MDBX requires no maintenance * during operation. Both write-ahead loggers and append-only databases * require periodic checkpointing and/or compaction of their log or database - * files otherwise they grow without bound. LMDB tracks free pages within + * files otherwise they grow without bound. MDBX tracks free pages within * the database and re-uses them for new write operations, so the database * size does not grow without bound in normal use. * @@ -44,7 +44,7 @@ * store, you may find the \ref starting page to be helpful. * * @section caveats_sec Caveats - * Troubleshooting the lock file, plus semaphores on BSD systems: + * Troubleshooting the lock file: * * - A broken lockfile can cause sync issues. * Stale reader transactions left behind by an aborted program @@ -53,30 +53,15 @@ * * Fix: Check for stale readers periodically, using the * #mdb_reader_check function or the \ref mdb_stat_1 "mdb_stat" tool. - * Stale writers will be cleared automatically on most systems: - * - BSD, systems using SysV semaphores - automatic - * - Linux, systems using POSIX mutexes with Robust option - automatic + * Stale writers will be cleared automatically on Linux + * using POSIX mutexes with Robust option. * Otherwise just make all programs using the database close it; * the lockfile is always reset on first open of the environment. * - * - On BSD systems or others configured with MDB_USE_SYSV_SEM, - * startup can fail due to semaphores owned by another userid. - * - * Fix: Open and close the database as the user which owns the - * semaphores (likely last user) or as root, while no other - * process is using the database. * * Restrictions/caveats (in addition to those listed for some functions): * - * - Only the database owner should normally use the database on - * BSD systems or when otherwise configured with MDB_USE_POSIX_SEM. - * Multiple users can cause startup to fail later, as noted above. - * - * - There is normally no pure read-only mode, since readers need write - * access to locks and lock file. Exceptions: On read-only filesystems - * or with the #MDB_NOLOCK flag described under #mdb_env_open(). - * - * - An LMDB configuration will often reserve considerable \b unused + * - An MDBX configuration will often reserve considerable \b unused * memory address space and maybe file size for future growth. * This does not use actual memory or disk space, but users may need * to understand the difference so they won't be scared off. @@ -97,7 +82,7 @@ * * - Use an MDB_env* in the process which opened it, without fork()ing. * - * - Do not have open an LMDB database twice in the same process at + * - Do not have open an MDBX database twice in the same process at * the same time. Not even from a plain open() call - close()ing it * breaks flock() advisory locking. * @@ -123,7 +108,7 @@ * - If you do that anyway, do a periodic check for stale readers. Or * close the environment once in a while, so the lockfile can get reset. * - * - Do not use LMDB databases on remote filesystems, even between + * - Do not use MDBX databases on remote filesystems, even between * processes on the same host. This breaks flock() on some OSes, * possibly memory map sync, and certainly sync between programs * on different hosts. @@ -131,9 +116,37 @@ * - Opening a database can fail if another process is opening or * closing it at exactly the same time. * - * @author Howard Chu, Symas Corporation. + * @author Leonid Yuriev, 'ReOpen' initiative . + * Howard Chu, Symas Corp. All rights reserved. * - * @copyright Copyright 2011-2016 Howard Chu, Symas Corp. All rights reserved. + * @copyright 2015,2016 Leonid Yuriev . + * 2011-2014 Howard Chu, Symas Corp. All rights reserved. + * + * --- + * + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * --- + * @par Derived From: + * This code is derived from LMDB engine written by Howard Chu, Symas Corporation. + * + * Copyright 2011-2014 Howard Chu, Symas Corp. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted only as authorized by the OpenLDAP @@ -160,6 +173,7 @@ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ + #ifndef _LMDB_H_ #define _LMDB_H_ @@ -206,7 +220,7 @@ typedef int mdb_filehandle_t; MDB_VERINT(MDB_VERSION_MAJOR,MDB_VERSION_MINOR,MDB_VERSION_PATCH) /** The release date of this library version */ -#define MDB_VERSION_DATE "2016-04-06" +#define MDB_VERSION_DATE "2016-06-09" /** A stringifier for the version info */ #define MDB_VERSTR(a,b,c,d) "MDBX " #a "." #b "." #c ": (" d ", https://github.com/ReOpen/libmdbx)" @@ -300,8 +314,8 @@ typedef void (MDB_rel_func)(MDB_val *item, void *oldptr, void *newptr, void *rel /** tie reader locktable slots to #MDB_txn objects instead of to threads */ #define MDB_NOTLS 0x200000 /** don't do any locking, caller must manage their own locks - * WARNING: MDBX and ReOpenLDAP don't support this mode. */ -#define MDB_NOLOCK__UNSUPPORTED 0x400000 + * WARNING: ReOpenMDBX don't support this mode. */ +#define MDB_NOLOCK__UNSUPPORTED 0x400000 /** don't do readahead */ #define MDB_NORDAHEAD 0x800000 /** don't initialize malloc'd memory before writing to datafile */ diff --git a/mdb.c b/mdb.c index 7d09d923..91e71e49 100644 --- a/mdb.c +++ b/mdb.c @@ -4,8 +4,29 @@ * A Btree-based database management library modeled loosely on the * BerkeleyDB API, but much simplified. */ + /* - * Copyright 2011-2016 Howard Chu, Symas Corp. + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * --- + * + * Copyright 2011-2014 Howard Chu, Symas Corp. * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -37,6 +58,36 @@ # define MDB_DEBUG 0 #endif +#ifndef _GNU_SOURCE +# define _GNU_SOURCE +#endif + +/* LY: Please do not ask us for Windows support, just never! + * But you can make a fork for Windows, or become maintainer for FreeBSD... */ +#ifndef __gnu_linux__ +# warning "ReOpenMDBX supports only GNU Linux" +#endif + +#include + +#if !defined(__GNUC__) || !__GNUC_PREREQ(4,2) + /* LY: Actualy ReOpenMDBX was not tested with compilers + * older than GCC 4.4 (from RHEL6). + * But you could remove this #error and try to continue at your own risk. + * In such case please don't rise up an issues related ONLY to old compilers. + */ +# warning "ReOpenMDBX required at least GCC 4.2 compatible C/C++ compiler." +#endif + +#if !defined(__GNU_LIBRARY__) || !__GLIBC_PREREQ(2,12) + /* LY: Actualy ReOpenMDBX was not tested with something + * older than glibc 2.12 (from RHEL6). + * But you could remove this #error and try to continue at your own risk. + * In such case please don't rise up an issues related ONLY to old systems. + */ +# warning "ReOpenMDBX required at least GLIBC 2.12." +#endif + #include "./reopen.h" #include "./barriers.h" @@ -108,6 +159,11 @@ # define MDBX_DBG_EXTRA 0 # define MDBX_DBG_AUDIT 0 # define MDBX_DBG_EDGE 0 +# define mdb_runtime_flags 0 +# define mdb_debug_logger ((void (*)(int, ...)) NULL) +# define MDBX_ONLY_FEATURE static +#else +# define MDBX_ONLY_FEATURE #endif /* ! MDBX_MODE_ENABLED */ #if (BYTE_ORDER == LITTLE_ENDIAN) == (BYTE_ORDER == BIG_ENDIAN) @@ -149,11 +205,8 @@ #ifndef MDB_USE_ROBUST /* Howard Chu: Android currently lacks Robust Mutex support */ # if defined(EOWNERDEAD) && !defined(ANDROID) \ - /* LY: glibc before 2.10 has a troubles with Robust Mutex too. - * But more over: - * - we couldn't test code with glibc < 2.12; - * - we won't provide compatibility with old systems. */ \ - && !(defined(__GLIBC__) && ((__GLIBC__ << 16)|__GLIBC_MINOR__) < 0x02000c) + /* LY: glibc before 2.10 has a troubles with Robust Mutex too. */ \ + && __GLIBC_PREREQ(2,10) # define MDB_USE_ROBUST 1 # else # define MDB_USE_ROBUST 0 @@ -920,7 +973,7 @@ typedef struct MDB_pgstate { } MDB_pgstate; /** Context for deferred cleanup of reader's threads. - * to avoid https://github.com/ReOpen/ReOpenLDAP/issues/48 */ + * to avoid https://github.com/ReOpen/ReOpenLDAP/issues/48 */ struct MDB_rthc { MDB_reader *rc_reader; }; @@ -1146,28 +1199,8 @@ mdb_strerror(int err) } #if MDBX_MODE_ENABLED - -int mdb_runtime_flags = MDBX_DBG_PRINT -#if MDB_DEBUG - | MDBX_DBG_ASSERT -#endif -#if MDB_DEBUG > 1 - | MDBX_DBG_TRACE -#endif -#if MDB_DEBUG > 2 - | MDBX_DBG_AUDIT -#endif -#if MDB_DEBUG > 3 - | MDBX_DBG_EXTRA -#endif - ; - -static MDBX_debug_func *mdb_debug_logger; - -#else /* MDBX_MODE_ENABLED */ -# define mdb_runtime_flags 0 -# define mdb_debug_logger ((void (*)(int, ...)) NULL) -#endif /* ! MDBX_MODE_ENABLED */ +static txnid_t mdbx_oomkick(MDB_env *env, txnid_t oldest); +#endif /* MDBX_MODE_ENABLED */ #if MDB_DEBUG static txnid_t mdb_debug_edge; @@ -1205,24 +1238,6 @@ static MDBX_debug_func *mdb_debug_logger; __assert_fail(msg, __FILE__, line, func) #endif /* MDB_DEBUG */ -#if MDBX_MODE_ENABLED -int __cold -mdbx_setup_debug(int flags, MDBX_debug_func* logger, long edge_txn) { - unsigned ret = mdb_runtime_flags; - if (flags != (int) MDBX_DBG_DNT) - mdb_runtime_flags = flags; - if (logger != (MDBX_debug_func*) MDBX_DBG_DNT) - mdb_debug_logger = logger; -#if MDB_DEBUG - if (edge_txn != (long) MDBX_DBG_DNT) - mdb_debug_edge = edge_txn; -#else - (void) edge_txn; -#endif - return ret; -} -#endif /* MDBX_MODE_ENABLED */ - static void __cold mdb_debug_log(int type, const char *function, int line, const char *fmt, ...) @@ -1980,70 +1995,6 @@ txnid_t mdb_find_oldest(MDB_env *env, int *laggard) return env->me_pgoldest = oldest; } -static txnid_t __cold -mdbx_oomkick(MDB_env *env, txnid_t oldest) -{ - mdb_debug("DB size maxed out"); -#if MDBX_MODE_ENABLED - int retry; - txnid_t snap; - mdb_debug("DB size maxed out"); - - for(retry = 0; ; ++retry) { - int reader; - - if (mdb_reader_check(env, NULL)) - break; - - snap = mdb_find_oldest(env, &reader); - if (oldest < snap || reader < 0) { - if (retry && env->me_oom_func) { - /* LY: notify end of oom-loop */ - env->me_oom_func(env, 0, 0, oldest, snap - oldest, -retry); - } - return snap; - } - - MDB_reader *r; - pthread_t tid; - pid_t pid; - int rc; - - if (!env->me_oom_func) - break; - - r = &env->me_txns->mti_readers[ reader ]; - pid = r->mr_pid; - tid = r->mr_tid; - if (r->mr_txnid != oldest || pid <= 0) - continue; - - rc = env->me_oom_func(env, pid, (void*) tid, oldest, - mdb_meta_head_w(env)->mm_txnid - oldest, retry); - if (rc < 0) - break; - - if (rc) { - r->mr_txnid = ~(txnid_t)0; - if (rc > 1) { - r->mr_tid = 0; - r->mr_pid = 0; - mdbx_coherent_barrier(); - } - } - } - - if (retry && env->me_oom_func) { - /* LY: notify end of oom-loop */ - env->me_oom_func(env, 0, 0, oldest, 0, -retry); - } -#else - (void) oldest; - (void) mdb_reader_check(env, NULL); -#endif /* MDBX_MODE_ENABLED */ - return mdb_find_oldest(env, NULL); -} - /** Add a page to the txn's dirty list */ static void mdb_page_dirty(MDB_txn *txn, MDB_page *mp) @@ -2346,8 +2297,14 @@ mdb_page_alloc(MDB_cursor *mc, int num, MDB_page **mp, int flags) } if (rc == MDB_MAP_FULL) { +#if MDBX_MODE_ENABLED txnid_t snap = mdbx_oomkick(env, oldest); +#else + mdb_debug("DB size maxed out"); + txnid_t snap = mdb_find_oldest(env, NULL); +#endif /* MDBX_MODE_ENABLED */ if (snap > oldest) { + oldest = snap; continue; } } @@ -3997,21 +3954,6 @@ fail: return rc; } -#if MDBX_MODE_ENABLED -int __cold -mdbx_env_set_syncbytes(MDB_env *env, size_t bytes) -{ - if (unlikely(!env)) - return EINVAL; - - if(unlikely(env->me_signature != MDBX_ME_SIGNATURE)) - return MDB_VERSION_MISMATCH; - - env->me_sync_threshold = bytes; - return env->me_map ? mdb_env_sync(env, 0) : 0; -} -#endif /* MDBX_MODE_ENABLED */ - /** Read the environment parameters of a DB environment before * mapping it into memory. * @param[in] env the environment handle @@ -4815,17 +4757,27 @@ mdb_env_setup_locks(MDB_env *env, char *lpath, int mode, int *excl) return errno; if (*excl > 0) { - pthread_mutexattr_t mattr; + /* Solaris needs this before initing a robust mutex. Otherwise + * it may skip the init and return EBUSY "seems someone already + * inited" or EINVAL "it was inited differently". + */ + memset(&env->me_txns->mti_rmutex, 0, sizeof(env->me_txns->mti_rmutex)); + memset(&env->me_txns->mti_wmutex, 0, sizeof(env->me_txns->mti_wmutex)); + + pthread_mutexattr_t mattr; + rc = pthread_mutexattr_init(&mattr); + if (rc) return rc; + + rc = pthread_mutexattr_setpshared(&mattr, PTHREAD_PROCESS_SHARED); - if ((rc = pthread_mutexattr_init(&mattr)) - || (rc = pthread_mutexattr_setpshared(&mattr, PTHREAD_PROCESS_SHARED)) #if MDB_USE_ROBUST - || (rc = pthread_mutexattr_setrobust(&mattr, PTHREAD_MUTEX_ROBUST)) + if(! rc) rc = pthread_mutexattr_setrobust(&mattr, PTHREAD_MUTEX_ROBUST); #endif /* MDB_USE_ROBUST */ - || (rc = pthread_mutex_init(&env->me_txns->mti_rmutex, &mattr)) - || (rc = pthread_mutex_init(&env->me_txns->mti_wmutex, &mattr))) - return rc; + if (! rc) rc = pthread_mutex_init(&env->me_txns->mti_rmutex, &mattr); + if (! rc) rc = pthread_mutex_init(&env->me_txns->mti_wmutex, &mattr); + pthread_mutexattr_destroy(&mattr); + if (rc) return rc; env->me_txns->mti_magic = MDB_MAGIC; env->me_txns->mti_format = MDB_LOCK_FORMAT; @@ -4865,10 +4817,7 @@ mdb_env_setup_locks(MDB_env *env, char *lpath, int mode, int *excl) # error "Persistent DB flags & env flags overlap, but both go in mm_flags" #endif -#if ! MDBX_MODE_ENABLED -static -#endif /* ! MDBX_MODE_ENABLED */ -int __cold +MDBX_ONLY_FEATURE int __cold mdbx_env_open_ex(MDB_env *env, const char *path, unsigned flags, mode_t mode, int *exclusive) { int oflags, rc, len, excl = -1; @@ -5099,10 +5048,7 @@ mdb_env_close0(MDB_env *env) } } -#if ! MDBX_MODE_ENABLED -static -#endif /* ! MDBX_MODE_ENABLED */ -int __cold +MDBX_ONLY_FEATURE int __cold mdbx_env_close_ex(MDB_env *env, int dont_sync) { MDB_page *dp; @@ -9196,7 +9142,7 @@ mdb_env_cthr_toggle(mdb_copy *my, int st) static int __cold mdb_env_cwalk(mdb_copy *my, pgno_t *pg, int flags) { - MDB_cursor mc = {0}; + MDB_cursor mc; MDB_txn *txn = my->mc_txn; MDB_node *ni; MDB_page *mo, *mp, *leaf; @@ -9208,6 +9154,7 @@ mdb_env_cwalk(mdb_copy *my, pgno_t *pg, int flags) if (*pg == P_INVALID) return MDB_SUCCESS; + memset(&mc, 0, sizeof(mc)); mc.mc_snum = 1; mc.mc_txn = txn; @@ -9713,10 +9660,7 @@ mdb_stat0(MDB_env *env, MDB_db *db, MDB_stat *arg) return MDB_SUCCESS; } -#if ! MDBX_MODE_ENABLED -static -#endif /* ! MDBX_MODE_ENABLED */ -int __cold +MDBX_ONLY_FEATURE int __cold mdbx_env_stat(MDB_env *env, MDBX_stat *arg, size_t bytes) { MDB_meta *meta; @@ -9736,10 +9680,7 @@ mdb_env_stat(MDB_env *env, MDB_stat *arg) return mdbx_env_stat(env, (MDBX_stat *) arg, sizeof(MDB_stat)); } -#if ! MDBX_MODE_ENABLED -static -#endif /* ! MDBX_MODE_ENABLED */ -int __cold +MDBX_ONLY_FEATURE int __cold mdbx_env_info(MDB_env *env, MDBX_envinfo *arg, size_t bytes) { MDB_meta *meta; @@ -9951,10 +9892,7 @@ int mdb_dbi_open(MDB_txn *txn, const char *name, unsigned flags, MDB_dbi *dbi) return rc; } -#if ! MDBX_MODE_ENABLED -static -#endif -int __cold +MDBX_ONLY_FEATURE int __cold mdbx_stat(MDB_txn *txn, MDB_dbi dbi, MDBX_stat *arg, size_t bytes) { if (unlikely(!arg || !txn)) @@ -10459,321 +10397,6 @@ static void mdb_mutex_unlock(MDB_env *env, pthread_mutex_t *mutex) { (void) rc; } -#if MDBX_MODE_ENABLED - -void __cold -mdbx_env_set_oomfunc(MDB_env *env, MDBX_oom_func *oomfunc) -{ - if (likely(env && env->me_signature == MDBX_ME_SIGNATURE)) - env->me_oom_func = oomfunc; -} - -MDBX_oom_func* __cold -mdbx_env_get_oomfunc(MDB_env *env) -{ - return likely(env && env->me_signature == MDBX_ME_SIGNATURE) - ? env->me_oom_func : NULL; -} - -struct mdb_walk_ctx { - MDB_txn *mw_txn; - void *mw_user; - MDBX_pgvisitor_func *mw_visitor; -}; - -typedef struct mdb_walk_ctx mdb_walk_ctx_t; - - -/** Depth-first tree traversal. */ -static int __cold -mdb_env_walk(mdb_walk_ctx_t *ctx, const char* dbi, pgno_t pg, int flags, int deep) -{ - MDB_page *mp; - int rc, i, nkeys; - unsigned header_size, unused_size, payload_size, align_bytes; - const char* type; - - if (pg == P_INVALID) - return MDB_CORRUPTED; - - rc = mdb_page_get(ctx->mw_txn, pg, &mp, NULL); - if (rc) - return rc; - if (pg != mp->mp_p.p_pgno) - return MDB_CORRUPTED; - - nkeys = NUMKEYS(mp); - header_size = IS_LEAF2(mp) ? PAGEHDRSZ : PAGEBASE + mp->mp_lower; - unused_size = SIZELEFT(mp); - payload_size = 0; - - /* LY: Don't use mask here, e.g bitwise (P_BRANCH|P_LEAF|P_LEAF2|P_META|P_OVERFLOW|P_SUBP). - * Pages should not me marked dirty/loose or otherwise. */ - switch (mp->mp_flags) { - case P_BRANCH: - type = "branch"; - if (nkeys < 1) - return MDB_CORRUPTED; - break; - case P_LEAF: - type = "leaf"; - break; - case P_LEAF|P_SUBP: - type = "dupsort-subleaf"; - break; - case P_LEAF|P_LEAF2: - type = "dupfixed-leaf"; - break; - case P_LEAF|P_LEAF2|P_SUBP: - type = "dupsort-dupfixed-subleaf"; - break; - case P_META: - case P_OVERFLOW: - default: - return MDB_CORRUPTED; - } - - for (align_bytes = i = 0; i < nkeys; - align_bytes += ((payload_size + align_bytes) & 1), i++) { - MDB_node *node; - - if (IS_LEAF2(mp)) { - /* LEAF2 pages have no mp_ptrs[] or node headers */ - payload_size += mp->mp_ksize; - continue; - } - - node = NODEPTR(mp, i); - payload_size += NODESIZE + node->mn_ksize; - - if (IS_BRANCH(mp)) { - rc = mdb_env_walk(ctx, dbi, NODEPGNO(node), flags, deep); - if (rc) - return rc; - continue; - } - - assert(IS_LEAF(mp)); - if (node->mn_ksize < 1) - return MDB_CORRUPTED; - if (node->mn_flags & F_BIGDATA) { - MDB_page *omp; - pgno_t *opg; - size_t over_header, over_payload, over_unused; - - payload_size += sizeof(pgno_t); - opg = NODEDATA(node); - rc = mdb_page_get(ctx->mw_txn, *opg, &omp, NULL); - if (rc) - return rc; - if (*opg != omp->mp_p.p_pgno) - return MDB_CORRUPTED; - /* LY: Don't use mask here, e.g bitwise (P_BRANCH|P_LEAF|P_LEAF2|P_META|P_OVERFLOW|P_SUBP). - * Pages should not me marked dirty/loose or otherwise. */ - if (P_OVERFLOW != omp->mp_flags) - return MDB_CORRUPTED; - - over_header = PAGEHDRSZ; - over_payload = NODEDSZ(node); - over_unused = omp->mp_pages * ctx->mw_txn->mt_env->me_psize - - over_payload - over_header; - - rc = ctx->mw_visitor(*opg, omp->mp_pages, ctx->mw_user, dbi, - "overflow-data", 1, over_payload, over_header, over_unused); - if (rc) - return rc; - continue; - } - - payload_size += NODEDSZ(node); - if (node->mn_flags & F_SUBDATA) { - MDB_db *db = NODEDATA(node); - char* name = NULL; - - if (NODEDSZ(node) < 1) - return MDB_CORRUPTED; - if (! (node->mn_flags & F_DUPDATA)) { - name = NODEKEY(node); - int namelen = (char*) db - name; - name = memcpy(alloca(namelen + 1), name, namelen); - name[namelen] = 0; - } - rc = mdb_env_walk(ctx, (name && name[0]) ? name : dbi, - db->md_root, node->mn_flags & F_DUPDATA, deep + 1); - if (rc) - return rc; - } - } - - return ctx->mw_visitor(mp->mp_p.p_pgno, 1, ctx->mw_user, dbi, type, - nkeys, payload_size, header_size, unused_size + align_bytes); -} - -int __cold -mdbx_env_pgwalk(MDB_txn *txn, MDBX_pgvisitor_func* visitor, void* user) -{ - mdb_walk_ctx_t ctx; - int rc; - - if (unlikely(!txn)) - return MDB_BAD_TXN; - if (unlikely(txn->mt_signature != MDBX_MT_SIGNATURE)) - return MDB_VERSION_MISMATCH; - - ctx.mw_txn = txn; - ctx.mw_user = user; - ctx.mw_visitor = visitor; - - rc = visitor(0, 2, user, "lmdb", "meta", 2, sizeof(MDB_meta)*2, PAGEHDRSZ*2, - (txn->mt_env->me_psize - sizeof(MDB_meta) - PAGEHDRSZ) *2); - if (! rc && txn->mt_dbs[FREE_DBI].md_root != P_INVALID) - rc = mdb_env_walk(&ctx, "free", txn->mt_dbs[FREE_DBI].md_root, 0, 0); - if (! rc && txn->mt_dbs[MAIN_DBI].md_root != P_INVALID) - rc = mdb_env_walk(&ctx, "main", txn->mt_dbs[MAIN_DBI].md_root, 0, 0); - if (! rc) - rc = visitor(P_INVALID, 0, user, NULL, NULL, 0, 0, 0, 0); - return rc; -} - -/* attribute support functions for Nexenta ***********************************/ - -static __inline int -mdbx_attr_peek(MDB_val *data, mdbx_attr_t *attrptr) -{ - if (unlikely(data->mv_size < sizeof(mdbx_attr_t))) - return MDB_INCOMPATIBLE; - - if (likely(attrptr != NULL)) - *attrptr = *(mdbx_attr_t*) data->mv_data; - data->mv_size -= sizeof(mdbx_attr_t); - data->mv_data = likely(data->mv_size > 0) - ? ((mdbx_attr_t*) data->mv_data) + 1 : NULL; - - return MDB_SUCCESS; -} - -static __inline int -mdbx_attr_poke(MDB_val *reserved, MDB_val *data, mdbx_attr_t attr, unsigned flags) -{ - mdbx_attr_t *space = reserved->mv_data; - if (flags & MDB_RESERVE) { - if (likely(data != NULL)) { - data->mv_data = data->mv_size ? space + 1 : NULL; - } - } else { - *space = attr; - if (likely(data != NULL)) { - memcpy(space + 1, data->mv_data, data->mv_size ); - } - } - - return MDB_SUCCESS; -} - -int -mdbx_cursor_get_attr(MDB_cursor *mc, MDB_val *key, MDB_val *data, - mdbx_attr_t *attrptr, MDB_cursor_op op) -{ - int rc = mdbx_cursor_get(mc, key, data, op); - if (unlikely(rc != MDB_SUCCESS)) - return rc; - - return mdbx_attr_peek(data, attrptr); -} - -int -mdbx_get_attr(MDB_txn *txn, MDB_dbi dbi, - MDB_val *key, MDB_val *data, uint64_t *attrptr) -{ - int rc = mdbx_get(txn, dbi, key, data); - if (unlikely(rc != MDB_SUCCESS)) - return rc; - - return mdbx_attr_peek(data, attrptr); -} - -int -mdbx_put_attr(MDB_txn *txn, MDB_dbi dbi, - MDB_val *key, MDB_val *data, mdbx_attr_t attr, unsigned flags) -{ - MDB_val reserve = { - .mv_data = NULL, - .mv_size = (data ? data->mv_size : 0) + sizeof(mdbx_attr_t) - }; - - int rc = mdbx_put(txn, dbi, key, &reserve, flags | MDB_RESERVE); - if (unlikely(rc != MDB_SUCCESS)) - return rc; - - return mdbx_attr_poke(&reserve, data, attr, flags); -} - -int mdbx_cursor_put_attr(MDB_cursor *cursor, MDB_val *key, MDB_val *data, - mdbx_attr_t attr, unsigned flags) -{ - MDB_val reserve = { - .mv_data = NULL, - .mv_size = (data ? data->mv_size : 0) + sizeof(mdbx_attr_t) - }; - - int rc = mdbx_cursor_put(cursor, key, &reserve, flags | MDB_RESERVE); - if (unlikely(rc != MDB_SUCCESS)) - return rc; - - return mdbx_attr_poke(&reserve, data, attr, flags); -} - -int mdbx_set_attr(MDB_txn *txn, MDB_dbi dbi, - MDB_val *key, MDB_val *data, mdbx_attr_t attr) -{ - MDB_cursor mc; - MDB_xcursor mx; - MDB_val old_data; - mdbx_attr_t old_attr; - int rc; - - if (unlikely(!key || !txn)) - return EINVAL; - - if (unlikely(txn->mt_signature != MDBX_MT_SIGNATURE)) - return MDB_VERSION_MISMATCH; - - if (unlikely(!TXN_DBI_EXIST(txn, dbi, DB_USRVALID))) - return EINVAL; - - if (unlikely(txn->mt_flags & (MDB_TXN_RDONLY|MDB_TXN_BLOCKED))) - return (txn->mt_flags & MDB_TXN_RDONLY) ? EACCES : MDB_BAD_TXN; - - mdb_cursor_init(&mc, txn, dbi, &mx); - rc = mdb_cursor_set(&mc, key, &old_data, MDB_SET, NULL); - if (unlikely(rc != MDB_SUCCESS)) { - if (rc == MDB_NOTFOUND && data) { - mc.mc_next = txn->mt_cursors[dbi]; - txn->mt_cursors[dbi] = &mc; - rc = mdbx_cursor_put_attr(&mc, key, data, attr, 0); - txn->mt_cursors[dbi] = mc.mc_next; - } - return rc; - } - - rc = mdbx_attr_peek(&old_data, &old_attr); - if (unlikely(rc != MDB_SUCCESS)) - return rc; - - if (old_attr == attr && (!data || - (data->mv_size == old_data.mv_size - && memcpy(data->mv_data, old_data.mv_data, old_data.mv_size) == 0))) - return MDB_SUCCESS; - - mc.mc_next = txn->mt_cursors[dbi]; - txn->mt_cursors[dbi] = &mc; - rc = mdbx_cursor_put_attr(&mc, key, data ? data : &old_data, attr, MDB_CURRENT); - txn->mt_cursors[dbi] = mc.mc_next; - return rc; -} - -#endif /* MDBX_MODE_ENABLED */ - /** @} */ #include "./midl.c" diff --git a/mdb_chk.c b/mdb_chk.c index 241105d3..75320105 100644 --- a/mdb_chk.c +++ b/mdb_chk.c @@ -1,23 +1,26 @@ -/* - Copyright (c) 2015,2016 Leonid Yuriev . - Copyright (c) 2015,2016 Peter-Service R&D LLC. - - This file is part of ReOpenLDAP. - - ReOpenLDAP is free software; you can redistribute it and/or modify it under - the terms of the GNU Affero General Public License as published by - the Free Software Foundation; either version 3 of the License, or - (at your option) any later version. - - ReOpenLDAP is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU Affero General Public License for more details. - - You should have received a copy of the GNU Affero General Public License - along with this program. If not, see . -*/ /* mdbx_chk.c - memory-mapped database check tool */ + +/* + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + */ + #include #include #include diff --git a/mdb_copy.1 b/mdb_copy.1 index 258affbf..6f566aab 100644 --- a/mdb_copy.1 +++ b/mdb_copy.1 @@ -1,6 +1,26 @@ -.TH MDB_COPY 1 "2014/06/20" "LMDB 0.9.14" -.\" Copyright 2012-2016 Howard Chu, Symas Corp. All Rights Reserved. +.\" Copyright (c) 2015,2016 Leonid Yuriev . +.\" Copyright (c) 2015,2016 Peter-Service R&D LLC . +.\" +.\" This file is part of ReOpenMDBX. +.\" +.\" ReOpenMDBX is free software; you can redistribute it and/or modify it under +.\" the terms of the GNU Affero General Public License as published by +.\" the Free Software Foundation; either version 3 of the License, or +.\" (at your option) any later version. +.\" +.\" ReOpenMDBX is distributed in the hope that it will be useful, +.\" but WITHOUT ANY WARRANTY; without even the implied warranty of +.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +.\" GNU Affero General Public License for more details. +.\" +.\" You should have received a copy of the GNU Affero General Public License +.\" along with this program. If not, see . +.\" +.\" --- +.\" +.\" Copyright 2012-2014 Howard Chu, Symas Corp. All Rights Reserved. .\" Copying restrictions apply. See COPYRIGHT/LICENSE. +.TH MDB_COPY 1 "2014/06/20" "LMDB 0.9.14" .SH NAME mdb_copy \- LMDB environment copy tool .SH SYNOPSIS diff --git a/mdb_copy.c b/mdb_copy.c index 348bea9e..f2db960e 100644 --- a/mdb_copy.c +++ b/mdb_copy.c @@ -1,6 +1,27 @@ /* mdb_copy.c - memory-mapped database backup tool */ + /* - * Copyright 2012-2016 Howard Chu, Symas Corp. + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * --- + * + * Copyright 2012-2014 Howard Chu, Symas Corp. * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -11,6 +32,7 @@ * top-level directory of the distribution or, alternatively, at * . */ + #include #include #include diff --git a/mdb_dump.1 b/mdb_dump.1 index 19a7fd26..ead2d92a 100644 --- a/mdb_dump.1 +++ b/mdb_dump.1 @@ -1,6 +1,26 @@ -.TH MDB_DUMP 1 "2014/06/20" "LMDB 0.9.14" -.\" Copyright 2014-2016 Howard Chu, Symas Corp. All Rights Reserved. +.\" Copyright (c) 2015,2016 Leonid Yuriev . +.\" Copyright (c) 2015,2016 Peter-Service R&D LLC . +.\" +.\" This file is part of ReOpenMDBX. +.\" +.\" ReOpenMDBX is free software; you can redistribute it and/or modify it under +.\" the terms of the GNU Affero General Public License as published by +.\" the Free Software Foundation; either version 3 of the License, or +.\" (at your option) any later version. +.\" +.\" ReOpenMDBX is distributed in the hope that it will be useful, +.\" but WITHOUT ANY WARRANTY; without even the implied warranty of +.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +.\" GNU Affero General Public License for more details. +.\" +.\" You should have received a copy of the GNU Affero General Public License +.\" along with this program. If not, see . +.\" +.\" --- +.\" +.\" Copyright 2014 Howard Chu, Symas Corp. All Rights Reserved. .\" Copying restrictions apply. See COPYRIGHT/LICENSE. +.TH MDB_DUMP 1 "2014/06/20" "LMDB 0.9.14" .SH NAME mdb_dump \- LMDB environment export tool .SH SYNOPSIS diff --git a/mdb_dump.c b/mdb_dump.c index aeb57801..b5f3af6c 100644 --- a/mdb_dump.c +++ b/mdb_dump.c @@ -1,6 +1,27 @@ /* mdb_dump.c - memory-mapped database dump tool */ + /* - * Copyright 2011-2016 Howard Chu, Symas Corp. + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * --- + * + * Copyright 2011-2014 Howard Chu, Symas Corp. * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -11,6 +32,7 @@ * top-level directory of the distribution or, alternatively, at * . */ + #include #include #include diff --git a/mdb_load.c b/mdb_load.c index a883e011..2d89a501 100644 --- a/mdb_load.c +++ b/mdb_load.c @@ -1,6 +1,27 @@ /* mdb_load.c - memory-mapped database load tool */ + /* - * Copyright 2011-2016 Howard Chu, Symas Corp. + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * --- + * + * Copyright 2011-2014 Howard Chu, Symas Corp. * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -11,6 +32,7 @@ * top-level directory of the distribution or, alternatively, at * . */ + #include #include #include diff --git a/mdb_stat.1 b/mdb_stat.1 index b5ab6a9d..f2739bb4 100644 --- a/mdb_stat.1 +++ b/mdb_stat.1 @@ -1,6 +1,26 @@ -.TH MDB_STAT 1 "2014/06/20" "LMDB 0.9.14" -.\" Copyright 2012-2016 Howard Chu, Symas Corp. All Rights Reserved. +.\" Copyright (c) 2015,2016 Leonid Yuriev . +.\" Copyright (c) 2015,2016 Peter-Service R&D LLC . +.\" +.\" This file is part of ReOpenMDBX. +.\" +.\" ReOpenMDBX is free software; you can redistribute it and/or modify it under +.\" the terms of the GNU Affero General Public License as published by +.\" the Free Software Foundation; either version 3 of the License, or +.\" (at your option) any later version. +.\" +.\" ReOpenMDBX is distributed in the hope that it will be useful, +.\" but WITHOUT ANY WARRANTY; without even the implied warranty of +.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +.\" GNU Affero General Public License for more details. +.\" +.\" You should have received a copy of the GNU Affero General Public License +.\" along with this program. If not, see . +.\" +.\" --- +.\" +.\" Copyright 2012-2014 Howard Chu, Symas Corp. All Rights Reserved. .\" Copying restrictions apply. See COPYRIGHT/LICENSE. +.TH MDB_STAT 1 "2014/06/20" "LMDB 0.9.14" .SH NAME mdb_stat \- LMDB environment status tool .SH SYNOPSIS diff --git a/mdb_stat.c b/mdb_stat.c index b3533fdd..ab523467 100644 --- a/mdb_stat.c +++ b/mdb_stat.c @@ -1,6 +1,27 @@ /* mdb_stat.c - memory-mapped database status tool */ + /* - * Copyright 2011-2016 Howard Chu, Symas Corp. + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * --- + * + * Copyright 2011-2014 Howard Chu, Symas Corp. * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -11,6 +32,7 @@ * top-level directory of the distribution or, alternatively, at * . */ + #include #include #include diff --git a/mdbx.c b/mdbx.c new file mode 100644 index 00000000..05161b5a --- /dev/null +++ b/mdbx.c @@ -0,0 +1,447 @@ +/* + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + */ + +#include "mdbx.h" + +int mdb_runtime_flags = MDBX_DBG_PRINT +#if MDB_DEBUG + | MDBX_DBG_ASSERT +#endif +#if MDB_DEBUG > 1 + | MDBX_DBG_TRACE +#endif +#if MDB_DEBUG > 2 + | MDBX_DBG_AUDIT +#endif +#if MDB_DEBUG > 3 + | MDBX_DBG_EXTRA +#endif + ; + +static MDBX_debug_func *mdb_debug_logger; + +int mdbx_setup_debug(int flags, MDBX_debug_func* logger, long edge_txn); + +#include "mdb.c" + +int __cold +mdbx_setup_debug(int flags, MDBX_debug_func* logger, long edge_txn) { + unsigned ret = mdb_runtime_flags; + if (flags != (int) MDBX_DBG_DNT) + mdb_runtime_flags = flags; + if (logger != (MDBX_debug_func*) MDBX_DBG_DNT) + mdb_debug_logger = logger; +#if MDB_DEBUG + if (edge_txn != (long) MDBX_DBG_DNT) + mdb_debug_edge = edge_txn; +#else + (void) edge_txn; +#endif + return ret; +} + +static txnid_t __cold +mdbx_oomkick(MDB_env *env, txnid_t oldest) +{ + mdb_debug("DB size maxed out"); +#if MDBX_MODE_ENABLED + int retry; + txnid_t snap; + mdb_debug("DB size maxed out"); + + for(retry = 0; ; ++retry) { + int reader; + + if (mdb_reader_check(env, NULL)) + break; + + snap = mdb_find_oldest(env, &reader); + if (oldest < snap || reader < 0) { + if (retry && env->me_oom_func) { + /* LY: notify end of oom-loop */ + env->me_oom_func(env, 0, 0, oldest, snap - oldest, -retry); + } + return snap; + } + + MDB_reader *r; + pthread_t tid; + pid_t pid; + int rc; + + if (!env->me_oom_func) + break; + + r = &env->me_txns->mti_readers[ reader ]; + pid = r->mr_pid; + tid = r->mr_tid; + if (r->mr_txnid != oldest || pid <= 0) + continue; + + rc = env->me_oom_func(env, pid, (void*) tid, oldest, + mdb_meta_head_w(env)->mm_txnid - oldest, retry); + if (rc < 0) + break; + + if (rc) { + r->mr_txnid = ~(txnid_t)0; + if (rc > 1) { + r->mr_tid = 0; + r->mr_pid = 0; + mdbx_coherent_barrier(); + } + } + } + + if (retry && env->me_oom_func) { + /* LY: notify end of oom-loop */ + env->me_oom_func(env, 0, 0, oldest, 0, -retry); + } +#else + (void) oldest; + (void) mdb_reader_check(env, NULL); +#endif /* MDBX_MODE_ENABLED */ + return mdb_find_oldest(env, NULL); +} + +int __cold +mdbx_env_set_syncbytes(MDB_env *env, size_t bytes) +{ + if (unlikely(!env)) + return EINVAL; + + if(unlikely(env->me_signature != MDBX_ME_SIGNATURE)) + return MDB_VERSION_MISMATCH; + + env->me_sync_threshold = bytes; + return env->me_map ? mdb_env_sync(env, 0) : 0; +} + +void __cold +mdbx_env_set_oomfunc(MDB_env *env, MDBX_oom_func *oomfunc) +{ + if (likely(env && env->me_signature == MDBX_ME_SIGNATURE)) + env->me_oom_func = oomfunc; +} + +MDBX_oom_func* __cold +mdbx_env_get_oomfunc(MDB_env *env) +{ + return likely(env && env->me_signature == MDBX_ME_SIGNATURE) + ? env->me_oom_func : NULL; +} + +struct mdb_walk_ctx { + MDB_txn *mw_txn; + void *mw_user; + MDBX_pgvisitor_func *mw_visitor; +}; + +typedef struct mdb_walk_ctx mdb_walk_ctx_t; + + +/** Depth-first tree traversal. */ +static int __cold +mdb_env_walk(mdb_walk_ctx_t *ctx, const char* dbi, pgno_t pg, int flags, int deep) +{ + MDB_page *mp; + int rc, i, nkeys; + unsigned header_size, unused_size, payload_size, align_bytes; + const char* type; + + if (pg == P_INVALID) + return MDB_CORRUPTED; + + rc = mdb_page_get(ctx->mw_txn, pg, &mp, NULL); + if (rc) + return rc; + if (pg != mp->mp_p.p_pgno) + return MDB_CORRUPTED; + + nkeys = NUMKEYS(mp); + header_size = IS_LEAF2(mp) ? PAGEHDRSZ : PAGEBASE + mp->mp_lower; + unused_size = SIZELEFT(mp); + payload_size = 0; + + /* LY: Don't use mask here, e.g bitwise (P_BRANCH|P_LEAF|P_LEAF2|P_META|P_OVERFLOW|P_SUBP). + * Pages should not me marked dirty/loose or otherwise. */ + switch (mp->mp_flags) { + case P_BRANCH: + type = "branch"; + if (nkeys < 1) + return MDB_CORRUPTED; + break; + case P_LEAF: + type = "leaf"; + break; + case P_LEAF|P_SUBP: + type = "dupsort-subleaf"; + break; + case P_LEAF|P_LEAF2: + type = "dupfixed-leaf"; + break; + case P_LEAF|P_LEAF2|P_SUBP: + type = "dupsort-dupfixed-subleaf"; + break; + case P_META: + case P_OVERFLOW: + default: + return MDB_CORRUPTED; + } + + for (align_bytes = i = 0; i < nkeys; + align_bytes += ((payload_size + align_bytes) & 1), i++) { + MDB_node *node; + + if (IS_LEAF2(mp)) { + /* LEAF2 pages have no mp_ptrs[] or node headers */ + payload_size += mp->mp_ksize; + continue; + } + + node = NODEPTR(mp, i); + payload_size += NODESIZE + node->mn_ksize; + + if (IS_BRANCH(mp)) { + rc = mdb_env_walk(ctx, dbi, NODEPGNO(node), flags, deep); + if (rc) + return rc; + continue; + } + + assert(IS_LEAF(mp)); + if (node->mn_ksize < 1) + return MDB_CORRUPTED; + if (node->mn_flags & F_BIGDATA) { + MDB_page *omp; + pgno_t *opg; + size_t over_header, over_payload, over_unused; + + payload_size += sizeof(pgno_t); + opg = NODEDATA(node); + rc = mdb_page_get(ctx->mw_txn, *opg, &omp, NULL); + if (rc) + return rc; + if (*opg != omp->mp_p.p_pgno) + return MDB_CORRUPTED; + /* LY: Don't use mask here, e.g bitwise (P_BRANCH|P_LEAF|P_LEAF2|P_META|P_OVERFLOW|P_SUBP). + * Pages should not me marked dirty/loose or otherwise. */ + if (P_OVERFLOW != omp->mp_flags) + return MDB_CORRUPTED; + + over_header = PAGEHDRSZ; + over_payload = NODEDSZ(node); + over_unused = omp->mp_pages * ctx->mw_txn->mt_env->me_psize + - over_payload - over_header; + + rc = ctx->mw_visitor(*opg, omp->mp_pages, ctx->mw_user, dbi, + "overflow-data", 1, over_payload, over_header, over_unused); + if (rc) + return rc; + continue; + } + + payload_size += NODEDSZ(node); + if (node->mn_flags & F_SUBDATA) { + MDB_db *db = NODEDATA(node); + char* name = NULL; + + if (NODEDSZ(node) < 1) + return MDB_CORRUPTED; + if (! (node->mn_flags & F_DUPDATA)) { + name = NODEKEY(node); + int namelen = (char*) db - name; + name = memcpy(alloca(namelen + 1), name, namelen); + name[namelen] = 0; + } + rc = mdb_env_walk(ctx, (name && name[0]) ? name : dbi, + db->md_root, node->mn_flags & F_DUPDATA, deep + 1); + if (rc) + return rc; + } + } + + return ctx->mw_visitor(mp->mp_p.p_pgno, 1, ctx->mw_user, dbi, type, + nkeys, payload_size, header_size, unused_size + align_bytes); +} + +int __cold +mdbx_env_pgwalk(MDB_txn *txn, MDBX_pgvisitor_func* visitor, void* user) +{ + mdb_walk_ctx_t ctx; + int rc; + + if (unlikely(!txn)) + return MDB_BAD_TXN; + if (unlikely(txn->mt_signature != MDBX_MT_SIGNATURE)) + return MDB_VERSION_MISMATCH; + + ctx.mw_txn = txn; + ctx.mw_user = user; + ctx.mw_visitor = visitor; + + rc = visitor(0, 2, user, "lmdb", "meta", 2, sizeof(MDB_meta)*2, PAGEHDRSZ*2, + (txn->mt_env->me_psize - sizeof(MDB_meta) - PAGEHDRSZ) *2); + if (! rc && txn->mt_dbs[FREE_DBI].md_root != P_INVALID) + rc = mdb_env_walk(&ctx, "free", txn->mt_dbs[FREE_DBI].md_root, 0, 0); + if (! rc && txn->mt_dbs[MAIN_DBI].md_root != P_INVALID) + rc = mdb_env_walk(&ctx, "main", txn->mt_dbs[MAIN_DBI].md_root, 0, 0); + if (! rc) + rc = visitor(P_INVALID, 0, user, NULL, NULL, 0, 0, 0, 0); + return rc; +} + +/* attribute support functions for Nexenta ***********************************/ + +static __inline int +mdbx_attr_peek(MDB_val *data, mdbx_attr_t *attrptr) +{ + if (unlikely(data->mv_size < sizeof(mdbx_attr_t))) + return MDB_INCOMPATIBLE; + + if (likely(attrptr != NULL)) + *attrptr = *(mdbx_attr_t*) data->mv_data; + data->mv_size -= sizeof(mdbx_attr_t); + data->mv_data = likely(data->mv_size > 0) + ? ((mdbx_attr_t*) data->mv_data) + 1 : NULL; + + return MDB_SUCCESS; +} + +static __inline int +mdbx_attr_poke(MDB_val *reserved, MDB_val *data, mdbx_attr_t attr, unsigned flags) +{ + mdbx_attr_t *space = reserved->mv_data; + if (flags & MDB_RESERVE) { + if (likely(data != NULL)) { + data->mv_data = data->mv_size ? space + 1 : NULL; + } + } else { + *space = attr; + if (likely(data != NULL)) { + memcpy(space + 1, data->mv_data, data->mv_size ); + } + } + + return MDB_SUCCESS; +} + +int +mdbx_cursor_get_attr(MDB_cursor *mc, MDB_val *key, MDB_val *data, + mdbx_attr_t *attrptr, MDB_cursor_op op) +{ + int rc = mdbx_cursor_get(mc, key, data, op); + if (unlikely(rc != MDB_SUCCESS)) + return rc; + + return mdbx_attr_peek(data, attrptr); +} + +int +mdbx_get_attr(MDB_txn *txn, MDB_dbi dbi, + MDB_val *key, MDB_val *data, uint64_t *attrptr) +{ + int rc = mdbx_get(txn, dbi, key, data); + if (unlikely(rc != MDB_SUCCESS)) + return rc; + + return mdbx_attr_peek(data, attrptr); +} + +int +mdbx_put_attr(MDB_txn *txn, MDB_dbi dbi, + MDB_val *key, MDB_val *data, mdbx_attr_t attr, unsigned flags) +{ + MDB_val reserve = { + .mv_data = NULL, + .mv_size = (data ? data->mv_size : 0) + sizeof(mdbx_attr_t) + }; + + int rc = mdbx_put(txn, dbi, key, &reserve, flags | MDB_RESERVE); + if (unlikely(rc != MDB_SUCCESS)) + return rc; + + return mdbx_attr_poke(&reserve, data, attr, flags); +} + +int mdbx_cursor_put_attr(MDB_cursor *cursor, MDB_val *key, MDB_val *data, + mdbx_attr_t attr, unsigned flags) +{ + MDB_val reserve = { + .mv_data = NULL, + .mv_size = (data ? data->mv_size : 0) + sizeof(mdbx_attr_t) + }; + + int rc = mdbx_cursor_put(cursor, key, &reserve, flags | MDB_RESERVE); + if (unlikely(rc != MDB_SUCCESS)) + return rc; + + return mdbx_attr_poke(&reserve, data, attr, flags); +} + +int mdbx_set_attr(MDB_txn *txn, MDB_dbi dbi, + MDB_val *key, MDB_val *data, mdbx_attr_t attr) +{ + MDB_cursor mc; + MDB_xcursor mx; + MDB_val old_data; + mdbx_attr_t old_attr; + int rc; + + if (unlikely(!key || !txn)) + return EINVAL; + + if (unlikely(txn->mt_signature != MDBX_MT_SIGNATURE)) + return MDB_VERSION_MISMATCH; + + if (unlikely(!TXN_DBI_EXIST(txn, dbi, DB_USRVALID))) + return EINVAL; + + if (unlikely(txn->mt_flags & (MDB_TXN_RDONLY|MDB_TXN_BLOCKED))) + return (txn->mt_flags & MDB_TXN_RDONLY) ? EACCES : MDB_BAD_TXN; + + mdb_cursor_init(&mc, txn, dbi, &mx); + rc = mdb_cursor_set(&mc, key, &old_data, MDB_SET, NULL); + if (unlikely(rc != MDB_SUCCESS)) { + if (rc == MDB_NOTFOUND && data) { + mc.mc_next = txn->mt_cursors[dbi]; + txn->mt_cursors[dbi] = &mc; + rc = mdbx_cursor_put_attr(&mc, key, data, attr, 0); + txn->mt_cursors[dbi] = mc.mc_next; + } + return rc; + } + + rc = mdbx_attr_peek(&old_data, &old_attr); + if (unlikely(rc != MDB_SUCCESS)) + return rc; + + if (old_attr == attr && (!data || + (data->mv_size == old_data.mv_size + && memcpy(data->mv_data, old_data.mv_data, old_data.mv_size) == 0))) + return MDB_SUCCESS; + + mc.mc_next = txn->mt_cursors[dbi]; + txn->mt_cursors[dbi] = &mc; + rc = mdbx_cursor_put_attr(&mc, key, data ? data : &old_data, attr, MDB_CURRENT); + txn->mt_cursors[dbi] = mc.mc_next; + return rc; +} diff --git a/mdbx.h b/mdbx.h index db5bc55b..1477d953 100644 --- a/mdbx.h +++ b/mdbx.h @@ -1,22 +1,23 @@ /* - Copyright (c) 2015,2016 Leonid Yuriev . - Copyright (c) 2015,2016 Peter-Service R&D LLC. - - This file is part of ReOpenLDAP. - - ReOpenLDAP is free software; you can redistribute it and/or modify it under - the terms of the GNU Affero General Public License as published by - the Free Software Foundation; either version 3 of the License, or - (at your option) any later version. - - ReOpenLDAP is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU Affero General Public License for more details. - - You should have received a copy of the GNU Affero General Public License - along with this program. If not, see . -*/ + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + */ /* This is solution to provide flexible compatibility with the original liblmdb. diff --git a/midl.c b/midl.c index 8fae4693..9dcad71a 100644 --- a/midl.c +++ b/midl.c @@ -1,9 +1,28 @@ /** @file midl.c * @brief ldap bdb back-end ID List functions */ -/* $OpenLDAP$ */ -/* This work is part of OpenLDAP Software . + +/* + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. * - * Copyright 2000-2016 The OpenLDAP Foundation. + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * --- + * + * Copyright 2000-2014 The OpenLDAP Foundation. * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/midl.h b/midl.h index e2eb9780..2d6f2753 100644 --- a/midl.h +++ b/midl.h @@ -8,10 +8,29 @@ * This file is only used internally in libmdb and its definitions * are not exposed publicly. */ -/* $OpenLDAP$ */ -/* This work is part of OpenLDAP Software . + +/* + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. * - * Copyright 2000-2016 The OpenLDAP Foundation. + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * --- + * + * Copyright 2000-2014 The OpenLDAP Foundation. * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/mtest0.c b/mtest0.c index d47445f1..9b86cc0c 100644 --- a/mtest0.c +++ b/mtest0.c @@ -1,6 +1,26 @@ /* mtest.c - memory-mapped database tester/toy */ + /* - * Copyright 2011-2016 Howard Chu, Symas Corp. + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * + * Copyright 2011-2014 Howard Chu, Symas Corp. * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -11,6 +31,7 @@ * top-level directory of the distribution or, alternatively, at * . */ + #include #include #include diff --git a/mtest1.c b/mtest1.c index e0b00b01..61e09e83 100644 --- a/mtest1.c +++ b/mtest1.c @@ -1,22 +1,23 @@ /* - Copyright (c) 2015,2016 Leonid Yuriev . - Copyright (c) 2015,2016 Peter-Service R&D LLC. - - This file is part of ReOpenLDAP. - - ReOpenLDAP is free software; you can redistribute it and/or modify it under - the terms of the GNU Affero General Public License as published by - the Free Software Foundation; either version 3 of the License, or - (at your option) any later version. - - ReOpenLDAP is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU Affero General Public License for more details. - - You should have received a copy of the GNU Affero General Public License - along with this program. If not, see . -*/ + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + */ /* Based on mtest2.c - memory-mapped database tester/toy */ diff --git a/mtest2.c b/mtest2.c index 1247fecd..f9860c66 100644 --- a/mtest2.c +++ b/mtest2.c @@ -1,6 +1,27 @@ /* mtest2.c - memory-mapped database tester/toy */ + /* - * Copyright 2011-2016 Howard Chu, Symas Corp. + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * --- + * + * Copyright 2011-2014 Howard Chu, Symas Corp. * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/mtest3.c b/mtest3.c index 0c23ef0a..b1d30ca3 100644 --- a/mtest3.c +++ b/mtest3.c @@ -1,6 +1,27 @@ /* mtest3.c - memory-mapped database tester/toy */ + /* - * Copyright 2011-2016 Howard Chu, Symas Corp. + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * --- + * + * Copyright 2011-2014 Howard Chu, Symas Corp. * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/mtest4.c b/mtest4.c index ff146191..5cec46b7 100644 --- a/mtest4.c +++ b/mtest4.c @@ -1,6 +1,27 @@ /* mtest4.c - memory-mapped database tester/toy */ + /* - * Copyright 2011-2016 Howard Chu, Symas Corp. + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * --- + * + * Copyright 2011-2014 Howard Chu, Symas Corp. * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/mtest5.c b/mtest5.c index 6dc36126..31c92f01 100644 --- a/mtest5.c +++ b/mtest5.c @@ -1,6 +1,27 @@ /* mtest5.c - memory-mapped database tester/toy */ + /* - * Copyright 2011-2016 Howard Chu, Symas Corp. + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * --- + * + * Copyright 2011-2014 Howard Chu, Symas Corp. * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/mtest6.c b/mtest6.c index a1ac3abe..b150f1c4 100644 --- a/mtest6.c +++ b/mtest6.c @@ -1,6 +1,27 @@ /* mtest6.c - memory-mapped database tester/toy */ + /* - * Copyright 2011-2016 Howard Chu, Symas Corp. + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * --- + * + * Copyright 2011-2014 Howard Chu, Symas Corp. * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/reopen.h b/reopen.h index 733886f9..5c3ab4e0 100644 --- a/reopen.h +++ b/reopen.h @@ -1,44 +1,40 @@ /* - Copyright (c) 2015,2016 Leonid Yuriev . - Copyright (c) 2015,2016 Peter-Service R&D LLC. - - This file is part of ReOpenLDAP. - - ReOpenLDAP is free software; you can redistribute it and/or modify it under - the terms of the GNU Affero General Public License as published by - the Free Software Foundation; either version 3 of the License, or - (at your option) any later version. - - ReOpenLDAP is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU Affero General Public License for more details. - - You should have received a copy of the GNU Affero General Public License - along with this program. If not, see . -*/ - -/* LY: Please do not ask us for Windows support, just never! - * But you can make a fork for Windows, or become maintainer for FreeBSD... */ -#ifndef __gnu_linux__ -# error "ReOpenLDAP branch supports only GNU Linux" -#endif + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + */ #ifndef _REOPEN_H #define _REOPEN_H -#ifndef _GNU_SOURCE -# define _GNU_SOURCE -#endif +#ifndef __CLANG_PREREQ +# ifdef __clang__ +# define __CLANG_PREREQ(maj,min) \ + ((__clang_major__ << 16) + __clang_minor__ >= ((maj) << 16) + (min)) +# else +# define __CLANG_PREREQ(maj,min) (0) +# endif +#endif /* __CLANG_PREREQ */ #ifndef __has_attribute # define __has_attribute(x) (0) #endif -#if !defined(GCC_VERSION) && defined(__GNUC__) -# define GCC_VERSION (__GNUC__ * 1000 + __GNUC_MINOR__) -#endif /* GCC_VERSION */ - #if !defined(__thread) && (defined(_MSC_VER) || defined(__DMC__)) # define __thread __declspec(thread) #endif diff --git a/sample-bdb.txt b/sample-bdb.txt index 11aff13f..165585b9 100644 --- a/sample-bdb.txt +++ b/sample-bdb.txt @@ -2,8 +2,29 @@ * * Do a line-by-line comparison of this and sample-mdb.txt */ + /* - * Copyright 2012-2016 Howard Chu, Symas Corp. + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * --- + * + * Copyright 2012-2014 Howard Chu, Symas Corp. * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -14,6 +35,7 @@ * top-level directory of the distribution or, alternatively, at * . */ + #include #include #include diff --git a/sample-mdb.txt b/sample-mdb.txt index 5d337373..7ad5769d 100644 --- a/sample-mdb.txt +++ b/sample-mdb.txt @@ -2,8 +2,29 @@ * * Do a line-by-line comparison of this and sample-bdb.txt */ + /* - * Copyright 2012-2016 Howard Chu, Symas Corp. + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * --- + * + * Copyright 2012-2014 Howard Chu, Symas Corp. * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -14,6 +35,7 @@ * top-level directory of the distribution or, alternatively, at * . */ + #include #include "lmdb.h" diff --git a/tooltag b/tooltag deleted file mode 100644 index cf24ba7e..00000000 --- a/tooltag +++ /dev/null @@ -1,27 +0,0 @@ - - - mdb_copy_1 - mdb_copy - environment copy tool - mdb_copy.1 - - - mdb_dump_1 - mdb_dump - environment export tool - mdb_dump.1 - - - mdb_load_1 - mdb_load - environment import tool - mdb_load.1 - - - mdb_stat_1 - mdb_stat - environment status tool - mdb_stat.1 - - - mdb_chk_1 - mdb_chk - environment check tool - mdb_chk.1 - - diff --git a/wbench.c b/wbench.c index ed16099b..7901177c 100644 --- a/wbench.c +++ b/wbench.c @@ -1,22 +1,24 @@ /* - Copyright (c) 2015,2016 Leonid Yuriev . - Copyright (c) 2015,2016 Peter-Service R&D LLC. + * Copyright (c) 2015,2016 Leonid Yuriev . + * Copyright (c) 2015,2016 Peter-Service R&D LLC. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + */ - This file is part of ReOpenLDAP. - - ReOpenLDAP is free software; you can redistribute it and/or modify it under - the terms of the GNU Affero General Public License as published by - the Free Software Foundation; either version 3 of the License, or - (at your option) any later version. - - ReOpenLDAP is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU Affero General Public License for more details. - - You should have received a copy of the GNU Affero General Public License - along with this program. If not, see . -*/ #include #include #include diff --git a/yota_test1.c b/yota_test1.c index f1664d26..84ac0227 100644 --- a/yota_test1.c +++ b/yota_test1.c @@ -1,3 +1,24 @@ +/* + * Copyright (c) 2016 Leonid Yuriev . + * Copyright (c) 2015 Vladimir Romanov , Yota Lab. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + */ + #include #include diff --git a/yota_test2.c b/yota_test2.c index 26bd1334..1cf7df69 100644 --- a/yota_test2.c +++ b/yota_test2.c @@ -1,3 +1,24 @@ +/* + * Copyright (c) 2016 Leonid Yuriev . + * Copyright (c) 2015 Vladimir Romanov , Yota Lab. + * + * This file is part of ReOpenMDBX. + * + * ReOpenMDBX is free software; you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by + * the Free Software Foundation; either version 3 of the License, or + * (at your option) any later version. + * + * ReOpenMDBX is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + */ + #include #include