[Zope-DB] Re: BTreeFolder

[Shane Hathaway]

Toby Dickenson wrote:
> BTreeFolder has a problem that it doesnt store *all* of its data in the BTree. 
> It still has the tuple caching id and meta-type, thanks to it ObjectManager 
> base class. In this case it is a 40,000 element tuple. That would be enough 
> to get me worried.
> 
> (ps; I worry easily)

I've been updating BTreeFolder lately.  The latest code is called 
BTreeFolder2, though the reason I changed the name is now gone and I may 
decide to rename it back to BTreeFolder.  It's available at cvs.zope.org 
under /Products.

The newest code stores *all* subobject data in BTrees; no more giant 
tuple.  That giant tuple turned out to be a bit of a problem for very 
large folders, since every time you add or remove an item, a new 
multi-megabyte pickle is generated, transferred, and appended to a file. 
   That problem is gone now.  The newest code also features unit tests, 
a CMF-friendly version, conflict prevention, and a unique ID generation 
utility.

BTreeFolder is useful where you need something like a big dictionary 
that rarely gets exposed to the user, but you want it to remain 
discoverable and easily manipulated in emergencies.  It doesn't always 
take the place of something customized, but it's definitely more 
scalable than ever.

Shane