[ZODB-Dev] BTrees questions

[Jim Fulton]

Martijn Faassen wrote:
> 
> Jeremy Hylton wrote:
> > >>>>> "MF" == Martijn Faassen <faassen@vet.uu.nl> writes:
> [snip]
> >   MF> What happens if I keep a BTree in-memory anyway? I mean, if I
> >   MF> just create a BTree as a local variable, does the entire
> >   MF> structure exist in memory then? And how efficient is this? :)
> >
> > A BTree is a persistent object and so are each of its buckets.  If you
> > have a large BTree and lookup a specific key, you'll only touch a very
> > small number of nodes or buckets.  So that's very memory efficient.
> 
> But what if I don't attach the BTree to anything else? Will it still
> be persistent?

No.

> I guess the same applies to any object inheriting from
> Persistent, and I think they just stay in memory until they're attached
> to anything.

Right.

Note that BTrees offer advantages over buckets even in memory because, for
large data sets, the insertion cost is lower for BTrees due to some
memory-management details.

 
> [snip]
> > If you can use one of the I BTrees (IO, OI, II), you'll save memory
> > because the int part is stored directly in the BTree instead of using
> > a PyObject * to a Python int object.
> 
> Right, I'm using those a lot.
> 
> Sets by the way seem to give me the same problem as Buckets; when I
> try to access their keys they give me the same list assignment error.

That's not surprising since they share the same code as much as possible.
 
> Perhaps we should doublecheck whether sets and buckets should really not be
> used;

There is no reason why they should not be used, except for the bug you
found, of course. :)

> sets in particular seem useful and why are they exposed to Python
> (and have documentation in Interfaces.py) at all if we're not supposed to
> use them?

They are supposed to be used.

> Sets have their own single argument .insert() method and everything.

Yup.

Sets have a bonus feature that they can be used as sequences too.

> Sometimes I only care whether an item is there, and there's no associated
> value, and sets seem to fit,

Yes. Note that TreeSets have the same semantics (minus the sequence behavior).

> except that they're either buggy

I hope they are not buggy. They are obviously not bug free, as you've
demonstrated. 

> or I'm misusing them..

No, you aren't.

Jim

--
Jim Fulton           mailto:jim@zope.com       Python Powered!        
CTO                  (888) 344-4332            http://www.python.org  
Zope Corporation     http://www.zope.com       http://www.zope.org