[ZODB-Dev] IStorageIteration

[Christian Theune]

Hi,

Christian Theune schrieb:
> Hi,
> 
>> On Feb 11, 2008, at 4:19 PM, Christian Theune wrote:
>>> Context could be given as a reference date that is opaque to the
>>> client
>>> and can differ from storage to storage, a file pointer could serve
>>> this
>>> purpose. The API still might include the `length` of the data returned
>>> to minimize round-trips.
>>
>> Actually, a file position doesn't work either, because the storage
>> server can't know that much about what it is serving.
> 
> I would have thought that this wouldn't matter if we declare the context
> `opaque`. As long as the storage can hand out a token that represents
> context and that can be transferred through zrpc ...
> 
>> I suspect the only sane approach is to make the storage server
>> maintain the iterator returned by the underlying storage on behalf of
>> the client.  This will cause resources to be allocated on the server
>> that might only be freed when the client disconnects.
> 
> Right, now this makes me feel uneasy. :)

We started sketching out what we need to do, but are blocked because we 
need to decide on the API

a) Do we want to change the iterator() method to not return an iterator 
but a list?

b) Can we add a new parameter to the iterator() method that limits the 
length of the list in addition to giving an upper bound through the stop 
argument?

Our proposal would be to:

def iterator(start=None, stop=None, count=None):
     """...

     If the count argument is not None, then iteration will end after at
     most `count` transactions.

     Returns a list of IStorageTransactionInformation objects.

     """


stop and count given together would limit the list to the condition that 
matches first.

Unfortunately this would mean:

a) a method called `iterator` does not return an iterator
b) an existing method changes its signature

The first effect is an argument for making ZEO work hard enough to pass 
the iterator through transparently.

The second effect (especially the count argument) is a functional 
requirement that we have, to be able to say `please give me 100 
transactions` instead of providing a time frame with an unknown amount 
of transactions in it. This would be an argument for a new interface.

Anyway, we need to get consent first before going forward.

Christian

-- 
gocept gmbh & co. kg - forsterstrasse 29 - 06112 halle (saale) - germany
www.gocept.com - ct at gocept.com - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development