[ZODB-Dev] ZODB memory problems

[Toby Dickenson]

On Tuesday 31 May 2005 18:43, Shane Hathaway wrote:
> Jeremy Hylton wrote:
> > It's really too bad that ZEO only allows a single outstanding request.
> >  Restructuring the protocol to allow multiple simulatenous requests
> > was on the task list years ago, but the protocol implementation is so
> > complex I doubt it will get done :-(.  I can't help but think building
> > on top of an existing message/RPC layer would be profitable.  (What's
> > twisted's RPC layer?

pb; perspective broker. I thought that suggestion was crazy when Jeremey first 
presented it in his blog, but increasing exposure to other twisted tools 
makes it seem more reasonable.

In short, Ive not yet been disappointed by anything coming from the twisted 
camp.

> > )  Or at least something less difficult to use 
> > than asyncore.
> 
> Do you think the RPC layer is the source of the problem?  I feel like
> the way ZODB uses references in pickles is the main thing that slows it
> down.  Even if you have a protocol that can request many objects at
> once, the unpickling machinery only asks for one at a time.

Both problems seem interestingly non-trivial; a storage layer that will 
prefetch related pickles, and an rpc/storage layer that can express those 
prefetching requests with appropriate security, concurrency, and 
prioritization.


-- 
Toby Dickenson