[ZODB-Dev] Blobs, Copies, and Exports (Zope 3)

[Christian Theune]

On Fri, 2008-10-24 at 11:45 -0600, Jeff Shell wrote:
> On Oct 24, 2008, at 8:36 AM, Christian Theune wrote:
> > Again: blobs export as expected.
> >
> > However, Jeff pickles/unpickles the subtree to get rid of the
> > '__parent__' attribute. This and blobs' inability to get copied around
> > in this specific manner make him end up with empty blobs.
> 
> 
> This is the real issue. What can be done to address it? This sounds  
> like something that should be handled at/near the ZODB/Pickle/Copy  
> (deep copy?) level.
> 
> While doing some google searches on this issue, I saw others having  
> same or similar concerns when using Blobs with other tools that did  
> deep copies of objects, although their bugs seemed related more to the  
> issue of having 'None' values in the readers and writers attributes of  
> the Blob file (a bug which has been fixed).
> 
> The initial traceback in this bug report is tied to that issue:
> 
> https://bugs.launchpad.net/zope3/+bug/240381
> 
> But then things swing around to actual content copying. Christophe  
> Combelles last comment (from June) shows manually patching  
> 'zope.location....locationCopy' to address his particular situation.  
> 'loc' is the original 'located' object being copied.
> 
>      copied = unpickler.load()
>      if hasattr(copied, '_data') and type(copied._data) is Blob:
>          copied._data._create_uncommitted_file()
>          targetfile = open(copied._data._current_filename(), 'w')
>          sourcefile = loc.openDetached()
>          chunk = sourcefile.read(1000000)
>          while chunk:
>              targetfile.write(chunk)
>              chunk = sourcefile.read(1000000)
>          targetfile.close()
>      return copied
> 
> That's obviously not a general solution, or the most efficient, but  
> I'm guessing it's pretty close to what needs to happen...
> 
> I just verified that with our CMS (built on Zope 3) and custom  
> 'BlobFile' content type object, copying the BlobFile content type  
> object copies all of the other persistent data of the object, but the  
> Blob contents are indeed empty. This is using a copy/paste system  
> built on top of zope.copypastemove.

There are two general approaches that I can imagine:

or a blob that is a copy of another, keep the original oid/tid pair
around from which it was copied implementing 'copy on write' semantics.
This has packing issues from what I can see.

The other would be to create a hard link to the file that matched the
content at the given point in time. This has transaction race conditions
from what I see.

I'd like to avoid copies though.

-- 
Christian Theune · ct at gocept.com
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 7 · fax +49 345 1229889 1
Zope and Plone consulting and development
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url : http://mail.zope.org/pipermail/zodb-dev/attachments/20081025/1b60afa9/attachment.bin