[ZODB-Dev] [Plone-Users] question about using repozo and blob storage

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ralf Hemmecke wrote:

> Subject: Re: [Plone-Users] question about using repozo and blob storage
> Date: Fri, 7 May 2010 10:05:59 -0700 (PDT)
> From: ajung <lists at zopyx.com>
> To: plone-users at lists.sourceforge.net
> 
> 
> This list is the wrong place for such deep ZODB related discussions. Suggest
> this on the zodb-dev list.
> 
> -aj
> 
> 
> Ralf Hemmecke-2 wrote:
>> On 05/07/2010 06:40 PM, ajung wrote:
>>> Blobs are - as David explained - a _core_ component of the ZODB. Content
>>> stored through the file storage or blob storage are handled and treated
>>> in
>>> the same way to the application. If you want to write a SCM-based backend
>>> for ZODB....I am not sure how far you will get and if it would be
>>> reasonably
>>> fast and without major architectural issues.
>> I understand that meta data lives still in ZODB. And I wouldn't want to
>> suggest that people deal with the blobs in any other way than through
>> plone. One should only be able to have read access to the blobstorage
>> (if it's not through plone, otherwise disaster may happen).
>>
>> I haven't actually looked into how blobs are actually stored on the file
>> system, i.e., in particular what name they will have and whether the
>> file system structure is the same as the folder hierarchy inside plone.
>> But that doesn't actually matter.
>>
>> I assume that they are stored in some directory in a directory for
>> blobs, let's call it blobdir.
>>
>> I speak now for git, but other SCMs are quite similar (maybe not svn).
>>
>> This blobdir is turned into a repository by simply saying "git init".
>>
>> Then when plone stores a file, it will write it into (a subdir of)
>> blobdir. With GIT behind that should read:
>>
>>   (1)  write it into blobdir and then
>>   (2) git add FILENAME"
>>   (3) git commit -m "some message"
>>
>> That's it.
>>
>> If the user uploads a new file to plone, then plone doesn't have to care
>> about renaming the old file and storing the new, it simply has to do THE
>> SAME steps (1)..(3) as above. I call that easier.
>>
>> Maybe packing is a bit more difficult and depends on the SCM in
>> question. For git it would be a simple "git filter-branch" (of course
>> with appropriate parameters) followed by "git gc".
>>
>> Just my two cents. I would find such a GIT backend wonderful. But I've
>> no experience with how that blob stuff is actually done through
>> plone/zope. So sorry if I cannot provide any code.

Sharing the blobs using a DVCS pull sounds like an interesting approach.
 One flaw with your outlined implementation is that ZODB blob files are
*never* updated:  rather, if a ZODB object updates the blob object, it
creates a new blob file and stores its ID.  Old blob files are cleaned
up during a pack.

If you want to experiment with the idea anyway, then writing a "wrapper"
storage is probably the way to go:  DemoStorage is one such beast, as is
the BlobStorage itself.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkvoTrgACgkQ+gerLs4ltQ5HBgCgwXNqzE4KYWAQZYEJMPSoi18t
09EAniD4ucOThPmJRJ8A9TjmVuQv3gQV
=QUON
-----END PGP SIGNATURE-----