[Zope3-dev] Re: RFC: File-system synchronization proposal: ExternalFile?

[Craeg K Strong]

Hello:

I am the author of ExternalFile, and I agree with what Jim says below.

I do, however, believe that there will be overlap between the
pluggable persistence adaptor and the synchronization stuff.  Or
at least that you will want to use them together.

In an ideal world, they would both be pluggable and I would
simply use them "together" so that I could influence the representation
of Zope objects in the filesystem and still get automatic
synchronization.

The forces are tough to balance:
a) loss-less file system representation
b) ability to use standard file-system based tools
c) round-trip synchronization (changes in one representation get propagated to 
the other)

But there has been work in this area.  For example, the W3C's annotea
effort concerns metadata about a document (dublin core type stuff)
separately from its file representation, and links it together automatically in 
Mozilla using annozilla and XPointers.  It is a pretty cool idea.

We have experimented with keeping the metadata as part of the file.
As in the following example:   myarticle.xml:
<?xml version="1.0"?>
<!DOCTYPE Article>
<Document>
   <MetaData>
     ...
    </MetaData>
    <Article>
    ...
     </Article>
</Document>

As well as keeping it separate.  The problem is that some metadata
feels like part of the content of the document (like the title) but other
metadata really feels more separate (like the list of editors, edit
dates, comments, etc.).  One man's metadata is another man's data.

For example, I could store a single Zope object in two separate files,
one for "data" and one for "metadata."  That way the "data" file
could be structured in a way that was natural and easy for humans
to read and tools to process.  I can always combine them easily with
XSLT.  The name of the metadata file could be constructed heuristically
from the name of the data file (which corresponds to the zope ID...)

In any event, I don't see any _one_ answer to this problem.  That is why I
would like to see flexibility in how objects are represented.
That way the default representation could be something convenient
for ZServer, but those of us who use lots of non-Zope filesystem-oriented
tools could plug in our own special schemas as needed.


--Craeg



Jim Fulton wrote:
> Jean Jordaan wrote:
> 
>> Jim Fulton wrote:
>>  > Chris Withers wrote:
>>  >> Is there overlap here with the AdaptableStorage stuff Shane did and
>>  >> the stuff the Python Persistence SIG is working on?
>>  >
>>  > Not really. This is a system for:
>>  >
>>  > - Representing Zope objects on the file system without loss of 
>> information
>>
>> How about ExternalFile?
>> "ExternalFile extends Zope to work with files in the filesystem."
>> http://www.zope.org/Members/arielpartners/ExternalFile
> 
> 
> ExternalFile does the opposite. It represents a file-system file as a
> Zope object.
> 
> What we need is a way to represent a Zope object as a file-system
> file (or directory). This is a bit harder, since Zope objects are a lot
> more structured than files.  An additional challenge is that we want to
> represent objects on the file system in a way that will be useful to
> human editors of the files and to file-system based tools, especially tools
> like CVS.
> 
> Jim