From mj at zope.com Tue Oct 16 13:48:17 2001 From: mj at zope.com (Martijn Pieters) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] Test Message-ID: <20011016134814.I4414@zope.com> This is the standard first-email-to-the-list test message. -- Martijn Pieters | Software Engineer mailto:mj@zope.com | Zope Corporation http://www.zope.com/ | Creators of Zope http://www.zope.org/ --------------------------------------------- From faassen at vet.uu.nl Wed Oct 17 18:32:51 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] Kickoff message Message-ID: <20011018003251.A32744@vet.uu.nl> Hello everybody, I see this list is gaining subscribers rapidly, and since I said I'd try to make it interesting, I feel responsible to send a message. :) I'll start with a status update on various Zope/XML matters that I know something about. Firstly there's ParsedXML; it's been languishing away for months now without much happening to it at all. I was sitting on a large patch that speeds up the DOM/Zope interface by a lot, and wanted to clean up other stuff, but I couldn't do it very well as I didn't have CVS access. Recently that changed. :) So, my speedup patch is now sitting in a branch in the CVS, waiting to be integrated with the ParsedXML mainline. In a sandbox on my machine I've also restructured the unit test suite of ParsedXML to be more conformant with the Zope guidelines (as soon as I bugged people enough so they would actually describe the guidelines on this :). Lots of tests currently fail with Python 2.1 (and thus Zope 2.4), and we need to fix that too. I hope I can move things so we can do a new release by next week sometime, hopefully working a bit better with Zope 2.4, but that doesn't depend on me only so I may be completely off. Another recent development has been XPath Methods. If you manage to install all the dependencies for PyXML and 4Suite and the phase of the moon is right, XPath Methods will allow you to do XPath queries against ParsedXML and possibly in the future also other DOMs, maybe even an all new ZDOM layer over common Zope objects.. Anyway, all of this needs lots of help and there are many other things to be integrated well (XSLT comes to mind), so I will leave the floor for a while to let you all discuss how we're to approach things. :) I hope we'll have a healthy cooperation! Let's hope the whole Zope XML integration can sustain some development momentum this time. For this open source style participation seems to be needed. Regards, Martijn From fslager at phil.uu.nl Thu Oct 18 05:26:46 2001 From: fslager at phil.uu.nl (Felix Slager) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] sideline question about some ambiguity Message-ID: <200110180926.f9I9Ql503980@siberia.cats.nl> Hi everybody, after installing XPath, ParsedXML i got the following feedback: PROBLEM(100) Init Ambiguous name for method of Products.ParsedXML.ManageableDOM.DOMManageable: "manage_main" != "manage_DOMTree" It might be that this is not related to the XPath ParsedXML products, but due to some garbled installation of the broader environment (i.e. Zope 2.4.1 (source release, python 2.1, linux2), python 2.1.1, linux2). What can i do about this small hitch? Okay moderator - eat your heart out ... should this posting go to the ParsedXML list instead? From jeffh at sctconsulting.com Thu Oct 18 05:19:41 2001 From: jeffh at sctconsulting.com (Jeffrey Hulten) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] sideline question about some ambiguity In-Reply-To: <200110180926.f9I9Ql503980@siberia.cats.nl>; from fslager@phil.uu.nl on Thu, Oct 18, 2001 at 11:26:46AM +0200 References: <200110180926.f9I9Ql503980@siberia.cats.nl> Message-ID: <20011018021941.B22803@sctconsulting.com> I get that on several packages. Was there a change in 2.4 that influenced this? Thus spake Felix Slager (fslager@phil.uu.nl): > Hi everybody, > > after installing XPath, ParsedXML i got the following feedback: > > PROBLEM(100) Init Ambiguous name for method of > Products.ParsedXML.ManageableDOM.DOMManageable: "manage_main" != > "manage_DOMTree" > > It might be that this is not related to the XPath ParsedXML products, but due > to some garbled installation of the broader environment (i.e. Zope 2.4.1 > (source release, python 2.1, linux2), python 2.1.1, linux2). > > What can i do about this small hitch? > > > > Okay moderator - eat your heart out ... should this posting go to the > ParsedXML list instead? > > _______________________________________________ > Zope-xml mailing list > Zope-xml@zope.org > http://lists.zope.org/mailman/listinfo/zope-xml -- Jeffrey Hulten SCT Consulting (206) 853-5216 From faassen at vet.uu.nl Thu Oct 18 08:22:31 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] sideline question about some ambiguity In-Reply-To: <20011018021941.B22803@sctconsulting.com> References: <200110180926.f9I9Ql503980@siberia.cats.nl> <20011018021941.B22803@sctconsulting.com> Message-ID: <20011018142231.A2574@vet.uu.nl> Jeffrey Hulten wrote: > I get that on several packages. Was there a change in 2.4 that > influenced this? This is a harmless 2.4 glitch as far as I know; you get it without XPath as well. We should be able to eliminate this by the next release of ParsedXML. There's somekind of magic incantation I need to look up necessary to quiet Zope. I'm not quite sure why the warning was introduced in the first place. Hoi Felix by the way! :) Groeten, Martijn From mj at zope.com Thu Oct 18 10:02:11 2001 From: mj at zope.com (Martijn Pieters) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] sideline question about some ambiguity In-Reply-To: <200110180926.f9I9Ql503980@siberia.cats.nl> References: <200110180926.f9I9Ql503980@siberia.cats.nl> Message-ID: <20011018100207.F13783@zope.com> On Thu, Oct 18, 2001 at 11:26:46AM +0200, Felix Slager wrote: > after installing XPath, ParsedXML i got the following feedback: > > PROBLEM(100) Init Ambiguous name for method of > Products.ParsedXML.ManageableDOM.DOMManageable: "manage_main" != > "manage_DOMTree" > > It might be that this is not related to the XPath ParsedXML products, but due > to some garbled installation of the broader environment (i.e. Zope 2.4.1 > (source release, python 2.1, linux2), python 2.1.1, linux2). > > What can i do about this small hitch? Uhm, if I can remember the fix again, it's very simple really. IIRC one of the methods mentioned is an alias for the other method, but there is no security assertion for one of them. Otherwise this message is harmless and you can ignore it. -- Martijn Pieters | Software Engineer mailto:mj@zope.com | Zope Corporation http://www.zope.com/ | Creators of Zope http://www.zope.org/ --------------------------------------------- From delza at landry.alliances.org Thu Oct 18 11:13:52 2001 From: delza at landry.alliances.org (delza@landry.alliances.org) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] XPath Methods and namespaces Message-ID: <20011018151352.7E8834E98E@landry.alliances.org> Hi folks, Does the XPath Methods product for ParsedXML support XML namespaces? I've had a look through the code and don't see anything, just curious. --Dethe ========================================================================== | dethe@burningtiger.com | http://livingcode.manilasites.com | ========================================================================== "Well I've wrestled with reality for thirty-five years now, doctor, and I'm happy to state I've finally won out over it." -- Elwood P. Dowd, Harvey From faassen at vet.uu.nl Thu Oct 18 13:16:38 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] XPath Methods and namespaces In-Reply-To: <20011018151352.7E8834E98E@landry.alliances.org> References: <20011018151352.7E8834E98E@landry.alliances.org> Message-ID: <20011018191638.A4149@vet.uu.nl> delza@landry.alliances.org wrote: > Does the XPath Methods product for ParsedXML support XML namespaces? > > I've had a look through the code and don't see anything, just curious. Um, I don't know. It should to the extent that ParsedXML and 4Suite's XPath implementation support namespaces, but I'm not much of a namespce guru so perhaps there are special UI or API requirements you have? We could see about adding something. Regards, Martijn From delza at landry.alliances.org Thu Oct 18 13:11:54 2001 From: delza at landry.alliances.org (delza@landry.alliances.org) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] XPath Methods and namespaces In-Reply-To: <20011018191638.A4149@vet.uu.nl> from "Martijn Faassen" at Oct 18, 2001 07:16:38 PM Message-ID: <20011018171154.2CE224E98E@landry.alliances.org> Martijn Faassen wrote: > Um, I don't know. It should to the extent that ParsedXML and 4Suite's > XPath implementation support namespaces, but I'm not much of a namespce > guru so perhaps there are special UI or API requirements you have? > We could see about adding something. 4Suite supports XML namespaces, but you have to add them when you create the Context object. I found this very confusing at first, especially since documents which have a default namespace can't use a "default" prefix (a null string). Prefixes used to define namespaces in the original document do not have to match prefixes in the XPath context (but the namespace strings, the URLs, do). If all of this is just adding to the confusion, I'll try to post my XPath for ZPT tonight, which uses 4Suite and namespaces. I've tested it with ParsedXML, but not with ParsedXML + Namespaces yet due to a problem with my ParsedXML install--I can't create new instances. I'll try to test in anyway on the train home and let you know the results. --Dethe ========================================================================== | dethe@burningtiger.com | http://livingcode.manilasites.com | ========================================================================== "Well I've wrestled with reality for thirty-five years now, doctor, and I'm happy to state I've finally won out over it." -- Elwood P. Dowd, Harvey From faassen at vet.uu.nl Thu Oct 18 15:00:38 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] XPath Methods and namespaces In-Reply-To: <20011018171154.2CE224E98E@landry.alliances.org> References: <20011018191638.A4149@vet.uu.nl> <20011018171154.2CE224E98E@landry.alliances.org> Message-ID: <20011018210038.A4826@vet.uu.nl> delza@landry.alliances.org wrote: > Martijn Faassen wrote: > > Um, I don't know. It should to the extent that ParsedXML and 4Suite's > > XPath implementation support namespaces, but I'm not much of a namespce > > guru so perhaps there are special UI or API requirements you have? > > We could see about adding something. > > 4Suite supports XML namespaces, but you have to add them when you create the > Context object. I found this very confusing at first, especially since > documents which have a default namespace can't use a "default" prefix (a null > string). Prefixes used to define namespaces in the original document do not > have to match prefixes in the XPath context (but the namespace strings, the > URLs, do). > If all of this is just adding to the confusion, I'll just say you're for now the Zope-XML XPath namespace guru. I myself at least will depend on your zen. :) > I'll try to post > my XPath for ZPT tonight, which uses 4Suite and namespaces. I've tested it > with ParsedXML, but not with ParsedXML + Namespaces yet due to a problem with > my ParsedXML install--I can't create new instances. I'll try to test in > anyway on the train home and let you know the results. Okay, sounds good. Looking forward to seeing it! Regards, Martijn From delza at alliances.org Fri Oct 19 01:13:41 2001 From: delza at alliances.org (Dethe Elza) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] ANNOUNCE: PT_XPath References: <20011018191638.A4149@vet.uu.nl> <20011018171154.2CE224E98E@landry.alliances.org> <20011018210038.A4826@vet.uu.nl> Message-ID: <3BCFB685.7010105@alliances.org> Thanks to Martijn F. and Evan I have posted my first Zope product: http://www.zope.org/Members/DaddyGravity/PT_XPath/index_html It adds an xpath: expression type to TALES expressions in Zope Page Templates. If you don't know what XPath is, it might not help you much %-) Currently supports ParsedXML and well-formed XML in a string. Namespaces should be fully supported. I'm still working on Zope Hierarchy-as-XML and other document types. Feedback encouraged and welcome. Thanks guys! --Dethe -- Dethe Elza (delza@burningtiger.com) Chief Mad Scientist Burning Tiger Technologies (http://burningtiger.com) Living Code Weblog (http://livingcode.ca) From faassen at vet.uu.nl Fri Oct 19 06:26:42 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] ANNOUNCE: PT_XPath In-Reply-To: <3BCFB685.7010105@alliances.org> References: <20011018191638.A4149@vet.uu.nl> <20011018171154.2CE224E98E@landry.alliances.org> <20011018210038.A4826@vet.uu.nl> <3BCFB685.7010105@alliances.org> Message-ID: <20011019122642.A7331@vet.uu.nl> Dethe Elza wrote: > Thanks to Martijn F. and Evan I have posted my first Zope product: > > http://www.zope.org/Members/DaddyGravity/PT_XPath/index_html > > It adds an xpath: expression type to TALES expressions in Zope Page > Templates. If you don't know what XPath is, it might not help you much %-) > > Currently supports ParsedXML and well-formed XML in a string. Namespaces > should be fully supported. Cool! I'll take a peek. :) > I'm still working on Zope Hierarchy-as-XML > and other document types. I'm also working on ZDOM, i.e. Zope Hierarchy-as-XML. Right now I'm focusing on making it work read-only, and I do have a list of what methods need to be implemented for a read-only DOM. Care to work together? See also here: http://dev.zope.org/Wikis/DevSite/Proposals/ZDOMXPath Jay Dylan is also on this list, I believe. What we'd need is somekind of registry where for an arbitrary Zope object a DOM object will be returned, if possible. If not possible, some kind of DOM object should still be returned, I'm thinking about something like an element node. Element name would often be determined by meta_type, perhaps with spaces replaced by underscores, as I'm not sure on spaces in element names. Most of the work will go into designing the wrappers. I'd like all of this to be New Component Religion (like the piece of code that looks up the right DOM wrapper), but since that's not done let's just implement something that works, and care about new religion specifics later.. Regards, Martijn From delza at landry.alliances.org Fri Oct 19 13:26:41 2001 From: delza at landry.alliances.org (delza@landry.alliances.org) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] ANNOUNCE: PT_XPath In-Reply-To: <20011019122642.A7331@vet.uu.nl> from "Martijn Faassen" at Oct 19, 2001 12:26:42 PM Message-ID: <20011019172641.2D3C64E98D@landry.alliances.org> Martijn Faassen wrote: > > I'm still working on Zope Hierarchy-as-XML > > and other document types. > > I'm also working on ZDOM, i.e. Zope Hierarchy-as-XML. Right now I'm focusing > on making it work read-only, and I do have a list of what methods need > to be implemented for a read-only DOM. Care to work together? I'd love to help. I'll try to get up to speed with what's been done so far first, then see what I can add. I just finished up a big contract, so hopefully I will have more time to put into this. Where can I find the list of methods that need to be implemented. > See also here: > > http://dev.zope.org/Wikis/DevSite/Proposals/ZDOMXPath Wow. Very ambitious. This would turn Zope into a very full-featured XML respository. I wonder if it would be make sense to push some of the DOM support down into ZODB? I can't claim to understand all the issues there, though. > What we'd need is somekind of registry where for an arbitrary Zope object a > DOM object will be returned, if possible. If not possible, some kind of > DOM object should still be returned, I'm thinking about something like > an element node. I'm afraid I don't understand the need for a registry. Shouldn't we be patching low-level Zope objects with DOM methods so all objects inherit the DOM API? I've just started reading the ZDOMIssues document, so undoubtably I'm missing something. > Element name would often be determined by meta_type, perhaps with spaces > replaced by underscores, as I'm not sure on spaces in element names. That's correct, no XML whitespace (space, tab, various line breaks, etc.) is allowed in element names. > Most of the work will go into designing the wrappers. I'd like all of this > to be New Component Religion (like the piece of code that looks up the > right DOM wrapper), but since that's not done let's just implement > something that works, and care about new religion specifics later.. I've read the manifesto for the new religion, but I don't really have a clue about the actual implementation of it, I'm afraid. Looking forward to working together. --Dethe ========================================================================== | dethe@burningtiger.com | http://livingcode.manilasites.com | ========================================================================== "Well I've wrestled with reality for thirty-five years now, doctor, and I'm happy to state I've finally won out over it." -- Elwood P. Dowd, Harvey From faassen at vet.uu.nl Fri Oct 19 14:21:03 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] ANNOUNCE: PT_XPath In-Reply-To: <20011019172641.2D3C64E98D@landry.alliances.org> References: <20011019122642.A7331@vet.uu.nl> <20011019172641.2D3C64E98D@landry.alliances.org> Message-ID: <20011019202103.A9215@vet.uu.nl> delza@landry.alliances.org wrote: > > http://dev.zope.org/Wikis/DevSite/Proposals/ZDOMXPath > > Wow. Very ambitious. This would turn Zope into a very full-featured > XML respository. I wonder if it would be make sense to push some of the > DOM support down into ZODB? I can't claim to understand all the issues > there, though. How do you see pushing that into the ZODB? The ZODB just stores objects, right, DOM or not? > > What we'd need is somekind of registry where for an arbitrary Zope object a > > DOM object will be returned, if possible. If not possible, some kind of > > DOM object should still be returned, I'm thinking about something like > > an element node. > > I'm afraid I don't understand the need for a registry. Shouldn't we be > patching low-level Zope objects with DOM methods so all objects inherit > the DOM API? I've just started reading the ZDOMIssues document, so > undoubtably I'm missing something. If you mean by 'low level Zope objects' things like Folder and DTML Method, I'm much in favor in making new objects that layer over these Zope objects. That'd be a much cleaner design. The *old* ZDOM works with mixin classes, but it never got very far and I think it's just too complicated. So I see somekind of registry that knows how to map any Zope object it encounters to a DOM object (usually an element with some attributes and probably some other nodes inside). This way you could even have different registries that do different mappings, some more detailed than the other. Some vague pondering of mine about this in the context of new religion can be found here: http://dev.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/ExplicitVersusImplicitFeatureWrapping and also here: http://dev.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/FeatureFinding [snip] > > Most of the work will go into designing the wrappers. I'd like all of this > > to be New Component Religion (like the piece of code that looks up the > > right DOM wrapper), but since that's not done let's just implement > > something that works, and care about new religion specifics later.. > > I've read the manifesto for the new religion, but I don't really have a clue > about the actual implementation of it, I'm afraid. There doesn't seem to be any actual implementation yet, so your lack of clue is understandable. ;) I do have a bit of non functional documentation sitting on my computer here that I hope to whip into shape enough for demo purposes soon. Regards, Martijn From delza at landry.alliances.org Fri Oct 19 14:15:48 2001 From: delza at landry.alliances.org (delza@landry.alliances.org) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] ANNOUNCE: PT_XPath In-Reply-To: <20011019202103.A9215@vet.uu.nl> from "Martijn Faassen" at Oct 19, 2001 08:21:03 PM Message-ID: <20011019181548.67F314E98D@landry.alliances.org> > > Wow. Very ambitious. This would turn Zope into a very full-featured > > XML respository. I wonder if it would be make sense to push some of the > > DOM support down into ZODB? I can't claim to understand all the issues > > there, though. > > How do you see pushing that into the ZODB? The ZODB just stores objects, > right, DOM or not? Well, I'm thinking in terms of a generic XML repository, like Tamino or XML-DB. If we're using XPath as a general query mechanism over a database, it makes sense to put it *in* the database. On the other hand, this wouldn't necessarily have to be baked into the ZODB, but could be an API which sits between the ZODB and Zope. Maybe I'm confusing the goals with those of other projects and this doesn't make sense at all. One of the things I like about ParsedXML, btw, is its balance of effeciency with persistence: Store one object in the ZODB, but make it look like an entire DOM tree to the user (at least, that's how I think it works!). Part of the reason I start thinking about the ZODB when we talk about querying for objects as if they were all XML is that it would be handy to do even outside of Zope, in other projects which use the ZODB. I would like to be able to use ParsedXML for general XML persistence from Python, but it seems to be closely tied to Zope. > > I'm afraid I don't understand the need for a registry. Shouldn't we be > > patching low-level Zope objects with DOM methods so all objects inherit > > the DOM API? I've just started reading the ZDOMIssues document, so > > undoubtably I'm missing something. > > If you mean by 'low level Zope objects' things like Folder and DTML Method, > I'm much in favor in making new objects that layer over these > Zope objects. That'd be a much cleaner design. The *old* ZDOM works > with mixin classes, but it never got very far and I think it's > just too complicated. > > So I see somekind of registry that knows how to map any Zope object it > encounters to a DOM object (usually an element with some attributes > and probably some other nodes inside). This way you could even have > different registries that do different mappings, some more detailed > than the other. I can see how a registry could be useful, but I think it should be used to *override* some reasonable default behavior. That is, any Zopish object should be accessible via ZDOM, and objects which have special needs should be able to register those to override the defaults. I'll read up on your thoughts on the matter. --Dethe ========================================================================== | dethe@burningtiger.com | http://livingcode.manilasites.com | ========================================================================== "Well I've wrestled with reality for thirty-five years now, doctor, and I'm happy to state I've finally won out over it." -- Elwood P. Dowd, Harvey From faassen at vet.uu.nl Fri Oct 19 18:54:36 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] ANNOUNCE: PT_XPath In-Reply-To: <20011019181548.67F314E98D@landry.alliances.org> References: <20011019202103.A9215@vet.uu.nl> <20011019181548.67F314E98D@landry.alliances.org> Message-ID: <20011020005436.A10255@vet.uu.nl> delza@landry.alliances.org wrote: > > > Wow. Very ambitious. This would turn Zope into a very full-featured > > > XML respository. I wonder if it would be make sense to push some of the > > > DOM support down into ZODB? I can't claim to understand all the issues > > > there, though. > > > > How do you see pushing that into the ZODB? The ZODB just stores objects, > > right, DOM or not? > > Well, I'm thinking in terms of a generic XML repository, like Tamino or > XML-DB. If we're using XPath as a general query mechanism over a database, it > makes sense to put it *in* the database. On the other hand, this wouldn't > necessarily have to be baked into the ZODB, but could be an API which sits > between the ZODB and Zope. Maybe I'm confusing the goals with those of > other projects and this doesn't make sense at all. You could use the plain ZODB as a DOM repository, which is of course nice and would be of use in Zope. What would also be nice however is to enable the use of XPath (and perhaps later XSLT) in other contexts, such as existing Zope databases. For that you either need to equip current object with DOM powers (using mixin classes), or you create a new set of objects that knows how to give DOM interfaces to the existing data. > One of the things I like about ParsedXML, btw, is its balance of effeciency > with persistence: Store one object in the ZODB, but make it look like an > entire DOM tree to the user (at least, that's how I think it works!). Right; the entire tree is currently persistent only as a whole, not as the individual objects. This has advantages, mostly I think storage efficiency and probably also some use efficiency. The disadvantage is that it doesn't scale all the way up to huge documents and its' probably relatively inefficient for DOM manipulations, as changes to the DOM cause the entire tree to be stored again in a new transaction.. Eventually a hybrid segmented DOM approach may be the best solution. > Part of > the reason I start thinking about the ZODB when we talk about querying for > objects as if they were all XML is that it would be handy to do even outside > of Zope, in other projects which use the ZODB. I would like to be able to use > ParsedXML for general XML persistence from Python, but it seems to be closely > tied to Zope. So you're thinking about a generic way to represent arbitrary Python objects as XML. It's an interesting idea that I need to think about a bit more. I'm talking about a higher level representation of particular existing Zope objects. The advantage I see in the latter is that it may make more sense to a human than the direct representation of object internals. An advantage of a more generic representation would be that you represent absolutely everything, but of course that can also be a weakness -- it is what makes XML .zexps rather hard to read. > > > I'm afraid I don't understand the need for a registry. Shouldn't we be > > > patching low-level Zope objects with DOM methods so all objects inherit > > > the DOM API? I've just started reading the ZDOMIssues document, so > > > undoubtably I'm missing something. > > > > If you mean by 'low level Zope objects' things like Folder and DTML Method, > > I'm much in favor in making new objects that layer over these > > Zope objects. That'd be a much cleaner design. The *old* ZDOM works > > with mixin classes, but it never got very far and I think it's > > just too complicated. > > > > So I see somekind of registry that knows how to map any Zope object it > > encounters to a DOM object (usually an element with some attributes > > and probably some other nodes inside). This way you could even have > > different registries that do different mappings, some more detailed > > than the other. > > I can see how a registry could be useful, but I think it should be used to > *override* some reasonable default behavior. That is, any Zopish object > should be accessible via ZDOM, and objects which have special needs should be > able to register those to override the defaults. That would be an interesting compromise, though I'm still not convinced you could actually expose the implementation details... hm, what about security issues, for instance? Regards, Martijn From doc at pcola.gulf.net Fri Oct 19 22:51:15 2001 From: doc at pcola.gulf.net (Watson, Brian) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] pyexpat.c help Message-ID: <3BD0E6A3.5CE72DB2@pcola.gulf.net> Hello, I'm trying to get ParsedXML to work in Zope 2.4.1 with Python 2.1. At the ParsedXML site at zope.org, there is a zipe file with the VC++ project file to build pyexpat.pyd. I get this error when building: Creating library Debug/pyexpat.lib and object Debug/pyexpat.exp pyexpat.obj : error LNK2001: unresolved external symbol __Py_Dealloc pyexpat.obj : error LNK2001: unresolved external symbol __Py_RefTotal pyexpat.obj : error LNK2001: unresolved external symbol __Py_NoneStruct pyexpat.obj : error LNK2001: unresolved external symbol _PyExc_IOError pyexpat.obj : error LNK2001: unresolved external symbol _PyExc_TypeError pyexpat.obj : error LNK2001: unresolved external symbol _PyFile_Type pyexpat.obj : error LNK2001: unresolved external symbol _PyExc_ValueError pyexpat.obj : error LNK2001: unresolved external symbol _PyString_Type pyexpat.obj : error LNK2001: unresolved external symbol _PyExc_AttributeError pyexpat.obj : error LNK2001: unresolved external symbol _PyExc_RuntimeError pyexpat.obj : error LNK2001: unresolved external symbol _Py_InitModule4TraceRefs pyexpat.obj : error LNK2001: unresolved external symbol _PyType_Type ../pyexpat.pyd : fatal error LNK1120: 12 unresolved externals Error executing link.exe. pyexpat.pyd - 13 error(s), 0 warning(s) Is this .c written for python1.5 only? I put py2.1 files in the include/lib path. Do I need a ver made to compile with 2.1? Also, what is pyexpat.pyd's role in zope (what does it do/used for)? ANY help would be very much appreciated; I've been trying for hours to make it work... Thanks in advance, Brian W. From faassen at vet.uu.nl Sat Oct 20 08:29:08 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] pyexpat.c help In-Reply-To: <3BD0E6A3.5CE72DB2@pcola.gulf.net> References: <3BD0E6A3.5CE72DB2@pcola.gulf.net> Message-ID: <20011020142908.A11915@vet.uu.nl> Hey, I've sent a cc of this reply to the ParsedXML-dev and Fred Drake. Fred hopes to start looking at the parser issues next week. Watson, Brian wrote: > I'm trying to get ParsedXML to work in Zope 2.4.1 with Python 2.1. At > the ParsedXML site at zope.org, there is a zipe file with the VC++ > project file to build pyexpat.pyd. I get this error when building: > > Creating library Debug/pyexpat.lib and object Debug/pyexpat.exp > pyexpat.obj : error LNK2001: unresolved external symbol __Py_Dealloc > pyexpat.obj : error LNK2001: unresolved external symbol __Py_RefTotal > pyexpat.obj : error LNK2001: unresolved external symbol __Py_NoneStruct > pyexpat.obj : error LNK2001: unresolved external symbol _PyExc_IOError > pyexpat.obj : error LNK2001: unresolved external symbol _PyExc_TypeError > pyexpat.obj : error LNK2001: unresolved external symbol _PyFile_Type > pyexpat.obj : error LNK2001: unresolved external symbol > _PyExc_ValueError > pyexpat.obj : error LNK2001: unresolved external symbol _PyString_Type > pyexpat.obj : error LNK2001: unresolved external symbol > _PyExc_AttributeError > pyexpat.obj : error LNK2001: unresolved external symbol > _PyExc_RuntimeError > pyexpat.obj : error LNK2001: unresolved external symbol > _Py_InitModule4TraceRefs > pyexpat.obj : error LNK2001: unresolved external symbol _PyType_Type > ../pyexpat.pyd : fatal error LNK1120: 12 unresolved externals > Error executing link.exe. > > pyexpat.pyd - 13 error(s), 0 warning(s) > > Is this .c written for python1.5 only? I put py2.1 files in the > include/lib path. Do I need a ver made to compile with 2.1? Also, what > is pyexpat.pyd's role in zope (what does it do/used for)? It's used by ParsedXML itself to do its parsing (translating the raw XML data into a DOM tree). I'm not entirely sure why we can't use Python 2.1's expat, but it seems it may miss some features that ParsedXML needs (and some Python builds apparently don't come with it at all). > ANY help > would be very much appreciated; I've been trying for hours to make it > work... Weird; it does compile for Python 2.1 on Linux so I'm at a loss why it wouldn't compile under Windows. However, even on Linux the compiled pyexpat seems to trigger segfault errors in the parser test suite, so something is still wrong (but some limited testing showed the expat that comes with Python 2.1 *also* suffers these problems..) I have no experience with compilation on Windows however. I do have an interest in seeing a Windows binary version of ParsedXML for Zope 2.4, so I hope we can figure it out.. Regards, Martijn From felix.slager at burningmail.com Sat Oct 20 09:30:24 2001 From: felix.slager at burningmail.com (felix slager) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] xml standard char set? (& minor zope thing ...) Message-ID: <20011020133024.DAD9B36EE@sitemail.everyone.net> Hi all, I was wondering about python and the Unicode - ISO/IEC 10646 charset standard for XML. Legal characters are tab, carriage return, line feed, and the legal characters of Unicode and ISO/IEC 10646. There's a problem when XML is manipulated via browser forms, like f.i. with XMLWidgets. Different browsers, even different OS versions of the same firm (i will use no names but ...) used in different countries (codepages) might well lead to difficulties. I haven't read everything about it yet - and before i try to do that, is there anyone out there who has? And knows a solution for this? By the way, just to be the naggin' horse i am, zope is inserting a nasty sub standard entity in my otherwise perfect html-4.01 template: The -->' /> '<-- part is not part of html. Shall i post a fix or will Zope 2.4.2 get rid of this ugly bird. Greetings _____________________________________________________________ Get Your Private, Free Email at http://www.BurningMail.com From mj at zope.com Sun Oct 21 09:26:43 2001 From: mj at zope.com (Martijn Pieters) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] xml standard char set? (& minor zope thing ...) In-Reply-To: <20011020133024.DAD9B36EE@sitemail.everyone.net> References: <20011020133024.DAD9B36EE@sitemail.everyone.net> Message-ID: <20011021092639.F2230@zope.com> On Sat, Oct 20, 2001 at 06:30:24AM -0700, felix slager wrote: > By the way, just to be the naggin' horse i am, zope is inserting a nasty > sub standard entity in my otherwise perfect html-4.01 template: > > > > The -->' /> '<-- part is not part of html. Shall i post a fix or will > Zope 2.4.2 get rid of this ugly bird. The trailing slash is there is to make it compatible with XHTML. The lash will be seen as an unknown attribute by non-XHTML browsers and ignored. See the W3C XHTML recommendations. -- Martijn Pieters | Software Engineer mailto:mj@zope.com | Zope Corporation http://www.zope.com/ | Creators of Zope http://www.zope.org/ --------------------------------------------- From djay at avaya.com Sun Oct 21 19:31:15 2001 From: djay at avaya.com (Jay, Dylan) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] ANNOUNCE: PT_XPath Message-ID: > -----Original Message----- > From: Martijn Faassen [mailto:faassen@vet.uu.nl] > Sent: Saturday, 20 October 2001 8:55 AM > To: delza@landry.alliances.org > Cc: Dethe Elza; zope-xml@zope.org > Subject: Re: [Zope-xml] ANNOUNCE: PT_XPath > > > > One of the things I like about ParsedXML, btw, is its > balance of effeciency > > with persistence: Store one object in the ZODB, but make it > look like an > > entire DOM tree to the user (at least, that's how I think > it works!). > > Right; the entire tree is currently persistent only as a whole, not as > the individual objects. This has advantages, mostly I think storage > efficiency and probably also some use efficiency. The disadvantage is > that it doesn't scale all the way up to huge documents and its' > probably relatively inefficient for DOM manipulations, as changes to > the DOM cause the entire tree to be stored again in a new > transaction.. > > Eventually a hybrid segmented DOM approach may be the best solution. Which is a much better idea if your treating a document as a database. Much more efficient for heavy querying. The only problem is you need a flexible way to represent the policy of what becomes objects and what elements are stored as one object. Perhaps some extension DTD could mark where the chunking occurs. > > Part of > > the reason I start thinking about the ZODB when we talk > about querying for > > objects as if they were all XML is that it would be handy > to do even outside > > of Zope, in other projects which use the ZODB. I would > like to be able to use > > ParsedXML for general XML persistence from Python, but it > seems to be closely > > tied to Zope. > > So you're thinking about a generic way to represent arbitrary Python > objects as XML. It's an interesting idea that I need to think > about a bit > more. I'm talking about a higher level representation of particular > existing Zope objects. The advantage I see in the latter is that it > may make more sense to a human than the direct representation > of object > internals. An advantage of a more generic representation would be > that you represent absolutely everything, but of course that can > also be a weakness -- it is what makes XML .zexps rather hard to > read. I think this problem will be solved to some extent if the component architecture is going to be a level between zope and the ZODB. I'm not sure if it is or not but this would mean you could request a given ZODB object as XML and it would happen for you. As to what the default view for unknown object types is... it's kind of abitrary. You can't make much guarentees about what the XML would look like for some new object type so why put anything in at all. I prefere the object representation. > > I can see how a registry could be useful, but I think it > should be used to > > *override* some reasonable default behavior. That is, any > Zopish object > > should be accessible via ZDOM, and objects which have > special needs should be > > able to register those to override the defaults. > > That would be an interesting compromise, though I'm still not > convinced > you could actually expose the implementation details... hm, > what about security > issues, for instance? A registry seems to be how the component archetecture is going to work so I'd say that would be the way to go. Less change when the component archecture does appear. Plus it was the advantage that no internal zope code needs to be modified which makes this much better for distribution as a product. From faassen at vet.uu.nl Mon Oct 22 02:48:53 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] xml standard char set? (& minor zope thing ...) In-Reply-To: <20011021092639.F2230@zope.com> References: <20011020133024.DAD9B36EE@sitemail.everyone.net> <20011021092639.F2230@zope.com> Message-ID: <20011022084853.A16317@vet.uu.nl> Martijn Pieters wrote: > On Sat, Oct 20, 2001 at 06:30:24AM -0700, felix slager wrote: > > By the way, just to be the naggin' horse i am, zope is inserting a nasty > > sub standard entity in my otherwise perfect html-4.01 template: > > > > > > > > The -->' /> '<-- part is not part of html. Shall i post a fix or will > > Zope 2.4.2 get rid of this ugly bird. > > The trailing slash is there is to make it compatible with XHTML. The lash > will be seen as an unknown attribute by non-XHTML browsers and ignored. See > the W3C XHTML recommendations. Isn't the actual backwards compatibility that there's to be a space between the / and the >? This ensures that it will be ignored. If there's actually no space added by Zope I think we should make sure it does. Regards, Martijn From sean.upton at uniontrib.com Mon Oct 22 11:38:40 2001 From: sean.upton at uniontrib.com (sean.upton@uniontrib.com) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] xml standard char set? (& minor zope thing ...) Message-ID: Right; my understanding is that this is fairly common and supposed to work with almost every browser, because it is treated as just another "unknown" attribute, because the lexical rules for what constitutes an attribute in a tag within most browser's parsers is farily forgiving to compensate for mistakes. Sean -----Original Message----- From: Martijn Faassen [mailto:faassen@vet.uu.nl] Sent: Sunday, October 21, 2001 11:49 PM To: Martijn Pieters Cc: felix slager; zope-xml@zope.org Subject: Re: [Zope-xml] xml standard char set? (& minor zope thing ...) Martijn Pieters wrote: > On Sat, Oct 20, 2001 at 06:30:24AM -0700, felix slager wrote: > > By the way, just to be the naggin' horse i am, zope is inserting a nasty > > sub standard entity in my otherwise perfect html-4.01 template: > > > > > > > > The -->' /> '<-- part is not part of html. Shall i post a fix or will > > Zope 2.4.2 get rid of this ugly bird. > > The trailing slash is there is to make it compatible with XHTML. The lash > will be seen as an unknown attribute by non-XHTML browsers and ignored. See > the W3C XHTML recommendations. Isn't the actual backwards compatibility that there's to be a space between the / and the >? This ensures that it will be ignored. If there's actually no space added by Zope I think we should make sure it does. Regards, Martijn _______________________________________________ Zope-xml mailing list Zope-xml@zope.org http://lists.zope.org/mailman/listinfo/zope-xml From mj at zope.com Mon Oct 22 11:40:15 2001 From: mj at zope.com (Martijn Pieters) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] ANNOUNCE: PT_XPath In-Reply-To: References: Message-ID: <20011022114007.C24158@zope.com> On Mon, Oct 22, 2001 at 09:31:15AM +1000, Jay, Dylan wrote: > > Eventually a hybrid segmented DOM approach may be the best solution. > > Which is a much better idea if your treating a document as a database. Much > more efficient for heavy querying. The only problem is you need a flexible > way to represent the policy of what becomes objects and what elements are > stored as one object. Perhaps some extension DTD could mark where the > chunking occurs. I think that XPath expressions would be a better siolution for specifying the chunking borders. > > > Part of the reason I start thinking about the ZODB when we talk about > > > querying for objects as if they were all XML is that it would be handy > > > to do even outside of Zope, in other projects which use the ZODB. I > > > would like to be able to use ParsedXML for general XML persistence > > > from Python, but it seems to be closely tied to Zope. Note that in my opinion, XML is not really suitable as a storage format. It is great for interchange between different entities because of the efficient and standard choices for parsing XML, but as soon as you rely on XML for storage, things become tricky and inefficient. You would be much better off converting from and to XML, which is what most storage solutions do. I really h?ve yet to come across convincing reasons for having a DOM on the server side, which is not a good situation to be in, seeing I am the owner of ParsedXML at Zope Corp... :| XMLPath expressions in TAL could just maybe be a reason.. > > So you're thinking about a generic way to represent arbitrary Python > > objects as XML. It's an interesting idea that I need to think about a > > bit more. I'm talking about a higher level representation of particular > > existing Zope objects. The advantage I see in the latter is that it may > > make more sense to a human than the direct representation of object > > internals. An advantage of a more generic representation would be that > > you represent absolutely everything, but of course that can also be a > > weakness -- it is what makes XML .zexps rather hard to read. To me the advantage would be for you to be able to use a standard, widespread API for object tree manipulation, the W3C DOM. Except that the DOM was designed for a particalr type of object tree, with a well-defined and finite set of object types and manipulations. A ZODB-stored object tree will almost always *not* fit that model, and making it fit will always necessitate cutting out functionality and power. -- Martijn Pieters | Software Engineer mailto:mj@zope.com | Zope Corporation http://www.zope.com/ | Creators of Zope http://www.zope.org/ --------------------------------------------- From sean.upton at uniontrib.com Mon Oct 22 11:48:24 2001 From: sean.upton at uniontrib.com (sean.upton@uniontrib.com) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] ANNOUNCE: PT_XPath Message-ID: What would be nice, perhaps is a new product: 'bigXML' is a folderish object containing 1..n ParsedXML objects, and acts as a proxy component that uses each ParsedXML instance to store DocumentFragments, as delineated by rules written as a bunch of properties in that folder that constitute rules for what gets stored where (perhaps expressed using XPath as a notation). This would allow for use of the existing ParsedXML framework, and allow improvements made to it to trickle into any product that proxied it. I know there is only so many degrees of 'proxying' that we should do, but this seems like a simple enough approach to get the job done? It would allow any document adminstrator that could write XPath to manage how branches are stored, which means they can control how the XML is optimized for their particular use case. Sean -----Original Message----- From: Jay, Dylan [mailto:djay@avaya.com] Sent: Sunday, October 21, 2001 4:31 PM To: zope-xml@zope.org Subject: RE: [Zope-xml] ANNOUNCE: PT_XPath > -----Original Message----- > From: Martijn Faassen [mailto:faassen@vet.uu.nl] > Sent: Saturday, 20 October 2001 8:55 AM > To: delza@landry.alliances.org > Cc: Dethe Elza; zope-xml@zope.org > Subject: Re: [Zope-xml] ANNOUNCE: PT_XPath > > > > One of the things I like about ParsedXML, btw, is its > balance of effeciency > > with persistence: Store one object in the ZODB, but make it > look like an > > entire DOM tree to the user (at least, that's how I think > it works!). > > Right; the entire tree is currently persistent only as a whole, not as > the individual objects. This has advantages, mostly I think storage > efficiency and probably also some use efficiency. The disadvantage is > that it doesn't scale all the way up to huge documents and its' > probably relatively inefficient for DOM manipulations, as changes to > the DOM cause the entire tree to be stored again in a new > transaction.. > > Eventually a hybrid segmented DOM approach may be the best solution. Which is a much better idea if your treating a document as a database. Much more efficient for heavy querying. The only problem is you need a flexible way to represent the policy of what becomes objects and what elements are stored as one object. Perhaps some extension DTD could mark where the chunking occurs. > > Part of > > the reason I start thinking about the ZODB when we talk > about querying for > > objects as if they were all XML is that it would be handy > to do even outside > > of Zope, in other projects which use the ZODB. I would > like to be able to use > > ParsedXML for general XML persistence from Python, but it > seems to be closely > > tied to Zope. > > So you're thinking about a generic way to represent arbitrary Python > objects as XML. It's an interesting idea that I need to think > about a bit > more. I'm talking about a higher level representation of particular > existing Zope objects. The advantage I see in the latter is that it > may make more sense to a human than the direct representation > of object > internals. An advantage of a more generic representation would be > that you represent absolutely everything, but of course that can > also be a weakness -- it is what makes XML .zexps rather hard to > read. I think this problem will be solved to some extent if the component architecture is going to be a level between zope and the ZODB. I'm not sure if it is or not but this would mean you could request a given ZODB object as XML and it would happen for you. As to what the default view for unknown object types is... it's kind of abitrary. You can't make much guarentees about what the XML would look like for some new object type so why put anything in at all. I prefere the object representation. > > I can see how a registry could be useful, but I think it > should be used to > > *override* some reasonable default behavior. That is, any > Zopish object > > should be accessible via ZDOM, and objects which have > special needs should be > > able to register those to override the defaults. > > That would be an interesting compromise, though I'm still not > convinced > you could actually expose the implementation details... hm, > what about security > issues, for instance? A registry seems to be how the component archetecture is going to work so I'd say that would be the way to go. Less change when the component archecture does appear. Plus it was the advantage that no internal zope code needs to be modified which makes this much better for distribution as a product. _______________________________________________ Zope-xml mailing list Zope-xml@zope.org http://lists.zope.org/mailman/listinfo/zope-xml From mj at zope.com Mon Oct 22 11:48:48 2001 From: mj at zope.com (Martijn Pieters) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] xml standard char set? (& minor zope thing ...) In-Reply-To: References: Message-ID: <20011022114845.F24158@zope.com> > > The trailing slash is there is to make it compatible with XHTML. The > > lash will be seen as an unknown attribute by non-XHTML browsers and > > ignored. See the W3C XHTML recommendations. > > Isn't the actual backwards compatibility that there's to be a space > between the / and the >? This ensures that it will be ignored. If there's > actually no space added by Zope I think we should make sure it does. The space is there. Note that Image objects generate an tag as well. -- Martijn Pieters | Software Engineer mailto:mj@zope.com | Zope Corporation http://www.zope.com/ | Creators of Zope http://www.zope.org/ --------------------------------------------- From elena.schulz at gmx.net Mon Oct 22 12:53:01 2001 From: elena.schulz at gmx.net (Elena Schulz) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] parsedXML-Installation Problem References: Message-ID: <000801c15b1a$35701620$c205e0d4@Sebastian> Hi everybody, I tried to install ParsedXML-1.1b1-win32-x86 on my Zope2.4.2 on a win98-pc under python2.1 and I get problems when I tried to build the Expat-parser as written in the readme. Executing the setup.py and after the expression: "Building static Expat library..." python cannot find the modules os.system. Looking in my python-library: \bin\lib\ I cannot locate a function system() in the module os and the module sys.py that is also required in the setup.py. Can anybody tell me what is wrong or missing? Has anybody experience with ParsedXML-1.1b1-win32-x86 on my Zope2.4.2 on a win98-pc? Thanks for some hints, Elena From sean.upton at uniontrib.com Mon Oct 22 13:40:09 2001 From: sean.upton at uniontrib.com (sean.upton@uniontrib.com) Date: Sun Aug 10 16:54:49 2008 Subject: storing XML can be good... was Re: [Zope-xml] ANNOUNCE: PT_XPath Message-ID: I hear more and more often that XML should not be stored, and just be an interchange format. And in most cases, that is likely to be correct, but... I think there are some very convincing reasons to, indeed, store XML on the server for particular use cases. The best possible illustration that I can think of is a use case that I have; sorry that this 'manifesto for storing XML' is so lengthy, but I think this position needs definitive justification - and I think that for this use case below, I'm right... ;) Eventually, for our internal use, my company will likely be creating a product that will support the use and storage of NITF (News Industry Text Format) and NewsML - both standards for news content defined by the IPTC (International Press Telecommunications Council) and/or the Newspaper Association of America (NAA). These DTDs were designed to address deficiencies in current information flow in the newspaper industry; NITF, in particular, began in the pre-XML days, as an SGML DTD with the specific goal of allowing news producers/vendors to scrap the 150 or so different proprietary text interchange and storage formats used by the industry. The idea was that in translation (filtering) between lots of formats, information is almost always lost, and always costly. One primary idea behind the migration to NITF XML was the idea of 'value added' data continuing to be added at every step of the workflow. That is, rather than losing quality of content at every step of the workflow, content gains quality: more markup, more specificity, more info to help increase the quality of the content, which CAN be monetized, because it avoids human labor. Two things prevent this: filtering between formats (not lossless), and 'flattening' data to a prescribed set of flat fields. If your DTD supports hundreds of fields, and less than 'tabular' datasets, flattening to say 30 or 40 fields is a bad, bad thing, but to date, AFAIK, all news vendors supporting this standard do it. The lifeline of a news story clearly demonstrates the need to add value (not subtract it with flattening) at every step: >News Assignment created by editor, kept as metadata ->Reporter creates story [Editorial System] -->Editor edits story [Editorial System] --->Copy is edited for print [Pagination System] ---->Edited Copy is fed back (merged) into editorial system from paginated output ----->Stories are imported into online CMS system for www/wireless/PDA/web-services, etc publishing ------>Stories edited by online staff (adding data like links to photos, cleaning up cutlines to make them useable). ------->Stories are merged back into database or archive system either in editorial system or 3rd party archive/library system. -------->Librarians add lots of data and subject/topic info and --------->Librarians export stories to vendors like AOL, Lexis-Nexis, and Wire Services --------->Wire services add markup to story for re-use ---------->Newspaper X uses story from wire service, edits and adds their own text and metadata. ----------->[Cycle is repeated until story end-of-life] At every step of the way, revision history metadata is kept in the XML. If at every step of the way, the XML was NOT flattened, the maximum amount of possible data is kept within this lifeline. No news vendor systems (that I know of) do this yet, which is disappointing, considering that this was the 'vision' for the creation of this DTD (NITF) in the first place. If I republish a syndicated story from, say, the Washington Post or AP, there should be no reason that meta-data and/or unused content that my APIs do not support to be stripped out. The problem is, IMHO, ignorant newspaper production system vendors, who love relational data stores so much, that they would love nothing more than to flatten XML into a tabular structure, with finite n number of relations between tables which cannot possibly express a multi-dimensional XML structure hierarchy without loss. This is the promise that Zope with ParsedXML has, by coupling a post-parsed DOM with object persistence machinery, we are allowed to: - Treat the XML as a online document - Treat the XML as a queryable fielded database - Use the XML without data loss due to DOM-based storage - Bolt-on APIs including web-services APIs allowing the reuse of the document via 'flattened' field queries (like an RDB) but storing the most possible data without loss. XPath, Object Query languages, and custom extraction and editing APIs would provide for this need. These APIs could be used by TAL, search/catalog systems, and distributed / web services based clients. Frankly, this is the holy grail of newspaper production technology: the centralized database for content, which to date, are usually RDB stores with lost of complex relations constituting lossy systems; the crazy thing about it is that newspapers are paying tens of millions of dollars for these centralized database systems for running their editorial and pagination systems. I think that XML storage done right (DOM storage coupled with and ODB and a data access API accessible to a broad range of client possibilities) is really something that needs to be done for the news media industry, and I strongly feel that any advances made in this manner in online production will trickle into the print production systems of newspapers and magazines eventually. And, for that, it seems to be enough justification to store XML on the server as XML (or DOM, which is even better). And in the realm of possibility, this is one thing that differentiates Zope from almost any other technology platform; I hope that's a good thing... Anyway, that's my rant, Sean ========================= Sean Upton Senior Programmer/Analyst SignOnSanDiego.com The San Diego Union-Tribune 619.718.5241 sean.upton@uniontrib.com ========================= From mj at zope.com Mon Oct 22 14:26:41 2001 From: mj at zope.com (Martijn Pieters) Date: Sun Aug 10 16:54:49 2008 Subject: storing XML can be good... was Re: [Zope-xml] ANNOUNCE: PT_XPath In-Reply-To: References: Message-ID: <20011022142637.J24158@zope.com> On Mon, Oct 22, 2001 at 10:40:09AM -0700, sean.upton@uniontrib.com wrote: > I hear more and more often that XML should not be stored, and just be an > interchange format. And in most cases, that is likely to be correct, but... > > I think there are some very convincing reasons to, indeed, store XML on the > server for particular use cases. The best possible illustration that I can > think of is a use case that I have; sorry that this 'manifesto for storing > XML' is so lengthy, but I think this position needs definitive justification > - and I think that for this use case below, I'm right... ;) > > Eventually, for our internal use, my company will likely be creating a > product that will support the use and storage of NITF (News Industry Text > Format) and NewsML - both standards for news content defined by the IPTC > (International Press Telecommunications Council) and/or the Newspaper > Association of America (NAA). These DTDs were designed to address > deficiencies in current information flow in the newspaper industry; NITF, in > particular, began in the pre-XML days, as an SGML DTD with the specific goal > of allowing news producers/vendors to scrap the 150 or so different > proprietary text interchange and storage formats used by the industry. The > idea was that in translation (filtering) between lots of formats, > information is almost always lost, and always costly. To me, this is still no reason to justify an XML storage. Note that Zope doesn't force you to use a fixed number of fields; the object tree and python provide you with much more flexibility. I am convinced that it is posible to build a Zope app with the same fidelity as the NITF, with lossless conversion between the internal storage format and NITF. Using Zope objects instead of XML would buy you speed and scalability. DOMs are memory hogs, and CPU intensive to build and manipulate. Only use a DOM when a custom API isn't feasable. Don't carry around all that extra weight if you can avoid it! -- Martijn Pieters | Software Engineer mailto:mj@zope.com | Zope Corporation http://www.zope.com/ | Creators of Zope http://www.zope.org/ --------------------------------------------- From sean.upton at uniontrib.com Mon Oct 22 14:57:32 2001 From: sean.upton at uniontrib.com (sean.upton@uniontrib.com) Date: Sun Aug 10 16:54:49 2008 Subject: storing XML can be good... was Re: [Zope-xml] ANNOUNCE: PT_XP ath Message-ID: But for value-added applications, I just don't have time to implement custom objects for every type of field I would need to store. If my organization only uses 12 fields (reads/writes to) out of 200 possible fields, there is no sense in having me value-subtact from the data that is on a complext journey between complex systems. DOM provides me the ability to have an object model, and not have to sacrifice date integrity, especially if Zope is a middle-point between several systems needing to intercahge data. Could I build my own object model and be able to expect better performance? Of course, but do I have time? I've got to be a pragmitist on this one most of the time. Caching mitigates some of the read-oriented performance concerns, anyway. I would rather just throw more hardware at Zope (i.e. more ZEO node boxes, more RAM & CPU speed) than have to lose this flexibility. What would be nice is if the likelyhood of write conflicts in a write-intensive system could be reduced by not needing to lock the whole DOM object, but this, seems to be only a minor issue for my particular application, so I think I'm willing to live with this as a limitaion, becuase adding useable fields becomes simply a matter of writing an API method (well 2 - one each for read / write), and not a whole new data structure / class. This means that if a user of software (not a core developer) doing this needed to extend the API on the fly with some TTW python scripts, they could quickly and easily if they could speak DOM. Sean -----Original Message----- From: Martijn Pieters [mailto:mj@zope.com] Sent: Monday, October 22, 2001 11:27 AM To: sean.upton@uniontrib.com Cc: zope-xml@zope.org Subject: Re: storing XML can be good... was Re: [Zope-xml] ANNOUNCE: PT_XPath On Mon, Oct 22, 2001 at 10:40:09AM -0700, sean.upton@uniontrib.com wrote: > I hear more and more often that XML should not be stored, and just be an > interchange format. And in most cases, that is likely to be correct, but... > > I think there are some very convincing reasons to, indeed, store XML on the > server for particular use cases. The best possible illustration that I can > think of is a use case that I have; sorry that this 'manifesto for storing > XML' is so lengthy, but I think this position needs definitive justification > - and I think that for this use case below, I'm right... ;) > > Eventually, for our internal use, my company will likely be creating a > product that will support the use and storage of NITF (News Industry Text > Format) and NewsML - both standards for news content defined by the IPTC > (International Press Telecommunications Council) and/or the Newspaper > Association of America (NAA). These DTDs were designed to address > deficiencies in current information flow in the newspaper industry; NITF, in > particular, began in the pre-XML days, as an SGML DTD with the specific goal > of allowing news producers/vendors to scrap the 150 or so different > proprietary text interchange and storage formats used by the industry. The > idea was that in translation (filtering) between lots of formats, > information is almost always lost, and always costly. To me, this is still no reason to justify an XML storage. Note that Zope doesn't force you to use a fixed number of fields; the object tree and python provide you with much more flexibility. I am convinced that it is posible to build a Zope app with the same fidelity as the NITF, with lossless conversion between the internal storage format and NITF. Using Zope objects instead of XML would buy you speed and scalability. DOMs are memory hogs, and CPU intensive to build and manipulate. Only use a DOM when a custom API isn't feasable. Don't carry around all that extra weight if you can avoid it! -- Martijn Pieters | Software Engineer mailto:mj@zope.com | Zope Corporation http://www.zope.com/ | Creators of Zope http://www.zope.org/ --------------------------------------------- From wade at lightlink.com Mon Oct 22 15:11:02 2001 From: wade at lightlink.com (Wade Leftwich) Date: Sun Aug 10 16:54:49 2008 Subject: storing XML can be good... was Re: [Zope-xml] ANNOUNCE: PT_XPath Message-ID: <200110221907.PAA20733@emerald.lightlink.com> I am in a situation similar to Sean's, sending and receiving a bunch of newsfeeds in NITF format. (Hi Sean, i think we met up on the NITF list a while back.) Because I'm adding on to (and hoping to replace) an ASP system, I've got all my data in MSSQL. My tables have the usual columns that Sean alluded to: headline, byline, date, etc. I also keep the original NITF- XML in a bigtext field. I run a cron job to rewrite the XML if any of the other fields gets edited. Right now I don't think I would want to give up the RDB storage, even if I didn't need it for historical reasons. While each article is a tree, the organization at higher levels is definitely tabular. And many people understand RDB's. I would love to be able to access the tree structure of each article within Zope, without first parsing XML to a DOM. Having done a fair amount of work with DOM, I hear what Martijn is saying about it being fat and slow. What about an object representation, like Martijn suggested, that could live in a BLOB in an RDB? Wade Leftwich Ithaca, NY 10/22/2001 2:26:41 PM, Martijn Pieters wrote: >On Mon, Oct 22, 2001 at 10:40:09AM -0700, sean.upton@uniontrib.com wrote: >> I hear more and more often that XML should not be stored, and just be an >> interchange format. And in most cases, that is likely to be correct, but... >> >> I think there are some very convincing reasons to, indeed, store XML on the >> server for particular use cases. The best possible illustration that I can >> think of is a use case that I have; sorry that this 'manifesto for storing >> XML' is so lengthy, but I think this position needs definitive justification >> - and I think that for this use case below, I'm right... ;) >> >> Eventually, for our internal use, my company will likely be creating a >> product that will support the use and storage of NITF (News Industry Text >> Format) and NewsML - both standards for news content defined by the IPTC >> (International Press Telecommunications Council) and/or the Newspaper >> Association of America (NAA). These DTDs were designed to address >> deficiencies in current information flow in the newspaper industry; NITF, in >> particular, began in the pre-XML days, as an SGML DTD with the specific goal >> of allowing news producers/vendors to scrap the 150 or so different >> proprietary text interchange and storage formats used by the industry. The >> idea was that in translation (filtering) between lots of formats, >> information is almost always lost, and always costly. > >To me, this is still no reason to justify an XML storage. Note that Zope >doesn't force you to use a fixed number of fields; the object tree and >python provide you with much more flexibility. I am convinced that it is >posible to build a Zope app with the same fidelity as the NITF, with >lossless conversion between the internal storage format and NITF. > >Using Zope objects instead of XML would buy you speed and scalability. DOMs >are memory hogs, and CPU intensive to build and manipulate. Only use a DOM >when a custom API isn't feasable. Don't carry around all that extra weight >if you can avoid it! > >-- >Martijn Pieters >| Software Engineer mailto:mj@zope.com >| Zope Corporation http://www.zope.com/ >| Creators of Zope http://www.zope.org/ >--------------------------------------------- > >_______________________________________________ >Zope-xml mailing list >Zope-xml@zope.org >http://lists.zope.org/mailman/listinfo/zope-xml > > From sean.upton at uniontrib.com Mon Oct 22 17:52:06 2001 From: sean.upton at uniontrib.com (sean.upton@uniontrib.com) Date: Sun Aug 10 16:54:49 2008 Subject: storing XML can be good... was Re: [Zope-xml] ANNOUNCE: PT_XP ath Message-ID: I think this is an interesting approach that would also work: merge your flat changes into righ XML as needed, preserving the XML structure and previously value added data... The advantage to ParsedXML, of course, is that for read purposes, it is already parsed into a DOM, and shouldn't require parsing, if I understand correctly, except for the occasional re-read after modification. I never understood the use of BLOBs in RDB when using coupled with an object database. Perhaps one solution is to have a container-type object in the ZODB that contains both a dumb-slow-DOM and a bunch of smart-fielded-objects, and a merge with the dumb-slow-DOM could be done from the custom objects as needed... In my case, though, I would want people other than myself to be able to extend the API as needed, so I still like the simplicity of being able to support new behaviors in TTW python scripts that could be written by a slightly techical person with enough experience playing with DOM to get what they want. Sean -----Original Message----- From: Wade Leftwich [mailto:wade@lightlink.com] Sent: Monday, October 22, 2001 12:11 PM To: Martijn Pieters; sean.upton@uniontrib.com Cc: zope-xml@zope.org Subject: Re: storing XML can be good... was Re: [Zope-xml] ANNOUNCE: PT_XPath I am in a situation similar to Sean's, sending and receiving a bunch of newsfeeds in NITF format. (Hi Sean, i think we met up on the NITF list a while back.) Because I'm adding on to (and hoping to replace) an ASP system, I've got all my data in MSSQL. My tables have the usual columns that Sean alluded to: headline, byline, date, etc. I also keep the original NITF- XML in a bigtext field. I run a cron job to rewrite the XML if any of the other fields gets edited. Right now I don't think I would want to give up the RDB storage, even if I didn't need it for historical reasons. While each article is a tree, the organization at higher levels is definitely tabular. And many people understand RDB's. I would love to be able to access the tree structure of each article within Zope, without first parsing XML to a DOM. Having done a fair amount of work with DOM, I hear what Martijn is saying about it being fat and slow. What about an object representation, like Martijn suggested, that could live in a BLOB in an RDB? Wade Leftwich Ithaca, NY 10/22/2001 2:26:41 PM, Martijn Pieters wrote: >On Mon, Oct 22, 2001 at 10:40:09AM -0700, sean.upton@uniontrib.com wrote: >> I hear more and more often that XML should not be stored, and just be an >> interchange format. And in most cases, that is likely to be correct, but... >> >> I think there are some very convincing reasons to, indeed, store XML on the >> server for particular use cases. The best possible illustration that I can >> think of is a use case that I have; sorry that this 'manifesto for storing >> XML' is so lengthy, but I think this position needs definitive justification >> - and I think that for this use case below, I'm right... ;) >> >> Eventually, for our internal use, my company will likely be creating a >> product that will support the use and storage of NITF (News Industry Text >> Format) and NewsML - both standards for news content defined by the IPTC >> (International Press Telecommunications Council) and/or the Newspaper >> Association of America (NAA). These DTDs were designed to address >> deficiencies in current information flow in the newspaper industry; NITF, in >> particular, began in the pre-XML days, as an SGML DTD with the specific goal >> of allowing news producers/vendors to scrap the 150 or so different >> proprietary text interchange and storage formats used by the industry. The >> idea was that in translation (filtering) between lots of formats, >> information is almost always lost, and always costly. > >To me, this is still no reason to justify an XML storage. Note that Zope >doesn't force you to use a fixed number of fields; the object tree and >python provide you with much more flexibility. I am convinced that it is >posible to build a Zope app with the same fidelity as the NITF, with >lossless conversion between the internal storage format and NITF. > >Using Zope objects instead of XML would buy you speed and scalability. DOMs >are memory hogs, and CPU intensive to build and manipulate. Only use a DOM >when a custom API isn't feasable. Don't carry around all that extra weight >if you can avoid it! > >-- >Martijn Pieters >| Software Engineer mailto:mj@zope.com >| Zope Corporation http://www.zope.com/ >| Creators of Zope http://www.zope.org/ >--------------------------------------------- > >_______________________________________________ >Zope-xml mailing list >Zope-xml@zope.org >http://lists.zope.org/mailman/listinfo/zope-xml > > From sean.upton at uniontrib.com Mon Oct 22 20:38:08 2001 From: sean.upton at uniontrib.com (sean.upton@uniontrib.com) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] RE: ZOPE/XML Message-ID: You won't be able to use XMLDocument for Unicode apps, and currently, not ParsedXML either... Zope Unicode support exists in Zope 2.4, but ParsedXML unicode is currently lacking. This is being addressed, AFAIK, but I don't know the time-frame, given that this is one the horizon, and not definitively prioritized by any person or entity. My guess is that other DOM work to solve any minor compliance work in ParsedXML unit-tests using 8-bit character sets will happen first before any Unicode work. In the long-term, Unicode will likely be supported in ParsedXML, but I have absolutely no idea on the time-frame. XML node search would most certainly be possible in ZCatalog, provided you use some sort of method for accessing your data. An example might either be a python script via DOM, or an XPath query, via the 'XPath Methods' support recently released for Zope. ZCatalog would work well for this. Perhaps the problem with your code might be a code typo; the list 'name' is initialized, but 'names' is not, so this may not be a DOM problem... You may want to post this to the zope-xml list. I have cc'd this to that list. Sean -----Original Message----- From: Sunil Mundari [mailto:sunil_stylusinc@yahoo.com] Sent: Friday, October 19, 2001 9:55 PM To: sean.upton@uniontrib.com Subject: ZOPE/XML Hi, I read ur article, that ur developing large editorial system by using latest techonology Zope and XML. I am also intrested in zope and xml to develop my portal. But i don't know how reliable it will be. after reading your article which is in zope.org/members, i thought it is possible through zope and fullfill my requirements. My project is to develop a powerfull search engine which supports all languages(UNICODE). And my data will be in XML. So i want to know how i will start my project. Please answer some queries :- Is zope support Unicode on linux and windows2000 ? Is XML text node search is possible through ZCatalog ? i download zope document from zope.org and trying using xml with Python example :- XML Document id - addressbook.xml Bob
2118 Wildhare st.
 Randolf
13 E. Roundway
 id - query.py parameter list - self def query(self): import string doc=getattr(self, 'addressbook.xml') book=doc.documentElement print "Number of items", len(book.childNodes) name=[] for item in book.childNodes: name=item.firstChild names.append(name.firstChild.nodeValue) print "Names ", string.join(names, ",") return printed but after quering it is showing error. same i tried with parsed xml also then also it is not working. i need ur suggestions. Thanks, Sunil __________________________________________________ Do You Yahoo!? Make a great connection at Yahoo! Personals. http://personals.yahoo.com From djay at avaya.com Mon Oct 22 20:33:25 2001 From: djay at avaya.com (Jay, Dylan) Date: Sun Aug 10 16:54:49 2008 Subject: storing XML can be good... was Re: [Zope-xml] ANNOUNCE: PT_XP ath Message-ID: > -----Original Message----- > From: Martijn Pieters [mailto:mj@zope.com] > Sent: Tuesday, 23 October 2001 4:27 AM > To: sean.upton@uniontrib.com > Cc: zope-xml@zope.org > Subject: Re: storing XML can be good... was Re: [Zope-xml] ANNOUNCE: > PT_XPath > > > On Mon, Oct 22, 2001 at 10:40:09AM -0700, > sean.upton@uniontrib.com wrote: > > I hear more and more often that XML should not be stored, > and just be an > > interchange format. And in most cases, that is likely to > be correct, but... > > > > I think there are some very convincing reasons to, indeed, > store XML on the > > server for particular use cases. The best possible > illustration that I can > > think of is a use case that I have; sorry that this > 'manifesto for storing > > XML' is so lengthy, but I think this position needs > definitive justification > > - and I think that for this use case below, I'm right... ;) > > > > Eventually, for our internal use, my company will likely be > creating a > > product that will support the use and storage of NITF (News > Industry Text > > Format) and NewsML - both standards for news content > defined by the IPTC > > (International Press Telecommunications Council) and/or the > Newspaper > > Association of America (NAA). These DTDs were designed to address > > deficiencies in current information flow in the newspaper > industry; NITF, in > > particular, began in the pre-XML days, as an SGML DTD with > the specific goal > > of allowing news producers/vendors to scrap the 150 or so different > > proprietary text interchange and storage formats used by > the industry. The > > idea was that in translation (filtering) between lots of formats, > > information is almost always lost, and always costly. > > To me, this is still no reason to justify an XML storage. > Note that Zope > doesn't force you to use a fixed number of fields; the object tree and > python provide you with much more flexibility. I am convinced > that it is > posible to build a Zope app with the same fidelity as the NITF, with > lossless conversion between the internal storage format and NITF. > > Using Zope objects instead of XML would buy you speed and > scalability. DOMs > are memory hogs, and CPU intensive to build and manipulate. > Only use a DOM > when a custom API isn't feasable. Don't carry around all that > extra weight > if you can avoid it! The actualy physical storage is kind of irrelevent. If it's stored as objects vs XML text fragments make no difference (other than efficency of course). What is important is two things: - It can be represented as WC3 DOM so it can be manipulated using a standard API - There is a easy flexible mechanizm for adjusting its storage policy. eg I can say that all Blah elements need to be indexed, or all foo sub trees can be chunked as one object since they will always be accessed togeather. ParsedXML isn't this since it only has one storage, one large text fragment. XMLDocument also had one policy of one object per element. Custom Zope objects each with DOM interfaces can be flexible however it would be very time consuming to change from one policy to another. You would have to difine new sets of objects and then write conversion scripts that delete the old objects and create new ones. So what does that leave? A hypothetical product. Let's call it FlexiXML. I can import a lot of XML and it will use a defualt policy, let's say that is to store the a compact parsed representation and do the DOM or XML on the fly. It would also give a folder, document, properties Zope like API. Then later I want to optimize this store for both speed and storage efficency. I specify a couple of XPath queries to select elements that should be treated as chunked into one ZODB object so to improve storage efficiency (eg a one whole HR employee record). This then changes the underlying stucture of the database. I specify another XPath that will nominate the HR employee key to be indexed. Some of my XPath queries would then become much faster. On top of this you can then sepecify a mapping between element types and Zope interfaces. Then you can use the component archtecture to specify additional views, UI, adapters etc for different element types. This gives you the XMLWidgets functionality++. What's important about all of this? A developer can concentrate on a data structure for the data first and then optimize it easily later. IMHO that was the killer functionality of RDBMS and that is something zope is weak on. From grant7 at sbcglobal.net Mon Oct 22 21:44:05 2001 From: grant7 at sbcglobal.net (Grant K Rauscher) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] Re: Zope-xml digest, Vol 1 #7 - 9 msgs References: Message-ID: <048f01c15b64$36881630$b2be7b42@geekier.org> > Can anybody tell me what is wrong or missing? Elena, actually I experienced the same issue on RH Linux 7.1 for ParsedXML 1.0 - the docs are confusing, but I think you need to run setup.py for the Expat that goes with ParsedXML using Python 1.5.x here's what worked for me, since I have Python 2.1 installed and am using it to run ZOPE (source install)... /usr/bin/python1.5 setup.py Grant K Rauscher GeeKieR Enterprises http://www.geekier.com/ From sean.upton at uniontrib.com Tue Oct 23 12:54:06 2001 From: sean.upton at uniontrib.com (sean.upton@uniontrib.com) Date: Sun Aug 10 16:54:49 2008 Subject: storing XML can be good... was Re: [Zope-xml] ANNOUNCE: PT_XP ath Message-ID: I'll have to take some time to digest this fully, but my initial reactions: - Isn't parsedXML's single storage NOT 'one large text fragment' but a DOM; the only thing is that the DOM is one Zope object, and must be completely rewritten on every write? 'One large text fragment' - i.e. what you see on edit - is just a rendering of the DOM? - The quickest route to a solution now will likely be the best. Zope's XML support is strong for several application use-cases now. I have used it for several applications using ParsedXML, and am relatively happy. That said, I know that there are improvements that need to be made, especially with Big documents. The obvious solution that a lot of people are agreeing with here is chunking using path expressions as boundaries. I might suggest (perhaps naively) like I did yesterday, that the easiest route to doing this might be to solidify ParsedXML as it is, get it to pass the unit-tests, etc, and build a proxy container object around it around it. The container would contain 1..n number of ParsedXML objects, and a bunch of properties containing chunking borders expressed as XPath statements. The container folder object (lets call it 'BigXML') would act as a proxy with traffic director responsibilities, to read and write from the correct underlying DocumentFragments stored as ParsedXML... Sean -----Original Message----- From: Jay, Dylan [mailto:djay@avaya.com] Sent: Monday, October 22, 2001 5:33 PM To: 'Martijn Pieters'; sean.upton@uniontrib.com Cc: zope-xml@zope.org Subject: RE: storing XML can be good... was Re: [Zope-xml] ANNOUNCE: PT_XP ath > -----Original Message----- > From: Martijn Pieters [mailto:mj@zope.com] > Sent: Tuesday, 23 October 2001 4:27 AM > To: sean.upton@uniontrib.com > Cc: zope-xml@zope.org > Subject: Re: storing XML can be good... was Re: [Zope-xml] ANNOUNCE: > PT_XPath > > > On Mon, Oct 22, 2001 at 10:40:09AM -0700, > sean.upton@uniontrib.com wrote: > > I hear more and more often that XML should not be stored, > and just be an > > interchange format. And in most cases, that is likely to > be correct, but... > > > > I think there are some very convincing reasons to, indeed, > store XML on the > > server for particular use cases. The best possible > illustration that I can > > think of is a use case that I have; sorry that this > 'manifesto for storing > > XML' is so lengthy, but I think this position needs > definitive justification > > - and I think that for this use case below, I'm right... ;) > > > > Eventually, for our internal use, my company will likely be > creating a > > product that will support the use and storage of NITF (News > Industry Text > > Format) and NewsML - both standards for news content > defined by the IPTC > > (International Press Telecommunications Council) and/or the > Newspaper > > Association of America (NAA). These DTDs were designed to address > > deficiencies in current information flow in the newspaper > industry; NITF, in > > particular, began in the pre-XML days, as an SGML DTD with > the specific goal > > of allowing news producers/vendors to scrap the 150 or so different > > proprietary text interchange and storage formats used by > the industry. The > > idea was that in translation (filtering) between lots of formats, > > information is almost always lost, and always costly. > > To me, this is still no reason to justify an XML storage. > Note that Zope > doesn't force you to use a fixed number of fields; the object tree and > python provide you with much more flexibility. I am convinced > that it is > posible to build a Zope app with the same fidelity as the NITF, with > lossless conversion between the internal storage format and NITF. > > Using Zope objects instead of XML would buy you speed and > scalability. DOMs > are memory hogs, and CPU intensive to build and manipulate. > Only use a DOM > when a custom API isn't feasable. Don't carry around all that > extra weight > if you can avoid it! The actualy physical storage is kind of irrelevent. If it's stored as objects vs XML text fragments make no difference (other than efficency of course). What is important is two things: - It can be represented as WC3 DOM so it can be manipulated using a standard API - There is a easy flexible mechanizm for adjusting its storage policy. eg I can say that all Blah elements need to be indexed, or all foo sub trees can be chunked as one object since they will always be accessed togeather. ParsedXML isn't this since it only has one storage, one large text fragment. XMLDocument also had one policy of one object per element. Custom Zope objects each with DOM interfaces can be flexible however it would be very time consuming to change from one policy to another. You would have to difine new sets of objects and then write conversion scripts that delete the old objects and create new ones. So what does that leave? A hypothetical product. Let's call it FlexiXML. I can import a lot of XML and it will use a defualt policy, let's say that is to store the a compact parsed representation and do the DOM or XML on the fly. It would also give a folder, document, properties Zope like API. Then later I want to optimize this store for both speed and storage efficency. I specify a couple of XPath queries to select elements that should be treated as chunked into one ZODB object so to improve storage efficiency (eg a one whole HR employee record). This then changes the underlying stucture of the database. I specify another XPath that will nominate the HR employee key to be indexed. Some of my XPath queries would then become much faster. On top of this you can then sepecify a mapping between element types and Zope interfaces. Then you can use the component archtecture to specify additional views, UI, adapters etc for different element types. This gives you the XMLWidgets functionality++. What's important about all of this? A developer can concentrate on a data structure for the data first and then optimize it easily later. IMHO that was the killer functionality of RDBMS and that is something zope is weak on. From mj at zope.com Tue Oct 23 13:56:14 2001 From: mj at zope.com (Martijn Pieters) Date: Sun Aug 10 16:54:49 2008 Subject: storing XML can be good... was Re: [Zope-xml] ANNOUNCE: PT_XP ath In-Reply-To: References: Message-ID: <20011023135609.S24158@zope.com> On Tue, Oct 23, 2001 at 09:54:06AM -0700, sean.upton@uniontrib.com wrote: > - Isn't parsedXML's single storage NOT 'one large text fragment' but a DOM; > the only thing is that the DOM is one Zope object, and must be completely > rewritten on every write? 'One large text fragment' - i.e. what you see on > edit - is just a rendering of the DOM? Indeed, ParsedXM Lonly holds a DOM, the original text string is never stored, and it'll serialize the DOM into a string only if requested to do so. > - The quickest route to a solution now will likely be the best. Zope's XML > support is strong for several application use-cases now. I have used it for > several applications using ParsedXML, and am relatively happy. That said, I > know that there are improvements that need to be made, especially with Big > documents. The obvious solution that a lot of people are agreeing with here > is chunking using path expressions as boundaries. I might suggest (perhaps > naively) like I did yesterday, that the easiest route to doing this might be > to solidify ParsedXML as it is, get it to pass the unit-tests, etc, and > build a proxy container object around it around it. The container would > contain 1..n number of ParsedXML objects, and a bunch of properties > containing chunking borders expressed as XPath statements. The container > folder object (lets call it 'BigXML') would act as a proxy with traffic > director responsibilities, to read and write from the correct underlying > DocumentFragments stored as ParsedXML... It'll be just as easy to modify nodes down the tree by swapping them with a class that includes Persistant in it's bases. Porbably easier, in fact. This will have the effect of it being stored in its own transaction. We may even be able to modify the classes on the fly and have them persist as sperate records that way. BigXML would bring the implementation too much too the foreground, with lots of added overhead and too much UI. -- Martijn Pieters | Software Engineer mailto:mj@zope.com | Zope Corporation http://www.zope.com/ | Creators of Zope http://www.zope.org/ --------------------------------------------- From faassen at vet.uu.nl Tue Oct 23 14:07:20 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] xml standard char set? (& minor zope thing ...) In-Reply-To: <20011022114845.F24158@zope.com> References: <20011022114845.F24158@zope.com> Message-ID: <20011023200720.A22269@vet.uu.nl> Martijn Pieters wrote: > > > The trailing slash is there is to make it compatible with XHTML. The > > > lash will be seen as an unknown attribute by non-XHTML browsers and > > > ignored. See the W3C XHTML recommendations. > > > > Isn't the actual backwards compatibility that there's to be a space > > between the / and the >? This ensures that it will be ignored. If there's > > actually no space added by Zope I think we should make sure it does. > > The space is there. Note that Image objects generate an tag as well. Formulator does too. I hope I do add the space there too. :) Regards, Martijn From sean.upton at uniontrib.com Tue Oct 23 16:39:05 2001 From: sean.upton at uniontrib.com (sean.upton@uniontrib.com) Date: Sun Aug 10 16:54:49 2008 Subject: storing XML can be good... was Re: [Zope-xml] ANNOUNCE: PT_XP ath Message-ID: I think that makes much more sense. I told you my idea was naive ;) Sean -----Original Message----- From: Martijn Pieters [mailto:mj@zope.com] Sent: Tuesday, October 23, 2001 10:56 AM To: sean.upton@uniontrib.com Cc: djay@avaya.com; zope-xml@zope.org Subject: Re: storing XML can be good... was Re: [Zope-xml] ANNOUNCE: PT_XP ath On Tue, Oct 23, 2001 at 09:54:06AM -0700, sean.upton@uniontrib.com wrote: > - Isn't parsedXML's single storage NOT 'one large text fragment' but a DOM; > the only thing is that the DOM is one Zope object, and must be completely > rewritten on every write? 'One large text fragment' - i.e. what you see on > edit - is just a rendering of the DOM? Indeed, ParsedXM Lonly holds a DOM, the original text string is never stored, and it'll serialize the DOM into a string only if requested to do so. > - The quickest route to a solution now will likely be the best. Zope's XML > support is strong for several application use-cases now. I have used it for > several applications using ParsedXML, and am relatively happy. That said, I > know that there are improvements that need to be made, especially with Big > documents. The obvious solution that a lot of people are agreeing with here > is chunking using path expressions as boundaries. I might suggest (perhaps > naively) like I did yesterday, that the easiest route to doing this might be > to solidify ParsedXML as it is, get it to pass the unit-tests, etc, and > build a proxy container object around it around it. The container would > contain 1..n number of ParsedXML objects, and a bunch of properties > containing chunking borders expressed as XPath statements. The container > folder object (lets call it 'BigXML') would act as a proxy with traffic > director responsibilities, to read and write from the correct underlying > DocumentFragments stored as ParsedXML... It'll be just as easy to modify nodes down the tree by swapping them with a class that includes Persistant in it's bases. Porbably easier, in fact. This will have the effect of it being stored in its own transaction. We may even be able to modify the classes on the fly and have them persist as sperate records that way. BigXML would bring the implementation too much too the foreground, with lots of added overhead and too much UI. -- Martijn Pieters | Software Engineer mailto:mj@zope.com | Zope Corporation http://www.zope.com/ | Creators of Zope http://www.zope.org/ --------------------------------------------- _______________________________________________ Zope-xml mailing list Zope-xml@zope.org http://lists.zope.org/mailman/listinfo/zope-xml From webmaven at lvcm.com Tue Oct 23 17:32:56 2001 From: webmaven at lvcm.com (Michael R. Bernstein) Date: Sun Aug 10 16:54:49 2008 Subject: [Zope-xml] Zope-XML use case Message-ID: <1003872777.732.161.camel@fiawol> While I don't feel i have much to contribute to the ongoing discussion from a technical perspective, i'd like to mention what it is that I hope to be able to build using Zope XML support, perhaps using XMLWidgets on top. I'd like to build a writing environment. I want to be able to use an (initially) web-based authoring environment that allows me to create book-length documents of arbitrary depth and complexity. I'd like the storage to use the DocBook XML DTD, and I'd like it to be able to produce PDF and HTML renderings (probably cached, rather than on-the fly). When editing the document, i'd like to be able to specify (by clicking) the scope of the element that I am editing (ie. a whole section, or a single paragraph). Ultimately, once the web-based interface is usable, i'd like to create a dedicated client app for editing the document in outline (kind of like Radio Userland) I'd like other users of the browser of desktop editor to add arbitrary markup and comments to my documents, that can be shown or hidden while I am doing my own editing, without those elements being discarded. I don't know how to resolve simultaneous edits to different parts of the document, much less how to resolve conflicting edits to the same element. diff is meant to be used on a serialized text format, not an object tree. I don't know if this set of use-cases helps inform this debate, but that's what I'm (eventually) shooting for. Thanks, Michael Bernstein. From faassen at vet.uu.nl Wed Oct 24 11:02:13 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Zope-XML use case In-Reply-To: <1003872777.732.161.camel@fiawol> References: <1003872777.732.161.camel@fiawol> Message-ID: <20011024170213.A26144@vet.uu.nl> Michael R. Bernstein wrote: [snip] > I'd like to build a writing environment. > > I want to be able to use an (initially) web-based authoring environment > that allows me to create book-length documents of arbitrary depth and > complexity. I'd like the storage to use the DocBook XML DTD, and I'd > like it to be able to produce PDF and HTML renderings (probably cached, > rather than on-the fly). > > When editing the document, i'd like to be able to specify (by clicking) > the scope of the element that I am editing (ie. a whole section, or a > single paragraph). While I'm not aiming initially at supporting a complicated DT like DocBook, this is also a very important use case of mine. I believe there are some advantages in layering this over XML and DOM directly, instead of going the alternate route of creating a bunch of custom Zope objects. DOM is perfectly capable of tree management already and has a fairly straightforward documented API. You can generally combine the best of both worlds by layering objects on top (dynamically). Plus you always have a clear representation as XML, which makes import and export more easy. It is also always possible to directly edit the XML. And then you can leverage the whole set of XML standards (once they're integrated with Zope, but XPath is already very interesting). > Ultimately, once the web-based interface is usable, i'd like to create a > dedicated client app for editing the document in outline (kind of like > Radio Userland) Not a nearby usecase of mine, though it may become necessary eventually. > I'd like other users of the browser of desktop editor to add arbitrary > markup and comments to my documents, that can be shown or hidden while I > am doing my own editing, without those elements being discarded. I think you can do this by working with XPointers of some kind. The problem of course is that XPointers are unstable to document edits, unless you manage to uniquely identify an element, for instance by ID. > I don't know how to resolve simultaneous edits to different parts of the > document, much less how to resolve conflicting edits to the same > element. diff is meant to be used on a serialized text format, not an > object tree. I don't know either. Simultaneous edits to different parts of the tree were more or less possible with XMLDocument as they were seperate persistent objects. With ParsedXML this seems harder; right now everything would be a conflict like an edit to the same element. Resolving that seems difficult. > I don't know if this set of use-cases helps inform this debate, but > that's what I'm (eventually) shooting for. I think it is helpful, but that's because it's very similar to my own use case. :) Regards, Martijn From faassen at vet.uu.nl Wed Oct 24 11:35:29 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] straw poll: ParsedXML requirements Message-ID: <20011024173529.A26346@vet.uu.nl> Hi there, I'd like to hold an informal poll on requirements for any new release of ParsedXML. There was some discussion on whether it should continue to support Zope 2.3 and Python 1.5.2, or if it would be okay if it supported Zope 2.4 and Python 2.1 only. The latter option would make life easier on the developers, but if there are a lot of users who want to upgrade ParsedXML but not their Zope yet, we should continue to support Python 1.5.2 for a while longer. So, what do people think? I'm especially interested in hearing from current users of ParsedXML, though input from others would be interesting as well. Please send your replies to the list also so we can discuss things further. Regards, Martijn From dethe at burningtiger.com Wed Oct 24 11:44:48 2001 From: dethe at burningtiger.com (Dethe Elza) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] straw poll: ParsedXML requirements References: <20011024173529.A26346@vet.uu.nl> Message-ID: <3BD6E1F0.3030503@burningtiger.com> +1 Zope 2.4/Python 2.1 Burning Tiger is hoping to use ParsedXML extensively as the basis for supporting a whole slew of XML document types. We've already made the move to Zope 2.4 and Python 2.1, so only supporting those levels and above is OK with us. And I'm looking forward to a new release, as I'm currently having trouble with ParsedXML and Zope 2.4... I'm sure the answer is in the archives, but I haven't had the time lately for archive mining, and my headlamp is busted. %-) --Dethe -- Dethe Elza (delza@burningtiger.com) Chief Mad Scientist Burning Tiger Technologies (http://burningtiger.com) Living Code Weblog (http://livingcode.ca) From sean.upton at uniontrib.com Wed Oct 24 12:09:49 2001 From: sean.upton at uniontrib.com (sean.upton@uniontrib.com) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] straw poll: ParsedXML requirements Message-ID: You will get conservative-implementation folks who have a lot of production infrastructure still on Zope 2.3.x and Python 1.5.2 (like me, right now); that said, I think the best approach is to release an 2.3.x/1.5.2 compatible interim release with your proxy speedups, see if the expat leak bug (is that fixed yet, and better yet, what is the nature of the problem?) can be fixed. Label that release as the 'last' ParsedXML release supporting Python 1.5.2 and Zope 2.3.x. I think that is fair, then move on, supporting only new releases of Zope/Python. Sean -----Original Message----- From: Martijn Faassen [mailto:faassen@vet.uu.nl] Sent: Wednesday, October 24, 2001 8:35 AM To: zope-xml@zope.org Subject: [Zope-xml] straw poll: ParsedXML requirements Hi there, I'd like to hold an informal poll on requirements for any new release of ParsedXML. There was some discussion on whether it should continue to support Zope 2.3 and Python 1.5.2, or if it would be okay if it supported Zope 2.4 and Python 2.1 only. The latter option would make life easier on the developers, but if there are a lot of users who want to upgrade ParsedXML but not their Zope yet, we should continue to support Python 1.5.2 for a while longer. So, what do people think? I'm especially interested in hearing from current users of ParsedXML, though input from others would be interesting as well. Please send your replies to the list also so we can discuss things further. Regards, Martijn _______________________________________________ Zope-xml mailing list Zope-xml@zope.org http://lists.zope.org/mailman/listinfo/zope-xml From jmax at h24-76-105-229.vc.shawcable.net Wed Oct 24 12:29:24 2001 From: jmax at h24-76-105-229.vc.shawcable.net (John Maxwell) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] straw poll: ParsedXML requirements In-Reply-To: ; from zope-xml-request@zope.org on Wed, Oct 24, 2001 at 09:01:14 -0700 References: Message-ID: <20011024092924.A3137@h24-76-105-229.vc.shawcable.net> Martijn Faassen wrote: > I'd like to hold an informal poll on requirements > Zope 2.4 and Python 2.1 only > So, what do people think? We're using ParsedXML to hold large-ish XML documents (10-100k) and provide a number of different renderings on the fly (it's a Learning Management System, different users get different views. Python 2 and Zope 2.4 is what we're using, and the closer we get to Unicode support, the easier our jobs will be (as we're trying to support a whole external authoring cycle in this.) Performance is a big issue for us. ----------------------------- - John Maxwell http://www.entity-x.ca jmax@portal.ca From bkc at murkworks.com Wed Oct 24 13:12:48 2001 From: bkc at murkworks.com (Brad Clements) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] parsedXML in ZClass loses permission mapping on restart Message-ID: <3BD6BD63.5517.4349A71E@localhost> Can anyone answer this? I'm using ParsedXML (last release on Zope site) in Zope 2.3.3 I have a ParsedXML instance in the methods list of a ZClass. under define permissions for this object, I have "access contents information = access contents information" However when the server is restarted, this mapping is lost every time. Then, when a PythonScript (not in the zclass) references the ParsedXML object in an instance of the ZClass it gets a Unauth error (even if I'm Manager), so I have to go back and manually set this permission mapping every time. Is this a bug in ParsedXML (forgetting it's permission mapping), or a ZClass problem? (btw, the PythonScript is not in the zclass because I want it's output to be cached by the RAM cache manager, seems I can't do that with zclass instance methods) Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax netmeeting: ils://ils.murkworks.com AOL-IM: BKClements From webmaven at lvcm.com Wed Oct 24 13:37:32 2001 From: webmaven at lvcm.com (Michael R. Bernstein) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] straw poll: ParsedXML requirements In-Reply-To: References: Message-ID: <1003945052.948.12.camel@fiawol> On Wed, 2001-10-24 at 09:09, sean.upton@uniontrib.com wrote: > You will get conservative-implementation folks who have a lot of production > infrastructure still on Zope 2.3.x and Python 1.5.2 (like me, right now); And me. > that said, I think the best approach is to release an 2.3.x/1.5.2 compatible > interim release with your proxy speedups, see if the expat leak bug (is that > fixed yet, and better yet, what is the nature of the problem?) can be fixed. > Label that release as the 'last' ParsedXML release supporting Python 1.5.2 > and Zope 2.3.x. I think that is fair, then move on, supporting only new > releases of Zope/Python. This sounds like a good plan, except that the 1.5.2/2.3.x compatible branch should still get bugfixes if bugs are found (unlikely, I know). In short, the release/branching strategy should take it's cue from Zope's. Michael Bernstein (past user of XMLDocument and XMLWidgets, future user of ParsedXML) From wade at lightlink.com Wed Oct 24 13:54:31 2001 From: wade at lightlink.com (Wade Leftwich) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] straw poll: ParsedXML requirements Message-ID: <200110241750.NAA27860@emerald.lightlink.com> +1 Zope 2.4 & Python 2.1 Between Unicode support and overall better XML tools in Python 2+, I think it would be a real drain on resources to be backwards-compatible. Wade Leftwich Ithaca, NY From webmaven at lvcm.com Wed Oct 24 14:12:11 2001 From: webmaven at lvcm.com (Michael R. Bernstein) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Arbitrary dynamic granularity In-Reply-To: <20011024170213.A26144@vet.uu.nl> References: <1003872777.732.161.camel@fiawol> <20011024170213.A26144@vet.uu.nl> Message-ID: <1003947132.948.28.camel@fiawol> On Wed, 2001-10-24 at 08:02, Martijn Faassen wrote: > Michael R. Bernstein wrote: > [snip] > > I'd like to build a writing environment. > > [snippage] > > When editing the document, i'd like to be able to specify (by clicking) > > the scope of the element that I am editing (ie. a whole section, or a > > single paragraph). > > While I'm not aiming initially at supporting a complicated DT like > DocBook, this is also a very important use case of mine. The point I was trying to make was that whatever Zope XML solution is created, I think it's going to need to support arbitrary dynamic granularity somehow. XMLDocument supported by storing objects at the finest granularity it could, and then aggregating and serializing them together if you wanted a containing object. I don't know how ParsedXML supports this, if at all. Michael Bernstein. From faassen at vet.uu.nl Wed Oct 24 15:23:16 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] straw poll: ParsedXML requirements In-Reply-To: References: Message-ID: <20011024212316.A30944@vet.uu.nl> sean.upton@uniontrib.com wrote: > You will get conservative-implementation folks who have a lot of production > infrastructure still on Zope 2.3.x and Python 1.5.2 (like me, right now); > that said, I think the best approach is to release an 2.3.x/1.5.2 compatible > interim release with your proxy speedups, see if the expat leak bug (is that > fixed yet, and better yet, what is the nature of the problem?) I know there's an expat segfault bug, but what is this expat leak bug? > can be fixed. > Label that release as the 'last' ParsedXML release supporting Python 1.5.2 > and Zope 2.3.x. I think that is fair, then move on, supporting only new > releases of Zope/Python. The main sticking point would be the unicode support. I know some code in ParsedXML currently works with both non-unicode strings and unicode strings, but some obviously doesn't. Not resolving this issue and fixing all the unit tests would be releasing a buggy version of ParsedXML (for Zope 2.3 or Zope 2.4 or worse, both :). Anyway, something like this would be my own preference if we can't move on to Zope 2.4 directly. (now you know my agenda if it wasn't obvious anyway :) Regards, Martijn From sean.upton at uniontrib.com Wed Oct 24 15:43:37 2001 From: sean.upton at uniontrib.com (sean.upton@uniontrib.com) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] straw poll: ParsedXML requirements Message-ID: Well, I would think that slightly non-compliant (i.e. no unicode support) for old releases of Zope would have to do. I'm not sure about the expat bug; perhaps I am mistaken... just remember hearing something about something like that a while back... Sean -----Original Message----- From: Martijn Faassen [mailto:faassen@vet.uu.nl] Sent: Wednesday, October 24, 2001 12:23 PM To: sean.upton@uniontrib.com Cc: zope-xml@zope.org Subject: Re: [Zope-xml] straw poll: ParsedXML requirements sean.upton@uniontrib.com wrote: > You will get conservative-implementation folks who have a lot of production > infrastructure still on Zope 2.3.x and Python 1.5.2 (like me, right now); > that said, I think the best approach is to release an 2.3.x/1.5.2 compatible > interim release with your proxy speedups, see if the expat leak bug (is that > fixed yet, and better yet, what is the nature of the problem?) I know there's an expat segfault bug, but what is this expat leak bug? > can be fixed. > Label that release as the 'last' ParsedXML release supporting Python 1.5.2 > and Zope 2.3.x. I think that is fair, then move on, supporting only new > releases of Zope/Python. The main sticking point would be the unicode support. I know some code in ParsedXML currently works with both non-unicode strings and unicode strings, but some obviously doesn't. Not resolving this issue and fixing all the unit tests would be releasing a buggy version of ParsedXML (for Zope 2.3 or Zope 2.4 or worse, both :). Anyway, something like this would be my own preference if we can't move on to Zope 2.4 directly. (now you know my agenda if it wasn't obvious anyway :) Regards, Martijn From kra at monkey.org Wed Oct 24 17:26:00 2001 From: kra at monkey.org (Karl Anderson) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] ANNOUNCE: PT_XPath In-Reply-To: Martijn Pieters's message of "Mon, 22 Oct 2001 11:40:15 -0400" References: <20011022114007.C24158@zope.com> Message-ID: Martijn Pieters writes: > I really hæve yet to come across convincing reasons for having a DOM on the > server side [...] > To me the advantage would be for you to be able to use a standard, > widespread API for object tree manipulation, the W3C DOM. That's not a convincing reason? :) I personally am only excited about the DOM because it is a standard - a DOM interface allows tools that consume a DOM tree, such as XPath and XSLT engines, to work with your objects. > Except that the > DOM was designed for a particalr type of object tree, with a well-defined > and finite set of object types and manipulations. A ZODB-stored object tree > will almost always *not* fit that model, and making it fit will always > necessitate cutting out functionality and power. Really? I'm not coming up with a good example of that. The DOM fulfills the XML Infoset, the Infoset represents tree-based data - an XML document also represents an Infoset instance, modulo syntactic noise such as whitespace and entity references. Any Python object can be represented by XML, and unless your XML representation relies on that syntactic noise (and it shouldn't), it can also be represented by a DOM tree. Of course, any one-size-fits-all XML object representation will be unwieldy for many objects, but I don't see why that matters (currently). If you want XPath, don't look at the ugly DOM tree, let the XPath processor do that for you. If you do need a cleaner DOM tree - for my favorite example, if you want a StructuredText representation of a Zope object that reads and writes via its DOM, and you want a cleaner representation presented before the StructuredText layer - then clean up what your processor sees with DOM Traversal or XSLT. Or, if you have a specialized XML or DOM usage, or need more efficiency, use a specialized DOM representation for your class. That will be an issue anyway, I think, since the DOM is document-centric and people won't want to use the entire ZODB as a single document. There will be times when the expected usage will influence the representation. But these are all efficiency issues, not functionality issues. -- Karl Anderson kra@monkey.org http://www.monkey.org/~kra/ From jmunoz at softhome.net Thu Oct 25 20:32:18 2001 From: jmunoz at softhome.net (=?ISO-8859-1?Q?Juli=E1n_Mu=F1oz_Dom=EDnguez?=) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Message in a bottle... Message-ID: I have subclassed the ZSyncer product, adding properties "cp" and "rcp", to copy a folder content from a local folder to a remote folder (cp) or from a remote folder to a local folder (rcp). rcp really calls cp on the remote host via xml-rpc. cp works, but rcp does not !! And I think it should, this is pretty simple, all I do in rcp is: serverconn = self._serverConn(self._check_http(srv)) result = serverconn.cp(remote_path,local_path,type) _serverConn is internal to the product, and is based in # Thanks Amos and Chris at DC # http://www.zope.org/Members/Amos/XML-RPC Well, the error I have calling rcp method from a python script is the following, BUT ZOPE DOESN'T GIVE ME THE ERROR MESSAGE on the top the screen, so its hard to know WHY it is failing. I think it is permission related, because the remote method is not launched. But I don't understand why cp methods works (it uses also xml-rpc to copy the objects), and not rcp. I have been fetching in Zope code all the afternoon, something is failing in the unmarshaling of the response of the xml-call, but I have no idea what. Again, the xml-rpc procedures are working ok for transfering objects from local to remote host. The permissions _seem_ correctly configured, I have no error browsing the folders from the ZMI. Traceback (innermost last): File E:\Archivos de programa\zwin\lib\python\ZPublisher\Publish.py, line 223, in publish_module File E:\Archivos de programa\zwin\lib\python\ZPublisher\Publish.py, line 187, in publish File E:\Archivos de programa\zwin\lib\python\Zope\__init__.py, line 226, in zpublisher_exception_hook (Object: LockableItem) File E:\Archivos de programa\zwin\lib\python\ZPublisher\Publish.py, line 171, in publish File E:\Archivos de programa\zwin\lib\python\ZPublisher\mapply.py, line 160, in mapply (Object: copia) File E:\Archivos de programa\zwin\lib\python\ZPublisher\Publish.py, line 112, in call_object (Object: copia) File E:\Archivos de programa\zwin\lib\python\Shared\DC\Scripts\Bindings.py, line 324, in __call__ (Object: copia) File E:\Archivos de programa\zwin\lib\python\Shared\DC\Scripts\Bindings.py, line 354, in _bindAndExec (Object: copia) File E:\Archivos de programa\zwin\lib\python\Products\PythonScripts\PythonScript.py, line 363, in _exec (Object: copia) (Info: ({'script': , 'context': , 'container': , 'traverse_subpath': []}, (), {}, None)) File Script (Python), line 1, in copia (Object: guarded_getattr) File E:\Archivos de programa\zwin\lib\python\Products\ZSyncerMute\ZSyncerMute.py, line 51, in rcp (Object: ZSyncerMute) File E:\Archivos de programa\zwin\lib\python\xmlrpclib.py, line 547, in __call__ File E:\Archivos de programa\zwin\lib\python\xmlrpclib.py, line 630, in __request File E:\Archivos de programa\zwin\lib\python\Products\ZSyncer\xmlrpclibBasicAuth.py, line 48, in request File E:\Archivos de programa\zwin\lib\python\xmlrpclib.py, line 601, in parse_response File E:\Archivos de programa\zwin\lib\python\xmlrpclib.py, line 371, in close Fault: (see above) HELP!!!!!!!!!!!! Thank you very much !! -- __o _ \<_ (_)/(_) Saludos de Juli?n EA4ACL -.- From cchoi at sonicsinc.com Fri Oct 26 05:03:49 2001 From: cchoi at sonicsinc.com (Charles Y. Choi) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Parsed XML Requirements Feedback Message-ID: <3BD926F5.8030308@sonicsinc.com> Hello All! First off, many thanks to Martijn for taking on Parsed XML - this product even with its flaws has been too useful not to use and deserves a lot of love. We have been using Parsed XML 1.1b1 on sites using 2.3/1.5.2 and 2.4/2.1. For the most part, I have no bonds to 2.3/1.5.2, and would favor seeing Parsed XML development using the 2.4/2.1 releases. We've used Parsed XML largely as a container for content, which we map to HTML using DTML, Python DOM, and most recently ZPT calls. Visually editing the DOM tree via the Zope Management Interface has not really been useful to us. Honestly, IMHO there are better tools for editing XML (Emacs, XML Spy, ArborText, etc.) than ZMI, especially through a DOM tree. On the other hand, making the DOM tree a HTML form with text boxes populated with the node values could be very useful. Fixing bugs with Parsed XML is high on my wish list: * Copy and Paste of Parsed XML objects is buggy (though Karl mentioned that a fix was made but not released a while back). * Editing of Parsed XML objects though FTP doesn't work. * Instantiation of a Parsed XML object by importing a file requires that you explicitly type an id for it, rather than using the existing file name for the id. * Performance can always improve. Questions on where Parsed XML is going, esp. with Zope pushing Zope Page Templates (ZPT): Will Parsed XML have XSLT support? Will it really matter if integration with ZPT is good enough? So far in working with Parsed XML and ZPT, we've gotten this far: ZPT making DOM calls to Parsed XML Object: Good ZPT calling Python script with DOM calls to Parsed XML Object: Good What is not clear is how to create a ZPT-style method (arguably not possible, since ZPTs are objects) to directly render a Parsed XML object, like a DTML-method: object/viewXML ^ ^ | | parsed xml dtml method Anyone out there with some insights into this? Thanks! -Charles From dethe at burningtiger.com Fri Oct 26 13:04:00 2001 From: dethe at burningtiger.com (Dethe Elza) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Parsed XML Requirements Feedback References: <3BD926F5.8030308@sonicsinc.com> Message-ID: <3BD99780.3090409@burningtiger.com> Charles Y. Choi wrote: > > Will Parsed XML have XSLT support? Will it really matter if > integration with ZPT is good enough? So far in working with Parsed XML > and ZPT, Tying ParsedXML to XSLT is very easy (well, it's easy once you've installed 4Suite, but that's another matter). I've done it already as an External Method, but I want to come up with a better user interface. I'm looking at Martijn's XPath Methods for ParsedXML as a model for adding XSLT to ParsedXML. The tricky bit is that, a) A single XSLT object may be used for multiple XML objects, and b) A single XML object may be used with multiple XSLT objects. What I'm thinking right now is this: folder -+ +- XMLinstance (ParsedXML) +- XSLstylesheet (ParsedXML) so the path to the XML instance would be: /root/folder/XMLinstance and to apply the stylesheet: /root/folder/XMLinstance/XSLstylesheet Any comments on that interface? > What is not clear is how to create a ZPT-style method (arguably not > possible, > since ZPTs are objects) to directly render a Parsed XML object, like a > DTML-method: Render how? As XML? You should be able to do that directly, and you can do it indirectly with the PT_XPath add-on for ZPT (print the results of the path expression /*), which should also work with the XPath Methods for ParsedXML (as long as the XML instances don't use namespaces). HTH --Dethe -- Dethe Elza (delza@burningtiger.com) Chief Mad Scientist Burning Tiger Technologies (http://burningtiger.com) Living Code Weblog (http://livingcode.ca) From dethe at burningtiger.com Fri Oct 26 17:35:46 2001 From: dethe at burningtiger.com (Dethe Elza) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Python -> XML and vice versa Message-ID: <3BD9D732.3090304@burningtiger.com> Relating to recent discussion of whether arbitrary Zope objects can be represented meaningfully in XML, there are a few articles and scripts by David Mertz that are interesting in the same vein: XML Matters #1: On the 'Pythonic' treatment of XML documents as objects http://www-106.ibm.com/developerworks/library/xml-matters1/index.html XML Matters #2: On the 'Pythonic' treatment of XML documents as objects(II) http://www-106.ibm.com/developerworks/library/xml-matters2/index.html XML Matters #11: Revisiting xml_pickle and xml_objectify http://www-106.ibm.com/developerworks/library/x-matters11.html#h27947 Basically there are two scripts, xml_pickle which serializes and deserializes Python objects as XML instead of in the Pickle format, and xml_objectify which reads in a dom, but converts it to an ordinary python object so you don't have to walk through all the DOM rigamarole just to grab the attributes and sub-elements. This is similar in flavor to JDOM, which presents the DOM using standard Java tools and idioms, but for Python. Since ordinary Python objects can be treated as XML, it seems to me that it shouldn't be hard to treat random Zope objects as DOM in ZDOM, which is where I was coming from when I suggested that we have a default representation for all objects and only use a registry for objects which need (or want) to be treated specially. If there has to be a registry entry describing how to handle every Zope object in use, well, that just doesn't seem realistic or feasible. My $0.02CAD --Dethe -- Dethe Elza (delza@burningtiger.com) Chief Mad Scientist Burning Tiger Technologies (http://burningtiger.com) Living Code Weblog (http://livingcode.ca) From faassen at vet.uu.nl Fri Oct 26 18:10:33 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Python -> XML and vice versa In-Reply-To: <3BD9D732.3090304@burningtiger.com> References: <3BD9D732.3090304@burningtiger.com> Message-ID: <20011027001033.A15424@vet.uu.nl> Dethe Elza wrote: > Relating to recent discussion of whether arbitrary Zope objects can be > represented meaningfully in XML, there are a few articles and scripts by > David Mertz that are interesting in the same vein: [snip] Interesting. I think I had seen one or two of these before, but I hadn't thought about them yet in the light of Zope and XPath. The way to have a simplified more pythonic access to an underlying DOM is neat. We should consider offering such access to DOMs in Zope as well, if we can work out the security implications. > Since ordinary Python objects can be treated as XML, it seems to me that > it shouldn't be hard to treat random Zope objects as DOM in ZDOM, which > is where I was coming from when I suggested that we have a default > representation for all objects and only use a registry for objects which > need (or want) to be treated specially. If there has to be a registry > entry describing how to handle every Zope object in use, well, that just > doesn't seem realistic or feasible. The problem is that David Mertz' XML pickle strategy pickles object attributes, but doesn't do much about methods. The equivalent would be exposing to the DOM (as attributes or perhaps as sub element nodes or a mixture) the attributes of some persistent Zope object. That's not very useful in the Zope content for a number of reasons: * security; the innards shouldn't be exposed like that generally. Normally you get to this by using methods which do security checks, etc. * semantics; the innards don't mean much to Zope users; we're far more used to the Zope APIs, not to the way Zope actually stores things. Do you know how ObjectManager stores its contents exactly? I don't, I do know I can use objectIds() and objectValues() and such to get a list of what's there, though. What needs to happen, and why I think a registry of adapters is unavoidable, is that we expose what's accessible through object's APIs to the DOM somehow. So, an objectValues() would turn into a list of childNodes. Properties and other attributes can often become attribute nodes. Text contained by some object can become a text node. The representation to the DOM should: * follow security; go through the Zope APIs and check whether this user can do this. * have a comprehensible semantics. The represenation really needs to be human readable and should just not be a simple reflection of what can be pickled. I do think we can get quite a long way by providing some standard adapters for a set of core components. Even doing Folders with properties and perhaps a listing of contents can be helpful already, after all. I'd consider exposing the innards of objects to the DOM as not very useful, even if you can get the security details worked out. Eventually there should be an easy way for product developers to expose their stuff to the DOM. There should be product guidelines so product authors can write something up themselves. There should also be a set of standard adapters that product authors can choose to reuse, so that they can get this done relatively quickly. Perhaps there is a way to translate common APIs to DOM access almost automatically, with only a few hints here and there from a developer. Ideas? Regards, Martijn From faassen at vet.uu.nl Fri Oct 26 18:22:29 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Parsed XML Requirements Feedback In-Reply-To: <3BD926F5.8030308@sonicsinc.com> References: <3BD926F5.8030308@sonicsinc.com> Message-ID: <20011027002229.B15424@vet.uu.nl> Charles Y. Choi wrote: > First off, many thanks to Martijn for taking on Parsed XML - this product > even with its flaws has been too useful not to use and deserves a lot of > love. Thanks! At least if you mean me and not the Other Martijn. :) [snip] > We've used Parsed XML largely as a container for content, which we map to > HTML using DTML, Python DOM, and most recently ZPT calls. Is the mapping to HTML through DTML fast enough currently? (should get faster in the future though with my patch) > Visually editing the DOM tree via the Zope Management Interface has not > really been useful to us. Honestly, IMHO there are better tools for editing > XML (Emacs, XML Spy, ArborText, etc.) than ZMI, especially through a DOM > tree. I like the ability to type in a bit of XML through the web. The tree view itself isn't that useful to me either, though I imagine it would be useful in case of large documents, where I want to edit only a small section. > On the other hand, making the DOM tree a HTML form with text boxes populated > with the node values could be very useful. I'm having trouble envisioning what such a form should look like, could you give more details? > Fixing bugs with Parsed XML is high on my wish list: > > * Copy and Paste of Parsed XML objects is buggy (though Karl mentioned that > a fix was made but not released a while back). I hadn't heard of this one before -- see how much I've used them! :) Let's hope the fix is in the CVS. If not, we can always email Karl and ask. > * Editing of Parsed XML objects though FTP doesn't work. I'd be very happy to see patches for this. :) > * Instantiation of a Parsed XML object by importing a file requires that > you explicitly type an id for it, rather than using the existing file > name for the id. This seems to be the case in other Zope objects as well. Anyway, patches are welcome if this is actually possible (there must be a reason why some of these other objects don't do this). > * Performance can always improve. The next release should have increased performance for access through Zope, though the raw Python performance of the underlying DOM won't change. > Questions on where Parsed XML is going, esp. with Zope pushing Zope Page > Templates (ZPT): > > Will Parsed XML have XSLT support? I hope so; I'm not the one developing it but Dethe is looking into it, as you saw in his reply. > Will it really matter if integration with ZPT is good enough? I think XSLT support is important. I haven't used XSLT myself yet, but I imagine usages which are harder to do with ZPT. With XSLT you could potentially report on *any* underlying data that is exposed to a DOM, such as the mythical ZDOM we're looking into. If the ZDOM exposes a comprehensible DOM view on underlying Zope objects, it could be possible that using XSLT becomes one of the easiest ways to generate HTML for complicated object structures. > So far in working with Parsed XML and > ZPT, we've gotten this far: > > ZPT making DOM calls to Parsed XML Object: Good > ZPT calling Python script with DOM calls to Parsed XML Object: Good > > What is not clear is how to create a ZPT-style method (arguably not > possible, > since ZPTs are objects) to directly render a Parsed XML object, like a > DTML-method: > > object/viewXML > ^ ^ > | | > parsed xml dtml method > > Anyone out there with some insights into this? Hm, I haven't tried this yet, but I don't quite understand what the difficulties would be. ZPT objects are acquired, right? Regards, Martijn From dethe at burningtiger.com Fri Oct 26 18:49:44 2001 From: dethe at burningtiger.com (Dethe Elza) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Python -> XML and vice versa References: <3BD9D732.3090304@burningtiger.com> <20011027001033.A15424@vet.uu.nl> Message-ID: <3BD9E888.7010108@burningtiger.com> Martijn Faassen wrote: > The problem is that David Mertz' XML pickle strategy pickles object > attributes, but doesn't do much about methods. The equivalent would be > exposing to the DOM (as attributes or perhaps as sub element nodes or > a mixture) the attributes of some persistent Zope object. I didn't mean to imply we could use Mertz's classes directly, after all they are focussed on XML as text file and we are focussed as XML as DOM API (with the additional considerations of Zope security and semantics, etc). Only that a similar approach would be worth considering, or if not worth considering, at least gives insight into my earlier suggestions. > I do think we can get quite a long way by providing some standard adapters > for a set of core components. Even doing Folders with properties and > perhaps a listing of contents can be helpful already, after all. I'd > consider exposing the innards of objects to the DOM as not very useful, > even if you can get the security details worked out. Yes, I think listing folders and their contents would be a helpful first step. Actually, listing *folderish* and their contents would be even better. > Eventually there should be an easy way for product developers to expose > their stuff to the DOM. There should be product guidelines so product > authors can write something up themselves. There should also be a set of > standard adapters that product authors can choose to reuse, so that they > can get this done relatively quickly. Product guidelines? Standard adapters? That's crazy talk! If we did that then *anybody* could use it. I think we should follow the CMF guidelines and make everyone read the implementation code from CVS. (Sorry, been struggling with underdocumented Zope features quite a lot lately, I'm not usually given to fits of irony.) > Perhaps there is a way to translate common APIs to DOM access almost > automatically, with only a few hints here and there from a developer. > Ideas? I'm afraid my ideas are limited to, "I'm sure there's a way, but my understanding of the underbelly of Zope is too limited to know what that way is." Which isn't really helpful, so I withdraw it. --Dethe -- Dethe Elza (delza@burningtiger.com) Chief Mad Scientist Burning Tiger Technologies (http://burningtiger.com) Living Code Weblog (http://livingcode.ca) From faassen at vet.uu.nl Fri Oct 26 20:22:44 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] keeping references to nodes Message-ID: <20011027022244.A15657@vet.uu.nl> Hi there, As if I haven't posted enough to the parsed-xml-dev list yet, here some more.. What I want to have is an ability to keep a reference to a unique node in a document, without using a reference to the node itself directly. This because I want to store these references in say a session. By using such a reference I can recognize whether I've run into a node before and take action (such as pull things from the cache, or render it differently as I've selected that node for special operations in the GUI, etc) With XMLDocument, URLs to nodes were stable enough to make this possible. I'd simply get the absolute_url to some node and I had my reference. Unfortunately ParsedXML URLs to nodes are very unstable with respect to changes to the DOM, so that won't work. So, I've been pondering a way to get this for ParsedXML. One way would be to change the way URLs work to be more stable towards changes, but how? One simple way is to simply keep a reference to the node directly. Unfortunately that seems hard to do for sessions and would cause a lot of growth of the ZODB if references are kept there. Nodes may never get garbage collected. Another way is to give each node that is created a unique id. For that we'd need to change the DOM in some places and we'll have to carry the unique id around, but that'd certainly be possible. A hybrid approach I tried but didn't get far with is to keep a weak value dictionary. The key is some unique id that a manager class can assign, and the value is the weak ref to the node. This way, if a node goes away, it won't be in the dictionary anymore either. Unfortunately I get: TypeError: 'ExplicitAcquirerWrapper' objects are not weakly referencable when I try to put nodes in a WeakValueDictionary, so this seems to be a no-go. Yet another way would be to exploit some properties of the ZODB to get a unique id to the node object. Since nodes aren't persistent objects directly, I guess that won't work, right? Anyway, some feedback on this matter would be welcome. I'm also sending a cc to the zope-xml list to get some more discussion on this, hopefully. Perhaps it's a use case others have run into before. (should I move this kind of discussion in general to the zope-xml list? Sometimes I don't know where I should be posting my stuff) Regards, Martijn From dethe at burningtiger.com Fri Oct 26 23:42:42 2001 From: dethe at burningtiger.com (Dethe Elza) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] keeping references to nodes References: <20011027022244.A15657@vet.uu.nl> Message-ID: <3BDA2D32.9080001@burningtiger.com> > Another way is to give each node that is created a unique id. For that > we'd need to change the DOM in some places and we'll have to carry the > unique id around, but that'd certainly be possible. This would be my choice, it also requires a check to make sure the node is still *there* before you use it. > (should I move this kind of discussion in general to the zope-xml list? > Sometimes I don't know where I should be posting my stuff) I can't speak for anyone else, but I'm certainly willing to see ParsedXML discussed in zope-xml. In fact, I'd like to see all the various XML projects within zope discussed in zope-xml: We can work towards a common vision and all that. Individual lists would still be OK for specific bug reports, CVS checkin reports, etc., but direction and strategy would seem to be right at home on this list. --Dethe -- Dethe Elza (delza@burningtiger.com) Chief Mad Scientist Burning Tiger Technologies (http://burningtiger.com) Living Code Weblog (http://livingcode.ca) From faassen at vet.uu.nl Sat Oct 27 09:31:13 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] keeping references to nodes In-Reply-To: <3BDA2D32.9080001@burningtiger.com> References: <20011027022244.A15657@vet.uu.nl> <3BDA2D32.9080001@burningtiger.com> Message-ID: <20011027153113.A16823@vet.uu.nl> Dethe Elza wrote: > >Another way is to give each node that is created a unique id. For that > >we'd need to change the DOM in some places and we'll have to carry the > >unique id around, but that'd certainly be possible. > > This would be my choice, it also requires a check to make sure the node > is still *there* before you use it. Right, although such a reverse mapping isn't part of my requirements just yet (that brings back the whole stuff about keeping references to nodes, the dictionary that maps id to nodes would need to do something like that). What I think is enough for my purposes so far is to store the id only, and then when I encounter a node, I ask it what its id is, and compare the two. That way I know I've seen it before. Resolving the ids to nodes in an efficient way seems harder. Of course you can hunt the entire DOM tree for it, but that's not efficient. You can keep a potentially huge mapping of all ids to all nodes, but then we want that to be a mapping of weak references, and that doesn't seem to work with the acquisition base classes.. Hmm...what *would* work is the original url mapping thing, though. You can now make a stable url into the document, using the ids. It's also reasonably efficient to look up such a node by URL. It's also easy to verify whether a node doesn't exist anymore in that location. Of course this is slightly different from finding out whether a node exists at all anymore (it may have moved somewhere else), but would that be important? > >(should I move this kind of discussion in general to the zope-xml list? > >Sometimes I don't know where I should be posting my stuff) > > I can't speak for anyone else, but I'm certainly willing to see > ParsedXML discussed in zope-xml. In fact, I'd like to see all the > various XML projects within zope discussed in zope-xml: We can work > towards a common vision and all that. Individual lists would still be > OK for specific bug reports, CVS checkin reports, etc., but direction > and strategy would seem to be right at home on this list. Okay, then I'll be posting more often to this list when stuff like this comes up. Regards, Martijn From webmaven at lvcm.com Sun Oct 28 14:22:16 2001 From: webmaven at lvcm.com (Michael R. Bernstein) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Python -> XML and vice versa In-Reply-To: <3BD9E888.7010108@burningtiger.com> References: <3BD9D732.3090304@burningtiger.com> <20011027001033.A15424@vet.uu.nl> <3BD9E888.7010108@burningtiger.com> Message-ID: <1004296938.5217.17.camel@fiawol> On Fri, 2001-10-26 at 15:49, Dethe Elza wrote: > > I'm afraid my ideas are limited to, "I'm sure there's a way, but my > understanding of the underbelly of Zope is too limited to know what that > way is." Which isn't really helpful, so I withdraw it. Don't feel bad, I've encountered that particular situation so many times in the past that I routinely self-censor these days. Of course, sometimes this leads to long term frustration. The ZMI still doesn't have a built-in way of representing many-to-many relationships between objects, for instance, and I first started a discussion about that over two years ago. The most recent time I participated in that conversation, it seemed like it devolved to "oh, we'll just use XPath when Zope's XML support incldes it". Michael Bernstein. From faassen at vet.uu.nl Sun Oct 28 17:43:31 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Python -> XML and vice versa In-Reply-To: <1004296938.5217.17.camel@fiawol> References: <3BD9D732.3090304@burningtiger.com> <20011027001033.A15424@vet.uu.nl> <3BD9E888.7010108@burningtiger.com> <1004296938.5217.17.camel@fiawol> Message-ID: <20011028234331.A20712@vet.uu.nl> Michael R. Bernstein wrote: > On Fri, 2001-10-26 at 15:49, Dethe Elza wrote: > > > > I'm afraid my ideas are limited to, "I'm sure there's a way, but my > > understanding of the underbelly of Zope is too limited to know what that > > way is." Which isn't really helpful, so I withdraw it. > > Don't feel bad, I've encountered that particular situation so many times > in the past that I routinely self-censor these days. I didn't think Dethe's ideas were unhelpful at all though; I think the idea of doing an automatic translation of Zope API to DOM representation is interesting. We need to flesh this out further and I'm quite interested in seeing more Stuff on this, myself. I don't think this is so much a Zope problem in itself; it's more of a python problem, though the security part and the ZODB part do come in. We have a bunch of Python objects with some API consisting of methods (perhaps attributes, such as id, but that's deprecated). The goal here would be to have a system which can mostly automatically somehow map this onto a human-sensible DOM interface. We need some examples of what such a mapping looks like, to analyze the story further. To make it easy we can simply write out the XML that the DOM would present itself, so we could say something like: objectIds() foo bar baz or more complex: ... ... Perhaps. It would then be possible to do XPath queries. ZDOM manipulation support sounds far harder to make automatic than just ZDOM readonly support, by the way, but I think there's some hope to make readonly support more automatic. So, this discussion in my opinion is fruitful. Even if the idea is finally rejected we will have learned something. > Of course, sometimes this leads to long term frustration. The ZMI still > doesn't have a built-in way of representing many-to-many relationships > between objects, for instance, and I first started a discussion about > that over two years ago. The most recent time I participated in that > conversation, it seemed like it devolved to "oh, we'll just use XPath > when Zope's XML support incldes it". > > I hope I didn't participate in that discussion. :) Zope-XML seems to be having a very very difficult birth. Anyway, I'm sorry if I gave the impression to people that their ideas make no sense and that they know nothing. That I find objections doesn't mean I want to reject ideas outright or something; if you go on a bit then we can frequently get somewhere. I tend to argue (too much) to get my point across, but I'm all for cooperation and consensus overall. Perhaps it's a hacker culture type thing; blunt objection doesn't mean the idea is necessarily rejected, just not perfect yet. Regards, Martijn From kra at monkey.org Sun Oct 28 20:40:47 2001 From: kra at monkey.org (Karl Anderson) Date: Sun Aug 10 16:54:50 2008 Subject: storing XML can be good... was Re: [Zope-xml] ANNOUNCE: PT_XP ath In-Reply-To: "Jay, Dylan"'s message of "Tue, 23 Oct 2001 10:33:25 +1000" References: Message-ID: "Jay, Dylan" writes: > The actualy physical storage is kind of irrelevent. If it's stored as > objects vs XML text fragments make no difference (other than efficency of > course). Well, so long as you don't mind losing any non-XML information if and when you change from rich objects to pure XML-oriented objects. For example, right now you have to give up using any Zope attributes, such as permissions, within a ParsedXML tree. You have to apply that kind of attribute externally. > What is important is two things: > > - It can be represented as WC3 DOM so it can be manipulated using a > standard API > > - There is a easy flexible mechanizm for adjusting its storage policy. eg I > can say that all Blah elements need to be indexed, or all foo sub trees can > be chunked as one object since they will always be accessed togeather. > > ParsedXML isn't this since it only has one storage, one large text fragment. > XMLDocument also had one policy of one object per element. Custom Zope > objects each with DOM interfaces can be flexible however it would be very > time consuming to change from one policy to another. You would have to > difine new sets of objects and then write conversion scripts that delete the > old objects and create new ones. I would like to be able to choose my storage according to my needs - single Zope DOM objects with ParsedXML subtrees, for example. I don't see it as being especially hard for the developer - all of these objects need to be XML-centric, so, for example, initializing one DOM subtree from an existing one and replacing the original doesn't sound like a big deal. You'll have to be prepared to lose any non-XML properties if you move from a Zope-rich to XML-bare environment, or assign properties if you go in the other direction, but that's unavoidable. I never thought that it was a requirement that the change be efficient time or space wise, however. I see it as an architectural change. -- Karl Anderson kra@monkey.org http://www.monkey.org/~kra/ From kra at monkey.org Sun Oct 28 20:47:17 2001 From: kra at monkey.org (Karl Anderson) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Parsed XML Requirements Feedback In-Reply-To: Martijn Faassen's message of "Sat, 27 Oct 2001 00:22:29 +0200" References: <3BD926F5.8030308@sonicsinc.com> <20011027002229.B15424@vet.uu.nl> Message-ID: Martijn Faassen writes: > > * Copy and Paste of Parsed XML objects is buggy (though Karl mentioned that > > a fix was made but not released a while back). > > I hadn't heard of this one before -- see how much I've used them! :) > Let's hope the fix is in the CVS. If not, we can always email Karl and > ask. ParsedXML instances were losing some Zope attributes, such as ID, between txns, and that was showing up during cut and paste. If that's what you're talking about, it should be fixed. There are some tests for Zope interactions such as that, so it should be easy to add one that demonstrates any other problems. > > * Editing of Parsed XML objects though FTP doesn't work. > > I'd be very happy to see patches for this. :) There's commented out stubs in the code that I never got around to filling out. -- Karl Anderson kra@monkey.org http://www.monkey.org/~kra/ From djay at avaya.com Sun Oct 28 21:16:04 2001 From: djay at avaya.com (Jay, Dylan) Date: Sun Aug 10 16:54:50 2008 Subject: storing XML can be good... was Re: [Zope-xml] ANNOUNCE: PT_XP ath Message-ID: > -----Original Message----- > From: Karl Anderson [mailto:kra@monkey.org] > Sent: Monday, 29 October 2001 12:41 PM > To: Jay, Dylan > Cc: 'Martijn Pieters'; sean.upton@uniontrib.com; zope-xml@zope.org > Subject: Re: storing XML can be good... was Re: [Zope-xml] ANNOUNCE: > PT_XP ath > > > "Jay, Dylan" writes: > > > The actualy physical storage is kind of irrelevent. If it's > stored as > > objects vs XML text fragments make no difference (other > than efficency of > > course). > > Well, so long as you don't mind losing any non-XML information if and > when you change from rich objects to pure XML-oriented objects. For > example, right now you have to give up using any Zope attributes, such > as permissions, within a ParsedXML tree. You have to apply that kind > of attribute externally. I wa just talking about transforming some stored XML with one storage policy to another storage policy. I wasn't talking about transforming other kinds of zope content to XML. That would be an interesting use case however. From kra at monkey.org Sun Oct 28 21:14:42 2001 From: kra at monkey.org (Karl Anderson) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Parsed XML Requirements Feedback In-Reply-To: Dethe Elza's message of "Fri, 26 Oct 2001 10:04:00 -0700" References: <3BD926F5.8030308@sonicsinc.com> <3BD99780.3090409@burningtiger.com> Message-ID: Dethe Elza writes: > Charles Y. Choi wrote: > > > > > Will Parsed XML have XSLT support? Will it really matter if > > integration with ZPT is good enough? So far in working with Parsed XML > > and ZPT, > > > Tying ParsedXML to XSLT is very easy (well, it's easy once you've > installed 4Suite, but that's another matter). I've done it already as > an External Method, but I want to come up with a better user interface. Tying ParsedXML to XSLT by printing XML and giving that text to the processor is easy, but inefficient. I presume that you didn't get 4Suite's XSLT processor to work on ParsedXML's DOM - they're moving away from the DOM and towards a more efficient storage. PyXML intends to be able to do this for any compliant DOM, however, and that might have gotten closer while I wasn't looking. > I'm looking at Martijn's XPath Methods for ParsedXML as a model for > adding XSLT to ParsedXML. The tricky bit is that, a) A single XSLT > object may be used for multiple XML objects, and b) A single XML object > may be used with multiple XSLT objects. > > What I'm thinking right now is this: > > folder -+ > +- XMLinstance (ParsedXML) > +- XSLstylesheet (ParsedXML) > > so the path to the XML instance would be: > > /root/folder/XMLinstance > > and to apply the stylesheet: > > /root/folder/XMLinstance/XSLstylesheet > > Any comments on that interface? That's what I had XSLTemplate doing (http://www.zope.org/Wikis/zope-xml/XSLTemplate). You can give the product the IDs of your sheets and source, or you can acquire the product from your source and its sheets will transform it. You can also acquire the product from a subnode of your source, very cool. To pass several XSLT sheets, you have tell the processor their IDs and have the main sheet use them as params. XSLTemplate was quite alpha when I left it. It used sab-pyth when it was released, but the CVS version used 4Suite's transformer when I last touched it. I think that XSLTemplate illustrates some current drawbacks for Zope and XML. You want some cacheing of the XML - you don't want to generate the XML from the ParsedXML each time you transform. Right now cacheing is really aimed at the web page level, which works fine if it lets you avoid transforming altogether because your server cached the output, but loses otherwise. We need a ZODB cache to complement the RAM cache. I was cacheing the XML within XSLTemplate, but I yanked it out in CVS - half the code was about cacheing and I thought it was a net loss. The cache manager framework is really there, just needs someone to want a ZODB version. -- Karl Anderson kra@monkey.org http://www.monkey.org/~kra/ From faassen at vet.uu.nl Tue Oct 30 19:06:52 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Status report Message-ID: <20011031010652.A28964@vet.uu.nl> Hi there, I've just finished merging two things into the ParsedXML mainline, so it's time for a status update: - the DOM proxy speedups This aims to speed up access to DOM from within the Zope security framework. I haven't done a lot of measuring, but the measuring that I did showed human noticable increases in speed, so that's good. I'd like to see some more tests with rendering or accessing larger documents, however. - test cleanups I've cleaned up the unit tests to be more compliant with the Zope unit test guidelines. Basically I've ripped out some magic and made each test set runnable by itself (before you'd need to use the magic domtester.py framework). The testing guidelines are here: http://dev.zope.org/CVS/ZopeTestingGuidelines I've also just now discovered a bug that at least appears on my machine when I try to run setup.py of the included Expat with python 1.5; because was not included by pyexpat.c anymore I got an ImportError when trying to import pyexpat on my machine. We should have the bugfix (#include somewhere on top :) in CVS soon as well, I hope. It probably hadn't been compiling straight (at least on my setup) for about 3 months, though it's been working fine with Python 1.5. Another tidbit: I've done some intensive reading of the DOM recommendation again to determine whether some namespace related test failures are in fact testing the wrong thing. I suspect that they are, and once I get some independent feedback on this I'll alter the tests. Concerning the tests, I've also started the process to donate the DOM tests to the pyxml group, so that they can be used to check other DOMs as well (such as 4DOM). What I'm currently working on (not yet merged with the trunk in CVS) is unique element ids. This will enable a much more stable way to access nodes through URLs than before; a simple insert or append into the DOM tree now won't change the entire node URL so easily anymore. This is still work in progress but I'd like to know what people think. Currently URLs to nodes look like this: http://foobar.com/path/to/doc/0/5/3 where the 0/5/3 part means: go to the 0th node of the document, take its 5th child node, take its 3rd childnode. This is very unstable to changes in the DOM structure. If I insert a node, I might make node 5 node 6, breaking the URL right away. What I'm playing with is a way to add unique ids to element nodes, so that at least inserting an element won't break everything anymore. Perhaps it is a good idea to introduce unique ids to other nodes as well, instead of only to elements.. for some reason I didn't do that but I forget now why not. Having such a unique id per node does cost a bit of extra memory per node to store the ids. So, your feedback and opinions, please. Regards, Martijn From kra at monkey.org Tue Oct 30 18:54:44 2001 From: kra at monkey.org (Karl Anderson) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Python -> XML and vice versa In-Reply-To: Martijn Faassen's message of "Sat, 27 Oct 2001 00:10:33 +0200" References: <3BD9D732.3090304@burningtiger.com> <20011027001033.A15424@vet.uu.nl> Message-ID: Martijn Faassen writes: > The problem is that David Mertz' XML pickle strategy pickles object > attributes, but doesn't do much about methods. The equivalent would be > exposing to the DOM (as attributes or perhaps as sub element nodes or > a mixture) the attributes of some persistent Zope object. > > That's not very useful in the Zope content for a number of reasons: > > * security; the innards shouldn't be exposed like that generally. > Normally you get to this by using methods which do security checks, > etc. Well, I never thought that ZDOM would interact with the security mechanism in a DOMish way. A DOM method such as attributes() would return whatever the security context allowed, just like any other method called with a security context, but the restricting wouldn't have anything to do with how the DOM calls are invoked. In other words, a Python call of a ZDOM method would be as unrestricted as any other Python access of the DB, while a hypothetical XPath access of ZDOM invoked via a web interface would only see what was allowed by that interface's security context. But that security isn't expressed meaningfully by the DOM, it's expressed by the fact that these are still Zope objects - that is, a ZDOM object might have an ac_permissions DOM attribute, but security isn't handled by accessing that interface thru the DOM, and the custom adapter (like mentioned below) might not even expose it. IOW, security doesn't change. I never thought too hard about it, though, is this a naive? > What needs to happen, and why I think a registry of adapters is unavoidable, > is that we expose what's accessible through object's APIs to the DOM somehow. > So, an objectValues() would turn into a list of childNodes. Properties > and other attributes can often become attribute nodes. Text contained by > some object can become a text node. The representation to the DOM should: Adapters are unavoidable because the DOM is document-centric. Otherwise the only way to look at the ZODB would be as one large document rooted at Zope.app(). IIRC, WhatIsADocument talks about this. -- Karl Anderson kra@monkey.org http://www.monkey.org/~kra/ From faassen at vet.uu.nl Tue Oct 30 19:47:44 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Re: [Zope] dynamic update of XML-Documents from local FileSystem In-Reply-To: <001601c16187$0ac2c3a0$a705e0d4@Sebastian> References: <001601c16187$0ac2c3a0$a705e0d4@Sebastian> Message-ID: <20011031014744.A29261@vet.uu.nl> Elena Schulz wrote: > is it possible to link xml-documents residing in the local filesystem to > zope via the XML-Document Product dynamically. It means that changes in > the xml-document are reflected in zope on the fly? > > If somebody knows about that, I would be very thankful for some hints how > to do it? This sounds tricky to do. Both XMLDocument (which isn't maintained anymore, btw) as well as its successor ParsedXML (which is being maintained and developed further) store the XML as a tree of objects, not as text. When you view the contents for instance in the Zope management screens, it'll ask the tree to render itself as XML text on the fly each time. The tree in XMLDocument follows loosely the w3c DOM level 1 recommendation, while the tree in ParsedXML strictly implements the w3c DOM level 2 recommendation. So, if a file on the local file system contains XML and changes what would need to happen in Zope is more than a simple access to the text data and passing it through XMLDocument. What would need to happen is a reparse of the XML text, storing it in the XMLDocument. It should be possible to either adapt XMLDocument (or ParsedXML) or write some new product that does something like this, though: Each document in Zope would carry with it a path to the filesystem file, and a last changed datetime of the file. Then, on each access to the document object (this is the tricky part; I am not sure how to do this without some inefficiency, unless you make sure the user code is responsible to trigger this check instead), you check the file in the filesystem, compare the last change datetime with the one you stored, and if the file on the filesystem is newer, load the file and call a method to reparse the entire tree into XMLDocument (or ParsedXML). Why do you want to do this by the way? Would it help if ParsedXML had working FTP support, for instance? I've sent a cc of this reply to the zope-xml list as well; it is a good place to discuss XML questions. Regards, Martijn From faassen at vet.uu.nl Tue Oct 30 20:04:35 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Python -> XML and vice versa In-Reply-To: References: <3BD9D732.3090304@burningtiger.com> <20011027001033.A15424@vet.uu.nl> Message-ID: <20011031020434.A29297@vet.uu.nl> Karl Anderson wrote: > Martijn Faassen writes: > > > The problem is that David Mertz' XML pickle strategy pickles object > > attributes, but doesn't do much about methods. The equivalent would be > > exposing to the DOM (as attributes or perhaps as sub element nodes or > > a mixture) the attributes of some persistent Zope object. > > > > That's not very useful in the Zope content for a number of reasons: > > > > * security; the innards shouldn't be exposed like that generally. > > Normally you get to this by using methods which do security checks, > > etc. > > Well, I never thought that ZDOM would interact with the security > mechanism in a DOMish way. A DOM method such as attributes() would > return whatever the security context allowed, just like any other > method called with a security context, but the restricting wouldn't > have anything to do with how the DOM calls are invoked. Right, I'm not sure I was claiming it would. What the DOM adapter would have to do is make sure the user is actually allowed to use the methods the adapter is calling to construct the DOM representation. But here I was objecting to the idea to generate a DOM representation automatically by looking at the object innards (i.e. private attributes), instead of using an adapter approach. An compromise approach later discussed in this thread would be semi automatic way to translate Zope API to DOM representation. This would make make adapter construction a lot easier. > In other words, a Python call of a ZDOM method would be as > unrestricted as any other Python access of the DB, while a > hypothetical XPath access of ZDOM invoked via a web interface would > only see what was allowed by that interface's security context. Right, if you use unrestricted Python it should just return as much as possible. The implementation would be using unrestricted Python, however, wouldn't it? So we'd need to check there explicitly, (or pass some security context along?) unless I'm missing something. Is this automatic? > But > that security isn't expressed meaningfully by the DOM, it's expressed > by the fact that these are still Zope objects - that is, a ZDOM object > might have an ac_permissions DOM attribute, but security isn't handled > by accessing that interface thru the DOM, and the custom adapter (like > mentioned below) might not even expose it. > > IOW, security doesn't change. I never thought too hard about it, > though, is this a naive? Well, it's basically what I'd like to implement, but wouldn't the ZDOM implementation be unrestricted Python, just like an external method or any product method? I.e. if you can bypass the security by going through ZDOM that'd be bad. You can't just restrict access to ZDOM methods with security declarations to fix this, either. The ZDOM methods themselves need to check for permission, right? > > What needs to happen, and why I think a registry of adapters is unavoidable, > > is that we expose what's accessible through object's APIs to the DOM somehow. > > So, an objectValues() would turn into a list of childNodes. Properties > > and other attributes can often become attribute nodes. Text contained by > > some object can become a text node. The representation to the DOM should: > > Adapters are unavoidable because the DOM is document-centric. > Otherwise the only way to look at the ZODB would be as one large > document rooted at Zope.app(). Because you'd need to construct the entire XML representation of the entire tree in one go otherwise? Yes, adapters are 'lazy', which is good, but I was talking about the need for a *registry* of adapters instead of having some form of super adapter that does the translation fully automatically (or adding a super mixin to objects to do this DOM representation). Regards, Martijn From cstrong at arielpartners.com Tue Oct 30 21:30:25 2001 From: cstrong at arielpartners.com (Craeg K. Strong) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Any interest in a set of Zope products to support CVS-versioned, XML/XSLT-based Zope development? Message-ID: <3BDF6241.9010509@arielpartners.com> Greetings! We at Ariel have recently started using Zope and, needless to say, we are enjoying a large increase in productivity compared to our previous Apache-Cocoon2-Tomcat-Servlet-dbXML-Java environment. In our opinion, Zope is about a year or two ahead of where these guys are in many respects. Kudos! The main problems we faced while moving to Zope were these: - how to support version control, branching, merging, tagging, etc. etc. (we use CVS) - how to support our XML/XSLT-pipeline based document publishing paradigm (cf Cocoon2) We have seen several efforts underway to address these issues within the Zope community. We have gone a slightly different direction, but hope to contribute to these efforts in some way. In our environment: - multiple author/developers work on the same documents at the same time - we work on content for several clients at the same time - web pages are cobbled together from content pulled from several XML documents, according to a PagePolicy (similar to the way articles are collected and pasted into a newspaper page). - the PagePolicies can be changed independent of the underlying content (header-body-leftmargin-footer, header-body, etc) - all content is in XML, transformed into HTML/DTML by XSLT. - we use lots of tools other than Zope, so we want to keep our content in XML, not pickled Zope objects. (hold the pickles ;-) In order to make all this work, we created three Zope Products: ExternalFile: Points to a file in the filesystem. Unlike ExtFile, it is not copied into a repository directory, but remains "in situ" Behaves similarly to a symbolic link in UNIX or windows shortcut. Metadata stays in Zope, but content stays in the file. index_html and __call__ return the content as if it were a real Zope object, so Zope doesn't know the difference. :-) Due to permissioning and other possible problems, this is probably only appropriate for development, not production. In future, we hope to add an option to copy in the content to ZODB, essentially converting it to a DTMLDocument. At that point we will probably rename the class "ExternalizableFile" more on this later..... CVSFile: inherits from ExternalFile. Same thing, but the file is assumed to be inside of a CVS sandbox. Includes buttons for doing normal cvs commands. Sandboxdir is stored in a cookie, so each developer can access his own sandbox via THE SAME ZOPE OBJECT. That is, all developers can share a single Zope server but each see content from his own sandbox. (play in your own sandbox :-) This avoids the locking issues and other problems inherent in things like CVSwebClient. It also enables the content to stay "native" rather than becoming a Zope extract like in ZCVSMixin/ZCVSFolder. XMLDocument: (Sorry for the name clash). Inherits from CVSFile. Has a notion of a "transformer" property that points to an XSLT transformer. This enables it to be rendered automatically into HTML when __call__ or index_html is called. The underlying XML file can either be external or not. The XMLDocument object represents the entire file, as opposed to the parsedXML stuff that explodes a single document into multiple Zope objects for each XML node (correct me if I am wrong...) There is lots more to do, but we do have initial development versions of all three working in the lab. Is there interest in this kind of Product? Would it be considered heretical? ;-) Are others working on the same thing? Thanks! --Craeg From cstrong at arielpartners.com Tue Oct 30 21:50:26 2001 From: cstrong at arielpartners.com (Craeg K. Strong) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Any interest in a set of Zope products to support CVS-versioned, XML/XSLT-based Zope development? References: <3BDF6241.9010509@arielpartners.com> <15327.22053.333953.296807@mcallister.cyc.com> Message-ID: <3BDF66F2.2010801@arielpartners.com> Daniel Mahler wrote: >Hi, > >For what it is worth >"I am interested!" >Can I obtain a copy somwhere? >Do you use 4suite at all? > >D > Which part? Would you use CVS or no? XML/XSLT? Both? Definately would help to get a sense of how we should factor this stuff. My guess is they should be as separable as possible, b/c lots of people out there use other version control tools (PVCS, SourceSafe, ClearCase, etc.) The other thing is that we are currently using saxon to do our XSLT transformation http://saxon.sourceforge.net Because saxon is written in java, we are using XMLRPC to get there. We copied the stuff from XSLTConnector but b/c we had to change it so much we subsumed it fully into our stuff. The reason we are using saxon is that in our experience it is the most advanced-- has many features from XSLT 1.1 and all. It is also really fast and very stable. BUT the call out to XML-RPC is a pain-- it sure would be nice to remove this dependency. We looked at 4suite but it looked like it was not ready for prime time. What do you think? --Craeg From anthony at interlink.com.au Tue Oct 30 23:54:14 2001 From: anthony at interlink.com.au (Anthony Baxter) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Re: [Zope-dev] Any interest in a set of Zope products to support CVS-versioned, XML/XSLT-based Zope development? In-Reply-To: Message from "Craeg K. Strong" of "Tue, 30 Oct 2001 21:30:25 CDT." <3BDF6241.9010509@arielpartners.com> Message-ID: <200110310454.f9V4sEZ18906@mbuna.arbhome.com.au> You might want to look at the fishbowl project on this that was recently announced: http://dev.zope.org/Projects/Wikis/DevSite/Projects/ZopeVersionControl/ (URL copied from Brian Lloyd's earlier email - with a 7s delay in the AU-US net link at the moment, I can't check it's correct, unfortunately...) Anthony -- Anthony Baxter It's never too late to have a happy childhood. From dethe at burningtiger.com Wed Oct 31 00:19:32 2001 From: dethe at burningtiger.com (Dethe Elza) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Canonical Python DOM Interface Message-ID: <3BDF89E4.5010205@burningtiger.com> In Java-land there's an official interface which the DOM must implement, so there are certainties you can count on with all the various DOM implementations (modulo explicitly non-standard ones such as JDOM). Here in Python-land, there is 4DOM, minidom, ParsedXML's DOM, pulldom, javadom, and I'm sure I'm missing a few (maybe more than a few). Is there a canonical interface defined anywhere? I realize that there is a canonical OMG IDL interface, and a canonical IDL -> Python binding, but is that what is being used? And if so, does anyone have it mapped to Python, or is there some relatively easy way to do this? Given that, I think I can figure out a way to test a given DOM implementation for conformance to that interface and we can get some data on how widely the various implementations diverge from each other (unless someone has already done that as well). Any advice would be appreciated. TIA -- Dethe Elza (delza@burningtiger.com) Chief Mad Scientist Burning Tiger Technologies (http://burningtiger.com) Living Code Weblog (http://livingcode.ca) From romain at zzict.com Wed Oct 31 03:54:52 2001 From: romain at zzict.com (Romain Slootmaekers) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Re: [Zope-dev] Any interest in a set of Zope products to support CVS-versioned, XML/XSLT-based Zope development? In-Reply-To: <3BDF6241.9010509@arielpartners.com> Message-ID: > In our opinion, Zope > is about a year or two ahead of where these guys are in many respects. > Kudos! When I asked the tomcat mailing list for Zope like functionality in tomcat (some year ago) nobody even answered. The product probably wasn't known or nobody saw the possibilities. I still think cocoon rocks, and session managament in Tomcat is way better than in Zope. The problem is that Tomcat demands more skills from the webbies. and Zope allows easy maintainance of 'live' sites, on site. Anyway, > > The main problems we faced while moving to Zope were these: > > - how to support version control, branching, merging, tagging, etc. > etc. (we use CVS) > - how to support our XML/XSLT-pipeline based document publishing > paradigm (cf Cocoon2) Having moved ZZICT towards zope from jsp. we faced the same XSL/XSLT problems. I often wondered: how come they have xml-rpc but not XSLT? > > XMLDocument: > (Sorry for the name clash). Inherits from CVSFile. Has a notion > of a "transformer" property that points to an XSLT transformer. > This enables it to be rendered automatically into HTML when __call__ or > index_html is called. The underlying XML file > can either be external or not. The XMLDocument object represents the > entire file, as opposed to the parsedXML stuff that explodes > a single document into multiple Zope objects for each XML node (correct > me if I am wrong...) We developed more or less the same thingy: a ZZXML product which is an XML file which upon rendering can be transformed by any number of XSLTs. the way the XML and XSL are linked is that of the XSLT spec: ... I personally hacked in into a Zope 2.4 with the python xml.xslt and xml.dom.ext packages and some 200 lines of glue code. The problems with the thingy are: - xml authoring :the xml documents are only editable as text - xslt is not context sensitive. fi I'd like to be able to apply different style sheets for (fe) different authorization roles - for some reason the webbies don't really like an XML/XSLT pipeline, because it is rather difficult to get the separation of concerns right. (fe content:XML only, first style sheet: structure of page second style sheet: markup elements , third style sheet: browser specifics aso) Web publishing still remains rather hard. > There is lots more to do, but we do have initial development versions of > all three working in the lab. Is there interest in this kind > of Product? Would it be considered heretical? ;-) Are others working > on the same thing? In short: yes, donno, yes. Romain. From rik.hoekstra at inghist.nl Wed Oct 31 06:17:22 2001 From: rik.hoekstra at inghist.nl (Rik Hoekstra) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Status report References: <20011031010652.A28964@vet.uu.nl> Message-ID: <3BDFDDC2.2080109@inghist.nl> > What I'm currently working on (not yet merged with the trunk in CVS) > is unique element ids. This will enable a much more stable way to > access nodes through URLs than before; a simple insert or append into the > DOM tree now won't change the entire node URL so easily anymore. This > is still work in progress but I'd like to know what people think. Currently > URLs to nodes look like this: > > http://foobar.com/path/to/doc/0/5/3 > > where the 0/5/3 part means: go to the 0th node of the document, take its > 5th child node, take its 3rd childnode. > > This is very unstable to changes in the DOM structure. If I insert a node, > I might make node 5 node 6, breaking the URL right away. I see your point, but this is not undesirable behaviour per definition. It is if your want your url to be persistent, but not if you actually want to resolve http://foobar.com/path/to/doc/0/5/3 to "the 0th node of the document, take its 5th child node, take its 3rd childnode" > > What I'm playing with is a way to add unique ids to element nodes, so > that at least inserting an element won't break everything anymore. taking that http://foobar.com/path/to/doc/0/5/3 would still resolve in the way expressed above, so that it will still be possible to retrieve an element of which we just know that it is the first child node of a given DOM structure I take it that the id would be something random, and not meaningful? A url like http://foobar.com/path/to/doc/0/5/3, or worse something like http://foobar.com/path/to/doc/9198274/2394837/9192877 does not sound very attractive to me. And even then: at what point is a node with a given identity 919287576 still the same, and at what point will it be decided that a change in the underlying XML document (and the DOM) that a node will no longer remain the same? After a change in its content? After a change in its element name (even if its content is still the same, just as its place in the DOM tree)? etc > Perhaps it is a good idea to introduce unique ids to other nodes > as well, instead of only to elements.. for some reason I didn't > do that but I forget now why not. > > Having such a unique id per node does cost a bit of extra memory per > node to store the ids. > > So, your feedback and opinions, please. > my 0.02 EUR (euro that is, in case it get scrambled along the way) Rik From faassen at vet.uu.nl Wed Oct 31 07:28:55 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Canonical Python DOM Interface In-Reply-To: <3BDF89E4.5010205@burningtiger.com> References: <3BDF89E4.5010205@burningtiger.com> Message-ID: <20011031132855.A30669@vet.uu.nl> Dethe Elza wrote: > In Java-land there's an official interface which the DOM must implement, > so there are certainties you can count on with all the various DOM > implementations (modulo explicitly non-standard ones such as JDOM). > > Here in Python-land, there is 4DOM, minidom, ParsedXML's DOM, pulldom, > javadom, and I'm sure I'm missing a few (maybe more than a few). > > Is there a canonical interface defined anywhere? I realize that there > is a canonical OMG IDL interface, and a canonical IDL -> Python binding, > but is that what is being used? This is what is being used by 4DOM and ParsedXML's DOM, and presumably also (though perhaps incompletely) by the other Python DOMs. > And if so, does anyone have it mapped > to Python, or is there some relatively easy way to do this? What do you mean by 'have it mapped to Python'? I do have a Python file on my computer that implements the read-only part of such a mapping as a skeleton. It is relatively easy to just type it all in, after all. :) > Given that, I think I can figure out a way to test a given DOM > implementation for conformance to that interface and we can get some > data on how widely the various implementations diverge from each other > (unless someone has already done that as well). That's already done as well, by the ParsedXML project (mostly to the credit of Martijn Pieters I understand). In ParsedXML, in tests/domapi there's a complete testsuite for DOM lvl 2. I'm in the process of trying to donate this testsuite to the PyXML project, so that they can start regular validation against 4DOM as well. tests/test_dom.py tests the DOM interface of ParsedXML tests/test_wrappeddom.py tests the wrapped DOM interface tests/test_pyxmldom.py tests 4DOM as included by pyxml (if available). If I run this, I get a lot more errors untils it hangs during some test as well. This is one reason I want to donate the tests to PyXML. :) If they can fix up 4DOM they may also find assumptions in 4XPath that are wrong, and in 4XSLT. This will make the DOM implementations a lot more interoperable. (note that you can't run the tests individually if you use an older version of ParsedXML; use domtester.py..the newer version now in CVS has cleanups which require you to set your pythonpath to your Zope's lib/python, though (see the README). One thing I'd like to make work eventually is a way to test a readonly subset of the DOM with these unit tests as well. Of course there is no such thing as a readonly DOM standard, but readonly ought to be enough for 4XPath and 4XSLT use. A readonly ZDOM would therefore be worth a lot already. Regards, Martijn From faassen at vet.uu.nl Wed Oct 31 10:51:38 2001 From: faassen at vet.uu.nl (Martijn Faassen) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Status report In-Reply-To: <3BDFDDC2.2080109@inghist.nl> References: <20011031010652.A28964@vet.uu.nl> <3BDFDDC2.2080109@inghist.nl> Message-ID: <20011031165138.A31588@vet.uu.nl> Rik Hoekstra wrote: > > > >What I'm currently working on (not yet merged with the trunk in CVS) > >is unique element ids. This will enable a much more stable way to > >access nodes through URLs than before; a simple insert or append into the > >DOM tree now won't change the entire node URL so easily anymore. This > >is still work in progress but I'd like to know what people think. Currently > >URLs to nodes look like this: > > > >http://foobar.com/path/to/doc/0/5/3 > > > >where the 0/5/3 part means: go to the 0th node of the document, take its > >5th child node, take its 3rd childnode. > > > >This is very unstable to changes in the DOM structure. If I insert a node, > >I might make node 5 node 6, breaking the URL right away. > > I see your point, but this is not undesirable behaviour per definition. It > is if your want your url to be persistent, but not if you actually want to > resolve http://foobar.com/path/to/doc/0/5/3 to "the 0th node of the > document, take its 5th child node, take its 3rd childnode" That's true; as long as the document doesn't change this is actually *more* stable in some circumstances, for instance when there's a reparse. I'd like a way to get stable references into a document somehow for all kinds of purposes, such as annotation and 'hey, we saw this node before'. > >What I'm playing with is a way to add unique ids to element nodes, so > >that at least inserting an element won't break everything anymore. > > taking that http://foobar.com/path/to/doc/0/5/3 would still resolve in the > way expressed above, so that it will still be possible to retrieve an > element of which we just know that it is the first child node of a given DOM > structure You're correct. Perhaps I need another way to arrive at semi-stable references to nodes, possibly based on XPointer (unfortunately those are just XPath expressions, and they may be as unstable as anything). The requirement then would be to be stable through minor edits both through reparse and DOM manipulation. Perhaps this isn't easily possible, though. :) > I take it that the id would be something random, and not meaningful? A url > like http://foobar.com/path/to/doc/0/5/3, or worse something like > http://foobar.com/path/to/doc/9198274/2394837/9192877 does not sound very > attractive to me. In the test implementation I'm simply using a document global counter that gives each element a unique id. So the first element encountered during document construction will be e0, the second e1, then e2, etc. > And even then: at what point is a node with a given identity 919287576 still > the same, and at what point will it be decided that a change in the > underlying XML document (and the DOM) that a node will no longer remain the > same? After a change in its content? After a change in its element name > (even if its content is still the same, just as its place in the DOM tree)? > etc Yup, this is a tricky problem. Good points. :) > >Perhaps it is a good idea to introduce unique ids to other nodes > >as well, instead of only to elements.. for some reason I didn't > >do that but I forget now why not. > > >Having such a unique id per node does cost a bit of extra memory per > >node to store the ids. > > > >So, your feedback and opinions, please. > > my 0.02 EUR (euro that is, in case it get scrambled along the way) I just see EUR here. :) I suppose the only way to get a semi-stable link into a document is to use some heuristics to construct an XPath expression. Of course the *certain* way is to use actual id attributes embedded in the document, but I'd really like to leave the document alone if at all possible. So, what kind of heuristics do we need? Let's restrict the problem to references to element nodes for now; the problem is hard enough and that would tackle most of the requirements people have in my opinion. Element name seems a good idea to start with. Then name of the parent element is probably a good idea, perhaps a few levels up. This remains stable under fairly many document edits and changes. Attributes used and value is also helpful, I think, and again remains relatively stable. Next we can move on to determine text node contents. If this is non-whitespace content we could extract a bit of the text, say the first n characters, to get even more of a match. This isn't always possible; perhaps the first element child node can also give us more of a contextual match, though this I think is less stable under changes. Sibling nodes is something else to consider. Even if we have a heuristic which works well, we have some other difficulties: * our XPointer/XPath expression probably becomes horribly long and complicated * it is relatively slow to construct and resolve these things A completely different approach that I experimented a bit with is somekind of bookmark ability. This approach would allow one to use an API to 'bookmark' a reference to a node. You get somekind of number back, but you just treat it as a token. Put the number back in, and you'll get a reference to a node again. Internally you'd have a dictionary mapping bookmark tokens to nodes. A couple of problems with this though; once a node is bookmarked it won't be garbage collected even if not in the tree anymore, as it's always referenced from the dictionary. The other problem would be that this isn't stable over reparses. One could of course store the bookmark as URLs to nodes (either using the simple 0/1/2 approach or using the complicated heuristic approach) just before any reparse, and try to reestablish the bookmark dictionary afterwards. Even the same tokens would be preserved. The garbage collecting problem seems like the hardest to crack, though perhaps we can come at a fairly simple solution. I tried using a weak dictionary but that didn't seem to want to work with Zope's extensionclass mechanism. I could use some form of manual refcount approach, but how? Another way could be to regularly purge the dictionary of any nodes that aren't connected to the tree. Hmm... I need to think more about this, and more feedback and ideas here would certainly help! Regards, Martijn From sean.upton at uniontrib.com Wed Oct 31 12:17:58 2001 From: sean.upton at uniontrib.com (sean.upton@uniontrib.com) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Canonical Python DOM Interface Message-ID: I think it is just the IDL that is used as a reference. There could potentially be more than that, but see my rant below. If scarecrow interfaces supported CONSTANTS, you could port the DOM IDL to useable Python interfaces. I have found this to be a slight annoyance in trying to port Java code to Python, in that Java supports constants in interfaces, just like IDL. Sean -----Original Message----- From: Dethe Elza [mailto:dethe@burningtiger.com] Sent: Tuesday, October 30, 2001 9:20 PM To: zope-xml@zope.org Subject: [Zope-xml] Canonical Python DOM Interface In Java-land there's an official interface which the DOM must implement, so there are certainties you can count on with all the various DOM implementations (modulo explicitly non-standard ones such as JDOM). Here in Python-land, there is 4DOM, minidom, ParsedXML's DOM, pulldom, javadom, and I'm sure I'm missing a few (maybe more than a few). Is there a canonical interface defined anywhere? I realize that there is a canonical OMG IDL interface, and a canonical IDL -> Python binding, but is that what is being used? And if so, does anyone have it mapped to Python, or is there some relatively easy way to do this? Given that, I think I can figure out a way to test a given DOM implementation for conformance to that interface and we can get some data on how widely the various implementations diverge from each other (unless someone has already done that as well). Any advice would be appreciated. TIA -- Dethe Elza (delza@burningtiger.com) Chief Mad Scientist Burning Tiger Technologies (http://burningtiger.com) Living Code Weblog (http://livingcode.ca) _______________________________________________ Zope-xml mailing list Zope-xml@zope.org http://lists.zope.org/mailman/listinfo/zope-xml From adrian at haqa.co.uk Wed Oct 31 15:05:48 2001 From: adrian at haqa.co.uk (Adrian Hungate) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Re: [Zope-dev] Any interest in a set of Zope products to support CVS-versioned, XML/XSLT-based Zope development? References: <3BDF6241.9010509@arielpartners.com> Message-ID: <02d501c16247$6eadf9a0$0280a8c0@armadillo> I'll leave the CVS to others, XML stuff is right where I am looking at the moment. My XML kit is not "standards" based (And anyone who has spoken to me about XMLKit will know that I don't like the current SAX/DOM implementations in python as they try to make Python think like Java...). I have been looking to greatly improve the proccessing in my XMLDocument because it is currently both counter-intuitive and clumsy to use. I may have misunderstood, but my XMLFile and XMLProxy objects my provide some of what you were taking about with the Zope access to XML both inside and outside of your Zope. Maybe we can move XMLKit in the directions you are talking of aswell? The two criteria that I have are: 1) The code needs to be Python, not C (For now at least) for the platform independance. (A number of Zope products I would love to use will only run on one platform and I need complete platform independance for most of my installations). 2) The product needs to be easy enough to use that a someone who is not an XML-guru can use it to good effect. Adrian... -- The difficulty of tactical maneuvering consists in turning the devious into the direct, and misfortune into gain. - Sun Tzu ----- Original Message ----- From: "Craeg K. Strong" To: ; ; Sent: Wednesday, October 31, 2001 2:30 AM Subject: [Zope-dev] Any interest in a set of Zope products to support CVS-versioned, XML/XSLT-based Zope development? > Greetings! > > We at Ariel have recently started using Zope and, needless to say, we > are enjoying a large increase in productivity > compared to our previous Apache-Cocoon2-Tomcat-Servlet-dbXML-Java > environment. In our opinion, Zope > is about a year or two ahead of where these guys are in many respects. > Kudos! > > The main problems we faced while moving to Zope were these: > > - how to support version control, branching, merging, tagging, etc. > etc. (we use CVS) > - how to support our XML/XSLT-pipeline based document publishing > paradigm (cf Cocoon2) > > We have seen several efforts underway to address these issues within the > Zope community. We have gone a slightly different > direction, but hope to contribute to these efforts in some way. In > our environment: > > - multiple author/developers work on the same documents at the same time > - we work on content for several clients at the same time > - web pages are cobbled together from content pulled from several XML > documents, according to a PagePolicy > (similar to the way articles are collected and pasted into a > newspaper page). > - the PagePolicies can be changed independent of the underlying content > (header-body-leftmargin-footer, header-body, etc) > - all content is in XML, transformed into HTML/DTML by XSLT. > - we use lots of tools other than Zope, so we want to keep our content > in XML, not pickled Zope objects. (hold the pickles ;-) > > In order to make all this work, we created three Zope Products: > > ExternalFile: > Points to a file in the filesystem. Unlike ExtFile, it is not copied > into a repository directory, but remains "in situ" > Behaves similarly to a symbolic link in UNIX or windows shortcut. > Metadata stays in Zope, but content stays in the file. > index_html and __call__ return the content as if it were a real Zope > object, so Zope doesn't know the difference. :-) > Due to permissioning and other possible problems, this is probably only > appropriate for development, not production. > In future, we hope to add an option to copy in the content to ZODB, > essentially converting it to a DTMLDocument. At that > point we will probably rename the class "ExternalizableFile" more on > this later..... > > CVSFile: > inherits from ExternalFile. Same thing, but the file is assumed to be > inside of a CVS sandbox. Includes buttons for doing > normal cvs commands. Sandboxdir is stored in a cookie, so each > developer can access his own sandbox via THE SAME ZOPE OBJECT. > That is, all developers can share a single Zope server but each see > content from his own sandbox. (play in your own sandbox :-) > This avoids the locking issues and other problems inherent in things > like CVSwebClient. It also enables the content to > stay "native" rather than becoming a Zope extract like in > ZCVSMixin/ZCVSFolder. > > XMLDocument: > (Sorry for the name clash). Inherits from CVSFile. Has a notion > of a "transformer" property that points to an XSLT transformer. > This enables it to be rendered automatically into HTML when __call__ or > index_html is called. The underlying XML file > can either be external or not. The XMLDocument object represents the > entire file, as opposed to the parsedXML stuff that explodes > a single document into multiple Zope objects for each XML node (correct > me if I am wrong...) > > There is lots more to do, but we do have initial development versions of > all three working in the lab. Is there interest in this kind > of Product? Would it be considered heretical? ;-) Are others working > on the same thing? > > Thanks! > > --Craeg > > > _______________________________________________ > Zope-Dev maillist - Zope-Dev@zope.org > http://lists.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > http://lists.zope.org/mailman/listinfo/zope-announce > http://lists.zope.org/mailman/listinfo/zope ) > From djay at avaya.com Wed Oct 31 17:39:51 2001 From: djay at avaya.com (Jay, Dylan) Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Status report Message-ID: Why not use ID attributes if they are defined in the DTD? That way if someone wants a stable url for every time they parse it then its their responsibility to put in the ids. Just use your other schemes for things that don't have ID attributes. > -----Original Message----- > From: Martijn Faassen [mailto:faassen@vet.uu.nl] > Sent: Thursday, 1 November 2001 2:52 AM > To: Rik Hoekstra > Cc: zope-xml@zope.org > Subject: Re: [Zope-xml] Status report > > > Rik Hoekstra wrote: > > > > > > >What I'm currently working on (not yet merged with the > trunk in CVS) > > >is unique element ids. This will enable a much more stable way to > > >access nodes through URLs than before; a simple insert or > append into the > > >DOM tree now won't change the entire node URL so easily > anymore. This > > >is still work in progress but I'd like to know what people > think. Currently > > >URLs to nodes look like this: > > > > > >http://foobar.com/path/to/doc/0/5/3 > > > > > >where the 0/5/3 part means: go to the 0th node of the > document, take its > > >5th child node, take its 3rd childnode. > > > > > >This is very unstable to changes in the DOM structure. If > I insert a node, > > >I might make node 5 node 6, breaking the URL right away. > > > > I see your point, but this is not undesirable behaviour per > definition. It > > is if your want your url to be persistent, but not if you > actually want to > > resolve http://foobar.com/path/to/doc/0/5/3 to "the 0th node of the > > document, take its 5th child node, take its 3rd childnode" > > That's true; as long as the document doesn't change this is actually > *more* stable in some circumstances, for instance when > there's a reparse. > I'd like a way to get stable references into a document > somehow for all > kinds of purposes, such as annotation and 'hey, we saw this > node before'. > > > >What I'm playing with is a way to add unique ids to > element nodes, so > > >that at least inserting an element won't break everything anymore. > > > > taking that http://foobar.com/path/to/doc/0/5/3 would still > resolve in the > > way expressed above, so that it will still be possible to > retrieve an > > element of which we just know that it is the first child > node of a given DOM > > structure > > You're correct. Perhaps I need another way to arrive at semi-stable > references to nodes, possibly based on XPointer (unfortunately those > are just XPath expressions, and they may be as unstable as anything). > The requirement then would be to be stable through minor edits both > through reparse and DOM manipulation. Perhaps this isn't easily > possible, though. :) > > > I take it that the id would be something random, and not > meaningful? A url > > like http://foobar.com/path/to/doc/0/5/3, or worse something like > > http://foobar.com/path/to/doc/9198274/2394837/9192877 does > not sound very > > attractive to me. > > In the test implementation I'm simply using a document global > counter that > gives each element a unique id. So the first element > encountered during > document construction will be e0, the second e1, then e2, etc. > > > And even then: at what point is a node with a given > identity 919287576 still > > the same, and at what point will it be decided that a change in the > > underlying XML document (and the DOM) that a node will no > longer remain the > > same? After a change in its content? After a change in its > element name > > (even if its content is still the same, just as its place > in the DOM tree)? > > etc > > Yup, this is a tricky problem. Good points. :) > > > >Perhaps it is a good idea to introduce unique ids to other nodes > > >as well, instead of only to elements.. for some reason I didn't > > >do that but I forget now why not. > > > > >Having such a unique id per node does cost a bit of extra > memory per > > >node to store the ids. > > > > > >So, your feedback and opinions, please. > > > > my 0.02 EUR (euro that is, in case it get scrambled along the way) > > I just see EUR here. :) > > I suppose the only way to get a semi-stable link into a document is to > use some heuristics to construct an XPath expression. Of course the > *certain* way is to use actual id attributes embedded in the > document, but > I'd really like to leave the document alone if at all possible. > > So, what kind of heuristics do we need? Let's restrict the problem to > references to element nodes for now; the problem is hard enough and > that would tackle most of the requirements people have in my opinion. > > Element name seems a good idea to start with. Then name of the parent > element is probably a good idea, perhaps a few levels up. This remains > stable under fairly many document edits and changes. > > Attributes used and value is also helpful, I think, and again remains > relatively stable. > > Next we can move on to determine text node contents. If this > is non-whitespace > content we could extract a bit of the text, say the first n > characters, > to get even more of a match. This isn't always possible; perhaps the > first element child node can also give us more of a contextual match, > though this I think is less stable under changes. Sibling nodes is > something else to consider. > > Even if we have a heuristic which works well, we have some other > difficulties: > > * our XPointer/XPath expression probably becomes horribly > long and complicated > > * it is relatively slow to construct and resolve these things > > A completely different approach that I experimented a bit with is > somekind of bookmark ability. This approach would allow one to use > an API to 'bookmark' a reference to a node. You get somekind of number > back, but you just treat it as a token. Put the number back in, and > you'll get a reference to a node again. Internally you'd have a > dictionary mapping bookmark tokens to nodes. > > A couple of problems with this though; once a node is bookmarked it > won't be garbage collected even if not in the tree anymore, as it's > always referenced from the dictionary. The other problem would be > that this isn't stable over reparses. One could of course store the > bookmark as URLs to nodes (either using the simple 0/1/2 approach or > using the complicated heuristic approach) just before any reparse, and > try to reestablish the bookmark dictionary afterwards. Even the same > tokens would be preserved. > > The garbage collecting problem seems like the hardest to crack, though > perhaps we can come at a fairly simple solution. I tried using a > weak dictionary but that didn't seem to want to work with > Zope's extensionclass mechanism. I could use some form of manual > refcount approach, but how? Another way could be to regularly purge > the dictionary of any nodes that aren't connected to the tree. > > Hmm... I need to think more about this, and more feedback and ideas > here would certainly help! > > Regards, > > Martijn > > > _______________________________________________ > Zope-xml mailing list > Zope-xml@zope.org > http://lists.zope.org/mailman/listinfo/zope-xml > From faassen at vet.uu.nl Wed Oct 31 17:47:40 2001 From: faassen at vet.uu.nl ('Martijn Faassen') Date: Sun Aug 10 16:54:50 2008 Subject: [Zope-xml] Status report In-Reply-To: References: Message-ID: <20011031234740.A766@vet.uu.nl> Jay, Dylan wrote: > Why not use ID attributes if they are defined in the DTD? That way if > someone wants a stable url for every time they parse it then its their > responsibility to put in the ids. I agree that when the ID attribute *can* be used it should be used; that's what it is therefore of course. > Just use your other schemes for things that don't have ID attributes. Perhaps we should design a form of scheme where these different types of URLs into ParsedXML's DOM can all happily live together. IDs, child node #s, heuristics, bookmarks, etc. Anyone have any ideas? Regards, Martijn