[Zope-dev] Massive scalability

Steve Spicklemire steve@spvi.com
Wed, 17 Jan 2001 06:43:52 -0500 (EST)


Hi Andy,

   I'm not sure what you mean by 'interface/way', so.. I'm going to 
guess at two possible interpretations.

1) Basically ZPatterns allows you to define classes (DataSkins)
instances of which can optionally be used to view/create/change/delete
external data through methods of the class ( + a little SkinScript ).

If your store your instance data in SQL you can use SQL queries,
masked from the application behind some generic method (e.g.,
getFooIdsWithText( textToFind )") to find the id(s) of the instance(s)
you're after.  You can then get the instance from the ZPatterns
machinery and, once gotten, display it, change it, call it's methods,
and/or delete it. The way these actions on the object interact with
the data in the external database is all defined in 'SkinScript' which is
hidden away as a PlugIn of a Rack deep inside the ZPatterns guts. At
the Zope application level you don't really *know* where/how the data
is stored. Best of all you, or your Product's customers can easily
customize that part *after* your product is plugged into *their*
application, without changing the basic application level logic
and design of your product.

It's the coolest. ;-)

2) To get folks started with moving objects from ZODB to SQL I've
found ZFormulator handy as a tool to get folks quickly up to speed
in how SQL 'works'. 

http://www.zope.org/Members/faassen/ZFormulator

If they already have ZClasses, they can use this to 'automatically' 
generate starting point queries to match their class propertysheets.
Of course... it probably won't be normalized/optimized/etc.. but
it's better than doing it all for them! ;-)

-steve

>>>>> "Andy" == Andy McKay <andym@ActiveState.com> writes:

    Andy> Does ZPatterns provide a nice interface / way for storing
    Andy> classes in a RDBMS? I have to say using an RDBMS is not as
    Andy> transparent as I would like, this may may improve
    Andy> it. Finally a reason for me to ZPatterns...

    Andy> -- Andy McKay.


    Andy> ----- Original Message ----- From: "John Eikenberry"
    Andy> <jae@kavi.com> To: <zope-dev@zope.org> Cc: "Michael
    Andy> Bernstein" <webmaven@lvcm.com> Sent: Tuesday, January 16,
    Andy> 2001 3:22 PM Subject: Re: [Zope-dev] Massive scalability


    >> Michael Bernstein wrote:
    >> 
    >> > So, again: Has anyone run up against any performance or >
    >> other limitations regarding large numbers (hundreds of >
    >> thousands or more) of objects stored within the ZODB either >
    >> in a BTree Folder or a Rack?
    >> 
    >> I was looking into the same issues recently, but for a much
    >> smaller set of data (50000ish). In my tests
    >> ZPatterns/binary-trees scaled well for
    Andy> storage
    >> and retrieval. But ZCatalog did not. It was basically useless
    >> for partial matching searches (taking many minutes for searches
    >> that retrieved more than 100 matches). I was also concerned
    >> about the indexing overhead. It doesn't scale well when
    >> changing/adding many things at a time (we might have bulk
    >> adds/changes).
    >> 
    >> I ended up deciding to go with a RDBMS backend for data storage
    >> with a ZPatterns interface. SkinScripts work so well for this
    >> that I'm actually glad I switched. It simplified my design and
    >> implementation immensely.
    >> 
    >> --
    >> 
    >> John Eikenberry [jae@zhar.net - http://zhar.net]
    >> ______________________________________________________________
    >> "A society that will trade a little liberty for a little order
    >> will deserve neither and lose both."  --B. Franklin
    >> 
    >> _______________________________________________ Zope-Dev
    >> maillist - Zope-Dev@zope.org
    >> http://lists.zope.org/mailman/listinfo/zope-dev ** No cross
    >> posts or HTML encoding!  ** (Related lists -
    >> http://lists.zope.org/mailman/listinfo/zope-announce
    >> http://lists.zope.org/mailman/listinfo/zope )
    >> 


    Andy> _______________________________________________ Zope-Dev
    Andy> maillist - Zope-Dev@zope.org
    Andy> http://lists.zope.org/mailman/listinfo/zope-dev ** No cross
    Andy> posts or HTML encoding!  ** (Related lists -
    Andy> http://lists.zope.org/mailman/listinfo/zope-announce
    Andy> http://lists.zope.org/mailman/listinfo/zope )