[Zope] please help with membership design and DCOracle

[Brian Lloyd]

> We are trying to build a web application that needs to have _many_
> members (kind of like a portal).  We need a way to store large numbers
> of member profiles, each with their own username password, 
> and a way to
> track usage statistics.
> 
> Zope appears to be very well suited for caching and displaying content
> on the site, and we have built most of the application so far with
> relatively little hassle.  This is our first project using Zope and we
> like it a lot.  kudos!  
> 
> There is a reason for this post, however :)  We need some help
> validating our approach for handling members and some help with a
> couple of specific problems. Please help!

Sure :)

> MEMBERS
> The next step is to add members, which requires large numbers 
> of writes
> to the system, particularly since the data generated by the 
> application
> (~200K of HTML per usage) must be saved. I am looking at two 
> approaches
> and am desperately wondering if anyone has an opinion.  
> 
> A) Store all member data as folders and properties in Zope. 
> B) Use a RDBMS for member data storage
> 
> Assumptions: (please correct me if I am wrong about any of these)
> I.   Zope can select cached data faster than a RDBMS.
> II.  A RDBMS can handle frequent and/or large writes of data faster
> than Zope.

These are good assumptions for your situation. Also, the Zope object
database is a multiple-revision db that stores updates as appends,
not overwrites, so you _really_ want to avoid doing things like
logging usage stats into the Zope odb in most cases.


> PROBLEMS  
> 
> [ Environment:  Zope 1.10.3, Linux Redhat 6,  Oracle 8,  
> DCOracle 2.0.0
> ]
> 
> 1. One big problem we are currently running into involves member
> authentication. If we are going to avoid writing membership data into
> Zope, that means usernames & passwords need to be stored in the RDBMS.
> We can get this information from the user and authenticate 
> him/her, but
> we then need a way to assign him/her a role so that we can 
> use standard
> Zope security to protect our site.

Have you looked at the UserDb product? It was built specifically to
pull users (including roles and such) from an RDBMS. You can get
the UserDb product from the Zope site at:

http://www.zope.org/Download/Prereleases/UserDB/

It should be fairly easy figure out how to do this sort of thing
by looking at the way UserDb works -- if you have any questions,
let me know.


> 2. Writing large amounts of data into the database seems to be a
> problem as well. We are flexible about the datatype used, so 
> long as it
> can hold ~100K of html, but are having problems with the
> implementation. We tried using a LONG datatype, and then trying to
> store the html using a ZSQL method containing the following code
> snippets (in separate tests):
> a) BODY='<!--#var body sql_quote-->'
> b) BODY=<!--#sqlvar body type=string-->
> We get back an oracle error "string literal too long" if the data
> exceeds 4000 characters. Is there another way to write this that will
> work?

LONG/BLOB support is still in the process of being worked out for
most of the Zope database adapters - hopefully there will be a
patch for this soon. I can't really put a date on it, since we're
pretty swamped with consulting work at the moment :) I'd be happy
to suggest some alternative designs that you could use in the 
interim though, if you want them :)

Hope this helps!


Brian Lloyd        brian@digicool.com
Software Engineer  540.371.6909              
Digital Creations  http://www.digicool.com