[ZODB-Dev] Re: so your coming from the rdbms world tutorial?

[Sean Allen]

On Mar 23, 2008, at 3:59 AM, Roché Compaan wrote:

> On Sat, 2008-03-22 at 15:49 -0400, Sean Allen wrote:
>>>> Ha! Ok.
>>>>
>>>> Questions then from there.
>>>>
>>>> Why not store each object type,
>>>>
>>>> customer, order etc in their own folders?
>>>
>>> You could, and I understand that you want to do so coming from a  
>>> RDBMS
>>> world, but if you are working with objects then you have the  
>>> luxury of
>>> organising content in a way that is closer to reality. I think it is
>>> rather convenient to store a customer's orders inside it.
>>>
>>
>> I can see the convenience. Ok, so here is a big question...
>> Assuming you did store each object in its own folder and you had
>>
>> Customer
>>     has many orders
>>
>> At what point do you have to hook into zodb so that if you say did  
>> this:
>>
>> customer = Customer( ... )
>> customer.addOrder( Order( ... ) )
>>
>> that the order in question when customer is saved, gets saved to its
>> own folder
>> and then customer gets a reference to that now saved persistent order
>> before it itself is saved.
>
> You first create the order inside the orders folder, then you  
> reference
> it from the customer:
>
> order = Order()
> orderfolder['order123'] = order
> # assuming you have a folder for orders in each customer, the  
> following
> # will create a reference since the above code already persisted the
> # object.
> customer.orders['order123'] = order
>
> I don't see why you need the reference though. Why not just create the
> order directly in the order folder for the relevant customer:
>
> order = Order()
> customer.orders['order123'] = order

Can that be wrapped in a transaction so that if something goes wrong
with customer creation, the order gets rolled back as well?

Or put higher level, do transactions only work within a single folder
or across folders?

>
>
>>
>>> There is another good reason - it makes a lot more sense to  
>>> distribute
>>> object creation to different containers in the ZODB to avoid write
>>> conflict errors.
>>
>> I think I understand this but can you elaborate?
>>
>> By 'distribute object creation to different containers' do you mean
>> store each
>> object in own folders?
>
> No that would lead to an unnecessary proliferation of folders and it
> wouldn't actually remove conflict errors.
>
> Conflicts errors occur when two concurrent transactions modify the  
> same
> attribute. When you are putting objects in a folder you are  
> essentially
> modifying the same attribute (unless you are using a btree). A good
> application design strategy for the ZODB is to ensure that concurrent
> transactions are modifying attributes on different objects. If you are
> putting all your orders in one folder you are increasing the chances  
> of
> conflict errors.
>

If you dont put all orders in a single folder, when it comes time to
search orders, how would you know where to find?

Do you sequentially search the multiple folders for a given object?

> Like I mentioned above you could use a btree folder since it has a
> fairly good conflict resolution strategy. But given a high enough
> insertion rate even a btree will not protect you against conflict
> errors. I would recommend you create orders inside a btree-based order
> folder inside each customer. This will significantly reduce the
> likeliness of conflict errors.
>

So if this occurs, you get an exception ( basically optimistic locking )
which you then deal with on a higher level in your application?

>>>
>>>>
>>>> --
>>>>
>>>> the database i am looking at moving over has
>>>>
>>>> 1.2 million entries in a transactions table
>>>> 980,000 orders
>>>> 775,000 customers
>>>> 1.5 million order items
>>>
>>> I think these numbers are quite manageable. But think carefully what
>>> kind of queries you want to do on the data. You have a very rich  
>>> query
>>> language in SQL that allows the construction of complex queries  
>>> and it
>>> will come naturally to you. To do the same in the ZODB will take
>>> careful
>>> planning.
>>
>> Are there any mistakes that people usually make when doing this sort
>> of mental context switch that you can make me aware of now?
>>
>> Any good reading on the subject?
>>
>> Either there isnt a ton of information on this on the net or my  
>> google
>> skills
>> are slipping.
>
> The most valuable resource is the mailing list. Unfortunately nobody  
> has
> documented their experience elsewhere before.
>

Well, if I go with zeo/zodb and python, I'll be sure to document the  
entire thing.