[ZODB-Dev] nontransactional, oid, and thread safety. oh my!

[Dmitry Vasiliev]

Tim Peters wrote:
> [Randy]
>>I could implement a daily pack() strategy to minimize the size,
> 
> Many do.
> 
> 
>>but that's not the solution to fit my problem.
>>
>>Transactions are not a requirement for my application; is there way to
>>disable transaction in FileStorage?
> 
> No.
> 
> 
>>If not, are they any good Storage implementations that don't do
>>transactions?
> 
> No, and not even bad ones <wink>, because it wouldn't make sense.  The only
> way to get anything *into* a storage is to commit a transaction.  So a
> storage implementation without transactions couldn't store any data -- such
> a storage would be useless.
> 
> 
>>Or should I roll my own?
> 
> I'm lost as to what your requirements are, but if you've got no use for
> transactions then you've also got no use for database technology.  Maybe
> save your objects to a file using the pickle module from time to time?
> Maybe don't use files at all?

I guess Randy told about some sort of "transparently autopacking", for 
example in Berkeley DB you can set DB_LOG_AUTOREMOVE flag and DB will 
automatically remove log files that are no longer needed.

>>3).  Thread safety.  What are some best practices can follow to make
>>database writes and reads thread safe?
> 
> Safest is not to use threads at all.  That isn't entirely flippant:  if it's
> possible for your app, the best approach on many counts is to share a
> database across distinct single-thread processes, each process making its
> own connection via ZEO.  Then you avoid all thread traps by virtue of not
> having multiple threads.  Maximally simple, maximally safe.
> 
> If you think you have to use multiple threads, then each thread should open
> its own connection, and an object loaded in one thread should never be used
> or modified in a different thread.  That doesn't mean threads T1 and T2
> can't both do, e.g.,
> 
>     obj = my_thread_connect.root()['some_object']
> 
> That's fine.  They each get a *distinct* copy of obj in that case.  It means
> that T1 can't do
> 
>     obj = conn.root()['some_object']
> 
> and then "pass" (make available, in any way) the Python object bound to
> 'obj' to T2.  ZODB's object model isolates threads from each other at the
> database level, but can't stop clients from screwing it up at the Python
> level.

Is it safe to use code like this:

def do_changes():
     lock.acquire()
     try:
         do_obj_changes()
         transaction.commit()
     finally:
         lock.release()

?

-- 
Dmitry Vasiliev (dima at hlabs.spb.ru)
     http://hlabs.spb.ru