[ZODB-Dev] [Enhancement Proposal] Memory size limited Cache

[Dieter Maurer]

Jim Fulton wrote at 2006-10-9 13:37 -0400:
>Dieter Maurer wrote:
>> Jim Fulton wrote at 2006-10-6 17:04 -0400:
>...
>>> Or consider object activation and deactivation. If a ghost is
>>> shared among multiple threads, then __setstate__ could
>>> be called from separate threads.
>> 
>> But, why should this be a problem? They would install the same
>> state.
>
>Because __setstate__ is written assuming that no other thread is
>calling it.

And where is the this assumption used?

  The object is already created. "__setstate__" must store
  the state given in its argument in the instance dict or
  slots.
  This looks harmless, even in a multi threaded environment.

> Threaded programming is tricky.  Code designed for a
>threaded environment has to take the possibility of of multiple
>simultaneous calls into account.  Traditionally, persistent object
>implementors haven't had to do that.

Yes.

If we really implement this idea, then we have to look
into the implementation carefully.

>>> And of course, there's the possibility that buggy software
>>> might try to mutate the objects.  How would you prevent it?
>> 
>> The special read-only connection with shared cache would raise 
>> an exception instead of joining the transaction.
>
>That's too late.

I have not looked at the "Persistent.__setattr__", but we
could easily arrange the code that this is *BEFORE* the actual
assignment. It is difficult to believe that this could be too late.

>> Of course, other threads might already have been affected
>> (e.g. in the example above of modifying non-persistent data).
>> But, bugs can have nasty effects....
>
>ZODB applications count on isolation.  IMO, it isn't acceptable
>to give up on the isolation guarantee.

If part of the application is buggy, then some guarantees are no
longer available.

> ....
>What if the exception is swallowed?

Then the attribute might retain its old (unchanged) value (see above).

However, swallowing essential exceptions is always dangerous.
Currently (without the feature),
you can get inconsistent persistent state -- which is
comparably nasty to multi thread problems.

> What would you do? Reload the state?

As mentioned above: perform the connection callback before the actual
assignment. Then, there will be no assignment when the connection
is unwilling to let it through.

>> This fails for:
>> 
>>   * volatile attributes -- I have no easy solution
>
>Conceivably the persistence machinery could disallow
>assignments to _v_ attrs.

But, it should do this only for some special connections...
>...
>My intuition is still that sharing objects between threads will
>introduce a host of subtle bugs.

Yes, you may be right...



-- 
Dieter