[ZODB-Dev] shared cache when no write?

[Leonardo Rochael Almeida]

Hi

In ERP5, the rule is that you should never talk to external systems as
a synchronous response to a user request, and you should avoid, at all
costs, writing to ZODB at the same time as talking to external systems
(or the external system must be able to handle this gracefully).

Of course, it helps a lot if you have a reliable background task mechanism.

In the example given, the click to send the tweet would cause an app
written the ERP5 way to simply register all the information necessary
to send the tweet into ZODB, trigger a background activity to send the
tweet later, and immediately return to the user (browser).

Then, one of the nodes dedicated to executing background activities
pick up the tweet sending activity and can spend as much time as
necessary in the venture, and finally, trigger another background
activity to actually write to the ZODB that the sending was
successful.

This last part is to avoid sending the tweet twice on account of a
conflict-error.

You can do something similar with zc.async and friends (like
plone.app.async), but make sure you got your configuration right so
that conflict errors are resolved automatically in the storage of
background activities (ERP5 uses MySQL for storage and coordination of
background activities, so no fear of conflict errors there).

Cheers,

Leo

On Thu, Dec 13, 2012 at 10:07 AM, Jim Fulton <jim at zope.com> wrote:
> On Wed, Dec 12, 2012 at 6:31 PM, Dylan Jay <djay at pretaweb.com> wrote:
>> Hi,
>>
>
>> I've been working with zope for over 12 years and something that
>>  keeps coming up is sacling IO bound operations in Zope. The typical
>>  example is where you build an app that calls external apis. While
>>  this is happening a zope thread isn't doing any other processing
>>  and because there is a 1 thread 1 zodb cache limit.  You can run
>>  into scalability problems as you can only have as many threads your
>>  RAM / average cache size. The end result is low throughput while
>>  still having low CPU. I've consulted on some $$$ sites where others
>>  have made this mistake. It's an easy mistake to make as SQL/PHP
>>  systems don't tend to have this limitation so new developers to
>>  zope often don't to think of it.
>
> I was listening to a talk by a Java guy on Friday where he warned that
> a common newbie mistake was to have too large a database connection
> pool, causing lots of RAM usage.  I expect though that ZODB caches,
> consisting of live Python objects exacerbate this effect.
>
>
>>  The possible workarounds aren't
>>  pretty. You can segregate your api calling requests to zeo clients
>>  with large numbers of threads with small caches using some fancy
>>  load balancing rules. You can rework that part of your application
>>  to not use zope, perhaps using edge side includes to make it seem p
>>  art of the same app.
>
>> Feel free to shoot down the following makes no sense.  What if two
>> or more threads could share a zodb cache up until the point at which
>> one wants to write. This is the point at which you can't share a
>> cache in a consistent manner in my understanding. At that point the
>> transaction could be blocked until other readonly transactions had
>> finished and continue by itself? or perhaps the write transaction
>> could be aborted and restarted with a special flag to ensure it was
>> processed with the cache to itself. As long as requests which
>> involve external access are readonly with regard to zope then this
>> would improve throughput. This might seem an edge case but consider
>> where you want to integrate an external app into a zope or Plone
>> app. Often the external api is doing the writing not the zope
>> part. For example clicking a button on a plone site to make plone
>> send a tweet. It might also improve throughput on zope requests
>> which involve zodb cache misses as they are also IO bound.
>
> A simpler approach might be to manage connections better at the
> application level so you don't need so many of them.  If you're goinng
> to spend a lot of time blocked waiting on some external service, why
> not close the database connection and reopen it when you need
> it? Then you could have a lot more threads than database connections.
>
> It's possible that ZODB could help at the savepoint level.  For
> example, maybe you could somehow allow savepoints to be used accross
> tranasctions and connections.  This would be a lot saner that tring to
> share a cache accross threads.
>
> Jim
>
> --
> Jim Fulton
> http://www.linkedin.com/in/jimfulton
> Jerky is better than bacon! http://zo.pe/Kqm
> _______________________________________________
> For more information about ZODB, see http://zodb.org/
>
> ZODB-Dev mailing list  -  ZODB-Dev at zope.org
> https://mail.zope.org/mailman/listinfo/zodb-dev