[ZODB-Dev] question about connections

[Tim Peters]

...

[Tim Peters]
>> Heh.  Is there a secret agenda here, or is that line an elaborate way
>> to spell "time.sleep(random())"?

[Victor Safronovich]
>    yes, this is an elaborate way :)). Do you think that use "time.sleep"
> is more correct way to freeze the thread?

I'm not entirely sure what you're trying to accomplish.  time.sleep(x) is
the _natural_ way to spell "this thread wants to yield to other threads for
at least the next x wall-clock seconds".  If that's what you want to do,
time.sleep(x) is the clearest way to spell it.

In contrast,

    threading.Event().wait(x)

creates an Event that can never be set, waits for it despite that it can
never be set, and sticks a timeout of x on it I guess because it _knows_
it's waiting for an Event that can never get set ;-)

Under the covers, it does the same thing, but more expensively (does a
sequence of time.sleep()s, waking up now and again to check whether the
Event has been set, which never happens, and then finally times out).

...

>> All ZODBs before 3.3 work that way.  As an extreme example, if one of
>> your threads never modifies a persistent object, it will never see
>> changes made TP> by other threads.

> Is this mean that i  explicitly must call scheduler._p_changed = 1
> to got the lastest states of the objects?

No, and I have no idea whether that would work at all for you.  For example,
I don't know whether the object you call "scheduler" there is even a
persistent object.  Did you try the call to .sync() I suggested later?
While a bit of a hack, calling sync() is a straightforward way to force a
Connection to process invalidation messages.

(re)open()'ing a Connection is another to accomplish that, although it's not
guaranteed you'll get back the same Connection object if other code is also
opening and closing Connections at the same time.  Whether that's better or
worse (if either) depends on app details.

> ...
> I know about a pool of the available connections, but is it correct
> to have not-closed connection for a long time?

That's up to you.

> I.e. is it obligatory to return the connection to the pool?

No.  Of course you should close() a Connection if you're not going to use it
again; and, because a DB has a limit on the number of simultaneous
connections it's willing to create, most apps will close a Connection if
they don't think they're going to use it again "soon".

> I have a question about the pool, under heavy load Zope have 4
> connections to zodb, + 1 connection from scheduler, and each scheduler
> element runs in separate thread ( this needed for runs elements in
> identical times ).
>
> And when 2 scheduler elements runs with heavy load from ZServer, i have
> >=7 connections. From this point the Zope behave the uncorrectly
> ( appeared slow speed, disappeared answers from server ). I try to place
> following code in my product:
>
>       from Zope import DB
>       DB.setPoolSize( 15 )
>
>    But this isn`t helped.  i added some 'print's in Connection.py, and
> this 'print's  show  2  numbers of the pool size. one number is 15, the
> second is 7.
>
>    But i don`t know why '7' is still a pool size.

The code you showed sets the pool size to 15 on the single, specific
*instance* of the DB class exposed by Zope.__init__.py (it's an instance
named "DB" of a class that's also named "DB").  It has no effect on any
other instance of the DB class that may be created by you, Zope, or any
other product you may be using.  There is no way to set the pool size to 15
globally (for _all_ instances of the DB class); you can only set it on one
DB instance at a time.  If you see 15 sometimes and 7 other times, that
means more than one DB instance exists and is begin used.  Sorry, I can't
tell you where or how from what you've said; what's certain is that code
somewhere is using an instance of the DB class other than the Zope.DB
instance.

> So i replace number 7 to 15 in
>    DB.__init__ constructor.

Sick, but effective ;-)

>    Another problem that I have not-closed connections which i knew
> should be closed for a some time ago.
>    But i think this is problem of my code :)),

I hope so, because I couldn't understand the sentence above ;-)

> because usually connection closed by the REQUEST.close method,
> which placed in finally section in ZPublisher.Publish.publish_module
> function. But when catched  the ConflictError, REQUEST closed in
> ZPublisher.Publish.publish function  too. And i didn`t find the code
> where the connection is reopened, i think somewhere in the
> ZODB.ZApplication module.

Maybe someone more familiar with Zope (or Zope 2.6.1) than I am can comment.
If not, I'd ask about this part on a Zope list instead.

...

>    As i understand i must explicitly call the app._p_jar.sync() method
> in the begining of each iteration of my infinite loop.

I don't know whether this will help you:  you have to try it.  The thing to
try is to replace:

            while self.running:
                get_transaction().begin()

with:

            while self.running:
                app._p_jar.sync()

There's no need to do get_transaction().begin() if you're calling sync().
It's OK if you do both (it will just waste some time).

>  English is not my native laguage, and very sorry for my mistakes and
> typos :(.

You're doing fine, Victor!  You've already seen my best attempt to converse
in Russian ;-)