[ZODB-Dev] Memory out of control when *NOT* changing objects

[Malcolm Cleaton]

I'm having a memory problem with a script for updating objects in the ZODB.

I'm running on Zope 2.7.3b1, and invoking the script using zopectl run.

The loop looks like this, although some method names have been
changed to protect the innocent:

results = portal_catalog.searchResults(...)
num = 0
for brain in results:
	ob = brain.getObject()
	if needsChanging(ob):
		change(ob)
	# else
	# 	ob._p_deactivate()

	num += 1
	if num % 1000 == 0:
		get_transaction().commit()

With the code as shown, memory usage is fine if the objects actually need
changing. However, if they don't, it spirals out of control, straight for
swap death; I suspect the script may eventually finish but I don't have
the patience.

With the two commented lines uncommented, so objects get explicitly
deactivated when no longer needed, memory usage is fine.

But, why is this necessary? I thought the only circumstance that would
bring swap death to the ZODB was a super-sized transaction, full of
changed objects. However, in this case, loading the objects but not
changing them appears to be unsafe.

Is this a bug, or am I misunderstanding the rules of the ZODB?

Thanks,
Malcolm.