[ZODB-Dev] Anyone mind looking over a bit of code?

[Kenneth Miller]

All,

      If it's ok, I'd like to post a bit of code on here with the hope  
of getting a bit of help. What i'm attempting to do is create a simple  
database layer for all of my applications to traffic data in and out  
of my ZODB. I'm writing quite a few different apps, 10+ and growing,  
that all work with the same data. I've created a simple "newData"  
function that should decide what sort of data the incoming data is and  
arrange it in the database in an organized manner. The idea with all  
of this is if I want to change the way my database works, I don't have  
to change it across 10+ applications. I've taken a lot of the other  
methods out to simplify the post.

My current problem with the code is that as I'll try to insert say 150  
objects into the db, and then verify that those objects have been  
inserted into the db with another application. It appears as if  
they're being inserted in the application that's doing the insertions,  
but the "reader" app can only see a handful (<10) objects have been  
inserted. The odd thing is that every time i ask the "insertion"  
application to insert these 150 objects, the "reader" can see a  
handful more. It's as if the ZODB listens to my request to insert the  
first few objects, but the newer transactions after a certain point.  
Any help is really appreciated!

Regards,
Kenneth Miller


from This.That import Thing,Blirp,Thingy,Thingamajig,Something

class Layer:
     def __init__(self,host,port):
         """A layer for trafficing Teledrill data types in and out of  
a ZODB through a ZEO client.
         host - the hostname
         port - the port
         """

         self.host = host
         self.port = port
         self.addr = self.host,self.port
         self.lf = '/tmp/mwdlf'

         self.connect()

         #the below is an excellect way to wipe out your DB
         #if not self.verifyIntegrity():
         #    self.createDBSystem()

     def connect(self):
         """Connect the layer to the underlying database."""
         from ZEO import ClientStorage
         from ZEO import Exceptions
         from ZODB import DB

         self.storage =  
ClientStorage.ClientStorage(self.addr,wait=False) #try to connect,  
don't wait
         self.db = DB(self.storage)
         self.conn = self.db.open()
         self.root = self.conn.root()


     def disconnect(self):
         self.conn.close()
         self.db.close()
         self.storage.close()

     def verifyIntegrity(self):
         root = self.connect()

         #sync the DB
         #get the BTrees
         from BTrees import OOBTree
         from persistent.list import PersistentList
         #make sure the proper data containers exist
         try:
             if not isinstance(self.root['rawdata'],OOBTree.OOBTree):
                 return False
             if not isinstance(self.root['things1'],OOBTree.OOBTree):
                 return False
             if not isinstance(self.root['things2'],OOBTree.OOBTree):
                 return False
             if not isinstance(self.root['things3'],OOBTree.OOBTree):
                 return False
             if not isinstance(self.root['things4'],OOBTree.OOBTree):
                 return False
             if not isinstance(self.root['things5'],OOBTree.OOBTree):
                 return False
         except KeyError:
             return False

         #self.disconnect()
         #if all else
         return True

     def createDBSystem(self):
         #get the transaction module

         #root = self.connect()

         import transaction
         #and our BTrees and PL
         from BTrees import OOBTree
         from persistent.list import PersistentList
         #Create the containers
         self.root['rawdata'] = OOBTree.OOBTree()
         self.root['things1'] =  OOBTree.OOBTree()
         self.root['things2'] = OOBTree.OOBTree()
         self.root['things3'] = OOBTree.OOBTree()
         self.root['things4'] = OOBTree.OOBTree()
         self.root['things5'] = OOBTree.OOBTree()
         #make the changes permanent
         transaction.commit()



     def newData(self,data):

         print "Incoming Data!"
         #if we can iterate over the data
         try:
             for d in data:
                 self.newData(d) #iterate over the elements individual
             return True
         except TypeError:
             pass

         import transaction

         #validate the object
         myErr = ValueError('Data has no timeStamp attached!')

         try:
             if data.timeStamp==None:
                 raise myErr
         except AttributeError, e:
             print e
             raise myErr

         newKey = data.timeStamp

         if isinstance(data,Blirp.Blirp):
             print "It's a blirp!"
             if self.root['blirps'].has_key(newKey):
                 print "They key already exists!", newKey
                 raise KeyError('Data Exists!',newKey,data)
             else:
                 print "Trying to insert!"
                 while(not self.root['blirps'].insert(newKey,data)):  
print 'Try!'
                 print 'Done!'
             print len(self.root['blirps'])

         print "Committing"
         transaction.commit()

         return True