<html><head><style type="text/css"><!-- DIV {margin:0px;} --></style></head><body><div style="font-family:times new roman, new york, times, serif;font-size:12pt"><DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">Sorry Andreas..</DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif"> </DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">My product is rewritten in python placed in the product folder. So yes, it is a instance of a class. </DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif"> </DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">In the future, I will be storing more data into one instance from other dbase file (The total columns may vary).</DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif"> </DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">I'm still in the developing stage and this is just a test run to know that is the length of processing time I'm look at. I want each entry to be an instance because I'm planning to create other interactive functions (edit and query(ZCatalog maybe, I'm not sure yet) and more ) for particular info.</DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif"> </DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">I've never thought of using a BTree because I don't know enough about it. I'll look into it but will BTree still be a better choice than making instance if I'm going to make interactive functions?</DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif"> </DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">Any other suggestions?</DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif"><BR><BR> </DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">----- Original Message ----<BR>From: Andreas Jung <lists@zopyx.com><BR>To: Allen Huang <swapp0@yahoo.com>; Zope <zope@zope.org><BR>Sent: Tuesday, January 9, 2007 1:55:34 PM<BR>Subject: Re: [Zope] need advice on mass data processing<BR><BR>
<DIV>--On 8. Januar 2007 19:28:32 -0800 Allen Huang <swapp0@yahoo.com> wrote:<BR><BR>> I have a data file that has over 110000 entry of 3 column data (string,<BR>> float, float)<BR>> currently I have written my program so it will do an entry by entry<BR>> processing with zope. This operation is like this<BR>> 1. read data (the data file)<BR>> 2. create product (a python product that store three field data: one<BR>> string and two float data) 3. update product (update the three field<BR>> entries)<BR><BR>Please name things the right way. A "Product" is basically a Zope/Python <BR>package that contains definitions of classes, scripts, templates etc.<BR><BR>You mean instances of a particular class?<BR><BR>><BR>> when I first tried it out with the first 1000 entries it took about 30<BR>> seconds. That means its going to take 50 ~ 60 minutes for 110000 entries.<BR><BR>You're creating 110k instances for storing a string and two floats?<BR>If
yes, that's stupid idea.<BR><BR>You can persistent large amounts of data within a single instances<BR>by using Zope BTrees.<BR><BR>> It not every day that you have to process over 110000 data entries but<BR>> processing over 60 minutes is still kind of long.<BR>><BR><BR>What kind of processing?<BR><BR>-aj</DIV></DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif"><BR></DIV></div><br>__________________________________________________<br>Do You Yahoo!?<br>Tired of spam? Yahoo! Mail has the best spam protection around <br>http://mail.yahoo.com </body></html>