<HTML><BODY style="word-wrap: break-word; -khtml-nbsp-mode: space; -khtml-line-break: after-white-space; ">Quick note...<DIV><BR class="khtml-block-placeholder"></DIV><DIV>Smaller buckets, fewer conflicts, more overhead on reading and writing.<DIV>Larger buckets, more conflicts, less overhead on reading and writing.</DIV><DIV>One bucket ... constant conflicts.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>I'd bet that the additional tree with tiny buckets would be best. Transfer them into the normal tree once the overhead starts rising.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>How to transfer them? You would not want a single transaction to take the hit for the whole transfer, so have a low and high water mark. When hitting HWM, transfer only until LWM is reached. Or, just focus on transferring some items out of a single tree, to avoid the cost of tree rebalancing on the additional tree at the same time as rebalancing on the main tree.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>Sounds like a fin project,</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>--r.</DIV><DIV><BR><DIV><DIV>On 1 Nov 2007, at 21:00, David Binger wrote:</DIV><BR class="Apple-interchange-newline"><BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">On Nov 1, 2007, at 4:25 PM, Matt Hamilton wrote:</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV> <BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">David Binger <dbinger <at> mems-exchange.org> writes:</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV> <BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">On Nov 1, 2007, at 7:05 AM, Matt Hamilton wrote:</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV> <BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Ie we perhaps look at a catalog data structure</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">in which writes are initially done to some kind of queue then moved</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">to the</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">BTrees at a later point.</DIV> </BLOCKQUOTE><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">A suggestion: use a pair of BTrees, one with a high branching factor</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">(bucket size)</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">and one with a very low branching factor.<SPAN class="Apple-converted-space"> </SPAN>Force all writes into the</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">tree with little</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">buckets.<SPAN class="Apple-converted-space"> </SPAN>Make every search look in both trees.<SPAN class="Apple-converted-space"> </SPAN>Consolidate</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">occasionally.</DIV> </BLOCKQUOTE><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">An interesting idea.<SPAN class="Apple-converted-space"> </SPAN>Surely we need the opposite though, and that is an</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">additional BTree with a very large bucket size, as we want to minimize the</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">chance of a bucket split when inserting?<SPAN class="Apple-converted-space"> </SPAN>Then we occasionally consolidate and</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">move the items in the original BTree with the regular bucket size/branch factor.</DIV> </BLOCKQUOTE><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">You may be right about that.<SPAN class="Apple-converted-space"> </SPAN>Conflict resolution makes it harder for</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">me to predict which way is better.<SPAN class="Apple-converted-space"> </SPAN>If you don't have conflict resolution</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">for insertions, then I think the smaller buckets are definitely better</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">for avoiding conflicts.<SPAN class="Apple-converted-space"> </SPAN>In either case, smaller buckets reduce the</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">size and serialization time of the insertion transactions, and that alone</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">*might* be a reason to favor them.<SPAN class="Apple-converted-space"> </SPAN>I think I'd still bet on smaller</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">buckets, but tests would expose the trade-offs.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">_______________________________________________</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">For more information about ZODB, see the ZODB Wiki:</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><A href="http://www.zope.org/Wikis/ZODB/">http://www.zope.org/Wikis/ZODB/</A></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">ZODB-Dev mailing list<SPAN class="Apple-converted-space"> </SPAN>-<SPAN class="Apple-converted-space"> </SPAN><A href="mailto:ZODB-Dev@zope.org">ZODB-Dev@zope.org</A></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><A href="http://mail.zope.org/mailman/listinfo/zodb-dev">http://mail.zope.org/mailman/listinfo/zodb-dev</A></DIV> </BLOCKQUOTE></DIV><BR><DIV> <SPAN class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Verdana; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><SPAN class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Verdana; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><SPAN class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Verdana; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><DIV>Russ Ferriday</DIV><DIV><A href="mailto:russ@indepthoptics.com">russ@indepthoptics.com</A></DIV><DIV><FONT class="Apple-style-span" size="2"><SPAN class="Apple-style-span" style="font-size: 10px; line-height: 13px;"><B style="font-size: 10px; font-weight: bold; line-height: 13px; "><SPAN class="Apple-style-span" style="font-size: 10px; font-weight: bold; line-height: 13px; "><SPAN class="Apple-style-span" style="font-size: 10px; font-weight: bold; line-height: 13px; "><SPAN class="Apple-style-span" style="font-size: 10px; font-weight: bold; line-height: 13px; ">office: +44 118 3217026</SPAN></SPAN></SPAN></B></SPAN></FONT></DIV><DIV><FONT class="Apple-style-span" size="2"><SPAN class="Apple-style-span" style="font-size: 10px; line-height: 13px;"><B style="font-size: 10px; font-weight: bold; line-height: 13px; "><SPAN class="Apple-style-span" style="font-size: 10px; font-weight: bold; line-height: 13px; "><SPAN class="Apple-style-span" style="font-size: 10px; font-weight: bold; line-height: 13px; "><SPAN class="Apple-style-span" style="font-size: 10px; font-weight: bold; line-height: 13px; ">mobile: +44 7789 338868</SPAN></SPAN></SPAN></B></SPAN></FONT></DIV><DIV><FONT class="Apple-style-span" size="2"><SPAN class="Apple-style-span" style="font-size: 10px; line-height: 13px;"><B style="font-size: 10px; font-weight: bold; line-height: 13px; "><SPAN class="Apple-style-span" style="font-size: 10px; font-weight: bold; line-height: 13px; "><SPAN class="Apple-style-span" style="font-size: 10px; font-weight: bold; line-height: 13px; "><SPAN class="Apple-style-span" style="font-size: 10px; font-weight: bold; line-height: 13px; ">skype: ferriday </SPAN></SPAN></SPAN></B></SPAN></FONT></DIV><DIV><BR class="khtml-block-placeholder"></DIV><BR class="Apple-interchange-newline"></SPAN></SPAN></SPAN> </DIV><BR></DIV></DIV></BODY></HTML>