[Checkins] SVN: Sandbox/adamg/ocql/trunk/src/ocql/ improving documentation

Adam Groszer agroszer at gmail.com
Thu Aug 21 04:48:36 EDT 2008


Log message for revision 90051:
  improving documentation

Changed:
  A   Sandbox/adamg/ocql/trunk/src/ocql/LICENSE.txt
  U   Sandbox/adamg/ocql/trunk/src/ocql/compiler/compiler.txt
  U   Sandbox/adamg/ocql/trunk/src/ocql/compiler/compiler_optimized.txt

-=-
Added: Sandbox/adamg/ocql/trunk/src/ocql/LICENSE.txt
===================================================================
--- Sandbox/adamg/ocql/trunk/src/ocql/LICENSE.txt	                        (rev 0)
+++ Sandbox/adamg/ocql/trunk/src/ocql/LICENSE.txt	2008-08-21 08:48:36 UTC (rev 90051)
@@ -0,0 +1,54 @@
+Zope Public License (ZPL) Version 2.1
+-------------------------------------
+
+A copyright notice accompanies this license document that
+identifies the copyright holders.
+
+This license has been certified as open source. It has also
+been designated as GPL compatible by the Free Software
+Foundation (FSF).
+
+Redistribution and use in source and binary forms, with or
+without modification, are permitted provided that the
+following conditions are met:
+
+1. Redistributions in source code must retain the
+   accompanying copyright notice, this list of conditions,
+   and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the accompanying
+   copyright notice, this list of conditions, and the
+   following disclaimer in the documentation and/or other
+   materials provided with the distribution.
+
+3. Names of the copyright holders must not be used to
+   endorse or promote products derived from this software
+   without prior written permission from the copyright
+   holders.
+
+4. The right to distribute this software or to use it for
+   any purpose does not give you the right to use
+   Servicemarks (sm) or Trademarks (tm) of the copyright
+   holders. Use of them is covered by separate agreement
+   with the copyright holders.
+
+5. If any files are modified, you must cause the modified
+   files to carry prominent notices stating that you changed
+   the files and the date of any change.
+
+Disclaimer
+
+  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS''
+  AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT
+  NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
+  AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.  IN
+  NO EVENT SHALL THE COPYRIGHT HOLDERS BE
+  LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+  EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+  LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+  LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+  HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+  CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
+  OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
+  DAMAGE.


Property changes on: Sandbox/adamg/ocql/trunk/src/ocql/LICENSE.txt
___________________________________________________________________
Name: svn:keywords
   + Date Author Id Revision
Name: svn:eol-style
   + native

Modified: Sandbox/adamg/ocql/trunk/src/ocql/compiler/compiler.txt
===================================================================
--- Sandbox/adamg/ocql/trunk/src/ocql/compiler/compiler.txt	2008-08-21 07:31:22 UTC (rev 90050)
+++ Sandbox/adamg/ocql/trunk/src/ocql/compiler/compiler.txt	2008-08-21 08:48:36 UTC (rev 90051)
@@ -1,8 +1,15 @@
 
+Compiler
+========
+
+The task of the compiler is to compile the inputted algebra tree to python code.
+
 I'm sorry, but I'm lazy here, in reality we would have to create
 stubs for all ocql.rewriter.algebra classes and use those...
 But it's a lot faster to reuse the parser and the existing classes.
 
+We need some imports and setup for things to work:
+
     >>> from ocql.parser.queryparser import QueryParser
     >>> from ocql.qoptimizer.qoptimizer import QueryOptimizer
     >>> from ocql.rewriter.rewriter import Rewriter
@@ -10,109 +17,101 @@
     >>> from ocql.compiler.compiler import AlgebraCompiler
 
     >>> from ocql.testing.database import TestMetadata
-
     >>> metadata = TestMetadata()
-    >>> qo = QueryParser("set [ | 1 ]")(metadata)
-    >>> opt = QueryOptimizer(qo)()
-    >>> alg = Rewriter(opt)()
-    >>> aopt = AlgebraOptimizer(alg)(metadata)
-    >>> run = AlgebraCompiler(aopt)(metadata, alg)
-    >>> print str(run)
-    RunnableQuery: set([1])
 
-    >>> metadata = TestMetadata()
-    >>> qo = QueryParser("list [ | 1 ]")(metadata)
-    >>> opt = QueryOptimizer(qo)()
-    >>> alg = Rewriter(opt)()
-    >>> aopt = AlgebraOptimizer(alg)(metadata)
-    >>> run = AlgebraCompiler(aopt)(metadata, alg)
-    >>> print str(run)
-    RunnableQuery: [1]
+    >>> def make(query):
+    ...     qo = QueryParser(query)(metadata)
+    ...     opt = QueryOptimizer(qo)()
+    ...     alg = Rewriter(opt)()
+    ...     aopt = AlgebraOptimizer(alg)(metadata)
+    ...     run = AlgebraCompiler(aopt)(metadata, alg)
+    ...     return aopt, run
 
-    >>> metadata = TestMetadata()
-    >>> qo = QueryParser("set [ | 1 ] union set [|2]")(TestMetadata())
-    >>> opt = QueryOptimizer(qo)()
-    >>> alg = Rewriter(opt)()
-    >>> aopt = AlgebraOptimizer(alg)(metadata)
-    >>> run = AlgebraCompiler(aopt)(metadata, alg)
-    >>> print str(run)
-    RunnableQuery: set.union(set([1]), set([2]))
+We just want to see how the algebra compiles to code:
 
-    >>> metadata = TestMetadata()
-    >>> qo = QueryParser("list [ | 1 ] union list [|2]")(TestMetadata())
-    >>> opt = QueryOptimizer(qo)()
-    >>> alg = Rewriter(opt)()
-    >>> aopt = AlgebraOptimizer(alg)(metadata)
-    >>> run = AlgebraCompiler(aopt)(metadata, alg)
-    >>> print str(run)
-    RunnableQuery: ([1]+filter(lambda x:x not in [1],[2]))
+    >>> aopt, run = make("set [ | 1 ]")
+    >>> aopt
+    Head(Single(<type 'set'>, `1`))
+    >>> run.code
+    'set([1])'
 
-    >>> qo = QueryParser("set [ | 1 ] differ set [|2]")(TestMetadata())
-    >>> opt = QueryOptimizer(qo)()
-    >>> alg = Rewriter(opt)()
-    >>> aopt = AlgebraOptimizer(alg)(metadata)
-    >>> run = AlgebraCompiler(aopt)(metadata, alg)
-    >>> print str(run)
-    RunnableQuery: set.difference(set([1]), set([2]))
 
-    >>> qo = QueryParser("list [ | 1 ] differ list [|2]")(TestMetadata())
-    >>> opt = QueryOptimizer(qo)()
-    >>> alg = Rewriter(opt)()
-    >>> aopt = AlgebraOptimizer(alg)(metadata)
-    >>> run = AlgebraCompiler(aopt)(metadata, alg)
-    >>> print str(run)
-    RunnableQuery: (filter(lambda x:x not in [2],[1]))
+    >>> aopt, run = make("list [ | 1 ]")
+    >>> aopt
+    Head(Single(<type 'list'>, `1`))
+    >>> run.code
+    '[1]'
 
-    >>> metadata = TestMetadata()
-    >>> qo = QueryParser("set [ i in ICourse | i ]")(TestMetadata())
-    >>> opt = QueryOptimizer(qo)()
-    >>> alg = Rewriter(opt)()
-    >>> aopt = AlgebraOptimizer(alg)(metadata)
-    >>> run = AlgebraCompiler(aopt)(metadata, alg)
-    >>> print str(run)
-    RunnableQuery: reduce(set.union, map(lambda i: set([i]), set(metadata.getAll("ICourse"))), set())
 
-    >>> metadata = TestMetadata()
-    >>> qo = QueryParser("list [ i in ICourse | i ]")(TestMetadata())
-    >>> opt = QueryOptimizer(qo)()
-    >>> alg = Rewriter(opt)()
-    >>> aopt = AlgebraOptimizer(alg)(metadata)
-    >>> run = AlgebraCompiler(aopt)(metadata, alg)
-    >>> print str(run)
-    RunnableQuery: reduce(operator.add, map(lambda i: [i], list(metadata.getAll("ICourse"))), [])
+    >>> aopt, run = make("set [ | 1 ] union set [|2]")
+    >>> aopt
+    Head(Union(<type 'set'>, Single(<type 'set'>, `1`), Single(<type 'set'>, `2`)))
+    >>> run.code
+    'set.union(set([1]), set([2]))'
 
-    >>> metadata = TestMetadata()
-    >>> qo = QueryParser("len(set [ i in ICourse | i ])")(TestMetadata())
-    >>> opt = QueryOptimizer(qo)()
-    >>> alg = Rewriter(opt)()
-    >>> aopt = AlgebraOptimizer(alg)(metadata)
-    >>> run = AlgebraCompiler(aopt)(metadata, alg)
-    >>> print str(run)
-    RunnableQuery: reduce(operator.add, map(lambda i: 1, reduce(set.union, map(lambda i: set([i]), set(metadata.getAll("ICourse"))), set())), 0)
 
-    >>> metadata = TestMetadata()
-    >>> qo = QueryParser("len(list [ i in ICourse | i ])")(TestMetadata())
-    >>> opt = QueryOptimizer(qo)()
-    >>> alg = Rewriter(opt)()
-    >>> aopt = AlgebraOptimizer(alg)(metadata)
-    >>> run = AlgebraCompiler(aopt)(metadata, alg)
-    >>> print str(run)
-    RunnableQuery: reduce(operator.add, map(lambda i: 1, reduce(operator.add, map(lambda i: [i], list(metadata.getAll("ICourse"))), [])), 0)
+    >>> aopt, run = make("list [ | 1 ] union list [|2]")
+    >>> aopt
+    Head(Union(<type 'list'>, Single(<type 'list'>, `1`), Single(<type 'list'>, `2`)))
+    >>> run.code
+    '([1]+filter(lambda x:x not in [1],[2]))'
 
-    >>> metadata = TestMetadata()
-    >>> qo = QueryParser("set [ c in ICourse | c.code ]")(TestMetadata())
-    >>> opt = QueryOptimizer(qo)()
-    >>> alg = Rewriter(opt)()
-    >>> aopt = AlgebraOptimizer(alg)(metadata)
-    >>> run = AlgebraCompiler(aopt)(metadata, alg)
-    >>> print str(run)
-    RunnableQuery: reduce(set.union, map(lambda c: set([c.code]), set(metadata.getAll("ICourse"))), set())
 
-    >>> metadata = TestMetadata()
-    >>> qo = QueryParser("list [ c in ICourse | c.code ]")(TestMetadata())
-    >>> opt = QueryOptimizer(qo)()
-    >>> alg = Rewriter(opt)()
-    >>> aopt = AlgebraOptimizer(alg)(metadata)
-    >>> run = AlgebraCompiler(aopt)(metadata, alg)
-    >>> print str(run)
-    RunnableQuery: reduce(operator.add, map(lambda c: [c.code], list(metadata.getAll("ICourse"))), [])
+    >>> aopt, run = make("set [ | 1 ] differ set [|2]")
+    >>> aopt
+    Head(Differ(<type 'set'>, Single(<type 'set'>, `1`), Single(<type 'set'>, `2`)))
+    >>> run.code
+    'set.difference(set([1]), set([2]))'
+
+
+    >>> aopt, run = make("list [ | 1 ] differ list [|2]")
+    >>> aopt
+    Head(Differ(<type 'list'>, Single(<type 'list'>, `1`), Single(<type 'list'>, `2`)))
+    >>> run.code
+    '(filter(lambda x:x not in [2],[1]))'
+
+
+
+    >>> aopt, run = make("set [ i in ICourse | i ]")
+    >>> aopt
+    Head(Iter(<type 'set'>, Lambda i: Single(<type 'set'>, i), Make(<type 'set'>, <type 'set'>, ICourse)))
+    >>> run.code
+    'reduce(set.union, map(lambda i: set([i]), set(metadata.getAll("ICourse"))), set())'
+
+
+    >>> aopt, run = make("list [ i in ICourse | i ]")
+    >>> aopt
+    Head(Iter(<type 'list'>, Lambda i: Single(<type 'list'>, i), Make(<type 'list'>, <type 'set'>, ICourse)))
+    >>> run.code
+    'reduce(operator.add, map(lambda i: [i], list(metadata.getAll("ICourse"))), [])'
+
+
+
+    >>> aopt, run = make("set [ c in ICourse | c.code ]")
+    >>> aopt
+    Head(Iter(<type 'set'>, Lambda c: Single(<type 'set'>, c.code), Make(<type 'set'>, <type 'set'>, ICourse)))
+    >>> run.code
+    'reduce(set.union, map(lambda c: set([c.code]), set(metadata.getAll("ICourse"))), set())'
+
+
+    >>> aopt, run = make("list [ c in ICourse | c.code ]")
+    >>> aopt
+    Head(Iter(<type 'list'>, Lambda c: Single(<type 'list'>, c.code), Make(<type 'list'>, <type 'set'>, ICourse)))
+    >>> run.code
+    'reduce(operator.add, map(lambda c: [c.code], list(metadata.getAll("ICourse"))), [])'
+
+
+
+
+    >>> aopt, run = make("len(set [ i in ICourse | i ])")
+    >>> aopt
+    Head(Reduce(<type 'set'>, `0`, Lambda i: `1`, +, Iter(<type 'set'>, Lambda i: Single(<type 'set'>, i), Make(<type 'set'>, <type 'set'>, ICourse))))
+    >>> run.code
+    'reduce(operator.add, map(lambda i: 1, reduce(set.union, map(lambda i: set([i]), set(metadata.getAll("ICourse"))), set())), 0)'
+
+
+    >>> aopt, run = make("len(list [ i in ICourse | i ])")
+    >>> aopt
+    Head(Reduce(<type 'list'>, `0`, Lambda i: `1`, +, Iter(<type 'list'>, Lambda i: Single(<type 'list'>, i), Make(<type 'list'>, <type 'set'>, ICourse))))
+    >>> run.code
+    'reduce(operator.add, map(lambda i: 1, reduce(operator.add, map(lambda i: [i], list(metadata.getAll("ICourse"))), [])), 0)'

Modified: Sandbox/adamg/ocql/trunk/src/ocql/compiler/compiler_optimized.txt
===================================================================
--- Sandbox/adamg/ocql/trunk/src/ocql/compiler/compiler_optimized.txt	2008-08-21 07:31:22 UTC (rev 90050)
+++ Sandbox/adamg/ocql/trunk/src/ocql/compiler/compiler_optimized.txt	2008-08-21 08:48:36 UTC (rev 90051)
@@ -1,6 +1,23 @@
 
-Checking here how an optimized algebra for zope indexes compiles into python code
+Optimized compilation
+=====================
 
+Checking here how an optimized algebra for zope indexes compiles into python code.
+
+Actually the algebra optimizer does much of the work. It identifies the
+possibility of the optimization and replaces the objects in the algebra
+tree for the optimized ones.
+
+Some support is needed from the database (metadata class).
+It employs a method to support the retrieval from the index, otherwise it would
+be quite difficult to do the retrieval inline.
+
+Note: this optimization is very much preliminary.
+
+
+
+We need some imports and setup:
+
     >>> from ocql.engine import OCQLEngine
 
     >>> from ocql.testing.utils_opt import setupInterfaces
@@ -19,13 +36,16 @@
     >>> run = OCQLEngine().compile(query)
 
 Here is the runnable code:
+It uses metadata.getAll, which simply returns all objects of an IF.
 
     >>> run
-    RunnableQuery: reduce(set.union, map(lambda i: set([i]), set(metadata.getAll("IUnOptimizedClass"))), set())
+    RunnableQuery: reduce(set.union, map(lambda i: set([i]),
+    set(metadata.getAll("IUnOptimizedClass"))), set())
 
     >>> result = run.execute()
     >>> sorted(list(result), key=lambda x:x.value)
-    [UnOpt: 0, UnOpt: 1, UnOpt: 2, UnOpt: 3, UnOpt: 4, UnOpt: 5, UnOpt: 6, UnOpt: 7, UnOpt: 8, UnOpt: 9]
+    [UnOpt: 0, UnOpt: 1, UnOpt: 2, UnOpt: 3, UnOpt: 4, UnOpt: 5, UnOpt: 6,
+    UnOpt: 7, UnOpt: 8, UnOpt: 9]
     >>> type(result)
     <type 'set'>
 
@@ -37,6 +57,8 @@
     >>> run = OCQLEngine().compile(query)
 
 Here is the runnable code:
+It still uses metadata.getAll, because there is no index for this class and
+property.
 
     >>> run
     RunnableQuery:
@@ -59,6 +81,7 @@
     >>> run = OCQLEngine().compile(query)
 
 Here is the runnable code:
+It uses now metadata.getFromIndex as there is an index for the class and property.
 
     >>> run
     RunnableQuery:



More information about the Checkins mailing list