[Checkins] SVN: Sandbox/adamg/ocql/trunk/src/ocql/ improving documentation
Adam Groszer
agroszer at gmail.com
Thu Aug 21 04:48:36 EDT 2008
Log message for revision 90051:
improving documentation
Changed:
A Sandbox/adamg/ocql/trunk/src/ocql/LICENSE.txt
U Sandbox/adamg/ocql/trunk/src/ocql/compiler/compiler.txt
U Sandbox/adamg/ocql/trunk/src/ocql/compiler/compiler_optimized.txt
-=-
Added: Sandbox/adamg/ocql/trunk/src/ocql/LICENSE.txt
===================================================================
--- Sandbox/adamg/ocql/trunk/src/ocql/LICENSE.txt (rev 0)
+++ Sandbox/adamg/ocql/trunk/src/ocql/LICENSE.txt 2008-08-21 08:48:36 UTC (rev 90051)
@@ -0,0 +1,54 @@
+Zope Public License (ZPL) Version 2.1
+-------------------------------------
+
+A copyright notice accompanies this license document that
+identifies the copyright holders.
+
+This license has been certified as open source. It has also
+been designated as GPL compatible by the Free Software
+Foundation (FSF).
+
+Redistribution and use in source and binary forms, with or
+without modification, are permitted provided that the
+following conditions are met:
+
+1. Redistributions in source code must retain the
+ accompanying copyright notice, this list of conditions,
+ and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the accompanying
+ copyright notice, this list of conditions, and the
+ following disclaimer in the documentation and/or other
+ materials provided with the distribution.
+
+3. Names of the copyright holders must not be used to
+ endorse or promote products derived from this software
+ without prior written permission from the copyright
+ holders.
+
+4. The right to distribute this software or to use it for
+ any purpose does not give you the right to use
+ Servicemarks (sm) or Trademarks (tm) of the copyright
+ holders. Use of them is covered by separate agreement
+ with the copyright holders.
+
+5. If any files are modified, you must cause the modified
+ files to carry prominent notices stating that you changed
+ the files and the date of any change.
+
+Disclaimer
+
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS''
+ AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT
+ NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
+ AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN
+ NO EVENT SHALL THE COPYRIGHT HOLDERS BE
+ LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
+ OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
+ DAMAGE.
Property changes on: Sandbox/adamg/ocql/trunk/src/ocql/LICENSE.txt
___________________________________________________________________
Name: svn:keywords
+ Date Author Id Revision
Name: svn:eol-style
+ native
Modified: Sandbox/adamg/ocql/trunk/src/ocql/compiler/compiler.txt
===================================================================
--- Sandbox/adamg/ocql/trunk/src/ocql/compiler/compiler.txt 2008-08-21 07:31:22 UTC (rev 90050)
+++ Sandbox/adamg/ocql/trunk/src/ocql/compiler/compiler.txt 2008-08-21 08:48:36 UTC (rev 90051)
@@ -1,8 +1,15 @@
+Compiler
+========
+
+The task of the compiler is to compile the inputted algebra tree to python code.
+
I'm sorry, but I'm lazy here, in reality we would have to create
stubs for all ocql.rewriter.algebra classes and use those...
But it's a lot faster to reuse the parser and the existing classes.
+We need some imports and setup for things to work:
+
>>> from ocql.parser.queryparser import QueryParser
>>> from ocql.qoptimizer.qoptimizer import QueryOptimizer
>>> from ocql.rewriter.rewriter import Rewriter
@@ -10,109 +17,101 @@
>>> from ocql.compiler.compiler import AlgebraCompiler
>>> from ocql.testing.database import TestMetadata
-
>>> metadata = TestMetadata()
- >>> qo = QueryParser("set [ | 1 ]")(metadata)
- >>> opt = QueryOptimizer(qo)()
- >>> alg = Rewriter(opt)()
- >>> aopt = AlgebraOptimizer(alg)(metadata)
- >>> run = AlgebraCompiler(aopt)(metadata, alg)
- >>> print str(run)
- RunnableQuery: set([1])
- >>> metadata = TestMetadata()
- >>> qo = QueryParser("list [ | 1 ]")(metadata)
- >>> opt = QueryOptimizer(qo)()
- >>> alg = Rewriter(opt)()
- >>> aopt = AlgebraOptimizer(alg)(metadata)
- >>> run = AlgebraCompiler(aopt)(metadata, alg)
- >>> print str(run)
- RunnableQuery: [1]
+ >>> def make(query):
+ ... qo = QueryParser(query)(metadata)
+ ... opt = QueryOptimizer(qo)()
+ ... alg = Rewriter(opt)()
+ ... aopt = AlgebraOptimizer(alg)(metadata)
+ ... run = AlgebraCompiler(aopt)(metadata, alg)
+ ... return aopt, run
- >>> metadata = TestMetadata()
- >>> qo = QueryParser("set [ | 1 ] union set [|2]")(TestMetadata())
- >>> opt = QueryOptimizer(qo)()
- >>> alg = Rewriter(opt)()
- >>> aopt = AlgebraOptimizer(alg)(metadata)
- >>> run = AlgebraCompiler(aopt)(metadata, alg)
- >>> print str(run)
- RunnableQuery: set.union(set([1]), set([2]))
+We just want to see how the algebra compiles to code:
- >>> metadata = TestMetadata()
- >>> qo = QueryParser("list [ | 1 ] union list [|2]")(TestMetadata())
- >>> opt = QueryOptimizer(qo)()
- >>> alg = Rewriter(opt)()
- >>> aopt = AlgebraOptimizer(alg)(metadata)
- >>> run = AlgebraCompiler(aopt)(metadata, alg)
- >>> print str(run)
- RunnableQuery: ([1]+filter(lambda x:x not in [1],[2]))
+ >>> aopt, run = make("set [ | 1 ]")
+ >>> aopt
+ Head(Single(<type 'set'>, `1`))
+ >>> run.code
+ 'set([1])'
- >>> qo = QueryParser("set [ | 1 ] differ set [|2]")(TestMetadata())
- >>> opt = QueryOptimizer(qo)()
- >>> alg = Rewriter(opt)()
- >>> aopt = AlgebraOptimizer(alg)(metadata)
- >>> run = AlgebraCompiler(aopt)(metadata, alg)
- >>> print str(run)
- RunnableQuery: set.difference(set([1]), set([2]))
- >>> qo = QueryParser("list [ | 1 ] differ list [|2]")(TestMetadata())
- >>> opt = QueryOptimizer(qo)()
- >>> alg = Rewriter(opt)()
- >>> aopt = AlgebraOptimizer(alg)(metadata)
- >>> run = AlgebraCompiler(aopt)(metadata, alg)
- >>> print str(run)
- RunnableQuery: (filter(lambda x:x not in [2],[1]))
+ >>> aopt, run = make("list [ | 1 ]")
+ >>> aopt
+ Head(Single(<type 'list'>, `1`))
+ >>> run.code
+ '[1]'
- >>> metadata = TestMetadata()
- >>> qo = QueryParser("set [ i in ICourse | i ]")(TestMetadata())
- >>> opt = QueryOptimizer(qo)()
- >>> alg = Rewriter(opt)()
- >>> aopt = AlgebraOptimizer(alg)(metadata)
- >>> run = AlgebraCompiler(aopt)(metadata, alg)
- >>> print str(run)
- RunnableQuery: reduce(set.union, map(lambda i: set([i]), set(metadata.getAll("ICourse"))), set())
- >>> metadata = TestMetadata()
- >>> qo = QueryParser("list [ i in ICourse | i ]")(TestMetadata())
- >>> opt = QueryOptimizer(qo)()
- >>> alg = Rewriter(opt)()
- >>> aopt = AlgebraOptimizer(alg)(metadata)
- >>> run = AlgebraCompiler(aopt)(metadata, alg)
- >>> print str(run)
- RunnableQuery: reduce(operator.add, map(lambda i: [i], list(metadata.getAll("ICourse"))), [])
+ >>> aopt, run = make("set [ | 1 ] union set [|2]")
+ >>> aopt
+ Head(Union(<type 'set'>, Single(<type 'set'>, `1`), Single(<type 'set'>, `2`)))
+ >>> run.code
+ 'set.union(set([1]), set([2]))'
- >>> metadata = TestMetadata()
- >>> qo = QueryParser("len(set [ i in ICourse | i ])")(TestMetadata())
- >>> opt = QueryOptimizer(qo)()
- >>> alg = Rewriter(opt)()
- >>> aopt = AlgebraOptimizer(alg)(metadata)
- >>> run = AlgebraCompiler(aopt)(metadata, alg)
- >>> print str(run)
- RunnableQuery: reduce(operator.add, map(lambda i: 1, reduce(set.union, map(lambda i: set([i]), set(metadata.getAll("ICourse"))), set())), 0)
- >>> metadata = TestMetadata()
- >>> qo = QueryParser("len(list [ i in ICourse | i ])")(TestMetadata())
- >>> opt = QueryOptimizer(qo)()
- >>> alg = Rewriter(opt)()
- >>> aopt = AlgebraOptimizer(alg)(metadata)
- >>> run = AlgebraCompiler(aopt)(metadata, alg)
- >>> print str(run)
- RunnableQuery: reduce(operator.add, map(lambda i: 1, reduce(operator.add, map(lambda i: [i], list(metadata.getAll("ICourse"))), [])), 0)
+ >>> aopt, run = make("list [ | 1 ] union list [|2]")
+ >>> aopt
+ Head(Union(<type 'list'>, Single(<type 'list'>, `1`), Single(<type 'list'>, `2`)))
+ >>> run.code
+ '([1]+filter(lambda x:x not in [1],[2]))'
- >>> metadata = TestMetadata()
- >>> qo = QueryParser("set [ c in ICourse | c.code ]")(TestMetadata())
- >>> opt = QueryOptimizer(qo)()
- >>> alg = Rewriter(opt)()
- >>> aopt = AlgebraOptimizer(alg)(metadata)
- >>> run = AlgebraCompiler(aopt)(metadata, alg)
- >>> print str(run)
- RunnableQuery: reduce(set.union, map(lambda c: set([c.code]), set(metadata.getAll("ICourse"))), set())
- >>> metadata = TestMetadata()
- >>> qo = QueryParser("list [ c in ICourse | c.code ]")(TestMetadata())
- >>> opt = QueryOptimizer(qo)()
- >>> alg = Rewriter(opt)()
- >>> aopt = AlgebraOptimizer(alg)(metadata)
- >>> run = AlgebraCompiler(aopt)(metadata, alg)
- >>> print str(run)
- RunnableQuery: reduce(operator.add, map(lambda c: [c.code], list(metadata.getAll("ICourse"))), [])
+ >>> aopt, run = make("set [ | 1 ] differ set [|2]")
+ >>> aopt
+ Head(Differ(<type 'set'>, Single(<type 'set'>, `1`), Single(<type 'set'>, `2`)))
+ >>> run.code
+ 'set.difference(set([1]), set([2]))'
+
+
+ >>> aopt, run = make("list [ | 1 ] differ list [|2]")
+ >>> aopt
+ Head(Differ(<type 'list'>, Single(<type 'list'>, `1`), Single(<type 'list'>, `2`)))
+ >>> run.code
+ '(filter(lambda x:x not in [2],[1]))'
+
+
+
+ >>> aopt, run = make("set [ i in ICourse | i ]")
+ >>> aopt
+ Head(Iter(<type 'set'>, Lambda i: Single(<type 'set'>, i), Make(<type 'set'>, <type 'set'>, ICourse)))
+ >>> run.code
+ 'reduce(set.union, map(lambda i: set([i]), set(metadata.getAll("ICourse"))), set())'
+
+
+ >>> aopt, run = make("list [ i in ICourse | i ]")
+ >>> aopt
+ Head(Iter(<type 'list'>, Lambda i: Single(<type 'list'>, i), Make(<type 'list'>, <type 'set'>, ICourse)))
+ >>> run.code
+ 'reduce(operator.add, map(lambda i: [i], list(metadata.getAll("ICourse"))), [])'
+
+
+
+ >>> aopt, run = make("set [ c in ICourse | c.code ]")
+ >>> aopt
+ Head(Iter(<type 'set'>, Lambda c: Single(<type 'set'>, c.code), Make(<type 'set'>, <type 'set'>, ICourse)))
+ >>> run.code
+ 'reduce(set.union, map(lambda c: set([c.code]), set(metadata.getAll("ICourse"))), set())'
+
+
+ >>> aopt, run = make("list [ c in ICourse | c.code ]")
+ >>> aopt
+ Head(Iter(<type 'list'>, Lambda c: Single(<type 'list'>, c.code), Make(<type 'list'>, <type 'set'>, ICourse)))
+ >>> run.code
+ 'reduce(operator.add, map(lambda c: [c.code], list(metadata.getAll("ICourse"))), [])'
+
+
+
+
+ >>> aopt, run = make("len(set [ i in ICourse | i ])")
+ >>> aopt
+ Head(Reduce(<type 'set'>, `0`, Lambda i: `1`, +, Iter(<type 'set'>, Lambda i: Single(<type 'set'>, i), Make(<type 'set'>, <type 'set'>, ICourse))))
+ >>> run.code
+ 'reduce(operator.add, map(lambda i: 1, reduce(set.union, map(lambda i: set([i]), set(metadata.getAll("ICourse"))), set())), 0)'
+
+
+ >>> aopt, run = make("len(list [ i in ICourse | i ])")
+ >>> aopt
+ Head(Reduce(<type 'list'>, `0`, Lambda i: `1`, +, Iter(<type 'list'>, Lambda i: Single(<type 'list'>, i), Make(<type 'list'>, <type 'set'>, ICourse))))
+ >>> run.code
+ 'reduce(operator.add, map(lambda i: 1, reduce(operator.add, map(lambda i: [i], list(metadata.getAll("ICourse"))), [])), 0)'
Modified: Sandbox/adamg/ocql/trunk/src/ocql/compiler/compiler_optimized.txt
===================================================================
--- Sandbox/adamg/ocql/trunk/src/ocql/compiler/compiler_optimized.txt 2008-08-21 07:31:22 UTC (rev 90050)
+++ Sandbox/adamg/ocql/trunk/src/ocql/compiler/compiler_optimized.txt 2008-08-21 08:48:36 UTC (rev 90051)
@@ -1,6 +1,23 @@
-Checking here how an optimized algebra for zope indexes compiles into python code
+Optimized compilation
+=====================
+Checking here how an optimized algebra for zope indexes compiles into python code.
+
+Actually the algebra optimizer does much of the work. It identifies the
+possibility of the optimization and replaces the objects in the algebra
+tree for the optimized ones.
+
+Some support is needed from the database (metadata class).
+It employs a method to support the retrieval from the index, otherwise it would
+be quite difficult to do the retrieval inline.
+
+Note: this optimization is very much preliminary.
+
+
+
+We need some imports and setup:
+
>>> from ocql.engine import OCQLEngine
>>> from ocql.testing.utils_opt import setupInterfaces
@@ -19,13 +36,16 @@
>>> run = OCQLEngine().compile(query)
Here is the runnable code:
+It uses metadata.getAll, which simply returns all objects of an IF.
>>> run
- RunnableQuery: reduce(set.union, map(lambda i: set([i]), set(metadata.getAll("IUnOptimizedClass"))), set())
+ RunnableQuery: reduce(set.union, map(lambda i: set([i]),
+ set(metadata.getAll("IUnOptimizedClass"))), set())
>>> result = run.execute()
>>> sorted(list(result), key=lambda x:x.value)
- [UnOpt: 0, UnOpt: 1, UnOpt: 2, UnOpt: 3, UnOpt: 4, UnOpt: 5, UnOpt: 6, UnOpt: 7, UnOpt: 8, UnOpt: 9]
+ [UnOpt: 0, UnOpt: 1, UnOpt: 2, UnOpt: 3, UnOpt: 4, UnOpt: 5, UnOpt: 6,
+ UnOpt: 7, UnOpt: 8, UnOpt: 9]
>>> type(result)
<type 'set'>
@@ -37,6 +57,8 @@
>>> run = OCQLEngine().compile(query)
Here is the runnable code:
+It still uses metadata.getAll, because there is no index for this class and
+property.
>>> run
RunnableQuery:
@@ -59,6 +81,7 @@
>>> run = OCQLEngine().compile(query)
Here is the runnable code:
+It uses now metadata.getFromIndex as there is an index for the class and property.
>>> run
RunnableQuery:
More information about the Checkins
mailing list