[Zope3-checkins] CVS: Zope3/src/zope/tal - dummyengine.py:1.11.20.1 interfaces.py:1.9.16.1 taldefs.py:1.6.22.1 talgenerator.py:1.9.6.1 talgettext.py:1.14.22.1 talinterpreter.py:1.22.6.1 history.txt:NONE readme.txt:NONE

Stephan Richter srichter at cosmos.phy.tufts.edu
Wed Aug 20 10:00:35 EDT 2003


Update of /cvs-repository/Zope3/src/zope/tal
In directory cvs.zope.org:/tmp/cvs-serv5066

Modified Files:
      Tag: srichter-inlinepython-branch
	dummyengine.py interfaces.py taldefs.py talgenerator.py 
	talgettext.py talinterpreter.py 
Removed Files:
      Tag: srichter-inlinepython-branch
	history.txt readme.txt 
Log Message:
This commit is a bit chaotic, since I accidently created my branch on an old 
checkout.

So the first thing this does is an update with HEAD.

The second part I got the following working:

  <p tal:script="server-python">print "hello"</p>

  Result <p>hello\n</p>\n

This fails the requirement that we eventually want this syntax:

  <script lang="server-python">
    print "hello"
  </script>

It will be fairly difficult to do this, since we have to introduce sort of 
a new namespace to handle this.

Other open issues (I try to solve today):

- How to store global versus local variables. There are two models I can
  see being practical:

  (a) Make all declared variables inside the code global. This would be 
      acceptable, since we want to keep it simple and the feature is aimed
      at the scripter, who usually does not think carefully about namespaces
      anyways.

  (b) Have a special markup for making variables global, maybe by reusing 
      the global keyword?

- How to handle indentation. I have not explored this, but plan to do this 
  today. I also do not know what will happen to multiline code at this 
  point.

- What happens if the code contains other tags? I don't know, but I think
  we can reuse the Javascript trick and place <!-- --> inside the script
  tag to avoid any issues. Of course we would need to handle this case too.

- Security. Currently I call the exec keyword directly without worrying 
  about any intrusive code. Since the TTW module has the same problem I 
  will look there for an answer. However, for the DummyEngine the plain 
  exec is enough I think.



=== Zope3/src/zope/tal/dummyengine.py 1.11 => 1.11.20.1 ===
--- Zope3/src/zope/tal/dummyengine.py:1.11	Tue Jun  3 11:20:06 2003
+++ Zope3/src/zope/tal/dummyengine.py	Wed Aug 20 08:59:55 2003
@@ -1,6 +1,6 @@
 ##############################################################################
 #
-# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
+# Copyright (c) 2001, 2002, 2003 Zope Corporation and Contributors.
 # All Rights Reserved.
 #
 # This software is subject to the provisions of the Zope Public License,
@@ -12,14 +12,16 @@
 #
 ##############################################################################
 """Dummy TAL expression engine so that I can test out the TAL implementation.
-"""
 
+$Id$
+"""
 import re
 
 from zope.interface import implements
 from zope.tal.taldefs import NAME_RE, TALExpressionError, ErrorInfo
 from zope.tal.interfaces import ITALExpressionCompiler, ITALExpressionEngine
 from zope.i18n.interfaces import ITranslationService
+from zope.i18n.messageid import MessageID
 
 Default = object()
 
@@ -87,6 +89,7 @@
         else:
             type = "path"
             expr = expression
+
         if type in ("string", "str"):
             return expr
         if type in ("path", "var", "global", "local"):
@@ -127,6 +130,8 @@
 
     def evaluateText(self, expr):
         text = self.evaluate(expr)
+        if isinstance(text, (str, unicode, MessageID)):
+            return text
         if text is not None and text is not Default:
             text = str(text)
         return text
@@ -192,6 +197,17 @@
         return self.translationService.translate(
             msgid, domain, mapping, default=default)
 
+    def evaluateCode(self, lang, code):
+        assert lang == 'server-python'
+        import sys, StringIO
+        tmp = sys.stdout
+        sys.stdout = StringIO.StringIO()
+        try:
+            exec code in self.globals, self.locals
+        finally:
+            result = sys.stdout
+            sys.stdout = tmp
+        return result.getvalue()
 
 class Iterator:
 
@@ -227,6 +243,11 @@
         # If the domain is a string method, then transform the string
         # by calling that method.
 
+        # MessageID attributes override arguments
+        if isinstance(msgid, MessageID):
+            domain = msgid.domain
+            mapping = msgid.mapping
+            default = msgid.default
 
         # simulate an unknown msgid by returning None
         if msgid == "don't translate me":
@@ -237,6 +258,6 @@
             text = msgid.upper()
 
         def repl(m):
-            return mapping[m.group(m.lastindex).lower()]
+            return unicode(mapping[m.group(m.lastindex).lower()])
         cre = re.compile(r'\$(?:([_A-Za-z][-\w]*)|\{([_A-Za-z][-\w]*)\})')
         return cre.sub(repl, text)


=== Zope3/src/zope/tal/interfaces.py 1.9 => 1.9.16.1 ===
--- Zope3/src/zope/tal/interfaces.py:1.9	Tue Jul  1 13:31:03 2003
+++ Zope3/src/zope/tal/interfaces.py	Wed Aug 20 08:59:55 2003
@@ -149,9 +149,16 @@
         """
 
     def translate(msgid, domain=None, mapping=None, default=None):
+        """See ITranslationService.translate()"""
+
+    def evaluateCode(lang, code):
+        """Evaluates code of the given language.
+
+        Returns whatever the code outputs. This can be defined on a
+        per-language basis. In Python this usually everything the print
+        statement will return.
         """
-        See ITranslationService.translate()
-        """
+        
 
 class ITALIterator(Interface):
     """A TAL iterator
@@ -164,7 +171,6 @@
 
         Return a true value if it was possible to advance and return
         a false value otherwise.
-
         """
 
 


=== Zope3/src/zope/tal/taldefs.py 1.6 => 1.6.22.1 ===
--- Zope3/src/zope/tal/taldefs.py:1.6	Tue Jun  3 11:20:06 2003
+++ Zope3/src/zope/tal/taldefs.py	Wed Aug 20 08:59:55 2003
@@ -48,6 +48,7 @@
     "on-error",
     "omit-tag",
     "tal tag",
+    "script"
     ]
 
 KNOWN_I18N_ATTRIBUTES = [


=== Zope3/src/zope/tal/talgenerator.py 1.9 => 1.9.6.1 ===
--- Zope3/src/zope/tal/talgenerator.py:1.9	Tue Jul 29 10:15:22 2003
+++ Zope3/src/zope/tal/talgenerator.py	Wed Aug 20 08:59:55 2003
@@ -64,6 +64,7 @@
             self.source_file = source_file
             self.emit("setSourceFile", source_file)
         self.i18nContext = TranslationContext()
+        self.i18nLevel = 0
 
     def getCode(self):
         assert not self.stack
@@ -277,7 +278,7 @@
             else:
                 self.emit("setGlobal", name, cexpr)
 
-    def emitOnError(self, name, onError):
+    def emitOnError(self, name, onError, TALtag, isend):
         block = self.popProgram()
         key, expr = parseSubstitution(onError)
         cexpr = self.compileExpression(expr)
@@ -286,7 +287,10 @@
         else:
             assert key == "structure"
             self.emit("insertStructure", cexpr, {}, [])
-        self.emitEndTag(name)
+        if TALtag:
+            self.emitOptTag(name, (None, 1), isend)
+        else:
+            self.emitEndTag(name)
         handler = self.popProgram()
         self.emit("onError", block, handler)
 
@@ -315,7 +319,11 @@
             assert key == "structure"
             self.emit("insertStructure", cexpr, attrDict, program)
 
-    def emitI18nVariable(self, varname, action, expression):
+    def emitEvaluateCode(self, lang):
+        program = self.popProgram()
+        self.emit('evaluateCode', lang, program)
+
+    def emitI18nVariable(self, stuff):
         # Used for i18n:name attributes.  arg is extra information describing
         # how the contents of the variable should get filled in, and it will
         # either be a 1-tuple or a 2-tuple.  If arg[0] is None, then the
@@ -328,6 +336,7 @@
         # calculate the contents of the variable, e.g.
         # "I live in <span i18n:name="country"
         #                  tal:replace="here/countryOfOrigin" />"
+        varname, action, expression = stuff
         m = _name_rx.match(varname)
         if m is None or m.group() != varname:
             raise TALError("illegal i18n:name: %r" % varname, self.position)
@@ -347,8 +356,6 @@
             assert action == I18N_EXPRESSION
             key, expr = parseSubstitution(expression)
             cexpr = self.compileExpression(expr)
-        # XXX Would key be anything but 'text' or None?
-        assert key in ('text', None)
         self.emit('i18nVariable', varname, program, cexpr)
 
     def emitTranslation(self, msgid, i18ndata):
@@ -509,6 +516,7 @@
         repeat = taldict.get("repeat")
         content = taldict.get("content")
         replace = taldict.get("replace")
+        script = taldict.get("script")
         attrsubst = taldict.get("attributes")
         onError = taldict.get("on-error")
         omitTag = taldict.get("omit-tag")
@@ -521,6 +529,11 @@
         varname = i18ndict.get('name')
         i18ndata = i18ndict.get('data')
 
+        if varname and not self.i18nLevel:
+            raise I18NError(
+                "i18n:name can only occur inside a translation unit",
+                position)
+
         if i18ndata and not msgid:
             raise I18NError("i18n:data must be accompanied by i18n:translate",
                             position)
@@ -604,7 +617,11 @@
             todo["scope"] = 1
         if onError:
             self.pushProgram() # handler
+            if TALtag:
+                self.pushProgram() # start
             self.emitStartTag(name, list(attrlist)) # Must copy attrlist!
+            if TALtag:
+                self.pushProgram() # start
             self.pushProgram() # block
             todo["onError"] = onError
         if define:
@@ -619,22 +636,28 @@
             if repeatWhitespace:
                 self.emitText(repeatWhitespace)
         if content:
-            todo["content"] = content
-        if replace:
+            if varname:
+                todo['i18nvar'] = (varname, I18N_CONTENT, None)
+                todo["content"] = content
+                self.pushProgram()
+            else:
+                todo["content"] = content
+        elif replace:
             # tal:replace w/ i18n:name has slightly different semantics.  What
             # we're actually replacing then is the contents of the ${name}
             # placeholder.
             if varname:
-                todo['i18nvar'] = (varname, replace)
+                todo['i18nvar'] = (varname, I18N_EXPRESSION, replace)
             else:
                 todo["replace"] = replace
             self.pushProgram()
         # i18n:name w/o tal:replace uses the content as the interpolation
         # dictionary values
         elif varname:
-            todo['i18nvar'] = (varname, None)
+            todo['i18nvar'] = (varname, I18N_REPLACE, None)
             self.pushProgram()
         if msgid is not None:
+            self.i18nLevel += 1
             todo['msgid'] = msgid
         if i18ndata:
             todo['i18ndata'] = i18ndata
@@ -671,13 +694,19 @@
         if replace:
             todo["repldict"] = repldict
             repldict = {}
+        if script:
+            todo["script"] = script    
         self.emitStartTag(name, self.replaceAttrs(attrlist, repldict), isend)
         if optTag:
             self.pushProgram()
-        if content:
+        if content and not varname:
             self.pushProgram()
         if msgid is not None:
             self.pushProgram()
+        if content and varname:
+            self.pushProgram()
+        if script:
+            self.pushProgram()            
         if todo and position != (None, None):
             todo["position"] = position
         self.todoPush(todo)
@@ -700,6 +729,7 @@
         repeat = todo.get("repeat")
         content = todo.get("content")
         replace = todo.get("replace")
+        script = todo.get("script")
         condition = todo.get("condition")
         onError = todo.get("onError")
         repldict = todo.get("repldict", {})
@@ -720,12 +750,11 @@
             raise exc("%s attributes on <%s> require explicit </%s>" %
                       (what, name, name), position)
 
+        if script:
+            self.emitEvaluateCode(script)
         # If there's no tal:content or tal:replace in the tag with the
         # i18n:name, tal:replace is the default.
-        i18nNameAction = I18N_REPLACE
         if content:
-            if varname:
-                i18nNameAction = I18N_CONTENT
             self.emitSubstitution(content, {})
         # If we're looking at an implicit msgid, emit the insertTranslation
         # opcode now, so that the end tag doesn't become part of the implicit
@@ -733,8 +762,14 @@
         # the opcode after the i18nVariable opcode so we can better handle
         # tags with both of them in them (and in the latter case, the contents
         # would be thrown away for msgid purposes).
-        if msgid is not None and not varname:
-            self.emitTranslation(msgid, i18ndata)
+        #
+        # Still, we should emit insertTranslation opcode before i18nVariable
+        # in case tal:content, i18n:translate and i18n:name in the same tag
+        if msgid is not None:
+            if (not varname) or (
+                varname and (varname[1] == I18N_CONTENT)):
+                self.emitTranslation(msgid, i18ndata)
+            self.i18nLevel -= 1
         if optTag:
             self.emitOptTag(name, optTag, isend)
         elif not isend:
@@ -751,26 +786,30 @@
         if replace:
             self.emitSubstitution(replace, repldict)
         elif varname:
-            if varname[1] is not None:
-                i18nNameAction = I18N_EXPRESSION
             # o varname[0] is the variable name
-            # o i18nNameAction is either
+            # o varname[1] is either
             #   - I18N_REPLACE for implicit tal:replace
             #   - I18N_CONTENT for tal:content
             #   - I18N_EXPRESSION for explicit tal:replace
-            # o varname[1] will be None for the first two actions and the
+            # o varname[2] will be None for the first two actions and the
             #   replacement tal expression for the third action.
-            self.emitI18nVariable(varname[0], i18nNameAction, varname[1])
+            assert (varname[1]
+                    in [I18N_REPLACE, I18N_CONTENT, I18N_EXPRESSION])
+            self.emitI18nVariable(varname)
         # Do not test for "msgid is not None", i.e. we only want to test for
         # explicit msgids here.  See comment above.
-        if msgid is not None and varname:
-            self.emitTranslation(msgid, i18ndata)
+        if msgid is not None: 
+            # in case tal:content, i18n:translate and i18n:name in the
+            # same tag insertTranslation opcode has already been
+            # emitted
+            if varname and (varname[1] <> I18N_CONTENT):
+                self.emitTranslation(msgid, i18ndata)
         if repeat:
             self.emitRepeat(repeat)
         if condition:
             self.emitCondition(condition)
         if onError:
-            self.emitOnError(name, onError)
+            self.emitOnError(name, onError, optTag and optTag[1], isend)
         if scope:
             self.emit("endScope")
         if i18ncontext:


=== Zope3/src/zope/tal/talgettext.py 1.14 => 1.14.22.1 ===
--- Zope3/src/zope/tal/talgettext.py:1.14	Tue Jun  3 11:20:06 2003
+++ Zope3/src/zope/tal/talgettext.py	Wed Aug 20 08:59:55 2003
@@ -36,10 +36,11 @@
 
 from zope.interface import implements
 from zope.tal.htmltalparser import HTMLTALParser
-from zope.tal.talinterpreter import TALInterpreter
+from zope.tal.talinterpreter import TALInterpreter, normalize
 from zope.tal.dummyengine import DummyEngine
 from zope.tal.interfaces import ITALExpressionEngine
 from zope.tal.taldefs import TALExpressionError
+from zope.i18n.messageid import MessageID
 
 __version__ = '$Revision$'
 
@@ -110,11 +111,20 @@
                   # XXX position is not part of the ITALExpressionEngine
                   #     interface
                   position=None):
-        # assume domain and mapping are ignored; if they are not,
-        # unit test must be updated.
-        if msgid not in self.catalog:
-            self.catalog[msgid] = []
-        self.catalog[msgid].append((self.file, position))
+
+        # Make the message is a MessageID object, if the default differs
+        # from the value, so that the POT generator can put the default
+        # text into a comment.
+        if default is not None and normalize(default) != msgid:
+            msgid = MessageID(msgid, default=default)
+        
+        if domain not in self.catalog:
+            self.catalog[domain] = {}
+        domain = self.catalog[domain]
+
+        if msgid not in domain:
+            domain[msgid] = []
+        domain[msgid].append((self.file, position))
         return 'x'
 
 
@@ -273,7 +283,11 @@
     else:
         outfile = file(outfile, update_mode and "a" or "w")
 
-    messages = engine.catalog.copy()
+    catalog = {}
+    for domain in engine.catalog.keys():
+        catalog.update(engine.catalog[domain])
+
+    messages = catalog.copy()
     try:
         messages.update(engine.base)
     except AttributeError:
@@ -282,7 +296,8 @@
         print >> outfile, pot_header % {'time': time.ctime(),
                                         'version': __version__}
 
-    msgids = engine.catalog.keys()
+    msgids = catalog.keys()
+    # XXX: You should not sort by msgid, but by filename and position. (SR)
     msgids.sort()
     for msgid in msgids:
         positions = engine.catalog[msgid]


=== Zope3/src/zope/tal/talinterpreter.py 1.22 => 1.22.6.1 ===
--- Zope3/src/zope/tal/talinterpreter.py:1.22	Tue Jul 15 02:06:56 2003
+++ Zope3/src/zope/tal/talinterpreter.py	Wed Aug 20 08:59:55 2003
@@ -14,7 +14,6 @@
 """
 Interpreter for a pre-compiled TAL program.
 """
-
 import sys
 
 # Do not use cStringIO here!  It's not unicode aware. :(
@@ -68,7 +67,7 @@
 
     def emit(self, *args):
         if self.enabled:
-            apply(TALGenerator.emit, (self,) + args)
+            TALGenerator.emit(self, *args)
 
     def emitStartElement(self, name, attrlist, taldict, metaldict, i18ndict,
                          position=(None, None), isend=0):
@@ -155,18 +154,7 @@
         self.macroStack.append([macroName, slots, entering, self.i18nContext])
 
     def popMacro(self):
-        stuff = self.macroStack.pop()
-        self.i18nContext = stuff[3]
-        return stuff
-
-    def macroContext(self, what):
-        macroStack = self.macroStack
-        i = len(macroStack)
-        while i > 0:
-            i = i-1
-            if macroStack[i][0] == what:
-                return i
-        return -1
+        return self.macroStack.pop()
 
     def __call__(self):
         assert self.level == 0
@@ -337,16 +325,16 @@
                 value = None
             else:
                 ok = 0
-        else:
-            if expr is not None:
-                evalue = self.engine.evaluateText(item[3])
-                if evalue is self.Default:
-                    if action == 'insert': # Cancelled insert
-                        ok = 0
-                else:
-                    if evalue is None:
-                        ok = 0
-                    value = evalue
+        elif expr is not None:
+            evalue = self.engine.evaluateText(item[3])
+            if evalue is self.Default:
+                if action == 'insert': # Cancelled insert
+                    ok = 0
+            else:
+                if evalue is None:
+                    ok = 0
+                value = evalue
+                
         if ok:
             if xlat:
                 translated = self.translate(msgid or value, value, {})
@@ -356,7 +344,6 @@
                 value = name
             value = "%s=%s" % (name, quote(value))
         return ok, name, value
-
     bytecode_handlers["<attrAction>"] = attrAction
 
     def no_tag(self, start, program):
@@ -388,14 +375,6 @@
             self.do_optTag(stuff)
     bytecode_handlers["optTag"] = do_optTag
 
-    def dumpMacroStack(self, prefix, suffix, value):
-        sys.stderr.write("+---- %s%s = %s\n" % (prefix, suffix, value))
-        for i in range(len(self.macroStack)):
-            what, macroName, slots = self.macroStack[i][:3]
-            sys.stderr.write("| %2d. %-12s %-12s %s\n" %
-                             (i, what, macroName, slots and slots.keys()))
-        sys.stderr.write("+--------------------------------------\n")
-
     def do_rawtextBeginScope(self, (s, col, position, closeprev, dict)):
         self._stream_write(s)
         self.col = col
@@ -507,6 +486,12 @@
             # Evaluate the value to be associated with the variable in the
             # i18n interpolation dictionary.
             value = self.engine.evaluate(expression)
+
+            # evaluate() does not do any I18n, so we do it here. 
+            if isinstance(value, MessageID):
+                # Translate this now.
+                value = self.engine.translate(value)
+
         # Either the i18n:name tag is nested inside an i18n:translate in which
         # case the last item on the stack has the i18n dictionary and string
         # representation, or the i18n:name and i18n:translate attributes are
@@ -600,6 +585,21 @@
         p.parseFragment('</foo>', 1)
         program, macros = gen.getCode()
         self.interpret(program)
+
+    def do_evaluateCode(self, stuff):
+        lang, program = stuff
+        # Use a temporary stream to capture the interpretation of the
+        # subnodes, which should /not/ go to the output stream.
+        tmpstream = StringIO()
+        self.pushStream(tmpstream)
+        try:
+            self.interpret(program)
+        finally:
+            self.popStream()
+        code = tmpstream.getvalue()
+        output = self.engine.evaluateCode(lang, code)
+        self._stream_write(output)        
+    bytecode_handlers["evaluateCode"] = do_evaluateCode
 
     def do_loop(self, (name, expr, block)):
         self.interpret(block)

=== Removed File Zope3/src/zope/tal/history.txt ===

=== Removed File Zope3/src/zope/tal/readme.txt ===




More information about the Zope3-Checkins mailing list