[Checkins] Re: [Zope-Checkins] SVN: Zope/trunk/ support for a configurable resolver for UnicodeDecodeErrors

[Chris Withers]

I must have missed it, but where are your tests for this and the other 
unicode changes?

Andreas Jung wrote:
> Log message for revision 71753:
>   support for a configurable resolver for UnicodeDecodeErrors
>   
> 
> Changed:
>   U   Zope/trunk/doc/CHANGES.txt
>   U   Zope/trunk/lib/python/Products/PageTemplates/Expressions.py
>   A   Zope/trunk/lib/python/Products/PageTemplates/configure.zcml
>   A   Zope/trunk/lib/python/Products/PageTemplates/interfaces.py
>   A   Zope/trunk/lib/python/Products/PageTemplates/unicodeconflictresolver.py
> 
> -=-
> Modified: Zope/trunk/doc/CHANGES.txt
> ===================================================================
> --- Zope/trunk/doc/CHANGES.txt	2007-01-07 10:25:20 UTC (rev 71752)
> +++ Zope/trunk/doc/CHANGES.txt	2007-01-07 10:32:39 UTC (rev 71753)
> @@ -48,6 +48,10 @@
>          'output_encodings' property that controls the conversion from/to unicode
>          for WebDAV/FTP operations.
>  
> +      - the ZPT implementation has now a configurable option in order how to deal 
> +        with UnicodeDecodeErrors. A custom UnicodeEncodingConflictResolver can
> +        be configured through ZCML (see Products/PageTemplates/(configure.zcml,
> +        unicodeconflictresolver.py, interfaces.py)
>  
>      Bugs Fixed
>  
> 
> Modified: Zope/trunk/lib/python/Products/PageTemplates/Expressions.py
> ===================================================================
> --- Zope/trunk/lib/python/Products/PageTemplates/Expressions.py	2007-01-07 10:25:20 UTC (rev 71752)
> +++ Zope/trunk/lib/python/Products/PageTemplates/Expressions.py	2007-01-07 10:32:39 UTC (rev 71753)
> @@ -17,6 +17,10 @@
>  
>  $Id$
>  """
> +
> +import logging
> +
> +from zope.component import getUtility
>  from zope.interface import implements
>  from zope.tales.tales import Context, Iterator
>  from zope.tales.expressions import PathExpr, StringExpr, NotExpr
> @@ -31,13 +35,17 @@
>  from MultiMapping import MultiMapping
>  from Acquisition import aq_base
>  from zExceptions import NotFound, Unauthorized
> +
>  from Products.Five.browser.providerexpression import Z2ProviderExpression
>  from Products.PageTemplates import ZRPythonExpr
>  from Products.PageTemplates.DeferExpr import LazyExpr
>  from Products.PageTemplates.GlobalTranslationService import getGlobalTranslationService
> +from Products.PageTemplates.interfaces import IUnicodeEncodingConflictResolver
>  
>  SecureModuleImporter = ZRPythonExpr._SecureModuleImporter()
>  
> +LOG = logging.getLogger('Expressions')
> +
>  # BBB 2005/05/01 -- remove after 12 months
>  import zope.deprecation
>  from zope.deprecation import deprecate
> @@ -173,6 +181,44 @@
>              domain, msgid, mapping=mapping,
>              context=context, default=default)
>  
> +
> +    def evaluateText(self, expr):
> +        """ customized version in order to get rid of unicode
> +            errors for all and ever
> +        """
> +
> +        text = self.evaluate(expr)
> +
> +        if text is self.getDefault() or text is None:
> +            # XXX: should be unicode???
> +            return text
> +
> +        if isinstance(text, unicode):
> +            # we love unicode, nothing to do
> +            return text
> +
> +        elif isinstance(text, str):
> +            # bahh...non-unicode string..we need to convert it to unicode
> +            resolver = getUtility(IUnicodeEncodingConflictResolver)
> +
> +            try:
> +                return resolver.resolve(self.contexts['context'], text, expr)
> +            except UnicodeDecodeError,e:
> +                LOG.error("""UnicodeDecodeError detected for expression "%s"\n"""
> +                          """Resolver class: %s\n"""
> +                          """Exception text: %s\n"""
> +                          """Template: %s\n"""
> +                          """Rendered text: %r"""  % \
> +                          (expr, resolver.__class__, e, 
> +                            self.contexts['template'].absolute_url(1), text))
> +                raise 
> +        else:
> +
> +            # This is a weird culprit ...calling unicode() on non-string
> +            # objects
> +            return unicode(text)
> +
> +
>  class ZopeEngine(zope.app.pagetemplate.engine.ZopeEngine):
>  
>      _create_context = ZopeContext
> 
> Copied: Zope/trunk/lib/python/Products/PageTemplates/configure.zcml (from rev 71752, Zope/branches/ajung-death-to-unicode-errors/lib/python/Products/PageTemplates/configure.zcml)
> 
> Copied: Zope/trunk/lib/python/Products/PageTemplates/interfaces.py (from rev 71752, Zope/branches/ajung-death-to-unicode-errors/lib/python/Products/PageTemplates/interfaces.py)
> 
> Copied: Zope/trunk/lib/python/Products/PageTemplates/unicodeconflictresolver.py (from rev 71752, Zope/branches/ajung-death-to-unicode-errors/lib/python/Products/PageTemplates/unicodeconflictresolver.py)
> 
> _______________________________________________
> Zope-Checkins maillist  -  Zope-Checkins at zope.org
> http://mail.zope.org/mailman/listinfo/zope-checkins
> 

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk