[Zope-CMF] Re: Decode for Record objects

[Hanno Schlichting]

Charlie Clark wrote:
> If I have non-ASCII data in an RDBMS that I wish to use in a CMF site or 
> even straight ZPT's you get a UnicodeError when accessing the non-ASCII 
> Values which you get around using the decode utility.

ZPT uses Unicode internally in Zope 2.10 and newer, so you need to 
decode your non-unicode values before passing them into a Page Template.

Plone hacks the TAL engine in horrible ways so you can still use utf-8 
encoded strings as well, but I wouldn't recommend this approach for a 
small application.

> However, I think the error is raised because utils.decode simply returns 
> any values untouched it doesn't know what to do with which is why I 
> think it might be worth extending. OTOH it might something that is 
> better fixed in the Shared.RDBMS.Results module

The database connection layer is the I/O boundary in this case and the 
decoding to Unicode should happen here. Pretty much all code in Zope2's 
SQL or database layers doesn't do this and I haven't seen much interest 
from anyone to fix this.

I'd highly recommend using one of the approaches based on sqlalchemy, as 
otherwise you'll probably end up fixing a number of bugs and adding 
missing features to the various products responsible for Zope2's current 
DB interconnection features.

Hanno