[Zope-dev] Non-Latin-1 properties

[Yves Bastide]

Hi!

Sorry to beat a dead horse: the ZMI doesn't allow using non-Latin-1 
properties.

* properties are stored using the default-zpublisher-encoding (iso-8859-15);
* manage_propertiesForm is input--output'ed using 
management_page_charset (UTF-8) ...
* ... but built using join_unicode, which calls PyUnicode_DecodeLatin1.

To see the problem:

* with the default default-zpublisher-encoding (Latin-9),
   - set a property's value to '€' (not in Latin-1)
   - Save Changes: it will be displayed as '¤' (the corresponding character)
   - Save Changes again: 'charmap' codec can't encode character u'\xa4' 
in position 0: character maps to <undefined> (because '¤' is not in 
(Latin-9).

* with default-zpublisher-encoding set to UTF-8
   - set a property's value to '€'
   - Save Changes: it will be displayed as '€'
   - Save Changes again: it becomes '€'


Note that this has repercussions not only on the ZMI but on other 
products, such as GenericSetup.


Some possible fixes:

* do nothing. This will please those wishing to deprecate the ZMI, but 
not the rest of us :-)

* rewrite properties.dtml as a ZPT?

* change join_unicode to use either default-zpublisher-encoding or yet 
another setting (which would default to Latin-1 to stay compatible)?


Flames, comments?

Thanks!

Yves