[Zope-CMF] Charsets

[Charlie Clark]

Am 18.01.2009 um 23:00 schrieb yuppie:

> Hi Charlie!

Hiya Yuppie,

> Charlie Clark wrote:
>> Am 29.12.2008 um 15:01 schrieb Charlie Clark:
>>
>> CMFDefault.utils
>>
>> def getBrowserCharset(request):
>>     """ Get charset preferred by the browser.
>>     """
>>     envadapter = IUserPreferredCharsets(request)
>>     charsets = envadapter.getPreferredCharsets() or ['utf-8']
>>     return charsets[0]
>>
>> This will always be iso-8859-1 for Opera and Firefox because all
>> charsets have the same quality, again even if UTF-8 encoding is
>> specified.
>
> getBrowserCharset does almost the same as
> zope.publisher.http.getCharsetUsingRequest. And it is only used for
> encoding and decoding 'portal_status_message'. It is not relevant for
> the issue you noticed.

Okay.

>> I haven't been able to track where the decoding of form
>> data occurs for Zope 2 stuff but I can identify the problem in
>> zpublisher.browser.BrowserRequest
>
> You mean zope.publisher.browser.BrowserRequest. The Zope 2 version  
> is in
> Products.Five.browser.decode.

Thanks - I thought it must have been in Five but didn't know where to  
look.

> AFAICS the fallback to other charsets is usually not required in  
> Zope 3.
> If the publisher encodes responses using
> zope.publisher.http.getCharsetUsingRequest, the first charset will be
> the right one.

That seems reasonable.

>> I would suggest that we work towards enforcing UTF-8 in where  
>> possible
>> but at the very least add the accept-charset attribute to forms and
>> use the portal's default_charset for this.
>>
>> I'd very much appreciate your comments on this.
>
> I can't see a need to implement this in a different way than Zope 3.  
> So
> I propose to fix the encoding of forms sent to the browser.


I agree that there shouldn't be implemented in a different way than  
for Zope 3. And if we can solve the problems by fixing form encoding  
I'm happy. Although I'd like to see UTF-8 always the first charset  
returned if * the quality is the same.

One thing that did strike me when working on this is quite how often  
getPreferredCharsets() is called on single request.

Charlie
--
Charlie Clark
Helmholtzstr. 20
Düsseldorf
D- 40215
Tel: +49-211-938-5360
GSM: +49-178-782-6226