<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hello list.<br>
<br>
I'm having an issue with ZTUtils.Zope.make_query when providing
Unicode values.<br>
<br>
Specifically, We're building search results pagination using
CMFPlone.PloneBatch<br>
<br>
Here's the scenario:<br>
<br>
The search form is provided a Unicode string. The query is properly
written as "?search_text:utf8:ustring=value"<br>
The search will return many results.<br>
<br>
Our searchResults browser view will provide a PloneBatch object,
making it very easy to display only, for example, 30 results, then
provide page links at the bottom of the page for the next 30
results, or whatever.<br>
<br>
These links will contain the original query string, but when they
are built using make_query(), the result is "?search_text=value" <br>
- What follows is Unicode decode errors when the 'next' link is
followed - the search_text is marshaled as a 'str', not as a
'unicode'<br>
<br>
The bottom line is that make_query (actually, complex_marshal &
simple_marshal) doesn't seem to apply the ':ustring' type prefix
when creating query strings.<br>
<br>
Kind of important to us at Radio Free Asia, considering we host
Cantonese and Korean languages, for example.<br>
<br>
This was fixed by using sys.setdefaultencoding('utf-8'), but that
seems like quite a sledgehammer fix.<br>
<br>
----------<br>
<br>
I have a proposed fix, branched from the master branch on Github.<br>
Diff here:
<meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1">
<a href="https://github.com/zopefoundation/Zope/pull/4/files">https://github.com/zopefoundation/Zope/pull/4/files</a>
<br>
<br>
Questions:<br>
1. Is "Master" the correct place to fork from? <br>
<br>
2. Is is safe - at least to start - to assume utf8 encoding when
constructing query strings? <br>
<br>
3. Dangerous idea: If simpleMarshal is passed a unicode object
that can be represented as ASCII, should we "magically" turn it into
a string? I don't like magic, but I also know it's common (and
good) practice to use unicode in ASCII-based languages, making this
exercise an overkill for those languages.<br>
<br>
Any comments or criticisms (not to hard please!) would be
appreciated!<br>
<br>
-Flip<br>
<br>
<pre class="moz-signature" cols="72">--
Mike McFadden
Radio Free Asia
Technical Operations Division
2025 M Street NW
Washington DC 20036 USA
This e-mail message is intended only for the use of the addressee and may contain information that is privileged and confidential. Any unauthorized dissemination, distribution or copying is strictly prohibited. If you receive this transmission in error, please contact <a class="moz-txt-link-abbreviated" href="mailto:network@rfa.org">network@rfa.org</a>.
</pre>
</body>
</html>