1 2 Previous Next 16 Replies Latest reply on Aug 12, 2016 9:44 AM by mayerw01

Undertow encoding problem

nickarls Aug 3, 2016 3:36 AM

We're testing Oracle APEX on WF10 (Windows) and noticed that some special chars (åäö) now show up as upside-down question marks (classical encoding problem).

I've tried setting the undertow servlet default-encoding to UTF-8 and even tried passing in -Dfile.encoding=UTF-8 but I still see the problem. The HTTP response header says UTF-8. Pointers?

Thanks in advance,

Nik

1. Re: Undertow encoding problem

ctomc Aug 3, 2016 8:35 AM (in response to nickarls)

Is the "static" (jsp/servlet/...) code that wrongly displays chars or is it dynamic content from database?
if such you could look at datasource driver configuration.
Actions
2. Re: Undertow encoding problem

nickarls Aug 3, 2016 9:01 AM (in response to ctomc)

Well, with APEX you never really know, everything is sort of DB ;-)

But I've already tried adding the ora18n.jar to the driver module (which usually helps), but in this case it didn't...
Actions
3. Re: Undertow encoding problem

nickarls Aug 9, 2016 2:21 AM (in response to nickarls)

It's a bit strange. The page contains special chars, works fine. If I submit the form with special chars and use the Chrome console, I see the form data f02: åäö with content type application/x-www-f-u.
But when the page is updated, the search text is garbled. Somehow it looks like the encoding is messed up when/after the data the data is submitted.

Pointers?
Actions
4. Re: Undertow encoding problem

ctomc Aug 9, 2016 8:42 AM (in response to nickarls)

reading from database would be my go-to for next research.
maybe some special settings for jdbc driver, some extra parameter in the url. or maybe even different jdbc driver all together.
Actions
5. Re: Undertow encoding problem

nickarls Aug 10, 2016 1:36 AM (in response to ctomc)

I don't think it's that. It's working against the same database as before and I used the same driver module as in the old EAP6.1. Besides, I'm almost sure the search criteria as never stored in the DB and whatever roundtrip to the DB the rendering takes, it works for the non-submitted stuff (labels etc)...
Actions
6. Re: Undertow encoding problem

jaikiran Aug 10, 2016 2:03 AM (in response to nickarls)

What exactly does this page(?) involve? JSP? Can you post a sample field on that page so that we know what kind of a form element it is and how it's created? Also, where is the data sourced from, for that page (I don't mean the DB part but any web layer frameworks etc).
Actions

7. Re: Undertow encoding problem

nickarls Aug 10, 2016 2:28 AM (in response to jaikiran)

There is an apex.war deployed that connects through a connection pool to the DB and does the interaction. The view, fields etc. are all defined in fields in the DB. This is a Chrome HAR of the exchange. The "f02" is the submitted data and the content/text is the truncated response for the offending label.

{
  "log": {
    "version": "1.2",
    "creator": {
      "name": "WebInspector",
      "version": "537.36"
    },
    "pages": [],
    "entries": [
      {
        "startedDateTime": "2016-08-10T06:14:14.279Z",
        "time": 192.99899972975254,
        "request": {
          "method": "POST",
          "url": "http://mare12:8008/apex/wwv_flow.show",
          "httpVersion": "HTTP/1.1",
          "headers": [
            {
              "name": "Origin",
              "value": "http://mare12:8008"
            },
            {
              "name": "Accept-Encoding",
              "value": "gzip, deflate"
            },
            {
              "name": "Host",
              "value": "mare12:8008"
            },
            {
              "name": "Accept-Language",
              "value": "en-US,en;q=0.8,fi-FI;q=0.6,fi;q=0.4"
            },
            {
              "name": "User-Agent",
              "value": "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2783.4 Safari/537.36"
            },
            {
              "name": "Content-Type",
              "value": "application/x-www-form-urlencoded"
            },
            {
              "name": "Accept",
              "value": "*/*"
            },
            {
              "name": "Referer",
              "value": "http://mare12:8008/apex/f?p=100:8:4251609565487::::VA:4"
            },
            {
              "name": "Cookie",
              "value": "PVEKAAPPI_SISNO=128; SAFIIRI_USR_8008=jas_v2; SAFIIRI_SISNO_8008=1; SAFIIRI_DOM_8008=KAKE; ORA_WWV_APP_100=ORA_WWV-hONgJCXzx+g7OhkXtH5odXDH"
            },
            {
              "name": "Connection",
              "value": "keep-alive"
            },
            {
              "name": "Content-Length",
              "value": "330"
            }
          ],
          "queryString": [],
          "cookies": [
            {
              "name": "PVEKAAPPI_SISNO",
              "value": "128",
              "expires": null,
              "httpOnly": false,
              "secure": false
            },
            {
              "name": "SAFIIRI_USR_8008",
              "value": "jas_v2",
              "expires": null,
              "httpOnly": false,
              "secure": false
            },
            {
              "name": "SAFIIRI_SISNO_8008",
              "value": "1",
              "expires": null,
              "httpOnly": false,
              "secure": false
            },
            {
              "name": "SAFIIRI_DOM_8008",
              "value": "KAKE",
              "expires": null,
              "httpOnly": false,
              "secure": false
            },
            {
              "name": "ORA_WWV_APP_100",
              "value": "ORA_WWV-hONgJCXzx+g7OhkXtH5odXDH",
              "expires": null,
              "httpOnly": false,
              "secure": false
            }
          ],
          "headersSize": 618,
          "bodySize": 327,
          "postData": {
            "mimeType": "application/x-www-form-urlencoded",
            "text": "p_request=APXWGT&p_instance=4251609565487&p_flow_id=100&p_flow_step_id=8&p_widget_num_return=15&f01=apexir_CURRENT_SEARCH_COLUMN&f01=apexir_SEARCH&f01=apexir_NUM_ROWS&f02=&f02=åäö&f02=15&p_widget_name=worksheet&p_widget_mod=ACTION&p_widget_action=QUICK_FILTER&p_widget_action_mod=ADD&x01=50377881313248334&x02=50381983327248344",
            "params": [
              {
                "name": "p_request",
                "value": "APXWGT"
              },
              {
                "name": "p_instance",
                "value": "4251609565487"
              },
              {
                "name": "p_flow_id",
                "value": "100"
              },
              {
                "name": "p_flow_step_id",
                "value": "8"
              },
              {
                "name": "p_widget_num_return",
                "value": "15"
              },
              {
                "name": "f01",
                "value": "apexir_CURRENT_SEARCH_COLUMN"
              },
              {
                "name": "f01",
                "value": "apexir_SEARCH"
              },
              {
                "name": "f01",
                "value": "apexir_NUM_ROWS"
              },
              {
                "name": "f02",
                "value": ""
              },
              {
                "name": "f02",
                "value": "åäö"
              },
              {
                "name": "f02",
                "value": "15"
              },
              {
                "name": "p_widget_name",
                "value": "worksheet"
              },
              {
                "name": "p_widget_mod",
                "value": "ACTION"
              },
              {
                "name": "p_widget_action",
                "value": "QUICK_FILTER"
              },
              {
                "name": "p_widget_action_mod",
                "value": "ADD"
              },
              {
                "name": "x01",
                "value": "50377881313248334"
              },
              {
                "name": "x02",
                "value": "50381983327248344"
              }
            ]
          }
        },
        "response": {
          "status": 200,
          "statusText": "OK",
          "httpVersion": "HTTP/1.1",
          "headers": [
            {
              "name": "X-DB-Content-length",
              "value": "10506"
            },
            {
              "name": "Date",
              "value": "Wed, 10 Aug 2016 06:14:14 GMT"
            },
            {
              "name": "Server",
              "value": "WildFly/10"
            },
            {
              "name": "Connection",
              "value": "keep-alive"
            },
            {
              "name": "X-Powered-By",
              "value": "Undertow/1"
            },
            {
              "name": "Content-Length",
              "value": "10526"
            },
            {
              "name": "Content-Type",
              "value": "text/html;charset=UTF-8"
            }
          ],
          "cookies": [],
          "content": {
            "size": 10526,
            "mimeType": "text/html",
            "compression": 0,
            "text": "<td class=\"fielddata\">Rivin teksti sis&#xE4;lt&#xE4;&#xE4; &#x27;&#xBF;&#xBF;&#xBF;&#xBF;&#xBF;&#xBF;&#x27;</td>"
          },
          "redirectURL": "",
          "headersSize": 216,
          "bodySize": 10526,
          "_transferSize": 10742
        },
        "cache": {},
        "timings": {
          "blocked": 2.03599967062473,
          "dns": 0.6890003569424201,
          "connect": 1.3849996030330596,
          "send": 0.18600001931191024,
          "wait": 188.0570002831519,
          "receive": 0.6459997966885282,
          "ssl": -1
        },
        "serverIPAddress": "192.130.222.243",
        "connection": "1322310"
      }
    ]
  }
}

The submitting field is a

<input title="Hae raportin näkyvillä olevista sarakkeista
" type="text" size="30" maxlength="4000" value="" id="apexir_SEARCH" onkeydown="if($f_Enter(event)){gReport.search('SEARCH'); return false;}">

in a form

<form action="wwv_flow.accept" method="post" name="wwv_flow" id="wwvFlowForm" novalidate="">

8. Re: Undertow encoding problem

mayerw01 Aug 10, 2016 4:53 AM (in response to nickarls)

So you deployed the APEX Listener to use WildFly as HTTP server?
I understand the APEX listener is quite old and had been replaced by Oracle REST Data Services (ORDS) in between.
Therefore your problem may be related to APEX. Did you verify if your problem persists when you use ORDS instead of APEX:
Actions
9. Re: Undertow encoding problem

nickarls Aug 10, 2016 5:24 AM (in response to mayerw01)

We're on the 4.x-series but I guess we're going to upgrade to 5.x soonish so I guess we could try.
I must say that I don't know about the listener changes but 4.2.6 is under two years old so one would think that there wouldn't be a problem with basic HTTP/Servlet operations...
Actions
10. Re: Undertow encoding problem

mayerw01 Aug 10, 2016 9:44 AM (in response to nickarls)

I think you refer to the Oracle Application Express version which is currently on 5.0.4.
The listener (APEX or ORDS) is not dependent on the OE version (actually Oracle Application Express is not a prerequisite for using Oracle REST Data Services).
The latest APEX listener I am aware of is 2.0.5 (which is about 2 years old)
Oracle REST Data Services which is currently on version 3.0.6 was released in July 2016.
Actions
11. Re: Undertow encoding problem

nickarls Aug 11, 2016 3:52 AM (in response to mayerw01)

I must confess I don't have much experience in the version history etc. The case just landed on my desk because "it worked before, now it doesn't" ;-)

Can anyone point me to the location where I should put my breakpoint if I want to debug Undertow at the point where the decoding of the parameters has occurred?
Actions
12. Re: Undertow encoding problem

jaikiran Aug 11, 2016 10:15 AM (in response to nickarls)

From what I understand, it's the response whose content uses a wrong encoding. If that's the case perhaps io.undertow.servlet.spec.HttpServletResponseImpl might be good place to start adding breakpoints.
Actions
13. Re: Undertow encoding problem

nickarls Aug 11, 2016 2:44 PM (in response to jaikiran)

I think the jury is still out on that one, I guess if the stuff is still decoded correctly at the point where it is finally passed off to the application is has to be in the encoding phase...
Actions
14. Re: Undertow encoding problem

mayerw01 Aug 12, 2016 6:40 AM (in response to nickarls)

I still don't think that this is related to WildFly. In my environment the special characters are shown properly:
- Application Express: build 5.0.4.00.12
- APEX_LISTENER_VERSION   3.0.6.176.08.46
- PLSQL_GATEWAY   WebDb
- wildfly-10.0.0.Final

{
"log": {
    "version": "1.1",
    "creator": {
      "name": "Firefox",
      "version": "48.0"
    },
    "browser": {
      "name": "Firefox",
      "version": "48.0"
    },

....
"entries": [
      {
        "pageref": "page_4",
        "startedDateTime": "2016-08-12T11:04:10.090+02:00",
        "time": 181,
        "request": {
          "bodySize": 900,
          "method": "POST",
          "url": "http://localhost:8080/ords/wwv_flow.accept",
          "httpVersion": "HTTP/1.1",
          "headers": [
            {
              "name": "Host",
              "value": "localhost:8080"
            },
            {
              "name": "User-Agent",
              "value": "Mozilla/5.0 (X11; Linux x86_64; rv:48.0) Gecko/20100101 Firefox/48.0"
            },
            {
              "name": "Accept",
              "value": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
            },

....
"queryString": [],
          "postData": {
            "mimeType": "application/x-www-form-urlencoded",
            "params": [
...
{
                "name": "p_t02",
                "value": "Jürgen"
              },
              {
                "name": "p_arg_names",
                "value": "7470154439861862127"
              },
              {
                "name": "p_t03",
                "value": "Böße-Bären"
              },

....
},
        "response": {
          "status": 200,
          "statusText": "OK",
          "httpVersion": "HTTP/1.1",
          "headers": [
            {
              "name": "Connection",
              "value": "keep-alive"
            },
            {
              "name": "X-Powered-By",
              "value": "Undertow/1"
            },
            {
              "name": "Server",
              "value": "WildFly/10"
            },
....
"content": {
            "mimeType": "text/html; charset=UTF-8",
            "size": 22709,
            "text": "<!--[if HTML5
....
\">Tags</a></th></tr>\n<tr ><td class=\" u-tL\" headers=\"CUSTOMER_NAME\"><a href=\"f?p=110:7:5717697975157::NO::P7_CUSTOMER_ID,P7_BRANCH:100,2&cs=15E03E4CD071461359F6F0475F3496D58\" >Böße-Bären, Jürgen</a></td><td class=\" u-tL\" headers=\"CUSTOMER_ADDRESS\"></td><td class=\" u-tL\" headers=\"CUST_CITY\"></td><td class=\" u-tL\" headers=\"CUST_STATE\">AK</td><td class=\" u-tL\" headers=\"CUST_POSTAL_CODE\">7455</td><td class=\" u-tL\" headers=\"TAGS\"></td></tr></table></div><div class=\"a-IRR-paginationWrap a-IRR-paginationWrap--bottom\"><ul class

But there is something else strange:
Your special characters appear as "name": "f02". Did you enter these characters in the search field like
In my environment these characters are encoded as hexadecimal values in this case
"queryString": [],
          "postData": {
            "mimeType": "application/x-www-form-urlencoded",
            "params": [],
            "text": "p_request=PLUGIN%3D29A7AFE9DE9D66DE991787CDEE424EAC58BBDE5795FEE6EA6AF299C0FF2E254A&p_flow_id=110&p_flow_step_id=2&p_instance=2998691333482&p_debug=&p_widget_name=worksheet&p_widget_mod=ACTION&p_widget_action=QUICK_FILTER&p_widget_num_return=15&x01=7325972842842603813&x02=7179925510617646276&f01=R7325972741959603813_column_search_current_column&f01=R7325972741959603813_search_field&f01=R7325972741959603813_row_select&f02=&f02=%C3%A5%C3%A4%C3%B6&f02=15"
          },
          "headersSize": 795
        },

....
while the response looks OK again

"response": {
          "status": 200,
          "statusText": "OK",
          "httpVersion": "HTTP/1.1",
          "headers": [
....
controls-cell--label\"><span id=\"control_text_3204685503669598\" class=\"a-IRR-controlsLabel\">Row text contains 'åäö'</span></span><span class
1 of 1 people found this helpful
Actions

1 2 Previous Next

Go to original post