1 2 Previous Next 16 Replies Latest reply on Aug 12, 2016 9:44 AM by Wolfgang Mayer

    Undertow encoding problem

    Nicklas Karlsson Master

      We're testing Oracle APEX on WF10 (Windows) and noticed that some special chars (åäö) now show up as upside-down question marks (classical encoding problem).

      I've tried setting the undertow servlet default-encoding to UTF-8 and even tried passing in -Dfile.encoding=UTF-8 but I still see the problem. The HTTP response header says UTF-8. Pointers?

       

      Thanks in advance,

      Nik

        • 1. Re: Undertow encoding problem
          Tomaz Cerar Master

          Is the "static" (jsp/servlet/...) code that wrongly displays chars or is it dynamic content from database?

          if such you could look at datasource driver configuration.

          • 2. Re: Undertow encoding problem
            Nicklas Karlsson Master

            Well, with APEX you never really know, everything is sort of DB ;-)

             

            But I've already tried adding the ora18n.jar to the driver module (which usually helps), but in this case it didn't...

            • 3. Re: Undertow encoding problem
              Nicklas Karlsson Master

              It's a bit strange. The page contains special chars, works fine. If I submit the form with special chars and use the Chrome console, I see the form data f02: åäö with content type application/x-www-f-u.

              But when the page is updated, the search text is garbled. Somehow it looks like the encoding is messed up when/after the data the data is submitted.

               

              Pointers?

              • 4. Re: Undertow encoding problem
                Tomaz Cerar Master

                reading from database would be my go-to for next research.

                maybe some special settings for jdbc driver, some extra parameter in the url. or maybe even different jdbc driver all together.

                • 5. Re: Undertow encoding problem
                  Nicklas Karlsson Master

                  I don't think it's that. It's working against the same database as before and I used the same driver module as in the old EAP6.1. Besides, I'm almost sure the search criteria as never stored in the DB and whatever roundtrip to the DB the rendering takes, it works for the non-submitted stuff (labels etc)...

                  • 6. Re: Undertow encoding problem
                    jaikiran pai Master

                    What exactly does this page(?) involve? JSP? Can you post a sample field on that page so that we know what kind of a form element it is and how it's created? Also, where is the data sourced from, for that page (I don't mean the DB part but any web layer frameworks etc).

                    • 7. Re: Undertow encoding problem
                      Nicklas Karlsson Master

                      There is an apex.war deployed that connects through a connection pool to the DB and does the interaction. The view, fields etc. are all defined in fields in the DB. This is a Chrome HAR of the exchange. The "f02" is the submitted data and the content/text is the truncated response for the offending label.

                       

                      {
                        "log": {
                          "version": "1.2",
                          "creator": {
                            "name": "WebInspector",
                            "version": "537.36"
                          },
                          "pages": [],
                          "entries": [
                            {
                              "startedDateTime": "2016-08-10T06:14:14.279Z",
                              "time": 192.99899972975254,
                              "request": {
                                "method": "POST",
                                "url": "http://mare12:8008/apex/wwv_flow.show",
                                "httpVersion": "HTTP/1.1",
                                "headers": [
                                  {
                                    "name": "Origin",
                                    "value": "http://mare12:8008"
                                  },
                                  {
                                    "name": "Accept-Encoding",
                                    "value": "gzip, deflate"
                                  },
                                  {
                                    "name": "Host",
                                    "value": "mare12:8008"
                                  },
                                  {
                                    "name": "Accept-Language",
                                    "value": "en-US,en;q=0.8,fi-FI;q=0.6,fi;q=0.4"
                                  },
                                  {
                                    "name": "User-Agent",
                                    "value": "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2783.4 Safari/537.36"
                                  },
                                  {
                                    "name": "Content-Type",
                                    "value": "application/x-www-form-urlencoded"
                                  },
                                  {
                                    "name": "Accept",
                                    "value": "*/*"
                                  },
                                  {
                                    "name": "Referer",
                                    "value": "http://mare12:8008/apex/f?p=100:8:4251609565487::::VA:4"
                                  },
                                  {
                                    "name": "Cookie",
                                    "value": "PVEKAAPPI_SISNO=128; SAFIIRI_USR_8008=jas_v2; SAFIIRI_SISNO_8008=1; SAFIIRI_DOM_8008=KAKE; ORA_WWV_APP_100=ORA_WWV-hONgJCXzx+g7OhkXtH5odXDH"
                                  },
                                  {
                                    "name": "Connection",
                                    "value": "keep-alive"
                                  },
                                  {
                                    "name": "Content-Length",
                                    "value": "330"
                                  }
                                ],
                                "queryString": [],
                                "cookies": [
                                  {
                                    "name": "PVEKAAPPI_SISNO",
                                    "value": "128",
                                    "expires": null,
                                    "httpOnly": false,
                                    "secure": false
                                  },
                                  {
                                    "name": "SAFIIRI_USR_8008",
                                    "value": "jas_v2",
                                    "expires": null,
                                    "httpOnly": false,
                                    "secure": false
                                  },
                                  {
                                    "name": "SAFIIRI_SISNO_8008",
                                    "value": "1",
                                    "expires": null,
                                    "httpOnly": false,
                                    "secure": false
                                  },
                                  {
                                    "name": "SAFIIRI_DOM_8008",
                                    "value": "KAKE",
                                    "expires": null,
                                    "httpOnly": false,
                                    "secure": false
                                  },
                                  {
                                    "name": "ORA_WWV_APP_100",
                                    "value": "ORA_WWV-hONgJCXzx+g7OhkXtH5odXDH",
                                    "expires": null,
                                    "httpOnly": false,
                                    "secure": false
                                  }
                                ],
                                "headersSize": 618,
                                "bodySize": 327,
                                "postData": {
                                  "mimeType": "application/x-www-form-urlencoded",
                                  "text": "p_request=APXWGT&p_instance=4251609565487&p_flow_id=100&p_flow_step_id=8&p_widget_num_return=15&f01=apexir_CURRENT_SEARCH_COLUMN&f01=apexir_SEARCH&f01=apexir_NUM_ROWS&f02=&f02=åäö&f02=15&p_widget_name=worksheet&p_widget_mod=ACTION&p_widget_action=QUICK_FILTER&p_widget_action_mod=ADD&x01=50377881313248334&x02=50381983327248344",
                                  "params": [
                                    {
                                      "name": "p_request",
                                      "value": "APXWGT"
                                    },
                                    {
                                      "name": "p_instance",
                                      "value": "4251609565487"
                                    },
                                    {
                                      "name": "p_flow_id",
                                      "value": "100"
                                    },
                                    {
                                      "name": "p_flow_step_id",
                                      "value": "8"
                                    },
                                    {
                                      "name": "p_widget_num_return",
                                      "value": "15"
                                    },
                                    {
                                      "name": "f01",
                                      "value": "apexir_CURRENT_SEARCH_COLUMN"
                                    },
                                    {
                                      "name": "f01",
                                      "value": "apexir_SEARCH"
                                    },
                                    {
                                      "name": "f01",
                                      "value": "apexir_NUM_ROWS"
                                    },
                                    {
                                      "name": "f02",
                                      "value": ""
                                    },
                                    {
                                      "name": "f02",
                                      "value": "åäö"
                                    },
                                    {
                                      "name": "f02",
                                      "value": "15"
                                    },
                                    {
                                      "name": "p_widget_name",
                                      "value": "worksheet"
                                    },
                                    {
                                      "name": "p_widget_mod",
                                      "value": "ACTION"
                                    },
                                    {
                                      "name": "p_widget_action",
                                      "value": "QUICK_FILTER"
                                    },
                                    {
                                      "name": "p_widget_action_mod",
                                      "value": "ADD"
                                    },
                                    {
                                      "name": "x01",
                                      "value": "50377881313248334"
                                    },
                                    {
                                      "name": "x02",
                                      "value": "50381983327248344"
                                    }
                                  ]
                                }
                              },
                              "response": {
                                "status": 200,
                                "statusText": "OK",
                                "httpVersion": "HTTP/1.1",
                                "headers": [
                                  {
                                    "name": "X-DB-Content-length",
                                    "value": "10506"
                                  },
                                  {
                                    "name": "Date",
                                    "value": "Wed, 10 Aug 2016 06:14:14 GMT"
                                  },
                                  {
                                    "name": "Server",
                                    "value": "WildFly/10"
                                  },
                                  {
                                    "name": "Connection",
                                    "value": "keep-alive"
                                  },
                                  {
                                    "name": "X-Powered-By",
                                    "value": "Undertow/1"
                                  },
                                  {
                                    "name": "Content-Length",
                                    "value": "10526"
                                  },
                                  {
                                    "name": "Content-Type",
                                    "value": "text/html;charset=UTF-8"
                                  }
                                ],
                                "cookies": [],
                                "content": {
                                  "size": 10526,
                                  "mimeType": "text/html",
                                  "compression": 0,
                                  "text": "<td class=\"fielddata\">Rivin teksti sis&#xE4;lt&#xE4;&#xE4; &#x27;&#xBF;&#xBF;&#xBF;&#xBF;&#xBF;&#xBF;&#x27;</td>"
                                },
                                "redirectURL": "",
                                "headersSize": 216,
                                "bodySize": 10526,
                                "_transferSize": 10742
                              },
                              "cache": {},
                              "timings": {
                                "blocked": 2.03599967062473,
                                "dns": 0.6890003569424201,
                                "connect": 1.3849996030330596,
                                "send": 0.18600001931191024,
                                "wait": 188.0570002831519,
                                "receive": 0.6459997966885282,
                                "ssl": -1
                              },
                              "serverIPAddress": "192.130.222.243",
                              "connection": "1322310"
                            }
                          ]
                        }
                      }
                      

                       

                      The submitting field is a

                       

                      <input title="Hae raportin näkyvillä olevista sarakkeista
                      " type="text" size="30" maxlength="4000" value="" id="apexir_SEARCH" onkeydown="if($f_Enter(event)){gReport.search('SEARCH'); return false;}">
                      

                       

                      in a form

                       

                      <form action="wwv_flow.accept" method="post" name="wwv_flow" id="wwvFlowForm" novalidate="">
                      
                      • 8. Re: Undertow encoding problem
                        Wolfgang Mayer Master

                        So you deployed the APEX Listener to use WildFly as HTTP server?

                        I understand the APEX listener is quite old and had been replaced by Oracle REST Data Services (ORDS) in between.

                        Therefore your problem may be related to APEX. Did you verify if your problem persists when you use ORDS instead of APEX:

                        • 9. Re: Undertow encoding problem
                          Nicklas Karlsson Master

                          We're on the 4.x-series but I guess we're going to upgrade to 5.x soonish so I guess we could try.

                          I must say that I don't know about the listener changes but 4.2.6 is under two years old so one would think that there wouldn't be a problem with basic HTTP/Servlet operations...

                          • 10. Re: Undertow encoding problem
                            Wolfgang Mayer Master

                            I think you refer to the Oracle Application Express version which is currently on 5.0.4.

                            The listener (APEX or ORDS) is not dependent on the OE version (actually Oracle Application Express is not a prerequisite for using Oracle REST Data Services).

                            The latest APEX listener I am aware of is 2.0.5 (which is about 2 years old)

                            Oracle REST Data Services which is currently on version 3.0.6 was released in July 2016.

                            • 11. Re: Undertow encoding problem
                              Nicklas Karlsson Master

                              I must confess I don't have much experience in the version history etc. The case just landed on my desk because "it worked before, now it doesn't" ;-)

                               

                              Can anyone point me to the location where I should put my breakpoint if I want to debug Undertow at the point where the decoding of the parameters has occurred?

                              • 12. Re: Undertow encoding problem
                                jaikiran pai Master

                                From what I understand, it's the response whose content uses a wrong encoding. If that's the case perhaps io.undertow.servlet.spec.HttpServletResponseImpl might be good place to start adding breakpoints.

                                • 13. Re: Undertow encoding problem
                                  Nicklas Karlsson Master

                                  I think the jury is still out on that one, I guess if the stuff is still decoded correctly at the point where it is finally passed off to the application is has to be in the encoding phase...

                                  • 14. Re: Undertow encoding problem
                                    Wolfgang Mayer Master

                                    I still don't think that this is related to WildFly. In my environment the special characters are shown properly:

                                    - Application Express: build 5.0.4.00.12

                                    - APEX_LISTENER_VERSION   3.0.6.176.08.46

                                    - PLSQL_GATEWAY   WebDb

                                    - wildfly-10.0.0.Final

                                     

                                    {

                                      "log": {

                                        "version": "1.1",

                                        "creator": {

                                          "name": "Firefox",

                                          "version": "48.0"

                                        },

                                        "browser": {

                                          "name": "Firefox",

                                          "version": "48.0"

                                        },

                                     

                                    ....

                                    "entries": [

                                          {

                                            "pageref": "page_4",

                                            "startedDateTime": "2016-08-12T11:04:10.090+02:00",

                                            "time": 181,

                                            "request": {

                                              "bodySize": 900,

                                              "method": "POST",

                                              "url": "http://localhost:8080/ords/wwv_flow.accept",

                                              "httpVersion": "HTTP/1.1",

                                              "headers": [

                                                {

                                                  "name": "Host",

                                                  "value": "localhost:8080"

                                                },

                                                {

                                                  "name": "User-Agent",

                                                  "value": "Mozilla/5.0 (X11; Linux x86_64; rv:48.0) Gecko/20100101 Firefox/48.0"

                                                },

                                                {

                                                  "name": "Accept",

                                                  "value": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"

                                                },

                                     

                                    ....

                                    "queryString": [],

                                              "postData": {

                                                "mimeType": "application/x-www-form-urlencoded",

                                                "params": [

                                    ...

                                    {

                                                    "name": "p_t02",

                                                    "value": "Jürgen"

                                                  },

                                                  {

                                                    "name": "p_arg_names",

                                                    "value": "7470154439861862127"

                                                  },

                                                  {

                                                    "name": "p_t03",

                                                    "value": "Böße-Bären"

                                                  },

                                     

                                    ....

                                    },

                                            "response": {

                                              "status": 200,

                                              "statusText": "OK",

                                              "httpVersion": "HTTP/1.1",

                                              "headers": [

                                                {

                                                  "name": "Connection",

                                                  "value": "keep-alive"

                                                },

                                                {

                                                  "name": "X-Powered-By",

                                                  "value": "Undertow/1"

                                                },

                                                {

                                                  "name": "Server",

                                                  "value": "WildFly/10"

                                                },

                                    ....

                                    "content": {

                                                "mimeType": "text/html; charset=UTF-8",

                                                "size": 22709,

                                                "text": "<!--[if HTML5

                                    ....

                                    \">Tags</a></th></tr>\n<tr ><td class=\" u-tL\" headers=\"CUSTOMER_NAME\"><a href=\"f&#x3F;p&#x3D;110&#x3A;7&#x3A;5717697975157&#x3A;&#x3A;NO&#x3A;&#x3A;P7_CUSTOMER_ID,P7_BRANCH&#x3A;100,2&#x26;cs&#x3D;15E03E4CD071461359F6F0475F3496D58\" >Böße-Bären, Jürgen</a></td><td class=\" u-tL\" headers=\"CUSTOMER_ADDRESS\"></td><td class=\" u-tL\" headers=\"CUST_CITY\"></td><td class=\" u-tL\" headers=\"CUST_STATE\">AK</td><td class=\" u-tL\" headers=\"CUST_POSTAL_CODE\">7455</td><td class=\" u-tL\" headers=\"TAGS\"></td></tr></table></div><div class=\"a-IRR-paginationWrap a-IRR-paginationWrap--bottom\"><ul class

                                     

                                     

                                    But there is something else strange:

                                    Your special characters appear as "name": "f02". Did you enter these characters in the search field like

                                    In my environment these characters are encoded as hexadecimal values in this case

                                    "queryString": [],

                                              "postData": {

                                                "mimeType": "application/x-www-form-urlencoded",

                                                "params": [],

                                                "text": "p_request=PLUGIN%3D29A7AFE9DE9D66DE991787CDEE424EAC58BBDE5795FEE6EA6AF299C0FF2E254A&p_flow_id=110&p_flow_step_id=2&p_instance=2998691333482&p_debug=&p_widget_name=worksheet&p_widget_mod=ACTION&p_widget_action=QUICK_FILTER&p_widget_num_return=15&x01=7325972842842603813&x02=7179925510617646276&f01=R7325972741959603813_column_search_current_column&f01=R7325972741959603813_search_field&f01=R7325972741959603813_row_select&f02=&f02=%C3%A5%C3%A4%C3%B6&f02=15"

                                              },

                                              "headersSize": 795

                                            },

                                     

                                    ....

                                    while the response looks OK again

                                     

                                    "response": {

                                              "status": 200,

                                              "statusText": "OK",

                                              "httpVersion": "HTTP/1.1",

                                              "headers": [

                                    ....

                                    controls-cell--label\"><span id=\"control_text_3204685503669598\" class=\"a-IRR-controlsLabel\">Row text contains &#x27;åäö&#x27;</span></span><span class

                                    1 of 1 people found this helpful
                                    1 2 Previous Next