12 Replies Latest reply on Feb 28, 2008 11:14 AM by ropalka

    Signatures and CRs

    asoldano

      Hi Folks,
      I spent some time on http://jira.jboss.org/jira/browse/JBWS-2011, I think there aren't straightforward solutions for this, may be somebody else have interesting idea ;-)
      Here is my analysis: the reason why the signature verification fails when signing messages containing CR (\r i.e. #x0d) is that the WS-Security implementation works with the soap message element that is created parsing the incoming xml message. As you can read here http://www.w3.org/TR/REC-xml/#sec-line-ends CRs must be converted in line feeds when parsing. Xerces does this of course, thus the message bytes change and signature (which is obtained using the message containing the CR) is not valid anymore when processing the request message at server side.
      We might strip out CR from messages before signing but that would be a poor fix since thirdparty implementations could still produce signatures over messages containing CRs.
      Btw, if you're wondering whether this should be done by the canonicalization happening before the actual signature is computed, here http://www.w3.org/TR/2001/REC-xml-c14n-20010315#Example-Chars is an example showing the CR is processed and not discarder (we actually use the Exclusive XML canonicalization, but afaik this shouldn't change).

      Fortunately this issue only happens when the message is not encrypted. Signature+encryption prevents this since the xml parser works with the encrypted element data.

        • 1. Re: Signatures and CRs
          ropalka

           

          "alessio.soldano@jboss.com" wrote:
          As you can read here http://www.w3.org/TR/REC-xml/#sec-line-ends CRs must be converted in line feeds when parsing. Xerces does this of course, thus the message bytes change and signature (which is obtained using the message containing the CR) is not valid anymore when processing the request message at server side.


          From my understanding this have nothing to do with our problem. The above specification says what to do with new-lines only. Our use-case is different. We have element containing
          sequence of characters enclosing 0xD character:

          <some_element>asdf\rjklo</some_element>
          


          • 2. Re: Signatures and CRs
            ropalka

            I'm not 100 % sure but IMHO such enclosed 0xD character won't be converted to new line by the parser.

            • 3. Re: Signatures and CRs
              ropalka

               

              "alessio.soldano@jboss.com" wrote:

              We might strip out CR from messages before signing but that would be a poor fix since thirdparty implementations could still produce signatures over messages containing CRs.

              Stripping out the CR character from such elements violates the message integrity.

              • 4. Re: Signatures and CRs
                ropalka

                 

                "alessio.soldano@jboss.com" wrote:

                Fortunately this issue only happens when the message is not encrypted. Signature+encryption prevents this since the xml parser works with the encrypted element data.

                If we're not able to fix the problem we could suggest our users to use base64 encoding of such strings before putting them to the message payload (the workaround)

                • 5. Re: Signatures and CRs
                  asoldano

                   

                  "richard.opalka@jboss.com" wrote:
                  I'm not 100 % sure but IMHO such enclosed 0xD character won't be converted to new line by the parser.


                  Try this:
                  public void testParser() throws Exception
                   {
                   String s = "<env:Envelope xmlns:env='http://schemas.xmlsoap.org/soap/envelope/'><env:Body><a>He\rllo</a></env:Body></env:Envelope>";
                   Logger.getLogger(this.getClass()).info("before: "+s);
                   ByteArrayInputStream bis = new ByteArrayInputStream(s.getBytes());
                   Element el = DOMUtils.parse(bis);
                   Element body = DOMUtils.getChildElements(el).next();
                   Element a = DOMUtils.getChildElements(body).next();
                   Logger.getLogger(this.getClass()).info(a.getTextContent());
                   Logger.getLogger(this.getClass()).info(a.getChildNodes().item(0));
                   String res = DOMWriter.printNode(el, false);
                   Logger.getLogger(this.getClass()).info("after: "+res);
                   assertEquals(s, res);
                   }
                  


                  The \r is converted into in a new line (and the logs show the text node already contains the new line, thus it's created while parsing).

                  • 6. Re: Signatures and CRs
                    asoldano

                     

                    "richard.opalka@jboss.com" wrote:
                    "alessio.soldano@jboss.com" wrote:
                    As you can read here http://www.w3.org/TR/REC-xml/#sec-line-ends CRs must be converted in line feeds when parsing. Xerces does this of course, thus the message bytes change and signature (which is obtained using the message containing the CR) is not valid anymore when processing the request message at server side.


                    From my understanding this have nothing to do with our problem. The above specification says what to do with new-lines only. Our use-case is different. We have element containing
                    sequence of characters enclosing 0xD character:


                    Sorry, may be I'm missing something, why this should have nothing to do with our issue? The spec says: "To simplify the tasks of applications, the XML processor MUST behave as if it normalized all line breaks in external parsed entities (including the document entity) on input, before parsing, by translating both the two-character sequence #xD #xA and any #xD that is not followed by #xA to a single #xA character." We have a #xD not followed by #xA that is converted to #xA...

                    • 7. Re: Signatures and CRs
                      asoldano

                       

                      "richard.opalka@jboss.com" wrote:
                      "alessio.soldano@jboss.com" wrote:

                      Fortunately this issue only happens when the message is not encrypted. Signature+encryption prevents this since the xml parser works with the encrypted element data.

                      If we're not able to fix the problem we could suggest our users to use base64 encoding of such strings before putting them to the message payload (the workaround)


                      Yep, this is a workaround too, you're right.

                      • 8. Re: Signatures and CRs
                        ropalka

                         

                        "alessio.soldano@jboss.com" wrote:

                        Sorry, may be I'm missing something, why this should have nothing to do with our issue? The spec says: "To simplify the tasks of applications, the XML processor MUST behave as if it normalized all line breaks in external parsed entities (including the document entity) on input, before parsing, by translating both the two-character sequence #xD #xA and any #xD that is not followed by #xA to a single #xA character." We have a #xD not followed by #xA that is converted to #xA...


                        I was wrong, you are true

                        • 9. Re: Signatures and CRs
                          ropalka

                           

                          "alessio.soldano@jboss.com" wrote:
                          "richard.opalka@jboss.com" wrote:
                          "alessio.soldano@jboss.com" wrote:

                          Fortunately this issue only happens when the message is not encrypted. Signature+encryption prevents this since the xml parser works with the encrypted element data.

                          If we're not able to fix the problem we could suggest our users to use base64 encoding of such strings before putting them to the message payload (the workaround)


                          Yep, this is a workaround too, you're right.


                          I'd like wonder whether other WS-Security implementations don't have the same problem ;)

                          • 10. Re: Signatures and CRs
                            thomas.diesler

                            IMHO, we should generally decide whether a combination of CR+LF is valid for an incomming message.

                            If it is valid (even for a normal incoming message) the stack should preserve it.

                            It can possibly be invalid for an incoming message that uses WS-Security, in which case we should have a clear error message if possible.

                            If it is invalid, our stack should not produce it.

                            Generally, I believe we should be strict rather than lenient with respect to message integrity. Axis opted to be lenient and tried to tolerate all sorts of inconsistencies - AFAICS they ended up with an unmaintainable code base.

                            • 11. Re: Signatures and CRs
                              asoldano

                              Here are my thoughs and what I did: generally speaking I think a message containing CRLF is valid. Parsers are able to deal with them and according to the specs CR must be converted to LF when parsing. When sending a message you should thus know that if there's a CR in the document's content, it should not be send as a literal otherwise it will be normalized to LF when the document is parsed; it should be encoded as & # x D ; instead (sorry for the padding spaces ;-) ).
                              As a matter of fact, according to my tests, if you don't run into any transition to DOM_VALID representation of the soap content (need to comment out most of the message logging stuff to achieve this), you already get this (the & # x D ; ) using our stack when outputting messages containing \r in a string content. Thus IMHO the problem was in the DOMWriter's normalize method which didn't convert \r into & # x D ; . So I fixed it. This way, along with the changes done for JBWS-1974, the wsse implementation computes the signature over an element whose content is what the user specified and will not be altered during parsing on client side.

                              • 12. Re: Signatures and CRs
                                ropalka

                                I like this fix