1 2 Previous Next 15 Replies Latest reply on Mar 13, 2009 10:17 AM by Maurice Zeijen

    Feedback on the Smooks editor

    Maurice Zeijen Master

      The last couple of days I have been evaluating the Smooks Eclipse plugin in JBoss Tools RC2. You may have noticed that from the number of Jira issues I have raised for the plugin ;). I would like to share some of my feedback. My interest for the editor comes from the fact that I'm one of the core developers on Smooks, plus the fact that the company I work for uses Smooks and JBoss ESB extensively in a number of production environments.

      I must say that I find it great that JBoss is developing the Eclipse tooling for Smooks. A good tool should make it a lot easier to use Smooks and should open Smooks up for a lot more people. Doing transformations within the ESB should also become a lot easier.

      While using the editor I ran into quite a few issues while trying to do some quite basic things. There where a number of bugs that made the editor practically unusable. I understand that some of these bugs probably only appear in some environments and that there aren't a lot of people out there that are testing it. I added Jira bug reports for these issues. I hoped that the most critical ones would be fixed in the version that gets released with JBoss Tools 3.0.0 GA. Some of the bugs I've added have been fixed already, which is great, but the fix version is for the Smooks editor that gets released in JBoss Tools 3.1.0 GA. Does that mean that JBoss Tools 3.0.0.GA is going to be released with a very buggy Smooks editor?

      I noticed that the editor at this moment for the biggest part concentrate around the XML/Java to Java mapping. Javabean binding is only a small part of what Smooks has to offer. Is it really valid then that the editor gets a 1.0.0 release number? If I think of a 1.0.0 release of the Smooks editor then I would expect that the editor has support for most of the features that Smooks has to offer. It is understandable that the first stable public releases of the editor don't have all the features. But wouldn't it be better to give this editor a version number like 0.1? That at least indicates that the editor doesn't have all the features one can expect.

      Regards,

      Maurice

        • 1. Re: Feedback on the Smooks editor
          Max Rydahl Andersen Master


          First off, I really appreciate we are now getting feedback for the editor even though it is hard to use, but it as a start :)

          Will 3.0.0.GA go out with an incomplete smooks editor ? Yes - I don't want to delay GA months just for one plugin.

          Should it be named 0.1 instead of 1.0 ? Yes, it should but we started with 1.0 since we assumed we would have a more complete editor at the end of development - we assumed wrong but going back in version numbers is en evil sin in context of eclipse updates!

          We will continue to do development on the smooks editor but that will be for the 3.1 development so we don't break 3.0.x users.


          • 2. Re: Feedback on the Smooks editor
            Maurice Zeijen Master

            Thanks for the answer. That explains a lot. I understand that you can't hold back JBoss Tools just for one editor. Isn't it maybe an option to not release the Smooks editor in the next JBoss Tools? Because what is the point of releasing an editor when it is so buggy that it is almost unusable.

            Now that I have the editor in my radar I am going to keep a close eye on it ;). I hope to provide you guys with more feedback in the future. That last couple of days, because I was so busy with the editor, I got a couple of feature idea's myself. At the moment I don't have time to work them out for you guys (I am at my job now) but here are some points I am thinking about:

            * Support for every Smooks input type, without having to code special support for them in the editor. You can use any Smooks reader to find out the input format, as long as you got an example input file. However for those cases where you want to use mapping files (e.g. XML schema or WSDL file) you do need to have some special Smooks reader. But the Smooks cartridge itself could provide that reader instead that the editor needs to provide it.
            * XML configuration editor enhancements:
            * Selector path code completion
            * Bean id code completion
            * Smart configuration templates for all core resource configurations
            * (Inline) Freemarker editor with variable code completion using the selector paths.

            Some of these points need more explanation. If you guys are interested then I will certainly be happy to provide it.

            • 3. Re: Feedback on the Smooks editor
              Max Rydahl Andersen Master

              We don't include Smooks in the upcoming JBoss Developer Studio 2 for this reason, but if we also keeping it out from JBoss Tools we are never going to get any feedback - i.e. you would never have seen this if we hadn't start bundling it, right ?

              With respect to your suggestions about using Smooks reader then I got one question for you: How are we going to handle multiple versions of Smooks if the tooling become dependent on the runtime it self ?

              • 4. Re: Feedback on the Smooks editor
                Max Rydahl Andersen Master

                ..and is the Smooks readers capable of handling partial documents which is how users normally edit files. ?

                • 5. Re: Feedback on the Smooks editor
                  Maurice Zeijen Master

                  I understand that you want to release it. Most developers probably don't try Beta or RC releases. I do hope that not everybody runs into the same critical problems with the graph editor as I did.

                  Regarding to your questions:

                  How are we going to handle multiple versions of Smooks if the tooling become dependent on the runtime it self?
                  That is a very good question. I have an idea how to solve that. By creating a custom classloader repository for each version of Smooks, you can provide support for multiple Smooks versions. In the java project the user needs to configure which Smooks version he uses. The Smooks editor can then dynamically use the classloader repository that belongs to that version. I believe that creating this classloader system is probably easier then creating Smooks independent readers, for every input source that Smooks supports.

                  Are the Smooks readers capable of handling partial documents which is how users normally edit files.
                  I hope I get what you mean with the question. As long as the reader gets a syntax correct file, it will be able to create a data structure from it. It is then up to the editor to decide if the structure is compatible enough against the data structure it may already have. But that wouldn't probably be any different then what it now already does for it's XML or Java source.

                  • 6. Re: Feedback on the Smooks editor
                    Max Rydahl Andersen Master

                    No matter if we use a custom classloader we would need to have a *very* stable API to do this since we would compile against liet say Smooks 1.0 would our code be able to use Smooks 1.1, 1.2 etc. without updating the binaries ? - does Smooks provide that ?

                    And what about the model that we load - is that stable ? I'm asking since I don't know :)

                    About partial documents then users don't have syntax correct files - they type and it will be imperfect. That would mean as soon as the user type the graphical editor would not be able to show anything - that might be ok as you write it.

                    Another issue is that Smooks might not be able to load any of the related classes referred to in the editor because the classes are not compiled yet - does it support that ?

                    • 7. Re: Feedback on the Smooks editor
                      Maurice Zeijen Master

                       

                      No matter if we use a custom classloader we would need to have a *very* stable API to do this since we would compile against liet say Smooks 1.0 would our code be able to use Smooks 1.1, 1.2 etc. without updating the binaries ? - does Smooks provide that ?


                      I believe you need to implement a strategy pattern that can handle every supported Smooks version. For example something like this:

                      - SmooksReaderStrategy interface
                      - Smooks1_0ReaderStrategy class
                      - Smooks1_1ReaderStrategy class

                      Every one of those implementations knows how to use one specific version of Smooks. So they also have dependencies on that specific version. That is why you can't build against only one version of Smooks, you need to build against all of them. Because you can't do that within one bundle, you need to split it up in multiple bundles. The parts of the Smooks editor could then look something like this:

                      - SmookEditor
                      - SmooksReader API
                      - Smooks1_0Reader implementation
                      - Smooks1_1Reader implementation

                      If I understand OSGI correctly then you wouldn't even need to worry to much about classloading, because every bundle can have it's own independent bundle dependencies. The SmooksEditor would have dependencies on the SmooksReader API and all it's implementations. Every implementation would a dependency on one specific version of Smooks.

                      You could even make every reader a separate Eclipse plugin. That way you could implement new Readers for new versions of Smooks and use them with older versions of the editor. You only need to install the plugin.

                      And what about the model that we load - is that stable ? I'm asking since I don't know :)


                      The model that the Smooks editor gets from the Readers wouldn't change in between Smooks editors because they would belong to SmooksReader API. The Reader implementations would take care of the proper translation between the model that Smooks uses and the model that the Editor uses.

                      About partial documents then users don't have syntax correct files - they type and it will be imperfect. That would mean as soon as the user type the graphical editor would not be able to show anything - that might be ok as you write it.


                      Yeah, you would have that same problem with the current Smooks editor. The editor would need to tell the user that the source model can't be read because of a syntax problem.

                      Another issue is that Smooks might not be able to load any of the related classes referred to in the editor because the classes are not compiled yet - does it support that ?


                      No, Smooks doesn't currently support that. In this special case we would need a special reader. Maybe we can reuse the code that is now used by the editor to read the java model?

                      I think that the Smooks team would be happy to provide support for implementing any special readers like that special Java reader or a XSD reader. I know I would at least ;).

                      • 8. Re: Feedback on the Smooks editor
                        Max Rydahl Andersen Master


                        Warning - I am a smooks noob, just have that in mind when reading my answers :)

                        "mzeijen" wrote:
                        No matter if we use a custom classloader we would need to have a *very* stable API to do this since we would compile against liet say Smooks 1.0 would our code be able to use Smooks 1.1, 1.2 etc. without updating the binaries ? - does Smooks provide that ?


                        I believe you need to implement a strategy pattern that can handle every supported Smooks version. For example something like this:

                        - SmooksReaderStrategy interface
                        - Smooks1_0ReaderStrategy class
                        - Smooks1_1ReaderStrategy class


                        Of course, the problem is just that the two models of Smooks 1_0 and Smooks 1_1 is vastly different so I'm not sure any strategy can be defined that would work. But of course if it is doable then sure.


                        Every one of those implementations knows how to use one specific version of Smooks. So they also have dependencies on that specific version.


                        Of course they have an dependency to a specific version (or rather version range Smooks 1.1.x and 1.0.x) but that does not mean they have a compile nor runtime dependency on the Smooks binaries.


                        That is why you can't build against only one version of Smooks, you need to build against all of them.


                        I develop an IDE that is compiled against Java 5 and it supports Java 1.1 to Java 6 without any build dependency to each specific version.

                        Same goes for our AS, jbpm, hibernate (to some extent), esb, jbossws annd other tooling. If we had to depend on *every* runtime version of these we would have a gigantic monster ;)


                        Because you can't do that within one bundle, you need to split it up in multiple bundles. The parts of the Smooks editor could then look something like this:

                        - SmookEditor
                        - SmooksReader API
                        - Smooks1_0Reader implementation
                        - Smooks1_1Reader implementation

                        If I understand OSGI correctly then you wouldn't even need to worry to much about classloading, because every bundle can have it's own independent bundle dependencies.


                        Sure, but we would now be bundling how many versions of Smooks ?
                        What about smooks 1.0.1, 1.0.2, 1.1.0, 1.1.1 etc ?


                        The SmooksEditor would have dependencies on the SmooksReader API and all it's implementations. Every implementation would a dependency on one specific version of Smooks.


                        which means if smooks 1.2 comes out we would need to redo/adjust the tooling before the tooling can read understand it even if the syntax is 90% the same ...


                        You could even make every reader a separate Eclipse plugin. That way you could implement new Readers for new versions of Smooks and use them with older versions of the editor. You only need to install the plugin.


                        If we somehow can create a model from the internal smooks model that is independent of it then I agree this could work - but users who have to have perfect syntactically correct files all the time.


                        And what about the model that we load - is that stable ? I'm asking since I don't know :)


                        The model that the Smooks editor gets from the Readers wouldn't change in between Smooks editors because they would belong to SmooksReader API. The Reader implementations would take care of the proper translation between the model that Smooks uses and the model that the Editor uses.


                        The Reader you mention in that paragraph is from Smooks or from the Smooks plugin ?


                        About partial documents then users don't have syntax correct files - they type and it will be imperfect. That would mean as soon as the user type the graphical editor would not be able to show anything - that might be ok as you write it.


                        Yeah, you would have that same problem with the current Smooks editor. The editor would need to tell the user that the source model can't be read because of a syntax problem.


                        the current editor at least in theory can survive broken parts and still read the rest.


                        Another issue is that Smooks might not be able to load any of the related classes referred to in the editor because the classes are not compiled yet - does it support that ?


                        No, Smooks doesn't currently support that. In this special case we would need a special reader. Maybe we can reuse the code that is now used by the editor to read the java model?


                        If we can pass in a strategy then sure.


                        I think that the Smooks team would be happy to provide support for implementing any special readers like that special Java reader or a XSD reader. I know I would at least ;).


                        Great :)

                        p.s. I'll meet up with Dart who have been doing the smooks editor at EclipseCon in 2 weeks time where we will discuss this so keep the ideas flowing ;)

                        • 9. Re: Feedback on the Smooks editor
                          Maurice Zeijen Master

                           

                          Warning - I am a smooks noob, just have that in mind when reading my answers :)


                          No problem. But I must confess that I am wondering now if we are talking about the same thing ;).

                          I am also just brainstorming here, so my theories can also be wrong one or more points ;).

                          Of course, the problem is just that the two models of Smooks 1_0 and Smooks 1_1 is vastly different so I'm not sure any strategy can be defined that would work. But of course if it is doable then sure.


                          To make sure that we are talking about the same thing I am going to explain my idea in little more detail.. I am proposing to use Smooks its readers (XML, EDI, CSV, JSON, Java reader) to analyze an example file of the source data, that the final Smooks configuration is going to process. Because of Smooks its flexibility you can let Smooks do the reading but you provide your own code to build the data model. The resulting model will probably look a lot like a DOM model. The visitor classes, provided by the Smooks editor, that build the model, will be Smooks depended. But because Smooks is backwards compatible you can use the same classes for Smooks 1.0 as for Smooks 1.2.

                          I think what you mean is Smooks its configuration model. That changed a lot between version 1.0 and 1.1. But it is still possible to use the 1.0 configuration model with Smooks 1.1.


                          Sure, but we would now be bundling how many versions of Smooks ?
                          What about smooks 1.0.1, 1.0.2, 1.1.0, 1.1.1 etc ?


                          Smooks is backwards compatible at every level. Even when you would provide different strategies for different versions of Smooks, they probably all can use the same version of Smooks. So you don't need to include Smooks 4x times. Only in the theoretical case that Smooks wouldn't be backwards compatible between minor versions, then you would have the possibility to use two different versions of Smooks. That makes sure that the Eclipse Editor isn't Smooks version specific.

                          The Reader you mention in that paragraph is from Smooks or from the Smooks plugin ?


                          I meant the Smooks editor implementation of the SmooksReaderStrategy.

                          the current editor at least in theory can survive broken parts and still read the rest.


                          Personally I am not sure if surviving a broken part would be the correct thing to do. But that is a something for a different discussion. If surviving a broken input source would be needed then we probably could implement such a feature in our current readers.

                          The biggest point of my idea is that you don't need to re-implement something that is already provided. I don't see the point that you guys need to re-implement the XML, EDI, JSON, etc readers.

                          Max, would you be interested to talk about this topic during a chat session? That would go a lot faster and probably prevent a lot of misunderstandings then discussing this via the forum. Do you use Skype? If you are then maybe Tom Fennelly would also be interested to join the conversation.

                          • 10. Re: Feedback on the Smooks editor
                            Max Rydahl Andersen Master

                            I think we are talking about the same thing and yes I would love to go on a chat with you guys - but not until we get GA out; i'm fully booked until then, so some time next week.

                            With respect to broken syntax readers imagine if the java editor in eclipse would stop being able to code complete, syntax color etc. if you did not make sure every key you pressed were correct ?

                            Anyhow - I agree it might be ok for a smooks editor to be more strict if we can get a lot of features "for free" by giving up some others.

                            Next after binding to specific versions of Smooks (which you say is not so big a deal because Smooks is built to be backwards and possibly also forwards compatible - thanks for that!) then my biggest issue is being able to read smooks configuration files without having all the listed classes/resources available to the reader (i.e. classes might not be compiled, templates might not be where you want it to be etc.)

                            But let's chat about that next week.

                            • 11. Re: Feedback on the Smooks editor
                              Tom Fennelly Master

                              Hi guys. Sorry for being late in joining this conversation :)

                              I think we are definitely talking about different things here. My 2c worth...

                              Configuration Model
                              In terms of Smooks configurations, the latest version of Smooks (v1.1) is backward compatible (BC) with versions back to v0.9 (and probably before). So you can throw the v0.9 DTD, the v1.0 XSD or the v1.1 XSD at it and it will be able to generate it's internal config model from it.

                              As far as v1.0 and v1.1 are concerned, what was added here in v1.1 was the ability to define extended namespaces i.e. to extend the Smooks base configuration in order to define a configuration namespace for specific features e.g. for java binding configurations, for applying freemarker templates to a fragment of the input data, for triggering a read on a DB in order to enrich the source message etc etc.

                              My understand is that this extended config namespace approach should help with tooling because it allows for the creation of stricter/tighter configs. I might be wrong there.

                              In v1.0, all of these configuration "resources" had to be defined in the base config model, which made the configs very verbose... poor XSD based help from your IDE etc. In Smooks v1.1, all of these resource types have a namespace of their own so they are far more strongly typed, less verbose etc etc. That said... Smooks v1.1 would still be able to read a v1.0 style config no problem. We made a lot of effort to try and ensure this, so I'd be very p*ssed off if we found somewhere here that we broken BC.

                              I think this was the subject of Max's thoughts, right?

                              Data Source Model
                              Then you have the source data that Smooks is going to process in some way e.g. XML, EDI, JSON, CSV, Java etc. To process these, Smooks needs to be configured with a "Reader", whose responsibility is to generate a stream of SAX events from the source data model, no matter what it's format is i.e. anything that's structured/hierarchical - not just XML. The smooks runtime processes this stream of events and, at the appropriate times, fires the resources configured in the smooks config.

                              So the source data may be an EDI/JSON/CSV message, but the data model "seen" by the smooks runtime is a stream of SAX events generated by the EDI reader that was plugged into Smooks for reading the source. The reader is defined in the smooks config. If not defined, it defaults to whatever is configured in the JAXP environment e.g. Xerces.

                              The supplied Reader must implement org.xml.sax.XMLReader (JDK).

                              I think this was the subject of Maurice's thoughts, right?

                              Smooks API
                              The Smooks "API" is actually very simple and hasn't changed in a few versions and I don't see any reason to change it in the future.

                              In terms of the configuration API... Smooks receives the XML config as a InputStream via it's constructor. I don't think there's any need to discuss programmatic configuration since the editor will not be using that, right?

                              In terms of the execution API... there are 2 methods on the Smooks class that are of interest:
                              - createExecutionContext()
                              - filter()

                              The filter method takes 3 params:
                              - javax.xml.transform.Source (JDK).
                              - javax.xml.transform.Result (JDK).
                              - ExecutionContext (Smooks core/runtime). This interface would be stable/BC (v1.0, v1.1).

                              So going forward, I'd be confident we can maintain backward compatibility on at an API level.

                              • 12. Re: Feedback on the Smooks editor
                                Maurice Zeijen Master

                                 

                                max.andersen@jboss.com wrote:
                                yes I would love to go on a chat with you guys - but not until we get GA out; i'm fully booked until then, so some time next week.


                                Ok, that is cool. Could you send me an e-mail so that we can schedule it? My address is maurice(at)smies(point)com. It would need to be in the evening, because I am at work during the day.

                                tfennelly wrote:
                                I think we are definitely talking about different things here.


                                Thanks for explaining clearly what we are talking about ;). I was indeed talking about the Data Source Model.

                                • 13. Re: Feedback on the Smooks editor
                                  Tom Fennelly Master

                                   

                                  "mzeijen" wrote:
                                  Ok, that is cool. Could you send me an e-mail so that we can schedule it? My address is maurice(at)smies(point)com. It would need to be in the evening, because I am at work during the day.


                                  We also need to make sure that John (Graham) is involved in any discussions. Max, John and Dart can work out a time that suits them and get back to me.

                                  1 2 Previous Next