13 Replies Latest reply on Nov 1, 2016 12:34 PM by jamezp

    Adding Logging Subsystem Resources At Boottime

    jfisherdev

      I am currently working to solve the following problem and I am hoping someone here may have some ideas:

       

      Some, but not all, applications deployed to a standalone WildFly 9.0.2 server define custom logging configuration using log4j.xml documents. The set of applications deployed to the server may vary. I would like the appropriate logging subsystem resources to be created at boottime so the custom logging configuration is in place before application deployment. I do not want to modify the standalone.xml before starting the server, so my default server configuration is separate from the dynamic logging configuration and manually mapping log4j.xml to logging subsystem XML configuration is not something I want to attempt. I also want the benefits of having the logging subsystem management resources for this configuration [runtime modification, visibility in the management console], which the approach of packaging the configuration with the deployment and setting the use-deployment-logging-config option to true does not provide.

       

      My plan is to write an extension that does the following at boottime:

      1. Scans for log4j.xml configuration files in a directory that contains application configuration files.
      2. Calls the appropriate Logging subsystem operations to create the appropriate resources and to do the appropriate configuration from the log4j.xml.

       

      The issue I am having is that there is not an obvious entry point into the Logging subsystem from my extension.

       

      I have written an extension that do this sort of thing for data sources and I am working on one for JMS destinations. However, there was an obvious entry point in those cases. Specifically, there was an OperationStepHandler class, DataSourceAdd or JMSTopicAdd for example, that I was able to use in conjunction with OperationContext.addStep().

       

      I would like to know if it possible to do what I am doing with an extension or some other means.

       

      Any information on this would be appreciated.

        • 1. Re: Adding Logging Subsystem Resources At Boottime
          ctomc

          You shouldn't be "plugging in" in extension code of logging or any other subsystem for that matter.

           

          what you should do is generate list of add/remove/write-attribute operations and pass that to server to process.

          this way this operations will than get executed by subsystems you want them to execute in.

           

          pretty much same as you would do with cli operations or parsing ones.

          • 2. Re: Adding Logging Subsystem Resources At Boottime
            jfisherdev

            To clarify, you are suggesting going through the CLI/management API to add these resources? If I remember correctly, the management API cannot be used from extension code, so this would not be done from an extension. I would certainly prefer to use the management API. My question is can this be done at boottime/before application deployment?

             

            I have not used the CLI or management API extensively, so I am sure it has many capabilities I am not aware of and could possibly provide better solutions to some of the problems I am facing with deployment.

            • 3. Re: Adding Logging Subsystem Resources At Boottime
              jamezp

              If you're writing the extension you can definitely use the ModelControllerClient from the extension. At least I can't think of an issue using it. You'll more than likely need to do this in a DeploymentUnitProcessor that runs before the logging subsystem processors. You'll want it to run before because you should probably use logging-profiles so configurations won't clash and each deployment would get it's own log context.

               

              You'll also want to make sure to remove the configuration during undeploy. This is another reason to use a logging-profile so you don't remove a handler that two deployments are using.

               

              To get the logging profile approach to work you'll likely have to manipulate the MANIFEST.MF to add a Logging-Profile entry as well. It's not really writable so this is something that might need to be done at build time of the deployment. At least writing the manifest entry at build time would be the safest approach.

               

              manually mapping log4j.xml to logging subsystem XML configuration is not something I want to attempt

               

              This is going to have to be done to a point. All log4j appenders will need to be mapped to a custom-handler. I don't believe layouts can be mapped to the subsystem model at all unless you write you're own custom-formatter mapping and place that in a module.

               

              In theory this should work. I'd advise against hacking into the logging subsystem internals. Using operations or CLI commands should be much safer. Let me know if you have further questions on this.

               

              --

              James R. Perkins

              • 4. Re: Adding Logging Subsystem Resources At Boottime
                ctomc

                James Perkins wrote:

                 

                If you're writing the extension you can definitely use the ModelControllerClient from the extension. At least I can't think of an issue using it. You'll more than likely need to do this in a DeploymentUnitProcessor that runs before the logging subsystem processors. You'll want it to run before because you should probably use logging-profiles so configurations won't clash and each deployment would get it's own log context.

                 

                Rather than using ModelControllerClient, I would go via OperationContext#addStep

                 

                Given that you already used this, I think it will be even easier for your.

                addStep() is basically just internal call to say, execute this mgmt operation as well.

                 

                so when add steps you just tell other subsystems what to do.

                so if you would do something like

                ModelNode op = new ModelNode();

                op.get("address").set(PathAddress.pathAddress("subsystem", "logging").append("logger", "mylogger").toModelNode(); //or whatever other address

                op.get("operation").set("add");

                op.get("level","trace"); //and other attributes

                context.addStep(op ,(c, o) -> {} , OperationContext.Stage.MODEL);//add it to queue for execution

                 

                • 5. Re: Adding Logging Subsystem Resources At Boottime
                  jamezp

                  Well this will have to be done in a DUP because it scans the deployment for a logging configuration file. In this case a ModelControllerClient will be the only way to update the configuration.

                   

                  --

                  James R. Perkins

                  • 6. Re: Adding Logging Subsystem Resources At Boottime
                    jfisherdev

                    Thank you for responding.

                     

                    My plan was to use OperationStepHandlers for configuring the resources, but the operation handlers for the logging subsystem are not exposed, which maybe is a good thing as it sounds like these generally should not be visible outside of the extension module. Is there one I am not aware of that I should be using?

                    • 7. Re: Adding Logging Subsystem Resources At Boottime
                      jamezp

                      As I understand you want the log4j.xml be part of the deployment correct? If so you'll need to use a DeploymentUnitProcessor to read the configuration and execute operations to configure the logging subsystem. An OperationStepHandler will not work for you in this case.

                       

                      Yes the OSH's from the logging subsystem should not be exposed or used outside of the logging subsystem.

                       

                      --

                      James R. Perkins

                      • 8. Re: Adding Logging Subsystem Resources At Boottime
                        jfisherdev

                        Thank you for responding.

                         

                        I should clarify my statement about not wanting to map log4j XML configuration to logging subsystem XML configuration. One option I was considering was transforming the XML configuration in the standalone.xml before starting the server. I would much rather use the CLI/management API to do this.

                         

                        I think when I wrote my extension for loading data sources at boottime I initially tried to use a ModelControllerClient, but I could not get it to work. I am not sure how one would do this, because the only usages of the ModelControllerClient I can recall seeing were for a RemotingModelControllerClient created via the ModelControllerClient factory method(s). If I remember correctly, this cannot be used at boottime.

                         

                        To be clear, my plan is not to package the log4j.xml configuration with the deployment artifacts [otherwise I would probably be using the use-deployment-config option]. Instead, I want to place these files in a configuration directory that can the server can access at boottime. I don't think using a deployment processor will work, because I want the logging configuration to be registered with the management model and changes to the management model are not allowed at deployment time if I remember correctly.

                         

                        Ultimately what I am trying to do here [and with data sources and JMS destinations] is have my resource configuration defined outside of the standalone.xml file and create the management resources at boottime/before application deployment using the management API.

                        • 9. Re: Adding Logging Subsystem Resources At Boottime
                          jamezp

                          Hmm.. ..okay. So you want to have a log4j.xml to be read at boot time and update the logging subsystem configuration. Is there a reason not to just use CLI or management operations to just update the configuration without having some kind of external file? Not everything will map correctly without some kind of custom module that maps log4j types to JUL types. You can use a custom-handler to map a log4j appender to a java.util.logging.Handler, but that won't work for layouts.

                           

                          In theory this is possible using OperationStepHandlers, but it seems more complicated to me than just using a CLI script or something similar. Since the boot is parallel too you'd need to somehow make sure the logging subsystem boots before your extension. If the log4j.xml format is important to you, you could just write a utility that will map the XML to CLI commands and then just execute the configuration commands.

                           

                          Maybe I just don't understand the use-case, but it seems like a lot of effort for little gain.

                           

                          --

                          James R. Perkins

                          • 10. Re: Adding Logging Subsystem Resources At Boottime
                            jfisherdev

                            Let me contextualize this by saying I am migrating to WildFly 9.0.2 from JBoss AS 4.2.2 and explaining what I am dealing with. There is much I have yet to learn about the CLI, which I feel like I need to learn more about now.

                             

                            As I said in my original post, the set of apps deployed to the WildFly server varies, so we only want to configure application-specific logging resources and other resources based on what will be deployed to the server. The applications with custom logging configuration currently express this via log4j.xml configuration. The log4j.xml configuration is NOT included in the deployment archive. Whether we continue to do this or use some other configuration format, it will ultimately need to be converted to the appropriate format for logging subsystem configuration.

                             

                            In JBoss 4.2.2, these log4j.xml resources are merged into the main log4j.xml before the server is started via XML transformations. This is something I want to get away from, which is why I am looking to the WildFly management API/CLI. On JBoss and WildFly, for the time being, we take a "cold" deployment approach where the deployment archives are pushed to the deployment directory before the server is started. This means that startup and application deployment are triggered by a single command, which means that resources that applications depend on, such as data sources and logging configuration, need to be configured before application deployment.

                             

                            For data sources, I ultimately solved this problem with an extension that runs at boottime and configures the resources for the data sources using externally defined configuration. It allowed me to define the configuration externally and separate from the standalone.xml, which is most important to me, while also retaining the cold deployment approach, which is not as important but would be a nice option. In retrospect, while the extension approach worked, I am not sure that was the best one, and I may reevaluate this in the future. I was thinking of doing something similar with logging, where I would scan an external app configuration directory structure for log4j.xml files, translate the configuration to logging subsystem configuration, and call the logging subsystem management operations to register the configuration resources. I think these same basic steps could be used to do this, but not from an extension, as that will not work, maybe using the CLI, but I am not sure.

                             

                            I would also like to eventually get away from using the deployment scanner, but I am not sure if the CLI can be used to deploy applications as a group, as there are many using the jboss-all.xml descriptor to ensure proper deployment ordering. Otherwise, this would need to be considered properly if this were done as a sequence of single deployment operations.

                             

                            I am open to the idea [in fact, I like it] of breaking up startup, resource configuration, and application deployment into distinct steps/stages using CLI operations/scripting; however, I would need some direction on the matter. The only requirement is that the resource configuration and application deployment steps would be triggered automatically. This would give us what the cold deployment approach was used to provide and what we ultimately want, which is the ability to start the server, configure resources, and to deploy the applications with a single "start" command.

                            • 11. Re: Adding Logging Subsystem Resources At Boottime
                              jamezp

                              Thanks for the very detailed description of what you're doing. It's very helpful to understand a use-case like this. Instead of using custom extensions I would suggest using CLI or at least management operations. It will be easier moving forward with an approach like that. I'll give you a couple options below to describe how you might be able to do this.

                               

                              Admin Only

                              The first approach would be to use --admin-only mode when starting the server. This works for both standalone and domain and allows you do execute any management operations, including deploy operations, with the runtime of the server actually being started. At the end of your configuration script you can use a :reload(admin-only=false), false is the default BTW, which will reload the server and execute the runtime stages which include deploying your applications.

                               

                              With admin-only mode the management port is also up and listening so you could also use operations rather than CLI. Really could just start a WildFly process in admin-only, execute your operations and then shut it down. Here's a simple example:

                              final StandaloneCommandBuilder commandBuilder = StandaloneCommandBuilder.of(Paths.get("/home/jperkins/servers/wildfly-10.1.0.Final"))
                                      .setAdminOnly();
                              final Process process = Launcher.of(commandBuilder)
                                      .inherit()
                                      .launch();
                              try (final ModelControllerClient client = ModelControllerClient.Factory.create(InetAddress.getLocalHost(), 9990)) {
                                  ServerHelper.waitForStandalone(client, 10);
                                  final ModelNode op = Operations.createReadResourceOperation(Operations.createAddress("subsystem", "logging"));
                                  final ModelNode result = client.execute(op);
                                  if (!Operations.isSuccessfulOutcome(result)) {
                                      throw new RuntimeException(String.format("Failed to execute operation %s:%n%s", op, Operations.getFailureDescription(result).asString()));
                                  }
                                  System.out.println(Operations.readResult(result));
                              } finally {
                                  process.destroy();
                              }
                              process.waitFor();
                              

                               

                              The launcher API is part of WildFly Core, though it's not really documented at this point. The ServerHelper piece is part of a core split with the maven plugin.

                               

                              Offline CLI

                              The second approach would be offline CLI. In WildFly 10 this works for both standalone and domain servers. In WildFly 9 it only works for standalone. The advantage to this approach is it doesn't open up any management ports. Everything is embedded and you can configure your server with CLI commands. Like with admin-only you can use the reload command to reload WildFly and the runtime will startup. The process will still be embedded in the CLI process though.

                               

                              You can start the offline CLI process withing a CLI script as well.

                              embed-server
                              /subsystem=logging:read-resource
                              stop-embedded-server
                              

                               

                              You can then just pass the script to CLI and it will execute all the CLI commands for you.

                              $JBOSS_HOME/bin/jboss-cli.sh --file=configure.cli
                              

                               

                               

                              Either of these approaches will work for you and are likely better than using a custom extension. Either way you need to translate your current configuration into some kind of management operation. Hopefully these descriptions of the options help and let me know if you have more questions or want more details on either approach.

                               

                              --

                              James R. Perkins

                              1 of 1 people found this helpful
                              • 12. Re: Adding Logging Subsystem Resources At Boottime
                                jfisherdev

                                Thank you for sharing this information.

                                 

                                While I will need to do further research and I will likely have some questions, I think the provided CLI/Management API capabilities you describe will allow me to achieve what I want to.

                                 

                                I will consider my question(s) for this thread answered. Any further questions will be the subject of another thread.

                                 

                                Again, thank you to both you and to ctomc for your prompt and helpful responses to my questions.

                                • 13. Re: Adding Logging Subsystem Resources At Boottime
                                  jamezp

                                  Excellent. Feel free to visit us on HipChat too.

                                  --

                                  James R. Perkins