2 Replies Latest reply on Aug 12, 2010 5:21 PM by brian.stansberry

    Slogging through Logging

    dmlloyd

      To my deep sadness, and complete lack of surprise, nobody has volunteered to take up logging integration while I was off working on Modules, so here I am again.

       

      There are a bunch of logging topics to cover but I think we need to cover them now, since lack of good logging is already making life difficult on the front lines.

       

      Domain Environment

       

      Console Logging

       

      We don't have a strategy for console logging in the domain environment.  I think that whatever else may be decided, ultimately people will want to see important messages logged to the console, whether they have 5 servers or 500.  If I'm right, then this implies that we need an aggregation strategy.  We don't want to intermix a bunch of different formatting rules for example.  Here's what I'm thinking:

       

      1. At the domain level, we have a "virtual console" handler type which uses the process manager messaging scheme to pass log messages on to the server manager using the PM API.  This handler would have a very simple configuration consisting only of a level and optional filters.
      2. At the server manager level, we have a logging configuration which consists of either configuration at the host.xml level (which implies a bit of a can of worms relating to subsystems/profiles in host.xml), or just keeping the simple logging.properties from bootstrap.  The server manager also receives and logs incoming messages from the server (logmanager already has all the stuff necessary to do this).

       

      That's pretty much it.  The message information can simply be serialized between the two systems.  Because of the potential for a lot of messages, on the server side the level should be restricted as much as possible (WARN or higher I'm thinking).  Also, the category name should be transformed or supplemented somehow with the server name to make filtering easier to accomplish on the server manager.

       

      Bootstrap Logging

       

      Right now everything that boots up via modules is using the same logger.properties to bootstrap, which is ugly at best.  Ideally I'd like to see this happen:

       

      1. The server manager (SM) process is bootstrapped in one of the following ways:
        1. It uses a logging.properties for boot logging (which is a very short window) until it reads its host.xml and derives a logging configuration from it.
        2. Alternatively, it could use a logging.properties which exists solely for use of the SM in the configuration area instead.  This would eliminate the two-phased logging thing which I think is a pain for users.
      2. The server's logging is bootstrapped in one of the following ways:
        1. The SM reads and understands the logger config from domain.xml, and passes it in at bootup, which immediately configures the server's logger environment in accordance with domain.xml.
        2. Each server has its own logging.properties which is used to perform bootstrap logging config until the logger subsystem is activated using the regular subsystem activation we use for everything else.
        3. Same as #2 only all servers share a common logging.properties.

      For SM, I think option #2 is probably most practical.  It'd only be a problem if we want to be able to manage the SM logging environment, which I view as non-required.

       

      For Server bootstrapping, #1 is the best solution, but it can only work if the SM already understands the domain config.  If so it'd be easy to put in special code to assemble a message or object model or something for the server to consume from main().

       

      Standalone Environment

       

      I think standalone ought to work much like AS6: a logging.properties is read and its configuration is used until the server logging subsystem comes up.  No reason I can see to really deviate here.

       

      Per-Deployment Logging Configuration

       

      Back in the early days, if you wanted per-deployment logging config you basically had to include a log4j.jar in your deployment unit.  With AS6 we became somewhat more sophisticated by detecting a logging deployment descriptor (jboss-logging.xml) and using information in that descriptor to create and configure an independent logging "context" without including a separate logging implementation.

       

      In order to keep things managed under the domain model, it seems to me that we would have to put all this logging configuration into domain.xml.  So if you want per-app logging you'd have to add another log context into the domain and then somehow tell the deployment system that you want your deployment to be attached to the specific alternative logging context.  I think this would allow users to continue doing everything they do today; they may be opposed to the "how" though.  I don't know.

       

      Comments?

        • 1. Re: Slogging through Logging
          aloubyansky

          David Lloyd wrote:

          Domain Environment

           

          Console Logging

           

          We don't have a strategy for console logging in the domain environment.  I think that whatever else may be decided, ultimately people will want to see important messages logged to the console, whether they have 5 servers or 500.  If I'm right, then this implies that we need an aggregation strategy.  We don't want to intermix a bunch of different formatting rules for example.  Here's what I'm thinking:

           

          1. At the domain level, we have a "virtual console" handler type which uses the process manager messaging scheme to pass log messages on to the server manager using the PM API.  This handler would have a very simple configuration consisting only of a level and optional filters.
          2. At the server manager level, we have a logging configuration which consists of either configuration at the host.xml level (which implies a bit of a can of worms relating to subsystems/profiles in host.xml), or just keeping the simple logging.properties from bootstrap.  The server manager also receives and logs incoming messages from the server (logmanager already has all the stuff necessary to do this).

           

          That's pretty much it.  The message information can simply be serialized between the two systems.  Because of the potential for a lot of messages, on the server side the level should be restricted as much as possible (WARN or higher I'm thinking).  Also, the category name should be transformed or supplemented somehow with the server name to make filtering easier to accomplish on the server manager.

          When you are talking about the console you mean the PM's console, right? I am not sure logs from the SM and servers (even if trimmed/filtered) should be logged here. IMO, it would be more convenient if PM, SM and server processes had its own logging config and a separate output for it, each logging messages it's producing itself (i.e. w/o relay). Otherwise, why would we need same log messages duplicated in different logs?

           

          Bootstrap Logging

           

          Right now everything that boots up via modules is using the same logger.properties to bootstrap, which is ugly at best.  Ideally I'd like to see this happen:

           

          1. The server manager (SM) process is bootstrapped in one of the following ways:
            1. It uses a logging.properties for boot logging (which is a very short window) until it reads its host.xml and derives a logging configuration from it.
            2. Alternatively, it could use a logging.properties which exists solely for use of the SM in the configuration area instead.  This would eliminate the two-phased logging thing which I think is a pain for users.
          2. The server's logging is bootstrapped in one of the following ways:
            1. The SM reads and understands the logger config from domain.xml, and passes it in at bootup, which immediately configures the server's logger environment in accordance with domain.xml.
            2. Each server has its own logging.properties which is used to perform bootstrap logging config until the logger subsystem is activated using the regular subsystem activation we use for everything else.
            3. Same as #2 only all servers share a common logging.properties.

          For SM, I think option #2 is probably most practical.  It'd only be a problem if we want to be able to manage the SM logging environment, which I view as non-required.

           

          For Server bootstrapping, #1 is the best solution, but it can only work if the SM already understands the domain config.  If so it'd be easy to put in special code to assemble a message or object model or something for the server to consume from main().

          I agree: 2 for the SM and 1 for the server.

          Standalone Environment

           

          I think standalone ought to work much like AS6: a logging.properties is read and its configuration is used until the server logging subsystem comes up.  No reason I can see to really deviate here.

          Agreed. Should stay the same as AS6.

          Per-Deployment Logging Configuration

           

          Back in the early days, if you wanted per-deployment logging config you basically had to include a log4j.jar in your deployment unit.  With AS6 we became somewhat more sophisticated by detecting a logging deployment descriptor (jboss-logging.xml) and using information in that descriptor to create and configure an independent logging "context" without including a separate logging implementation.

           

          In order to keep things managed under the domain model, it seems to me that we would have to put all this logging configuration into domain.xml.  So if you want per-app logging you'd have to add another log context into the domain and then somehow tell the deployment system that you want your deployment to be attached to the specific alternative logging context.  I think this would allow users to continue doing everything they do today; they may be opposed to the "how" though.  I don't know.

           

          Comments?

          Per deployment logging in the domain maybe a good idea. Since deployments are already mentioned/managed there, maybe deployment element can also be extended to include/reference some logging config for it?

          • 2. Re: Slogging through Logging
            brian.stansberry

            On the console logging, I'm not sure how much benefit there is to trying to combine/filter. We really don't want anything from the servers at less than WARN level going to the PM. And how useful is a console that only shows WARN and above? Maybe just not worth the hassle. The PM itself can log the critical events related to starting/stopping processes.

             

            Agreed as well on 2 for the SM and 1 for the server. My only caveat on that is if

             

            1) we end up with a server-manager element in the host.xml, to configure the socket it uses for communication with the DC (or with the servers if stdio isn't used)

            2) a typical SM logging config is really trivial

            3) the SM can parse that part of host.xml and configure logging *very* early in bootstrap (i.e. before there's much need for logging)

             

            then it's simple for a user to understand having the SM logging config in host.xml.  But, having all 3 of those end up being true is quite iffy, so +1 for using a logging.properties.

             

            +1 on logging.properties for standalone.

             

            Re: per deployment logging, being able to configure that at the domain sounds really great, but we'd better get a lot of feedback before making that the only way.