11 Replies Latest reply on Feb 15, 2005 4:04 PM by Scott Stark

    How do we start adding jdk 5 based features

    Scott Stark Master

      So some projects have some conditional compiles based on whether or not a jdk 5 compiler is in use, but do we have a general template for how to add jdk 5 specific features into the legacy build as well as the new build?

        • 1. Re: How do we start adding jdk 5 based features
          Adrian Brock Master

          I don't llike the way Bill has done this, essentially creating two parallel
          codebases for the java5 and pre-java5 builds.
          This seems error prone and laborious to me.

          I quite like the idea of using something like retroweaver, developing
          everything with java5 and weaving in replacements that work with jdk1.3.

          + Only one codebase
          + Take advantage of java5 features
          + The process is simple to understand
          + Removes the need for annotationc (now included in the retroweave)
          + java5's lint picks up a lot more problems

          - Do all developers have java5 development support? e.g. eclipse does not have
          an official release with java5 support
          - Requires a post compile
          - Retroweaver is incomplete - no support for annotations, java.util.concurrent
          - Requires runtime binaries for java5 replacements
          - Need to spend time completing Retroweaver for the java5 features we want
          to use
          - Some java5 features are not replaceable in older versions, e.g. the jvm mbeans
          (but these could be "stubbed out" or replaced with crippled versions
          if the usage was optional)

          Retroweaver has an interesting validation routine that basically
          checks for class usage that isn't supported on earlier jdk releases.
          (I think it should also check for methods as well).

          • 2. Re: How do we start adding jdk 5 based features
            Adrian Brock Master

            Oh no, I have become an American :-)

            -laborious (US)
            +labourious (UK)

            • 3. Re: How do we start adding jdk 5 based features
              Scott Stark Master

              We are a virus.

              Ok, so let's try the retroweaver approach as I'm definitely not keen on duplicate codebases either. In terms of jdk 5 specific mbeans I guess we need a jdk5forjdk1.42 module for these classes.

              There is a backport of the java.util.concurrent for jdk 1.4.x:
              http://altair.cs.oswego.edu/pipermail/concurrency-interest/2004-September/001035.html


              Hello everybody,

              I am happy to announce availability of a backport of
              java.util.concurrent API to Java 1.4. The backport is based on sources
              from JSR 166 CVS repository (September 2004), and dl.util.concurrent
              1.3.4. The backport is nearly complete; unsupported functionality is
              limited mostly to the following classes: 1) requiring explicit JVM
              support (e.g. nanosecond clock), 2) some non-essential functions
              described as "designed primarily for use in monitoring in system state,
              not for synchronization control", 3) functionality that would affect
              performance, or 4) functionality that would require development of
              substantial amount of new code.

              The backport will probably go into the Emory Utilities package
              (http://www.mathcs.emory.edu/dcl/util/). However, the interest expressed
              earlier in this list motivates releasing the library as soon as
              possible. Hence I am hereby releasing the "still warm" version. The
              purpose of this library is to enable gradual transition from Java 1.4 to
              5.0: the library allows to develop concurrent applications for Java 1.4
              that will work with Java 5.0 by simply changing package names.

              So here it is:
              http://www.mathcs.emory.edu/dcl/util/util-concurrent.jar (binaries)
              http://www.mathcs.emory.edu/dcl/util/util-concurrent-src.jar (source code)
              http://www.mathcs.emory.edu/dcl/util/util-concurrent-doc.jar (javadoc)

              This is NOT an emulation: it is adaptation of JSR 166 and dl.u.c. code
              to Java 1.4. Performance levels of this backport are comparable to dl.u.c.

              Overview of supported functionality:
              all JSR 166 executors, utilities, and everything related (thread pools,
              FutureTask, scheduled tasks and executors, etc)
              Locks: ReentrantLock, Semaphore, ReentrantReadWriteLock (see remarks
              below), Conditions
              Queues: synchronous, array, linked, delay, and priority queues
              Atomics: everything except reflection-based updaters
              Other concurrency utils: CountDownLatch, CyclicBarrier
              Collections: ConcurrentHashMap, CopyOnWriteArrayList, CopyOnWriteArraySet

              As far as stability and reliability is considered: the library passes
              all of attempted 941 tests from tck test package designed for
              java.util.concurrent (the tests of unsupported funcionality were
              skipped). Occasionally, there are failures of "condition timeout" test
              cases of boolean Condition.await(timeout) and boolean
              Condition.awaitUntil(Date), but it is because the tests assume stronger
              semantics than required by the specification (see comments on Condition
              below).

              The following classes were unit-tested:
              AbstractExecutorService
              AbstractQueue
              ArrayBlockingQueue
              AtomicBoolean
              Atomic[Integer,Long]Array
              Atomic[Integer,Long]
              AtomicMarkableReference
              AtomicReferenceArray
              AtomicReference
              AtomicStampedReference
              ConcurrentHashMap
              CopyOnWriteArray[List,Set]
              CountDownLatch
              CyclicBarrier
              DelayQueue
              Exchanger
              Executors
              ExecutorCompletionService
              FutureTask
              LinkedBlockingQueue
              PriorityBlockingQueue
              PriorityQueue
              ReentrantLock
              ReentrantReadWriteLock
              ScheduledExecutor
              Semaphore
              SynchronousQueue
              ThreadLocal
              ThreadPoolExecutor
              TimeUnit

              It is also important to stress out that vast proportion of this library
              is based on source code from JSR 166 and dl.util.concurrent, both very
              well tested. Whenever possible, the JSR 166 code was used. In cases when
              it was infeasible (e.g. the JSR 166 code had strong dependencies on
              native JVM support), the dl.util.concurrent code was adapted. The new
              code was introduced only when absolutely neccessary, e.g. to make
              dl.u.c. code conforming to JSR 166 interfaces and semantics.

              However, as any software, it may still contain bugs. I will conduct more
              tests, but I would like to welcome everybody to try it out, report any
              issues, or contribute source code. There is still a couple of things
              missing (e.g. JSR 166 functionality that didn't make it to
              java.util.concurrent yet, or some of things listed below), and I welcome
              collaboration. The library is released to public domain and can be used
              without acknowledgement and for any purpose.

              While substantial effort was made to ensure reliability and conformance
              to specifications, the library is provided AS IS and without any express
              or implied warranty.


              -------------------------------------------------------------------------------------------------------------------------
              The detailed explanation of what JSR 166 functionality is not supported:

              Condition:
              long awaitNanos(long nanosTimeout) is not supported, since it cannot
              accurately report remaining times with nanosecond precision.
              boolean await(timeout) and boolean awaitUntil(Date), called on
              conditions obtained from locks, may sometimes wake up spuriously. This
              is allowed by the Condition specification. However, it causes them to
              occassionally fail tck unit tests. Maybe the tests are too rigorous, or
              maybe the semantics of conditions returned from lock classes in
              java.util.concurrent are stronger than required by the specification.
              Nevertheless, this implementation still conforms to the specification.

              ReentrantLock:
              the following monitoring methods are not supported: boolean
              hasWaiters(Condition), int getWaitQueueLength(Condition), Collection
              getWaitingThreads(Condition).
              the following monitoring methods are supported only for fair locks:
              boolean hasQueuedThreads(), int getQueueLength(), Collection
              getQueuedThreads(), boolean isQueued().

              ReentrantReadWriteLock:
              this implementation is based on dl.u.c.
              ReentrantWriterPreferenceReadWriteLock, and thus is a slight departure
              from java.util.concurrent that does not specify acquisition order but
              allows to enable/disable fairness. This implementation does not have a
              single-parameter constructor allowing to specify fairness policy; it
              always behaves like writer-preference lock with no fairness guarantees.
              Bacause of these characteristics, this class is compliant with JSR 166
              specification of non-fair reentrant read-write locks, while the exact
              semantics of fair locks are not supported (and the appropriate
              constructor is missing).
              Also, the following instrumentation and status methods are not
              supported: Collection getQueuedWriterThreads(), Collection
              getQueuedReaderThreads(), boolean hasQueuedThreads(), boolean
              hasQueuedThread(Thread), Collection getQueuedThreads(), boolean
              hasWaiters(Condition), int getWaitQueueLength(Condition), Collection
              getWaitingThreads(Condition).

              Semaphore:
              Atomic multi-acquires: tryAcquire(int permits) and tryAcquire(int
              permits, long timeout, TimeUnit unit) are not supported.

              The following platform-level functionality is unsupported:
              System.nanoTime(), UncaughtExceptionHandlers

              The following low-level concurrency classes are not supported:
              LockSupport, AbstractQueuedSynchronizer.

              The following "atomic" utilities are not supported:
              Atomic[Integer,Long,Reference]FieldUpdater.

              The following collection classes are not supported: LinkedList,
              ConcurrentLinkedQueue.

              Note on nanosecond precision: although nanoTime() is not supported, the
              library strives to honor nanosecond timeouts, if such are requested, by
              using two-parameter variant of Object.wait(). Note, however, that most
              JVMs prior to 5.0 will round up the timeout to full milliseconds anyway.


              I kindly ask members of JSR 166 expert group to straighten out any
              inaccuracies that I may have made in the above description.

              Regards,
              Dawid Kurzyniec



              • 4. Re: How do we start adding jdk 5 based features
                Adrian Brock Master

                I'd like to get Bill's and Chiba's feedback first,
                since they have the most experience with weaving.

                Remember we have to support these binaries in production environments.

                I even jokingly suggested to Bill that we write our own compiler,
                but this might actually be possible using the eclipse compiler.
                It is designed to be pluggable, e.g. aspectj sits on top of it.

                "AspectJ 1.1 is built on Eclipse/JDT's Java compiler but is distributed standalone and can run standalone."

                This would allow compiles to specify a source target of 1.5, but a binary target of 1.3
                and do everything in one step. But that is probably too radical for the short term.

                Mostly what we want (annotations, generics?, enums, new loop) is binary
                compatibility, rather than the 1.5 apis.

                Bill already has the annotations stuff with annotationc, retroweaver has the
                others or if it doesn't it should be easy to fix.

                • 5. Re: How do we start adding jdk 5 based features
                  Bill Burke Master

                  a) The java.util.concurrent lives in a different package. I don't see the point of using it unless we recompile it with a package name change. Also, AFAIK, there are tiny API changes as well. We will need to make sure we don't use those API changes. Concurrent makes heavy use of generics. We need to make sure that

                  b) If we're gonna do the retroweaver approach, we need to make sure that JDK 1.4 compiled classes run on JDK 1.5. This means annotations, enums, etc...

                  c) It may be much easier, and less error prone, less of a headache, to just require JDK 5 for JBoss 5, aspect library, etc... J2EE 1.5 will require JDK 5.

                  Bill

                  • 6. Re: How do we start adding jdk 5 based features
                    Adrian Brock Master

                    One issue I came across is that retroweaving does not work incrementally.

                    If you compile against a binary that has already been retroweaved it does not work.

                    e.g. suppose you have a method
                    public void doSomething(java.lang.StringBuilder);
                    This is retroweaved to
                    public void doSomething(java.lang.StringBuffer);
                    or
                    public void doSomething(org.jboss.lang.StringBuilder);

                    If somebody then tries to use that method, it won't compile without
                    support in the compiler for the knowledge of retroweaving:
                    doSomething(new java.lang.StringBuilder());
                    Error: Cannot locate method

                    Another solution is to only retroweave when you copy the jars to
                    buiild/output.

                    This would still mean the javadoc is misleading for people
                    compiling against the jboss binaries.

                    • 7. Re: How do we start adding jdk 5 based features
                      Adrian Brock Master

                       


                      c) It may be much easier, and less error prone, less of a headache, to just require JDK 5 for JBoss 5, aspect library, etc... J2EE 1.5 will require JDK 5.


                      There are two problems with that solution.
                      1) Standalone projects need to run on JBoss3/4 and so cannot have JDK5
                      specific features.
                      2) For non standalone projects it is not trivial to backport changes.

                      • 8. Re: How do we start adding jdk 5 based features
                        Adrian Brock Master

                        Just to clarify, I don't see a problem to require JBoss to be developed with Java5
                        (source level), only that the binaries produced should run on jdk1.3.

                        • 9. Re: How do we start adding jdk 5 based features
                          Scott Stark Master

                          jdk1.3 runtime comatbility is generally too strict in my view as this is an obsolete platform with numerous bugs and limitations. Its a requirement for 3.2.x, but not 4.x. If a module should have portability across all three branches, then perhaps this can be enforced by setting the source.target to 1.3?

                          Simplying requiring jdk5 for head its runtime is also not an option as you state there has to be some thought to maintence between the 4.x and head branches.

                          The concurrent backport classes live in the edu.emory.mathcs.backport.java.util namespace. We should update retroweaver to reference those classes when mapping from the jdk5 versions. In terms of avoiding non-usable jdk1.4 features, the retroweaver tool should simply fail to convert those and fail with an error to flag their use.

                          Only retroweaving on the final build and when explicitly requested seems like what we should do. The overhead of xdoclet has demonstrated that doing excessive traversals through the code is just too time consuming.

                          • 10. Re: How do we start adding jdk 5 based features
                            Adrian Brock Master

                            There does need to be some sanity check the developer
                            or cruise control can perform before the final release, to make
                            sure the java5 feature is supported by the retroweave.

                            With the new build, adding a retroweave target should be enough.
                            This would then work its way through the jars going into the release
                            and make sure they have weaved and validated for supported features.

                            Getting cruise control to retroweave and run the testsuite against jdk1.4
                            should catch errors where developers aren't testing this scenario themselves
                            (most of them).

                            • 11. Re: How do we start adding jdk 5 based features
                              Scott Stark Master

                              Defintely. If retroweave is not run during compilation it must be run as part of the daily compilation via a seperate target to identify backward compatibility problems. It just another compiler to add to jdk matrix runs we have now to identity problem.