-
1. Re: How do we start adding jdk 5 based features
I don't llike the way Bill has done this, essentially creating two parallel
codebases for the java5 and pre-java5 builds.
This seems error prone and laborious to me.
I quite like the idea of using something like retroweaver, developing
everything with java5 and weaving in replacements that work with jdk1.3.
+ Only one codebase
+ Take advantage of java5 features
+ The process is simple to understand
+ Removes the need for annotationc (now included in the retroweave)
+ java5's lint picks up a lot more problems
- Do all developers have java5 development support? e.g. eclipse does not have
an official release with java5 support
- Requires a post compile
- Retroweaver is incomplete - no support for annotations, java.util.concurrent
- Requires runtime binaries for java5 replacements
- Need to spend time completing Retroweaver for the java5 features we want
to use
- Some java5 features are not replaceable in older versions, e.g. the jvm mbeans
(but these could be "stubbed out" or replaced with crippled versions
if the usage was optional)
Retroweaver has an interesting validation routine that basically
checks for class usage that isn't supported on earlier jdk releases.
(I think it should also check for methods as well). -
2. Re: How do we start adding jdk 5 based features
Oh no, I have become an American :-)
-laborious (US)
+labourious (UK) -
3. Re: How do we start adding jdk 5 based features
We are a virus.
Ok, so let's try the retroweaver approach as I'm definitely not keen on duplicate codebases either. In terms of jdk 5 specific mbeans I guess we need a jdk5forjdk1.42 module for these classes.
There is a backport of the java.util.concurrent for jdk 1.4.x:
http://altair.cs.oswego.edu/pipermail/concurrency-interest/2004-September/001035.html
Hello everybody,
I am happy to announce availability of a backport of
java.util.concurrent API to Java 1.4. The backport is based on sources
from JSR 166 CVS repository (September 2004), and dl.util.concurrent
1.3.4. The backport is nearly complete; unsupported functionality is
limited mostly to the following classes: 1) requiring explicit JVM
support (e.g. nanosecond clock), 2) some non-essential functions
described as "designed primarily for use in monitoring in system state,
not for synchronization control", 3) functionality that would affect
performance, or 4) functionality that would require development of
substantial amount of new code.
The backport will probably go into the Emory Utilities package
(http://www.mathcs.emory.edu/dcl/util/). However, the interest expressed
earlier in this list motivates releasing the library as soon as
possible. Hence I am hereby releasing the "still warm" version. The
purpose of this library is to enable gradual transition from Java 1.4 to
5.0: the library allows to develop concurrent applications for Java 1.4
that will work with Java 5.0 by simply changing package names.
So here it is:
http://www.mathcs.emory.edu/dcl/util/util-concurrent.jar (binaries)
http://www.mathcs.emory.edu/dcl/util/util-concurrent-src.jar (source code)
http://www.mathcs.emory.edu/dcl/util/util-concurrent-doc.jar (javadoc)
This is NOT an emulation: it is adaptation of JSR 166 and dl.u.c. code
to Java 1.4. Performance levels of this backport are comparable to dl.u.c.
Overview of supported functionality:
all JSR 166 executors, utilities, and everything related (thread pools,
FutureTask, scheduled tasks and executors, etc)
Locks: ReentrantLock, Semaphore, ReentrantReadWriteLock (see remarks
below), Conditions
Queues: synchronous, array, linked, delay, and priority queues
Atomics: everything except reflection-based updaters
Other concurrency utils: CountDownLatch, CyclicBarrier
Collections: ConcurrentHashMap, CopyOnWriteArrayList, CopyOnWriteArraySet
As far as stability and reliability is considered: the library passes
all of attempted 941 tests from tck test package designed for
java.util.concurrent (the tests of unsupported funcionality were
skipped). Occasionally, there are failures of "condition timeout" test
cases of boolean Condition.await(timeout) and boolean
Condition.awaitUntil(Date), but it is because the tests assume stronger
semantics than required by the specification (see comments on Condition
below).
The following classes were unit-tested:
AbstractExecutorService
AbstractQueue
ArrayBlockingQueue
AtomicBoolean
Atomic[Integer,Long]Array
Atomic[Integer,Long]
AtomicMarkableReference
AtomicReferenceArray
AtomicReference
AtomicStampedReference
ConcurrentHashMap
CopyOnWriteArray[List,Set]
CountDownLatch
CyclicBarrier
DelayQueue
Exchanger
Executors
ExecutorCompletionService
FutureTask
LinkedBlockingQueue
PriorityBlockingQueue
PriorityQueue
ReentrantLock
ReentrantReadWriteLock
ScheduledExecutor
Semaphore
SynchronousQueue
ThreadLocal
ThreadPoolExecutor
TimeUnit
It is also important to stress out that vast proportion of this library
is based on source code from JSR 166 and dl.util.concurrent, both very
well tested. Whenever possible, the JSR 166 code was used. In cases when
it was infeasible (e.g. the JSR 166 code had strong dependencies on
native JVM support), the dl.util.concurrent code was adapted. The new
code was introduced only when absolutely neccessary, e.g. to make
dl.u.c. code conforming to JSR 166 interfaces and semantics.
However, as any software, it may still contain bugs. I will conduct more
tests, but I would like to welcome everybody to try it out, report any
issues, or contribute source code. There is still a couple of things
missing (e.g. JSR 166 functionality that didn't make it to
java.util.concurrent yet, or some of things listed below), and I welcome
collaboration. The library is released to public domain and can be used
without acknowledgement and for any purpose.
While substantial effort was made to ensure reliability and conformance
to specifications, the library is provided AS IS and without any express
or implied warranty.
-------------------------------------------------------------------------------------------------------------------------
The detailed explanation of what JSR 166 functionality is not supported:
Condition:
long awaitNanos(long nanosTimeout) is not supported, since it cannot
accurately report remaining times with nanosecond precision.
boolean await(timeout) and boolean awaitUntil(Date), called on
conditions obtained from locks, may sometimes wake up spuriously. This
is allowed by the Condition specification. However, it causes them to
occassionally fail tck unit tests. Maybe the tests are too rigorous, or
maybe the semantics of conditions returned from lock classes in
java.util.concurrent are stronger than required by the specification.
Nevertheless, this implementation still conforms to the specification.
ReentrantLock:
the following monitoring methods are not supported: boolean
hasWaiters(Condition), int getWaitQueueLength(Condition), Collection
getWaitingThreads(Condition).
the following monitoring methods are supported only for fair locks:
boolean hasQueuedThreads(), int getQueueLength(), Collection
getQueuedThreads(), boolean isQueued().
ReentrantReadWriteLock:
this implementation is based on dl.u.c.
ReentrantWriterPreferenceReadWriteLock, and thus is a slight departure
from java.util.concurrent that does not specify acquisition order but
allows to enable/disable fairness. This implementation does not have a
single-parameter constructor allowing to specify fairness policy; it
always behaves like writer-preference lock with no fairness guarantees.
Bacause of these characteristics, this class is compliant with JSR 166
specification of non-fair reentrant read-write locks, while the exact
semantics of fair locks are not supported (and the appropriate
constructor is missing).
Also, the following instrumentation and status methods are not
supported: Collection getQueuedWriterThreads(), Collection
getQueuedReaderThreads(), boolean hasQueuedThreads(), boolean
hasQueuedThread(Thread), Collection getQueuedThreads(), boolean
hasWaiters(Condition), int getWaitQueueLength(Condition), Collection
getWaitingThreads(Condition).
Semaphore:
Atomic multi-acquires: tryAcquire(int permits) and tryAcquire(int
permits, long timeout, TimeUnit unit) are not supported.
The following platform-level functionality is unsupported:
System.nanoTime(), UncaughtExceptionHandlers
The following low-level concurrency classes are not supported:
LockSupport, AbstractQueuedSynchronizer.
The following "atomic" utilities are not supported:
Atomic[Integer,Long,Reference]FieldUpdater.
The following collection classes are not supported: LinkedList,
ConcurrentLinkedQueue.
Note on nanosecond precision: although nanoTime() is not supported, the
library strives to honor nanosecond timeouts, if such are requested, by
using two-parameter variant of Object.wait(). Note, however, that most
JVMs prior to 5.0 will round up the timeout to full milliseconds anyway.
I kindly ask members of JSR 166 expert group to straighten out any
inaccuracies that I may have made in the above description.
Regards,
Dawid Kurzyniec -
4. Re: How do we start adding jdk 5 based features
I'd like to get Bill's and Chiba's feedback first,
since they have the most experience with weaving.
Remember we have to support these binaries in production environments.
I even jokingly suggested to Bill that we write our own compiler,
but this might actually be possible using the eclipse compiler.
It is designed to be pluggable, e.g. aspectj sits on top of it.
"AspectJ 1.1 is built on Eclipse/JDT's Java compiler but is distributed standalone and can run standalone."
This would allow compiles to specify a source target of 1.5, but a binary target of 1.3
and do everything in one step. But that is probably too radical for the short term.
Mostly what we want (annotations, generics?, enums, new loop) is binary
compatibility, rather than the 1.5 apis.
Bill already has the annotations stuff with annotationc, retroweaver has the
others or if it doesn't it should be easy to fix. -
5. Re: How do we start adding jdk 5 based features
a) The java.util.concurrent lives in a different package. I don't see the point of using it unless we recompile it with a package name change. Also, AFAIK, there are tiny API changes as well. We will need to make sure we don't use those API changes. Concurrent makes heavy use of generics. We need to make sure that
b) If we're gonna do the retroweaver approach, we need to make sure that JDK 1.4 compiled classes run on JDK 1.5. This means annotations, enums, etc...
c) It may be much easier, and less error prone, less of a headache, to just require JDK 5 for JBoss 5, aspect library, etc... J2EE 1.5 will require JDK 5.
Bill -
6. Re: How do we start adding jdk 5 based features
One issue I came across is that retroweaving does not work incrementally.
If you compile against a binary that has already been retroweaved it does not work.
e.g. suppose you have a method
public void doSomething(java.lang.StringBuilder);
This is retroweaved to
public void doSomething(java.lang.StringBuffer);
or
public void doSomething(org.jboss.lang.StringBuilder);
If somebody then tries to use that method, it won't compile without
support in the compiler for the knowledge of retroweaving:
doSomething(new java.lang.StringBuilder());
Error: Cannot locate method
Another solution is to only retroweave when you copy the jars to
buiild/output.
This would still mean the javadoc is misleading for people
compiling against the jboss binaries. -
7. Re: How do we start adding jdk 5 based features
c) It may be much easier, and less error prone, less of a headache, to just require JDK 5 for JBoss 5, aspect library, etc... J2EE 1.5 will require JDK 5.
There are two problems with that solution.
1) Standalone projects need to run on JBoss3/4 and so cannot have JDK5
specific features.
2) For non standalone projects it is not trivial to backport changes. -
8. Re: How do we start adding jdk 5 based features
Just to clarify, I don't see a problem to require JBoss to be developed with Java5
(source level), only that the binaries produced should run on jdk1.3. -
9. Re: How do we start adding jdk 5 based features
jdk1.3 runtime comatbility is generally too strict in my view as this is an obsolete platform with numerous bugs and limitations. Its a requirement for 3.2.x, but not 4.x. If a module should have portability across all three branches, then perhaps this can be enforced by setting the source.target to 1.3?
Simplying requiring jdk5 for head its runtime is also not an option as you state there has to be some thought to maintence between the 4.x and head branches.
The concurrent backport classes live in the edu.emory.mathcs.backport.java.util namespace. We should update retroweaver to reference those classes when mapping from the jdk5 versions. In terms of avoiding non-usable jdk1.4 features, the retroweaver tool should simply fail to convert those and fail with an error to flag their use.
Only retroweaving on the final build and when explicitly requested seems like what we should do. The overhead of xdoclet has demonstrated that doing excessive traversals through the code is just too time consuming. -
10. Re: How do we start adding jdk 5 based features
There does need to be some sanity check the developer
or cruise control can perform before the final release, to make
sure the java5 feature is supported by the retroweave.
With the new build, adding a retroweave target should be enough.
This would then work its way through the jars going into the release
and make sure they have weaved and validated for supported features.
Getting cruise control to retroweave and run the testsuite against jdk1.4
should catch errors where developers aren't testing this scenario themselves
(most of them). -
11. Re: How do we start adding jdk 5 based features
Defintely. If retroweave is not run during compilation it must be run as part of the daily compilation via a seperate target to identify backward compatibility problems. It just another compiler to add to jdk matrix runs we have now to identity problem.