In my previous entry, I wrote about how how there is a difference in behavior between Class.forName and ClassLoader.loadClass. Since then I wrote a simple (for class loaders :-) test case to demonstrate the difference.
When calling ClassLoader.loadClass to load a class, the initiating class loader delegates to another class loader which actually defines the class. The defined class is only added to the defining class loader's cache. The cache of the initiating class loader's cache is not altered. So if the initiating class loader delegates to a different defining class loader on a future request for the class, the class is always returned from the defining class loader to which the delegation occurred. This is of course how the ContextFinder from Eclipse is intended to work. ContextFinder is the initiating class loader which uses context from the call stack to select the right bundle class loader to delegate the actual class load request.
However, if Class.forName is used to call the initiating class loader, the behavior with respect to caching and the returned class is quite different. In this case, when the class is first defined, it is cached by the defining class loader as expected. But it is also cached by the initiating class loader which is not expected. Even more unusual and unexpected is that Class.forName, through its native implementation, seems to consult the initiating class loader's cache directly before calling loadClass on the initiating class loader which is the normal place where the class loader's cache is consulted (via ClassLoader.findLoadedClass). As a result, all calls to Class.forName to a initiating class loader always return the same class object (the first one loaded), even if the implementation of the initiating class loader does not define classes or directly consult its own cache.
The test case also showed that ClassLoader.loadClass always works as expected even when interleaved with calls to Class.forName.
If every one always used ClassLoader.loadClass to consult the Thread Context Class Loader (TCCL), then a ContextFinder style TCCL choice would work very well in OSGi (or any similar module system) . However a lot of code uses Class.forName to consult the TCCL which means that a ContextFinder style TCCL is not going to help those callers.
The test case also includes a test to see whether having Class.forName add the class object to the initiating class loader's cache would result in pinning the class in the heap after the class and its defining class loader became garbage collected. This would also be a problem for OSGi since it would cause a ContextFinder style TCCL (which would have a lifetime of the framework) to potentially pin a bundle's class loader and all loaded classes in the heap. Fortunately, this was not an issue. The class object was removed from the initiating class loader's cache once the class and its defining class loader were garbage collected. So, interestingly enough, the reference to the class from the initiating class loader's cache must be a sort of weak reference which allows the class to be garbage collected.
This unexpected behavior of Class.forName does not seem to be documented or explained anywhere that I have located. If you know of any such documentation, please let me know! In any case, there is a problem in designing a useful TCCL solution for OSGi.
Wednesday, September 19, 2007
Thursday, July 19, 2007
Why do Class.forName and ClassLoader.loadClass behave different?
ClassLoader.loadClass or Class.forName seem to be synonyms for the same basic operation: request a dynamic class load. Yet calling Class.forName does additional "checking" which is not very useful (certainly in OSGi).
When doing a dynamic class load, the returned type has no implied type by the code. The code must use reflection to access static members or to create an instance. A created instance can either be reflected upon or cast to a type which must already implicitly know by the code. This cast will result in a runtime type check which will ensure type safety.
This is very different than an implicit class load done by the VM to resolve a class constant pool entry. Since it is the goal of these implicit class loads to avoid runtime type checks, the loader constraints are used to ensure type safety.
It does not seem necessary or reasonable to impose loader constraint checks on some dynamic class load requests. That is, code calling Class.forName (or ClassLoader.loadClass) and the VM resolving a class constant pool entry have different type safety needs. The former does not require loader constraint checks since that will be done at runtime by a type cast if needed. The latter does require loader constraint checks to avoid the need for runtime type checks.
So it does seem reasonable to have Class.forName behavior altered to avoid loader constraint checks. Only internal VM class load requests need to check loader constraints.
[2013-10-14 - Updated links to fix link rot from Oracle acquisition of Sun.]
When doing a dynamic class load, the returned type has no implied type by the code. The code must use reflection to access static members or to create an instance. A created instance can either be reflected upon or cast to a type which must already implicitly know by the code. This cast will result in a runtime type check which will ensure type safety.
This is very different than an implicit class load done by the VM to resolve a class constant pool entry. Since it is the goal of these implicit class loads to avoid runtime type checks, the loader constraints are used to ensure type safety.
It does not seem necessary or reasonable to impose loader constraint checks on some dynamic class load requests. That is, code calling Class.forName (or ClassLoader.loadClass) and the VM resolving a class constant pool entry have different type safety needs. The former does not require loader constraint checks since that will be done at runtime by a type cast if needed. The latter does require loader constraint checks to avoid the need for runtime type checks.
So it does seem reasonable to have Class.forName behavior altered to avoid loader constraint checks. Only internal VM class load requests need to check loader constraints.
[2013-10-14 - Updated links to fix link rot from Oracle acquisition of Sun.]
Wednesday, July 18, 2007
ContextFinder in Eclipse is broken
[Updated (19 July 2007): Further discussion with Glyn Normington and Tom Watson indicates it is the Class.forName(...,TCCL) form of loading a class from the TCCL that presents the issues since it triggers the class loading constraints while the ClassLoader.loadClass form does not.]
I was doing some reading and thinking about the ThreadContextClassLoader (TCCL) issue in OSGi environments. Many libraries use the TCCL to load classes. The real problem comes when the library ONLY uses Class.forName(...,TCCL) to load a class and doesn't try it's own class loader first or TCCL.loadClass.
Eclipse created the ContextFinder to try and address the TCCL issue in OSGi. It's goal is to find a bundle's class loader on the call stack and then delegate to that class loader to handle the load request. The normal OSGi bundle class loader rules as well as a further Buddy Policy extension is then used to find the requested class.
However, it turns out there are problems with this approach when Class.forName(...,TCCL) is used. One of which is class loader constraint violation. The other is inadvertent pinning of classes in memory as they are added to the constraint table. After reading a classloader paper (from 1998!), I think the ContextFinder model (that is a single, framework-wide shared TCCL) used by Eclipse is fatally flawed. It is virtually guaranteed to violate class loading constraints in the face of multiple package versions. Not to mention the fact that it can pin classes and class loaders in memory for as long as the ContextFinder is reachable.
Specifying a way to handle TCCL in OSGi may not be possible. Given 3 bundles: A, B, C having two versions of each: A1, A2, B1, B2, C1, C2. Imagine C1 imports a package from B1 and B1 imports a package from A1. Further C2 imports a package from B2 and B2 imports a package from A2. A type from A1 does not appear in the signature of B1 and a type from A2 does not appear in the signature of B2. Thus via B1, C1 cannot "see" A1 and via B2, C2 cannot "see" A2. Also imagine that C1 import the package from A2 used by B2 and C2 imports the package from A1 used by B1. We now have the wiring depicted below:
This is entirely possible and works fine in normal (non-TCCL) class loading. C1 is not exposed to A1 via B1, so it's use of the package from A2 results in no conflict. If C1 loads a class from B1 which results in the load of a class from A1, then B1 is the initiating class loader of the request to A1. When C1 loads a class from A2, C1 is the initiating class loader. So C1 is never the initiating class loader for loads from both A1 and A2.
However, when we have a single, shared TCCL and bundle D calls Class.forName(...,TCCL) to load a class, the shared TCCL will be the initiating class loader for loads from both A1 and A2 (or B1 and B2, or C1 and C2) depending upon whatever context is used to select the defining class loader. C1 or C2 may have caused D to request the class load. Since the TCCL is the initiating class loader, any class load initiated by it for some class P must always return the same class object. In the presence of multiple versions (and even without multiple versions if two bundles are unlucky enough to choose the same fully qualified class name), loader constraints will eventually be violated.
Thus the Eclipse ContextFinder model is broken and we obviously should not spec it in OSGi. However, I am currently at a loss for a reliable solution to the TCCL problem for OSGi. We can certainly recommend TCCL.loadClass is used instead of Class.forName(...,TCCL) but there is a large body of code already out there which already uses the latter form.
I was doing some reading and thinking about the ThreadContextClassLoader (TCCL) issue in OSGi environments. Many libraries use the TCCL to load classes. The real problem comes when the library ONLY uses Class.forName(...,TCCL) to load a class and doesn't try it's own class loader first or TCCL.loadClass.
Eclipse created the ContextFinder to try and address the TCCL issue in OSGi. It's goal is to find a bundle's class loader on the call stack and then delegate to that class loader to handle the load request. The normal OSGi bundle class loader rules as well as a further Buddy Policy extension is then used to find the requested class.
However, it turns out there are problems with this approach when Class.forName(...,TCCL) is used. One of which is class loader constraint violation. The other is inadvertent pinning of classes in memory as they are added to the constraint table. After reading a classloader paper (from 1998!), I think the ContextFinder model (that is a single, framework-wide shared TCCL) used by Eclipse is fatally flawed. It is virtually guaranteed to violate class loading constraints in the face of multiple package versions. Not to mention the fact that it can pin classes and class loaders in memory for as long as the ContextFinder is reachable.
Specifying a way to handle TCCL in OSGi may not be possible. Given 3 bundles: A, B, C having two versions of each: A1, A2, B1, B2, C1, C2. Imagine C1 imports a package from B1 and B1 imports a package from A1. Further C2 imports a package from B2 and B2 imports a package from A2. A type from A1 does not appear in the signature of B1 and a type from A2 does not appear in the signature of B2. Thus via B1, C1 cannot "see" A1 and via B2, C2 cannot "see" A2. Also imagine that C1 import the package from A2 used by B2 and C2 imports the package from A1 used by B1. We now have the wiring depicted below:
C1 C2
^^ ^^
| \ / |
| \ / |
B1 \/ B2
^ /\ ^
| / \ |
| / \ |
A1 A2
This is entirely possible and works fine in normal (non-TCCL) class loading. C1 is not exposed to A1 via B1, so it's use of the package from A2 results in no conflict. If C1 loads a class from B1 which results in the load of a class from A1, then B1 is the initiating class loader of the request to A1. When C1 loads a class from A2, C1 is the initiating class loader. So C1 is never the initiating class loader for loads from both A1 and A2.
However, when we have a single, shared TCCL and bundle D calls Class.forName(...,TCCL) to load a class, the shared TCCL will be the initiating class loader for loads from both A1 and A2 (or B1 and B2, or C1 and C2) depending upon whatever context is used to select the defining class loader. C1 or C2 may have caused D to request the class load. Since the TCCL is the initiating class loader, any class load initiated by it for some class P must always return the same class object. In the presence of multiple versions (and even without multiple versions if two bundles are unlucky enough to choose the same fully qualified class name), loader constraints will eventually be violated.
Thus the Eclipse ContextFinder model is broken and we obviously should not spec it in OSGi. However, I am currently at a loss for a reliable solution to the TCCL problem for OSGi. We can certainly recommend TCCL.loadClass is used instead of Class.forName(...,TCCL) but there is a large body of code already out there which already uses the latter form.
Tuesday, July 03, 2007
Eclipse Europa on Java 1.4
Like half the rest of the programming universe, I downloaded Eclipse Europa yesterday to update my development environment with the latest and greatest released version of the Eclipse IDE. I spent a few moments figuring out what kind of developer I was given the choices offered. Since I develop OSGi code, I figured RCP/Plug-in developer was the best choice to start with. It would have been nice if OSGi bundle development was mentioned somewhere in there :-)
In any case, the "installation" (unzip is my favorite installer!) went fine. Since I deal with a few XML files, I like WST for the XML editors, etc. it provides. However, when I started Eclipse, even though WST is included in the RCP/Plug-in developer download, none of the WST stuff was functional! After some time trying to figure out what was up, I eventually figured out that it was because the default JRE on my system is 1.4! (Yes, I know it is rather retro, but I still use it as the default. I have Java 5 installed which I use when I need it.) WST just silently does not work when you start Eclipse with 1.4. No warnings or errors in the log view. I guess the WST bundles simply don't resolve because they probably require the Java 5 execution environment. Starting Eclipse with Java 5 solved the problem (and even opened the Welcome screen which I realized did not show up when I started with 1.4!).
I eventually found the Eclipse JRE page which states that the Java EE developer version requires Java 5 but I picked the RCP developer version. And while the RCP version started on 1.4, some significant function was not operational. Perhaps the website should just state that Java 5 is required for all downloads or provide some warning to the user that on Java 1.4 functions X, Y and Z don't work in Eclipse 3.3.
In any case, the "installation" (unzip is my favorite installer!) went fine. Since I deal with a few XML files, I like WST for the XML editors, etc. it provides. However, when I started Eclipse, even though WST is included in the RCP/Plug-in developer download, none of the WST stuff was functional! After some time trying to figure out what was up, I eventually figured out that it was because the default JRE on my system is 1.4! (Yes, I know it is rather retro, but I still use it as the default. I have Java 5 installed which I use when I need it.) WST just silently does not work when you start Eclipse with 1.4. No warnings or errors in the log view. I guess the WST bundles simply don't resolve because they probably require the Java 5 execution environment. Starting Eclipse with Java 5 solved the problem (and even opened the Welcome screen which I realized did not show up when I started with 1.4!).
I eventually found the Eclipse JRE page which states that the Java EE developer version requires Java 5 but I picked the RCP developer version. And while the RCP version started on 1.4, some significant function was not operational. Perhaps the website should just state that Java 5 is required for all downloads or provide some warning to the user that on Java 1.4 functions X, Y and Z don't work in Eclipse 3.3.
2007 OSGi Community Event was a Success
Last week I was in Munich, Germany for the 2007 OSGi Community Event as well as a face 2 face OSGi Enterprise Expert Group meeting.
The OSGi Alliance had held congresses in the past (2005 and prior) but did not hold one in 2006. This year we tried for a smaller and more intimate event. I think it was a great meeting. 100+ people over the span of two days heard many interest presentations on OSGi technology. Some sessions were more business oriented and some more technical. It was a pretty decent blend. You can view the presentations online.
Overall the Community Event was an excellent complement to the OSGi sessions at EclipseCon and JavaOne each year. I am sure the OSGi Alliance will have another Community Event next year and I hope to see you there!
The OSGi Alliance had held congresses in the past (2005 and prior) but did not hold one in 2006. This year we tried for a smaller and more intimate event. I think it was a great meeting. 100+ people over the span of two days heard many interest presentations on OSGi technology. Some sessions were more business oriented and some more technical. It was a pretty decent blend. You can view the presentations online.
Overall the Community Event was an excellent complement to the OSGi sessions at EclipseCon and JavaOne each year. I am sure the OSGi Alliance will have another Community Event next year and I hope to see you there!
Friday, May 25, 2007
Version 4.1 changes to OSGi Core Specification
Version 4.1 of the OSGi Core specification is a small incremental improvement to the already quite functional Release 4 specs. We made a number of minor enhancements, clarifications and errata fixes. But the 2 major changes in Version 4.1 occurred in the life cycle layer.
The two changes are (1) adding a transient flag to the Bundle start and stop methods and (2) adding a lazy activation mode to Bundle start.
1. Normally when Bundle.start() or Bundle.stop() are called, the framework must persistently remember the autostart setting of the bundle. That is, if a bundle is started with Bundle.start(), then, when the framework is next restarted, that bundle will automatically be restarted*. The start and stop methods have now been overloaded with new variants that take option flags. Two of these option flags control whether the start or stop action is transient. That is, whether the framework must alter the persistent autostart setting of the bundle. If the transient option is specified, then the framework does not alter the persistent autostart setting when performing the start or stop operation. See the new Bundle.start(int) and Bundle.stop(int) methods.
2. The other new feature is lazy activation. This allows the activation of a bundle (i.e. calling of the BundleActivator) to be delayed until the first class load is made from the bundle. This feature is based in large part upon the Eclipse-LazyStart capability. However there are a few important differences in the OSGi spec. In particular, we are clear to state that it is the activation that is lazy since starting bundle is still necessary, either explicitly via Bundle.start or implicitly because of the persistent autostart setting*. A bundle that is not started will never lazy activate. First it must be started, then it can lazy activate.
The spec also allows the caller of Bundle.start to ignore the lazy activation policy as declared in the bundle's manifest by using a new start option. This allows an admin agent to require "eager" activation even when the manifest says "lazy".
Finally, there is a new bundle event type that can be received by SynchronousBundleListeners to advise them that a bundle has started and is lazily awaiting activation. This can be used by extender bundles like OSGi Declarative Services, Spring OSGi and iPOJO to receive notification that the bundle has been started and now has a valid BundleContext. The extender bundle can then act on behalf of the bundle and do things like register services for the bundle.
Check out the new Version 4.1 spec for more details. It is now available for download.
* Assuming the bundle's startlevel is met.
The two changes are (1) adding a transient flag to the Bundle start and stop methods and (2) adding a lazy activation mode to Bundle start.
1. Normally when Bundle.start() or Bundle.stop() are called, the framework must persistently remember the autostart setting of the bundle. That is, if a bundle is started with Bundle.start(), then, when the framework is next restarted, that bundle will automatically be restarted*. The start and stop methods have now been overloaded with new variants that take option flags. Two of these option flags control whether the start or stop action is transient. That is, whether the framework must alter the persistent autostart setting of the bundle. If the transient option is specified, then the framework does not alter the persistent autostart setting when performing the start or stop operation. See the new Bundle.start(int) and Bundle.stop(int) methods.
2. The other new feature is lazy activation. This allows the activation of a bundle (i.e. calling of the BundleActivator) to be delayed until the first class load is made from the bundle. This feature is based in large part upon the Eclipse-LazyStart capability. However there are a few important differences in the OSGi spec. In particular, we are clear to state that it is the activation that is lazy since starting bundle is still necessary, either explicitly via Bundle.start or implicitly because of the persistent autostart setting*. A bundle that is not started will never lazy activate. First it must be started, then it can lazy activate.
The spec also allows the caller of Bundle.start to ignore the lazy activation policy as declared in the bundle's manifest by using a new start option. This allows an admin agent to require "eager" activation even when the manifest says "lazy".
Finally, there is a new bundle event type that can be received by SynchronousBundleListeners to advise them that a bundle has started and is lazily awaiting activation. This can be used by extender bundles like OSGi Declarative Services, Spring OSGi and iPOJO to receive notification that the bundle has been started and now has a valid BundleContext. The extender bundle can then act on behalf of the bundle and do things like register services for the bundle.
Check out the new Version 4.1 spec for more details. It is now available for download.
* Assuming the bundle's startlevel is met.
Thursday, May 24, 2007
JSR 291 is final
As mentioned on Glyn's blog JSR 291 is now final. The most important point here is that OSGi Service Platform Release 4 Core Specification Version 4.1 (quite a mouthful) is done and published. JSR 291 adds some icing to that cake by having a final JSR specification for the OSGi core spec. So, for those who like their Java specifications with a JCP flavor, OSGi technology is now also available in that flavor. :-)
You can access the JSR 291 materials online now.
You can access the JSR 291 materials online now.
SD Times article on OSGi
An SD Times article on OSGi just came out in which I was interviewed. It is a really good article on OSGi and its stealthy uptake in the market. I really enjoyed chatting with the article's author.
Saturday, May 12, 2007
What have I done?
Goodness! I've finally broken down and created a blog. To what is the world coming? After JavaOne this year, I realized I need to say some things about Java and OSGi futures and what better place than a blog? It is how we all communicate in the web 2.0 world.
Subscribe to:
Posts (Atom)
