Java 9: Why Modules?

What Are Modules?

The Java Module System, a.k.a. Project Jigsaw, is an effort to introduce a higher-level structural concept into Java. In a nutshell: Classes assemble into packages, and packages assemble into modules. Modules declare on which packages they depend, and in turn export packages for others to depend on.

Modules are a fundamental new concept going deep into the JDK, and have been under development for years. Jigsaw was originally planned for Java 7. As work began it became clear that this was a huge undertaking reaching deep into the internals of the JVM and the Java language itself. Yet, despite being slipped to Java 8 and then to Java 9, the need for it has only increased. Why do we need modules in Java? Why is Project Jigsaw such a big deal?

Why Modules?

Mark Reinhold, the Chief Architect of the Java Platform Group at Oracle, gave a keynote speech at Devoxx Belgium where he described the purpose of a module system and why it is so important.

There are many use cases that modules solve, but they all boil down to two primary pain points:

  1. Brittle, error-prone classpath (AKA JAR Hell)
  2. Monolithic JDK

Let’s look more at each of these.

Brittle Classpath

Have you ever seen a ClassNotFoundException? How about NoClassDefFoundError, NoSuchFieldError, or NoSuchMethodError? Maybe even a ClassCastException: “Cannot cast com.foo.Bar to com.foo.Bar”? All of these are the results of a brittle classpath. What that means is that ClassLoaders don’t have a good mechanism for distinguishing one loaded class from another class of the same name, or for isolating classes loaded by one ClassLoader from classes loaded by another.

From the Wikipedia entry on JAR Hell, problems from a brittle classpath arise from one of these situations:

  • Accidental presence of two different versions of a library installed on a system. This will not be considered an error by the system. Rather, the system will load classes from one or the other library. Adding the new library to the list of available libraries instead of replacing it may result in the application still behaving as though the old library is in use, which it may well be.
  • Multiple libraries or applications require different versions of library foo. If versions of library foo use the same class names, there is no way to load the versions of library foo with the same classloader.
  • The most complex JAR hell problems arise in circumstances that take advantage of the full complexity of the classloading system. A Java program is not required to use only a single “flat” classloader, but instead may be composed of several (potentially very many) nested, cooperating classloaders. Classes loaded by different classloaders may interact in complex ways not fully comprehended by a developer, leading to errors or bugs that are difficult to analyze, explain, and resolve

In some cases, build tools may help you detect some of these situations (such as incompatible dependency graphs). But the fact remains that tools will not prevent you from doing dangerous things, and they do not protect you from all of the situations described above.

Java Modules will provide reliable configuration and strong encapsulation. If you have incompatibilities, you will discover these at build time instead of some indeterminate time after your application has started running in production.

Monolithic JDK

The JDK has been big for years. At this point in 2016, the JDK is too big to fit comfortably on small devices, and we would really like it to be easy to use embedded devices and other devices on the Internet of Things.

Now you might say “But I run my software on a large server, we are not constrained by space.” However, even on large servers we want to run lots of VMs and optimize memory use. If every instance you have requires an extra 200MB of memory that it doesn’t need, that adds up across hundreds of instances, and those resources are something you pay for every second of every day. Additionally, with a monolithic JDK startup performance is slower because of loading unnecessary things. We would like to measure our startup time in milliseconds, not thousands of milliseconds.

Java Modules will allow us to break up the JDK into modules so that we only use what we need. We will be able to deploy modular runtime images that take less space and that start up faster.

Conclusion

The changes for Java 9 to support modules are far reaching, and it will take some work because we will have new language features and syntax to learn. But I think this will be worth it to make Java applications stronger, faster, and more reliable. In a subsequent post we will look more at the details of how modules work and how we can start to incorporate them into our own software.

What do you think? Have you had the problems that Jigsaw is trying to solve? Or are these non-issues for you?

Leave a comment

Filed under Java 9

Java 9: Multi-Release JAR Files, Part II

In Part I on multi release JAR Files, we learned what it is and why it is useful. In this part we will learn how to actually construct such a JAR and test it out.

For this demonstration we will be using just the command line as opposed to Maven or Gradle. Tooling support for Java 9 is still ramping upūüôā

in a src folder, we have two classes

public class Application {
   public static void main(String[] args) throws IOException {
      Generator gen = new Generator();
      System.out.println("Generated strings: " + gen.createStrings());
   }
}

and

public class Generator {
   public Set<String> createStrings() {
      Set strings = new HashSet();
      strings.add("Java");
      strings.add("7");
      return strings;
   }
}

Then in another src folder we have another implementation of the class using Java 9 features:

public class Generator {
   public Set<String> createStrings() {
      return Set.of("Java", "9");
   }
}

Finally, we need to specify a file containing a new manifest attribute to indicate that the JAR is a multi-release JAR. The manifest needs to include:

Multi-Release: true

And now, to compile! From the command line:

javac -d build -release 7 src/*.java
javac -d build/META-INF/versions/9 -release 9 src9/*.java
jar --create --file mrjar.jar --manifest MANIFEST.MF --main-class=Application -C build .

There are a couple things going on here: We need to compile in multiple phases like this because we literally need to generate different versions of bytecode and put it in different places. Also note that we are compiling with the new -release argument instead of -source and -target. Refer to JEP-247 for more details, but essentially it guarantees the class to be compatible with a given version of the JVM in a way that -source and -target can’t.

Finally, if we set our path to use the appropriate JVM, when running on Java 9 we see

java -jar mrjar.jar 
Generated strings: [Java, 9]

But running the same JAR on Java 7 we see

java -jar mrjar.jar 
Generated strings: [Java, 7]

Proving that a different class was used for each version of Java that was running it!

Admittedly this is still a bit crude, with the non-standard file locations and dropping to the command line to do everything. We will build up to a Gradle example soon!

Leave a comment

Filed under Java 9

Java 9: Multi-Release JAR files, Part I

Is your software directly installed by your end users? Do you write libraries that could be used on any version of the JVM?

In either of these cases, you don’t have control over which JVM your code is running on, so you need to target your code to the lowest version of the JVM that you can support. In some cases, this could even be Java 6. Even in early 2016, 10% of commercial JVM installations ran on Java 6. Not a lot, but… you know… Enough for you to have to consider it.

This is a problem

I have lived with this situation myself. Besides just keeping us from using the latest language features, supporting older versions allows the possibility that bugs that our users complain about in our software might actually be attributable to underlying JVM bugs. For example, there are known issues with Java.io.File that are fixed with NIO2, older JVM’s generally have lower performance, and finally older JVM’s lack certain security features and other bug fixes. Running on older JVM’s can even expose issues in our code that are not JVM specific but that would be easier to fix on later versions, such as hard-to-find multithreading bugs for parallelism that would be moot if we could just use parallel streams.

Wouldn’t it be nice if we could start using a later version of Java while still supporting older JVM’s for your users? We can!

Multi-Release JAR Files

Allow me to introduce you to JEP-238: Multi-Release JAR Files.

The idea behind Multi-Release JAR Files is that within a single JAR we can include multiple versions of class file for a single class. A version of the class can be selected and loaded that is specific to the version of the JVM running it. Each class file could have a different version of byte code, so later versions of the class will be ignored by older JVM’s. This allows you to begin migration efforts earlier and fix version specific bugs as outlined above, all while still only delivering a single JAR file as usual.

How does this work?

Java 9 defines an extension of the Jar file format. See the diagram below, a picture is worth a thousand words:

jar root
   - A.class
   - B.class
   - C.class
   - D.class
   - META-INF
      - versions
         - 8
            - A.class
            - B.class
         - 9
            - A.class

As we can see, the normal class files are present right where we’d expect them. But now there are additional class files that represent the same class when run on a specific version of the JVM. So if you run this JAR on Java 6 or 7, you will get the “normal” version of the class. If you run it on Java 8, you will get those versions of class A and B only, and if you run on Java 9, you will get the Java 8 version of B and the Java 9 version of A.

Keep in mind: at the time of this writing, Java 8 is not MRJAR-aware. So the Java 8 portion of the example described in the JEP and shown in this diagram is not actually possible because the Java 8 classes can never be recognized and loaded on Java 8. But they will be loaded on Java 9 and there’s nothing to prevent future versions of the Java 8 JVM from becoming MRJAR aware.

In Part II we will look at an actual implementation that you can do with the early access release of Java 9!

Leave a comment

Filed under Java 9

Move Deliberately and Don’t Break Anything

Move Fast and Break Things¬†was first coined at Facebook, and has since become a startup mantra. The idea is that if you’re not occasionally breaking things, you’re not moving fast enough. And of course as a business if you’re not moving fast enough you’ll die.

I recently watch a talk by Brian Goetz (Java Language Architect at Oracle) called “Move Deliberately and Don’t Break Anything.” This post is a recap of that talk and what I thought were important points.

As with most things, Move Fast and Break Things does not apply to everything. Success comes at a price and that price is you have to stop breaking things. At a certain scale (millions of users? billions?) the cost of breaking something is higher than the cost of not moving at a particular speed. Facebook, the originator of the idea, has realized this too and had to modify their philosophy accordingly.

So what does this mean for the Java language?

In Java, there is a tension between Progress and¬†Preservation.¬†Pushing for progress are developers who tend to overestimate the value of code and underestimate the value of working software. Pushing for preservation are the decision makers of the business who need software to not break, period. Some people would like to change the Java language more quickly, but changes can play out in unpredictable ways. A feature is not just a feature, it’s the interaction with every future feature… forever! You are really locked into¬†your decisions¬†if you need to preserve, which you have to do if you are successful and have a billion users.

So the designers and maintainers of the Java language have to move carefully and deliberately. According to Brian Goetz, if you don’t¬†yet¬†know what is the right thing to do, don’t do anything. Just don’t do the wrong thing. This gives you a chance to still do the right thing in the future! I think he makes a good argument for why you can’t have that language feature you really want.ūüôā

What do you think? Is the Java language moving too slowly to compete in the modern software engineering world, or moving at the just the right speed to ensure our critical systems stay up and running?

 

Leave a comment

Filed under Uncategorized

Lagom Is Coming For Your Monolith

The Philly JUG had another successful meetup, this time about Lightbend’s new open source project Lagom. It was a timely presentation, as Lagom 1.0 had¬†just been released the day before (on July 26, 2016).

The Lagom website¬†has much better descriptions of what it’s all about, but I’ll give you my first impressions.

Lagom is a microservices framework¬†which is¬†designed from the ground up to be reactive, resilient, and elastic. Event Sourcing¬†and CQRS¬†are implemented as first class citizens, and all calls are asynchronous with first class support for streaming of request and response. It’s easy to see that Lagom is positioned as a high-throughput framework.

Besides the design for high performance, the big advantages I saw were that service registration and location are built in. The framework is designed to work with sets of services at once, which is a plus when dealing with a large project composed of many services.

The downside is that this framework is very new and there is a lot of work to be done. For instance, it only builds with SBT (not Maven or Gradle), and there is no security model.

One thing that is¬†confusing at first is¬†that it builds with SBT and runs with Akka and Play,¬†implying this is a Scala framework. But the only way to use it is via a¬†Java API, and if you dig in you’ll find there is no Scala API. So where is the Scala API? A Scala enthusiast might say “Where is it indeed?!?” ¬†A quick conversation with the Lagom presenters revealed that – to the chagrin of the Scala base – Lightbend has decided that it wants to be involved in the enterprise space, and the opportunity for adoption there lies with producing a Java API first. So although that’s good news for Java developers, Scala developers will unfortunately have to wait.

Give Lagom a try, and let me know what you think in the comments!

1 Comment

Filed under Uncategorized

Java 9: Convenience Factory Methods for Collections

If you are following the feature set to be included in Java 9, you may have noticed¬†JEP 269: Convenience Factory Methods for Collections. According to the proposal, this feature is to “provide static factory methods on the collection interfaces that will create compact, unmodifiable collection instances. The API is deliberately kept minimal.”

The expert group¬†discusses alternatives, and explains in particular : “The Google Guava libraries have a rich set of utilities for creating immutable collections, including a builder pattern, and for creating a wide variety of mutable collections. The Guava libraries are very useful and general but are perhaps overkill for inclusion into the Java SE Platform.”¬†However, I would argue that if Guava is “very useful” and “general”, that makes it completely appropriate for inclusion in Java SE proper. But that’s just my opinion!

If you look at the Java 9 API for Set, one¬†thing that may jump out at you is the plethora of “.of()” methods, such as this .of() method¬†with 10 arguments. The argument for this (!) is that¬†“While this introduces some clutter in the API, it avoids array allocation, initialization, and garbage collection overhead that is incurred by varargs calls.” So here is a real-world instance where there was an evaluation between clarity of API design vs performance considerations, and in this case, the performance considerations won.

Without any further ado, here is an example of the new API:

Set strings = Set.of("a", "b");

Reading the Set javadocs with all the .of() methods is a little confusing at first (as to why they did that) but the nice part is that the code as used reads quite nicely.

Enjoy your tidy new Sets!

Leave a comment

Filed under Java 9

Drawbacks of Sprint Zero, or, How To Do Just Enough Design Up Front

I read this article recently about drawbacks of sprint zero and thought it was highly relevant to us as Software Engineers. Please read this article then come back here and we can talkūüôā

There is a balance to be had between doing ALL design up front, NO design up front, and (as this article suggests) SOME design up front. I have seen all of these approaches, and it is admittedly tricky at times to pick the right balance. For new product development a design phase does seem to work well in my experience. And for existing product development I would say a healthy dose of “big picture thinking” is appropriate when designing the feature (perhaps at story grooming time) and an entire design phase might be a bit much at that point. Again, as with most things, I think the answer to the question of how much design to do up front is “It depends.”

How do you design your products? How much time do you spend designing your features and how do you know it’s the right amount of time? Let me know what you think!

Leave a comment

Filed under Software Engineering