Supporting multiple platforms – sharing our experience

This post is part of our open culture. We do not believe into closed processes, so we decided to share some interesting concepts about our development infrastructure. Or more precisely – how do we support multiple platforms in Plumbr development.

Lets start with our goals in mind. Among other goals we are aiming towards, the one adding vasts amounts of complexity is the number of platforms we need to support. If you have thought that for Java it will all be about Write Once, Run Anywhere then – make a second guess.

It all falls apart the very same minute you create something truly low-level and optimized to the last possible bit. Suddenly all kind of unexpected small details start to matter, such as the garbage collector selection, minor differences in the classloader implementations or the way JVM vendors have decided to interpret certain parts of the JVM specification.

Now having to support this low-level software running in all kind of unimaginable environments you have set up a strong foundation to failure:

• Operating system: Windows, Linux, Mac, Solaris x86, Solaris SPARC.
• Underlying architecture: 32-bit or 64-bit.
• JVM vendor: Oracle HotSpot, Oracle jRockit, IBM JDK
• Java version: 5, 6, 7 and 8.
• Application Server: Tomcat, Jetty, JBoss, Weblogic, Websphere, etc.

Most of the Java developers are lucky and have never had to dig into the hairy details of supporting multiple platforms. After all, most of the work being done by the Java developers is related to enterprise applications being deployed to a single production environment. Even if the development / staging environments differ from the production, it is usually solved in ad-hoc manner of figuring out whether to use forward or backward slashes in a particular file system.

But as we were not so lucky as we needed to support as many different environments as possible. And as our early releases demonstrated, it is nowhere near OK to test the code just on our Mac OS X development environments nor in our original Ubuntu testing environment. The bugs just kept surfacing.

So we had to figure out what to do with the aforementioned configuration matrix. Which is actually only a subset of what will be needed eventually, as we have not differentiated between the Windows releases nor different flavours of Linux nor the minor JVM releases. But even if we multiply the different configurations in the far-from-complete set of platforms listed above we reach to the 600 different configurations we have to support. To give you an idea about the configurations:

32-bit Windows XP with Oracle Hotspot 1.5.0_10 JVM running a Weblogic 8 application server

versus

64-bit Ubuntu 12.10 with Oracle jRockit running an Apache Tomcat 6.0.23 application server

From here it is relatively easy to fill out the remaining 598 configurations. But somewhere along this boring process you will start discovering close-to-impossible combinations, such as

• IBM Java running Oracle application servers.
• Oracle jRockit running IBM Websphere
• SPARC boxes with Java 7 and above

After understanding this, our original goal of “supporting as many possible configurations as possible” was quickly replaced by “support as many users as possible”. Digging into the statistics submitted by our users also gave confidence in doing so – the truly exotic combinations are close to non-existent.

So you can reduce the size of the matrix somewhat. In our case we were able to cut the required infrastructure to more than half of its theoretical size. But it still left us with more than 200 environments to support. Which, as you can imagine is still way too much for a human being to handle without some clever tricks and techniques in your belt.

But this will be the subject for the next post covering our internal goals and implementation details about:

• Automated infrastructure. Build infrastructure has to be able to launch and destroy different configurations automatically.
• Automated tests. Tests run on different environments have to be automated
• Support multiple versions. We need to support older releases independent of the ongoing future development
• Backward traceability. When receiving an error report from an earlier version we need to be able to link it with the exact version.
• Obfuscation map compliancy. As we use code obfuscation we need to be able to use the proper obfuscation map for proper version.

If you wish to be alerted about our next posts then subscribe to our Twitter feed

The post might seem familiar to those of of you who were participating in JavaOne Moscow this year. Indeed, you had a chance to hear me on stage with the same presentation. But the rest of the ~8,310,700 who did not have a chance to be present I hope I was able to introduce some interesting concepts.