Welcome!

Linux Containers Authors: Liz McMillan, Carmen Gonzalez, Elizabeth White, Pat Romanski, Jyoti Bansal

Related Topics: Linux Containers, Java IoT

Linux Containers: Article

Improving Swing Performance: JIT vs AOT Compilation

Linux native band plays fast

The JFC/Swing API, natively precompiled on Linux for the first time, delivers measurable improvement in Java GUI performance.

The Excelsior Engineering Team has ported Excelsior JET, a Java Virtual Machine (JVM) with an ahead-of-time compiler, to the Linux/x86 platform. As the JET JVM supports the entire J2SE platform API including the Java Foundation Classes (JFC/Swing), Excelsior engineers had an opportunity to evaluate the response time of natively compiled JFC/Swing on Linux. The results of the comparison with conventional, dynamically optimizing JVMs were encouraging: response time has improved by 40% or even doubled on some benchmarks. What's more important is that real-world Swing applications performed perceivably faster.

This article describes Excelsior JET JVM and JFCMark, free benchmark software by Excelsior that measures Swing-based GUI performance. Moreover, the authors share their technical experience in optimizing JFC/Swing and argue why ahead-of-time Java compilation has advantages over dynamic compilation for certain application types.

Two Recipes for Java

The definition of the term "virtual machine" has been revised in the past few years. Modern Java Virtual Machines are no longer just interpreters of the Java bytecode. High performance, state-of-the-art implementations are made up of optimizing compilers that translate bytecode instructions down to the native code that runs directly on the hardware. However, one technical decision distinguishes contemporary JVMs: what's the best time to run the performance engine, an optimizing native compiler? Two options exist: run it before or after the application starts.

Most JVMs initially interpret the program and then analyze how it runs by looking for hot spots, that is, frequently executed portions of bytecode. Hot spots are then compiled to optimized native code during program execution. This approach is called Just-In-Time (JIT) compilation. Other JVMs feature static native compilers similar to traditional C/C++ compilers, enabling developers to optimize their Java applications before execution. For Java, this old trick has a new name - Ahead-Of-Time (AOT) compilation. However, solely static compilation is not enough for Java compatibility. Remember that many Java applications use custom classloaders to load some components or plug-ins at runtime. To have the "J" in "JVM," such virtual machines must be supplied with an interpreter or JIT compiler to handle classes that could not be precompiled.

Either approach alone is not a silver bullet for Java performance. JIT-oriented JVMs can "see through" program execution, which may help them optimize hot code better than static compilers do. In return, AOT-oriented JVMs do not spend execution time on interpretation, profiling, and compilation, so optimized programs run fast from start-up. Not surprisingly, single-loop benchmarks fail to reveal a clear performance winner between the two approaches. Instead of carrying the "microbenchmark war" into the Linux camp, let's take a look at a vital example: the performance of Java GUI applications based on JFC/Swing.

Penguin-Driven JVMs

Both JIT and AOT-oriented implementations are available for Linux. The well-known Sun HotSpot Client and Server JVMs are powered by JIT compilers. BEA WebLogic JRockit and IBM Java 2 Runtime Environment also play in the JIT team. GCJ, the GNU compiler for Java, now supported by Red Hat, and Excelsior JET feature AOT compilation.

At the core of Excelsior JET is a static optimizing compiler that enables developers to transform their Java applications into native executables or shared libraries (.so) on a Linux flavor. The AOT compiler comes with a JET Control Panel (see Figure 1), a graphical front end that makes the product easy to learn and use. The command-line interface provides the integration of Excelsior JET into automated builds. The redistributable JET runtime system includes a JIT compiler to support Java dynamic classloading. Another graphical tool, JetPackII (see Figure 2), enables the rapid creation of installation packages for optimized applications. Excelsior JET supports all Java 2 platform packages up to version 1.4.2.

Another static Java compiler is GCJ, a member of the GNU Compiler Collection. Following the FSF philosophy, GCJ uses a clean-room, free implementation of the Java 2 Platform API. Although most packages are now supported, there are noticeable exceptions such as the Abstract Window Toolkit (AWT), Swing, and some of the APIs introduced to J2SE 1.4. Contributors to the GNU Classpath project are currently implementing the missing packages. The GNU Interpreter for Java complements GCJ to enable Java dynamic loading. The implementation of a JIT compiler is planned for the future.

Accelerated Swing Tempo

GUI response time is in the eye of the beholder. As a result, it's tough to obtain GUI performance scores. To address this problem, Excelsior has developed JFCMark, a free benchmark suite to measure the performance of the JFC/Swing API. The included tests are manipulating with frames, trees, and tables; switching look-and-feels; decoding and drawing images; displaying and scrolling HTML texts; and using Swing layout managers. JFCMark requires JVMs that support the Java 2 Platform at the level of J2SE 1.3, 1.4, or 1.5.

Each test performs its scenario in the main loop through a given number of iterations. This allows you to obtain performance scores in short- and long-running modes. Upon completion, these tests report performance measured in units specific to their specific scenarios, for instance, frames opened per second. When designing JFCMark, we paid close attention to benchmarking accuracy. For a particular configuration, every benchmark always performs the same number of operations independently of the JVM under test. This requirement is achieved through synchronous processing of Swing events, that is, a next event is sent only after the previous one is processed. Therefore, a possible difference in reported speed depends solely on the time of the benchmark execution.

Let's consider the performance of the Swing windowing system that provides operations with frames. We have used a part of JFCMark to test typical manipulations with frames such as opening/closing, dragging, and selecting. The testbed configuration was as follows.]

  • CPU: AMD Athlon running at 1,8 GHz
  • RAM: 512MB DDR SDRAM
  • Video: NVidia GeForce2 MX-200 at 1024x768x65536c
  • OS: Red Hat Linux release 8.0 (Psyche)
  • Linux Kernel: 2.4.18-14
We've run these benchmarks using a spectrum of JVMs available for Linux:
  • Excelsior JET 3.6 Professional Edition with JRE 1.4.2_04
  • Sun Java HotSpot Client VM 1.4.2_04
  • IBM J2RE 1.4.2 Classic with JIT enabled
  • Sun Java HotSpot Server VM 1.4.2_04
  • BEA WebLogic JRockit 8.1 with JRE 1.4.2_04
The server-oriented JVMs (Sun HotSpot Server and BEA WebLogic JRockit) are included for your reference. They're not expected to really shine in client-side application performance. GCJ is not included because it does not yet have support for JFC/Swing. To play fair, we provided results for both long- and short-running configuration of the benchmarks. Figures show the composite performance index in comparison with the reference implementation (Sun HotSpot Client VM). Longer bars mean better performance.

Figure 3 shows Swing performance scores for the long-playing version of JFCMark. For example, 600 frames are opened and closed by one of the tests. In this scenario, JIT-based JVMs have a chance to "warm up," that is, to optimize hot code for maximum speed. Note, however, that this level of performance may be reached only after doing a "good amount of mouse-clicking" in real-world Java GUI applications. Nevertheless, Excelsior JET still outperforms by 40% the first runner-up (HotSpot Client VM).

To see how Swing works on dynamic JVMs that have not been "warmed up" yet, look at Figure 4. It shows out-of-the-box performance scores for the short-running configuration. This case would correspond to the way Java GUI applications perform right after start-up. As can be seen, the JET-compiled Swing runs fast and it runs fast from the start. Other test participants work at least twice as slow for this scenario. Note that short-running benchmarks are mostly important for the client-side application's performance. For instance, if you start a GUI application and drag something with the mouse, you don't want to see it stumbling just because the JVM has not yet done its job. People talking about "snail Java" often ignore the fact that many JVMs take time to warm up. It's not an issue for server-side applications, which typically run for hours and days. However, the performance of client-side applications is most heavily impacted by the JVM warm-up cycle.

JEdit, a Practical Example

If you don't trust vendor benchmarks, check the results yourself. One of the samples that comes with Excelsior JET is a project for compiling jEdit, an open source, cross-platform text editor written in Java. jEdit has many advanced features that make text editing easier, such as syntax highlighting, auto indent, abbreviation expansion, registers, macros, regular expressions, and multiple file search/replace. It's a good example of a full-featured Swing-based application to evaluate GUI response time.

Behind the Performance Figures

An interesting question is why don't the JIT-powered JVMs hit the performance bar raised by Excelsior JET? Of course, aggressive static optimizations and the removal of bytecode interpretation make Swing work smoothly. However, it seems that the long-running mode of JFCMark should be comfortable for the JIT compilers. Where does the 40% performance win come from? We found the answer unexpectedly when we aimed at further improving Swing performance: the absence of hot methods.

Under the covers, JFC/Swing is quite a complex event-driven system implemented on top of AWT. The implementation consists of several hundred classes which, in turn, use a variety of other core classes. Many thousands of Java methods are executed when, for example, you open a Swing frame. When we obtained the Swing execution profile, it proved to be almost flat. Lots of methods were executed but each of them took hundredths of one percent in total execution time. Only a few methods took more than 1-2%. At this point, we encountered an interesting problem: what should we improve in the absence of clear performance bottlenecks? As you may have guessed, the same problem exists for profile-guided JIT compilers. Instead of a few hot spots to be aggressively optimized, there are plenty of "warm spots" that are left intact. The flat execution profile is an application-specific property that some JVMs cannot effectively manage to achieve top performance.

Conclusion

This article is not about fast and slow JVMs. Rather, it demonstrates that for some Java applications, a JVM with AOT compilation can work faster than JIT-based JVMs. The main lesson we have learned from this study is that one size does not fit all and JFC/Swing is not the only example. One way or another, the Java platform wins and we were happy to make Excelsior JET JVM available to Java/Linux developers.

Resources

  • Java Foundation Classes (JFC/Swing): http://java.sun.com/products/jfc/
  • Excelsior JET: www.excelsior-usa.com/jet.html
  • JFCMark: JFC/Swing benchmark with source code: www.excelsior-usa.com/jfcmark.html
  • Sun HotSpot JVMs: http://java.sun.com/j2se/1.4.2
  • IBM J2RE: www-106.ibm.com/developerworks/java/jdk/linux140/
  • BEA WebLogic JRockit: www.bea.com/framework.jsp?CNT=index.htm&FP=/content/products/jrockit
  • GCJ, the GNU compiler for Java: http://gcc.gnu.org/java/
  • GNU Classpath: www.gnu.org/software/classpath/
  • jEdit: www.jedit.org
  • More Stories By Vitaly Mikheev

    Vitaly Mikheev is the chief technology officer for Excelsior, LLC, a company focusing on design and development of optimizing compilers. Vitaly has been involved in software development since 1987 and focused on compiler construction technologies for the last decade. He started working with Java in 1998 as the architect of the Excelsior Java Virtual Machine. Before that, he worked on proprietary optimizing compilers for Nortel Networks. Vitaly is a member of ACM and a co-author of the patent on the garbage collector algorithm implemented in the Samsung's J2ME CDC virtual machine. He holds an MS in computer science from the Novosibirsk State University, Russia.

    Comments (5) View Comments

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    Most Recent Comments
    Blind Earl 11/08/04 09:04:35 AM EST

    Once I actually found the article it was an interesting read. However, this website looks *terrible*. When a column is only 11 characters wide due to advertising squishing it down, it is time to find some other magazine to read.

    the author 11/08/04 07:14:08 AM EST

    To angel'o'sphere:

    1)

    >>A VM based on a JIT compiler loads always only byte code
    >>and compiles the byte-code to native code on the fly
    >>depending on several algorithms/heuristics.
    >>A AOT compiler does JUST THE SAME on the first invocation
    >>of the byte code program, but it saves the generated
    >>native code to be available for later invocations of the
    >>same program. So for the later invocations, the
    >>compilation was ahead of time, not for the first one

    The thing you are talking about is called Caching JIT compilation (at least HotSpot and BEA engineers called it so when I talked with them at JavaOne 2004)

    Besides, Caching JIT compilation does not occur on the first invocation - it would take long long time and miss profile information useful for optimizations.

    2)

    >>The Excelsior JET is no AOT compiler but just a static
    >>compiler. However they seem to jump on the bandwagon
    >>of a new buzzword, because the term AOT is only used
    >>in academic circles currently

    visit http://gcc.gnu.org/java/ (GNU compiler for Java). It's described as an AOT compiler...

    3)

    >>The next flaw is this: they measure a GUI application
    >>over only 600 "loops". As the exact form of the loop
    >>is not published, I have the impression they made the
    >>loop in a way that a "standard" HotSpot JIT does
    >>predictable not even attempt to compile much of the
    >>covered code

    In this test, HotSpot shows 28,9 frames/sec for short-running configuration and 44,6 frames/sec for long-running one. It does not look like "not even attempt to compile much of the covered code". Increasing the number of iteration to open more frames does not improve the HotSpot results. BTW, did you ever open 600+ frames in a Java application?

    4)

    >>In the "benchmark" of the article,
    >>you neither have a loop (that one is in the external
    >>driver program hidden) nor stack samples which occur
    >>often enough to cause a JIT compilation.

    Check the source code of the benchmark. It includes the loop

    for(int j=0;j>IMHO: the Excelsior people tricked the HotSpot/JIT VMs
    >>in not doing their job, and thus you see the big
    >>performance gain.
    >> There are other tricks thinkable, e.g. not using
    >>the standard SWING library but a tweaked one,
    >>just remove some "synchronized" keywords if
    >> you are sure the benchmark does not need them,
    >> and voila ...

    The standard Swing library and the public JET version were used for benchmarking. As for "synchronized" keywords - some of them are *safely* removed by the JET compiler during the course of escape analysis. There is excellent literature about this optimization technique (for example, see ACM OOPSLA'99 - about 5 papers were devoted to it).

    Take care,

    --Vitaly

    P.S. JITs work well on servers and reusing the same machinery on the desktop is not always appropriate.

    angel'o'sphere 11/08/04 05:28:31 AM EST

    I don't like the article.
    For several reasons I think they cheated and they coin old terms into new terms for no reason (except publicity). E.g. they mix up AOT with static compilation.
    AOT is an extension of JIT compilation, and not a compilation done ahead of the first execution.
    A VM based on a JIT compiler loads always only byte code and compiles the byte-code to native code on the fly depending on several algorithms/heuristics.
    A AOT compiler does JUST THE SAME on the first invocation of the byte code program, but it saves the generated native code to be available for later invocations of the same program. So for the later invocations, the compilation was ahead of time, not for the first one.
    The Excelsior JET is no AOT compiler but just a static compiler. However they seem to jump on the bandwagon of a new buzzword, because the term AOT is only used in academic circles currently (besides that Java 5.0 tries to cash compile informations and even native code over invocations).
    The next flaw is this: they measure a GUI application over only 600 "loops". As the exact form of the loop is not published, I have the impression they made the loop in a way that a "standard" HotSpot JIT does predictable not even attempt to compile much of the covered code.
    To understand that you have to know how a HotSpot JIT works. Two "big" strategies are used: fiddling with loops, this is a short code fragment, inside of one method, consider it as a for loop or while loop. You can do different things with it ... not interesting here.
    The second big thing is call stack sampling. Suppose you have very often the method calls A, B, C on the call stack, the VM decides to check if anything can be inlined or if any of those methods is jitted.
    In the "benchmark" of the article, you neither have a loop (that one is in the external driver program hidden) nor stack samples which occur often enough to cause a JIT compilation.
    Excellent literature about JIT compilation and strategies can be found here: http://www.research.ibm.com/jalapeno/publication.h tml and here: http://www-124.ibm.com/developerworks/oss/jikesrvm /info/papers.shtml
    IMHO: the Excelsior people tricked the HotSpot/JIT VMs in not doing their job, and thus you see the big performance gain. There are other tricks thinkable, e.g. not using the standard SWING library but a tweaked one, just remove some "synchronized" keywords if you are sure the benchmark does not need them, and voila ...

    Seun Osewa 11/07/04 02:32:53 PM EST

    combining the two approaches will lead to overhead similar to a JIT approach (hence, there's nothing to be gained for a long-running program)

    RAMMS+EIN 11/06/04 05:40:50 AM EST

    Why compare JIT against AOT? Why not have both?

    AOT compilation makes for fast start up time and fast run time. JIT advocates claim that it can lead to better performance, as more optimizations can be performed with run-time information. So why not combine the two? Compile it before the first run, and further optimize it at run-time where appropriate. That way, you get the best of both worlds.

    @ThingsExpo Stories
    "There's a growing demand from users for things to be faster. When you think about all the transactions or interactions users will have with your product and everything that is between those transactions and interactions - what drives us at Catchpoint Systems is the idea to measure that and to analyze it," explained Leo Vasiliou, Director of Web Performance Engineering at Catchpoint Systems, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York Ci...
    In his General Session at 16th Cloud Expo, David Shacochis, host of The Hybrid IT Files podcast and Vice President at CenturyLink, investigated three key trends of the “gigabit economy" though the story of a Fortune 500 communications company in transformation. Narrating how multi-modal hybrid IT, service automation, and agile delivery all intersect, he will cover the role of storytelling and empathy in achieving strategic alignment between the enterprise and its information technology.
    The 20th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held June 6-8, 2017, at the Javits Center in New York City, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Containers, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportunity. Submit your speaking proposal ...
    Discover top technologies and tools all under one roof at April 24–28, 2017, at the Westin San Diego in San Diego, CA. Explore the Mobile Dev + Test and IoT Dev + Test Expo and enjoy all of these unique opportunities: The latest solutions, technologies, and tools in mobile or IoT software development and testing. Meet one-on-one with representatives from some of today's most innovative organizations
    20th Cloud Expo, taking place June 6-8, 2017, at the Javits Center in New York City, NY, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy.
    SYS-CON Events announced today that Super Micro Computer, Inc., a global leader in Embedded and IoT solutions, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 7-9, 2017, at the Javits Center in New York City, NY. Supermicro (NASDAQ: SMCI), the leading innovator in high-performance, high-efficiency server technology, is a premier provider of advanced server Building Block Solutions® for Data Center, Cloud Computing, Enterprise IT, Hadoop/Big Data, HPC and E...
    SYS-CON Events announced today that Linux Academy, the foremost online Linux and cloud training platform and community, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Linux Academy was founded on the belief that providing high-quality, in-depth training should be available at an affordable price. Industry leaders in quality training, provided services, and student certification passes, its goal is to c...
    WebRTC is the future of browser-to-browser communications, and continues to make inroads into the traditional, difficult, plug-in web communications world. The 6th WebRTC Summit continues our tradition of delivering the latest and greatest presentations within the world of WebRTC. Topics include voice calling, video chat, P2P file sharing, and use cases that have already leveraged the power and convenience of WebRTC.
    "Matrix is an ambitious open standard and implementation that's set up to break down the fragmentation problems that exist in IP messaging and VoIP communication," explained John Woolf, Technical Evangelist at Matrix, in this SYS-CON.tv interview at @ThingsExpo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
    Buzzword alert: Microservices and IoT at a DevOps conference? What could possibly go wrong? In this Power Panel at DevOps Summit, moderated by Jason Bloomberg, the leading expert on architecting agility for the enterprise and president of Intellyx, panelists peeled away the buzz and discuss the important architectural principles behind implementing IoT solutions for the enterprise. As remote IoT devices and sensors become increasingly intelligent, they become part of our distributed cloud enviro...
    WebRTC sits at the intersection between VoIP and the Web. As such, it poses some interesting challenges for those developing services on top of it, but also for those who need to test and monitor these services. In his session at WebRTC Summit, Tsahi Levent-Levi, co-founder of testRTC, reviewed the various challenges posed by WebRTC when it comes to testing and monitoring and on ways to overcome them.
    "A lot of times people will come to us and have a very diverse set of requirements or very customized need and we'll help them to implement it in a fashion that you can't just buy off of the shelf," explained Nick Rose, CTO of Enzu, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
    DevOps is being widely accepted (if not fully adopted) as essential in enterprise IT. But as Enterprise DevOps gains maturity, expands scope, and increases velocity, the need for data-driven decisions across teams becomes more acute. DevOps teams in any modern business must wrangle the ‘digital exhaust’ from the delivery toolchain, "pervasive" and "cognitive" computing, APIs and services, mobile devices and applications, the Internet of Things, and now even blockchain. In this power panel at @...
    Every successful software product evolves from an idea to an enterprise system. Notably, the same way is passed by the product owner's company. In his session at 20th Cloud Expo, Oleg Lola, CEO of MobiDev, will provide a generalized overview of the evolution of a software product, the product owner, the needs that arise at various stages of this process, and the value brought by a software development partner to the product owner as a response to these needs.
    WebRTC services have already permeated corporate communications in the form of videoconferencing solutions. However, WebRTC has the potential of going beyond and catalyzing a new class of services providing more than calls with capabilities such as mass-scale real-time media broadcasting, enriched and augmented video, person-to-machine and machine-to-machine communications. In his session at @ThingsExpo, Luis Lopez, CEO of Kurento, introduced the technologies required for implementing these idea...
    The WebRTC Summit New York, to be held June 6-8, 2017, at the Javits Center in New York City, NY, announces that its Call for Papers is now open. Topics include all aspects of improving IT delivery by eliminating waste through automated business models leveraging cloud technologies. WebRTC Summit is co-located with 20th International Cloud Expo and @ThingsExpo. WebRTC is the future of browser-to-browser communications, and continues to make inroads into the traditional, difficult, plug-in web co...
    Internet of @ThingsExpo, taking place June 6-8, 2017 at the Javits Center in New York City, New York, is co-located with the 20th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. @ThingsExpo New York Call for Papers is now open.
    Who are you? How do you introduce yourself? Do you use a name, or do you greet a friend by the last four digits of his social security number? Assuming you don’t, why are we content to associate our identity with 10 random digits assigned by our phone company? Identity is an issue that affects everyone, but as individuals we don’t spend a lot of time thinking about it. In his session at @ThingsExpo, Ben Klang, Founder & President of Mojo Lingo, discussed the impact of technology on identity. Sho...
    Technology vendors and analysts are eager to paint a rosy picture of how wonderful IoT is and why your deployment will be great with the use of their products and services. While it is easy to showcase successful IoT solutions, identifying IoT systems that missed the mark or failed can often provide more in the way of key lessons learned. In his session at @ThingsExpo, Peter Vanderminden, Principal Industry Analyst for IoT & Digital Supply Chain to Flatiron Strategies, will focus on how IoT depl...
    Data is an unusual currency; it is not restricted by the same transactional limitations as money or people. In fact, the more that you leverage your data across multiple business use cases, the more valuable it becomes to the organization. And the same can be said about the organization’s analytics. In his session at 19th Cloud Expo, Bill Schmarzo, CTO for the Big Data Practice at Dell EMC, introduced a methodology for capturing, enriching and sharing data (and analytics) across the organization...