Welcome!

Linux Containers Authors: Elizabeth White, Pat Romanski, Yeshim Deniz, Liz McMillan, Amit Gupta

Related Topics: Linux Containers, Java IoT

Linux Containers: Article

Improving Swing Performance: JIT vs AOT Compilation

Linux native band plays fast

The JFC/Swing API, natively precompiled on Linux for the first time, delivers measurable improvement in Java GUI performance.

The Excelsior Engineering Team has ported Excelsior JET, a Java Virtual Machine (JVM) with an ahead-of-time compiler, to the Linux/x86 platform. As the JET JVM supports the entire J2SE platform API including the Java Foundation Classes (JFC/Swing), Excelsior engineers had an opportunity to evaluate the response time of natively compiled JFC/Swing on Linux. The results of the comparison with conventional, dynamically optimizing JVMs were encouraging: response time has improved by 40% or even doubled on some benchmarks. What's more important is that real-world Swing applications performed perceivably faster.

This article describes Excelsior JET JVM and JFCMark, free benchmark software by Excelsior that measures Swing-based GUI performance. Moreover, the authors share their technical experience in optimizing JFC/Swing and argue why ahead-of-time Java compilation has advantages over dynamic compilation for certain application types.

Two Recipes for Java

The definition of the term "virtual machine" has been revised in the past few years. Modern Java Virtual Machines are no longer just interpreters of the Java bytecode. High performance, state-of-the-art implementations are made up of optimizing compilers that translate bytecode instructions down to the native code that runs directly on the hardware. However, one technical decision distinguishes contemporary JVMs: what's the best time to run the performance engine, an optimizing native compiler? Two options exist: run it before or after the application starts.

Most JVMs initially interpret the program and then analyze how it runs by looking for hot spots, that is, frequently executed portions of bytecode. Hot spots are then compiled to optimized native code during program execution. This approach is called Just-In-Time (JIT) compilation. Other JVMs feature static native compilers similar to traditional C/C++ compilers, enabling developers to optimize their Java applications before execution. For Java, this old trick has a new name - Ahead-Of-Time (AOT) compilation. However, solely static compilation is not enough for Java compatibility. Remember that many Java applications use custom classloaders to load some components or plug-ins at runtime. To have the "J" in "JVM," such virtual machines must be supplied with an interpreter or JIT compiler to handle classes that could not be precompiled.

Either approach alone is not a silver bullet for Java performance. JIT-oriented JVMs can "see through" program execution, which may help them optimize hot code better than static compilers do. In return, AOT-oriented JVMs do not spend execution time on interpretation, profiling, and compilation, so optimized programs run fast from start-up. Not surprisingly, single-loop benchmarks fail to reveal a clear performance winner between the two approaches. Instead of carrying the "microbenchmark war" into the Linux camp, let's take a look at a vital example: the performance of Java GUI applications based on JFC/Swing.

Penguin-Driven JVMs

Both JIT and AOT-oriented implementations are available for Linux. The well-known Sun HotSpot Client and Server JVMs are powered by JIT compilers. BEA WebLogic JRockit and IBM Java 2 Runtime Environment also play in the JIT team. GCJ, the GNU compiler for Java, now supported by Red Hat, and Excelsior JET feature AOT compilation.

At the core of Excelsior JET is a static optimizing compiler that enables developers to transform their Java applications into native executables or shared libraries (.so) on a Linux flavor. The AOT compiler comes with a JET Control Panel (see Figure 1), a graphical front end that makes the product easy to learn and use. The command-line interface provides the integration of Excelsior JET into automated builds. The redistributable JET runtime system includes a JIT compiler to support Java dynamic classloading. Another graphical tool, JetPackII (see Figure 2), enables the rapid creation of installation packages for optimized applications. Excelsior JET supports all Java 2 platform packages up to version 1.4.2.

Another static Java compiler is GCJ, a member of the GNU Compiler Collection. Following the FSF philosophy, GCJ uses a clean-room, free implementation of the Java 2 Platform API. Although most packages are now supported, there are noticeable exceptions such as the Abstract Window Toolkit (AWT), Swing, and some of the APIs introduced to J2SE 1.4. Contributors to the GNU Classpath project are currently implementing the missing packages. The GNU Interpreter for Java complements GCJ to enable Java dynamic loading. The implementation of a JIT compiler is planned for the future.

Accelerated Swing Tempo

GUI response time is in the eye of the beholder. As a result, it's tough to obtain GUI performance scores. To address this problem, Excelsior has developed JFCMark, a free benchmark suite to measure the performance of the JFC/Swing API. The included tests are manipulating with frames, trees, and tables; switching look-and-feels; decoding and drawing images; displaying and scrolling HTML texts; and using Swing layout managers. JFCMark requires JVMs that support the Java 2 Platform at the level of J2SE 1.3, 1.4, or 1.5.

Each test performs its scenario in the main loop through a given number of iterations. This allows you to obtain performance scores in short- and long-running modes. Upon completion, these tests report performance measured in units specific to their specific scenarios, for instance, frames opened per second. When designing JFCMark, we paid close attention to benchmarking accuracy. For a particular configuration, every benchmark always performs the same number of operations independently of the JVM under test. This requirement is achieved through synchronous processing of Swing events, that is, a next event is sent only after the previous one is processed. Therefore, a possible difference in reported speed depends solely on the time of the benchmark execution.

Let's consider the performance of the Swing windowing system that provides operations with frames. We have used a part of JFCMark to test typical manipulations with frames such as opening/closing, dragging, and selecting. The testbed configuration was as follows.]

  • CPU: AMD Athlon running at 1,8 GHz
  • RAM: 512MB DDR SDRAM
  • Video: NVidia GeForce2 MX-200 at 1024x768x65536c
  • OS: Red Hat Linux release 8.0 (Psyche)
  • Linux Kernel: 2.4.18-14
We've run these benchmarks using a spectrum of JVMs available for Linux:
  • Excelsior JET 3.6 Professional Edition with JRE 1.4.2_04
  • Sun Java HotSpot Client VM 1.4.2_04
  • IBM J2RE 1.4.2 Classic with JIT enabled
  • Sun Java HotSpot Server VM 1.4.2_04
  • BEA WebLogic JRockit 8.1 with JRE 1.4.2_04
The server-oriented JVMs (Sun HotSpot Server and BEA WebLogic JRockit) are included for your reference. They're not expected to really shine in client-side application performance. GCJ is not included because it does not yet have support for JFC/Swing. To play fair, we provided results for both long- and short-running configuration of the benchmarks. Figures show the composite performance index in comparison with the reference implementation (Sun HotSpot Client VM). Longer bars mean better performance.

Figure 3 shows Swing performance scores for the long-playing version of JFCMark. For example, 600 frames are opened and closed by one of the tests. In this scenario, JIT-based JVMs have a chance to "warm up," that is, to optimize hot code for maximum speed. Note, however, that this level of performance may be reached only after doing a "good amount of mouse-clicking" in real-world Java GUI applications. Nevertheless, Excelsior JET still outperforms by 40% the first runner-up (HotSpot Client VM).

To see how Swing works on dynamic JVMs that have not been "warmed up" yet, look at Figure 4. It shows out-of-the-box performance scores for the short-running configuration. This case would correspond to the way Java GUI applications perform right after start-up. As can be seen, the JET-compiled Swing runs fast and it runs fast from the start. Other test participants work at least twice as slow for this scenario. Note that short-running benchmarks are mostly important for the client-side application's performance. For instance, if you start a GUI application and drag something with the mouse, you don't want to see it stumbling just because the JVM has not yet done its job. People talking about "snail Java" often ignore the fact that many JVMs take time to warm up. It's not an issue for server-side applications, which typically run for hours and days. However, the performance of client-side applications is most heavily impacted by the JVM warm-up cycle.

JEdit, a Practical Example

If you don't trust vendor benchmarks, check the results yourself. One of the samples that comes with Excelsior JET is a project for compiling jEdit, an open source, cross-platform text editor written in Java. jEdit has many advanced features that make text editing easier, such as syntax highlighting, auto indent, abbreviation expansion, registers, macros, regular expressions, and multiple file search/replace. It's a good example of a full-featured Swing-based application to evaluate GUI response time.

Behind the Performance Figures

An interesting question is why don't the JIT-powered JVMs hit the performance bar raised by Excelsior JET? Of course, aggressive static optimizations and the removal of bytecode interpretation make Swing work smoothly. However, it seems that the long-running mode of JFCMark should be comfortable for the JIT compilers. Where does the 40% performance win come from? We found the answer unexpectedly when we aimed at further improving Swing performance: the absence of hot methods.

Under the covers, JFC/Swing is quite a complex event-driven system implemented on top of AWT. The implementation consists of several hundred classes which, in turn, use a variety of other core classes. Many thousands of Java methods are executed when, for example, you open a Swing frame. When we obtained the Swing execution profile, it proved to be almost flat. Lots of methods were executed but each of them took hundredths of one percent in total execution time. Only a few methods took more than 1-2%. At this point, we encountered an interesting problem: what should we improve in the absence of clear performance bottlenecks? As you may have guessed, the same problem exists for profile-guided JIT compilers. Instead of a few hot spots to be aggressively optimized, there are plenty of "warm spots" that are left intact. The flat execution profile is an application-specific property that some JVMs cannot effectively manage to achieve top performance.

Conclusion

This article is not about fast and slow JVMs. Rather, it demonstrates that for some Java applications, a JVM with AOT compilation can work faster than JIT-based JVMs. The main lesson we have learned from this study is that one size does not fit all and JFC/Swing is not the only example. One way or another, the Java platform wins and we were happy to make Excelsior JET JVM available to Java/Linux developers.

Resources

  • Java Foundation Classes (JFC/Swing): http://java.sun.com/products/jfc/
  • Excelsior JET: www.excelsior-usa.com/jet.html
  • JFCMark: JFC/Swing benchmark with source code: www.excelsior-usa.com/jfcmark.html
  • Sun HotSpot JVMs: http://java.sun.com/j2se/1.4.2
  • IBM J2RE: www-106.ibm.com/developerworks/java/jdk/linux140/
  • BEA WebLogic JRockit: www.bea.com/framework.jsp?CNT=index.htm&FP=/content/products/jrockit
  • GCJ, the GNU compiler for Java: http://gcc.gnu.org/java/
  • GNU Classpath: www.gnu.org/software/classpath/
  • jEdit: www.jedit.org
  • More Stories By Vitaly Mikheev

    Vitaly Mikheev is the chief technology officer for Excelsior, LLC, a company focusing on design and development of optimizing compilers. Vitaly has been involved in software development since 1987 and focused on compiler construction technologies for the last decade. He started working with Java in 1998 as the architect of the Excelsior Java Virtual Machine. Before that, he worked on proprietary optimizing compilers for Nortel Networks. Vitaly is a member of ACM and a co-author of the patent on the garbage collector algorithm implemented in the Samsung's J2ME CDC virtual machine. He holds an MS in computer science from the Novosibirsk State University, Russia.

    Comments (5) View Comments

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    Most Recent Comments
    Blind Earl 11/08/04 09:04:35 AM EST

    Once I actually found the article it was an interesting read. However, this website looks *terrible*. When a column is only 11 characters wide due to advertising squishing it down, it is time to find some other magazine to read.

    the author 11/08/04 07:14:08 AM EST

    To angel'o'sphere:

    1)

    >>A VM based on a JIT compiler loads always only byte code
    >>and compiles the byte-code to native code on the fly
    >>depending on several algorithms/heuristics.
    >>A AOT compiler does JUST THE SAME on the first invocation
    >>of the byte code program, but it saves the generated
    >>native code to be available for later invocations of the
    >>same program. So for the later invocations, the
    >>compilation was ahead of time, not for the first one

    The thing you are talking about is called Caching JIT compilation (at least HotSpot and BEA engineers called it so when I talked with them at JavaOne 2004)

    Besides, Caching JIT compilation does not occur on the first invocation - it would take long long time and miss profile information useful for optimizations.

    2)

    >>The Excelsior JET is no AOT compiler but just a static
    >>compiler. However they seem to jump on the bandwagon
    >>of a new buzzword, because the term AOT is only used
    >>in academic circles currently

    visit http://gcc.gnu.org/java/ (GNU compiler for Java). It's described as an AOT compiler...

    3)

    >>The next flaw is this: they measure a GUI application
    >>over only 600 "loops". As the exact form of the loop
    >>is not published, I have the impression they made the
    >>loop in a way that a "standard" HotSpot JIT does
    >>predictable not even attempt to compile much of the
    >>covered code

    In this test, HotSpot shows 28,9 frames/sec for short-running configuration and 44,6 frames/sec for long-running one. It does not look like "not even attempt to compile much of the covered code". Increasing the number of iteration to open more frames does not improve the HotSpot results. BTW, did you ever open 600+ frames in a Java application?

    4)

    >>In the "benchmark" of the article,
    >>you neither have a loop (that one is in the external
    >>driver program hidden) nor stack samples which occur
    >>often enough to cause a JIT compilation.

    Check the source code of the benchmark. It includes the loop

    for(int j=0;j>IMHO: the Excelsior people tricked the HotSpot/JIT VMs
    >>in not doing their job, and thus you see the big
    >>performance gain.
    >> There are other tricks thinkable, e.g. not using
    >>the standard SWING library but a tweaked one,
    >>just remove some "synchronized" keywords if
    >> you are sure the benchmark does not need them,
    >> and voila ...

    The standard Swing library and the public JET version were used for benchmarking. As for "synchronized" keywords - some of them are *safely* removed by the JET compiler during the course of escape analysis. There is excellent literature about this optimization technique (for example, see ACM OOPSLA'99 - about 5 papers were devoted to it).

    Take care,

    --Vitaly

    P.S. JITs work well on servers and reusing the same machinery on the desktop is not always appropriate.

    angel'o'sphere 11/08/04 05:28:31 AM EST

    I don't like the article.
    For several reasons I think they cheated and they coin old terms into new terms for no reason (except publicity). E.g. they mix up AOT with static compilation.
    AOT is an extension of JIT compilation, and not a compilation done ahead of the first execution.
    A VM based on a JIT compiler loads always only byte code and compiles the byte-code to native code on the fly depending on several algorithms/heuristics.
    A AOT compiler does JUST THE SAME on the first invocation of the byte code program, but it saves the generated native code to be available for later invocations of the same program. So for the later invocations, the compilation was ahead of time, not for the first one.
    The Excelsior JET is no AOT compiler but just a static compiler. However they seem to jump on the bandwagon of a new buzzword, because the term AOT is only used in academic circles currently (besides that Java 5.0 tries to cash compile informations and even native code over invocations).
    The next flaw is this: they measure a GUI application over only 600 "loops". As the exact form of the loop is not published, I have the impression they made the loop in a way that a "standard" HotSpot JIT does predictable not even attempt to compile much of the covered code.
    To understand that you have to know how a HotSpot JIT works. Two "big" strategies are used: fiddling with loops, this is a short code fragment, inside of one method, consider it as a for loop or while loop. You can do different things with it ... not interesting here.
    The second big thing is call stack sampling. Suppose you have very often the method calls A, B, C on the call stack, the VM decides to check if anything can be inlined or if any of those methods is jitted.
    In the "benchmark" of the article, you neither have a loop (that one is in the external driver program hidden) nor stack samples which occur often enough to cause a JIT compilation.
    Excellent literature about JIT compilation and strategies can be found here: http://www.research.ibm.com/jalapeno/publication.h tml and here: http://www-124.ibm.com/developerworks/oss/jikesrvm /info/papers.shtml
    IMHO: the Excelsior people tricked the HotSpot/JIT VMs in not doing their job, and thus you see the big performance gain. There are other tricks thinkable, e.g. not using the standard SWING library but a tweaked one, just remove some "synchronized" keywords if you are sure the benchmark does not need them, and voila ...

    Seun Osewa 11/07/04 02:32:53 PM EST

    combining the two approaches will lead to overhead similar to a JIT approach (hence, there's nothing to be gained for a long-running program)

    RAMMS+EIN 11/06/04 05:40:50 AM EST

    Why compare JIT against AOT? Why not have both?

    AOT compilation makes for fast start up time and fast run time. JIT advocates claim that it can lead to better performance, as more optimizations can be performed with run-time information. So why not combine the two? Compile it before the first run, and further optimize it at run-time where appropriate. That way, you get the best of both worlds.

    @ThingsExpo Stories
    Recently, REAN Cloud built a digital concierge for a North Carolina hospital that had observed that most patient call button questions were repetitive. In addition, the paper-based process used to measure patient health metrics was laborious, not in real-time and sometimes error-prone. In their session at 21st Cloud Expo, Sean Finnerty, Executive Director, Practice Lead, Health Care & Life Science at REAN Cloud, and Dr. S.P.T. Krishnan, Principal Architect at REAN Cloud, will discuss how they b...
    Enterprises have taken advantage of IoT to achieve important revenue and cost advantages. What is less apparent is how incumbent enterprises operating at scale have, following success with IoT, built analytic, operations management and software development capabilities – ranging from autonomous vehicles to manageable robotics installations. They have embraced these capabilities as if they were Silicon Valley startups. As a result, many firms employ new business models that place enormous impor...
    SYS-CON Events announced today that Dasher Technologies will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Dasher Technologies, Inc. ® is a premier IT solution provider that delivers expert technical resources along with trusted account executives to architect and deliver complete IT solutions and services to help our clients execute their goals, plans and objectives. Since 1999, we'v...
    SYS-CON Events announced today that MIRAI Inc. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MIRAI Inc. are IT consultants from the public sector whose mission is to solve social issues by technology and innovation and to create a meaningful future for people.
    SYS-CON Events announced today that TidalScale, a leading provider of systems and services, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TidalScale has been involved in shaping the computing landscape. They've designed, developed and deployed some of the most important and successful systems and services in the history of the computing industry - internet, Ethernet, operating s...
    SYS-CON Events announced today that TidalScale will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TidalScale is the leading provider of Software-Defined Servers that bring flexibility to modern data centers by right-sizing servers on the fly to fit any data set or workload. TidalScale’s award-winning inverse hypervisor technology combines multiple commodity servers (including their ass...
    Amazon is pursuing new markets and disrupting industries at an incredible pace. Almost every industry seems to be in its crosshairs. Companies and industries that once thought they were safe are now worried about being “Amazoned.”. The new watch word should be “Be afraid. Be very afraid.” In his session 21st Cloud Expo, Chris Kocher, a co-founder of Grey Heron, will address questions such as: What new areas is Amazon disrupting? How are they doing this? Where are they likely to go? What are th...
    SYS-CON Events announced today that IBM has been named “Diamond Sponsor” of SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California.
    Infoblox delivers Actionable Network Intelligence to enterprise, government, and service provider customers around the world. They are the industry leader in DNS, DHCP, and IP address management, the category known as DDI. We empower thousands of organizations to control and secure their networks from the core-enabling them to increase efficiency and visibility, improve customer service, and meet compliance requirements.
    Join IBM November 1 at 21st Cloud Expo at the Santa Clara Convention Center in Santa Clara, CA, and learn how IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Cognitive analysis impacts today’s systems with unparalleled ability that were previously available only to manned, back-end operations. Thanks to cloud processing, IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Imagine a robot vacuum that becomes your personal assistant tha...
    SYS-CON Events announced today that IBM has been named “Diamond Sponsor” of SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California.
    In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, will lead you through the exciting evolution of the cloud. He'll look at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering ...
    SYS-CON Events announced today that N3N will exhibit at SYS-CON's @ThingsExpo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. N3N’s solutions increase the effectiveness of operations and control centers, increase the value of IoT investments, and facilitate real-time operational decision making. N3N enables operations teams with a four dimensional digital “big board” that consolidates real-time live video feeds alongside IoT sensor data a...
    In a recent survey, Sumo Logic surveyed 1,500 customers who employ cloud services such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP). According to the survey, a quarter of the respondents have already deployed Docker containers and nearly as many (23 percent) are employing the AWS Lambda serverless computing framework. It’s clear: serverless is here to stay. The adoption does come with some needed changes, within both application development and operations. Tha...
    SYS-CON Events announced today that Avere Systems, a leading provider of enterprise storage for the hybrid cloud, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Avere delivers a more modern architectural approach to storage that doesn't require the overprovisioning of storage capacity to achieve performance, overspending on expensive storage media for inactive data or the overbui...
    With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
    Smart cities have the potential to change our lives at so many levels for citizens: less pollution, reduced parking obstacles, better health, education and more energy savings. Real-time data streaming and the Internet of Things (IoT) possess the power to turn this vision into a reality. However, most organizations today are building their data infrastructure to focus solely on addressing immediate business needs vs. a platform capable of quickly adapting emerging technologies to address future ...
    Digital transformation is changing the face of business. The IDC predicts that enterprises will commit to a massive new scale of digital transformation, to stake out leadership positions in the "digital transformation economy." Accordingly, attendees at the upcoming Cloud Expo | @ThingsExpo at the Santa Clara Convention Center in Santa Clara, CA, Oct 31-Nov 2, will find fresh new content in a new track called Enterprise Cloud & Digital Transformation.
    Most technology leaders, contemporary and from the hardware era, are reshaping their businesses to do software. They hope to capture value from emerging technologies such as IoT, SDN, and AI. Ultimately, irrespective of the vertical, it is about deriving value from independent software applications participating in an ecosystem as one comprehensive solution. In his session at @ThingsExpo, Kausik Sridhar, founder and CTO of Pulzze Systems, will discuss how given the magnitude of today's applicati...
    SYS-CON Events announced today that NetApp has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. NetApp is the data authority for hybrid cloud. NetApp provides a full range of hybrid cloud data services that simplify management of applications and data across cloud and on-premises environments to accelerate digital transformation. Together with their partners, NetApp emp...