Welcome!

Linux Containers Authors: Liz McMillan, Pat Romanski, Elizabeth White, XebiaLabs Blog, Bob Gourley

Related Topics: Linux Containers, Java IoT

Linux Containers: Article

Improving Swing Performance: JIT vs AOT Compilation

Linux native band plays fast

The JFC/Swing API, natively precompiled on Linux for the first time, delivers measurable improvement in Java GUI performance.

The Excelsior Engineering Team has ported Excelsior JET, a Java Virtual Machine (JVM) with an ahead-of-time compiler, to the Linux/x86 platform. As the JET JVM supports the entire J2SE platform API including the Java Foundation Classes (JFC/Swing), Excelsior engineers had an opportunity to evaluate the response time of natively compiled JFC/Swing on Linux. The results of the comparison with conventional, dynamically optimizing JVMs were encouraging: response time has improved by 40% or even doubled on some benchmarks. What's more important is that real-world Swing applications performed perceivably faster.

This article describes Excelsior JET JVM and JFCMark, free benchmark software by Excelsior that measures Swing-based GUI performance. Moreover, the authors share their technical experience in optimizing JFC/Swing and argue why ahead-of-time Java compilation has advantages over dynamic compilation for certain application types.

Two Recipes for Java

The definition of the term "virtual machine" has been revised in the past few years. Modern Java Virtual Machines are no longer just interpreters of the Java bytecode. High performance, state-of-the-art implementations are made up of optimizing compilers that translate bytecode instructions down to the native code that runs directly on the hardware. However, one technical decision distinguishes contemporary JVMs: what's the best time to run the performance engine, an optimizing native compiler? Two options exist: run it before or after the application starts.

Most JVMs initially interpret the program and then analyze how it runs by looking for hot spots, that is, frequently executed portions of bytecode. Hot spots are then compiled to optimized native code during program execution. This approach is called Just-In-Time (JIT) compilation. Other JVMs feature static native compilers similar to traditional C/C++ compilers, enabling developers to optimize their Java applications before execution. For Java, this old trick has a new name - Ahead-Of-Time (AOT) compilation. However, solely static compilation is not enough for Java compatibility. Remember that many Java applications use custom classloaders to load some components or plug-ins at runtime. To have the "J" in "JVM," such virtual machines must be supplied with an interpreter or JIT compiler to handle classes that could not be precompiled.

Either approach alone is not a silver bullet for Java performance. JIT-oriented JVMs can "see through" program execution, which may help them optimize hot code better than static compilers do. In return, AOT-oriented JVMs do not spend execution time on interpretation, profiling, and compilation, so optimized programs run fast from start-up. Not surprisingly, single-loop benchmarks fail to reveal a clear performance winner between the two approaches. Instead of carrying the "microbenchmark war" into the Linux camp, let's take a look at a vital example: the performance of Java GUI applications based on JFC/Swing.

Penguin-Driven JVMs

Both JIT and AOT-oriented implementations are available for Linux. The well-known Sun HotSpot Client and Server JVMs are powered by JIT compilers. BEA WebLogic JRockit and IBM Java 2 Runtime Environment also play in the JIT team. GCJ, the GNU compiler for Java, now supported by Red Hat, and Excelsior JET feature AOT compilation.

At the core of Excelsior JET is a static optimizing compiler that enables developers to transform their Java applications into native executables or shared libraries (.so) on a Linux flavor. The AOT compiler comes with a JET Control Panel (see Figure 1), a graphical front end that makes the product easy to learn and use. The command-line interface provides the integration of Excelsior JET into automated builds. The redistributable JET runtime system includes a JIT compiler to support Java dynamic classloading. Another graphical tool, JetPackII (see Figure 2), enables the rapid creation of installation packages for optimized applications. Excelsior JET supports all Java 2 platform packages up to version 1.4.2.

Another static Java compiler is GCJ, a member of the GNU Compiler Collection. Following the FSF philosophy, GCJ uses a clean-room, free implementation of the Java 2 Platform API. Although most packages are now supported, there are noticeable exceptions such as the Abstract Window Toolkit (AWT), Swing, and some of the APIs introduced to J2SE 1.4. Contributors to the GNU Classpath project are currently implementing the missing packages. The GNU Interpreter for Java complements GCJ to enable Java dynamic loading. The implementation of a JIT compiler is planned for the future.

Accelerated Swing Tempo

GUI response time is in the eye of the beholder. As a result, it's tough to obtain GUI performance scores. To address this problem, Excelsior has developed JFCMark, a free benchmark suite to measure the performance of the JFC/Swing API. The included tests are manipulating with frames, trees, and tables; switching look-and-feels; decoding and drawing images; displaying and scrolling HTML texts; and using Swing layout managers. JFCMark requires JVMs that support the Java 2 Platform at the level of J2SE 1.3, 1.4, or 1.5.

Each test performs its scenario in the main loop through a given number of iterations. This allows you to obtain performance scores in short- and long-running modes. Upon completion, these tests report performance measured in units specific to their specific scenarios, for instance, frames opened per second. When designing JFCMark, we paid close attention to benchmarking accuracy. For a particular configuration, every benchmark always performs the same number of operations independently of the JVM under test. This requirement is achieved through synchronous processing of Swing events, that is, a next event is sent only after the previous one is processed. Therefore, a possible difference in reported speed depends solely on the time of the benchmark execution.

Let's consider the performance of the Swing windowing system that provides operations with frames. We have used a part of JFCMark to test typical manipulations with frames such as opening/closing, dragging, and selecting. The testbed configuration was as follows.]

  • CPU: AMD Athlon running at 1,8 GHz
  • RAM: 512MB DDR SDRAM
  • Video: NVidia GeForce2 MX-200 at 1024x768x65536c
  • OS: Red Hat Linux release 8.0 (Psyche)
  • Linux Kernel: 2.4.18-14
We've run these benchmarks using a spectrum of JVMs available for Linux:
  • Excelsior JET 3.6 Professional Edition with JRE 1.4.2_04
  • Sun Java HotSpot Client VM 1.4.2_04
  • IBM J2RE 1.4.2 Classic with JIT enabled
  • Sun Java HotSpot Server VM 1.4.2_04
  • BEA WebLogic JRockit 8.1 with JRE 1.4.2_04
The server-oriented JVMs (Sun HotSpot Server and BEA WebLogic JRockit) are included for your reference. They're not expected to really shine in client-side application performance. GCJ is not included because it does not yet have support for JFC/Swing. To play fair, we provided results for both long- and short-running configuration of the benchmarks. Figures show the composite performance index in comparison with the reference implementation (Sun HotSpot Client VM). Longer bars mean better performance.

Figure 3 shows Swing performance scores for the long-playing version of JFCMark. For example, 600 frames are opened and closed by one of the tests. In this scenario, JIT-based JVMs have a chance to "warm up," that is, to optimize hot code for maximum speed. Note, however, that this level of performance may be reached only after doing a "good amount of mouse-clicking" in real-world Java GUI applications. Nevertheless, Excelsior JET still outperforms by 40% the first runner-up (HotSpot Client VM).

To see how Swing works on dynamic JVMs that have not been "warmed up" yet, look at Figure 4. It shows out-of-the-box performance scores for the short-running configuration. This case would correspond to the way Java GUI applications perform right after start-up. As can be seen, the JET-compiled Swing runs fast and it runs fast from the start. Other test participants work at least twice as slow for this scenario. Note that short-running benchmarks are mostly important for the client-side application's performance. For instance, if you start a GUI application and drag something with the mouse, you don't want to see it stumbling just because the JVM has not yet done its job. People talking about "snail Java" often ignore the fact that many JVMs take time to warm up. It's not an issue for server-side applications, which typically run for hours and days. However, the performance of client-side applications is most heavily impacted by the JVM warm-up cycle.

JEdit, a Practical Example

If you don't trust vendor benchmarks, check the results yourself. One of the samples that comes with Excelsior JET is a project for compiling jEdit, an open source, cross-platform text editor written in Java. jEdit has many advanced features that make text editing easier, such as syntax highlighting, auto indent, abbreviation expansion, registers, macros, regular expressions, and multiple file search/replace. It's a good example of a full-featured Swing-based application to evaluate GUI response time.

Behind the Performance Figures

An interesting question is why don't the JIT-powered JVMs hit the performance bar raised by Excelsior JET? Of course, aggressive static optimizations and the removal of bytecode interpretation make Swing work smoothly. However, it seems that the long-running mode of JFCMark should be comfortable for the JIT compilers. Where does the 40% performance win come from? We found the answer unexpectedly when we aimed at further improving Swing performance: the absence of hot methods.

Under the covers, JFC/Swing is quite a complex event-driven system implemented on top of AWT. The implementation consists of several hundred classes which, in turn, use a variety of other core classes. Many thousands of Java methods are executed when, for example, you open a Swing frame. When we obtained the Swing execution profile, it proved to be almost flat. Lots of methods were executed but each of them took hundredths of one percent in total execution time. Only a few methods took more than 1-2%. At this point, we encountered an interesting problem: what should we improve in the absence of clear performance bottlenecks? As you may have guessed, the same problem exists for profile-guided JIT compilers. Instead of a few hot spots to be aggressively optimized, there are plenty of "warm spots" that are left intact. The flat execution profile is an application-specific property that some JVMs cannot effectively manage to achieve top performance.

Conclusion

This article is not about fast and slow JVMs. Rather, it demonstrates that for some Java applications, a JVM with AOT compilation can work faster than JIT-based JVMs. The main lesson we have learned from this study is that one size does not fit all and JFC/Swing is not the only example. One way or another, the Java platform wins and we were happy to make Excelsior JET JVM available to Java/Linux developers.

Resources

  • Java Foundation Classes (JFC/Swing): http://java.sun.com/products/jfc/
  • Excelsior JET: www.excelsior-usa.com/jet.html
  • JFCMark: JFC/Swing benchmark with source code: www.excelsior-usa.com/jfcmark.html
  • Sun HotSpot JVMs: http://java.sun.com/j2se/1.4.2
  • IBM J2RE: www-106.ibm.com/developerworks/java/jdk/linux140/
  • BEA WebLogic JRockit: www.bea.com/framework.jsp?CNT=index.htm&FP=/content/products/jrockit
  • GCJ, the GNU compiler for Java: http://gcc.gnu.org/java/
  • GNU Classpath: www.gnu.org/software/classpath/
  • jEdit: www.jedit.org
  • More Stories By Vitaly Mikheev

    Vitaly Mikheev is the chief technology officer for Excelsior, LLC, a company focusing on design and development of optimizing compilers. Vitaly has been involved in software development since 1987 and focused on compiler construction technologies for the last decade. He started working with Java in 1998 as the architect of the Excelsior Java Virtual Machine. Before that, he worked on proprietary optimizing compilers for Nortel Networks. Vitaly is a member of ACM and a co-author of the patent on the garbage collector algorithm implemented in the Samsung's J2ME CDC virtual machine. He holds an MS in computer science from the Novosibirsk State University, Russia.

    Comments (5) View Comments

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    Most Recent Comments
    Blind Earl 11/08/04 09:04:35 AM EST

    Once I actually found the article it was an interesting read. However, this website looks *terrible*. When a column is only 11 characters wide due to advertising squishing it down, it is time to find some other magazine to read.

    the author 11/08/04 07:14:08 AM EST

    To angel'o'sphere:

    1)

    >>A VM based on a JIT compiler loads always only byte code
    >>and compiles the byte-code to native code on the fly
    >>depending on several algorithms/heuristics.
    >>A AOT compiler does JUST THE SAME on the first invocation
    >>of the byte code program, but it saves the generated
    >>native code to be available for later invocations of the
    >>same program. So for the later invocations, the
    >>compilation was ahead of time, not for the first one

    The thing you are talking about is called Caching JIT compilation (at least HotSpot and BEA engineers called it so when I talked with them at JavaOne 2004)

    Besides, Caching JIT compilation does not occur on the first invocation - it would take long long time and miss profile information useful for optimizations.

    2)

    >>The Excelsior JET is no AOT compiler but just a static
    >>compiler. However they seem to jump on the bandwagon
    >>of a new buzzword, because the term AOT is only used
    >>in academic circles currently

    visit http://gcc.gnu.org/java/ (GNU compiler for Java). It's described as an AOT compiler...

    3)

    >>The next flaw is this: they measure a GUI application
    >>over only 600 "loops". As the exact form of the loop
    >>is not published, I have the impression they made the
    >>loop in a way that a "standard" HotSpot JIT does
    >>predictable not even attempt to compile much of the
    >>covered code

    In this test, HotSpot shows 28,9 frames/sec for short-running configuration and 44,6 frames/sec for long-running one. It does not look like "not even attempt to compile much of the covered code". Increasing the number of iteration to open more frames does not improve the HotSpot results. BTW, did you ever open 600+ frames in a Java application?

    4)

    >>In the "benchmark" of the article,
    >>you neither have a loop (that one is in the external
    >>driver program hidden) nor stack samples which occur
    >>often enough to cause a JIT compilation.

    Check the source code of the benchmark. It includes the loop

    for(int j=0;j>IMHO: the Excelsior people tricked the HotSpot/JIT VMs
    >>in not doing their job, and thus you see the big
    >>performance gain.
    >> There are other tricks thinkable, e.g. not using
    >>the standard SWING library but a tweaked one,
    >>just remove some "synchronized" keywords if
    >> you are sure the benchmark does not need them,
    >> and voila ...

    The standard Swing library and the public JET version were used for benchmarking. As for "synchronized" keywords - some of them are *safely* removed by the JET compiler during the course of escape analysis. There is excellent literature about this optimization technique (for example, see ACM OOPSLA'99 - about 5 papers were devoted to it).

    Take care,

    --Vitaly

    P.S. JITs work well on servers and reusing the same machinery on the desktop is not always appropriate.

    angel'o'sphere 11/08/04 05:28:31 AM EST

    I don't like the article.
    For several reasons I think they cheated and they coin old terms into new terms for no reason (except publicity). E.g. they mix up AOT with static compilation.
    AOT is an extension of JIT compilation, and not a compilation done ahead of the first execution.
    A VM based on a JIT compiler loads always only byte code and compiles the byte-code to native code on the fly depending on several algorithms/heuristics.
    A AOT compiler does JUST THE SAME on the first invocation of the byte code program, but it saves the generated native code to be available for later invocations of the same program. So for the later invocations, the compilation was ahead of time, not for the first one.
    The Excelsior JET is no AOT compiler but just a static compiler. However they seem to jump on the bandwagon of a new buzzword, because the term AOT is only used in academic circles currently (besides that Java 5.0 tries to cash compile informations and even native code over invocations).
    The next flaw is this: they measure a GUI application over only 600 "loops". As the exact form of the loop is not published, I have the impression they made the loop in a way that a "standard" HotSpot JIT does predictable not even attempt to compile much of the covered code.
    To understand that you have to know how a HotSpot JIT works. Two "big" strategies are used: fiddling with loops, this is a short code fragment, inside of one method, consider it as a for loop or while loop. You can do different things with it ... not interesting here.
    The second big thing is call stack sampling. Suppose you have very often the method calls A, B, C on the call stack, the VM decides to check if anything can be inlined or if any of those methods is jitted.
    In the "benchmark" of the article, you neither have a loop (that one is in the external driver program hidden) nor stack samples which occur often enough to cause a JIT compilation.
    Excellent literature about JIT compilation and strategies can be found here: http://www.research.ibm.com/jalapeno/publication.h tml and here: http://www-124.ibm.com/developerworks/oss/jikesrvm /info/papers.shtml
    IMHO: the Excelsior people tricked the HotSpot/JIT VMs in not doing their job, and thus you see the big performance gain. There are other tricks thinkable, e.g. not using the standard SWING library but a tweaked one, just remove some "synchronized" keywords if you are sure the benchmark does not need them, and voila ...

    Seun Osewa 11/07/04 02:32:53 PM EST

    combining the two approaches will lead to overhead similar to a JIT approach (hence, there's nothing to be gained for a long-running program)

    RAMMS+EIN 11/06/04 05:40:50 AM EST

    Why compare JIT against AOT? Why not have both?

    AOT compilation makes for fast start up time and fast run time. JIT advocates claim that it can lead to better performance, as more optimizations can be performed with run-time information. So why not combine the two? Compile it before the first run, and further optimize it at run-time where appropriate. That way, you get the best of both worlds.

    @ThingsExpo Stories
    Internet of @ThingsExpo has announced today that Chris Matthieu has been named tech chair of Internet of @ThingsExpo 2017 New York The 7th Internet of @ThingsExpo will take place on June 6-8, 2017, at the Javits Center in New York City, New York. Chris Matthieu is the co-founder and CTO of Octoblu, a revolutionary real-time IoT platform recently acquired by Citrix. Octoblu connects things, systems, people and clouds to a global mesh network allowing users to automate and control design flo...
    Unless your company can spend a lot of money on new technology, re-engineering your environment and hiring a comprehensive cybersecurity team, you will most likely move to the cloud or seek external service partnerships. In his session at 18th Cloud Expo, Darren Guccione, CEO of Keeper Security, revealed what you need to know when it comes to encryption in the cloud.
    With 15% of enterprises adopting a hybrid IT strategy, you need to set a plan to integrate hybrid cloud throughout your infrastructure. In his session at 18th Cloud Expo, Steven Dreher, Director of Solutions Architecture at Green House Data, discussed how to plan for shifting resource requirements, overcome challenges, and implement hybrid IT alongside your existing data center assets. Highlights included anticipating workload, cost and resource calculations, integrating services on both sides...
    "We're a cybersecurity firm that specializes in engineering security solutions both at the software and hardware level. Security cannot be an after-the-fact afterthought, which is what it's become," stated Richard Blech, Chief Executive Officer at Secure Channels, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
    According to Forrester Research, every business will become either a digital predator or digital prey by 2020. To avoid demise, organizations must rapidly create new sources of value in their end-to-end customer experiences. True digital predators also must break down information and process silos and extend digital transformation initiatives to empower employees with the digital resources needed to win, serve, and retain customers.
    The WebRTC Summit New York, to be held June 6-8, 2017, at the Javits Center in New York City, NY, announces that its Call for Papers is now open. Topics include all aspects of improving IT delivery by eliminating waste through automated business models leveraging cloud technologies. WebRTC Summit is co-located with 20th International Cloud Expo and @ThingsExpo. WebRTC is the future of browser-to-browser communications, and continues to make inroads into the traditional, difficult, plug-in web co...
    In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
    The Internet of Things (IoT) promises to simplify and streamline our lives by automating routine tasks that distract us from our goals. This promise is based on the ubiquitous deployment of smart, connected devices that link everything from industrial control systems to automobiles to refrigerators. Unfortunately, comparatively few of the devices currently deployed have been developed with an eye toward security, and as the DDoS attacks of late October 2016 have demonstrated, this oversight can ...
    What happens when the different parts of a vehicle become smarter than the vehicle itself? As we move toward the era of smart everything, hundreds of entities in a vehicle that communicate with each other, the vehicle and external systems create a need for identity orchestration so that all entities work as a conglomerate. Much like an orchestra without a conductor, without the ability to secure, control, and connect the link between a vehicle’s head unit, devices, and systems and to manage the ...
    "Once customers get a year into their IoT deployments, they start to realize that they may have been shortsighted in the ways they built out their deployment and the key thing I see a lot of people looking at is - how can I take equipment data, pull it back in an IoT solution and show it in a dashboard," stated Dave McCarthy, Director of Products at Bsquare Corporation, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
    Data is the fuel that drives the machine learning algorithmic engines and ultimately provides the business value. In his session at Cloud Expo, Ed Featherston, a director and senior enterprise architect at Collaborative Consulting, discussed the key considerations around quality, volume, timeliness, and pedigree that must be dealt with in order to properly fuel that engine.
    Businesses and business units of all sizes can benefit from cloud computing, but many don't want the cost, performance and security concerns of public cloud nor the complexity of building their own private clouds. Today, some cloud vendors are using artificial intelligence (AI) to simplify cloud deployment and management. In his session at 20th Cloud Expo, Ajay Gulati, Co-founder and CEO of ZeroStack, will discuss how AI can simplify cloud operations. He will cover the following topics: why clou...
    As data explodes in quantity, importance and from new sources, the need for managing and protecting data residing across physical, virtual, and cloud environments grow with it. Managing data includes protecting it, indexing and classifying it for true, long-term management, compliance and E-Discovery. Commvault can ensure this with a single pane of glass solution – whether in a private cloud, a Service Provider delivered public cloud or a hybrid cloud environment – across the heterogeneous enter...
    Everyone knows that truly innovative companies learn as they go along, pushing boundaries in response to market changes and demands. What's more of a mystery is how to balance innovation on a fresh platform built from scratch with the legacy tech stack, product suite and customers that continue to serve as the business' foundation. In his General Session at 19th Cloud Expo, Michael Chambliss, Head of Engineering at ReadyTalk, discussed why and how ReadyTalk diverted from healthy revenue and mor...
    The many IoT deployments around the world are busy integrating smart devices and sensors into their enterprise IT infrastructures. Yet all of this technology – and there are an amazing number of choices – is of no use without the software to gather, communicate, and analyze the new data flows. Without software, there is no IT. In this power panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, Dave McCarthy, Director of Products at Bsquare Corporation; Alan Williamson, Principal...
    The 20th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held June 6-8, 2017, at the Javits Center in New York City, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Containers, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportunity. Submit your speaking proposal ...
    You have great SaaS business app ideas. You want to turn your idea quickly into a functional and engaging proof of concept. You need to be able to modify it to meet customers' needs, and you need to deliver a complete and secure SaaS application. How could you achieve all the above and yet avoid unforeseen IT requirements that add unnecessary cost and complexity? You also want your app to be responsive in any device at any time. In his session at 19th Cloud Expo, Mark Allen, General Manager of...
    Successful digital transformation requires new organizational competencies and capabilities. Research tells us that the biggest impediment to successful transformation is human; consequently, the biggest enabler is a properly skilled and empowered workforce. In the digital age, new individual and collective competencies are required. In his session at 19th Cloud Expo, Bob Newhouse, CEO and founder of Agilitiv, drew together recent research and lessons learned from emerging and established compa...
    "IoT is going to be a huge industry with a lot of value for end users, for industries, for consumers, for manufacturers. How can we use cloud to effectively manage IoT applications," stated Ian Khan, Innovation & Marketing Manager at Solgeniakhela, in this SYS-CON.tv interview at @ThingsExpo, held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA.
    Financial Technology has become a topic of intense interest throughout the cloud developer and enterprise IT communities. Accordingly, attendees at the upcoming 20th Cloud Expo at the Javits Center in New York, June 6-8, 2017, will find fresh new content in a new track called FinTech.