News, introduction and a bit of History
Hotspots, sunspots, solar flares et al have been in the news since the last few months, first a major one in November 2012, followed by another one in January 2013.
Whilst I have your attention on this topic, did you know the JVM (Java Virtual Machine) also goes by the name Hotspot™, find out why it is called so! HotSpot. Unlike the sun’s solar hotspots, the JVM i.e. Hotspot™ is a much gentler entity! With the new build system i.e. build-infra it is now possible to build and run Hotspot from within Eclipse in a matter of minutes. Further you can even launch your own java-based program using gamma – the Hotspot launcher.
The steps are similar to that of building it via the command-line interface (CLI) – but with the advantage of being able to Run and Debug any line in the Hotspot source-code (it is a C/C++ based component of the OpenJDK project). You can also pass in the same arguments as you would to the java command-line.
In this blog we will cover the following in two parts, a simple import-build-run steps for busy readers (first 8 to 9 sections) followed by a simple hack-build-run-examine (the additional sections) for ones with more spare time in hand:
Building the latest version of OpenJDK locally
Installing Eclipse Juno under Ubuntu 12.04
Importing the C/C++ Hotspot project into Eclipse Juno
Downloading the Hotspot ready-to-go project files (and supporting script files)
Applying the downloaded files to the imported Eclipse project files
Running Eclipse with the Hotspot project loaded
Building Hotspot from within Eclipse
Running Hotspot from within EclipseHacking the java.c program – add your own code to it
Running Hotspot (and loading a simple Demo class) from within Eclipse
Putting breakpoints in java.c within Eclipse
Tracing code and inspecting variables in java.c within Eclipse
Examining the different logs generated during the build and Run/Debug launching processes
Most of these sections will have screen-shots to illustrate the actions to be taken to achieve the results. We cover only basic aspects of the topic so advanced topics won’t be covered here. Having said that it is a step up from the Old Build system, as in the new Build-infra incremental builds have made things faster.
Pre-requisites
The below programs and environments are required, including basic operational knowledge and understanding of them:
Eclipse Juno
Eclipse CDT 6.0
Ubuntu 12.04 (CLI & GUI familiarity)
OpenJDK 8 (a version that builds)
OpenJDK 7 or Java / Javac 7.0 (JRE/JDK 1.7.013 – at the time of writing)
Any class or jar that can be run using the java command.
Basic knowledge of C/C++ (if you are going to make changes to Hotspot)
Basic knowledge of Bash scripting
Building the latest version of OpenJDK locally
In order to download and install the latest OpenJDK system, follow build instructions at http://java.net/projects/adoptopenjdk/pages/AdoptOpenJDKVMBuild.
Installing Eclipse Juno under Ubuntu 12.04
Please ensure you do not have a previous installation of Eclipse on your system, if you do uninstall it before performing the below instructions.
Find out whether you are running a 32- or 64-bits system with the below command:
$ file /bin/bash
Sample output indicating the platform of the system
/bin/bash: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=0xf199a4a89ac968c2e0e99f2410600b9d7e995187, stripped
Quick method
Download the script that does that automatically for us:
$ wget http://bit.ly/YBS8ARRun it with the correct platform parameter using the feedback from the previous step:
$ bash installEclipseJunoCDTForUbuntu12.04.sh 32
or
$ bash installEclipseJunoCDTForUbuntu12.04.sh 64To create a icon in the Unity Dashboard, etc… please refer to the last section of the original blog
— OR —
Step-by-step method
Go to the downloads folder under your $HOME directory:
$ cd ~/Downloads
Download the necessary Eclipse Juno CDT package
32 bits
$ wget http://bit.ly/11QStqT
– or –
64 bits
$ wget http://bit.ly/WSadhy
Untar the downloaded file:
32 bits
$ tar -zxvf eclipse-cpp-juno-SR1-linux-gtk.tar.gz
– or –
64 bits
$ tar -zxvf eclipse-cpp-juno-SR1-linux-gtk-x86_64.tar.gz
This results in a folder named “eclipse”, which should be copied to /opt with
$ sudo mv “eclipse” /opt
Now add a link to the executable in /usr/bin to “/opt/eclipse/eclipse” for easier access.
$ sudo ln -s “/opt/eclipse/eclipse” /usr/bin/eclipse
To do the rest of the optional steps i.e. create a icon in the Unity Dashboard, etc… please refer to the last section of the original blog.
Importing the C/C++ Hotspot project into Eclipse Juno
Run Eclipse
Make it a point to create your own Eclipse workspace for OpenJDK projects calling it something like ‘Eclipse_OpenJDK_Projects’ (without quotes).
File > Import… > select a import source > Existing Code as Makefile Project
Select the hotspot project from the hotspot sub-folder
Select ‘Cross GCC’ as the Toolchain
Once imported, the hotspot project appears in the Project Explorer panel at the left-hand-side.
Now shutdown Eclipse.
Downloading the Hotspot ready-to-go project files (and supporting script files)
Go to the command-prompt and navigate to where the ‘hotspot’ folder is situated (i.e. ~/sources/jdk8_tl/hotspot).
Run this command at the CLI:
$ wget http://bit.ly/Vreesk
Download takes under a minute and the download progress bar should show 100%.
Applying the downloaded files to the imported Eclipse project files
Run the below command from within the hotspot folder:
$ bash downloadEclipseProjectFiles.sh ~/Eclipse_OpenJDK_Projects ~/Eclipse_OpenJDK_Projects/
Once all the scripts and supporting files are downloaded the following visible and hidden files and folders should become available:
.cproject (hidden) .project (hidden) .settings (hidden) .metadata (moved to destination) eclipseScripts
The local copies of .cproject and .project will be overwritten by the above action, along with that the .metadata folder in the Eclipse workspace for the project will also be updated.
Navigate to the eclipseScripts folder and run:
$ bash runEclipseForHotspot.sh ~/Eclipse_OpenJDK_Projects
This should launch Eclipse and take you to your imported project (always launch Eclipse via the script file).
Building Hotspot from within Eclipse
Select the project in the Project Explorer and select the menu option Project > Build Project (in case of first-time build from within Eclipse, please do a Clean Build from within Eclipse before doing a full-build).
Now shutdown Eclipse.
Restart Eclipse using the script mentioned in the above section (see Applying the downloaded…), always launch Eclipse via this script file.
Select the project in the Project Explorer and select the menu option Project > Build Project.
A successful build will result in messages in the build output console:
Running Hotspot from within Eclipse
Select the ‘hotspot’ project from the Project Explorer and select the menu option Run > Run.
Basically running ‘gamma’ through the Eclipse Run/Debug launcher should print out the gamma usage screen which looks like the below (snapshot) and exactly the same as the ‘java –help’ command run from the command prompt:
[Loaded sun.misc.Launcher$AppClassLoader$1 from shared objects file]
[Loaded java.lang.SystemClassLoaderAction from shared objects file]
Usage: gamma [-options] class [args...]
(to execute a class)
or gamma [-options] -jar jarfile [args...]
(to execute a jar file)
where options include:
-cp <class search path of directories and zip/jar files>
-classpath <class search path of directories and zip/jar files>
A : separated list of directories, JAR archives,
and ZIP archives to search for class files.
.
.
.
[Loaded java.lang.Shutdown from shared objects file]
[Loaded java.lang.Shutdown$Lock from shared objects file]
See section “Examining all kinds of logs…” for a detailed version of the above snapshot of the log.
_______________________________________________________________________________________________________________________________________________________________________________
Hacking the java.c program – add your own code to it
The C/C++ code in Hotspot, in this case java.c (found in the hotspot/src/share/launcher folder) isn’t as scary as C/C++ code can be deemed to be – I hope it removes such a phobia about system-level languages. Locate hotspot/src/share/launcher/java.c in Eclipse, open it and go to the section of the code between line numbers 388 and 389 and insert the below block of code:
printf("**********************************\n");
printf("* Simple java.c hack \n");
printf("**********************************\n");
printf("* jre path: %s \n", jrepath);
printf("* jvm path: %s \n", jvmpath);
printf("* Jarfile: %s \n", jarfile);
printf("* Classname: %s \n", classname);
printf("**********************************\n");
which on completion should look like:
Running Hotspot (and loading a simple Demo class) from within Eclipse
If you don’t have a sample class or jar to hand, create one, here is a snapshot of the code behind the demo HelloWorld.java program:
public class HelloWorld {
public static void main(String[] args) {
System.out.println("************************");
System.out.println("* Hello, World *");
System.out.println("************************");
}
}
Save this program say in the $HOME folder, then edit ../hotspot/eclipseScripts/updateEnvVarsForEclipseForHotspot.sh to enable passing the HelloWorld or any other class to the hotspot program’s Run/Debug Launcher – instructions in the file, look for the below block of code (comment and uncomment the relevant lines). If another name or location is chosen for the demo class or program, then make the necessary amendment in the above script file – comments available in the file to guide you (see below).
# Uncomment this line when you have a sample class or jar to pass to gamma # export DEMOCLASS_OR_JAR_ARG="-cp $HOME HelloWorld" # This will invoke gamma to display the usage screen export DEMOCLASS_OR_JAR_ARG="" }
Restart Eclipse using the script mentioned in the above section (see Applying the downloaded…), always launch Eclipse via this script file.
The above lines should produce the below output (snapshot) between two blocks of verbose messages from gamma, which also contains the print messages to the console that were inserted into the java.c unit:
Using java runtime at: /usr/lib/jvm/java-7-openjdk-i386/jre/ ********************************** * Simple java.c hack ********************************** * jre path: /usr/lib/jvm/java-7-openjdk-i386/jre/ * jvm path: /home/saiubuntu/sources/jdk8_tl/build/linux-x86-normal-server-release/hotspot/linux_i486_compiler2/product/libjvm.so * Jarfile: (null) * Classname: HelloWorld ********************************** [Loaded java.lang.Object from shared objects file] [Loaded java.io.Serializable from shared objects file] . . . ************************ * Hello, World * ************************ [Loaded java.lang.Shutdown from shared objects file] [Loaded java.lang.Shutdown$Lock from shared objects file]
See the below section on “Examining all kinds of logs…” for a detailed version of the above snapshot.
Putting breakpoints in java.c within Eclipse
Open java.c in the Eclipse editor, position the cursor on line 392 (or any other line) and double click on the left-hand-side border/bevel of the editor window:
Tracing code and inspecting variables in java.c within Eclipse
Now select the project in the Project Explorer and run the program in Debug mode by clicking on the menu option Run > Debug to launch the Debug perspective:
Adding a Watch Expression and Tracing through the lines of code shows the current value being populated in the Watch Expression window (top right corner). A watch expression can be added by merely select the field or variable in the editor, right-mouse click and selecting Watch Expression from the pop-up menu:
Examining the different logs generated during the build and Run/Debug launching processes
A number of log files were created during the whole process and can be examined, see below (the names of the files describe their contents):
hotspot.eclipse.clean.build.log (console messages generated on Clean Hotspot Build action)
hotspot.eclipse.full.build.log (console messages generated on Full Hotspot Build action)
hotspot.eclipse.incremental.build.log (console messages generated on Incremental Hotspot Build action)
gamma.run_or_debug.Usage.output.log (console messages generated on running gamma in verbose mode, and not launching any program)
gamma.run_or_debug.HelloWorld.Verbose.output.log (console messages generated on running gamma in verbose mode, and launching the HelloWorld program)
A number of advanced hacks (assignment for readers!)
1) Insert debug-level log messages into java.c throughout the unit, rebuild gamma and run the Demo class or any other java-based.
2) Refactor java.c and insert debug-level log messages throughout the unit, rebuild gamma and run the Demo class or any other java-based program.
3) After step 2) above, load a low-latency, GC-tuned java based program, with GC-logs enabled and examine the GC-logs produced, to see if there is any change in performance (for performance tuning buffs).
4) Apply the Elvis operator to javac (a good way to get exposure to ‘how to modify javac?’) and compile a java program with the Elvis operator implemented in it.
5) GC-fun: replace the existing garbage collector(s) with a custom one. Resurrect PermGen or iCMS in the existing code. Add your change you always wanted to, to the existing version of Hotspot (for GC buffs).
6) Change javac to be able to parse and compile new language features or understand another dialect of JVM-based languages or maybe even older programming languages like C, Assembly, Scheme or Smalltalk.
7) Replace the built-in class-loader with your custom version.
References
(1) OpenJDK: README for the New Build System
(2) Hotspot Runtime Overview
(3) HotSpot Internals
(4) Hotspot Docs
(5) OpenJDK build instructions (old build)
(6) Hacking Hotspot in Eclipse – Roman Kennke (old build system)
(7) Biased Locking in HotSpot
(8) HotSpot source: command line arguments
(9) Memory leak profiling with netbeans
(10) HotSpot Tools – HotSpot Internals for OpenJDK – Oracle Wiki
(11) HotSpot development on Linux with NetBeans
(12) Building Hotspot in Eclipse under Ubuntu 12.04 (Old Build system)
Related references
Read about the Adopt OpenJDK program at the the Adopt OpenJDK java.net project website. Join the Adopt OpenJDK mailing list at the google group or send an email to adopt-openjdk@googlegroups.com to subscribe to the mailing list. Find out how to join a Google group.
@adoptopenjdk – follow us on twitter, get the latest Adopt OpenJDK community news!Read about the Adopt-a-JSR program at the java.net project website. Join the Adopt-a-JSR mailing list at the java.net mailing list. Send an email to members@adoptajsr.java.net to subscribe to the mailing list.
@adoptajsr – follow us on twitter, get the latest Adopt-a-JSR community news!
Finally
These instructions will be converted to much compact ‘wiki’ instructions without any of the narrations explanations as mentioned in the blog above – “post any feedback and improvements from the users of the instructions!”. You can post them at the bottom of this blog or tweet them to me at @theNeomatrix369.
‘Thank you’ to those who have helped in the process (@karianna, @RichardWarburto, Girish Balakrishnan, @teozaurus, @SamirTalwar, @sandromancuso and @stPundit). Any (constructive) comments or request for changes are welcome.
I have learnt a lot about the topics covered here i.e. Java, Eclipse and Ubuntu! For those who wish to dive deeper in Hotspot and Java, make use of all the links in the blog, study the bash scripts written to automate the process, above all look into the hotspot folder and take interest in all the files it contains.
Feel free to approach the list of ‘assignment’ hacks, I’ll be more than happy to add links from here to your page if you accomplish any one of them. You will also be mentioned in our community news relayed regularly.
Mani's fun & useful blogs 