Why not build #OpenJDK 9 using #Docker ? – Part 2 of 2

…continuing from Why not build #OpenJDK 9 using #Docker ? – Part 1 of 2.

I ran into a number of issues and you can see from my commits, I pulled myself out of it, but to run this Dockerfile from the command-line I used this instruction:

$ docker build -t neomatrix369/openjdk9 .

you can also do it using the below if you have not set your permissions:

$ sudo docker build -t neomatrix369/openjdk9 .

and get the below (summarised) output:

Sending build context to Docker daemon 3.072 kB
Sending build context to Docker daemon 
Step 0 : FROM phusion/baseimage:latest
 ---> 5a14c1498ff4
Step 1 : MAINTAINER Mani Sarkar (from @adoptopenjdk)
 ---> Using cache
 ---> 95e30b7f52b9
Step 2 : RUN apt-get update &&   apt-get install -y     libxt-dev zip pkg-config libX11-dev libxext-dev     libxrender-dev libxtst-dev libasound2-dev libcups2-dev libfreetype6-dev &&   rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 1ea3bbb15c2d
Step 3 : RUN apt-get update
 ---> Using cache
 ---> 6c3938f4d23d
Step 4 : RUN apt-get install -y mercurial ca-certificates-java build-essential
 ---> Using cache
 ---> e3f99b5a3bd3
Step 5 : RUN cd /tmp &&   hg clone http://hg.openjdk.java.net/jdk9/jdk9 openjdk9 &&   cd openjdk9 &&   sh ./get_source.sh
 ---> Using cache
 ---> 26cfaf16b9fa
Step 6 : RUN apt-get install -y wget &&   wget --no-check-certificate --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u45-b14/jdk-8u45-linux-x64.tar.gz
 ---> Using cache
 ---> 696889250fed
Step 7 : RUN tar zxvf jdk-8u45-linux-x64.tar.gz -C /opt
 ---> Using cache
 ---> c25cc9201c1b
Step 8 : RUN cd /tmp/openjdk9 &&   bash ./configure --with-cacerts-file=/etc/ssl/certs/java/cacerts --with-boot-jdk=/opt/jdk1.8.0_45
 ---> Using cache
 ---> 4e425de379e6
Step 9 : RUN cd /tmp/openjdk9 &&   make clean images
 ---> Using cache
 ---> 2d9e17c870be
Step 10 : RUN cd /tmp/openjdk9 &&   cp -a build/linux-x86_64-normal-server-release/images/jdk     /opt/openjdk9
 ---> Using cache
 ---> 9250fac9b500
Step 11 : RUN cd /tmp/openjdk9 &&   find /opt/openjdk9 -type f -exec chmod a+r {} + &&   find /opt/openjdk9 -type d -exec chmod a+rx {} +
 ---> Using cache
 ---> d0c597d045d4
Step 12 : ENV PATH /opt/openjdk9/bin:$PATH
 ---> Using cache
 ---> 3965c3e47855
Step 13 : ENV JAVA_HOME /opt/openjdk9
 ---> Using cache
 ---> 5877e8efd939
Successfully built 5877e8efd939

The above action creates an image which is stored in your local repository (use docker images to enlist the images in the repo). If you want to load the image into a container, and access the files it has built or see anything else, do the below:

$ sudo docker run -it --name openjdk9 neomatrix369/openjdk9 /bin/bash

this will take you to a bash prompt into the container and you can run any of your linux commands and access the file system.

Explaining docker run

$ sudo docker run -it --name openjdk9 neomatrix369/openjdk9 java -version

will show you this

openjdk version "1.9.0-internal"
OpenJDK Runtime Environment (build 1.9.0-internal-_2015_06_04_06_46-b00)
OpenJDK 64-Bit Server VM (build 1.9.0-internal-_2015_06_04_06_46-b00, mixed mode)

Here’s a breakdown of the docker run command:

docker run The command to create and start a new Docker container.
-it To run in interactive mode, so you can see the after running the container.
neomatrix369/openjdk9 This is a reference to the image tag by name (which we created above).
java -version Runs the java command asking its version, inside the containing, which is assisted by the two environment variables PATH and JAVA_HOME which was set in the Dockerfile above.

Footnotes

You might have noticed I grouped very specific instructions with each step, especially the RUN commands, its because, each time I got one of these wrong, it would re-execute the step again, including the steps that ran fine and didn’t need re-executing. Not only is this unnecessary its not using our resources efficiently which is what Docker brings us. So any addition, edition or deletion to any step will only result in that step being executed, and not the other steps that are fine.

So one of the best practises is to keep the steps granular enough and pre-load files and data beforehand and give it to docker. It has amazing caching and archiving mechanisms built in.

Save our work

As we know if we do not save the container into the image, our changes are lost.

If I didn’t use the docker build command I used earlier I could have, after the build process was completed and image created, used the below command:

$ sudo docker commit [sha of the image] neomatrix369/openjdk9

Sharing your docker image on Docker hub

Once you are happy with your changes, and want to share it with community at large, do the below:

$ sudo docker push neomatrix369/openjdk9

and you will see these depending on which of your layers have been found in the repo and which ones are new (this one is an example snapshot of the process):

The push refers to a repository [neomatrix369/openjdk9] (len: 1)
5877e8efd939: Image already exists 
3965c3e47855: Image already exists 
d0c597d045d4: Image already exists 
9250fac9b500: Image already exists 
2d9e17c870be: Buffering to Disk
.
.
.

There is plenty of room for development and improvement of this Docker script. So happy hacking and would love to hear your feedback or contributions from you.

BIG Thanks

Big thanks to the below two who proof-read my post and added value to it, whilst enjoying the #Software #Craftsmanship developer community (organised and supported by @LSCC):
Oliver Nautsch – @ollispieps (JUG Switzerland)
Amir Bazazi (@Codurance) – @amirbazazi

Special thanks to Roberto Cortez (@radcortez) for your Docker posts, these inspired and helped me write my first Docker post.

Resources

[1] Docker
[2] Get into Docker – A Guide for Total Newbies
[3] Docker for Total Newbies Part 2: Distribute Your Applications with Docker Images
[4] Docker posts on Voxxed
[5] OpenJDK
[6] Building OpenJDK
[7] Building OpenJDK on Linux, MacOs and Windows
[8] Virtual Machines (OpenJDK)
[9] Build your own OpenJDK
[10] Vagrant script (OpenJDK)
[11] YOUR DOCKER IMAGE MIGHT BE BROKEN without you knowing it
[12] Dockerfile on github
[13] Adopt OpenJDK: Getting Started Kit
[14] London Java Community

Where in the world is Carmen Sandiego?

Is that a quiz question? Okay here are some hints:
1) Is it some place on the world map or are we talking of a real person?
2) Is it a movie?
3) Is it a video / computer game?
4) Code name for a covert operation!

Hmmm, one more hint Brøderbund! Rings a bell now? I remember playing this detective game on a 8-bit Intel 8088 processor with 640kb RAM, and no HDD, yes we did have B&W CRT monitors, thank you! We had to boot up the system using 5 1/4 inch floppy disks and load a virtual drive in memory (thanks to HIMEM; to run games that ran slow when they were executed from floppy disks, and also to dim those grinding noises they made whenever we needed to move to the next level or save your game state). I used to love that game and would play it endlessly, till the new version of Prince Of Persia came along, another one of the wonders from Brøderbund Software! By the way take a look at what Carmen looks like, go to this link ;)!

Let’s say you ran a pretty neat operation, a central CVS system and gave access to folks (developers) who would commit to your repo now and then. It’s all happy-go-lucky when everyone behaves and everything goes as planned.

But when you wake up one morning and find your CI system coming to a grind-halt – throwing reds and oranges on the screen you would get shirty with anyone. Only later do you find out that this well-meant chap submitted a patch, and one of the classes broke the build. But how do you know what did it and who behind the what did it if you didn’t have some sort of a system?

You do know that by looking at the error message in the log (if you generate log files, if you even look at them and read them, lol), then pick up on the cause of the failure, say in this case it was a class, take that name and run it against a list of last contributors to your repo! Voila! That matches with a so-called Carmen Sandiego! So now you know what we were eluding to?

But where do we get such a program from and what about maintaining a database with a list of developers – lets not rant about it yet, we have a simple solution that could solve it – that’s where our good old “Java” comes in. Have a look at OpenJDK Tool github repo and you will find something of that sort. Have a look at the program requirements on the Adopt OpenJDK mailing list.

So you see we need to keep such tools in hand when you have to run a neat operation, so to seek out the operative who might be hiding behind emails and it would be moments before you got to the person! 😉 It would be interesting if you could do something like this:

$ openJDKProdTool -coActionEvent

and get the below response, with details of the contributor:

[20/09/2012 22:36:50,Carmen Sandiego,Carmen.Sandiego@gmail.com,java.awt.event.ActionEvent,fallthro,pending review,/javac/warnings/core/java/awt/event/, ActionEvent.java.patch]

BTW, if you think you might be able to hook up with Carmen Sandiego – you could try emailing her, but at your own peril 😉 !

Actually if you look closely in the example-scripts folder, you will find something that helps do it. You will need the necessary .jars and .txt files in the local folder for it to work.

It all started when I joined the OpenJDK initiative in 2011, and then came across two interesting techniques called TDD and BDD, and thought why not fuse all of them together to teach myself Java, TDD, and BDD and make a project out of it. And voila, months later we have a maven project on github with stories and scenarios (still learning how to do these, just scratched the surface, so to speak). I’m still a novice at these techniques and learning them as we go along, but very inspired and passionate about learning and using them as I write more code. Please feel free to correct or bring an improvement to anything you find does not look right – but please do also explain why, so that I can learn from it. If you are a TDD or JBehave buff or know how to guard functionalities with unit tests or fill the gaps in an application using TDD techniques I would love to hear from youu to learn those techniques.

Whilst we got busy with all this buzz, my intel tells me that there’s a “Hunt for Red October” on the loose – is it another one of those covert ops or is “Red October” a code name for someone or something. We won’t know till we find out! We know we are in November, so whats this thing about October – apologies for the delayed announcement but due to tactical reasons the news couldn’t be released in October, ;)! So you see we might have to do some time-travelling! lol! The time-machine comes handy when we have to roll-back to that specific configuration that worked for us!

Okay, okay I’ll let you out on the news, I was talking about @kittylyst’s OpenJDK initiative called the “OpenJDK Build&Test system”, you know I’m talking about Ben Evans right! You can read all about it at OpenJDK Build&Test system github repo, above all support him and us on this Adopt OpenJDK journey! It starts like this

Scope

It is envisaged that the ecosystem would enable a user of the system to make requests…

the rest can be found at the Scope.md document on the repo. We all need you, just as much we need Java!

Any constructive comments and feedback are always welcome! Thanks for reading and hope to see you join us and support our operations!

Communications & Awards
Adopt OpenJDK Mailing list
Adopt OpenJDK Github repo
IRC: #adoptopenjdk on irc.freenode.net channel using your favourite IRC client
Adopt OpenJDK wiki page
JavaOne 2012 Talks & Presentation (feedback)
LJC & Adopt OpenJDK win awards!

Credits & Kudos
London Java Community meetup site (LJC)
London Java Community blog site
London Software Craftsmanship (LSCC)
The Adopt OpenJDK team

Thanks to #LJC, #LSCC and the #AdoptOpenJDK team, without which I wouldn’t be able to do a lot of the things I can do now, some of the things I have benefitted from and for which I’m grateful:

Learn and develop my skills, learn about software techniques like TDD & BDD (thanks to @sandromancuso, @SamirTalwar & ‏@londonswcraft, for organising those much-needed talks and hands-on sessions) and many other tiny but important things a developer should know.

Contribute to the Adopt OpenJDK project and learn and improve my “Java” skills – thanks to @karianna, @kittylyst, @bcrecworks (Barry) & @recworks (the whole team) for creating this opportunity for all of us and making continued efforts to move the initiative forward.

Others who helped me with the github project are @RichardWarburto (from the #adoptopenjdk team, thanks for your feedback and suggestions – I’m still working on them), Vaibhav Gowadia (thanks for reviewing my code, it looks much better now and I’m java-wiser after all the amendments 😉 ).

And to everyone else I must have missed out, where the credit is due, who has positively contributed and supported me.