Digital Catapult | Machine Intelligence Garage: the best-kept secret yet in the open

Introduction

I was at a meetup when Simon Knowles, CTO of Graphcore was giving his talk on the latest development at Graphcore and that is also where I met with Peter Bloomfield from Digital Catapult(@digicatapult). Peter was spreading the word about Machine Intelligence Garage which is an amazing opportunity created by a collaboration between the government and industry leaders like Google, Nvidia, AWS, etcetera to help startups and small businesses access compute resources which they would have otherwise never been able to get hold of.

Digital Catapult - Get Involved

Our conversation

Sometime later, we decided to have a chat discussing the usual questions like what is Machine Intelligence Garage, what is the history, why, how, who, when, and at what stage is the programme at and how do people get involved? As you would imagine, I found our conversation interesting and informative and hence decided to write about it and share it with the rest of us.

Mani: Hey, Peter great meeting you and learning about the initiative from Digital Catapult, can you please share with me the history about this initiative?

Peter: Hi Mani, thanks for dropping in! Our programme, Machine Intelligence Garage was born from a piece of research we conducted last summer. In this report, we explore barriers facing AI startups, and we wanted to test the hypotheses that: Access to the right data, technical talent and adequate computational resources were the main things holding startups back. We collaborate a bit on the first two barriers with government and academic institutes, like the Turing, but the Machine Intelligence Garage programme was designed to provide startups with access to cloud computing vouchers, novel chipsets and HPC facilities. It is through our brilliant partners that we can offer these resources.

Mani: Are there other sister and daughter initiatives or programs, related to Digital Catapult, that everyone would benefit knowing about?

Peter: We have a whole range of digital programmes, across three core tech layers (future networks, AI/ML and Immersive tech as well as some cybersecurity and blockchain initiatives). The full details of our opportunities can be found through the ‘Get Involved’ section of our website.

Mani: Who have been and are going to benefit from this setup that you have in place?

Peter: Our Machine Intelligence Garage programme is designed to benefit early-stage startups who are data ready and need compute power to scale faster. Our collaborative programmes provide opportunities for larger corporations to get involved with startups to address industry-specific challenges.

Mani: How much does this access cost and for how long are they available for access?

Peter: Everything we provide on the Machine Intelligence Garage programme is free to use. The programme was set up using public funding from InnovateUK and CAP.AI. We are able to deliver the compute resources through our work with a wonderful set of partners.

Mani: Can you tell us the process from start to finish?

Peter: When I meet a new company, I have a chat with them about the things they are trying to do, the infrastructure they currently use and the sorts of ML approaches they are using to solve the problem. If the company needs our support and are developing a product with commercial viability and have both strong technical skills and domain expertise I encourage them to apply. The application form asks questions about the product, training data, compute power requirements. If we like the idea, we invite a company to an interview and if successful onboard them with the most suitable resource. The process generally takes 3 weeks from the close of the application call to onboarding with a resource.

gathered-around-table-classic

Mani: What other benefits do startups get from being involved? Are you able to introduce them to partners who can help them run trials and give feedback on the products or services they are building?

Peter: We have a large network that we encourage all our companies to take advantage of. Digital Catapult is an innovation centre and meeting the right people at the right time is key to success for many startups. We run a range of workshops, from business growth and pitch training to deep dives to learn more about technical resources and we make sure our startups benefit from all of these. If a startup wants to put some of the new knowledge into practice we are always very keen to facilitate it!

Mani: This is a lot of information, are there any resources on your website that can help. Anything to sign up to, to keep in touch?

Peter: We have a general technology Digital Catapult newsletter (sign-up form at the bottom of the page) and an AI specific Machine Intelligence Garage newsletter (sign-up form at the bottom of the page). If a startup wants to chat about the programme, they can send me an e-mail: peter.bloomfield@digicatapult.org.uk. We announce all our calls and opportunity through our twitter account too @DigiCatapult.

Mani: Can you please touch on the specifics of what the startups will get access to and how it can benefit them?

Peter: We have three main resources available:

  • Cloud Computing vouchers, either through AWS or Google Cloud Platform

To find out the exact amounts and specifics of access, please do get in touch.

Mani: If someone needed to find out about the benchmarks between different compute resources available to the participants, who or where would they look for the information? Do you have a team that does these measurements on a daily basis?

Peter: We do our own benchmarking of the facilities available and our data engineer on the programme can advise on this, as well as point you in the right direction for more literature!

immersive-lab-entrance

Mani: I have been to the Digital Catapult HQ at 101 Euston Road, and was blown away with all the tech activities happening there, can you please share details about it with our readers

Peter: Digital Catapult was set up four years ago and is the UK’s leading innovation centre for advanced digital technologies.  We have seen a number of changes over the years but our core values of opening up markets and making businesses more competitive and productive remain. We are incredibly lucky to have some amazing facilities to help companies develop new products and services and get their products to market faster, including a nationwide network of Immersive Labs [see launch photos, photos in 2018], an LPWAN network and the new 5G Brighton Testbed.

Mani: Can you name a few startups that are currently going through your programs and the ones who have already been through it?

Peter: We currently have 25 start-ups on our programme. The reams are a range of sizes, some just 2 people, others 20+. The thing they all have in common is that they are developing some really exciting commercial products/solutions with deep learning and have an immediate need for the computational resources we offer. The full list of start-ups can be found on our cohort page on the Machine Intelligence Garage website.

Mani: I really appreciate the time you have taken to answer my questions and this has definitely helped the readers know more about what you do and how they can benefit from this great government-driven initiative.

Peter: Thank you very much for coming in. It’s great to be able to reach a wider audience and grow our community! See you soon!

15307393344_f6881df22a_k

Closing note

I was shown around a number of facilities at their centre (two floors) i.e. the Immersive lab (yes plenty of VR headsets to play with), the server area where all the HPC hardware is kept (at low room temperature), a spacious conference room where meetups are held, a small library full of interesting books and also a hot-desking area shared by both internal staff, partners and friends of Digital Catapult/Machine Intelligence Garage. Looking at the two websites I found the news and views, events and workshops and Digital Catapult | MI Garage blog sites interesting to keep track of activities in this space.

immersive-lab-man-headgear

I’m sure after reading about the conversation, you must be wondering how you could take advantage of these facilities out there meant for you and ones in your network who could benefit from it.

Readers should go to the links mentioned above to learn about this program and how they can go about taking advantage of it, or recommend it to their friends in the community who would be more suitable for it.

Please do let me know if this is helpful by dropping a line in the comments below, and I would also welcome feedback, see how you can reach me, above all please check out to the links mentioned above and also reach out to the folks behind this great initiative.

 

Adopt OpenJDK & Java community: how can you help Java !

Introduction

I want to take the opportunity to show what we have been doing in last year and also what we have done so far as members of the community. Unlike other years I have decided to keep this post less technical compare to the past years and compared to the other posts on Java Advent this year.

inthebeginning

This year marks the fourth year since the first OpenJDK hackday was held in London (supported by LJC and its members) and also when the Adopt OpenJDK program was started. Four years is a small number on the face of 20 years of Java, same goes to the size of the Adopt OpenJDK community which forms a small part of the Java community (9+ million users). Although the post is non-technical in nature, the message herein is fairly important for the future growth and progress of our community and the next generation developers.

Creations of the community

Creations from the community

Over the many months a number of members of our community contributed and passed on their good work to us. In no specific order I have enlisted these picking them from memory. I know there are more to name and you can help us by sharing those with us (we will enlist them here).  So here are some of those that we can talk about and be proud of, and thank those who were involved:

  • Getting Started page – created to enabled two way communication with the members of the community, these include a mailing list, an IRC channel, a weekly newsletter, a twitter handle, among other social media channels and collaboration tools.
  • Adopt OpenJDK project: jitwatch – a great tool created by Chris Newland, its one of its kind, ever growing with features and helping developers fine-tune the performance of your Java/JVM applications running on the JVM.
  • Adopt OpenJDK: GSK – a community effort gathering knowledge and experience from hackday attendees and OpenJDK developers on how to go about with OpenJDK from building it to creating your own version of the JDK. Many JUG members have been involved in the process, and this is now a e-book available in many languages (5 languages + 2 to 3 more languages in progress).
  • Adopt OpenJDK vagrant scripts – a collection of vagrant scripts initially created by John Patrick from the LJC, later improved by the community members by adding more scripts and refactoring existing ones. Theses scripts help build OpenJDK projects in a virtualised container i.e. VirtualBox, making building, and testing OpenJDK and also running and testing Java/JVM applications much easier, reliable and in an isolated environment.
  • Adopt OpenJDK docker scripts – a collection of docker scripts created with the help of the community, this is now also receiving contributions from a number of members like Richard Kolb (SA JUG). Just like the vagrant scripts mentioned above, the docker scripts have similar goals, and need your DevOps foo!
  • Adopt OpenJDK project: mjprof – mjprof is a Monadic jstack analysis tool set. It is a fancy way to say it analyzes jstack output using a series of simple composable building blocks (monads). Many thanks to Haim Yadid for donating it to the community.
  • Adopt OpenJDK project: jcountdown – built by the community that mimics the spirit of ie6countdown.net. That is, to encourage users to move to the latest and greatest Java! Many thanks to all those involved, you can already see from the commit history.
  • Adopt OpenJDK CloudBees Build Farm – thanks to the folks at CloudBees for helping us host our build farm on their CI/CD servers. This one was initially started by Martijn Verburg and later with the help of a number of JUG members have come to the point that major Java projects are built against different versions of the JDK. These projects include building the JDKs themselves (versions 1.7, 1.8, 1.9, Jigsaw and Shenandoah). This project has also helped support the Testing Java Early project and Quality  Outreach program.

These are just a handful of such creations and contributions from the members of the community, some of these projects would certainly need help from you. As a community one more thing we could do well is celebrate our victories and successes, and especially credit those that have been involved whether as individuals or a community. So that our next generation contributors feel inspired and encourage to do more good work and share it with us.

Contributions from the community

contribution_header-700x325In a recent tweet and posts to various Java / JVM and developer mailing lists, I requested the community to come forward and share their contribution stories or those from others with our community. The purpose was two-fold, one to share it with the community and the other to write this post (which in turn is shared with the community). I was happy to see a handful of messages sent to me and the mailing lists by a number of community members. I’ll share some of these with you (in the order I have received them).

 

Sebastian Daschner:

I don’t know if that counts as contribution but I’ve hacked on the
OpenJDK compiler for fun several times. For example I added a new
thought up ‘maybe’ keyword which produces randomly executed code:
https://blog.sebastian-daschner.com/entries/maybe_keyword_in_java

Thomas Modeneis:

Thanks for writing, I like your initiative, its really good to show how people are doing and what they have been focusing on. Great idea.
From my part, I can tell about the DevoxxMA last month, I did a talk on the Hacker Space about the Adopt the OpenJDK and it was really great. We had about 30 or more attendees, it was in a open space so everyone that was going to any talk was passing and being grabbed to have a look about the topic, it was really challenging because I had no mic. but I managed to speak out loud and be listen, and I got great feedback after the session. I’m going to work over the weekend to upload the presentation and the recorded video and I will be posting here as soon as I have it done! :)

Martijn Verburg:

Good initiative.  So the major items I participated in were Date and Time and Lambdas Hackdays (reporting several bugs), submitted some warnings cleanups for OpenJDK.  Gave ~10 pages of feedback for jshell and generally tried to encourage people more capable than me to contribute :-).

Andrii Rodionov:

Olena Syrota and Oleg Tsal-Tsalko from Ukraine JUG: Contributing to JSR 367 test code-base (https://github.com/olegts/jsonb-spec), promoting ‘Adopt a JSR’ and JSON-B spec at JUG UA meetings (http://jug.ua/2015/04/json-binding/) and also at JavaDay Lviv conference (http://www.slideshare.net/olegtsaltsalko9/jsonb-spec).

Contributors

contributorAs you have seen that from out of a community of 9+ million users, only a handful of them came forward to share their stories. While I can point you out to another list of contributors who have been paramount with their contributions to the Adopt OpenJDK GitBook, for example, take a look at the list of contributors and also the committers on the git-repo. They have not just contributed to the book but to Java and the OpenJDK community, especially those who have helped translate the book into multiple languages. And then there are a number of them who haven’t come forward to add their names to the list, even though they have made valuable contributions.

From this I can say contributors can be like unsung heroes, either due their shy or low-profile nature or they just don’t get noticed by us. So it would only be fair to encourage them to come forward or share with the community about their contributions, however simple or small those may be. In addition to the above list I would like to also add a number of them (again apologies if I have missed out your name or not mentioned about you or all your contributions). These names are in no particular order but as they come to my mind as their contributions have been invaluable:

  • Dalibor Topic (OpenJDK Project Lead) & the OpenJDK team
  • Mario Torre & the RedHat OpenJDK team
  • Tori Wieldt (Java Community manager) and her team
  • Heather Vancura & the JCP team
  • NightHacking, vJUG and RebelLabs (and the great people behind them)
  • Nicolaas & the team at Cloudbees
  • Chris Newland (JitWatch developer)
  • Lucy Carey, Ellie & Mark Hazell (Devoxx UK & Voxxed)
  • Richard Kolb (JUG South Africa)
  • Daniel Bryant, Richard Warburton, Ben Evans, and a number of others from LJC
  • Members of SouJava (Otavio, Thomas, Bruno, and others)
  • Members of Bulgarian JUG (Ivan, Martin, Mitri) and neighbours
  • Oti, Ludovic & Patrick Reinhart
  • and a number of other contributors who for some reason I can’t remember…

I have named them for their contributions to the community by helping organise Hackdays during the week and weekends, workshops and hands-on sessions at conferences, giving lightening talks, speaking at conferences, allowing us to host our CI and build farm servers, travelling to different parts of the world holding the Java community flag, writing books, giving Java and advance-level training, giving feedback on new technologies and features, and innumerable other activities that support and push forward the Java / JVM platform.

How you can make a difference ? And why ?

make_a_differenceYou can make a difference by doing something as simple as clicking the like button (on Twitter, LinkedIn, Facebook, etc…) or responding to a message on a mailing list by expressing your opinion about something you see or read about –as to why you think about it that way or how it could be different.

The answer to the question “And why ?” is simple, because you are part of a community and ‘you care’ and want to share your knowledge and experience with others — just like the others above who have spared free moments of their valuable time for us.

Is it hard to do it ? Where to start ? What needs most attention ?

important-checklist The answer is its not hard to do it, if so many have done it, you can do it as well. Where to start and what can you do ? I have written a page on this topic. And its worth reading it before going any further.

There is a dynamic list of topics that is worth considering when thinking of contributing to OpenJDK and Java. But recently I have filtered this list down to a few topics (in order of precedence):

We need you!

With that I would like to close by saying:

i_need_you_duke3

Not just “I”, but we as a community need you

This post have been re-blogged from the Java Advent Calendar 2015 site. Many thanks to its organisers and writers.

This post is part of the Java Advent Calendar and is licensed under the Creative Commons 3.0 Attribution license. If you like it, please spread the word by sharing, tweeting, FB, G+ and so on!

My first couple of months at Codurance

Some background

Five Characteristics of a Great Company CultureSome of you may know me from the various meetups in the city, especially my attendance at a number of LJC and LSCC meetup events. Attending these events I learnt about various conferences like Devoxx, SoCraTes, JAX LondonJava2Days, OpenFest, and I ended up attending and later presenting on various topic including Adopt OpenJDK.

During this time I met a lot of people with various levels of experience and my interest and urge to learn more about the Java/JVM platform, Code Quality, Software Design, XP Practices, Software Craftsmanship, etc…, were on the rise and saw no end. And whilst attending these events I came across Sandro and Mash, who were in those days hosting LSCC events. I went to many of LSCC events, especially liked the hands-on sessions (which are still my favourite).

I also noticed that many things I learnt at such events and conferences wouldn’t always be immediately recognised or accepted at the workplace. And moving to another work environment didn’t always solve this problem fully. I found that I wasn’t learning what I wanted from my peers and the things I learnt from the community I couldn’t apply at work. Besides very few were really in tuned with what the community was about. So one fine day I decided to take charge of my career and make a serious decision and take up the Apprenticeship program offered by Codurance and go through the process.

I was urged to go this way after being inspired by Sandro’s book: The Software Craftsman, attending all the SoCraTes UK conferences, and meeting with developers who valued and took pride of their work namely their craft.

I was urged to go this way after being inspired by Sandro’s book: The Software Craftsman, attending all the SoCraTes UK conferences, and meeting with developers who valued and took pride of their work namely their craft.

Where we are just now

It’s now been nearly two months since I have been working for Codurance, a formidable force. And so it’s also about time that I share my experiences with my fellow mates and the community around me.

During my first few weeks at Codurance, I have been busy learning various things that have been chalked out for becoming a craftsman.

When working on a kata or learning a concept, we paired or did what is known as ‘mob programming’ along with other apprentices and craftsmen. And most of the time used the pomodoro technique. Time boxing our work in intervals is something done both in groups and working individually. We would have a lot of discussions and retrospectives after working on a problem or writing some code from scratch.

Structure of my program

We used an internal tool based on the concept of Impact Mapping. I soon got interested in it when I saw my colleague Franzi (who is now a craftswoman) had used it to plan out her Apprenticeship route. Such a tool helps map out our goals and the tasks we need to perform to achieve it. And this can differ from person-to-person, depending on what they want to work on (driven by the Apprentice).

My mentor and other craftsmen reviewed them to get an idea of what I wanted to achieve for myself. And then its up to me to apply my own drive and perseverance to achieve the individual stories. My mentor and I meet and talk informally on a regular basis, many times pairing on a kata or a project or on the white board trying to get my head around a concept.

Days in the life of an Apprentice

I found the working hours quite flexible, remote working is also an option (when you are on the bench or if the client allows, if you are in a project). Our co-founders are understanding and compassionate about our individual situations.

Meetings are at their minimum, except for a weekly Apprentices meeting (run by an Apprentice and guided by at least one Craftsperson) and a bi-monthly company-wide catchup.

The Apprentices meetings are full of fun — we are accompanied by at least one Craftsperson, who disperses their knowledge and experience from a wide variety of topics designed to help us in the journey and fill the gaps in our knowledge and experience.

A bi-monthly catchup involves sharing of knowledge via lightning talks, discussions and pairing sessions on pet projects over pizzas and beer (and of course veggies and non-alcoholic beverages for the teetotalers).

Katas, code reviews, mob programming and projects make up a learning week – all of these done individually or when pairing with another.

Katas

On a daily basis I have worked on different katas or try to solve the same kata in various different ways (using different testing and refactoring approaches). This in turn gave me better insights into designing and refactoring techniques. Trying to solve the same problem in different ways has a positive impact on our problem solving skills especially when writing code. In my case I also learnt how to use the different libraries and methods to write tests. I would like to cite Samir, thanks to you, for the suggesting this approach during the first week of my Apprenticeship.

Code reviews

Just last week we did a group code review and time-boxed ourselves, performed a retrospective at the end of each interval and ensured we delivered a good chunk of the feedback before close of play. Such regular code review exercises are helping all of us learn about how to code better as we are not only learning from feedback from the tools we used, but also through exchange of feedback from our peers who were involved in the group code review session.

Software Design, Specification Gathering & Communication

Recently we had an interesting mob-programming session where we were trying to model and write a game. At the end of the session, we had a retrospective, discussing the things we did well and didn’t do well. Each of the apprentices and craftsmen were performing a specific role i.e. Developer, Domain Expert, etc… We learnt in retrospective, about areas where we could have done better and should focus on. That any test written gives immediate feedback about how well we have understood the domain and if we were taking the right approach. Why a certain approach when starting a project is more advantageous than another approach. What questions to ask and why it is important to ask the right questions to the domain expert or to give the right level of information to another developer and vice-versa. Sandro has described this process in detail in his blog post recently.

Fun, socialising and sharing

I found our office environment to be conducive to learning, sharing and collaboration. We even have a pairing rota that we use from time-to-time to record or suggest pairing sessions during the week.

We share links to events, conferences, tweets, interesting articles, videos, blog posts, etc… via slack, document discussions and brain dumps via Google doc, huddles during lunch- and tea- breaks to talk about anything we are working on. Thanks to the library of printed and digital books to our disposal, the huge collection of blog posts and videos on our site.

The apprentices and some craftsmen have collectively started a social event which of course happens every Friday, sometimes it’s dinner at a nearby restaurant, while at other times an indoor movie over snacks and drinks at our office premises.

It is worthwhile and that’s why we are here

It is a privilege to be able to work alongside very experienced craftsmen from our industry. We are very lucky and thankful to have the opportunity to be guided and mentored by talented and like minded developers.

This is my first job where the company has a completely flat hierarchy and where we share similar values.

greatCompanyCulture

Closing note

Work is fun and learning is enjoyable when we love what we do and are amongst friends with similar goals and aspirations.

Thank you for taking the time to read this post and I hope it was interesting. Looking forward to write more and share such experiences in future posts.

Many thanks to Sandro, Tomaz, Alex, Franzi and David for all the feedback provided for this blog post.

 

Why not build #OpenJDK 9 using #Docker ? – Part 2 of 2

…continuing from Why not build #OpenJDK 9 using #Docker ? – Part 1 of 2.

I ran into a number of issues and you can see from my commits, I pulled myself out of it, but to run this Dockerfile from the command-line I used this instruction:

$ docker build -t neomatrix369/openjdk9 .

you can also do it using the below if you have not set your permissions:

$ sudo docker build -t neomatrix369/openjdk9 .

and get the below (summarised) output:

Sending build context to Docker daemon 3.072 kB
Sending build context to Docker daemon 
Step 0 : FROM phusion/baseimage:latest
 ---> 5a14c1498ff4
Step 1 : MAINTAINER Mani Sarkar (from @adoptopenjdk)
 ---> Using cache
 ---> 95e30b7f52b9
Step 2 : RUN apt-get update &&   apt-get install -y     libxt-dev zip pkg-config libX11-dev libxext-dev     libxrender-dev libxtst-dev libasound2-dev libcups2-dev libfreetype6-dev &&   rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 1ea3bbb15c2d
Step 3 : RUN apt-get update
 ---> Using cache
 ---> 6c3938f4d23d
Step 4 : RUN apt-get install -y mercurial ca-certificates-java build-essential
 ---> Using cache
 ---> e3f99b5a3bd3
Step 5 : RUN cd /tmp &&   hg clone http://hg.openjdk.java.net/jdk9/jdk9 openjdk9 &&   cd openjdk9 &&   sh ./get_source.sh
 ---> Using cache
 ---> 26cfaf16b9fa
Step 6 : RUN apt-get install -y wget &&   wget --no-check-certificate --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u45-b14/jdk-8u45-linux-x64.tar.gz
 ---> Using cache
 ---> 696889250fed
Step 7 : RUN tar zxvf jdk-8u45-linux-x64.tar.gz -C /opt
 ---> Using cache
 ---> c25cc9201c1b
Step 8 : RUN cd /tmp/openjdk9 &&   bash ./configure --with-cacerts-file=/etc/ssl/certs/java/cacerts --with-boot-jdk=/opt/jdk1.8.0_45
 ---> Using cache
 ---> 4e425de379e6
Step 9 : RUN cd /tmp/openjdk9 &&   make clean images
 ---> Using cache
 ---> 2d9e17c870be
Step 10 : RUN cd /tmp/openjdk9 &&   cp -a build/linux-x86_64-normal-server-release/images/jdk     /opt/openjdk9
 ---> Using cache
 ---> 9250fac9b500
Step 11 : RUN cd /tmp/openjdk9 &&   find /opt/openjdk9 -type f -exec chmod a+r {} + &&   find /opt/openjdk9 -type d -exec chmod a+rx {} +
 ---> Using cache
 ---> d0c597d045d4
Step 12 : ENV PATH /opt/openjdk9/bin:$PATH
 ---> Using cache
 ---> 3965c3e47855
Step 13 : ENV JAVA_HOME /opt/openjdk9
 ---> Using cache
 ---> 5877e8efd939
Successfully built 5877e8efd939

The above action creates an image which is stored in your local repository (use docker images to enlist the images in the repo). If you want to load the image into a container, and access the files it has built or see anything else, do the below:

$ sudo docker run -it --name openjdk9 neomatrix369/openjdk9 /bin/bash

this will take you to a bash prompt into the container and you can run any of your linux commands and access the file system.

Explaining docker run

$ sudo docker run -it --name openjdk9 neomatrix369/openjdk9 java -version

will show you this

openjdk version "1.9.0-internal"
OpenJDK Runtime Environment (build 1.9.0-internal-_2015_06_04_06_46-b00)
OpenJDK 64-Bit Server VM (build 1.9.0-internal-_2015_06_04_06_46-b00, mixed mode)

Here’s a breakdown of the docker run command:

docker run The command to create and start a new Docker container.
-it To run in interactive mode, so you can see the after running the container.
neomatrix369/openjdk9 This is a reference to the image tag by name (which we created above).
java -version Runs the java command asking its version, inside the containing, which is assisted by the two environment variables PATH and JAVA_HOME which was set in the Dockerfile above.

Footnotes

You might have noticed I grouped very specific instructions with each step, especially the RUN commands, its because, each time I got one of these wrong, it would re-execute the step again, including the steps that ran fine and didn’t need re-executing. Not only is this unnecessary its not using our resources efficiently which is what Docker brings us. So any addition, edition or deletion to any step will only result in that step being executed, and not the other steps that are fine.

So one of the best practises is to keep the steps granular enough and pre-load files and data beforehand and give it to docker. It has amazing caching and archiving mechanisms built in.

Save our work

As we know if we do not save the container into the image, our changes are lost.

If I didn’t use the docker build command I used earlier I could have, after the build process was completed and image created, used the below command:

$ sudo docker commit [sha of the image] neomatrix369/openjdk9

Sharing your docker image on Docker hub

Once you are happy with your changes, and want to share it with community at large, do the below:

$ sudo docker push neomatrix369/openjdk9

and you will see these depending on which of your layers have been found in the repo and which ones are new (this one is an example snapshot of the process):

The push refers to a repository [neomatrix369/openjdk9] (len: 1)
5877e8efd939: Image already exists 
3965c3e47855: Image already exists 
d0c597d045d4: Image already exists 
9250fac9b500: Image already exists 
2d9e17c870be: Buffering to Disk
.
.
.

There is plenty of room for development and improvement of this Docker script. So happy hacking and would love to hear your feedback or contributions from you.

BIG Thanks

Big thanks to the below two who proof-read my post and added value to it, whilst enjoying the #Software #Craftsmanship developer community (organised and supported by @LSCC):
Oliver Nautsch – @ollispieps (JUG Switzerland)
Amir Bazazi (@Codurance) – @amirbazazi

Special thanks to Roberto Cortez (@radcortez) for your Docker posts, these inspired and helped me write my first Docker post.

Resources

[1] Docker
[2] Get into Docker – A Guide for Total Newbies
[3] Docker for Total Newbies Part 2: Distribute Your Applications with Docker Images
[4] Docker posts on Voxxed
[5] OpenJDK
[6] Building OpenJDK
[7] Building OpenJDK on Linux, MacOs and Windows
[8] Virtual Machines (OpenJDK)
[9] Build your own OpenJDK
[10] Vagrant script (OpenJDK)
[11] YOUR DOCKER IMAGE MIGHT BE BROKEN without you knowing it
[12] Dockerfile on github
[13] Adopt OpenJDK: Getting Started Kit
[14] London Java Community

Why not build #OpenJDK 9 using #Docker ? – Part 1 of 2

Introduction

I think I have joined the Docker [1] party a bit late but that means by now everyone knows what Docker is and all the other basic fundamentals which I can very well skip, but if you are still interested, please check these posts Get into Docker – A Guide for Total Newbies [2] and Docker for Total Newbies Part 2: Distribute Your Applications with Docker Images [3]. And if you still want to know more about this widely spoken topic, check out these Docker posts on Voxxed [4].

Why ?

Since everyone has been doing some sort of provisioning or spinning up of dev or pre-prod or test environments using Docker [1] I decided to do the same but with my favourite project i.e. OpenJDK [5].

So far you can natively build OpenJDK [6] across Linux, MacOs and Windows [7], or do the same things via virtual machines or vagrant instances, see more on then via these resources Virtual Machines, [8] Build your own OpenJDK [9] and this vagrant script [10]. All part of the Adopt OpenJDK initiative lead by London Java Community [14] and supported by JUGs all over the world.

Requirements

Most parts of post is for those using Linux distributions (this one was created on Ubuntu 14.04). Linux, MacOS and Windows users please refer to Docker‘s  Linux, MacOS and Windows instructions respectively.

Hints: MacOS and Windows users will need to install Boot2Docker and remember to run the below two commands (and check your Docker host environment variables):

$ boot2docker init
$ boot2docker up 
$ boot2docker shellinit 

For the MacOS, if the above throw FATA[…] error messages, please try the below:

$ sudo boot2docker init
$ sudo boot2docker up 
$ sudo boot2docker shellinit 

For rest of the details please refer to the links provided above. Once you have the above in place for the Windows or MacOS platform, by merely executing the Dockerfile using the docker build and docker run commands you can create / update a container and run it respectively.

*** Please refer to the above links and ensure Docker works for you for the above platforms – try out tutorials or steps proving that Docker run as expected before proceeding further. ***

Building OpenJDK 9 using Docker

Now I will show you how to do the same things as mentioned above using Docker.

So I read the first two resource I shared so far (and wrote the last ones). So lets get started, and I’m going to walk you through what the Dockerfile looks like, as I take you through each section of the Dockerfile code.

*** Please note the steps below are not meant to be executed on your command prompty, they form an integral part of the Dockerfile which you can download from here at the end of this post. ***

You have noticed unlike everyone else I have chosen a different OS image i.e. phusion/baseimage, why? Read YOUR DOCKER IMAGE MIGHT BE BROKEN without you knowing it [11], to learn more about it.

FROM phusion/baseimage:latest

Each of the RUN steps below when executed becomes a Docker layer in isolation and gets assigned a SHA like this i.e. 95e30b7f52b9.

RUN \
  apt-get update && \
  apt-get install -y \
    libxt-dev zip pkg-config libX11-dev libxext-dev \
    libxrender-dev libxtst-dev libasound2-dev libcups2-dev libfreetype6-dev && \
  rm -rf /var/lib/apt/lists/*

The base image is updated and a number of dependencies are installed i.e. Mercurial (hg) and build-essential.

RUN \
  apt-get update && \
  apt-get install -y mercurial ca-certificates-java build-essential

Clone the OpenJDK 9 sources and download the latest sources from mercurial. You will notice that each of these steps are prefixed by this line cd /tmp &&, this is because each instruction is run in its own layer, as if it does not remember where it was when the previous instruction was run. Nothing to worry about, all your changes are still intact in the container.

RUN \
  cd /tmp && \
  hg clone http://hg.openjdk.java.net/jdk9/jdk9 openjdk9 && \
  cd openjdk9 && \
  sh ./get_source.sh

Install only what you need when you need them, see below I downloaded wget and then the jdk binary. I also learnt how to use wget by passing the necessary params and headers to make the server give us the binary we request. Finally un-tar the file using the famous tar command.

RUN \
  apt-get install -y wget && \
  wget --no-check-certificate --header "Cookie: oraclelicense=accept-securebackup-cookie" \ 
http://download.oracle.com/otn-pub/java/jdk/8u45-b14/jdk-8u45-linux-x64.tar.gz

RUN \
  tar zxvf jdk-8u45-linux-x64.tar.gz -C /opt

Run configure with the famous –with-boot-jdk=/opt/jdk1.8.0_45 to set the bootstrap jdk to point to jdk1.8.0_45.

RUN \
  cd /tmp/openjdk9 && \
  bash ./configure --with-cacerts-file=/etc/ssl/certs/java/cacerts --with-boot-jdk=/opt/jdk1.8.0_45

Now run the most important command:

RUN \  
  cd /tmp/openjdk9 && \
  make clean images

Once the build is successful, the artefacts i.e. jdk and jre images are created in the build folder.

RUN \  
  cd /tmp/openjdk9 && \
  cp -a build/linux-x86_64-normal-server-release/images/jdk \
    /opt/openjdk9

Below are some chmod ceremonies across the files and directories in the openjdk9 folder.

RUN \  
  cd /tmp/openjdk9 && \
  find /opt/openjdk9 -type f -exec chmod a+r {} + && \
  find /opt/openjdk9 -type d -exec chmod a+rx {} +

Two environment variable i.e. PATH and JAVA_HOME are created with the respective values assigned to them.

ENV PATH /opt/openjdk9/bin:$PATH
ENV JAVA_HOME /opt/openjdk9

You can find the entire source for the entire Dockerfile on github [12].

…more of this in the next post, Why not build #OpenJDK 9 using #Docker ? – Part 2 of 2, we will use the docker build, docker run commands and some more docker stuff.

What is being a good developer? How can we work to become better?

TL;DR

  • future of the Java ecosystem
  • JavaSE, Java EE workshops at Devoxx UK 2015
  • OpenJDK and JSRs
  • extraordinary developers
  • hands-on sessions, hackdays and panel
  • learn, collaborate and share
  • developer communities, JUG leaders, and corporate participations
  • thought leaders and tomorrow’s leaders of Java related topics
  • agenda, costs, tickets and discount codes

The Adopt team are running workshops at Devoxx UK next month that will help to answer the questions posed above. Why should you get along to a workshop? Well, here’s an imagined conversation between you and the Adopt team to explain. The Adopt Team are an ever growing community from various parts of the developer community, mainly the JUGs worldwide, members of the JCP, developers, evangelists and technical representatives of various companies supporting and extending the Java ecosystem via the JCP, Adopt-a-JSR  and Adopt OpenJDK programs.


You: I hear that there are a number of workshops going to be held during the first day of Devoxx UK 2015. What are these?

Adopt team: Yes, you heard right, we have a whole day (Wednesday) dedicated to it. In brief, the adopt team at Devoxx UK this year is comprised of experienced developers and leaders of various Java SE and Java EE topics i.e. OpenJDK, and a number of JSRs. These include members of the JCP as well. The topics range from learning how to build OpenJDK, working with devops tools, quality analysis tools, contributing and working with JSRs (guided by spec leads of the respective JSRs). Learning how to run a hackday if you want to start your own community where you are. And towards the end of the session you can get answers to the questions you have been pondering upon before or at the conference.

Have a look at the agenda to find out more.


You: Apart from OpenJDK, can you name some JSRs we will be involved with?

Adopt team: Good questions. We will have amongst us spec leads from the various JSRs:

  • JSR 363, Units of Measurement API – Leonardo Lima
  • JSR JSR 365, CDI 2.0 – Antoine Sabot-Durand
  • Java EE 8, and some of the proposed component JSRs, including JSR 371 MVC, the Security JSR, and JMS – David Delabassee

You: I’m a busy person, and there’s lots on at Devoxx. Why would I spend time at these workshops rather than doing something else? What do I gain from it?

Adopt team: Depending the workshop you attend you attend and participate in, the benefits can be any or all of these:

  • a great opportunity to feedback on tomorrow’s technology
  • learn and contribute to these technologies even before they are released
  • contributing to the future of the Java / Java platform
  • learn how to build your own Java / JVM platform
  • learn new technologies and improve your confidence,; including automation, code-coverage, testing technologies etc.
  • be able to make contact with and talk to the developers who are leading one or more of the technologies
  • be able to share with your own local user group / community the things you learn in the workshop
  • become a part of the adopt community and keep in touch about current and future developments, hackdays and other workshops

Contributions can be in any form, from as simple as speaking about it to another colleague, writing a post about it, or even mentioning it on a relevant mailing list.


You: So who is this aimed at? Who is your target audience?

Adopt team: Our primary goal is to propagate the know-how and experience various Java User Group leaders all across the globe, but we see everyone else benefiting from it as well: students, professionals, tech leads, devops, trainers, experts, etc…


You: What if my company wishes to be involved? Can you give me examples of companies who are already involved in this program?

Adopt team: Yes, companies, user groups, dev communities are all welcome. A number of companies are involved with the adoption program including: Oracle, IBM, RedHat, SAP, Google, Twitter, Goldman Sachs, Credit Suisse, among many others.


You: Can I or my company make contributions to any of the projects?

Adopt team: Yes, of course, that’s the whole point. One of the reasons is not just spread awareness and know-how of the upcoming technologies in the Java/JVM world but also to show how you can do it yourself. The topics and subtopics are numerous and this is an opportunity to pick one and lead it. You can drive these technologies with the help other community members, helping move your vision forward. Come along to a workshop to find out more!


You: So how much more does it cost me or my company if I have to attend these workshops?

Adopt team: Have a look at the ticket prices for the university days and the combi-tickets. You can also take advantage of special discount codes from us to get further discounts on the final ticket prices (ranging from 10 to 20%).


You:  What do we take along with us to be able to participate in the workshops ?

Adopt team: Bring along a laptop and a fresh mind to absorb all the knowledge and experience we have to share.


You: I hear this theme ‘The Extraordinary…. what is it about, tell me more” Why is this post so titled?

Adopt team: Another good question: to know more about the theme have a look at this http://www.devoxx.co.uk/2015/03/whats-theme/.

Good developers do what every developer does and more. They come to conferences like Devoxx, attend local dev events and talks, participate in workshops similar to the ones this post speaks about and then go out and share many of those things with other fellow developers in the form of blog posts, code snippets or even local community presentations.


You: What about those who can’t get to the workshop. How can they find out more and get involved?

Adopt team: These three websites are a great starting point to get to know more about the adopt programs: http://www.jcp.org, http://adoptopenjdk.java.net and http://adoptajsr.java.net/, but if you are based in London, the LJC organises a hackday every month, see http://www.meetup.com/Londonjavacommunity/events/222368734/, in the past developers from outside London and even outside the UK have visited the day-long hackdays.


You:  So tell me how did this come about? Who do you credit the initiative to?

Adopt team: Nice question: credit must go to where it is due. Thanks to the support from the JCP organisation i.e. Heather VanCura and Mark Hazell for making it happen at Devoxx UK this year. Not forgetting that any event like this one isn’t possible without the help of local and worldwide developer communities, with the likes of London Java Community (@LJCJug), continuously supporting good developer endeavours.


You: How to get the code?

Adopt team: That’s an important question, get in touch with Heather VanCura (heather@jcp.org) or Mark Hazell (mark.hazell@devoxx.co.uk) and they should help you sort out a discount code for you, but hurry are they will get exhausted soon.


How is Java / JVM built ? Adopt OpenJDK is your answer!

Introduction & history
As some of you may already know, starting with Java 7, OpenJDK is the Reference Implementation (RI) to Java. The below time line gives you an idea about the history of OpenJDK:
OpenJDK history (2006 till date)
If you have wondered about the JDK or JRE binaries that you download from vendors like Oracle, Red Hat, etcetera, then the clue is that these all stem from OpenJDK. Each vendor then adds some extra artefacts that are not open source yet due to security, proprietary or other reasons.


What is OpenJDK made of ?
OpenJDK is made up of a number of repositories, namely corba, hotspot, jaxp, jaxws, jdk, langtools, and nashorn. Between OpenjJDK8 and OpenJDK9 there have been no new repositories introduced, but lots of new changes and restructuring, primarily due to Jigsaw – the modularisation of Java itself [2] [3] [4] [5].
repo composition, language breakdown (metrics are estimated)
Recent history
OpenJDK Build Benchmarks – build-infra (Nov 2011) by Fredrik Öhrström, ex-Oracle, OpenJDK hero!

Fredrik Öhrström visited the LJC [16] in November 2011 where he showed us how to build OpenJDK on the three major platforms, and also distributed a four page leaflet with the benchmarks of the various components and how long they took to build. The new build system and the new makefiles are a result of the build system being re-written (build-infra).


Below are screen-shots of the leaflets, a good reference to compare our journey:

How has Java the language and platform built over the years ?

Java is built by bootstrapping an older (previous) version of Java – i.e. Java is built using Java itself as its building block. Where older components are put together to create a new component which in the next phase becomes the building block. A good example of bootstrapping can be found at Scheme from Scratch [6] or even on Wikipedia [7].


OpenJDK8 [8] is compiled and built using JDK7, similarly OpenJDK9 [9] is compiled and built using JDK8. In theory OpenJDK8 can be compiled using the images created from OpenJDK8, similarly for OpenJDK9 using OpenJDK9. Using a process called bootcycle images – a JDK image of OpenJDK is created and then using the same image, OpenJDK is compiled again, which can be accomplished using a make command option:
$ make bootcycle-images       # Build images twice, second time with newly built JDK

make offers a number of options under OpenJDK8 and OpenJDK9, you can build individual components or modules by naming them, i.e.

$ make [component-name] | [module-name]
or even run multiple build processes in parallel, i.e.
$ make JOBS=<n>                 # Run <n> parallel make jobs
Finally install the built artefact using the install option, i.e.
$ make install


Some myths busted
OpenJDK or Hotspot to be more specific isn’t completely written in C/C++, a good part of the code-base is good ‘ole Java (see the composition figure above). So you don’t have to be a hard-core developer to contribute to OpenJDK. Even the underlying C/C++ code code-base isn’t scary or daunting to look at. For example here is an extract of a code snippet from vm/memory/universe.cpp in the HotSpot repo –
.
.
.
Universe::initialize_heap()

if (UseParallelGC) {
#ifndef SERIALGC
Universe::_collectedHeap = new ParallelScavengeHeap();
#else // SERIALGC
fatal("UseParallelGC not supported in this VM.");
#endif // SERIALGC

} else if (UseG1GC) {
#ifndef SERIALGC
G1CollectorPolicy* g1p = new G1CollectorPolicy();
G1CollectedHeap* g1h = new G1CollectedHeap(g1p);
Universe::_collectedHeap = g1h;
#else // SERIALGC
fatal("UseG1GC not supported in java kernel vm.");
#endif // SERIALGC

} else {
GenCollectorPolicy* gc_policy;

if (UseSerialGC) {
gc_policy = new MarkSweepPolicy();
} else if (UseConcMarkSweepGC) {
#ifndef SERIALGC
if (UseAdaptiveSizePolicy) {
gc_policy = new ASConcurrentMarkSweepPolicy();
} else {
gc_policy = new ConcurrentMarkSweepPolicy();
}
#else // SERIALGC
fatal("UseConcMarkSweepGC not supported in this VM.");
#endif // SERIALGC
} else { // default old generation
gc_policy = new MarkSweepPolicy();
}

Universe::_collectedHeap = new GenCollectedHeap(gc_policy);
}
. . .
(please note that the above code snippet might have changed since published here)
The things that appears clear from the above code-block are, we are looking at how pre-compiler notations are used to create Hotspot code that supports a certain type of GC i.e. Serial GC or Parallel GC. Also the type of GC policy is selected in the above code-block when one or more GC switches are toggled i.e. UseAdaptiveSizePolicy when enabled selects the Asynchronous Concurrent Mark and Sweep policy. In case of either Use Serial GC or Use Concurrent Mark Sweep GC are not selected, then the GC policy selected is Mark and Sweep policy. All of this and more is pretty clearly readable and verbose, and not just nicely formatted code that reads like English.


Further commentary can be found in the section called Deep dive Hotspot stuff in the Adopt OpenJDK Intermediate & Advance experiences [11] document.


Steps to build your own JDK or JRE
Earlier we mentioned about JDK and JRE images – these are no longer only available to the big players in the Java world, you and I can build such images very easily. The steps for the process have been simplified, and for a quick start see the Adopt OpenJDK Getting Started Kit [12] and Adopt OpenJDK Intermediate & Advance experiences [11] documents. For detailed version of the same steps, please see the Adopt OpenJDK home page [13]. Basically building a JDK image from the OpenJDK code-base boils down to the below commands:


(setup steps have been made brief and some commands omitted, see links above for exact steps)
 $ hg clone http://hg.openjdk.java.net/jdk8/jdk8 jdk8  (a)...OpenJDK8
or
$ hg clone http://hg.openjdk.java.net/jdk9/jdk9 jdk9  (a)...OpenJDK9
$ ./get_source.sh                                     (b)
$ bash configure                                      (c)
$ make clean images                                   (d)
(setup steps have been made brief and some commands omitted, see links above for exact steps)


To explain what is happening at each of the steps above:
(a) We clone the openjdk mercurial repo just like we would using git clone ….
(b) Once we have step (a) completed, we change into the folder created, and run the get_source.sh command, which is equivalent to a git fetch or a git pull, since the step (a) only brings down base files and not all of the files and folders.
(c) Here we run a script that checks for and creates the configuration needed to do the compile and build process
(d) Once step (c) is success we perform a complete compile, build and create JDK and JRE images from the built artefacts


As you can see these are dead-easy steps to follow to build an artefact or JDK/JRE images [step (a) needs to be run only once].


Benefits
– contribute to the evolution and improvement of the Java the language & platform
– learn about the internals of the language and platform
– learn about the OS platform and other technologies whilst doing the above
– get involved in F/OSS projects
– stay on top the latest changes in the Java / JVM sphere
– knowledge and experience that helps professionally but also these are not readily available from other sources (i.e. books, training, work-experience, university courses, etcetera).
– advancement in career
– personal development (soft skills and networking)


Contribute
Join the Adopt OpenJDK [13] and Betterrev [15] projects and contribute by giving us feedback about everything Java including these projects. Join the Adoption Discuss mailing list [14] and other OpenJDK related mailing lists to start with, these will keep you updated with latest progress and changes to OpenJDK. Fork any of the projects you see and submit changes via pull-requests.


Thanks and support

Adopt OpenJDK [13] and umbrella projects have been supported and progressed with help of JCP [21], the Openjdk team [22], JUGs like London Java Community [16], SouJava [17] and other JUGs in Brazil, a number of JUGs in Europe i.e. BGJUG (Bulgarian JUG) [18], BeJUG (Belgium JUG) [19], Macedonian JUG [20], and a number of other small JUGs. We hope in the coming time more JUGs and individuals would get involved. If you or your JUG wish to participate please get in touch.

—-

Credits
Special thanks to +Martijn Verburg (incepted Adopt OpenJDK),+Richard Warburton, +Oleg Shelajev, +Mite Mitreski, +Kaushik Chaubal and +Julius G for helping improve the content and quality of this post, and sharing their OpenJDK experience with us.

—-

How to get started ?
Join the Adoption Discuss mailing list [14], go to the Adopt OpenJDK home page [13] to get started, followed by referring to the Adopt OpenJDK Getting Started Kit [12] and Adopt OpenJDK Intermediate & Advance experiences [11] documents.


Please share your comments here or tweet at @theNeomatrix369.

Resources
[17] SouJava

This post is part of the Java Advent Calendar and is licensed under the Creative Commons 3.0 Attribution license. If you like it, please spread the word by sharing, tweeting, FB, G+ and so on!

SoCrates UK 2013 – my experiences!

Refactoring TDD habits

It’s about 15:21 on 19th September, a group of us from the London Software Craftsmanship Community gathered to leave for SoCrates UK 2013. We all gathered in the same and luckily found empty seats next to each other.

Amongst a lot of jokes, we suggested lets do a small dojo in the two hours we will be travelling. Wifi is free but terrible, on this train – nevertheless we tethered our phones and enabled Wifi on our laptops to make the most of the bandwidth.

This was also about the time when others sitting around me took notice of the desktop wallpaper, it looked something like this:

13 habits of good TDD programming

13 habits of good TDD programming

One thing lead to another, and before we knew, we were going through the above list of habits – some found the habits difficult to remember, others found duplicates and overlaps. There was this tendency that, this is a big list to remember when doing TDD!

So like developers we said we can “refactor” these habits into something more meaningful, maybe even organise them as per our thinking process.

From this conversation the below list was born:

12 habits to good TDD programming:
  1) Write the assertion first and work backwards
  2) Test should test one thing only
  3) See the test fail
  4) Write the simplest code to pass the test
  5) Refactor to remove duplications
  6) Don’t refactor with failing test(s)
  7) Write meaningful tests
  8) Triangulate
  9) Keep your test and model code separate (except when practising TDD-as-if-you-meant-it)
10) Isolate your tests
11) Organise your tests to reflect model code
12) Maintain your tests

You can notice already that the 13…habits shrunk to 12 habits, and their order changed in comparison to the original.

Then @Frankie mentions the difference between Test-first development and Test driven development, no one claimed to know it, but suggested the below:

Test-first development
– you start the development with tests, and change the tests if goal is not achieved via model (domain) code. – No guidelines about design, flexible approach.

Test driven development
– test drives the model code, you never change the test, only model code to reflect any changes that does not satisfy the tests! Guided by design.
Model code = production code or implementation, domain (better term)

Soon we arrived at Moreton-in-Marsh and our focussed changed to getting a taxi to the venue.

I continued contemplating with the idea of collaborating with other developers and ironing our this list – so it can be helpful at the least.

I met @gonsalo and @sandromancuso and ran the idea by them and they thought it was certainly an idea to present and get others involved to see what the final outcome could be.

Next morning everyone was proposing their presentations at the “Open Session”, and I took the chance of presenting – Refactoring TDD habits… in the Cheltham Loft room in the house called Coach at the Cotwold estate!

30 minutes into the conversation and we already attracted discussions between @sleepyfox and @sandromancuso, we did try to persuade them to avoid tangenting from our core topic of discussion.

@sandromancuso also shared with us one of the rules of simple design – as these have evolved over time, these sorta look like this on the flip chart he scribbled on:

4 rules of simple design

4 rules of simple design (thanks @racheldavies for taking this pic at the event, I have cropped it to fit it in here)

– passes all tests
– minimises duplication
– maximises clarity (clear, expressive, consistent)
– has fewer elements

The idea is that all your actions and practises could use these as guiding light to keep on track.

An hour and few minutes later and we have refactored and cleansed the original 13 habits down to 12 habits – reshuffled and rejuvenated. They may not be perfect but close (Kent Beck: use ‘perfect’ as a verb, not a noun – its a journey not a destination!

12 habits of good TDD programming

I sincerely hope that these list of habits do cover the essence of the principles, values and practises of TDD programming.

Prepare yourself with things you should or should not do, and then perform the Red-Green-Refactor actions to satisfy the TDD process.

The things some of the attendees not like is the way of the first list of habits were worded:

  • definition of the word ‘model’ or ‘domain’
  • use of the term duplication
  • the habits being assigned with numbers
  • use of the term assert – one assert per test or groups of assert per logical test

I also brought up the topic of the difference between Test-first development and Test driven development – and there were disagreements about it amongst the attendees, on the meanings of the definition itself. Please share with me your input!

Back to the final list of habits, @CarlosBle brought up a very valid point that some of the habits on the list might be time-based rather than relevant all the time – they were not linear but appear and disappear from the list depending on the current task in hand. We agreed we would sit together and work it out, but @sleepyfox was ahead of us and kindly drew this flow / state diagram on the flip chart that illustrated the list of habits but in a diagrammatic format:

Flow diagram of TDD habits

State/flow diagram of TDD habits (my apologies if its not clear, sometime down the line one of us could come up with a digital version of the state/flow diagram)

@CarlosBle – please feel free to come up with your time-based / non-linear list and share it with us when you get a chance.

Although we got a lot out of the session with discussions on various topics, we haven’t covered everything and finished discussing everything!

I’m more of the idea, that we could try to look at the above habits as trigger points (practical and pragmatic use) to make us do the right things when we are writing code with the intention of writing good code…clean code…quality code…whatever the terminology be in your environment.

At the end of the day, good practises become habitual only when practised repeatedly, with focus and intent.

Please do provide constructive criticism, if any, such feedback to the above are very welcome as they help improve the quality of the post!

Thanks to @sandromancuso & @sleepfox for helping and participating during my session. Big thanks to Socrates UK – its host, facilitators, organisors and sponsors for making this event happen!

Heroku API using RESTAPIUnifier

Background

I was encouraged and inspired by fellow developers and supporters at the recent LSCC event to blog about my experiences at Hacked.IO that took place a couple of weeks ago (20-21 July ’13), at the O2 dome in London.

A whopping 500+ developers in an huge auditorium, all set to compete for a challenge organised by O2’s fantastic team of geeks! It was one of the biggest hack events I have been to, my team was I and me ;)! It felt like I was amongst London’s cream-of-the-crop, some mega-talented folks with great ideas! A number of kids (under 10) put all of us to shame with their focus, talent and know-how on how to use tech gadgets and hack with them!

The motto of the event was “Learn – Build – Share” precisely what we all did during the weekend – unlike other events this wasn’t a mere competition, the organisers and sponsors were interested in seeing our creativity and anything innovative using things we had at our disposal!

c54cd-hacked-prizes-heroku-excitement

— All these dynamics certainly gave the judges a hard time to decide who were the best — (Read one of the judge’s experiences at the event)


Timeline

20/7 Sat. 10:00 am – we all arrive, get scanned, get our badges, have some breakkie! Mingle about, select our seats, try to connect with the Wifi – but no luck, this goes on for a while.

20/7 Sat. 12:00 noon – sponsors talk about what they do, and announce the challenges and prizes they are giving away!

After all the talks and workshops, I decided to narrow my focus down to one idea for which I could do something using a set of tools I already have – given at a hackathon we barely have time to make a full-fledged app, the best is to leave prepared from home! Do the bare-minimum with the best tools and ideas you have and present something that can be made production-ready later on!

I was surrounded by a number of other hackers who were working on anything from a flying drone to raspberry pi to Philips hue lamps! Not forgetting ones hacking away with python, java, javascript, php, etc… despite the lack of a decent Wifi connection!

20/7 Sat. 6-ish – Wifi is now up and running, slowly routers and modems are made available! We all switched from our USB-teethers to the now existent Wifi!

My Story


After going through the code base and fixing a number of things in the RESTAPIUnifier library, I had a “eureka” moment to make a Heroku API client available using the wrapper library written in Java – to craft API clients!

At this time, I had semi-interested contributors but zero commitment or dedication from them! So I went off writing the API client using the RESTAPIUnifier, and struggled for some time, between a bad Wifi connection, loads of API documentation, failing tests, clarity in thoughts, lots of noise around me and cramped up seats!

Towards midnight I had a couple of passing tests, which gave me confidence that there was hope, I was able to connect to the Heroku API end-point and use my API key to gain access and receive an {OK} against the OAuth API command. Everything is much easier, when you get authenticated by the end-point – first major step achieved!

My strengths: able to stay awake till late, past hackathon experience, past API experience, handy toolbox…., my dev. friends & supporters, ability to observe & learn from experiences, learn & share, stay positive, work independently & in teams, motivate myself and others!

What I learnt: keep chugging away without thinking of the outcome, participate and compete even though you might seem like an “odd” one out, watch how others present their ideas, everyone’s view counts, support the best and the best will support you, its not about winning but about what you have to give and share with others, be modest & humble!

21/7 Sun. 10:00 am onwards – what worked last night does not work fully this morning, now lets implement a new command for the same API and see if that works! An hour later, yay! that worked!

21/7 Sun. 11:15 am – back to the failing tests, tried to revive my project from a failing API call – switch to version 3.0 of Heroku API, that’s the latest, use that one says @neilmiddleton !

21/7 Sun. 11:30 am – 30 minutes before the submission closes, here’s the submission on Hackerleague: https://www.hackerleague.org/hackathons/hacked/hacks/heroku-api-using-restapiunifier.

21/7 Sun. 12noon onwards – presentations and judging starts

21/7 Sun. 4:30 pmHeroku API using RESTAPIUnifier bags the “star” prize among the five prizes in the Heroku category. Other submitters get the Heroku prizes as well!

The aim of the idea was to highlight the fact that Heroku API functionalities which are available from CLI (command-line interface), are now available programmatically through your applications – able to control the functionalities through your own programs.

Find out more about the work, by following the links mentioned above, contributors are welcome! This is just the beginning and there’s plenty of work to be done on the library and the dependent example API clients!

Github repo: https://github.com/neomatrix369/RESTAPIUnifier


Thanks

Thanks to @angiemae, @antonio, @emmanuel for their moral support! The @Heroku team also helped with knowledge and support when the tests were failing and I needed to know about their latest API!

Thank you @Heroku for selecting my submission and the other Heruko winners from amongst the 70+ submissions! Not forgetting John Stevenson (@jr0cket) and his team who were present both days sharing their knowledge and supporting us developers.

Signup on Heroku at https://id.heroku.com/signup/hackedio – to generate your API key, host your app and hack away!


Next time

Hmmmm, it was fun, not just ’cause I was able to present and win, but I met so many developers, many known faces and saw so much talent. I was inspired and wished the weekend didn’t end!

I’m looking forward to the next one, as it feels I’m getting better at it. Are you game? Yes, then join me, my closing quote: Together we are a formidable force! As individuals we contribute to the best of our abilities!

See you all at another hack / community event!

DevoxxUK 2013 aware-ne·ces·si·ty !


You are here because you love and care about software and have a passion for Java/JVM and Web technologies! You are familiar with software, smart devices or even robotics and when you see one you want to take it home! – You are an an inhabitant of planet DevoxxUK !

Post a comment at the bottom of the blog, saying what DevoxxUK means to you and why you are part of it, for e.g.:
– You have been to Devoxx conferences before.
– You are a speaker / presenter at DevoxxUK (I’ll add a speaker badge to your message banner)
– You have passion and love for technology and want to share it with other like minded folks
– You are facilitating the DevoxxUK event
– You may even be a sponsor to the event
– Or you can’t make it and sorry about it, but given a chance you would love to come (maybe next year)!
– Or maybe even wanting to say “Hi” and encourage others to participate in the event!

Just post a message similar to one(s) below with your message, your name, your twitter handle, blog link, a link to your photo (if available), or anything relevant you wish to share with the rest of the community – including details enlisted above. Selected shout-outs will appear in the body of the blog, others will remain in the comments section! Shout-outs will be posted and shared with community far and wide!

Shout-outs that stand-out will be highlighted and given focus!

Follow DevoxxUK on twitter at @DevoxxUK.

DevoxxUK Newsletters!
Newsletter #10 (March 19th)
Newsletter #9 (March 13th) ——– Newsletter #8 (March 7th)
Newsletter #7 (February 26th) ——– Newsletter #6 (February 19th)
Newsletter #5 (February 13th) ——– Newsletter #4 (February 5th)
Newsletter #3 (January 29th) ——– Newsletter #2 (January 22nd)
Newsletter #1 (January 8th)

DevoxxUK full schedule is out! —- Get your ticket today!

Adopt OpenJDK and Adopt-a-JSR events and speakers at DevoxxUK 2013 – Bring your Java queries realted to JSRs, the JCP, the OpenJDK, excetera to these events.

Adopt OpenJDK

Adopt-a-JSR

Speakers / Presenters

#DevoxxUK Shout-outs!

BeJUG

Devoxx

Devoxx UK is a great opportunity for the London Java Community (and beyond) to learn, network and get inspired during this event. It’s also a very nice extension to the already sold out Devoxx Belgium and France editions.

Attending a conference by developers for developers (at a very democratic price) can really bring value to your career. Of course the things you learn and pick up but more importantly the peer-to-peer contacts & conversations often inspire you to take your ideas (and even your code) to the next level.

Enjoy our first Devoxx UK edition.

— Stephan Janssen (@Stephan007) * java developer * BeJUG, Devoxx, Parleys and Playpass + Java Champion * Belgium


We are really looking forward to DevoxxUK. Great to get the conference over here and for the LJC to have such a big part in it.

At RecWorks we are always encouraging developers to engage with the community, learn more and get involved in what’s going on. DevoxxUK seems a perfect place to do that and at a price point that is open to everyone. See you at the LJC stand!

— Barry Cranford (@bcrecworks) * Founder, leader, organiser, community herder * London Java Community JUG (LJC JUG): @ljcjug, Graduate Developers Community: @GDCldn, RecWorks: @RecWorks, Samantha Hepburn: @SHRecWorks * London

European Platinum Partners

ImSpeakingAtDevoxxUK4
I have been to several Devoxx conferences (in Belgium) and to several other conferences all over Europe as well. I’m even going to be a speaker at Devoxx UK this year. So if you’re interested in creating Word/Excel documents with Java, come to my quickie. I’m also going to be co-host of the Duchess Women in IT BoF and maybe you’ll find me on a panel about teaching kids how to code.
ImSpeakingAtDevoxxUK2
— Linda van der Pal (@DuchessFounder) * java developer, leader, speaker * Belgium

ImSpeakingAtDevoxxUK4
Needless to say, I’m obviously very excited about Devoxx UK! Devoxx has always been the premier Java conference in Europe and it’s a fantastic opportunity for Londoners to get a piece of that. I’m particularly excited about some of the deep dive and topical talks on Java and Cloud security, and the Hands on Labs for Java EE7 and Java 8 are also major highlights for me. Get your ticket today!

ImSpeakingAtDevoxxUK2— Martijn Verburg (@karianna) * The Diabolical Developer, CTO – jClarity, London JUG co-leader (LJC), Speaker, Author, Javaranch Mod, PCGen & Adopt OpenJDK / A-JSR Cat herder, Java Champion, Cat herder 😉 * London


DevoxxUK is going to be my first proper software conference where I’m involved with the top brass (if I may) and their supporters, from world of developers ! The two days of DevoxxUK will be filled with excitement, throwing a lot of attention on technologies that developers are passionate about, that make them proud and that they hold close to their hearts ! Remember it is an event for developers by developers, for the community by the community. In the last so many years, the age range of developers have breached its limits again-and-again – today we have kids (Devoxx 4 Kids) as young as 10 are being introduced to robotics, smart devices and programming. But lets not forget the veterans of the software industry in their 50s and 60s who have left behind system-level and functional languages for us to cherish! DevoxxUK is hosting and welcoming many such inhabitants from a growing and maturing planet of developers and creators of our past, present and future!

— mani (@theNeomatrix369) * blog * #java developer * Agile (TDD, BDD), Bash, full of innovative ideas, thinker, creator! Blogger! LJC & LSCC member. LJC Advocate (@adoptopenjdk and @adoptajsr programs), MutabilityDetector! * london

Silver Partner

My very first software conference was LJC Conf 2012 and ever since then I have been bitten by the bug! Devoxx UK is a fantastic opportunity to learn about all the latest technologies and pick the brains of some of the greatest developers in our industry. I’m extremely excited and I simply can’t wait 😀

ImSpeakingAtDevoxxUK2

— Edward Yue Shung Wong (@arkangelofkaos) * passionate java developer * London

Great event!!!! I would like to go there this year!!!


— Emanuel Cordeiro (@emanuelcordeiro) * blog * java developer * CEJUG member * Brazil ____________________________________________________________________________________________________________________________________

The first time I heard about DevoxxUK was during Devoxx 2012 in Belgium. I’ve got immediately excited! No doubt that it´s going to be a great edition! I will attend on behalf of CEJUG (cejug.org) and my goal, besides attending all those great conference sessions, is to get in touch with AdoptOpenJDK and AdoptaJSR folks! Those guys are doing a great job by moving Java forward! CEJUG Community heavily supports these initiatives!


— Hildeberto Mendonca (@htmfilho) * java developer * CEJUG member * Brazil

____________________________________________________________________________________________________________________________________

Grande Evento, uma oportunidade para todos aqueles que amam tecnologia! A comunidade CEJUG estará bem representado com Hildeberto. (Great Event, an opportunity for all those who love technology! The CEJUG community will be well represented by Hildeberto.)

— Jardel Rodrigues (@JardelJava) * java developer * CEJUG member * Brazil

____________________________________________________________________________________________________________________________________

Yes CEJUG has the passion for java / JVM and Web technologies ! Manda Bala DevoxxUK (“Go go go DevoxxUK“) !!!


— Hélio Frota (@hf_metal) * enthusiastic java developer * CEJUG leader * Brazil

____________________________________________________________________________________________________________________________________

Vamos que vamos porque os cães ladram mas a caravana não para (PH) [“Come on everyone let’s go !”]

— Eduardo Vieira (@dudurct) * sysadmin * CEJUG * Brazil

____________________________________________________________________________________________________________________________________

É uma excelente oportunidade para a nossa comunidade (CEJUG) absorver um conteúdo tão bem elaborado para a rotina atual e corrida de um desenvolvedor. Desejo que os participantes aproveitem ao máximo o partilhar dos “Caras do desenvolvimento” e estreitem laços entre outros desenvolvedores de uma forma qualitativa e eficaz.
De fato é um evento da comunidade para a comunidade!
Ao infinito… e a Londres! (Rsrs)

It is an excellent opportunity for our community (CEJUG) to absorb content as well prepare for the development of the future of a developer. I wish the participants make the most of the presence of you “developers” and establish closer links between other developers in a qualitative and effective manner.
In fact it is a community event for the community!
To infinity … and London! (Lol)


— Israel Bruno (@ibmarques) * java developer * CEJUG * Brazil


Bronze Partners

I am so exited to know about this great conference, I was looking forward to participate in this year Devoxx UK, but I participated in Devoxx France but I have submitted lately and also for my bad chance it only has a limited slots for English sessions.

This conference gather the top quality speakers, some of them I know well, specially for their high activity and professionalization in JCP as we worked together in adopt-a-JSR program. Mani Sarkar, Somay, Hildeberto Mendonca, Arun Gupta, Adam Bien and Martijn Verburg.

I am looking forward to speak at next year Devoxx UK 2014 😉


— Mohamed Taman (@_tamanm) * systems architect * java team leader * MoroccoJUG * Morocco


ImSpeakingAtDevoxxUK4 Being part of Devoxx UK is very important to me. Not every developer works for a company that can afford to send them on expensive overseas or local conferences. And that’s where Devoxx is different, we strongly believe in catering to and the support of the local communities and want to offer a real value for money experience in the heart of London – home to 1000s of developers. Naturally I’m also really excited by the content; it’s one thing to be able to read things in blogs, it’s another to have a dynamic and engaging speaker serve up novel or challenging ideas like a pro at Wimbledon. So make sure you get a ticket and grab a seat in the front row!




— James McGivern (Twitter Id pending!) * mathematician turned programmer, software engineer * London


====>> Vacant space to be filled by you <<<====
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx!

ImSpeakingAtDevoxxUK2
— xxx (@xxxxx) * yyyy zzzzz * ppppp


====>> Vacant space to be filled by you <<<====
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx!

ImSpeakingAtDevoxxUK2
— xxx (@xxxxx) * yyyy zzzzz * ppppp



Starters

       
         
   

 

– Devoxx Party Sponsors –