jnaapti

April 14, 2015September 5, 2017

A Docker Based Development Environment

At jnaapti, we have been using containerization technologies since almost the very early days. In 2011, when I was evaluating a solution to provide light-weight containers to our learners in the Virtual Coach, I was told that the only mature solution was to use Virtualization solutions. But Virtualization was too slow (in terms of boot up time) and I didn’t have enough resources to keep stand-by nodes running all the time. So after some evaluation, I decided to use LXC and it served its purpose. However, there were several features missing and I was in the verge of building a few of them myself.

So it’s not surprising that when I discovered Docker, I just fell in love with it. One of the first things we did was we moved our LXC based learner containers to Docker. We then slowly started to migrate portions of our infrastructure to Docker. We achieved full Docker migration last November, and then also moved our staging/testing systems in the cloud to Docker.

The last in the list was to migrate our development environments. The initial migration wasn’t too hard, because of 2 things:

We already had Docker in production – so it was a matter of working off our production Dockerfiles
We were already using KVM in our development – so we already had a clear idea about what containers our system should be running.

Why the move away from KVM based development environments? Simple! KVM’s disk usage is too much to suit our requirements. We had our development images of 5G each and you have 10’s of such images and are soon out of disk space. I am not sure how many Docker fans will approve some of the things I discuss here but I believe that this is a much leaner solution than using Virtualization and I don’t see any issues in the way I am doing it.

So with the migration from KVM to Docker, there were a few additional things that I wanted to handle:

Can we use Desktop tools like text-editors with data in Docker? Imagine I am writing a NodeJS application. I want to use Atom installed in my host to write code. However, I want to run NodeJS inside the container.
It is very easy to work with command line utilities (which don’t need X) in containers, but how about Desktop utilities like Eclipse? Can we run this in a Docker container and still have the same user experience as a regular app? What are some best practices to do so?
Is it possible to expose devices into Docker containers – this is required for eg, if we are doing Android development and want to debug our app in an actual Android device

The first 2 were rather easy and I sailed through it. The third one, I struggled a little, but I finally made some head-way.

This post attempts to capture some of my learnings in this entire process in case you want to build a similar environment. So let me answer these questions:

Using Desktop tools with Docker containers

This one is easy. A practice we follow is all the code that we write is inside a mounted volume. Containers are used to run processes in a contained fashion, but the processes are manipulating files that are in our host (and not in the underlying diff file system). We can as well remove our containers and we don’t lose anything.

Here is a sample run to demonstrate this:

Start a Docker container that has NodeJS installed in it. Make sure that this container has access to a local host directory (in this case /home/gautham/Desktop/node-data):

gautham@ananya:~|? docker run -d -P -v /home/gautham/Desktop/node-data:/data --name "node-example" ananya-nodejs:0.0.1 820e105db5061b380e6117e42a0cabad5f00c54e54f5016aefc18399e2a2eb25

Check if the container is running

gautham@ananya:~|? docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 820e105db506 ananya-nodejs:0.0.1 "/usr/sbin/sshd -D" 9 seconds ago Up 8 seconds 0.0.0.0:49155->22/tcp node-example

Inside the container (which is accessible via SSH):
gautham@ananya:~|? ssh -p 49155 ubuntu@localhost ubuntu@localhost's password: Last login: Tue Apr 14 14:13:12 2015 from 172.17.42.1 ubuntu@820e105db506:~$ cd /data/ ubuntu@820e105db506:/data$ ls hello.js ubuntu@820e105db506:/data$ node hello.js Hello World!

Using Desktop tools inside Docker containers

This one initially seemed a little difficult, but I figured out soon.

Create an image that has lxde-core package installed in it.

Now, there are 2 options:

Connect to lxde running inside the Docker container

Run these commands in host:

docker run -d -P ananya-desktop:0.0.1 sudo su xinit -- :1 &

This will now switch you to a different terminal (accessible at Ctrl+Alt+F8). You will also see a white terminal. Type the following in this terminal to start LXDE:

docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 38280dd66b98 ananya-android:0.0.1 "/usr/sbin/sshd -D" 4 minutes ago Up 4 minutes 0.0.0.0:49156->22/tcp, 0.0.0.0:49157->5901/tcp hopeful_lumiere ssh -X -p 49156 ubuntu@localhost

In the Docker container run

startlxde

You should now see a full fledged Desktop running like this!

So your host is running at Ctrl+Alt+F7 and your Docker container is at Ctrl+Alt+F8. Use this option in case you are running many Desktop applications and you want to be totally isolated from the host when working with the applications in the container (i.e you are not using any host applications in conjunction with the applications in the Docker container).

Only run the application that you are interested in

I found this option to be better in some ways. I have my Android Studio setup using this option now.

gautham@ananya:~|? docker run -d -P ananya-desktop:0.0.1 gautham@ananya:~|? ssh -X -p 49156 ubuntu@localhost ubuntu@localhost's password: Welcome to Ubuntu 14.04.2 LTS (GNU/Linux 3.16.0-34-generic x86_64)

* Documentation: https://help.ubuntu.com/ Last login: Tue Apr 14 14:25:47 2015 from 172.17.42.1 ubuntu@38280dd66b98:~$ cd android-studio/ ubuntu@38280dd66b98:~/android-studio$ ls ubuntu@38280dd66b98:~/android-studio$ bin/studio.sh

And lo and behold!

Accessing devices within Docker container

A final requirement was whether we can get Docker to detect USB devices. I found that if you pass a –privileged flag and mount the /dev/ device appropriately, you can then access it in the Docker container. I was able to successfully use adb along with my Docker container.

docker run --privileged -v /data:/data -v /dev/bus/usb:/dev/bus/usb -d -P ananya-android:0.0.1

Docker has been a boon and although there are several area of improvement, I see that it has a future. It has become an indispensable tool in our software tools arsenal in jnaapti.

Cross-posted here: http://buzypi.in/2015/04/14/a-docker-based-development-environment/

July 31, 2014November 3, 2019

Jnaapti’s Day Out @ FSMK 2014

In the last 3 years of Jnaapti’s functioning, we have visited more than 20 Engineering colleges in South India and one thing that we see in common is the lack of motivation in engineering students.

So when this is what seems like a norm, it’s always good to see organizations like Free Software Movement of Karnataka (FSMK) trying out initiatives like the FSMK Summer Camp 2014.

So when one of our Meetup participants asked us if we are interested in conducting sessions on free software technologies for this upcoming camp, we blindly accepted to be a part of this.

What the FSMK team was trying out was something really grand! Getting 185 participants (and another 20 odd volunteers) under one roof for a period of 9 days during the vacations and keeping their motivation levels high during the entire period is not an easy task. But like, Sarath MS (one of the key people behind FSMK) put it, “It’s better to try something this big and fail than to not give it a shot”.

Continue reading “Jnaapti’s Day Out @ FSMK 2014”

September 22, 2013November 8, 2016

The Story of Engineering Education in India

During the last 2 years of Jnaapti’s functioning, we have faced a lot of criticism from many of the stake-holders in this field. And we are not surprised.

This is an account of why this is so, and why Jnaapti functions the way it functions.

My intention of writing this post is to only help parents and students make an informed decision about their careers and make better choices for themselves.

People have criticized us and will continue to criticize us because they have a certain mindset when looking at the success of the IT industry over the last decade.

We have been approached by many parents when their sons/daughters are in the final year of Engineering hoping that we can get their children a job. They say, “My son just finished his Engineering and is now looking for a job. I believe that you can get him one?”. And it hurts us that we can’t help most of them because of the sheer lack of quality. In a lot of ways we feel that these people have been duped. If someone had done this outside the context of an Engineering degree, such institutions would have been forced to close and the people behind this would have been taken to the courts.

Continue reading “The Story of Engineering Education in India”

August 23, 2012November 8, 2016

Jnaapti – The Problems We’re After

Check out this statement from a report on National Employability by Aspiring Minds [pdf]:

Even though India produces more than five lakh engineers annually, only 17.45% of them are employable for the IT services sector, while a dismal 3.51% are appropriately trained to be directly deployed on projects. Further, only 2.68% are employable in IT product companies, which require greater understanding of computer science and algorithms. – National Employability Report by Aspiring Minds

While on one side, there is an ever increasing demand for skilled employees in organizations, the unemployment situation is getting out of control.

Concentrating on increasing quantity of engineers has impacted quality drastically. – National Employability Report by Aspiring Minds

Continue reading “Jnaapti – The Problems We’re After”

June 19, 2012November 8, 2016

Jnaapti – One Year On…

Time flies!

It has been a year since I started Jnaapti and it is time to take a step back and reflect on how things have fared during the last one year. I was reading through the mails and the Jnaapti weekly reports and was wondering if there is anything that I have to add to the 6 month report that I wrote back in December or if there is anything that I don’t agree to among the things I mentioned in that list.

So here goes:

Naming: Somehow, I don’t recall reading this anywhere, or atleast, didn’t think it was important. But, just like you do usability testing of your product, I think it is important to get the name tested with a few people before you christen your company. While choosing an Indian name, also remember to include people from various parts of India and abroad.
The name Jnaapti is a Sanskrit word which means to “acquire knowledge”. I had a few things in mind when choosing the name:
1. I wanted an Indian name – because the target segment was Indian to begin with
2. I didn’t want to have a problem with branding, in acquiring domains etc
3. Finally, I wanted the name to be generic enough to be used until the vision of the company has been achieved and shouldn’t require a change as I pivot.
Turns out, people don’t get it.

Here are some suggestions to people about naming a company:
Continue reading “Jnaapti – One Year On…”

December 1, 2011November 8, 2016

Jnaapti – A 6 month Report

This post is not quite about Jnaapti‘s vision and why I am doing it, but more about things that I have learnt since I founded Jnaapti.

I can’t believe it has been more than 6 months already and what an experience it has been! I can easily blog a few hundred pages about my experiences, but here is the MVP (Minimum Valuable Post) – the key things that other wantrepreneurs and newbie entrepreneurs can learn from:

The art of self-discipline: If there is one thing that I should pick among the various things that I learnt in the last 6 months, I would pick being self-disciplined and maintaining a rhythm in the face of distractions that are not under your control. Time management is extremely important. When you are handling all aspects of business, there is a good chance that you need to meet someone at their convenience, or you are asked to provide a service (in my case training) and these are not under your control. So you need to get really good at handling these, and getting back to your work as soon as this is complete. For productivity freaks, I use a combination of Pomodoro technique, Hamster, Workrave, this hack and a simple Libreoffice document to manage all my data. I wish it could have been simpler and someday, I hope to continue my efforts in building productivity tools.

Continue reading “Jnaapti – A 6 month Report”