Jump in to the world of containers with docker

Why we need containers?

If we consider SDLC there are number of parties involve in different activities on our application.

Eg :- Developers, Testers, SysAdmins etc

There are various problems which these people need to address, because of platform and dependency standardization problems.

Docker address this problem with an open platform which "Build, Ship and Run Any App, Anywhere". Why docker VM's have already solved this problem? yes but with a virtual machine we are shipping our application+dependencies (assume 200MB) + guest OS (> 1GB) ouch :( that is painful.

Compared to VM, docker container includes only our application+dependencies :) Docker engine runs on top of the host OS and it's portable.

Let's jump in to our container, in this sample I'm going to create a docker container to ship WSO2 Enterprise store.

udara@udara-home:~$ sudo docker images
[sudo] password for udara:
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
ubuntu              latest              86ce37374f40        4 days ago          192.7 MB

I have ubuntu docker image with me and I'm going to start it.

sudo docker run -i -t ubuntu /bin/bash

With the above docker command I can login to the terminal of our newly created node,

udara@udara-home:~$ sudo docker run -i -t ubuntu /bin/bash
root@df8485e65e93:/#

Before running Enterprise Store inside our container I need to follow installation guidelines available here. So in this scenario we need to install JDK within our container.

Run following commands within your container terminal to install Java, zip and unzip.

apt-get install software-properties-common
add-apt-repository ppa:webupd8team/java
apt-get update
apt-get install oracle-java7-installer zip unzip

During this process if get "apt-get can't find the package" error similar to following as in my case use following command to lists to use old-releases.ubuntu.com,

sudo sed -i -e 's/archive.ubuntu.com\|security.ubuntu.com/old-releases.ubuntu.com/g' /etc/apt/sources.list

Err http://archive.ubuntu.com/ubuntu/ quantal/main openssl amd64 1.0.1c-3ubuntu2
  404  Not Found [IP: 91.189.91.14 80]
Err http://archive.ubuntu.com/ubuntu/ quantal/main ca-certificates all 20120623
  404  Not Found [IP: 91.189.91.14 80]
Err http://archive.ubuntu.com/ubuntu/ quantal/main libgirepository-1.0-1 amd64 1.33.14-1
  404  Not Found [IP: 91.189.91.14 80]
Err http://archive.ubuntu.com/ubuntu/ quantal/main gir1.2-glib-2.0 amd64 1.33.14-1
  404  Not Found [IP: 91.189.91.14 80]
Err http://archive.ubuntu.com/ubuntu/ quantal/main iso-codes all 3.38-1
  404  Not Found [IP: 91.189.91.14 80]
Err http://archive.ubuntu.com/ubuntu/ quantal/main python-apt-common all 0.8.7ubuntu4
  404  Not Found [IP: 91.189.91.14 80]
Err http://archive.ubuntu.com/ubuntu/ quantal/main python3-apt amd64 0.8.7ubuntu4
  404  Not Found [IP: 91.189.91.14 80]
Err http://archive.ubuntu.com/ubuntu/ quantal/main python3-dbus amd64 1.1.1-1
  404  Not Found [IP: 91.189.91.14 80]
Err http://archive.ubuntu.com/ubuntu/ quantal/main python3-gi amd64 3.4.0-1
  404  Not Found [IP: 91.189.91.14 80]
Err http://archive.ubuntu.com/ubuntu/ quantal/main unattended-upgrades all 0.79.3ubuntu4
  404  Not Found [IP: 91.189.91.14 80]
Err http://archive.ubuntu.com/ubuntu/ quantal/main python3-software-properties all 0.92.9
  404  Not Found [IP: 91.189.91.14 80]
Err http://archive.ubuntu.com/ubuntu/ quantal/main software-properties-common all 0.92.9

Set JAVA_HOME property using,

export JAVA_HOME=/usr/lib/jvm/java-7-oracle

Since Java is the only dependency to start any WSO2 product, I'm going to save the container status so I can create a container with any other product in future without installing Java.

Before Creating a new image from the above changes, we need to get our container ID,

How to list existing containers ID?

udara@udara-home:~$ sudo docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
df8485e65e93        ubuntu:latest       "/bin/bash"         6 minutes ago       Up 6 minutes                            compassionate_elion  
10a4800560de        ubuntu:latest       "/bin/bash"         27 minutes ago      Up 27 minutes                           dreamy_nobel

Let's find out changes we have done to the existing container by running,

sudo docker diff <container_id>
Eg :- sudo docker diff e69ccb432919

Ok, now I'm going to create new docker image by providing name "udara/java7"

sudo docker commit e69ccb432919 udara/java7

Our new image "udara/java7" is listed if I run command sudo docker images

udara@udara-home:~$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
udara/java7         latest              76f948ba0aff        24 seconds ago      762.3 MB
ubuntu              latest              86ce37374f40        4 days ago          192.7 MB
base                latest              b750fe79269d        20 months ago       175.3 MB
base                ubuntu-12.10        b750fe79269d        20 months ago       175.3 MB
base                ubuntu-quantal      b750fe79269d        20 months ago       175.3 MB
base                ubuntu-quantl       b750fe79269d        20 months ago       175.3 MB

Lets start Enterprise Store instance within our initial docker image,

Before that we need to copy Enterprise Store distribution from our host OS to docker image. You need to run following commands within your host OS.

Run following command to get the full container ID

sudo docker inspect -f  '{{.Id}}'<container_id>
Eg:- sudo docker inspect -f  '{{.Id}}' e69ccb432919

This command will return the full container ID, in my case its,

udara@udara-home:~$ sudo docker inspect -f   '{{.Id}}' e69ccb432919
e69ccb432919f7432e1cf804693662a9adc61fc5a10f16d1a87b0ff7993b1254

Let's copy ES to the docker image using following command.

sudo cp wso2store-1.0.0.zip /var/lib/docker/aufs/mnt/<full_container_id>/tmp/

Eg:-

sudo cp wso2store-1.0.0.zip /var/lib/docker/aufs/mnt/e69ccb432919f7432e1cf804693662a9adc61fc5a10f16d1a87b0ff7993b1254/tmp/

I copied wso2store-1.0.0.zip in to the /tmp directory of my docker instance. Go into the /tmp directory an unzip ES distribution. Change directory to wso2store-1.0.0/bin and start ES instance using sh wso2server.sh

Now let's create a separate Docker image consists of ES distribution which we can share among core developers/QA team. They can just start ES instance without worrying about underlying dependencies. (this example only Java but in a real world project there can be thousands of dependencies)

If you are new to docker follow this try-it tool available on your browser. If you want to try docker on your local machine here is the link to installation.Docker is available on almost all environments.

Path to linux screen

Screen makes dev/devops life easy in many ways,

1. If you want to keep a shell active, even with network/ssh interrupts.
2. Resume previous shell from another location
3. Keep a long running process

i). Start named screen, I will explain why we need a name in later step.

screen -R <name> 
Eg : - screen -R udara

If you don't have screen installed on your system run,

sudo apt-get install screen

ii). De-attach from the current screen

ctrl+a+d , this will return you back to the default shell( where you start screen)

This is the interesting and powerful feature of shell, where you can De-attach or Re-attach to the same screen.

iii). Re-attach to the same screen

screen -r will re-attach you to the previous screen.

But let's assume we have multiple De-attached screens, screen -r will list down all existing screens in that case.

Eg :-

udara@udara-home:~$ screen -r
There are several suitable screens on:
    5192.udara    (2014-11-25 10:41:15)    (Detached)
    5135.JMeter    (2014-11-25 10:40:59)    (Detached)
Type "screen [-d] -r [pid.]tty.host" to resume one of them.

This is where screen name comes handy :) You can easily recognize which screen you need to Re-attach. So use a meaningful name when starting your screen.

iv). Re-attach to a screen, when there are multiple De-attached screens

screen -r <screen-name>
Eg :-  screen -r 5192.udara

v). List existing screens 
Yes, If you have multiple screens screen -r will list down all existing screens. But there is a specific list command to do this.

screen -ls
screen -list


udara@udara-home:~$ screen -ls
There are screens on:
    5410.pts-7.udara-home    (2014-11-25 10:51:20)    (Detached)
    5192.udara    (2014-11-25 10:41:15)    (Detached)
    5135.JMeter    (2014-11-25 10:40:59)    (Detached)
3 Sockets in /var/run/screen/S-udara.


These are some basic commands which we can use to make our life easy, refer screen man page for more commands/shortcuts.

JMeter Loop Controller & Counter

Why we need loop controller[1] in JMeter?
Assume you need to run the same HTTP request(sampler) multiple times, rather than creating duplicate samplers we can use this handy logic controller available in JMeter.

Eg:- Publish 1000 APIs to WSO2 API Manager, Benchmark web-server concurrent requests handle

Lets take the first sample scenario, We can create a HTTP request sampler which publishes API. (before that we need to create a HTTP request to login which is out of loop controller scope)

1. Create a Loop Controller using Add> Logic Controllers> Loop Controller,

2. Add a Counter to above Controller using, Add> Config Element> Counter,

 3. Create sampler(s) according to your requirement, here I'm creating Add API HTTP Request Sampler, (Add> Sampler> HTTP Request),

You can see I'm appending ${api_id} for some request parameter values. This value will be initialized and incremented within our Counter(step 2). In my scenario I can't publish multiple APIs with the same name & context.

I tried this loop 20 times by updating Loop Count and Maximum values to 20. You can see multiple "Add API HTTP Request" & "Publish API HTTP Request" sampler requests,

[1] http://jmeter.apache.org/usermanual/component_reference.html#Loop_Controller

Run JMeter in command line

There are use-cases where you can't use JMeter GUI to run the script.

Eg :- during distributed JMeter load testing, run jmx from a remote machine (just using command line)

Following are the steps which I followed to run JMeter script in command line.

1. Create JMeter script in your local machine.(yes, using the GUI ;) )
2. Setup JMeter in the remote machine.

•  SCP apache-jmeter-X.XX.zip and created jmx in step 1.

3. Access remote machine using command line. (ssh ;))
4. Move and setup apache-jmeter-X.XX.zip and jmx according to your preference.

I have extracted  apache-jmeter-X.XX.zip inside my home directory(remote machine) and able to move to bin directory using,

cd /home/udara-aws/apache-jmeter-2.11/bin

Copied test.jmx script in to the same location.

Lets run this script in command line now.

sh jmeter.sh -n -t test.jmx -l log.jtl

Here we have used -n -t and -l arguments.

-n denotes –nongui run JMeter in nongui mode
-t denotes –testfile  jmx file which needs to run
-l denotes  –logfile file to put sampler logs (jtl)

Following is the output I got, If you are not getting summery results make sure to attach "Generate Summary Results" listener to relevant samplers.

udara@udara-aws:~/apache-jmeter-2.11/bin$ sh jmeter.sh -n -t test.jmx -l log.jtl
Creating summariser <summary>
Created the tree successfully using test.jmx
Starting the test @ Tue Nov 11 19:29:28 UTC 2014 (1415734168842)
Waiting for possible shutdown message on port 4445
summary +      1 in   3.3s =    0.3/s Avg:  3324 Min:  3324 Max:  3324 Err:     0 (0.00%) Active: 1 Started: 1 Finished: 0
summary +      6 in   4.1s =    1.5/s Avg:   676 Min:   124 Max:  1756 Err:     0 (0.00%) Active: 0 Started: 1 Finished: 1
summary =      7 in     8s =    0.9/s Avg:  1054 Min:   124 Max:  3324 Err:     0 (0.00%)
Generate Summary Results for list APIs =      1 in   1.1s =    0.9/s Avg:  1065 Min:  1065 Max:  1065 Err:     0 (0.00%)
Generate Summary Results for List Subscriptions =      1 in     1s =    1.6/s Avg:   637 Min:   637 Max:   637 Err:     0 (0.00%)
Tidying up ...    @ Tue Nov 11 19:29:36 UTC 2014 (1415734176890)
... end of run

Measure tempreture using TMP35 sensor

After receiving my LCD this came to my mind, as a weekend project :)

Ingredients

1. Arduino (I'm using uno here)
2. Breadboard with jumper cables (create the prototype)
3. LCD (I'm using 2*16)
4. TMP35 Sensor
5. USB cable

Other than that I have used a potentiometer(variable resistor) to control brightness of the LCD.

Let's configure our LCD first, I have used the following circuit found here[1].

Next step is to configure our temperature sensor. I'm using a TMP35 sensor for this, this provides a voltage output linearly proportional to the temperature. So we have to do some math within our program to output temperature in Celsius.

One more thing, unlike our LCD TMP35 provides analog signal as it's output. So we have to plug in to an analog pin located in Arduino.

Connect Analog out pin (middle one) to the A0 analog pin. Then + pin to the positive bus of our breadboard and - end to the ground bus. (make sure to plug these two pins in correct way, refer following image) 

Now we are good to proceed to our program since we are done with the circuit prototype. Connect Arduino into a computer and open ArduinoIDE to write the sketch.You can find complete sketch here.

#Adding LCD library
#include <LiquidCrystal.h>
#Init LCD library with digital pin 12, 11, 5, 4, 3 and #2   
LiquidCrystal lcd(12, 11, 5, 4, 3, 2);                           
const int sensorPin = A0;
void setup() {
    lcd.begin(16, 2);
    Serial.begin(9600);
}
void loop() {
    #Read the sensor value
    int sensorVal = analogRead(sensorPin);
    #Convert reading to voltage
    float voltage = (sensorVal/1024.0) * 5.0;
    #convert millivolts into temperature
    float temperature = (voltage - .5) * 100;
    lcd.print("Room Temp: ");
    lcd.print(temperature);
    delay(1000);
    lcd.clear();
} 

Now upload this sketch into your Arduino and here is our final outcome :)

[1] http://arduino.cc/en/Tutorial/LiquidCrystal

AV cable and raspberry pi B+

I'm able to taste the latest revision of Raspberry Pi, the model B+ recently. Compared to the previous version this has more advantages,

1) 40 GPIO pins
2) 4 USB ports
3) Neat micro SD slot with push to release capability
4) 3.5mm integrated Audio/video port

We can find tons of information available on-line about first three, but found the last point bit complicated.

If you have a nice HDMI supported monitor or TV no need to worry about this. But if you are going to plug your Raspberry pi B+ to an old fashioned TV with a AV cable this post might helps you :)

Selecting a cable,
This is bit tedious task since there is no standard for AV cable pinouts. Found this post which is really helpful for this task.

FYI : I bought zune AV cable with the correct pinout configuration for Raspberry pi B+.

Even with the correct cable you won't get this to work in first attempt :(

I tried with the NOOBS installation, by default NOOBS configures HDMI output with higher priority than our poor AV out.

You can find this configuration in /boot/config.txt file,

hdmi_force_hotplug=1 config_hdmi_boost=4

 I have commented out above default configuration and added following, with this configuration even if you have a HDMI monitor plugged in it will use the composite output.

hdmi_ignore_hotplug=1
#hdmi_force_hotplug=1
#config_hdmi_boost=4

Optimizing eclipse IDE on a Ubuntu box

I'm currently using eclipse Luna on my Ubuntu 13.10, IDE crashes after few seconds.
While searching google for a fix I found number of SO replies with following update to eclipse.ini ,

-Dorg.eclipse.swt.browser.DefaultType=mozilla

But it does not seem to fix my issue, then I tweaked existing eclipse.ini as follows,

-startup
plugins/org.eclipse.equinox.launcher_1.3.0.v20140415-2008.jar
--launcher.library
plugins/org.eclipse.equinox.launcher.gtk.linux.x86_64_1.1.200.v20140603-1326
-product
org.eclipse.epp.package.java.product
--launcher.defaultAction
openFile
-showsplash
org.eclipse.platform
--launcher.XXMaxPermSize
512m
--launcher.defaultAction
openFile
--launcher.appendVmargs
-vmargs
-Dosgi.requiredJavaVersion=1.6
-XX:MaxPermSize=512m
#-Xms40m
#-Xmx512m
-Dorg.eclipse.swt.browser.DefaultType=mozilla

I have commented out ms & mx parameters to pick my system parameters, then increased MaxPermSize to 512 MB.

Eclipse issue seems to be fixed now !!