Wednesday, November 13, 2013

Benchmark your HTTP server

Problem : Send 10,000 requests to a simple node.js server with variable number of concurrent requests.

1st test  - send 100 concurrent requests
2nd test -  send 500 concurrent requests

Following is my simple node.js server,
var http = require('http');

http.createServer(function (req, res) {
        res.writeHead(200, {'Content-Type': 'text/plain'});
        res.end('Hello Blogger\n');
        }).listen(8000, '127.0.0.1');

console.log('Server running at http://127.0.0.1:8000/');
To overcome my initial problem I found this simple http benchmarking tool apache-bench aka ab. This comes packed with apache installation or you can simply install by giving sudo apt-get install apache2-utils on debian based distro.

Save above node server in ~/node_test/server/ directory as server.js.
cd to the server directory using terminal.
Before running the next step make sure you have node.js installed in your environment. If not use this git repo and the README to install node.js.

We bind our node based HTTP server to port 8000 because binding to ports lower than 1000, we need root access.

If you don't want to use node.js, you can use any HTTP server to try out apache-bench. I will move with my node server,

start node server using following command,
node server.js
You will see something similar,
udara@thinkPad:~/node_test/server$ node server.js
Server running at http://127.0.0.1:8000/
Lets run a simple test on our node server. Open another terminal and run,
ab -n 100 -c 50 http://127.0.0.1:8000/
-n Denotes number of requests to perform.
-c Denotes number of multiple requests to make(concurrency).
http://127.0.0.1:8000/ is my node server running in the same machine, on port 8000.

*Make sure you have the trailing / after port number.

This will run ab for 100 requests with 50 concurrent requests. If you have done everything right you will get something similar informative output,

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient).....done


Server Software:       
Server Hostname:        127.0.0.1
Server Port:            8000

Document Path:          /
Document Length:        12 bytes

Concurrency Level:      50
Time taken for tests:   0.022 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      11300 bytes
HTML transferred:       1200 bytes
Requests per second:    4624.06 [#/sec] (mean)
Time per request:       10.813 [ms] (mean)
Time per request:       0.216 [ms] (mean, across all concurrent requests)
Transfer rate:          510.27 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.4      0       1
Processing:     1    9   4.1      9      15
Waiting:        1    9   4.1      9      15
Total:          2    9   3.8     10      15

Percentage of the requests served within a certain time (ms)
  50%     10
  66%     11
  75%     13
  80%     14
  90%     15
  95%     15
  98%     15
  99%     15
 100%     15 (longest request)

Now we can run all different combinations of actual benchmarking tests against our own node server.
ab -n 10000 -c 100 http://127.0.0.1:8000/
ab -n 10000 -c 500 http://127.0.0.1:8000/ 
You can use apache-bench to benchmark any HTTP server as I have mentioned earlier, this is pretty helpful tool to benchmark, compare your HTTP server environments.

run  man ab  to find out other options available in Apache HTTP server benchmarking tool ab.

Happy benchmarking !!!

User groups in Linux

While performing particular task in Linux based distro one might need to verify if that user belongs to a group with right permissions.

You can verify this using two simple commands.Open your terminal and run groups,
udara@thinkPad:~$ groups
udara adm cdrom sudo dip plugdev lpadmin sambashare
This will show the groups which logged-in user belong to.

If you wish to find the groups of any other user just run, groups {user_name},
udara@thinkPad:~$ groups udara
udara : udara adm cdrom sudo dip plugdev lpadmin sambashare
udara@thinkPad:~$ groups root
root : root

Friday, November 8, 2013

WSO2 products & H2 Console

How to query H2 Console while working with WSO2 carbon based product.

There are basically two ways to accomplish above mentioned.

1st method
  • Download H2 database from[1].
  • Extract and cd into the h2/bin directory.
  • Stop your carbon server.
  • Run sh h2.sh.
  • You need to enter correct JDBC URL, User Name and Password and press Connect.
In this approach each time you connect/query H2 Console you have to stop carbon server. (Bit annoying!!!)

2nd method
This is what interests everyone. In this method you don't need to download anything as we are going to use H2 Console embedded in WSO2 Product.
  • Update carbon.xml resides in CARBON_SERVER/repository/conf directory.
  • Search for "H2DatabaseConfiguration" and uncomment configuration parameters as in following screen-shot.
  • Save the configuration file and start carbon server.
  • Browse http://localhost:8082/ in your favorite browser.
  •  Enter correct JDBC URL, User Name and Password and press Connect. In my scenario,
JDBC URL : jdbc:h2:/home/udara/wso2/git/enterprise-store/modules/distribution/target/wso2store-1.0.0/repository/database/WSO2CARBON_DB
User Name : wso2carbon
Password : wso2carbon

You will get the H2 Console now.



[1] http://www.h2database.com/html/download.html

Wednesday, November 6, 2013

VIM as an IDE

Why vim? There are lots of text-editors & IDEs out there.Over last 7 years I have used several text-editors & IDEs.But for me simply I'm not a fan of IDEs.

I love the ability of customization and the lightweight of VIM over other text-editors and IDEs. If you are a newbie to VIM, you might feel uncomfortable until you get familiar with basic commands. You can find number of VIM cheat sheets here[1] to learn all VIM commands.
 
You may feel few OOTB IDE features missing in VIM, once you start to code with VIM. BIG NOOOO !!! How to browse files, how to search, search and replace, how to work with multiple files etc. I will address those further within this post.

When I want to browse project directories(file system) while coding simply use the command :NERDTree. You can download this plugin from[2] and extract the downloaded content to .vim directory located under your home directory. I'm using nerdtree 4.2.0 latest version available at the moment.After extracting you can find several directories under .vim as in following screen shot.


Thats it. Now simply open a text file from your project directory, and type :NERDTree hit enter.


You can navigate between directories/files in file-system with the use of arrow keys or with the mouse and select files by hitting enter. This plugin is really helpful when you are working with a huge code base within your project.

Now lets move to the find/replace in VIM, following are few commands we can use.
/text      - search for text
?text      - search backward for text
n          - repeat search in same direction
:%s/old/new- replace all old with new throughout file
Lets talk about syntax highlighting ability of VIM now. At the moment I'm working on WSO2 Enterprise Store project which written on top of the WSO2 Carbon platform. If you browse though this product you can find that VIM needs to coupe with number of file types here. For eg :- .jag (jaggery.js file extention), .js, .hbs, .css etc.

For unknown(unknown to VIM) file types like .hbs you can enable syntax highlighting(as HTML in my scenario) by simply adding, au BufReadPost *.hbs set syntax=html  to your .vimrc. You can find .vimrc under your home directory.

Before enabling syntax highlighting for .hbs file format,


after enabling syntax highlighting,


And the best thing is that this VIM configuration is so portable, you only need to replace default .vimrc with your customized version in any distribution.

You can download and install VIM from here[3].

[1] http://www.worldtimzone.com/res/vi.html
      http://www.fprintf.net/vimCheatSheet.html
[2] http://www.vim.org/scripts/script.php?script_id=1658
[3] http://www.vim.org/download.php

Monday, November 4, 2013

image sprites- Optimize your webApp

What is an image sprite? If you need a number of images regularly to render your web-app on a browser, you can have those in a single image rather than having multiple images.This single image is known as a sprite image.

What you gain from this approach? It will take only one HTTP request to render all your images so you can reduce the time to load your web-app.This approach will give a better image compression also.

How to use image sprites,



Lets say you are going to develop a web-app with the use of above sample sprite.Lets assume this sample contains 60, 40X40 icons in a single 480X200 file.

If we require the facebook icon, we can use CSS to show the correct image by positioning the background:

#facebook{
left:80px;
width: 40px;
background: url("sprite.jpg") 0 0;
}

Friday, October 25, 2013

MongoDB - usage of Primary Key

Like other databases MongoDB also uses primary keys to distinguish documents.
We can provide our own, otherwise MongoDB will create itself a primary key for each document. This key is a Object which consists of time-stamp and information about the machine which document created.

How to provide our own primary key

$connection = new Mongo( "10.100.0.128:27017" );
$db = $connection->selectDB("primary_key_blog");
$collection = $db->blog_collection;
$collection.insert({ _id: 1, author: "UdaraR", blog: "http://udarakr.blogspot.com" });

else we can just say,

$collection.insert({author: "UdaraR", blog: "http://udarakr.blogspot.com" });

If we read the document we stored back,
 $collection->findone( array( "author" => "UdaraR" ) );
You can see that MongoDB itself added the 12 bytes ObjectId.

The greatest thing in the second method is that you can extract created-time without storing it separately. You can simply use ObjectId.getTimestamp() method to get this done.
ObjectId("507c7f79bcf86cd7994f6c0e").getTimestamp();
Other than the above usage, as in all other databases the primary usage will be to query using the ObjectId.

$id = new MongoId("507c7f79bcf86cd7994f6c0e");
$collection->findone( array( "_id" => $id ) );
Keep in mind this is not equal to string "507c7f79bcf86cd7994f6c0e".

Other than findone, we can provide this ObjectId with find, remove, update etc.

Tried this sample on linux environment with the use of mongo driver.

Friday, October 4, 2013

Git push requires username and password ??

This post will help to overcome Github username password request during every git push. Instead of using SSH we use HTTPS to clone the repo and then every time you try to push to Github it will ask for your username and password for https://github.com.









To overcome this situation you can change the URL of your origin remote by using,

git remote set-url origin git@github.com:username/repo.git
In my scenario its,

git remote set-url origin git@github.com:wso2/enterprise-store.git
You can find the "git@github.com:username/repo.git" by browsing the Github repository and copying the SSH clone URL.