Wednesday, November 27, 2013

Explore WSO2 Carbon admin Web services

The Carbon Management console is popular among WSO2 Product users.Without making any file level configuration changes, one can use web based admin console to interact with the Carbon installation.

As a Carbon developer one can ask, "Is there any other way to access those admin services rather than using the management console?"

Yes you can, Management functionalities inside the console UI communicate with back end web-services which you can use directly. All these web services are secured, you can access them with either HTTP Basic Authentication or session based authentication.

1. First you need to change the visibility of  admin web service WSDLs.
Open CARBON_HOME/repository/conf/carbon.xml and search for the HideAdminServiceWSDLs property.

Change the value of HideAdminServiceWSDLs property as follows,
<HideAdminServiceWSDLs>false</HideAdminServiceWSDLs>
2. Start Carbon server with –DosgiConsole parameter.
CARBON_HOME/bin$ sh wso2server.sh -DosgiConsole
3. Once server started in OSGI-Console mode just run listAdminServices command.
osgi> listAdminServices
Above command will list down all available admin services in your Carbon server.

I'm taking TenantMgtAdminService located at https://10.100.0.128:9443/admin/services/TenantMgtAdminService/ as a sample and explore this service within SoupUI.

4. Browse https://10.100.0.128:9443/admin/services/TenantMgtAdminService?wsdl

You can simply import this WSDL to SoapUI and check for all available services.

Create new soapui project with a name you want and https://10.100.0.128:9443/admin/services/TenantMgtAdminService?wsdl as the initial WSDL.



Happy Exploring !!!



Thursday, November 21, 2013

Deploy your own Enterprise Store [part 3]

I will discuss about few UI modifications you can do to make your own Music Store(or whatever) from what you have deployed, using WSO2 ES while following previous blog posts.You can find previous posts from [1] and [2].

At the moment we have our own mp3 asset type plus few other default asset types (gadget, site, e-book) in the ES.


First I'm going to hide all default asset types from the Store UI, there are number of ways to accomplish this task.

1. Get a fresh WSO2-ES pack and remove all default asset rxt's , asset types from store-tenant.json, publisher-tenant.json files while adding our new mp3 asset.

2. By modifying  /_system/config/store/configs/store.json in our registry. You can find this under Resources > Browse > _system > config >store > configs in our management console https://localhost:9443/admin/carbon/admin/login.jsp.

Under content menu you can update store.json with the following content by pressing Edit as text. We are going to remove all default asset types from the config.
{
  "assets": [
    "mp3"
  ],
  "assetsPageSize": 12,
  "commentsPageSize": 10,
  "userSpace": "\/_system\/governance\/users",
  "roles": {
    "Internal\/store": {
      "\/permission\/admin\/login": [
        "ui.execute"
      ]
    }
  },
  "userRoles": [
    "Internal\/store"
  ],
  "permissions": {
    "login": {
      "\/permission\/admin\/login": [
        "ui.execute"
      ]
    }
  },
  "paths": {
    "RXT_EXTENSION_PATH": "\/config\/ext\/"
  }
}
Just save it and browse our store http://localhost:9763/store/



You can update the icon-sprite.png resides under repository/deployment/server/jaggeryapps/store/extensions/assets/mp3/resources/  with a suitable one for our MP3 store.

Without writing a single line of code we can configure our simple digital asset store by using WSO2 ES :)

[1]. http://udarakr.blogspot.com/2013/11/deploy-your-own-enterprise-store-part-1.html
[2]. http://udarakr.blogspot.com/2013/11/deploy-your-own-enterprise-store-part-2.html

Wednesday, November 20, 2013

Deploy your own Enterprise Store [part 2]

I have explained how to install WSO2 ES with bundled default assets in the previous blog post.In this post I will explain how to introduce your own digital asset type to the ES.

Before going though those steps we need to understand a bit about WSO2 Governance Registry. WSO2 GREG enables storing meta-data of our digital assets and assets themselves. You can find more information about WSO2 GREG from here.

Make sure you have WSO2 ES up and running for now,
Browse following URL and sign-in to the management console with the use of given user credentials(default).
Management Console : https://localhost:9443/admin/carbon/admin/login.jsp
Username :admin
Password :admin 
If you browse through following, Main>Resources > Browse> _system> governance > sites > admin > Facebook you will see something similar to the following screen-shot.


This is exactly where all information of Facebook (site-asset) stored. If you select the version 1.0.0 you can see meta-data as well as the properties of this asset.


Lets move to the publisher for a second, browse https://localhost:9443/publisher/asset/site and you will get,


If you compare form fields of add-site form with the properties of above Facebook site-asset, you can clearly understand the relationship between them.

Lets add our own asset type to the ES now. Make sure you proceed with a fresh WSO2 ES pack now on, just unzip already downloaded zip to a different location.

All the resources you need to create are available @ mp3-store git repo.
You can either replace existing files or update as explain in following steps.

1. Introduce meta-data model(rxt file)
If you browse repository/resources/rxts/ directory inside ES distribution, you can see number of .rxt files there. Open site.rxt file using your favorite text editor.

(Yeah !! now you know exactly where all those add-site form fields came from.)

You may create your own meta-data model now.
Create mp3.rxt (Yes im going to create a Music Store) with the following content.
<?xml version="1.0"?>
<artifactType type="application/vnd.wso2-mp3+xml" shortName="mp3" singularLabel="MP3" pluralLabel="MP3s"
              hasNamespace="false" iconSet="10">
    <storagePath>/mp3s/@{overview_provider}/@{overview_name}/@{overview_version}</storagePath>
    <nameAttribute>overview_name</nameAttribute>
    <ui>
        <list>
            <column name="Provider">
                <data type="path" value="overview_provider" href="@{storagePath}"/>
            </column>
            <column name="Name">
                <data type="path" value="overview_name" href="@{storagePath}"/>
            </column>
            <column name="Version">
                <data type="path" value="overview_version" href="@{storagePath}"/>
            </column>
        </list>
    </ui>
    <content>
        <table name="Overview">
            <field type="text" required="true">
                <name>Provider</name>
            </field>
            <field type="text" required="true">
                <name>Name</name>
            </field>
            <field type="text" required="true">
                <name>Version</name>
            </field>
            <field type="text">
                <name>Createdtime</name>
            </field>
        <field type="options">
                <name label="Category">Category</name>
                <values>
                    <value>Country</value>
                    <value>Pop</value>
                    <value>Rock</value>
                </values>
            </field>
        <field type="text" url="true">
                <name>URL</name>
            </field>
        <field type="text" required="true">
                <name>Artists</name>
            </field>
            <field type="text-area">
                <name>Description</name>
            </field>
        </table>
        <table name="Images">
            <field type="text" required="true">
                <name>Thumbnail</name>
            </field>
            <field type="text" required="true">
                <name>Banner</name>
            </field>
        </table>
    </content>

</artifactType>

2. Introduce new asset type to the store.
Go to the repository/deployment/server/jaggeryapps/store/config/ directory inside youe ES distribution.
Open store-tenant.json file and update following line with the new asset type.
"assets": ["gadget", "site", "ebook","mp3"],
3. Go to repository/deployment/server/jaggeryapps/store/extensions/assets directory inside youe ES distribution.
Create a directory called mp3 inside assets directory and copy/paste content from already existing asset type.You can update the content if you wish to do so.

Create mp3.json inside repository/deployment/server/jaggeryapps/store/config/ext/ directory with the following content. You can refer existing default artifact json while creating this.

Sample

{
    "applyTo": "mp3",


    "import": [
        "/modules/rxt/ext/scripts/form.overview.exporter.js",
        "/modules/rxt/ext/scripts/form.exporter.js"],

    "fieldPropertyRules": [
        "overview.name:order=1",
        "overview.provider:order=2",
        "overview.description:order=3",
        "overview.version:order=4"
    ],

    "tables": [

        {
            "name": "*"
        }

    ],

    "fieldProperties": [

        {
            "field": "*",
            "name": "order",
            "value": ""
        },
        {
            "field": "*",
            "name": "hidden",
            "value": "false"
        },
        {
            "field": "*",
            "name": "visible",
            "value": "true"
        },
        {
            "field": "overview.*",
            "name": "save",
            "value": "default"
        },
        {
            "field": "overview.*",
            "name": "editable",
            "value": true
        },
         {
            "field": "images.*",
            "name": "editable",
            "value": true
        },
        {
            "field": "*.lifeCycle",
            "name": "save",
            "value": "asset.lifecycle.action.save"
        },
        {
            "field": "overview.provider",
            "name": "readOnly",
            "value": "true"
        },
         {
            "field":"overview.provider",
            "name":"editable",
            "value":false
        },
         {
            "field":"overview.name",
            "name":"editable",
            "value":false
        },
        {
            "field": "overview.createdtime",
            "name": "hidden",
            "value": "true"
        },
        {
            "field":"overview.version",
            "name":"editable",
            "value":false
        },

    ],

    "fields": [
        {
            "name": "lifeCycle",
            "table": "*",
            "label": "Life Cycle",
            "value": "SampleLifeCycle2"

        } ,
        {
            "name": "banner",
            "table": "images",
            "type": "file"
        },
        {
            "name": "thumbnail",
            "table": "images",
            "type": "file"
        },
        {
            "name": "url",
            "table": "overview",
            "type": "file"
        },
        {
            "name": "description",
            "table": "overview",
            "value": "The provider has not given a description."
        }

    ],

    "permissions": {
        "created": ["Internal/private_{overview_provider}"],
        "in-review": ["Internal/reviewer", "Internal/private_{overview_provider}"],
        "published": ["Internal/private_{overview_provider}"],
        "unpublished": ["Internal/private_{overview_provider}"]
    },


    "storage": {

        "images_banner": {

            "lifecycle": {
                "created": ["Internal/private_{overview_provider}"],
                "in-review": ["Internal/reviewer", "Internal/private_{overview_provider}"],
                "published": ["Internal/everyone", "Internal/private_{overview_provider}", "Internal/reviewer"],
                "unpublished": ["Internal/private_{overview_provider}"]
            }
        },

        "images_thumbnail": {

            "lifecycle": {
                "created": ["Internal/private_{overview_provider}"],
                "in-review": ["Internal/reviewer", "Internal/private_{overview_provider}"],
                "published": ["Internal/everyone", "Internal/private_{overview_provider}", "Internal/reviewer"],
                "unpublished": ["Internal/private_{overview_provider}"]
            }
        }

    }
}



4.  Introduce new asset type to the publisher.
Go to the  repository/deployment/server/jaggeryapps/publisher/config/ directory inside youe ES distribution.
 Open publisher-tenant.json file and update following line with the new asset type.
"collections": ["gadgets", "sites", "ebooks","mp3s"],


5. Go to the repository/deployment/server/jaggeryapps/publisher/config/ext/ directory inside youe ES distribution. create mp3.json file with the following content.


{
    "applyTo": "mp3",
    "import": [
        "/modules/ext/scripts/rxt.importer.js",
        "/modules/ext/scripts/form.exporter.js"],
    "fieldPropertyRules": [
        "overview.name:order=1",
        "overview.provider:order=2",
        "overview.description:order=3",
        "overview.version:order=4"
    ],
    "tables": [
        {
            "name": "*"
        }
    ],
    "fieldProperties": [
        {
            "field": "*",
            "name": "order",
            "value": ""
        },
        {
            "field": "*",
            "name": "hidden",
            "value": "false"
        },
        {
            "field": "*",
            "name": "visible",
            "value": "true"
        },
        {
            "field": "overview.*",
            "name": "save",
            "value": "default"
        },
        {
            "field": "overview.*",
            "name": "editable",
            "value": true
        },
         {
            "field": "images.*",
            "name": "editable",
            "value": true
        },
        {
            "field": "*.lifeCycle",
            "name": "save",
            "value": "asset.lifecycle.action.save"
        },
        {
            "field": "overview.provider",
            "name": "readOnly",
            "value": "true"
        },
         {
            "field":"overview.provider",
            "name":"editable",
            "value":false
        },
         {
            "field":"overview.name",
            "name":"editable",
            "value":false
        },
        {
            "field": "overview.createdtime",
            "name": "hidden",
            "value": "true"
        },
        {
            "field":"overview.version",
            "name":"editable",
            "value":false
        },

    ],
    "fields": [
        {
            "name": "lifeCycle",
            "table": "*",
            "label": "Life Cycle",
            "value": "SampleLifeCycle2"
        } ,
        {
            "name": "banner",
            "table": "images",
            "type": "file"
        },
        {
            "name": "thumbnail",
            "table": "images",
            "type": "file"
        },
        {
            "name": "url",
            "table": "overview",
            "type": "file"
        },
        {
            "name": "description",
            "table": "overview",
            "value": "The provider has not given a description."
        }
    ],
    "permissions": {
        "created": ["Internal/private_{overview_provider}"],
        "in-review": ["Internal/reviewer", "Internal/private_{overview_provider}"],
        "published": ["Internal/private_{overview_provider}"],
        "unpublished": ["Internal/private_{overview_provider}"]
    },
    "storage": {
        "images_banner": {
            "lifecycle": {
                "created": ["Internal/private_{overview_provider}"],
                "in-review": ["Internal/reviewer", "Internal/private_{overview_provider}"],
                "published": ["Internal/everyone", "Internal/private_{overview_provider}", "Internal/reviewer"],
                "unpublished": ["Internal/private_{overview_provider}"]
            }
        },
        "images_thumbnail": {
            "lifecycle": {
                "created": ["Internal/private_{overview_provider}"],
                "in-review": ["Internal/reviewer", "Internal/private_{overview_provider}"],
                "published": ["Internal/everyone", "Internal/private_{overview_provider}", "Internal/reviewer"],
                "unpublished": ["Internal/private_{overview_provider}"]
            }
        }

    }
}
Now you can test the set-up by publishing a mp3.



Start your fresh ES with above changes and browse our mp3 asset type using, https://localhost:9443/publisher/asset/mp3. Yes!! you will get something similar to the following screen-shot.


Just fill those form fields with your own values and create the first mp3 asset on ES :)

You will redirect to a page, where you can see something similar to this.
(Here I'm using the same Thumbnail and Banner created for the CNN site)


You can select your mp3, browse life-cycle and change its state to In-review -> published from created state.( If you do not have permission to run above step just log-in as admin to the ES)


Lets, log-in to the store and browse our newly added asset type using following URL http://localhost:9763/store/assets/mp3


And if you select already published MP3,in my case skyfall :). You will see something similar to the following where you can download, comment and rate the asset.


Now, you are good to go with your own asset (mp3) store.

Deploy your own Enterprise Store [part 1]

Why enterprise store?

We are living in a digital-era where we create and use thousands of digital assets. There are number of tools around us to create assets, but very few resources we have to store, publish, maintain which bring immense value addition to our digital assets. This is the enterprise wide problem solved by the WSO2 Enterprise Store. Organize your assets at one place, access from everywhere !!

ES powered by two major apps, store and publisher. Lets talk about the publisher first. As the back office of the store, publisher is the platform which create, manage life-cycle and monitor statistics of assets. You can provision those published assets though the store front-end.

Lets dive in-to the ES world by running the product, first download the WSO2 ES product from here and unzip the product.You can find following directory structure inside the ES.

├── bin
├── dbscripts
├── INSTALL.txt
├── lib
├── LICENSE.txt
├── modules
├── README.txt
├── release-notes.html
├── repository
├── resources
├── tmp
└── webapp-mode
Here I'm working in a linux environment, you can follow official installation guide here if you are using/familiar with any other environment.

Navigate to the bin directory and start the server using following commands within your terminal window.
cd bin/
sh wso2server.sh 
Once the server started you can browse,

Store URL : http://localhost:9763/store
Publisher URL : http://localhost:9763/publisher
Mgt Console URL  : https://localhost:9443/admin/carbon/


Use the following default user credentials to login, or else you can create your own user.

username : admin
password : admin

By default ES 1.0.0 shipped with provision for gadgets, websites and eBooks, you can browse, search, download or bookmark, manage life-cycle, view stats of those default digital assets in this ES installation.

Few screen-shots from the ES 1.0.0.

Store home

Site-Asset home


Sample- FaceBook asset page

In the following screen-shot you can find out some interesting social features of the WSO2 ES such as commenting, rating and tagging of assets.



Store-My items page

This is where you can see all assets that you have either download or bookmark from the store.


Publisher home

Asset Statistics

This is where you one can get a insight of his/her assets.
Number of bookmarks/downloads per asset, number of bookmarks/downloads during given period of time etc.


Asset Life-cycle management

Add new asset(gadget asset type)

Lets talk about how to install your own digital asset(rather than using gadgets,sites and e-books) in coming blog post.

If you wish to contribute to the WSO2 Enterprise-Store Product, yes its on github. You can clone ES from here and play/contribute with/to the source.

Wednesday, November 13, 2013

Benchmark your HTTP server

Problem : Send 10,000 requests to a simple node.js server with variable number of concurrent requests.

1st test  - send 100 concurrent requests
2nd test -  send 500 concurrent requests

Following is my simple node.js server,
var http = require('http');

http.createServer(function (req, res) {
        res.writeHead(200, {'Content-Type': 'text/plain'});
        res.end('Hello Blogger\n');
        }).listen(8000, '127.0.0.1');

console.log('Server running at http://127.0.0.1:8000/');
To overcome my initial problem I found this simple http benchmarking tool apache-bench aka ab. This comes packed with apache installation or you can simply install by giving sudo apt-get install apache2-utils on debian based distro.

Save above node server in ~/node_test/server/ directory as server.js.
cd to the server directory using terminal.
Before running the next step make sure you have node.js installed in your environment. If not use this git repo and the README to install node.js.

We bind our node based HTTP server to port 8000 because binding to ports lower than 1000, we need root access.

If you don't want to use node.js, you can use any HTTP server to try out apache-bench. I will move with my node server,

start node server using following command,
node server.js
You will see something similar,
udara@thinkPad:~/node_test/server$ node server.js
Server running at http://127.0.0.1:8000/
Lets run a simple test on our node server. Open another terminal and run,
ab -n 100 -c 50 http://127.0.0.1:8000/
-n Denotes number of requests to perform.
-c Denotes number of multiple requests to make(concurrency).
http://127.0.0.1:8000/ is my node server running in the same machine, on port 8000.

*Make sure you have the trailing / after port number.

This will run ab for 100 requests with 50 concurrent requests. If you have done everything right you will get something similar informative output,

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient).....done


Server Software:       
Server Hostname:        127.0.0.1
Server Port:            8000

Document Path:          /
Document Length:        12 bytes

Concurrency Level:      50
Time taken for tests:   0.022 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      11300 bytes
HTML transferred:       1200 bytes
Requests per second:    4624.06 [#/sec] (mean)
Time per request:       10.813 [ms] (mean)
Time per request:       0.216 [ms] (mean, across all concurrent requests)
Transfer rate:          510.27 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.4      0       1
Processing:     1    9   4.1      9      15
Waiting:        1    9   4.1      9      15
Total:          2    9   3.8     10      15

Percentage of the requests served within a certain time (ms)
  50%     10
  66%     11
  75%     13
  80%     14
  90%     15
  95%     15
  98%     15
  99%     15
 100%     15 (longest request)

Now we can run all different combinations of actual benchmarking tests against our own node server.
ab -n 10000 -c 100 http://127.0.0.1:8000/
ab -n 10000 -c 500 http://127.0.0.1:8000/ 
You can use apache-bench to benchmark any HTTP server as I have mentioned earlier, this is pretty helpful tool to benchmark, compare your HTTP server environments.

run  man ab  to find out other options available in Apache HTTP server benchmarking tool ab.

Happy benchmarking !!!

User groups in Linux

While performing particular task in Linux based distro one might need to verify if that user belongs to a group with right permissions.

You can verify this using two simple commands.Open your terminal and run groups,
udara@thinkPad:~$ groups
udara adm cdrom sudo dip plugdev lpadmin sambashare
This will show the groups which logged-in user belong to.

If you wish to find the groups of any other user just run, groups {user_name},
udara@thinkPad:~$ groups udara
udara : udara adm cdrom sudo dip plugdev lpadmin sambashare
udara@thinkPad:~$ groups root
root : root

Friday, November 8, 2013

WSO2 products & H2 Console

How to query H2 Console while working with WSO2 carbon based product.

There are basically two ways to accomplish above mentioned.

1st method
  • Download H2 database from[1].
  • Extract and cd into the h2/bin directory.
  • Stop your carbon server.
  • Run sh h2.sh.
  • You need to enter correct JDBC URL, User Name and Password and press Connect.
In this approach each time you connect/query H2 Console you have to stop carbon server. (Bit annoying!!!)

2nd method
This is what interests everyone. In this method you don't need to download anything as we are going to use H2 Console embedded in WSO2 Product.
  • Update carbon.xml resides in CARBON_SERVER/repository/conf directory.
  • Search for "H2DatabaseConfiguration" and uncomment configuration parameters as in following screen-shot.
  • Save the configuration file and start carbon server.
  • Browse http://localhost:8082/ in your favorite browser.
  •  Enter correct JDBC URL, User Name and Password and press Connect. In my scenario,
JDBC URL : jdbc:h2:/home/udara/wso2/git/enterprise-store/modules/distribution/target/wso2store-1.0.0/repository/database/WSO2CARBON_DB
User Name : wso2carbon
Password : wso2carbon

You will get the H2 Console now.



[1] http://www.h2database.com/html/download.html

Wednesday, November 6, 2013

VIM as an IDE

Why vim? There are lots of text-editors & IDEs out there.Over last 7 years I have used several text-editors & IDEs.But for me simply I'm not a fan of IDEs.

I love the ability of customization and the lightweight of VIM over other text-editors and IDEs. If you are a newbie to VIM, you might feel uncomfortable until you get familiar with basic commands. You can find number of VIM cheat sheets here[1] to learn all VIM commands.
 
You may feel few OOTB IDE features missing in VIM, once you start to code with VIM. BIG NOOOO !!! How to browse files, how to search, search and replace, how to work with multiple files etc. I will address those further within this post.

When I want to browse project directories(file system) while coding simply use the command :NERDTree. You can download this plugin from[2] and extract the downloaded content to .vim directory located under your home directory. I'm using nerdtree 4.2.0 latest version available at the moment.After extracting you can find several directories under .vim as in following screen shot.


Thats it. Now simply open a text file from your project directory, and type :NERDTree hit enter.


You can navigate between directories/files in file-system with the use of arrow keys or with the mouse and select files by hitting enter. This plugin is really helpful when you are working with a huge code base within your project.

Now lets move to the find/replace in VIM, following are few commands we can use.
/text      - search for text
?text      - search backward for text
n          - repeat search in same direction
:%s/old/new- replace all old with new throughout file
Lets talk about syntax highlighting ability of VIM now. At the moment I'm working on WSO2 Enterprise Store project which written on top of the WSO2 Carbon platform. If you browse though this product you can find that VIM needs to coupe with number of file types here. For eg :- .jag (jaggery.js file extention), .js, .hbs, .css etc.

For unknown(unknown to VIM) file types like .hbs you can enable syntax highlighting(as HTML in my scenario) by simply adding, au BufReadPost *.hbs set syntax=html  to your .vimrc. You can find .vimrc under your home directory.

Before enabling syntax highlighting for .hbs file format,


after enabling syntax highlighting,


And the best thing is that this VIM configuration is so portable, you only need to replace default .vimrc with your customized version in any distribution.

You can download and install VIM from here[3].

[1] http://www.worldtimzone.com/res/vi.html
      http://www.fprintf.net/vimCheatSheet.html
[2] http://www.vim.org/scripts/script.php?script_id=1658
[3] http://www.vim.org/download.php

Monday, November 4, 2013

image sprites- Optimize your webApp

What is an image sprite? If you need a number of images regularly to render your web-app on a browser, you can have those in a single image rather than having multiple images.This single image is known as a sprite image.

What you gain from this approach? It will take only one HTTP request to render all your images so you can reduce the time to load your web-app.This approach will give a better image compression also.

How to use image sprites,



Lets say you are going to develop a web-app with the use of above sample sprite.Lets assume this sample contains 60, 40X40 icons in a single 480X200 file.

If we require the facebook icon, we can use CSS to show the correct image by positioning the background:

#facebook{
left:80px;
width: 40px;
background: url("sprite.jpg") 0 0;
}