Thursday, September 12, 2013

ActiveMQ with PHP

First you need to download and configure ActiveMQ, You can find latest ActiveMQ release from[1].

Download latest release from http://www.apache.org/dyn/closer.cgi?path=/activemq/apache-activemq/5.8.0/apache-activemq-5.8.0-bin.tar.gz


Step1 : svn co http://svn.apache.org/repos/asf/activemq/tags/activemq-x.x.x

At the date of writing this post , ActiveMQ 5.8.0 is the latest release. So here after lets replace x.x.x with 5.8.0.


Step 2 : go to the activemq directory cd activemq-5.8.0/

Step 3 : Run mvn clean install to build ActiveMQ from source. This step will take a while.

Step 4 :  Edit <ACTIVEMQ_HOME>/conf/activemq.xml and add following transport connector.

 <transportConnector name="stomp" uri="stomp://localhost:61613"/>
Step5 : Start ActiveMQ: <ACTIVEMQ_HOME>/bin/activemq console
Step6 : Install PHP stomp extension pear install stomp.If you don't have pear package installed, run sudo apt-get install php-pear first.

Lets move to the PHP side now, we need to write two snippets one to publish messages another one to subscribe previously published messages.

First the publisher, create publisher.php inside /var/www/activemqsample/ directory with following content.

<?php
$queue  = 'ActiveMQ';
$msg    = 'ActiveMQ with PHP';

try {
    $stomp = new Stomp('tcp://localhost:61613');
    while (true) {
      $stomp->send($queue, $msg);
      sleep(1);
    }
} catch(StompException $e) {
    die('Connection error: '.$e));
}
?>
 Then the subscriber, create subscriber.php inside /var/www/activemqsample/ directory with following content.
<?php
$queue  = 'ActiveMQ';

try {
    $stomp = new Stomp('tcp://localhost:61613');
    $stomp->subscribe($queue);
    while (true) {
       if ($stomp->hasFrame()) {
           $frame = $stomp->readFrame();
           if ($frame != NULL) {
               print "Received: ".$frame->body;
               $stomp->ack($frame);
           }
       }

    }
} catch(StompException $e) {
    die('Connection error: ' . $e);
}
?>
Now you can test the sample php application by running,
php publisher.php and php subscriber.php within /var/www/activemqsample/ directory.


[1] http://activemq.apache.org/download.html

Wednesday, September 11, 2013

How to overcome WSO2 message broker OutOfMemory error

You many face difficulties when running WSO2 Message Broker with in-built Cassandra server as you are unable to allocate min memory requirement.

TID: [0] [MB] [2013-09-10 07:31:15,892] ERROR {org.wso2.andes.transport.network.mina.MinaNetworkHandler} - Exception caught by Mina {org.wso2.andes.transport.network.mina.MinaNetworkHandler} java.io.IOException: An established connection was aborted by the software in your host machine at sun.nio.ch.SocketDispatcher.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:25) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:198) at sun.nio.ch.IOUtil.read(IOUtil.java:171) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:243) at org.apache.mina.transport.socket.nio.SocketIoProcessor.read(SocketIoProcessor.java:218) at org.apache.mina.transport.socket.nio.SocketIoProcessor.process(SocketIoProcessor.java:198) at org.apache.mina.transport.socket.nio.SocketIoProcessor.access$400(SocketIoProcessor.java:45) at org.apache.mina.transport.socket.nio.SocketIoProcessor$Worker.run(SocketIoProcessor.java:485) at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51) at java.lang.Thread.run(Thread.java:662)
TID: [0] [MB] [2013-09-10 07:33:59,226] WARN {org.wso2.andes.server.queue.QueueEntryImpl} - Requesting rejection by null subscriber:org.wso2.andes.server.queue.QueueEntryImpl@3daf57b4 {org.wso2.andes.server.queue.QueueEntryImpl} TID: [0] [MB] [2013-09-10 07:33:59,226] WARN {org.wso2.andes.server.queue.QueueEntryImpl} - Requesting rejection by null subscriber:org.wso2.andes.server.queue.QueueEntryImpl@42a70a0f {org.wso2.andes.server.queue.QueueEntryImpl} TID: [0] [MB] [2013-09-10 07:33:59,242] WARN {org.wso2.andes.server.queue.QueueEntryImpl} - Requesting rejection by null subscriber:org.wso2.andes.server.queue.QueueEntryImpl@648580f8 {org.wso2.andes.server.queue.QueueEntryImpl} TID: [0] [MB] [2013-09-10 07:33:59,242] WARN {org.wso2.andes.server.queue.QueueEntryImpl} - Requesting rejection by null subscriber:org.wso2.andes.server.queue.QueueEntryImpl@78d64d56 {org.wso2.andes.server.queue.QueueEntryImpl} TID: [0] [MB] [2013-09-10 07:33:59,242] WARN {org.wso2.andes.server.queue.QueueEntryImpl} - Requesting rejection by null subscriber:org.wso2.andes.server.queue.QueueEntryImpl@3d6f8714 {org.wso2.andes.server.queue.QueueEntryImpl} TID: [0] [MB] [2013-09-10 07:33:59,242] WARN {org.wso2.andes.server.queue.QueueEntryImpl} - Requesting rejection by null subscriber:org.wso2.andes.server.queue.QueueEntryImpl@650d518e {org.wso2.andes.server.queue.QueueEntryImpl} TID: [0] [MB] [2013-09-10 07:33:59,242] WARN {org.wso2.andes.server.queue.QueueEntryImpl} - Requesting rejection by null subscriber:org.wso2.andes.server.queue.QueueEntryImpl@4d657576 {org.wso2.andes.server.queue.QueueEntryImpl} TID: [0] [MB] [2013-09-10 07:33:59,242] WARN {org.wso2.andes.server.queue.QueueEntryImpl} - Requesting rejection by null subscriber:org.wso2.andes.server.queue.QueueEntryImpl@4002bdac {org.wso2.andes.server.queue.QueueEntryImpl} TID: [0] [MB] [2013-09-10 07:33:59,242] WARN {org.wso2.andes.server.queue.QueueEntryImpl} - Requesting rejection by null subscriber:org.wso2.andes.server.queue.QueueEntryImpl@73f28808 {org.wso2.andes.server.queue.QueueEntryImpl} TID: [0] [MB] [2013-09-10 07:33:59,242] WARN {org.wso2.andes.server.queue.QueueEntryImpl} - Requesting rejection by null subscriber:org.wso2.andes.server.queue.QueueEntryImpl@37c100b0 {org.wso2.andes.server.queue.QueueEntryImpl} TID: [0] [MB] [2013-09-10 07:33:59,242] WARN {org.wso2.andes.server.queue.QueueEntryImpl} - Requesting rejection by null subscriber:org.wso2.andes.server.queue.QueueEntryImpl@60143f68 {org.wso2.andes.server.queue.QueueEntryImpl}


TID: [0] [MB] [2013-09-10 07:55:59,188] ERROR {org.wso2.andes.server.protocol.AMQProtocolEngine} - IOException caught in/192.168.2.84:50034(admin), session closed implictly: java.io.IOException: An existing connection was forcibly closed by the remote host {org.wso2.andes.server.protocol.AMQProtocolEngine} TID: [0] [MB] [2013-09-10 08:12:15,079] INFO {org.wso2.andes.server.stats.PerformanceCounter} - PerfCountFDAutoRenewalWriteQueue_1:14000/0 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], tot=15035/5751 {org.wso2.andes.server.stats.PerformanceCounter} TID: [0] [MB] [2013-09-10 08:20:22,065] ERROR {org.wso2.andes.server.cassandra.QueueDeliveryWorker} - Error running Cassandra Message FlusherJava heap space {org.wso2.andes.server.cassandra.QueueDeliveryWorker} java.lang.OutOfMemoryError: Java heap space at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:39) at java.nio.ByteBuffer.allocate(ByteBuffer.java:312) at org.wso2.andes.server.store.CassandraMessageStore$StoredCassandraMessage.<init>(CassandraMessageStore.java:4122) at org.wso2.andes.server.store.CassandraMessageStore$StoredCassandraMessage.<init>(CassandraMessageStore.java:4106) at org.wso2.andes.server.store.CassandraMessageStore.getMessagesFromNodeQueue(CassandraMessageStore.java:873) at org.wso2.andes.server.cassandra.QueueDeliveryWorker.run(QueueDeliveryWorker.java:241) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662



So the best practice is to have an external Cassandra server configured with WSO2 Message Broker, where you can have at least "2048M" for MAX_HEAP_SIZE or more if you face OutOfMemory exceptions.

You can find the complete installation guide [1].

[1] http://docs.wso2.org/display/MB210/Standalone+Deployment+on++External+Cassandra+Server

Tuesday, September 10, 2013

Query cassandra within jaggery.js

jaggery.js at 0.9.0-ALPHA-3 does not have Cassandra database engine support out of the box.

So you need to place cassandra connector libs within {JAGGERY_HOME}/carbon/repository/components/lib/ directory or use Cassandra feature enabled carbon product.

I'm going to use enterprise-store[1] from the branch social to develop this sample script.

Create jaggery file select.jag inside your jaggery app.

and put the following script inside select.jag,

<%
var db = new Database("jdbc:cassandra://localhost:9160/EVENT_KS","admin","admin",{"driverClassName":"org.apache.cassandra.cql.jdbc.CassandraDriver"});
var result;
db.query("USE EVENT_KS");
result = db.query("SELECT * FROM org_wso2_bam_phone_retail_store_kpi");
print(result);

%>

This[2] might help to publish data into Cassandra within jaggery.

[1] https://github.com/wso2/enterprise-store
[2] http://udarakr.blogspot.com/2013/09/publish-data-to-cassandra-from.html

Monday, September 9, 2013

Publish data to cassandra from jaggery.js using WSO2 BAM DataPublisher

To achieve the subject, I used kpi-definition sample app[1] with WSO2 BAM 2.3.0.

First I managed to install few BAM features within WSO2 enterprise-store[2].
You can get those features with the enterprise-store social branch, or install by yourself using following.

Update <enterprise_store_home>/modules/p2-profile-gen/pom.xml with BAM-cassandra feature.

Define following <featureArtifactDef> under <featureArtifacts> ,

org.wso2.carbon:org.wso2.carbon.cassandra.explorer.feature                                    org.wso2.carbon:org.wso2.carbon.cassandra.feature
org.wso2.carbon:org.wso2.carbon.bam.cassandra.data.archive.feature
org.wso2.carbon:org.wso2.carbon.databridge.datapublisher.feature
org.wso2.carbon:org.wso2.carbon.databridge.datareceiver.feature                                    org.wso2.carbon:org.wso2.carbon.databridge.cassandra.feature
org.wso2.carbon:org.wso2.carbon.module.mgt.server.feature                                    org.wso2.carbon:org.wso2.carbon.databridge.streamdefn.cassandra.feature

Define following features under <features> ,

org.wso2.carbon.cassandra.explorer.feature.group
org.wso2.carbon.bam.cassandra.data.archive.feature.group
org.wso2.carbon.cassandra.feature.group
org.wso2.carbon.databridge.datapublisher.feature.group
org.wso2.carbon.databridge.datareceiver.feature.group
org.wso2.carbon.databridge.cassandra.feature.group

Run mvn clean install from within your <enterprise_store_home>.
If have defined above features with correct versions you will get the BUILD SUCCESS with all newly added features.

Lets start the server,
cd modules/distribution/target/
unzip wso2store-1.0.0.zip
sh wso2store-1.0.0/bin/wso2server.sh

Keep in mind if you do have your own Cassandra server up and running make sure you stop that before above step.

Next lets write the same above kpi-definition Java sample in JavaScript, and save it as Stream.js

var host = "localhost";
var url = "tcp://" + host + ":" + "7611";
var username = "admin";
var password = "admin";

var PHONE_RETAIL_STREAM = "org.wso2.bam.phone.retail.store.kpi";
var VERSION = "1.0.0";

var Agent = org.wso2.carbon.databridge.agent.thrift.Agent;
var AgentConfiguration = org.wso2.carbon.databridge.agent.thrift.conf.AgentConfiguration;
var DataPublisher = org.wso2.carbon.databridge.agent.thrift.DataPublisher;
var Event = org.wso2.carbon.databridge.commons.Event;

agentConfiguration = new AgentConfiguration();
agent = new Agent(agentConfiguration);

dataPublisher = new DataPublisher(url, username, password, agent);
var streamId = null;
  try {
     streamId = dataPublisher.findStream(PHONE_RETAIL_STREAM, VERSION);

      } catch (e) {
            streamId = dataPublisher.defineStream("{" +
                                                  "  'name':'" + PHONE_RETAIL_STREAM + "'," +
                                                  "  'version':'" + VERSION + "'," +
                                                  "  'nickName': 'Phone_Retail_Shop'," +
                                                  "  'description': 'Phone Sales'," +
                                                  "  'metaData':[" +
                                                  "          {'name':'clientType','type':'STRING'}" +
                                                  "  ]," +
                                                  "  'payloadData':[" +
                                                  "          {'name':'brand','type':'STRING'}," +
                                                  "          {'name':'quantity','type':'INT'}," +
                                                  "          {'name':'total','type':'INT'}," +
                                                  "          {'name':'user','type':'STRING'}" +
                                                  "  ]" +
                                                  "}");
        }

           
        if(streamId){
            for (var i = 0; i < 100; i++) {
                publishEvents(dataPublisher, streamId, i);
                print("Events published : " + (i + 1)+"<br/>");
            }
            dataPublisher.stop();
        }

    function  publishEvents(dataPublisher, streamId, i){
       var quantity = 1000;
       var ext_str = new java.lang.String("external");
       var nokia_str = new java.lang.String("Nokia");
       var total_int = new java.lang.Integer(123);
       var quantity_int = new java.lang.Integer(12300);
       var user_str = new java.lang.String("UdaraR");
       eventOne = new Event(streamId, Date.now(),[ext_str], null,[nokia_str,total_int, quantity_int, user_str]);
       dataPublisher.publish(eventOne);
    }

Now you can use this within the jaggery script,

var Stream = require("Stream.js");
 

[1] http://docs.wso2.org/display/BAM230/KPI+Monitoring+Sample
[2] https://github.com/wso2/enterprise-store

Friday, September 6, 2013

How to secure API invocation inside an iframe with WSO2 API Manager

First you need to get started with the WSO2 API Manager, I'm using API Manager version 1.4.0 in this post.You can find API Manager latest version from here[1] and you can find installation guide for your platform here[2].

As I'm going to use a sample which is shipped with API Manager 1.4.0 by default, follow this guide[3] in order to set up the Samples.

Now I need the API Access-Token before invoking the API within iframe.

You can login to API store (https://localhost:9443/store) with the following credentials,username: subscriber1/Password: subscriber1 ,find YoutubeFeeds API under the APIs tab and subscribe to the API.

Click on the generate button within My Subscriptions tab. Now you will see something similar to this,



 After obtaining the application key you can test sample API invocation using curl command, E9Cx0Iyu20sOAZXLE5ltvkfWgWMa is the Access Token here.

 curl -H "Authorization :Bearer E9Cx0Iyu20sOAZXLE5ltvkfWgWMa" http://localhost:8280/youtube/1.0.0/most_popular

You will see results from the YouTube.

Now lets move to the iframe part.

I'm going to use LAMPP in order to host my sample html content.

Lets create a simple html file index.html under my /var/www/oauth_iframe/ directory and and set iframe content using following ajax request,

$.ajax({
    type: "GET",
    url: "http://localhost:8280/youtube/1.0.0/most_popular",
    beforeSend: function(xhr, settings){
    xhr.setRequestHeader('Authorization', 'Bearer E9Cx0Iyu20sOAZXLE5ltvkfWgWMa');},
    success: function(data){
        $("#iframe").attr('src',"data:text/html;charset=utf-8," + escape(data));
    }
});
If you need to restrict your API invocation from other domains,  
you can set your domain in allowed domains, this will ensure API content access only through the given domain.

[1] http://wso2.com/products/api-manager
[2] http://docs.wso2.org/display/AM140/Installing+the+Product
[3] http://docs.wso2.org/display/AM140/Setting+up+the+Samples