IshaniQ: An Open-Source Java Platform for Quantitative Analysis

There is something fascinating about how students are learning software development today. Learning Java no longer needs to stop at loops, arrays, inheritance, and textbook exercises. With open-source libraries, public APIs, and modern visualization tools readily available, a student can turn those fundamentals into something surprisingly real.

IshaniQ is one such project! It is an open-source Java quantitative analysis and financial data exploration platform developed by a student from the University of Washington while learning software programming.

What began as an exercise in Java programming has gradually evolved into an application that retrieves real financial market data, persists and analyzes it, calculates technical indicators, and presents the results through interactive visualizations.

IshaniQ brings together several interesting open-source and developer technologies, including Java, TA4J, FINOS Perspective, OpenXava, JPA/Hibernate, Gson, and Maven, with Alpha Vantage providing market data.

Using TA4J, the project currently performs technical analysis with indicators such as Simple Moving Average (SMA), Relative Strength Index (RSI), and Average True Range (ATR). FINOS Perspective adds interactive visualization and exploratory analysis on top of the resulting datasets.

What I find particularly interesting about projects like IshaniQ is not simply the finished software. It is the way students can learn today.

A concept such as an array can lead to JSON parsing. JSON leads to REST APIs. APIs lead to object modelling and persistence. Market data introduces statistics and quantitative analysis. That leads naturally to visualization, open-source libraries, architecture, Git, documentation, and eventually publishing software for others to inspect and use.

The classroom concepts have not become less important. Instead, students now have an extraordinary opportunity to discover why those concepts matter by building something tangible with them.

IshaniQ remains an evolving project, and that is part of what makes it interesting. Its source code is publicly available, making it possible to follow its development, experiment with it, contribute ideas, or simply use it as an example of building a quantitative analysis application with Java.

Explore IshaniQ on GitHub:

• IshaniQ - Open-Source Java Quantitative Analysis
• A Medium article series introducing the core concepts

Perhaps the most encouraging aspect of projects like this is seeing students move from learning a programming language to using programming as a tool for exploration and creation. That is a meaningful difference in how the next generation of developers can learn.

A Brief Journey Through Web Services - SOAP, REST, and Beyond

Introduction

In this post, we’ll dive deep into the fascinating evolution of web services—from their early SOAP-based beginnings, through the rise of WebDAV and REST, and all the way to modern solutions like GraphQL and gRPC. Each approach addressed a particular set of challenges, and all of them left a lasting mark on the way applications talk to each other over the Internet.

Let’s explore how web services went from clunky XML exchanges to streamlined JSON endpoints to high-performance, real-time systems. Regardless of whether you are a seasoned developer or just starting your journey, understanding these historical shifts will give you valuable insights into the “why” behind today’s popular API styles.

Early Days with SOAP and RPC-Focused Services

The Advent of Simple Object Access Protocol (SOAP)

In the late 1990s, Simple Object Access Protocol (SOAP) emerged as a game-changer for exchanging data across different platforms. It mainly ran over HTTP (though it wasn’t strictly limited to HTTP), using XML to structure requests and responses in a standard way.

One of SOAP’s hallmark features was Remote Procedure Call (RPC) support, wherein you could call methods on a server like calling functions in your local code. While that felt intuitive, it also led to tight coupling between clients and servers. Plus, SOAP’s love for XML often caused messages to balloon in size, resulting in heavier network usage and slower processing times.

Challenges with SOAP

As cool as SOAP was in providing a universal format, it had a few pain points; some of which are -

• Excessive Verbosity: XML messages could become huge and weren’t always easy to parse or debug.
• Performance Hit: Parsing large XML documents demanded extra CPU cycles.
• RPC Coupling: Rigid, function-style calls made it trickier to evolve services without breaking clients.

These drawbacks prompted many developers to experiment with simpler, more flexible approaches.

WebDAV’s Subtle but Crucial Role

A Quick Look at WebDAV

Though not always labeled a web-service, Web Distributed Authoring and Versioning (WebDAV) expanded HTTP to support collaborative editing, file locking, and version management. Essentially, WebDAV took the standard HTTP methods—like GET and POST—and extended them so multiple users could work on the same files and resources.

This was particularly handy for document management, where version control mattered a lot. Even if WebDAV itself wasn’t about structured data or JSON, it showed how HTTP could be leveraged for more than just simple page fetches, paving the way for resource-focused patterns later on.

WebDAV’s Influence

WebDAV introduced the idea that HTTP methods could be used to manipulate resources in a standardized way. That concept would turn into a core principle of the REST approach—treating everything as an addressable resource and letting the standard HTTP verbs handle reading, writing, and deleting.

The Rise of Representational State Transfer (REST)

REST in a Nutshell

Around the early 2000s, Roy Fielding presented Representational State Transfer (REST) as a simpler architectural style that fully embraced the capabilities of HTTP.

Key Representational State Transfer (REST) principles include -

1. Stateless Interactions: Each request has all the info needed for the server to process it, making the server more scalable.
2. Uniform Interface: You use the same set of HTTP verbs (like GET, POST, PUT, DELETE) for consistent operations across different resources.
3. Layered System: Caches, load balancers, and proxies can sit between client and server for higher efficiency and flexibility.
4. Resource Representation: A resource can be delivered in different formats—JSON, XML, or even HTML—and the client picks what it needs.

Why REST Became So Popular?

Representational State Transfer (REST) was an instant hit for a few obvious reasons.

• Simplicity: The reliance on HTTP kept the technology stack lean and familiar.
• Less Overhead: Switching from XML to JSON in many cases significantly cut down message size and parsing difficulty.
• Loose Coupling: Clients interact with resources via well-known URIs, so servers can evolve behind the scenes without constantly breaking client integrations.

As a result, RESTful APIs quickly became the go-to option for web-based communication, especially for public APIs—think social media platforms, payment gateways, and mapping services.

Where REST Falls Short?

Like every architecture, REST isn’t perfect! Following are some of the most common complaints -

1. Overfetching & Underfetching: You might get too much or too little data by hitting a single endpoint, forcing multiple calls or extra filtering.
2. Complex Queries: Dealing with nested or related resources can get messy, often leading to lots of endpoint variations.
3. No Real-Time Model: REST’s request-response pattern isn’t inherently set up for real-time data pushes. You’d have to bolt on WebSockets or another trick for two-way communication.

For many straightforward CRUD tasks, REST still rules. But as applications got more complex—especially on mobile or in richly interactive front-ends—developers began looking for solutions that could solve these particular pain points.

Evolving Past REST with GraphQL and gRPC

GraphQL - A Query Language for APIs

Developed at Facebook (now Meta) and open-sourced in 2015, GraphQL quickly gained traction for solving the overfetching and underfetching problems. Instead of juggling many REST endpoints, GraphQL condenses everything behind a single endpoint.

How GraphQL Works?

• Clients define exactly what data they need in a nested, declarative format. No more extra payload, no more missing data.
• The server has a strongly typed schema that outlines the data’s structure, enabling auto-documentation and robust validation.
• It’s particularly loved by front-end teams, as they can shape queries to match UI needs without waiting for the back-end to create new endpoints.

Trade-Offs

• It introduces a new query language and a more complex server-side resolver system.
• Traditional caching strategies based on resource URIs aren’t straightforward with a single GraphQL endpoint, so new caching patterns or frameworks are often required.

gRPC - High-Speed, Low-Latency Communication

While GraphQL tackles data-fetching issues, gRPC, originating from Google, focuses on performance and streaming scenarios. It’s built on HTTP/2 and uses Protocol Buffers (Protobuf) for a compact, binary approach.

Standout Features

• Small, Efficient Messages: Protobuf keeps data slim, reducing bandwidth usage and boosting speed.
• Streaming Support: gRPC can handle bidirectional streaming natively, making it perfect for real-time data and microservices that need constant updates.
• Contract-First Development: Defining a .proto file sets the structure for both client and server, reducing inconsistencies and integration errors.

Downsides

• Requires learning Protobuf schemas and dealing with a separate toolchain.
• Debugging binary messages is less intuitive than reading JSON or XML if something goes wrong.

Epilogue

From the XML heavy days of SOAP to the resource-friendly nature of REST, and finally, to the more refined problem-solving approaches of GraphQL and gRPC, web services have continually adapted to meet changing requirements. SOAP demonstrated the value of standardized protocols, while WebDAV paved the way for using HTTP verbs to manipulate resources. REST then capitalized on HTTP’s simplicity, but its one-size-fits-all approach sometimes struggled when data relationships got complex or real-time updates were crucial.

GraphQL stepped up to reduce excessive network calls and offer a front-end friendly API, while gRPC brought lightning-fast performance and streaming capabilities to microservices. Each style - whether SOAP, REST, GraphQL, or gRPC still has its place, catering to different technical and organizational needs. By understanding how these approaches evolved, developers can make more informed decisions, matching the right tool to the right job. After all, tech moves fast - but knowing our history helps us chart a better course forward.

Dynamic Traits with Java

Traits are reusable components representing a set of methods or behaviors that can be used to extend the functionality of multiple classes. Some programming languages like Groovy and Scala have been supporting Traits for a long time, and allow dynamic association of Traits to any object, at runtime.

Java being a strongly typed programming language, enforces a very strict discipline of type association, upfront whilst declaring any language element, e.g. variables, functions, etc; and the same is applicable for Classes too. Therefore when you implement a class in Java, the associative types for the class, must be specified as a part of the declaration of the class, which strongly binds the type interfaces to the class, and cannot be changed in runtime.

The concept of Traits is implemented in Java using Interfaces, which contain default methods and variables to allow maintaining an object's state. However, these interfaces then have to be tightly coupled with the class declaration, and as discussed above one cannot dynamically modify an interface association.

Java-DynaTraits is a very simple and minimalist library, that allows dynamic association of Traits/Interfaces; such that new interfaces could be added to your class at runtime, without really having to specify them, with the declaration of the class.

The core concept behind the code is to use a subscription model, which allows registering interfaces (or in other words Traits) with a class, in runtime, and then using an Invocation Handler to delegate function calls to the relevant subscriptions.

An Example Code

Following is a sample code, that demonstrates an example usage -

var catalog = (CatalogInterface & NamedEntity & PricedEntity) 
(new Catalog()).withTraits(new NamedEntity(){}, new PricedEntity(){});

// The setName function is rendered by the "NamedEntity" trait.

catalog.setName("Dynamic Traits with Java");

System.out.println(catalog.getName());

// The setPrice function is rendered by the "PricedEntity" trait

catalog.setPrice(20.00);

System.out.println(catalog.getPrice());

Please visit the project page on Github for further details.

Java 7, brings a simple yet a long awaited feature...

Would you be surprised to know that the API of java.io.File only supported getting the last modified time, and NOT the file's creation time, until Java 7...? Well, not many people seem to be aware of this and surprisingly even the Internet was very quiet on this topic all the while.

So if you are using anything below Java 7 and are desperate to fetch a file's creation time then, one solution would be to write some native code to call system routines and then call the native code using JNI. Most of this work seems to be already done for you in a library called JNA, though. Nevertheless, you will still need to do a little OS specific coding in Java for this, though, as you'll probably not find the same system calls available in Windows and Unix/Linux/BSD/OS X.

VMWare Cloud Foundry - Developing "cloud-ready" web applications has never been so easy

A lot has been said and done about services on the cloud lately and "Platform as a Service" (PaaS) is probably one of the most popular buzz-word that's been around for quite sometime now. However, I sincerely feel that the common man (read as "developer") is yet to get a taste of what it really means. Now, this very paradigm seems to be soon changing, with VMWare Cloud Foundry. Cloud Foundry is the world’s very first open Platform as a Service (PaaS). It is designed to help developers easily create web applications, using multiple programming frameworks including Spring for Java, Ruby on Rails and Sinatra for Ruby; that can run upon public and private cloud environments, with just about no additional learning curve. So, by now if you are dreaming of a possibility to port your existing web-applications to the cloud with minimal efforts, then let me give you the good news - "it's all possible here..."

Well, looks like i've done a lot of sales talks in favour of VMWare Cloud Foundry. Now lets get under the hood and get hands-on with developing an extreamly simple web application that demonstrates the following -

1. Setting up VMWare Cloud Foundry
2. Consuming Cloud Foundry's MySQL service in a JSP, using the standard JDBC way
3. Deploying a web archive and running the JSP on Cloud Foundry

Setting up VMWare Cloud Foundry

Windows Setup

1. Apply for a Cloud Foundry account at www.cloudfoundry.com; you will be notified by email when your account is activated
2. Install Ruby from www.rubyinstaller.org; The Cloud Foundry cloud-controller command line is built in Ruby so this is required. As the installer runs, make sure to check the boxes to add the ruby directory to your command path
3. Start the Windows command line client (Start Menu -> Run -> "cmd")
4. To install the cloud-controller tool type : gem install vmc (If you are behind a firewall then : gem install --http-proxy http://proxy.vmware.com:3128 vmc)

Congratulations! The Cloud Foundry Cloud Controller is now installed. From here on, you may type Cloud Foundry commands into the Windows command window.

1. Inform Cloud-Foundry which cloud you want to connect to : vmc target api.cloudfoundry.com
2. To login to Cloud Foundry type : vmc login (enter your account credentials when prompted)

Linux Setup

The document here is the best source of information for setting up the cloud-controller upon various Linux flavours.

Consuming Cloud Foundry's MySQL service

There are plenty of elaborate tutorials available on the web, that illustrate accessing the MySQL service at Cloud foundry using Spring. However, this one is a bit different, because it does not use Spring at all and demonstrates the same functionality with plain and simple Java and standard JDBC API.

The following code snippet demonstrates the approach -

<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>

<%@page import="java.sql.*,javax.sql.*"%>
<%@page import="org.cloudfoundry.services.*"%>
<%
String query = "Select * FROM users";
%>

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Cloud Foundry with Simple Java</title>
</head>
<body>
Welcome to the simple web application on cloud-foundry...
<%
Connection connection = null;

try {
// establish connection to MySQL Service
ServiceManager services = ServiceManager.INSTANCE;
connection = (Connection) services.getInstance(CloudFoundryServices.MYSQL);

if (connection != null && !connection.isClosed()) {
out.println("<p>Successfully connected to MySQL service</p>");

// creating a database table and populating some values
Statement s = connection.createStatement();
int count;
s.executeUpdate("DROP TABLE IF EXISTS animal");
s.executeUpdate("CREATE TABLE animal ("
+ "id INT UNSIGNED NOT NULL AUTO_INCREMENT,"
+ "PRIMARY KEY (id),"
+ "name CHAR(40), category CHAR(40))");

out.println("<p>[1] Table successfully created.</p>");

count = s.executeUpdate("INSERT INTO animal (name, category)"
+ " VALUES"
+ "('snake', 'reptile'),"
+ "('frog', 'amphibian'),"
+ "('tuna', 'fish'),"
+ "('racoon', 'mammal')");

out.println("<p>[2] " + count + " rows were inserted.</p>");

count = 0;
ResultSet rs = s.executeQuery("select * from animal");
while (rs.next()) {
count++;
}
out.println("<p>[3] " + count + " rows were fetched.</p>");

s.close();
}
} catch (Exception e) {
out.println(e.getMessage());
} finally {
if (connection != null && !connection.isClosed()) {
connection.close();
}

connection = null;
}
%>
</body>
</html>

In the above code, org.cloudfoundry.services is a custom package which primarily contains a Singleton implementation, namely "ServiceManager"; and an interface to hold the constants, namely "CloudFoundryServices". Following are the sources for the same -

CloudFoundryServices.java

package org.cloudfoundry.services;

public interface CloudFoundryServices {
public static final int MYSQL = 1;
}

ServiceManager.java

package org.cloudfoundry.services;

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;

import argo.jdom.JdomParser;
import argo.jdom.JsonNode;
import argo.jdom.JsonRootNode;

public enum ServiceManager implements CloudFoundryServices {

INSTANCE;

private static final String NULL_STRING = "";

public Object getInstance(int service_type) throws Exception {
if (service_type == MYSQL) {
return getMySQLConnection();
} else {
throw new IllegalArgumentException("Service for id " + service_type + " not found...");
}
}

/*
* This method is responsible for establishing a valid connection to the MySQL service,
* using the credentials available in the environment variable, namely "VCAP_SERVICES".
*
* The content of VCAP_SERVICES environment variable is a JSON string, thus this method
* uses standard interfaces from the Argo JSON parsing API to extract the credentials.
*/

private Object getMySQLConnection() throws SQLException {
String vcap_services = System.getenv("VCAP_SERVICES");

String hostname = NULL_STRING;
String dbname = NULL_STRING;
String user = NULL_STRING;
String password = NULL_STRING;
String port = NULL_STRING;

if (vcap_services != null && vcap_services.length() > 0) {
try {
JsonRootNode root = new JdomParser().parse(vcap_services);

JsonNode mysqlNode = root.getNode("mysql-5.1");
JsonNode credentials = mysqlNode.getNode(0).getNode("credentials");

dbname = credentials.getStringValue("name");
hostname = credentials.getStringValue("hostname");
user = credentials.getStringValue("user");
password = credentials.getStringValue("password");
port = credentials.getNumberValue("port");

String dbUrl = "jdbc:mysql://" + hostname + ":" + port + "/" + dbname;

Class.forName("com.mysql.jdbc.Driver");
Connection connection = DriverManager.getConnection(dbUrl, user, password);
return connection;
} catch (Exception e) {
throw new SQLException(e);
}
}

return null;
}
}

Now all you need to do now is, bundle everything together and get yourself a standard WAR (web archive) file.

Deploying a web archive and running the JSP on Cloud Foundry

Once you have the WAR file in place, browse to the location where the file is placed, using your command-line console; and execute the following cloud-controller commands -

• In case you are not yet logged in to the cloud, type : vmc login (and, provide the required credentials when prompted)
• vmc push
• Enter a unique application name when prompted, could be anything like in my case its my name "rahulr"
• Select 'Y' when prompted to bind a service and select the MySQL service from the offered list of services

Finally the status "Starting Application : OK", at the end of the command execution, indicates a successful deployment. So, now you are all set to start-up a browser and fire-up your first "cloud-enabled" web application, at the url <app-name>.cloudfoundry.com. Optionally, you might want to check the one that I've hosted at http://rahulr.cloudfoundry.com

Thats all for now, I hope you liked the post and should you have any issues getting this up and running feel free to reach out to me.

Additional Resources

Following are some links which I found useful -

• A Guide to creating your own services
• The "vmc" command internals

Demystifying Enterprise Architecture with TOGAF - Understanding the Architecture Development Methodology (ADM)

In the previous article, we glanced through Enterprise Architecture as a whole and also discussed the need for an Enterprise Architecture Framework, like TOGAF.

With this article we shall continue exploring TOGAF further, and discusses TOGAF's Architecture Development Methodology (ADM).

The Architecture Development Methodology (ADM)

The Architecture Development Methodology (ADM) provides a proven and repeatable process for developing architectures.

The Scope

The scope of ADM includes or encompasses the below listed activities, which are generally carried out in iterative cycles of continuous architecture definition and realization; thus aiding a controlled transformation of an enterprises in response to business goals and opportunities -

• Establishing an architecture framework
• Developing architecture content
• Transitioning
• Governing the realization of architectures

Implementation Phases

The implementation of ADM could be envisioned across the following phases -

Preliminary Initiation
The Preliminary Initiation phase describes the preparation and initiation activities, required for meeting the business directive for a new enterprise architecture, including the definition of an Organization-Specific Architecture framework and the definition of core principles.

Architecture Vision Setup
The Architecture Vision Setup phase describes the initial phase of an architecture development cycle. It includes information about the following activities -

• Defining the scope
• Identifying the stakeholders
• Creating the architectural vision statement
• Obtaining the initial approval

Business Architecture Evolution
The Business Architecture Evolution phase describes the development of a Business Architecture and its alignment to support an agreed Architecture Vision.

Information Systems Architecture Definition
The Information Systems Architecture Definition phase describes the development of Information Systems Architectures required to support the Architecture Vision. This phase typically involves -

• Identification and development of Data Architectures
• Development of various Application Architectures

Technology Architecture Evolution
The Technology Architecture Evolution phase describes identification and development of the requisite Technology Architecture, with respect to supporting the Information Systems Architecture.

Opportunities & Solutions Identification
The Opportunities & Solutions Identification phase typically involves the following activities -

• Conducting initial implementation planning
• Identifying single or multiple delivery channels for the requisite architectures

Migration Planning
The Migration Planning phase addresses the formulation of a set of detailed sequence of transition architectures with a supporting Implementation and Migration Plan.

Implementation Governance
The Implementation Governance phase involves providing an architectural oversight of the implementation.

Demystifying Enterprise Architecture with TOGAF

Lately I've been studying/reading The Open Group Architecture Framework (TOGAF), and I simply could not resist from appreciating the structural approach that it introduces towards envisioning and capturing Enterprise Architecture. I therefore thought of initiating a series of articles herein, with the sole purpose of sharing, simplifying and promoting the framework especially amongst the architecture aspirants out there who follow and read my blog.

This write-up, which is the first one in the series, intends to provide a brief overview on Enterprise Architecture and then further goes on to illustrate the need for an Enterprise Architecture Framework.

Preface

ISO/IEC 42010: 2007 defines "architecture" as:

"The fundamental organization of a system, embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution."

TOGAF embraces but does not strictly adhere to ISO/IEC 42010: 2007 terminology. In TOGAF, "architecture" has two meanings depending upon the context:

• A formal description of a system, or a detailed plan of the system at component level to guide its implementation
• The structure of components, their inter-relationships, and the principles and guidelines governing their design and evolution over time

Understanding Enterprise Architecture

The Definition of Enterprise

The term "enterprise" refers to any collection of organizations that has a common set of goals. For example, an enterprise could be a government agency, a whole corporation, a division of a corporation, a single department, or a chain of geographically distant organizations linked together by common ownership.

It is important to here note that the term "enterprise" in the context of "enterprise architecture" can be used to denote both [1] an entire enterprise as a collection encompassing all of its information and technology services, processes, and infrastructure — and [2] a specific domain within the enterprise. Nevertheless in both cases, the architecture crosses multiple systems and multiple functional groups within the enterprise.

Defining Enterprise Architecture

Enterprise architecture is a structural approach that optimizes the often fragmented organization wide processes (both manual and automated) into an integrated environment that is responsive to change and enables delivering the enterprise's business strategy.

Domains of Enterprise Architecture

There are four architecture domains that are commonly accepted as subsets of an overall enterprise architecture, all of which TOGAF is designed to support.

• Business Architecture
  The Business Architecture defines the business strategy, governance, organization, and key business processes.
  
  
• Data Architecture
  The Data Architecture describes the structure of an organization’s logical and physical data assets and data management resources.
  
  
• Application/Solution Architecture
  The Application or Solution Architecture provides a blueprint for the individual application systems to be deployed, their interactions, and their relationships to the core business processes of the organization.
  
  
• Technology/Deployment Architecture
  The Technology or Deployment Architecture describes the logical software and hardware capabilities that are required to support the deployment of business, data, and application services. This includes IT infrastructure, middleware, networks, communications, processing, standards, etc.

Understanding TOGAF

The Open Group Architecture Framework (TOGAF) provides the core methods and tools for assisting in the acceptance, production, use, and maintenance of an enterprise architecture. It is based on an iterative process model supported by best practices and a re-usable set of existing architecture assets.

Illustrating the need for TOGAF

Architecture design is a technically complex process, and the design of heterogeneous, multi-vendor architectures is particularly complex. TOGAF plays an important role in helping to de-mystify and de-risk the architecture development process. TOGAF provides a platform for adding value, and enables users to build genuinely open systems-based solutions to address their business issues and needs.

Easy 3D with WebGL and Three.js

Lately a lot has been happening in 3D around the world. From movies to electronic gadgets like televisions, projectors and gaming consoles, everything seems to be heading toward rendering that additional 3rd dimension. A 3D experience in itself so very enticing that one would just falling it love with it; and thus the creative ones amongst us are constantly thinking about various areas wherein it could be applied. Some of the obvious areas of application would be bio-technology, clinical research, engineering drawings and entertainment media; but meanwhile some creative minds amongst us have gone ahead thinking differently and even made simple software applications that we use in our everyday lives, to render a user interface with 3D. For instance, a few months ago I came across a program called 3DNA Desktop which improves the way we work with Windows and the Web. With 3DNA Desktop one can choose from different 3D worlds to explore and customize the same, to create an immersive and entertaining 3D desktop usage experience.

Screenshot of 3DNA Desktop

As a developer, I've always thought about how these awesome experiences are created. For a while I did try to play around with Adobe Flash, but to by candor found it too exhaustive to understand in a short time; therefore all the time continued to wonder, if there is something simpler, and above all free of cost, which could yield the same results or even better. Until recently, that I came across WebGL.

WebGL is a cross-platform, royalty-free web standard for a low-level 3D graphics API based on OpenGL ES 2.0, exposed through the HTML-5's Canvas element in a Document Object Model (DOM). So, what it essentially means is - now you can have Adobe Flash like experiences created within standard HTML using the Canvas element. You must have seen so many fancy user experiences built atop Adobe Flash, so obviously are wondering how exactly does WebGL compare against Adobe Flash? Therefore, to augment your curiosity here is an example that I built to demonstrate the capabilities WebGL, which shows a real-time day light view of our planet. Just try interacting with the globe using your mouse or touch-pad to get the real feel of the thing. Meanwhile, a quick note about the support – you need to make sure that your browser supports WebGL content rendering for running this example. Ideally one would need Google Chrome, Firefox 4+ or IE 9+ to run this one at its best.

Screenshot of the WebGL and three.js example

A software programming enthusiast would furthermore be amazed to know that this is entirely done using standard Javascript and Mr.Doob's awesome script library called three.js, in just over 80 lines of code (LoC); you might want to right-click and choose to view the source on the example page to have a look at the code for yourselves. Three.js is a framework library developed by Mr. Doob that skillfully abstracts away the complex mathematics involved in implementing 3D, and provides simple APIs with which you can create cameras, objects, lights, materials and more; with a choice of renderers, which means you can decide if you want your scene to be drawn using HTML-5's canvas, WebGL or SVG within a standard browser.

Now, should this little article and the example inspire you to get your hands dirty with some code. The best place to start would be the three.js sources and bundled examples. You may download the latest release of three.js from GitHub (https://github.com/mrdoob/three.js) and get started with simple examples in absolutely no time. :)

Wishing you happy exploration of WebGL and three.js...

Have fun!!!

Table2Visualization : Rendering Google Visualization Charts with HTML Tables

With the advent of the analytics age, one would find applications spring-up every month that lets you track, monitor, and analyze just about anything, from human habits to system behaviors. Such applications generally have a standard implementation pattern wherein they primarily collect, upload, process, and display data. Now, somebody trying to develop one such application would obviously face challenges, of different kinds, at each of these points depending upon the domain of applicability. However, the challenges in displaying the data, in a way that it is understandable to non-professionals, are the same for all of them.

Oftentimes, we rely upon statistical charts and graphs to render such data. These days one would find a plenty of charting libraries that make this possible easily (and some of them without any cost), for example Visualize, MooCharts, ProtoChart etc. One such player in the world of data visualization is Google itself.

Google Visualization is a set of JavaScript libraries that offer a wide range of data representation options with an easily pluggable API. It enables a developer to render statistical charts and graphs from a wide range of data-sources. However, as a developer, I always felt that a very simple and basic data source was always missing in that exhaustive list of data sources; and that is HTML tables. My searched across the web for an answer, either resulted in vein or a complex process with involved importing the data in a Google Docs spreadsheet and then rendering a chart from there. In simple words, there was nothing “Simple and Sweet”.

So, here I had a problem in-hand and decided to solve it for myself and all my fellow developers around the world. The implementation is now available on Google Code at the following location, with a sample to help you jump-start

http://code.google.com/p/table2visualization

GMail does not understand dots...

I recently discovered some little-known ways to use a GMail address that can give you greater control over your inbox and save you some time. Actually, when you choose a Gmail address, you actually get more than just "yourusername@gmail.com"; here are two different ways you can modify your Gmail address and still continue getting your mails:

• Append a plus ("+") sign and any combination of words or numbers after your email address. For example, if your name was yourusername@gmail.com, you could send mail to yourusername+friends@gmail.com or yourusername+mailinglists@gmail.com.
• Insert one or several dots (".") anywhere in your email address. Gmail doesn't recognize periods as characters in addresses -- Google mail just ignores them. For example, you could tell people your address was hikingfan@gmail.com, hiking.fan@gmail.com or hi.kin.g.fan@gmail.com.
  

For me, the real value in being able to manipulate your email address is that it makes it really easy to filter on those variants. For example you could use yourusername+bank@gmail.com when you sign up for online banking and then set up a filter to automatically star, archive or label emails addressed to yourusername+bank. You can also use this when you register for a service and think they might share your information. For example, I added "+yahoogrp" when I subscribed to some yahoo groups once, and now when I see emails from other groups to that address, I know how they got it and can apply a filter to auto-delete the mails.

Simple Serialization

The Simple XML serialization framework has released version 1.2. Simple is a serialization framework for Java that intends to provide an XML serialization framework that requires no configuration or mappings to serialize objects bi-directionally; i.e. to and from standard XML. Below is a list of some of the capabilities of the framework.

Simple framework with powerful capabilities
The framework used to provide XML serialization is simple to use and revolves around several annotations an a single persister object used to read and write objects to and from XML. Both serialization and deserialization are fully supported.

It requires absolutely no configuration
Unlike many of the XML frameworks for Java there are no mappings or configuration required to serialize objects regardless of its complexity, not even code to establish mappings. Everything is done at runtime, only annotations are required to perform the object to XML structure. The XML schema is represented using field and method annotations.

Converts to and from human editable XML
A primary goal for the framework is that the XML data used to deserialize and serialize objects is human readable. All XML elements and attributes take a simple structure that can be easily created with a text editor. Also the serialization format is compatible with the C# XML serialization format.

Contains an XML templating system
As part of the deserialization process template markers within the XML elements and attributes can be replaced with variables. This allows the system to be easily adapted for use as a configuration system with dynamic value substitution. Stock template filers include OS environment variable substitution, system properties, as well as an API that enables custom filters to be introduced.

Provides persister callback methods
During serialization and deserialization an object can recieve persister callbacks, to validate and prepare data during the process. Each callback allows the object to manipulate the persistance session, which allows values to be passed in to and out of the templating engine, and allows arbitrary attributes to be passed between the persisable objects.

Allows XML to drive composition
The serialization and deserialization process is controlled by an exensible strategy API, which allows objects to be created based on custom XML attributes. This allows interception of the object creation in order to delegate instantiation to object factories.

Fully self contained
For Java 6 the framework has no external dependancies and is completely self contained. For Java 5 two external JAR files are required, the StAX API and the StAX implementation, both are provided with the download.

Open source
Released under the LGPL and so can be fully integrated or adapted for both commercial and open source projects.

The Simple Home : http://simple.sourceforge.net

Is Java String really immutable...?

In many texts String is cited as the ultimate benchmark of Java's various immutable classes. Well, I'm sure you'd have to think the other way once you have read this article.

To start with, let's get back to the books and read the definition of immutability. The Wikipedia defines it as follows -

'In object-oriented and functional programming, an immutable object is an object whose state cannot be modified after it is created.'

I personally find this definition good as it mentions that an immutable instance's state should not be allowed to be modified after it's construction.

Now keeping this in the back of our minds, let's decompile Java's standard String implementation and peep into the hashCode() method -

public int hashCode() {
int h = hash;
if (h == 0) {
int off = offset;
char val[] = value;
int len = count;
for (int i = 0; i < len; i++) {
h = 31*h + val[off++];
}
hash = h;
}
return h;
}

A detailed look at the above code reveals this to be a classic implementation of the 'lazy evaluation' pattern; i.e. the String class instead of computing the hash value [1] during an instance's construction or [2] computing it each time the hashCode() method is called; computes it once, i.e. on the first call to hashCode(), and saves the computed value in the hash attribute.

Now, here is a test to prove the above quoted statement -

String string = "MyDearString";

Field field = String.class.getDeclaredField("hash");
field.setAccessible(true);

System.out.println("Before: " + field.getInt(string));

string.hashCode();

System.out.println("After: " + field.getInt(string ));

Anxious to know what the output would looks like? Here it is...

Before: 0
After: -1554135584

So this infers that for a given String instance which we create, but for which we never call hashCode(), the private hash field will remain to be zero. It is only changed when we call hashCode().

Well, don't you find this in contrast with the basic definition of an object's immutability?

One might argue that we can not observe a String instance in a different state without a reflective read of String's hash field and because the call to retrieve the state actually modifies it; if we can't observe it changing.

Well, if one doesn't observe a change does that mean it never happened...?

This reminds me about the other day when i scratched my new car, from below, against an irregular speed-breaker and my friend Manish said that I should not be worried about that because if a scratch is not visible its never been there. Well, is it really never there...?

However it seems like String is not immutable. Although it seems safe, ignoring reflective access, it definitely looks incorrect.

Post me your views around this...

Schematron - validating web-forms and java objects

Whilst developing web applications, more than often we are confronted with the need to validate user inputs. On the client-side (browsers) we generally do that with scripting languages; and for those users who have perhaps deliberately disabled their browser script support; we perform the same set validation on the server side, and this time we write some lines-of-code to extract the user input from the HTTP request and validate the same. Well, although ironic; yet it seems to be a de-facto that - we have to specify the validation rules separately on the client and server side, for eventually validating something that is semantically same.

A technical analysis into the core reason reveals that – although the semantics of the herein discussed "thing" (i.e. the user input) are the same, yet the representations of the semantic on the client and server side are different; which enforces us to implement the extraction and validation logic differently. So going by that theory - if we can represent the semantics of the user input in a single unified format, we can potentially have a single definition and implementation of the validation rules too.

So, what should our choice of a single unified data representation format? Well, there are a plethora of data representation formats out there; and you may be attracted to choose one that best suits your industry. However, as this article intends to project Schematron as a common validation solution across the client and server; I limited my choice to XML with intent to convey the concept with simplicity.

Dear JSON lovers, here is an excellent library by Stefan Goessner, which provides helper function for converting JSON to XML and visa-versa; So, that means - Schematron can be potentially used to validate JSON representations too.

So, let’s apply the concept to practice -

Before we apply Schematron to validate the user inputs on the client side; one must first convert the web form input to XML. The following archive illustrates an approach for converting a web form to XML and validating the same against a standard Schematron based schema –

• schematron-sample.rar

Once we have validated the user input on the client, we forward the same to the server; wherein we should re-use our validation rules and this time execute the same on the server side. For many of you out there this is perhaps a no-brainer. Nevertheless, we will discuss this aspect in similar details with the next post in this series.

Until then wish you a merry Christmas and a safe holiday season.

Introducing Schematron

Schematron is a rule-based validation language for making assertions about patterns found in XML documents. It is a simple language which is based very much on XML itself and uses standard XPath to specify the assertion statements. The Schematron definations (a.k.a Schema) can be processed with standard XSL templates; which makes Schematron applicable is a variety of scenarios.

Although a Schematron defination is referred as a Schema, but one must understand that Schematron differs in the basic concept from other schema languages; i.e. it is not based on grammars but on finding tree patterns in the parsed document. This approach allows many kinds of structures to be represented which could be difficult with grammar-based schema languages. For instance - imagine how would a typical schema be, for the following XML document -

<?xml version="1.0" encoding="UTF-8"?>
<instance>
####<person>
########<fname/>
########<lname/>
####</person>
</instance>

Guess its a no-brainer! It would be something like this -

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
####<xs:element name="instance">
########<xs:complexType>
############<xs:sequence>
################<xs:element name="person">
####################<xs:complexType>
########################<xs:sequence>
############################<xs:element name="fname" type="xs:string"/>
############################<xs:element name="lname" type="xs:string"/>
########################</xs:sequence>
####################</xs:complexType>
################</xs:element>
############</xs:sequence>
########</xs:complexType>
####</xs:element>
</xs:schema>

You must be already wondering about how different would the same schema look like in the "Schematron" world, right? Well, here is the answer -

<?xml version="1.0" encoding="UTF-8"?>
<schema xmlns="http://www.ascc.net/xml/schematron" >
####<pattern name="assert validity">
########<rule context="instance">
############<assert test="person">person element is missing.</assert>
########</rule>
########<rule context="person">
############<assert test="fname">fname element is missing.</assert>
############<assert test="lname">lname element is missing.</assert>
########</rule>
####</pattern>
</schema>

Now, isn't that a much better and understandable version of a schema?

A closer look at the above document would reveal how Schematron differs in the fundamentals of validating a document; the crux herein is not to define the structure of the document (that is what the traditional schema types do), but is to assert the structure. Imagine it to be something like JUnit or NUnit for the XML world; wherein one puts assert statements to check the validity of an object's state. And just like it happens in the JUnit or NUnit world; herein with Schematron also, one can have custom messages for assert conditions.

The Schematron can render custom messages in two cases, v.i.z. -

1. "Report" the presence of a pattern
2. and "Assert" the absence of a pattern

The following Schematron document illustrates the usage of the above discussed features -


<?xml version="1.0" encoding="UTF-8"?>
<schema xmlns="http://www.ascc.net/xml/schematron" >

####<!-- Render messages in case the elements are found. -->
####<pattern name="report validity">
########<rule context="instance">
############<report test="person">person element is present.</assert>
########</rule>
########<rule context="person">
############<report test="fname">fname element is present.</assert>
############<report test="lname">lname element is present.</assert>
########</rule>
####</pattern>

####<!-- Render messages in case the elements are not found. -->
####<pattern name="assert validity">
########<rule context="instance">
############<assert test="person">person element is missing.</assert>
########</rule>
########<rule context="person">
############<assert test="fname">fname element is missing.</assert>
############<assert test="lname">lname element is missing.</assert>
########</rule>
####</pattern>

</schema>


Now that we know how Schematron based schemas look like; the next obvious question is - "How to validate a document against a Schematron schema...?" Well, I already addressed that question in the first paragraph itself, with the following statement -

"The Schematron definations (a.k.a Schema) can be processed with standard XSL templates."

Following statements illustrate the basic processing involved in validating a document with Schematron -

xslt -stylesheet schematron-message.xsl SchematronRules.xml > compiled-SchematronRules.xsl
xslt -stylesheet compiled-SchematronRules.xsl TestData.xml

Now if that looks a little complex; here is a simple Schematron document validator (implemented in Java) which I developed to ease this complexity -

- Schematronize.java

The above listed validator requires the following XSL templates to be locally available -

- skeleton1-5.xsl
- schematron-message.xsl

Thats all for now... I’d leave it here for you to play. Hope you enjoyed reading this article. I will look forward to reading your feedbacks and implementation details around Schematron, so please do take a moment and drop in a note.

Adieu.

Some References -

- The Schematron Website
- The Academia Sinica Computing Centre's Schematron Home Page

Scarlet – A Scala DSL for web applications

Work in Progress The Scarlet project is currently under development. Please keep an eye on the sourceforge project site (sourceforge.net/projects/scarlets) or contact personally for updates.

Introduction to Domain Specific Language (DSL)

Domain-specific languages have lately become a hot topic and a lot is being spoken and written about them all over the internet; much of the buzz around functional languages is their applicability to (once again) put the power and control of an application in exactly the right place where it belongs, i.e. in the hands of its users.

By defining a simple language that the application users can understand, feel comfortable to communicate and use, programmers can effectively reduce their involvement in the never-ending cycle of UI requests and enhancements and can let users create scripts and other tools that allow them to create newer behavior in the applications they use. A perfect example of a wildly successful DSL is the Microsoft® Office Excel "language" used to express the various calculations and contents of the cells in the spreadsheets.

Well, some might even go far and suggest that SQL itself is a DSL, this time a language aimed specifically at interacting with the relational database (Imagine if programmers had to fetch data out of Oracle via traditional API read()/write() calls — Eshhhh! Now that’s a nightmare).

What is Scarlet…?

To be quick to the point: Scarlets is an attempt to create “platform-independent”, “compilable” and “executable” pseudo-codes.

Pseudo-codes are a kind of structured english for describing algorithms; which enables the designer to focus on the logic of the algorithm without being distracted by details of language syntax. The intent is to describe the entire logic of the algorithm with finer details so that implementation becomes a rote mechanical task of translating line by line into source code.

We have known pseudo-codes probably since the beginning of times at software engineering. And although we understand that pseudo-codes are capable of describing the entire logic of an algorithm to such an extent that implementation remains to be a simple mechanical task of translating lines of plain english into source code; yet, this so-called simple mechanical task is till date performed with human efforts.

The Scarlets project intends minimize this human intervention by defining a Scala based domain-specific language which is very much like structured english and is a narrative for someone who knows the requirements (problem domain); so that he/she can write a pseudo code and see it run with zero or minimal involvement of a programmer who knows the programming language (implementation domain).

A Sample Scarlet

The following is a sample implementation of Scarlet –

AccountApp.scala :

val userName = theParameter named "user" in request if (userName equals "nothing") { ~~~this renders "AccountUserForm.jsp" } else { ~~~theAttribute named "LoggedInUser" in session is userName ~~~val accountHolder = new AccountHolder("B123456", "Rahul", "Roy") ~~~val account = new Account("SA-001", accountHolder, 100.00) ~~~theAttribute named "AccountObject" in session is account ~~~this renders "AccountDisplay.jsp" }

Do we need a description for the above implementation…? Well, I’d rather leave it for you to understand. Meanwhile here is how the AccountUserForm and AccountDisplay Java server-pages (JSP) look like.

AccountUserForm.jsp :

<html> ~~~<head> ~~~~~~<title>Scarlet Sample</title> ~~~</head> ~~~<body> ~~~~~~<form action="AccountApp" method="GET"> ~~~~~~~~~Username : <input type="text" name="user" /> ~~~~~~~~~<input type=submit value="Login" /> ~~~~~~</form> ~~~</body> </html>

AccountDisplay.jsp :

<% ~~~~~~account = session.getAttribute("AccountObject"); ~~~~~~LoggedInUser = session.getAttribute("LoggedInUser").toString(); %> <html> ~~~<head> ~~~<head> ~~~~~~<title>Scarlet Sample</title> ~~~</head> ~~~<body> ~~~~~~<%=LoggedInUser%>'s Account : <%=account%> ~~~</body> </html>

I hope you found Scarlets to be interesting; anyways please do feel free to share your thoughts and comments here so that I can better it with your cooperation.

Web4J : An Interesting framework

Its been quite a while since my last post here. I understand that my sudden disappearing act must have annoyed a many of my regular visitors; especially the ones who are expecting the second part of the XRX tutorial.


Whilst you wait for the concluding part of the XRX article, I thought of sharing somethings that i found to be interesting -


As Java Web application frameworks have become more powerful and flexible, they've also become more complex. John O'Hanley's WEB4J framework in many ways flies in the face of this trend: it offers few customization options, but is easy to learn and work with. Read on to learn about the unusual (and somewhat contrarian) development principles behind WEB4J. You'll also have the opportunity to test out the framework by building a feature for a sample Web application... [more]

Part I : XRX Architecture - Unleashed

During my tenure at Software AG, I was introduced to the power of XML. To my candor, i was very impressed to see the wide range of problems that XML promised to solve. Especially, to mention the performance power-punch that XML databases unleash when combined with XQuery and REST.

A couple of years later, I moved into the consultancy world. Being back in the consultancy world and seeing business systems with my renewed XML-ized eyes (all credits to Software AG Tamino API and tools laboratory), I was highly dismayed to find that business oriented systems seldom use the real power of XML. The usage of XML herein has been more-so confined to defining application configurations and bare minimum transformation to either render interface elements or reports. I agree, there have been a few exceptions to this statement too.

After spending quite some years revolting the general underutilization of XML; apparently I found my preferred architecture – i.e. XRX (XForms/REST/XQuery).

What is so unique about XRX architecture...? Generally, web application architectures accept inputs from HTML forms, which provide the user inputs in flat key-value pairs, thereafter these data structures are converted to middle tier objects such as Java or .Net and then transformed into tabular data streams so they can be stored in relational databases. Once in the relational database, the data must then be re-serialized by doing SELECT statements, converted into objects and the objects then converted back into HTML forms. This is four-step translation architecture. In contrast to the above discussed conventional approach, XRX uses the zero-translation architecture. Zero translation implies that XML is stored in the web client, transmitted to a middle tier validation rules engine in XML and then stored in its entirety in an XML database. The storage in a single XML object is also known as a zero-shredding process since the data files are not separated into Third Normal Form (3NF) data structures. The key here is to break the myth and realize that 3NF shredding does not add any business value to the system.

Over the years, I have preached XForms as a very solid architectural composite; as it offers an order-of-magnitude improvement over other web front-end development architectures. Now, XForms when combined with the traditional XML performance power-punch i.e. REST and XQuery, defines a radical architecture that works wonders.

I'm sure you must be already wondering: how to build this magical combination of XForms - REST - XQuery (XRX)? The following architecture view depicts the approach at an abstract, to facilitate and initiate your quest -


Following are some of the distinct advantages rendered by XRX architecture -

• A web development architecture with a 10x productivity improvement over traditional Javascript/Object Oriented/RDBMS methods

• A development architecture based on international standards that is designed to minimize the probability of vendor-locking

• An architecture that gives a rich user experience without creating mountains of spaghetti procedural code on either the client or the server

• A system that leverages the REST architecture to take advantage of high-performance and simple interfaces using web standards

• Portability on both the client and the server using a variety of forms players and XQuery databases

• The option of avoiding costly shredding (and reconstitution) of complex XML documents into RDBMS tables

• A community of standards/tools and a "complete solution" ecosystem that can give you a proven ROI on IT investment

The forthcoming post in this series, i.e. "Part II : XRX Architecture - Illustrated", will walk you through an elaborate tutorial on building a sample application that implements the XRX architecture.

Adieu for now...