Yesterday,  Following Foursquare’s announcement of their decision to switch to OpenStreetMap,  I had a long discussion on Twitter about the merits of open and “closed” map systems.

I am a big fan of Google maps. I like using the website both from my pc and my iPhone.
The only interaction I have had with OpenStreetMap has been through the Skobbler navigation app for the iPhone. I had even paid to get the offline maps. My experience was a thoroughly disappointing one. Partly because of the maps not being very accurate but mainly because of the frustration with the application. After spending the money to get the offline maps the app kept telling me to connect to the internet to navigate – which entirely defeats the purpose of paying for the offline maps.

Normally I’m a big supporter of openness. However, in this particular care, I find myself thinking that a “closed” system is probably better. I am using the word close losely here as what I mean is not a system unaccessible to developers but rather a system maintained and paid for by a private corporation.
My reasons are the following:

1. “Mapping” is an expensive business – A private corporation fully invested in this business is likely to spend considerable sums of money making sure its maps are accurate and reliable, globally. Globally here is a keyword for me. OpenStreetMap is not likely to get very accurate data in areas which are not easily reachable or don’t have easy access to the internet (and that’s where I like to travel and use my maps)
2. People are lazy – We see it all the time. Wikipedia (which I love) is updated only by a handful of users  while being consumed by billions – Creating content on Wikipedia requires relatively little (physical) effort, on the other hand creating content for OpenStreetMaps includes going out, driving and therefore spending money. Granted there will always be a handful of people eager to create and share this information, but is that enough to keep a world atlas up to date?
3. Being popular online is expensive – OpenStreetMap to keep growing in popularity will have to raise an ever larger amount of money to keep its hosting facilities up to scratch to deal with the demand. Of course a business has the same issue, but they make money out of it and this money is invested in keeping the facilities up-to-date.

I love the idea of OpenStreetMap being an open data source and API rather than a product (like Google maps). I love how customisable it is and how easy it is for developers to integrate. Nevertheless I am not sure it will be able to stay accurate in the long run. Yesterday Skobbler’s creator claimed that OpenStreetMap is already better than Google maps in Germany. Just out of curiosity I went to google maps, picked a random small town in the middle of Germany then went to have a look at it on OpenStreetMap – quite a few roads where missing (especially dead end roads) – Urgh.

What do you think?

Bookmark It

Over the past couple of weeks I’ve been interviewing candidates for a Java developer position.
I can’t find words to describe my feelings now so I’ll recycle something a friend of mine, who’s also hiring, said:

It’s next to impossible to hire anyone useful now.

My complaint with most of the applicants is that they behave – there’s no other way to say this – machines.

I did what I did because I was told to it. Never really thought about it.

This infuriates me beyond belief.

So here a few tips for your (and my) next interview:

1. Don’t make up stuff on your CV. Sounds obvious but you’ll be surprised by how many people list technologies they have barely used on their CV and when asked about it don’t have a clue. You have no idea how bad this looks. (The red bullshitter light starts flashing in my head instantly)
2. Show that you have passion for what you do. Being a Sun certified Java developer is all well and good but when I ask you what you have been looking into recently, or if you’ve worked on something on your own make sure to have an answer; or, if you don’t, make sure you have a good excuse.
3. If you have 1000 years of experience with something I expect you to have some thoughts about it. Once again, Sun certified Java developer, if I ask you where do you see java going in the next 5 years have an answer! There must be something you think needs improving in Java or that you’d do differently. I’ll go wherever Sun takes me is just not an acceptable answer.
4. Have an opinion! For God’s sake have one. I’ll even settle for half an opinion. If not about a technology you have used for 10 years at least about the coffee I just bought you during the interview. There’s no point in talking for an hour with somebody who has nothing to tell me other than “I’ll do everything you want, I always did.”
5. Know what the company you have applied for does! This sounds stupid but I have called people after 1 week of their application and after 10 minutes with them over the phone I found out that they have no idea what the company they applied at was doing. That’s an instant goodbye!
6. Fill the gaps in your CV, and proof-read it before sending it out for goodness sake. It’s fine that you have taken one year off to travel the world. If your CV does not list anything for the whole of 2009 I’m going to ask you about it. Give me a sincere instant answer and there’ll be no problem. Give me random excuses and it’s goodbye. As for proof-reading I had somebody applying who had listed “quick apprehension” as a skill. I’m not into hiring failed super-heroes. (I assume they ment quick comprehension)
7. Apply for jobs you are genuinely interested in. You are not likely to get a job if your interviewer notices you don’t give a toss about what you do and who you do it for, and it shows. So Unless you are applying for a position as a nut-packer make sure you’re going for a job you’d actually enjoy doing and are interested in.

Needless to say my search continues.

After reviewing more than 60 CVs I have almost completely given up on websites such as Monster and CWJobs and I’m going to go entirely through connections now.

Bookmark It

The new version of MapKit in iOS 4 supports polylines, finally. MKPolyline to be precise.

I have been working with the Google Maps API for a while and now I’m learning Objective-C and writing an iPhone app ro Route.ly

Since the best way to send around the net the data to draw a polyline on a map is in encoded format I decided to write a small Objective-C function to decode the encoded string. Since I couldn’t find this anywhere on the net I decided to publish the code here.

You’ll notice that at the beginning of the function I replace “\\\\” with “\\”. This is because I use the encoded data both in web pages with javascript and my iPhone app. In javascript “\\” would be considered an escape and would therefore break my array of points. This is not the case in Objective-C so I switch back to the normal “\\”.

Waypoints is a simple model class I’ve written with two NSString properties: lat and lng.

Bookmark It

Brilliant video posted by Fred Wilson on his blog.

If you can spare half an hour definitely watch it!

Bookmark It

I love calculating driving directions on Google Maps and then drag the blue line marking my directions to change the route. Everything is updated automatically on the page and the directions are re-calculated to go through the new point I defined.

I’ve been playing around with the Google Maps APIs recently and imagine my disappointment when I found out that Google does not allow directions calculated through the APIs to be dragged around.

Not put off by this I decided to try and replicate the directions-dragging myself. How hard can it be?
As it turns out. Very, at least if you want to make it look as smooth as Google’s own solution.

When generating directions Google Maps adds a GPolyline element overlay to your map. You can set the returned line to be editable but this makes an awful lot of vertices appear on it, which makes reading your directions quite hard. Even so, once you dragged one of this vertices around you are not editing the route but just changing the shape of the line.

Mine is not a complete solution and I’m interested in feedback and ideas on how to improve it.

First off calculate your directions:

This will calculate your driving directions and plot a GPolyline on your map. The line is easily accessible in the GDirections object once the directions are calculated. To intercept this I have decided to use the addoverlay event on the GMap object. This event is triggered every time something is plotted over the map (says on the tin).

At this point we can ask the APIs to make the GPolyline editable. This will make the vertices appear on the line and make then draggable. As I said before this only changes the shape of the line and doesn’t actually affect your directions object. Luckily the GPolyline comes with a nifty event called lineupdated.
This is triggered once the user has finished dragging a vertex. By intercepting this we can look through the vertices and know what’s been changed on the line and where the vertex has been moved to.
In order to do this we must also know the previous position of the vertices (latitude and longitude) to be able to compare the old and the new “edited” line.
Another challenge is the fact that the GDirections object can accept only so many waypoints (25 if I’m not wrong). Which means we’ll have to add only the vertex that has changed to the directions and not all of them.

This works quite well but does not look as smooth as Google’s solution.
The problem is that while you are dragging a vertex only that bit of the GPolyline moves and the rest stays in its original position. which makes the shape of your directions quite awkward while you are dragging. Unforunately the GPolyline does not come with a “startdragging” event, otherwise we could just recalculate the route every few seconds while the vertex is being dragged.

This is not the most elegant of solutions but it does the job.

Bookmark It

What do you need to know? Java Enterprise Edition. That’s the simple answer.
If you read between the lines what I’m really saying is: “You better make sure you really don’t give a flying toss about technology and don’t care about how frustrating what you spend most of your day doing is.”

A few days ago I was talking with a friend who’s in the process of interviewing with a few big financial institutions in London. The most common questions according to him. Struts, JEE.
Spring was also mentioned by I don’t have much against Spring. If used properly it really does keep your code tidy and avoids duplication.

You see a bank’s idea of a high performance reliable system is a monolithic Java application running a million-billion threads at the same time on a huge machine. Maybe that’s a bit too harsh. Sometimes they do distribute. They just add 15 layers shooting messages at each other in between the 2 they actually need.
Then they wonder why the best talent winds up working for Google, and their budget to deliver 1 application is twice what Facebook spent to build its entire social network.

What I have a problem with is the architects who design these systems. And the clearly-drunken bank managers who hired them.

The most common profile for architects in investment banks (and I met quite a few) is 50-something, stopped following/reading about technology in 1995 and prior to being employed by Mr Mighty WankBank (Thanks City Boy for the name) was VP of over-engineering at the Apache Software Foundation (focused on Struts).
First thing in their TODO list is hire some bright spark straight out of university who has a keen interest in over-engineering and never had to deliver anything in less than a geological era.
That’s when an overpaid team of consultants comes in to “sort-it-out”, at which point the bank loses control over its source-code and none of their developers can understand it any more.

Oh, and then there’s the process, oh the bureaucracy.

Fortunately I also know that in London there are a few sunny spots (techonology-wise) such as Hedge funds who started using Erlang (for example) to distribute their heavy computations over a network.

Now don’t get me wrong I have nothing against JEE per-se. Some of the libraries are actually very useful and improve a developer’s quality of life quite a lot. Just think about JTA.
What is wrong is the way they are used by people who never left the nineties.

Bookmark It

While skimming through my RSS feed I stumbled upon an interesting article on Ajaxian about server side JavaScript programming.

I really, really enjoy writing JavaScript code so I decided to read through the article and take a look at the node.js library linked there.

Node is a framework to build server-side event-driven JavaScript applications. Developed in C++ on top of Google’s V8 JavaScript engine and accompanied by a set of JavaScript libraries Node seems to make building distributed (over a network), fast applications a piece of cake even for inexperienced developers.
Event-driven here really is the keyword because it represent a big change in the way applications are built (and architected in your brain).

Node is similar in design to and influenced by systems like Ruby’s Event Machine or Python’s Twisted. Node takes the event model a bit further—it presents the event loop as a language construct instead of as a library. In other systems there is always a blocking call to start the event-loop. Typically one defines behavior through callbacks at the beginning of a script and at the end starts a server through a blocking call like

EventMachine::run()

. In Node there is no such start-the-event-loop call. Node simply enters the event loop after executing the input script.

The example below (taken from Node’s website) will make everything clear… hopefully.

Can you see the beauty?!

I have only one worry. This is an open-source effort. The community behind it on Google groups is just 181 members strong (so far). What if node.js suddenly stops being the cool thing and the community disappears.

As much as I love writing JavaScript and I can really see the value in what they are building I’ll still wait until there are 100 Google Groups for node.js and a trillion members in each before using it in anything close to a production system.

Having said that I’m going back to writing silly node.js apps now. Well done to all the developers involved in the project and keep it up!

Bookmark It

When I started developing TweetSentiment I decided that the interface should have as little text as possible. Most of the information I was interested in could be displayed graphically, with a chart.

So I looked at all the options available for chart generation.

1. Backend code to generate a static image (JFreeChart or PHP)
2. Flash object to draw a chart retrieving the data from a URL
3. Draw charts in JavaScript directly on the client’s browser

I’m not a huge fan of option one. Primarily because TweetSentiment is hosted on a tiny linux box which would not be able to handle the load for the traffic the site gets, also because it’s a static image – it’s just not very funky – no interaction possible.

Option two would certainly create spectacular looking charts but I have almost no experience with flash and I wasn’t about to start learning a new language/technology. Plus I’m not into browser plugins if I can avoid them.

JavaScript is a language I’m familiar with and I remember seeing some cool-looking charts generated with Dojo. Unfortunately for TweetSentiment I have used jquery since the most important thing for me there was DOM manipulation (and jquery is just better for that).
I then started shopping around for jQuery plugins to generate charts. There are a few around but none of them impressed me. They just weren’t as good looking as I’d hoped nor they were interactive.

By coincidence I stumbled on RaphaelJS. A JavaScript library to draw Scalable Vector Graphics directly from JavaScript based on jQuery. I tested the samples on the website with a few browsers and I was happy to discover it worked just fine with all of them.

Scalable Vector Graphics (SVG) is a family of specifications of an XML-based file format for describing two-dimensional vector graphics, both static and dynamic (i.e. interactive or animated).

The SVG specification is an open standard that has been under development by the World Wide Web Consortium (W3C) since 1999.

I also discovered that there is a charting library built on top of RaphaelJS, which is exactly what I was looking for. However, being a geek, I decided to go ahead and try to develop something on my own. You know, just for kicks.

As I delved deeper into RaphaelJS I found the library to be incredibly powerful. It’s a shame that the documentation provided on the website lets it down a bit.
The most powerful bit is the ability to extend objects and attach new functions to them. Something scarcely mentioned in the available documentation.

For example if you need to use curved lines (paths as SVG calls them) you can just defined a default function you can then call from your code simply by adding it to the el “object” in RaphaelJS

Another very useful function I found in one of their samples is the andClose() This is used to close a polygon you have started drawing with paths. No matter where you got to it will reconnect to the initial point.

This can then be used this way using chaining.

I’m still developing the chart library I used in TweetSentiment and I’m planning to publish it here with some documentation under MIT licence.

Bookmark It

I have just stumbled on an article on Slashdot about Google Go. A programming language which was initially developed internally and it’s now been open-sourced.

Over the past few months I have been working with Erlang on multi-process distributed systems. Concurrency in Go is based on the same model Erlang uses: Hoare’s Communicating Sequential Processes, or CSP.

Go’s concurrency primitives derive from a different part of the family tree whose main contribution is the powerful notion of channels as first class objects.

The thing I appreciate the most of Erlang is how easy it is to distribute – even over a network – and monitor processes. After skimming through Go’s documentation I couldn’t help but notice the lack of integrated network-distribution of processes or goroutines, as they are called.

This for me means that while Go, exactly like Erlang and Occam, simplifies the approach to multi-threaded programming (at least from a developer’s point of view), it doesn’t go all the way and leaves developers to deal with distribution of processes over a network on their own.

This is a perfectly acceptable design choice in the name of flexibility, however it creates a whole new bunch of use cases for the developers to deal with, such as sending channels/channels-data over the network).

What I don’t like of Erlang is its handling of strings, which makes it unsuitable for a number of application that could benefit greatly from its multi-processing capabilities. Very few people would recommend Erlang as a high performance string manipulation language. To quote Sendmail’s case study in implementing their Sendmail load balancing “Client Daemon” in Erlang:

…But Erlang’s treatment of strings as lists of bytes is as elegant as it is impractical. The factor-of-eight storage expansion of text, as well as the copying that occurs during message-passing, cripples Erlang for all but the most performance-insensitive text-processing applications.

Go totally wins on this.

If you now asked me to pick one to use. I’d still go for Erlang.
Functional languages and CSP are changing the way big companies approach the development of business critical applications. In the last year alone I have seen the number of positions advertised for Erlang developers in the financial sector multiply.
Go, if it continues on this path definitely has a chance of making it there. Open-sourcing the language is the right decision to give it the exposure to real-life scenarios that tend to accelerate the growth and adoption rate.

Bookmark It

Perhaps I’m bored. Maybe I’m just too single.
Anyway. A couple of days ago I was looking at StockTwits again and saw a considerable amount of activity. There and then I decided that it would have been pretty cool to check whether there was any correlation between the activity about a stock on Twitter and the actual trading volume on the market.

I set off to build a small system to do just that. A couple of groovy scripts to collect the data and save it plus some JavaScript and HTML to display it.

So here I am a couple of days later talking about TweetSentiment.

Collecting the data I needed was pretty trivial thanks to the fact that StockTwits asks all its members to tweet the symbols they are trading preceded by a dollar sign ($).
My first task was to build a list of securities I was interested in. I decided to go to Covestor.com and pick the most traded securities list.  Once that list was ready all I had to do was call the Twitter Search APIs for each stock and store the results.
Groovy made these tasks incredibly simple.

I’m not doing much with the tweets. At the moment I just count them and look for “buy” and “sell”. I’m looking into smarter ways of analysing the text and look for positive/negative opinions. If you have any suggestion on this please do leave a comment.

Next in my TODO list was getting some market data. For that I’m using Yahoo finance asking for previous closing price and average volume for each security. Once again, piece of cake with Groovy. Same thing as before just different URL.
Yahoo finance actually has quite a simple interface to let you grab the data. Check out this page on Cliff’s Notes.

Now I had all the data I needed. Only thing left was to build some sort of interface to look at it. I wanted to display all of the data on a chart to be able to make sense of it as quickly as possible.
My server (i.e. the machine running this blog) with its puny power could have never handled the traffic I get while generating charts. Therefore what I decided to do is to export all the data in text files (JSON format) and do all of the chart generation with JavaScript.

I browsed the web for a bit looking for charting components for jQuery. Couldn’t find anything I was happy with. Not functionality-wise but aesthetically. Or maybe I just felt like playing around with JS for some time – unfortunately as it often happens with JS I spent the best part of the last 2 days working on it.

I decided to use a library called Raphaël. This library makes working with vector graphics incredibly simple and the results just look amazing. I know that they are in the process of building a charting library on top of it however… well I have no excuse now other than I wanted to work with JS.

What I did is build a very simple jQuery component called SAPchart. It only draws line charts but it’s quite simple and I think the results look pretty good.

You can download the source here. It requires jQuery, Raphaël, and raphael path methods.

I’m now wondering whether I should keep working on it and improve it.

Anyway TweetSentiment is now available here. I’m planning to keep it running and collecting data for a while. I’d say you need at least 3/4 months worth of data before you can make any useful observations.

Bookmark It