I stumbled across this idea the other day, when I was wondering what was going on in town, so as you do, I went to the facebook page for hashigo zake, then bebemos, then little beer quarter, then goldings free dive. And I thought, there must be a better way to do this.

So, short of adding every single bar and restaurant in town as a facebook friend, and then having my newsfeed spammed with hundreds of updates, I decided to build a tool that shows updates from all the bars and restaurants in Wellington.

http://www.venues.pw/

I’m really interested in this idea. The vision I’m working towards would have a filter for bars | cafes | restaurants, the ability to sort by distance or time (so you can see what’s been posted nearby you), and a search function so you can look up if anyone is selling chilli or stout.

I think this could be a really great tool, and if I can get it working well in Wellington, I’ll try get it going in the Mission SF and Melbourne next.

I’ve been working on placepinner exclusively lately, to the expense of my other web property, ZoomIn. ZoomIn is still making a little bit of adsense revenue though, so it’s ok just to tick over in the background.

So since I’m investing so much time in my trip planning website (since that’s what placepinner will eventually be), I thought I should do a bunch of research on other trip planning sites and learn what they do well, what they lack at and try and work out exactly where the niche is that I want to occupy with Placepinner. I’d love to build the be-all singing-and-dancing trip planning tool from nam that did everything to everyone, but I’m one guy working 20 hours (one full day plus a few evenings) a week, so I have to be very particular with what features I’m going to try and implement. So, here’s a run down of a few sites that do trip planning and what I think of them.

TripIt

Great site, doesn’t deal with bars, museums, restaurants, ‘things-i-might-do’. It’s only for confirmed things that have dates or reservations. Great tool, people love it.

Kayak trip planning

Very similar to TripIt, doesn’t have the ability to add places i might-want-to-go-to.

Trippy

Great site. Has gone for the pinterest-of-travel angle. It has a bookmarklet for grabbing photos off website and create a travel wall of places you might want to go. Doesn’t seem to be widely used to bookmark what I’m focussing on (eg bars, galleries, cafes, public spaces). Doesn’t do much with their geodata, eg no ‘what’s nearby links’, no iphone app for exploring the places you bookmarked when you’re actually in a city. This seems like my closest competitor, since they do actually gather geodata, and it would be easy for them to pivot to creating personal city guides based on their data.

Dcovery

This is a great iphone app that has a really good bookmarklet. The bookmarklet is actually better than placepinner in that it looks really nice, and handles multiple places in one webpage (eg reading an article listing places to go in bangkok, rather than the foursquare venue page for a restaurant in bangkok) - but it’s worse than placepinner, in that it doesn’t automagically grab the map location or address on pages that have the address. They also don’t have any web front-end, or ability to edit places and add notes about places. Editing places and adding notes is something that’s coming in the next version of their iphone app, and they have become very popular on the iphone app store. They also have a business model, in that the app is a paid-app (99cents on special at the moment). Great competitor, they’re not in a hurry to go android, so maybe I’ll just duplicate them for android. ;)

Rego

Just linking these guys because they got a bit of press recently. Haven’t tried out their app since my ipad doesn’t run ios 6, but it looks like a nice tool. You can’t add places by name (searching via google places or foursquare), so it’s not hugely useful as a trip planning tool, but it’s got a nice looking interface, and they understand the potential as a trip planning tool so you can’t discount them.

Wanderfly

Wanderfly is an ajax-heavy web app for planning your trips. However, it falls in the trap that a lot of trip planning sites do, in that it only recommends places that already exist in it’s database. It has no easy way to add new places to your trip. It does a good job of recommending places, but a lot of sites do a good job of recommending places, and I still think there is a need for an app that easily lets you grab all these recommendations and jam them together onto one map. Wanderfly doesn’t do this yet, but maybe it will.

Foursquare

The almighty foursquare. With an awesome venue database, a bookmarklet for saving places (that’s not quite as good as the placepinner one, but is pretty good), and good mobile apps. Foursquare could totally own trip planning, but it’s not their sole dedicated focus at the moment, so they don’t solve it that well. eg, viewing places on my todolist on mobile isn’t that easy to do. To be honest, you could most of what I’m imaging placepinner to be as foursquare add-ons, eg a list of cities in which you have places to-do and not visited,

I’ve been doing about two weeks of android development now, on and off. It’s been interesting. The things I like:

Eclipses autocompletion and code highlighting

The code intellisense that eclipse provides is really great. I like how it compiles on the fly and shows unused and uninitialized variables, and the autocompleter works really well (coming from a ruby world, it’s a pretty nice feature of a strongly typed language).

Sensible exceptions

When you try and do some network activity on the main thread, android throws an exception. You can disable this, but you really shouldn’t, since all it’s doing is forcing you to make a responsive app.

AsyncTask

This is a really nice little class that starts up a new thread, runs some code in the thread, then rejoins your main thread and runs some other code in the main thread. That means that you can easily do some network activity (or anything else slow), and easily put the result back into your main UI code. Obviously you shouldn’t try and manipulate your UI from two threads at once, so the onPostExecute code that runs in the main thread is cool.

Nice animations and performance

Coming from a phonegap background, it’s nice all the little built-in animations that come with android (for example moving between activities), and the performance of list views and page rendering is much quicker than anything you could hope to do with phonegap.

Things I don’t like…

Eclipse

It’s still an ugly troll. And it’s one monolithic window, you can’t break it up and spread it around multiple monitors.

The Android emulator is slow

I’ve been told to use the intel-based (atom) emulator, using the Intel VT-X extensions, I haven’t got around to trying that yet (I do most of my testing live on my phone), but the default ARM-based emulator is slow as a dog.

Having to support 2.3

If you look at the android versions dashboard, you’ll see that you have to support Android 2.3 (API version 9) to get more than 50% compatibility with Android devices. 2.3 is okay, and most of the good features you want are backported by Google so that you can use them. I’m still learning what is and isn’t available, but at the moment I’m just using list views, text views and layout components and don’t need to do anything fancy.

Theming

I’m not sure how theming works at the moment, for example @ristaris 2.3 device has a very different theme to my 4.0 holo themed device. I’m not sure how I’m meant to make UI that looks native on both devices, or whether to make my own style toolbars that don’t really look the part on Android, I guess I’ll find out.

Potential revenue

From reading different Android forums (and my own experience with the Rankers app), it seems like there’s very little revenue to made from advertising networks, or even selling pro versions of apps on Android. The iOS store is a lot more lucrative. Sadly, there’s already a really good app that does trip planning / place bookmarking for iOS (it’s called dcovery), and I really want to make an Android app, since I’m an Android fan myself. I’m just not sure how I’m going to make a return on all the work that I’ve put into placepinner so far. Maybe sell the app, maybe only allow 100 bookmarks for free users, then encourage people to upgrade to pro. Something anyway.

It’s been an interesting time, I’ve also done a bunch of work on the web-site part of Plcaepinner (it’s still not ready for people to see, but it’s getting there). The three things I need to finish before I can promote placepinner are 1. Chrome extension (for bookmarking places), 2. Website (marketing site basically), 3. Android app. Then I can start to show people what I’ve been working on for the last few months.

I’ve started working on a native android app for placepinner. I decided to go this track after trying out a quick phonegap app using ratchet and backbone.js. Even though it’s super easy to develop, and you can do some quite nice UI with CSS, I couldn’t help but notice how laggy scrolling was using Phonegap.

So I’ve decided to go native. I’m also taking this as a good opportunity to try out Java (which I’ve never used in anger before), and learn some android mobile development (in case the ruby on rails job scene dries up).

My experience so far:

• Eclipse is an ugly tool
• Java is a sweet language
• The android emulator is much slower than the ios simulator

It’s a bit funny doing all the typecasting and type coercion that’s required to pass stuff around. For example in ruby:

JSON.parse(IO.readlines('http://placepinner.com/places.json'))

Versus this java code to do part of the same…

InputStream stream;
    
try {
    URL url = new URL(urls[0]);
    URLConnection urlConnection = url.openConnection();
    urlConnection.setConnectTimeout(5000);
    stream = urlConnection.getInputStream();
} catch (Exception ex) {
    Log.i ("Exception!", ex.toString() );
    return null;
}

String str = "";

try{
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    byte[] buffer = new byte[1024];
    int length = 0;
    while ((length = stream.read(buffer)) != -1) {
        baos.write(buffer, 0, length);
    }
    str = new String(baos.toByteArray());
} catch (Exception ex) {
    Log.i ("Exception!", ex.toString() );
}

return str; 

We’ll see how far I get. I’m using dcovery as inspiration for this app. They have a great travel planning bookmarklet and app for IOS. I’m trying to do the same thing for Android, but I’ll probably also have a website (Dcovery doesn’t have any website trip planning tools, so you can’t review / edit your trip on your PC), so that you can view and edit places you have bookmarked, and eventually maybe split places you have ‘pinned’ into seperate days (actually plan a trip day by day).

But yeah - android for now.

I released the first iteration of a mobile site for ZoomIn last night. Try it out on your iphone or android phone and you should get a mobile-designed stylesheet. It’s not complete, the recent / popular and group pages in particular aren’t done very well, and I want to add a view that shows you ‘what is nearby’ (using the html5 geolocation api), but it’s a first cut, and the place pages in particular are a lot more usable on mobile than they were before. I haven’t tested it thoroughly on iphone yet (I don’t have the simulator installed on my home laptop), but I’ll probably be doing more mobile work over the coming week, so things will slowly improve.

I’ve been working on the ZoomIn mobile view, on and off for a few months now. I initially started by creating a :mobile format in rails, so that I could render a totally different view for mobile users. Eg layout.mobile.erb and show.mobile.erb. This worked pretty well, and I got off to a flying start, styling up the mobile view and copying view code across from the desktop view.

This worked well for prototyping, but as I spent time redesigning the rest of the site, I realised that there are a lot of views in ZoomIn, and duplicating each one in a mobile view would take forever, and double my maintenance costs. So instead, I decided to go for responsive design, where I serve a different stylesheet for mobile devices. I’m not using media queries at the moment, I’m instead detecting mobile devices based on the user agent, but I think I’ll change this to use screen-width media queries and some javascript to hide / show bits of the UI that aren’t working in mobile yet.

The biggest problem at the moment is that I haven’t built in support for all the popovers that are used for login and most of the interactivity with the site. So it’s a ready only view of the app at the moment. I’ll probably wait and just implement facebook auth for mobile and skip support for the current username / password login. One of the things that has worked well has been using google maps and streetview on mobile, they both run really nicely on my nexus s, which isn’t a super powerful phone. I was going to use leaflet and cloudmade maps since I have experience of that on mobile, but it seems that the google maps implementation is really nice on touch devices.

For design, I’ve been copying lots of bits of UI from ratchet, so I’ve got a nice blue header bar and some good looking buttons and form elements.

I’ve been working on a trip planner, on and off for quite a long time now. Probably since back in 2008, when I started work on ‘weheartplaces’. The original idea was to make a site for bookmarking places you want to go, like delicious, except for places. Basically you’d use it to make a list of places you might want to go one day. Foursquare does a pretty good job of this with their ‘want to go’ feature. And trippy does a pretty good job of being pinterest for places, you can create a wall of all the pretty places you want to visit one day.

Me personally, I still want to create a chrome extension, that lets you quickly save the current page to your travel map. I’m not sure what the next step should be though, whether to create a fully fledged travel planner where you can divide places into days (I’ll go to volkspark on tuesday and gorlitzerpark on wednesday), or whether just to make a minimal ‘places i’ve starred nearby’ app for iphone and android (using ratcher and backbone.js and phonegap).

So yeah, I haven’t decided where exactly I’m going with this project at the moment, and ZoomIn is taking up most of my free development time (especially now that its growing, slightly, week on week) - but watch this space, there might be a cool chrome extension / travel app coming from me. If I can decide what it is exactly that I want to build.

This post partly inspired by utrip - with their cool travel planning app:

Tuesday to Friday I work at Powershop, on their large rails app that runs most of the business. They’re a great team and I really enjoy working on such a gnarly rails project.

However - Mondays are my work-from-home-on-ZoomIn day, and this Monday I worked on two things.

Rails 2.3 migration

ZoomIn is written on ruby on rails, starting with version 0.6 back in 2005. Over time it has been upgraded, but when I took over ZoomIn last year, it was stuck back on Rails 2.1. This was a bit of a pain, because some of the extensions I wanted to use (eg Facebook connect) aren’t available for such an old version of rails, and secondly, there are a whole swathe of security issues with Rails apps that were fixed in 2.3.17. So, after updating my gems, copying in the initializers for the new app, I was mostly ready to deploy the new version. I used a git branch to test out the app against Rails 2.3. One of the biggest problems I had was replacing classic_pagination with will_paginate, but in doing so, I sped up a few queries and redesigned the pagination links on a few pages, so it worked out to be quite a benefit in the end.

So as of Monday night, ZoomIn is now running on the latest 2.3 version of Rails. Score!

Groups page redesign

After getting the rails migration mostly done, I was fiddling around with the groups functionality in ZoomIn. This has been a part of the site that I was always very excited about. I had modelled the groups on Flickr groups, which are a very social part of that site. So groups have a name, a description, an owner, and members, places and topics. My first priority was to update the groups so that they used the new bootstrap theme that I had applied to the rest of the site. This was pretty straightforward.

Next up, I was looking at a group that I created (places I have lived) and decided that it would be cool to add comments to each of the places that were included in a group. So, I made it that under each place that is listed as part of the group, it looks for comments on that place, that were made by a member of the group. If a comment is found, it’s then displayed under the place, so that you can get a bit of a narrative going on of why that place is part of the group.

Next up is to work on Facebook integration I think, since people seem to have problem with the current authentication on ZoomIn. Also it’d be good to make it easier for businesses to add or update (or even promote?) their business on ZoomIn, since I get a lot of emails from people that can’t work out how to add places. But that’s for another day…

The lifeblood of a website is it’s traffic. If no one is using your site, it’s basically dead. One of the things that interested me most when I was considering purchasing ZoomIn, was it’s traffic stats. On a typical Monday, ZoomIn has around 10,000 visitors, 85% of which are from New Zealand.

This doesn’t make ZoomIn a top 100 website in New Zealand in terms of rank, in fact it comes it at number 660 according to the Alexa ranking. That’s even further down the list than Zenbu, a similair site that has been around for about as long.

However, those visitors every day are interested in New Zealand, and are interested in maps, so they’re a really great sounding board to try new ideas and try and find one that sticks. Something that I noticed early on (back in 2006) in ZoomIn, was that every time we made a release, we would get a slight bump in traffic. I guess this is because we had interested users, and that whenever they noticed a change to the site, they’d dig around and see what had changed.

I sort of foolishly assumed that I’d get the same little traffic bump every time I do a release to ZoomIn now. Sadly, that doesn’t seem to be the case. I’ve done about 50 releases to ZoomIn in the past 6 months, many small changes, and a few large ones (redesigning to use twitter bootstrap for example), and none of these releases have made a significant difference to visitors, page views, or pages per visit.

I’m working my way down my list of things to do to ZoomIn, hopefully I can report back in 6 months and show a slow growth in the traffic stats above.

I’ve been thinking of upgrading ZoomIn to some dedicated hardware. I currently use a 4gb ram VM hosted by rimuhosting in Auckland. They’re a pretty good crowd and the performance has been ok, but IO is especially slow, so anything which hits disk is slow. I’ve fixed this by caching expensive things (for example the json used in the new search sidebar) in memcached, and adding indexes wherever explain analyze showed a sequential scan, but it’s annoying to have to do so much performance work in production, when performance is perfectly acceptable in my development version (running on a SSD and 4gb of RAM). And so I’ve been thinking about moving to some dedicated hardware - maybe something like this:

• Poweredge R410 $1499
• 16gb RAM $501
• 120gb SSD $176

And then the monthly costs, hosting with xtreme, who have a Wellington based data center - which is important since the majority of ZoomIns visitors are from New Zealand, and things like the autocompleter require ultra low latency.

• 1U Rackspace $199
• Extra 20gb intl. traffic $99

Which comes in to at least $300 per month, after the capital costs of buying the server. Another option, but I’m not sure of the quality of their services, is HD in Auckland. I’m not going to rush into this, but if I get the advertising revenue up a bit on ZoomIn, it might be worth upgrading to some more chunky hardware.

When we first wrote ZoomIn, back in the day, the built in search that came with Postgres (TSearch) was implemented as a bunch of stored procedures, and generally sucked. So we used a high performance C++ reverse index search library called Xapian. Why did we need full text search in ZoomIn? Well, the plan is that people would tag places with keywords, and then you could generate a searchable column on each place, that contained all the tags, the address and any other information we had about the place, so a search like:

beer bar wellington

Would surface Hashigo Zake and Bebemos, just using the magic of full text search.

Note that we didn’t use FTS for our address parser, instead we used a bayesian search tool that I wrote from scratch in a bit of gut-driven intuition, and then Nick rewrote after doing the maths to work out exactly how it should work.

Our Xapian integration was pretty cool, we wrote a module for Postgres that exposed Xapian query results as a temporary postgres table (that we could then join with our Places table). But when I took over ZoomIn last year and moved everything onto new infrastructure, I wanted to reduce the number of moving parts, and one of the easiest ways to do that was to remove Xapian. Luckily, Full Text Search in postgres has come a long way, and the new (built in) text searching functionality is fast enough on the data set I use. So now I just have an after_save hook that rebuilds the tsvector column, which updates the index and makes sure we can always search quickly and freshly on any of the 100,000+ places in the ZoomIn dataset.

I’ve been working on ZoomIn a lot in the past few weeks. I need to start writing more newsletters about what I’m doing. But anyway, there’s a technical aspect of ZoomIn that I want to highlight, since it’s really great.

Back in 2006, when John, Nick and I were working on the first version of ZoomIn, we were discussing ways of storing the ZoomIn url hierarchy in the database. So for example, we had nodes like:

• /wellington/
• /wellington/newtown/
• /wellington/newtown/coromandel-street/

And in the database, we wanted to store that newtown was a child of wellington, and coromandel street a child of newtown. The easiest way to do that would have been just randomly inserting nodes into the database and building a tree structure where each node has a parent_id. The only downside to this, is that I wanted a really efficient way of finding all the nodes, comments, or photos, under a node in the tree.

eg - we wanted to be able to do a single query (without any crazy joins) that would return all comments on any streets, places or street addresses under /wellington/newtown/.

We went away for a while, and bounced over a bunch of ideas, then eventually stumbled on the inet6 datatype that is native to postgresql. So after a bit of research, we worked out that we could use an inet6 as a primary key and foreign key, rails worked well with it, and indexes operated fine as well. So what we ended up doing was something like this:

• /wellington/ => 1::
• /wellington/newtown/ => 1::1::
• /wellington/newtown/coromandel-street/ => 1::1::1::
• /wellington/newtown/hanson-street/ => 1::1::2::
• /wellington/newtown/hanson-street/43/ => 1::1::2::1::
• /wellington/berhampore/ => 1::2::

And we wrote two helper methods on each Place (a node in the tree), left and right.

• wellington.left => 1::1::
• wellington.right => 1::FFFF::

So - if for example, you want to find all the comments under Newtown (to show the new forum page), you do a query like this:

['select * from comments where place_id between ? and ?', newtown.left, newtown.right]

In the end, we actually used a variable sized structure (so we don’t use a whole subnet per node, but instead a specific bit-depth), but the basic idea is as above.

I’m not sure what kind of data structure this is (I think maybe it’s a fixed tree?), but it’s worked excellently for over 7 years on ZoomIn. Combined with the GIST 2-dimensional index that PostGIS supplies, we’ve been able to do most all the queries you want on a geo-social site without relying on denormalizing the data or super expensive mega-joins or subselects.

Edit Stig from SilverStripe informed me that this technique is called a Nested Set.

I’m not sure if I’ve already shared my rendition of the Imperial March.

I give you a time of day, you give me a name for the appropriate meal. I was trying to work out how to do this with a case / switch statement - then came across this syntax which is kinda nice:

def time_of_day_to_meal_name(t = Time.now)
   {
     0..5 => 'Midnight snack',
     6..10 => 'Breakfast',
     11..14 => 'Lunch',
     15..16 => 'Afternoon tea',
     17..21 => 'Dinner',
     22..24 => 'Late night snack'
   }.find { |k,v| k.include? t.hour}.last
 end

What would you like for lunch?