I know rambling on about something that isn't yet live or on display isn't terribly exciting, but I wanted to give a quick update on JungleDragon v2 anyway. This time I'll express the update as seen from a user.

So you've uploaded an image and want to identify the specie on it. Actually, getting you to want this is another challenge but for now I assume this is true :)

There's currently four major challenges in doing a succesful specie identification: 

1. Being specific enough

JungleDragon is only able to match a specie if what you enter actually is a specie. Quite often what you think is a specie in reality is a family of species, and that's not specific enough for JungleDragon. It's all or nothing: you match a specie or none at all, a partial identification at a higher level is not impossible.

This "limitation" is a conscious design decision, as partial identifications will make things a lot more complex and I want to move forward and deploy this feature soon. Therefore, the only "fix" I can offer is to educate users in this area.

2. Entering a specie for which there actually is a Wikipedia page

It is very well possible that you correctly identify a specie, yet Wikipedia has no page for it. In that case, JungleDragon is unable to find any information and identification as a whole will fail.

In the longer term, I may offer a fix by searching for the specie in other sources, but for now, this too is a limitation of JungleDragon.

3. Spelling and casing the specie name correctly

Wikipedia has case sensitive pages, and worse, there's no consistency in them either. Without intervention this could mean that identify a specie by wrong casing will lead to failure. Actually, it could be that your casing is correct, just different from the specie casing in Wikipedia. From the Wikipedia perspective, this problem is sometimes solved by making hard redirect pages. My engine can detect that, and will resolve the redirect.

But it doesn't always work. Many pages do not have such a redirect and will simply fail. "Milky stork" will fail and there's no redirect, whilst "Milky Stork" will work. Could I simply uppercase each word to find a match? No, since there's no consistency at the Wikipedia end. 

The update in this area: I've found a way to largely fix this. If no direct match is found, I fall back to the opensearch API of Wikipedia and it's working pretty good so far. All case variations of "Milky Stork" lead to a successful match so the engine is a lot less sensitive to your input. It's even able to fix small spelling variations, such as "red-bellied piranha" vs "red bellied piranha". 

I need to do further testing, but already this new mechanism has drastically improved the success rate in testing.

4. Parsing the results correctly

The final part of specie identification from the JungleDragon perspective is to correctly parse the Wikipedia page content if a page is found. This part is complex because I want selective information and I want it formatted in a specific way. Thus, I need to be able to handle the numerous markup forms that Wikipedia page authors use.

The update in this area: things are stabilizing. Where during initial testing 1 in 2 pages had parsing or rendering problems, it's now 1 in 10. Further testing and fixing should improve that rate even further.

Planning

The way things are going now, I'm hoping to deploy a first version by the end of this month. It will not be perfect, but hopefully usable enough to start identifying species. Identifying species is the basis of a new world of possibilities for JungleDragon.

Some people enjoy tool hunting, constantly searching for the latest and best tools to optimize their workflow, meanwhile undoing those optimizations by constantly switching tools ;)

For many needs that I have, I'm not like that. I like to minimize the tools in my belt, stick to them and utilize them. I had been using Bloglines to read blog feeds for many years, until it was shut down/taken over, and I was forced to move. I switched to Google Reader. Now, I'm in a similar boat concerning my bookmarks.

For years, I have been using Google Bookmarks. It's a little-known service, and it is very basic:

It's not great, but it works. Until Google in their relentless cleanup momentum decided to drop support for the Google toolbar, as of Firefox 4 I believe. It continued to work for a while, but not anymore.  I can't access my bookmarks from the toolbar anymore, nor can I add new ones.

Unwanting to deal with the problem, I went even more basic by accessing my bookmarks from the webpage, which is a bit painful. Until adding new bookmarks started to fail a few days ago there as well. These are enough signs to get the hell out of the service.

Opportunistic as I am in such unwanted tasks, I was looking for the least painful way to get similar functionality, my requirements to be summarized as follow:

Accessing and adding (private) bookmarks easily from any device, browser or location

Oh, and for free of course. I eventually settled on Delicious, which lets me import all my exported Google Bookmarks in one go, auto tag them and set them to private. That's all I wanted, and it took less time than writing this blog post. Such a relief, as I was thinking the conversion would take me precious hours. 

As a final note, I want to add that due to search and social networking, the importance of bookmarks has decreased for me. Still, I do have a lot of work-related bookmarks that cannot be retrieved or remembered easily, as they are not searchable.

The above meme sums up what I'm currently doing in the JungleDragon area. I've been demonstrating basic specie identification and visualization earlier. That part is fit for production. The back-end of the so-called specie engine isn't yet. I'm hardening it as follow by testing my own photos sets:

Each successful identification makes the specie engine better and more reliable. Currently, the fail rate is still quite high, too high to deploy this, so I'm thinking it will be a few more weeks before I release it. Here are some of the more common problems I encounter:

So the feature is forthcoming within a matter of weeks. It will be awesome, but not perfect upon release. It will always have some degree of problems, which I'm hoping to minimize over time.

Yesterday I explained my progress concerning specie integration in JungleDragon v2. Much to my delight, I received a few very positive comments. I need those to fuel my motivation. This part is hard, complex work with constant setbacks.

Anyway, I also explained I was playing with the idea to automatically integrate Youtube videos with the specie page. Quite a simple, yet powerful way to see a specie in action. My theory was to simply use the Youtube API and to pass it a search string, in my case the name of the specie.

You can see above how this is not going to work. The Impala, other than a specie, is also a car. A Polar bear, other than a specie, is also a cartoon, dance event, and part of the song title of various music tracks. In these two simple test cases, about 90% of the search results have nothing to do with the specie.

So this is a failed experiment. Funny, though: Henriette asked what I was doing and I explained the problem to her. She suggested to use the scientific name as a search string. As this was a 5 second code change, we could instantly see the improvement in relevancy: up to about 80%. Much better, but I'm thinking it's still not good enough. I then had the task of explaining to my beloved one how brilliant her remark was, yet how I was going to ignore it anyway. I'll spare the details. 

How to move forward in this area? I still believe in the power of video to enrich specie information, so here's my plan:

What do you think?

Life and work is quite hectic at the moment, which explains my relative silence. Yet I keep pushing JungleDragon v2 development and hereby I want to give you a more in-depth teaser. Let's get to it. Note that the below screenshots are of the development environment.

Here's a photo uploaded to JungleDragon, of which the specie is identified as a "Polar Bear":

As a result of this identification, an interesting side panel appears. Let's have a closer look at it:

This essentially is a summary of this specie, with the following data visualized:

There's a lot more that JungleDragon knows about this specie. To learn more about it, there's the obvious button "Learn more about this specie". Clicking the photo of the specie will also you bring you to the specie page, which currently looks like this:

So this is the dedicated specie page, which tells everything JungleDragon knows about it. This is only a fragment of the page, in reality it is about 10 times longer.

First, we start with a large header containing a larger version of the most popular image, as well as the full description of the specie.

Below that are a set of dynamic links, the ones with the arrows pointing down. These represent the areas of which JungleDragon has information about the specie. For example, rather than scrolling the entire page, you can instantly jump to "Diet" and from there go back to the top. JungleDragon does not copy all text blocks from Wikipedia, only a select few. 

The left column below the header contains the actual text blocks. These come from Wikipedia, yet they are transformed so that it becomes natural language with various Wikipedia elements removed (such as links, citations, etc). Another interesting element to the text display is that it is mixed with JungleDragon photos, not Wikipedia photos. You'll notice the first text block containing a left-aligned photo. The next text block will have a right-aligned photo. This keeps going until we run out of text blocks or photos. It makes for an attractive format of reading and an interesting way to mix Wikipedia content with JungleDragon content, or so I think.

Moving on the right column, we have a "Distribution and status" block which contains an image of this species' distribution (if available on Wikipedia) and again the IUCN status, richly illustrated.

Below that is the taxonomy block which classifies the specie in the biological "tree". As you can see, the elements in the tree currently are not links. Initially they will not be, even when I bring this live. If I can, I will implement this though. It would allow for tree-like browsing. For example, you could jump to a page of the family "Bears", to see all bears, not just Polar Bears.

So for now this concerns the overview tab of the specie page. There's also a photos tab:

It does what you expect: it shows photos of that specie, and you can sort them by popularity or upload date. 

The third tab "videos" is not implemented yet. It's just an idea I'm having. The idea being that I use the Youtube API to shows videos of this specie based on a search string. I will give this a try to see if I can get it to work reliably.

There's about 10 other things to implement to fully implement the v2 vision of JungleDragon, but this teaser definitely represents the core functionality. Your feedback is highly wanted!

Continue reading...