Ed's Big Plans

Computing for Science and Awesome

Wanted: Semiotics Search Tool

with 2 comments

Brief: One of the problems that I’ve encountered is the complete and utter inability to search for symbols online. One can enter keywords, but there doesn’t seem to be a good generative grammar or stick-figure search to specify the symbol that you’ve seen so that you can ask “what is this figure called”, “what does this figure mean?”, “who does it belong to?” — the closest I’ve found has been this reference, fittingly called symbols.com — but it only offers you a symbol whose name you already know. There are also a few tools that will present several chemical compounds that match a query sketch the user inputs — here’s a structure search by PubChem and another one by eMolecules.

So what figures do I want to be able to search for? Here are a few example queries …

  • A circle is drawn with a freehand curve separating the figure roughly into two halves — should turn up the Ying Yang along with a few other circular bipartitioned figures.
  • Two arcs are drawn beside one another concave inward with a staff in the middle — should turn up the symbol for Sikhism and the Caduceus.
  • Two to four stick figure humans are placed inside a box — should turn up the symbol for an elevator or washrooms etc.

If anyone knows of a good sketch-based semiotic search tool, please let me know. Or conversely, if anyone’s interested in having one developed — I’d be interested in helping šŸ˜›

Eddie Ma

August 3rd, 2010 at 1:57 pm

Posted in Web Programming

Tagged with , ,

Kamil Slowikowski says...

Hi Ed,

I’ve been following your blog for a little while now, good stuff.

I found something on LifeHacker for you: a Hitachi-powered search called GazoPa.

Here’s your first example, it works pretty well! http://bit.ly/yy001

I was searching for something less relevant and found GazoPa instead. I was looking for a project where you could draw some stick figures in a scene (for example, a man on a house) and it would be rendered with real images. I think it was a limited research project, perhaps by Adobe, never released.

Eddie Ma says...

That’s really neat actually. I bet GazoPa and the project you mentioned could use the same image-recognization-to-descriptor front end. There was a grad student at Guelph that was working on the opposite half of what we’re after: she was working on that terribly intractable space of describing the components of an image for images returned from a search engine. I haven’t caught up with her yet, I think her name was Melanie Veltman.