Image Recognition and Tagging Technology in 2018

In this article we’ll be checking out some of the top image recognition software currently available—specifically concerning image tagging APIs. Most of these companies have demos on their sites, so I’ll be putting them to test with two pictures of my own—one easy, one a bit tougher. Here they are.

First up will be this guy, because the internet loves cats wearing hats .

Then we’ll give this mosaic a go.

Clarifai

First on the list is Clarifai, one of the top image recognition APIs available today. It gives companies and developers an easy yet powerful way to integrate visual search into their apps. This includes image tagging and similar image search. They also offer “custom training so that you can train your own recognition models (on anything) and also search through them.” Oh, and this also goes for videos too.

So let’s try it out—first with our cat, then the mosaic.

Results:

There were actually more but you get the gist of it.

You can also check for faces, colors, and NSFW content. This pic came in SWF, no face, and gave five pretty accurate colors. Not too bad, but you’ll have to decide for yourself if these results are accurate enough for your needs.

When I tried the next pic I got the following: Oops, something went wrong. Try reloading the model.

wah wah wah :(

Turns out the file was too big, though I had to figure that out on my own. I uploaded a smaller version and presto:

Not really the best results, but depending on your needs Clarifai could be a good fit. They offer free, pay as you go, and enterprise plans, so pick which is right for you.

CloudSight

Let’s try another. With CloudSight you can keep it simple—send an HTTP request with your image and you'll receive a description of its contents. Sounds good to me. So let’s dive right into the results.

I uploaded the image and got back some JSON:

{
   "token": "G_h3bhJITmdtBMOrsHcCrQ",
   "url": "http://assets.cloudsight.ai/uploads/image_request/image/353/353008/353008767/IMG_20140726_001419.jpg",
   "ttl": 54,
   "status": "completed",
   "name": "white cat"
}

White cat—yeah that pretty much sums it up. Now the second pic:

{
   "token": "TblM2W9AaORdhmW0O3kA-A",
   "url": "http://assets.cloudsight.ai/uploads/image_request/image/353/353009/353009932/IMG_0345.jpg",
   "ttl": 54,
   "status": "completed",
   "name": "brown and yellow mosaic bird decor"
}

Brown and yellow mosaic bird decor. Impressive! But these results do come at a cost, namely latency. While other APIs return results in measurements of milliseconds, CloudSight is more in the seconds range—up to 15 actually.

Plans range from free for up to 500 requests and then paid plans from $99 per month.

If you need super high accuracy and are willing to wait a bit longer, CloudSight is the way to go.

Cloud Vision API

Google’s Cloud Vision API gives you powerful machine learning in an easy to use REST API. You’ll get image labeling, face and landmark detection, optical character recognition (OCR), and tagging of explicit content. No surprise, being Google it’s pretty slick.

It returns JSON if you like, or just get the labels, NSFW, and color results. For our cat pic we got:

The JSON returns all the info.

On the tougher mosaic?

Not bad I guess, but not as good as CloudSight. It also has logo detection to “detect popular product logos within an image.”

Pricing ranges from free for the first thousand units and then $1.50 per 1000 units for most features. A unit is one application of a feature to an image. So if one image gets face detection and let’s say logo detection, that’s two units.

Microsoft’s Computer Vision API

Not to be outdone by Google, Microsoft has their own Computer Vision API. By now I think we all get how these things work so let’s just dive in.

We get back a description:

{ "tags": [ "cat", "indoor", "sitting", "white", "table", "top", "computer", "black", "bed", "keyboard", "desk", "laptop", "standing", "room", "man", "umbrella", "laying" ], "captions": [ { "text": "a cat sitting on a desk", "confidence": 0.8545866 } ] }

and tags:

[ { "name": "cat", "confidence": 0.9951088 }, { "name": "indoor", "confidence": 0.9361982 } ]

We also get an adult/NSFW result of false, a racy score of false, and foreground, background, and accent colors. If you’re wondering, turns out a girl in a bikini will score false for adult but true for racy, so there’s that.

Pretty good results for our furry friend, but let’s see how it handles the mosaic.

Description:

{ "tags": [ "building", "standing", "cat", "group" ], "captions": [ { "text": "a close up of a cat", "confidence": 0.0968242 } ] }

Tags:

[]

Second wah wah wah :( of the article. Close up of a cat? Yeah, not so good Microsoft.

This API also has OCR features and can even read handwriting in an image. It also has celebrity and landmark recognition, and real-time video analysis. If you think it’ll be accurate enough for you and you like Microsoft stuff, give it a try.

It’s free for the first 5000 transactions per month and then, well it gets complicated but it’s on par with Google’s pricing.

Imagga

The last of our image tagging APIs is Imagga, another easy to implement, easy to use API. Their angle is their auto-tagging technology which can be trained. You can define tags for your specific use case and Imagga will learn to recognize and suggest them.

Alright, last one. Here we go:

No surprises here. As expected you get some JSON back with all the details. But did you notice that “show me more tags?” Well they mean it; the full list was about 85 tags!

Now the mosaic:

Some of the best results we’ve seen for my ancient mosaic. Way to go Imagga! Notice how the confidence percentages are lower.

Pretty good stuff from Imagga. They don’t have a free plan but the cheapest starts at $14 per month, and then they go up from there.

Try it yourself

Here are links to all the different demos so you can play around with them yourself. Just try and keep it SFW. Or don’t, either way. Have fun!

• Clarifai Demo
• CloudSight Demo
• Google Cloud Vision Demo
• Microsoft’s Computer Vision Demo
• Imagga Demo

Looking for more? Check out our Image Recognition Category page for lots more cool stuff.