rclip Update: Mixing Text and Image Queries for the Ultimate Photo Search

Hi! Today, I am introducing a fantastic new feature of rclip (an AI-powered command-line photo search tool I built): the ability to add and subtract different queries. It’s fun to play around with different concepts in this way. This feature is also handy when you want to find similar images. If it’s the first time you read about rclip, check out this and this blog posts.

The feature works like this. If you want to add two queries together, add a “plus” sign between them like this:

cd photos && rclip horse + stripes

If you want to subtract one query from the other, use the “minus” sign:

cd photos && rclip apple - fruit

It works with image queries too:

cd photos && rclip "./new york city.jpg" + night

When combining image and text queries, you may notice that sometimes your text queries are seemingly ignored by rclip. This can happen because CLIP (an AI model that rclip uses) weights features extracted from images more than features extracted from text. Luckily, rclip provides a way to fix this. You can control the influence the query has on the result by prefixing it with a multiplier. It can be any positive number, and you can multiply both image and text queries. Here is an example which doubles the influence of the “golden retriever” query:

cd photos && rclip "2:golden retriever" + "./swimming pool.jpg"

And last but not least, when combining queries like this, you are not limited to two queries. Combine as many as you want. For example:

cd photos && rclip "./racing car.jpg" - "2:sports car" + "2:snow"

If you are interested in seeing how all these queries perform when executed on the 1.28 million images ImageNet-1k dataset, check out the video demo on YouTube:

This is another rclip feature initially introduced by a GitHub user ramayer. Huge thanks to ramayer for this incredible contribution! Speaking of contributors, I always appreciate the enthusiasm and fresh perspectives that new contributors bring! If you’re considering getting involved in open-source projects and making a positive impact, a number of issues on the rclip GitHub page could benefit from your attention and expertise. Why not give them a look?

That’s all you need to know about the new feature. You can download the latest version of rclip by following the instructions in the repository’s README. Try it out and share your thoughts. Stay curious, and enjoy your photo searches with rclip!