Search by photo and shop the look in Magento: what it takes
Some shoppers can show you what they want but cannot name it. Text search cannot help them. How visual search and shop the look work, and what to sort out first.
A shopper saw a bag on a train. Tan, soft leather, a flap with a metal clasp, about the size of a laptop. She has a photo of it on her phone. On your shop she types "brown leather bag", gets 60 results, scrolls two pages and gives up. Your shop may well sell something very close to it. She just has no way of telling the search what she saw.
The same thing happens on product pages. Someone likes the shape of a jacket but not the colour or the price, and wants to see what else looks like it. Whether they find anything depends on which related products someone in your team picked by hand.
The short answer is that Magento's search and product lists work with words and with links people set up. To search by appearance you need image embeddings: a model that turns each product photo into numbers that can be compared. A lot of the problem can be reduced without that, with good attribute data and filters. The rest is what our AI Visual Search & Shop the Look module is for.
What Magento gives you without an extension
Magento's catalogue search matches the words in a query against the text of your products. There is no way to search with an image.
On the product page, Magento has related products, up-sells and cross-sells. You add them per product in the "Related Products, Up-Sells, and Cross-Sells" section of the product1. In Adobe Commerce, related product rules can fill those lists by conditions, much like a catalogue price rule2, and Adobe's Product Recommendations service documents a "Visual similarity" type that recommends similar-looking products to the one being viewed3. If you are on Adobe Commerce, check those first.
Without any of that, the honest fix is better data. Fill in colour, material, pattern and style as filterable attributes with plain values, and make sure layered navigation shows them on search results and category pages. A shopper who can click "Bags, Leather, Tan" is most of the way to the bag on the train. Then spend time on related products for your best sellers, because those pages get the most visits. This costs nothing but time, and it helps every shopper, not only the ones with a photo.
How image search works
An image embedding model turns a picture into a long list of numbers, so that pictures that look alike end up close together. Colour, shape, texture and the type of object all play a part. You embed every product image once, and embed the shopper's photo when they search. Then you look for the product images whose numbers are closest to the photo's.
"Find similar" on a product page is the same idea, but cheaper: both sides are product images you have already embedded, so no new request to the AI is needed when the page is viewed.
Shop the look adds one step. A lifestyle photo shows several things at once: a jacket, a scarf, a bag. Compared as a whole, it mostly matches whatever dominates the picture. So a vision model first finds the separate items and where they are in the photo, each item is cut out, and each cut-out is matched on its own.
None of this reads a product code from a photo. It finds products that look alike, which is what the shopper usually wants, but the closest match can be a similar item rather than the exact one in the picture.
What our module adds
A camera button sits in the search box, on Luma and Hyvä. On a phone it opens the camera or the photo library. The photo is scaled down in the browser, checked on the server by its content (not its file name), stripped of EXIF data including location, and re-encoded. It is not written to disk unless you switch on a troubleshooting setting. The results page uses your shop's own product cards, with the customer group's prices and add to cart, and category buttons to narrow the results without another AI request.
On product pages there is a "Find similar" button on the gallery and a "Visually similar" list below the product. It uses the stored vectors, is loaded on demand and is cached per product and customer group.
For shop the look you can build looks yourself: upload a photo, click to place numbered hotspots and pick the products. Looks appear through a CMS widget and on the pages of the products in them, with add to cart per item and "Add all to cart". In the admin, "Detect Items with AI" asks a vision model for the items and suggests matching products for each; you confirm, swap or remove each suggestion. They stay suggestions until you confirm them, which matters because a wrong match on a merchant look would be shown to every visitor. Shoppers can also use shop the look on their own photo, which costs one vision request per photo.
Product images are indexed through our AI Core module: the base image, the small image or up to four gallery images per product. An image is embedded again only when it changes. There is an hourly limit per visitor (20 searches by default), a daily limit per website (500 by default) and AI Core's monthly budget. The search log is anonymous: it stores no photo, IP address or customer.
Limits worth knowing
Image embeddings come from Voyage AI, so you need a Voyage AI key. Item detection for shop the look needs a vision-capable model from Anthropic or OpenAI. The shopper's photo is sent to those providers to be processed, so your privacy notice should say so.
Results depend on your photos. Plain packshots on a clean background match best. Lifestyle shots with several products match the main one, which is why looks exist. HEIC photos work where the browser converts them (iOS Safari does); otherwise shoppers are asked for JPEG, PNG or WebP. Behind a proxy or CDN, Magento has to see the real visitor IP address, or the hourly limit counts all visitors together; the daily limit and the budget still apply.
And it is not for every shop. If you sell spare parts, books or anything people choose by specification rather than by look, visual search adds little. It makes sense where appearance drives the choice: fashion, bags, shoes, homeware, furniture, jewellery.
AI Visual Search & Shop the Look is not on sale yet. We are testing it on our demo store, and will write about it again when it is available.
Sources
- Adobe Commerce Admin documentation, "Related Products, Up-Sells, and Cross-Sells", https://experienceleague.adobe.com/en/docs/commerce-admin/catalog/products/settings/related-products-up-sells-cross-sells, checked 10 October 2026. ↩
- Adobe Commerce Admin documentation, "Related product rules", https://experienceleague.adobe.com/en/docs/commerce-admin/marketing/promotions/product-relationships/product-related-rules, checked 10 October 2026. ↩
- Adobe Commerce documentation, "Recommendation Types", https://experienceleague.adobe.com/en/docs/commerce/product-recommendations/admin/type, checked 10 October 2026. ↩