Introduction
10 years ago, when I was in graduate school studying an aging-friendly industry, I googled everything for my research. I spent 10 hours picking up good articles and research. After digging up some good articles and research, I can finally start my paperwork. Can you imagine how short my hours were?
E.g., googling the best beer.
But now, instead of googling, I open Gemini or ChatGPT and ask, "Hey, can you find reliable and trustworthy articles for my aging-friendly IoT research?" or give the AI some pictures related to my research and ask, "Do you have any similar items to this one?" My AI secretary finds them within a minute. Now I can start my paperwork without spending 10 hours.
.gif) |
E.g., searching for the best beer in Ontario! |
People usually ask Gemini, "What is this?" instead of googling it. Now, the Search does not start with fingers but with eyes. This change occurs in the minimalism UI; all the complications are off.
What is Visual Search?
Visual Search does not simply compare the picture. The computer interprets the items as human eyes would, converts them into data, and finds the correct answer.
How does it work?
Step 1: Image Capture & Preprocessing
When a user takes a photo or uploads it, the system refines the image to make it easier to analyze.
- Noise Reduction: Compensates for shaking or poor lighting.
- Object Detection: Focuses on key subjects, such as a 'bottle' or 'label', from the entire photo.
Step 2: Feature Extraction & Embedding
Computer Vision AI analyzes images pixel by pixel and converts them into numerical data.
- Feature Analysis: Reads the colour, texture, shape, and pattern of an object.
- Vector Embedding: The extracted features are converted into a "vector" consisting of tens of thousands of numbers. The photo now becomes a single "coordinate" that the computer can calculate.
Step 3: Similarity Matching
Compare the coordinates of my photo with the coordinates of numerous photos stored in the database.
- Nearest Neighbour Search: Finds the data closest to my photo vector.
- Results Ranking: Displays information to the user in order of highest similarity (e.g., "There's a 98% chance this beer is Muskoka Brewery's Mad Tom IPA!")
What problem does it solve?
Visual Search solves the 3 big problems that have limited the effectiveness of Search.
- The "I can't describe it" Problem: Visual Search can find things that can't be described. For example, a "Vibrant orange can with a skeleton wearing a cap." Human memory is based on visuals, but the search tab only wants text.
- Zero-friction Experience: Instead of 5 steps,
[unlock -> browser -> search -> compare -> confirm],
[Run camera -> take a shot].
- Automation of Manual Entry: It solves 'input fatigue'. A single photo automatically fills in all metadata. The user changes from an inputter to a checker.
Summary
This shift in how we interact with technology isn't just a personal convenience; it is a massive architectural change being led by tech giants. A recent TechRadar report highlights this exact evolution, specifically how Google is transforming its interface to meet the "visual-first" demand.
According to Eric Hal Schwartz in his article, "Google Adds Eyes to AI Mode with New Visual Search Features" (TechRadar, 2025), Google is moving beyond traditional Search by integrating a "Visual Search Fan-Out" approach.
- Visual Search Fan-out: Moving beyond simple pixel matching, the AI now deconstructs images into multiple dimensions such as texture, colour, and background context. This fan-out approach allows the system to interpret the user's true intent, delivering contextually relevant results rather than merely visually identical ones.
- Integration with Natural Language: By merging vision with conversational AI, complex search filters are being replaced by natural dialogue. Users can simply snap a photo and refine their search with commands like, Show me this style in a lighter shade," creating a seamless bridge between visual input and specific desires.
- Real-time Shopping Graph integration: By indexing over 50 billion projects updated hourly, visual search provides instant access to live pricing, reviews, and local inventory. A single photo transforms the smartphone into a real-time personal shopping assistant.
- The criticality of Design: The article emphasized that to remain discoverable in this new era, websites must prioritize Clean Visuals and accurate metadata. As AI becomes the primary curator of information, high-quality imagery is no longer just an aesthetic choice ot os a functional necessity for search visibility.
How it applies to the mobile development industry
In the Mobile Development Industry, Visual Search is not just adding a search function. This changes the UX (User experience) architecture entirely.
Here are the 3 cases that I researched for:
 |
| e.g., using a camera lens for searching for lamps. |
- Pinterest Lens (The Pioneer of Visual Discovery): Pinterest has solved the problem of finding inspiration for things that cannot be expressed in words. When a user takes a photo of a piece of furniture they see on the street, it instantly displays a list of similar interior design pins. This allows users to begin their customer journey without knowing the exact brand name.
 |
| The Vivino app just scans the wine label and gives you all the information you need. |
- Vivino (Solving ' Input Fatigue' in Niche markets): Vivino, a wine app, is a very successful app for using visual search. Wine labels are difficult for people to understand. Through visual search, the user can instantly find the rating, taste, and pairing of food.
- IKEA Place (Visual Search meets AR): IKEA combines the visual search with AR tech. Customers can place the furniture in their room virtually before buying it. It proposes a model in which search does not end with acquiring information but extends to ‘interaction with real space.’
My Opinion
Convenience at a Cost: The Thin Line Between Efficiency and Thinking
Now, we are witnessing another monumental shift: the transition from active input to a single shutter click. We no longer need to struggle with questions like, 'How do I put my thoughts into words?' or 'How do I explain what I'm looking for?' Instead, we simply open the camera and take a shot.
This change is incredibly powerful, yet I find myself feeling a mix of excitement and apprehension. On one hand, visual search is a liberation; it allows us to 'ask' about things that are impossible to describe in words. It bridges the gap between our visual senses and digital data.
In my view, the true power of visual search lies in its ability to translate the untranslatable. As a non-native English speaker, I often feel a "vocabulary gap" when trying to describe specific aesthetics or feelings. For example, a word like '청량하다' (Cheongnyang-hada) carries a complex sense of coolness and clarity that the English word 'refreshing' can't fully capture. Similarly, '푸르스름하다' is so much more nuanced than just 'bluish.'
When we search with text, we are limited by the words we know. But when we search with our eyes, those linguistic boundaries disappear. Visual search allows us to communicate with AI through "Visual Emotions" rather than just "Vocabulary."
But on the other hand, it makes me wonder: In this world of effortless convenience, what will ultimately remain of our own thought processes? If we stop searching for the 'right words,' do we also lose a bit of the critical thinking that comes with defining our own desires?
The Minimalism of Evolution: From Complexity to Transparency
In the past, "good UI" meant organizing complex search filters and categories into a neat layout. But the evolution of visual search suggests a different path: the most sophisticated UI is the one that disappears. When the camera becomes the primary input, the screen no longer needs a cluttered search bar or dozens of buttons. This is the "Minimalism of Evolution." We are stripping away the digital noise to focus on the physical subject in front of us.
As a designer and developer, I believe that as technology becomes more complex (like the backend of visual search), the UI must become even more transparent and minimal. I am currently in the UX/UI design phase of my personal project, 'To the Taste.' While researching and writing this post, I had a definitive 'Aha!' moment: incorporating a visual search feature is not just an option—it is a necessity.
In an era where users are accustomed to effortless convenience, 'Manual Entry' is a significant friction point that leads to user churn. Therefore, I have decided to implement a visual search function that automatically populates all essential metadata from a single label image. My goal with 'To the Taste' is to create an environment where the interface doesn't stand in the way. I hope to introduce my project here within a few months.
Reference
Schwartz, E. H. (2025, October 2). Google adds eyes to AI Mode with new visual search features. TechRadar. https://www.techradar.com/ai-platforms-assistants/gemini/google-adds-eyes-to-ai-mode-with-new-visual-search-features/
Comments
Post a Comment