Dr. Udi Manber has been a member of Twiggle's board from its early days. Udi is a computer scientist known for his work on search. He is also one of the authors of agrep and GLIMPSE. He actually started to develop search software in the late 80s when nearly no one was interested, but he was patient enough to wait 10 years when it finally paid off in a big way. He joined Amazon as Chief Algorithm officer in 2002, and has lead the search groups at Yahoo, Google and YouTube.
Amir: What defines good search?
Udi: Good search is search that helps people find what they need.
Amir: How should online retailers evaluate their search experience?
Udi: This is more tricky than it seems. The first instincts are usually to evaluate search based on sales. A search that led to sales probably gave people what they wanted. But there are problems with that approach. First, instead of volume of sales, retailers often optimize revenues or profits, and that often leads to search that promotes what retailers want to sell rather than what people want to buy. Soon enough people realize it, and go to a competitor. Any time you introduce goals into search that are different from giving people what they need, they will eventually lose trust. It is critical to find a balance.
In the pre-Google days, search companies optimized their search based on ad revenues (and other goals unrelated to the actual search). Google came along and based everything exclusively on what users needed. That was one huge reason that Google attracted so many users. Search companies that optimized their revenues for the short term lost everything long term.
We will still see incredible improvements in search. Search today will look primitive 20 years from now.
Amir: What most excites you about the future of search?
Udi: Search today is good for one fact at a time, one point in the knowledge space, so to speak. You have to collect all the needed points and draw all the lines yourself. I am excited about the possibility of search enabling genuine research. It will make it easier to connect facts, make inferences, and understand things in much deeper ways. For example, I can find prices and dozens of other pieces of data about products, but if I want to know why they are made in a particular way, what distinguishes them, how they differ from other products, and most important, will they work for me, that’s much harder.
Amir: What is the biggest misconception about search?
Udi: Many people believe that search is essentially a solved problem, and it just works. I have heard that sentiment since the early 2000s. We will still see incredible improvements in search. Search today will look primitive 20 years from now.
Amir: When it comes to search, you must have seen it all. What makes Twiggle special?
Udi: You and I have seen nothing yet. Twiggle is special because it does not rely on keywords and popularity alone, the hallmarks of product search, but it tries to actually understand both what users want and what products you offer.
Amir: Why is e-commerce search so difficult?
Udi: There are several reasons. First, expectations are high, because it seems easy. How hard can it be to find a medium size blue shirt with a pocket? (Neither Macy's nor Eddie Bauer, for example, can find any.) Second, the data is usually incomplete and unreliable. This is true for many types of information, but it is especially bad for product information. In many cases, the official product information has more about how great the product is and how it can solve all your problems, rather than any useful information about it (or even what it is). Not to mention the still widely-spread practice of naming products with cryptic long strings, making you decide whether the FJU65-RF/b is better for you than the FJV75-164X. You have to rely too often on users' comments, and not all of those come from unbiased users.
Amir: What frustrates people the most when they search?
Udi: When the search engine does not understand them or ignores their requests. It is one thing not to know the right terms to use, but when you give the correct terms and you still get different results it is very frustrating. In my experience this is still quite common.
Amir: What is the biggest mistake that companies make when they think about search?
Udi: By far the biggest mistake is that they don't think about search. Here's a simple piece of advice to anyone who runs a retail web site. Ask your search engineering team (even if it's only one person) to present you every day with 5 random queries that were asked the day before together with the results. It will take you 5 minutes to go through them every day, and you will get an invaluable understanding of what works and what doesn't work. Then you can devise much better tests, better dashboards, and some solutions. But that's a good start. Another mistake is to let marketing be in charge of search. I mean no disrespect for marketing, but good search requires independence from sales goals.