The Evolution of Smarter Search: A Convo With Stefan Weitz

by Elizabeth King on September 21, 2011

barn owl South Florida rodent control

recipe fast sweet peas artichoke pasta

best tires minivan city driving

If you use the web with regularity, which you probably do, you’ve likely developed fluency in Ad Hoc Searchese, that loose, disjointed language we all use to tease out search results that suit our needs. Whether you’re searching for restaurant vegan Upper West Side brunch or cancer treatment homeopathy stage 3, your use of minced up English is ubiquitous.

The practice is born of necessity: even with your supposedly refined search phrase, you’ll probably end up searching more than once for the same information. Think about how often you refine or rephrase your queries when your search results are way off base. The struggle to find great search results echoes one of the most challenging aspects of learning in any arena: the effort to precisely describe that which you do not already know.

Search engines can only deliver when you perfectly describe what you don't know.Think about it. At this point, search engines can really only offer query results if you’ve perfectly described for the engine what it is you don’t know. I had a conversation this summer with the Director of Bing, Stefan Weitz, about education, intelligence, and search’s Big Brother hand in it all.

“…The key challenge of search is that it doesn’t understand language all that well. It’s a really hard problem to solve: it’s semantics. We’re trying to give machines the ability to understand all the colloquialisms and  idioms that we use—the different ways that we connect words and objects together—so that [those machines] can begin to understand and have those kinds of conversations with us.

Today, the machine is all based on math. If you punch in ‘cognitive neuroscience,’ you’ll get a bunch of results, but those results will [be based on] the simple math of how many times other words appear in the context of those other two words. So, there’s not a lot of ‘human’ behind those results. “

If you’re not already considering the significance of this, it’s time to start. Why? Search engines have become the de facto libraries of our age and largely drive our self-education. It’s not the way we used to learn, and the good news is that it won’t always be like this. We’re in a time of transition, and it’s imperative that we understand the nature of the transition so that we can move into the next phase of search without falling apart as learners in the meantime.

Any child of the ’80s or earlier probably recalls a class trip to the school library during which the librarian delivered a crash course on how to use the card catalogue. The catalogue really only offered three ways to search for books: title, author, and topic. For today’s students, though, the web is a cross-referenced card catalogue of the world—and it’s cross-referenced by every word in the language. There’s very little in the way of a definitive starting place for research; using the internet for information and education isn’t a title, author, topic practice.

As it turns out, your pidgin search queries will eventually become a thing of the past: search engines are working to offer a more natural, holistic approach to user’s information acquisition. Lessons on creating long tail search terms will (happily) become obsolete; the introduction to research as we know it will become a thing of the past.

Is Easier Search Better Search?

Sure, it’ll be easier, and easier is nice, but there’s much more to future search than ease. This more holistic, semantically driven search engine also means much broader access to information that’s both deeply meaningful and likely to promote consideration and further exploration. Instead of the frenetic experience of continually rephrasing search terms and hoping to find relevant facts, search is going to be bent on delivering ideas and solutions. In essence, the very same search that’s been accused of making us stupid [see The Shallows: What the Internet Is Doing to Our Brains] may actually begin to help make us smarter—and wiser.

“For those who have a hunger for wisdom…[this holistic, semantics-driven] search can help. In order to make a good decision you have to have as many facts or as many viewpoints as possible. You have to be able to look at multiple viewpoints, rationalize them, [and] maybe have some cognitive dissonance as you can’t quite make sense of these two things…but at least you understand the choice you’ve made,” says Weitz.

For those who have a hunger for wisdom, search can help. Stefan WeitzAs an example, Weitz offers a hypothetical story about a controversial issue, like deforestation or extinction. A holistic search engine, like those engines currently in development, will be able to find those outlier pieces of information that are relevant to our decision-making—like an article that holds a linchpin piece of information about rebuilding habitats—and it will bring that article to the querier’s attention no matter how inconsequential or repetitious the rest of the article is. Today’s search engines would leave that article in the dust, but in a few years that article will show up in results because it’s truly relevant.

If you’re searching with current technology,

“…your actually finding that little tiny piece of data that’s only mentioned in, maybe, one in a thousand articles is unlikely, whereas a search engine, [as it begins to] make these correlations and build these bridges between pieces of information, could begin to expose that to you. The magic of search in helping people get smarter or contributing to their wisdom is that it can begin to expose things and bubble things up that you wouldn’t necessarily have found otherwise,” said Weitz.

Still, to anyone interested in promoting critical thinking, removing the need to describe what one doesn’t know—potentially eliminating the need for deductive, Socratic inquiry—can be disarming. It’s the funny, beneficial underbelly of our ad hoc search language: it makes us work to find information. It compels–forces, even–to assess what we know and what we don’t know. In fact, using crummy search in its current form is probably good mental exercise

According to Weitz, in the coming years search will become so integral to our daily lives that we won’t even be saying “search” anymore.

“Remember how we used to have to “dial up” to the web to ‘get on’?” he quips. “Now you’re always on the web. It’s just what you do… and the same thing will happen with search. You’re always kind of doing it, whether you know it or not.”

So if the search engine is going to become so effortlessly incorporated into our lives and so semantically driven that it’s actually synthesizing ideas for us, where will we come in? What does this mean for thinking and teaching others to think?

“I think, for teachers, there will be less instruction around how to create a proper Boolean query…” says Weitz. Future searches will, “be much more of a ‘speak to it as you’d speak to anybody else’” experience for the searcher. “I think what’s going to become incredibly important in search for kids is not so much the act of searching–it’s  knowing what to do with the results once you get them.”

For the first time, you really will have information right at your fingertips—cohesive, connected, meaningful information. There won’t be anything standing between anyone with an internet connection and virtually anything he or she wants to know. Preparing students for that reality—a web experience that can serve to make them smarter—is the task at hand.

“You probably do hundreds of searches within a given week– you get all these different results, and [it’s most important to] actually know how to capture the ones that are most relevant and most interesting to you. What do you do with them? How do you store them? How do you tag them? How do you curate them? How do you link them to something else you saw previously to contribute to a larger understanding of a topic?

“Critical thinking… simply becomes more important as you have to work less hard to find data and services on the web. That’s, to me, probably the  most important thing teachers can teach six year olds right now. I know with my six year old, I’m teaching her that every single day.”

{ 1 comment… read it below or add one }

Jeff jacobs September 21, 2011 at 11:34 pm

Really never had considered “crummy search” as a form of good brain exercise. Like most of your posts, THIS has been a good brain exercise, as well. Thanks for a trip back in time to the library, too- can you explain the Dewey Decimal system, as well?

Reply

Leave a Comment

{ 1 trackback }

Previous post:

Next post: