Searching for that needle in the haystack? There is help.

Answer honestly. How

often do you use search engines? I

know. That’s an incredibly broad

question. Sure, I use Google, MSN or Yahoo

to find information on web sites just like millions of other web surfers. But how often do you use Enterprise Search tools to find data relevant to performing your

job? That is, disparate data stored not

only on the Internet and your intranet, but across data and email repositories,

content management systems and databases to name a few. Admittedly, the closest I’ve come to using an

enterprise search tool is Google Desktop or our company’s intranet. Finding files on my workstation or links on

the Internet? Yawn. I want to see something that helps me find

that long lost box of CDs in my garage.

Okay, there is nothing that can help me find what I’m

looking for in my garage. (In fact, I

think my wife secretly threw the box away and gets great enjoyment from

watching me fruitlessly search for it.) So,

short of that miracle, there are some very intriguing advancements in search

technology that will become more important as the volumes of data which

companies store continues to grow.

When you stop and consider all the places your company

stores data, it’s daunting to think about how long it would take to search through

each individual location. Today’s

enterprise search tools can not only search every nook and cranny of your

network, but can search for data regardless of the format it’s stored. Tools such as Vivisimo’s Search Engine can search for

data within relational databases, PDF, ZIP and GZIP files, email, MS Word,

Excel, file servers, intranets. The list

goes on and on. Search results aren’t

returned in the common linear lists which make it difficult and time consuming

to sift through either. Instead, premium

search engines can offer results in clustered and conceptual categories. Take a look at some of the online demos by Vivisimo and you

immediately get that “Aha!” reaction about what should be the norm when you

execute a search.

Another major player, if not THE player in enterprise

search, is a company called Autonomy. It incorporates what it calls Meaning Based

Computing which takes unstructured and disparate data, and enables computers to

create meaningful relationships. This

goes beyond just performing a keyword lookup which only works to find all

documents with an exact match. Instead,

Autonomy also attempts to find results which contain the same or similar

meaning to what is queried and can execute searches based on pattern matching

and language translation.

Many corporate solutions don’t rely on only one search

technology. This is where federated

search systems come in. Federated search

takes one familiar front-end search box and passes the query to multiple search

engines. So, say, a search on your company’s

intranet may use Vivisimo or Autonomy technologies to locate data within your

network, while a “hook” on the backend may also return results using Google or

MSN. This could also have applications for

companies who wish to point search engines toward data repositories on company

extranets. For instance, a search for a company

purchase order would not only return results from your corporate CRM system,

but also the invoice stored on the outside vendor’s data storage system.

Yes, there are also enterprise search engines such as Fast Search & Transfer (FAST) which

cater solutions toward specific niche markets such as corporate regulatory

compliance, criminal investigation and litigation compliance protection. FAST search engines even have the ability to

search video and audio formats and return results based on closed caption text,

voice-to-text recognition and sound recognition.

Speaking of niche search engines, Splunk is aimed at IT analysts who have spent

countless hours searching for the source of an IT problem. Pretty much nailed all of you, didn’t

it? Splunk allows you to search all data

being “logged by any service, application or device within your data center,

regardless of source or format” in real time.

When an IT problem is reported, Splunk aims to reduce the mean time to

recovery through live reporting. Perform

a search for a specific error message or incident type and turn it into an

alert that constantly monitors for the event and notifies you when it occurs.

I am pleasantly surprised at the advancements being made in enterprise

search technologies. With the sheer

volume of data being collected and held onto by companies, it makes sense that

IT has rushed in to fill yet another void.

And since the main hurdle of how to index disparate data types seems to

have been conquered, the direction now seems to be focused on what to do with

the search results returned. This is why

so many of the companies mentioned also offer collaboration with Business

Intelligence tools to provide greater data analysis. This should be interesting to watch as the

market continues to mature. Now, if

someone could please help me with my garage…