Taxonomy & Content Classification: Market Milestone Report

A Delphi Group White Paper
Taxonomy
& Content
Classification
Market Milestone Report
Table of Contents
Introduction 1
Irrelevant Information and Infoglut 2
Context 2
Ambiguity 3
Browsing vs. Searching 3
Report Scope 3 Do You Know What You Know?
Evolution of Taxonomy Technologies 4
Is “Taxonomy” a Misnomer? 5
Using Categories in the Search Process 5 Introduction
Browsing Process 6
Taxonomy and Search 7 Cataloguing unstructured information is a chronic
Benefits of Taxonomy 7 problem, that if not adequately addressed can be
Taxonomy Software Integration 7 terminal for your organization. Today we have many
Market Survey 9
Survey Results 10 easy-to-use and accessible tools to create and publish
“Infoglut” and Knowledge Worker Efficiency 10 information electronically. Examples are: the
Current Software and Manual Systems Not Adequate 11 ubiquitous suite of Microsoft office products like
Enterprise Organizations Deomonstrate Interest 13 Word and PowerPoint, Adobe Portable Document Files
Definition of Terms 14 (PDF), Web Pages (HTML files), e-mail, news feeds, and
Taxonomy Market Landscape 16 the like.
Stages of Taxonomy 19
Vendor Assessment Reports:
Autonomy 21 Lack of information is no longer the problem—but
Convera 24 lack of time to correlate, categorize, analyze and act on
Entopia 28 the information is a crucial problem. The information
Mohomine 31 is there, hidden in reports and e-mails and published
Quiver 33 on the corporate Web site. We are placed in the
Semio 35
Stratify 37 position of being unable to find applicable and
Textology 40 pertinent information to make timely business
TopicalNet 43 decisions. This comes at a time when the agility to
Verity 45 quickly make fast, informed decisions is increasingly
Wherewithal 48 critical to survival and prosperity. As the volume of
Controversies & Pitfalls 52 opportunities increases, the duration of the time to act
Manual vs. Automatic 52
Maintenance and Dynamic Information 53 on each opportunity decreases. The information-
Librarians 53 based economy is in danger of drowning in a sea of
Directory Building vs. Hierarchical Categories 53 irrelevant, unstructured data.
Granularity of the Taxonomy Structure 53
Users Needs and Personalized Taxonomies 53 A new segment of software has emerged to help with
Speed, Accuracy, Robustness and Scalability 54
Future trends 54
the task of combating “infoglut.” For example, there is
End Note 56 software that enhances the performance of search
Vendor Contact Information 58 engines, text mining, natural language search
2
applications, ontology, summarization and the documents contained information about –

taxonomy. Taxonomy software correlates and chocolate chips, potato chips, buffalo chips, wood
groups unstructured information from a myriad chips, poker chips, an old TV series (ChiPS), or
of sources. Taxonomy software also helps us in integrated circuits. “Chips” is one of those words
the automation of this process. The software’s that have multiple ambiguous meanings like “java,”
fundamental challenge is to understand the “can,” “branches” and “boot” (noun or a verb).
concepts and ideas that group like documents
together and separate unlike documents. Think People can distinguish concepts from each other
of taxonomies as computer-generated card based on context and the specific meaning of a
catalogs that allow us to locate, retrieve and word as it should be applied to the situation.
cross-reference information in our digital libraries. Computers cannot. Other methods must be used
by computers to give us the results we want. But
Taxonomy software can reduce our reaction time before we explore these methods, let’s look at
to make informed and timely business decisions some examples of how people think versus how
based on knowledge and information contained computers “think.”
within the unstructured data of an organization’s
digital documents. This software helps us form Context
ideas from information we didn’t know we had
while revealing relationships and correlations The definition of context is that which surrounds,
that were submerged or lost in the depths of the and gives meaning to, something else. People
ocean of information overload. This happens on explore concepts; computers primarily search for
an individual basis and a community basis. We key words. Relevancy is entirely subjective to the
work in groups and we must be able to individual who is performing the search. Only
communicate with the constituents of the group each individual can judge how relevant a
so we can have actionable knowledge. particular bit of information is to what they are
Productivity comes from seeing connections, attempting to discover. The document may be too
evaluating importance, recognizing context, technical or out of date or too general for your
understanding the implications and understanding needs. Context is the determining factor.
the correlations of data and information. Machines can’t distinguish between “John Smith
to marry Mary Jones” vs. “Reverend Billy Graham
Although taxonomy software cannot stem the to marry Bruce Springsteen.” Only people with
tide of “infoglut,” it can help us find the the proper context can know that Reverend Billy
information we need to survive and prosper in Graham will perform the ceremony and not be
the new knowledge-based economy – to truly the recipient of Bruce’s ardor.
know what we know.
“Customers” may be a descriptor for an employee
Irrelevant Information and Infoglut in an HR document about medical benefits, but
“customers” may mean something different in the
An ancient Greek philosopher and ascetic, context of the sales department. If a given
Diogenes searched for an honest man and never document mentions “customers” where should it
fulfilled his quest. You may feel like Diogenes be categorized?
when trying to search for and retrieve
information from your enterprise portal or the In the above example about “chips,” you might
Web to make a business decision. have been looking for integrated circuit “chips”
and not a recipe to make chocolate “chip” cookies.
Lets use the example of searching for If you had only searched in a category such as
information about chips. A search on the Web for computers or electronics, you would have found
“chips” (using the Google search engine) fewer documents, but more precise and relevant
returned 2,430,000 references. Even if only 1% of information. If you have categories and
these documents were relevant, that is over hierarchical structures of information you will be
24,000 documents, which is way beyond the able to narrow the search field and find relevant
capability of most of us to wade through. Some of information faster.
©2002 Delphi Group • Ten Post Office Square, Boston, MA 02109-4603 • v(617) 247-1511 • www.delphigroup.com
Reproduction authorization required.
3
Ambiguity need is about flash memory “chips” It is much

easier to discover information about a particular
The beauty of the English language is that it has subject if you see it in the context of related
many words to describe the same thing. The information. Browsing encourages associative
corollary is that the same word may have thought. Browsing in categories can guide you
different meanings. “Chips” is one example of an through the information discovery process.
ambiguous word. “Java” is another. Java could be
an island, a cup of coffee or a computer Report Scope
programming language. “Kick the bucket” is
another phrase that could have multiple The objectives of this report are to:
meanings depending on the context. Consider the
sentence, “Joe kicked the bucket and the water • Inform about the current state of the market
spilled out.” The question is did Joe “buy the for taxonomy software
farm” (drop dead) or did Joe violently place his
foot in contact with a container of water. The • Explain with examples how these technologies
context of the surrounding phrases in the rest of are applied to real life business issues
the document will clarify this. So the question
becomes, how do you categorize the documents • Report the findings of a market survey of over
that has words like “chips” or “java” or phrases 450 organizations about taxonomy
like “kick the bucket.” • Understand how those organizations are
planning to implement taxonomy technology
Browsing vs. Searching
• Analyze trends of taxonomy software
To search effectively you must know the terms technology, development and implementation
you want to use before you see what is in the
collection of documents. Key word search • Describe the technological landscape of the
however assumes you know what you are looking taxonomy software market
for and that is often an erroneous assumption.
Knowledge workers are not always exactly sure • Compare the technologies’ similarities
what they are looking for but, “when they see it, and differences
they know what it is.” More than 25% of the day is
involved in searching for information on the Taxonomies
knowledge workers computer system. About 70%
of that time is spent browsing for information1.
75% of the people surveyed during a Yahoo Why Now?
market research project preferred browsing
to searching. The first question we have to ask is, “Why the
current high interest about taxonomy?”
From our previous example about “chips,” you
may not have known there were many types of Delphi Group’s research on user experiences with
computer “chips,” such as processor “chips,” corporate Webs reveals that lack of organization
application specific (ASIC) “chips” or memory of information is in fact the number one problem
“chips.” If there were categories of computer in the opinion of business professionals. These
“chips” such as: processor, ASIC and memory professionals may include the customer service
“chips”; you may find all the information you representatives, the sales team, the financial
2
Delphi Group Research
4
services professionals, the R&D engineers and Evolution of Taxonomy Technologies

the health care professionals that work in our
organizations. If these professionals are spending People are natural categorizers. We tend to group
25% of their time or more looking for similar documents into categories by conceptual
information, then this results in an opportunity subject matter. We have developed filing systems
cost and represents a runaway expense item in for our paper documents. Examples of these are
many organizations. Our ability to create found in every office. We have filing cabinet filled
information has substantially outpaced our with paper documents that are grouped together
ability to retrieve relevant information. Delphi by some system:
calls this information explosion “digital sprawl.”
The impact of this situation has dramatically • Categorized by type e.g. contracts, marketing
affected the way we work. collateral, or invoices
We now have many tools to aid in the creation of • Alphabetized by company or individual’s
electronic information. It is easy to create an last name
electronic report or presentation and convert
those to PDF files for easy viewing and sharing. • Arranged by date
E-mails are the ubiquitous mechanism for • Grouped by department e.g. legal, HR,
communication within business. An estimated sales, accounting
eight billion e-mail messages will flood U.S.
corporations each day in 2002 say leading We do this so that when we can find the relevant
industry sources. Proposals, resumes and information to complete the actions or make the
contracts represent other sources of decisions – in short to run our business. We call
unstructured documents. Portals, Web sites, this “actionable information.”
visible and invisible2, and intranets have also
made it easy to place and share information. Web We have structured databases to keep track of our
pages are increasing at the rate of 7 million pages transactions and customer lists. This data is in
per day.3 rows and columns that tell us who to contact, how
much to pay, how many we sold, etc. But this
Since it is easy to create, share and store structured information doesn’t tell us what will
information, the rate of unstructured likely happen or why. The “why” is usually buried
information is now growing exponentially and in our “corporate landfill” of associated e-mails,
turning into what one taxonomy vendor calls reports, or presentations.
knowledge asset “landfills.” There are 250
megabytes of information for every man, woman, The first step was search and retrieval software.
and child on earth.4 The percentage of Unstructured documents were “scanned” and
unstructured data to the total amount of data is indexed and key words along with their
estimated at 85% and is growing. One of the frequency of occurrence were placed into
defining challenges of this era of enterprise databases. Search engines were developed so that
computing is just this: How do we find the we could scan the index and find the documents
relevant and pertinent information to do our jobs and create pointers or hyperlinks to the
and make informed business decisions? The documents that contained the key words that we
answer is at once obvious and elusive. We must thought were relevant. The simple search engines
harness the computer to help fix the problem it and indexers soon developed into complex ones
has helped us create. that could support Boolean logic such as “and,”
2
Invisible refers to dynamically generated Web like
Amazon.com catalog pages, Wall Street Journal Archives. –
Invisible Web, Chris Sherman
3
UC Berkeley
4
UC Berkeley SIMS How much information.
5
“or,” “not” type of qualifiers. These functions were Is “Taxonomy” a Misnomer?

expanded to include “wild card” searches and
“fuzzy” searches like “near.” When sources were The word taxonomy evolved from the life
relatively limited these tools helped you find the sciences where a plant or animal is placed in a
information you wanted. single spot describing its hierarchical
relationship to other plants and animals. One of
Now, however, instead of returning too small a list the interesting ironies that developed during the
of relevant documents, the search engines return research phase of the project was the realization
too many choices, as in our previous example of that the description of this category of
searching for “chips” with 2,400,000 possible technology was somewhat misleading. Since we
documents. These are brain-dead lists that insult are talking about semantics and language here,
our intelligence. When Boolean logic searches there is an inherent problem with using the word
turned out to be too complicated for most people taxonomy to describe this type of technology.
to use, natural language query software helped When applied to digital information, taxonomy is
develop key word searches that reflected our a systematic classification of a conceptual space.
vernacular and intentions when searching for When categorizing a document, it could and
information. The same problem soon developed – might be placed in multiple categories depending
too much recall, and too many references, even on the context.
when ranked by relevancy percentages.
For example at a pharmaceutical company the
Another mechanism soon evolved, that of research scientists may place a particular
metadata, or “data about data.” With this document about a new drug in a category based
approach, when a document is created the author on its chemical composition. The marketing and
is prompted to list the creation date, title, subject sales people may want to place that same
matter, synopsis and a few keywords. Since document in a category called “competition.” The
authors are very close to the document as the same document can be classified into two or
creator, the relevancy and the consistency of this more categories by different groups or under
information were questionable. different contexts or conditions. ( You can see
where the analogy breaks down: scientific
The next step in retrieval technology was a taxonomies are fundamentally rigid, while
methodology called link ranking. The more the categorization is personal and subjective and
Web page in question is linked to other systems sometimes even arbitrary).
and searched by other systems, the more
probable it is important to more people. Each Using Categories in the Search Process
time a link is clicked, the click is recorded in the
database and the client is redirected to the proper People search for information in two different
site, creating a simple click thru monitor that ways. The first is the process you use when you
records the popularity of links. The flaw here is know what you are looking for. You know the
that this becomes a popularity poll or self- answer, now you need to find more information
fulfilling mechanism; the more your site is visited about the subject. Keyword search with Boolean
the more likely it is to be visited. logic and traditional search engines are good for
this type of approach.
The limitations of these various approaches were
recognized by a number of “information and Many times people don’t know the answer they
language” experts and a new generation of are looking for when they begin the search for
technology has emerged. Taxonomy software was information. In fact, as discussed above, they
originally developed to help speed the process of often don’t know what they don’t know yet.
creating hierarchical structures. Let’s use the example of doing research on
categorization technologies. Here is an
example of a possible hierarchy of categorization
information. Each of these categories can also be
called a node.
6
The methodologies used in the automatic process

Categorization Approaches
of categorization are likewise not something you
would know about at the beginning of your
Manual search. By learning more about the various
methods of automatic categorization (i.e., rules,
Disadvantages example learning, semantic analysis and
clustering analysis) you can better understand
Does Not Scale Well how each of the methodologies may be applied to
your particular situation.
Does Not Keep With Rapid Change of Info
The point we are making here is that when you
Inconsistency When Classifying started your quest, you were not thinking about
cost-justifying a software solution, nor were you
Classifiers May Find Work Boring aware of the various methodological approaches
to the problem. This information was discovered
Classifiers are Expensive Assets either during or after your search—maybe.
Wouldn’t it be better to have such information at
Advantages your disposal at the beginning of your project?
Scales Well... Visual, hierarchical arrangements of subject
categorization trigger associations and
Automatic relationships that are not obvious when searching
for keywords. This distinction is important and
implies yet another reason why categorization
Rules Based
can be critically important to the productivity of
knowledge workers.
Training by Example
Browsing Process
Semantic Analysis
As you browse for relevant information about
Clustering Analysis your particular subject, you will find three more
characteristics that describe the process:
Combination browsing is dynamic, interactive and iterative.
Browsing is dynamic. Information changes all the

As you start the search process for time. In today’s world, virtually any search on a
categorization software you soon realize there complex topic becomes a hunt for a moving
is one alternative, the purely manual approach target. For example, my search for “chips” today
to categorization. There are several reasons, now yields 2,490,000 hits—60,000 more instances
however, why an exclusively manual system of information than just a few days previously.
may not be as effective: the chief drawbacks are Versions will change, articles will be removed,
that human classifiers are expensive, information added, etc. Delphi’s research shows
inconsistent, and obviously not scalable to the that at least 10% of enterprise information
same degree as automated approaches. changes on a monthly basis. This is a
Although these considerations were probably conservative finding—Delphi Group believes that
not foremost in your mind when you started the dynamic, volatile nature of information
your search, they certainly can support your sources is the number one reason that knowledge
argument in favor of the purchase of a software workers have difficulty finding the information
system. they are looking for.
7
Browsing is an interactive process. As you To paraphrase a quote, “to search via a computer
navigate a well-designed interface to information, without a taxonomy system is like trying to find
you will automatically be directed to other your way around an unfamiliar country without a
relevant topics. If you search and browse through map.” Taxonomy helps delineate the conceptual
information about categorization software, for relationships that exist within and between
example, you will find reviews, analysis, white various topics contained in the multitude of
papers and commentaries with information unstructured data within various
about other technologies, companies or related enterprise documents.
topics of information that may be worth
investigating. The benefits are:
Browsing is an iterative process. Repeating the • Discovering information you didn’t know
process refines your focus while broadening your you had
knowledge. Accessing relevant information and • Avoiding duplicate efforts within large
interrelated ideas and concepts supports a organizations where independent groups
fundamental change in your activity—from “reinvent the wheel” over and over again
simply searching, to finding and discovering.
• Not repeating the same mistake
Taxonomy and Search • Reports are better prepared if the author
really expects to be read.5
One purpose of taxonomy is to aid in the retrieval
of relevant information. An intrinsic benefit of • Provide overview as well as details about
the hierarchical structure of categorization is that a subject
links and summaries of information are rendered • Demonstrate relationships
in the context of their unique “parent-child”
relationships. Relevant information is more likely • Reduce complexity
to be found when specific content filters are
employed. For example, if we had had a general Taxonomy Software Integration With
category like “computers” in our search for Other Applications
“chips,” we would not have wasted any time with
false returns from the “recipes” category. Taxonomy can impact many aspects of your
organization. As organizations implement
Benefits of Taxonomy various software solutions to manage their
knowledge assets, taxonomy can dramatically
Finding relevant information quicker is the key increase the effectiveness of such solutions. All
benefit, especially when it provides immediate the software features in the world won’t matter if
access to the right information that allows the they don’t facilitate “just-in-time” knowledge
user to take effective actions. Equipping retrieval. Software applications such as portals,
enterprise knowledge workers with the tools to content management systems, knowledge
make faster and better-informed decisions is a management systems, search and retrieval
strategic imperative in today’s economy. Jakob software, personalization software, data
Nielsen, the guru of usability, estimates that poor extraction, and data mining can all benefit from
classification costs a 10,000 user organization taxonomy. Many taxonomy solutions are sold
$10M annually. with Application Programming Interfaces to
integrate into these existing applications.
5
There is a product marketing manager that places a peach cobbler recipe in his market research reports. He places the
recipe in a relatively arcane section of the report. At the end of the recipe, he offers to whomever finds the recipe a bottle
of Don Perignon champagne. In his career, producing many reports, he never has had to buy the champagne.
8
Taxonomy Is Not Just For summer “monsoon” season in Arizona corre-

Enhancing Search sponding with an unusually dry summer in the
Northeast. Now you have a “why” to substantiate
Let’s apply taxonomy to other applications. a business decision to increase
Taxonomy technology is currently making search rain-related inventory in a desert region.
engines more efficient by, as we’ve seen above,
limiting a search to a subset of relevant topics. Determining Your Categorization
Browsing topics categorized by hierarchical
relationships will allow new insights and
Requirements
correlations when searching for information. These questions will help you quantify how a
Imagine the impact of applying this technology taxonomy system can be applied to your
to other applications that must access and organization. The primary strategic
manipulate masses of unstructured data. consideration for any firm is what kind of
resources to commit to the process and in what
One taxonomy vendor’s customer is a proportions.
professional services organization. This
organization is constantly developing proposals Suggested Questions
for its customers. Since the organization is
national in scope, at any given time there are • Can employees, partners and customers find
many offices developing proposals for many the information they want?
customers—often in the same vertical industry. • How are you currently categorizing your
The professional services organization has an information?
existing-proposal generating application. They
are now using a taxonomy classification tool • What is the diversity of the documents?
to discover: • How many documents will you categorize?
• Proposals covering similar industries that have • Are there isolated areas that have a minimal
been successfully presented to similar customers number of documents? (e.g. small intranets
• Sections of proposals that can be reused within the organization that don’t have enough
documents to develop categories for them)
• Research data from previous proposals that are
pertinent to the new customer • Will this number grow over time or remain
constant?
• The correlation between different proposal
patterns or approaches and the success or • What rate of growth do you expect—linear or
failure of the proposal exponential?
• Up-to-date info from Internet news feeds of • What file types and formats are the
particular relevance to prospective customers information stored in? (e.g., text, PDF,
PowerPoint, HTML, etc)
In another example, imagine categorizing un- • How volatile and dynamic is the information?
structured data for the benefit of your CRM Does it change hourly, daily, monthly?
application. You might readily identify patterns
and correlations that bridge seemingly disparate • When does it change?
occurrences that are in fact tied together by some • What is your official document publishing
previously unknown common factor. process and policy?
Finally, consider the case of a data mining appli- • What are the life cycle parameters?
cation that discovers, for example, that sales of • Who categorizes information now?
rain gear declines in the northeast while increas-
ing in Arizona. Data mining is notoriously good • Who sets up the categories?
at telling you the “what” but not the “why.” By • How available are they for this task?
browsing a taxonomy-enhanced knowledge base,
however, you may identify related e-mails or • How do categories get updated and expanded?
reports that point to the existence of an annual
9
Survey Summary
Market Survey:
Enterprises know they have a serious and
Results and Analysis growing problem with unstructured data, and
that the problem is dramatically impacting their
During the first week of February 2002, Delphi ability to make rapid and effective business
conducted an extensive survey of approximately decisions. Current systems are not adequate.
450 end user organizations on the subject of Organizations planning on developing a
categorization and taxonomy management. taxonomy strategy remain unsure about how to
Several dozen questions were asked in regards to do so. This presents a significant opportunity for
the business issues surrounding the evaluation, the technology providers seeking to fill this need.
planning and implementation of taxonomy
technology. Profile of Respondents
The survey of 450-plus respondents represents a
The objectives of the survey include: fair sampling of enterprise organizations, with
over half the organizations having revenues of
• Validate the extent of the unstructured data over $100 million. 73% are located in North
problem faced by knowledge workers in today’s America. The respondents were either executives,
organization IT, or LOB or had project management
responsibilities. Respondents’ role in determining
• Determine the relative importance of the taxonomy software was primarily as a sponsor or
business issues surrounding the retrieval of project lead, or involved in defining need or
information from unstructured data sources specifications.
• Understand the scope of the problem and the Survey Methodology
perceived impediments associated with job Individuals identified by Delphi’s analyst team
performance and unstructured data. were contracted directly and asked to answer a
series of structured survey questions. The survey
• Confirm the characteristics of unstructured format was primarily multiple-choice, with either
information sources in terms of size, volatility, single or multiple answers possible depending on
and language the question. Respondents were also provided the
opportunity to volunteer more detailed and
• Verify if there are classification processes and otherwise restricted answers (i.e,. “fill-in-the-
policies in place today blank”) which are quoted at various points
throughout this report. The results from the
• Ascertain if there are taxonomy software survey are tabulated and graphed in the sections
projects underway or pending, their relative to follow. Listed below are the profiles of the
importance, and the proposed budgets for respondents, the questions, the results and,
implementation and maintenance of taxonomy finally, Delphi’s analysis of this market survey.
• Find out who will be responsible for defining Survey Limitations and Risks
and then maintaining the taxonomy software The survey resulted in over 450 respondents.
While this number is sufficient to develop quan-
• Clarify how the taxonomy software should be titative and qualitative trends, variations may be
configured and deployed found within individual deployments or taxono-
my initiatives. While respondents represents a
• Discover to what extent the market recognizes valid cross-section of enterprise-class organiza-
the leading providers of taxonomy software tions, the population’s preestablished interest in
this or similar technology may distinguish this
group as more knowledgeable and aware than a
similar group of randomly selected enterprise
respondents.
10
Question #2 – “How much time do you spend

Survey Results each day searching for information that is
“Infoglut” and Knowledge Worker critical to your job performance?”
Efficiency
Time Spent Looking
A number of questions were asked to validate the
extent of infoglut faced by survey respondents
and to verify the environmental causes.
“Infoglut,” the business problem of too much

information, is real, growing and recognized as
an important issue in today’s enterprise. The 2-4 hours
exponential rise of unstructured data is having a 37%
major detrimental impact on the efficiency of
enterprise organizations. Most respondents to
this survey (with job descriptions such as
executives, IT management, Line-Of-Business
managers and project managers) spend more han 2 hours
than two hours a day (25% or more of an 8-hour 49%
day) searching for information to perform their 4-6
jobs. More than 60% agree that finding 1
Greater than
information was a difficult process, and much of 6 Hours
2%
the time they cannot find the information they
need to do their jobs. More than 60% of the respondents agreed or
strongly agreed that finding information was a
Question #1 – “Finding the information I need to difficult process and over 50% of the respondents
do my job is difficult: were spending 2 or more hours each day
searching for information. This is consistent with
agree – disagree” Delphi direct client experience and various other
surveys, which of shown knowledge workers
Finding Information is Difficult typically spend 20% to 30% of their time
searching for business-related information, the
Strongly majority of which is stored electronically and
Agree
should otherwise be easily identified.
Strongly The issue of “search time” is one of the

Disagree fundamental symptoms of infoglut, and is at the
2%
heart of how most organizations measure the
business impact of taxonomy management. While
the positives impact information accessibly on
Disagree productivity is at times overrated (i.e., simply
13%
because information is accessible is no guarantee
that work is being done), there is no denying the
negative impact resulting from the lack of
e
information. Simply put, very few individual are
able to generate business value through the act of
N searching for information – this frequently
required task is what economists refer to as a
“transaction cost” or otherwise a drain on
productivity. And it is what the rest of us would
call “a waste of time.”
11
Question #3 – “The biggest impediment to frequently sited impediment to finding

finding the information I need to do my job is:” information. Over 95% of the respondents
reported that more than 10% of their documents
change on monthly basis. In a separate series of
Impediments to Find Info questions, more than half (59%) stated that they
have over 50,000 documents in the corpus of
information contained in their environment.
Unavailable
Compounding the problem, the rate of
information volatility is certain to accelerate.
This study shows that respondents have access
to at least 50,000 documents within their
Don't Know immediate organization— with often 10,000 or
15% ta Changes more of those documents changing on at least a
35%
monthly basis.
Question #4 – “What percentage of
No Skills unstructured data changes at least monthly?”
3%
Monthly Document Turn-Over

ccess
% 4%
51-75% 26-50%
As respondents looked and searched for

information, 61% of the time they had a 75%
chance or less of finding the information they
needed. The two main impediments to finding 10% 5
the information they were seeking:
25% of
ernal Documents
• bad tools 28% of the time
ange Monthly
• volatility of the data 35% of the time.
The two most frequently cited reasons for this are

“Bad Tools” and “Data Changes” or the concern
that information is changing too fast. Even with
the wide array of available search tools and
content management solutions, knowledge
workers either can’t find the information they
need to do their jobs, or they are spending an Current Software and Manual Systems
inordinate amount of time looking for that Not Adequate
information. Enterprise organizations are looking to
taxonomy software to help solve this problem.
Document Turn-Over and Data Change Most of the organizations surveyed had some
Another area of focus for the survey was an type of system for classifying information
examination of information volatility, consistent within their organization, but most had no
with the finding that “data change” was the most formal policy for tagging the documents and
left the classifying up to the author. The survey’s
12
central message—that finding information is “Question #6 – What type of system for

extraordinarily difficult and time-consuming – classification of documents?”
point to the fundamental inadequacy of current
approaches. Type of System for Classifying
It’s hardly surprising, then, that even where

Don't Know
there is some type of software in place to help 9%
with classifying unstructured data, most (over Other
55%) respondents felt it was necessary and 5%
important to put a taxonomy software system in
place. Over 90% plan to have a taxonomy
strategy in place within the next 24 months.
The objective of this set of questions is to

understand if there are systems currently in
place for categorizing documents and who
within the organization is responsible.
Question #5 – “Does your organization provide

a system for classifying the information you
work with?” are
%
System for Classifying Documents
Don't Know For a slight majority, 55%, there is some system

3% for classifying documents used in the
organization. Although not a true taxonomy
Pen
system, over half were using some type of
16 software or a combination of manual approaches
and software for classifying documents.
In contrast, only a third of respondents had a
system in place for tagging documents.
Tagging System for Documents
Ye
31
Don't Know
4%
13
Enterprise Organizations Deomonstrate Question #8 – “How important is taxonomy

Interest; But Unclear on Timing and software to your organization’s business
How To Implement Taxonomy strategy?”
Taxonomy Important to Business Strategy

This next set of questions explores specific plans
for implementing taxonomy software within the
respondents’ organizations.
Very I
nt
Question #7 – “Is anyone in your organization
working with taxonomy software today?”
Over 75% of the respondents either don’t have or
Imperative
Working With Taxonomy Software 7%
Don't Know
16%
at
No 8%
3%
Question #9 – “When will your organization

develop a taxonomy management strategy?”
Don't K
22%
Time Frame for Taxonomy Strategy
don’t know if they have taxonomy software being Next 6 nths

16
used or proposed within the organization. Yet
55%—a clear majority—feel it is important to
have a taxonomy system.
Already
The time frame for 90% of the respondents is to 12% -18 Months
have a taxonomy strategy within the next 2 years 13%
or less, indicating the likelihood that many
organization will follow-through on the priority
placed on taxonomy, as indicated by Question #8.
The delta between those who indicate someone Neve
working with taxonomy software and those who 10%
have an enterprise strategy in place (about 2:1
hs-2yrs
respectively) indicate that the deployment of %
taxonomy management software is still very
much in the experimental or pilot stages, a notion 17%
consistent with other finds throughout the study.
14
There are a number of taxonomy-related issues

that the enterprise is unclear about. This is Definition of Terms Used in
shown by both the diversity of answers and the Discussions on Taxonomy
preponderance of “I don’t know” responses. The
data from this survey shows that organizations
are particularly uncertain about:
Recall, Relevancy and Precision
• Who will be responsible for determining Recall is inversely proportional to precision. The
enterprise strategy for choosing, implementing more precise you are about defining what you
and maintaining taxonomy software. want to look for, the fewer documents are
recalled. Using our “chips” example, 2,430,000
• The costs of acquiring or maintaining the documents found represents “recall,” answering
taxonomy software. (This is especially the basic question of “How much is out there?”
troublesome given that most respondents The subset of 24,000 documents indicates
have a role in determining needs and/or relevancy. The 10 documents found regarding the
specifications for the software.) newest Intel processor reflect precision. The
ultimate measure of a taxonomy’s success or
• The topology of the vendor landscape. No failure, precision addresses whether or not you
vendor is recognized as a leader in this found the answer to your question.
technology segment. The vast majority of
respondents could not answer that question Hierarchies
nor could they identify any company in this
space in any consistent way. This obviously As discussed before, building hierarchical
translates to a significant opportunity for one relationships is something that humans do
or more vendor organizations to take on this inherently. Every piece of information we gather
leadership challenge. gets placed in a virtual cubbyhole in our brain.
That information is not useful until we can relate
it to something else. Since many of us are visually
oriented, the process of building graphic
representations of hierarchical relationships
helps us build those relationships faster, allowing
us to more readily envision how items are related
to each other. Hierarchies also allow us to see
both the big picture (with just the main topics),
and the details
and Top Tier
relationships
contained in
each “nested” Subtopic 1
subtopic.
The number of
nested levels Subtopic a
varies greatly
according to
subject matter, Subtopic i
user preference,
and the built-in Subtopic a)
parameters of a
given system.
The question of Subtopic 2
how many
15
levels of information are either necessary or Controlled Vocabulary and Thesauri

pragmatic is open to debate among systems
designers. A controlled vocabulary is a finite set of terms.
The typical application for a controlled
Bottom-Up or Top-Down vocabulary is for lists of allowed values in
structured record systems. A thesaurus is a
Methods that are used to build categories vary. particular type of controlled vocabulary that
One approach is to take the details and start represents a formalized description containing a
placing them in “buckets.” This approach is finite set of terms and relations between terms,
termed “Bottom-Up.” Returning to our “chips” and frequently also contains information on how
example, a Bottom-Up approach would start by the terms are to be applied. The following topics
separating a subset of documents into either the are included in a thesaurus:
integrated circuit category or the food category.
If you run into too many documents falling into • preferred terms
different kinds of food categories, you may, for
example, need to divide that group into both • non-preferred terms
potato chips and chocolate chips. The Bottom-Up • semantic relations between terms
approach thus expands categories iteratively as
part of the classification process. • how to apply terms
The inverse process is called “Top-Down,” and is Thesauri are usually in vertical industries such as
how people traditionally categorize. In this legal, medicine, and pharmaceutical. An example
approach, domain experts analyze the subject might be a music thesaurus: Soprano: Broader
matter at hand, and determine that the terms = vocalist, singer; Narrower terms = lyric
documents will fall into, for example, 10 general soprano, coloratura soprano; Related terms =
topics. Each document is then examined and placed mezzo-soprano, treble.
them in its appropriate, preexisting category.
Example-Based
Clustering Another approach to classifying unstructured
Clustering is a technique for partitioning data is to develop a subset of documents that
documents/words into subsets of similar preestablish categories defined by a set of
documents/words based on the identification reference content. These “training sets” can be
of common elements between the documents/ automatic or supervised. The software analyzes
words. Each document can be considered a new documents in comparison to the training set
“bag of words.”; clustering essentially groups and searches for similar concepts and ideas. This
the similar words contained in each bag. approach is also referred to as “machine
learning.”
Pattern Matching
A limitation of the example-based taxonomy
Pattern matching is the process of looking for method is that the resulting classification is
groups of words that are often grouped together. totally dependent on the breadth and precision of
One example is “Business Unit Manager” as a title. the training set.
(Note: this is also a “noun phrase” that could be
processed using semantic analysis.) Other Neural Networks
recognized patterns include frequency of words
used in a document, placement of words, Artificial Intelligence as applied to a computer
proximity of words to each other, and clusters of system is modeled after the neurons (nerve cells)
related words. Pattern matching is inherently in a biological nervous system. A neural network
language-independent. is designed as an interconnected system of
processing elements, each with a limited number
of inputs and outputs. Rather than being
programmed, these systems learn to recognize
patterns. Neural networks are an information
16
processing technique based on the way biological There are a number of algorithms that
nervous systems, such as the brain, process technology vendors customize, optimize, combine
information. Composed of a large number of and patent in order to categorize digital
highly interconnected processing elements, a documents. For a variety of reasons each vendor
neural network system uses the human-like has chosen a particular algorithm method or
technique of learning by example to resolve combination of methods. This is a list of each of
problems. The neural network is configured for a the methods that are discussed in more detail in
specific application, such as data classification or the following sections:
pattern recognition, through a learning process
called “training.” • Rules-based
Spiders • Bayesian
Spiders are automated processes used to feed • Linguistic and Semantic
pages to data extraction and parsing engines. It’s
called a spider because it “crawls” over the data. • Support Vector Machine
Another term for these programs is crawler. • Pattern Matching and Other Statistical
Algorithms
Taxonomy Market Landscape • Neural Networks
To help understand these methodologies,

Segmenting the Market consider the example of the various types of
engines used by car manufacturers today. There
The market for classification software is are 4-, 6-, 8-, or 12- cylinder internal combustion
exceptionally dynamic. It is evolving and gasoline engines. There are also diesel, electric,
changing on a daily basis. An overview analysis hydrogen or natural-gas powered engines.
like this one, therefore, can at best hope to take a Increasingly, a car’s power plant might be a
snapshot in time. There are many companies hybrid of any of these. Some of these engines are
supplying this software. The companies more suitable for the race track than a commute
discussed in this section are market leaders and to work, but all will get you to the grocery store
are fairly representative of the myriad of and back. These engines have many pros and
approaches and technologies offered within the cons associated with them, and their use depends
market. The organizations featured in this on a design perspective and what performance
section are listed alphabetically, and chose to characteristics the engineers want to provide to
participate in this project with Delphi. the users.
Methodology Algorithms Similarly, there are many approaches to building
and populating a taxonomy. No one method or
This section will deal with the technologies even a combination of methods seems to yield
underneath the technologies. As you will see particularly superior results. Many of the
there are many approaches to tackling the companies profiled below are very technology-
problem of building automatic or centric, and spend much of their marketing effort
semiautomatics taxonomies. We will examine trying to convince us of the advantages of their
each of the technologies from an overview approach or methodology. The bottom line is to
perspective and talk about them in more detail as understand how these differences affect system
we examine the individual products that utilize performance in the only environment that
them. These technologies are based on various matters—your unique data environment.
algorithms using statistical analysis, semantics
and neural networks.
17
Keyword Because of the proliferation of unstructured data,

the problem we now face is not finding information
Let’s start with a brief analysis of search and about a particular subject, but finding relevant
retrieval as we know it today (that is, without an information. In a recent Dartmouth College study,
accompanying classification solution). We will it was found that one in five or about 20% of all
then have a basis for understanding how other Web pages are less than 12 days old. When you
technologies try to improve on its weaknesses. consider that Internet spiders take 3 to 4 weeks to
index information, the implication here is that you
Search engines look for keywords in the title, are automatically by default of the process missing
synopsis or abstract, body of a document, or the 20% of the most current data.6
meta-tag (i.e., “data about data”) section. A program
“crawls” through the subject document, and each Keyword search systems are actually a subset of
instance of a keyword is put into an indexing rules-based systems.
database that describes how many times it found
that particular word and where that word was Rule-Based
located within the paragraph, page or document.
The “crawling” software looks for unique words but An example of “rule-based” taxonomy could be
does not include prepositions and conjunctions that all documents that include the terms “San
such as “to,” “in,” “or,” “and,” etc. Francisco,” “Chicago,” “New Orleans” be listed in a
category called “Cities, USA.” The rules break
The results are analogous to a book index down when an example like “Cambridge” is used.
alphabetizing all the words in the book. This is Is this Cambridge in Massachusetts or in England?
obviously not the most efficient way to locate
pertinent information—you may as well just read Taxonomies were developed to aid in the search
the book itself. for information. This has been done manually in
libraries for hundreds of years. Various systems
Advanced functions of search engines, however, were developed to apply particular “rules” on
allow one to apply Boolean logic to the search how such taxonomies have traditionally been
process. For example, multiple words can be built. Rule-based taxonomy classifies documents
searched by looking for instances where all the based on the existence or absence of pre-above.
words are present, or where some words are present Rule-based classification requires experts to
and others are “not.” create and maintain a rule for a document to be
included in a given category. Experts organize
Some search engines also “stem” the word. concepts into categories using “If-Then” rules.
“Stemming” is process of extracting the root of the These rules can support complex operation and
word and ignoring plural versions or other decision trees and are very accurate. Rule-based
modifications of the word. For example, the root of systems have their supporters, then, because
jumped, jumping and jumps is jump. these systems can precisely define the criteria by
which a document is classified. The rule
Accurate spelling and unambiguous terms (not like measures how well a given document meets the
our “chips” example) are required for keyword criteria for membership in that topic.
searches to be successful. The correct spelling
requirement applies to the document as well as the Besides the content of documents, rules can be
search criteria. applied to metadata and even business policies:
for instance, a rule might specify that only PDF
documents created since January 2000 should be
6
Mohomine
7
Verity http://www.verity.com/techbuzz/content.html
18
included in a particular category. Thus rules are c) Semantic and Linguistic Clustering
a powerful and flexible means for automatically
classifying content based on not just content Semantic analysis depends on a particular
itself but the metadata that describes the language and dialect. Documents are clustered or
content’s business context.7 The down side of grouped depending on meaning of words using
rule-based system is that expensive human thesauri, custom dictionaries (e.g. a dictionary of
domain experts have to write and maintain the abbreviations), parts-of-speech analyzers, rule-
rules. Other examples of rules are source of based and probabilistic grammar, recognition of
document, age, size and document type. idioms, verb chain recognition, and noun phrase
identifiers (e.g. “business unit manager”).
a) Statistical Text Analysis and Clustering Linguistic software also analyzes the structure of
the sentences identifying the subject, verbs and
This technology observes and measures co- objects, like you did when you first studied
occurrences of words. For example, “Java” used in grammar in grade school. Then sentence
connection with Starbucks probably relates to a structure analysis is applied to extract the
document about coffee instead of a programming meaning. Stemming or reducing a word to its root
language. Relative placement of words is also helps linguistic or semantic clustering.
important. Words in the first lines of a document
are likely more important than information d) Support Vector Machine
contained in the copyright section. Statistical
analysis and clustering also look for word Support Vector Machine (SVM) is a refinement of
frequency, placement and grouping, as well as the taxonomy-by-example. These algorithms are
distance between words in a document. Pattern derived from statistical learning theory. SVM’s
analysis improves precision by resolving calculate the maximum “separation,” in multiple
ambiguous or multiple meetings. dimensions of one document from another. Each
document—essentially a collection of words and
b) Bayesian Probability phrases that together have meaning—can be
represented as a vector. The direction of the
The Bayesian approach attempts to learn the vector is determined by the words (dimension) it
probabilities of words for a given category. An spans. The magnitude of the vector is determined
example of Bayesian probability applied is that if by how many times each word occurs in the
a given document contains the words “apples” document (distance traveled in each
and “oranges” it is more than likely this dimension).8 As this iterative method
document is about fruit, which leads to the continuously analyses documents, it separates
assumption that other fruit nouns such as them into either the “relevant” side or the
“grapes” or “tangerines” will occur. “irrelevant” space. By repeating the process it
categorizes those documents that are “relevant”
Applying a Bayesian algorithm sorts documents into like categories, but more importantly learns
by examining the terms, words and phrases how they are different.
contained therein. Bayesian probability uses
statistical models from words in training sets, Combining Methodologies
and uses pattern analysis to assign the
probability of correlation. This is one of the more Of course, no single taxonomy methodology,
common methods applied to building categories algorithm, or technology is superior to another
and taxonomy structures. for every possible application. The trend by more
and more taxonomy software companies is to
combine multiple methods to categorize the
corpus of documents to increase the accuracy
and the relevancy of grouping similar documents.
8
Mohomine White Papers
19
Stages of Taxonomy Process

There are four stages of the taxonomy process. To
help clarify this process, consider the example of There are a number of subtleties that are not
a traditional book library. readily apparent when first reading through the
descriptions of how these various taxonomy
1. Development of the taxonomy structure—the software solutions operate.
equivalent of the Dewey Decimal System
One major distinction often lost on the first time
2. The process of categorizing the content and investigator is the differentiation between
placing the pointers to the documents in the building the taxonomy structure and then
hierarchical structure—analogous to putting populating the resulting hierarchical tree with
the books on the right shelves pointers to the source of the information—in
other words, “populating the tree.” These are two
3. Presentation of the information or the interface separate and distinct operations and one should
that helps you find the information—the card not assume that these processes are happening
catalog would be the library equivalent simultaneously or concurrently. Rather, these are
serial steps.
4. Monitoring new input and maintaining
knowledge assets—analogous to the daily work Below is a diagram showing the general process
of librarians; classifying new books, writing up or workflow of taxonomy software as it
new card catalogs, updating periodicals, and categorizes unstructured data. Vendors have their
putting returned books back on the proper variations on this, but this schematic will give
shelves. you an overview of how the software basically
works. Some vendors leave out steps and other
add steps in between these depending on the
design approach and architecture of the
taxonomy process.
Process concepts to Develop

Select data to use
Spider data to develop clusters of Taxonomy
as basis of taxonomy
extract concepts related concepts Structure
structure or documents
Classify corpus of data Display hierarchal Administration

Develop and to populate taxonomy for modifying
structure. This meta
submit training sets structure and provide
data is usually in structure, renaming
to structure links and pointers
to data XML format. and linking
20
API or Standalone Application The companies reviewed here run the gamut
from Mohomine—which markets just a
As with many new technologies, a subset of the development framework for the OEM market—to
vendors have developed standalone applications Quiver, which markets an application suite as a
that come complete with end-user browser-based complete end-to-end solution for the enterprise.
clients and MS Windows-based clients. Users Most companies offer both approaches, and
and/or administrators can point the software at depending on their orientation emphasize either
the body of documents to be classified on hard a strongly developed GUI as an application or a
disks, servers, intranet sites, portals and Web more extensive API for integration.
sites. The taxonomy engines residing on servers
perform the categorization process and usually Company and Product Profiles
populate a database of metadata. The next section details the companies that
participated in this project with Delphi Group.
Taxonomy software can be an enabling The company sections are organized in the
technology. It aids and enhances other following way:
applications that the user interacts with through
a GUI. Most of the vendors support this concept • Introduction – short description of the
and offer an API to integrate into other company and its origins
applications. Since many enterprise
applications are custom-built this is an • Technology approach – a basic description of
important consideration. the underlying technology of the software
A number of vendors view this technology as • Product – a description of the functions and
eventually being an even more fundamental features of the taxonomy product
component of the information infrastructure.
Just as relational databases are a fundamental • Vision or customer study – a statement by the
infrastructure component of applications such as executives of the technology company relative
accounting, CRM, and other enterprise to where they see the market heading or an
applications, taxonomy software will be the example of a taxonomy implementation
infrastructure component that correlate • Assessment – a short evaluation of the
unstructured data. This design philosophy technology and product approach by the
positions taxonomy software as a core module in Delphi Group
a suite of products that work on the unstructured
data within an organization. The companies listed in this section are arranged
alphabetically based on the name of the
Another series of vendors mentioned in this organization. This arrangement does not imply a
report have added and tightly integrated the ranking or endorsement by Delphi Group.
functionality of taxonomy into the search and
retrieve application. Here are various ways we
can group them:
• Application With GUI

• API Integration OEM
• Integrated Within Search Applications
21
presence or absence of individual words doesn’t

Vendor Assessment Report: significantly change the probability of the idea
Autonomy being expressed. This makes matching in the idea
space more reliable and accurate than in the
word space.
Introduction
Because it does not rely on key words, it can work
with any language independent of slang or
Autonomy does not require a mutually exclusive regional variations. It treats words as abstract
decision between automation and manual symbols of meaning, deriving its understanding
control. Combining automatic processing with a through context of their occurrence rather than a
variety of human controllable overrides removes rigid definition of grammar.
Autonomy from the purely automatic category of
taxonomy vendors. Users can add rules, rename Autonomy’s Classification Server is part of a
or modify the taxonomy structure to suit their wider infrastructure, the Intelligent Data
needs. Autonomy’s holistic approach to resolving Operating Layer (IDOL). IDOL provides an
the business issues associated with all types of integrated platform that integrates content
unstructured data is one of the reasons the through an understanding of it. Because
company is growing and profitable. Autonomy’s technology understands the content
it can do much more than simply classifying it.
Autonomy is one of the veterans in the business IDOL allows content to be cross- referenced,
of developing a software infrastructure to retrieved, classified and compared with other
manage unstructured data. Their enterprise content. In addition, it profiles users based on an
corporate customers are companies such as understanding of the content they are working
General Motors, PepsiCo, Lucent, Reuters, and the with. These profiles of users are technically
U.S. Department of Defense. They were founded identical to profiles of documents or taxonomy
in 1996 and have been an OEM supplier of subjects, meaning that anything that can be done
taxonomy software to vendors like BEA Systems, with content can also be done with users’ profiles
Business Objects, Sybase and Vignette who have (e.g. a taxonomy of users and their interests can
embedded Autonomy’s core technology into be created as well).
their products
The IDOL infrastructure does two fundamental
things: aggregates content from all repositories
Technology Approach regardless of format or type (such as speech,
video, text, XML etc); and builds a mathematical
Autonomy’s strength lies in a unique conceptual understanding of each piece of
combination of technologies that employ content for later use. It also allows each item to
advanced pattern matching techniques have an unlimited amount of metadata (source,
(nonlinear adaptive digital signal processing), date, author, subject heading, etc) to assist in later
utilizing Bayesian Inference and Claude manipulation. A subject understanding can then
Shannon’s principles of information theory. be created by example from one or more pieces
Bayesian inference is a way of inferring an idea of content, or explicitly by sets of rules linked to
from a set of evidence, and Shannon’s theory metadata or keywords. Each of these would be a
allows you to rank the importance of ideas. In the taxonomy element, or category, which is indexed
case of text, for example, the words ‘black’, ‘white’, into a server. As new information comes in, it is
‘sea’, bird’, ‘flightless’ being present provide matched against the server’s conceptual index
strong evidence of the idea ‘penguin’ being (while still using all legacy boolean constructs
expressed. Each word on its own is a fairly weak and keywords a user may wish to define) as one
indicator of the idea to the extent that a single operation (a highly scalable operation – each
word is ambiguous (even the word ‘penguin’ is document in effect ‘queries’ the categories), and a
ambiguous – English chocolate bar, Batman decision is made based on the most closely
character, Italian air conditioner, book publisher, matching categories. This enables the user to
bird). The strength of this method is that the exercise whatever level of individual control they
22
may want over a taxonomy component, while Clustering

allowing them to make it a completely automatic
process. The Classification Server delivers the ability to
automatically cluster information. Clustering is
Product the process of taking a large repository of
unstructured data, agents or profiles and
Autonomy Classification Server provides a automatically portioning the data so similar
completely automated access to all of the information is clustered together. Each cluster
information within an organization. It aggregates represents a concept area within the knowledge
the information from its respective repositories base and contains a set of items with comm
while respecting all of the underlying security on properties.
and organizes it into a taxonomy, which is either Autonomy has two types of automatic clustering:
manually defined or automatically generated, or
a blend of the two. Autonomy’s infrastructure 1. Hot News Clustering – identifies the main topics
technology deals with a static starting point of of information present within the knowledge base
information as well as automatically dealing with
2. Breaking News Clustering – compares clusters
ongoing changes in the information. In addition,
from the previous periods and compares them
it identifies new threads or themes of
to the current one. By identifying new clusters
information.
allows the automation of breaking news.
Autonomy Server had numerous modules:
Cluster Mapping
• Automates categorization, cross-referencing,
hyper-linking and presentation of information The ability to visualize clusters is aided by two
applets that identify the relationships between
• Dynamic updates: can be retrained on the fly information clusters in one time period or
eSummarize documents and recommend between successive periods and sets of data.
related article via hypertext These are displayed as either a spectrograph or
a 2D cluster map.
• Provide an easy-to-navigate visual interface for
searching providing a unified view.
After structuring the data into taxonomies, the

Classification server tags the data automatically,
adding relevant metadata tags based on an
understanding of the content. Using these meta-
data tags a hierarchical tree can be generated
pointing to the relevant documents.
23
Customer Quotes Assessment

“There is a lot of useful information out there, Autonomy is one of the veterans in the taxonomy
but the key to using it is to first make sense of it. software market segment. Working daily with real
This entails understanding the content, sending it problems encountered at their installed base of
to the right person and linking it to other over 500 enterprise customers, gives Autonomy
pertinent information and people. Speed is the pragmatic experience of making taxonomy
another critical factor, thus all this should happen technology work to solve real business problems.
in a non-labour-intensive manner - that is, Their experience in the enterprise is reflected in
completely automatically. That is what Autonomy, the integrated support for over 28 different
uniquely, now enables us to do. It is the engine of applications with security-aware plug-ins.
our KnowledgeNavigator.”
Realizing that building a taxonomy structure and
Jean-Pierre Krause, of Zurich Risk Engineering classifying unstructured data is only one of the
Group’s Head Office tools that will help businesses deal with the ever
increasing amount of unstructured data
“Autonomy enables our software engineers and generated within and from outside the
contractors to retrieve accurate and personalized organization. Autonomy also applies the same
information, which helps them design our earth- technology to understanding the expertise of
and star-observing platforms. The amount of individuals within the organization. Any
information in our organization is akin to the enterprise that chooses Autonomy as a
number of stars in the universe – and we technology provider for understanding the
encounter more and more each day. Autonomy’s capabilities and expertise of individuals within
technology automatically processes growing the organization, will be working with a company
volumes of data and easily integrates with other that not only knows what it knows, but that
products. The implementation has been smooth recognizes how to apply what it knows to real
and straightforward throughout.” business issues.
Steve Nauss, associate head of NASA’s Computing

Environment &Technology Branch for Goddard Space
Flight Center’s Information System Center
“Novartis is a world leader in pharmaceuticals,
healthcare, agribusiness and nutrition. That
means we handle a vast amount of complex
information daily, both from internal and
external sources. Until now, that has cost us a
great deal of time and money in manual
processes, but Autonomy ’s technology can
automate the whole process from start to finish. ”
John McCulloch, Manager, Executive

Information System, Novartis
“Autonomy is a very powerful solution. It’s easy to

maintain, requiring little to no maintenance.
We’re now able to focus our resources towards
growing our business while saving time and
money.”
Bill Gafney, Web Specialist at Boston.Herald.com
24
Convera’s approach also means that

Vendor Assessment Report: organizations need not rely upon the vagaries
Convera associated with authors keyword-tagging
documents for allocation to particular categories.
Authors may well understand a particular subject
Introduction but often lack “the big picture” when it comes to
how a particular piece of information relates to
As an industry pioneer, Convera has been other topics. Convera’s categorization approach
creating technologies for the search and retrieval frees authors and organizations from this burden
of text-based information for more than 20 years. by providing virtual domain expertise through
Today, the company is marketing and developing the use of synchronized taxonomies and
intelligent information indexing, retrieval and semantic networks in various languages.
organization technologies that allow enterprises Convera’s solution also provides categorization of
to take all forms of content—text, images, audio documents across languages so that in
and video—and transform it into valuable multilingual environments users can intuitively
information that can be shared among employees navigate repositories of information that contain
and business partners across language barriers. documents in multiple languages.
Convera’s categorization solution provides a Convera has recognized the amount of confusion
modular platform for secure organization and in the marketplace that surrounds the topic of
search of rapidly changing information across information organization and has responded by
repositories, across languages and across media/ bringing to market a solution that includes not
file types. only search and categorization engines but also
a categorization personalization feature, domain-
Convera’s modular architectural approach allows specific taxonomies, and related domain-specific
IT department’s efficiency in deploying and semantic networks for most languages. Convera’s
managing a categorization project while their objective is to reduce the customer’s confusion
cross-lingual, concept-based categorization and risk by providing a single stop for both
solution allows knowledge workers to find the information organization and search solutions.
information they need as this information is
dynamically organized by how they conceptualize Convera’s basic technology includes the
the knowledge. following items:
Technology Approach • Categorization to support browsing

and search
Convera’s categorization technology leverages • Profiling (personal categories) to alert users to
established semantic models that mirror the way new relevant documents and changes
people associate different words and concepts. in documents
Convera’s approach means that information
organized by a given taxonomy reflects how • Industry specific semantic networks that
people think about the ideas embodied in related support concept-based categorization
documents. The result is that user expectations and search
are met as they navigate a taxonomy, browsing
their way towards the information they need, • Pattern search through Convera’s proprietary
ultimately finding the information where they Adaptive Pattern Recognition Processing
would expect to find it. Convera has modeled (APRP) for recall
how they think about ideas in their language,
relative to a given taxonomy. This approach is • Synchronizers that access content in a wide
able to respond dynamically to changes that range of repositories to find and update all the
occur in document repositories as well as in the content in the enterprise lowered.
manner in which people think about information.
25
• A security model that supports secure access to are organized by concept. Although a program
those disparate repositories can use surrounding words and linguistic
analysis to guess at the meaning of a term in a
• Multilingual support through a plug-in document or query, it is important to provide a
architecture for categorization across category administrator the choice to select the
languages meaning of a term. RetrievalWare’s PowerSearch
feature allows users to control word expansion of
• A component architecture query terms in the Semantic Network, which
means they choose which, and how many, related
• Software development kits (SDKs) and terms to include in the category rules. Users can
application programming interfaces (APIs) that also choose specific meanings for their query
allow customization and integration of terms. For example, when using the term for
categorization and search banks in a category’s rule set, an administrator
can easily instruct RetrievalWare to use only
Concept-Based Categorization those other meanings associated with financial
institutions and not those dealing with the bank
of a river or a turning aircraft. In addition it is
Concept-based categorization accounts for the possible to select individual terms within a
fact that there may be several different words that meaning to sharpen the rules even more.
can be used to express essentially the same
concept. It is often the case that industry-specific Support of Languages and Subject Domains
jargon is developed to more describe objects,
concepts or processes that might otherwise be
expressed through the use of simple layman’s RetrievalWare includes modular support for
terms. Concept-based categorization enables the more than 25 languages and many domain-
controlled expansion of terms within a category specific semantic networks through a Plug-in
rule set e.g., a search for “international architecture. This feature acts like a multilingual
commerce” will find documents that contain subject expert that not only understands the
terms such as foreign trade, import, export global concepts and terms in a subject area, but also
mercantilism and free trade. By using a concept- knows how they are used across languages.
based rules engine, the administrator can take
advantage of subject matter expertise to support Concept-based categorization that takes
a more accurate rule set that is automatically advantage of cross-lingual search and domain-
updated and improved with subsequent versions specific semantic networks reduces the need for
of the semantic network. In other words, a authors or other people to tag the documents
standard taxonomy can be imported and then because this feature automatically provides these
modified by subject matter experts to match your relationships. For example, if a category rule uses
organization’s needs. the term “central” meaning in or near a middle
position, not only could documents containing
Concept search also does what we naturally do in middle, medial, midway and other relevant
conversations with each other: it clarifies the English terms be included, but also it could
meaning of query words through analysis of include documents in other languages using
surrounding words (e.g., the word tank when terms such as middelpunt (Dutch); milieu
surrounded by words such as military and vehicle (French); Mittelpunkt (German); medio (Spanish)
is more likely to be a fighting vehicle and less and mezzo (Italian). If the medical domain-
likely to be a container for holding fuel). specific term bronchus was used in a category
rule, other English terms such as windpipe,
The core support behind concept-based respiratory tract and bronchial tubes could be
categorization is the standard RetrievalWare automatically be included in the rules, along with
Semantic Network, a collection of approximately terms from other languages such as bovenlip
500,000 English words that expands to over 1.6 (Dutch); arrière-gorge (French); Atemwege
million semantic relationships and idioms that (German); bocado de Adán (Spanish) and bocca
(Italian).
26
This architecture allows Convera’s RetrievalWare Products

to not only reduce the need for tedious and
inconsistent tagging of documents but also
provides cross-lingual and cross-domain Convera RetrievalWare is an enterprise-class
categorization and search. For example, when an cross-lingual multimedia categorization and
English-speaking financial analyst wants to retrieval solution for organizations that have
research the potential risk of making a loan to a large amounts of information they need to make
German pharmaceutical company, there may be searchable. RetrievalWare can index, organize
relevant documents created in other languages and securely retrieve contents from a wide range
containing technical terms that could impact his of repositories and document types, including file
decision. With cross-lingual and domain-specific systems, over 200 document types, groupware
categorization, the analyst may find articles systems, Web pages, XML, relational databases,
about this company in English, German, Italian leading document management systems and even
and French newspapers that discuss recent loan video and scanned paper documents. All of this
defaults and technical advances. information can be browsed and searched
simultaneously from any Internet-enabled
Security computer running either Internet Explorer or
Netscape. RetrievalWare allows users to use
simple natural language queries as a basis for
One very important aspect of categorization, categorization and retrieval without concern for
particularly in intranets, is security. Now that search syntax, word usage and other details for
most modern information management content in over 25 languages.
solutions provide easy access to enterprise
information, there is an increased need to control Tools for integration and modification of
access to sensitive data. And, the security system RetrievalWare
must be able to keep pace with changes to access
control lists (ACLs). RetrievalWare provides a
security infrastructure that respects the native RetrievalWare offers a series of toolkits that
security of the disparate document repositories enable RetrievalWare to also be used by System
accessed by RetrievalWare through a single log-on. Integrators, OEMs, corporate developers, and
Convera’s Integration Services Group to modify,
RetrievalWare addresses the issues of extend, or embed RetrievalWare for more custom
authentication, distributed, cross-repository solutions. The RetrievalWare SDK contains
and document / library level security for toolkits to enable the integration of
categorization and search. RetrievalWare’s cross RetrievalWare servers as well as the creation of
repository, library and document-level security custom client interfaces or modifications to out-
ensures that only the documents that a given user of-the-box interface clients. Convera’s
should see are displayed, thus preserving the RetrievalWare runs on most major operating
total security of the system. platforms (Microsoft windows based and UNIX)
and uses industry standard Web servers, RDBMS
and browsers. Although almost any standard
business computer fulfills the minimum system
requirements, each customer’s requirements and
computing environment are unique and therefore
may need additional resources to achieve the
desired level of performance and functionality.
27
Vision Assessment
To efficiently provide users with knowledge Convera has long been a pioneer in this market
appropriate for their needs and roles, based on all and their products reflect the upcoming
available information, regardless of location, requirements of the enterprise as the next
language or form, and through a system that generation of search and categorization software
interacts naturally with the user at his, her or its emerges.
convenience, anticipating the user’s needs when
appropriate and desired. The latest release of their product incorporates
many of the future trends Delphi Group sees
• Provision of domain-specific taxonomies and emerging in this market segment such as:
semantic networks
• Provision of domain-specific taxonomies and
• Support for multiple languages in semantic networks
categorization and in search
• Support for multiple languages in categorization
• Robust security provisions that recognize and and in search
respect access restrictions associated with
particular repositories and/or documents • Security provisions that recognize and respect
access restrictions associated with particular
• Support for Multimedia information such as repositories and/or documents.
image, audio and video files
• Support for Multimedia information such as
image, audio and video files
Convera’s advantage is that it embeds multiple

information management tools into a unified
suite of products that integrate with other
business applications in the enterprise.
Ultimately, Convera’s software makes employees
and customers more productive by helping them
manage, locate and access information –
regardless of format, location or language.
28
builder adds contextual information and pairs it

Vendor Assessment Report: with the reduced content. The coupled content
Entopia TM and context is added to the knowledge base into
an invisible taxonomy. The entire first stage
occurs in the background of user activity,
Introduction including adding files, comments, queries, etc.
Where does the knowledge reside in the enterprise? The second stage is a dynamic profiling of users
and content that occurs when an individual taps
The answer is a seemingly intuitive one – into the knowledge base. User and source profiles
primarily in the minds of the individuals who are generated at the time of the query,
form that organization. Yet most software guaranteeing the most up-to-date and relevant
applications are designed to address the results for the user.
organization’s needs and not the needs of the
individuals. EntopiaTM has developed software The first stage of Entopia’s dynamic semantic
that leverages the concept that the individual’s profiling process is centered on a series of
knowledge is the basis of the enterprise linguistic algorithms as well as language-
organization’s explicit and tacit knowledge. independent statistical techniques. This process
starts with a rule-based contextual linguistic
Rather than focus on taxonomy as the end, parser that removes words without
Entopia software uses taxonomy as a means to discriminatory semantic content, taking into
transform information into knowledge and account the context in which they appear. Then
ultimately into business action. Individuals the software uses a morphological engine in
classify documents into categories as a by- order to regroup terms in equivalent classes. The
product of the process of collecting and creating document that has now been reduced to its
the information. When the user finds the essence is fed to two Neural Networks, which have
information they need, they categorize it either been designed and trained to extract key
on their own computer system, a shared system concepts of documents and adjust their relative
of files and folders or the enterprise system data importance to represent the document. The
repository. Entopia software automatically extraction is not only based on the presence of
suggests the category based on the semantic individual terms but also looking at relationships
engine’s transparent analysis of the document, between them, constrained within grammatical
but the individual controls where the and linguistic rules. These key concepts are
information is placed in the knowledge tree and amassed within the invisible taxonomy.
who has access to this view of the document and
its relationships to other text data. The second stage initiates when a user queries
the system to capitalize on the collected
Technology Approach information. At the time of the query, the system
recognizes the user on the basis of all work done
Entopia’s design goal is that the taxonomy up to that moment. The invisible taxonomy
structure generation and population of that adjusts to present relevant results personalized to
hierarchy should be a by-product of an each user by taking into consideration the
individual’s normal work process. As relevant structure of the query, the user’s unique profile,
documents are found or created, Entopia and the visual taxonomies.
software starts a transparent process of
extracting key concepts from the document to be On the visual front end, the user is prompted to
the basis of search and a taxonomy. This process place the information that is collected into one of
is called dynamic semantic profiling. three hierarchical zones:
Entopia’s dynamic semantic profiling process is a 1. Their personal zone
two-stage process. The first stage consists of a 2. Work group zone
semantic engine that reduces the collected
content to its bare concepts, while a metadata 3. Enterprise zone
29
As contributors add information, the Smart The result is a Quantum File, or Q-File, that contains:
Classification feature suggests “best fit” •The content
destinations, based upon the first-stage semantic
analysis. The visualization of these personalized •Enrichments (highlights, annotations) and
taxonomies will be made possible through a comments
Virtual Classification Structure. It allows a single •The semantic profile of the content and its
user to personalize his virtual overlay (based on enrichments
the invisible taxonomy) upon a corpus of
documents which may reside in the different •Contextual metadata
zones.
Collaborate
Products This module is a server infrastructure design
Entopia’s Quantum is a software suite designed to that enables users within the same organization
help enterprise organizations improve individual to share information and collaborate with each
productivity, manage information and facilitate the other through the workgroup zone in the visible
transfer of knowledge among individuals and taxonomy. A host of collaboration features are
workgroups. Entopia characterizes its software by offered including voice and text anchored
features that fall into three modules: Collect, comments and threaded discussion, e-mail
Collaborate, and Capitalize. notification, document check-in / check-out, and
HTML publishing. In the process of collaboration
metadata is also added to the enterprise
knowledge base. This collaboration module
encourages the synthesis between concepts and
the relationships between people that uncover
tacit and explicit knowledge within the
organization.
Capitalize
The third module is comprised of tools to
capitalize on the accumulated knowledge base
initially created during the Quantum Collection
and Collaboration processes. These tools include
Smart Classification, the Virtual Classification
Collect System (described above), auto-summarization,
The Collect module is the front end of the suite, and a next-generation search engine, the
accessed through a browser-based “thin client” or Knowledge Locator.
Windows desktop client. Entopia Quantum allows
users to gather and save information, from any The Knowledge Locator is a search solution that
digital source, into the visible taxonomy. These allows users to find not only documents but also
documents can be MS Office files, PDF files, Web experts and sources of knowledge within the
pages, e-mails, and many more file types. enterprise. Aware that different goals require
Through collection, metadata is automatically different search tools, the Knowledge Locator
generated for indexing and retrieving utilizes both a familiar full-text keyword or
information and added to the invisible taxonomy. phrase search, and a more advanced “semantic
search.” To track down documents containing
specific terms, only a full-text search is adequate.
For “fuzzier,” more conceptual searches, one
needs semantic search that yield results “about”
or “around” the query. This also reduces the noise
by listing only very relevant documents.
30
While other available search tools may stop there, Assessment

Entopia’s Knowledge Locator also takes into
consideration the context of the search. Entopia’s software is targeted at, among others,
the following verticals:
Vision •Financial Services
Entopia’s vision is to provide the best place in the •Professional Services
enterprise to locate information, which is also the •Pharmaceutical / Biotech
genesis of the company name Entopia, derived
from the Greek word entopizo (v. to locate or •Government
bring to light) and the more commonly •Education
recognized utopia (n. any ideal place or state).
Following Entopia’s core belief that knowledge and these horizontal roles:
management software will only be adopted when •Research-Oriented Users
it helps individuals directly and conforms to
their needs, Entopia’s view of taxonomies is that •Cross Functional Project Leaders
they must:
•Marketing Professionals
•Reflect individuals’ thinking first, as a visual
“front end.” •Industry Analysts
•Be customizable based upon individual •Knowledge Leaders
preference. •Corporate Learning Managers
•Allow multiple views of the same data (by
content, concept, intent, date, etc.). Entopia’s product incorporates taxonomy
technology in a knowledge management solution
Secondly, taxonomies must fulfill the application that empowers the individual to:
requirements of an enterprise, which means that •Discover information
taxonomies should:
•Convert that information to knowledge
• Have an invisible “back end” allowing differing
organizational views •Expedite decisions
• Be dynamic and evolve over time
Rather than isolating knowledge in silos of
• Be “soft” (objects may exist in multiple enterprise repositories, Entopia leverages the way
categories) the individual works to collect, collaborate and
• Grow with the size of the knowledge base capitalize on the tacit and explicit knowledge
within the organization. By beginning with
• Be in tune with natural patterns of the personal productivity, Entopia’s approach
knowledge base effectively captures tacit and often times
Entopia believes that the user is central to any disconnected information. Since it is
successful knowledge management system. While implemented on the user’s computer and helps
many organizations view taxonomies as merely them become more productive while controlling
search and navigation tools, Entopia believes that their own privacy, it delivers organizational
taxonomies are important enabling components productivity as a by-product.
of the knowledge worker’s daily decision process.
For an individual, the process of understanding Since most organizational data resides on
and integrating new information is significantly individual computers and hard drives, this
eased with an appropriate taxonomy; a taxonomy information is often lost when an employee
that is in tune with the individual, their current leaves, is promoted or moved. Entopia’s software
interest and the new information. In order to be brings back this orphaned information to the
efficient in this new role, the system has to be organization because it “selfishly” serves the
flexible and continuously adapt to the current individual where the organizational knowledge
needs of the knowledge worker. has always resided.
31
advantages of neural network are speeds up to

Vendor Assessment Report: 300 documents per second and accuracy levels
Mohomine between 90-95%.
Although Mohomine promotes the advantages of
automatic classification, the API’s provides tools
Introduction to change the names of the nodes in the
Mohomine’s strategy is to be a supplier of hierarchy to reflect the users needs.
taxonomy technology that is tightly integrated
into other applications. Mohomine is focusing on Mohomine’s technology is language independent
the OEM market segment of enterprise with support for Western European and Asian
application vendors, infrastructure vendors, languages. The same engine can classify western
system integrators and custom applications European languages with custom linguistic
within enterprise organizations. processing for each supported language. This
linguistic supports removes articles such as “a,”
Mohomine believes that the exponential rise of “the” and “and.” Stemming the words to their
unstructured data and the volatility of the data roots helps identify like concepts for example
requires an automatic solution that minimizes jump is the stem of “jumping,”“jumped” and “jumps.”
manual intervention and increases the end user’s
efficiency to realize a greater return on investment. The original design intent of their product is to
be integrated into other applications, thus the
Technology Approach supplied API can be programmed into another
application within a very short period of time.
Mohomine sees classification as one of the
components along with text extraction, clustering Products
and summarization as part of an underlying Mohomine’s Classification software,
framework designed to allow enterprises to mohoClassifier 2.0, automates the process of
control the flood of unstructured data. Through a populating customer-defined taxonomies with
common set of API’s these components are documents. Mohomine’s products are targeted at
exposed and tightly integrated with enterprise enterprise applications such as: Content
applications. By building a common framework Management, Search & Retrieval, Enterprise
for these modules efficient integration can Information Portals, Human Resource
be achieved. Management Systems, Supply Chain applications
and Customer Relationship Management applications.
The method Mohomine uses for building and
populating a taxonomy structure is their unique The current product is a Microsoft based
development of the neural network approach development environment with Linux and Solaris
supported by seven patents currently filed. They platform support under development. Mohomine
fuse pattern recognition technology with provides a C++ , com, and http interface to the
machine learn by example techniques. There are classifier. Java interfaces will be available soon.
numerous benefits to this technology. One of the
primary advantages is that this approach requires Mohomine uses a technology that learns the
few training documents to yield accurate results. boundaries between categories rather than just
The training material can be a minimum set of 8 the similarities. Analogous to the Sesame Street
documents to an entire custom predefined TV game –“Two of these three things are the
taxonomy. That means that the classifier can same and one is different. Can you tell which one
adapt to changes in relevance and new topics is different?” Where the viewer is shown two very
easily and quickly, thus lowering expenses similar objects and a third that is different. The
associated with maintaining a taxonomy. The viewer is taught to recognize the differentiators,
neural network separates unlike documents from so too does this classifier methodology. This
each other by learning the boundaries between process is applied to text patterns within the
categories instead of looking for probable unstructured data. This technology needs fewer
training documents to “understand” and develop
reoccurrences of concepts. Other cited
the categorization groups or nodes.
32
The process is self-teaching through a machine Vision

learning function that returns more and more
differentiated categories. Because this is a form As one of their early customers and investors ,
of pattern recognition, Mohomine’s approach is Gilman Louie, president and CEO of In-Q-Tel said
language independent. As the body of documents “Mohomine’s technology gets to the heart
grows and changes dynamically the “machine of what we see as the biggest problem facing
learning” function will grow and change with enterprises and national security organizations
documents it analyzes. today — information overload.”
To augment the neural network methodology, Mohomine’s senior management vision is to

Mohomine uses a linguistic processing module build a horizontal, enterprise-wide integrated
for each language supported. This process solution for unstructured data management
understands the morphology of words by a (UDM) unstructured, semi-structured or
stemming process and sentence structure to structured data sources. The UDM platform
provide disambiguation. enables the classifying and extracting of concepts
and relevant information from unstructured
mohoClassifier’s architecture and design goals corporate repositories and Web sites
are focused at providing the customer with the automatically, quickly and accurately. The
fastest and most adaptable classification system combination of classification and text extraction
possible. There are a variety of features that and summarization is the key for enterprises to
achieve this goal: capitalize on their core competencies. Giving the
enterprise the tools and the platform to build
• Flexible Data Classification Options: applications that can take advantage of changing
mohoClassifier can be run in a Server or Client information into actionable knowledge is where
configuration. The Server configuration is best Mohomine sees the future.
used for batch classification of many
documents. The Client configuration is best Assessment
used by remote clients for lower volume, single
item classification. The Client configuration Mohomine appears to see the potential of being
interface is typically used by applications and the enabling technology to add structure to
ASPs. unstructured data. By combining complimentary
technologies such as extraction and
• Scalability: support for taxonomies of thousands summarization with classification, enterprise
of categories and millions of documents. applications can automatically supply actionable
information to the majority of an enterprise’s
• Readily Trainable: been designed to be readily users. This will help users find, file and process
trainable, allowing the user to specify a custom information in a smarter way. Increasing the
taxonomy and requiring typically between 5-8 value of applications such as Human Resource
items per category to achieve optimal Management Systems, Content Management
classification accuracy. Systems, Enterprise Portals and Customer
• Multiple Categories: documents can be Relationship Management software by correlating
classified into multiple categories. the unstructured information buried inside, the
value of classification technology increases
• Dynamic updates: can be retrained on the fly dramatically. A tightly integrated classification
enabling fine tuning of document classification capability will be standard functionality for
even after your taxonomy structure has been enterprise applications by 2004.
defined. Additionally, mohoClassifier
automatically supports alterations to a
taxonomy.
• Multiple Languages: mC supports English and
Western European languages.
33
1. Parsing or tokenizing converts the documents

Vendor Assessment Report: into a “bag of words” via standard vector format
Quiver 2. Feature selection scores how informative that
word or phrase is for that particular topic
Introduction 3. By choosing 10-15 documents per topic, these
There is more to taxonomy than applying documents teach the machine learning
classification algorithms or clustering documents Categorization Engine about each topic and
in a hierarchical structure. Since concepts and how to classify other documents that match
ideas are relative and most documents contain 4. Each document is then tokenized into a
complex concepts, it is essential to give people the hierarchy of topics. Documents can belong to
control of where and how these documents fit one topic, many topics or no topics at all.
into a categorization schema.
5. Then through a proprietary technology the
Quiver’s approach to this problem is an software assigns a numeric relevance score
application that allows for the hands-on control based on multiple criteria. This relevancy score
and maintenance of the process of taxonomy. In aids in the classification process. For example,
their approach they combine the strength of a an administrator can set a threshold for all
manual system – accuracy, with the strength of documents with a relevancy score of 85 or
an automatic system – efficiency. higher to be automatically categorized into the
Delphi Group and Quiver agree that taxonomy is machine chosen topics.
not a onetime process. When investing in But categorization is only the first step, a
taxonomy software users should also think about workflow system was developed by Quiver to
investing in a taxonomy process. The value of permit the user to determine how much of the
your information is only as valuable as your taxonomy is automatic and how much is in the
ability to utilize it to take business actions. manual override mode. A major value
Because of the inherent volatility and subjective proposition of Quiver’s product configuration
nature of the documents, a key to a successful lies in its administration and collaboration
taxonomy implementation is placing the environment. The distributed workflow is all
documents where users would expect to find organized from a single panel for nontechnical
them. If the documents are mis-classified, users users. This puts the decisions about how
will stop using the service. The Quiver system documents should be classified in the hands of
was designed for easy maintenance and adapting the content managers and subject manager
the taxonomy to the users and their current and experts. Workflow Memory increases editorial
ever changing needs. productivity by remembering decisions
Technological Approach previously made by information managers, on a
document-by-document basis allowing a
QKS Classifier is a truly hybrid solution to the machine learning process to make classification
information overload problem by providing more efficient.
human oversight of the decisions made by the
categorization engine in an easy-to-use toolset. The browser-based client houses the directory
with the nested hierarchy of related topics. By
Implementing their variation on the Naïve simply clicking and expanding the topics and
Bayesian methodology Quiver processes the text subtopics one can quickly navigate to the
contained in unstructured documents and groups appropriate concept. When integrated with your
related documents in clusters of hierarchical corporate search engine via the supplied API,
nodes through an automated multi-step process. finding the relevant topics is fast and efficient.
Products & Services
The QKS Classifier™ is an application designed
for managing the taxonomy process. The primary
34
differentiator for Quiver is their modular design, objectives, advise and set an appropriate timeline
which has three components: and goals accordingly. They also assist in
deployment and integration and ensure long-
1. Categorization engine discussed in the term, flexible and reliable customized solutions
previous section that fit your specific business needs. Quiver
2. Directory Management Toolset (DMT™) services range from taxonomy consulting to UI
3. Output and Display Interface customization, custom reporting, training and
systems integration.
From the Directory Management toolset module Vision
via an intuitive single panel:
Quiver’s unique approach augments advanced
• Document locations are chosen, which contain auto-categorization technology with an intuitive
the documents to be categorized, directory and workflow management toolset to
• Training sets developed and run teaching the provide clear visibility into classification decisions
classifier how documents are related. and maximum control of the end-user experience.
• Domain experts can then be assigned to review Quiver sees as their vision to be the provider of
documents based on qualification criteria. information management solutions to maximize
• Administrators can assign rights, permissions enterprise leverage of corporate information
and set limits and hours for classification. assets. Quiver’s view is that technology alone
cannot replace the accuracy of human expertise
This tool is very flexible allowing multiple and contextual analysis. Quiver combines the
individuals to approve the classification of the efficiency of technology with accuracy of human
document with optional administrative control oversight to deliver optimal knowledge solutions
for managing the people and the process. for intuitive and comprehensive access to
Following the MS Outlook GUI metaphor, this enterprise information. Whether Quiver is
presents a hierarchical structure of folders and deployed standalone or as a part of an employee
sub-folders and files is on the left which can be portal, a customer extranet, or a public website,
expanded and contracted with simple mouse an organized view of content is crucial to
clicks. The list of documents for review is listed successful information sharing and productivity
in a window to the right, with metadata improvement.
information managed in the panel directly below
the main window. With administrator rights Assessment
users can assign documents for review by your To make taxonomy useful for your constituents it
domain experts. These are the people in your must apply their view of the world. A highly
organization who understand their area of interactive application like Quiver will assure
expertise – be it budgets, or marketing those who want control in the creation and
information or technical support procedures. The maintenance of a taxonomy system to have it
toolset also allows administrators to filter out easily customized for your users. This approach
documents on the basis of age, size, source and assumes that the accuracy of the relevancy of the
document type. categorization is very important to your
organization. If close is good enough, then
Targeting enterprise applications such as Search, Quiver’s approach may be overkill.
CRM, content management and ERP systems,
Quiver API’s deliver standard XML output to Quiver products have a solid methodology for
portals, intranets, extranets and other enterprise developing a useful taxonomy with as much or as
applications. This output can be either a XML little control as your users and management
output, standalone or integrated directory view wants. If what you are looking for is an
application to develop and maintain your
Drawing on a deep understanding of information taxonomy with a distributed group of domain
classification, ranking and retrieval products, experts within your organization, then Quiver is a
Quiver’s professional services group works choice to consider.
closely with each client to identify project
35
multi-tiered hierarchy. The methods of

Vendor Assessment Report: categorization are governed by rules the user
Semio can see and modify.
Keeping up-to-date is no problem as the design

Introduction allows for continuous processing so as
Like the taxonomies they build, Semio is evolving, documents are added, changed, and updated they
changing and adapting to the needs of their can be re-indexed to keep the taxonomy relevant
users. Semio was founded in 1996 and is on their and current.
third generation of taxonomy products. Currently
they offer a multi-product line of taxonomy Pre-Built Taxonomies and Thesauri
solutions that will expand in the near future. Semio has also developed a number of
taxonomies for specific vertical markets based on
Founded on information gathered during industry standard thesauri that can jump-start
academic linguistic research, the Semio approach
the process and provide standardized structures
has evolved as experience was gained working for the taxonomy. Some of these currently
with customers in real life enterprise
available taxonomies are:
environments. Their suite of products and
services will help companies get their
• IT
information sprawl under control, quickly and - Hardware
efficiently. Realizing that one size does not fit all,
- Software
Semio has applications, API’s, interfaces and
administrative and maintenance tools to get • Healthcare
started quickly and then expand and scale to - Pharmaceuticals
enterprise environments.
• Telecom
Technology Approach
• HR
Semio’s approach is to analyze the text of
unstructured information via a series of • Financial
linguistic patented algorithms that identify and - Investment Banking
extract concepts (key phrases and terms) in - Bankruptcy
documents and statistical clustering techniques.
Some of the linguistic methods are identifying • Legal
noun phrases and proper names, analyzing the • Sales & Marketing
recurrence of words and their proximity to each
other and the uniqueness of the words to a • Geography
particular document. Another method would be a
comparison of concepts from an imported
lexicon or previously defined taxonomy. Vertical taxonomies under construction are:
petrochemical, sports, entertainment, religion,
No document training sets are required for this education, sciences, Internet, insurance, construction,
linguistic method, which produces a hierarchy of paper products, food, and government.
concepts that are very granular and multi-tiered.
This approach has yielded as many as 50,000 Semio has over 30 sets of pre-designed
categories with multiple levels depending on the categorization templates addressing various
corpus of documents within the organization. organizations. Since no pre-built taxonomy will
Designed to scale through distributed and fit seamlessly into your organization, Semio
parallel processing architecture, some Semio provides an extensive tool set to modify, edit, and
customers have over 13 million documents in rename these structures and approved word lists
their corpus that have been categorized into a to customize for your organization.
36
Products Customer Study

• SemioTaxonomy™ - applications such “Semio picked up a tape of corporate data and
corporate sales portals and R&D portals, returned it in less than two days with this same
as well as integrations with Plumtree data organized in a taxonomical structure.
and Autonomy Through this exercise we were able to
demonstrate to our board that we could
• SemioMap™ - visualization of content categorize our data with minimal human
via graphical maps of concepts and their intervention,” said John Walford, IT
inter-relations infrastructure manger, Highways Agency.
• SemioSkyline ™ - a viewer designed to add The project started on January 2001and the pilot
browsing and searching capabilities as well as was initiated in March of 2001. This was an
information about the document. This can be integration with Plumtree to provide a portal for
customized to reflect your organization’s look 2500 users distributed across 11 Highways Agency
and feel. sites. Use of Semio indexing and categorization
has stripped away and exposed bad data,
• SemioTagger™ - is the categorization engine highlighting what needs to be done and where.
that supports over 200 file formats including It has uncovered data previously hidden or
Lotus Notes and Documentum and runs on unavailable for analysis. Semio’s software has
Windows 2000 SP1 server with output to either reduced the time and cost associated with
XML, Microsoft SQL or SemioSkyline. This knowledge delivery and has improved
component contains the Knowledge decision support.
Engineering workbench which is a module for
editing the rules that govern the linguistic Assessment
categorization process. Other modules are
crawlers, converters, categorizers, exporters With over 100 implementations Semio
and an administration tool. understands what enterprise customers need and
want to turn information into knowledge. Semio
A key differentiator for Semio is a “Starter File” realizes that extracting knowledge from information
in the KE Workbench module in the SemioTagger overload is a chronic problem that could be
engine. This “Starter File” can contain 2,000 to terminal in today’s knowledge-based economy.
10,000 predefined categories based on your
organization’s requirements. This “Starter File” Semio has a suite of proven products that have
answers the basic question of where and how do been refined by working with enterprise
I start this taxonomy process. customers to resolve real business issues
involving categorizing unstructured information.
Addressing the evolving needs of customers
Semio will be expanding their product’s
capabilities to address significant business
issues such as security and personalization.
37
Technology Approach
Vendor Assessment Report:
Stratify automatically creates a hierarchy of
Stratify concepts important to a given business. By
applying pattern-matching algorithms to a
Introduction sample set of documents, Stratify aggregates
individual documents into clusters and arranges
Stratify is highly attuned to the context and these clusters into a topic hierarchy. The software
associative meanings of words. The company continuously adjusts the composition of the
recently changed its name from PurpleYogi clusters, identifying outlying documents and
(memorable but not relevant to their business) to duplicates as it proceeds. Stratify can also import
Stratify. They did this to reflect the company’s an existing taxonomy or provide a pre-built
focus on creating order (stratification) out of the taxonomy containing more than 15,000 topics to
chaos of unstructured data. jump-start an implementation.
An aptly named product, the Stratify Discovery Any hierarchy can be enhanced using additional
System helps customers discover knowledge rules and keywords to define topics. Software
hidden within unstructured data scattered across tools allow nontechnical people to edit the
the enterprise. Consisting of HTML pages, taxonomy to meet their specific needs. Users can
reports, proposals and e-mails, these text add documents to training sets and test the
documents are located on file systems, Web results of these changes in real time. The tools
servers, databases and content repositories. help the user diagnose and resolve problems in
Stratify sees its products as helping executives classification. By applying business rules and
make better informed decisions by finding the different filters, the user can specify which
“whys” (supporting e-mails, reports, news items) sources of information are more important to
and correlating them to the “whats” derived from them. Users can add, delete and link concepts as
applications drawing on relational databases. they wish since this information is stored in a
metadata repository.
Stratify believes that a taxonomy of important
business topics should be an integral part of Taxonomies evolve with time as topics and
every major enterprise application. By creating relationships change. Unstructured content has
logical topic hierarchies and accurately implicit meanings that people interpret in
classifying enterprise documents into them, different ways depending on the current context
Stratify provides business critical information to and their individual interests. Stratify allows the
a variety of enterprise applications. Examples of easy manipulation and re-linking of concepts via
these types of applications are: Enterprise Search the metadata contained in the hierarchy, instead
Solutions, Customer Relationship Management, of having to reclassify an entire corpus of
Content and Document Management, Corporate documents. For example, as references to Clinton
Portals, News and Information Aggregation and change from president to ex-president, the same
ultimately Business Intelligence applications. documents can now be classified together and yet
Access to information in an organized and be found under another category as well. This is a
contextually relevant manner will dramatically good balance of hands-on approach and
enhance the value of those applications for automatic functionality.
today’s businesses.
Consistent with Stratify’s belief that different
categorization algorithms and methodologies
have different strengths and limitations, Stratify
uses a variety of classifier engines to analyze and
categorize documents into appropriate topics.
The Stratify product is designed so that multiple
classification methodologies run in parallel. The
classification analyses are compared and chosen
by using the results of the individual processes
38
with patented technology built into the Products

Combiner module. This modular architecture
allows Stratify to incorporate new methodologies Stratify’s Discovery System has a modular
as they are developed. architecture. Stratify packages their product as an
application with Web browser and Windows
Stratify uses industry standard SQL databases for clients. This application consists of the following
its metadata repository, exposing APIs that allow components:
the import and export of metadata in standard
XML format. The ability to easily integrate • Crawlers to collect and extract the text
Stratify’s technology into existing enterprise information;
applications is a feature that will make corporate • Hierarchy Builder to cluster the sample set of
IT groups appreciate Stratify’s adherence to open documents into relevant hierarchical groups
standards. Stratify servers are J2EE compliant based on common concepts, statistical patterns,
and can integrate with legacy systems via APIs key words and other algorithms;
that accommodate XML, Java, SOAP and COM.
• The Classifier analyzes documents based on
independent, multiple classifiers, allocates
Another unique feature is the advanced
them to appropriate topics in the hierarchy, and
personalization capability of the Windows Client
that learns users’ interests and alerts them to new aggregates critical information such as
document source, title, time/date stamp and the
documents matching their profile. While the user
is in the process of writing a document, the location into the metadata repository. This
eliminates the need for manual tagging;
Stratify system classifies that document and
presents related documents to the user. This • Management Tools allow users to modify and
happens automatically, but users can modify maintain the categories;
their interest profile as they desire. In this way, • Metadata database and server to store the data
the user can be made aware of interesting about the unstructured data so that
documents without sifting through irrelevant applications can access and view documents in
search results. standard XML format; and
• Web Access and Windows Client interfaces
allow users and administrators easy access
through an intuitive interface.
Stratify also provides a Classification Server that

can be used to scale the system, or for use as an
embedded product.
To help customers get started, a 15,000-node

taxonomy is available with the application. Users
can run their internal document against this pre-
built taxonomy and then customize it to their
environment and needs.
39
Customer Case Study and Testimonials Assessment

One of Stratify’s customers shows that taxonomy The Stratify target market is the enterprise
is not just a search engine enhancement. Infosys organization. The ability to run in a Microsoft or
Technologies Limited, a leading consulting and IT J2EE development environment with either Web
services provider, needed to respond more browser or Windows clients will appeal to large
quickly to its prospects and customers to win organizations with mixed legacy systems.
new opportunities and cut costs. They chose the Focusing on key vertical markets such as
Stratify Discovery System software to launch a professional services, financial services,
data management system to organize and classify government agencies, the pharmaceutical
unstructured marketing documentation, such as industry and biotech market, gives Stratify the
reports, customer proposals, case studies and reach to a broad base of customers while
competitive research. remaining focused with a consistent message and
knowledge of business issues specific to those
By incorporating the Discovery System into their vertical markets.
existing set of enterprise software applications,
Infosys’ sales and marketing teams are able to Pre-built reference taxonomies tailored to
create proposals faster. Reusing applicable data specific industries appeal to customers who want
while integrating new information makes the to kick-start their initiative and show immediate
proposal generating process more efficient. The results. Software management tools give
Stratify system also gave Infosys insightful nontechnical customers the ability to customize
competitive intelligence. The return on and tailor the topic hierarchy to their specific
investment was made in less than five months. needs as those needs change and evolve over
time. Example: do you place documents about Al
Infosys is integrating the Stratify Discovery Gore under politics, education or both, and then
System technology into its growing Knowledge link them together?
Management practice. The Infosys Knowledge
Management Service focuses on linking Stratify’s use of multiple classifier methods and
knowledge management to overall business algorithms to achieve more accurate and relevant
strategy and is holistic in character; it addresses categories is an approach Delphi Group sees
people and processes with technology tools, such more suppliers adopting.
as the Stratify Discovery System, playing the role
of a key enabler. Delphi Group agrees with Stratify that adding
structure to unstructured data helps customers
“Harnessing the intellectual capital within a make strategic decisions and discover new
company has become essential for competitive ways to utilize the knowledge assets within
advantage and even survival,” said “Kris” their organizations.
Gopalakrishnan S., Deputy Managing Director
and Head- Customer Service and Technology at
Infosys. “The only effective implementation of a
knowledge management strategy is one that
captures the intellectual capital throughout the
entire organization. The Stratify Discovery
System is an important tool for our own
consultants in enhancing client relationships and
as part of the KM service offering to our clients.”
40
Getting value out of text mining and

Vendor Assessment Report: summarization is only possible if you have
Textology enough granularity and detail in the concepts you
can identify. You must be able to distinguish a
large number of concepts in the corpus of text,
Introduction and avoid false overlaps by understanding the
After your organization has categorized the context in which they are being used. Having
information into a structure – then what? done this, you are positioned to gain greater
value from documents.
There are a large variety of solution providers
addressing the problems of unstructured text For example, one of the functions Textology can
categorization and classification via taxonomy provide is to summarize information linking the
software. Textology drills a layer deeper into primary concepts contained in a document to the
documents to add valuable information that key sentences supporting them. This lets users
helps companies identify, prioritize, and use quickly see if a document contains the
critical information extracted from text sources. information they are looking for, and helps them
In addition to producing precise categorization, rapidly scan texts by highlighting the most
this detailed understanding allows Textology to relevant sentences. They can also select the
extract key sentences and combine them to concepts they are interested in and generate
produce concise document summaries, and unique summaries based on the key sentences
comprehensive meta-tag data. Textology has related to those concepts. For example, a
approached many of the technology companies quarterly financial report might generate a
mentioned in this report as potential partners summary highlighting sales figures for one user,
and not as potential competitors. There are and a summary highlighting restructuring costs
multiple reasons for this. for another user.
1. Textology’s approach improves the granularity Textology was founded as a joint venture between
of taxonomy structures by basing classification Elbit, Ltd and Assa-or Systems. Assa-or has
on identifying key concepts and the context in produced products for the American and Israeli
which they are used. This provides greater defense and intelligence communities. Elbit
precision then subjects based on statistical makes focused investments in the e-business and
aggregations of key words, and allows difficult m-commerce markets with infrastructure
categories like “explanations of market technologies, cellular applications, and value-
conditions” to be accurately handled. added services.
2. Organizing documents into a formal taxonomy

is only a partial solution. To achieve greater
business value, important details within texts
must also be identified and extracted so that
they can be used to enable better business
decisions. Textology’s summarization, meta-
tagging and extraction capabilities provide this
capability.
41
Technology Approach Products

Textology’s approach to developing a taxonomy is Textology products are a family of modular
a hybrid method based on combining the best components and API’s for integration into
features of linguistic capabilities, machine enterprise applications. The company is focused
learning and statistical algorithms. It leverages on specific customer types where analysis and
linguistic patterns to add context that improves knowledge extraction from high volumes of high
the detail and accuracy of statistical techniques, value text is critical to the business processes and
and uses statistical methods to automatically decision making, such as:
identify and evolve these linguistic rules. These
algorithms can be tuned for specific applications • Financial Services and News Wire
to produce optimal results. Taxonomy also give Organizations
users control over whether they want to increase
precision and narrow their results to see only the • News Aggregators
most relevant items, or do a broader analysis that
ensures coverage of all related items. A text, thus • Business News Publishing and Services
analyzed, is then assigned to one, none or many
categories based on its identified concepts and • Insurance and Legal Organizations
their context. Textology’s hybrid method also
minimizes the need for large training sets to • Healthcare and Pharmaceutical and
achieve acceptable levels of accuracy. Biotech Firms
Extracting concepts on a fine granular level • Defense and Intelligence Services

produces additional consequences such as:
• Precise text summaries with unique versions This market focus allows performance
created for each concept the user optimization for targeted applications.
is interested in
Java-based architecture ensures platform
• Dynamically individualized summaries to independence and embeddable components
reduce reading time for easy integration into OEM products. The
technical requirements are:
• User selectable summary views (list of key
sentences, paragraphs, or simply highlighting Windows NT or Solaris.
sentences within document view)
Typical NT server is a 4 CPU Pentium 4 with
• Granular taxonomy and easy cross- 512 MB RAM.
categorization to other taxonomies
Textology provides connectors to standard text
• Comprehensive meta-tag information and Web document formats. Textology output of
meta-tag data supports standard databases and
• Customizable relevant document headlines applications. In addition to training and support,
or titles Textology provides professional services to
install and setup Textology products, and to
• Detailed data extracts that allow data to integrate them into customer application
populate other applications data requirements. environments.
42
Vision Assessment
Textology is targeting its solutions to customers How much detail is too much and how much is
where there is a high volume of dynamic, volatile not enough is a sticky problem to solve.
text information and high value associated with Textology’s approach is to provide a very detailed
extracting detailed concepts from this classification and extraction of concepts and
unstructured text. Delivering innovative business ideas contained within large volumes of
solutions for enhancing productivity and unstructured text. Solving the problem of
creating business value in text critical information overload becomes more critical
environments is Textology’s mission. By focusing when the volume and volatility of the
on a select group of customers, Textology can information is very large. Textology’s solution is
leverage their expertise and value to address for those organizations who need a very detailed,
critical business issues. granular taxonomy and the ability to extract
information and summarize it to make critical
Example Application business decisions. Textology solutions requires
serious consideration.
Most financial newswire services use a dedicated
staff to daily review hundreds of Web pages, press
releases, internal documents, and other potential
news sources. For each identified news item they
must read it, classify it for relevancy to one or
more channel topics, construct a short text
summary and/or news headline, create a list of
key words associated with the article, identify a
list of related articles. They then enter this
information along with the URL or location
identity of the source document into a company
specified meta-tag format. Finding related
articles is particularly a time-consuming process
due to the imprecise process of sifting through
the large number of hits returned by entering the
identified key words into an internal Web site
search engine.
Staffing is limited by budget constraints, so there

isn’t sufficient time to review all the potential
sources completely. As a result, quality suffers
from broad holes in coverage of potentially
important items. New products are difficult to
add because they would add additional work to
already strained resources. Revenue
opportunities are lost because there is no way to
expand their services to include articles from
additional content providers. In addition,
consumers are continually asking for new ways to
create even more personalized channels that
require much greater detail and flexibility in the
way news topics are identified.
43
Technological Approach
TopicalNet’s technology uses semantic and
TopicalNet syntactic knowledge to classify documents. The
system understands that “biology,” “biological,”
Introduction “bio” and “biologists” are all related. It knows
that “textbooks” and “text books” are lexical
“Quick start” and automatic taxonomy generation variants. It also knows that “Linux operating
are two descriptors that summarize TopicalNet’s system” and “Linux” are semantic variations.
approach. The quick start comes from a pre-built
taxonomy of over 1 million categories that is an Each quarter, TopicalNet fetches over 60 million
integral value of the TopicalNet solution. This
Web pages. Specialized software analyzes these
extensive pre-built taxonomy eliminates the need pages to extract knowledge about the
for developing a set of training documents for the
relationships between words, phrases and
software. For some organizations trying to find classifications. One of the outputs from this
the “right” training documents becomes a analysis is a taxonomy of about 1.5 million
circular problem. You can’t find out what you richly-connected categories. Another output is a
don’t know because you need to use known set of over 1 million semantic relationships
documents for training the software. Simple between individual words. TopicalNet also finds
installation requirements make this solution
and analyzes over 2.3 billion distinct word
typically up and running in one day. phrases in the document set. This automatically
generated base of knowledge allows TopicalNet’s
Quick start is an essential element to TopicalNet’s software to classify new documents “out-of-the-box.”
ROI argument, but so are features to allow the use
of existing taxonomies within an organization.
Through its mapping technology, they are able to Products
leverage the depth and accuracy of their TopicalNet Classifier is the core product. This is
underlying million plus categories while an out-of-the-box solution. A major differentiator
expressing the classifications using the names is that little training for either the users or the
and organization of a company’s own taxonomy. system is required. This means quick deployment
and quick ROI. The system is very scalable from
Automatic categorization is based on TopicalNet’s one extreme of 60 million pages down to the files
extensive pre-built taxonomy. TopicalNet has over on a single PC. Currently, the TopicalNet product
a million categories with an interconnected offers both an API aimed at custom integration
matrix of topic relationships. The software crawls into enterprise applications and a packaged user
through your unstructured data, parses the interface for smaller organizations looking for a
concepts and matches and places them into the quick return on investment.
pre-built categories or the companies’ categories.
The product consists of several modules:
Designed to work with multiple types of
environments that range from a single desktop • Data acquisition to harvest data from many file
machine, to file servers, to an enterprise types and formats such as HTML, Text, MS
information system, TopicalNet is quick to Office products, presentations and PDF files.
deploy and quarterly updates keep it “topical.”
• The Classifier that develops the hierarchical
and interrelated categorizations and scores the
documents relevancy.
• The presentation module that shows a
browsable directory structure.
44
A unique application of this technology that the

company shows is reclaiming data and
knowledge assets that would normally be lost on
personal computers of former employees.
TopicalNet assures their users that they have

access only to documents allowed by the current
security system. The system can run on a number
of operating systems and environments such as
Unix, Windows, Macintosh and Linux.
Vision
TopicalNet’s vision is that classification adds a
dimension of clarity, fidelity, and actionability to
the analysis of most types of unstructured
business data. The software has the capability of
defining the thematic attributes to answer the
question, “What is this data about?” The software
facilitates collaboration between employees on
disparate information by organizing the
information into recognizable categories.
TopicalNet’s mission is to develop and market
software applications with the lowest possible
cost and deployment effort to be applied to a
broad range of customer problems associated
with unstructured data.
Assessment
Getting started quickly and scaling up to your
enterprise's capacity usually are two
diametrically opposed features. TopicalNet has
been able to address both of these issues in one
solution. Using the pre-built taxonomy eliminates
the need for developing training sets and the
quarterly experience with very large corpora
(over 60 million Web pages analyzed) combines
scalability with being current and topical.
TopicalNet software has the ability to tune and/or
train the system to do a better job for specific
clients' content and taxonomy. TopicalNet has
shifted the burden of responsibility for a
successful implementation from the client to
this impressive product that does the bulk of
the hard work.
45
Technological Approach
Verity breaks content organization down into
Verity four steps:
Introduction 1. Build the Taxonomy.

Verity recognizes that different businesses are
Verity’s three-tier portal infrastructure software at different stages and levels of content
integrates discovery, content organization and organization. Some may have existing
social network technology into corporate taxonomies (the skeletal navigation structure
portals, e-commerce portals and business-to- of nodes/categories), while others don’t know
business portals, along with a wide range of e- what categories they need. To accommodate
business applications. this, Verity has the flexibility to build
taxonomies in a number of ways. Existing
Verity currently has over 1500 customers, corporate taxonomies can be imported,
including 80% of the Fortune 50 companies, and industry taxonomies can be used, new
is the OEM search provider of over 200 leading taxonomies can be built, imported taxonomies
software vendors. In the Delphi report Market can be modified, and/or the corpus of
Analysis: Portal Software Leaders, released in May information can be analyzed and relevant
2001, Verity was the market leader in the portal concepts automatically extracted.
infrastructure software category.
2. Build the Model.
Before a taxonomy can
be populated with documents, a model
defining each category must be built.
Companies with limited resources may require
a completely automatic solution, while others
may choose to augment automatic methods
with human intellect to achieve specific
business goals. With Verity, models can be built
using a number of methods: rules defining
categories can be generated automatically,
rules can be imported from existing taxonomy
models or industry taxonomies, and/or domain
experts can build new rules or modify
imported and automatically generated rules.
3. Populate the Taxonomy.
Using the model built in step 2, Verity
automatically populates the taxonomy with
documents during the indexing process.
4. Use the Taxonomy.
Populated taxonomies can be used in a number
of ways. End users can browse through them by
drilling down through broader categories to
more focused concepts and individual
documents, or they can narrow search results
by limiting queries to specific categories.
Taxonomy models can also be used to enable
content notification applications.
46
Products Customer Case Study – The Gale Group

Verity Intelligent Classifier is the application that Gale Group Case Study
enables Verity’s back-end content organization The Gale Group is a world-leader in
capabilities. Intelligent Classifier can be e-information publishing for libraries, schools
purchased as a component of Verity K2 and businesses. Best known for its accurate and
Enterprise and Verity K2 Catalog, in conjunction authoritative reference content, as well as its
with the Verity Profiling Engine for content intelligent organization of full-text magazine and
routing and notification, or as a standalone newspaper articles, the company creates and
content organization suite. maintains more than 600 databases that are
published online, in print and in microform.
Using Intelligent Classifier, businesses can
combine what Verity calls the ABCs of The Gale Group aggregates the widest range and
Content Organization: most up-to-date periodical articles on the market.
But it wants to offer even more. That’s why Gale
A. Automatic Classification. selected Verity’s next-generation content
Positive and negative exemplary documents can organization technology. Verity will help Gale’s
be used to automatically generate the rules that editorial team organize the company’s growing
define categories. This employs Verity’s collection of content in categories for millions of
Logistic Regression Classification (LRC) end-users worldwide.
technology.
The Challenge
B. Business Rules.
Domain experts can manually create new rules, With as many as 30,000 new articles being
or modify imported rules and/or automatically aggregated into Gale’s proprietary databases on a
created rules to enhance accuracy or meet daily basis, the company knows the success of its
specific business goals. The ability to easily offerings hinge on how easy it is for the millions
modify and fine-tune the rules that make up of students, library users, and business people
the model of each category is essential to who access Gale products to find the information
classifying mission critical information. they need. That’s why the company demands that
its content is organized in the most precise
C. Concept Extraction. manner possible so users can intuitively locate
Verity’s Thematic Mapping can be used to the valuable content Gale offers.
analyze the entire corpus of documents to
reveal themes and concepts. This can be used Currently, Gale harnesses the editorial expertise
to generate entire taxonomies, break populated of more than 150 of its employees to manually
categories down into subcategories or to mine classify articles from more than 7,000 periodicals
enterprise knowledge for new insights. Once into as many as 40,000 categories and
Thematic Mapping has identified concepts, it subcategories. But with plans to expand the
labels them with names that are easy for people volume of content it aggregates and grow its
to make sense of. business even further, the company realized it
needed to augment its manual classification
efforts with a more cost-effective solution.
47
The Solution Assessment

After a thorough evaluation of several Verity is a recognized market leader in search
classification technologies on the market, Gale and classification, with a long history of
selected Verity, the portal infrastructure company addressing business issues surrounding
with the three-tier foundation of next-generation unstructured and semi-structured information.
business portals. Of paramount importance to Verity’s content organization technology
Gale was Verity’s second tier, content effectively and accurately classifies the mission-
organization, which combines the ABCs of critical information that is spread across most
Classification—Automatic rules, Business rules enterprises, delivering solid return on investment.
and Concept extraction.
The company will take advantage of a

combination of Verity automatic classification
capabilities, including our third-generation
Logistic Regression Classification (LRC)
technology to create the rules that define
categories using both positive and negative
examples. In addition, Gale Group editors will
fine-tune and customize the rules created by LRC.
The Benefits of Verity

Verity will help end-users—the students, library
users and business people who access content
from Gale databases around the world—to
navigate their way through content from an even
larger number of publications.
Verity technology will benefit Gale’s editorial

team by helping it increase the efficiency of Gale’s
already stellar classification efforts. Documents
that Gale’s editors previously would have read
and sorted through themselves can now be
handled by Verity technology. This allows the
editors to either focus on classification tasks for
new material, or to augment the results of Verity’s
LRC technology with business rules that capture
and utilize their domain knowledge to
continually improve results.
Additionally, Gale will be in a position to grow its
business even more by expanding its offerings
and strengthening its product line now that it is
using Verity technology to handle more content
on a cost-effective basis. This is an important step
in the company’s ongoing quest to serve existing
clients better—and to continue to build its
impressive market-share around the world.
48
Technical Approach
Wherewithal has developed a multi-threaded
Wherewithal software engine that holds the structure of the
taxonomy in a proprietary format designed for
fast response times under very heavy load and
Introduction extremely large numbers of categories and items.
There are various approaches to addressing the This is called the Collaborative Taxonomy
issues of constructing and maintaining an Engine™ (CTE) and has proven its capability by
enterprise taxonomy. The continuum ranges being the core technology of the Xoron.com, an
from a fully automatic approach with minimal Internet search site.
human intervention to Wherewithal’s approach A patent pending search algorithm-called the
which is a collaborative effort that makes every Hierarchical Search Algorithm™-uses the
member of your team—or your entire hierarchical structure itself to resolve the search.
company—contributors who leverage their own This algorithm is designed to both create the
unique knowledge to create a taxonomy for your most relevant search results for users, and also to
intranet. Wherewithal uses human judgment to create a system by which many contributors (viz.
create and maintain the taxonomy. This approach “tens of thousands” or more) would be able to
ensures the taxonomy is relevant, personal and collaborate on keywords without the system
up-to-date. getting bogged down with duplicated effort. This
structure is based on the human placement of the
Wherewithal’s approach allows category owners document, page and information pointers using
to “opt in.” What this means is that the domain the Category Owner Toolbox™ that is used to
experts can contribute to the maintenance and create and maintain the taxonomy category.
construction of their category as they have time
and need. This encourages a sense of internal The result of a “search” on this taxonomy
“pride of ownership” and an external community structure gives multiple hits based on the
pressure to maintain the relevancy of the category. keywords placed in the structure by human
A check and balance approach to taxonomy. contributors and the relationship of those
keywords to others used in the taxonomy. To
The advantages are that the maintenance and ensure results are relevant, and to enable the
construction of the taxonomy becomes self- scalable system of collaboration, results are
fulfilling. The most popular categories will get scored using multiple criteria. These criteria include:
the most searches and the most positive or
negative feedback based on how well the • Lexical “closeness” of the terms used by the
searchers believe the category is maintained. The searcher to the keyword found in the category
other advantage is that this approach alleviates item (i.e. word pattern matching-”computers”
the IT bottleneck. The contributors to the vs.”computing,” etc.)
taxonomy structure are the ones that have a
vested interest in the category, not someone in • How many times the keyword is found within
the overburdened IT department. the taxonomy hierarchy-if the result is found
several times within the hierarchy, it is thought
of as more relevant to that search.
• Keywords found at higher levels within the
hierarchy are given more weight in the scoring
process. This means that contributors at
“higher levels” of the taxonomy have greater
control over search results than those at lower
levels.
49
• The order of the keywords-if they are found in Products

the same order as the user entered them, they
are more relevant. Wherewithal uses a collaborative approach. They
integrate the needs of organization and retrieval,
• The “closeness” of the words within the allowing users to create a searchable and
hierarchy. For example if the keywords skip browsable structure. This in turn creates a
multiple hierarchical levels, the score is resource where end users are enabled to make
lowered. faster and better business decisions. Contributors
across the enterprise opt in to take ownership of
• The number of keywords used in the categories of their favorite subjects (using
contributing categories. The fewer the Wherewithal tools and an ordinary Web browser).
keywords the higher the score. In other words,
if a result is found and the item has only the
desired keyword, then it is thought of as being The process is:
very relevant, whereas if the item is described 1. Each category gets an “owner” (employee
with a larger number of keywords, the item is contributor), and that category lives
less relevant to that particular search. The underneath a higher-level category, and so on.
engine judges both the item itself and the
collection of keywords within the given 2. Owners create categories, subcategories, and
hierarchy for each potential result. enter items (URLs or other kinds of data)
within these categories, along with keywords
The Hierarchical Search Algorithm also allows for each item and category.
categories to be “repositioned” seamlessly in 3. Together category owners create a knowledge
other category hierarchies without changing its base in the Collaborative Taxonomy Engine,
keywords. In other words, it frees the contributor creating a living snapshot of the company’s
to concentrate only on their particular subject business decisions and structure
versus worrying about that of the entire category
hierarchy. This in turn allows others to use that 4. Programmers now use the CTE database to
category in other contexts. build navigation into the intranet, because this
database is dynamic and changes with the
This structure is based on the human placement business. The site is now integrated with site’s
of the document, page and information pointers search functionality.
using the Category Owner Toolbox that is used to 5. Using the search box on the intranet Web page,
create and maintain the taxonomy category. The users can search through the collaboratively-
system also includes a feature called built taxonomy, producing search results for
SecondOpinions™, which allows users to both categories and individual items, all scored
customize their view of the taxonomy at each level. according to relevance.
The selected SecondOpinion is scored higher in The benefits are:

the search results than opinions that are not
selected. This way users’ choices of “opinions” in • A living snapshot of the company’s structure
the search engine gradually customize it to show and business goals, which changes dynamically
the results that are more relevant to them (but with the business
still include all possible “opinions” in case the • Site structure and search functionality are
selected one doesn’t have the necessary results) seamlessly integrated
• Improving the site’s structure also improves its
search results
• Search results are under the control of
individual stakeholders across the organization
50
• A scalable system that allows for reuse of other Wherewithal’s perspective on taxonomy can be
people’s work and reduces or eliminates summed up:
duplication of effort
• Knowledge as a “non-fixed taxonomy”
Wherewithal has packaged the software in a suite – Also called a “hierarchical matrix”
of tools and applications for the enterprise – Also called a “hierarchy with infinite
market in a package called Wherewithal Enter- contextual variant branches”
prise Web Directory.
In other words, that things are classified
This suite consists of: according to a certain context, and that things
• The Collaborative Taxonomy Engine should be able to be classified in number of ways
(described above) • Indexing cannot be done “automatically”
• The Category Owner Toolbox for maintaining – It must be done by real people to have
and creating categories within the taxonomy real relevance
• Custom Directory™ - a way to customize the – Given enough people (1% of users in
look and feel of directory content any given base), people will make
crawlers irrelevant
• MultiSearch™ programmable meta-search
engine - a way to integrate existing enterprise Wherewithal’s value proposition is:
search functionality into a single set of search
results • A more complete and relevant corporate
taxonomy means a better-organized and easier
• The Taxonomy Service API, an XML-based to search intranet, leading to faster searching
interface into the CTE and browsing
This product is designed to develop categories to • The collaborative approach allows IT and
make searching on intranets and portals more management to maintain control but saves
efficient. time in meetings deciding on taxonomy and
Intranet structure
Executive Vision
• Business decisions are reflected instantly to the
Wherewithal envisions a world in which the Intranet when users decide on new structure,
classification and organization of items for use and the Intranet changes as fast as the business
by others is an integral part of everybody’s daily does
job. The company’s technology focus is
specifically on the issues of bringing together • Search results are under complete control of
large groups of people to collaborate on a single the business, vs. a random crawler
knowledge base. Wherewithal’s executive team
says, “Leverage the Internet’s power to gather • Programmers can use a single repository for
knowledge from individuals to create it’s index.” structural information versus separate
“Someday every person on earth will be a part databases and static HTML Web pages
time or full time infomediary-a knowledge
worker dedicated to classifying parts of the
internet or a corporate Intranet”
51
Assessment
Delphi Group sees Wherewithal’s products as
overcoming the common obstacles of making a
taxonomy timely and relevant to the users by
using the collaborative model for constructing
and maintaining the taxonomy. This community
effort will ensure that the documents and
information that most people care about will be
placed in the categories where most people will
want to find them. An example will be that Bill
Clinton will be moved from the category of
“president” to the category of “former president.”
This technology has proven itself to scale and

work well on the categorizing the entire Internet.
A key challenge in implementing the product is
integrating its approach with the existing
corporate culture and document publishing
process. The community method with its carrot
and stick approach has a very good chance at
keeping the flood of information controlled by
the users and the authors of the information.
Self-serving as well as visionary this is a
technology worth serious investigation.
52
parent-child hierarchy to Wherewithal’s and

Controversies & Pitfalls Entopia’s approach, which entail collaboration of
participants regarding both the construction of
Taxonomy Strategy: The Time is Now! the taxonomy and actually populating the
As an organization you need to start organizing resulting hierarchical structure.
your data now. The longer you wait, the more It’s critical to bear in mind that the end result of
unwieldy and overwhelming the task will be. any taxonomy initiative is a human interface:
Although manual classification works well for concepts and ideas are inherently relative,
“small” volumes of data, how small is small and personal and subject to change. Consequently,
how big is big, depends on your resources, virtually all taxonomy vendors supply some type
requirements and expectations. And due to the of tool to customize and rename the nodes of the
inherently ubiquitous nature of Infoglut, even taxonomy structure to suit individual needs. The
heretofore “small” classification projects are difference between these applications and tool
becoming evermore unmanageable. sets is one of degree.
Our recommendation is that you start with an One of the key advantages of an automatic system
expert on taxonomy and classification – people vs. a manual system is consistency. An important
with extensive training in the library sciences. perspective here is to consider whether the
Then, interview your internal domain experts people doing the classifying have
and start them setting up agreed-upon the same criteria for assigning categories as do
categories. the users. Categorizing the same concepts into the
same place is what automatic systems do well.
Then decide how big a problem this is. How If the automatic systems misunderstand a
volatile is the information? How much time do concept, they will at least mis-categorize all
your people spend looking for information? related documents and not scatter them in
How much do you lose in opportunity costs multiple categories.
because employees can’t make an informed
decision quickly? The decision to adopt a manual, automatic or
hybrid approach is a complex one. Whatever your
Manual vs. Automatic choice, though, your organization should commit
There seems to be a lot of “controversy” about sufficient budget and human resources to
manual taxonomy, versus automatic, versus a maintaining an accurate, up-to-date and relevant
hybrid of the two. Delphi believes this is a taxonomy system. The trend in the industry is to
tempest in a teapot and not really a relevant combine machines and human processing to
issue, because in order to make the taxonomy develop and maintain taxonomies.
relevant to the users, it must match their needs
and unique rules for relevancy. This is a good Maintenance and Dynamic Information.
time to reemphasize that there are two distinct As you become more involved in the process of
steps in constructing a useful taxonomy. The first designing and deploying a taxonomy, you will
is the design of the structure of the taxonomy soon realize that this is not a onetime effort.
(e.g. the Dewey Decimal System). The second is Taxonomy is an ongoing process that requires a
populating the structure. The range of taxonomy long-term investment— business priorities,
software providers is a continuum of approaches. technology, language, and human interest are in a
This range is exemplified from Mohomine constant state of flux. The more volatile the
representing a completely automatic, strictly information, the more the need for a systematic
process to keep the
Manual Automatic information categorized
Pros Cons Pros Cons
and relevant. New
documents are
Accurate, Logical Inefficient Efficient Limited Accuracy constantly being added
Controllable Does not scale Scalable Lacks Control to repositories, while
Inconsistent Resource Intensive Difficult to Train new versions of old
53
documents are released, and out-of-date The third interpretation here is that individuals
documents are removed from circulation. or work groups can develop their own relevant
Changing strategies, evolving products, and taxonomies as a subset of large taxonomies.
advancing technologies all drive changes.
Taxonomies may be industry- or even Librarians
department-specific. Information today is by Librarians are, first and foremost, people who
nature dynamic—consequently, categorization help you find information. Corporate librarians
systems must be dynamic as well. are experts on how various categorization
schema are designed. They know how to find
Directory Building vs. Hierarchical information—that “needle in a haystack.” The
Categories idea of an automatic taxonomy system may at
Another argument in this field emphasizes the first seem threatening to them. The reality is that,
fact that directories are about stored things as because of the dynamic nature of information,
opposed to related concepts. Directories are these experts will become more and more
virtual bins and do not necessarily reflect a valuable as the amount of information
hierarchical relationship. Proponents of this view expands exponentially.
would simply ask you to look at the directory
structure on your personal computer. Carrying Users Needs and Personalized Taxonomies
this argument forward supports the idea of The needs of individual users represent another
customizing directories to help reflect your major aspect to examine. Will one comprehensive
unique view of the world. The alternative enterprise taxonomy address everyone’s needs, or
approach is to implement a strict hierarchy with will you need departmental taxonomies as well?
topics and subtopics organized in a strict Or will individual workers require their own
grandparent-parent-child structure. Delphi unique taxonomies? Or will your environment
believes that each approach has advantages and require a blend of all of the above? As you
disadvantages. The choice is one of working style investigate different taxonomy products, be sure
more than substance. to investigate how flexible the products are for
generating multiple taxonomies.
Granularity of the Taxonomy Structure
Although there are two distinct sides to this Speed, Accuracy, Robustness and
debate, your decision will place you somewhere Scalability
along a continuum of alternatives. For the There is no universally accepted standard for
purpose of this discussion, taxonomies can be evaluating the various algorithms or software
arbitrarily divided into three sizes by the number configurations in regard to speed, accuracy, and
of nodes or headings and subheadings: scalability. When your organization is in the final
• Small - 1,000 or less stages of evaluation and has developed its short
• Medium - 1,001 t0 20,000 list of vendors, Delphi Group recommends
testing the different solutions against a
• Large - +20,000 significant portion of your unstructured data,
There are many very large taxonomies. The letting your users verify that the documents are
proponents of large taxonomies say that more is categorized quickly and accurately and on a scale
better. Since the organization of information is that meets your needs.
hierarchical, the users can drill down to as much
detail as they wish. Levels of hierarchy greater
than 10 are not uncommon in implementations
on this scale.
At the other end of spectrum, proponents of
small taxonomies argue that more than five levels
once again confront users with a kind of Infoglut,
receiving too many hits on a search, too much
irrelevant information, etc.
54
discover information about a particular process

Future trends or a new product. If that same individual is doing
a competitive analysis, then they want to look at
Topic Maps information from that perspective and context. A
“Topic maps are a new ISO standard for number of companies (like Stratify, Semio, and
describing knowledge structures and associating Wherewithal) have added personalization
them with information resources. As such they functionality to their taxonomy products. Delphi
constitute an enabling technology for knowledge Group expects this capability to be incorporated
management. Dubbed ‘the GPS of the in more and more products in the future.
information universe,’ topic maps are also
destined to provide powerful new ways of Vertical Taxonomies
navigating large and interconnected corpora.”9 Some taxonomy providers (such as Semio) are
developing and marketing vertical taxonomies
The best way to describe how topic maps are geared toward a particular vertical market such
different from taxonomies is to go back to our as pharmaceuticals, financial, etc. The proponents
“chips” example. We may find that when we of such an approach say this will “jump start” the
develop a topic map around chips, there are process for your organization. Detractors,
recipes for chocolate chip cookies. A taxonomy however, maintain that each organization has its
would list the various recipes and supply the own culture and its own way of categorizing.
pointers to them. A topic map would “associate” a
white chocolate chip recipe with dark chocolate Trying to use someone else’s taxonomy can be
recipes and milk chocolate chip recipes. Then a like wearing someone else’s shoes—shoes that are
topic map is formed between “chips,” recipes for already molded to that person’s feet and will not
cookies, and various types of chocolate chips. fit another person even if the size is the same.
Despite this, each of the vendors supplying pre-
The components of a topic map are: built taxonomies do allow customization of
Topics - A “subject” (or more generally any organization and the naming of the nodes for
“thing”) associated with the topic is the name. each of the categories. Some of these currently
There are also distinct types of names – base available taxonomies are:
names, display names, and sort names.
• Hardware/Software
Occurrences - These are topics linked to one or • Healthcare/Pharmaceuticals
more information sources. • Telecom
Associations - An association is a link element • HR
that asserts a relationship between two or more • Financial/Investment Banking
topics, e.g. ‘Tosca’ was written by ‘Puccini,’ ‘Tosca’ • Legal
takes place in Rome, Rome is in Italy. • Sales & Marketing
For more information on this subject visit http:// • Geography
www.topicmap.com/, a site dedicated to the use of
topic maps. Other vertical taxonomies are: petrochemical,
sports, entertainment, religion, education,
Personalization sciences, Internet, insurance, construction,
paper products, food, and government.
“The right information, at the right time, for the
right person” has been the mantra of knowledge
management software developers for some time. Taxonomy Integrated With Applications
Taxonomies are by their very nature volatile, Delphi Group believes taxonomy software is an
contextually relevant and personal. An enabling technology. The long-term evolution of
individual’s shifting priorities and goals can at the market for taxonomy will be its integration
any given time change the way they want to view into applications such as Enterprise Portals and
information categories. If an individual is in an Content Management. The first integration of
education mode, for example, they may want to taxonomy technology will be with search and
55
retrieval software. We are already seeing this For instance, you probably do not want all your
trend as companies such as Semio and employees to see the list of personnel files that
Mohomine develop aggressive OEM programs. are assigned to specific categories of diseases. If
you look up oncology you certainly don’t want to
Expect to see taxonomy software increasingly see John Doe’s personnel file linked to that topic.
integrated with applications like these: Even if you don’t have access to John Doe’s
personnel file, the fact of its potential association
• Search & Retrieval with sensitive topics such as an employee’s
• Internet & Intranet medical condition is of obvious concern. A
• Portals number of technology providers (including
Autonomy, Convera, Semio, Stratify and Verity)
• Content & Document Management are expanding their security functionality.
• Supply Chain
• CRM & Business Intelligence Ontologies
An Enterprise Ontology is a collection of terms
Security and definitions relevant to business enterprises.10
Security issues are among the key topics on the An ontology is more than a taxonomy or
minds of information managers. Most taxonomy classification of terms. Although taxonomy
applications follow the standard security model contributes to the semantics of a term in a
for the server they run on within the enterprise vocabulary, ontologies include richer
system. This approach can be summarized by this relationships between terms. It is these rich
simple rule: if you have access to the area where relationships that enable the expression of
the document is stored, you have security domain-specific knowledge, without the need to
clearance to see the document. Delphi Group include domain-specific terms.11
believes this type of security implementation is
the minimum level allowable. Security issues An ontology is more than an agreed-upon
relative to unstructured data in the future will vocabulary, however. The terms in an ontology
involve at least three different aspects: are selected with great care, ensuring that the
most basic (abstract) foundational concepts and
distinctions are defined and specified. The terms
1. Security attributes of each individual document.
When the document is created, various
chosen form a complete set, whose relationship
properties will be assigned as to which one to another is defined using formal techniques.
individuals, which set of clearances, which It is these formally defined relationships that
departments, etc., will be permitted to view or provide the semantic basis for the terminology chosen.
edit the documents. In the context of knowledge sharing, an ontology
2. Security attributes of the individual. This type is a description (like a formal specification of a
of security will be based on clearance levels, program) of the concepts and relationships that
membership in particular departments or can exist for an agent or a community of agents.
assigned role(s) in the organization.
An ontology often takes the form of an extremely
3. Security issues regarding the overall operating large database of words and phrases, their
environment. Examples of these considerations meanings and their conceptual relationships.
will be factors such as what time of day access
is allowed, access from inside or outside the Examples of conceptual relationships are: a
firewall, number of documents accessed, etc. “commissioner” is a member of a “commission”;
“good” is an antonym to “bad”; and “lumber” has
substance, i.e. “wood.”
9
http://www.gca.org/papers/xmleurope2000/papers/s11-01.html
10
http://www.aiai.ed.ac.uk/~entprise/enterprise/ontology.html
11
http://www.ontology.org/main/papers/faq.html
56
Taxonomies in effect are simplified ontologies. End Note

Where taxonomies generally classify categories
in “broader” or “narrower” terms, ontologies can “Knowing what you know” —that is, enjoying
include more descriptive classifiers such as ready access to actionable information— is one
“located in” or “part of.” of the major factors determining success in
today’s knowledge-driven economy. The ability to
Beyond Text correlate the “whats” from relational structured
A few of the companies mentioned in this report databases to the “whys” embedded in the
(Convera and Autonomy, for example) are unstructured data of e-mails, reports, contracts,
expanding the concept of taxonomy to embrace presentations, and Web pages will be a primary
types of data other than text. Although somewhat driver of innovation and competitive advantage.
governed by the limitation of accurate As in the past, when just-in-time inventory was
conversion of multimedia files (consisting of critical to success, so today is “just in time
video and audio) into text, the classification knowledge” the critical factor.
engines can still perform adequately to
categorize these files by their content. The Delphi Group’s research shows that the
normal requirements of accurate conversion unproductive time spent looking for information
from audio content to text are not as stringent within the digital repositories of the enterprise is
when pattern-matching analysis is applied. For growing, and affecting more and more of the
example, one might view a news clip about expensive personnel we have managing our
former president Clinton and understand its organizations. Executives, managers and middle-
theme without accurately knowing each word office knowledge workers alike require
that was spoken. sophisticated new tools for the delivery of
relevant information.
Taxonomy software can reduce our reaction time

to make informed and timely business decisions
based on the knowledge and information
contained within the unstructured data of an
organization’s digital documents. This software
helps us form ideas from information we didn’t
know we had, while revealing relationships and
correlations that would otherwise be lost in the
ocean of information overload. Productivity and
innovation come from seeing connections,
evaluating importance, recognizing context, and
understanding the implications of these
correlations.
Although taxonomy software cannot completely

stem the tide of “Infoglut,” it can help us find the
information we need to survive and prosper in
the new knowledge-based economy—to truly
“know what we know.”
57
Vendor Contact Information:
Autonomy, Inc.
301 Howard Street, 22nd Floor
San Francisco, California 94105
Phone: (415) 243-9955
Fax: (415) 243-9984
E-mail: info@us.autonomy.com
http://www.autonomy.com
Convera
1921 Gallows Road, Suite 200
Vienna, Virginia 22182
Toll-free: (800) 788-7758
Phone: (703) 761-3700
Fax: (703) 761-1990
E-mail: info@convera.com
http://www.convera.com
Entopia, Inc.
1301 Shoreway Road, Suite 302
Belmont, California 94002
Toll-free: (866) ENTOPIA
Phone: (650) 632-0101
Fax: (650) 802-6709
E-mail: info@entopia.com
http://www.entopia.com
continued next page
58
Mohomine, Inc.
5120 Shoreham Place, Suite 200,
San Diego, California 92122
Phone: (858) 362-3000
Fax: (858) 362-3099
E-mail: mohoinfo@mohomine.com
http://www.mohomine.com
Quiver, Inc.
2121 El Camino Real, Suite 300
San Mateo, California 94403
Phone: (650) 653-3577
Fax: (650) 653-3578
E-mail: info@quiver.com
http://www.quiver.com
Semio Corporation
1730 South Amphlett Boulevard, Suite 101
San Mateo, California 94402
Phone: (650) 638-3330
Fax: (650) 638-3339
E-mail: info@semio.com
http://www.semio.com
Stratify, Inc.
501 Ellis Street
Mountain View, California 94043
Phone: (650) 988-2000
Fax: (650) 988-2159
E-mail: info@stratify.com
http://www.stratify.com
59
Textology, Inc.
150 Post Street, Suite 400
San Francisco, California 94108
Phone: (415) 710-6589
Fax: (253) 369-0253
E-mail: info@textology.com
http://www.textology.com
TopicalNet, Inc.
800 W Cummings Park, Suite 2900
Woburn, Massachusetts 01801
Phone: (781) 932-8400
Fax: (781) 932-2558
E-mail: info@topicalnet.com
http://www.topicalnet.com
Verity, Inc.
894 Ross Drive
Sunnyvale, California 94089
Phone: (408) 541-1500
Fax: (408) 541-1600
E-mail: info@verity.com
http://www.verity.com
Wherewithal, Inc.
2995 Woodside Road, Suite 400
Woodside, California 94062-2446
Phone: (415) 705-0726
Fax: (415) 723-7498
E-mail: info@wherewithal.com
http://www.wherewithal.com/
About This Document:
The product-specific
information contained in this
document is intended to
provide an overview of a
specific product and vendor
at the date of publishing.
Facts presented have been
verified to the best of our
ability with the vendor and
actual users of the product
where indicated, however,
Delphi cannot insure the
accuracy of this information
since products, vendors, and
market conditions change
rapidly. Delphi Group makes
no implied or explicit
warranties, endorsements, or
recommendations in this
report nor should such
warranties be inferred from
its contents. A complete
assessment of your specific
application, the method
of implementation for a given
product or technology, and
the current state of that
product must be considered
in order for a
recommendation to be made
on any product’s suitability
for your purpose, needs and
requirements.
Delphi Group is a leading

provider of business and
technology advisory services
to Global 2000
organizations. With offices
established around the
world, Delphi has assisted A Delphi Group
professionals across White Paper
disciplines and industries at
nearly every major national Ten Post Office Square
and global organization and
branch of government. Its
DELPHI
G R O U P
®
Boston, MA 02109-4603
v (617) 247.1511
clients and subscribers f (617) 247.4957
include more than half of the www.delphigroup.com
Global 2000.

Taxonomy & Content Classification: Market Milestone Report

Uploaded by

Document Informationclick to expand document information

Document Informationclick to expand document information

Copyright:

Available Formats

Taxonomy & Content Classification: Market Milestone Report

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Taxonomy & Content Classification: Market Milestone Report

Uploaded by

Copyright:

Available Formats

A Delphi Group White Paper

applications, ontology, summarization and the documents contained information about –

Ambiguity need is about flash memory “chips” It is much

services professionals, the R&D engineers and Evolution of Taxonomy Technologies

“or,” “not” type of qualifiers. These functions were Is “Taxonomy” a Misnomer?

The methodologies used in the automatic process

Browsing is dynamic. Information changes all the

Taxonomy Is Not Just For summer “monsoon” season in Arizona corre-

Question #2 – “How much time do you spend

“Infoglut,” the business problem of too much

Strongly The issue of “search time” is one of the

Question #3 – “The biggest impediment to frequently sited impediment to finding

Monthly Document Turn-Over

As respondents looked and searched for

The two most frequently cited reasons for this are

central message—that finding information is “Question #6 – What type of system for

It’s hardly surprising, then, that even where

The objective of this set of questions is to

Question #5 – “Does your organization provide

System for Classifying Documents

Don't Know For a slight majority, 55%, there is some system

Enterprise Organizations Deomonstrate Question #8 – “How important is taxonomy

Taxonomy Important to Business Strategy

Question #9 – “When will your organization

don’t know if they have taxonomy software being Next 6 nths

There are a number of taxonomy-related issues

levels of information are either necessary or Controlled Vocabulary and Thesauri

Taxonomy Market Landscape • Neural Networks

To help understand these methodologies,

Keyword Because of the proliferation of unstructured data,

Stages of Taxonomy Process

Process concepts to Develop

Classify corpus of data Display hierarchal Administration

• Application With GUI

presence or absence of individual words doesn’t

may want over a taxonomy component, while Clustering

After structuring the data into taxonomies, the

Customer Quotes Assessment

Steve Nauss, associate head of NASA’s Computing

John McCulloch, Manager, Executive

“Autonomy is a very powerful solution. It’s easy to

Bill Gafney, Web Specialist at Boston.Herald.com

Convera’s approach also means that

Technology Approach • Categorization to support browsing

This architecture allows Convera’s RetrievalWare Products

Convera’s advantage is that it embeds multiple

builder adds contextual information and pairs it

While other available search tools may stop there, Assessment

advantages of neural network are speeds up to

The process is self-teaching through a machine Vision

To augment the neural network methodology, Mohomine’s senior management vision is to

1. Parsing or tokenizing converts the documents

multi-tiered hierarchy. The methods of

Keeping up-to-date is no problem as the design

Products Customer Study

with patented technology built into the Products

Stratify also provides a Classification Server that