Microsoft

Configure Windows XP's Indexing Service the right way

Little used and much maligned, Windows XP's Indexing Service can really help you find data on workstations if you use it properly.

Posted by Mark Kaelin

This article was originally published on TechRepublic in February 2005 by a guest contributor.

If you're like most Windows XP users, you've probably read that the Indexing Service is unnecessary and that disabling it will improve the operating system's overall performance. Although these statements aren't entirely true, most people buy in to them because they really don't understand the benefits of using the Indexing Service.

If you regularly search for specific content in data files on your multigigabyte hard disk, the Indexing Service will definitely save you time and therefore can be considered a necessary service. In fact, I have literally thousands of articles in Word documents, and I've discovered that searching for content with the Indexing Service turns up results hundreds of times faster than using a standard search engine.

Furthermore, like a screen saver, the Indexing Service is designed to perform its intensive hard-disk indexing operations when the computer is idle. The service pauses whenever it senses activity from the keyboard or mouse, so it will never drag down system performance when you're using your computer.

Of course, like all tools in the Windows operating system, the Indexing Service's default settings can be tweaked in order to improve performance. For example, in its default configuration, the service is designed to search through all the files on your entire hard disk as it performs its indexing operation. However, the service is much more efficient if you redirect the indexing operations toward the directories, or folders, in which you store your data files.

In this blog post, we'll show you how to redirect the focus of the indexing operations and discuss other ways you can configure the Indexing Service so that it performs more efficiently.

How it works

Let's begin with a brief look at how the Indexing Service works. To provide such fast turnaround time, the service actually performs a comprehensive search operation during the indexing phase, when it goes through the files on your hard disk and gathers detailed information on each file. As it does, it creates a catalog, or database, of the results. When you run a search operation, you're actually searching through the database rather than searching through the files on your hard disk.

To determine what types of files to search for and how to search them, the Indexing Service uses a set of preconfigured filters. By default, the service comes with filters for HTML files, text files, and all files created by Microsoft Office, including e-mail. The filter also allows the service to isolate the meaningful text from other content in the document, such as header and formatting information.

In addition to using filters to determine how to search a file, the service has a list of noise words -- words that don't need to be indexed. Words in the noise list include prepositions, common verbs, conjunctions, pronouns, individual letters, and numbers.

The service also gathers other pertinent information about the file. For example, it indexes standard information, such as file size, time and date stamp, and file type. More detailed information about a document includes word count, author, or even the last time the document was printed.

Works best on NTFS

While the Indexing Service will run on any hard disk regardless of the file system, it runs best on NTFS volumes, where it can take advantage of several advanced NTFS features. The most beneficial of these features is the NTFS Change Journal.

As its name implies, the Change Journal keeps a constantly updated log of any file deletions, additions, or modifications to all folders on the NTFS volume. The Indexing Service can consult with the Change Journal to help determine when new files have been added or when existing files have been modified; it can then proceed to update the index.

As you can imagine, a database of all the files on your hard disk could become extremely large. To deal with the size issue, the service stores its database using NTFS Sparse File, a feature that stores files in a way that requires much less disk space than if they were stored on a FAT-formatted hard disk.

Taking a look around

Now that you have a good idea of how the Indexing Service works, let's take a look at its default configuration via the Indexing Service console.

To launch this console, go to the Run dialog box on the Start menu, type Ciadv.msc in the Open text box, and click OK. Once the console is up and running, select the Show/Hide Console Tree button on the toolbar to show the console tree. You'll then see the screen shown in Figure A.

Figure A

By default, the Indexing Service is configured to index the entire hard disk.

Select the Directories folder in the tree to see a list of all the directories the service is configured to monitor. While these default settings ensure that data files on this computer will be included in the catalog, it forces the service to spend a lot of time searching through folders containing files that might not have the data you want to index.

For example, the C:\Windows folder on this particular system contains close to 17,000 files, which would take a lot of unnecessary time and disk space to index. The C:\Documents and Settings folder comes much closer to the data I'd like to index; however, this folder contains nearly 34,000 files, many of which aren't data files.

Redirecting the indexing operations

To make the Indexing Service as efficient as possible, you should redirect the indexing operations to the My Documents folder, which contains all your data files. This will make the service much more efficient.

First, delete all the current entries in the Directories list by right-clicking on each and selecting the Delete command from the menu. As you do, the Indexing Service will prompt you to confirm the deletion.

To add an entry for the My Documents folder, go to the Action menu and select the New | Directory command. In the Add Directory dialog box, select the Browse button and type the path to the My Documents folder into the Path text box, as shown in Figure B. Then, click OK.

Figure B

You can redirect the Indexing Service so that it focuses on the folders containing your data files.
At this point, minimize the Indexing Service console, launch Windows Explorer, and locate the My Documents folder. Now, right-click on the My Documents folder to access the Properties dialog box. Click the Advanced button. In the Advanced Attributes dialog box, shown in Figure C, select the For Fast Searching, Allow Indexing Service to Index This Folder check box, and click OK.

Figure C

To ensure that the folders and files are ready for the indexing operations, you need to set the Index attribute.
To continue, click OK in the My Documents Properties dialog box. You'll then see the Confirm Attribute Changes dialog box, shown in Figure D. Make sure the Apply Changes to This Folder, Subfolders and Files check box is selected and click OK.

Figure D

You'll be prompted to confirm that you want to set the Index attribute for all subfolders and files.
Return to the Indexing Service console and select the Indexing Service on Local Machine folder in the tree. You'll then see the main System catalog, as shown in Figure E. Now click the Start Indexing button on the toolbar.

Figure E

To initiate the indexing operation, just click the Start Indexing button on the toolbar.
The Indexing Service will begin the indexing operation, and you should leave your system site idle for a while. Ideally, you'll allow the service to run while you're at lunch or even overnight in order to give it time to build the index. While the indexing operation is running, the service will keep you apprised of its status, as shown in Figure F.

Figure F

While the indexing operation is ongoing, the Status column will show that it is scanning.
When the indexing operation is complete, the Status column will list the operation as Started, as shown in Figure G. You can then close the Indexing Service, launch the Search tool, and begin searching your indexed files.

Figure G

Once indexing is complete, the Status column will indicate that the index is ready to be searched.

Adding file types

As I mentioned, by default the Indexing Service comes with filters for HTML files, text files, and all files created by Microsoft Office. If you want to index other types of files, you can find and install custom content filters, also called IFilters. Because Microsoft's SharePoint Server also uses IFilters, you can find them for a lot of common file types.

For example, you can download and install a free PDF IFilter from the IT solutions page on the Adobe Web site. You can download and install a free Visio IFilter from the Downloads page on the Microsoft Web site. From the 4-Share Web site, you can download a 30-day evaluation copy of an IFilter for ZIP files. You can also purchase a number of IFilters on the IFilterShop Web site.

Adding unknown extensions

You can configure the Indexing Service to index files with unknown extensions. In this case, the service will analyze the content of the files and try to locate pertinent information.

To do so, run the Indexing Service console as described earlier. Select the Show/Hide Console Tree button on the toolbar to show the console tree. Right-click on the Indexing Service on Local Machine folder in the tree and select the Properties command. In the Indexing Service on Local Machine Properties dialog box, select the Index Files with Unknown Extensions check box, as shown in Figure H, and click OK.

Figure H

If you want the Indexing Service to attempt to pull relevant information from unknown file types, you can enable the Index Files with Unknown Extensions check box.

Stay on top of the latest XP tips and tricks with TechRepublic's Windows XP newsletter, delivered every Thursday. Automatically sign up today!

35 comments
jeslurkin
jeslurkin

I don't use it,... and may never. I, nonetheless, enjoy reading the pro and con opinions.

glnz
glnz

Does Indexing Service index Outlook Express? (Outlook Express emails are in .dbx files.) If not, how can I make it do so? Need step-by-step instructions. (I know where the .dbx files are located and added that Directory. Question is whether Indexing can then read them at all.) Great article, by the way!

techrepublic
techrepublic

I think it's useful for some, but not all users. I don't use it personally, but I may start using it for my documents.

1PebKac
1PebKac

Just use Agent Ransack - MUCH faster, - even finds text and even displays the line with the text searched for in the window! This has been installed on my machine since XP came out and they changed the default search!

gromit
gromit

I keep no data under MY DOCUMENTS (they are on a separate hdd) but I do want to include my Outlook daya in the indices. Can you tell me where that data is kept (so I can include it in the Search indexing)? Many thanks.

RudHud
RudHud

First of all, count me among those who find typing Ciadv.msc into the Run box to be unacceptable. I am not a UNIX nerd who gets off on memorizing obscure codes. Someone should inform MS that Windows has a GUI that handles this. My problem with Windows Search is that typing in conditions in the search box yields instant-run weird garbage, and hitting "Help" gets me stuck in endless loops of "click here for details". Organized sub-directories are the best solution. This fails so seldom that, when a search is necessary, some sort of advanced grammar is needed. If Windows Search has this, good luck finding it in the help. But thanks for the reminder to uninstall it!

listerfarrar
listerfarrar

I use google desktop everyday, and find it pretty good at finding what I need on my home laptop I use for consulting. How does Windows indexing this compare? Also, I've heard government and business systems administrators won't let users download google desktop, not sure why. Is windows indexing more acceptable should I go back to a company computer?

fedm235
fedm235

Does this information apply to Windows Search 4.0? The controls are not the same on my XP system with 4.0 installed.

jf
jf

Although indexing theoretically cuts in only when the system is idle, this is not entirely true. If you are using a programme that needs as much active memory as possible indexing seems to keep cutting in and out to check whether it is alllowed to get going again. I use Dragon Dictate for long documents because I am slow and inaccurate typist. Indexing seems to disturb this programme a lot and since disabling indexing I have had significantly faster and more accurate dictation results. I have a computer with a lot of avlaible RAM specifically to benefit Dragon Dictate but indexing seemed to completely negate the benefits of having all that RAM available. That is really the problelm of having buckets of stuff ticking away in the background; if you want one particular programme to get as much working memory as possible you cant do it.

Ajax4Hire
Ajax4Hire

Another example of the frustration that is Microsoft. Although Microsoft CE,ME,NT(cement),XP,Vista,7 may be the default operating system, the fact that you have to Start/Run ciadv.msc says that Microsoft does not trust users. How many hundreds-100s of .msi and .exe is in the /windows directory that could be useful? If it is considered a useful feature that users would like or need to use, put it in the Start/ Administrator or Accessories. More importantly, because of the drag on system resources, I cannot run Indexing Services. Try explaining your instructions to the rest of my family or your grandmother. No, better to full disable Indexing and use GoogleSearch. Not everyone drives a Ferrari of a PC that Microsoft develpoers have. Where Windows falls down is not the the uber-sleek Gaming/Developer PC but in the millions of older Dell/HP/WhiteBox hand-me-downs that all of our family, friends and neighbors have. You sly dog, now you got me monologging!

Rob C
Rob C

Banish it to the sh...house There is a free program ('Everything') that finds files by name. You just type part of the name at the top, and it instantly lists matching files - http://www.voidtools.com/ It does not search file contents. If you need that (which we all do), I use XYPlorerFree. You may have to use sites that have lists of last free versions. It is a brilliant program that leaves Windows Explorer for dead. (Multiple Tabs, Favorites, Coloring of various things, etc) You just position on the drive or folder you wish to search, and then hit F12 A large dialog appears at the foot of the screen, allowing you to set your search preferences. (MS should study it, and learn from it) When you are finished, F12 closes the Search dialog. It does not require indexing, so it is not instant. However it is unbelievable fast, for a raw search. The download at the top of this page, appears to be that last free version (5.55) - http://www.portablefreeware.com/index.php?id=579 Grab it while you can. For those that miss out, it is definitely worth buying.

ian
ian

I always have and find it very useful, but sometimes a bit resource hungry.

nw372
nw372

I do now. Noticeably quicker search time.

CharlieSpencer
CharlieSpencer

I don't think indexing is possible in Express out of the can. Maybe there's a third-party app or plug-in.

seanferd
seanferd

In the user profile folder → \Local Settings\Application Data\Microsoft\Outlook\ I don't know if it can index inside Outlook storage files, though.

Tank252ca
Tank252ca

"If it is considered a useful feature that users would like or need to use, put it in the Start/ Administrator or Accessories." Start / Control_Panel / Administrative_Tools / Computer_Management. Expand Services and Applications and Indexing Service is listed there. Took me 10 seconds to find it, although the average user would not know to look there. Your other points are valid. It's tucked away in Computer Management because it's too complex for the average user to configure, and it would probably consume less resources if the default configuration only included data files in My_Documents.

CharlieSpencer
CharlieSpencer

Man, I'd rather talk someone through configuring Indexing than install a third-party desktop search app. If you think Indexing is a drag, wait until that app calls home to the mothership to invite its little friends.

In2TheBlues
In2TheBlues

I've been using Everything for quite some time know but had never heard of XYPlorer before. I just followed the link that you provided and tried it. I think it's awesome! I will use both now...thank you very much for the reference.

wwgorman
wwgorman

I couldn't find a "safe" site to download a free copy (Be aware of possible viruses and Trojan horses from some sites) so I went to the authors site and downloaded the trial version which is good for 30 days and then costs $29. Unfortunately, it uses the Windows Explorer as its engine so when backing up multiple files onto multiple drives you must laboriously confirm each file backup or transfer. This is not like Norton File Manager which was designed for Windows 95 but still works as written in Windows XP and with a bothersome meaningless error message in Vista. Norton gives you a choice of "Yes to All" and you backup multiple files on a single click. Too bad Norton hasn't been updated although at least one German company has "adopted" a similar name for its File Manager version which doesn't work either. For search "Everything" is better than XYPlorer.

mik3
mik3

Please Remove the SPAM posts to this article !!! Don't give them free advertising !!!

dougogd
dougogd

If i need to find files the file i need to find are always in system directories because they are files that should not even be on my computer in the first place. They are usually programs or files installed by a program which were installed against my wishes and i have to search them down and delete them. As far as searching for my files I don't have a need to do that since i keep my files organized and in certain folders so the indexing for me is a total waste of time, as well as a search program that refuses to search in system folders(Like windows 7 and vistas search).

sgivens
sgivens

I attempted to run through the steps in the article but in our environment the My Documents folder actually sits on a network drive and it wouldn't accept "F:\" for a directory. Is there any way around that?

IndianaTux
IndianaTux

I disable it on all my user's PCs for one simple reason. All of our users either keep their important documents in their My Documents folder, on their desktop, or in a folder on a network share. They very rarely ever use the search function within Windows to find files. Plus, our desktop PC users have their My Documents folder redirected to their home directory on the network. I'm sure if you have a lot of files on your hard drive and have to search through them frequently this has the potential to save you some time, but for our specific situation there is no value added.

rickclark9
rickclark9

Let's see, turn on indexing and slow down already 7 year old computers...what an idea!

Rolland St-Onge
Rolland St-Onge

In the past, I found it to be a major pain in the ***. Since I got me a new PC (Dual core - 4GB ram - 500GB HDD) and started to build the company's knowledge base and file management server, I've ended up filling my drive with 285GB of documents, MS office templates, marketing panphlets and various database file. I then read about the benefit of index service and decide to tweak it for better use and you can't imagine how easy it is now to search thru all the documents and data on my drive. I can find specific documents based on content in 25% of the time it would have taken me if I'd use the old search engine without the indexing service. Overall I think that if you have a huge amount of data and documents on any drive (internal or external), using indexing service is a plus.

CharlieSpencer
CharlieSpencer

I disable it on all client systems here. Everyone's My Documents are redirected to a network share.

stuartc
stuartc

Since the start of XP I've never used Indexing Service, and never will. I find that giving the folders and file names descriptive name is much beter than relying on IS. And the best software that I've seen in a very long time is Everything (www.viodtools.com), gives all the files on your NTFS drive in a split second, Bril!

Rob C
Rob C

There is a link lower in that page, that may not be trustworthy. However there is the 'official' download button at the top of the page which gives this download link - http://www.portablefreeware.com/download.php?id=579 You don't trust www.portablefreeware.com download ? PS That version of XYPlorerFree (5.5), does not require installation.

J.C.Alexandres
J.C.Alexandres

Use the UNC path ... It will let you add it then.

CharlieSpencer
CharlieSpencer

I can't speak for Vista; I didn't try it on the single Vista system we had. You can index a network drive in W7; I tripped over it when investigating Libraries. I didn't turn it on since I don't search by content, so I don't know about the performance impact.

wwgorman
wwgorman

Thank you for the reference. The Everything search engine scanned my 10 (yes, TEN) NTFS hard drives in less than 3 minutes and came up with 927,566 "objects." Actually, it is still working, slowly, and finding more "objects." By-the-Way I have back up the many drives for my data files. Some of the drives are resurrected from "dead" and that's why so many. In a flash one day I lost a main hard drive and three external hard drives----with all my photos. I was able to recover most of the data files and I think all my photos to now be placed on CD's or DVD's for the future. Several of the hard drives are ones I carry with me when I travel with my Netbook which has only a 160GB hard drive that I try to keep relatively free for faster operation

beck.joycem
beck.joycem

I'm not denying the occasional potential usefulness for one minute, but SO often, when we're overhauling people's PCs, an instant stunning performance improvement is gained just by unticking that little box! The problem is that the default setting - indexing every little bit of drive c: - is barmy. Why can't it default to indexing My Documents only? No, wait a minute, then we wouldn't look so good when we take it out!

wwgorman
wwgorman

I than you for the reference. I've downloaded it.

Editor's Picks