Remote Search Solutions for Different Sized Organizations

Introduction

Business thrives on information, so keeping track of information is a key for success in business. Unfortunately the amount of information businesses need to keep track of has been growing exponentially in recent years. It wasn’t that long ago that one gigabyte was considered a huge capacity for a hard drive—how could the average PC user ever hope to fill up such space? Today hard drives a terabyte or more are common, and with advanced format drives now becoming the norm, it may not be long until the average PC’s hard drive is in the low double-digit terabyte range.

Many business environments are configured so that data is stored on central servers instead of locally on the hard drives of end-user PCs. This is done to facilitate backing up critical business data, and to provide the business with tighter control over who can access the data. The servers used for storing data can be file servers, Web servers, SharePoint servers, and so on. Network Attached Storage (NAS) and Storage Area Networks (SAN) may also be used.

More and more businesses today are storing their data in the cloud. This may be a private cloud (an enterprise datacenter at a remote location and owned by the business) or public cloud via storage services provided by a third-party such as Microsoft, Google, Amazon, and so on.

Given the complexity of today’s business environments, which can include sensitive business data stored on PCs, on servers, or in the cloud, how can businesses implement a solution that enables their users to quickly search for and find the information they need? The reality is that there isn’t a one-size-fits-all solution for this. Instead, the search technology you use should be chosen to match the size and complexity of your information environment. But regardless of the size and complexity of the business environment, the best search solutions involve having Windows 7 deployed on end-user PCs, so each of the solutions described in detail below assume that this is the case.

Small Business Solution: Offline Files

If the users store business data on their own PCs (which although not a best practice is nevertheless quite common in very small business environments that use workgroups instead of domains) then Windows Search, the built-in search capability found in Windows 7, together with Offline Files, which has been enhanced with new capabilities in Windows 7, is a simple and effective solution. And if the business stores user data in shared folders on file servers, for example by using Folder Redirection, then Windows Search together with Offline Files is again a simple and effective solution.

Key to this simplicity and effectiveness is that Windows Search in Windows 7 integrates with Libraries, a feature first introduced in Windows 7. A library acts as a kind of virtual container that can include content from both local and remote locations. For example, the Documents library on a user’s PC could include:

• Folders on local hard drives such as C:\Data
• Folders such as My Documents that have been redirected to a file server using Folder Redirection
• Shared folders on file servers

Network locations such as shared folders on file servers can only be added to libraries as locations if the content in the folder is indexed, either on the user’s local machine as described below or on the file server as described later on in this article. 

Implementing this solution works something like this:

1. The administrator must first make the remote content available over the network, for example by sharing a folder on a file server. The file server can be running any version of Windows Server and can even be a third-party network storage appliance.
2. Once the user can access the remote content using Windows Explorer, the user must manually mark the files as “always available offline” using the Offline Files feature of Windows 7. When a file is marked as “always available offline” a copy of the file is present in the client side cache (CSC) on the user’s computer. Since the CSC is included by default as a search scope in Windows 7, the cached file is indexed locally on the user’s machine. 
3. Once the remote content is indexed locally on the user’s machine, the user can manually add the remote folder as a location of their Documents library. Then when the user searches for content in their Documents library, the query is performed against the search index on the local machine, which includes the remote folder as one of the indexed locations.

The main benefit of this solution is that he file server can be running any version of Windows Server and can even be a third-party network storage appliance.

The downsides of this solution are:

• It doesn’t scale very well and works best when using Folder Redirection of known folders such as My Documents, My Pictures, and so on.
• Some manual effort involving expertise on the part of the user is needed, but this is often the case in small business environments.
• This approach doesn’t work with DFS—you can only query remote folders directly, you can’t query DFS links. So no fault tolerance is provided by this solution, but small businesses typically don’t use DFS anyways.
• It doesn’t work with Windows clusters either. So no high availability either with this solution, but small businesses typically don’t use clustering either.
• It doesn’t work with SharePoint sites, Exchange mailboxes, Windows Live SkyDrive, and so on. So not everything can be indexed.

Small- to Mid-Sized Solution: Server Indexing

For businesses that are somewhat larger and more complex, Windows Search together with server indexing may be the best solution. Server indexing refers to using the Windows Search Service on the server to index content stored in shared folders on the server. Unlike the previous scenario where the remote content is indexed locally on the user’s computer by caching it using Offline Files, here the remote content is indexed remotely on the file server itself.

Implementing this solution works something like this:

1. The administrator begins by installing the Windows Search Service on the file server. The Windows Search Service is an optional role service of the File Services server role of Windows Server 2008 R2.
2. The shared folder on the file server that contains the content that users need to be able to search must then be manually added as a search scope for the Windows Search Service on the server.
3. The administrator then uses Group Policy to add the shared folder as a location of the Documents library on users’ computers. Alternatively, the user can manually add the remote folder as a location of their Documents library.
4. Now when the user searches for content in their Documents library, the query is performed against both the search index on the local machine (for local content) and against the search index on the file server (for the remote content).

The benefits of this solution are:

• It works with any storage technology that lets the server volumes where the remote folders reside appear as local drives attached to the server. For example, this approach will work with iSCSI.
• As an administrator you have a great deal of control over how you implement this solution for your users.
• Little or no manual effort or expertise on the part of the user is needed to get things working.

The downsides of this solution are:

• Your file server must be running Windows Server 2008 R2 to take full advantage of the powerful capabilities of Windows Search.
• It still doesn’t scale very well—the Windows Search Service was originally developed as a desktop search technology and is therefore best suited for desktop search scenarios or for small file servers. So don’t try and use this approach in a large enterprise environment.
• It doesn’t work with DFS, clustering, SharePoint, Exchange, SkyDrive, and so on.

Large-Scale Solution: SharePoint or Search Server

If your business needs a search solution that scales to the level of enterprise environments, then you need something more than the built-in search and organization features found in Windows 7 and Windows Server 2008 R2. Here are the options available:

• SharePoint Server 2010 – You can let SharePoint index all your content or store your content in SharePoint itself. But make sure you choose the SharePoint edition that has the search capabilities you need. That’s because different SharePoint editions—Foundation, Standard and Enterprise—have different search and organization capabilities. See here for details.
• FAST Search Server for SharePoint 2010 – FAST Search Server builds upon SharePoint technologies by including a connector framework that lets you index files and metadata from various types of content repositories, stores the indexed data in an efficient manner, and provides query servers, object models, and user interfaces to facilitate searching the indexed data. In other words, if your enterprise data environment is complex and varied and you want to build a custom search solution, FAST Search Server may be the best choice.
• Search Server 2010 Express – This is a free solution you can download and quickly customize to meet your enterprise search needs. It lacks integration with SharePoint’s My Site feature, can’t do people search, and has a few other limitations compared with SharePoint Server itself, but it’s free and may be a good choice that can meet your business needs.

The main downside of these large-scale search solutions is that they don’t integrate out-of-the-box with the Libraries feature of Windows 7. But you can however use the new Federated Search capabilities of Windows 7 and Windows Server 2008 R2 to create search connectors that can enable your users to search SharePoint sites directly from the familiar Windows Explorer user interface. 

Conclusion

Choosing the best search technology for your business depends on many factors including the size and organization of your business, the scope and nature of the corpus that needs indexing, the level of control you want both administrators and end-users to have on how searches are performed, and many other considerations. This article should help provide you with a starting point on your journey towards implementing a search solution that best meets the needs of your business.