Enterprise storage best practices

If you would like to read previous articles in this series, please go to:

• Enterprise hard drives
• Choosing a storage solution
• Data tiering and overprovisioning
• Data tiering strategies
• Data tiering and service level agreements
• Avoiding storage failures

In this series of articles we’ve been examining various problems that organizations often face when it comes to data storage. We’ve also examined some possible solutions to these problems and how they can be implemented for both large enterprises and smaller businesses.

It’s time now to pull all this information together and establish some best practices for data storage in enterprise environments. We’ll do this by considering strategies and recommendations that address five common concerns that businesses have concerning their data:

• Performance
• Availability
• Reliability
• Searchability
• Cost

The infographic below summarizes some best practices for these five areas, while the sections that follow go into more detail concerning each of these areas.

Figure 1: Some best practices for enterprise storage.

Performance

Performance is a critical issue when it comes to storing and accessing business data. If information about a product or service can’t be quickly retrieved when a customer requests it from your website, the customer is likely to become impatient and leave to look for a different company whose site performs better.

Performance is important throughout the entire process by which a business transaction takes place. This means that for optimal data storage, transfer and processing performance your infrastructure must have:

• Storage devices that can quickly perform reads and writes.
• Server systems that can quickly process business transactions.
• A networking infrastructure that can move data quickly between servers with low latency.
• An authentication infrastructure that can quickly determine whether or not a user should be allowed to access a particular data entity.
• A high-speed corporate connection to the Internet that can route customer requests/responses to/from your infrastructure without noticeable delay.

Also important of course are qualified IT staff that can quickly resolve infrastructure problems, and trained helpdesk personnel who can quickly resolve customer issues. These two last things are often not considered when it comes to the performance of business applications and services, but they are definitely important.

What are some strategies and steps you can take to increase your storage performance?

• Enterprise hard drives – Use 15,000 RPM hard drives in specific areas of your infrastructure where they can provide benefit.
• SSDs – Solid state disk (SSD) drives are rapidly increasing in capacity and coming down in prices. This makes SSDs a promising alternative to enterprise hard drives for high transaction volume servers in the datacenter.
• Storage consolidation – Consolidate any direct attached storage (DAS) and internal server RAID storage to a storage area network (SAN).
• Data tiering – Migrate stale data to slower, less expensive Tier 2 storage devices to reduce the amount of data stored on faster, more expensive Tier 1 storage devices. Offloading stale data like this from mission-critical servers will reduce the workload on these servers and thus increase performance for data processing.
• Storage monitoring – Implement performance monitoring for your storage infrastructure and devices. SAN vendors typically include such monitoring software as part of their solution, so all you need to do is to use it.

Availability

Being able to quickly read, write, store or transfer data is useless if your storage devices are offline when you need to access them. Ensuring high availability is therefore clearly another best practice when it comes to enterprise storage.

What are some strategies and steps you can take to increase your storage availability?

• Retire old storage hardware – Get rid of old storage hardware before it fails on you and wreaks havoc with your business. Use SMART monitoring and other tools to identify and weed out hard drives before they fail.
• Standardize on new storage hardware – Replace your mixed bag of old types of storage hardware with standardized new types of hardware. It’s much easier to troubleshoot storage failure when you only have a handful of different types of devices in your enterprise, and faster troubleshooting means increased uptime.
• Failover clustering and load balancing – Deploy solutions like Failover Clustering and Network Load Balancing (NLB), both of which are included with the Windows Server platform, to ensure your servers and the storage infrastructure they access are redundantly and highly available.
• Service level agreement – Implement SLAs for your storage infrastructure. Achieving 100% availability is an unrealistic goal and is also unnecessary for stale data stored on Tier 2 or 3. Set appropriate SLAs for each storage tier and processes for monitoring and ensuring these SLAs are met.

Reliability

Data that is always available and can be quickly accessed is useless if the data is lost or corrupt. Ensuring the reliability of your business data is therefore another important best practice for enterprise data storage.

What are some strategies and steps you can take to increase your storage reliability?

• Retire old storage hardware – Aging storage hardware often becomes more and more unreliable as time goes on. Solve the problem before it blows up on your by replacing it with newer standardized storage hardware.
• Storage monitoring – Monitor your storage infrastructure to ensure your data is not becoming corrupt. This can be as simple as running Chkdsk periodically to remap bad sectors on hard drives, or you can use software provided by your SAN vendor for automatic monitoring and repair of data corruption.

Note:
Chkdsk performance has been drastically improved in Windows Server 2012. You can find more information about these improvements here.

Searchability

Your storage infrastructure is reliable, highly available, and performs well. But what if you can’t find data when you need it? Searchability is an issue of growing importance for enterprises as the volume of business data they need to store and process continues to grow exponentially. Searchability is also tightly linked to performance, because customers are likely to abandon you if it takes too long a time for a business transaction to be completed on your website because of difficulty finding the data involved.

What are some strategies and steps you can take to increase your storage searchability?

• Storage consolidation – Instead of using DAS or internal server RAID and storing your business data across a number of different silos, you should consolidate all business data into a single storage array to make it easier to find data when you need it.
• Data tiering – It’s much easier to find a needle in a handful of straw than it is to find it in a haystack. Tier 1 storage is typically much smaller than Tier 2 storage and is therefore easier to search when you need to find some particular data. Searchability and search performance therefore can both improve if you implement a data tiering solution for your organization.
• Enterprise search tools – Big data needs big search tools to be able to drill down and find exactly what you’re looking for. Microsoft SharePoint 2013 includes powerful tools for enterprise search. You can find more information about SharePoint 2013 here.

Cost

Finally, a good enterprise search infrastructure shouldn’t just be highly available, reliable, searchable, and perform well. It also has to not cost an arm and a leg. Of course cost is a relative thing because it relates to both how big you are and how important your data is. But all businesses need to do more with less nowadays, so cost is always a key consideration when it comes to choosing an enterprise storage solution.

What are some strategies and steps you can take to control your storage costs?

• Data tiering – Keeping all your business data on enterprise HDDs or SSDs is not only very expensive but also unnecessary. Move your stale data to cheaper, slower Tier 2 large capacity HDDs, and only keep hot data on your smaller Tier 1 storage.
• Storage Spaces – Storage Spaces is a new storage virtualization technology included in Windows Server 2012 that lets you provision storage easily from pools of storage backed by a mix of different types storage devices such as SATA and SAS drives or even USB drives. Storage Spaces is a good option for smaller companies to consider and can be much cheaper than a SAN while providing similar levels of performance. For more information on Storage Spaces, see this link.
• Cloud storage – Instead of building and managing your own storage infrastructure, why not consider moving some or all of it to the cloud? In fact, you can move your whole data processing infrastructure into the cloud and have your line of business applications running in the cloud. Microsoft has a whole range of cloud solutions which are nicely summarized here.

Conclusion

This series of articles has examined some of the problem areas and pain points of enterprise storage and has offered some recommendations on how to address these areas.

For some additional articles on Windows Server storage solutions, see my series of articles on iSCSI Storage.