Quick guide to data retention tiers

Data retention is now an umbrella term applied to the two different types of ‘data storage’, back up and archival.  Most companies now require some sort of data retention system which allows for both back up and archival, but often only have back up capability.  This quick guide will help to clarify the three different tiers involved in a complete data retention environment.

Tier 1 (On-line storage)

Tier 1 is really meant to be for back up.  The kind of data you would normally store here is information you need to access infrequently but may need to change, also data which needs to be kept for a short time and then deleted, e.g. if you’re backing up a database every day, you would normally not wish to keep hundreds of copies and would overwrite the previous days back up.  Traditionally the best system to use for tier 1 is RAID (redundant array of independent discs), RAID being a collection of hard drives with different RAID levels being available (striping, parity, fault tolerance etc) Unfortunately for many companies this is where their data retention ends, which results in clogged up hard drives populated with data that must be kept and active data.  In this instance, performance suffers (drive failure being a common problem), and because of the cheap price of discs, companies simply add more and migrate.  This turns into a vicious circle with continual expense, migration and management needed.
To really get the best out of a tier 1 system, it needs to combined with tiers 2 & 3.

Tier 2 (Near-line Storage)

This tier is the first level of archival.  Data held on tier 2 systems must be kept (for compliance reasons, value to the company etc), but doesn’t need to be regularly accessed and shouldn’t be changed.  This is usually financial records, transactions, CAD, company records and the like.  Near-line storage has many benefits; less stress on the server, content held on tier 1 is moved and frees up more space, data can be quickly accessed among others.  The preferred media of Near-line storage is either Tape or Optical.  The advantages of Optical over Tape are discussed in the article ‘Five Step Storage Strategy’ on this site.  The most important things to consider when looking at Near-line Storage systems are; how often will I need to migrate data and how much management does this system need?  An important feature of Near-line storage is that most systems will appear as a shared drive, so employees within an organisation will not be aware that their data is sitting on a tape or Blu-day disc.  This may not seem a big deal, but in many organisations when personnel have been made aware they are putting data onto an archive media; they actually stop and start using the RAID system again because of fear of retrieval, wasting space etc.

Tier 3 (Off-line Storage)

Off-line storage is extremely important to any organisation.  Providing it is properly implemented, off-line storage will serve as your disaster recovery should you have a fire, flood or suffer any other disaster which results in data loss.  This works very effectively when combined with a Near-line system which offers mirroring.  When data is written to the Near-line system, a duplicate copy is created allowing the IT Manager (or whoever is managing the system) to remove the duplicate and store it off-site.  Again many companies will only create one copy of data they need to keep and you can equate it to putting all your eggs in one basket.  Having tiers 2 and 3 really is the best way of ensuring you have copies of your valuable data.