Saturday, December 28, 2013

Data Deduplication

Data de-duplication is a technique to reduce storage needs by eliminating redundant data in backup environment .
data deduplication involves finding and removing duplication within data without compromising its integrity or fidelity .
only one copy of the data retained on storage media and redundant data will replace with a pointer to unique data copy .

How data deduplication works?
Deduplication segments an incoming data stream, uniquely identifies data segments, and then compares the segments to previously stored data. If it found the segment is unique , it store on disk , if an incoming data segment is duplicate of already stored data , it create a reference pointer instead of store whole data .
Dedupe technology divide data into chunk and use a algorithm to assign a hash identifier which it compare to previously store to determine if the data chunk already been stored .Each chunk of data is processed using a hash algorithm such as MD5 or SHA-1. This process generates a unique number for each piece which is then stored in an index.

Deduplication Benefit -
         -Eliminating duplicate data will shrink storage requirement and bandwidth efficiency .

Vendor -
1.-EMC Data Domain - application-agnostic deduplication storage system can attachable as virtual tape library over FC or iSCSI  . Its Real-time Inline DeDuplication and is targeted for secondary (backup) data and optimized for this flavor of data
2. Netapp ASIS IS Deduplication.

Faq-
- Data Deduplication won’t work with compressed and encrypted data.

No comments:

Post a Comment