Isilon SmartDedupe

Isilon SmartDedupe allows your cluster to be space efficient by removing redundant data ( it takes less amount of space to save files with similar blocks. SmartDedupe scans the cluster for identical data blocks and moves the single copy of redundant block to a hidden file called a shadow store. SmartDedupe then deletes the duplicated block and replaces the block with the pointer to the shadow store. Each shadow store consists of 255 blocks and each block can be referenced 32000 times.

Deduplication is applied to the directory level and files; targeting all files & directories underneath that directory.

Note: Dedupe should only be configured on data (folders) that are NOT compressed. Also don't configure dedupe for /ifs, always configure for the high level directory structure eg: /ifs/data/project_tango/

Steps to configure SmartDedupe via CLI on Isilon Cluster:

1) Check if the license is added or not as SmartDedupe is a licensed feature.

# isi license

2) Specify the file system or source directory:

# isi dedupe setting modify --path=/ifs/data/project1, /ifs/data/project2

 3) Schedule the deduplication job:

# isi job types Dedupe --schedule "Every Saturday at 01:00 AM"

4) Check the stats:

# isi dedupe stats

5) Once job is completed, you can check its report:

# isi dedupe reports list

# isi dedupe reports view <report number>

6) To check settings, use:

# isi dedupe settings view

7) To check the events, user:

# isi job events list --job-type dedupe



Happy Learning!