AWS - Listing users who have not enabled MFA for thier AWS account

Welcome Back!

Well, It's been years since I wrote my last blog.. Life is busy but there is nothing to complain about, as they say - Love the life you live and live the life you love!

Coming back to the topic; today's topic is about a task which I was trying to do since last couple of days..

Task was simple  - List the users who haven't enabled MFA for their AWS account. But integrating boto3 framework with Lambda was the trick.. Hence, thought of sharing the piece of code that I wrote for this task.

Note: You may run this with normal python IDE as well but that will require the AWS credentials.

I hope you will find it useful.

Python code for Lambda function:

##########################

import boto3


def lambda_handler(event, context):
    client = boto3.client('iam')
    users = client.list_users()

    user_list = []
    virutal_enabled =[]


    for key in users['Users']:
        user_list.append(str(key['UserName']))


    for key in users['Users']:
        List_of_MFA_Devices = client.list_mfa_devices(UserName=key['UserName'])
        for key in List_of_MFA_Devices['MFADevices']:
        virutal_enabled.append(str(key['UserName']))
     
    diff_list = [item for item in user_list if not item in virutal_enabled]

    for p in diff_list: print (p)

Procedure: Mount S3 Bucket on AWS EC2 Instance

A quick handy how-to:

How to mount S3 bucket on EC2 instance of AWS

S3FS is the S3 file system that will be required here.

Firstly update the packages on Ubuntu machine

Step 1:

# sudo apt-get install build-essential git libfuse-dev libcurl4-openssl-dev libxml2-dev mime-support automake libtool
# sudo apt-get install pkg-config libssl-dev
# git clone https://github.com/s3fs-fuse/s3fs-fuse
# cd s3fs-fuse/
# ./autogen.sh
# ./configure --prefix=/usr --with-openssl
# make
# sudo make install

Step 2:

S3FS needs AWS access key id and secret key to work. Create "passwd-s3fs" file at /etc/passwd-s3fs with content

<AWS Access Key ID>:<AWS Secret Access Key>

Step 3: Change permission

# chmod 640 /etc/passwd-s3fs

Step 4: Create mountpoint:

# mkdir <mount_point>

Step 5: Now mount the s3 bucket

# sudo s3fs bucketname /mys3bucket

Step 6: To check mounted s3 bucket

# df -Th /mys3bucket

# mkdir /mys3bucket

In most cases, you will want S3 to be mounted automatically after each reboot. So, add below line to /etc/fstab

# s3fs#<buket_name> /mys3bucket fuse allow_other,use_cache=/tmp/cache 0 0

For istance: s3fs#jarvis.pc /mys3bucket fuse allow_other,use_cache=/tmp/cache 0 0

Isilon API

Welcome to the world of API!  

It took a  evening to configure a ISILON sdk on ubuntu and bringing up a two node cluster on VMware workstation; but it is worth doing it. 

Need to spend more time with it.

#api #python #isilon

Isilon OneFS upgrade to 7.2.1.0

It's the time.. Time to UPGRADE

I recently performed a OneFS upgrade to 7.2.1. The entire process is categorized in three phase and below are the steps:

Pre-upgrade tasks:

Use the below commands and do the overall health check of the cluster

• isi status -D
•  isi_for_array -s 'isi devices | grep -v HEALTHY'
•  isi_for_array -s 'gmirror status'
•  In the OneFS web administration interface, click Dashboard > Cluster Overview > Cluster Status.In the Status area, click the ID of a node.In the Chassis and drive status area, click Platform
•  isi_for_array -s 'df -h'  - Check the utilization of all file system and confirm that it is below 90%
•  isi events list

Back up data.
• Backup all important data like corn jobs, etc.
• Ensure synciq jobs completed sucessfully

Collect cluster status.
isi status -D -w

Complete system jobs.
• isi services -a isi_job_d disable

Pause or cancel SyncIQ jobs.
• isi sync policy list -v
• isi sync jobs list"
• isi sync jobs pause --all

Cancel all running jobs:  
Use command (isi job jobs list) to view any running jobs, use command ( isi job jobs cancel JobIDNumber) to cancel by ID number

Implementation plan

Check the upgrade image:

# isi update --check-only
when asked to specify  the image to update then provide the path where the OneFS image is stored.

Check if you have any job running

# isi job jobs list

Flush the cache 

# isi_for_array isi_flush

Perform the upgrade

# isi update

when asked to specify  the image to update then provide the path where the OneFS image is stored.

Are you sure you wish to upgrade (yes/no)? yes

Reboot to complete the process? (Yes/No) Yes

Now system run the upgrade process and you may need to reconnect to the cluster

Check the version

# isi version

Check the events

# isi event list -w

Install the patch if required/needed

Change to the directory where the patch image is stored

# cd <path>

# isi pkg install <file name>

Some times few drives may need firmware upgarde, run below command to do the same

# isi_disk_firmware_reboot -p

Check the services and enable if you have disabled it before performing the upgrade

# isi services -a supportiq enable
# isi services -a nfs enable
# isi services -a smb enable

Cancel all the events

#isi events cancel all

Gather the info and gave it to EMC for review

# isi_gather_info


Happy Learning!

Isilon Drive firmware upgrade

And i love this photo :-D


Well i recently performed a Drive firmware upgrade for one of our Isilon cluster and below are the steps.

Procedure for Isilon Drive Firmware Upgrade:

1. Download the package from support.emc.com link

2. Copy the package to Install directory of the cluster, example:

<cluster>:/ifs/data/2015/upgrade/software

3. Disable the notifications:

a.  isi services -a celog_notification disable

b.  isi services -a supportiq disable

4. Install the package (once per cluster)
       Command: isi pkg install /ifs/data/2015/upgrade/software/<package_name.tar>

5. Flush the cache and reboot each node; normally prefer doing it in a bottom-up fashion

a.     Run  isi_flush step prior rebooting each node.

b.     After this comand runs, it will reboot the node.

                                                   i.   isi_disk_firmware_reboot –p

6. After the node comes back online, verify the firmware status on that node

                  i. isi devices

                 ii. isi_disk_firmware_reboot -p

Note: if all drives have been updated then the message "no disks needs firmware update" will be displayed

7. Repeat step 5 & 6 for the rest of the nodes in the cluster

8. Clear the alerts with command

a.  isi events quiet all

b.     isi events cancel all

9.  Enable the notifications with commands:

a.  isi services -a celog_notification enable

b.     isi services -a supportiq enable     

Happy Learning!

SyncIQ Failover and Failback

SyncIq automated data failover & failback provides the high availability to OneFS which means in the event that a primary culster becomes unavailable, SyncIQ provides the ability to failover to a mirrored, DR cluster with minimal interruption.

Please note that: SyncIQ follows asynchronous replication not synchronous.

Below are the steps that needs to be executed for Isilon SyncIQ failover and failback; this can also be used for DR testing.

To initiate failover:

1) Set the target cluster as read-write

# isi sync recovery allow-write <policy_name>

Failback can be accomplished if the policy meets the following criteria:

1) source directory is not a smartlock directory

2) The policy has been failed over

3) The policy is a Sync Policy and not a copy policy

Steps in failback:

1) Resync-prep on the policy

# isi sync recovery resync-prep <policy_name>

2) Run a typical job on the mirror policy, a final failback differential sync is preformed:

# isi sync job start <policy_name_mirror>

3) Change the mirror local target to allow writes

# isi sync recovery allow-write <policy_name_mirror>

4) Run Resync-prep again on the mirror policy

# isi sync recovery resync-prep <policy_name_mirror>

5) Delete the mirror policy

# isi sync policies delete <policy_name_mirror>

Also read Isilon SyncIQ

Happy Learning!

Object Storage for dummies