How to check if a file is locked in Python

Posted on December 14th, 2010 webmaster 1 comment          Print

My colleague showed me a trick on how to do this earlier.  What I was trying to do is poll a folder for a bunch of files and if they’re not there keep polling until they arrive, then process them.  The problem is before they can be processed we need to be sure that the file transfer is complete.

The trick on how to do this is to try and open the file once it arrives in ‘append’ mode.  This will fail if another process is using the file.  So what we can do in this case is then keep trying to open it in append mode until it’s successful and at that point we know that there is no longer a lock on the file.

Sounds a bit hacky but we couldn’t think of a more elegant way to do it.

import os, time

def is_locked(filepath):
    """Checks if a file is locked by opening it in append mode.
    If no exception thrown, then the file is not locked.
    """
    locked = None
    file_object = None
    if os.path.exists(filepath):
        try:
            print "Trying to open %s." % filepath
            buffer_size = 8
            # Opening file in append mode and read the first 8 characters.
            file_object = open(filepath, 'a', buffer_size)
            if file_object:
                print "%s is not locked." % filepath
                locked = False
        except IOError, message:
            print "File is locked (unable to open in append mode). %s." % \
                  message
            locked = True
        finally:
            if file_object:
                file_object.close()
                print "%s closed." % filepath
    else:
        print "%s not found." % filepath
    return locked

def wait_for_files(filepaths):
    """Checks if the files are ready.

    For a file to be ready it must exist and can be opened in append
    mode.
    """
    wait_time = 5
    for filepath in filepaths:
        # If the file doesn't exist, wait wait_time seconds and try again
        # until it's found.
        while not os.path.exists(filepath):
            print "%s hasn't arrived. Waiting %s seconds." % \
                  (filepath, wait_time)
            time.sleep(wait_time)
        # If the file exists but locked, wait wait_time seconds and check
        # again until it's no longer locked by another process.
        while is_locked(filepath):
            print "%s is currently in use. Waiting %s seconds." % \
                  (filepath, wait_time)
            time.sleep(wait_time)

# Test
if __name__ == '__main__':
    files = [r"C:\testfolder\testfile1.txt",
             r"C:\testfolder\testfile2.txt"]
    print wait_for_files(files)

Output:

>>>

Trying to open C:\testfolder\testfile1.txt.

File is locked (unable to open in append mode). [Errno 13] Permission denied: ‘C:\\testfolder\\testfile1.txt’.

C:\testfolder\testfile1.txt is currently in use. Waiting 5 seconds.

Trying to open C:\testfolder\testfile1.txt.

C:\testfolder\testfile1.txt is not locked.

C:\testfolder\testfile1.txt closed.

Trying to open C:\testfolder\testfile2.txt.

C:\testfolder\testfile2.txt is not locked.

C:\testfolder\testfile2.txt closed.

>>>

How to convert a Large Integer value to normal date format using PowerShell

Posted on December 6th, 2010 webmaster No comments          Print

My co-worker was wondering last week whether an old Windows domain user account was still being used by someone.  Having managed Windows domain environments at my previous jobs, the first thing I did of course was go to Active Directory and check the LastLogonTimestamp attribute.  This attribute is stored in the Active Directory database as a Large Integer so it will need to be converted to a normal date format to make sense of it.

The following PowerShell command can be used to do the conversion (forgot the website where I got this from, will reference it here if I find it again):

$lastLogonTimestamp = "129358017032999046"

[DateTime]::FromFiletime([Int64]::Parse($lastLogonTimestamp))

Python function for displaying a list of dictionaries in table format

Posted on December 3rd, 2010 webmaster 2 comments          Print

I just spent like 2 hours writing this function, but it was worth it!

This cut down a lot of duplicate code on the project I’m working on.  I figured I’d post it here in case someone is trying to do the same thing.  I’m sure my code is not optimal so if you have any suggestions on how to improve it please feel free to comment .

from operator import itemgetter

def format_as_table(data,
                    keys,
                    header=None,
                    sort_by_key=None,
                    sort_order_reverse=False):
    """Takes a list of dictionaries, formats the data, and returns
    the formatted data as a text table.

    Required Parameters:
        data - Data to process (list of dictionaries). (Type: List)
        keys - List of keys in the dictionary. (Type: List)

    Optional Parameters:
        header - The table header. (Type: List)
        sort_by_key - The key to sort by. (Type: String)
        sort_order_reverse - Default sort order is ascending, if
            True sort order will change to descending. (Type: Boolean)
    """
    # Sort the data if a sort key is specified (default sort order
    # is ascending)
    if sort_by_key:
        data = sorted(data,
                      key=itemgetter(sort_by_key),
                      reverse=sort_order_reverse)

    # If header is not empty, add header to data
    if header:
        # Get the length of each header and create a divider based
        # on that length
        header_divider = []
        for name in header:
            header_divider.append('-' * len(name))

        # Create a list of dictionary from the keys and the header and
        # insert it at the beginning of the list. Do the same for the
        # divider and insert below the header.
        header_divider = dict(zip(keys, header_divider))
        data.insert(0, header_divider)
        header = dict(zip(keys, header))
        data.insert(0, header)

    column_widths = []
    for key in keys:
        column_widths.append(max(len(str(column[key])) for column in data))

    # Create a tuple pair of key and the associated column width for it
    key_width_pair = zip(keys, column_widths)

    format = ('%-*s ' * len(keys)).strip() + '\n'
    formatted_data = ''
    for element in data:
        data_to_format = []
        # Create a tuple that will be used for the formatting in
        # width, value format
        for pair in key_width_pair:
            data_to_format.append(pair[1])
            data_to_format.append(element[pair[0]])
        formatted_data += format % tuple(data_to_format)
    return formatted_data

# Test
if __name__ == '__main__':
    header = ['Name', 'Age', 'Sex']
    keys = ['name', 'age', 'sex']
    sort_by_key = 'age'
    sort_order_reverse = True
    data = [{'name': 'John Doe', 'age': 37, 'sex': 'M'},
            {'name': 'Lisa Simpson', 'age': 17, 'sex': 'F'},
            {'name': 'Bill Clinton', 'age': 57, 'sex': 'M'}]

    print format_as_table(data,
                          keys,
                          header,
                          sort_by_key,
                          sort_order_reverse)

Output:

Name         Age Sex
----         --- ---
Bill Clinton 57  M   
John Doe     37  M   
Lisa Simpson 17  F