Utilpy Packages

utilpy.utils

utilpy.utils.clean_str(string)

Tokenization/string cleaning for all datasets except for SST. Original taken from https://github.com/yoonkim/CNN_sentence/blob/master/process_data.py

Args:

string:str

string of sentences

Returns:

Lowercase cleaned string:str

utilpy.utils.download_file(url, save_dir='tmp/', filename=None)

download and save the file

Args:

url:str

url of file

save_dir:str

directory to save

filename:str

Name of file to save as

utilpy.utils.download_files(urls, save_dir='tmp/')

download all files from urls

Args:

urls:list

urls of file

save_dir:str

directory to save

utilpy.utils.format_filename(s)

Take a string and return a valid filename constructed from the string. Uses a whitelist approach: any characters not present in valid_chars are removed. Also spaces are replaced with underscores.

Args:

s:str

string

Returns:

Valid filename : str

utilpy.utils.logger(logger_name='utilpy.utils', filename='log.log', level=10)

logger for logging

Args:

logger_name:str

name of logger

filename:str

path and file name

level:str

logging level

Return:

logger funcation for logging:logger

utilpy.utils.page_soup(url)

Make BeautifulSoup of page

Args:

url:str

url of page

Returns:

BeautifulSoup of page:soup

utilpy.utils.read_config(filename)

Parse config yaml file

Args:

filename:str

path of config file

utilpy.utils.top_n(lis, n)

Get index of top n item from list with value

Args:

lis:list

list

n:int

number of top value

Return:

dict of top value and index: dict

utilpy.utils.walk_directory(dir_path, files_only=True)

Get list of all file in directory recursive

Args:

dir_path:str

parent directory to start

files_only:bool

get list of files only other

Return:

If file only than list of all files otherwise files and list of directories

utilpy.decorator

utilpy.decorator.profile(fnc)

A decorator that uses cProfile to profile a function And print the result

utilpy.decorator.threading_d(func)

A decorator to run function in background on thread

Args:

func:function

Function with args

Return:

background_thread: Thread

utilpy.decorator.timeout(seconds=10, error_message='Timer expired')

Decorator to throw timeout error, if function doesnt complete in certain time

Args:

seconds:int

No of seconds to wait

error_message:str

Error message

utilpy.log

Logger function that print time required in each session of logger

class utilpy.log.Logger

Bases: object

Print the log message with time required between each session New session should start with “<” and end with “>” session ends with “</>”

log(message)

log message new session starts with < and ends with > session end with </>

utilspy.files

utilpy.files.delete_dirs(folder_names)

delete folder and all its contents

Args: `

folder_names:str` or list

string or list of folders

utilpy.files.make_dirs(folder_names)

make folder if it doesnot exist

Args:

folder_names:str or list

string or list of folders

utilpy.files.move_file(filename, out_dir)

move file/dir

Args:

filename:str

filename of file to be moved or name of dir to be moved

out_dir:str

output directory

utilpy.files.parse_path(path, fn_only=False, ext=False, al=True)

get the directory from filename,

Args:

path:str

path of file

fn_only:bol

get the file name only from path

ext:bol

split file name into name and extension

al: bol

get list of dir and file name

Returns:

list of value: list

utilpy.files.unzip_file(filename, save_dir='data')

unzip all file

Args:

filename:str

filename of zip file

sav_dir:str

parent directory to zip

Warning

for rar file type, install rar and unrar

apt install rar && apt install unrar

utilpy.files.zip_folder(dir_path, filename=None, exclude=None, include=None)

zip all file in folder,

Args:

dir_path:str

parent directory to zip

filename:str

filename of zip file

exclude:list

list of file to exclude

include:list

list of file to include