Friday, 1 January 2021

Data Structures and Algorithms: Learning Resources

 There are many resources available to learn Data Structures and Algorithms (DSA), including:

  1. "Introduction to Algorithms" by Cormen, Leiserson, Rivest, and Stein: This is a classic textbook that covers a wide range of DSA topics, including sorting algorithms, data structures, and algorithm design techniques.
  2. "Algorithms, Part I" and "Algorithms, Part II" on Coursera: These online courses, taught by Robert Sedgewick and Kevin Wayne of Princeton University, cover a wide range of DSA topics and include video lectures, quizzes, and programming assignments.
  3. "Data Structures and Algorithms" on Udemy: This online course covers a wide range of DSA topics and includes video lectures, quizzes, and coding exercises.
    1.  Master the coding interview data structures Algorithms 
    2. Data structure and Algorithm
  1. "Data Structures and Algorithms" on HackerRank/LeetCode: This website provides a wide range of coding challenges and tutorials on DSA topics, with a focus on practical implementation.
  2. "The Algorithm Design Manual" by Steven Skiena: This book provides an in-depth look at algorithm design techniques and strategies.
  3. "Algorithms" by Robert Sedgewick and Kevin Wayne, on the book website or on the publisher's website, it includes the full content of the book, as well as additional materials, such as lecture slides and exercises.
  4. "Algorithms and Data structures" from OpenCourseWare from MIT: This resource provides free online access to a wide range of DSA materials, including lecture notes, assignments, and exams.

Some of the courses are not in the language you want to learn, but I would suggest giving it a try in what language the instructor is saying to get out the most from the course.

These resources are widely used and recommended, but you should pick the one that suits you best, based on your learning style, schedule, and budget.

Do Solve DSA's Problems on HackerRank/LeetCode. 

Let us know in comment section about your favorite course/book or even any question. 

Monday, 3 February 2020

Running a Python or Django Application with Supervisor

Running a Python or Django Application with Supervisor

Making use of Supervisor

I have lurking around to find a example how can I get my python application work with Supervisor and not able to find some good examples so decided to write one. it will be brief.

Install supervisor

Go with

pip install superviosr

and it’s done. If not please check the official documentation.

Configure the Supervisor

Supervisor have 3 utility.

  1. supervisord
  2. supervisorctl
  3. Web server

supervisord (supervisor daemon) run in background and process all your configuration.
supervisorctl a CLI that can be used to to access some information regarding running process, configuration change, re-start the supervisor daemon etc.
Web server can check the stack/log for the running services.

After installing the supervisor run

echo_supervisord_conf

It will show a sample of supervisor config. Read through it to get some insight.

Let’s create a configuration file that can be used to start the supervisor. (you can copy-past echo_supervisord_conf content and play around that).

In the Installation folder (/etc/supervisor in Ubuntu), do following:–>

create a supervisor.conf file (vim supervisor.conf) with following content

let’s get it one by one :–>

  1. unix_http_server

    Start of configuration

    ; supervisor config file
    [unix_http_server]
    file=/var/run/supervisor.sock
    chmod=0700
    
  2. supervisord

    This section content the functioning configuration of supervisor daemon. Here we have configured below
    a. logfile --> path of file to record log of supervisor daemon
    b. pidfile --> file in which the process id will be logged.
    c. childlogdir --> path of file to record the child process that are running through supervisor daemon.

    [supervisord]
    logfile=/var/log/supervisor/supervisord.log
    pidfile=/var/run/supervisord.pid 
    childlogdir=/var/log/supervisor 
    
  3. inet_http_server and rpcinterface:supervisor:

    This section is used to define the web interface of the supervisor. Here we are defining the port and rpinterface

     [inet_http_server]
     port = 127.0.0.1:9001
     
     [rpcinterface:supervisor]
     supervisor.rpcinterface_factory=supervisor.rpcinterface:make_main_rpcinterface
    
  4. supervisorctl

    Define configuration related to supervisor. Let’s define the server URL for now.

    [supervisorctl]
    serverurl = http://127.0.0.1:9001
    
  5. Include section

    Include section to include the addition programs/process related configuration that we are going to run using the supervisor.

     [include]
     files = /etc/supervisor/conf.d/*.ini
    

Running a Python Script using Supervisor

Create a simple python script (sampleapp.py)

import logging
from random import randint
from time import sleep

def main():
    while True:
        i = randint(1,100)
        # If get a digit in multiple of 10 then crash
        if i in [10, 20, 30, 40, 50, 60, 70, 80, 90, 100]:
            logging.error('Generated {}. Application Crashing'.format(i))
            raise Exception('Application Crashing')
        else:
            print('Generated {}. Sleeping for {}'.format(i, str(i//10)))
        sleep(i//10)

if __name__ == "__main__":
    print('Starting the sample app')
    main()

Now define a sampleapp.ini file

  • define a program using [program:program_name]

  • directory - define the directory where your script is ( Program will change to mentioned directory while running the program.

  • command- What to run

    [program:sample_app]
    directory=/path_to_script_containing_folder/
    command=python3 sample_app.py

In Include section we have mentioned that /etc/supervisor/conf.d/*ini will be considered. So either put this sampleapp.ini file in /etc/supervisor/conf.d or create a symlink.

Now run

supervisord

It will start and run in background.

Now Run

supervisorctl

If everything is alright then you will see a response similar to this and we will enter inside the CLI utility of supervisor.

sajjan# supervisorctl
sample_app   RUNNING   pid 25366, uptime 0:01:34
supervisor> 
supervisor> help
default commands (type help <topic>):
=====================================
add    exit      open  reload  restart   start   tail   
avail  fg        pid   remove  shutdown  status  update 
clear  maintail  quit  reread  signal    stop    version

Play around it. Now to check the web interface of the supervisor, visit http://127.0.0.1:9001

Running a Django Application using Supervisor

Similarly, we create a django.ini file for our Django application.

[program:application_name]
directory=/path_to_your_project/
command=/path_to_your_virtual_enviroment/python3 manage.py runserver

Restart the daemon and it’s done. Play around it’s different parameter in configuration file.

Saturday, 12 January 2019

F-1 Score, Precision and Recall

Hi,

Initially, when I started to learn machine learning, I was not able to grasp the terms F-1 Score, precision and recall along with False Positive and False Negative( ya these 2 are more confusing then True Positive and True Negative).

Reading post per post and still not able to get anything in my mind. So One day I sat and just cleared it out. Ok, Fine Here are the Details. Let's start with abbreviation.
Let's Create the basic table which we see in every post.

 Predicted 
  Yes No
ActualYesTPFN
 NoFPTN
TP -  True  Positive      ----> Predicted is Positive and that is True
TN - True  Negative   ----> Predicted is Negative and that is True
FP - False Positive    ----> Predicted is Positive and that is False
FN - False Negative  ----> Predicted is Negative and that is False

So In TP/TN/FP/FN ending Positive, Negative tell us about the prediction. and the Initial True, False tell us "Is that Correct or Not?"

So False Positive means that the prediction is "Positive" but and False Indicate it is Wrong. For example an data corresponds to  class 0 but our classifier predicted the class 1.

Now same can go for False negative. I am leaving that up to you.

Now Here comes the Precision and Recall.

Precision

precison = TP/(TP+FP)

                = (Correct Yes Predicted by Model)/ (Total Yes predicted by Model [TP+FP])

This tells us "How much our model predicted correctly out of Total True predicted by Model(TP+FP)"

Recall

recall = TP/(TP+FN)
         
         = (Correct Yes Predicted by Model)/ (Total Yes in Actual Data[TP+FN])

This tells us "How much our model predict correctly out of total True in Data"


Recall address the question: "Given the positive example, will the classifier detect it".
While Precision address the question: "Given a positive prediction from classifier, how likely is it to be correct".

The image is just for my personal purpose.