#6 Two Toolbox

date
May 10, 2023
slug
6-toolbox
status
Published
tags
summary
I completed the Python fundamentals track, and streamed too
time
5h-30m
type
Post
I accidently fell asleep early and would’ve just missed today’s progress but woke up sometime around around 2am, I decided to stream and continue learning. Finished Python Toolbox for DataScience part 1 and 2. There were a lot of interesting challenges and I learned a lot about iteartors and generators, I have taken a few notes but It’s already 6:45am as the time of writing. I need to get some sleep. I have already pushed myself. Will process these notes and update this post later.
 
 
 
💡
Tuple use cases ??
  • Arguments vs Parameters
    • Arguments are passed into the function
    • Parameters are written in function header
  • Docstrings
    • Used to describe the function
    • Placed immediately after the function declaration
    • Written between triple quotes """
  • Multiple values return by function → Use a tuple
 
Tweets country counter project.
  • Read the tweets from tweets.csv file.
  • Load into data frame df
  • Create a dictionary to store the count
  • Iterate and keep updating the count
 
Name Scopes defines where certain objects are accessible
  • Global → Main body of script
  • Local → Inside a function
  • Built-in → Name in pre-defined built-in modules
 
Keywords to change scopes
  • global jumps to the global scope
  • nonlocal jumps to the next parent scope
💡
Tricky question related to Name Scopes
 
What’s the deal with built-ins
Here you're going to check out Python's built-in scope, which is really just a built-in module called builtins. However, to query builtins, you'll need to import builtins 'because the name builtins is not itself built in…No, I’m serious!'
 
Nested Function
  • Moves one level upper
💡
Case for Nested functions
 
Useful for higher order functions stuff
  • The function raise_val() returns a function rather than returning a value.
  • This function can be reused later
  • This is closure in computer science
 
LEGB rule, search starts with local and ends in built-in.
  • Local
  • Enclosing function
  • Global
  • Built-in
 
Python closure, nested function remember the state of its enclosing scope when called.
 
Arbitary number of arguments can be passed in python with *args and similarly keyword arguments can be with **kwargs.
  • args is a tuple
  • kwargs is a dictionary
 
 
Lambda function
 
map function
 
reduce
 
 
💡
Learn about the a
TIL repertoire stock of skills
 
Error Handling
 
 

Data Science Tool box 2

 

Iter(ables)(ators)

 
Iterable : Object that has an associated iter() method
  • Iterable is an object that can return an iterator
  • Lists, Strings, Dictionaries, File contents
  • When using a for loop, under the hood an iterator is created.
Iterator : Produces next value with next()
  • Iterator is an object that keeps state and produces the next value on being called next()
  • After the last value has been passed, calling next will result StopIteration error.
 
Splat operator is used to unpack all values at once
 
 
💡
How does the range function work
 
Range function
  • doesn’t actually create a list
  • instead created a range object with iterator
 
 
Enumerate
  • takes an iterable and return enumerate object that pairs up items with index
  • enumerate(iterable,start=5) starts counting from 5
 
Zip
  • Takes arbitary number of iterables and returns iterator of tuples
  • Use it with list to create a list of tuples
 
 
 
Chunksize
  • when data is too big to store in memory
  • use chunk argument in .read_csv() method of python
 
 

Comprehension & Generators

 
List comprehension is made up of
  • Iterable
  • Iterator
  • Output Expression
Code readability is a trade-off with one liners
 
Dictionary comprehension
  • Use {} brackets instead of []
  • key : val syntax is to be used
  • [ output expression for iterator variable in iterable if predicate expression ]
  • [output1 expression if (condition1) else output2 for iterator in iterable]
 
 
 
Generator object is created using () with the same syntax as list comprehension.
  • Doesn’t store in the memory, doesn’t construct the list
  • Object we can generate over to produce elements of list as required.
 
Lazy Evaluation
  • evaluation of expression is delayed until its value is needed
 
Generator Functions
  • just like regular function but instead of returning a value it yields a result
 
 
 
Range
  • works with generator behind the scenes
  • So, does .items() of a dictionary
 
List comprehension // Generator
 
 
 
  • Use df.head() to print the head of a DataFrame
 
 
Project file
  • Process data and create a graph
  • Load file chunk by chunk
  • create new column for urban population values
  • plotting the data
 
 
 
 
 
 
 

© ibrar 2023