Jekyll2023-11-05T15:55:29-08:00/KishStatsKishStats is a resource for Python development. View our Python Fundamentals course. More courses coming soon.Python Functools Module2023-10-27T10:13:00-07:002023-10-27T10:13:00-07:00/python/2023/10/27/python-functools<p><code class="highlighter-rouge">functools</code> is part of the Python standard library which is defined as being a “module is for higher-order functions” and that “functions that act on or return other functions”. In this post, we’ll go through a number examples incorporating the most common use cases.</p>
<p><a href="https://github.com/kishstats/Python-Functools-Module" class="source-code-link">
<button type="button">
<span class="icon icon--github"><svg viewBox="0 0 16 16" width="25px" height="25px"><path fill="#111" d="M7.999,0.431c-4.285,0-7.76,3.474-7.76,7.761 c0,3.428,2.223,6.337,5.307,7.363c0.388,0.071,0.53-0.168,0.53-0.374c0-0.184-0.007-0.672-0.01-1.32 c-2.159,0.469-2.614-1.04-2.614-1.04c-0.353-0.896-0.862-1.135-0.862-1.135c-0.705-0.481,0.053-0.472,0.053-0.472 c0.779,0.055,1.189,0.8,1.189,0.8c0.692,1.186,1.816,0.843,2.258,0.645c0.071-0.502,0.271-0.843,0.493-1.037 C4.86,11.425,3.049,10.76,3.049,7.786c0-0.847,0.302-1.54,0.799-2.082C3.768,5.507,3.501,4.718,3.924,3.65 c0,0,0.652-0.209,2.134,0.796C6.677,4.273,7.34,4.187,8,4.184c0.659,0.003,1.323,0.089,1.943,0.261 c1.482-1.004,2.132-0.796,2.132-0.796c0.423,1.068,0.157,1.857,0.077,2.054c0.497,0.542,0.798,1.235,0.798,2.082 c0,2.981-1.814,3.637-3.543,3.829c0.279,0.24,0.527,0.713,0.527,1.437c0,1.037-0.01,1.874-0.01,2.129 c0,0.208,0.14,0.449,0.534,0.373c3.081-1.028,5.302-3.935,5.302-7.362C15.76,3.906,12.285,0.431,7.999,0.431z"></path></svg>
</span>
Source Code
</button>
</a></p>
<iframe width="740" height="417" src="https://www.youtube.com/embed/qGE01LPxd2A" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen=""></iframe>
<h2 id="reduce">Reduce</h2>
<p>The <code class="highlighter-rouge">reduce</code> method is one of the most commonly used functions within the <code class="highlighter-rouge">functools</code> module. The name “reduce” implies taking in an iterable (often a list), and “reducing” it to a single value. This is also thought of as an “accumulator”.</p>
<p>In Python 2, the reduce method was available with no import statement necessary. Python 3, on the other hand, requires it to be imported from the <code class="highlighter-rouge">functools</code> library:</p>
<pre><code class="language-Python">from functools import reduce
</code></pre>
<h3 id="numbers">Numbers</h3>
<p>The <code class="highlighter-rouge">reduce</code> function takes a minimum of 2 arguments: a function and an iterable. The simplest example is to add up a list of numbers:</p>
<pre><code class="language-Python">from functools import reduce
lyst = [1, 2, 3, 4]
total = reduce(lambda x, y: x + y, lyst)
assert total == 10
</code></pre>
<p>As you can see, this is much more concise than having to create a variable, setting an initial value, then using a for loop to keep track of the total. Notice the use of a <code class="highlighter-rouge">lambda</code> function. This is how anonymous functions are implemented in Python. Inside the <code class="highlighter-rouge">lambda</code>, we are simply adding the value to the running total. We can also use a named function:</p>
<pre><code class="language-Python"># using a named function
def add_it(x, y):
return x + y
total = reduce(add_it, lyst)
assert total == 10
</code></pre>
<p>A named function is generally preferred for more complex functions.</p>
<p>For any list that is created dynamically, it’s important to point out what happens if the list is empty. It will result in an error if no initial value is set:</p>
<pre><code class="language-Python"># TypeError: reduce() of empty iterable with no initial value
total = reduce(lambda x, y: x + y, [])
</code></pre>
<p>One reason why this cannot work is that if there is nothing in the list, there is no way to tell what data type would be reduced.</p>
<p>We can eliminate this error by setting an initial value which is an optional, third argument of the <code class="highlighter-rouge">reduce</code> function:</p>
<pre><code class="language-Python">total = reduce(lambda x, y: x + y, [], 0)
assert total == 0
</code></pre>
<p>Although Python has built-in functions for <code class="highlighter-rouge">min</code> and <code class="highlighter-rouge">max</code>, we can perform the same task using <code class="highlighter-rouge">reduce</code> by modifying the return value using an <code class="highlighter-rouge">if</code> condition inside our <code class="highlighter-rouge">lambda</code> function.</p>
<pre><code class="language-Python">max_value = reduce(lambda x, y: x if x > y else y, lyst)
assert max_value == max(lyst)
</code></pre>
<h3 id="strings">Strings</h3>
<p>Iterating through strings can be done through both lists of string values and stings themselves. In the following example, the list of characters will be concatenated into a single string value:</p>
<pre><code class="language-Python"># iterate through a list of characters
lyst = ["a", "b", "c", "d", "e", "f"]
result = reduce(lambda x, y: x + y, lyst)
print(f"{result=}")
assert result == "abcdef"
</code></pre>
<p>We can also get the same result from a string itself. Since a string can be an iterable in Python, we will get the same value back by concatenating the characters:</p>
<pre><code class="language-Python"># iterate through a single string
lyst = "abcdef"
result = reduce(lambda x, y: x + y, lyst)
print(f"{result=}")
assert result == "abcdef"
</code></pre>
<h3 id="dictionaries-and-other-complex-types">Dictionaries and Other Complex Types</h3>
<p><code class="highlighter-rouge">reduce</code> can be used for pretty much anything that’s an iterable. In the next example, we will use a dictionary list where each item is a product for a specific order. We will attempt to add up the amounts of each order. Similar to the example of the empty list that caused an error, the same thing will happen here if we don’t include an initial value:</p>
<pre><code class="language-Python">order_items = [
{"item_id": 1, "name": "hiking boots", "amount": 150.00, "qty": 1},
{"item_id": 33, "name": "hiking poles", "amount": 95.00, "qty": 1},
{"item_id": 1, "name": "hydration pack", "amount": 52.00, "qty": 1},
{"item_id": 1, "name": "hiking shorts", "amount": 60.00, "qty": 2},
]
# missing initial value will cause an error
total_amount = reduce(lambda x, y: x + y["amount"], order_items)
# TypeError: unsupported operand type(s) for +: 'dict' and 'float'
</code></pre>
<p>Without an initial value, the <code class="highlighter-rouge">reduce</code> method will try to add the amount to a dictionary. We can fix this by setting an initial value of 0 since we know we’re adding up a numeric value:</p>
<pre><code class="language-Python">total_amount = reduce(lambda x, y: x + y["amount"], order_items, 0)
print(f"{total_amount=}")
assert total_amount == 357.0
</code></pre>
<h2 id="cache-decorators">Cache Decorators</h2>
<p><code class="highlighter-rouge">functools</code> provides a few decorators for caching. The <code class="highlighter-rouge">lru_cache</code> method uses the <a href="https://en.wikipedia.org/wiki/Cache_replacement_policies#Least_recently_used_(LRU)">Least Recently Used (LRU)</a> algorithm. This means that the cache will generally have a size limit and, once filled, the least recently used items will be discarded first.</p>
<p>The most commonly example cited for using <code class="highlighter-rouge">lru_cache</code> is a Fibonacci function. This function requires a lot of recursive function calls which makes maintaining a cache perform better. It also becomes obvious very quickly that there’s a performance drag once any sizeable number is passed in.</p>
<p>Our syntax for using the decorator will be <code class="highlighter-rouge">@lru_cache(maxsize=128, typed=False)</code>. Both the <code class="highlighter-rouge">maxsize</code> of 128 and the <code class="highlighter-rouge">typed</code> of <code class="highlighter-rouge">False</code> are default values which we can leave out. If we pass in <code class="highlighter-rouge">None</code> as the <code class="highlighter-rouge">maxsize</code>, the cache can grow to any size. It’s also worth noting that the <code class="highlighter-rouge">typed</code> parameter, if set to <code class="highlighter-rouge">True</code>, will also check the datatype of the parameter in addition to its value. The documentation states that 3.0 (a float) and 3 (an integer) will be treated as distinct calls with distinct results, thus expanding the cache.</p>
<p>In the code snippet below, we import <code class="highlighter-rouge">lru_cache</code> and create our function. Adding the decorator is very straightforward. We simply place it just above the function.</p>
<pre><code class="language-Python">from functools import lru_cache
@lru_cache(maxsize=128)
def fibonacci(n):
if n < 2:
return n
return fibonacci(n - 1) + fibonacci(n - 2)
if __name__ == "__main__":
res = fibonacci(40)
print(res) # 102334155
</code></pre>
<p>Executing the code snippet above will return the result very quickly. This is because it’s saving the result of each function call that includes the same parameter value passed into it. If the decorator is omitted/commented out, the performance will be noticeable with a seemingly modest value of 40. Even a fast machine will take over several seconds to get the result.</p>
<p>The wrapper created by the decorator also gives us a couple methods related to caching. The <code class="highlighter-rouge">cache_info</code> will provide us with caching statistics and the <code class="highlighter-rouge">cache_clear</code> method will reset the cache. The following example implements both methods:</p>
<pre><code class="language-Python">...
if __name__ == "__main__":
res = fibonacci(40)
print(res) # 102334155
print(
fibonacci.cache_info()
) # CacheInfo(hits=38, misses=41, maxsize=128, currsize=41)
fibonacci.cache_clear() # clears cache
</code></pre>
<p>The <code class="highlighter-rouge">CacheInfo</code> object that’s returned shows that the cache was used 38 times. The 41 “misses” is how many times our function actually ran. Since 41 is below our limit of 128, the current size matches the number of misses.</p>
<p>Finally, the <code class="highlighter-rouge">cache_clear</code> method will reset the cache and zero out all the data in the <code class="highlighter-rouge">CacheInfo</code> object.</p>
<h3 id="shorthand-for-lru-cache">Shorthand for LRU Cache</h3>
<p><code class="highlighter-rouge">functools</code> provides a <code class="highlighter-rouge">cache</code> decorator that’s a shorthand for <code class="highlighter-rouge">lru_cache</code>. It is equivalent to <code class="highlighter-rouge">@lru_cache(maxsize=None)</code> which means the size of the cache will be unbounded. Using our Fibonacci example, the implementation will be the following:</p>
<pre><code class="language-Python">from functools import cache
@cache # same as @lru_cache(maxsize=None)
def fibonacci(n):
if n < 2:
return n
return fibonacci(n - 1) + fibonacci(n - 2)
if __name__ == "__main__":
res = fibonacci(40)
print(res) # 102334155
print(
fibonacci.cache_info()
) # CacheInfo(hits=38, misses=41, maxsize=None, currsize=41)
</code></pre>
<h3 id="caching-class-properties">Caching Class Properties</h3>
<p>The final caching decorator provided by <code class="highlighter-rouge">functools</code> is <code class="highlighter-rouge">cached_property</code> which caches the property of a class. Consider the following example:</p>
<pre><code class="language-Python">class Customer:
def __init__(self, orders) -> None:
self._orders = orders
@property
def recent_orders(self):
orders = sorted(self._orders, key=lambda x: x["order_id"], reverse=True)
return orders[:3]
if __name__ == "__main__":
orders = [
{"order_id": 1, "name": "hiking boots", "amount": 150.00, "qty": 1},
{"order_id": 2, "name": "hiking poles", "amount": 95.00, "qty": 1},
{"order_id": 3, "name": "hydration pack", "amount": 52.00, "qty": 1},
{"order_id": 4, "name": "hiking shorts", "amount": 60.00, "qty": 2},
{"order_id": 5, "name": "hiking shorts", "amount": 60.00, "qty": 2},
{"order_id": 6, "name": "hiking shorts", "amount": 60.00, "qty": 2},
{"order_id": 7, "name": "hiking shorts", "amount": 60.00, "qty": 2},
]
customer = Customer(orders)
print(customer.recent_orders)
print(customer.recent_orders)
</code></pre>
<p>This <code class="highlighter-rouge">Customer</code> class takes in a list of orders when initialized. Getting the “recent orders” returns the last 3 orders. This simplified example assumes the orders with the highest <code class="highlighter-rouge">order_id</code> are the most recent. Using the <code class="highlighter-rouge">sorted</code> function, we sort the orders by <code class="highlighter-rouge">order_id</code> in descending order. Then, we return the first 3 records by index values.</p>
<p>At the bottom, we call the <code class="highlighter-rouge">recent_orders</code> property twice. In a realistic application, the <code class="highlighter-rouge">recent_orders</code> property could be used in multiple locations and a SQL query might be involved in getting the order data. In that case, we would want to cache the result instead of running unnecessary queries.</p>
<p>We can convert <code class="highlighter-rouge">recent_orders</code> to a cached property by importing <code class="highlighter-rouge">cached_property</code> from <code class="highlighter-rouge">functools</code> and replacing our <code class="highlighter-rouge">property</code> decorator:</p>
<pre><code class="language-Python">from functools import cached_property
class Customer:
def __init__(self, orders) -> None:
self._orders = orders
@cached_property
def recent_orders(self):
orders = sorted(self._orders, key=lambda x: x["order_id"], reverse=True)
return orders[:3]
if __name__ == "__main__":
orders = [
{"order_id": 1, "name": "hiking boots", "amount": 150.00, "qty": 1},
{"order_id": 2, "name": "hiking poles", "amount": 95.00, "qty": 1},
{"order_id": 3, "name": "hydration pack", "amount": 52.00, "qty": 1},
{"order_id": 4, "name": "hiking shorts", "amount": 60.00, "qty": 2},
{"order_id": 5, "name": "hiking shorts", "amount": 60.00, "qty": 2},
{"order_id": 6, "name": "hiking shorts", "amount": 60.00, "qty": 2},
{"order_id": 7, "name": "hiking shorts", "amount": 60.00, "qty": 2},
]
customer = Customer(orders)
print(customer.recent_orders)
print(customer.recent_orders) # from cache
</code></pre>
<h2 id="wrapped-functions">Wrapped Functions</h2>
<p>There are a number of uses for wrapping functions. This allows executing code automatically both before and after a certain function is called (pre and post-processing). Use cases can range from handling retries to isolating your code from third party API’s where only the wrapper requires updating, leaving the rest of the code untouched.</p>
<p>The <code class="highlighter-rouge">wraps</code> decorator offers an important benefit: maintaining the original function’s signature, name, and other attributes. That is, without using <code class="highlighter-rouge">wraps</code>, a function being wrapped will maintain the signature, name, and docstring of the wrapper, not the function itself. We’ll start with a simple example, then move on to a real world example.</p>
<p>First, we into import <code class="highlighter-rouge">wraps</code> from <code class="highlighter-rouge">functools</code> and create the wrapper:</p>
<pre><code class="language-Python">from functools import wraps
def my_decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
print("before executing function")
func(*args, **kwargs)
print("after executing function")
return wrapper
</code></pre>
<p>In the snippet above, we’ll use <code class="highlighter-rouge">print</code> statements in place of any pre and post-processing code. The <code class="highlighter-rouge">my_decorator</code> takes a function as an argument which is then passed into the <code class="highlighter-rouge">wraps</code> decorator function. The <code class="highlighter-rouge">wrapper</code> function itself takes in any arguments and keyword arguments setup in the target function, which we’ll add in the next step:</p>
<pre><code class="language-Python">from functools import wraps
def my_decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
print("before executing function")
func(*args, **kwargs)
print("after executing function")
return wrapper
@my_decorator
def say_hello(name):
"""Say Hello
Parameters:
name (string)
"""
print(f"hello, {name}")
</code></pre>
<p>We can now call our function by passing in a name. We can also print the double-under properties of the target function:</p>
<pre><code class="language-Python">...
# prints:
# before executing function
# hello, Joe Blow
# after executing function
say_hello("Joe Blow")
print(say_hello.__name__) # say_hello
# prints:
# Say Hello
# Parameters:
# name (string)
print(say_hello.__doc__)
</code></pre>
<p>As you can see, this implementation will maintain the same metadata as the original function. If we were, however, to comment out the line that reads <code class="highlighter-rouge">@wraps(func)</code>, <code class="highlighter-rouge">print(say_hello.__name__)</code> would return <code class="highlighter-rouge">wrapper</code>, the name of the wrapper function. And, <code class="highlighter-rouge">print(say_hello.__doc__)</code> would print <code class="highlighter-rouge">None</code> because the wrapper function doesn’t have its own docstring.</p>
<h3 id="a-more-realistic-example">A More Realistic Example</h3>
<p>The following example will entail using the <a href="https://pypi.org/project/requests/">requests</a> library to handle reties for failed API calls. External API requests can fail for a variety of reasons ranging from network issues to API’s being out of service. If you do not have <code class="highlighter-rouge">requests</code> installed, you can install it using <code class="highlighter-rouge">pip</code>: <code class="highlighter-rouge">pip install requests</code>.</p>
<p>To simulate our requests, we will use a website called <a href="https://httpbin.org">httpbin</a>. We can simulate any request method and get back any response code (i.e. 200, 400, etc.). The response code will be used to determine whether or not to retry a request. We will start with a simple request using the <code class="highlighter-rouge">requests</code> library, then make the necessary adjustments using a wrapper.</p>
<pre><code class="language-Python">import requests
httpbin_url = "https://httpbin.org/get"
response = requests.get(httpbin_url, params={"test": "true"})
result = response.json()
print("result", result)
</code></pre>
<p>This is a typical HTTP request implementation. By running this, you should see a JSON response that includes the params that were originally sent plus some other metadata.</p>
<p>Now, let’s say this request fails periodically and we want to implement retries. We can create a <code class="highlighter-rouge">retry_requests</code> function that wraps any <code class="highlighter-rouge">requests</code> method (i.e. GET, POST, etc.). We can also include default arguments for the max number of reties and the delay in seconds. We will be using the <code class="highlighter-rouge">sleep</code> method from the <code class="highlighter-rouge">time</code> module to implement the delay.</p>
<p>We will create a stub function along with how our <code class="highlighter-rouge">fetch_data</code> function will be used. The <code class="highlighter-rouge">retry_requests</code> function will not know which <code class="highlighter-rouge">requests</code> method will be used, so it must be flexible in that regard. We also have 2 sets of parameters:</p>
<ul>
<li><code class="highlighter-rouge">max_retries</code> and <code class="highlighter-rouge">delay</code></li>
<li><code class="highlighter-rouge">url</code> and <code class="highlighter-rouge">params</code></li>
</ul>
<pre><code class="language-Python">import requests
import time
from functools import wraps
def retry_requests(max_retries=3, delay=3):
# TODO
@retry_requests(max_retries=2, delay=3)
def fetch_data(url, params=None):
return requests.get(url, params)
httpbin_url = "https://httpbin.org/get"
response = fetch_data(httpbin_url, params={"test": "true"})
if response:
result = response.json()
print("result", result)
else:
print("No reponse received")
</code></pre>
<p>Next, we will setup the needed nested functions:</p>
<pre><code class="language-Python">...
def retry_requests(max_retries=3, delay=3):
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
return wrapper
return decorator
...
</code></pre>
<p>In the snippet above, we have 2 nested functions that enable us to maintain both our retries params and the params for the request itself. If we made this less flexible, we might be able to get by with a single nested function. But this gives the flexibility to pass in both sets of parameters.</p>
<p>Next, we will implement a for loop to keep track of our attempts. We will check the status code returned by the API call via <code class="highlighter-rouge">requests</code> module to determine whether the call succeeded for failed. For logging purposes, we will <code class="highlighter-rouge">print</code> this information.</p>
<pre><code class="language-Python">...
def retry_requests(max_retries=3, delay=3):
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
for attempt in range(max_retries):
try:
response = func(*args, **kwargs)
print(
f"Request to {response.url} returned status code {response.status_code}"
)
if response.status_code == 200:
return response
elif response.status_code == 429:
print("Rate limited. Waiting and retrying.")
time.sleep(delay)
elif 500 <= response.status_code < 600:
print("Server error. Retrying after delay.")
time.sleep(delay)
else:
response.raise_for_status()
except requests.RequestException as e:
if attempt < max_retries - 1:
print(
f"Request failed with error: {str(e)}. Retrying in {delay} seconds."
)
time.sleep(delay)
else:
print("Max retries reached. Request failed.")
raise
return None
return wrapper
return decorator
...
</code></pre>
<p>Notice that the line that reads <code class="highlighter-rouge">response = func(*args, **kwargs)</code> executes the target function. Once we have the response, we then check for the status code. If a 200 code is received, we assume the request was successful and simply return the response. However, for any 429 (too many requests) or 500-level status code, we apply the delay and retry. Any response code that doesn’t meet any of our conditions (since we’re not explicitly anticipating them) will be handled in the exception block. If the maximum number of retries is reached, <code class="highlighter-rouge">None</code> will be returned.</p>
<p>To test this out, we will create a new httpbin url that will return a 429 error. This should run through every attempt and return an empty result:</p>
<pre><code class="language-Python">...
httpbin_url = "https://httpbin.org/get"
httpbin_error_url = "https://httpbin.org/status/429"
# should return a successful result
response = fetch_data(httpbin_url, params={"test": "true"})
if response:
result = response.json()
print("result", result)
else:
print("No reponse received")
response = fetch_data(httpbin_error_url, params={"test": "true"})
if response:
result = response.json()
print("result", result)
else:
print("No reponse received")
# should print:
# Request to https://httpbin.org/status/429?test=true returned status code 429
# Rate limited. Waiting and retrying.
# Request to https://httpbin.org/status/429?test=true returned status code 429
# Rate limited. Waiting and retrying.
# No response received
</code></pre>
<p>Finally, we can add a docstring to the <code class="highlighter-rouge">fetch_data</code> function and the metadata should be that of the same function:</p>
<pre><code class="language-Python">@retry_requests(max_retries=2, delay=3)
def fetch_data(url, params=None):
"""Say Hello
Parameters:
url (string)
params (dict|None)
"""
return requests.get(url, params)
print(fetch_data.__name__) # fetch_data
print(fetch_data.__doc__)
# prints:
# Say Hello
# Parameters:
# url (string)
# params (dict|None)
</code></pre>
<h2 id="partial-function">Partial Function</h2>
<p>What we just implemented with the <code class="highlighter-rouge">wraps</code> function is actually a convenience function that applies the <code class="highlighter-rouge">partial</code> function.</p>
<p>In the following snippet from the <code class="highlighter-rouge">functools</code> library, note the use of <code class="highlighter-rouge">WRAPPER_ASSIGNMENTS</code> and <code class="highlighter-rouge">WRAPPER_UPDATES</code> constants which contain double-under values including <code class="highlighter-rouge">__name__</code> and <code class="highlighter-rouge">__doc__</code>. This is what enables <code class="highlighter-rouge">wraps</code> to maintain the original function metadata as described above.</p>
<pre><code class="language-Python">def wraps(wrapped,
assigned = WRAPPER_ASSIGNMENTS,
updated = WRAPPER_UPDATES):
"""Decorator factory to apply update_wrapper() to a wrapper function
Returns a decorator that invokes update_wrapper() with the decorated
function as the wrapper argument and the arguments to wraps() as the
remaining arguments. Default arguments are as for update_wrapper().
This is a convenience function to simplify applying partial() to
update_wrapper().
"""
return partial(update_wrapper, wrapped=wrapped,
assigned=assigned, updated=updated)
</code></pre>
<p>The main objective for using <code class="highlighter-rouge">partial</code> is to create a new function from another while freezing one or more of the original function’s arguments. Consider the following example:</p>
<pre><code class="language-Python">from functools import partial
def multiply(x, y):
return x * y
double_it = partial(multiply, 2)
print(double_it) # functools.partial(<function multiply at 0x10619cfe0>, 2)
print(double_it(5)) # 10
</code></pre>
<p>Above: we create a <code class="highlighter-rouge">multiply</code> function that takes in 2 parameters. Let’s say we want a doubling function that takes in any number and multiplies it by 2. We do this by using a <code class="highlighter-rouge">partial</code>, where “2” is always frozen as the <code class="highlighter-rouge">x</code> parameter. This simplifies our doubling case by having a function with fewer arguments.</p>
<h2 id="single-dispatch">Single Dispatch</h2>
<p>The final function from the <code class="highlighter-rouge">functools</code> module that we’ll be covering is <code class="highlighter-rouge">singledispatch</code>. This allows running different function implementations based on the data type of the first argument. To better illustrate this, we will implement a JSON converter. Let’s first setup our initial function:</p>
<pre><code class="language-Python">import json
def to_json(data):
return json.dumps(data)
item_1 = {"test": True}
print(to_json(item_1)) # {"test": true}
</code></pre>
<p>This code above works ok as-is given the example. However, let’s try converting a <code class="highlighter-rouge">set</code> to JSON:</p>
<pre><code class="language-Python">import json
def to_json(data):
return json.dumps(data)
item_1 = {"test": True}
print(to_json(item_1)) # {"test": true}
item_2 = {"one", "two", "three"}
print(to_json(item_2)) # TypeError: Object of type set is not JSON serializable
</code></pre>
<p>Since there are no “sets” in JSON, we have to do something like convert the <code class="highlighter-rouge">set</code> to a <code class="highlighter-rouge">list</code> before serializing it. We can use the <code class="highlighter-rouge">singledispatch</code> function to handle this. We will add <code class="highlighter-rouge">singledispatch</code> as a decorator to the original function. This will become the default option. We can create subsequent functions using a decorator like: <code class="highlighter-rouge">@to_json.register(type)</code>. Each of these can use an underscore in place of the function name because the decorator is used to reference the function. Any function that doesn’t match the data type passed into the <code class="highlighter-rouge">register</code> method will fall back to running the default function.</p>
<pre><code class="language-Python">import json
from functools import singledispatch
@singledispatch
def to_json(data):
return json.dumps(data)
@to_json.register(set)
def _(data):
return json.dumps(list(data))
</code></pre>
<p>In the <code class="highlighter-rouge">set</code> implementation, we simply convert the <code class="highlighter-rouge">set</code> to a <code class="highlighter-rouge">list</code> before serializing it. Although, the same result could have been handled using an <code class="highlighter-rouge">if</code> condition within a single function, this approach is often used in libraries where the differences between each function can become very extensive.</p>
<h3 id="single-dispatch-for-class-methods">Single Dispatch for Class Methods</h3>
<p>This approach is also available for class methods by using the <code class="highlighter-rouge">singledispatchmethod</code> function. Similar, to <code class="highlighter-rouge">singledispatch</code>, each subsequent method will use a <code class="highlighter-rouge">register</code> method with a data type passed in as an argument. Consider the following class:</p>
<pre><code class="language-Python">from functools import singledispatchmethod
class Math:
def __init__(self, value) -> None:
self.value = value
@singledispatchmethod
def add(self, item):
return self.value + item
@add.register(str)
def _(self, item):
return f"{self.value}, {item}"
@add.register(list)
def _(self, item):
return [self.value] + item
</code></pre>
<p>We set an initial value with a new instance of the “Math” class. Then, an <code class="highlighter-rouge">add</code> method will run depending on the data type of the argument. In the default method, we use the <code class="highlighter-rouge">+</code> operator to add to the initial value. However, if the value is a <code class="highlighter-rouge">string</code> or a <code class="highlighter-rouge">list</code>, we will either use string interpolation to combine the values or return a list with the initial value prepended to it.</p>
<pre><code class="language-Python">...
math = Math(7)
print(math.add(3)) # 10
print(math.add("lucky")) # 7, lucky
print(math.add([9, 11])) # [7, 9, 11]
</code></pre>functools is part of the Python standard library which is defined as being a “module is for higher-order functions” and that “functions that act on or return other functions”. In this post, we’ll go through a number examples incorporating the most common use cases.A Concise Guide to Using Pipenv2023-10-04T10:13:00-07:002023-10-04T10:13:00-07:00/python/2023/10/04/guide-to-using-pipenv<p>Python has multiple package managers to choose from, each having their own pros and cons. In this tutorial, we will explore <code class="highlighter-rouge">pipenv</code>. This option is especially powerful as it combines the use of <code class="highlighter-rouge">pip</code> and <code class="highlighter-rouge">virtualenv</code>.</p>
<h2 id="installing">Installing</h2>
<p>For Mac users, installing <code class="highlighter-rouge">pipenv</code> can simply be done using Homebrew:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>brew install pipenv
</code></pre></div></div>
<p>Alternatively, the installation can be done using <code class="highlighter-rouge">pip</code>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>pip install <span class="nt">--user</span> pipenv
</code></pre></div></div>
<h2 id="setting-up-a-new-project">Setting Up a New Project</h2>
<p>Now that <code class="highlighter-rouge">pipenv</code> is installed, it’s time to use it in a project. Create (or navigate) to a new folder for your project. Using the CLI, you can create a new virtual environment by running only <code class="highlighter-rouge">pipenv</code> command and passing in a Python version:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pipenv <span class="nt">--python</span> 3.11
</code></pre></div></div>
<p>In your project’s root directory, a new <code class="highlighter-rouge">Pipfile</code> should have been generated with contents similar to the following:</p>
<div class="language-toml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">[[source]]</span>
<span class="py">url</span> <span class="p">=</span> <span class="s">"https://pypi.org/simple"</span>
<span class="py">verify_ssl</span> <span class="p">=</span> <span class="kc">true</span>
<span class="py">name</span> <span class="p">=</span> <span class="s">"pypi"</span>
<span class="nn">[packages]</span>
<span class="nn">[dev-packages]</span>
<span class="nn">[requires]</span>
<span class="py">python_version</span> <span class="p">=</span> <span class="s">"3.11"</span>
<span class="py">python_full_version</span> <span class="p">=</span> <span class="s">"3.11.5"</span>
</code></pre></div></div>
<h3 id="installing-a-package">Installing a Package</h3>
<p>Obviously, we don’t have any packages installed yet. We’ll begin by installing <a href="https://flask.palletsprojects.com/en/3.0.x/">Flask</a> using the following CLI command:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pipenv install Flask
</code></pre></div></div>
<p>You’ll notice that in the <code class="highlighter-rouge">Pipfile</code>, the following package has been added:</p>
<p>Pipfile</p>
<div class="language-toml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">...</span>
<span class="nn">[packages]</span>
<span class="py">flask</span> <span class="p">=</span> <span class="s">"*"</span>
<span class="err">...</span>
</code></pre></div></div>
<p>The asterisk in the line that specifies <code class="highlighter-rouge">flask = "*"</code> refers to pulling the latest version. By adding a dependency, a separate <code class="highlighter-rouge">Pipfile.lock</code> file has bee created. At the time of writing, Flask 3.0.0 was the latest version which is what got installed.</p>
<p>The important thing to remember is that since this lock file has been created, version 3.0.0 will be used unless we deliberately update the version. If however, a new version comes out and we delete the lock file, we can install a newer version simply by running <code class="highlighter-rouge">pipenv install</code>.</p>
<h3 id="managing-sub-dependencies">Managing Sub-Dependencies</h3>
<p>Many packages have a number of dependencies they depend on. <code class="highlighter-rouge">pipenv</code> has a <code class="highlighter-rouge">graph</code> command that displays this in a readable format. This is great for tracking down issues like conflicts from different version requirements.</p>
<p>Running <code class="highlighter-rouge">pipenv graph</code></p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>brett$ pipenv graph
flask==3.0.0
- blinker [required: >=1.6.2, installed: 1.6.2]
- click [required: >=8.1.3, installed: 8.1.7]
- itsdangerous [required: >=2.1.2, installed: 2.1.2]
- Jinja2 [required: >=3.1.2, installed: 3.1.2]
- MarkupSafe [required: >=2.0, installed: 2.1.3]
- Werkzeug [required: >=3.0.0, installed: 3.0.0]
- MarkupSafe [required: >=2.1.1, installed: 2.1.3]
</code></pre></div></div>
<p>Notice that <code class="highlighter-rouge">Jinja2</code> and <code class="highlighter-rouge">Werkzeug</code> both have <code class="highlighter-rouge">MarkupSafe</code> as a dependency with slightly different version requirements. This can eventually get complicated with many installed packages, especially when attempting to update certain dependencies. Which is why having the graph command is useful.</p>
<h3 id="dev-dependencies">Dev Dependencies</h3>
<p>A common feature of package managers is to have separate dev dependencies. For running in production, it’s best to keep the application size as small as possible. Which means we generally don’t want to ship anything that’s not being used in production. This will often include things like packages for unit testing. Running the command below will install <code class="highlighter-rouge">pytest</code> as a dev dependency.</p>
<p>Pipfile</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pipenv install pytest <span class="nt">--dev</span>
</code></pre></div></div>
<p>Our <code class="highlighter-rouge">Pipfile</code> should now include the following:</p>
<div class="language-toml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">...</span>
<span class="nn">[packages]</span>
<span class="py">flask</span> <span class="p">=</span> <span class="s">"*"</span>
<span class="nn">[dev-packages]</span>
<span class="py">pytest</span> <span class="p">=</span> <span class="s">"*"</span>
<span class="err">...</span>
</code></pre></div></div>
<h3 id="installing-specific-versions">Installing Specific Versions</h3>
<p>Installing specific versions can be done by adding <code class="highlighter-rouge">==</code> operator followed by the version number:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pipenv install <span class="nv">requests</span><span class="o">==</span>2.31.0
</code></pre></div></div>
<p>You’ll notice the specific version specified in the <code class="highlighter-rouge">Pipfile</code>.</p>
<p>Pipfile</p>
<div class="language-toml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">...</span>
<span class="nn">[packages]</span>
<span class="py">flask</span> <span class="p">=</span> <span class="s">"*"</span>
<span class="py">requests</span> <span class="p">=</span> <span class="s">"==2.31.0"</span>
<span class="nn">[dev-packages]</span>
<span class="py">pytest</span> <span class="p">=</span> <span class="s">"*"</span>
<span class="err">...</span>
</code></pre></div></div>
<p>It’s important to note that you can also simply add/update the version in the <code class="highlighter-rouge">Pipfile</code> and run <code class="highlighter-rouge">pipenv install</code> to achieve the same result.</p>
<p>Even though the version gets locked in the lock file regardless, destroying the lock file in this case would now result in the same version being installed.</p>
<p>You can also specify a version range like the following:</p>
<div class="language-toml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">[packages]</span>
<span class="py">requests</span> <span class="p">=</span> <span class="s">">=2.31,<3"</span>
</code></pre></div></div>
<p>This is a more flexible approach that allows for updates but is safer than allowing any version (specified by <code class="highlighter-rouge">*</code>). A major version update (i.e. to version 3) often has more drastic changes that could break API’s used by the application.</p>
<h2 id="removing-packages">Removing Packages</h2>
<p>This is very straightforward. The <code class="highlighter-rouge">uninstall</code> CLI command can be used to remove packages:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pipenv uninstall requests
</code></pre></div></div>
<h2 id="finding-the-virtual-environment-path">Finding the Virtual Environment Path</h2>
<p>Obtaining the path to the virtual environment is often needed in certain cases. Adding the path to our editor is necessary for autocomplete and intellisense to work with imported packages. Running <code class="highlighter-rouge">pipenv --venv</code> will output the path in the terminal.</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>brett$ pipenv --venv
/Users/user/.local/share/virtualenvs/pipenvtest-u5Lgc-F2
</code></pre></div></div>
<h2 id="loading-the-virtual-environment">Loading the Virtual Environment</h2>
<p>The shell command allows you to load the virtual environment in your terminal window. When the virtual environment is loaded, you will see the name of the virtual environment on the left side of the terminal prompt in parentheses.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pipenv shell
</code></pre></div></div>
<p>Once the shell is loaded, any <code class="highlighter-rouge">python</code> command will use the version of Python from the virtual environment. Exiting the shell is as simple as entering an “exit” command.</p>
<p>Exit the shell:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">exit</span>
</code></pre></div></div>
<h2 id="running-python-from-outside-the-shell">Running Python from Outside the Shell</h2>
<p>There will be times you’ll want to use your virtual environment without having to load the shell. The <code class="highlighter-rouge">run</code> command handles this situation. If you just want to run an interactive Python console, that can be achieved with the following:</p>
<p>Python Console:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pipenv run python
</code></pre></div></div>
<p>The majority of the time, however, you’ll want to run a script or launch a web server. if the file you want to run is <code class="highlighter-rouge">main.py</code>, for example, that can be handled with the following:</p>
<p>Running a Python Script:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pipenv run python main.py
</code></pre></div></div>
<h2 id="install-packages-from-lockfile">Install Packages from Lockfile</h2>
<p>In certain environments, such as CI or production, you’ll want to install packages exactly as defined in the lock file. You can simply pass the <code class="highlighter-rouge">ignore-pipfile</code> option with the <code class="highlighter-rouge">install</code> command.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pipenv install <span class="nt">--ignore-pipfile</span>
</code></pre></div></div>
<h2 id="check-for-security-vulnerabilities">Check for Security Vulnerabilities</h2>
<p>The <code class="highlighter-rouge">check</code> command will check the <a href="https://safetycli.com/">PyUp/Safety</a> security vulnerability database to see if any of your installed packages are an issue.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pipenv check
</code></pre></div></div>
<p>Here’s an example for running the <code class="highlighter-rouge">check</code> command using an old project:</p>
<p><img src="/assets/2023/10/pipenv-check-vulnerability-issue.png" alt="pipenv security check vulnerability example" /></p>
<h2 id="switching-to-pipenv-from-pip">Switching to Pipenv from pip</h2>
<p>If you have an existing project you wish to convert to using <code class="highlighter-rouge">pipenv</code>, it can be easily done if you have a <code class="highlighter-rouge">requirements.txt</code> file. Simply add the <code class="highlighter-rouge">-r</code> option followed by the filename:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pipenv install <span class="nt">-r</span> requirements.txt
</code></pre></div></div>
<h3 id="generate-requirements-file">Generate Requirements File</h3>
<p>In certain cases, you’ll want to still create a <code class="highlighter-rouge">requirements.txt</code> file. Maybe you want to share the application with someone not using <code class="highlighter-rouge">pipenv</code>. The <code class="highlighter-rouge">requirements</code> command will create this file for you. The <code class="highlighter-rouge">requirements</code> command itself will output the requirements to the screen, so standard out must be used to actually write the requirements to a file.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pipenv requirements <span class="o">></span> requirements.txt
</code></pre></div></div>
<h2 id="cleanup-and-remove-virtual-environment">Cleanup and Remove Virtual Environment</h2>
<p>The <code class="highlighter-rouge">--rm</code> option will completely remove your virtual environment.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pipenv <span class="nt">--rm</span>
</code></pre></div></div>Python has multiple package managers to choose from, each having their own pros and cons. In this tutorial, we will explore pipenv. This option is especially powerful as it combines the use of pip and virtualenv.Insomnia REST Client Intro2019-04-20T11:24:00-07:002019-04-20T11:24:00-07:00/other/2019/04/20/insomnia-rest-client<p>REST Clients have long been a part of a software developer’s toolset. In a nutshell, they are a shortcut for testing and debugging API’s. Popular clients have consisted of browser extensions as well as desktop applications. In this post, I’ll will be covering Insomnia, a cross-platform REST Client which is packed full of great features. Making basic requests (i.e. GET, POST) is merely the tip of the iceberg.</p>
<p>If you don’t have Insomnia installed, you can access it <a href="https://insomnia.rest/">here</a>.</p>
<iframe width="740" height="417" src="https://www.youtube.com/embed/H16GUC9Svyk" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen=""></iframe>
<h2 id="creating-http-requests">Creating HTTP Requests</h2>
<p>For starters, making simple, straightforward HTTP requests are generally what REST clients are known for. For demonstrating some HTTP requests, we’ll cover some examples using <a href="https://httpbin.org/">httpbin.org</a>, which is an online HTTP request/response service.</p>
<h3 id="get-requests">GET Requests</h3>
<p>After opening the application, we will start by creating a new request. You can use the shortcut <code class="highlighter-rouge">⌘N</code>, or just click the “New Request” button. All that’s needed for the next step is to enter a name and a request method. Then, hit “Create”.</p>
<p><img src="/assets/2019/04/insomnia-new-request.png" alt="Insomnia new request" title="Insomnia new request" /></p>
<p>Notice that each time a new request is entered, you will see it populate in a list in the left column of the window. You can also have multiple workspaces to group your requests together.</p>
<p><img src="/assets/2019/04/insomnia-get-request.png" alt="Insomnia get request" title="Insomnia get request" />
After adding the URL, then clicking “Send”, a preview of the response will be displayed in a split window. It will automatically be pretty-printed to make JSON easy to read.</p>
<p>Clicking on the Query tab will allow adding query parameters. You can append the params to the URL, but using the tab will make it much easier to read and modify. Another added benefit is the ability to toggle invididual parameters on and off.</p>
<p><img src="/assets/2019/04/insomnia-get-query-params.png" alt="Insomnia get request with query parameters" title="Insomnia get request with query parameters" />
Above: a GET request with a single query parameter. Httpbin will return any GET params as part of the response in an “args” object.</p>
<h3 id="post-requests">Post Requests</h3>
<p>Insomnia is also great for posting data. We will cover 2 ways of posting data: 1) using multipart form data, and 2) using a JSON payload.</p>
<p>After choosing which POST method to use, you can click the dropdown next to the “Body” tab. We will begin by selecting “Multipart Form”.</p>
<p><img src="/assets/2019/04/insomnia-multipart-form.png" alt="Insomnia multipart form body" title="Insomnia multipart form body" /></p>
<p>In the next step, we’re going to add some form data. Then, click “Send”.</p>
<p><img src="/assets/2019/04/insomnia-post-request.png" alt="Insomnia POST Request" title="Insomnia POST Request" /></p>
<p>Again, httpbin.org will post our data back to use inside the response. This time, the data is in the “form” object.</p>
<p>The next POST example will consist of sending a JSON payload. Before we start, however, Insomnia has a nice “duplicate” feature. Since we’re really only changing one thing, we just as well take advantage of this:</p>
<p><img src="/assets/2019/04/insomnia-rest-client-duplicate.png" alt="Insomnia REST Client Duplicate" title="Insomnia REST Client Duplicate" /></p>
<p>The name of the duplicate will be “httpbin POST Form Data (copy)”. Renaming the duplicate is as simple as double-clicking it. We will also need to switch the “Body” dropdown to “JSON”:</p>
<p><img src="/assets/2019/04/insomnia-body-select-json.png" alt="Insomnia select JSON" title="Insomnia select JSON" /></p>
<p>Whenever this is switched, keep in mind that any POST data that was saved will be lost. In the top of the split window, you can now enter a JSON object. We will use the same data as the for POST:</p>
<p><img src="/assets/2019/04/insomnia-post-json.png" alt="Insomnia POST JSON payload" title="Insomnia POST JSON payload" /></p>
<p>Insomnia also has a built-in JSON validator to make sure you’ve entered valid JSON. For example, if we remove the comma after the username, we will be shown the following error:</p>
<p><img src="/assets/2019/04/insomnia-json-error.png" alt="Insomnia JSON parse error" title="Insomnia JSON parse error" />
These are just some of the powerful features Insomnia REST client has. In the next part, we will look at JSONpath to inspect data in the preview window. This is especially handy for API responses containing an overwhelming amount of data.</p>
<h2 id="using-jsonpath-to-filter-results">Using JSONpath to Filter Results</h2>
<p>JSONpath is a very nice tool for evaluating API responses. In this example, we will pull a list of countries from the <a href="http://www.worldbank.org/">World Bank</a>. Within our workspace, let’s setup a new GET request using this URL: <code class="highlighter-rouge">http://api.worldbank.org/countries?format=json</code>. The API will return to us a list of countries.</p>
<p><img src="/assets/2019/04/world-bank-api.png" alt="Insomnia World Bank Countries API" title="Insomnia World Bank Countries API" /></p>
<p>At the bottom-right, you will see an input field with the placeholder <code class="highlighter-rouge">$.store.books[*].author</code>. It is in this field where you can enter your JSONpath expression.</p>
<p>The API repsonse consists of an array with the first item being an meta data object and the second being another array of countries. Before we start filtering some data, you can find a concise JSONpath reference <a href="https://goessner.net/articles/JsonPath/">here</a>. The <code class="highlighter-rouge">$</code> symbol represents the root element. So, our expressions will begin with a <code class="highlighter-rouge">$</code>.</p>
<p>If we just want to extract the metadata by itself, we will need to grab the first item in the array. This is very simple. The expression <code class="highlighter-rouge">$[0]</code> will do the trick:</p>
<p><img src="/assets/2019/04/jsonpath-metadata.png" alt="JSONpath expression to get metadata" title="JSONpath expression to get metadata" />
As you can see from above, all we needed was to get the first item in the array because the array was our root element. That being said, the list of countries is much more interesting.</p>
<p>An expression of <code class="highlighter-rouge">$[1]</code> will give us the full list of countries from this API request (note: the results are paginated so the length of the full list will be 50). If we just want the first country we can get the first item in the countries array. Since we have a nested array, we will need to go through the root element first. The following expression will handle our case:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$[1][0]
</code></pre></div></div>
<p><img src="/assets/2019/04/jsonpath-first-item.png" alt="JSONpath get first item" title="JSONpath get first item" /></p>
<p>JSONpath also has some elegant solutions for getting ranges. You can use the colon operator <code class="highlighter-rouge">:</code> to select a range of countries by index. Grabbing the first 3 elements can be done with this expression:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$[1][0:3]
</code></pre></div></div>
<p>Let’s say we only want the first and third elements. The comma <code class="highlighter-rouge">,</code> is available as a union operator:</p>
<p>First and thid elements:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$[1][0,2]
</code></pre></div></div>
<p>We also know there are 50 countries in our response. If we were not given a count, we would still be able to find the last country in the list. Using the <code class="highlighter-rouge">@</code> operator, we can reference the current element. We can subtract 1 from the length property of the coutries array. This will enable us to exract just the last country, which in our case, is China:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$[1][(@.length-1)]
</code></pre></div></div>
<p><img src="/assets/2019/04/jsonpath-last-element.png" alt="JSONpath get last element" title="JSONpath get last element" />
Above: the parentheses are needed to express that its contents are a script expression, allowing us to use the length property and subtract 1.</p>
<p>For the next example, let’s grab only a list of country names. As you can see from the previous examples, each country is an object that contains data which includes things like income, location, and the capital city (in addition to the name). We can use the asterisk wildcard operator <code class="highlighter-rouge">*</code> to get the name property of every item in the list:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$[1].*.name
</code></pre></div></div>
<p><img src="/assets/2019/04/jsonpath-wildcard-operator.png" alt="JSONpath wildcard operator" title="JSONpath wildcard operator" />
One thing you’ll also notice from the list above is that not every result is a country. Some of these are regions, such as “East Asia & Pacific…”. If we want to filter these out, leaving just the names of actual countries, that is also possible with JSONpath. To accomplish this, we need to swap out our wildcard with a filter expression. A filter expression will consist of the following syntax:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>?(<boolean expr>)
</code></pre></div></div>
<p>Previously, we had <code class="highlighter-rouge">$[1].*.name</code>. We used the asterisk wildcard assuming that each item in the list was a country. Since this is not the case, we need to find something that’s different between countries and regions in the results. From inspecting the raw data, since regions do not capital cities, the <code class="highlighter-rouge">capitalCity</code> property is an empty string. So, this is what we will use for our filter expression:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$[1][?(@.capitalCity != '')].name
</code></pre></div></div>
<p><img src="/assets/2019/04/jsonpath-filter-expression.png" alt="JSONpath filter expression" title="JSONpath filter expression" /></p>
<p>Having JSONpath available is incredibly valueable for inspecting and filtering data. Another powerful feature of Insomnia is the ability to use variables; which is what we’ll cover next.</p>
<h2 id="using-variables">Using Variables</h2>
<p>Within a single workspace, you can setup variables which will help you become more productive. A workspace will generally have a collection of API calls that have many similarities. Perhaps they all share the same domain name. Or, each API call may use the same API key. There are a myraid of things you can do with this feature.</p>
<p>For the next several examples, we will switch to using the FRED database API’s. These API’s are available form the <a href="https://fred.stlouisfed.org/">Federal Reserve Bank of St. Louis</a>. (You can register and get an API key using this link.)</p>
<p>For our first API call, we will be getting the historic interest rate data for the US 30-year mortgage. Our full API will be:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>https://api.stlouisfed.org/fred/series/observations?api_key=af0b5e18a4931234d45828f6467fdec4&series_id=MORTGAGE30US&duration=30&file_type=json
</code></pre></div></div>
<p>NOTE: This is a made up API key. If you register and get your own API key, you will get a similar 32-character API key.</p>
<p>Running the request gives us the following result:</p>
<p><img src="/assets/2019/04/insomnia-fred-30-mortgage.png" alt="Insomnia FRED 30-year Mortgage" title="Insomnia FRED 30-year Mortgage" /></p>
<p>We can also add a similar request for 15-year mortgages. For adding this, we will use the duplicate feature (as mentioned earlier in this post, you can access it by hovering over the name of a request in the left column). A dropdown arrow will appear on the right and clicking it will reveal a list of choices:</p>
<p><img src="/assets/2019/04/insomnia-duplicate-request.png" alt="Insomnia Create Duplicate Request" title="Insomnia Create Duplicate Request" /></p>
<p>Initially, clicking “Duplicate” will name the new request “FRED 30-Year Mortgage (copy)”. You can rename this by simply double-clicking it. I will just change the 30 to 15. Within this API, the only 2 things that will be changing will be in the query params. The base URL will be the same, along with the API key. This is where variables come in handy.</p>
<p>To start using variables, we first need to setup an environment or use the base environment. You can use as many environments as you’d like within a single workspace. To keep things simple, we will use the base environment. Any variables added to the base environment will always be available throughout your workspace.</p>
<p>Access environment vaiables can be done by clicking the dropdown arrow next to “No Environment”. (Our text says “No Environment” because we have not setup an environment)</p>
<p><img src="/assets/2019/04/insomnia-environment-setup.png" alt="Insomnia New Environment Setup" title="Insomnia New Environment Setup" /></p>
<p>Since we’re using only the base environment, we can simply add our info to the empty JSON object that appears in the Environments window.</p>
<p><img src="/assets/2019/04/insomnia-base-environment-variables.png" alt="Insomnia Base Environment Variables" title="Insomnia Base Environment Variables" /></p>
<p>At this point, we can go back and modify our saved requests to use the variables we’ve just created. We will change the 30-year mortgage API first. These variables can be used in multiple places including the URL for the request and the query parameters:</p>
<p><img src="/assets/2019/04/insomnia-using-variables-in-requests.png" alt="Insomnia Using Custom Variables" title="Insomnia Using Custom Variables" /></p>
<p>Running the request should produce the same result as before, grabbing the historic interest rate data. We can also make the same changes to the 15-year mortgage API. Another import advantage to setting these up is that if one of these changes (i.e. we get a new API key), we will only have to apply the change in one place and all of our requests will be updated.</p>
<h2 id="exporting-requests-and-generating-code">Exporting Requests and Generating Code</h2>
<p>Insomina offers several nice options for exporing requests. You can generate CURL commands to run using the command line. You can also generate language-specific code for executing requests. Beyond generating code for individual requests, you can also export your entire workspace, which we’ll also cover in this section.</p>
<p>Copying as a CURL command is very easy. Simply click the down arrow next to the request in the left column. You will see a “Copy as CURL” option. Clicking this will automatically copy it to your clipboard. For this demo, I will use the Post JSON payload request from earlier. Since it’s hard to remember CURL syntax beyond a simple GET request (not to mention having to avoid typos), this makes things much easier.</p>
<p><img src="/assets/2019/04/insomnia-copy-as-curl.png" alt="Insomnia Copy as CURL" title="Insomnia Copy as CURL" /></p>
<p>Pasting the CURL command will provide the following output:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl <span class="nt">--request</span> POST <span class="se">\</span>
<span class="nt">--url</span> https://httpbin.org/post <span class="se">\</span>
<span class="nt">--header</span> <span class="s1">'content-type: application/json'</span> <span class="se">\</span>
<span class="nt">--data</span> <span class="s1">'{
"username": "johnsmith",
"email": "me@example.com"
}'</span>
</code></pre></div></div>
<p>Generating language-specific code can be done just as easily. Simply select “Generate Code” instead of “Copy as CURL”. A modal window will appear. In one dropdown, you can select your programming language. If there is more than 1 popular method of sending requests, you will have different options. For example, if your using Python’s popular <code class="highlighter-rouge">requests</code> library, that will be one of the options:</p>
<p><img src="/assets/2019/04/insomnia-generate-code.png" alt="Insomnia Generate Code option" title="Insomnia Generate Code" /></p>
<p><img src="/assets/2019/04/insomnia-generate-code-snippet.png" alt="Insomnia Generate Code Snippet" title="Insomnia Generate Code Snippet" /></p>
<h3 id="exporting-a-workspace">Exporting a Workspace</h3>
<p>There are multiple cases in which you’d want to export a workspace. You may want to share your workspace with someone else. Or, you may switch between machines such as your desktop and laptop, work and home computers, and so on. My most common case is the latter in which I keep exports of workspaces synched within a Dropbox folder.</p>
<p>Before we dive in, it’s important to note that there are more robust sharing/collaborating features available in the paid “Plus” version. Since we’re evaluating the free version, we won’t be covering any premium features. You can find more info regarding the Plus version here: <a href="https://insomnia.rest/pricing/">https://insomnia.rest/pricing/</a>.</p>
<p>We can export our current workspace by selecting the dropdown next to the workspace name at the top-left, then selecting “Import/Export”:</p>
<p><img src="/assets/2019/04/insomnia-import-export.png" alt="Insomnia Import/Export option" title="Insomnia Import/Export option" /></p>
<p>A modal window will appear. Click the “Export Data” dropdown followed by “Current Workspace”.</p>
<p><img src="/assets/2019/04/insomnia-export-current-workspace.png" alt="Insomnia Export Current Workspace" title="Insomnia Export Current Workspace" /></p>
<p>You will have 2 options: the Insomnia file type which can be imported into another Insomnia client. Or, a <a href="https://en.wikipedia.org/wiki/.har">HTTP Archive Format</a> (HAR) which is a more standard document type. Click “Done” will lead you to a Save As menu in which you can choose your storage location as well as a file name.</p>
<p>In this post, we’ve covered some of the best features Insomnia REST Client offers. Although there other popular options available, this one is among the most versatile and has many great features when it comes to using variables, generating client code, and exporting/sharing workspaces.</p>REST Clients have long been a part of a software developer’s toolset. In a nutshell, they are a shortcut for testing and debugging API’s. Popular clients have consisted of browser extensions as well as desktop applications. In this post, I’ll will be covering Insomnia, a cross-platform REST Client which is packed full of great features. Making basic requests (i.e. GET, POST) is merely the tip of the iceberg.Posting Data and Using Sessions with Requests2019-03-01T10:24:00-08:002019-03-01T10:24:00-08:00/python/2019/03/01/python-requests-posting-data<p>In the previous post, we covered downloading/pulling data using Requests. We downloaded and manipulated data from HTML web pages as well as API’s. Posting data is what will be covered here. We will be submitting data as if it were from an HTML form. We will also be posting data in JSON format in the form of a payload.</p>
<p><a href="https://github.com/kishstats/Python-Requests-Tutorial" class="source-code-link">
<button type="button">
<span class="icon icon--github"><svg viewBox="0 0 16 16" width="25px" height="25px"><path fill="#111" d="M7.999,0.431c-4.285,0-7.76,3.474-7.76,7.761 c0,3.428,2.223,6.337,5.307,7.363c0.388,0.071,0.53-0.168,0.53-0.374c0-0.184-0.007-0.672-0.01-1.32 c-2.159,0.469-2.614-1.04-2.614-1.04c-0.353-0.896-0.862-1.135-0.862-1.135c-0.705-0.481,0.053-0.472,0.053-0.472 c0.779,0.055,1.189,0.8,1.189,0.8c0.692,1.186,1.816,0.843,2.258,0.645c0.071-0.502,0.271-0.843,0.493-1.037 C4.86,11.425,3.049,10.76,3.049,7.786c0-0.847,0.302-1.54,0.799-2.082C3.768,5.507,3.501,4.718,3.924,3.65 c0,0,0.652-0.209,2.134,0.796C6.677,4.273,7.34,4.187,8,4.184c0.659,0.003,1.323,0.089,1.943,0.261 c1.482-1.004,2.132-0.796,2.132-0.796c0.423,1.068,0.157,1.857,0.077,2.054c0.497,0.542,0.798,1.235,0.798,2.082 c0,2.981-1.814,3.637-3.543,3.829c0.279,0.24,0.527,0.713,0.527,1.437c0,1.037-0.01,1.874-0.01,2.129 c0,0.208,0.14,0.449,0.534,0.373c3.081-1.028,5.302-3.935,5.302-7.362C15.76,3.906,12.285,0.431,7.999,0.431z"></path></svg>
</span>
Source Code
</button>
</a></p>
<iframe width="740" height="417" src="https://www.youtube.com/embed/Zmhka9oH3V4" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen=""></iframe>
<p>The examples covered below will be using <a href="https://httpbin.org/">httpbin.org</a>, which is an HTTP request & response service. This site has some nice features with one of them being that it will return the data you originally posted in the response. This can make debugging easier.</p>
<h2 id="posting-form-data">Posting Form Data</h2>
<p>If you have not installed <code class="highlighter-rouge">requests</code> already, it can be done easily using <code class="highlighter-rouge">pip</code>.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pip install requests
</code></pre></div></div>
<p>Next, we can create a new Python script to import <code class="highlighter-rouge">requests</code> and setup a variable for our target URL. We will also be using a dictionary to post form data.</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">requests</span>
<span class="n">url</span> <span class="o">=</span> <span class="s">'https://httpbin.org/post'</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">{</span><span class="s">'user'</span><span class="p">:</span><span class="s">'me@example.com'</span><span class="p">}</span>
</code></pre></div></div>
<p>Now, we’re ready to use <code class="highlighter-rouge">requests</code> to post the user data to our target URL:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">requests</span>
<span class="n">url</span> <span class="o">=</span> <span class="s">'https://httpbin.org/post'</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">{</span><span class="s">'user'</span><span class="p">:</span><span class="s">'me@example.com'</span><span class="p">}</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">post</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">data</span><span class="p">)</span>
<span class="k">print</span><span class="p">(</span><span class="n">response</span><span class="p">)</span> <span class="c"># <Response [200]></span>
</code></pre></div></div>
<p>Notice that our response variable is a <code class="highlighter-rouge">Response</code> object. To be able to use this data, we need to apply a method or a property. We will apply the <code class="highlighter-rouge">text</code> property in the next example. The text property will give us all the response data as a single string:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">requests</span>
<span class="n">url</span> <span class="o">=</span> <span class="s">'https://httpbin.org/post'</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">{</span><span class="s">'user'</span><span class="p">:</span><span class="s">'me@example.com'</span><span class="p">}</span>
<span class="c"># as form data</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">post</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">data</span><span class="p">)</span>
<span class="k">print</span><span class="p">(</span><span class="n">response</span><span class="p">)</span> <span class="c"># <Response [200]></span>
<span class="n">result</span> <span class="o">=</span> <span class="n">response</span><span class="o">.</span><span class="n">text</span>
<span class="k">print</span><span class="p">(</span><span class="nb">type</span><span class="p">(</span><span class="n">result</span><span class="p">))</span> <span class="c"># <class 'str'></span>
<span class="k">print</span><span class="p">(</span><span class="n">result</span><span class="p">)</span>
</code></pre></div></div>
<p>Printed results below:</p>
<p><img src="/assets/2019/02/requests-form-post-response.png" alt="Requests form post response" title="Requests form post response" />
Since this is merely a string (httpbin.org kindly returns a pretty-printed version with all the spacing), it’s difficult to make use of this data. However, as covered in the previous post, Requests has a built-in JSON decoder that we can use:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">from</span> <span class="nn">pprint</span> <span class="kn">import</span> <span class="n">pprint</span>
<span class="n">url</span> <span class="o">=</span> <span class="s">'https://httpbin.org/post'</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">{</span><span class="s">'user'</span><span class="p">:</span><span class="s">'me@example.com'</span><span class="p">}</span>
<span class="c"># as form data</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">post</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">data</span><span class="p">)</span>
<span class="k">print</span><span class="p">(</span><span class="n">response</span><span class="p">)</span> <span class="c"># <Response [200]></span>
<span class="n">result</span> <span class="o">=</span> <span class="n">response</span><span class="o">.</span><span class="n">json</span><span class="p">()</span>
<span class="k">print</span><span class="p">(</span><span class="nb">type</span><span class="p">(</span><span class="n">result</span><span class="p">))</span> <span class="c"># <class 'dict'></span>
<span class="n">pprint</span><span class="p">(</span><span class="n">result</span><span class="p">)</span>
</code></pre></div></div>
<p>Our output to the terminal window will look similar (we used <code class="highlighter-rouge">pprint</code> to pretty-print our dictionary data to help make it readable). The important difference is that we have dictionary variable in which we can work with to access our data:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">...</span>
<span class="k">print</span><span class="p">(</span><span class="n">result</span><span class="p">[</span><span class="s">'form'</span><span class="p">])</span> <span class="c"># {'user': 'me@example.com'}</span>
</code></pre></div></div>
<h2 id="posting-a-json-payload">Posting a JSON Payload</h2>
<p>Sending a POST request using a JSON payload is different from sending form data. Form data is sent using a series of key-value pairs. Alternatively, a payload consists of sending everything in one, single chunk of data. Here is a quick breakdown of the differences between sending form data versus a JSON payload:</p>
<p>Form Data:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>POST
Content-Type: application/x-www-form-urlencoded
user=me@example.com
</code></pre></div></div>
<p>JSON Payload:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>POST
Content-Type: application/json
{"user":"me@example.com"}
</code></pre></div></div>
<p>Notice that there is also a change to the <code class="highlighter-rouge">Content-Type</code> in the header. If we have our Python script setup the same way, except that we’re switching to a JSON Payload, we will need to convert our data into JSON. For this, we can use the built-in <code class="highlighter-rouge">json</code> Python library:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">import</span> <span class="nn">json</span>
<span class="kn">from</span> <span class="nn">pprint</span> <span class="kn">import</span> <span class="n">pprint</span>
<span class="n">url</span> <span class="o">=</span> <span class="s">'https://httpbin.org/post'</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">{</span><span class="s">'user'</span><span class="p">:</span><span class="s">'me@example.com'</span><span class="p">}</span>
<span class="c"># as payload</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">post</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">))</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">response</span><span class="o">.</span><span class="n">json</span><span class="p">()</span>
<span class="n">pprint</span><span class="p">(</span><span class="n">result</span><span class="p">)</span>
</code></pre></div></div>
<p>The data we intend to post is a dictionary. By using the <code class="highlighter-rouge">json.dumps</code> method, we can convert the dictionary into a JSON-formatted string to post as a payload. Also, as we did previously, we can apply the Requests JSON-decoder to convert our response info to a dictionary.</p>
<p>Previously, it was mentioned that it’s common to set the <code class="highlighter-rouge">Content-Type</code> in the header. We can setup a dictionary variable with our custom headers. We will do that by setting <code class="highlighter-rouge">Content-Type</code> to <code class="highlighter-rouge">application/json</code>:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">...</span>
<span class="n">url</span> <span class="o">=</span> <span class="s">'https://httpbin.org/post'</span>
<span class="n">headers</span> <span class="o">=</span> <span class="p">{</span>
<span class="s">'Content-Type'</span><span class="p">:</span> <span class="s">'application/json'</span><span class="p">,</span>
<span class="c"># additional headers here</span>
<span class="p">}</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">{</span><span class="s">'key'</span><span class="p">:</span><span class="s">'value'</span><span class="p">}</span>
<span class="c"># as payload</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">post</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">),</span> <span class="n">headers</span><span class="o">=</span><span class="n">headers</span><span class="p">)</span>
<span class="c"># headers that were originally sent</span>
<span class="k">print</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">headers</span><span class="p">)</span>
<span class="o">...</span>
</code></pre></div></div>
<p>With this added, you should be able to verify that this header was set because httpbin.org will return it back with the result. Alternatively, we can also access the original headers sent from the <code class="highlighter-rouge">response.request.headers</code> variable.</p>
<h2 id="working-with-request-sessions">Working with Request Sessions</h2>
<p>Eventually, you will run into situations where you must persist a user session. Let’s say you first have to log in/authenticate, which sets a browser cookie that must be sent with each subsequent request.</p>
<p>Using httpbin once again, we will save a cookie and then try to retrieve it. Consider the following:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">requests</span>
<span class="c"># loading this URL will set a cookie</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'https://httpbin.org/cookies/set/abc/123'</span><span class="p">)</span>
<span class="k">print</span><span class="p">(</span><span class="s">'res: {}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">res</span><span class="o">.</span><span class="n">text</span><span class="p">))</span>
<span class="c"># No cookie data found!</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'https://httpbin.org/cookies'</span><span class="p">)</span>
<span class="k">print</span><span class="p">(</span><span class="s">'res: {}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">res</span><span class="o">.</span><span class="n">text</span><span class="p">))</span>
<span class="c"># Outputs:</span>
<span class="c"># res: {</span>
<span class="c"># "cookies": {}</span>
<span class="c"># }</span>
</code></pre></div></div>
<p>In the example above, each subsequent request is starting a new session. We can get around this dilemma by using a <code class="highlighter-rouge">Session</code> object. We will use a context manager that will enclose all the HTTP requests made within our session:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">requests</span>
<span class="k">with</span> <span class="n">requests</span><span class="o">.</span><span class="n">Session</span><span class="p">()</span> <span class="k">as</span> <span class="n">s</span><span class="p">:</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">s</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'https://httpbin.org/cookies/set/abc/123'</span><span class="p">)</span>
<span class="k">print</span><span class="p">(</span><span class="s">'res: {}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">res</span><span class="o">.</span><span class="n">text</span><span class="p">))</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">s</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'https://httpbin.org/cookies'</span><span class="p">)</span>
<span class="k">print</span><span class="p">(</span><span class="s">'res: {}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">res</span><span class="o">.</span><span class="n">text</span><span class="p">))</span>
<span class="c"># Outputs</span>
<span class="c"># res: {</span>
<span class="c"># "cookies": {</span>
<span class="c"># "abc": "123"</span>
<span class="c"># }</span>
<span class="c"># }</span>
<span class="k">print</span><span class="p">(</span><span class="n">s</span><span class="o">.</span><span class="n">cookies</span><span class="p">)</span> <span class="c"># <RequestsCookieJar[<Cookie abc=123 for httpbin.org/>]></span>
<span class="k">print</span><span class="p">(</span><span class="s">'actual cookies: {}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">s</span><span class="o">.</span><span class="n">cookies</span><span class="o">.</span><span class="n">get_dict</span><span class="p">()))</span> <span class="c"># actual cookies: {'abc': '123'}</span>
</code></pre></div></div>
<p>Similar to the previous attempt, we’re setting cookie “abc” with the value of “123”. Since we’re using the same session to get our cookies, the data will be returned to us. Note that httpbin returns the cookie info in the response. To get the actual cookies, there is a <code class="highlighter-rouge">RequestsCookieJar</code> attached to the session. Using this, we can call the <code class="highlighter-rouge">get_dict</code> method to get our cookies in a dictionary format.</p>
<p>Setting a requests session is necessary when you need to make multiple requests. And, each subsequent request will require persisting data, such as a session cookie.</p>In the previous post, we covered downloading/pulling data using Requests. We downloaded and manipulated data from HTML web pages as well as API’s. Posting data is what will be covered here. We will be submitting data as if it were from an HTML form. We will also be posting data in JSON format in the form of a payload.Pulling Data with Requests2019-02-17T10:24:00-08:002019-02-17T10:24:00-08:00/python/2019/02/17/python-requests-pulling-data<p>Requests, being one of the most popular Python modules, is a highly regarded tool for sending HTTP requests. In this post, we will be pulling data from HTML pages, JSON API’s, and XML API’s.</p>
<iframe width="740" height="417" src="https://www.youtube.com/embed/jeZp5NE9lII" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen=""></iframe>
<h2 id="pulling-data-from-html-pages">Pulling Data from HTML Pages</h2>
<p>Requests makes downloading a webpage easy. For this example, we will pull in a list of countries from the <a href="https://www.cia.gov">cia.gov</a> website. Specifically, we will use this URL: <a href="https://www.cia.gov/library/publications/the-world-factbook/rankorder/2004rank.html">https://www.cia.gov/library/publications/the-world-factbook/rankorder/2004rank.html</a>.</p>
<p>If you have not installed <code class="highlighter-rouge">requests</code> already, it can be done easily using <code class="highlighter-rouge">pip</code>.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pip install requests
</code></pre></div></div>
<p>Next, we can create a new Python script to import <code class="highlighter-rouge">requests</code> and setup a variable for our target URL:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">requests</span>
<span class="n">url</span> <span class="o">=</span> <span class="s">'https://www.cia.gov/library/publications/the-world-factbook/rankorder/2004rank.html'</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="n">html</span> <span class="o">=</span> <span class="n">response</span><span class="o">.</span><span class="n">text</span>
<span class="k">print</span><span class="p">(</span><span class="n">html</span><span class="p">)</span> <span class="c"># will print out the full source of the webpage</span>
</code></pre></div></div>
<p>Once we use <code class="highlighter-rouge">requests</code> to fetch the HTML for us, we need some way of parsing it to get the specific data that we want. <a href="https://www.crummy.com/software/BeautifulSoup/">Beautiful Soup</a> is a great library for parsing HTML. We can also import Beautiful Soup using <code class="highlighter-rouge">pip</code>:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pip install bs4
</code></pre></div></div>
<p>For this page, let’s grab a list of country codes. If you load our target URL using your web browser, you will see that the table has an id of “rankOrder”:</p>
<p><img src="/assets/2019/02/cia-html-markup.png" alt="alt text" title="CIA webpage HTML markup" />
Inside each table row, there is also a column with a class of region as well as a link that contains the country code. Our next step will be to use Beautiful Soup to get all the ISO Codes. We can do this by feeding our HTML into Beautiful Soup, then find the table by it’s ID:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">from</span> <span class="nn">bs4</span> <span class="kn">import</span> <span class="n">BeautifulSoup</span>
<span class="n">url</span> <span class="o">=</span> <span class="s">'https://www.cia.gov/library/publications/the-world-factbook/rankorder/2004rank.html'</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="n">html</span> <span class="o">=</span> <span class="n">response</span><span class="o">.</span><span class="n">text</span>
<span class="n">soup</span> <span class="o">=</span> <span class="n">BeautifulSoup</span><span class="p">(</span><span class="n">html</span><span class="p">,</span> <span class="s">'html.parser'</span><span class="p">)</span>
<span class="n">table</span> <span class="o">=</span> <span class="n">soup</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="nb">id</span><span class="o">=</span><span class="s">'rankOrder'</span><span class="p">)</span>
<span class="n">rows</span> <span class="o">=</span> <span class="n">table</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="s">'tr'</span><span class="p">)</span>
</code></pre></div></div>
<p>When installing Beautiful Soup, our module was named <code class="highlighter-rouge">bs4</code>. And, from there we can import the <code class="highlighter-rouge">BeautifulSoup</code> object. We set the response text to our <code class="highlighter-rouge">html</code> variable earlier. So, now we can feed this in when creating a new <code class="highlighter-rouge">BeautifulSoup</code> instance. We have to set a parser so it can determine what it’s parsing (<code class="highlighter-rouge">html.parser</code> in this case). There are also parsers for other types of data such as XML.</p>
<p>After creating a new variable called <code class="highlighter-rouge">soup</code>, we can use Beautiful Soup’s library methods to parse out different elements. We created another variable, <code class="highlighter-rouge">table</code>. From there, we created a <code class="highlighter-rouge">rows</code> variable to find all the <code class="highlighter-rouge">tr</code> elements. We can now loop through this list:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">...</span>
<span class="n">soup</span> <span class="o">=</span> <span class="n">BeautifulSoup</span><span class="p">(</span><span class="n">html</span><span class="p">,</span> <span class="s">'html.parser'</span><span class="p">)</span>
<span class="n">table</span> <span class="o">=</span> <span class="n">soup</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="nb">id</span><span class="o">=</span><span class="s">'rankOrder'</span><span class="p">)</span>
<span class="n">rows</span> <span class="o">=</span> <span class="n">table</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="s">'tr'</span><span class="p">)</span>
<span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">rows</span><span class="p">:</span>
<span class="n">target_column</span> <span class="o">=</span> <span class="n">row</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="s">'td'</span><span class="p">,</span> <span class="s">'region'</span><span class="p">)</span>
</code></pre></div></div>
<p>As we loop through the rows, we can search for the column having an id of region. We created a new variable, <code class="highlighter-rouge">target_column</code>. Before attempting to use this variable, we should apply a guard to make sure it actually found the column. There may be instance where it doesn’t exist (i.e. the header row).</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">...</span>
<span class="n">soup</span> <span class="o">=</span> <span class="n">BeautifulSoup</span><span class="p">(</span><span class="n">html</span><span class="p">,</span> <span class="s">'html.parser'</span><span class="p">)</span>
<span class="n">table</span> <span class="o">=</span> <span class="n">soup</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="nb">id</span><span class="o">=</span><span class="s">'rankOrder'</span><span class="p">)</span>
<span class="n">rows</span> <span class="o">=</span> <span class="n">table</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="s">'tr'</span><span class="p">)</span>
<span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">rows</span><span class="p">:</span>
<span class="n">target_column</span> <span class="o">=</span> <span class="n">row</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="s">'td'</span><span class="p">,</span> <span class="s">'region'</span><span class="p">)</span>
<span class="k">if</span> <span class="n">target_column</span><span class="p">:</span>
<span class="k">pass</span> <span class="c"># find the anchor tag so we can parse the isocode</span>
</code></pre></div></div>
<p>In order to get the ISO Code, we will need to find the link, or <code class="highlighter-rouge">a</code> tag. We can parse the code from the <code class="highlighter-rouge">href</code> attribute:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>...
for row in rows:
target_column = row.find_all('td', 'region')
if target_column:
link = target_column[0].find('a')
href = link['href']
idx_html = href.index('.html')
country_code = href[idx_html-2:idx_html]
print('country_code: {}'.format(country_code))
</code></pre></div></div>
<p>Running the code above should display a list of ISO Codes for each country.</p>
<p>Using Beautiful Soup alongside Requests is a common approach to scraping html content.</p>
<h2 id="pulling-data-from-json-apis">Pulling Data from JSON API’s</h2>
<p>JSON API’s are probably the most common way of pulling data. JSON is much easier to parse and make use of downloaded data. Similar to the previous example, we will pull a list of countries and parse the ISO Codes. Except, this will be far less work. The <a href="http://www.worldbank.org/">World Bank</a> has an API for countries and we will use the following URL: <a href="http://api.worldbank.org/countries?format=json">http://api.worldbank.org/countries?format=json</a>.</p>
<p>Similar to the HTML example, we need to import <code class="highlighter-rouge">requests</code> and send a <code class="highlighter-rouge">GET</code> request to our target URL:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">requests</span>
<span class="n">url</span> <span class="o">=</span> <span class="s">'http://api.worldbank.org/countries?format=json'</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
</code></pre></div></div>
<p>From here, yes, we could use <code class="highlighter-rouge">response.text</code> and import the built-in <code class="highlighter-rouge">json</code> module to dump the data into a variable. However, <code class="highlighter-rouge">requests</code> has a built-in JSON decoder we can use simply by adding a method:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">requests</span>
<span class="n">url</span> <span class="o">=</span> <span class="s">'http://api.worldbank.org/countries?format=json'</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="c"># built-in JSON decoder</span>
<span class="n">country_data</span> <span class="o">=</span> <span class="n">response</span><span class="o">.</span><span class="n">json</span><span class="p">()</span>
</code></pre></div></div>
<p>If we inspect the data we’re getting back, it’s going to be an array. The first item in the array is meta data. The second item is another array containing the list of countries which is what we’re interested in:</p>
<p><img src="/assets/2019/02/json-api-data.png" alt="World Bank countries list data" title="World Bank countries list data" /></p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">requests</span>
<span class="n">url</span> <span class="o">=</span> <span class="s">'http://api.worldbank.org/countries?format=json'</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="c"># built-in JSON decoder</span>
<span class="n">country_data</span> <span class="o">=</span> <span class="n">response</span><span class="o">.</span><span class="n">json</span><span class="p">()</span>
<span class="n">countries</span> <span class="o">=</span> <span class="n">country_data</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
</code></pre></div></div>
<p>Above: since were only interested in the second item in the JSON array, we can set a new variable, <code class="highlighter-rouge">countries</code>. Since the Requests JSON decoder conveniently converted this to a list for us, we need to use an index of 1 to get the second item in the list.</p>
<p>From here, we can loop through the list of countries and find the ISO Codes.</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">requests</span>
<span class="n">url</span> <span class="o">=</span> <span class="s">'http://api.worldbank.org/countries?format=json'</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="c"># built-in JSON decoder</span>
<span class="n">country_data</span> <span class="o">=</span> <span class="n">response</span><span class="o">.</span><span class="n">json</span><span class="p">()</span>
<span class="n">countries</span> <span class="o">=</span> <span class="n">country_data</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="k">for</span> <span class="n">country</span> <span class="ow">in</span> <span class="n">countries</span><span class="p">:</span>
<span class="k">print</span><span class="p">(</span><span class="s">'country: {} {}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">country</span><span class="p">[</span><span class="s">'name'</span><span class="p">],</span> <span class="n">country</span><span class="p">[</span><span class="s">'iso2Code'</span><span class="p">]))</span>
</code></pre></div></div>
<h2 id="pulling-data-from-xml-apis">Pulling Data from XML API’s</h2>
<p>In this previous example we use the World Bank countries API to pull a list of countries. Recall that we had appended a query parameter (?format=json) to the end of the URL so the API would return our result back in JSON format. By default, World Bank will return data in XML format.</p>
<p>Getting our script setup will be the same as the previous example, except we won’t need any query params in our URL.</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">requests</span>
<span class="n">url</span> <span class="o">=</span> <span class="s">'http://api.worldbank.org/countries'</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
</code></pre></div></div>
<p>The next step will be to parse the XML to actually be able to use the data. While we can probably use Beautiful Soup again, Python has a built-in XML module that can help us with this. The ElementTree object has a <code class="highlighter-rouge">fromstring</code> method that will take our response text and convert it to an XML Element object.</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">import</span> <span class="nn">xml.etree.ElementTree</span> <span class="k">as</span> <span class="n">ET</span>
<span class="n">url</span> <span class="o">=</span> <span class="s">'http://api.worldbank.org/countries'</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="n">country_data</span> <span class="o">=</span> <span class="n">response</span><span class="o">.</span><span class="n">text</span>
<span class="n">countries</span> <span class="o">=</span> <span class="n">ET</span><span class="o">.</span><span class="n">fromstring</span><span class="p">(</span><span class="n">country_data</span><span class="p">)</span>
<span class="k">print</span><span class="p">(</span><span class="n">countries</span><span class="p">)</span> <span class="c"># XML Element Object</span>
</code></pre></div></div>
<p>Our XML element object contains a countries element in which all the individual country elements are nested:</p>
<p><img src="/assets/2019/02/countries-xml.png" alt="Countries XML data" title="Countries XML data" />
Notice also: there is a “wb” namespace for each element. In order to parse the data we want from this, we will have to pass a namespaces dictionary when parsing the elements:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">import</span> <span class="nn">xml.etree.ElementTree</span> <span class="k">as</span> <span class="n">ET</span>
<span class="n">url</span> <span class="o">=</span> <span class="s">'http://api.worldbank.org/countries'</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="n">country_data</span> <span class="o">=</span> <span class="n">response</span><span class="o">.</span><span class="n">text</span>
<span class="n">countries</span> <span class="o">=</span> <span class="n">ET</span><span class="o">.</span><span class="n">fromstring</span><span class="p">(</span><span class="n">country_data</span><span class="p">)</span>
<span class="k">print</span><span class="p">(</span><span class="n">countries</span><span class="p">)</span>
<span class="n">namespaces</span> <span class="o">=</span> <span class="p">{</span><span class="s">'wb'</span><span class="p">:</span> <span class="s">'http://www.worldbank.org'</span><span class="p">}</span>
<span class="k">for</span> <span class="n">country</span> <span class="ow">in</span> <span class="n">countries</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="s">'wb:country'</span><span class="p">,</span> <span class="n">namespaces</span><span class="p">):</span>
<span class="c"># each country will be an XML Element Object</span>
<span class="k">print</span><span class="p">(</span><span class="s">'country: {}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">country</span><span class="p">))</span>
</code></pre></div></div>
<p>Since each nested country element will also have a “wb” namespace, we will need to pass the namespaces dictionary into each method we use to find the country data we need:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">import</span> <span class="nn">xml.etree.ElementTree</span> <span class="k">as</span> <span class="n">ET</span>
<span class="n">url</span> <span class="o">=</span> <span class="s">'http://api.worldbank.org/countries'</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="n">country_data</span> <span class="o">=</span> <span class="n">response</span><span class="o">.</span><span class="n">text</span>
<span class="n">countries</span> <span class="o">=</span> <span class="n">ET</span><span class="o">.</span><span class="n">fromstring</span><span class="p">(</span><span class="n">country_data</span><span class="p">)</span>
<span class="k">print</span><span class="p">(</span><span class="n">countries</span><span class="p">)</span>
<span class="n">namespaces</span> <span class="o">=</span> <span class="p">{</span><span class="s">'wb'</span><span class="p">:</span> <span class="s">'http://www.worldbank.org'</span><span class="p">}</span>
<span class="k">for</span> <span class="n">country</span> <span class="ow">in</span> <span class="n">countries</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="s">'wb:country'</span><span class="p">,</span> <span class="n">namespaces</span><span class="p">):</span>
<span class="n">name</span> <span class="o">=</span> <span class="n">country</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="s">'wb:name'</span><span class="p">,</span> <span class="n">namespaces</span><span class="p">)</span><span class="o">.</span><span class="n">text</span>
<span class="n">code</span> <span class="o">=</span> <span class="n">country</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="s">'wb:iso2Code'</span><span class="p">,</span> <span class="n">namespaces</span><span class="p">)</span><span class="o">.</span><span class="n">text</span>
<span class="k">print</span><span class="p">(</span><span class="s">'country: {} - {}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">code</span><span class="p">))</span>
</code></pre></div></div>
<p>And now, we have successfully pulled data through HTML web pages, JSON API’s, and XML API’s. Python and Requests make this quick and easy for us. Along with some added help with libraries like Beautiful Soup.</p>Requests, being one of the most popular Python modules, is a highly regarded tool for sending HTTP requests. In this post, we will be pulling data from HTML pages, JSON API’s, and XML API’s.Intro to Using Python Type Hints2019-01-07T10:24:00-08:002019-01-07T10:24:00-08:00/python/2019/01/07/python-type-hinting<p>Type hinting was added to the Python standard library starting in version 3.5. Python, being a dynamically typed language, does not enforce data types. However, when argument types and return types for functions have type hints implemented, type hints can provide the following benefits:</p>
<h3 id="reviewing-old-code">Reviewing Old Code</h3>
<p>Most often, developers occasionally run into old code that hasn’t been touched in months, maybe longer. Type hints make this re-familiarizing process much easier.</p>
<h3 id="catch-bugs-that-are-often-overlooked">Catch Bugs that are Often Overlooked</h3>
<p>Many bugs you run into with dynamic languages are only evident at runtime. A common situation is where you expect a value to be set when the value is actually <code class="highlighter-rouge">None</code>.</p>
<h3 id="better-than-traditional-documentation">Better than Traditional Documentation</h3>
<p>With traditional documentation, particularly docstrings, you come across code that gets refactored but the docstrings weren’t updated. And, there is no enforcement in which a docstring has to be in lockstep with how the code is actually working.</p>
<h3 id="code-completion">Code Completion</h3>
<p>Traditionally for IDE’s, code completion has been a problem since the IDE has no way of knowing what accepted data types are appropriate when calling different functions.</p>
<p>Before we get into examples, it’s important to note that type checking is not done at run time because type hints are not enforced.</p>
<h2 id="static-type-checkers">Static Type Checkers</h2>
<p>In order to check our type hints, we need to run a static type checker. Remember, type hints are going to be ignored at runtime. IDE’s, like PyCharm, have type checkers built in. We can also use a Python package called <code class="highlighter-rouge">mypy</code>. <code class="highlighter-rouge">Mypy</code> is what we’ll be using in the examples below.</p>
<p>Before we jump in, it’s worth noting that the <code class="highlighter-rouge">mypy</code> documentation has a <a href="https://mypy.readthedocs.io/en/latest/cheat_sheet_py3.html">cheat sheet</a> handy.</p>
<p>Ok, let’s go ahead and install <code class="highlighter-rouge">mypy</code>:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>python <span class="nt">-m</span> pip install mypy
</code></pre></div></div>
<h3 id="type-hinting-variables">Type Hinting Variables</h3>
<p>Type hinting variables is entirely optional. Many devs will probably opt to focus mainly on functions arguments, return values, and classes. That being said, you are free to type hit local variables if you choose to. Here are some examples of type hinting local variables:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">a</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">b</span><span class="p">:</span> <span class="nb">float</span> <span class="o">=</span> <span class="mf">1.0</span>
<span class="n">c</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="bp">True</span>
<span class="n">d</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s">"test"</span>
<span class="n">e</span><span class="p">:</span> <span class="nb">bytes</span> <span class="o">=</span> <span class="n">b</span><span class="s">"test"</span>
</code></pre></div></div>
<p>Since we have <code class="highlighter-rouge">mypy</code> installed at this point, we can simply check the type hints with the following command:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>mypy my_script.py
</code></pre></div></div>
<p>If you see no output, that means all type hints are valid. You will only see output if there’s an error. For example, we can change one of our values to an incorrect data type:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">a</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mf">1.0</span> <span class="c"># will result in a mypy error</span>
<span class="n">b</span><span class="p">:</span> <span class="nb">float</span> <span class="o">=</span> <span class="mf">1.0</span>
<span class="n">c</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="bp">True</span>
<span class="n">d</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s">"test"</span>
<span class="n">e</span><span class="p">:</span> <span class="nb">bytes</span> <span class="o">=</span> <span class="n">b</span><span class="s">"test"</span>
</code></pre></div></div>
<p>Above: we changed <code class="highlighter-rouge">a</code> from 1 to 1.0, which forces it to be set as a float value. Running <code class="highlighter-rouge">mypy</code> again will show the following error:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>mypy my_script.py
error: Incompatible types <span class="k">in </span>assignment <span class="o">(</span>expression has <span class="nb">type</span> <span class="s2">"float"</span>, variable has <span class="nb">type</span> <span class="s2">"int"</span><span class="o">)</span>
</code></pre></div></div>
<p>Lists and other collection types can be type hinted as well. For this, we will have to import the <code class="highlighter-rouge">typing</code> module. In the example below, the type hint constrains <code class="highlighter-rouge">x</code> to being a list of integers:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">List</span>
<span class="n">x</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="nb">int</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">]</span>
</code></pre></div></div>
<p>All collection types work similarly. You can simply import the built-in type from the <code class="highlighter-rouge">typing</code> module (<code class="highlighter-rouge">Dict</code> for dictionaries, <code class="highlighter-rouge">Tuple</code> for tuples, and so on).</p>
<p>Because Python lists can hold items of different types, we can use the <code class="highlighter-rouge">Union</code> type to constrain the potential data types. Consider the following list:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">y</span> <span class="o">=</span> <span class="p">[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="s">"test"</span><span class="p">,</span> <span class="s">"fun"</span><span class="p">]</span>
</code></pre></div></div>
<p>Here, we have both integers and strings. If this list will only contain integers and strings, we can apply the following <code class="highlighter-rouge">Union</code> type hint:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">List</span><span class="p">,</span> <span class="n">Union</span>
<span class="n">y</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="nb">str</span><span class="p">]]</span> <span class="o">=</span> <span class="p">[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="s">"test"</span><span class="p">,</span> <span class="s">"fun"</span><span class="p">]</span>
</code></pre></div></div>
<h3 id="type-hinting-functions">Type Hinting Functions</h3>
<p>Providing type hinting for functions is what makes type hinting powerful. Consider the following function:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">get_batting_ave</span><span class="p">(</span><span class="n">hits</span><span class="p">,</span> <span class="n">at_bats</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">round</span><span class="p">(</span><span class="n">hits</span> <span class="o">/</span> <span class="n">at_bats</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
</code></pre></div></div>
<p>Above: to calculate a hitter’s batting average, we’re accepting 2 arguments: <code class="highlighter-rouge">hits</code> (an integer) and <code class="highlighter-rouge">at_bats</code> (an integer). In the return statement, we are dividing the 2 values, rounding the result to 3 decimal places. As a result, the function will be returning a float. To add type hints to this implementation, we can do the following:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">get_batting_ave</span><span class="p">(</span><span class="n">hits</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">at_bats</span><span class="p">:</span> <span class="nb">int</span><span class="p">)</span> <span class="o">-></span> <span class="nb">float</span><span class="p">:</span>
<span class="k">return</span> <span class="nb">round</span><span class="p">(</span><span class="n">hits</span> <span class="o">/</span> <span class="n">at_bats</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
</code></pre></div></div>
<p>Similar to the variables we were type hinting earlier, all we need to add is a colon followed by the type for each argument the function accepts. You’ll also notice we type hinted the return value, which was annotated by the arrow followed by the data type (<code class="highlighter-rouge">-> float</code>). At this point, running <code class="highlighter-rouge">mypy</code> will show that this functions type hints are valid.</p>
<p>There is a problem with this function, however. A potential edge case lies with players that have 0 at-bats. In this case the function would attempt to divide by 0. We can refactor the code with the following:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">Union</span>
<span class="k">def</span> <span class="nf">get_batting_ave</span><span class="p">(</span><span class="n">hits</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">at_bats</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="bp">None</span><span class="p">])</span> <span class="o">-></span> <span class="n">Union</span><span class="p">[</span><span class="nb">float</span><span class="p">,</span> <span class="bp">None</span><span class="p">]:</span>
<span class="k">if</span> <span class="n">at_bats</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span> <span class="ow">and</span> <span class="n">at_bats</span> <span class="o">></span> <span class="mi">0</span><span class="p">:</span>
<span class="k">return</span> <span class="nb">round</span><span class="p">(</span><span class="n">hits</span> <span class="o">/</span> <span class="n">at_bats</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">None</span>
</code></pre></div></div>
<p>This is much more flexible. The <code class="highlighter-rouge">Union</code> type, as mentioned earlier, means either type in the square brackets is acceptable. Our batting average can now handle the following cases:</p>
<ul>
<li>a value of 0 at-bats</li>
<li>a value of <code class="highlighter-rouge">None</code> for at-bats</li>
<li><code class="highlighter-rouge">None</code> type will be returned if at-bats is None or 0</li>
</ul>
<p>Since within both or our <code class="highlighter-rouge">Union</code> types, <code class="highlighter-rouge">None</code> is one of the options, we can actually simplify this further using the <code class="highlighter-rouge">Optional</code> type. In this case, <code class="highlighter-rouge">Optional[float]</code> is the same as <code class="highlighter-rouge">Union[float, None]</code>.</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">Optional</span>
<span class="k">def</span> <span class="nf">get_batting_ave</span><span class="p">(</span><span class="n">hits</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">at_bats</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">int</span><span class="p">])</span> <span class="o">-></span> <span class="n">Optional</span><span class="p">[</span><span class="nb">float</span><span class="p">]:</span>
<span class="k">if</span> <span class="n">at_bats</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span> <span class="ow">and</span> <span class="n">at_bats</span> <span class="o">></span> <span class="mi">0</span><span class="p">:</span>
<span class="k">return</span> <span class="nb">round</span><span class="p">(</span><span class="n">hits</span> <span class="o">/</span> <span class="n">at_bats</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">None</span>
</code></pre></div></div>
<p>As before, we should be able to run <code class="highlighter-rouge">mypy</code> at this point and the results should be error-free.</p>
<p>One nice feature regarding <code class="highlighter-rouge">mypy</code> is that it will only evaluate code in which type hints are being used. So, if you’re starting to apply type hints within a large code base, it won’t complain about functions and classes that don’t have any type hints applied yet.</p>
<p>Consider the following:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">Optional</span>
<span class="k">def</span> <span class="nf">get_batting_ave</span><span class="p">(</span><span class="n">hits</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">at_bats</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">int</span><span class="p">])</span> <span class="o">-></span> <span class="n">Optional</span><span class="p">[</span><span class="nb">float</span><span class="p">]:</span>
<span class="k">if</span> <span class="n">at_bats</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span> <span class="ow">and</span> <span class="n">at_bats</span> <span class="o">></span> <span class="mi">0</span><span class="p">:</span>
<span class="k">return</span> <span class="nb">round</span><span class="p">(</span><span class="n">hits</span> <span class="o">/</span> <span class="n">at_bats</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">None</span>
<span class="k">def</span> <span class="nf">get_slugging_pct</span><span class="p">(</span><span class="n">singles</span><span class="p">,</span> <span class="n">doubles</span><span class="p">,</span> <span class="n">triples</span><span class="p">,</span> <span class="n">home_runs</span><span class="p">,</span> <span class="n">at_bats</span><span class="p">):</span>
<span class="n">numerator</span> <span class="o">=</span> <span class="p">(</span><span class="n">singles</span> <span class="o">+</span> <span class="p">(</span><span class="mi">2</span> <span class="o">*</span> <span class="n">doubles</span><span class="p">)</span> <span class="o">+</span> <span class="p">(</span><span class="mi">3</span> <span class="o">*</span> <span class="n">triples</span><span class="p">)</span> <span class="o">+</span> <span class="p">(</span><span class="mi">4</span> <span class="o">*</span> <span class="n">home_runs</span><span class="p">))</span>
<span class="k">return</span> <span class="nb">round</span><span class="p">(</span><span class="n">numerator</span> <span class="o">/</span> <span class="n">at_bats</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
</code></pre></div></div>
<p>Now, we’re going to add a <code class="highlighter-rouge">get_stats</code> function that will call both of the functions above. In this walk-through, we’re going to highlight a situation where type hints can help prevent bugs that can easily go unnoticed.</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">Optional</span><span class="p">,</span> <span class="n">Tuple</span>
<span class="k">def</span> <span class="nf">get_batting_ave</span><span class="p">(</span><span class="n">hits</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">at_bats</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">int</span><span class="p">])</span> <span class="o">-></span> <span class="n">Optional</span><span class="p">[</span><span class="nb">float</span><span class="p">]:</span>
<span class="k">if</span> <span class="n">at_bats</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span> <span class="ow">and</span> <span class="n">at_bats</span> <span class="o">></span> <span class="mi">0</span><span class="p">:</span>
<span class="k">return</span> <span class="nb">round</span><span class="p">(</span><span class="n">hits</span> <span class="o">/</span> <span class="n">at_bats</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">None</span>
<span class="k">def</span> <span class="nf">get_slugging_pct</span><span class="p">(</span><span class="n">singles</span><span class="p">,</span> <span class="n">doubles</span><span class="p">,</span> <span class="n">triples</span><span class="p">,</span> <span class="n">home_runs</span><span class="p">,</span> <span class="n">at_bats</span><span class="p">):</span>
<span class="n">numerator</span> <span class="o">=</span> <span class="p">(</span><span class="n">singles</span> <span class="o">+</span> <span class="p">(</span><span class="mi">2</span> <span class="o">*</span> <span class="n">doubles</span><span class="p">)</span> <span class="o">+</span> <span class="p">(</span><span class="mi">3</span> <span class="o">*</span> <span class="n">triples</span><span class="p">)</span> <span class="o">+</span> <span class="p">(</span><span class="mi">4</span> <span class="o">*</span> <span class="n">home_runs</span><span class="p">))</span>
<span class="k">return</span> <span class="nb">round</span><span class="p">(</span><span class="n">numerator</span> <span class="o">/</span> <span class="n">at_bats</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">get_stats</span><span class="p">(</span><span class="n">singles</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">doubles</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">triples</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">home_runs</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">at_bats</span><span class="p">:</span> <span class="nb">int</span><span class="p">)</span> <span class="o">-></span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">float</span><span class="p">,</span> <span class="nb">float</span><span class="p">]:</span>
<span class="n">hits</span> <span class="o">=</span> <span class="n">singles</span> <span class="o">+</span> <span class="n">doubles</span> <span class="o">+</span> <span class="n">triples</span> <span class="o">+</span> <span class="n">home_runs</span>
<span class="n">batting_ave</span> <span class="o">=</span> <span class="n">get_batting_ave</span><span class="p">(</span><span class="n">hits</span><span class="p">,</span> <span class="n">at_bats</span><span class="p">)</span>
<span class="n">slugging_pct</span> <span class="o">=</span> <span class="n">get_slugging_pct</span><span class="p">(</span><span class="n">singles</span><span class="p">,</span> <span class="n">doubles</span><span class="p">,</span> <span class="n">triples</span><span class="p">,</span> <span class="n">home_runs</span><span class="p">,</span> <span class="n">at_bats</span><span class="p">)</span>
<span class="k">return</span> <span class="n">batting_ave</span><span class="p">,</span> <span class="n">slugging_pct</span>
</code></pre></div></div>
<p>In the <code class="highlighter-rouge">get_stats</code> function, we’re returning a tuple containing the batting average and the slugging percentage. If we run <code class="highlighter-rouge">mypy</code> at this point, we will get the following error:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>mypy my_script.py
error: Incompatible return value type (got "Tuple[Optional[float], Any]", expected "Tuple[float, float]")
</code></pre></div></div>
<p>Note that we’re calling the <code class="highlighter-rouge">get_batting_ave</code> function (from within <code class="highlighter-rouge">get_stats</code>), which is a function we type hinted earlier. Since <code class="highlighter-rouge">get_batting_ave</code> can potentially return <code class="highlighter-rouge">None</code>, we need to handle that case in <code class="highlighter-rouge">get_stats</code> return value. In order to get <code class="highlighter-rouge">mypy</code> to accept it, we need to modify the <code class="highlighter-rouge">get_stats</code> return type to <code class="highlighter-rouge">Tuple[Optional[float], float]</code> from <code class="highlighter-rouge">Tuple[float, float]</code>.</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">...</span>
<span class="k">def</span> <span class="nf">get_stats</span><span class="p">(</span><span class="n">singles</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">doubles</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">triples</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">home_runs</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">at_bats</span><span class="p">:</span> <span class="nb">int</span><span class="p">)</span> <span class="o">-></span> <span class="n">Tuple</span><span class="p">[</span><span class="n">Optional</span><span class="p">[</span><span class="nb">float</span><span class="p">],</span> <span class="nb">float</span><span class="p">]:</span>
<span class="n">hits</span> <span class="o">=</span> <span class="n">singles</span> <span class="o">+</span> <span class="n">doubles</span> <span class="o">+</span> <span class="n">triples</span> <span class="o">+</span> <span class="n">home_runs</span>
<span class="n">batting_ave</span> <span class="o">=</span> <span class="n">get_batting_ave</span><span class="p">(</span><span class="n">hits</span><span class="p">,</span> <span class="n">at_bats</span><span class="p">)</span>
<span class="n">slugging_pct</span> <span class="o">=</span> <span class="n">get_slugging_pct</span><span class="p">(</span><span class="n">singles</span><span class="p">,</span> <span class="n">doubles</span><span class="p">,</span> <span class="n">triples</span><span class="p">,</span> <span class="n">home_runs</span><span class="p">,</span> <span class="n">at_bats</span><span class="p">)</span>
<span class="k">return</span> <span class="n">batting_ave</span><span class="p">,</span> <span class="n">slugging_pct</span>
</code></pre></div></div>
<p>Re-running <code class="highlighter-rouge">mypy</code> now should result in no errors.</p>
<h3 id="any-type">“Any” Type</h3>
<p>The <code class="highlighter-rouge">Any</code> type is the most flexible type. It is completely unconstrained because it is compatible with every type. The problem with using <code class="highlighter-rouge">Any</code> is that you lose some of the benefits of type hinting. So, this should be used sparingly. For example, the code from the previous example would have worked if we switched the return type to <code class="highlighter-rouge">Tuple[Any, float]</code>.</p>
<h3 id="setting-a-return-type-outside-a-function">Setting a Return Type Outside a Function</h3>
<p>We can set return types outside of functions. In our <code class="highlighter-rouge">get_stats</code> function, let’s convert the return type to a named tuple and setup a new class for it:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">Optional</span><span class="p">,</span> <span class="n">NamedTuple</span>
<span class="k">class</span> <span class="nc">StatsResult</span><span class="p">(</span><span class="n">NamedTuple</span><span class="p">):</span>
<span class="n">batting_ave</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">float</span><span class="p">]</span>
<span class="n">slugging_pct</span><span class="p">:</span> <span class="nb">float</span>
</code></pre></div></div>
<p>Notice that we imported <code class="highlighter-rouge">NamedTuple</code> from <code class="highlighter-rouge">typing</code>, and not <code class="highlighter-rouge">namedtuple</code> from <code class="highlighter-rouge">collections</code>. The remaining step is to apply this to our <code class="highlighter-rouge">get_stats</code> function:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">Optional</span><span class="p">,</span> <span class="n">NamedTuple</span>
<span class="o">...</span>
<span class="k">class</span> <span class="nc">StatsResult</span><span class="p">(</span><span class="n">NamedTuple</span><span class="p">):</span>
<span class="n">batting_ave</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">float</span><span class="p">]</span>
<span class="n">slugging_pct</span><span class="p">:</span> <span class="nb">float</span>
<span class="k">def</span> <span class="nf">get_stats</span><span class="p">(</span><span class="n">singles</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">doubles</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">triples</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">home_runs</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">at_bats</span><span class="p">:</span> <span class="nb">int</span><span class="p">)</span> <span class="o">-></span> <span class="n">StatsResult</span><span class="p">:</span>
<span class="n">hits</span> <span class="o">=</span> <span class="n">singles</span> <span class="o">+</span> <span class="n">doubles</span> <span class="o">+</span> <span class="n">triples</span> <span class="o">+</span> <span class="n">home_runs</span>
<span class="n">batting_ave</span> <span class="o">=</span> <span class="n">get_batting_ave</span><span class="p">(</span><span class="n">hits</span><span class="p">,</span> <span class="n">at_bats</span><span class="p">)</span>
<span class="n">slugging_pct</span> <span class="o">=</span> <span class="n">get_slugging_pct</span><span class="p">(</span><span class="n">singles</span><span class="p">,</span> <span class="n">doubles</span><span class="p">,</span> <span class="n">triples</span><span class="p">,</span> <span class="n">home_runs</span><span class="p">,</span> <span class="n">at_bats</span><span class="p">)</span>
<span class="k">return</span> <span class="n">StatsResult</span><span class="p">(</span><span class="n">batting_ave</span><span class="p">,</span> <span class="n">slugging_pct</span><span class="p">)</span>
</code></pre></div></div>
<p>Adding extra classes for the purposes of type hints can be beneficial in some cases. The one drawback, however, is they can add more noise and clutter to your code.</p>Type hinting was added to the Python standard library starting in version 3.5. Python, being a dynamically typed language, does not enforce data types. However, when argument types and return types for functions have type hints implemented, type hints can provide the following benefits:Breakpoints in Chrome Developer Tools2018-12-22T10:24:00-08:002018-12-22T10:24:00-08:00/chrome/developer/tools/2018/12/22/breakpoints-chrome-developer-tools<iframe width="740" height="417" src="https://www.youtube.com/embed/AZIiJYSaio0" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen=""></iframe>
<h3 id="using-breakpoints-in-chrome-developer-tools">Using Breakpoints in Chrome Developer Tools</h3>
<h4 id="included">Included:</h4>
<ul>
<li>Conditional breakpoints</li>
<li>Exception breakpoints</li>
<li>Scope pane</li>
<li>Watch expressions</li>
</ul>JavaScript Console and Debugging in Chrome Developer Tools2018-12-22T10:24:00-08:002018-12-22T10:24:00-08:00/chrome/developer/tools/2018/12/22/JavaScript-Console<iframe width="740" height="417" src="https://www.youtube.com/embed/muXE2Wa7y8Q" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen=""></iframe>
<p>How to use the JavaScript Console in Chrome Developer Tools. Included are executing JavaScript, writing logs, and debugging.</p>Box Model Widget in Chrome Developer Tools2018-12-13T10:24:00-08:002018-12-13T10:24:00-08:00/chrome/developer/tools/2018/12/13/box-model-chrome-developer-tools<iframe width="740" height="417" src="https://www.youtube.com/embed/4Q5llID99Zc" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen=""></iframe>
<p>This video demo will cover the box model widget in Chrome Developer Tools. I will, very briefly, go over the box model then how to use the widget to edit CSS.</p>Picking a Color from a Webpage with Chrome Developer Tools2018-12-11T10:24:00-08:002018-12-11T10:24:00-08:00/chrome/developer/tools/2018/12/11/picking-color-from-webpage<iframe width="740" height="417" src="https://www.youtube.com/embed/Uc4NWHdX114" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen=""></iframe>
<p>Picking a color from a webpage is simple. There’s no need to install a Chrome Extension. Simply, open the Elements tab in Chrome Developer Tools.</p>
<h3 id="open-developer-tools">Open Developer Tools</h3>
<ul>
<li><code class="highlighter-rouge">Command+Option+I (Mac) or Control+Shift+I (Windows)</code></li>
</ul>
<p><strong>In the styles tab, select any CSS selector that has a color property</strong></p>
<ul>
<li>filter using “color” as search term</li>
</ul>
<p><strong>Click on the colored box</strong></p>
<ul>
<li>make sure eye dropper is enabled</li>
</ul>
<p><strong>Return to webpage</strong></p>
<ul>
<li>click on target to find color</li>
</ul>
<p><strong>Return to Styles section in Developer Tools</strong></p>
<ul>
<li>color should appear with the hex value</li>
<li>clicking arrows will switch between hex, RGB values, and HSL (Hue Saturation Luminosity)</li>
</ul>