Retry with Tenacity

In the world of software development, it's common to encounter situations where a piece of code must be retried multiple times due to temporary failures or issues beyond our control. This can include network errors, server timeouts, or API rate limits. To address these challenges, Python developers can use the powerful and flexible Tenacity library.

Tenacity is a widely-used and well-maintained library that simplifies the process of adding retry logic to Python applications. It provides an easy-to-use API, allowing developers to implement retries with minimal effort and maximum customization. In this book, we will explore the many features of Tenacity, from basic usage to advanced strategies, and demonstrate how to use them effectively in various scenarios.

Installing Tenacity

Before diving into the features of Tenacity, we need to install the library. You can install Tenacity using pip, the Python package manager, with the following command:

$ pip install tenacity

Basic Usage of Tenacity

Simple Retries

To start using Tenacity, import the retry decorator from the tenacity module and apply it to the function you want to retry. For example:

from tenacity import retry

@retry
def fetch_data():
    # Code to fetch data from a remote server

By default, Tenacity will retry the function indefinitely with a 1-second wait between attempts until it succeeds.

Customizing Retry Logic

Tenacity allows you to customize the retry behavior using various options. For example, you can set the number of retries, the wait time between attempts, and the conditions under which a retry should occur. Here's an example of a function that retries up to 5 times with a 2-second wait between attempts:

from tenacity import retry, stop_after_attempt, wait_fixed

@retry(stop=stop_after_attempt(5), wait=wait_fixed(2))
def fetch_data():
    # Code to fetch data from a remote server

Handling Exceptions with Tenacity

When working with retries, it's crucial to handle exceptions effectively. In this chapter, I will explore Tenacity's exception handling capabilities, including how to reraise exceptions and create custom exception handlers.

Reraise Exceptions

By default, Tenacity suppresses exceptions that occur during retries. This behavior can be problematic if you need to know when an exception has occurred, even after all retries have been exhausted. To configure Tenacity to reraise the last exception after all retries have been attempted, you can use the reraise=True option:

from tenacity import retry, stop_after_attempt

@retry(stop=stop_after_attempt(3), reraise=True)
def fetch_data():
    # Code to fetch data from a remote server

Now, if the fetch_data function continues to fail after three attempts, Tenacity will reraise the last exception, allowing you to handle it appropriately in your application.

Custom Exception Handling

Sometimes, you may want to perform specific actions when an exception occurs, such as logging the error or executing a fallback function. Tenacity provides a callback mechanism called retry_error_callback that allows you to define custom exception handling logic.

Here's an example of a custom error callback that logs the exception and returns a default value:

from tenacity import retry, stop_after_attempt

def error_callback(retry_state):
    print(f"An error occurred: {retry_state.outcome.exception()}")
    return "default_value"

@retry(stop=stop_after_attempt(3), retry_error_callback=error_callback)
def fetch_data():
    # Code to fetch data from a remote server

In this example, if the fetch_data function continues to fail after three attempts, the error_callback function will be called. This function logs the exception and returns a default value, which will be the result of the fetch_data function.

Keep in mind that when using retry_error_callback, Tenacity will not reraise the exception. If you want to both handle the exception and have it reraised, you can do so within the error callback:

def error_callback(retry_state):
    print(f"An error occurred: {retry_state.outcome.exception()}")
    raise retry_state.outcome.exception()

@retry(stop=stop_after_attempt(3), retry_error_callback=error_callback)
def fetch_data():
    # Code to fetch data from a remote server

References

https://tenacity.readthedocs.io/en/latest/
https://github.com/jd/tenacity