Automate FB video download with Python

If you are a regular user of Facebook you might often come across videos which you wish to download. Or you might wish to share the video with your friends outside Facebook directly rather than sending them the Facebook video URL. Well as a Pythoneer you can easily convert your wish into a real world utility web app to automate facebook video download with Python. Yes, we won’t just create a tool to download a facebook video from its URL but create a fully functional but minimalistic web app to allow yourself or anyone to easily download a video by just entering its URL. Let’s see how!

A standalone module to download facebook videos from given URL

Just a few days ago on r/python an entrepreneuring Pythoneer shared their module to automatically download a Facebook video from a given URL. So we will use this handy module and call it from our web app when a user asks to download the video for a given Facebook URL. The repository for this module can be found here.

Creating the web app to automate


The HTML for this super simple web app is literally 5 lines:

<h1>Download videos straight up from Facebook</h1>
<input type="text" size="64" id="fbUrl"
    placeholder="" />
<button onclick="dl()">Download!</button>
<p id="status"></p>

We add the HTML mentioned above inside the body tag. Basically we have created a heading and then a text input field for the user to enter the video URL to download, and a button to call a function when the user clicks it.


On the front-end of the app basically there are two actions to perform. One, when the user enters a URL to download the video from, we send the video URL to server to start its download. Second, we keep polling the server periodically asking about the status of the downloaded video. If the video has been downloaded we change our display on the front end showing the user the download button to watch and downlaod the video. If the video fails to download due to some reason, eg the URL being invalid, then the error infomormation is displayed instead.

    const SERVER_URL = "/fb/api";
    const statusEle = document.getElementById("status");
    let handle;

    function dl() {
        let url = document.getElementById("fbUrl").value;
        console.log(">>URL: " + url);
        if (!url) {
            alert("Please enter a URL!");
        fetch(SERVER_URL + "/dl",
            method: "POST",
            headers: {
            'Content-Type': 'application/json',
            body: JSON.stringify({url: url})
        .then(resp => resp.json())
        .then(data => {
            if (data.success) {
                statusEle.textContent = "Starting...";
            } else {
                statusEle.textContent = "Oops. Something went wrong :(";
        .catch(err => {
            statusEle.textContent = "Unable to talk to server :(";

        handle = setInterval(poll, 1000, url);

    function poll(url) {
        fetch(SERVER_URL + "/result",
            method: "POST",
            headers: {
            'Content-Type': 'application/json',
            body: JSON.stringify({url: url})
        }).then(r => r.json())
        .then(data => {
            if (data.failed) {
                statusEle.textContent = "Failed to download the video -- is the URL valid?";
            if (!data.done)
            statusEle.innerHTML = "Downloaded! Here's the <a href='" + data.url + "'>video</a>."

Server-side: API service

On the server side the API is pretty small. We need two endpoints for the two kinds of calls that the front end is making. The first one, which begins download, simply takes in the user-given URL and adds it to a queue for the worker to process.

The second endpoint responsible for providing the download status of a URL does just that by querying the results collection. The results collection is updated by the worker as the URL is processed for download.

Apart from this the worker picks the unprocessed tasks from the queue, uses the facebook video downloader module mentioned above to download the video, and then updates the result collection and marks the URL as processed.

And that’s all you need to get this web app up and running! Do let me know if you have any questions or feedback!

If you liked my work you can show your support by buying me a few cups of coffee!

Python Twitter Bot: Automatically Post Tweets

In this article I’m gonna explain how to create a Python Twitter bot which automates your browser and posts tweets automatically using the Selenium tool. This method doesn’t rely on any Twitter or third party API and rather posts tweets on your behalf directly using browser automation from scratch.

What will our Python Twitter bot do?

We are going to create a simple bot whose job is to pick a text block from a list of texts that we provide and then post a tweet containing a new text once everyday. The bot will launch the browser by itself, navigate to the Twitter homepage and then post the tweet from the web interface.

Tools to be Used

For creating this simple bot we only need a web browser, a webdriver to control the browser, a library to run that webdriver, and the mighty Python to glue it all together. If you are interested you might want to look into the Python Automation Tools article to learn about some of the most commonly used tools in Python automation.

Kicking off with creating our Python Twitter bot!

The first thing you’ll need to create a Python Twitter bot of course is to ensure that you have a Twitter account. We are going to need the username and password details of that account and we will assume the values to be EMAIL and PASSWORD respectively in the rest of this article.

Logging into our Twitter account

The first task we want to do with our Python Twitter bot is to log in to our account. You have to keep in mind that the web browser session that we will spawn using our Python program will run in an isolated mode, kind of like the incognito mode where each session is independent and the cookies or other information is not stored by the browser. This means every time you run the program you will have to log in again.

Let’s start with installing the two needed libraries: Selenium and PyAutoGUI

pip install selenium pyautogui

We are going to use Firefox and its webdriver known as geckodriver for this tutorial. You are free to use Chrome or any other browser and download the corresponding webdriver (eg Chromedriver). You can download the appropriate geckodriver binary for your Firefox version and operating system from this page.

After downloading and extracting the geckodriver binary put it in the same folder where you are going to keep your code for this Python twitter bot. Now let’s get to coding. I’ll share the entire code with you and then explain how I went about writing it.

import os
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import pyautogui

def find_by_text(driver, text):
    return driver.find_element_by_xpath("//*[contains(text(), '%s')]" % text)

def find_form_ele(driver, text):
    uele = driver.find_element_by_xpath("//*[contains(text(), '%s')]" % text)
    uele = uele.find_element_by_xpath("..")
    return uele.find_element_by_xpath("./following-sibling::div/div/input")

def run():
    usr_pwd = os.environ.get("PWD") or input("password: ")
    cur_dir = os.getcwd()
    geckopath = os.path.join(cur_dir, "geckodriver")
    driver = webdriver.Firefox(executable_path=geckopath)
    uele = find_form_ele(driver, "Phone, email, or username")
    uele = find_form_ele(driver, "Password")
    fele = driver.find_element_by_css_selector(".notranslate > div:nth-child(1) > div:nth-child(1)")
    fele = fele.find_element_by_xpath("./div")
“Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live”
-- John Woods 
    """)['tab'] * 6)'enter')

if __name__ == "__main__":

The first thing we do is import the webdriver package from selenium library. The entry point of code is in the run function. First we obtain the password from either the environment variable or ask user for the input. Then we initialize the driver by creating a Firefox webdriver object: driver = webdriver.Firefox(executable_path=geckopath)

The first thing we want our browser automater to do is visit driver.get("")

Then we look for the element in browser’s DOM containing the text “Phone, email, or username” as the corresponding element is situated adjacent to the username field which we need to fill. The username input field is a sibling of the parent of the aforementioned field and we have created a function which will find us the input field based on the adjacent element. After locating the element we automate sending key presses to the browser and enter the email: uele.send_keys(usr_pwd)

Similarly we locate the password field and fill the password. Following that we submit the associated log in form. We also put in random sleeps at a few places. Once the log in form is filled and submitted we’ll be redirected to the Twitter home page where we can post our tweet. We find the relevant element responsible for accepting the user input to post a tweet using these lines:

fele = driver.find_element_by_css_selector(".notranslate > div:nth-child(1) > div:nth-child(1)")
fele = fele.find_element_by_xpath("./div")

After locating the right element we click on it to get it on focus and be able to write in it. Here you will note that Twitter doesn’t make it easy to write in the form. Reason being that the box isn’t an input field but some complex JS element. Therefore we use another library called PyAutoGUI to enter our tweet as Selenium will otherwise complain that the input field isn’t visible. We type in the tweet using this statement: pyautogui.write(""" “Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live” -- John Woods """)

After that we locate the “Submit” button by navigating across the menu buttons by pressing TAB button six times. We follow it by pressing the ENTER button. And that’s it, our tweet is sent!

Tornado Use Case in Low Memory Environments

Tornado is a fantastic Python web framework and asynchronous networking library which sadly doesn’t get enough attention in the Python world. However I’ve found the framework to be an amazing choice for some of my requirements and I’m going to share some of my experiences with you. On a side note if you’re new to Python or want to master its fundamentals you may enjoy the article on best books to learn Python automation.

Why Use Tornado

I’m a developer who works on his own personal projects in free time. I often have multiple small ideas and I want to try them out and put them in front of the world as a functional product. However I quickly found out after I had a few projects running that the VPS I was renting wasn’t going to be able to handle all my projects. I had a couple of projects each running in their Docker container with a Django instance, a Celery worker, a Celery-beats daemon and the database servers. All this was quite a memory hog for me and I just wanted to be able to run more number of projects in the limited resources of my VPS. This is when Tornado started looking like a possible option for me as one of my apps was also extensively using websockets and Tornado is quite memory-efficient when it comes to keeping many long-running connections alive.

Celery used to be one of my favourite tools in web development as I have a penchant for ending up with many periodic tasks connected to the models involved in the web app. However during my analysis of memory-hogging processed I found that Celery was one of the major culprits. I was also intrigued by Tornado’s delayed callbacks and periodic task features. Finally I decided to give Tornado a go and also replace Celery with Tornado’s delayed callbacks and my own mini library for handling scheduled tasks with support for persistency. Not only this lets you avoid running Celery workers in your stack, it lets you have a complete task scheduling service running in the same process as your web app if you so desire, which can be quite valuable when your app is running in a context where memory is scarce. For the database I decided to switch from PostgreSQL to MongoDB because I felt it better suited for my new-found approach of extremely rapid prototyping and releasing updates. But that’s a discussion for another day. However this approach can work equally well (or better) in the case of an RDBMS as well.

A few nice things about Tornado

One great thing I liked when switching to Tornado was that it immediately made me feel more in control of the somewhat lower levels of how an HTTP server behaves. Don’t get me wrong, you can control these behaviours in frameworks like Django, Flask and Starlette as well, but generally these functionalities are somewhat hidden away as middlewares or there’s no easy and direct way to change the lower level behaviours of a set of routes. To take an example, if you had to allow CORS for a set of your endpoints in a language like Django or Flask, perhaps the most common approach is to install a corresponding package for the framework and add it to your application. So in Django you would install the django-cors-headers package and then add it to your installed apps, add it to your middlewares and then change the CORS policy in your setting. It works and solves your problem, but it hides away the inner details of its working and also makes your stack a bit heavier. However the approach that Tornado takes is a bit different and in practice helps you be more in control of the underlying network behaviour. So what do you do when you want to allow CORS for some of your endpoints in Tornado?

You extend the RequestHandler class and override its set_default_headers method to set the headers required for allowing CORS for the given request. It turns out that allowing CORS requires only modifying the value of one key in your HTTP header and can be done in just one line of Python. So no need to install a package and change your app configurations in different places to enable CORS, and you also get a much better understanding of what’s happening behind the scene and can use this lower level of control to make your app more efficient or simpler.

class MyHttpRequestHandler(tornado.web.RequestHandler):
    def set_default_headers(self):
        self.set_header("Content-Type", 'text/html')
        # allow CORS requests
        self.set_header("Access-Control-Allow-Origin", "*")
        self.set_header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept")
        self.set_header('Access-Control-Allow-Methods', 'POST, GET, OPTIONS')

That’s it for this blog post, Stay tuned for more tidbits about the world of Python! In the meanwhile if you want to read the article on Python automation tools!

Learn Python Automation: The Best Books

Python is a great tool to automate stuff because of its high productivity and the large ecosystem of excellent libraries and tools which you can easily use in your own programs. Python is not a difficult language to get started with and you can go quite far without spending too much time learning the ins and outs of the language. However there comes a point in your journey as a developer where you might be best off to learn Python automation in detail with all its nuances in more detail in order to write more organized and efficient code as well
as to understand the code written by others or the open source community. Let’s discuss some of the best books
that can help you become better at automation with Python.

Best Books to Learn Python Automation

Learning Python by Mark Lutz

This is one of the best books out there when it comes to teaching the fundamentals of Python in depth. The book is
very comprehensive and deals with all the topics a person unaware of the details of working with Python might need.
Learning the concepts mentioned in this book thoroughly should really bolster your Python understanding and be
completely sufficient to understand most of the Python code out in the wild and to write idiomatic and efficient
Python programs.

Programming Python by Mark Lutz

If the part 1 of Programming Python teaches you about the fundamentals of the Python language, part 2 is about applying the concepts of Python in real world. The book is divided into different domains, encompassing the most common use cases of Python – in system scripting, networking, web and so on.

Automate the Boring Stuff with Python by Al Sweigart

While the books mentioned above excellent at teaching you the ins and outs of Python from a theoretical perspective,
this book will teach you how to apply your expertise in Python on practical projects in a completely hands-on way.
It will also get you familiar with some of the most commonly used third party libraries and tools in Python ecosystem as well as many other practical details involved in building a real world project.

Test Driven Development with Python

While the other books mentioned till now talk about the specific concepts or features in Python and writing code for specific use cases, this book deals with the broader picture of software development, and specifically a particular approach to software development called test-driven development. It not only does a great job of instilling the mindset of test-driven development which can be a great ability to have as a developer, but it will also teach you the workflow involved in writing software in the real world including in a team setting. The book teaches you some essential practical skills such as using git and when to commit or not commit your code and the likes, deploying your code to the cloud and so on.

Head First Python

If you’re looking for a more fun to read and hands-on way of learning the basics of Python, then this book will probably be great for you. It goes over the basics of Python gradually in an extremely hands-on and tutorial-like manner and keeps the topics short and simple. If you want to avoid getting bogged down by dry text and want a lighter approach to learning the basics of Python then this is the right book for you!

Next Steps to Learn Python Automation

Once you have read some of these books or the parts of them you were most interested in, it’s natural for many developers to feel confused what their next step should be. This is a natural feeling when one transitions from one level of mastery of a topic to another. However in this case, I believe the solution is just to work on real world projects that interest you. It’s fine even if a similar project already exists or the idea doesn’t seem like the most awesome one in the world. The important thing is to apply your skills on a project that you personally find interesting and are excited by. You might also want to check out the article on the most popular Python automation tools to get an idea of how Python is being used to solve real world use cases.

Another opportunity for you to step up could be start working in a team rather than on your personal projects. The open source world is teeming with opportunities where you could contribute and learn to work as a member of team. And there is enough space for beginner or intermediate developers as well to contribute meaningfully in such projects. If you get stuck then communities like Stack Overflow and r/python are your friends. All the best in your journey to learn Python automation!

Python Automation Tools: Tools of the Trade

Python is an extremely powerful tool that allows you to quickly write scripts to perform the desired tasks and bring automation into different domains of life. Python has a large ecosystem of libraries and tools and sometimes it can be overwhelming to find and choose the right tools for the job. So let’s discuss some of the most commonly used and loved Python tools used in the field of automation. In case you are new to Python or want to master the fundamentals of the language you may want to check out the article on best books to learn automation with Python.

Python Automation Tools for the Web


requests is an amazing library which massively simplifies sending HTTP requests to any HTTP server via Python. Using requests library alone can get you very far and help you automate
many things. For example if you had a web app hosted on the free tier of Heroku where the app goes to sleep after half an hour of idle period, you could send a request every half an hour to stop the web app from going to sleep. requests library is very commonly used for accessing the REST API provided by other services to make use of their offerings. For example you can use requests to automatically upload files to Dropbox, send messages in a Discord channel, upload files to Google Drive and so on.


Generally requests and BeautifulSoup are used together. Whereas requests is used to send HTTP requests to servers and receive the response, BeautifulSoup allows you to extract the needed information from that response. BeautifulSoup is an HTML parser which can analyze an HTML document and lets you execute ad-hoc queries on it to get the desired information, such as fetching all the texts within the paragraph tags of class ‘item’, fetching all the links or image urls in a page and so on. BeautifulSoup lets you use different ways of querying for the data in HTML document, eg. using CSS selectors, HTML attributes such as id, class or tag, and so on.


While requests fetches the static response of an HTTP request, it’s not capable of executing the JavaScript code present in the response. In the modern web JavaScript is used excessively in most kinds of websites and often the desired action on a website can’t be automated without running those scripts. However there’s no practical way of running those scripts and providing those scripts the environment they need to run properly without spawning a browser instance. This is where Selenium comes into picture. Using Selenium you can spawn a browser instance such as that of Chrome or Firefox and programmatically inject keypresses, clicks, scrolls or other actions within the browser to automatically surf a website and perform the actions needed on it to accomplish your tasks.

Python Automation Tools for Desktop


While the libraries mentioned above are very powerful tools to automate web related tasks, there are sometimes cases when you need to automate parts of the GUI of your computer. Or may be the web app you’re targeting is highly graphics-dominated and you decided to spawn a browser from Selenium and set up the environment, and then work on the pixels of the website. Or you might wish to automate the paint application in your computer and draw images automatically using the app’s widgets just for fun. Also some cases can arise where a native application doesn’t provide any CLI or programmable API and interacting with it graphically is the only way to automate tasks on that app. In all these cases PyAutoGUI can be a very useful tool to analyze the pixels on a screen and automatically supply clicks, keystrokes and so on in the desired manner. Also this library is written by Al Sweigart himself, the author of the famous book in Python community, Automate the Boring Stuff with Python.


Celery is a powerful library often used in web development and other areas to run tasks asynchronously or to schedule tasks in future or periodically. You can use Celery to run a task periodically at a specified time eg. every Monday at 9 PM, or at every one hour interval and so on. This feature can be very helpful if you want to automate tasks such as taking periodic backups, checking for changes in your Google Drive/Dropbox storage, or sending alerts for new tweets containing a hashtag and so on.

Python automation tools for sending notifications

When executing long-running Python programs or running programs in the background, you’ll soon get to realize the importance of sending notifications to yourself regarding the status of your programs when important events take place. For example your programs can throw an error and fail due to a magnitude of reasons you couldn’t have predicted and in these cases it’s absolutely great to have the ability to be notified of the issue as well as the traceback and context of the program which led to the problem. There are different ways to let your Python script send notifications to you and let’s discuss about some of them.


Probably the simplest as well as an effective way to send notifications to yourself or your team from your programs is via email. It’s pretty straightforward to send email to yourself and also doesn’t require much setup to get going quickly with this system. All you need is an email account and some details of the email provider and you’re good to go using Python’s builtin smtplib and other libraries!


You can also send yourself notifications on a Discord channel very easily using Discord’s webhooks using the requests library we discussed earlier. You can also create a highly interactive interface for your program via Discord by creating a Discord bot using a library such as Likewise there are similar tools for Telegram which you can use to send messages to you or create interactive Telegram bots with. Using these tools you can easily send yourself rich and well-formatted alerts about different kinds of events.

Deploying your Python Automation Tools to the Cloud

Oftentimes the scripts you write to automate your tasks require running continuously or at specific times, or have other requirements such as good internet connectivity and so on. Depending on the needs it may be difficult or impossible to achieve that on your personal computer. In these cases it’s a good idea to deploy your script on the cloud and run it there. DigitalOcean is one of the best cloud based virtual server (VPS) providers in the market with a simple and developer-friendly platform to provision a server instance and get going. Other relevant providers in the market include Linode, AWS, GCP, Azure, Heroku and PythonAnywhere. Some of these services provide a free tier as well for hobby projects which might be sufficient for your needs.

Python Automation Tools for Testing

Testing is a great tool that helps developers write reliable, consistent and efficient software. In Python testing is especially important given its duck typing and interpreted nature where some of the more obvious bugs which would have been detected in the compile time in compiled languages can evade detection and lead to errors in deployment. However given its duck typing and other productivity-boosting features it also means that writing tests in Python is a faster and less painful process than it might be in more rigid languages. It’s also important to note that testing is more than just a tool and there are entire ideologies on software development in general which put testing at their core. Test-driven development is one such methodology. If you haven’t, I highly recommend reading the book Test-Driven Development with Python to learn more about testing and other good practices. Python has a great ecosystem when it comes to writing and running tests in a systematic and comprehensive manner and let’s discuss some of them.


unittest is a builtin library in Python which can take you quite far in your testing. It has a relatively lower learning curve than many other testing libraries and thus easier to start using, but is powerful enough to serve the purpose feasably in most cases.


pytest is currently probably the most popular Python testing framework. It has a more succint style and has more powerful features than unittest. It lets you write more compact and better organized tests and provides many excellent features that are needed in modern software development to write and run effective test suites.


tox is a more advanced testing framework which lets you run your tests in multiple environments which can be very useful if you’re going to package and distribute your Python application to other users who might be using different operating systems, Python versions and so on.

These are some of the Python automation tools which often come very handy when automating different kinds of tasks. However the Python ecosystem offers many more amazing and well-designed libraries and tools which can help you to automate different kinds of tasks. If you haven’t already, I highly recommend reading the Part 2 of the book Automate the Boring Stuff with Python (Part 1 as well if you are new to Python and want to master the basics!). The book provides a great introduction to automating a wide range of tasks using Python and will certainly get your mind coming up with new ideas and ways in which you could automate some of the tasks you perform in your daily life, at work and so on.