Up until this point, we have been able to request information from servers using only HTTP GET requests. These requests cover the vast majority of web scraping tasks that you will encounter as you build your own web scraper. However, there will come a time where you may need to submit some kind of form data in order to retrieve the information you are looking for. This form data could entail search queries, or a login screen, or any page that would require you to type into a box and click a Submit button.

For simple websites, this is done using an HTML <form> element, containing one or more <input> elements and a Submit button. This <form> element usually has attributes defining the action (where to send the <form> data), and a method (the HTTP method to use). By default, the web page will use an HTTP GET request to send the form data,...