Download a web page
Building a web crawler starts by fetching one page. It can be an HTML page. The RSS or Atom feed of a web site. Some site provide API where get the response as a JSON string.
It can be downloading an image or some other binary file.
The task is quite simple. Given a URL fetch the thing that is behind it and put it in a variable.
For a variation on this, you can write a function that will get a URL and a filename and will save the content it received in the given file.
Published on 2015-11-20