The URL class

Design and implement a class called URL containing a pointer to an HTMLPage object. Its functionality is to download an HTML file from the internet given its URL, to allow access to the links contained therein, and to return the hostname of the URL.

• We shall make use of the wget external utility to download the HTML file. This can be used from the shell: will download the specified URL to a local file called mypage.html. We can specify the output file with the option -O output_file_name. In order to call an external program from within C++, use the following code.

  #include<cstdlib>
  [...]
  char charQuote = '\"';
  string theURL = "http://127.0.0.1/index.html";
  string outputFile = ".wet.html";
  string cmd = "wget --quiet -O " + outputFile + " " + charQuote + theURL + charQuote;
  int status = system(cmd.c_str());
  

• You will need to delete the temporary file .wet.html after having parsed it -- use the following code.

  #include<unistd.h>
  [...]
  unlink(outputFile.c_str());
  

• The URL class will expose a method std::string getNextLink(void) to read the links in the downloaded HTML page in the order in which they appear. Each time the method is called, the next link is returned. When no more links are present, the empty string should be returned. Subsequent calls to getNextLink() should return the sequence of links again as in a cycle.
• The URL class will contain a pointer to an HTMLPage object which parses the HTML code and finds the relevant links. Note that this means that a user memory allocation/deallocation will need to be performed.
• Test your URL implementation using the driver program urldriver.cxx:
  

  #include<iostream>
  #include "url.h"
  
  int main(int argc, char** argv) {
    using namespace std;
    if (argc < 2) {
      cerr << "missing arg on cmd line" << endl;
      return 1;
    }
    URL url(argv[1]);
    url.download();
    string theLink = url.getNextLink();
    while(theLink.size() > 0) {
      cout << theLink << endl;
      theLink = url.getNextLink();
    }
    return 0;
  }
  

  Run the program

  ./urldriver http://www.enseignement.polytechnique.fr/profs/informatique/Leo.Liberti/test.html

  and verify that the output is

  http://www.enseignement.polytechnique.fr/profs/informatique/Leo.Liberti/test3.html
  http://www.enseignement.polytechnique.fr/profs/informatique/Leo.Liberti/test2.html