Putting it all together: main()

Code the main function and all auxiliary functions into the files wet.h (declarations) and wet.cxx (definitions). The program should: read the given URL and the desired downloading recursion depth limit from the command line, then recursively download the given URL and all meaningful sub-links up to the given depth limit, whilst storing them as vertices and arcs of a digraph, which will then be printed out in GraphViz format.

• The recursive part of the algorithm can be coded into a pair of functions (one is the ``driver'' function, the other is the truly recursive one) declared as follows:

    // retrieve the data, driver
    void retrieveData(std::string URL, int maxDepth, Digraph& G, 
                      bool localOnly, bool verbose);
    // retrieve the data, recursive
    void retrieveData(std::string URL, int maxDepth, Digraph& G, 
                      bool localOnly, bool verbose, 
                      int currentDepth, VertexURL* vParent);
  

  (also see Section 5.1.4).
• Endow your WET application with a command line option -v (stands for ``verbose'') that prints out the URLs as they are downloaded
• Endow your WET application with a command line option -l (stands for ``local'') that ignores all URLs referring to webservers other than localhost (127.0.0.1). Optional: also try to devise an option -L that ignores all URLs referring to webservers different from the one given on command line.
• The GraphViz system can read a text description of a given graph and output a graphical representation of the graph in various formats. The WET application needs to output the text description of the Digraph object. One self-explanatory example of the required format has been given at the beginning of this chapter. You can type at the shell to get more information. Implement WET so that it outputs the graph description in the correct format on cout. When your WET application is complete, run it so that it saves the output to a file with the .dot extension: . The command builds a GIF file which you can display using the command .

Test your WET application by running
[./wet http://www.enseignement.polytechnique.fr/profs/informatique/Leo.Liberti/test.html4] and verifying that the output is

# graphviz output by WET (L. Liberti 2006)
digraph www_1199989821 {
  0 [ label = "www.enseignement.polytechnique.fr" ];
  1 [ label = "www.enseignement.polytechnique.fr" ];
  2 [ label = "www.enseignement.polytechnique.fr" ];
  3 [ label = "Thu Jan 10 19:30:21 2008", color = red ];
   0 -> 1;
   1 -> 0;
   1 -> 2;
   2 -> 0;
   2 -> 1;
   0 -> 2;
}

By saving wet's output as wetout.dot and running we get Fig. 5.3, left. Reducing the depth level to 2 yields Fig. 5.3, right.

Figure 5.3: The neighbourhoods graphs (depth 4, left; depth 2, right).