cs1705.web
Class WebBot

java.lang.Object
  cs1705.web.WebBot

Direct Known Subclasses:

TurboWebBot

public class WebBot
extends Object

This class represents a robot that knows how to walk through a web page and identify headings and links. It will automatically transform "messy" real-world html into conforming XHTML as it visits pages, so all tag matching and other support should presume XHTML conventions.

Version:

2009.03.31

Author:

Stephen Edwards

Constructor Summary

WebBot()
Creates a new WebBot that is not yet viewing any web page.

WebBot(String url)
Creates a new WebBot for a given URL.

Method Summary

void	advanceToNextHeading() Advance the robot forward in the current document until it is looking at (or standing on) the next HTML heading element it can find.
void	advanceToNextLink() Advance the robot forward in the current document until it is looking at (or standing on) the next HTML anchor containing an href attribute that it can find.
void	echoCurrentElementText() Echo the text of the current HTML element (heading, link, etc.) to the robot's default output channel.
void	echoPageTitle() Echo the current web page title to the robot's default output channel.
HtmlElement	getCurrentElement() Get the HTML element of interest that the robot is currently standing on.
String	getCurrentElementText() Get the text of the current HTML element on this web page--i.e., the title of a heading or the text associated with a link.
int	getHeadingLevel() Get the heading level (1-6) of the current heading on this web page.
List<HtmlHeadingElement>	getHeadings() Get an iterator over all headings in the current document.
List<HtmlHeadingElement>	getHeadingsToLevel(int level) Get an iterator over all headings in the current document with a level less than or equal to the value specified.
List<URI>	getLinks() Get an iterator over all links in the current document.
List<URI>	getLinksOffServer() Get an iterator over all links in the current document that refer to pages on other servers.
List<URI>	getLinksToOtherPages() Get an iterator over all links in the current document that refer to other web pages.
URI	getLinkURI() Get the URI of the current link on this web page.
PrintWriterWithHistory	getOutputChannel() Get the output channel where this bot is sending its output.
String	getPageTitle() Get the title the current web page.
URL	getPageURL() Get the URL for the current web page.
boolean	hasPreviousPage() Check to see if this bot previously visited a different page that it can now return to.
boolean	hasVisitedPage(URI uri) Check whether this robot has visited this page before.
boolean	hasVisitedPage(URL url) Check whether this robot has visited this page before.
boolean	isLookingAtEndOfPage() Has the robot advanced through all the contents (headings and links) on the current page? Will also return true if isViewingWebPage() returns false.
boolean	isLookingAtHeading() Is the robot looking at (or standing on) an HTML heading element on the current page?
boolean	isLookingAtLink() Is the robot looking at (or standing on) an HTML anchor containing an href attribute (that is, a link to another web page) on the current page?
boolean	isViewingWebPage() Is the robot currently viewing a real web page with readable contents? Normally, this would be true, but may be false if the bot has not been given a web page to start on, or if it has been given a malformed or nonexistent URL address, or even if the server for the targeted page is not available.
void	jumpToLinkedPage() Causes the bot to temporarily leave the current page and hop over to the page at the end of the current link.
void	jumpToPage(String url) Causes the bot to temporarily leave the current page and hop over to the page specified by the URL (as a string).
void	jumpToPage(URI uri) Causes the bot to temporarily leave the current page and hop over to the page specified by the URL.
void	jumpToPage(URL url) Causes the bot to temporarily leave the current page and hop over to the page specified by the URL.
void	jumpToThisHTML(String html) Causes the bot to temporarily leave the current page and hop over to a specific HTML string provided as a parameter.
boolean	linkGoesToAnotherPage() Check whether the URL of the current link on this web page refers to a different page, or just another location within the current page.
boolean	linkGoesToAnotherServer() Check whether the URL of the current link on this web page refers to a page on a separate server, or simply another location on the same server.
int	numberOfPreviousPages() How deep is the stack of previous pages that this robot can return to? Each time the robot jumps to a new page, it remembers its previous page so you can returnToPreviousPage().
PrintWriterWithHistory	out() Get the output channel where this bot is sending its output.
boolean	outputIsHtml() Check whether this robot's output should be treated as plain text, or as HTML markup.
URI	resolveURIFromPage(String uri) Get a fully-resolved URI from a (possibly relative) string URI, such as the value of an anchor's href or an img's src attribute.
void	returnToPreviousPage() Causes the bot to leave the current page and return to the page it was previously visiting, at the location where it left off.
void	returnToStartOfPage() Moves the robot back to the start of the current page.
void	run() Execute this robot's built-in sequence of steps.
void	setOutputChannel(PrintWriter output) Tell this bot where to send its output.
void	setOutputIsHtml(boolean value) Set whether this robot's output should be treated as plain text, or as HTML markup.
String	toString() Get a printable summary of this robot.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

WebBot

public WebBot()

Creates a new WebBot that is not yet viewing any web page.

WebBot

public WebBot(String url)

Creates a new WebBot for a given URL.

Parameters:

url - The web page where the robot will start.

Method Detail

isViewingWebPage

public boolean isViewingWebPage()

Is the robot currently viewing a real web page with readable contents? Normally, this would be true, but may be false if the bot has not been given a web page to start on, or if it has been given a malformed or nonexistent URL address, or even if the server for the targeted page is not available.

Returns:

True if the robot is currently viewing a real web page with readable contents

isLookingAtEndOfPage

public boolean isLookingAtEndOfPage()

Has the robot advanced through all the contents (headings and links) on the current page? Will also return true if isViewingWebPage() returns false.

Returns:

True if the robot has advanced over all the headings and links in the current document, or false if there are more headings and/or links to visit.

returnToStartOfPage

public void returnToStartOfPage()

Moves the robot back to the start of the current page. Requires the bot to be viewing a web page.

getPageTitle

public String getPageTitle()

Get the title the current web page. Requires the bot to be viewing a web page.

Returns:

The page's title, or null if the page has no title.

echoPageTitle

public void echoPageTitle()

Echo the current web page title to the robot's default output channel. Requires the bot to be viewing a web page.

getPageURL

public URL getPageURL()

Get the URL for the current web page. Requires the bot to be viewing a web page.

Returns:

The page's URL, if it exists.

toString

public String toString()

Get a printable summary of this robot.

Overrides:

toString in class Object

Returns:

The page's content

getCurrentElement

public HtmlElement getCurrentElement()

Get the HTML element of interest that the robot is currently standing on. Requires the bot to be looking at an element on the current web page.

Returns:

The heading's title.

isLookingAtHeading

public boolean isLookingAtHeading()

Is the robot looking at (or standing on) an HTML heading element on the current page?

Returns:

True if the robot is positioned at a heading, or false otherwise.

advanceToNextHeading

public void advanceToNextHeading()

Advance the robot forward in the current document until it is looking at (or standing on) the next HTML heading element it can find. If there are no more headings in the document, it will end up looking at the end of the page. Requires the bot to be viewing a web page.

getHeadings

public List<HtmlHeadingElement> getHeadings()

Get an iterator over all headings in the current document. This method is designed to make it easy to write foreach-style loops over page headings. Requires the bot to be viewing a web page.

Returns:

an iterator of HtmlHeadingElement objects describing the headings in the page.

getHeadingsToLevel

public List<HtmlHeadingElement> getHeadingsToLevel(int level)

Get an iterator over all headings in the current document with a level less than or equal to the value specified. This method is designed to make it easy to write foreach-style loops over page headings. Requires the bot to be viewing a web page.

Parameters:

level - Only include headings at this level or above (i.e., numerically less than or equal to this number)

Returns:

an iterator of HtmlHeadingElement objects describing the headings in the page with levels less than or equal to the specified level.

echoCurrentElementText

public void echoCurrentElementText()

Echo the text of the current HTML element (heading, link, etc.) to the robot's default output channel. Requires the bot to be viewing an existing HTML element on the current web page.

getCurrentElementText

public String getCurrentElementText()

Get the text of the current HTML element on this web page--i.e., the title of a heading or the text associated with a link. Requires the bot to be looking at an element on the current web page.

Returns:

The text contained by this element on the web page.

getHeadingLevel

public int getHeadingLevel()

Get the heading level (1-6) of the current heading on this web page. Requires the bot to be looking at a heading element on the current web page.

Returns:

The heading's level.

isLookingAtLink

public boolean isLookingAtLink()

Is the robot looking at (or standing on) an HTML anchor containing an href attribute (that is, a link to another web page) on the current page?

Returns:

True if the robot is positioned at a link, or false otherwise.

advanceToNextLink

public void advanceToNextLink()

Advance the robot forward in the current document until it is looking at (or standing on) the next HTML anchor containing an href attribute that it can find. If there are no more headings in the document, it will end up looking at the end of the page. Requires the bot to be viewing a web page.

getLinkURI

public URI getLinkURI()

Get the URI of the current link on this web page. Requires the bot to be looking at a link (anchor) element on the current web page.

Returns:

The link's destination.

linkGoesToAnotherPage

public boolean linkGoesToAnotherPage()

Check whether the URL of the current link on this web page refers to a different page, or just another location within the current page. Requires the bot to be looking at a link (anchor) element on the current web page.

Returns:

True if the link refers to a different page

linkGoesToAnotherServer

public boolean linkGoesToAnotherServer()

Check whether the URL of the current link on this web page refers to a page on a separate server, or simply another location on the same server. Requires the bot to be looking at a link (anchor) element on the current web page.

Returns:

True if the link refers to a page located on a different server

getLinks

public List<URI> getLinks()

Get an iterator over all links in the current document. This method is designed to make it easy to write foreach-style loops over links. Requires the bot to be viewing a web page.

Returns:

an iterator of URI objects describing the links in the page.

getLinksToOtherPages

public List<URI> getLinksToOtherPages()

Get an iterator over all links in the current document that refer to other web pages. This is a subset of those returned by getLinks(), with any links to other locations within the same page filtered out. This method is designed to make it easy to write foreach-style loops over links. Requires the bot to be viewing a web page.

Returns:

an iterator of URI objects describing the links in the page.

getLinksOffServer

public List<URI> getLinksOffServer()

Get an iterator over all links in the current document that refer to pages on other servers. This is a subset of those returned by getLinks(), with any links to pages on the same server as the current page filtered out. This method is designed to make it easy to write foreach-style loops over links. Requires the bot to be viewing a web page.

Returns:

an iterator of URI objects describing the links in the page.

jumpToLinkedPage

public void jumpToLinkedPage()

Causes the bot to temporarily leave the current page and hop over to the page at the end of the current link. The bot will "remember" where it came from, keeping track of past pages in a stack. After working with the other page, you can use returnToPreviousPage() to come back to the point where you left off. Requires the bot to be looking at a link (anchor) element on the current web page.

returnToPreviousPage

public void returnToPreviousPage()

Causes the bot to leave the current page and return to the page it was previously visiting, at the location where it left off. The previous page is the one that was most recently "remembered", or alternatively, the one on top of the stack of previous pages that have been visited. Use this method in conjunction with jumpToLinkedPage() to explore multiple pages. Requires the bot to have some previous page to return to.

hasPreviousPage

public boolean hasPreviousPage()

Check to see if this bot previously visited a different page that it can now return to. Is the stack of previous pages empty or not?

Returns:

True if there is at least one previous page on the stack of previous visited pages, or false if there are none.

numberOfPreviousPages

public int numberOfPreviousPages()

How deep is the stack of previous pages that this robot can return to? Each time the robot jumps to a new page, it remembers its previous page so you can returnToPreviousPage(). These previous pages are remembered on a stack, and this method allows you to determine how deep this stack is--that is, how many times you can repeatedly call returnToPreviousPage() successfully.

Returns:

The depth of the previous page stack. This result is zero if the robot is on a page, but has not yet jumped to any others, or -1 if there is no current page at all.

jumpToPage

public void jumpToPage(String url)

Causes the bot to temporarily leave the current page and hop over to the page specified by the URL (as a string). The bot will "remember" where it came from, keeping track of past pages in a stack. After working with the other page, you can use returnToPreviousPage() to come back to the point where you left off.

Parameters:

url - The new page to jump to

jumpToPage

public void jumpToPage(URL url)

Causes the bot to temporarily leave the current page and hop over to the page specified by the URL. The bot will "remember" where it came from, keeping track of past pages in a stack. After working with the other page, you can use returnToPreviousPage() to come back to the point where you left off.

Parameters:

url - The new page to jump to

jumpToPage

public void jumpToPage(URI uri)

Causes the bot to temporarily leave the current page and hop over to the page specified by the URL. The bot will "remember" where it came from, keeping track of past pages in a stack. After working with the other page, you can use returnToPreviousPage() to come back to the point where you left off.

Parameters:

uri - The new page to jump to

jumpToThisHTML

public void jumpToThisHTML(String html)

Causes the bot to temporarily leave the current page and hop over to a specific HTML string provided as a parameter. Instead of reading web content from the internet, the text you pass in will be used instead. The bot will "remember" where it was before, keeping track of past pages in a stack. After working with the provided HTML content you pass in, you can use returnToPreviousPage() to come back to the point where you left off in the previous page.

Parameters:

html - A string containing an HTML document to treat as if it came from the web

resolveURIFromPage

public URI resolveURIFromPage(String uri)

Get a fully-resolved URI from a (possibly relative) string URI, such as the value of an anchor's href or an img's src attribute. If the input parameter is a relative URI, it will be converted into an appropriate absolute URI relative to the current page's web location. Requires the bot to be viewing a web page.

Parameters:

uri - The URI to convert to absolute form

Returns:

The equivalent, fully-resolved URI, or null if there is none.

hasVisitedPage

public boolean hasVisitedPage(URI uri)

Check whether this robot has visited this page before.

Parameters:

uri - The page to check

Returns:

True if this robot has previously visited (or is currently on) the given web page

hasVisitedPage

public boolean hasVisitedPage(URL url)

Check whether this robot has visited this page before.

Parameters:

url - The page to check

Returns:

True if this robot has previously visited (or is currently on) the given web page

setOutputChannel

public void setOutputChannel(PrintWriter output)

Tell this bot where to send its output. Whenever you tell the bot to echo content or headings, they will go to this destination. By default, output goes to the standard output channel, but you can change the destination here.

Parameters:

output - The output channel to send messages to

getOutputChannel

public PrintWriterWithHistory getOutputChannel()

Get the output channel where this bot is sending its output.

Returns:

The current output channel for this bot

out

public PrintWriterWithHistory out()

Get the output channel where this bot is sending its output. This is just a short convenience synonym for getOutputChannel().

Returns:

The current output channel for this bot

outputIsHtml

public boolean outputIsHtml()

Check whether this robot's output should be treated as plain text, or as HTML markup. The default is false (treat as plain text).

Returns:

True if the output should be treated as HTML markup

setOutputIsHtml

public void setOutputIsHtml(boolean value)

Set whether this robot's output should be treated as plain text, or as HTML markup.

Parameters:

value - True if the output should be treated as HTML markup, false if it should be treated as plain text

run

public void run()

Execute this robot's built-in sequence of steps. The default sequence is to do nothing, but subclasses can override this method to add their own behaviors. These behaviors will be automatically run if the robot is attached to a RobotViewer.