GetAlbumFromURI takes in an album page URL ie. http://www.pornhub.com/album/66212 and returns a corresponding *model.Album for that page

GetImageFromURI takes in a URI for an image page ie. http://www.pornhub.com/photo/540124 and returns a corresponding &model.Image struct for that page

CrawlPornHub crawls pages from pornhub for images, starting at 'start' (inclusive) and ending at 'stop' (exclusive). Pages are dumped to the filepath

* Currently only need GET but this can be extended easily */ Get takes in a HandlerFunc and makes it only accept GET requests

HTTPHandlerWithMethod takes in a handler func and wraps it so only requests with the specified method (POST, GET, etc) will go through to the handler.

ParseConfigFromFile takes the passed config file path and parses the config params from the file located there

getCommentFromSelection takes in a goquery selection and returns a corresponding model.Comment struct filled with information about the comment. Becau

GetBasePageByPageNumber takes in a page number and returns a BasePage struct (only descenced down into the album struct, not individual images) corres

init takes in the config path from the command line flags

mapper the map step takes in an image json object and maps it to comment information, holding onto information about the origi

mapper the map step splits images from their albums, storing the segment, album_id, and album_title for use later and yielding

mapper the map step takes each album from the page, finds it's album_id from the URL scheme and saves it, and maps each image

mapper the map step takes in an album and maps it to a reduced format comprised of the tag and all tags in the same al

mapper the map step takes in an album and maps it to a reduced format comprised of the tag and all tags in the same al

reducer the reduce step just passes all map output to the OUTPUT as a json object

reducer the reduce step just passes all map output to the OUTPUT as a json object

reducer the reduce step just passes all map output to the OUTPUT as a json object

reducer the reduce step takes all the tags and returns a key (tag) and the number of times the tag has been seen

reducer the reduce step takes all the tags and returns a key (tag) and an array of 2-tuples of tags seen with the key