Hax

Web - wget

"Making IT happen on the
Information Superhighway"

 

2600 | Antionline | Astalavista | Blacklisted 411 | Crypto-Gram | Defcon | Geeks Are Sexy | IT Conversations | Shoutwire | Slashdot | Tech Tales | The Daily WTF |

Forums | Linux | News |Telephone | Windows |

 


 

wget
  • wget
    Application Location
    C:\WINDOWS\system32
    Dump Location
    C:\Documents and Settings\User
  • wget in a batch format:
    wget -i c:\url.txt
  • WWget
  • Cygwin wget

    Dump Location:
    C:\cygwin\home\User Name

Arguments
  • Authentication
  • Concatenate group of files into a single document
  • Continue downloading, in event of failure
  • Delay, to avoid hammering
  • Do not overwrite existing files (file1, file2), etc
  • Download URLs from a file
  • Force filetype
  • Grab all files needed for proper viewing
  • Ignore Robots
  • Modify content for local viewing
  • Progress Bar
  • Put all files into same directory
  • Put files into specific directory
  • Times to attempt download

wget batch script, with comments
  • # Desired URL
    http://www.cryptome.org/

    # Sample script
    wget -P mydir -p -k -nd http://phoenix.csc.calpoly.edu/~st50/index.html

    # Positioning to construct a new script
    http://www.cryptome.org/
    wget -P mydir -p -k -nd http://phoenix.csc.calpoly.edu/~st50/index.html

    # Newly constructed script
    wget -P mydir -p -k -nd http://www.cryptome.org/

    # Add in a delay is -w 5

    # Add 5 second delay to newly constructed script
    wget -P mydir -p -k -nd -w 5 http://www.cryptome.org/

    # Try putting the file dump into a specific folder in a different root
    wget -P c:\mydir -p -k -nd -w 5 http://www.cryptome.org/
    | | | | | | The "w" command requires wget to WAIT for "x" seconds before requesting another file
    | | | | | The "nd" argument tells wget to put every file into the same directory (no folders)
    | | | | The "k" argument instructs wget to modify the content, so it can be locally viewed
    | | | The "p" argument instructs wget to grab ALL files needed for the page to properly display
    | | | for instance, graphics, "CSS" files or "Javascript" files, etc.
    | | The location for the "Put" command is specified here, as "c:\mydir"
    | The "Put" argument instructs wget to drop the files in the location specified
     
  •