Download files and web pages with WGet

wget The command can download web pages or remote files .

Log in and copy

      
      

       
       [[email protected] ~]# wget https://baidu.com
       
       
--2018-05-17 21:28:27--  https://baidu.com/
       
       
 Resolving host  baidu.com (baidu.com)... 220.181.57.216, 123.125.115.110
       
       
 on connection  baidu.com (baidu.com)|220.181.57.216|:443...  Connected .
       
       
 Already issued  HTTP  request , Waiting for response ... 302 Moved Temporarily
       
       
 Location ：http://www.baidu.com/ [ Follow the new  URL]
       
       
--2018-05-17 21:28:27--  http://www.baidu.com/
       
       
 Resolving host  www.baidu.com (www.baidu.com)... 115.239.210.27, 115.239.211.112
       
       
 on connection  www.baidu.com (www.baidu.com)|115.239.210.27|:80...  Connected .
       
       
 Already issued  HTTP  request , Waiting for response ... 200 OK
       
       
 length ：2381 (2.3K) [text/html]
       
       
 Saving to : “index.html”
       
       
100%[======================================>] 2,381       --.-K/s  when  0s      
       
       
2018-05-17 21:28:27 (326 MB/s) -  Saved  “index.html” [2381/2381])
      
      
      
      

       
       
1.
       
       
2.
       
       
3.
       
       
4.
       
       
5.
       
       
6.
       
       
7.
       
       
8.
       
       
9.
       
       
10.
       
       
11.
       
       
12.
       
       
13.
       
       
14.
      
      

You can also choose from multiple URL Address to download

Log in and copy

      
      

       
       $ wget url1 url2...
      
      
      
      

       
       
1.
      
      

Use options -O You can specify the file name of the output .

Log in and copy

      
      

       
       $ wget https://www.baidu.com/img/superlogo_c4d7df0a003d3db9b65e9ef0fe6da1ec.png -O baidu.png
      
      
      
      

       
       
1.
      
      

Use options -t We can specify the number of retries , Once the download is interrupted ,wget Will continue to try .

Log in and copy

      
      

       
       $ wget -t 5 https://baidu.com
      
      
      
      

       
       
1.
      
      

utilize --limit-rate Option we can limit the download speed

Log in and copy

      
      

       
       $ wget --limit-rate 100k  https://baidu.com
      
      
      
      

       
       
1.
      
      

utilize -c We can continue with breakpoints

wget There is a very useful option --mirror, seeing the name of a thing one thinks of its function , This is used to mirror the site .

Log in and copy

      
      

       
       $ wget --mirror https://baidu.com
      
      
      
      

       
       
1.
      
      

Of course , You can't really mirror Baidu , So you can use --quota perhaps -Q To limit download quotas .

Log in and copy

      
      

       
       $ wget -Q 100m https://baidu.com
      
      
      
      

       
       
1.
      
      

If you need more precise control over the mirror website ：

Log in and copy

      
      

       
       $ wget -r -N -l 5 https://baidu.com
      
      
      
      

       
       
1.
      
      

-r Is recursive option ,-N Allow timestamps on files ,-l Specify the number of pages to traverse .

If the web page or file you want to download needs authentication , have access to --user and --password To provide user name and password .

Log in and copy

      
      

       
       $ wget --user user --password password https://xuyaoyonghuminghemima.com
      
      
      
      

       
       
1.