curl and wget Examples: Download Files, Test APIs and Debug HTTP

curl and wget both fetch things over HTTP from the command line, but they have different strengths. wget is a downloader: it saves files by default, retries on its own and can mirror whole sites. curl is a transfer tool for talking to servers: it prints to the terminal by default and gives you control over methods, headers, request bodies and timing, which makes it the standard tool for testing APIs. Most systems have both; if not, install them with your package manager.

curl basics

$ curl https://example.com               # print the response body
$ curl -o page.html https://example.com  # save to a chosen name
$ curl -O https://www.gnu.org/licenses/gpl-3.0.txt   # save with the remote name

By default curl shows a progress meter when writing to a file. -s (silent) hides it, but also hides errors, so the common combination is -sS: silent, but still show errors.

curl does not follow redirects unless you ask. Without -L, a request to an http:// address often returns a short "301 Moved" page instead of the content:

$ curl -sL -o /dev/null -w '%{url_effective}\n' http://github.com
https://github.com/

Look at headers and status codes

$ curl -sI https://example.com
HTTP/2 200
date: Sun, 25 Oct 2026 06:36:32 GMT
content-type: text/html; charset=utf-8

-I sends a HEAD request and prints only the response headers. Some servers answer HEAD differently from GET; in that case use -i to include headers with a normal request, or -v to see the full conversation, including the request curl sent (lines starting with >) and the TLS handshake.

For scripts and monitoring, print just the status code with -w:

$ curl -s -o /dev/null -w '%{http_code}\n' https://example.com
200

Time a request

The -w option knows many timing variables, which makes curl a quick way to see where a slow page spends its time:

$ curl -s -o /dev/null -w 'dns %{time_namelookup} connect %{time_connect} ttfb %{time_starttransfer} total %{time_total}\n' https://example.com
dns 0.002896 connect 0.009879 ttfb 0.067515 total 0.067612

All times are seconds from the start. A big gap between connect and ttfb (time to first byte) means the server is slow to generate the page; a slow dns figure points at name resolution.

Send JSON and form data

To test an API, set the method with -X, add headers with -H and send a body with -d. These examples use httpbin.org, which echoes back what it receives:

$ curl -s -X POST -H 'Content-Type: application/json' \
    -d '{"name":"alice"}' https://httpbin.org/post

Using -d already implies POST, so -X POST is optional there; you need -X for PUT, PATCH and DELETE. Without the header, -d sends the data as a classic HTML form:

$ curl -s -d 'a=1&b=2' https://httpbin.org/post

Use -d @file.json to send a body from a file, and -F for a multipart upload like a file input on a web form:

$ curl -s -F '[email protected]' https://httpbin.org/post

When the response is JSON, pipe it to jq to pretty-print it or pull out a field: curl -s https://httpbin.org/json | jq -r '.slideshow.title'. If jq is not installed, python3 -m json.tool will at least format it.

Authentication, cookies and user agents

$ curl -s -u user:pass https://httpbin.org/basic-auth/user/pass
$ curl -s -H 'Authorization: Bearer TOKEN' https://httpbin.org/bearer
$ curl -s -A 'mybot/1.0' https://httpbin.org/user-agent

-u handles HTTP basic authentication; leave out the password (-u user) and curl prompts for it, which keeps it out of your shell history. For cookies, -c file saves cookies the server sets and -b file sends them back, so a login followed by a second request behaves like a browser session.

curl in scripts: failures, timeouts and retries

By default curl exits with status 0 even when the server returns 404 or 500, because the transfer itself worked. Add -f to make HTTP errors fail:

$ curl -sf https://httpbin.org/status/404 || echo "failed with $?"
failed with 22

Set limits so a hung server does not hang your script. --max-time 10 caps the whole request (exit code 28 when it trips), --connect-timeout 5 caps only the connection, and --retry 3 retries transient failures. A solid default for scripts is curl -fsSL --max-time 30 --retry 3.

To test a site on a new server before changing DNS, --resolve sends the request to an IP of your choice while keeping the right host name and TLS certificate: curl --resolve example.com:443:203.0.113.10 https://example.com.

Resume a download

$ curl -C - -O https://www.gnu.org/licenses/gpl-3.0.txt
$ wget -c https://www.gnu.org/licenses/gpl-3.0.txt

Both pick up where a partial file left off, as long as the server supports range requests (curl reports status 206 when it resumes). That is a lifesaver for large ISO images over a flaky connection.

wget for downloading

$ wget https://www.gnu.org/licenses/gpl-3.0.txt          # save with remote name
$ wget -O license.txt https://www.gnu.org/licenses/gpl-3.0.txt
$ wget -P downloads/ https://www.gnu.org/licenses/gpl-3.0.txt
$ wget -q -O - https://example.com | head          # print to stdout

wget follows redirects and retries by itself. If a file with the same name exists, it saves the new one as name.1 rather than overwriting it. -q makes it quiet and -nv prints one line per file.

Other useful options:

• -i urls.txt downloads every URL listed in a file.
• --tries=3 --timeout=10 controls retries and timeouts.
• --limit-rate=500k stops a big download from saturating your connection (curl has --limit-rate too).
• -b runs the download in the background and logs to wget-log.
• --spider checks that a URL exists without downloading it.

For parallel downloads with curl, see the xargs -P example in the pipes and xargs guide.

Mirror a page or site

$ wget -m -np -k -p --level=1 https://example.com/

-m turns on mirroring (recursion and timestamps), -np stops it climbing to parent directories, -k rewrites links so the copy works offline, and -p fetches the images, CSS and scripts each page needs. --level limits how deep it goes. Mirror only sites you have permission to copy, add --wait=1 to be polite, and expect modern JavaScript-heavy sites not to work well offline.

Quick reference