Key takeaways:
- cURL GET requests help you retrieve web pages, API data, or files from the command line.
- Query parameters help you control the data a server returns.
- HTTP headers help you specify your preferred response format.
To make a cURL GET request, run cURL followed by the URL of the resource you want to retrieve. You do not need to add -X GET to your command because cURL uses the GET method by default for HTTP and HTTPS requests.
This article will help you understand how to make these requests, pass parameters, add HTTP headers, work with cookies, and save the server's response to a file. You will also learn how to troubleshoot common errors and what security risks to consider when using cURL.
How cURL uses the GET method to fetch data
cURL is one of many command-line tools you can use from your terminal to interact with servers and retrieve resources. GET is an HTTP method you can use to request a resource from a server.
Here is an example of how cURL uses GET to request a resource:
curl https://httpbin.org/get
So, what happens when you run this cURL command?
- cURL reads the URL to identify the server you want to connect to and the resource
- cURL then resolves the server's hostname to an IP address and opens a network connection
- Because the URL uses HTTPS, cURL establishes a secure connection and verifies the server's certificate
- cURL then sends the GET request
- The server receives the request and decides what to return
- The server sends an HTTP response back to cURL
- cURL receives the response body and shows it in your terminal
A GET request does not necessarily retrieve a file. It asks the target server for the current representation of the resource at that URL. Depending on the resource you request, the server could return HTML, JSON, or another type of data.
Common use cases
Here are some of the most common use cases for cURL GET requests:
- Fetch web page content: cURL GET requests come in handy when you need to check the server's response for a web page without opening it in a browser.
- Retrieve data from an API: Use cURL to check whether an API endpoint is working, inspect the JSON it returns, or test query parameters. Our guide to using cURL for API requests covers other HTTP methods, authentication, and more advanced API requests.
- Download a file: cURL can help you download files directly from a remote server to your local device using output options like -o or -O.
In each case, cURL lets you send the GET request directly from the command line and work with the server's response.
Performing basic cURL GET requests
Here are a few examples showing how you can run cURL GET requests from your terminal.
Retrieve web page content
Run:
curl https://www.wikipedia.org
When you run this command, cURL sends a GET request to Wikipedia and returns the HTML source of the web page to your terminal. It does not render the page as you would see it in a browser.
Retrieve data from a public API
You can also make a GET request to a public API. For example:
curl https://api.github.com/repos/curl/curl
This cURL request asks the GitHub API for information about the curl/curl repository. The server's response is JSON and includes details about the repository.
Test a GET request
For a request you can inspect more easily, run:
curl https://httpbin.org/get
httpbin is designed for testing HTTP requests. This command returns JSON showing information about the GET request it received, including the request headers and URL:
{
"args": {},
"headers": {
"Accept": "*/*",
"Host": "httpbin.org",
"User-Agent": "curl/8.21.0",
"X-Amzn-Trace-Id": "Root=1-6a996f74-4245022838f60059124ef7b7"
},
"origin": "41.90.209.118",
"url": "https://httpbin.org/get"
}
These commands all use the same basic curl URL command format. What changes is the resource you request and the response the server sends back.
You can also route a cURL GET request through a proxy when you need the request to come from a different IP address. While there are different types of proxies you could use, residential proxies work best because they offer IP addresses from real residential connections.
Read our article on using a proxy with cURL to see how to route cURL GET requests through residential IPs.
How to pass parameters in cURL GET requests
Query parameters are useful when you are targeting a server that lets you narrow, search, sort, or otherwise change what it returns. A good example is when you want to use cURL to collect data from a website like Open Library.
If you want your cURL request to search for a particular term and return a set number of results, you need to add query parameters to the URL.
Here is the command you would use:
curl "https://openlibrary.org/search.json?q=tolkien&limit=3"
We introduced two query parameters here:
q=tolkien
This tells Open Library to use “tolkien” as the search query.
Then & separates that parameter from the second one:
limit=3
This tells Open Library to limit the results to three.
So essentially, query parameters let you control the results you want from a cURL request. But imagine you wanted to add several more parameters. Writing the full query string can quickly become cumbersome.
Fortunately, cURL can construct it for you. All you have to do is give it the pieces separately:
curl -G -d "q=tolkien" -d "limit=3" https://openlibrary.org/search.json
-G tells cURL to put your supplied data in the URL query string and send a GET request. Then, -d "q=tolkien" gives cURL the search query, while -d "limit=3" provides the maximum number of results.
cURL constructs:
?q=tolkien&limit=3
and attaches it to:
https://openlibrary.org/search.json
giving you:
https://openlibrary.org/search.json?q=tolkien&limit=3
In summary, the -G and -d options are simply an easier way to ensure your parameters are properly encoded in cURL instead of building everything manually.
Adding headers to cURL GET requests
If you're dealing with a website that supports different response formats, you can use an HTTP header to tell the server which format you prefer. Headers can also carry authentication details when the target server needs proof that you're authorized to access a particular resource.
For instance, if you want to receive data in JSON format with cURL, you can pass the Accept: application/json HTTP header to the server.
Open Library supports this on some of its resource endpoints. For example:
curl -H "Accept: application/json" https://openlibrary.org/books/OL1M
Here, -H tells cURL to add an HTTP header to the GET request. Read our article on sending HTTP headers using cURL for more tips on authentication, custom User-Agent headers, and other request requirements.
The header:
Accept: application/json
tells Open Library that you want the server's response in JSON format.
If you only need the response headers, use -I. For HTTP URLs, this tells cURL to send a HEAD request, which returns the headers without the normal response body.
Handling cookies in cURL GET requests
When working with cURL, cookies do not automatically persist between separate cURL commands the way they normally do in a browser. For instance, if a website gives you a session cookie after you log in, a later cURL request will not automatically have that cookie. The server may treat the second request as a new session because nothing connects it to the first one.
You can use the -c option to save cookies from the server to a file. For a later request, use -b followed by a string with the cookie or the filename containing the cookies to load and send them.
To see how this works with httpbin, first ask httpbin to set a cookie:
curl -c cookies.txt "https://httpbin.org/cookies/set?session=abc123"
The /cookies/set endpoint sends a cookie named session with the value abc123. The -c cookies.txt option tells cURL to save the cookie to a file called cookies.txt.
Then, send the stored cookie back with another GET request:
curl -b cookies.txt https://httpbin.org/cookies
Here, -b cookies.txt tells cURL to read the cookies stored in cookies.txt and send the ones that apply to this request. httpbin then shows the cookie it received:
{
"cookies": {
"session": "abc123"
}
}
You can also use -b and -c with the same file when you want cURL to read existing cookies and save any updates the server sends back:
curl -b cookies.txt -c cookies.txt https://example.com/account
How to save responses from cURL GET requests to a file
If you want to keep the server's response for later, use the -o or --output option to write the response to a file instead of standard output (stdout), which is usually your terminal.
Saving the response with -o is useful when you want to:
- Download a file from a URL
- Keep API output for later inspection
- Save web page HTML so you can compare or analyze it
- Save data for another script or program to use later
- Avoid writing a large response in your terminal
For example:
curl -o books.json "https://openlibrary.org/search.json?q=tolkien&limit=3"
Here, -o books.json tells cURL to save the response body to a file called books.json instead of displaying it in your terminal. Be careful when choosing the filename because -o can overwrite an existing file.
Troubleshooting common cURL GET request errors
Several things happen between sending a cURL GET request and receiving a response. cURL first has to find the server's IP address, connect to it, establish a secure connection for HTTPS, send the GET request, wait for the server, and receive the response.
Something could go wrong at any point in this process:
- cURL could fail to complete the connection because the server is taking too long to respond
- The request could fail or return an unexpected response
- The server could tell cURL that the resource is available at another URL
Here is how you can troubleshoot each of these situations:
If the server is taking too long to respond
If a cURL request is taking too long, set a cURL timeout with --connect-timeout or --max-time.
--connect-timeout limits how long cURL waits to establish the connection. For example:
curl --connect-timeout 2 httpbin.org
If cURL cannot connect within two seconds, it stops.
To test a slow server response, use httpbin:
curl https://httpbin.org/delay/5
This delays the server's response by five seconds.
You can limit the entire request with --max-time:
curl --max-time 2 https://httpbin.org/delay/5
If the request takes longer than two seconds, cURL stops it.
What if you don't know where the problem is?
If cURL is doing something you did not expect or showing an error you do not understand, use the -v option, short for --verbose. This shows you what cURL is doing during the request instead of just showing the response body. You will see things such as:
- How cURL connects to the server
- TLS and certificate information for HTTPS
- HTTP request headers
- Response headers the server sends back
If the server tells cURL the resource is at another URL
You will usually see this through a 3xx redirect status code, such as 301 Moved Permanently or 307 Temporary Redirect. Because cURL does not follow redirects by default, you need to tell it to do so with -L, short for --location:
curl -L https://example.com
With -L, cURL follows redirects until it reaches the final URL or hits its redirect limit.
Security considerations for cURL GET requests
Here are some security considerations to keep in mind when using cURL to send GET requests.
Use HTTPS for secure connections
Whenever the target server supports it, use HTTPS for your cURL request because TLS encrypts the connection. If you use HTTP, someone could potentially see or alter the request or the server's response, which can be a problem when working with sensitive data.
Handle API keys securely
Even if the target server accepts an API key in the URL, it is better to pass it in an HTTP header when the API supports it. Although HTTPS encrypts the query string in transit, the URL can still end up in places where you did not intend to expose the key, such as command history or server logs.
That’s why you should pass API keys and bearer tokens in request headers:
curl -H "Authorization: Bearer YOUR_TOKEN" https://api.example.com/data
The exact HTTP header you use depends on the API.
Verify TLS certificates
When you send a cURL request over HTTPS, cURL verifies the server's TLS certificate by default. If verification fails, cURL stops the connection and reports the error.
If you are connecting to an internal server that uses a certificate cURL does not already trust, you can provide the appropriate CA certificate with --cacert:
curl --cacert company-ca.pem https://internal.example.com
You can also tell cURL to skip certificate verification with -k or --insecure:
curl -k https://internal.example.com
Skipping verification makes the connection less secure, so using --insecure should generally be limited to testing or development.
Conclusion
To sum this all up, cURL GET requests let you fetch HTML page content or retrieve data from an API, among other use cases. Query parameters let you specify the data you want, while header specifies the preferred response format or carry API keys.
Just remember to use HTTPS, keep credentials out of URLs, and leave TLS certificate verification enabled. For more tips on using cURL, join our Discord channel.
