How to fetch images from a remote location

Sirv can automatically fetch images from your server by HTTP, HTTPS or S3.

Whenever an image is requested from your Sirv account, of it is missing, it will attempt to fetch it from your server or S3 bucket. Once fetched, Sirv stores the original image in your Sirv account.

HTTP/HTTPS fetching

Sirv can fetch images from a particular folder on your server by HTTP or HTTPS.

Let’s assume your source images are located in this folder of your website:

http://your-domain.com/my-images/

Now let’s say you want to serve this image from Sirv:

http://your-domain.com/my-images/example-folder/an-image.jpg

You can request that file from your Sirv account, like so:

https://your-account.sirv.com/example-folder/an-image.jpg

If it doesn’t already exist in your Sirv account, Sirv will fetch it from your server, then optimize and serve it. Fetching typically happens in less than 1 second, depending how quickly your server responds.

S3 fetching

Sirv can also fetch images from your Amazon S3 bucket or any non-Amazon S3 bucket.

Either the whole bucket or a specific folder can be fetched from.

You can use either virtual hosted-style or path-style URLs to access your S3 bucket:

  • Virtual hosted-style: http://<bucket>.<s3_server>/object
  • Path-style: http://<s3_server>/<bucket>/object

Enable fetching

To enable fetching, go to your Settings page in your Sirv account and click “Set up Auto-Fetch”:

Configure image fetching from S3 or HTTPS

Either configure HTTP fetching by entering your remote server URL, folder (optional) and username/password (if your domain uses authentication):

Configure HTTP image fetching

Or configure S3 fetching by entering the Amazon AWS S3 or other S3 bucket. Also enter your access key ID, secret access key and folder path (optional):

Configure S3 image fetching

File fetching is available to all paid Sirv plans. If you have a Trial or Free plan, switch to a paid plan to start using fetching immediately.

Image processing

All the usual image processing options can be applied to the image. For example:

http://your-account.sirv.com/example-folder/image.jpg?w=500&profile=xyz

Sirv will fetch this image from your server:

http://your-domain.com/my-images/example-folder/image.jpg

Then process and serve it with the options ?w=500&profile=xyz.

Updating a changed image

If an image changes on your server, the old image will continue to be served by Sirv until you delete it from your Sirv account. Once deleted, Sirv will refetch the image the next time it is requested.

Coming soon, Sirv will automatically check if the remote image has changed, according to the Expires header (TTL). For example, if your image had an expires header of 24 hours, Sirv will check the image next time it is requested after 24 hours.

Rate limit

Up to 2000 fetch requests can be sent per hour (4000 per hour for Enterprise accounts). If your account exceeds that, a 429 error (rate exceeded) will be served until the limit is reset (within 1 hour). If you need a larger limit, please contact us.

Troubleshooting

If the requested image cannot be fetched from your server, Sirv will try to download it later, in accordance with the images’ TTL. The minimum TTL is 15 minutes and increases with each new failed download attempt.

Failed fetches are displayed as notifications on your Events page. Notifications contain the source URL, status code and error code:

Fetch images automatically - error notifications

To help you investigate deeper, each attempted fetch is recorded in a JSON file containing the date, time and response code (error message) of each request. View the file by appending ?info to the image URL, for example:

http://test2.sirv.com/inner/NO_SUCH_IMAGE.JPG?info

You will see a “Fetching” section, which describes each fetching attempt. For example:

Fetch images automatically - screenshot of JSON information when 406 error

Possible causes of errors

  • 403 error – occurs if your server refuses access to a file. There are many possible reasons, such as forbidden read access, rejected IP address or invalid configuration.
  • 404 error – occurs if the image is missing from your server. Resolve this by correcting the URL or uploading the missing image to your server.
  • 504 error – occurs if your server didn’t respond within 60 seconds. Check if your server is using rate limiting. If so, either increase or remove the rate limit for requests from Sirv IP addresses.

Service information

If you require a list of Sirv servers (IP addresses) used for fetching, send us a request via your account.

The HTTP Referer header field for Sirv agents is ‘Sirv Image Service’.

The maximum file size Sirv will fetch by S3 is 300MB. The maximum file size Sirv will fetch by HTTP is 2MB. If you require a higher limit, please contact us.

Create your Sirv account

If you don’t have a Sirv account yet, create your account today.

Was this article helpful?