Search files with the API

The search API provides you with a powerful way to find files in your Sirv account, powered by the Elasticsearch engine. Common search options are described below, with code examples. For further reading, you can refer to the Elasticsearch documentation.

Search options

You can search any combination of these parameters:

• basename - file or folder name
• filename - file name and its folder path
• dirname - folder name in any part of path, including Trash
• dirname.paths - this folder and its sub-folders
• dirname.raw - strictly this folder only
• extension - file extension e.g. .jpg
• contentType - mime type e.g. image/jpeg
• mtime - modified time
• ctime - created time
• atime - last accessed time (by HTTP)
• size - file size (bytes)
• isDirectory - if the result is a folder
• meta.tags - files with a particular meta tag
• meta.description - files with a particular meta description
• meta.title - files with a particular meta title
• meta.width - images with certain width
• meta.height - images with certain height

Search by name

Three fields let you search by name: basename, filename and dirname.

basename contains the file (or folder) name. filename contains the entire folder path and file name. dirname contains the entire folder path only. You can see the three values in search results e.g.

"filename": "/Some Images/MY-file.jpg",
"dirname": "/Some Images",
"basename": "MY-file.jpg",

They are all case insensitive. Alphanumeric characters (a-z, 0-9) are considered, as well as the dot . character for filename searches. All other characters in the file name are ignored, but the following characters act as separators:

£ $ % & - _ + = ; @ # ' < > , . | ¬ ` space

Consider this file:

/Some Images/MY-file.jpg

That file would be found via a basename query for any of these terms:

m, my, y, f, fi, fil, file, i, il, ile, l, le, e, j, jp, jpg, p, pg, g

Or a filename query for any of these terms:

some, images, my, file.jpg

As you can see, basename searches are broad (fuzzy) whilst filename searches are specific.

You can wrap queries in " " to make them more specific.

Search by basename

This query will return all files containing Adidas, Gazelle and jpg:

"query": "basename:Adidas-Gazelle.jpg"

This query will return all files containing Adidas then Gazelle:

"query": "basename:*\"Adidas-Gazelle\""

This query will return all files ending 5.jpg and containing the word Ultraboost:

"query": "basename:*\"5.jpg\" AND Ultraboost"

Search by filename

Use filename as a more specific search.

This query will return all files containing Adidas and ending Gazelle.jpg:

"query": "filename:Adidas-Gazelle.jpg"

This query will return all files containing Gazelle and exclude any folders from the results:

"query": "filename:Gazelle NOT(isDirectory)"

Search by folder name

This query will return the contents (files, folders, sub-folders) for any folder named products:

"query": "dirname:products"

This query will return files and folders specifically within the root folder named /products:

"query": "dirname.paths:\\/Products"

Search by date/time

You can set a date range to find files within a particular created, modified or accessed time: ctime, mtime or atime. Last accessed time has been tracked since March 22, 2023 and is updated on a delayed basis - requests from the last 8 days may not be registered.

The following example will search for all files in the /Products folder, created between 28 March and 3 April 2023, in UTC time (Z):

"query": "dirname:\/Products AND ctime:[2023-03-28T00:00:00.000Z TO 2023-04-03T23:59:59.999Z]"

The following example uses now to find all files modified from 1435hrs on 15 April 2023 up until now:

"query": "mtime:[2023-04-15T14:35:00.000Z TO now]"

You can set a relative time in either minutes m, hours h, days d, weeks w, months m or years y. The following example will return all files modified in the last 48 hours:

"query": "mtime:[now-48h TO now]"

The following example uses * to set any time. It will return all files modified before 1 January 2020:

"query": "mtime:[* TO 2020-01-01]"

Search by file size

You can search for files by their size, in bytes. The following query will return files between 5 MB and 10 MB:

"query": "size:[5000000 TO 10000000]"

Search by image dimensions

You can use meta.width and meta.height to search for images by their dimensions.

The following query will return images of exactly 1920px width and 1080px height:

"query": "meta.width:1920 AND meta.height:1080"

To search for files smaller than 2500px width:

"query": "meta.width:<2500"

To find files between 400 and 850px height:

"query": "meta.height:[400 TO 850]"

Search by meta

You can use any of the other meta data fields to search via meta content. This query will search for all JPEG or PNG images that do not have a meta description:

"query": "contentType:image\\/jpeg OR image\\/png NOT (meta.description:*)"

Boolean operators

You can use AND, OR and NOT boolean operators to narrow down the files you require.

AND

Create a very specific search using AND or double ampersand && to search upon a variety of requirements. For example, this will return all JPEG files modified in the last 14 days and not in the Trash folder:

"query": "contentType:image\\/jpeg AND mtime:[now-14d TO now] AND -dirname:\/.Trash"

OR

Create a broad search with OR or double pipe || to return results containing either term. For example, this will return all files containing the name black or white:

"query": "basename:black OR white"

NOT

You can exclude certain results with NOT or minus - or exclamation !. For example, this will show all results for "Gazelle" that are not a folder:

"query": "basename:Gazelle NOT -isDirectory:true"

Escaping special characters

You can use special characters in your search, as long as they are escaped with a double backslash \\ beforehand.

These special characters should be escaped:

{ } / \

For example to search for /My-folder/2022/studio.jpg use the query:

"query": "filename:\\/My-folder\\/2022\\/studio.jpg"

If your file path contains a backslash \, then it should be escaped by a single backslash, thus your query should have two backslashes \\.

Grouping

Multiple terms or clauses can be grouped together with parentheses, to form sub-queries.

This will search for files with Gazelle in the name and with either a .psd or .eps extension:

"query": "extension:(.psd OR .eps) AND basename:Gazelle"

This will search for PDF files modified in the last 14 days:

"query": "(extension:.pdf OR contentType:application\\/pdf) AND mtime:[now-14d TO now]"

Wildcards

If you need to find all files that contain a partial string of characters, you can use a wildcard to fill in the gaps.

• * - represents a string of characters
• ? - represents a single character

For example, this will return files starting "Adidas":

"query": "basename.raw:Adidas*"

Exclude hidden folders

Your Sirv account has some special hidden folders, which might contain files that appear in search results:

• /Shared
• /Profiles
• /.Trash
• /.processed

To exclude those folders from your search, use - and AND like so:

"query": "extension:.tif AND -dirname:\/Shared AND -dirname:\/Profiles AND -dirname:\/.Trash AND -dirname:\/.processed"

Sorting results

The sort parameter will determine the order of files in the response. The default order is ascending asc filenames. The following example will sort by basename.raw, with descending results desc:

{
 "query": "extension:.pdf AND -dirname:\/.Trash",
 "sort": {
  "basename.raw": "desc"
 }
}

You can sort by any of these fields:

• filename.raw
• basename.raw
• dirname.raw
• contentType.raw
• mtime
• ctime
• size

You can apply multiple sort parameters, to sort by one field then another. The following example sorts by content type and then modified time:

{
 "query": "mtime:[now-2y TO now] AND -dirname:\/.Trash",
 "sort": {
  "contentType.raw": "asc",
  "mtime": "desc"
 },
}

For more sorting options, refer to the Elasticsearch sorting documentation.

Setting the number of results

You can receive up to 100 results at a time in each response. The number of results can be set with the optional size parameter. If you don't set the parameter, up to 100 results will be returned.

If your query contains more than 100 results, you can fetch the other results with additional requests by setting the from parameter. The count starts from 0, so the first 100 results are 0-99. The following query will fetch the next 100 (from 100 to 199):

{
 "query": "ctime:[now-1y TO now] AND -dirname:\/.Trash",
 "sort": {
  "dirname.raw": "asc"
 },
 "size": 100,
 "from": 100
}

The from and size parameters let you retrieve up to 1000 results. If you need more than 1000 results, use a scrolling search, described below.

Scroll through more than 1000 results

If you need to retrieve more than 1000 search results, you can use scrolling search to return them in batches of up to 100 at a time.

Enable a scrolling search by setting scroll to true, for example:

{
 "query": "ctime:[now-1y TO now] AND -dirname:/.Trash",
 "sort": {
  "dirname.raw": "asc"
 },
 "size": 100,
 "from": 0,
 "scroll": true
}

At the end of the response, you will receive a scrollId, which will look like this:

"scrollId": "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAZyDGkWWjlzbkU3cEFTcTZtTjF4OWRpVTdEdw=="

Use that scrollId over and over again to request additional batches of results, using the scroll API method.

The complete list of search results will be cached by Sirv for 20 minutes, then deleted, so keep submitting the scroll API requests until you have saved all the results locally. Use your locally saved search results to analyse the data as you wish.

A scrolling search represents a snapshot of results at that exact point in time. Any changes made to your files after that point (upload, delete, rename, move etc.) will not be reflected in the cached search results.

Query length

A search query string can contain up to 1024 characters. This lets you combine many AND, OR and NOT parameters in a single request.

Example scripts

The REST API search documentation contains an example payload, JSON schema, response and scripts for 10 popular programming languages:

• cURL search script
• Python search script
• Node.js search script | more Node.js scripts
• JavaScript search script
• Java search script
• PHP search script
• Ruby search script
• Swift search script
• Go search script
• C# search script

Example response

This example response shows you the file information, including all the searchable parameters and also some of the most important file meta.

The end of the response shows the total number of results (1 in this case):

{
  "hits": [
    {
      "_index": "sirvfs-v3",
      "_type": "_doc",
      "_id": "d575e090616438ded527bb40da3bfa0c6d5f05ff",
      "_score": null,
      "_routing": "sdulth0oi0t9zxpxqtxwkwvgipjgv6ud",
      "_source": {
        "accountId": "sdulth0oi0t9zxpxqtxwkwvgipjgv6ud",
        "filename": "/Products/trainers/Gazelle/Adidas-Gazelle.jpg",
        "dirname": "/Products/trainers/Gazelle",
        "basename": "Adidas-Gazelle.jpg",
        "extension": ".jpg",
        "id": "Qw5Z9DNgvwWRXWF8WEHlfpI6c3Doyx6w",
        "ctime": "2021-02-12T17:01:01.333Z",
        "mtime": "2021-02-12T17:01:01.446Z",
        "size": 1309461,
        "contentType": "image/jpeg",
        "meta": {
          "width": 3000,
          "height": 2400,
          "format": "JPEG",
          "duration": 0,
          "EXIF": {
            "DateTimeOriginal": "2019-06-19T00:11:38Z",
            "CreateDate": "2019-06-19T00:11:38Z",
            "ModifyDate": "2019-06-19T13:45:44Z",
            "ColorSpace": "Uncalibrated",
            "Make": "Canon",
            "Model": "Canon EOS 5D Mark II",
            "LensModel": "Canon EF 24-105mm f/4L IS USM"
          }
        }
      },
      "sort": [
        "/Products/trainers/Gazelle/Adidas-Gazelle.jpg"
      ]
    }
  ],
  "total": 1,
  "_relation": "eq"
}

Ask for help

If you need help with the search API, send a message to the Sirv support team. Describe your requirements and share a sample of your script, if possible.