Add image alt tags automatically with AI

Image alt tags are important for both accessibility and SEO. But writing alt tags and adding them to images can be quite a chore. What if there was a way to automate this process?

Luckily, image recognition services like Azure's Computer Vision have come a long way. Let's leverage the power of AI and a little bit of coding magic to automatically add alt tags to your images.

Adding alt tags with Sirv

All images hosted at Sirv have a special meta description field. It can be used as the alt text source in various Sirv components.

The description is used as the alt text in the Sirv web app.

To automatically add alt tags to images, we're going to use Azure Computer Vision, an AI image recognition tool. Then, we'll populate Sirv image descriptions with data returned by Azure Computer Vision. This comes with an added bonus of better search results in the Sirv app, because image descriptions are also searchable.

Here's how this can work with Sirv.

Auto alt tags in Sirv responsive images

Sirv automatically adds alt tags (if their value is empty) to responsive images. To maximize SEO benefits, Sirv responsive images are delivered in the most optimal file format and dimensions, which makes them load incredibly fast.

Sirv Media Viewer

Image alt tags are also automatically populated in gallery images in Sirv Media Viewer. This makes managing alt tags for e-commerce sites a breeze. This works great with our plugins for WordPress, WooCommerce, Magento and PrestaShop.

Getting started

To get started, we'll need API keys from Azure and Sirv.

Sign up for Azure Computer Vision

1. Sign up for a free Azure account.
2. Add Computer Vision
3. Create a Computer Vision instance
4. After your deployment is complete, save your API key and the endpoint address.
5. Set up Azure's Computer Vision SDK for the language of your choice. We'll be using Python as an example.

Get Sirv REST API keys

1. Sign up for Sirv if you haven't already.
2. Create a new REST API client from yor account settings.
3. Save your clientId and client secret keys.

Add an alt tag to an image automatically

Once you have the Azure Computer Vision SDK installed and all credentials saved, it's time for the good stuff. We'll use Python for this example.

1. Add required libraries

Create a new python file - 'autoalt.py', for example. Then open it in your favorite editor and add the required libraries.

from azure.cognitiveservices.vision.computervision import ComputerVisionClient
from azure.cognitiveservices.vision.computervision.models import OperationStatusCodes
from azure.cognitiveservices.vision.computervision.models import VisualFeatureTypes
from msrest.authentication import CognitiveServicesCredentials
from array import array
import os
import json
import requests
from PIL import Image
import sys
import time
from urllib.parse import urlparse

2. Set up variables

Then, add Computer Vision and Sirv API keys as variables:

# Your subscription key and endpoint
subscription_key = "YOUR AZURE SUBSCRIPTION KEY"
endpoint = "YOUR AZURE ENDPOINT URL, example - https://we.cognitiveservices.azure.com/"

#Sirv credentials
sirv_id = "YOUR SIRV CLIENT ID HERE"
sirv_secret = "YOUR SIRV CLIENT SECRET HERE"

#The image we're going to get a description for
remote_image_url = "https://demo.sirv.com/leopard.jpg"

We've also included an image URL under the 'remote_image_url' variable.

3. Authenticate

Let's create a Computer Vision client and get Sirv's authentication token.

#Create Azure client
computervision_client = ComputerVisionClient(endpoint, CognitiveServicesCredentials(subscription_key))

# Get Sirv auth token. API reference - https://apidocs.sirv.com/#connect-to-sirv-api
payload = {
  'clientId': sirv_id,
  'clientSecret': sirv_secret
}

headers = {'content-type': 'application/json'}
response = requests.request('POST', 'https://api.sirv.com/v2/token', data=json.dumps(payload), headers=headers)
token = response.json()['token']

Great, now we can finally interact with both APIs.

4. Get the image description and update it in Sirv

Here's how we can generate an image description:

# Create a Computer Vision client
description_results = computervision_client.describe_image(remote_image_url )

# Get the captions (descriptions) from the response, with confidence level
print("Description of remote image: ")
if (len(description_results.captions) == 0):
  print("No description detected.")
else:
  for caption in description_results.captions:
    print("'{}' with confidence {:.2f}%".format(caption.text, caption.confidence * 100))
    description = '{"description": "'+ caption.text + '"}'
    '''
    Update the image description in Sirv.
    API reference - https://apidocs.sirv.com/#set-meta-description
    '''
    # Grabbing the image path as the filename.
    params = {"filename": urlparse(remote_image_url).path}
    headers = {
      'content-type': "application/json",
      'authorization': 'Bearer %s' % token
    }
    response = requests.request('POST', 'https://api.sirv.com/v2/files/meta/description', data=description.encode('utf-8'), headers=headers, params=params)
    print(response)

Optionally, you can add a confidence check, to only update image description if the confidence level is high enough.

It'll look like this:

# Create a Computer Vision client
description_results = computervision_client.describe_image(remote_image_url )

# Get the captions (descriptions) from the response, with confidence level
print("Description of remote image: ")
if (len(description_results.captions) == 0):
  print("No description detected.")
else:
  for caption in description_results.captions:
    print("'{}' with confidence {:.2f}%".format(caption.text, caption.confidence * 100))
    description = '{"description": "'+ caption.text + '"}'

    # Change the number 60 here to a desired confidence percentage level.
    if (caption.confidence * 100 > 60):
      '''
      Update the image description in Sirv.
      API reference - https://apidocs.sirv.com/#set-meta-description
      '''
      # Grabbing the image path as the filename.
      params = {"filename": urlparse(remote_image_url).path}
      headers = {
        'content-type': "application/json",
        'authorization': 'Bearer %s' % token
      }
      response = requests.request('POST', 'https://api.sirv.com/v2/files/meta/description', data=description.encode('utf-8'), headers=headers, params=params)
      print(response)
    else:
      print("no captions with high enough confidence level detected")

Here's the result:

We're using Sirv responsive imaging here, the image is automatically resized, the alt is populated from the description and the image is delivered in WebP for supported browsers.

You can also check out the image meta by adding '?info' to the URL, like this - https://demo.sirv.com/leopard.jpg?info

5. Putting it all together

The full code looks like this:

  from azure.cognitiveservices.vision.computervision import ComputerVisionClient
  from azure.cognitiveservices.vision.computervision.models import OperationStatusCodes
  from azure.cognitiveservices.vision.computervision.models import VisualFeatureTypes
  from msrest.authentication import CognitiveServicesCredentials
  from array import array
  import os
  import json
  import requests
  from PIL import Image
  import sys
  import time
  from urllib.parse import urlparse

  # Your subscription key and endpoint
  subscription_key = "YOUR AZURE KEY"
  endpoint = "https://we.cognitiveservices.azure.com/"

  #Sirv credentials
  sirv_id = "Your CLIENT ID"
  sirv_secret = "YOUR CLIENT SECRET"


  #The image we're going to get a description for
  remote_image_url = "https://demo.sirv.com/leopard.jpg"

  #Create Azure client
  computervision_client = ComputerVisionClient(endpoint, CognitiveServicesCredentials(subscription_key))

  # Get Sirv auth token. API reference - https://apidocs.sirv.com/#connect-to-sirv-api
  payload = {
    'clientId': sirv_id,
    'clientSecret': sirv_secret
  }

  headers = {'content-type': 'application/json'}
  response = requests.request('POST', 'https://api.sirv.com/v2/token', data=json.dumps(payload), headers=headers)
  token = response.json()['token']

  # Create a Computer Vision client
  description_results = computervision_client.describe_image(remote_image_url )

  # Get the captions (descriptions) from the response, with confidence level
  print("Description of remote image: ")
  if (len(description_results.captions) == 0):
    print("No description detected.")
  else:
    for caption in description_results.captions:
      print("'{}' with confidence {:.2f}%".format(caption.text, caption.confidence * 100))
      description = '{"description": "'+ caption.text + '"}'
      '''
      Update the image description in Sirv.
      API reference - https://apidocs.sirv.com/#set-meta-description
      '''
      params = {"filename": urlparse(remote_image_url).path}
      headers = {
        'content-type': "application/json",
        'authorization': 'Bearer %s' % token
      }

      response = requests.request('POST', 'https://api.sirv.com/v2/files/meta/description', data=description.encode('utf-8'), headers=headers, params=params)
      print(response)

Adding image alt tags in bulk

Realistically, you'd want to add alt tags to images automatically in bulk. To pull this off, we need to get image URLs from your Sirv account. There are several ways to do this.

Export images as a CSV file

For the sake of simplicity, let's export your Sirv images using the web app.

Save image urls as a separate text file (we'll name it 'images.txt') and drop it in the same folder where the script is located.

Then we can loop through each image to describe it and populate the Sirv description field.
Here's the full code:

from azure.cognitiveservices.vision.computervision import ComputerVisionClient
from azure.cognitiveservices.vision.computervision.models import OperationStatusCodes
from azure.cognitiveservices.vision.computervision.models import VisualFeatureTypes
from msrest.authentication import CognitiveServicesCredentials
from array import array
import os
import json
import requests
from PIL import Image
import sys
import time
from urllib.parse import urlparse

# Your subscription key and endpoint
subscription_key = "COMPUTER VISION KEY"
endpoint = "https://we.cognitiveservices.azure.com/"
#Sirv credentials
sirv_id = "YOUR SIRV CLIENT ID"
sirv_secret = "YOUR SIRV CLIENT SECRET"

#Create Azure client
computervision_client = ComputerVisionClient(endpoint, CognitiveServicesCredentials(subscription_key))

# Get Sirv auth token. API reference - https://apidocs.sirv.com/#connect-to-sirv-api
payload = {
  'clientId': sirv_id,
  'clientSecret': sirv_secret
}

#We're opening the csv file and looping through each image in the file
with open('images.txt') as f:
  for image in f:
    headers = {'content-type': 'application/json'}
    response = requests.request('POST', 'https://api.sirv.com/v2/token', data=json.dumps(payload), headers=headers)
    token = response.json()['token']

    # Create a Computer Vision client
    description_results = computervision_client.describe_image(image)

    # Get the captions (descriptions) from the response, with confidence level
    print("Description of remote image: ")
    if (len(description_results.captions) == 0):
      print("No description detected.")
    else:
      for caption in description_results.captions:
        print("'{}' with confidence {:.2f}%".format(caption.text, caption.confidence * 100))
        description = '{"description": "'+ caption.text + '"}'
        '''
        Update the image description in Sirv.
        API reference - https://apidocs.sirv.com/#set-meta-description
        '''
        params = {"filename": urlparse(image).path.replace('n', '')}
        headers = {
          'content-type': "application/json",
          'authorization': 'Bearer %s' % token
        }

        response = requests.request('POST', 'https://api.sirv.com/v2/files/meta/description', data=description.encode('utf-8'), headers=headers, params=params)
        print(response)
        print('4 second pause')
        time.sleep(4)

Getting a list of files from your Sirv account via API

You can get a full list of files from a specific folder using this endpoint.
For more advanced use cases, you can use Sirv's search API endpoint to perform more sophisticated file searches, like "get all .jpg images added in the last 30 days from the 'products' folder".