Google custom search JSON API is a RESTFUL API that lets you develop applications to get and show search results (web search, images) from Google custom search programmatically. With this API, you can use RESTful requests to get either web search or image search results in JSON format.

The existing documentation on creating a Google custom search using the API method is confusing and we will try to simplify the process here. If you are not a big fan of Google custom search, you can try out other expertrec’s custom search engine.

In this article, we will see how we can create a custom search engine using the Google custom search API. We will also see what are the limitations of Google custom search API and how we can overcome them.

Create your custom search engine using expertrec

 

Custom Search JSON API

The Custom Search JSON API lets you develop websites and applications to retrieve and display search results from Programmable Search Engine programmatically. With this API, you can use RESTful requests to get either web search or image search results in JSON format.

Prerequisites

First Step- Create a custom search engine

This is the first step to obtain a Google custom search API. Go to cse.google.com, enter your website URL, and click on create Google custom search. If you would like to create a custom search engine on multiple URLs, have a list of URLs ready with you in a Google doc or excel sheet to readily copy paste.

The Custom Search JSON API lets you develop websites and applications to retrieve and display search results from Google custom search programmatically. With this API, you can use RESTful requests to get either web search or image search results in JSON format. With these results, you can build your own search front interface with the backend linked to Google custom search.

Get your Search engine ID

Before using the Custom Search JSON API you will first need to create and configure your Google custom search engine. If you have not already created a Google custom search engine, you can start by visiting the Google custom search engine control panel.

After you have created a Google custom search engine, visit the instruction towards the end of this blog to learn how to locate your Search engine ID.

Get your custom search API key

Custom Search JSON API requires the use of an API key. The API key uniquely identifies your API usage and make sure you don’t reveal your API key on any external forums as it might attract misuse. You can get your API key from here.

JSON API pricing

Custom Search JSON API provides 100 search queries per day for free. If you need more, you may sign up for billing in the API Console. Additional requests cost $5 per 1000 queries, up to 10k queries per day. If your site has 10,000 search queries in a day, it will cost 50$ per day. This is a major disadvantage of the Google custom search API, especially if you have significant traffic. If you fall under this category, you can sign up for expertrec custom search engine that is priced on the number of pages in your website.

If you need more than 10k queries per day and your Programmable Search Engine searches 10 sites or fewer, you may be interested in the Custom Search Site Restricted JSON API, which does not have a daily query limit. (the only catch with this is that you can search only Upton 10 websites).

JSON API Monitoring

Monitoring your API usage is a very useful feature that you should consider (for example when your search goes down, you can get an email alert).

Basic monitoring for the Custom Search JSON API is available through Cloud Platform Console’s API Dashboard. For more advanced monitoring Google Cloud’s Operations suite (formerly Stack driver) is available.

With Google Cloud Operations you can create custom dashboards, set up alerts, and access metrics data programmatically. To access Custom Search JSON API usage data in Google Cloud Operations, select “Resource type: Consumed API” and filter on “service = ‘customsearch.googleapis.com'” in the Query Builder.

See Monitoring Your API Usage for a discussion of the different monitoring and alerting capabilities provided by the API Dashboard and the Google Cloud Operations suite.

Google custom search JSON API parameters

You can retrieve results for a particular search by sending an HTTP GET request to its URI. You pass in the details of the search request as query parameters. The format for the Custom Search JSON API URI is:

The usual API call is

https://www.googleapis.com/customsearch/v1?parameters

Three query [parameters] are required with each search request:

  • API key – Use the key query parameter to identify your application.
  • Programmable Search Engine ID – Use cx to specify the Programmable Search Engine you want to use to perform this search. The search engine must be created with the Control Panel Note: The Search Engine ID (cx) can be of different format (e.g. 8ac1ab64606d234f1)
  • Search query – Use the q query parameter to specify your search expression.

All other query parameters are optional.

Here parameters could be –

  1. q– query string
  2. cx– Custom search engine ID.
  3. file Type– BMP, gif, jpg, ping, PDF etc
  4. num-number of search results to display
  5. searchType- search type.
  6. sort-Sort by functionality to use.

For a list of all query parameters, go to this link.

Google custom search JSON API simplified

A sample API call would be like this-

https://www.googleapis.com/customsearch/v1?key={YOUR_API_KEY}&cx={CUSTOM_SEARCH_ENGINE_ID}&q={KEYWORD}
https://www.googleapis.com/customsearch/v1?key=AIzaSyDOvbipdULbUpVm3HLjoX7bEEfDUKHELh4&cx=012278024424817052234:cimrvj4h4uk&q=jeff

The response of this API will be in JSON format for you to consume to build your search application.

{
   "kind": "customsearch#result",
   "title": "Jeff Pedowitz - President and CEO - The Pedowitz Group | LinkedIn",
   "htmlTitle": "\u003cb\u003eJeff\u003c/b\u003e Pedowitz - President and CEO - The Pedowitz Group | LinkedIn",
   "link": "https://www.linkedin.com/in/jeffpedowitz",
   "displayLink": "www.linkedin.com",
   "snippet": "View Jeff Pedowitz's full profile. It's free! Your colleagues, classmates, and 500 \nmillion other professionals are on LinkedIn. View Jeff's Full Profile ...",
   "htmlSnippet": "View \u003cb\u003eJeff\u003c/b\u003e Pedowitz's full profile. It's free! Your colleagues, classmates, and 500 \u003cbr\u003e\nmillion other professionals are on LinkedIn. View \u003cb\u003eJeff's\u003c/b\u003e Full Profile ...",
   "formattedUrl": "https://www.linkedin.com/in/jeffpedowitz",
   "htmlFormattedUrl": "https://www.linkedin.com/in/\u003cb\u003ejeff\u003c/b\u003epedowitz",
   "pagemap": {
    "cse_thumbnail": [
     {
      "width": "276",
      "height": "183",
      "src": "https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcT00CLj3qk3NDZp35KPvo4A2kaXF-FrhY7DwtGru9ikLSV2LDmaKneWJT2Z"
     }
    ],

Query parameters

There are two types of parameters that you can pass in your request:

  • API-specific parameters – define properties of your search, like the search expression, number of results, language etc.
  • Standard query parameters – define technical aspects of your request, like the API key.

All parameter values need to be URL encoded.

API-specific query parameters

Request parameters that apply specifically to the Custom Search JSON API and define your search request are summarized in the reference.

Standard query parameters

Query parameters that apply to all Custom Search JSON API operations are documented at System Parameters.

Response data

If the request succeeds, the server responds with a 200 OK HTTP status code and the response data in JSON format. You can look up the response data structure in the reference.

The response data is a JSON object that includes three types of properties:

  • Metadata describing the requested search (and, possibly, related search requests)
  • Metadata describing the Programmable Search Engine
  • Search results

 

Search request metadata

The search metadata includes:

  • url property, which has information about the OpenSearch template used for the results returned in this request.
  • queries property, which is an array of objects describing the characteristics of possible searches. The name of each object in the array is either the name of an OpenSearch query role or one of the two custom roles defined by this API: previousPage and nextPage . Possible query role objects include:
    • request: Metadata describing the query for the current set of results.
      • This role is always present in the response.
      • It is always an array with just one element.
      • nextPage: Metadata describing the query to use for the next page of results.
        • This role is not present if the current results are the last page. Note: This API returns up to the first 100 results only.
        • When present, it is always a array with just one element.
    • previousPage: Metadata describing the query to use for the previous page of results.
      • Not present if the current results are the first page.
      • When present, it is always a array with just one element.

Search engine metadata

The context property has metadata describing the search engine that performed the search query. It includes the name of the search engine, and any facet objects it provides for refining a search.

Search results

The items array contains the actual search results. The search results include the URL, title and text snippets that describe the result. In addition, they can contain rich snippet information, if applicable.

If the search results include a promotions property, it contains a set of promotions

 

Other Info

 

Getting your Google custom search API Key

  1. Go to https://developers.google.com/custom-search/json-api/v1/overview
  2. Click on get a key.Google custom search JSON API simplified
  3. Click create new project.Google custom search JSON API simplified
  4. This will generate your new API key.Google custom search JSON API simplified
  5. You can restrict the usage of this API key in your Google API console https://console.developers.google.com/apis
Google custom search JSON API simplified

Getting your CX / Search engine ID:

Go to your custom search / site search control panel-> edit search engine->Basics->Details->Search engine ID.

This is your search engine ID.

Google custom search JSON API

Testing your Google custom search JSON API-

You can test your Google custom search API using the Google API explorer 

google custom search apiYou could try out ExpertRec’s site search API, which is easier to implement with no coding needed at all. Expertrec’s site search API is also available if in case you want to go through the API route.

Limitations of Google custom search API-

  1. You can only retrieve a maximum of 10 search results at a time.
  2. Query limit of 100 per day.

Overcoming the limitations of the API

To over come the limitations of the Google custom search API, we have built a custom search engine at expertrec that is priced not based on the number of queries but the number of pages. If your website has 2000 pages, you can create a custom search engine with expertrec at just $9 per month.

Go to https://cse.expertrec.com?platform=cse  and enter your website and sitemap URLs to get your crawl done. Go and click on your demo link.

Get your expertrec site search API

Go to https://cse.expertrec.com/csedashboard/home/Status to get your API keygoogle custom search api

 

The API will be of this format-

https://searchv7.expertrec.com/v6/search/{org_id}/?q={keyword}&page={pagination}&size={number_of_search_results}
https://searchv7.expertrec.com/v6/search/ccdb9cb6-5380-11e8-a8e3-12b6486824f4/?q=a&page=0&size=16

Expertrec site search API parameters-

  1. org_id- Your unique ID that identifies your website/ websites.
  2. q- Keyword.
  3. page- pagination
  4. size- number of search results to be displayed.

The results will be in JSON format-

{  
   "0len":1,
   "1query":"a",
   "facet_interval":{  

   },
   "facets":{  
      "type":{  
         "webpage":160
      }
   },
   "fq":{  

   },
   "nf":{  

   },
   "numeric":{  

   },
   "res":{  
      "count":160,
      "st":0,
      "time":3
   },
   "results":[  
      {  
         "date":"2018-05-09T12:35:47.110Z",
         "host":"blog.expertrec.com",
         "id":"https://blog.expertrec.com/wordpress-custom-search/",
         "image":[  
            "https://blog.expertrec.com/wp-content/uploads/2018/04/6.png"
         ],
         "popularity_score":1.0,
         "score":1.25,
         "snippet":{  
            "content":[  
               "How to <span class=highlight>create</span> <span class=highlight>a</span> wordpress custom <span class=highlight>search</span> - 2018 Expertrec
 
Support
<span class=highlight>magento</span>
<span class=highlight>search</span> engine
Google site <span class=highlight>search</span> <span class=highlight>replacement</span>
Expertec
Toggle <span class=highlight>Navigation</span>
How to <span class=highlight>create</span> <span class=highlight>a</span> wordpress custom <span class=highlight>search</span>
Published by <span class=highlight>muthali</span> <span class=highlight>ganesh</span> on <span class=highlight>April</span> 8, 2018
<span class=highlight>Creating</span> wordpress custom <span class=highlight>search</span> using expertrec is <span class=highlight>as</span> <span class=highlight>easy</span> <span class=highlight>and</span> <span class=highlight>can</span> be <span class=highlight>taken</span> live in 5 minutes. "
            ],
            "metatag.description":[  

            ],
            "title":[  
               "How to <span class=highlight>create</span> <span class=highlight>a</span> wordpress custom <span class=highlight>search</span> - 2018 Expertrec"
            ]
         },
         "title":"How to create a wordpress custom search - 2018 Expertrec",
         "type":"webpage",
         "url":"https://blog.expertrec.com/wordpress-custom-search/"
      }
   ],
   "sfacets":{  
      "type":[  
         {  
            "count":160,
            "name":"webpage"
         }
      ]
   }
}

 

Create your expertrec custom search engine

muthali ganesh

Muthali loves writing about emerging technologies and easy solutions for complex tech issues. You can reach out to him through chat or by raising a support ticket on the left hand side of the page.

You may also like