Are you someone looking to search multiple sites on a regular basis? You could be a deal hunter looking to search multiple websites for the best price deals or a broker trying to find out the best house deals in your area. It would be practically impossible to visit each of these websites every day and find the best price. Creating a custom search engine could solve this issue. In this article, we will see various methods of creating a custom search engine that can help you search multiple websites.
Using Expertrec custom search engine ($9 per month)
ExpertRec adds a Google-like search box to your website using which you can search for multiple websites… It is priced at 9 USD per month. The biggest advantage is that it doesn’t contain Ads.
Before starting out to make a custom search engine using expertrec, make sure that the sites you are trying to search and are live and not behind a login.
- Go to https://cse.expertrec.com/?platform=cse.
- Login with your Gmail id.
- Enter your website URL. (enter your website URL in the format https://www.yourwebsite.com).
- Enter your sitemap URL. ( if you don’t have a sitemap, you can skip this step).
- Go to Crawl->What to crawl->URLs to add more URLs.
- Here you can add as many URLs as you like.
- Initiate a website crawl. (Now Expertrec’s crawler will start crawling your website. This could take some time depending on the size and number of pages on your website).
- Once the crawl is completed, you can make changes to your search engine’s look and feel and also to the search engine’s ranking algorithm by using the control panel. If in case you want certain websites to rank higher than others, you can use this option. There are more advanced features like search weights, promotions, synonyms, and more.
- the code contains two portions ( one within <script></script> and another piece of code <ci-search></ci-search>).
- The <ci-search></ci-search> code adds the search box to your website. The place where you add this code determines the location of the search box. Go to the code section and add it to your website. You can skip this step if you just want to have the search demo and not add the search engine to any website.
- It is advisable to make the code changes in a test mode or development website before taking it live on your live/ production site.
- As a final step, remove any website cache that you might have.
- If you would like to crawl the list of sites more frequently, you can go to recrawl frequency and adjust the crawl rate according to your need. Please note that faster crawl speeds are available only in the premium plans.
Advantages of Using Expertrec for searching multiple sites
- Control over which pages are crawled– You have very fine-tuned control over what is crawled. For example, you could only crawl pages that have /products/ in the URL.
- No Ads. As compared to Google custom search, Expertrec shows zero ads in their search bar.
- Affordable pricing plan. – Prices starting at $9 per month.
- Real-time support to help you take to make adjustments to your search algorithm.
Using Google custom search
Using Google custom search, you will be able to search multiple websites, however, this number(of websites to search from) is limited to 10. If your requirement is to search more than 10 websites, have a look at searching the entire web using Google custom search or expertrec custom search engine (understandable, since If Google allowed everyone to search unlimited websites, it will be easier to launch a Google alternative).
- Sign in to your Google custom search Control Panel using your Google Account (get an account if you don’t have one).
- In the Sites to search section, add the pages you want to include in your search engine. You can include any sites you want, not just the sites you own. You can include whole site URLs or individual page URLs. You can also use URL patterns. Here go to add and click on ” add sites in bulk.
- Enter a list of sites one per line.
- The name of your search engine will be automatically generated based on the URLs you select. You can change this name at any time.
- Select the language of your search engine. This defines the language of the buttons and other design elements of your search engine but doesn’t affect the actual search results.
- Click Create.
Your basic search engine that searches through multiple sites is ready to use! To see more configuration options, go to the Control Panel. The control panel also has a preview window that allows you to test and fine-tune your search results.
Using the OR operator inside Google search
To search on Google and limit the results to some websites follow the steps below:
- Go to Google and type your search as usual.
- Before firing the search, add site:site1 OR site:site2, etc as many sites as you want.
- Now do the search and see that the results have now been limited to site1 and site2.
- You can also use the pip symbol “|” to separate the websites.
Did you know that Duckduckgo has a custom search engine similar to Google custom search?
To search multiple websites using this, go to https://duckduckgo.com/search_box and enter the site URLs separated by a comma under the site search section. If you have a list of URLs in an excel sheet, you can use the concatenate function to append a comma at the end of each URL.
Under site search, enter the website URLs separated by commas. If you have the list of websites in an excel sheet in separate rows, use the concatenate function to append a comma to every URL. This will save a whole lot of time.
Using Yandex site search
In the first step, you define the search area (i.e., the set of pages covered by your search), search names, and other general properties. The only disadvantage of this method is that you won’t be able to enter more than 50 sites to search. If you are looking for more than 50 sites, try expertrec custom search engine.
Here are steps to follow to search multiples sites using Yandex:
- Go to yandex site search creator.
- Enter the technical name of your search.
- Enter Search name in the Open search standard –This is the name your visitors can use to add your search as a browser search engine.
- Add a family filter To exclude pages containing obscenities and pornography from the search area, set up a family filter. The filter settings are similar to the corresponding big search settings.
- Add a search area Specify the search area, i.e., a list of sites to include in the search. You can specify up to 50 addresses of sites or their sections.
- Click add sites and exclusions List the addresses of sites or their subsections space-separated.
Other Honorable mentions:
- Bing site search.
- Amazon cloudsearch.