Expertrec crawler allows some of the advanced options while crawling. These features mainly include:

Protected Pages:

Expertrec crawler asking login

You may need to search web pages that are protected such as user needs to login to see the content. Expertrec crawler can be guided with credentials to index these pages.

Expertrec crawler login details

As shown above, login_url will be the page url where user needs to put the login credentials. login_form_id is the html document id for this signin form( id given to the form).