Getting data from behind a login

In this article we will explain how to get data from a page when it sits behind a login.

Important: This option is only available for paid subscriptions

To extract data that is only available after logging in to a website is very simple. All we need is:

• The URL to log in to the website.
• A set of valid credentials (username and password)
• The URL from which we want to extract the data.

Once we have these items ready all we have to do is follow these simple steps:

Step 1. Go to the dashboard and create a new extractor

Step 2. Under the URL select the option Does the website require login and enter the login URL and valid credentials

Step 3. Select Go to trigger the login mechanism in the background

Step 4. After the login is completed check the screen capture that appears and confirm the login has been successful. This is important since otherwise data might not be available to extract later on.

Step 5. Train your extractor and save it to extract data from it

Step 6. Run the extractor and enter the valid credentials one more time

That's it! You should be able to download the data as with any other extractor.

Troubleshooting

I cannot see the authenticated page after selecting Go.

It could be the case that the authentication does not use a standard login form (e.g.: works on a modal window). These login types are not supported at the moment.

I don't get data and I see a "data not found" error in the logs when running my extractor

This means your credentials are invalid. Please double check your credentials are valid before running the extractor.