Scheduling your extractor

You can schedule your extractor to extract data from a website on a regular basis. You can set up schedules to keep your database or application updated with the latest information to monitor changes in data over time or just because you systematically need to get data from the web. You can set your extractor to run on an hourly, daily, weekly, or monthly basis and also write your own schedules using the Custom mode.

Important: This option is only available for paid subscriptions

To access the Scheduler Go to the Settings tab on your extractor and look for the "Schedule your Extractor" section at the bottom:

Hourly

Set your schedule to Hourly to run your extractor at a certain minute on every hour. Just pick a time and hit OK.

Daily

Set your schedule to Daily to run your extractor every day at a certain time of the day. Just pick a time and hit OK.

Weekly

Set your schedule to Weekly to run your extractor every week at a certain time of any given day. Just pick a time and hit OK.

Monthly

For those bigger datasets that only need to be updated once a month. Schedule will allow you to choose a start date and time for your extraction.

Advanced

Advanced scheduling brings maximum control when it comes to schedule your extractors, giving you full control on when your extractors will run, from months and days of the week up to the minute.

• Run every hour between 9am and 6pm on Monday, Wednesday and Friday
• Run once a day during the months of Jan, Feb and March
• or Run three times a week at 9:00am

This is done in this box. Which can be selected by changing the setting of your extractor scheduling to advanced.

When the value in the box is star it means the extractor will run at any of the times in that box. If there are all stars, the extractor would attempt to run every minute. A good way of quickly using your monthly quota. Instead let us say you wish to set it up so that it run only at 7:00 am each day. Then simply fill the minute box with 0 and the hour box with 7.

Of course this could setup without the advanced scheduler, but lets say you want to run the extractor at 5:00 and 7:00. We can select multiple values by using a comma, as shown below:

So this will extract every day at 5:00 and 7:00. All of the columns can be filled in the same way. For example, if you want to extract Mondays and Sunday of February at 5:00 and 7:00. You would input:

At the top of the box, when the extractor will run is always indicated. At the bottom of the box, the accepted values and operators for the box you are currently changing are listed.

Step values allows you to skip a day, for example starting from Monday you would input 1/2, in the day box.

This advanced scheduling can enable you to choose exactly when your extractors run. You can now update your extractors more often or less often, all based on when you wish to extract the data.

Tips and Tricks

It's important to know that ranges to not roll over so you have to be careful when using boundary values: