Haven OnDemand Dropbox Cloud Connector Quick Start
Index websites with the Dropbox Cloud Connector

Use the Dropbox Cloud Connector

Connectors allow you to ingest documents into Haven OnDemand text indexes from various sources.

The Dropbox Cloud Connector, lets you index everything in a specified Dropbox account, or create an App folder into which you manually copy only those files you want the Connector to process.

Dropbox: Creating the Application

Before configuring your Dropbox Cloud Connector, you must first log into the target Dropbox account and create an Application there.

The Application you create provides a token for the Connector to use while authenticating. It also sets whether the Connector can access the entire Dropbox, or only a folder. When you configure the Connector, you need to reflect this setting.

Create the Connector

Call the Create Connector API. Calling this API creates a Connector of the type of your choice.

The required parameters are:

  • connector : The name of your Connector. You use this name for further updates, status checks, and deletion of the Connector.

  • flavor : The type of Connector you want to use. For Dropbox Connector, this value is dropbox_cloud

  • config : This JSON parameter contains configuration relevant to the type of Connector.

  • destination : This JSON parameter specifies the text index into which you want to add files.

Example of a Dropbox Cloud Connector Configuration

/createconnector/v1?connector=havenondemand&flavor=dropbox_cloud
config={ "full_dropbox_access": false } 
destination={ "action" : "addtotextindex", "index" : "hodbox" }

This configuration creates a Dropbox Cloud Connector that indexes only the content of an App Folder into the hodbox text index.

Operate the Connector

Start and Stop a Connector Manually

The Start Connector API takes a connector name, and starts a crawl.

/startconnector/v1?connector=havenondemand

The Stop Connector API can stop a run.

Check Status and History

You can use the Connector Status API to check whether the connector is running.

/connectorstatus/v1?connector=havenondemand
{
  "connector": "havenondemand",
  "status": "PROCESSING",
  "token": "MTI3LjAuMS4xOjcyMDA6RkVUQ0g6LTEzMjcwNTE0NTU=",
  "queued_time": "24/04/2015 17:11:05 +00:00",
  "time_in_queue": 0,
  "process_start_time": "24/04/2015 17:11:05 +00:00",
  "time_processing": 0,
  "document_counts": {}
}

You can use the Connector History API to get a history of all activity on the Connector, with a range of convenient filters that you can set.

{
  "history": [
    {
      "connector": "havenondemand",
      "status": "FINISHED",
      "document_counts": {
        "errors": 0
      },
      "queued_time": "08/08/2016 05:22:26 +00:00",
      "time_in_queue": 0,
      "process_start_time": "08/08/2016 05:22:26 +00:00",
      "time_processing": 2,
      "process_end_time": "08/08/2016 05:22:28 +00:00",
      "start_time": "08/08/2016 05:22:26 +00:00",
      "token": "MTAuOC4xNi4xMDo3MjEwOkZFVENIOi0xODE5MDgwMjE1"
    },
    {
      "connector": "havenondemand",
      "status": "FINISHED",
      "document_counts": {
        "added": 2,
        "errors": 0,
        "ingest_added": 1,
        "ingest_failed": 1
      },
      "queued_time": "07/08/2016 23:21:50 +00:00",
      "time_in_queue": 0,
      "process_start_time": "07/08/2016 23:21:50 +00:00",
      "time_processing": 510,
      "process_end_time": "07/08/2016 23:30:20 +00:00",
      "start_time": "07/08/2016 23:21:50 +00:00",
      "token": "MTAuOC4xNi4xMDo3MjEwOkZFVENIOi04MDY3Njk3ODA="
    }

Start a Connector Run to Reindex a Site from Scratch

By default, the Connector crawls and indexes only new and modified pages. You can use the ignore_previous_state parameter to force the Connector to ignore previous runs, and index the site as though it was a first run. For example:

/startconnector/v1?connector=havenondemand&ignore_previous_state=true

Update the Connector Configuration

After you create a Connector, you might want to change the configuration.

The Retrieve Config API takes a Connector name, and returns its configuration.

/retrieveconfig/v1?connector=havenondemand

After you check the configuration, you can change it by using the Update Connector API.

Note:As a config object can be quite large, Haven OnDemand recommends using a POST method with Update Connector requests.

 POST --form "connector=havenondemand" --form "config=NewConfig" https://api.havenondemand.com/1/api/sync/updateconnector/v1

You can also change the destination and the schedule.

/updateconnector/v1?connector=havenondemand&destination=NewDestinationConfig
/updateconnector/v1?connector=havenondemand&schedule=NewScheduleConfig

If you just want to delete a schedule, you can use the Cancel Connector Schedule API.

/cancelconnectorschedule/v1?connector=havenondemand

Credentials and Credentials Policies

Credentials enable secure, encrypted connections between the Haven OnDemand APIs and the Dropbox repositories.

Credentials are required for access to Dropbox accounts. When you create the Application in Dropbox, you receive an Application Key and an Access Token which you use to configure the credentials.

Haven OnDemand also lets you to set up Credentials Policies to determine how the secured access behaves. With Credentials Policies you can, for example:

  • set up email notifications when the Connector decrypts and uses the credentials to log in to the Web site.
  • limit the number of times a given decryption token can be used before it must be updated.
  • configure start and end dates for the validity of the decryption token.

For more information, see Connector Credentials and the page for the Dropbox Connector flavor.