Back to home

Help Center

Getting Started

How to use the Precog Express API

The Precog public HTTP REST API.

Version: 1.0

http://apache.org/licenses/LICENSE-2.0.html

Methods

[ Jump to Models ]

Default

GET /destinations
GET /destinations/{destination-id}
GET /kinds/destination
GET /kinds/{kind-id}
GET /kinds/source
GET /pipelines
GET /pipelines/{pipeline-id}
GET /pipelines/{pipeline-id}/datasets
GET /pipelines/{pipeline-id}/details
GET /pipelines/{pipeline-id}/loads/latest/logs
GET /pipelines/{pipeline-id}/schedule
GET /pipelines/{pipeline-id}/status
GET /pipelines/status
GET /sources
GET /sources/{source-id}
GET /sources/{source-id}/datasets
PATCH /pipelines/{pipeline-id}/datasets
POST /destinations
POST /pipelines
POST /pipelines/{pipeline-id}/loads
POST /sources
POST /stripe/check
POST /stripe/session
PUT /destinations/{destination-id}/configuration
PUT /pipelines/{pipeline-id}/rename
PUT /pipelines/{pipeline-id}/schedule
POST /sources/{source-id}/refresh

Default

GET /destinations

Get Destinations (getDestinations)

Retrieve the configured destinations, optionally filtering by kind.

Query parameters

kind-id (optional)

Query Parameter —

Return type

array[connector-listing]

Example data

Content-Type: application/json

[ {
  "name" : "name",
  "id" : "id",
  "kindId" : "kindId"
}, {
  "name" : "name",
  "id" : "id",
  "kindId" : "kindId"
} ]

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

200

Example data

Content-Type: example-1

[{"id":"1","kindId":"14","name":"Account: Precog, Database: Raw, Region: west-us-2.azure"}]

401

Unauthorized

GET /destinations/{destination-id}

Get a destination (getDestinationsDestinationId)

Get a particular destination.

Path parameters

destination-id (required)

Path Parameter —

Return type

connector

Example data

Content-Type: application/json

{
  "auth" : {
    "details" : {
      "redirectUri" : "redirectUri",
      "authCode" : "authCode"
    },
    "type" : "OAuth 2.0 code authorization"
  },
  "name" : "name",
  "id" : "id",
  "config" : [ {
    "name" : "name",
    "value" : "value"
  }, {
    "name" : "name",
    "value" : "value"
  } ],
  "kindId" : "kindId"
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

200

OK connector

Example data

Content-Type: example-1

{"config":[],"id":"1","kindId":"14","name":"Account: Precog, Database: Raw, Region: west-us-2.azure"}

401

Unauthorized

404

Destination not found.

GET /kinds/destination

Get destination kinds (getKindsDestination)

Retrieve the available kinds of destinations.

Return type

array[connector-kind]

Example data

Content-Type: application/json

[ {
  "name" : "name",
  "id" : "id"
}, {
  "name" : "name",
  "id" : "id"
} ]

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

200

Example data

Content-Type: example-1

[{"id":"14","name":"Snowflake"},{"id":"15","name":"Panoply"},{"id":"16","name":"Redshift"},{"id":"17","name":"Google BigQuery"},{"id":"18","name":"Firebolt"}]

401

Unauthorized

GET /kinds/{kind-id}

Get connector schema (getKindsKindId)

Get the configuration schema for a particular kind of source.

Path parameters

kind-id (required)

Path Parameter —

Return type

inline_response_200

Example data

Content-Type: application/json

{
  "schema" : [ {
    "name" : "name",
    "description" : "description",
    "sensitive" : true
  }, {
    "name" : "name",
    "description" : "description",
    "sensitive" : true
  } ],
  "auth" : {
    "details" : {
      "uri" : "uri"
    },
    "type" : "OAuth 2.0 code authorization"
  }
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

200

OK inline_response_200

401

Unauthorized

404

Kind not found.

GET /kinds/source

Get source kinds (getKindsSource)

Retrieve the available kinds of sources.

Return type

array[connector-kind]

Example data

Content-Type: application/json

[ {
  "name" : "name",
  "id" : "id"
}, {
  "name" : "name",
  "id" : "id"
} ]

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

200

Example data

Content-Type: example-1

[{"id":"1","name":"Active Campaign"},{"id":"2","name":"Ariba Analytics"},{"id":"3","name":"Bing Ads"},{"id":"4","name":"Chargeback"},{"id":"5","name":"Clockify"},{"id":"6","name":"Everflow"},{"id":"7","name":"Facebook Ads"},{"id":"8","name":"Google Ads"},{"id":"9","name":"Gravity Forms"},{"id":"10","name":"Hubspot"},{"id":"11","name":"Salesforce"},{"id":"12","name":"Shopify"},{"id":"13","name":"WooCommerce"}]

401

Unauthorized

GET /pipelines

Get pipelines (getPipelines)

Get all pipelines.

Return type

array[pipeline-listing]

Example data

Content-Type: application/json

[ {
  "name" : "name",
  "id" : "id"
}, {
  "name" : "name",
  "id" : "id"
} ]

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

200

Example data

Content-Type: example-1

[{"id":"1","name":"Ecommerce"},{"id":"2","name":"Internal"},{"id":"3","name":"Staging environment"},{"id":"4","name":"Test environment"}]

401

Unauthorized

GET /pipelines/{pipeline-id}

Get a pipeline (getPipelinesPipelineId)

Get a particular pipeline.

Path parameters

pipeline-id (required)

Path Parameter —

Return type

pipeline

Example data

Content-Type: application/json

{
  "name" : "name",
  "id" : "id",
  "destinationId" : "destinationId"
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

200

OK pipeline

Example data

Content-Type: example-1

{"destinationId":"1","id":"1","name":"Ecommerce"}

401

Unauthorized

404

Pipeline not found.

GET /pipelines/{pipeline-id}/datasets

Get the pipeline datasets (getPipelinesPipelineIdDatasets)

Get the datasets in the provided pipeline.

Path parameters

pipeline-id (required)

Path Parameter —

Return type

array[dataset]

Example data

Content-Type: application/json

[ {
  "sourceId" : "sourceId",
  "name" : "name",
  "id" : "id"
}, {
  "sourceId" : "sourceId",
  "name" : "name",
  "id" : "id"
} ]

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

200

401

Unauthorized

404

Pipeline not found.

GET /pipelines/{pipeline-id}/details

Get pipeline details (getPipelinesPipelineIdDetails)

Get all the details for the provided pipeline.

Path parameters

pipeline-id (required)

Path Parameter —

Return type

pipeline-details

Example data

Content-Type: application/json

{
  "pipeline" : {
    "name" : "name",
    "id" : "id"
  },
  "schedule" : [ "schedule", "schedule" ],
  "destination" : {
    "name" : "name",
    "id" : "id"
  },
  "datasets" : [ {
    "dataset" : [ null, null ]
  }, {
    "dataset" : [ null, null ]
  } ]
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

200

OK pipeline-details

Example data

Content-Type: example-1

{"datasets":[{"dataset":[{"id":"1","name":"Abandoned checkouts"},{"id":"2","name":"Application charges"},{"id":"3","name":"Application credits"},{"id":"4","name":"Custom collections"},{"id":"5","name":"Customer saved searches"},{"id":"6","name":"Customers"},{"id":"7","name":"Locations"},{"id":"8","name":"Orders"},{"id":"9","name":"Price rules"},{"id":"10","name":"Products"},{"id":"11","name":"Recurring application charges"}],"source":{"id":"1","kind":{"id":"12","name":"Shopify"},"name":"Shop name: MyShop"}}],"destination":{"id":"1","kind":{"id":"14","name":"Snowflake"},"name":"Account: Precog, Database: Raw, Region: west-us-2.azure"},"pipeline":{"id":"1","name":"Ecommerce"}}

401

Unauthorized

404

Pipeline not found

GET /pipelines/{pipeline-id}/loads/latest/logs

Get pipeline logs for the latest load (getPipelinesPipelineIdLoadsLatestLogs)

Get pipeline logs for the latest load and provided dataset

Path parameters

pipeline-id (required)

Path Parameter —

Query parameters

dataset-id (required)

Query Parameter —

Return type

String

Example data

Content-Type: application/json

""

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

text/plain

Responses

200

OK String

204

This dataset has not yet been loaded

401

Unauthorized

404

Pipeline not found <OR> Dataset not found

GET /pipelines/{pipeline-id}/schedule

Get the pipeline schedule (getPipelinesPipelineIdSchedule)

Get the pipeline schedule.

Path parameters

pipeline-id (required)

Path Parameter —

Return type

pipeline-schedule

Example data

Content-Type: application/json

{
  "schedule" : [ "schedule", "schedule" ]
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

200

OK pipeline-schedule

401

Unauthorized

404

Pipeline not found

GET /pipelines/{pipeline-id}/status

Get the pipeline datasets status (getPipelinesPipelineIdStatus)

Get the status of the provided pipeline.

Path parameters

pipeline-id (required)

Path Parameter —

Return type

pipeline-dataset-status

Example data

Content-Type: application/json

{
  "datasets" : [ {
    "inBand" : {
      "at" : 6.027456183070403,
      "state" : "discovered"
    },
    "id" : "id",
    "outOfBand" : {
      "at" : 0.8008281904610115,
      "state" : "accepted"
    }
  }, {
    "inBand" : {
      "at" : 6.027456183070403,
      "state" : "discovered"
    },
    "id" : "id",
    "outOfBand" : {
      "at" : 0.8008281904610115,
      "state" : "accepted"
    }
  } ],
  "pipelineId" : "pipelineId",
  "status" : {
    "at" : 0.8008281904610115,
    "state" : "succeeded"
  }
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

200

OK pipeline-dataset-status

Example data

Content-Type: example-1

{"pipelineId":"1","status":{"state":"succeeded","at":0},"datasets":[{"id":"string","outOfBand":{"state":"staging","at":0},"inBand":{"state":"staging","at":0}}]}

401

Unauthorized

404

Pipeline not found

GET /pipelines/status

Get the pipeline status (getPipelinesStatus)

Get the status of all the pipelines.

Return type

array[pipeline-status]

Example data

Content-Type: application/json

[ {
  "pipelineId" : "pipelineId",
  "status" : {
    "at" : 0.8008281904610115,
    "state" : "succeeded"
  }
}, {
  "pipelineId" : "pipelineId",
  "status" : {
    "at" : 0.8008281904610115,
    "state" : "succeeded"
  }
} ]

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

200

Example data

Content-Type: example-1

[{"pipelineId":"1","status":{"at":1632428787341,"state":"loading"}},{"pipelineId":"3","status":{"at":1632128787341,"state":"succeeded"}},{"pipelineId":"4","status":{"at":1631251787341,"state":"failed"}}]

401

Unauthorized

GET /sources

Get Sources (getSources)

Retrieve the configured sources, optionally filtering by kind.

Query parameters

kind-id (optional)

Query Parameter —

Return type

array[connector-listing]

Example data

Content-Type: application/json

[ {
  "name" : "name",
  "id" : "id",
  "kindId" : "kindId"
}, {
  "name" : "name",
  "id" : "id",
  "kindId" : "kindId"
} ]

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

200

Example data

Content-Type: example-1

[{"id":"1","kindId":"12","name":"Shop name: MyShop"}]

401

Unauthorized

GET /sources/{source-id}

Get a source (getSourcesSourceId)

Get a particular source.

Path parameters

source-id (required)

Path Parameter —

Return type

connector

Example data

Content-Type: application/json

{
  "auth" : {
    "details" : {
      "redirectUri" : "redirectUri",
      "authCode" : "authCode"
    },
    "type" : "OAuth 2.0 code authorization"
  },
  "name" : "name",
  "id" : "id",
  "config" : [ {
    "name" : "name",
    "value" : "value"
  }, {
    "name" : "name",
    "value" : "value"
  } ],
  "kindId" : "kindId"
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

200

OK connector

Example data

Content-Type: example-1

{"config":[{"name":"Shop name","value":"MyShop"}],"id":"1","kindId":"12","name":"Shop name: MyShop"}

401

Unauthorized

404

Source not found.

GET /sources/{source-id}/datasets

Get datasets (getSourcesSourceIdDatasets)

Get all available datasets for the provided source.

Path parameters

source-id (required)

Path Parameter —

Return type

array[dataset]

Example data

Content-Type: application/json

[ {
  "sourceId" : "sourceId",
  "name" : "name",
  "id" : "id"
}, {
  "sourceId" : "sourceId",
  "name" : "name",
  "id" : "id"
} ]

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

200

Example data

Content-Type: example-1

[{"id":"1","name":"Abandoned checkouts","sourceId":"1"},{"id":"2","name":"Application charges","sourceId":"1"},{"id":"3","name":"Application credits","sourceId":"1"},{"id":"4","name":"Custom collections","sourceId":"1"},{"id":"5","name":"Customer saved searches","sourceId":"1"},{"id":"6","name":"Customers","sourceId":"1"},{"id":"7","name":"Locations","sourceId":"1"},{"id":"8","name":"Orders","sourceId":"1"},{"id":"9","name":"Price rules","sourceId":"1"},{"id":"10","name":"Products","sourceId":"1"},{"id":"11","name":"Recurring application charges","sourceId":"1"}]

204

Dataset hasn't been discovered yet

401

Unauthorized

404

Source not found.

PATCH /pipelines/{pipeline-id}/datasets

(patchPipelinesPipelineIdDatasets)

Add datasets to the provided pipeline.

Path parameters

pipeline-id (required)

Path Parameter —

Consumes

This API call consumes the following media types via the Content-Type request header:

application/json

Request body

body dataset-update (optional)

Body Parameter —

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

204

No Content

400

Bad Request error-message

401

Unauthorized

404

Pipeline not found. <or> Datasets not found.

422

Semantic error. error-message

POST /destinations

Create Destination (postDestinations)

Configure a new destination.

Consumes

This API call consumes the following media types via the Content-Type request header:

application/json

Request body

body connector (optional)

Body Parameter —

Return type

connector-listing

Example data

Content-Type: application/json

{
  "name" : "name",
  "id" : "id",
  "kindId" : "kindId"
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

201

Created connector-listing

Example data

Content-Type: example-1

{"id":"1","kindId":"14","name":"Account: Precog, Database: Raw, Region: west-us-2.azure"}

400

Bad Request error-message

401

Unauthorized

422

Destination kind unsupported or other semantic error. error-message

POST /pipelines

Create pipeline (postPipelines)

Create a new pipeline.

Consumes

This API call consumes the following media types via the Content-Type request header:

application/json

Request body

body pipeline (optional)

Body Parameter —

Return type

Example data

Content-Type: application/json

{
  "id" : "id"
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

201

Created id

400

Bad Request error-message

401

Unauthorized

422

Semantic error. error-message

POST /pipelines/{pipeline-id}/loads

Start a pipeline load (postPipelinesPipelineIdLoads)

Start a load for this pipeline.

Path parameters

pipeline-id (required)

Path Parameter —

Responses

202

Accepted - there are in-band datasets and a load has been scheduled to start

204

No Content - there are no in-band datasets

400

Bad Request - there are no datasets in pipeline

401

Unauthorized

404

Pipeline not found.

POST /sources

Create Source (postSources)

Configure a new source.

Consumes

This API call consumes the following media types via the Content-Type request header:

application/json

Request body

body connector (optional)

Body Parameter —

Return type

connector-listing

Example data

Content-Type: application/json

{
  "name" : "name",
  "id" : "id",
  "kindId" : "kindId"
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

201

Created connector-listing

400

Bad Request error-message

401

Unauthorized

422

Source kind unsupported or other semantic error. error-message

502

Problematic Request to External Server error-message

POST /stripe/check

Check Stripe subscription status (postStripeCheck)

Check the Stripe subscription status

Consumes

This API call consumes the following media types via the Content-Type request header:

application/json

Request body

body stripe_check_body (optional)

Body Parameter —

Return type

url

Example data

Content-Type: application/json

{
  "url" : "url"
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

200

url

204

Valid Subscription

400

Bad Request error-message

401

Unauthorized

422

Semantic error. error-message

502

Problematic Request to External Server error-message

POST /stripe/session

Create Stripe session (postStripeSession)

Create a Stripe customer portal session

Consumes

This API call consumes the following media types via the Content-Type request header:

application/json

Request body

body stripe_session_body (optional)

Body Parameter —

Return type

url

Example data

Content-Type: application/json

{
  "url" : "url"
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

200

url

400

Bad Request error-message

401

Unauthorized

404

Stripe customer not found

422

Semantic error. error-message

502

Problematic Request to External Server error-message

PUT /destinations/{destination-id}/configuration

Update a destination config (putDestinationsDestinationId)

Update a destination configuration for a particular destination .

Path parameters

destination-id (required)

Path Parameter —

Consumes

This API call consumes the following media types via the Content-Type request header:

application/json

Request body

body destinationid_configuration_body (optional)

Body Parameter —

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

204

No Content

400

Bad Request error-message

401

Unauthorized

404

Destination not found.

422

Semantic Error

PUT /pipelines/{pipeline-id}/rename

Rename a pipelne. (putPipelinesPipelineIdRename)

Rename a pipeline.

Path parameters

pipeline-id (required)

Path Parameter —

Consumes

This API call consumes the following media types via the Content-Type request header:

application/json

Request body

body pipeline-name (optional)

Body Parameter —

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

application/json

Responses

204

No Content

400

Bad Request error-message

401

Unauthorized

404

Pipeline not found

422

Semantic error. error-message

PUT /pipelines/{pipeline-id}/schedule

Schedule a pipelne. (putPipelinesPipelineIdSchedule)

Schedule a pipeline.

Path parameters

pipeline-id (required)

Path Parameter —

Consumes

This API call consumes the following media types via the Content-Type request header:

application/json

Request body

body pipeline-schedule (optional)

Body Parameter — A list of cron expressions. An empty list means that the pipeline will not be loaded on a schedule.

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.