Discovery Workflow

This guide walks you through the complete market research process using Forestreet’s API. The workflow consists of five main phases, with the third phase being optional for quality control.

Before You Start: Ensure you have proper authentication and access to the markets you want to research. All endpoints require valid authentication tokens.

Overview

The discovery workflow follows this sequence:

Search Criteria Generation - Define what you’re looking for
Company Discovery - Find relevant companies using AI
Company Review (Optional) - Review and include discovered companies
Market Publishing (Optional) - Publish your findings
Market Analysis - Gain insights from the discovered companies

Phase 1: Search Criteria Generation

Purpose: Convert your business question into structured search criteria that our AI can use to find relevant companies.

Step 1.1: Generate Search Criteria

Use the Generate Search Criteria endpoint to convert your natural language query into structured search parameters.

POST /v2/discovery/search-criteria
Content-Type: application/json

{
  "searchType": "marketSearch",
  "searchInput": "Find companies in the AI transcription services market that serve enterprise customers"
}

Response:

{
  "searchCriteria": {
    "userPrompt": "Find companies in the AI transcription services market that serve enterprise customers",
    "marketName": "AI Transcription Services",
    "marketType": "service",
    "marketRole": "serviceProvider",
    "keywords": {
      "features": ["AI transcription", "speech-to-text", "enterprise"],
      "user": ["enterprise", "business", "corporate"],
      "synonyms": ["transcription", "dictation", "voice-to-text"],
      "irrelevant": ["manual transcription", "human transcription"]
    },
    "seedDomains": [],
    "projectKind": "first_pass",
    "locations": [
      {
        "countryCode": "GB",
        "countryName": "United Kingdom",
      }
    ]
  }
}

Pro Tip: Be specific in your search input. Instead of “AI companies”, try “AI-powered transcription services for enterprise customers in the healthcare sector.”

Step 1.2: Refine Keywords (Optional)

Use the Generate Keywords endpoint to get AI-suggested keywords for your search criteria.

This step is optional but recommended for better discovery results. The AI will suggest additional relevant keywords and help refine your search.

Phase 2: Company Discovery

Purpose: Use the generated search criteria to discover relevant companies using AI-powered algorithms.

Step 2.1: Start Discovery Process

Use the Start Discovery endpoint to initiate the company discovery process.

If you set skipDiscoveryReview to true in pipelineChoices, the workflow will skip the company review phase and jump directly to Phase 5: Market Analysis.

POST /v2/discovery/12345/start
Content-Type: application/json

{
  "searchCriteria": {
    "marketName": "AI Transcription Services",
    "userPrompt": "Find companies in the AI transcription services market that serve enterprise customers",
    "marketType": "service",
    "marketRole": "serviceProvider",
    "keywords": {
      "features": ["AI transcription", "speech-to-text", "enterprise"],
      "user": ["enterprise", "business", "corporate"],
      "synonyms": ["transcription", "dictation", "voice-to-text"],
      "irrelevant": ["manual transcription", "human transcription"]
    },
    "seedDomains": [],
    "count": 50,
    "performSearch": true,
    "locations": [
      {
       "countryCode": "GB",
       "countryName": "United Kingdom",
      }
    ]
  },
  "pipelineChoices": {
    "skipDiscoveryReview": false
  }
}

Response:

{
  "metadata": {
    "marketId": 12345,
  },
  "data": [
    {
      "jobId": 67890,
      "workflowName": "first_pass",
      "startTime": "2024-01-15T10:30:00Z",
    },
  ]
}

Important: The discovery process runs asynchronously and can take 10-15 minutes. Do not make multiple requests for the same market while discovery is in progress.

Step 2.2: Monitor Discovery Progress

Use the Get Job Status endpoint to check the progress of your discovery job.

GET /v2/discovery/{marketId}/jobs

Response:

{
  "metadata": {
    "marketId": 12345,
  },
  "data": [
    {
      "jobId": 67890,
      "jobProgress": 0,
      "workflowName": "first_pass",
      "status": "running",
      "startTime": "2024-01-15T10:30:00Z",
      "endTime": null
    },
  ]
}

Polling Strategy: Check status every 30-60 seconds. The process typically takes 10-15 minutes depending on the complexity of your search criteria.

Phase 3: Company Review (Optional)

Purpose: Review and include discovered companies before publishing for quality control.

Optional Step: This phase is only needed if you set skipDiscoveryReview: false in your pipeline choices. If you set it to true, you can skip directly to Phase 4.

Step 3.1: Review Discovered Companies

Once discovery is complete, review the discovered companies and include them using the Update Company Review Status endpoint:

PATCH /v2/discovery/12345/companies/reviews
Content-Type: application/json

{
  "reviewStatus": "include",
  "target": {
    "mode": "partial",
    "domains": [
      "new-company.com",
      "startup-ai.com"
    ]
  }
}

Review Process: This step is optional but recommended for quality control. You can include or exclude discovered companies.

Step 3.2: Retrieve Search Criteria

Use the Get Search Criteria endpoint to review the criteria used for discovery.

GET /v2/discovery/12345/search-criteria

Step 3.3: Check Search History

Use the Get Search Criteria History endpoint to view previous search criteria iterations.

GET /v2/discovery/12345/search-criteria/history

Phase 4: Market Publishing

Purpose: Publish the discovered companies and trigger market analysis and enrichment.

Step 4.1: Publish Market

Optional Step: This phase is only needed if you set skipDiscoveryReview: false in your pipeline choices during Step 2 Start Discovery Process. If you set it to true, you can skip directly to Phase 5.

Once discovery is complete (and review is done if applicable), use the Publish Market endpoint to trigger market analysis and enrichment.

POST /v2/discovery/12345/publish

Response:

"metadata": {
  "marketId": 12345,
},
"data": [
  {
    "jobId": 67890,
    "workflowName": "enrichment",
    "startTime": "2024-01-15T10:30:00Z",
  },
  {
    "jobId": 67891,
    "workflowName": "scrape",
    "startTime": "2024-01-15T10:30:00Z",
  },
  {
    "jobId": 67892,
    "workflowName": "web_traffic",
    "startTime": "2024-01-15T10:30:00Z",
  },
],

Step 4.2: Monitor Publish Progress

Use the Get Job Status endpoint to check the progress of your publish job.

GET /v2/markets/{marketId}/jobs

Phase 5: Accessing Published Market

Purpose: Access and utilise the enriched data and insights from your published market.

Step 5: Access Your Published Market

Once publishing is complete, you can access your market through the Markets API endpoints:

# Get companies in the market
GET /v2/markets/12345/companies

# Get ESG data
GET /v2/markets/12345/esg

Error Handling

Common Issues: - 409 Conflict: Discovery already in progress for this market - 403 Forbidden: Insufficient permissions for the market - 404 Not Found: Market doesn’t exist or you don’t have access

Best Practices

For Better Results: 1. Be Specific: Use detailed, specific search criteria 2. Use Keywords: Leverage the keyword generation feature 3. Monitor Progress: Check status regularly during long-running processes 4. Review Companies: Take time to review discovered companies before publishing

Next Steps

Once your market is published, you can:

Export Data: Use the export endpoints to get CSV/Excel files
Track News: Monitor news and updates for companies in your market
Share Insights: Use the UI components to visualize and share your findings

For detailed API reference, see the Discovery Endpoints section.

Quick Start

Guides

API Reference

UI Components

Overview

Phase 1: Search Criteria Generation

Step 1.1: Generate Search Criteria

Step 1.2: Refine Keywords (Optional)

Phase 2: Company Discovery

Step 2.1: Start Discovery Process

Step 2.2: Monitor Discovery Progress

Phase 3: Company Review (Optional)

Step 3.1: Review Discovered Companies

Step 3.2: Retrieve Search Criteria

Step 3.3: Check Search History

Phase 4: Market Publishing

Step 4.1: Publish Market

Step 4.2: Monitor Publish Progress

Phase 5: Accessing Published Market

Step 5: Access Your Published Market

Error Handling

Best Practices

Next Steps

​Overview

​Phase 1: Search Criteria Generation

​Step 1.1: Generate Search Criteria

​Step 1.2: Refine Keywords (Optional)

​Phase 2: Company Discovery

​Step 2.1: Start Discovery Process

​Step 2.2: Monitor Discovery Progress

​Phase 3: Company Review (Optional)

​Step 3.1: Review Discovered Companies

​Step 3.2: Retrieve Search Criteria

​Step 3.3: Check Search History

​Phase 4: Market Publishing

​Step 4.1: Publish Market

​Step 4.2: Monitor Publish Progress

​Phase 5: Accessing Published Market

​Step 5: Access Your Published Market

​Error Handling

​Best Practices

​Next Steps

Overview

Phase 1: Search Criteria Generation

Step 1.1: Generate Search Criteria

Step 1.2: Refine Keywords (Optional)

Phase 2: Company Discovery

Step 2.1: Start Discovery Process

Step 2.2: Monitor Discovery Progress

Phase 3: Company Review (Optional)

Step 3.1: Review Discovered Companies

Step 3.2: Retrieve Search Criteria

Step 3.3: Check Search History

Phase 4: Market Publishing

Step 4.1: Publish Market

Step 4.2: Monitor Publish Progress

Phase 5: Accessing Published Market

Step 5: Access Your Published Market

Error Handling

Best Practices

Next Steps