Apify

Apify is a cloud platform for web scraping and data extraction, which provides an ecosystem of more than a thousand ready-made apps called Actors for various scraping, crawling, and extraction use cases.

This integration enables you run Actors on the Apify platform and load their results into LangChain to feed your vector indexes with documents and data from the web, e.g. to generate answers from websites with documentation, blogs, or knowledge bases.

Installation and Setup

• Install the LangChain Apify package for Python with:

pip install langchain-apify

• Get your Apify API token and either set it as an environment variable (APIFY_API_TOKEN) or pass it as apify_api_token in the constructor.

Tool

You can use the ApifyActorsTool to use Apify Actors with agents.

from langchain_apify import ApifyActorsTool

See this notebook for example usage.

For more information on how to use this tool, visit the Apify integration documentation.

Wrapper

You can use the ApifyWrapper to run Actors on the Apify platform.

from langchain_apify import ApifyWrapper

For more information on how to use this wrapper, see the Apify integration documentation.

Document loader

You can also use our ApifyDatasetLoader to get data from Apify dataset.

from langchain_apify import ApifyDatasetLoader

For a more detailed walkthrough of this loader, see this notebook.

Source code for this integration can be found in the LangChain Apify repository.