- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks diff --git a/posts/config/custom/index.html b/posts/config/custom/index.html index dc58974b..65a8613b 100644 --- a/posts/config/custom/index.html +++ b/posts/config/custom/index.html @@ -178,7 +178,7 @@ a {
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks @@ -290,7 +290,7 @@ a {
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks diff --git a/posts/config/init/index.html b/posts/config/init/index.html index 930cc414..4fc75745 100644 --- a/posts/config/init/index.html +++ b/posts/config/init/index.html @@ -178,7 +178,7 @@ a {
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks @@ -318,10 +318,10 @@ a {
settings.yaml- The configuration settings file. This file contains the configuration settings for GraphRAG..env- The environment variables file. These are referenced in thesettings.yamlfile.
-prompts/- The LLM prompts folder. This contains the default prompts used by GraphRAG, you can modify them or run the Auto Prompt Tuning command to generate new prompts adapted to your data.
+prompts/- The LLM prompts folder. This contains the default prompts used by GraphRAG, you can modify them or run the Auto Prompt Tuning command to generate new prompts adapted to your data.
Next Steps
-After initializing your workspace, you can either run the Prompt Tuning command to adapt the prompts to your data or even start running the Indexing Pipeline to index your data. For more information on configuring GraphRAG, see the Configuration documentation.
+After initializing your workspace, you can either run the Prompt Tuning command to adapt the prompts to your data or even start running the Indexing Pipeline to index your data. For more information on configuring GraphRAG, see the Configuration documentation.
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks diff --git a/posts/config/overview/index.html b/posts/config/overview/index.html index 77c3a370..d3e5d3e8 100644 --- a/posts/config/overview/index.html +++ b/posts/config/overview/index.html @@ -178,7 +178,7 @@ a {
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks @@ -293,12 +293,12 @@ a {
- Init command (recommended) -
- Purely using environment variables -
- Using JSON or YAML for deeper control +
- Init command (recommended) +
- Purely using environment variables +
- Using JSON or YAML for deeper control
Default Configuration Mode
The default configuration mode is the simplest way to get started with the GraphRAG system. It is designed to work out-of-the-box with minimal configuration. The primary configuration sections for the Indexing Engine pipelines are described below. The main ways to set up GraphRAG in Default Configuration mode are via:
-
-
Custom Configuration Mode
-Custom configuration mode is an advanced use-case. Most users will want to use the Default Configuration instead. The primary configuration sections for Indexing Engine pipelines are described below. Details about how to use custom configuration are available in the Custom Configuration Mode documentation.
+Custom configuration mode is an advanced use-case. Most users will want to use the Default Configuration instead. The primary configuration sections for Indexing Engine pipelines are described below. Details about how to use custom configuration are available in the Custom Configuration Mode documentation.
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks diff --git a/posts/developing/index.html b/posts/developing/index.html index e643c723..006b7808 100644 --- a/posts/developing/index.html +++ b/posts/developing/index.html @@ -178,7 +178,7 @@ a {
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks diff --git a/posts/get_started/index.html b/posts/get_started/index.html index 98c2fbba..f8e0b2f9 100644 --- a/posts/get_started/index.html +++ b/posts/get_started/index.html @@ -178,7 +178,7 @@ a {
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks @@ -294,12 +294,12 @@ a {
To get started with the GraphRAG system, you have a few options:
👉 Use the GraphRAG Accelerator solution
+👉 Use it from source
👉 Install from pypi.
-👉 Use it from source
Quickstart
To get started with the GraphRAG system we recommend trying the Solution Accelerator package. This provides a user-friendly end-to-end experience with Azure resources.
Top-Level Modules
-Indexing Pipeline Overview
+
-Query Engine OverviewIndexing Pipeline Overview
+Query Engine OverviewOverview
The following is a simple end-to-end example for using the GraphRAG system. It shows how to use the system to index some text, and then use the indexed data to answer questions about the documents.
@@ -313,7 +313,7 @@ It shows how to use the system to index some text, and then use the indexed data
- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {Running the Indexer
-Now we need to set up a data project and some initial configuration. Let's set that up. We're using the default configuration mode, which you can customize as needed using a config file, which we recommend, or environment variables.
+Now we need to set up a data project and some initial configuration. Let's set that up. We're using the default configuration mode, which you can customize as needed using a config file, which we recommend, or environment variables.
First let's get a sample dataset ready:
@@ -334,7 +334,7 @@ It shows how to use the system to index some text, and then use the indexed dataNext we'll inject some required config variables:
Set Up Your Workspace Variables
-First let's make sure to setup the required environment variables. For details on these environment variables, and what environment variables are available, see the variables documentation.
+First let's make sure to setup the required environment variables. For details on these environment variables, and what environment variables are available, see the variables documentation.
To initialize your workspace, let's first run the
@@ -368,9 +368,9 @@ Since we have already configured a directory named .ragtest` in the previous stegraphrag.index --initcommand. Since we have already configured a directory named .ragtest` in the previous step, we can run the following command:
- For more details about configuring GraphRAG, see the configuration documentation. -
- To learn more about Initialization, refer to the Initialization documentation. -
- For more details about using the CLI, refer to the CLI documentation. +
- For more details about configuring GraphRAG, see the configuration documentation. +
- To learn more about Initialization, refer to the Initialization documentation. +
- For more details about using the CLI, refer to the CLI documentation.
- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {-
-
Running the Indexing pipeline
Finally we'll run the pipeline!
@@ -382,7 +382,7 @@ Since we have already configured a directory named .ragtest` in the previous ste
- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {diff --git a/posts/config/template/index.html b/posts/config/template/index.html index 9e861f23..d1ae172a 100644 --- a/posts/config/template/index.html +++ b/posts/config/template/index.html @@ -178,7 +178,7 @@ a {- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {-
+

This process will take some time to run. This depends on the size of your input data, what model you're using, and the text chunk size being used (these can be configured in your
.envfile). Once the pipeline is complete, you should see a new folder called./ragtest/output/<timestamp>/artifactswith a series of parquet files.Using the Query Engine
@@ -412,7 +412,7 @@ Once the pipeline is complete, you should see a new folder called./ragtes
- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {diff --git a/posts/config/json_yaml/index.html b/posts/config/json_yaml/index.html index 089188c4..77752000 100644 --- a/posts/config/json_yaml/index.html +++ b/posts/config/json_yaml/index.html @@ -178,7 +178,7 @@ a {- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {-Please refer to Query Engine docs for detailed information about how to leverage our Local and Global search mechanisms for extracting meaningful insights from data after the Indexer has wrapped up execution.
+Please refer to Query Engine docs for detailed information about how to leverage our Local and Global search mechanisms for extracting meaningful insights from data after the Indexer has wrapped up execution.
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks diff --git a/posts/index/1-default_dataflow/index.html b/posts/index/1-default_dataflow/index.html index 7437a4b0..0bf919a0 100644 --- a/posts/index/1-default_dataflow/index.html +++ b/posts/index/1-default_dataflow/index.html @@ -178,7 +178,7 @@ a {
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks @@ -301,7 +301,7 @@ a {
Node- This table contains layout information for rendered graph-views of the Entities and Documents which have been embedded and clustered.
The Default Configuration Workflow
-Let's take a look at how the default-configuration workflow transforms text documents into the GraphRAG Knowledge Model. This page gives a general overview of the major steps in this process. To fully configure this workflow, check out the configuration documentation.
+Let's take a look at how the default-configuration workflow transforms text documents into the GraphRAG Knowledge Model. This page gives a general overview of the major steps in this process. To fully configure this workflow, check out the configuration documentation.
--- title: Dataflow Overview --- flowchart TB subgraph phase1[Phase 1: Compose TextUnits] documents[Documents] --> chunk[Chunk] chunk --> embed[Embed] --> textUnits[Text Units] end subgraph phase2[Phase 2: Graph Extraction] textUnits --> graph_extract[Entity & Relationship Extraction] graph_extract --> graph_summarize[Entity & Relationship Summarization] graph_summarize --> entity_resolve[Entity Resolution] entity_resolve --> claim_extraction[Claim Extraction] claim_extraction --> graph_outputs[Graph Tables] end subgraph phase3[Phase 3: Graph Augmentation] graph_outputs --> community_detect[Community Detection] community_detect --> graph_embed[Graph Embedding] graph_embed --> augmented_graph[Augmented Graph Tables] end subgraph phase4[Phase 4: Community Summarization] augmented_graph --> summarized_communities[Community Summarization] summarized_communities --> embed_communities[Community Embedding] embed_communities --> community_outputs[Community Tables] end subgraph phase5[Phase 5: Document Processing] documents --> link_to_text_units[Link to TextUnits] textUnits --> link_to_text_units link_to_text_units --> embed_documents[Document Embedding] embed_documents --> document_graph[Document Graph Creation] document_graph --> document_outputs[Document Tables] end subgraph phase6[Phase 6: Network Visualization] document_outputs --> umap_docs[Umap Documents] augmented_graph --> umap_entities[Umap Entities] umap_docs --> combine_nodes[Nodes Table] umap_entities --> combine_nodes endPhase 1: Compose TextUnits
The first phase of the default-configuration workflow is to transform input documents into TextUnits. A TextUnit is a chunk of text that is used for our graph extraction techniques. They are also used as source-references by extracted knowledge items in order to empower breadcrumbs and provenance by concepts back to their original source tex.
@@ -349,7 +349,7 @@ Entities and Relationships are extracted at once in our entity_extractIn this phase of the workflow, we create the Documents table for the knowledge model.
--- title: Document Processing --- flowchart LR aug[Augment] --> dp[Link to TextUnits] --> de[Avg. Embedding] --> dg[Document Table Emission]Augment with Columns (CSV Only)
-If the workflow is operating on CSV data, you may configure your workflow to add additional fields to Documents output. These fields should exist on the incoming CSV tables. Details about configuring this can be found in the configuration documentation.
+If the workflow is operating on CSV data, you may configure your workflow to add additional fields to Documents output. These fields should exist on the incoming CSV tables. Details about configuring this can be found in the configuration documentation.
Link to TextUnits
In this step, we link each document to the text-units that were created in the first phase. This allows us to understand which documents are related to which text-units and vice-versa.
Document Embedding
diff --git a/posts/index/2-cli/index.html b/posts/index/2-cli/index.html index bb0abafb..97bb1d3e 100644 --- a/posts/index/2-cli/index.html +++ b/posts/index/2-cli/index.html @@ -178,7 +178,7 @@ a {- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks diff --git a/posts/index/overview/index.html b/posts/index/overview/index.html index 6ad236ca..c4e6760e 100644 --- a/posts/index/overview/index.html +++ b/posts/index/overview/index.html @@ -178,7 +178,7 @@ a {
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks @@ -301,9 +301,9 @@ a {
The outputs of the pipeline can be stored in a variety of formats, including JSON and Parquet - or they can be handled manually via the Python API.
Getting Started
Requirements
-See the requirements section in Get Started for details on setting up a development environment.
+See the requirements section in Get Started for details on setting up a development environment.
The Indexing Engine can be used in either a default configuration mode or with a custom pipeline. -To configure GraphRAG, see the configuration documentation. +To configure GraphRAG, see the configuration documentation. After you have a config file you can run the pipeline using the CLI or the Python API.
Usage
CLI
@@ -359,10 +359,10 @@ pipeline_result = outputsgetting started
- - To understand the underlying concepts and execution model of the indexing library, see the architecture documentation +
- To start developing within the GraphRAG project, see getting started +
- To understand the underlying concepts and execution model of the indexing library, see the architecture documentation
- To get running with a series of examples, see the examples documentation -
- To read more about configuring the indexing engine, see the configuration documentation +
- To read more about configuring the indexing engine, see the configuration documentation
- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks @@ -292,7 +292,7 @@ a {
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks @@ -322,7 +322,7 @@ We provide a means for you to do this by allowing you to specify a custom prompt
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks @@ -299,9 +299,9 @@ a {
- Community Reports
Auto Templating
-Auto Templating leverages your input data and LLM interactions to create domain adaptive templates for the generation of the knowledge graph. It is highly encouraged to run it as it will yield better results when executing an Index Run. For more details about how to use it, please refer to the Auto Templating documentation.
+Auto Templating leverages your input data and LLM interactions to create domain adaptive templates for the generation of the knowledge graph. It is highly encouraged to run it as it will yield better results when executing an Index Run. For more details about how to use it, please refer to the Auto Templating documentation.
Manual Configuration
-Manual configuration is an advanced use-case. Most users will want to use the Auto Templating feature instead. Details about how to use manual configuration are available in the Manual Prompt Configuration documentation.
+Manual configuration is an advanced use-case. Most users will want to use the Auto Templating feature instead. Details about how to use manual configuration are available in the Manual Prompt Configuration documentation.
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks diff --git a/posts/query/1-local_search/index.html b/posts/query/1-local_search/index.html index 39fa44e9..06797109 100644 --- a/posts/query/1-local_search/index.html +++ b/posts/query/1-local_search/index.html @@ -178,7 +178,7 @@ a {
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks diff --git a/posts/query/2-question_generation/index.html b/posts/query/2-question_generation/index.html index 219e64ff..245b9886 100644 --- a/posts/query/2-question_generation/index.html +++ b/posts/query/2-question_generation/index.html @@ -178,7 +178,7 @@ a {
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks diff --git a/posts/query/3-cli/index.html b/posts/query/3-cli/index.html index e382fae7..a9550ddc 100644 --- a/posts/query/3-cli/index.html +++ b/posts/query/3-cli/index.html @@ -178,7 +178,7 @@ a {
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks diff --git a/posts/query/notebooks/global_search_nb/index.html b/posts/query/notebooks/global_search_nb/index.html index 65d68c25..0db8857e 100644 --- a/posts/query/notebooks/global_search_nb/index.html +++ b/posts/query/notebooks/global_search_nb/index.html @@ -178,7 +178,7 @@ a {
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks diff --git a/posts/query/notebooks/local_search_nb/index.html b/posts/query/notebooks/local_search_nb/index.html index 2d9fe95b..6d713f47 100644 --- a/posts/query/notebooks/local_search_nb/index.html +++ b/posts/query/notebooks/local_search_nb/index.html @@ -178,7 +178,7 @@ a {
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks diff --git a/posts/query/notebooks/overview/index.html b/posts/query/notebooks/overview/index.html index 28539fb1..8258a6a7 100644 --- a/posts/query/notebooks/overview/index.html +++ b/posts/query/notebooks/overview/index.html @@ -178,7 +178,7 @@ a {
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks @@ -291,10 +291,10 @@ a {
Query Engine Notebooks
For examples about running Query please refer to the following notebooks:
-The test dataset for these notebooks can be found here.
+The test dataset for these notebooks can be found here.
- -Welcome +Welcome
- -Get Started +Get Started -Developing +Developing
-
-Indexing
+Indexing
- -Architecture +Architecture
- -Dataflow +Dataflow
- -CLI +CLI
- -Configuration +Configuration
- -Prompt Tuning +Prompt Tuning @@ -258,26 +258,26 @@ a {
-
-Query
+Query
- -Local Search +Local Search
- -Question Generation +Question Generation
- -Global Search +Global Search
- -CLI +CLI
- -Notebooks +Notebooks @@ -289,7 +289,7 @@ a {
- Local Search @@ -298,13 +298,13 @@ It is responsible for the following tasks:
Query Engine 🔎
-The Query Engine is the retrieval module of the Graph RAG Library. It is one of the two main components of the Graph RAG library, the other being the Indexing Pipeline (see Indexing Pipeline). +
The Query Engine is the retrieval module of the Graph RAG Library. It is one of the two main components of the Graph RAG library, the other being the Indexing Pipeline (see Indexing Pipeline). It is responsible for the following tasks:
Local Search
Local search method generates answers by combining relevant data from the AI-extracted knowledge-graph with text chunks of the raw documents. This method is suitable for questions that require an understanding of specific entities mentioned in the documents (e.g. What are the healing properties of chamomile?).
-For more details about how Local Search works please refer to the Local Search documentation.
+For more details about how Local Search works please refer to the Local Search documentation.
Global Search
Global search method generates answers by searching over all AI-generated community reports in a map-reduce fashion. This is a resource-intensive method, but often gives good responses for questions that require an understanding of the dataset as a whole (e.g. What are the most significant values of the herbs mentioned in this notebook?).
-More about this can be checked at the Global Search documentation.
+More about this can be checked at the Global Search documentation.
Question Generation
This functionality takes a list of user queries and generates the next candidate questions. This is useful for generating follow-up questions in a conversation or for generating a list of questions for the investigator to dive deeper into the dataset.
-Information about how question generation works can be found at the Question Generation documentation page.
+Information about how question generation works can be found at the Question Generation documentation page.
- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {diff --git a/posts/query/overview/index.html b/posts/query/overview/index.html index b121f608..f0069cdf 100644 --- a/posts/query/overview/index.html +++ b/posts/query/overview/index.html @@ -178,7 +178,7 @@ a {- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {
- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {
- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {
- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {
- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {
- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {
Note: there is additional paramater for the
Claim Descriptionthat is used in claim extraction. The default value is
-"Any claims or facts that could be relevant to information discovery."See the configuration documentation for details on how to change this.
+See the configuration documentation for details on how to change this.
Generate Community Reports
Tokens (values provided by extractor)
diff --git a/posts/prompt_tuning/overview/index.html b/posts/prompt_tuning/overview/index.html index f6842c9f..ae3d82fa 100644 --- a/posts/prompt_tuning/overview/index.html +++ b/posts/prompt_tuning/overview/index.html @@ -178,7 +178,7 @@ a {- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {diff --git a/posts/query/0-global_search/index.html b/posts/query/0-global_search/index.html index dcfcb921..533a3c4f 100644 --- a/posts/query/0-global_search/index.html +++ b/posts/query/0-global_search/index.html @@ -178,7 +178,7 @@ a {- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {
GraphRAG provides the ability to create domain adaptive templates for the generation of the knowledge graph. This step is optional, though is is highly encouraged to run it as it will yield better results when executing an Index Run.
The templates are generated by loading the inputs, splitting them into chunks (text units) and then running a series of LLM invocations and template substitutions to generate the final prompts. We suggest using the default values provided by the script, but in this page you'll find the detail of each in case you want to further explore and tweak the template generation algorithm.
Prerequisites
-Before running the automatic template generation make sure you have already initialized your workspace with the
+graphrag.index --initcommand. This will create the necessary configuration files and the default prompts. Refer to the Init Documentation for more information about the initialization process.Before running the automatic template generation make sure you have already initialized your workspace with the
graphrag.index --initcommand. This will create the necessary configuration files and the default prompts. Refer to the Init Documentation for more information about the initialization process.Usage
You can run the main script from the command line with various options:
diff --git a/posts/prompt_tuning/manual_prompt_tuning/index.html b/posts/prompt_tuning/manual_prompt_tuning/index.html index 20ff9833..c8f9f8b7 100644 --- a/posts/prompt_tuning/manual_prompt_tuning/index.html +++ b/posts/prompt_tuning/manual_prompt_tuning/index.html @@ -178,7 +178,7 @@ a {- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {
- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {
- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {
Custom Configuration Mode
The primary configuration sections for Indexing Engine pipelines are described below. Each configuration section can be expressed in Python (for use in Python API mode) as well as YAML, but YAML is show here for brevity.
-Using custom configuration is an advanced use-case. Most users will want to use the Default Configuration instead.
+Using custom configuration is an advanced use-case. Most users will want to use the Default Configuration instead.
Indexing Engine Examples
The examples directory contains several examples of how to use the indexing engine with custom configuration.
Most examples include two different forms of running the pipeline, both are contained in the examples
diff --git a/posts/config/env_vars/index.html b/posts/config/env_vars/index.html index fc5f5d6d..0f104230 100644 --- a/posts/config/env_vars/index.html +++ b/posts/config/env_vars/index.html @@ -178,7 +178,7 @@ a {run.py- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {diff --git a/posts/index/0-architecture/index.html b/posts/index/0-architecture/index.html index 12185b9d..682bb1da 100644 --- a/posts/index/0-architecture/index.html +++ b/posts/index/0-architecture/index.html @@ -178,7 +178,7 @@ a {- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {
- GraphRAG + GraphRAG @@ -186,67 +186,67 @@ a {