Api overview
+ + + + + + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+In [1]:
+
+
+
+
+
+
+
+
+
+Copied!
+
+
+# Copyright (c) 2024 Microsoft Corporation.
+# Licensed under the MIT License.
+# Copyright (c) 2024 Microsoft Corporation.
+# Licensed under the MIT License.
+
+
+
+
+
+
+
+
+
+API Overview¶
This notebook provides a demonstration of how to interact with graphrag as a library using the API as opposed to the CLI. Note that graphrag's CLI actually connects to the library through this API for all operations.
+
+
+
+
+
+
+
+
+
+
+In [2]:
+
+
+
+
+
+
+
+
+
+Copied!
+
+
+import graphrag.api as api
+from graphrag.index.typing import PipelineRunResult
+import graphrag.api as api
+from graphrag.index.typing import PipelineRunResult
+
+
+
+
+
+
+
+
+
+
+
+++
+
+
+
+
+
+
+
+Load
+
+Prerequisite¶
As a prerequisite to all API operations, a GraphRagConfig object is required. It is the primary means to control the behavior of graphrag and can be instantiated from a settings.yaml configuration file.
Please refer to the CLI docs for more detailed information on how to generate the settings.yaml file.
Load settings.yaml configuration¶
+
+
+
+
+
+
+
+
+
+
+In [3]:
+
+
+
+
+
+
+
+
+
+Copied!
+
+
+import yaml
+
+settings = yaml.safe_load(open("<project_directory>/settings.yaml")) # noqa: PTH123, SIM115
+import yaml
+
+settings = yaml.safe_load(open("/settings.yaml")) # noqa: PTH123, SIM115
+
+
+
+
+
+
+
+
+
+
+
++--------------------------------------------------------------------------- +FileNotFoundError Traceback (most recent call last) +Cell In[3], line 3 + 1 import yaml +----> 3 settings = yaml.safe_load(open("<project_directory>/settings.yaml")) # noqa: PTH123, SIM115 + +File ~/.cache/pypoetry/virtualenvs/graphrag-F2jvqev7-py3.11/lib/python3.11/site-packages/IPython/core/interactiveshell.py:324, in _modified_open(file, *args, **kwargs) + 317 if file in {0, 1, 2}: + 318 raise ValueError( + 319 f"IPython won't let you open fd={file} by default " + 320 "as it is likely to crash IPython. If you know what you are doing, " + 321 "you can use builtins' open." + 322 ) +--> 324 return io_open(file, *args, **kwargs) + +FileNotFoundError: [Errno 2] No such file or directory: '<project_directory>/settings.yaml'+
+
+
+
+
+
+
+
+
+
+At this point, you can modify the imported settings to align with your application's requirements. For example, if building a UI application, the application might need to change the input and/or storage destinations dynamically in order to enable users to build and query different indexes.
+
+
+
+
+
+
+
+
+Generate a
+
+Generate a GraphRagConfig object¶
+
+
+
+
+
+
+
+
+
+
+In [4]:
+
+
+
+
+
+
+
+
+
+Copied!
+
+
+from graphrag.config.create_graphrag_config import create_graphrag_config
+
+graphrag_config = create_graphrag_config(
+ values=settings, root_dir="<project_directory>"
+)
+from graphrag.config.create_graphrag_config import create_graphrag_config
+
+graphrag_config = create_graphrag_config(
+ values=settings, root_dir=""
+)
+
+
+
+
+
+
+
+
+
+
+
++--------------------------------------------------------------------------- +NameError Traceback (most recent call last) +Cell In[4], line 4 + 1 from graphrag.config.create_graphrag_config import create_graphrag_config + 3 graphrag_config = create_graphrag_config( +----> 4 values=settings, root_dir="<project_directory>" + 5 ) + +NameError: name 'settings' is not defined+
+
+
+
+
+
+
+
+
+
+Indexing API¶
Indexing is the process of ingesting raw text data and constructing a knowledge graph. GraphRAG currently supports plaintext (.txt) and .csv file formats.
+
+
+
+
+
+
+
+
+Build an index¶
+
+
+
+
+
+
+
+
+
+
+In [5]:
+
+
+
+
+
+
+
+
+
+Copied!
+
+
+index_result: list[PipelineRunResult] = await api.build_index(config=graphrag_config)
+
+# index_result is a list of workflows that make up the indexing pipeline that was run
+for workflow_result in index_result:
+ status = f"error\n{workflow_result.errors}" if workflow_result.errors else "success"
+ print(f"Workflow Name: {workflow_result.workflow}\tStatus: {status}")
+index_result: list[PipelineRunResult] = await api.build_index(config=graphrag_config)
+
+# index_result is a list of workflows that make up the indexing pipeline that was run
+for workflow_result in index_result:
+ status = f"error\n{workflow_result.errors}" if workflow_result.errors else "success"
+ print(f"Workflow Name: {workflow_result.workflow}\tStatus: {status}")
+
+
+
+
+
+
+
+
+
+
+
++--------------------------------------------------------------------------- +NameError Traceback (most recent call last) +Cell In[5], line 1 +----> 1 index_result: list[PipelineRunResult] = await api.build_index(config=graphrag_config) + 3 # index_result is a list of workflows that make up the indexing pipeline that was run + 4 for workflow_result in index_result: + +NameError: name 'graphrag_config' is not defined+
+
+
+
+
+
+
+
+
+Query an index¶
To query an index, several index files must first be read into memory and passed to the query API.
+
+
+
+
+
+
+
+
+
+
+In [6]:
+
+
+
+
+
+
+
+
+
+Copied!
+
+
+import pandas as pd
+
+final_nodes = pd.read_parquet("<project_directory>/output/create_final_nodes.parquet")
+final_entities = pd.read_parquet(
+ "<project_directory>/output/create_final_entities.parquet"
+)
+final_communities = pd.read_parquet(
+ "<project_directory>/output/create_final_communities.parquet"
+)
+final_community_reports = pd.read_parquet(
+ "<project_directory>/output/create_final_community_reports.parquet"
+)
+
+response, context = await api.global_search(
+ config=graphrag_config,
+ nodes=final_nodes,
+ entities=final_entities,
+ communities=final_communities,
+ community_reports=final_community_reports,
+ community_level=2,
+ dynamic_community_selection=False,
+ response_type="Multiple Paragraphs",
+ query="Who is Scrooge and what are his main relationships?",
+)
+import pandas as pd
+
+final_nodes = pd.read_parquet("/output/create_final_nodes.parquet")
+final_entities = pd.read_parquet(
+ "/output/create_final_entities.parquet"
+)
+final_communities = pd.read_parquet(
+ "/output/create_final_communities.parquet"
+)
+final_community_reports = pd.read_parquet(
+ "/output/create_final_community_reports.parquet"
+)
+
+response, context = await api.global_search(
+ config=graphrag_config,
+ nodes=final_nodes,
+ entities=final_entities,
+ communities=final_communities,
+ community_reports=final_community_reports,
+ community_level=2,
+ dynamic_community_selection=False,
+ response_type="Multiple Paragraphs",
+ query="Who is Scrooge and what are his main relationships?",
+)
+
+
+
+
+
+
+
+
+
+
+
++--------------------------------------------------------------------------- +FileNotFoundError Traceback (most recent call last) +Cell In[6], line 3 + 1 import pandas as pd +----> 3 final_nodes = pd.read_parquet("<project_directory>/output/create_final_nodes.parquet") + 4 final_entities = pd.read_parquet( + 5 "<project_directory>/output/create_final_entities.parquet" + 6 ) + 7 final_communities = pd.read_parquet( + 8 "<project_directory>/output/create_final_communities.parquet" + 9 ) + +File ~/.cache/pypoetry/virtualenvs/graphrag-F2jvqev7-py3.11/lib/python3.11/site-packages/pandas/io/parquet.py:667, in read_parquet(path, engine, columns, storage_options, use_nullable_dtypes, dtype_backend, filesystem, filters, **kwargs) + 664 use_nullable_dtypes = False + 665 check_dtype_backend(dtype_backend) +--> 667 return impl.read( + 668 path, + 669 columns=columns, + 670 filters=filters, + 671 storage_options=storage_options, + 672 use_nullable_dtypes=use_nullable_dtypes, + 673 dtype_backend=dtype_backend, + 674 filesystem=filesystem, + 675 **kwargs, + 676 ) + +File ~/.cache/pypoetry/virtualenvs/graphrag-F2jvqev7-py3.11/lib/python3.11/site-packages/pandas/io/parquet.py:267, in PyArrowImpl.read(self, path, columns, filters, use_nullable_dtypes, dtype_backend, storage_options, filesystem, **kwargs) + 264 if manager == "array": + 265 to_pandas_kwargs["split_blocks"] = True # type: ignore[assignment] +--> 267 path_or_handle, handles, filesystem = _get_path_or_handle( + 268 path, + 269 filesystem, + 270 storage_options=storage_options, + 271 mode="rb", + 272 ) + 273 try: + 274 pa_table = self.api.parquet.read_table( + 275 path_or_handle, + 276 columns=columns, + (...) + 279 **kwargs, + 280 ) + +File ~/.cache/pypoetry/virtualenvs/graphrag-F2jvqev7-py3.11/lib/python3.11/site-packages/pandas/io/parquet.py:140, in _get_path_or_handle(path, fs, storage_options, mode, is_dir) + 130 handles = None + 131 if ( + 132 not fs + 133 and not is_dir + (...) + 138 # fsspec resources can also point to directories + 139 # this branch is used for example when reading from non-fsspec URLs +--> 140 handles = get_handle( + 141 path_or_handle, mode, is_text=False, storage_options=storage_options + 142 ) + 143 fs = None + 144 path_or_handle = handles.handle + +File ~/.cache/pypoetry/virtualenvs/graphrag-F2jvqev7-py3.11/lib/python3.11/site-packages/pandas/io/common.py:882, in get_handle(path_or_buf, mode, encoding, compression, memory_map, is_text, errors, storage_options) + 873 handle = open( + 874 handle, + 875 ioargs.mode, + (...) + 878 newline="", + 879 ) + 880 else: + 881 # Binary mode +--> 882 handle = open(handle, ioargs.mode) + 883 handles.append(handle) + 885 # Convert BytesIO or file objects passed with an encoding + +FileNotFoundError: [Errno 2] No such file or directory: '<project_directory>/output/create_final_nodes.parquet'+
+
+
+
+
+
+
+
+
+The response object is the official reponse from graphrag while the context object holds various metadata regarding the querying process used to obtain the final response.
+
+
+
+
+
+
+
+
+
+
+In [7]:
+
+
+
+
+
+
+
+
+
+Copied!
+
+
+print(response)
+print(response)
+
+
+
+
+
+
+
+
+
+
+
++--------------------------------------------------------------------------- +NameError Traceback (most recent call last) +Cell In[7], line 1 +----> 1 print(response) + +NameError: name 'response' is not defined+
+
+
+
+
+
+
+
+
+Digging into the context a bit more provides users with extremely granular information such as what sources of data (down to the level of text chunks) were ultimately retrieved and used as part of the context sent to the LLM model).
+
+
+
+
+
+
+
+
+
+
+In [8]:
+
+
+
+
+
+
+
+
+
+Copied!
+
+
+from pprint import pprint
+
+pprint(context) # noqa: T203
+from pprint import pprint
+
+pprint(context) # noqa: T203
+
+
+
+
+
+
+
+
+
+
+
++--------------------------------------------------------------------------- +NameError Traceback (most recent call last) +Cell In[8], line 3 + 1 from pprint import pprint +----> 3 pprint(context) # noqa: T203 + +NameError: name 'context' is not defined+