465 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Nathan Evans	780a46d1b1	Push drift_k_followups through to prompt	2026-01-12 18:00:21 -08:00
Nathan Evans	a380a58f4b	Fix DRIFT notebook	2026-01-12 17:22:48 -08:00
Nathan Evans	3d6b182a46	Fix mock embedder to match default embedding length	2026-01-12 16:38:39 -08:00
Nathan Evans	1fcb5cfaad	Add drift back to smoke tests	2026-01-12 16:01:33 -08:00
Nathan Evans	7ccf8d43a8	Fix lancedb insertion	2026-01-12 15:27:02 -08:00
Nathan Evans	f4a20cd73d	Remove embedding column from df loaders	2026-01-12 14:41:04 -08:00
Nathan Evans	a8c1772340	Remove deprecated title from embedding flow	2026-01-12 14:40:46 -08:00
Nathan Evans	a4d1278c2a	Use stable ids for community reports	2026-01-12 14:40:30 -08:00
Nathan Evans	710fdad6f0	Input factory (#2168) ... * Update input factory to match other factories * Move input config alongside input readers * Move file pattern logic into InputReader * Set encoding default * Clean up optional column configs * Combine structured data extraction * Remove pandas from input loading * Throw if empty documents * Add json lines (jsonl) input support * Store raw data * Fix merge imports * Move metadata handling entirely to chunking * Nicer automatic title * Typo * Add get_property utility for nested dictionary access with dot notation * Update structured_file_reader to use get_property utility * Extract input module into new graphrag-input monorepo package - Create new graphrag-input package with input loading utilities - Move InputConfig, InputFileType, InputReader, TextDocument, and file readers (CSV, JSON, JSONL, Text) - Add get_property utility for nested dictionary access with dot notation - Include hashing utility for document ID generation - Update all imports throughout codebase to use graphrag_input - Add package to workspace configuration and release tasks - Remove old graphrag.index.input module * Rename ChunkResult to TextChunk and add transformer support - Rename chunk_result.py to text_chunk.py with ChunkResult -> TextChunk - Add 'original' field to TextChunk to track pre-transform text - Add optional transform callback to chunker.chunk() method - Add add_metadata transformer for prepending metadata to chunks - Update create_chunk_results to apply transforms and populate original - Update sentence_chunker and token_chunker with transform support - Refactor create_base_text_units to use new transformer pattern - Rename pluck_metadata to get/collect methods on TextDocument * Back-compat comment * Align input config type name with other factory configs * Add MarkItDown support * Remove pattern default from MarkItDown reader * Remove plugins flag (implicit disabled) * Format * Update verb tests * Separate storage from input config * Add empty objects for NaN raw_data * Fix smoke tests * Fix BOM in csv smoke * Format	2026-01-12 12:47:57 -08:00
Nathan Evans	8fd7730067	Chunker factory (#2156) ... * Delete NoopTextSplitter * Delete unused check_token_limit * Add base chunking factory and migrate workflow to use it * Split apart chunker module * Co-locate chunking/splitting * Collapse token splitting functionality into one class/function * Restore create_base_text_units parameterization * Move Tokenizer base class to common package * Move pre-pending into chunkers * Streamline config * Fix defaults construction * Add prepending tests * Remove chunk_size_includes_metadata config * Revert ChunkingDocument interface * Move metadata prepending to a util * Move Tokenizer back to GR core * Fix tokenizer removal from chunker * Set defaults for chunking config * Move chunking to monorepo package * Format * Typo * Add ChunkResult model * Streamline chunking config * Add missing version updates for graphrag_chunking	2026-01-06 15:39:44 -08:00
gaudyb	fde14b63e5	Mismatch between header in community report generation prompt examples and input data (id vs human_readable_id) (#2161) ... * fix issue #860 for mismatch in prompts and input * fix format --------- Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>	2025-12-30 18:08:51 -06:00
gaudyb	c649d9f6ee	Issue #2004 fix (#2159) ... * fix issue #2004 using KeenhoChu idea in his PR * add unit test for dynamic community selection * add unit test for dynamic community selection implementing #2158 logic --------- Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>	2025-12-30 18:08:10 -06:00
Nathan Evans	c296f1ae15	Fix a bunch of module comments and function visibility (#2154)	2025-12-17 10:55:26 -08:00
Derek Worthen	3201f28bea	Add GraphRAG Cache package. (#2153) ... * Add GraphRAG Cache package.	2025-12-16 06:37:28 -08:00
Nathan Evans	bffa400c89	Python update (3.13) (#2149) ... * Update to python 3.14 as default, with range down to 3.10 * Fix enum value in query cli * Update pyarrow * Update py version for storage package * Remove 3.10 * add fastuuid * Update Python support to 3.11-3.14 with stricter dependency constraints - Set minimum Python version to 3.11 (removed 3.10 support) - Added support for Python 3.14 - Updated CI workflows: single-version jobs use 3.14, matrix jobs use 3.11 and 3.14 - Fixed license format to use SPDX-compatible format for Python 3.14 - Updated pyarrow to >=22.0.0 for Python 3.14 wheel support - Added explicit fastuuid~=0.14 and blis~=1.3 for Python 3.14 compatibility - Replaced all loose version constraints (>=) with compatible release (~=) for better lock file control - Applied stricter versioning to all packages: graphrag, graphrag-common, graphrag-storage, unified-search-app * update uv lock * Pin blis to ~=1.3.3 to ensure Python 3.14 wheel availability * Update uv lock * Update numpy to >=2.0.0 for Python 3.14 Windows compatibility Numpy 1.25.x has access violation issues on Python 3.14 Windows. Numpy 2.x has proper Python 3.14 support including Windows wheels. * update uv lock * Update pandas to >=2.3.0 for numpy 2.x compatibility Pandas 2.2.x was compiled against numpy 1.x and causes ABI incompatibility errors with numpy 2.x. Pandas 2.3.0+ supports numpy 2.x properly. * update uv.lock * Add scipy>=1.15.0 for numpy 2.x compatibility Scipy versions < 1.15.0 have C extensions built against numpy 1.x and are incompatible with numpy 2.x, causing dtype size errors. * update uv lock * Update Python support to 3.11-3.13 with compatible dependencies - Set Python version range to 3.11-3.13 (removed 3.14 support) - Updated CI workflows: single-version jobs use 3.13, matrix jobs use 3.11 and 3.13 - Dependencies optimized for Python 3.13 compatibility: - pyarrow~=22.0 (has Python 3.13 wheels) - numpy~=1.26 - pandas~=2.2 - blis~=1.0 - fastuuid~=0.13 - Applied stricter version constraints using ~= operator throughout - Updated uv.lock with resolved dependencies * Update numpy to 2.1+ and pandas to 2.3+ for Python 3.13 Windows compatibility Numpy 1.26.x causes access violations on Python 3.13 Windows. Numpy 2.1+ has proper Python 3.13 support with Windows wheels. Pandas 2.3+ is required for numpy 2.x compatibility. * update vsts.yml python version	2025-12-15 15:39:38 -08:00
Derek Worthen	4404668aa8	Add graphrag-storage. (#2127) ... * Add graphrag-storage.	2025-12-15 09:32:19 -08:00
Nathan Evans	20a96cb35e	Init command asks for models (#2137) ... * Add init prompting for models * Remove hard-coded model config validation * Switch to typer option prompt for full CLI use with models * Update getting started for init model input * Bump request timeout and overall smoke test timeout	2025-11-24 10:05:47 -08:00
Nathan Evans	7bf82b78ab	Add empty checks for NLP graphs (#2133)	2025-11-17 15:45:07 -08:00
Nathan Evans	d6e6191d84	Format	2025-11-17 13:59:58 -08:00
gaudyb	a4ffc3d34c	Remove embeddings optional new (#2128) ... * remove optional embeddings * fix test * fix tests * fix pipeline * fix test * fix test * fix test * fix tests --------- Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>	2025-11-17 13:10:54 -06:00
Nathan Evans	4512ce027f	Empty graph guards (#2126) ... * Remove networkx from graph_extractor and clean out redundancy * Bubble pipeline error to console	2025-11-11 10:11:08 -08:00
Derek Worthen	e0cce31f54	Graphrag config (#2119) ... * Add load_config to graphrag-common package.	2025-11-10 07:57:03 -08:00
Nathan Evans	ae1f5e1811	Nov 2025 housekeeping (#2120) ... * Remove gensim sideload * Split CI build/type checks from unit tests * Thorough review of docs to align with v3 * Format * Fix version * Fix type	2025-11-06 10:03:22 -08:00
Nathan Evans	6033e4ffa2	Storage fixes and cleanup (#2118) ... * Fix pipeline recursion * Remove base_dir from storage.find * Remove max_count from storage.find * Remove prefix on storage integ test * Add base_dir in creation_date test * Wrap base_dir in Path * Use constants for input/update directories	2025-11-05 13:06:09 -08:00
Nathan Evans	6b03af6277	Fix formatting	2025-11-04 10:48:09 -08:00
Derek Worthen	619269243d	Restructure project as monorepo. (#2111) ... * Restructure project as monorepo.	2025-11-04 09:51:56 -08:00
Gaudy Blanco	c43a58c353	fix for container name	2025-10-23 10:40:21 -06:00
gaudyb	542d3db2a9	Prefix vector store (#2106) ... * add prefix to vector store configuration and removal of container name * docs updated * change prefix property name * change prefix property name * feedback implemented --------- Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>	2025-10-21 15:30:50 -06:00
Nathan Evans	1bb9fa8e13	Unified factory (#2105) ... * Simplify Factory interface * Migrate CacheFactory to standard base class * Migrate LoggerFactory to standard base class * Migrate StorageFactory to standard base class * Migrate VectorStoreFactory to standard base class * Update vector store example notebook * Delete notebook outputs * Move default providers into factories * Move retry/limit tests into integ * Split language model factories * Set smoke test tpm/rpm * Fix factory integ tests * Add method to smoke test, switch text to 'fast' * Fix text smoke config for fast workflow * Add new workflows to text smoke test * Convert input readers to a proper factory * Remove covariates from fast smoke test * Update docs for input factory * Bump smoke runtime * Even longer runtime * min-csv timeout * Remove unnecessary lambdas	2025-10-20 12:05:27 -07:00
gaudyb	0436405962	Remove document overwrite (#2101) ... * remove document overwrite from vector store configuration * remove document overwrite and refactor load documents method * fix test * fix test * fix test --------- Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>	2025-10-16 07:56:54 -06:00
Nathan Evans	5ec49fd39c	V3 docs and cleanup (#2100) ... * Remove community contrib notebooks * Add migration notebook and breaking changes page edits * Update/polish docs * Make model instance name configurable * Add vector schema updates to v3 migration notebook * Spellcheck * Bump smoke test runtimes	2025-10-15 13:47:19 -07:00
Nathan Evans	b732445535	Remove multi search (#2093) ... * Remove multi-search from CLI * Remove multi-search from API * Flatten vector_store config * Push hydrated vector store down to embed_text * Remove outputs from config * Remove multi-search notebook/docs * Add missing response_type in basic search API * Fix basic search context and id mapping * Fix v1 migration notebook * Fix query entity search tests	2025-10-10 17:20:53 -07:00
Nathan Evans	715be61404	Sort deps alpha	2025-10-10 17:10:42 -07:00
Nathan Evans	f7a8a0888d	Merge branch 'main' into v3/main	2025-10-10 17:03:52 -07:00
Nathan Evans	6284cdd110	Remove fnllm (#2095)	2025-10-10 16:59:25 -07:00
Nathan Evans	eb0dfe376b	Remove strategy dicts (#2090) ... * Remove "strategy" from community reports config/workflow * Remove extraction strategy from extract_graph * Remove summarization strategy from extract_graph * Remove strategy from claim extraction * Strongly type prompt templates * Remove strategy from embed_text * Push hydrated params into community report workflows * Push hyrdated params into extract covariates * Push hydrated params into extract graph NLP * Push hydrated params into extract graph * Push hydrated params into text embeddings * Remove a few more low-level defaults * Semver * Remove configurable prompt delimiters * Update smoke tests	2025-10-10 12:15:23 -07:00
gaudyb	79ad9b96f3	reduce schema fields (#2089) ... * reduce schema fields * fix launch.json --------- Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>	2025-10-09 13:41:31 -06:00
Alonso Guevara	fdb7e3835b	Release v2.7.0 (#2087)	2025-10-08 21:33:34 -07:00
Nathan Evans	2b5284ca1b	Merge branch 'main' into v3/main	2025-10-07 16:24:15 -07:00
Nathan Evans	ac8a7f5eef	Housekeeping (#2086) ... * Add deprecation warnings for fnllm and multi-search * Fix dangling token_encoder refs * Fix local_search notebook * Fix global search dynamic notebook * Fix global search notebook * Fix drift notebook * Switch example notebooks to use LiteLLM config * Properly annotate dev deps as a group * Semver * Remove --extra dev * Remove llm_model variable * Ignore ruff ASYNC240 * Add note about expected broken notebook in docs * Fix custom vector store notebook * Push tokenizer throughout	2025-10-07 16:21:24 -07:00
Nathan Evans	4364d678dd	Merge branch 'main' into v3/main	2025-10-06 12:41:12 -07:00
Nathan Evans	6c86b0a7bb	Init config cleanup (#2084) ... * Spruce up init_config output, including LiteLLM default * Remove deployment_name requirement for Azure * Semver * Add model_provider * Add default model_provider * Remove OBE test * Update minimal config for tests * Add model_provider to verb tests	2025-10-06 12:06:41 -07:00
Nathan Evans	2bd3922d8d	Litellm auth fix (#2083) ... * Fix scope for Azure auth with LiteLLM * Change internal language on max_attempts to max_retries * Rework model config connectivity validation * Semver * Swtich smoke tests to LiteLLM * Take out temporary retry_strategy = none since it is not fnllm compatible * Bump smoke test timeout * Bump smoke timeout further * Tune smoke params * Update smoke test bounds * Remove covariates from min-csv smoke * Smoke: adjust communities, remove drift * Remove secrets where they aren't necessary * Clean out old env var references	2025-10-06 10:54:21 -07:00
Andres Morales	ebe959a9db	Update v3/main missing config + functions (#2082)	2025-09-30 15:48:40 -06:00
gaudyb	d7773bd15c	Clean vector store (#2077) ... * clean vector store code * fix * fix launch.json --------- Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>	2025-09-25 21:17:10 -06:00
Nathan Evans	d7516825e2	Update comment	2025-09-24 14:51:07 -07:00
Nathan Evans	de767cc881	Use 4.1 and text-embedding-3-large as defaults	2025-09-24 14:49:55 -07:00
Nathan Evans	7f996cf584	Docs/2.6.0 (#2070) ... * Add basic search to overview * Add info on input documents DataFrame * Add info on factories to docs * Add consumption warning and switch to "christmas" for folder name * Add logger to factories list * Add litellm docs. (#2058) * Fix version for input docs * Spelling --------- Co-authored-by: Derek Worthen <worthend.derek@gmail.com>	2025-09-23 14:48:28 -07:00
Nathan Evans	b73053010e	Merge branch 'main' into v3/main	2025-09-23 11:07:42 -07:00
Alonso Guevara	9bc899fe95	Release v2.6.0 (#2068)	2025-09-22 16:16:54 -06:00