Switch from Poetry to uv for package management (#2008)
Some checks are pending
gh-pages / build (push) Waiting to run
Python CI / python-ci (ubuntu-latest, 3.10) (push) Waiting to run
Python CI / python-ci (ubuntu-latest, 3.11) (push) Waiting to run
Python CI / python-ci (windows-latest, 3.10) (push) Waiting to run
Python CI / python-ci (windows-latest, 3.11) (push) Waiting to run
Python Integration Tests / python-ci (ubuntu-latest, 3.10) (push) Waiting to run
Python Integration Tests / python-ci (windows-latest, 3.10) (push) Waiting to run
Python Notebook Tests / python-ci (ubuntu-latest, 3.10) (push) Waiting to run
Python Notebook Tests / python-ci (windows-latest, 3.10) (push) Waiting to run
Python Publish (pypi) / Upload release to PyPI (push) Waiting to run
Python Smoke Tests / python-ci (ubuntu-latest, 3.10) (push) Waiting to run
Python Smoke Tests / python-ci (windows-latest, 3.10) (push) Waiting to run
Spellcheck / spellcheck (push) Waiting to run

* Initial plan

* Switch from Poetry to uv for package management

Co-authored-by: jgbradley1 <654554+jgbradley1@users.noreply.github.com>

* Clean up build artifacts and update gitignore

Co-authored-by: jgbradley1 <654554+jgbradley1@users.noreply.github.com>

* remove build artifacts

* remove hardcoded version string

* fix calls to pip in cicd

* Update gh-pages.yml workflow to use uv instead of Poetry

Co-authored-by: jgbradley1 <654554+jgbradley1@users.noreply.github.com>

* ruff formatting fixes

* update cicd workflow with latest uv action

* fix command to retrieve package version

* update development instructions

* remove Poetry references

* Replace deprecated azuright action with npm-based Azurite installation

Co-authored-by: jgbradley1 <654554+jgbradley1@users.noreply.github.com>

* skip api version check for azurite

* add semversioner file

* update more changes from switching to UV

* Migrate unified-search-app from Poetry to uv package management

Co-authored-by: jgbradley1 <654554+jgbradley1@users.noreply.github.com>

* minor typo update

* minor Dockerfile update

* update cicd thresholds

* update pytest thresholds

* ruff fixes

* ruff fixes

* remove legacy npm settings that no longer apply

* Update Unified Search App Readme

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: jgbradley1 <654554+jgbradley1@users.noreply.github.com>
Co-authored-by: Josh Bradley <joshbradley@microsoft.com>
Co-authored-by: Alonso Guevara <alonsog@microsoft.com>
This commit is contained in:
Copilot 2025-08-13 18:57:25 -06:00 committed by GitHub
parent 5713205210
commit 7c28c70d5c
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
28 changed files with 7407 additions and 11270 deletions

View File

@ -6,8 +6,7 @@ permissions:
contents: write contents: write
env: env:
POETRY_VERSION: '1.8.3' PYTHON_VERSION: "3.11"
PYTHON_VERSION: '3.11'
jobs: jobs:
build: build:
@ -29,18 +28,16 @@ jobs:
with: with:
python-version: ${{ env.PYTHON_VERSION }} python-version: ${{ env.PYTHON_VERSION }}
- name: Install Poetry ${{ env.POETRY_VERSION }} - name: Install uv
uses: abatilo/actions-poetry@v3.0.0 uses: astral-sh/setup-uv@v6
with:
poetry-version: ${{ env.POETRY_VERSION }}
- name: poetry intsall - name: Install dependencies
shell: bash shell: bash
run: poetry install run: uv sync --extra dev
- name: mkdocs build - name: mkdocs build
shell: bash shell: bash
run: poetry run poe build_docs run: uv run poe build_docs
- name: List Docsite Contents - name: List Docsite Contents
run: find site run: find site

View File

@ -26,9 +26,6 @@ concurrency:
# Only run the for the latest commit # Only run the for the latest commit
cancel-in-progress: true cancel-in-progress: true
env:
POETRY_VERSION: 1.8.3
jobs: jobs:
python-ci: python-ci:
# skip draft PRs # skip draft PRs
@ -51,7 +48,7 @@ jobs:
filters: | filters: |
python: python:
- 'graphrag/**/*' - 'graphrag/**/*'
- 'poetry.lock' - 'uv.lock'
- 'pyproject.toml' - 'pyproject.toml'
- '**/*.py' - '**/*.py'
- '**/*.toml' - '**/*.toml'
@ -64,30 +61,27 @@ jobs:
with: with:
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
- name: Install Poetry - name: Install uv
uses: abatilo/actions-poetry@v3.0.0 uses: astral-sh/setup-uv@v6
with:
poetry-version: $POETRY_VERSION
- name: Install dependencies - name: Install dependencies
shell: bash shell: bash
run: | run: |
poetry self add setuptools wheel uv sync --extra dev
poetry run python -m pip install gensim uv pip install gensim
poetry install
- name: Check - name: Check
run: | run: |
poetry run poe check uv run poe check
- name: Build - name: Build
run: | run: |
poetry build uv build
- name: Unit Test - name: Unit Test
run: | run: |
poetry run poe test_unit uv run poe test_unit
- name: Verb Test - name: Verb Test
run: | run: |
poetry run poe test_verbs uv run poe test_verbs

View File

@ -26,9 +26,6 @@ concurrency:
# only run the for the latest commit # only run the for the latest commit
cancel-in-progress: true cancel-in-progress: true
env:
POETRY_VERSION: 1.8.3
jobs: jobs:
python-ci: python-ci:
# skip draft PRs # skip draft PRs
@ -51,7 +48,7 @@ jobs:
filters: | filters: |
python: python:
- 'graphrag/**/*' - 'graphrag/**/*'
- 'poetry.lock' - 'uv.lock'
- 'pyproject.toml' - 'pyproject.toml'
- '**/*.py' - '**/*.py'
- '**/*.toml' - '**/*.toml'
@ -64,25 +61,24 @@ jobs:
with: with:
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
- name: Install Poetry - name: Install uv
uses: abatilo/actions-poetry@v3.0.0 uses: astral-sh/setup-uv@v6
with:
poetry-version: $POETRY_VERSION
- name: Install dependencies - name: Install dependencies
shell: bash shell: bash
run: | run: |
poetry self add setuptools wheel uv sync --extra dev
poetry run python -m pip install gensim uv pip install gensim
poetry install
- name: Build - name: Build
run: | run: |
poetry build uv build
- name: Install Azurite - name: Install and start Azurite
id: azuright shell: bash
uses: potatoqualitee/azuright@v1.1 run: |
npm install -g azurite
azurite --silent --skipApiVersionCheck --location /tmp/azurite --debug /tmp/azurite-debug.log &
# For more information on installation/setup of Azure Cosmos DB Emulator # For more information on installation/setup of Azure Cosmos DB Emulator
# https://learn.microsoft.com/en-us/azure/cosmos-db/how-to-develop-emulator?tabs=docker-linux%2Cpython&pivots=api-nosql # https://learn.microsoft.com/en-us/azure/cosmos-db/how-to-develop-emulator?tabs=docker-linux%2Cpython&pivots=api-nosql
@ -97,4 +93,4 @@ jobs:
- name: Integration Test - name: Integration Test
run: | run: |
poetry run poe test_integration uv run poe test_integration

View File

@ -26,9 +26,6 @@ concurrency:
# Only run the for the latest commit # Only run the for the latest commit
cancel-in-progress: true cancel-in-progress: true
env:
POETRY_VERSION: 1.8.3
jobs: jobs:
python-ci: python-ci:
# skip draft PRs # skip draft PRs
@ -54,7 +51,7 @@ jobs:
filters: | filters: |
python: python:
- 'graphrag/**/*' - 'graphrag/**/*'
- 'poetry.lock' - 'uv.lock'
- 'pyproject.toml' - 'pyproject.toml'
- '**/*.py' - '**/*.py'
- '**/*.toml' - '**/*.toml'
@ -66,18 +63,15 @@ jobs:
with: with:
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
- name: Install Poetry - name: Install uv
uses: abatilo/actions-poetry@v3.0.0 uses: astral-sh/setup-uv@v6
with:
poetry-version: $POETRY_VERSION
- name: Install dependencies - name: Install dependencies
shell: bash shell: bash
run: | run: |
poetry self add setuptools wheel uv sync --extra dev
poetry run python -m pip install gensim uv pip install gensim
poetry install
- name: Notebook Test - name: Notebook Test
run: | run: |
poetry run poe test_notebook uv run poe test_notebook

View File

@ -6,7 +6,6 @@ on:
branches: [main] branches: [main]
env: env:
POETRY_VERSION: "1.8.3"
PYTHON_VERSION: "3.10" PYTHON_VERSION: "3.10"
jobs: jobs:
@ -31,21 +30,19 @@ jobs:
with: with:
python-version: ${{ env.PYTHON_VERSION }} python-version: ${{ env.PYTHON_VERSION }}
- name: Install Poetry - name: Install uv
uses: abatilo/actions-poetry@v3.0.0 uses: astral-sh/setup-uv@v6
with:
poetry-version: ${{ env.POETRY_VERSION }}
- name: Install dependencies - name: Install dependencies
shell: bash shell: bash
run: poetry install run: uv sync
- name: Export Publication Version - name: Export Publication Version
run: echo "version=`poetry version --short`" >> $GITHUB_OUTPUT run: echo "version=$(uv version --short)" >> $GITHUB_OUTPUT
- name: Build Distributable - name: Build Distributable
shell: bash shell: bash
run: poetry build run: uv build
- name: Publish package distributions to PyPI - name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1 uses: pypa/gh-action-pypi-publish@release/v1

View File

@ -26,9 +26,6 @@ concurrency:
# Only run the for the latest commit # Only run the for the latest commit
cancel-in-progress: true cancel-in-progress: true
env:
POETRY_VERSION: 1.8.3
jobs: jobs:
python-ci: python-ci:
# skip draft PRs # skip draft PRs
@ -68,7 +65,7 @@ jobs:
filters: | filters: |
python: python:
- 'graphrag/**/*' - 'graphrag/**/*'
- 'poetry.lock' - 'uv.lock'
- 'pyproject.toml' - 'pyproject.toml'
- '**/*.py' - '**/*.py'
- '**/*.toml' - '**/*.toml'
@ -81,33 +78,32 @@ jobs:
with: with:
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
- name: Install Poetry - name: Install uv
uses: abatilo/actions-poetry@v3.0.0 uses: astral-sh/setup-uv@v6
with:
poetry-version: $POETRY_VERSION
- name: Install dependencies - name: Install dependencies
shell: bash shell: bash
run: | run: |
poetry self add setuptools wheel uv sync --extra dev
poetry run python -m pip install gensim uv pip install gensim
poetry install
- name: Build - name: Build
run: | run: |
poetry build uv build
- name: Install Azurite - name: Install and start Azurite
id: azuright shell: bash
uses: potatoqualitee/azuright@v1.1 run: |
npm install -g azurite
azurite --silent --skipApiVersionCheck --location /tmp/azurite --debug /tmp/azurite-debug.log &
- name: Smoke Test - name: Smoke Test
if: steps.changes.outputs.python == 'true' if: steps.changes.outputs.python == 'true'
run: | run: |
poetry run poe test_smoke uv run poe test_smoke
- uses: actions/upload-artifact@v4 - uses: actions/upload-artifact@v4
if: always() if: always()
with: with:
name: smoke-test-artifacts-${{ matrix.python-version }}-${{ matrix.poetry-version }}-${{ runner.os }} name: smoke-test-artifacts-${{ matrix.python-version }}-${{ runner.os }}
path: tests/fixtures/* path: tests/fixtures/*

2
.gitignore vendored
View File

@ -1,6 +1,8 @@
# Python Artifacts # Python Artifacts
python/*/lib/ python/*/lib/
dist/ dist/
build/
*.egg-info/
# Test Output # Test Output
.coverage .coverage

View File

@ -0,0 +1,4 @@
{
"type": "minor",
"description": "swap package management from Poetry -> UV"
}

6
.vscode/launch.json vendored
View File

@ -6,7 +6,7 @@
"name": "Indexer", "name": "Indexer",
"type": "debugpy", "type": "debugpy",
"request": "launch", "request": "launch",
"module": "poetry", "module": "uv",
"args": [ "args": [
"poe", "index", "poe", "index",
"--root", "<path_to_ragtest_root_demo>" "--root", "<path_to_ragtest_root_demo>"
@ -16,7 +16,7 @@
"name": "Query", "name": "Query",
"type": "debugpy", "type": "debugpy",
"request": "launch", "request": "launch",
"module": "poetry", "module": "uv",
"args": [ "args": [
"poe", "query", "poe", "query",
"--root", "<path_to_ragtest_root_demo>", "--root", "<path_to_ragtest_root_demo>",
@ -28,7 +28,7 @@
"name": "Prompt Tuning", "name": "Prompt Tuning",
"type": "debugpy", "type": "debugpy",
"request": "launch", "request": "launch",
"module": "poetry", "module": "uv",
"args": [ "args": [
"poe", "prompt-tune", "poe", "prompt-tune",
"--config", "--config",

37
.vscode/settings.json vendored
View File

@ -1,43 +1,8 @@
{ {
"search.exclude": {
"**/.yarn": true,
"**/.pnp.*": true
},
"editor.formatOnSave": false, "editor.formatOnSave": false,
"eslint.nodePath": ".yarn/sdks",
"typescript.tsdk": ".yarn/sdks/typescript/lib",
"typescript.enablePromptUseWorkspaceTsdk": true,
"javascript.preferences.importModuleSpecifier": "relative",
"javascript.preferences.importModuleSpecifierEnding": "js",
"typescript.preferences.importModuleSpecifier": "relative",
"typescript.preferences.importModuleSpecifierEnding": "js",
"explorer.fileNesting.enabled": true, "explorer.fileNesting.enabled": true,
"explorer.fileNesting.patterns": {
"*.ts": "${capture}.ts, ${capture}.hooks.ts, ${capture}.hooks.tsx, ${capture}.contexts.ts, ${capture}.stories.tsx, ${capture}.story.tsx, ${capture}.spec.tsx, ${capture}.base.ts, ${capture}.base.tsx, ${capture}.types.ts, ${capture}.styles.ts, ${capture}.styles.tsx, ${capture}.utils.ts, ${capture}.utils.tsx, ${capture}.constants.ts, ${capture}.module.scss, ${capture}.module.css, ${capture}.md",
"*.js": "${capture}.js.map, ${capture}.min.js, ${capture}.d.ts",
"*.jsx": "${capture}.js",
"*.tsx": "${capture}.ts, ${capture}.hooks.ts, ${capture}.hooks.tsx, ${capture}.contexts.ts, ${capture}.stories.tsx, ${capture}.story.tsx, ${capture}.spec.tsx, ${capture}.base.ts, ${capture}.base.tsx, ${capture}.types.ts, ${capture}.styles.ts, ${capture}.styles.tsx, ${capture}.utils.ts, ${capture}.utils.tsx, ${capture}.constants.ts, ${capture}.module.scss, ${capture}.module.css, ${capture}.md, ${capture}.css",
"tsconfig.json": "tsconfig.*.json",
"package.json": "package-lock.json, turbo.json, tsconfig.json, rome.json, biome.json, .npmignore, dictionary.txt, cspell.config.yaml",
"README.md": "*.md, LICENSE, CODEOWNERS",
".eslintrc": ".eslintignore",
".prettierrc": ".prettierignore",
".gitattributes": ".gitignore",
".yarnrc.yml": "yarn.lock, .pnp.*",
"jest.config.js": "jest.setup.mjs",
"pyproject.toml": "poetry.lock, poetry.toml, mkdocs.yaml",
"cspell.config.yaml": "dictionary.txt"
},
"azureFunctions.postDeployTask": "npm install (functions)",
"azureFunctions.projectLanguage": "TypeScript",
"azureFunctions.projectRuntime": "~4",
"debug.internalConsoleOptions": "neverOpen", "debug.internalConsoleOptions": "neverOpen",
"azureFunctions.preDeployTask": "npm prune (functions)", "python.defaultInterpreterPath": "${workspaceRoot}/.venv/bin/python",
"appService.zipIgnorePattern": [
"node_modules{,/**}",
".vscode{,/**}"
],
"python.defaultInterpreterPath": "python/services/.venv/bin/python",
"python.languageServer": "Pylance", "python.languageServer": "Pylance",
"cSpell.customDictionaries": { "cSpell.customDictionaries": {
"project-words": { "project-words": {

View File

@ -22,7 +22,7 @@ or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any addi
2. Create a new branch for your contribution: `git checkout -b my-contribution`. 2. Create a new branch for your contribution: `git checkout -b my-contribution`.
3. Make your changes and ensure that the code passes all tests. 3. Make your changes and ensure that the code passes all tests.
4. Commit your changes: `git commit -m "Add my contribution"`. 4. Commit your changes: `git commit -m "Add my contribution"`.
5. Create and commit a semver impact document by running `poetry run semversioner add-change -t <major|minor|patch> -d <description>`. 5. Create and commit a semver impact document by running `uv run semversioner add-change -t <major|minor|patch> -d <description>`.
6. Push your changes to your forked repository: `git push origin my-contribution`. 6. Push your changes to your forked repository: `git push origin my-contribution`.
7. Open a pull request to the main repository. 7. Open a pull request to the main repository.

View File

@ -5,29 +5,33 @@
| Name | Installation | Purpose | | Name | Installation | Purpose |
| ------------------- | ------------------------------------------------------------ | ----------------------------------------------------------------------------------- | | ------------------- | ------------------------------------------------------------ | ----------------------------------------------------------------------------------- |
| Python 3.10 or 3.11 | [Download](https://www.python.org/downloads/) | The library is Python-based. | | Python 3.10 or 3.11 | [Download](https://www.python.org/downloads/) | The library is Python-based. |
| Poetry | [Instructions](https://python-poetry.org/docs/#installation) | Poetry is used for package management and virtualenv management in Python codebases | | uv | [Instructions](https://docs.astral.sh/uv/) | uv is used for package management and virtualenv management in Python codebases |
# Getting Started # Getting Started
## Install Dependencies ## Install Dependencies
```shell ```shell
# (optional) create virtual environment
uv venv --python 3.10
source .venv/bin/activate
# install python dependencies # install python dependencies
poetry install uv sync --extra dev
``` ```
## Execute the indexing engine ## Execute the indexing engine
```shell ```shell
poetry run poe index <...args> uv run poe index <...args>
``` ```
## Execute prompt tuning ## Execute prompt tuning
```shell ```shell
poetry run poe prompt_tune <...args> uv run poe prompt_tune <...args>
``` ```
## Execute Queries ## Execute Queries
```shell ```shell
poetry run poe query <...args> uv run poe query <...args>
``` ```
## Repository Structure ## Repository Structure
@ -63,7 +67,7 @@ Where appropriate, the factories expose a registration method for users to provi
We use [semversioner](https://github.com/raulgomis/semversioner) to automate and enforce semantic versioning in the release process. Our CI/CD pipeline checks that all PR's include a json file generated by semversioner. When submitting a PR, please run: We use [semversioner](https://github.com/raulgomis/semversioner) to automate and enforce semantic versioning in the release process. Our CI/CD pipeline checks that all PR's include a json file generated by semversioner. When submitting a PR, please run:
```shell ```shell
poetry run semversioner add-change -t patch -d "<a small sentence describing changes made>." uv run semversioner add-change -t patch -d "<a small sentence describing changes made>."
``` ```
# Azurite # Azurite
@ -78,29 +82,29 @@ or by simply running `azurite` in the terminal if already installed globally. Se
# Lifecycle Scripts # Lifecycle Scripts
Our Python package utilizes Poetry to manage dependencies and [poethepoet](https://pypi.org/project/poethepoet/) to manage custom build scripts. Our Python package utilizes uv to manage dependencies and [poethepoet](https://pypi.org/project/poethepoet/) to manage custom build scripts.
Available scripts are: Available scripts are:
- `poetry run poe index` - Run the Indexing CLI - `uv run poe index` - Run the Indexing CLI
- `poetry run poe query` - Run the Query CLI - `uv run poe query` - Run the Query CLI
- `poetry build` - This invokes `poetry build`, which will build a wheel file and other distributable artifacts. - `uv build` - This invokes `uv build`, which will build a wheel file and other distributable artifacts.
- `poetry run poe test` - This will execute all tests. - `uv run poe test` - This will execute all tests.
- `poetry run poe test_unit` - This will execute unit tests. - `uv run poe test_unit` - This will execute unit tests.
- `poetry run poe test_integration` - This will execute integration tests. - `uv run poe test_integration` - This will execute integration tests.
- `poetry run poe test_smoke` - This will execute smoke tests. - `uv run poe test_smoke` - This will execute smoke tests.
- `poetry run poe check` - This will perform a suite of static checks across the package, including: - `uv run poe check` - This will perform a suite of static checks across the package, including:
- formatting - formatting
- documentation formatting - documentation formatting
- linting - linting
- security patterns - security patterns
- type-checking - type-checking
- `poetry run poe fix` - This will apply any available auto-fixes to the package. Usually this is just formatting fixes. - `uv run poe fix` - This will apply any available auto-fixes to the package. Usually this is just formatting fixes.
- `poetry run poe fix_unsafe` - This will apply any available auto-fixes to the package, including those that may be unsafe. - `uv run poe fix_unsafe` - This will apply any available auto-fixes to the package, including those that may be unsafe.
- `poetry run poe format` - Explicitly run the formatter across the package. - `uv run poe format` - Explicitly run the formatter across the package.
## Troubleshooting ## Troubleshooting
### "RuntimeError: llvm-config failed executing, please point LLVM_CONFIG to the path for llvm-config" when running poetry install ### "RuntimeError: llvm-config failed executing, please point LLVM_CONFIG to the path for llvm-config" when running uv sync
Make sure llvm-9 and llvm-9-dev are installed: Make sure llvm-9 and llvm-9-dev are installed:
@ -110,7 +114,7 @@ and then in your bashrc, add
`export LLVM_CONFIG=/usr/bin/llvm-config-9` `export LLVM_CONFIG=/usr/bin/llvm-config-9`
### "numba/\_pymodule.h:6:10: fatal error: Python.h: No such file or directory" when running poetry install ### "numba/\_pymodule.h:6:10: fatal error: Python.h: No such file or directory" when running uv sync
Make sure you have python3.10-dev installed or more generally `python<version>-dev` Make sure you have python3.10-dev installed or more generally `python<version>-dev`

View File

@ -5,27 +5,31 @@
| Name | Installation | Purpose | | Name | Installation | Purpose |
| ------------------- | ------------------------------------------------------------ | ----------------------------------------------------------------------------------- | | ------------------- | ------------------------------------------------------------ | ----------------------------------------------------------------------------------- |
| Python 3.10-3.12 | [Download](https://www.python.org/downloads/) | The library is Python-based. | | Python 3.10-3.12 | [Download](https://www.python.org/downloads/) | The library is Python-based. |
| Poetry | [Instructions](https://python-poetry.org/docs/#installation) | Poetry is used for package management and virtualenv management in Python codebases | | uv | [Instructions](https://docs.astral.sh/uv/) | uv is used for package management and virtualenv management in Python codebases |
# Getting Started # Getting Started
## Install Dependencies ## Install Dependencies
```sh ```sh
# Install Python dependencies. # (optional) create virtual environment
poetry install uv venv --python 3.10
source .venv/bin/activate
# install python dependencies
uv sync --extra dev
``` ```
## Execute the Indexing Engine ## Execute the Indexing Engine
```sh ```sh
poetry run poe index <...args> uv run poe index <...args>
``` ```
## Executing Queries ## Executing Queries
```sh ```sh
poetry run poe query <...args> uv run poe query <...args>
``` ```
# Azurite # Azurite
@ -40,31 +44,31 @@ or by simply running `azurite` in the terminal if already installed globally. Se
# Lifecycle Scripts # Lifecycle Scripts
Our Python package utilizes Poetry to manage dependencies and [poethepoet](https://pypi.org/project/poethepoet/) to manage build scripts. Our Python package utilize uv to manage dependencies and [poethepoet](https://pypi.org/project/poethepoet/) to manage build scripts.
Available scripts are: Available scripts are:
- `poetry run poe index` - Run the Indexing CLI - `uv run poe index` - Run the Indexing CLI
- `poetry run poe query` - Run the Query CLI - `uv run poe query` - Run the Query CLI
- `poetry build` - This invokes `poetry build`, which will build a wheel file and other distributable artifacts. - `uv build` - This will build a wheel file and other distributable artifacts.
- `poetry run poe test` - This will execute all tests. - `uv run poe test` - This will execute all tests.
- `poetry run poe test_unit` - This will execute unit tests. - `uv run poe test_unit` - This will execute unit tests.
- `poetry run poe test_integration` - This will execute integration tests. - `uv run poe test_integration` - This will execute integration tests.
- `poetry run poe test_smoke` - This will execute smoke tests. - `uv run poe test_smoke` - This will execute smoke tests.
- `poetry run poe test_verbs` - This will execute tests of the basic workflows. - `uv run poe test_verbs` - This will execute tests of the basic workflows.
- `poetry run poe check` - This will perform a suite of static checks across the package, including: - `uv run poe check` - This will perform a suite of static checks across the package, including:
- formatting - formatting
- documentation formatting - documentation formatting
- linting - linting
- security patterns - security patterns
- type-checking - type-checking
- `poetry run poe fix` - This will apply any available auto-fixes to the package. Usually this is just formatting fixes. - `uv run poe fix` - This will apply any available auto-fixes to the package. Usually this is just formatting fixes.
- `poetry run poe fix_unsafe` - This will apply any available auto-fixes to the package, including those that may be unsafe. - `uv run poe fix_unsafe` - This will apply any available auto-fixes to the package, including those that may be unsafe.
- `poetry run poe format` - Explicitly run the formatter across the package. - `uv run poe format` - Explicitly run the formatter across the package.
## Troubleshooting ## Troubleshooting
### "RuntimeError: llvm-config failed executing, please point LLVM_CONFIG to the path for llvm-config" when running poetry install ### "RuntimeError: llvm-config failed executing, please point LLVM_CONFIG to the path for llvm-config" when running uv install
Make sure llvm-9 and llvm-9-dev are installed: Make sure llvm-9 and llvm-9-dev are installed:
@ -74,12 +78,6 @@ and then in your bashrc, add
`export LLVM_CONFIG=/usr/bin/llvm-config-9` `export LLVM_CONFIG=/usr/bin/llvm-config-9`
### "numba/\_pymodule.h:6:10: fatal error: Python.h: No such file or directory" when running poetry install
Make sure you have python3.10-dev installed or more generally `python<version>-dev`
`sudo apt-get install python3.10-dev`
### LLM call constantly exceeds TPM, RPM or time limits ### LLM call constantly exceeds TPM, RPM or time limits
`GRAPHRAG_LLM_THREAD_COUNT` and `GRAPHRAG_EMBEDDING_THREAD_COUNT` are both set to 50 by default. You can modify these values `GRAPHRAG_LLM_THREAD_COUNT` and `GRAPHRAG_EMBEDDING_THREAD_COUNT` are both set to 50 by default. You can modify these values

View File

@ -202,10 +202,11 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"from graphrag.index.flows.generate_text_embeddings import generate_text_embeddings\n",
"\n",
"from graphrag.cache.factory import CacheFactory\n", "from graphrag.cache.factory import CacheFactory\n",
"from graphrag.callbacks.noop_workflow_callbacks import NoopWorkflowCallbacks\n", "from graphrag.callbacks.noop_workflow_callbacks import NoopWorkflowCallbacks\n",
"from graphrag.config.embeddings import get_embedded_fields, get_embedding_settings\n", "from graphrag.config.embeddings import get_embedded_fields, get_embedding_settings\n",
"from graphrag.index.flows.generate_text_embeddings import generate_text_embeddings\n",
"\n", "\n",
"# We only need to re-run the embeddings workflow, to ensure that embeddings for all required search fields are in place\n", "# We only need to re-run the embeddings workflow, to ensure that embeddings for all required search fields are in place\n",
"# We'll construct the context and run this function flow directly to avoid everything else\n", "# We'll construct the context and run this function flow directly to avoid everything else\n",

View File

@ -26,8 +26,7 @@ After you have a config file you can run the pipeline using the CLI or the Pytho
### CLI ### CLI
```bash ```bash
# Via Poetry uv run poe index --root <data_root> # default config mode
poetry run poe index --root <data_root> # default config mode
``` ```
### Python API ### Python API

View File

@ -29,6 +29,9 @@
"\n", "\n",
"import pandas as pd\n", "import pandas as pd\n",
"import tiktoken\n", "import tiktoken\n",
"from graphrag.query.llm.oai.chat_openai import ChatOpenAI\n",
"from graphrag.query.llm.oai.embedding import OpenAIEmbedding\n",
"from graphrag.query.llm.oai.typing import OpenaiApiType\n",
"\n", "\n",
"from graphrag.query.context_builder.entity_extraction import EntityVectorStoreKey\n", "from graphrag.query.context_builder.entity_extraction import EntityVectorStoreKey\n",
"from graphrag.query.indexer_adapters import (\n", "from graphrag.query.indexer_adapters import (\n",
@ -38,9 +41,6 @@
" read_indexer_reports,\n", " read_indexer_reports,\n",
" read_indexer_text_units,\n", " read_indexer_text_units,\n",
")\n", ")\n",
"from graphrag.query.llm.oai.chat_openai import ChatOpenAI\n",
"from graphrag.query.llm.oai.embedding import OpenAIEmbedding\n",
"from graphrag.query.llm.oai.typing import OpenaiApiType\n",
"from graphrag.query.structured_search.local_search.mixed_context import (\n", "from graphrag.query.structured_search.local_search.mixed_context import (\n",
" LocalSearchMixedContext,\n", " LocalSearchMixedContext,\n",
")\n", ")\n",

View File

@ -239,7 +239,7 @@ class GlobalSearchDefaults:
class StorageDefaults: class StorageDefaults:
"""Default values for storage.""" """Default values for storage."""
type = StorageType.file type: ClassVar[StorageType] = StorageType.file
base_dir: str = DEFAULT_OUTPUT_BASE_DIR base_dir: str = DEFAULT_OUTPUT_BASE_DIR
connection_string: None = None connection_string: None = None
container_name: None = None container_name: None = None

6301
poetry.lock generated

File diff suppressed because it is too large Load Diff

View File

@ -1,119 +1,110 @@
[tool.poetry] [project]
name = "graphrag" name = "graphrag"
# Maintainers: do not change the version here manually, use ./scripts/release.sh # Maintainers: do not change the version here manually, use ./scripts/release.sh
version = "2.4.0" version = "2.4.0"
description = "GraphRAG: A graph-based retrieval-augmented generation (RAG) system." description = "GraphRAG: A graph-based retrieval-augmented generation (RAG) system."
authors = [ authors = [
"Alonso Guevara Fernández <alonsog@microsoft.com>", {name = "Alonso Guevara Fernández", email = "alonsog@microsoft.com"},
"Andrés Morales Esquivel <andresmor@microsoft.com>", {name = "Andrés Morales Esquivel", email = "andresmor@microsoft.com"},
"Chris Trevino <chtrevin@microsoft.com>", {name = "Chris Trevino", email = "chtrevin@microsoft.com"},
"David Tittsworth <datittsw@microsoft.com>", {name = "David Tittsworth", email = "datittsw@microsoft.com"},
"Dayenne de Souza <ddesouza@microsoft.com>", {name = "Dayenne de Souza", email = "ddesouza@microsoft.com"},
"Derek Worthen <deworthe@microsoft.com>", {name = "Derek Worthen", email = "deworthe@microsoft.com"},
"Gaudy Blanco Meneses <gaudyb@microsoft.com>", {name = "Gaudy Blanco Meneses", email = "gaudyb@microsoft.com"},
"Ha Trinh <trinhha@microsoft.com>", {name = "Ha Trinh", email = "trinhha@microsoft.com"},
"Jonathan Larson <jolarso@microsoft.com>", {name = "Jonathan Larson", email = "jolarso@microsoft.com"},
"Josh Bradley <joshbradley@microsoft.com>", {name = "Josh Bradley", email = "joshbradley@microsoft.com"},
"Kate Lytvynets <kalytv@microsoft.com>", {name = "Kate Lytvynets", email = "kalytv@microsoft.com"},
"Kenny Zhang <zhangken@microsoft.com>", {name = "Kenny Zhang", email = "zhangken@microsoft.com"},
"Mónica Carvajal", {name = "Mónica Carvajal"},
"Nathan Evans <naevans@microsoft.com>", {name = "Nathan Evans", email = "naevans@microsoft.com"},
"Rodrigo Racanicci <rracanicci@microsoft.com>", {name = "Rodrigo Racanicci", email = "rracanicci@microsoft.com"},
"Sarah Smith <smithsarah@microsoft.com>", {name = "Sarah Smith", email = "smithsarah@microsoft.com"},
] ]
license = "MIT" license = "MIT"
readme = "README.md" readme = "README.md"
packages = [{ include = "graphrag" }] requires-python = ">=3.10,<3.13"
classifiers = [
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
]
[tool.poetry.urls] dependencies = [
"Source" = "https://github.com/microsoft/graphrag" "environs>=11.0.0",
# Vector Stores
"azure-search-documents>=11.5.2",
"lancedb>=0.17.0",
# Async IO
"aiofiles>=24.1.0",
# LLM
"fnllm[azure,openai]>=0.3.0",
"json-repair>=0.30.3",
"openai>=1.68.0",
"nltk==3.9.1",
"tiktoken>=0.9.0",
# Data-Science
"numpy>=1.25.2",
"graspologic>=3.4.1",
"networkx>=3.4.2",
"pandas>=2.2.3",
"pyarrow>=17.0.0",
"umap-learn>=0.5.6",
# Configuration
"pyyaml>=6.0.2",
"python-dotenv>=1.0.1",
"pydantic>=2.10.3",
"devtools>=0.12.2",
"typing-extensions>=4.12.2",
# Azure
"azure-cosmos>=4.9.0",
"azure-identity>=1.19.0",
"azure-storage-blob>=12.24.0",
"future>=1.0.0", # Needed until graspologic fixes their dependency
"typer>=0.16.0",
"tqdm>=4.67.1",
"textblob>=0.18.0.post0",
"spacy>=3.8.4",
]
[tool.poetry.scripts] [project.optional-dependencies]
dev = [
"coverage>=7.6.9",
"ipykernel>=6.29.5",
"jupyter>=1.1.1",
"nbconvert>=7.16.4",
"poethepoet>=0.31.1",
"pyright>=1.1.390",
"pytest>=8.3.4",
"pytest-asyncio>=0.24.0",
"pytest-timeout>=2.3.1",
"ruff>=0.8.2",
"semversioner>=2.0.5",
"update-toml>=0.2.1",
"deptry>=0.21.1",
"mkdocs-material>=9.5.48",
"mkdocs-jupyter>=0.25.1",
"mkdocs-exclude-search>=0.6.6",
"pytest-dotenv>=0.5.2",
"mkdocs-typer>=0.0.3",
]
[project.scripts]
graphrag = "graphrag.cli.main:app" graphrag = "graphrag.cli.main:app"
[tool.poetry-dynamic-versioning] [project.urls]
enable = true Source = "https://github.com/microsoft/graphrag"
style = "pep440"
vcs = "git"
bump = true
format-jinja = """
{%- if distance == 0 -%}
{{ serialize_pep440(base, stage, revision) }}
{%- else -%}
{{ serialize_pep440(base, stage, revision, dev=distance) }}
{%- endif -%}
"""
[tool.poetry.dependencies]
python = ">=3.10,<3.13"
environs = "^11.0.0"
# Vector Stores
azure-search-documents = "^11.5.2"
lancedb = "^0.17.0"
# Async IO
aiofiles = "^24.1.0"
# LLM
fnllm = {extras = ["azure", "openai"], version = "^0.3.0"}
json-repair = "^0.30.3"
openai = "^1.68.0"
nltk = "3.9.1"
tiktoken = "^0.9.0"
# Data-Science
numpy = "^1.25.2"
graspologic = "^3.4.1"
networkx = "^3.4.2"
pandas = "^2.2.3"
pyarrow = ">=17.0.0"
umap-learn = "^0.5.6"
# Configuration
pyyaml = "^6.0.2"
python-dotenv = "^1.0.1"
pydantic = "^2.10.3"
devtools = "^0.12.2"
typing-extensions = "^4.12.2"
# Azure
azure-cosmos = "^4.9.0"
azure-identity = "^1.19.0"
azure-storage-blob = "^12.24.0"
future = "^1.0.0" # Needed until graspologic fixes their dependency
typer = "^0.16.0"
tqdm = "^4.67.1"
textblob = "^0.18.0.post0"
spacy = "^3.8.4"
[tool.poetry.group.dev.dependencies]
coverage = "^7.6.9"
ipykernel = "^6.29.5"
jupyter = "^1.1.1"
nbconvert = "^7.16.4"
poethepoet = "^0.31.1"
pyright = "^1.1.390"
pytest = "^8.3.4"
pytest-asyncio = "^0.24.0"
pytest-timeout = "^2.3.1"
ruff = "^0.8.2"
semversioner = "^2.0.5"
update-toml = "^0.2.1"
deptry = "^0.21.1"
mkdocs-material = "^9.5.48"
mkdocs-jupyter = "^0.25.1"
mkdocs-exclude-search = "^0.6.6"
pytest-dotenv = "^0.5.2"
mkdocs-typer = "^0.0.3"
[build-system] [build-system]
requires = ["poetry-core>=1.0.0", "poetry-dynamic-versioning>=1.0.0,<2.0.0"] requires = ["setuptools>=64", "wheel"]
build-backend = "poetry_dynamic_versioning.backend" build-backend = "setuptools.build_meta"
[tool.setuptools.packages.find]
include = ["graphrag*"]
exclude = ["examples_notebooks*", "tests*"]
# Keep poethepoet for task management to minimize changes
[tool.poe.tasks] [tool.poe.tasks]
_sort_imports = "ruff check --select I --fix ." _sort_imports = "ruff check --select I --fix ."
_format_code = "ruff format ." _format_code = "ruff format ."
@ -123,7 +114,7 @@ _convert_local_search_nb = 'jupyter nbconvert --output-dir=docsite/posts/query/n
_convert_global_search_nb = 'jupyter nbconvert --output-dir=docsite/posts/query/notebooks/ --output="{notebook_name}_nb" --template=docsite/nbdocsite_template --to markdown examples_notebooks/global_search.ipynb' _convert_global_search_nb = 'jupyter nbconvert --output-dir=docsite/posts/query/notebooks/ --output="{notebook_name}_nb" --template=docsite/nbdocsite_template --to markdown examples_notebooks/global_search.ipynb'
_semversioner_release = "semversioner release" _semversioner_release = "semversioner release"
_semversioner_changelog = "semversioner changelog > CHANGELOG.md" _semversioner_changelog = "semversioner changelog > CHANGELOG.md"
_semversioner_update_toml_version = "update-toml update --path tool.poetry.version --value $(poetry run semversioner current-version)" _semversioner_update_toml_version = "update-toml update --path project.version --value $(semversioner current-version)"
semversioner_add = "semversioner add-change" semversioner_add = "semversioner add-change"
coverage_report = 'coverage report --omit "**/tests/**" --show-missing' coverage_report = 'coverage report --omit "**/tests/**" --show-missing'
check_format = 'ruff format . --check' check_format = 'ruff format . --check'
@ -169,6 +160,7 @@ ignore_fail = 'return_non_zero'
sequence = ['_test_all', 'coverage_report'] sequence = ['_test_all', 'coverage_report']
ignore_fail = 'return_non_zero' ignore_fail = 'return_non_zero'
# Keep all existing tool configurations
[tool.ruff] [tool.ruff]
target-version = "py310" target-version = "py310"
extend-include = ["*.ipynb"] extend-include = ["*.ipynb"]
@ -267,4 +259,4 @@ exclude = ["**/node_modules", "**/__pycache__"]
asyncio_default_fixture_loop_scope = "function" asyncio_default_fixture_loop_scope = "function"
asyncio_mode = "auto" asyncio_mode = "auto"
timeout = 1000 timeout = 1000
env_files = [".env"] env_files = [".env"]

View File

@ -4,7 +4,7 @@ has_change_doc=$(echo $changes | grep .semversioner/next-release)
has_impacting_changes=$(echo $changes | grep graphrag) has_impacting_changes=$(echo $changes | grep graphrag)
if [ "$has_impacting_changes" ] && [ -z "$has_change_doc" ]; then if [ "$has_impacting_changes" ] && [ -z "$has_change_doc" ]; then
echo "Check failed. Run 'poetry run semversioner add-change' to update the next release version" echo "Check failed. Run 'uv run semversioner add-change' to update the next release version"
exit 1 exit 1
fi fi
echo "OK" echo "OK"

View File

@ -32,7 +32,7 @@
"create_communities": { "create_communities": {
"row_range": [ "row_range": [
10, 10,
30 35
], ],
"max_runtime": 30, "max_runtime": 30,
"expected_artifacts": ["communities.parquet"] "expected_artifacts": ["communities.parquet"]
@ -40,7 +40,7 @@
"create_community_reports": { "create_community_reports": {
"row_range": [ "row_range": [
10, 10,
30 35
], ],
"nan_allowed_columns": [ "nan_allowed_columns": [
"title", "title",

View File

@ -130,7 +130,7 @@ class TestIndexer:
input_file_type: str, input_file_type: str,
): ):
command = [ command = [
"poetry", "uv",
"run", "run",
"poe", "poe",
"index", "index",
@ -202,7 +202,7 @@ class TestIndexer:
def __run_query(self, root: Path, query_config: dict[str, str]): def __run_query(self, root: Path, query_config: dict[str, str]):
command = [ command = [
"poetry", "uv",
"run", "run",
"poe", "poe",
"query", "query",

View File

@ -10,11 +10,10 @@ RUN apt-get update -y
# Install dependencies # Install dependencies
WORKDIR ./ WORKDIR ./
COPY . . COPY . .
RUN curl -sSL https://install.python-poetry.org | python - RUN curl -LsSf https://astral.sh/uv/install.sh | sh
ENV PATH="${PATH}:/root/.local/bin" ENV PATH="${PATH}:/root/.local/bin"
RUN poetry config virtualenvs.in-project true RUN uv sync --no-install-project
RUN poetry install --no-root
# Run application # Run application
EXPOSE 8501 EXPOSE 8501
ENTRYPOINT ["poetry","run","streamlit", "run", "./app/home_page.py"] ENTRYPOINT ["uv","run","poe","start_prod"]

View File

@ -5,14 +5,14 @@ Unified demo for GraphRAG search comparisons.
## Requirements: ## Requirements:
- Python 3.11 - Python 3.11
- Poetry - UV
This sample app is not published to pypi, so you'll need to clone the GraphRAG repo and run from this folder. This sample app is not published to pypi, so you'll need to clone the GraphRAG repo and run from this folder.
We recommend always using a virtual environment: We recommend always using a virtual environment:
- `python -m venv ./venv` - `uv venv --python 3.11`
- `source ./venv/bin/activate` - `source .venv/bin/activate`
## Run index ## Run index
Use GraphRAG to index your dataset before running Unified Search. We recommend starting with the [Getting Started guide](https://microsoft.github.io/graphrag/get_started/). You need to run GraphRAG indexing with graph embedding umap enabled to use the functionalities of Unified Search. Use GraphRAG to index your dataset before running Unified Search. We recommend starting with the [Getting Started guide](https://microsoft.github.io/graphrag/get_started/). You need to run GraphRAG indexing with graph embedding umap enabled to use the functionalities of Unified Search.
@ -91,9 +91,9 @@ You can host Unified Search datasets locally or in a blob.
# Run the app # Run the app
Install all the dependencies: `poetry install` Install all the dependencies: `uv sync --extra dev`
Run the project using streamlit: `poetry run poe start` Run the project using streamlit: `uv run poe start`
# How to use it # How to use it

File diff suppressed because it is too large Load Diff

View File

@ -1,32 +1,40 @@
[tool.poetry] [project]
name = "unified-copilot" name = "unified-copilot"
version = "1.0.0" version = "1.0.0"
description = "" description = ""
authors = ["GraphRAG team"] authors = [
{name = "GraphRAG team"},
]
readme = "README.md" readme = "README.md"
package-mode = false requires-python = ">=3.10,<3.12"
[tool.poetry.dependencies] dependencies = [
python = ">=3.10,<3.12" "streamlit==1.43.0",
streamlit = "1.43.0" "azure-search-documents>=11.4.0",
azure-search-documents = "^11.4.0" "azure-storage-blob>=12.20.0",
azure-storage-blob = "^12.20.0" "azure-identity>=1.16.0",
azure-identity = "^1.16.0" "graphrag==2.0.0",
graphrag = "2.0.0" "altair>=5.3.0",
altair = "^5.3.0" "streamlit-agraph>=0.0.45",
streamlit-agraph = "^0.0.45" "st-tabs>=0.1.1",
st-tabs = "^0.1.1" "spacy>=3.8.4,<4.0.0",
spacy = ">=3.8.4,<4.0.0" ]
[tool.poetry.group.dev.dependencies] [project.optional-dependencies]
poethepoet = "^0.26.1" dev = [
ipykernel = "^6.29.4" "poethepoet>=0.26.1",
pyright = "^1.1.349" "ipykernel>=6.29.4",
ruff = "^0.4.7" "pyright>=1.1.349",
"ruff>=0.4.7",
]
[build-system] [build-system]
requires = ["poetry-core"] requires = ["setuptools>=64", "wheel"]
build-backend = "poetry.core.masonry.api" build-backend = "setuptools.build_meta"
[tool.setuptools.packages.find]
include = ["app*"]
exclude = ["images*"]
[tool.poe.tasks] [tool.poe.tasks]
start = "streamlit run app/home_page.py" start = "streamlit run app/home_page.py"

2988
unified-search-app/uv.lock generated Normal file

File diff suppressed because it is too large Load Diff

4166
uv.lock generated Normal file

File diff suppressed because it is too large Load Diff