Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 5 additions & 9 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ name: CI
on:
push:
branches: [main]
# Child PRs target another stack branch and need the same checks.
pull_request:
branches: [main]
workflow_dispatch:

# Least privilege: enough to clone, nothing else. Checking out a private
Expand All @@ -24,10 +24,6 @@ concurrency:
group: ci-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

env:
# Fail rather than silently re-resolving when uv.lock is stale.
UV_FROZEN: "1"

jobs:
static:
name: Lint, format, docs and licences
Expand All @@ -37,7 +33,7 @@ jobs:

- uses: astral-sh/setup-uv@1c37ad07a6a961277cf70c0d37d6f313000f5884 # v10.3.0
with:
version: "latest"
version-file: "pyproject.toml"
python-version: "3.12"
enable-cache: true

Expand Down Expand Up @@ -67,13 +63,13 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
python-version: ["3.11", "3.12", "3.13", "3.14"]
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- uses: astral-sh/setup-uv@1c37ad07a6a961277cf70c0d37d6f313000f5884 # v10.3.0
with:
version: "latest"
version-file: "pyproject.toml"
python-version: ${{ matrix.python-version }}
enable-cache: true

Expand Down Expand Up @@ -101,7 +97,7 @@ jobs:

- uses: astral-sh/setup-uv@1c37ad07a6a961277cf70c0d37d6f313000f5884 # v10.3.0
with:
version: "latest"
version-file: "pyproject.toml"
python-version: "3.12"
enable-cache: true

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/health-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ jobs:

- uses: astral-sh/setup-uv@1c37ad07a6a961277cf70c0d37d6f313000f5884 # v10.3.0
with:
version: "latest"
version-file: "pyproject.toml"
python-version: "3.12"
enable-cache: true

Expand Down
6 changes: 1 addition & 5 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,6 @@ concurrency:
group: release-${{ github.ref }}
cancel-in-progress: false

env:
# Fail rather than silently re-resolving when uv.lock is stale.
UV_FROZEN: "1"

jobs:
build:
name: Build and check
Expand All @@ -45,7 +41,7 @@ jobs:

- uses: astral-sh/setup-uv@1c37ad07a6a961277cf70c0d37d6f313000f5884 # v10.3.0
with:
version: "latest"
version-file: "pyproject.toml"
python-version: "3.12"
# A cache written by another run could otherwise reach the
# published files.
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -66,3 +66,6 @@ Thumbs.db

# Cursor
.cursor/

# Local agent working notes
/.agent-work/
1 change: 1 addition & 0 deletions .python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.12
40 changes: 31 additions & 9 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,13 +136,34 @@ passes the check automatically.

## Getting set up

Requires Python 3.10 to 3.14 and [uv](https://docs.astral.sh/uv/) 0.13.4 or newer.
Supports Python 3.11 to 3.14. The default development interpreter is Python 3.12
from `.python-version`; CI tests every supported version. Start with an existing
[uv](https://docs.astral.sh/uv/) installation, then install the exact
`required-version` from `pyproject.toml`:

```bash
uv sync # installs the dev group: pytest, ruff, jupyter
make bootstrap # installs the declared uv; ensure its bin directory is on PATH
make install-dev # locked dev dependencies and pre-commit hook
uv run usersim smoke # offline self-check; no API key, no network
```

`make check-tools` rejects another uv version. `make check-lock` checks that
`uv.lock` agrees with dependency metadata without rewriting it. Both setup
targets use `--locked`, so stale metadata fails rather than updating the lock.
After a deliberate dependency change, run `uv lock`, review `uv.lock`, and
commit it with the metadata change. CI reads the same uv version declaration;
update it deliberately and validate the lock before committing a tool upgrade.

The 0.3 development branch temporarily pins Data Designer’s Python 3.11
transition commit through `[tool.uv.override-dependencies]`. Use `uv sync` or
the Make setup targets to consume that preview. Once DD 0.10.0 is published,
remove the Git overrides and `no-sources`, require `data-designer>=0.10,<0.11`, and regenerate
`uv.lock`.

`make install-ci` performs locked setup without installing hooks. In a Git
worktree, `make install-dev` finds the hook directory through Git rather than
assuming `.git` is a directory.

`smoke` is the fastest way to confirm an install is sound. It resolves the
model catalog, every probe's assets, and the plugin entry points without
making a single model call.
Expand Down Expand Up @@ -268,7 +289,8 @@ see [Issues before pull requests](#issues-before-pull-requests).

Every job in `.github/workflows/ci.yml` calls a `make` target, so anything CI
rejects reproduces locally with the same command. Tests and the dependency
license check run on Python 3.10 to 3.14.
license check run on Python 3.11 to 3.14. Pull-request checks run for every
base branch so stacked PRs receive CI; push checks remain scoped to `main`.

No job triggered by a pull request has access to a credential, so a PR from a
fork runs the complete suite. The consequence is that CI cannot tell you
Expand Down Expand Up @@ -301,9 +323,9 @@ Maintainers release from `main`:

```bash
git switch main && git pull --ff-only
make check-dist && make check-release-tag TAG=v0.2.0rc1
git tag -a v0.2.0rc1 -m "usersim 0.2.0rc1"
git push origin v0.2.0rc1
make check-dist && make check-release-tag TAG=v0.3.0rc1
git tag -a v0.3.0rc1 -m "usersim 0.3.0rc1"
git push origin v0.3.0rc1
```

3. `.github/workflows/release.yml` runs those checks and the test suite on the
Expand All @@ -312,14 +334,14 @@ Maintainers release from `main`:
pre-release for a release candidate, and writes their SHA-256 digests to
the run summary.
4. Merge a pull request that moves `version` to the next development version,
for example `0.2.0rc2.dev0`.
for example `0.3.1.dev0`.

If the workflow fails before the release is published, delete the tag
(`git tag -d v0.2.0rc1 && git push origin --delete v0.2.0rc1`) and any draft
(`git tag -d v0.3.0rc1 && git push origin --delete v0.3.0rc1`) and any draft
release it left, fix the cause and tag again. Once a release is published,
its tag and files stay as they are, and a fix ships as a new version. The
files attached to a release are the ones published to PyPI;
`gh release verify-asset v0.2.0rc1 <file>` checks a downloaded file against
`gh release verify-asset v0.3.0rc1 <file>` checks a downloaded file against
them.

## What the tests expect
Expand Down
25 changes: 20 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
SHELL := /bin/bash

UV ?= uv
UV_VERSION := $(shell sed -n 's/^required-version = "==\([^"]*\)"/\1/p' pyproject.toml)

# A top-level `_`-prefixed directory holds material that is not part of the
# distribution and may carry its own targets. Absent one, this expands to
Expand Down Expand Up @@ -57,7 +58,7 @@ COV_FAIL_UNDER ?= 84

.DEFAULT_GOAL := help

.PHONY: help install-dev install-ci install-kernel health-check check-notebooks-clean lint lint-fix format format-check check-all \
.PHONY: help bootstrap check-tools check-lock install-dev install-ci install-kernel health-check check-notebooks-clean lint lint-fix format format-check check-all \
check-all-fix test test-fast coverage smoke dist check-dist check-release-tag check-extensions check-doc-links clean-notebooks check-license-headers update-license-headers check-dependency-licenses clean

help: ## Show the available targets
Expand All @@ -67,11 +68,25 @@ help: ## Show the available targets

# --- setup -----------------------------------------------------------------

# Requires an existing uv installation; installs the declared release on PATH.
bootstrap: ## Install the repository's required uv version as a standalone tool
$(UV) tool install --force "uv==$(UV_VERSION)"

check-tools: ## Reject an unsupported uv version
@actual=$$($(UV) --version 2>/dev/null | awk '{print $$2}'); \
if [ -z "$(UV_VERSION)" ] || [ "$$actual" != "$(UV_VERSION)" ]; then \
echo "Expected uv $(UV_VERSION), found $${actual:-none}. Install uv, then run make bootstrap and ensure its bin directory is on PATH." >&2; \
exit 1; \
fi

check-lock: check-tools ## Check dependency metadata against uv.lock without rewriting it
$(UV) lock --check

# Reinstalling the project restores any file that removing another distribution
# took with it, such as the `usersim` command an earlier name also installed.
install-dev: ## Sync the workspace with dev tooling and install pre-commit hooks
$(UV) sync --reinstall-package usersim
@if [ ! -f .git/hooks/pre-commit ]; then \
install-dev: check-tools ## Sync the workspace with dev tooling and install pre-commit hooks
$(UV) sync --locked --reinstall-package usersim
@if [ ! -f "$$(git rev-parse --git-path hooks/pre-commit)" ]; then \
echo "Installing pre-commit hooks..."; \
$(UV) run pre-commit install; \
else \
Expand All @@ -81,7 +96,7 @@ install-dev: ## Sync the workspace with dev tooling and install pre-commit hook
# --locked installs strictly from uv.lock and refuses to re-resolve, so
# dependency drift becomes a build failure rather than a silent
# re-resolution. No pre-commit hooks -- CI has no working tree to guard.
install-ci: ## Sync exactly what CI needs, enforcing the lockfile
install-ci: check-tools ## Sync exactly what CI needs, enforcing the lockfile
$(UV) sync --locked

# --- lint / format ---------------------------------------------------------
Expand Down
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

[![CI](https://github.com/NVIDIA-NeMo/UserSim/actions/workflows/ci.yml/badge.svg)](https://github.com/NVIDIA-NeMo/UserSim/actions/workflows/ci.yml)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/NVIDIA-NeMo/UserSim/blob/main/LICENSE)
[![Python](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12%20%7C%203.13%20%7C%203.14-blue.svg)](https://github.com/NVIDIA-NeMo/UserSim/blob/main/pyproject.toml)
[![Python](https://img.shields.io/badge/python-3.11%20%7C%203.12%20%7C%203.13%20%7C%203.14-blue.svg)](https://github.com/NVIDIA-NeMo/UserSim/blob/main/pyproject.toml)

> Simulate a statistically representative population of users, sampled from
> persona datasets (e.g., census-grounded), having multi-turn conversations with
Expand Down Expand Up @@ -134,12 +134,13 @@ in the NeMo Gym repository.

<img src="https://raw.githubusercontent.com/NVIDIA-NeMo/UserSim/main/docs/images/getting_started.jpg" width="800" alt="From install to capability report in five commands: pip install usersim, download personas with data-designer, then usersim simulate, eval and report. Optional steps fix the panel, replay stored rows or select training data, with a locale picker and the probe groups below.">

Python 3.10+, in a virtual environment. Python 3.10 is supported through the
0.2 releases.
UserSim 0.3 requires Python 3.11–3.14. Create a virtual environment with a
supported interpreter before upgrading from Python 3.10. Python 3.10 users
should stay on the 0.2 release line (`usersim>=0.2,<0.3`).

```bash
python -m venv .venv && source .venv/bin/activate
pip install "usersim>=0.2.0" # in a uv project: uv add "usersim>=0.2.0"
pip install "usersim>=0.3.0" # in a uv project: uv add "usersim>=0.3.0"

# Verify the install offline: no LLM calls, no network, CI-ready.
usersim smoke
Expand Down
3 changes: 1 addition & 2 deletions dependency-license-policy.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,9 @@ reason = "Unmodified character-detection library installed as a separate transit
reports = [{ version = "5.2.0", license = "GNU Lesser General Public License v2 or later (LGPLv2+)" }]

[exceptions.numpy]
reason = "Core numerical dependency, BSD-3-Clause overall. The additional identifiers cover vendored components and are each permissive or public-domain-equivalent. Python 3.10 resolves 2.2.6, which reports the same licence as the generic BSD classifier."
reason = "Core numerical dependency, BSD-3-Clause overall. The additional identifiers cover vendored components and are each permissive or public-domain-equivalent."
reports = [
{ version = "2.4.3", license = "BSD-3-Clause AND 0BSD AND MIT AND Zlib AND CC0-1.0" },
{ version = "2.2.6", license = "BSD License" },
]

[exceptions.pathspec]
Expand Down
28 changes: 13 additions & 15 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
[project]
name = "usersim"
version = "0.2.1.dev0"
version = "0.3.0.dev0"
description = "Improve models and agents with population-grounded simulated users, even before production traffic exists."
readme = "README.md"
requires-python = ">=3.10"
requires-python = ">=3.11,<3.15"
keywords = [
"llm", "agents", "evaluation", "synthetic-data", "simulation", "user-simulation", "personas", "nemotron",
]
Expand All @@ -13,7 +13,6 @@ classifiers = [
"Intended Audience :: Developers",
"Intended Audience :: Science/Research",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
Expand All @@ -35,7 +34,7 @@ dependencies = [
# persona datasets for ``en_GB`` and ``de_DE``. 0.9.7 is the first whose
# ``data-designer-config`` requires ``certifi>=2026.10.19``, a bundle
# without the emSign ECC Root CA - G3.
"data-designer>=0.9.7,<0.10",
"data-designer>=0.9.7,<0.11",
# Language detection for the language_compliance scorer. Pure Python,
# MIT, accurate on short conversational text. Pinned to a major version
# to keep CI deterministic -- bump deliberately.
Expand Down Expand Up @@ -74,8 +73,6 @@ dependencies = [
# Reads credentials from a .env file so a run does not depend on what is
# exported in the calling shell. No required dependencies of its own.
"python-dotenv>=1.0,<2",
# Parses the models TOML on Python 3.10, which has no tomllib.
"tomli>=1.1.0; python_version < '3.11'",
]
[project.urls]
Homepage = "https://github.com/NVIDIA-NeMo/UserSim"
Expand Down Expand Up @@ -110,9 +107,6 @@ dev = [
# here. conftest.py fails the run if it is absent, because without it every
# async test is skipped with a warning rather than failing.
"pytest-asyncio>=1.0,<2",
# tests/test_network_guard.py unwraps exception groups, which Python 3.10
# has only through this backport.
"exceptiongroup>=1.0; python_version < '3.11'",
]
# Empty on purpose: the gate runs a pinned pip-licenses through `uv tool run`,
# so this group exists only to scope an isolated environment holding the
Expand All @@ -127,12 +121,16 @@ conversation-simulator = "usersim.engine.plugin:conversation_simulator"
trajectory-evaluator = "usersim.engine.plugin:trajectory_evaluator"

[tool.uv]
# uv checks TLS against certificates it bundles. Before 0.13.4 they include
# emSign ECC Root CA - G3, whose intermediates have compromised keys.
required-version = ">=0.13.4"
# Shared developer and CI pin; keep at or above main's 0.13.4 security floor.
required-version = "==0.13.4"
# Temporary immutable preview; replace all three overrides with the released
# DD >=0.10,<0.11 requirement when 0.10.0 is published.
# Ignore DD's workspace-only sources; the overrides below supply all siblings.
no-sources = true
override-dependencies = [
# Keep in sync with the ``dependencies`` pin above.
"data-designer>=0.9.7,<0.10",
"data-designer @ git+https://github.com/NVIDIA-NeMo/DataDesigner.git@7a62d75378bfc5b5ba7205eed6554959560deed9#subdirectory=packages/data-designer",
"data-designer-config @ git+https://github.com/NVIDIA-NeMo/DataDesigner.git@7a62d75378bfc5b5ba7205eed6554959560deed9#subdirectory=packages/data-designer-config",
"data-designer-engine @ git+https://github.com/NVIDIA-NeMo/DataDesigner.git@7a62d75378bfc5b5ba7205eed6554959560deed9#subdirectory=packages/data-designer-engine",
]
# The same root, for everything that verifies TLS through certifi. Constraints
# apply to this project's resolution only; installs from PyPI get the same
Expand Down Expand Up @@ -209,7 +207,7 @@ asyncio_mode = "auto"
[tool.ruff]
line-length = 120
indent-width = 4
target-version = "py310"
target-version = "py311"
# Ruff lints .ipynb by default. The notebooks are user-facing walkthroughs with
# committed outputs, so linting them means noise and rewritten cells. Excluding
# them keeps a bare `ruff check` equivalent to `make lint`.
Expand Down
6 changes: 1 addition & 5 deletions scripts/check_dependency_licenses.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,11 @@
import re
import subprocess
import sys
import tomllib
from dataclasses import dataclass
from pathlib import Path
from typing import Any

if sys.version_info >= (3, 11):
import tomllib
else:
import tomli as tomllib

PIP_LICENSES_VERSION = "5.5.5"
POLICY_PATH = Path(__file__).resolve().parents[1] / "dependency-license-policy.toml"

Expand Down
7 changes: 1 addition & 6 deletions src/usersim/cli/_models.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,18 +54,13 @@

import logging
import os
import sys
import tomllib
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Iterable

from usersim.cli._errors import ConfigError

if sys.version_info >= (3, 11):
import tomllib
else:
import tomli as tomllib

logger = logging.getLogger(__name__)


Expand Down
2 changes: 1 addition & 1 deletion tests/fixtures/extension_package/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ build-backend = "setuptools.build_meta"
name = "usersim-extension-fixture"
version = "0.1.0"
description = "Exercises every published extension seam against a real install."
requires-python = ">=3.10"
requires-python = ">=3.11,<3.15"

# One entry per published group. If a seam stops working, installing this
# package and asking the CLI what it can see is what notices.
Expand Down
Loading
Loading