From 10224bba60123f09347a66f1c490aacfee3224fd Mon Sep 17 00:00:00 2001 From: Henry Schreiner Date: Tue, 26 May 2026 17:57:48 -0400 Subject: [PATCH 01/15] feat(docs): migrate docs site from Jekyll to MyST (JupyterBook 2.0) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Replace Jekyll + just-the-docs with MyST using scientific-python-myst-theme - Add docs/myst.yml with full TOC structure (29 pages across 5 sections) - Add docs/config/scientific-python.yml for book-theme configuration - Add docs/rr-role.mjs custom MyST plugin replacing repo_review.rb Liquid tag - Add docs/assets/css/site.css with SP theme CSS and .rr-btn styles - Convert all Liquid tags: {%rr%}→{rr} role, tabs→tab-set/tab-item, details→{dropdown}, callouts→admonitions, links→relative paths - Inline interactive_repo_review.html as raw html block with Pyodide loading - Move _includes/pyproject.md to _partials/ and update include paths - Update .readthedocs.yaml from Ruby/Jekyll to Node.js/mystmd build - Update .pre-commit-config.yaml for new _partials/ path - Remove all Jekyll infrastructure: Gemfile, _config.yml, _plugins/, _sass/, _includes/, assets/js/tabs.js Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Assisted-by: copilot-cli:claude-sonnet-4.6 --- .pre-commit-config.yaml | 2 +- .readthedocs.yaml | 10 +- .ruby-version | 1 - AGENTS.md | 57 + Gemfile | 48 - Gemfile.lock | 243 ----- _config.yml | 57 - docs/_includes/head.html | 74 -- docs/_includes/head_custom.html | 20 - docs/_includes/interactive_repo_review.html | 25 - docs/_includes/toc.html | 7 - docs/{_includes => _partials}/pyproject.md | 0 docs/_plugins/details.rb | 22 - docs/_plugins/repo_review.rb | 19 - docs/_plugins/tabs.rb | 71 -- docs/_sass/color_schemes/skhep.scss | 1 - docs/_sass/custom/custom.scss | 155 --- docs/assets/css/site.css | 110 ++ docs/assets/js/tabs.js | 36 - docs/config/scientific-python.yml | 9 + docs/myst.yml | 61 ++ docs/pages/guides/coverage.md | 56 +- docs/pages/guides/docs.md | 165 ++- docs/pages/guides/gha_basic.md | 84 +- docs/pages/guides/gha_pure.md | 114 +- docs/pages/guides/gha_wheels.md | 44 +- docs/pages/guides/index.md | 60 +- docs/pages/guides/mypy.md | 8 +- docs/pages/guides/packaging_classic.md | 80 +- docs/pages/guides/packaging_compiled.md | 133 +-- docs/pages/guides/packaging_simple.md | 128 +-- docs/pages/guides/pytest.md | 59 +- docs/pages/guides/repo_review.md | 35 +- docs/pages/guides/style.md | 1052 ------------------- docs/pages/guides/tasks.md | 20 +- docs/pages/index.md | 68 +- docs/pages/patterns/backports.md | 6 - docs/pages/patterns/data_files.md | 12 +- docs/pages/patterns/exports.md | 11 +- docs/pages/patterns/index.md | 10 +- docs/pages/principles/design.md | 18 +- docs/pages/principles/index.md | 8 +- docs/pages/principles/process.md | 6 - docs/pages/principles/testing.md | 26 +- docs/pages/tutorials/dev-environment.md | 63 +- docs/pages/tutorials/docs.md | 10 +- docs/pages/tutorials/index.md | 8 +- docs/pages/tutorials/module.md | 8 +- docs/pages/tutorials/packaging.md | 8 +- docs/pages/tutorials/test.md | 8 +- docs/rr-role.mjs | 26 + 51 files changed, 783 insertions(+), 2579 deletions(-) delete mode 100644 .ruby-version create mode 100644 AGENTS.md delete mode 100644 Gemfile delete mode 100644 Gemfile.lock delete mode 100644 _config.yml delete mode 100644 docs/_includes/head.html delete mode 100644 docs/_includes/head_custom.html delete mode 100644 docs/_includes/interactive_repo_review.html delete mode 100644 docs/_includes/toc.html rename docs/{_includes => _partials}/pyproject.md (100%) delete mode 100644 docs/_plugins/details.rb delete mode 100644 docs/_plugins/repo_review.rb delete mode 100644 docs/_plugins/tabs.rb delete mode 100644 docs/_sass/color_schemes/skhep.scss delete mode 100644 docs/_sass/custom/custom.scss create mode 100644 docs/assets/css/site.css delete mode 100644 docs/assets/js/tabs.js create mode 100644 docs/config/scientific-python.yml create mode 100644 docs/myst.yml create mode 100644 docs/rr-role.mjs diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index caa7ca73..4242b8cb 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -92,5 +92,5 @@ repos: name: Cog the pages language: python entry: cog -P -r -I ./helpers - files: "^docs/pages/guides/(packaging_compiled|docs|tasks|gha_basic).md|^copier.yml|^docs/_includes/pyproject.md" + files: "^docs/pages/guides/(packaging_compiled|docs|tasks|gha_basic).md|^copier.yml|^docs/_partials/pyproject.md" additional_dependencies: [cogapp, cookiecutter, tomlkit] diff --git a/.readthedocs.yaml b/.readthedocs.yaml index eb3d433f..1e94b386 100644 --- a/.readthedocs.yaml +++ b/.readthedocs.yaml @@ -5,17 +5,13 @@ # Required version: 2 -# Set the version of Python and other tools you might need build: os: ubuntu-24.04 tools: - ruby: "3.4" + nodejs: "22" commands: - ./helpers/fetch_repo_review_app.sh - - bundle install - - > - JEKYLL_ENV=production bundle exec jekyll build --destination - _readthedocs/html --baseurl $(echo -n "$READTHEDOCS_CANONICAL_URL" | cut - -d '/' -f 4-) + - npm install -g mystmd + - cd docs && myst build --html --output _readthedocs/html diff --git a/.ruby-version b/.ruby-version deleted file mode 100644 index 47b322c9..00000000 --- a/.ruby-version +++ /dev/null @@ -1 +0,0 @@ -3.4.1 diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..db1c8485 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,57 @@ +# Agent Guide for scientific-python/cookie + +This repo has three distinct concerns: a **cookiecutter/copier template** for new Python projects (`{{cookiecutter.project_name}}/`), a **repo-review plugin** (`src/sp_repo_review/`), and a **Jekyll-based developer guide** (`docs/`). + +## Key commands + +### sp-repo-review (the package) + +- Run tests: `uv run pytest` (or `uvx nox -s rr_tests`) +- Run linting (pre-commit via prek): `uvx nox -s rr_lint` +- Run pylint: `uvx nox -s rr_pylint` +- Run the CLI: `uvx nox -s rr_run -- ` +- Regenerate README check list via cog: `uvx nox -s readme` +- Build wheel/sdist: `uvx nox -s rr_build` + +Important: tests run with `PYTHONWARNDEFAULTENCODING=1`. + +### Cookie template validation + +The noxfile generates temporary projects for **all 9 backends** × **vcs on/off** × **3 docs engines** (sphinx/mkdocs/zensical). These are slow. + +- `nox -s "tests(hatch)"` — run generated project tests for a single backend +- `nox -s "lint(hatch)"` — run pre-commit (`prek`) on generated project +- `nox -s "dist(hatch)"` — verify build output includes LICENSE +- `nox -s "native(hatch)"` — test hatch/pdm/poetry native test runners +- `nox -s compare_copier` — verify cookiecutter and copier produce identical files +- `nox -s compare_cruft` — verify cookiecutter and cruft produce identical files +- `nox -s gha_bump` — bump GitHub Actions versions across docs and templates +- `nox -s pc_bump` — bump pre-commit hook versions across docs and templates + +## Architecture notes + +- **Template directory**: `{{cookiecutter.project_name}}/` contains the cookiecuttter template. Copier reads `copier.yml`; cookiecutter reads `cookiecutter.json`. Keep them in sync; `compare_copier` checks this. +- **Entry points**: `sp_repo_review` registers `repo-review` checks/families/fixtures via `[project.entry-points]` in `pyproject.toml`. +- **Generated docs**: The README check list (line ~300+) is a cog block. Do not edit it by hand; run `nox -s readme` or cog will fail in CI. +- **Cookie template `.pre-commit-config.yaml`** uses `prek` (a Rust-based pre-commit alternative), not `pre-commit`. +- **Ruff hook ID**: `.pre-commit-config.yaml` uses `ruff-check` as the hook id (not `ruff`), for pre-commit 4.x compatibility. + +## Style and conventions + +- `tool.pytest.ini_options.norecursedirs` excludes `{{cookiecutter.project_name}}` so pytest does not descend into the template directory. +- `tool.ruff.extend-exclude` also excludes `\{\{cookiecutter.project_name\}\}` (double-escaped in TOML). +- `tool.mypy.python_version = "3.10"`; the `sp_repo_review.*` override enforces `disallow_untyped_defs=True`. +- `tool.pylint.master.ignore-paths` ignores `src/sp_repo_review/_version.py` (auto-generated by hatch-vcs). +- Ruff selects `ALL` with many ignores; notable: `S101` (assert) and `D` (docstrings) are globally disabled. + +## CI quirks + +- CI uses change detection to decide whether to run cookie tests or rr-tests. Both are required to pass for the `pass` job. +- rr-tests matrix runs on Python 3.10, 3.12, 3.14 across ubuntu/macos/windows. +- Cookie tests reuse the same `reusable-cookie.yml` workflow. + +## Docs site (Jekyll) + +- Ruby-based; uses `rbenv` + `bundle install` + `bundle exec jekyll serve --livereload`. +- There is a helper script `helpers/fetch_repo_review_app.sh`. +- Docs pages in `docs/pages/` contain inline cog blocks that auto-generate config examples from the template. diff --git a/Gemfile b/Gemfile deleted file mode 100644 index 01fee0f6..00000000 --- a/Gemfile +++ /dev/null @@ -1,48 +0,0 @@ -# frozen_string_literal: true - -source 'https://rubygems.org' - -# Hello! This is where you manage which Jekyll version is used to run. -# When you want to use a different version, change it below, save the -# file and run `bundle install`. Run Jekyll with `bundle exec`, like so: -# -# bundle exec jekyll serve -# -# This will help ensure the proper Jekyll version is running. -# Happy Jekylling! - -gem 'jekyll' - -# This is the theme -gem 'just-the-docs' - -# This is needed for GitHub Flavored Markdown -gem 'kramdown-parser-gfm' - -# Used to be in the stdlib -gem 'logger' - -# If you have any plugins, put them here! -group :jekyll_plugins do - gem 'jekyll-feed' - gem 'jekyll-seo-tag' -end - -group :development do - # Verify good coding practices in Ruby files - gem 'rubocop', '~>1.52', require: false - - # Check links. Use: - # bundle exec jekyll build - # bundle exec htmlproofer --assume_extension '.html' ./_site - gem 'html-proofer' -end - -# Windows and JRuby does not include zoneinfo files, so bundle the tzinfo-data gem -# and associated library. -platforms :mingw, :x64_mingw, :mswin, :jruby do - gem 'tzinfo' - gem 'tzinfo-data' -end - -gem 'wdm', install_if: Gem.win_platform? diff --git a/Gemfile.lock b/Gemfile.lock deleted file mode 100644 index 40ecaa71..00000000 --- a/Gemfile.lock +++ /dev/null @@ -1,243 +0,0 @@ -GEM - remote: https://rubygems.org/ - specs: - Ascii85 (2.0.1) - addressable (2.8.9) - public_suffix (>= 2.0.2, < 8.0) - afm (1.0.0) - ast (2.4.3) - async (2.38.1) - console (~> 1.29) - fiber-annotation - io-event (~> 1.11) - metrics (~> 0.12) - traces (~> 0.18) - base64 (0.3.0) - benchmark (0.5.0) - bigdecimal (3.3.1) - colorator (1.1.0) - concurrent-ruby (1.3.6) - console (1.34.3) - fiber-annotation - fiber-local (~> 1.1) - json - csv (3.3.5) - em-websocket (0.5.3) - eventmachine (>= 0.12.9) - http_parser.rb (~> 0) - ethon (0.18.0) - ffi (>= 1.15.0) - logger - eventmachine (1.2.7) - ffi (1.17.4-aarch64-linux-gnu) - ffi (1.17.4-aarch64-linux-musl) - ffi (1.17.4-arm-linux-gnu) - ffi (1.17.4-arm-linux-musl) - ffi (1.17.4-arm64-darwin) - ffi (1.17.4-x86_64-darwin) - ffi (1.17.4-x86_64-linux-gnu) - ffi (1.17.4-x86_64-linux-musl) - fiber-annotation (0.2.0) - fiber-local (1.1.0) - fiber-storage - fiber-storage (1.0.1) - forwardable-extended (2.6.0) - google-protobuf (4.34.1) - bigdecimal - rake (~> 13.3) - google-protobuf (4.34.1-aarch64-linux-gnu) - bigdecimal - rake (~> 13.3) - google-protobuf (4.34.1-aarch64-linux-musl) - bigdecimal - rake (~> 13.3) - google-protobuf (4.34.1-arm64-darwin) - bigdecimal - rake (~> 13.3) - google-protobuf (4.34.1-x86_64-darwin) - bigdecimal - rake (~> 13.3) - google-protobuf (4.34.1-x86_64-linux-gnu) - bigdecimal - rake (~> 13.3) - google-protobuf (4.34.1-x86_64-linux-musl) - bigdecimal - rake (~> 13.3) - hashery (2.1.2) - html-proofer (5.2.1) - addressable (~> 2.3) - async (~> 2.1) - benchmark (~> 0.5) - nokogiri (~> 1.13) - pdf-reader (~> 2.11) - rainbow (~> 3.0) - typhoeus (~> 1.3) - yell (~> 2.0) - zeitwerk (~> 2.5) - http_parser.rb (0.8.1) - i18n (1.14.8) - concurrent-ruby (~> 1.0) - io-event (1.14.5) - jekyll (4.4.1) - addressable (~> 2.4) - base64 (~> 0.2) - colorator (~> 1.0) - csv (~> 3.0) - em-websocket (~> 0.5) - i18n (~> 1.0) - jekyll-sass-converter (>= 2.0, < 4.0) - jekyll-watch (~> 2.0) - json (~> 2.6) - kramdown (~> 2.3, >= 2.3.1) - kramdown-parser-gfm (~> 1.0) - liquid (~> 4.0) - mercenary (~> 0.3, >= 0.3.6) - pathutil (~> 0.9) - rouge (>= 3.0, < 5.0) - safe_yaml (~> 1.0) - terminal-table (>= 1.8, < 4.0) - webrick (~> 1.7) - jekyll-feed (0.17.0) - jekyll (>= 3.7, < 5.0) - jekyll-include-cache (0.2.1) - jekyll (>= 3.7, < 5.0) - jekyll-sass-converter (3.1.0) - sass-embedded (~> 1.75) - jekyll-seo-tag (2.8.0) - jekyll (>= 3.8, < 5.0) - jekyll-watch (2.2.1) - listen (~> 3.0) - json (2.19.3) - just-the-docs (0.12.0) - jekyll (>= 3.8.5) - jekyll-include-cache - jekyll-seo-tag (>= 2.0) - rake (>= 12.3.1) - kramdown (2.5.2) - rexml (>= 3.4.4) - kramdown-parser-gfm (1.1.0) - kramdown (~> 2.0) - language_server-protocol (3.17.0.5) - lint_roller (1.1.0) - liquid (4.0.4) - listen (3.10.0) - logger - rb-fsevent (~> 0.10, >= 0.10.3) - rb-inotify (~> 0.9, >= 0.9.10) - logger (1.7.0) - mercenary (0.4.0) - metrics (0.15.0) - nokogiri (1.19.2-aarch64-linux-gnu) - racc (~> 1.4) - nokogiri (1.19.2-aarch64-linux-musl) - racc (~> 1.4) - nokogiri (1.19.2-arm-linux-gnu) - racc (~> 1.4) - nokogiri (1.19.2-arm-linux-musl) - racc (~> 1.4) - nokogiri (1.19.2-arm64-darwin) - racc (~> 1.4) - nokogiri (1.19.2-x86_64-darwin) - racc (~> 1.4) - nokogiri (1.19.2-x86_64-linux-gnu) - racc (~> 1.4) - nokogiri (1.19.2-x86_64-linux-musl) - racc (~> 1.4) - parallel (1.28.0) - parser (3.3.11.1) - ast (~> 2.4.1) - racc - pathutil (0.16.2) - forwardable-extended (~> 2.6) - pdf-reader (2.15.1) - Ascii85 (>= 1.0, < 3.0, != 2.0.0) - afm (>= 0.2.1, < 2) - hashery (~> 2.0) - ruby-rc4 - ttfunk - prism (1.9.0) - public_suffix (7.0.5) - racc (1.8.1) - rainbow (3.1.1) - rake (13.3.1) - rb-fsevent (0.11.2) - rb-inotify (0.11.1) - ffi (~> 1.0) - regexp_parser (2.11.3) - rexml (3.4.4) - rouge (4.7.0) - rubocop (1.86.0) - json (~> 2.3) - language_server-protocol (~> 3.17.0.2) - lint_roller (~> 1.1.0) - parallel (~> 1.10) - parser (>= 3.3.0.2) - rainbow (>= 2.2.2, < 4.0) - regexp_parser (>= 2.9.3, < 3.0) - rubocop-ast (>= 1.49.0, < 2.0) - ruby-progressbar (~> 1.7) - unicode-display_width (>= 2.4.0, < 4.0) - rubocop-ast (1.49.1) - parser (>= 3.3.7.2) - prism (~> 1.7) - ruby-progressbar (1.13.0) - ruby-rc4 (0.1.5) - safe_yaml (1.0.5) - sass-embedded (1.98.0-aarch64-linux-gnu) - google-protobuf (~> 4.31) - sass-embedded (1.98.0-aarch64-linux-musl) - google-protobuf (~> 4.31) - sass-embedded (1.98.0-arm-linux-gnueabihf) - google-protobuf (~> 4.31) - sass-embedded (1.98.0-arm-linux-musleabihf) - google-protobuf (~> 4.31) - sass-embedded (1.98.0-arm64-darwin) - google-protobuf (~> 4.31) - sass-embedded (1.98.0-x86_64-darwin) - google-protobuf (~> 4.31) - sass-embedded (1.98.0-x86_64-linux-gnu) - google-protobuf (~> 4.31) - sass-embedded (1.98.0-x86_64-linux-musl) - google-protobuf (~> 4.31) - terminal-table (3.0.2) - unicode-display_width (>= 1.1.1, < 3) - traces (0.18.2) - ttfunk (1.8.0) - bigdecimal (~> 3.1) - typhoeus (1.6.0) - ethon (>= 0.18.0) - unicode-display_width (2.6.0) - wdm (0.2.0) - webrick (1.9.2) - yell (2.2.2) - zeitwerk (2.7.5) - -PLATFORMS - aarch64-linux - aarch64-linux-gnu - aarch64-linux-musl - arm-linux-gnu - arm-linux-gnueabihf - arm-linux-musl - arm-linux-musleabihf - arm64-darwin - x86_64-darwin - x86_64-linux - x86_64-linux-gnu - x86_64-linux-musl - -DEPENDENCIES - html-proofer - jekyll - jekyll-feed - jekyll-seo-tag - just-the-docs - kramdown-parser-gfm - logger - rubocop (~> 1.52) - tzinfo - tzinfo-data - wdm - -BUNDLED WITH - 2.6.5 diff --git a/_config.yml b/_config.yml deleted file mode 100644 index f55fae1d..00000000 --- a/_config.yml +++ /dev/null @@ -1,57 +0,0 @@ -title: Scientific Python Development Guide -email: guide@scientific-python.org -description: >- - This guide is maintained by the scientific Python community for the benefit of - fellow scientists and research software engineers. -github_username: scientific-python -source: docs - -# Build settings -markdown: kramdown -theme: "just-the-docs" -plugins: - - jekyll-feed - -# Theme settings - -logo: "/assets/images/logo.svg" - -# Enable or disable the site search -search_enabled: true - -# Aux links for the upper right navigation -aux_links: - Scientific Python: - - https://scientific-python.org - Learn: - - https://learn.scientific-python.org - Cookie: - - https://github.com/scientific-python/cookie - -gh_edit_link: true -gh_edit_link_text: "View source for this page on GitHub." -gh_edit_repository: "https://github.com/scientific-python/cookie" -gh_edit_branch: "main" -gh_edit_source: "docs" -gh_edit_view_mode: "tree" # "tree" or "edit" - -callouts: - warning: - color: red - title: Warning - note: - color: yellow - title: Note - important: - color: green - title: Important - highlight: - color: blue - -# Workaround for dep warnings -sass: - quiet_deps: true - silence_deprecations: - - import - - color-functions - - global-builtin diff --git a/docs/_includes/head.html b/docs/_includes/head.html deleted file mode 100644 index 732e0399..00000000 --- a/docs/_includes/head.html +++ /dev/null @@ -1,74 +0,0 @@ - - - - - {% unless site.plugins contains "jekyll-seo-tag" %} - {{ page.title }} - {{ site.title }} - - {% if page.description %} - - {% endif %} {% endunless %} - - - - {% if false %} - - {% endif %} - - - - - - - {% if site.ga_tracking != nil %} - - - - {% endif %} {% if site.search_enabled != false %} - - {% endif %} - - - - - {% seo %} {% include head_custom.html %} - diff --git a/docs/_includes/head_custom.html b/docs/_includes/head_custom.html deleted file mode 100644 index 96fd84d7..00000000 --- a/docs/_includes/head_custom.html +++ /dev/null @@ -1,20 +0,0 @@ - - -{%- if page.interactive_repo_review %} - - - - - - - -{%- endif %} diff --git a/docs/_includes/interactive_repo_review.html b/docs/_includes/interactive_repo_review.html deleted file mode 100644 index ce4ff0dc..00000000 --- a/docs/_includes/interactive_repo_review.html +++ /dev/null @@ -1,25 +0,0 @@ -
Loading (requires javascript and WebAssembly)...
- - diff --git a/docs/_includes/toc.html b/docs/_includes/toc.html deleted file mode 100644 index fb09be8b..00000000 --- a/docs/_includes/toc.html +++ /dev/null @@ -1,7 +0,0 @@ - -
- Table of contents - - * TOC - {:toc .m-1} -
diff --git a/docs/_includes/pyproject.md b/docs/_partials/pyproject.md similarity index 100% rename from docs/_includes/pyproject.md rename to docs/_partials/pyproject.md diff --git a/docs/_plugins/details.rb b/docs/_plugins/details.rb deleted file mode 100644 index 25d4d35e..00000000 --- a/docs/_plugins/details.rb +++ /dev/null @@ -1,22 +0,0 @@ -# frozen_string_literal: true - -module SP - # Sets up a details / summary block - class Details < Liquid::Block - def initialize(tag_name, markup, tokens) - super - @title = markup.strip - end - - def render(context) - <<~RETURN -
- #{@title} - #{super} -
- RETURN - end - end -end - -Liquid::Template.register_tag('details', SP::Details) diff --git a/docs/_plugins/repo_review.rb b/docs/_plugins/repo_review.rb deleted file mode 100644 index 82882f56..00000000 --- a/docs/_plugins/repo_review.rb +++ /dev/null @@ -1,19 +0,0 @@ -# frozen_string_literal: true - -module SP - # Sets up a repo review badge - class RepoReview < Liquid::Tag - def initialize(tag_name, markup, tokens) - super - @code = markup.strip - raise SyntaxError, 'RepoReview requires a code' unless @code - raise SyntaxError, 'RepoReview code must not contain a space' if @code.include? ' ' - end - - def render(_context) - %(#{@code}) - end - end -end - -Liquid::Template.register_tag('rr', SP::RepoReview) diff --git a/docs/_plugins/tabs.rb b/docs/_plugins/tabs.rb deleted file mode 100644 index b182746d..00000000 --- a/docs/_plugins/tabs.rb +++ /dev/null @@ -1,71 +0,0 @@ -# frozen_string_literal: true - -module SP - # Sets up a tabs with top switcher bar - class Tabs < Liquid::Block - def initialize(tag_name, markup, tokens) - super - @group = markup.strip.empty? ? 'default' : markup.strip - end - - def render(context) - tab_bar_content = '' - context['tabs'] = [] - context['tab_group'] = @group - result = super - - context['tabs'].each_with_index do |(label, title), index| - res = index.zero? ? ' btn-purple' : '' - tab_bar_content += <<~CONTENT - - CONTENT - end - - <<~RETURN -
- #{tab_bar_content} -
- #{result} - RETURN - end - end - - # Sets up tabs without the top switcher bar - class TabBodies < Liquid::Block - def initialize(tag_name, markup, tokens) - super - @group = markup.strip.empty? ? 'default' : markup.strip - end - - def render(context) - context['tabs'] = [] - context['tab_group'] = @group - super - end - end - - # This is the content of each tab - class Tab < Liquid::Block - def initialize(tag_name, markup, tokens) - super - @label, @title = markup.strip.split(/ /, 2) - end - - def render(context) - raise SyntaxError, "'tab' be in 'tabs' or 'tabbodies'" unless context.key?('tabs') - - group = context['tab_group'] || 'default' - res = context['tabs'].empty? ? '' : ' style="display:none;"' - context['tabs'] << [@label, @title] - <<~RETURN -
- #{super} -
- RETURN - end - end -end - -Liquid::Template.register_tag('tabs', SP::Tabs) -Liquid::Template.register_tag('tabbodies', SP::TabBodies) -Liquid::Template.register_tag('tab', SP::Tab) diff --git a/docs/_sass/color_schemes/skhep.scss b/docs/_sass/color_schemes/skhep.scss deleted file mode 100644 index acf9f5b4..00000000 --- a/docs/_sass/color_schemes/skhep.scss +++ /dev/null @@ -1 +0,0 @@ -$link-color: #7092be; diff --git a/docs/_sass/custom/custom.scss b/docs/_sass/custom/custom.scss deleted file mode 100644 index 05d5b483..00000000 --- a/docs/_sass/custom/custom.scss +++ /dev/null @@ -1,155 +0,0 @@ -@media (min-width: 50rem) { - div.site-header { - height: 264px; - max-height: 264px; - } -} - -details > summary { - font-weight: bold; -} - -div.site-logo { - background-position: center center; -} - -a.site-title { - padding: 0px 6px 0px 6px; -} - -div.noted { - color: #ff6347; - font-weight: bold; - float: right; -} - -a.package { - font-size: larger; - font-weight: bold; -} - -div.project { - display: flex; - flex-direction: row; - flex-wrap: wrap; - justify-content: center; - box-shadow: 0px 5px 5px #88888840; - margin: 10px 0px; - padding: 2px; - border-radius: 5px; - background-color: #f5f6fa; - @media (prefers-color-scheme: dark) { - background-color: #000000; - box-shadow: 0px 0px 5px rgb(106, 143, 188); - } -} - -div.project.affiliated { -} - -div.project.deprecated { - background-color: #ffdad9; - @media (prefers-color-scheme: dark) { - background-color: #501109; - } -} - -div.project .nameorlogo { - flex-basis: 200px; - flex-shrink: 0; - padding: 2px; - text-align: center; -} - -div.project .nameorlogo a img { - vertical-align: middle; -} - -div.project .description { - flex-basis: 150px; - flex-shrink: 0; - flex-grow: 1; - padding: 2px; -} - -div.description .inner p { - display: inline; -} - -pre.highlight { - line-height: 1.2; -} - -details { - padding: 4px; - margin-bottom: 1.5rem; - border-radius: 4px; - box-shadow: - 0 2px 3px rgba(0, 0, 0, 0.12), - 0 3px 10px rgba(0, 0, 0, 0.08); - @media (prefers-color-scheme: dark) { - box-shadow: 0 0 4px rgba(255, 255, 255, 0.4); - background-color: black; - } -} - -details:not([open]) > summary::after { - content: "(expand)"; - float: right; -} -details[open] > summary::after { - content: "(close)"; - float: right; -} - -details[open] { - summary { - margin-bottom: 1em; - } - padding-bottom: 0.5em; -} - -blockquote { - display: block; - margin-block-start: 0; - margin-block-end: 0; - margin-inline-start: 0; - margin-inline-end: 0; - - padding: 4px 1em; - margin-bottom: 1.5rem; - border-radius: 4px; - box-shadow: - 0 2px 3px rgba(0, 0, 0, 0.12), - 0 3px 10px rgba(0, 0, 0, 0.08); - @media (prefers-color-scheme: dark) { - box-shadow: 0 0 4px rgba(255, 255, 255, 0.4); - background-color: black; - } -} - -blockquote > h2 { - font-size: 16px !important; - text-transform: uppercase; - line-height: 1.25; - font-weight: 300; - letter-spacing: 2px; -} - -.rr-btn { - border-radius: 4px; - background-color: #808080; - color: white; - padding: 1px 4px; - font-size: 9pt; - font-family: sans-serif; - display: inline-block; - text-align: center; - line-height: 1; -} -.rr-btn::before { - content: "sp-repo-review"; - font-size: 5pt; - display: block; - text-align: center; -} diff --git a/docs/assets/css/site.css b/docs/assets/css/site.css new file mode 100644 index 00000000..4b366d63 --- /dev/null +++ b/docs/assets/css/site.css @@ -0,0 +1,110 @@ +/* Scientific Python theme base + * Adapted from https://github.com/scientific-python/scientific-python-myst-theme + */ + +body { + display: flex; + flex-direction: column; + min-height: 100vh; +} + +main { + min-height: 0; + flex: 1 0 auto; +} + +.article.content { + min-height: 0; +} + +.footer { + flex-shrink: 0; + background: #013243; + color: white; + padding-left: 2rem; + padding-right: 2rem; + padding-left: 3.5rem; + padding-right: 3.5rem; + + & .outer-grid { + grid-template-columns: 3fr 3fr 4fr; + align-items: center; + margin-bottom: 0rem; + + & li { + list-style: none; + } + } + + @media (max-width: 640px) { + & .outer-grid { + grid-template-columns: 1fr; + justify-items: start; + } + } + + & a, + h1, + h2, + h3, + h4, + h5, + h6 { + color: white; + } + + & h1 { + font-size: 1.25rem; + font-weight: bold; + } +} + +.myst-fm-downloads-button { + display: none; +} + +.myst-home-link { + font-weight: bold; +} + +.bd-main .bd-content .bd-article-container { + max-width: 900px; +} + +h1, +h2, +h3, +h4 { + letter-spacing: -0.01em; +} + +.bd-header { + box-shadow: none; +} + +.bd-sidebar-primary { + border-right: 0; +} + +/* ------------------------------------------------------------------ */ +/* rr-btn: inline repo-review check badge, rendered by {rr} MyST role */ +/* ------------------------------------------------------------------ */ + +.rr-btn { + border-radius: 4px; + background-color: #808080; + color: white; + padding: 1px 4px; + font-size: 9pt; + font-family: sans-serif; + display: inline-block; + text-align: center; + line-height: 1; +} + +.rr-btn::before { + content: "sp-repo-review"; + font-size: 5pt; + display: block; + text-align: center; +} diff --git a/docs/assets/js/tabs.js b/docs/assets/js/tabs.js deleted file mode 100644 index 41b87fdb..00000000 --- a/docs/assets/js/tabs.js +++ /dev/null @@ -1,36 +0,0 @@ -function openTab(tabName, groupName = "default") { - var tab = document.getElementsByClassName("skhep-tab"); - for (const t of tab) { - if (t.classList.contains(`${groupName}-${tabName}-tab`)) { - t.style.display = "block"; - } else if ( - Array.from(t.classList).some( - (c) => c.startsWith(`${groupName}-`) && c.endsWith("-tab"), - ) - ) { - t.style.display = "none"; - } - } - var btn = document.getElementsByClassName("skhep-bar-item"); - for (const b of btn) { - if (b.classList.contains(`${groupName}-${tabName}-btn`)) { - b.classList.add("btn-purple"); - } else if ( - Array.from(b.classList).some( - (c) => c.startsWith(`${groupName}-`) && c.endsWith("-btn"), - ) - ) { - b.classList.remove("btn-purple"); - } - } -} -function ready() { - const urlParams = new URLSearchParams(window.location.search); - const tabs = urlParams.getAll("tabs"); - - for (const tab of tabs) { - openTab(tab); - } -} - -document.addEventListener("DOMContentLoaded", ready, false); diff --git a/docs/config/scientific-python.yml b/docs/config/scientific-python.yml new file mode 100644 index 00000000..00bf1391 --- /dev/null +++ b/docs/config/scientific-python.yml @@ -0,0 +1,9 @@ +version: 1 + +site: + template: book-theme + options: + hide_toc: false + hide_footer_links: true + folders: true + hide_myst_branding: true diff --git a/docs/myst.yml b/docs/myst.yml new file mode 100644 index 00000000..c5cc4d11 --- /dev/null +++ b/docs/myst.yml @@ -0,0 +1,61 @@ +version: 1 +extends: config/scientific-python.yml + +project: + title: Scientific Python Development Guide + github: https://github.com/scientific-python/cookie + plugins: + - rr-role.mjs + + toc: + - file: pages/index + - title: Tutorials + children: + - file: pages/tutorials/index + - file: pages/tutorials/dev-environment + - file: pages/tutorials/module + - file: pages/tutorials/packaging + - file: pages/tutorials/test + - file: pages/tutorials/docs + - title: Topical Guides + children: + - file: pages/guides/index + - file: pages/guides/pytest + - file: pages/guides/coverage + - file: pages/guides/docs + - file: pages/guides/packaging_simple + - file: pages/guides/packaging_compiled + - file: pages/guides/packaging_classic + - file: pages/guides/style + - file: pages/guides/mypy + - file: pages/guides/gha_basic + - file: pages/guides/gha_pure + - file: pages/guides/gha_wheels + - file: pages/guides/tasks + - file: pages/guides/repo_review + - title: Principles + children: + - file: pages/principles/index + - file: pages/principles/process + - file: pages/principles/design + - file: pages/principles/testing + - title: Patterns + children: + - file: pages/patterns/index + - file: pages/patterns/exports + - file: pages/patterns/backports + - file: pages/patterns/data_files + +site: + nav: + - title: Scientific Python + url: https://scientific-python.org + - title: Learn + url: https://learn.scientific-python.org + - title: Cookie + url: https://github.com/scientific-python/cookie + options: + favicon: assets/favicon.ico + logo: assets/images/logo.svg + logo_text: Scientific Python Development Guide + style: assets/css/site.css diff --git a/docs/pages/guides/coverage.md b/docs/pages/guides/coverage.md index d1469897..72b321aa 100644 --- a/docs/pages/guides/coverage.md +++ b/docs/pages/guides/coverage.md @@ -1,13 +1,7 @@ --- -layout: page title: "Code coverage" -permalink: /guides/coverage/ -nav_order: 3 -parent: Topical Guides --- -{% include toc.html %} - # Code Coverage The "Code coverage" value of a codebase indicates how much of the @@ -27,20 +21,16 @@ Tools and libraries used to calculate, read, and visualize coverage reports: - `GitHub Actions`: allows users to automatically upload coverage reports to `Codecov` -{: .note-title } - -> Are there any alternatives? -> -> Coveralls is an alternative coverage platform, but we recommend using Codecov -> because of its ease of use and integration with GitHub Actions. - -{: .highlight-title } +:::{note} Are there any alternatives? +Coveralls is an alternative coverage platform, but we recommend using Codecov +because of its ease of use and integration with GitHub Actions. +::: -> Should increasing the coverage value be my top priority? -> -> A low coverage percentage will definitely motivate you to add more tests, but -> adding weak tests just for coverage's sake is not a good idea. The tests -> should test your codebase thoroughly and should not be unreliable. +:::{tip} Should increasing the coverage value be my top priority? +A low coverage percentage will definitely motivate you to add more tests, but +adding weak tests just for coverage's sake is not a good idea. The tests +should test your codebase thoroughly and should not be unreliable. +::: ### Running your tests with coverage @@ -53,8 +43,8 @@ directly is hidden away from normal use, so we recommend that, but will show both methods below. If you are not running `pytest`, but instead are running an example or a script, you have to use `coverage` directly. -{% tabs %} {% tab cov coverage %} - +::::{tab-set} +:::{tab-item} coverage Make sure you install `coverage[toml]`. `coverage` has several commands; the most important one is `coverage run`. This @@ -75,9 +65,8 @@ coverage report This looks for a `.coverage` file and displays the result. There are many output formats for reports. - -{% endtab %} {% tab pycov pytest-cov %} - +::: +:::{tab-item} pytest-cov Make sure you install `pytest-cov`. `pytest` allows users to pass the `--cov` option to automatically invoke @@ -98,8 +87,8 @@ See the [docs](https://pytest-cov.readthedocs.io/en/latest/) for more options. Coverage pytest arguments can be placed in your pytest configuration file or in your task runner. It also will (mostly) respect the coverage configuration, shown below. - -{% endtab %} {% endtabs %} +::: +:::: ### Configuring coverage @@ -158,8 +147,8 @@ manually produce multiple files from task runner jobs. Here's an example nox job: -{% tabs %} {% tab cov coverage %} - +::::{tab-set} +:::{tab-item} coverage ```python @nox.session(python=ALL_PYTHONS) def tests(session: nox.Session) -> None: @@ -174,9 +163,8 @@ def tests(session: nox.Session) -> None: env={"COVERAGE_FILE": coverage_file}, ) ``` - -{% endtab %} {% tab pycov pytest-cov %} - +::: +:::{tab-item} pytest-cov ```python @nox.session(python=ALL_PYTHONS) def tests(session: nox.Session) -> None: @@ -190,8 +178,8 @@ def tests(session: nox.Session) -> None: env={"COVERAGE_FILE": coverage_file}, ) ``` - -{% endtab %} {% endtabs %} +::: +:::: #### Merging and reporting @@ -280,5 +268,3 @@ loss of coverage to fail. See the TODO --> [codecov/codecov-action]: https://github.com/codecov/codecov-action - - diff --git a/docs/pages/guides/docs.md b/docs/pages/guides/docs.md index bceb5281..6979ee28 100644 --- a/docs/pages/guides/docs.md +++ b/docs/pages/guides/docs.md @@ -1,13 +1,7 @@ --- -layout: page title: Writing documentation -permalink: /guides/docs/ -nav_order: 4 -parent: Topical Guides --- -{% include toc.html %} - # Writing documentation Documentation used to require learning reStructuredText (sometimes referred to @@ -15,45 +9,41 @@ as reST / rST), but today we have great choices for documentation in markdown, the same format used by GitHub, Wikipedia, and others. This guide covers Sphinx and Mkdocs, and uses the modern MyST plugin to get Markdown support. -{: .note-title } - -> Popular frameworks -> -> There are other frameworks as well; these often are simpler, but are not as -> commonly used, and have somewhat fewer examples and plugins. They are: -> -> - [Sphinx](https://www.sphinx-doc.org/en/master/): A popular documentation -> framework for scientific libraries with a history of close usage with -> scientific tools like LaTeX. Examples include -> [astropy](https://docs.astropy.org/en/stable/index_user_docs.html) and -> [corner](https://docs.astropy.org/en/stable/index_user_docs.html). -> - [MkDocs](https://www.mkdocs.org): A from-scratch new documentation system -> based on markdown and HTML. Less support for man pages & PDFs than Sphinx, -> since it doesn't use docutils. Has over -> [200 plugins](https://github.com/mkdocs/catalog) - they are much easier to -> write than Sphinx. Example sites include [hatch](https://hatch.pypa.io), -> [PDM](https://pdm.fming.dev), -> [cibuildwheel](https://cibuildwheel.readthedocs.io), -> [Textual](https://textual.textualize.io), -> [pipx](https://pypa.github.io/pipx/), -> [Pydantic](https://docs.pydantic.dev/latest/), -> [Polars](https://docs.pola.rs/), and -> [FastAPI](https://fastapi.tiangolo.com/) -> - [JupyterBook](https://jupyterbook.org): A powerful system for rendering a -> collection of notebooks using Sphinx internally. Can also be used for docs, -> though, see [echopype](https://echopype.readthedocs.io). - -{: .warning-title } - -> The Future of MkDocs -> -> The creators of `mkdocs-material` and `mkdocstrings` have come together to -> create a new documentation package called -> [Zensical](https://zensical.org/about/). The framework is still in alpha -> development, but aims to simplify the documentation process, be blazing fast, -> and move away from the limitations of MkDocs. This also means MkDocs's future -> is uncertain, and mkdocs-material will be minimally maintained until -> late 2026. +:::{note} Popular frameworks +There are other frameworks as well; these often are simpler, but are not as +commonly used, and have somewhat fewer examples and plugins. They are: + +- [Sphinx](https://www.sphinx-doc.org/en/master/): A popular documentation + framework for scientific libraries with a history of close usage with + scientific tools like LaTeX. Examples include + [astropy](https://docs.astropy.org/en/stable/index_user_docs.html) and + [corner](https://docs.astropy.org/en/stable/index_user_docs.html). +- [MkDocs](https://www.mkdocs.org): A from-scratch new documentation system + based on markdown and HTML. Less support for man pages & PDFs than Sphinx, + since it doesn't use docutils. Has over + [200 plugins](https://github.com/mkdocs/catalog) - they are much easier to + write than Sphinx. Example sites include [hatch](https://hatch.pypa.io), + [PDM](https://pdm.fming.dev), + [cibuildwheel](https://cibuildwheel.readthedocs.io), + [Textual](https://textual.textualize.io), + [pipx](https://pypa.github.io/pipx/), + [Pydantic](https://docs.pydantic.dev/latest/), + [Polars](https://docs.pola.rs/), and + [FastAPI](https://fastapi.tiangolo.com/) +- [JupyterBook](https://jupyterbook.org): A powerful system for rendering a + collection of notebooks using Sphinx internally. Can also be used for docs, + though, see [echopype](https://echopype.readthedocs.io). +::: + +:::{warning} The Future of MkDocs +The creators of `mkdocs-material` and `mkdocstrings` have come together to +create a new documentation package called +[Zensical](https://zensical.org/about/). The framework is still in alpha +development, but aims to simplify the documentation process, be blazing fast, +and move away from the limitations of MkDocs. This also means MkDocs's future +is uncertain, and mkdocs-material will be minimally maintained until +late 2026. +::: ## What to include @@ -67,12 +57,10 @@ Ideally, software documentation should include: - **Explanations** to convey deeper understanding of why and how the software operates the way it does. -{: .note-title } - -> The Diátaxis framework -> -> This overall framework has a name, [Diátaxis][], and you can read more about -> it if you are interested. +:::{note} The Diátaxis framework +This overall framework has a name, [Diátaxis][], and you can read more about +it if you are interested. +::: ` above) to mark where you want the docs portion to start. You can add the standard indices and tables at the end. - -{% endtab %} {% tab mkdocs MkDocs %} - +::: +:::{tab-item} MkDocs While the cookie cutter creates a basic structure for your MkDocs (a top level `mkdocs.yml` file and the `docs` directory), you can also follow the official [Getting started](https://squidfunk.github.io/mkdocs-material/getting-started/) @@ -477,17 +464,17 @@ nav: - Home: index.md - Python API: api.md ``` - -{% endtab %} {% endtabs %} +::: +:::: ### .readthedocs.yaml In order to use to build, host, and preview your -documentation, you must have a `.readthedocs.yaml` file {% rr RTD100 %} like +documentation, you must have a `.readthedocs.yaml` file {rr}`RTD100` like this: -{% tabs %} {% tab sphinx Sphinx %} - +::::{tab-set} +:::{tab-item} Sphinx - -{% endtab %} {% tab mkdocs MkDocs %} - +::: +:::{tab-item} MkDocs +::: +:::: -{% endtab %} {% endtabs %} - -This sets the Read the Docs config version (2 is required) {% rr RTD101 %}. +This sets the Read the Docs config version (2 is required) {rr}`RTD101`. The `build` table is the modern way to specify a runner. You need an `os` (a -modern Ubuntu should be fine) {% rr RTD102 %} and a `tools` table (we'll use -Python {% rr RTD103 %}, several languages are supported here). +modern Ubuntu should be fine) {rr}`RTD102` and a `tools` table (we'll use +Python {rr}`RTD103`, several languages are supported here). Finally, we have a `commands` table which describes how to install our dependencies and build the documentation into the ReadTheDocs output directory. @@ -560,8 +546,8 @@ dependencies and build the documentation into the ReadTheDocs output directory. Add a session to your `noxfile.py` to generate docs: -{% tabs %} {% tab sphinx Sphinx %} - +::::{tab-set} +:::{tab-item} Sphinx [diátaxis]: https://diataxis.fr/ @@ -795,5 +778,3 @@ show `mkdocs-gallery` in action. [sphinx-autodoc2]: https://sphinx-autodoc2.readthedocs.io/ [`sphinx.ext.napoleon`]: https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html - - diff --git a/docs/pages/guides/gha_basic.md b/docs/pages/guides/gha_basic.md index a161cd96..95baa036 100644 --- a/docs/pages/guides/gha_basic.md +++ b/docs/pages/guides/gha_basic.md @@ -1,17 +1,11 @@ --- -layout: page title: "GHA: GitHub Actions intro" -permalink: /guides/gha-basic/ -nav_order: 10 -parent: Topical Guides -custom_title: GitHub Actions introduction +short_title: GitHub Actions introduction --- -{% include toc.html %} - # GitHub Actions: Intro -{% rr GH100 %} The recommended CI for scientific Python projects is GitHub +{rr}`GH100` The recommended CI for scientific Python projects is GitHub Actions (GHA), although its predecessor Azure is also in heavy usage, and other popular services (Travis, Appveyor, and Circle CI) may be found in a few packages. GHA is preferred due to the flexible, extensible design and the tight @@ -47,7 +41,7 @@ on: jobs: ``` -This gives the workflow a nice name {% rr GH101 %}, and defines the conditions +This gives the workflow a nice name {rr}`GH101`, and defines the conditions under which it runs. This will run on all pull requests, or pushes to main. If you use a develop branch, you probably will want to include that. You can also specify specific branches for pull requests instead of running on all PRs (will @@ -59,8 +53,8 @@ If you use [prek][] or [pre-commit][] in CI, you can run it directly in GitHub Actions. Prek is a faster Rust rewrite of pre-commit that supports most real world usage and supports the same configuration and hooks. -{% tabs runner %} {% tab prek Prek %} - +::::{tab-set} +:::{tab-item} Prek Prek can run using the official action: ```yaml @@ -71,9 +65,8 @@ lint: - uses: actions/checkout@v6 - uses: j178/prek-action@v2 ``` - -{% endtab %} {% tab pre-commit Pre-commit %} - +::: +:::{tab-item} Pre-commit Pre-commit can run using the official action: ```yaml @@ -87,8 +80,8 @@ lint: python-version: "3.x" - uses: pre-commit/action@v3.0.1 ``` - -{% endtab %} {% endtabs %} +::: +:::: If you do use [pre-commit.ci](https://pre-commit.ci), but you need this job to run a manual check, like check-manifest, then you can keep it but just use @@ -169,7 +162,7 @@ static. And old versioned images are decommissioned. ## Updating -{% rr GH200 %} {% rr GH210 %} If you use non-default actions in your repository +{rr}`GH200` {rr}`GH210` If you use non-default actions in your repository (you will see some in the following pages), then it's a good idea to keep them up to date. GitHub provided a way to do this with dependabot. Just add the following file as `.github/dependabot.yml`: @@ -192,8 +185,8 @@ This will check to see if there are updates to the action weekly, and will make a PR if there are updates, including the changelog and commit summary in the PR. If you select a name like `v1`, this should only look for updates of the same form (since April 2022) - there is no need to restrict updates for "moving tag" -updates anymore {% rr GH211 %}. You can also use SHA's and dependabot will -respect that too. And `groups` will combine actions updates {% rr GH212 %}, +updates anymore {rr}`GH211`. You can also use SHA's and dependabot will +respect that too. And `groups` will combine actions updates {rr}`GH212`, which is both cleaner and sometimes required for dependent actions, like `upload-artifact`/`download-artifact`. @@ -344,7 +337,7 @@ These are some things you might need. ### Cancel existing runs -{% rr GH102 %} If you add the following, you can ensure only one run per +{rr}`GH102` If you add the following, you can ensure only one run per PR/branch happens at a time, cancelling the old run when a new one starts: {% raw %} @@ -468,15 +461,15 @@ You use the `python-path` output from `setup-python` to get the Python you activated. You use `github.action_path` to get the path to the checked-out action. -{: .highlight } +:::{tip} +Examples of custom composite actions include: -> Examples of custom composite actions include: -> -> - [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel/blob/main/action.yml) -> - [wntrblm/nox](https://github.com/wntrblm/nox/blob/main/action.yml) -> - [scientific-python/repo-review](https://github.com/scientific-python/repo-review/blob/main/action.yml) -> - [scientific-python/cookie](https://github.com/scientific-python/cookie/blob/main/action.yml) -> (This repo) +- [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel/blob/main/action.yml) +- [wntrblm/nox](https://github.com/wntrblm/nox/blob/main/action.yml) +- [scientific-python/repo-review](https://github.com/scientific-python/repo-review/blob/main/action.yml) +- [scientific-python/cookie](https://github.com/scientific-python/cookie/blob/main/action.yml) + (This repo) +::: ### Reusable workflows @@ -588,13 +581,12 @@ Everything else in the job is about getting the output from the step `changed-tests-files` to `tests-changes`, then from there into the reusable workflow output as `run-tests`. -{: .note } - +:::{note} Someone probably could write an action (maybe even an composite action using either `gh` or `shell: python`) that could directly report changes true/false instead of a file list, saving the two step process and greatly simplifying this. - +::: If you have more situations, you just repeat these two steps with different `id`s and inputs. @@ -650,13 +642,13 @@ if the "tests" job is being skipped into `allowed-skips`. {% endraw %} -{: .highlight } +:::{tip} +Some examples of repos using this method are: -> Some examples of repos using this method are: -> -> - [pypa/build](https://github.com/pypa/build/tree/main/.github/workflows) -> - [scientific-python/cookie](https://github.com/scientific-python/cookie/tree/main/.github/workflows) -> (this repo) +- [pypa/build](https://github.com/pypa/build/tree/main/.github/workflows) +- [scientific-python/cookie](https://github.com/scientific-python/cookie/tree/main/.github/workflows) + (this repo) +::: ### GitHub pages @@ -679,7 +671,7 @@ permissions: id-token: write ``` -{% rr GH103 %} You probably only want one deployment at a time, so you can use: +{rr}`GH103` You probably only want one deployment at a time, so you can use: ```yaml concurrency: @@ -739,14 +731,14 @@ The deploy-pages job gives a `page_url`, which is the same as `base_url` on the configure step, and can be set in the `environment`. If you want to do everything in one job, you only need one of these. -{: .highlight } +:::{tip} +See the +[official starter workflows](https://github.com/actions/starter-workflows/tree/main/pages) +for examples. Some other examples include: -> See the -> [official starter workflows](https://github.com/actions/starter-workflows/tree/main/pages) -> for examples. Some other examples include: -> -> - [CLIUtils.github.io/CLI11](https://github.com/CLIUtils/CLI11/blob/main/.github/workflows/docs.yml) -> - [iris-hep.org](https://github.com/iris-hep/iris-hep.github.io/blob/master/.github/workflows/deploy.yml) +- [CLIUtils.github.io/CLI11](https://github.com/CLIUtils/CLI11/blob/main/.github/workflows/docs.yml) +- [iris-hep.org](https://github.com/iris-hep/iris-hep.github.io/blob/master/.github/workflows/deploy.yml) +::: ### Changelog generation @@ -777,5 +769,3 @@ changelog: [gh-changelog]: https://docs.github.com/en/repositories/releasing-projects - - diff --git a/docs/pages/guides/gha_pure.md b/docs/pages/guides/gha_pure.md index 37d72572..55a084e8 100644 --- a/docs/pages/guides/gha_pure.md +++ b/docs/pages/guides/gha_pure.md @@ -1,14 +1,8 @@ --- -layout: page title: "GHA: Pure Python wheels" -permalink: /guides/gha-pure/ -nav_order: 11 -parent: Topical Guides -custom_title: GitHub Actions for pure Python wheels +short_title: GitHub Actions for pure Python wheels --- -{% include toc.html %} - # GitHub Actions: Pure Python wheels We will cover binary wheels [on the next page][], but if you do not have a @@ -17,23 +11,23 @@ procedure to make a "built" wheel is simple. At the end of this page, there is a recipe that can often be used exactly for pure Python wheels (if the previous recommendations were followed). -{: .note } +:::{note} +Why make a wheel when there is nothing to compile? There are a multitude of +reasons that a wheel is better than only providing an sdist: -> Why make a wheel when there is nothing to compile? There are a multitude of -> reasons that a wheel is better than only providing an sdist: -> -> - Wheels do not run `setup.py`, but simply install files into locations -> - Lower install requirements - users don't need your setup tools -> - Faster installs -> - Safer installs - no arbitrary code execution -> - Highly consistent installs -> - Wheels pre-compile bytecode when they install -> - Initial import is not slower than subsequent import -> - Less chance of a permission issue -> - You can look in the `.whl` (it's a `.zip`, really) and see where everything -> is going to go +- Wheels do not run `setup.py`, but simply install files into locations + - Lower install requirements - users don't need your setup tools + - Faster installs + - Safer installs - no arbitrary code execution + - Highly consistent installs +- Wheels pre-compile bytecode when they install + - Initial import is not slower than subsequent import + - Less chance of a permission issue +- You can look in the `.whl` (it's a `.zip`, really) and see where everything + is going to go +::: -[on the next page]: {% link pages/guides/gha_wheels.md %} +[on the next page]: pages/guides/gha_wheels ## Job setup @@ -105,37 +99,35 @@ GitHub Actions (in fact, they use it to setup other applications). We upload the artifact just to make it available via the GitHub PR/Checks API. You can download a file to test locally if you want without making a release. -{: .warning } - -> As of `upload-artifact@v4`, the artifact name must be unique. Extending an -> existing artifact is no longer supported. +:::{warning} +As of `upload-artifact@v4`, the artifact name must be unique. Extending an +existing artifact is no longer supported. +::: We also add an optional check using twine for the metadata (it will be tested later in the upload action for the release job, as well). -{: .highlight-title } - -> All-in-one action -> -> There is an -> [all-in-one action](https://github.com/hynek/build-and-inspect-python-package) -> that does all the work for you for a pure Python package, including extra -> pre-upload checks & nice GitHub summaries. -> -> ```yaml -> steps: -> - uses: actions/checkout@v6 -> - uses: hynek/build-and-inspect-python-package@v2 -> ``` -> -> The artifact it produces is named `Packages`, so that's what you need to use -> later to publish. This will be used instead of the manual steps below. +:::{tip} All-in-one action +There is an +[all-in-one action](https://github.com/hynek/build-and-inspect-python-package) +that does all the work for you for a pure Python package, including extra +pre-upload checks & nice GitHub summaries. -And then, you need a release job. Trusted Publishing is more secure and -recommended {% rr GH105 %}: +```yaml +steps: + - uses: actions/checkout@v6 + - uses: hynek/build-and-inspect-python-package@v2 +``` -{% tabs %} {% tab oidc Trusted Publishing (recommended) %} +The artifact it produces is named `Packages`, so that's what you need to use +later to publish. This will be used instead of the manual steps below. +::: +And then, you need a release job. Trusted Publishing is more secure and +recommended {rr}`GH105`: + +::::{tab-set} +:::{tab-item} Trusted Publishing (recommended) {% raw %} ```yaml @@ -172,9 +164,8 @@ accept your initial package publish. We are also generating artifact attestations, which can allow users to verify that the artifacts were built on your actions. - -{% endtab %} {% tab token Token %} - +::: +:::{tab-item} Token {% raw %} ```yaml @@ -199,18 +190,17 @@ If you cannot use Trusted Publishing, this publishes to PyPI with a token. You'll need to go to PyPI, generate a token for your user, and put it into `pypi_password` on your repo's secrets page. Once you have a project, you should delete your user-scoped token and generate a new project-scoped token. +::: +:::: -{% endtab %} {% endtabs %} - -{% details Complete recipe %} - +:::{dropdown} Complete recipe This can be used on almost any package with a standard `.github/workflows/cd.yml` recipe. This works because `pyproject.toml` describes exactly how to build your package, hence all packages build exactly via the same interface: -{% tabbodies %} {% tab oidc Trusted Publishing (recommended) %} - +::::{tab-set} +:::{tab-item} Trusted Publishing (recommended) {% raw %} ```yaml @@ -261,9 +251,8 @@ jobs: ``` {% endraw %} - -{% endtab %} {% tab token Token %} - +::: +:::{tab-item} Token {% raw %} ```yaml @@ -311,10 +300,9 @@ If you cannot use Trusted Publishing, this publishes to PyPI with a token. You'll need to go to PyPI, generate a token for your user, and put it into `pypi_password` on your repo's secrets page. Once you have a project, you should delete your user-scoped token and generate a new project-scoped token. - -{% endtab %} {% endtabbodies %} - -{% enddetails %} +::: +:::: +::: @@ -325,5 +313,3 @@ delete your user-scoped token and generate a new project-scoped token. - - diff --git a/docs/pages/guides/gha_wheels.md b/docs/pages/guides/gha_wheels.md index 6d4e6582..8b49e15d 100644 --- a/docs/pages/guides/gha_wheels.md +++ b/docs/pages/guides/gha_wheels.md @@ -1,14 +1,8 @@ --- -layout: page title: "GHA: Binary wheels" -permalink: /guides/gha-wheels/ -nav_order: 12 -parent: Topical Guides -custom_title: GitHub Actions for Binary Wheels +short_title: GitHub Actions for Binary Wheels --- -{% include toc.html %} - # GitHub Actions: Binary wheels Building binary wheels is a bit more involved, but can still be done effectively @@ -47,7 +41,6 @@ Finally, if you change the workflow itself in a PR, then rebuild the wheels too. [workflow_dispatch]: https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/ ### Useful suggestion: -{: .no_toc } Since these variables will be used by all jobs, you could make them available in @@ -168,10 +161,10 @@ You can skip specifying the `build[uv]` build-frontend option and pre-installing ## Publishing -Trusted Publishing is more secure and recommended {% rr GH105 %}: - -{% tabs %} {% tab oidc Trusted Publishing (recommended) %} +Trusted Publishing is more secure and recommended {rr}`GH105`: +::::{tab-set} +:::{tab-item} Trusted Publishing (recommended) {% raw %} ```yaml @@ -210,9 +203,8 @@ accept your initial package publish. We are also generating artifact attestations, which can allow users to verify that the artifacts were built on your actions. - -{% endtab %} {% tab token Token %} - +::: +:::{tab-item} Token {% raw %} ```yaml @@ -238,8 +230,8 @@ If you cannot use Trusted Publishing, this publishes to PyPI with a token. You'll need to go to PyPI, generate a token for your user, and put it into `pypi_password` on your repo's secrets page. Once you have a project, you should delete your user-scoped token and generate a new project-scoped token. - -{% endtab %} {% endtabs %} +::: +:::: If you have multiple jobs, you will want to collect your artifacts from above. If you only have one job, you can combine this into a single job like we did for @@ -248,16 +240,14 @@ multiple places, you can set `skip_existing` (but generally it's better to not try to upload the same file from two places - you can trick Travis into avoiding the sdist, for example). -{: .note-title } - -> Other architectures -> -> GitHub Actions supports ARM on Linux and Windows as well. On Travis, -> `cibuildwheel` even has the ability to create rarer architectures like PowerPC -> builds natively. IBM Z builds are also available but in beta. However, due to -> Travis CI's recent dramatic reduction on open source support, emulating these -> architectures on GHA or Azure is probably better. Maybe look into Cirrus CI, -> which has some harder-to-find architectures. +:::{note} Other architectures +GitHub Actions supports ARM on Linux and Windows as well. On Travis, +`cibuildwheel` even has the ability to create rarer architectures like PowerPC +builds natively. IBM Z builds are also available but in beta. However, due to +Travis CI's recent dramatic reduction on open source support, emulating these +architectures on GHA or Azure is probably better. Maybe look into Cirrus CI, +which has some harder-to-find architectures. +::: @@ -268,5 +258,3 @@ the sdist, for example). [pypi trusted publisher docs]: https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/ - - diff --git a/docs/pages/guides/index.md b/docs/pages/guides/index.md index 33e935ff..a1e10c8b 100644 --- a/docs/pages/guides/index.md +++ b/docs/pages/guides/index.md @@ -1,9 +1,5 @@ --- -layout: page title: Topical Guides -permalink: /guides/ -nav_order: 2 -has_children: true --- # Topical Guides @@ -28,40 +24,36 @@ and one for [compiled extensions][gha_wheels]. You can read about setting up good tests on the [pytest page][pytest], with [coverage][]. There's also a page on setting up [docs][], as well. -{: .highlight-title } +:::{tip} New project template +Once you have completed the guidelines, there is a +[copier][]/[cookiecutter][]/[cruft][] project, [scientific-python/cookie][], +that implements these guidelines and lets you setup a new package from a +template in less than 60 seconds! Nine build backends including compiled +backends, generation tested in Nox, and kept in-sync with the guide. +::: -> New project template -> -> Once you have completed the guidelines, there is a -> [copier][]/[cookiecutter][]/[cruft][] project, [scientific-python/cookie][], -> that implements these guidelines and lets you setup a new package from a -> template in less than 60 seconds! Nine build backends including compiled -> backends, generation tested in Nox, and kept in-sync with the guide. - -{: .important-title } - -> Checking an existing project -> -> We provide [sp-repo-review][], a set of [repo-review][] checks for comparing -> your repository with the guidelines, runnable [right in the guide][] via -> WebAssembly! All checks point to a linked badge in the guide. +:::{important} Checking an existing project +We provide [sp-repo-review][], a set of [repo-review][] checks for comparing +your repository with the guidelines, runnable [right in the guide][] via +WebAssembly! All checks point to a linked badge in the guide. +::: -[tutorials]: {% link pages/tutorials/index.md %} -[style]: {% link pages/guides/style.md %} -[mypy]: {% link pages/guides/mypy.md %} -[docs]: {% link pages/guides/docs.md %} -[simple packaging]: {% link pages/guides/packaging_simple.md %} -[compiled packaging]: {% link pages/guides/packaging_compiled.md %} -[classic packaging]: {% link pages/guides/packaging_classic.md %} -[coverage]: {% link pages/guides/coverage.md %} -[gha_basic]: {% link pages/guides/gha_basic.md %} -[gha_pure]: {% link pages/guides/gha_pure.md %} -[gha_wheels]: {% link pages/guides/gha_wheels.md %} -[pytest]: {% link pages/guides/pytest.md %} -[task runners]: {% link pages/guides/tasks.md %} -[right in the guide]: {% link pages/guides/repo_review.md %} +[tutorials]: pages/tutorials/index +[style]: pages/guides/style +[mypy]: pages/guides/mypy +[docs]: pages/guides/docs +[simple packaging]: pages/guides/packaging_simple +[compiled packaging]: pages/guides/packaging_compiled +[classic packaging]: pages/guides/packaging_classic +[coverage]: pages/guides/coverage +[gha_basic]: pages/guides/gha_basic +[gha_pure]: pages/guides/gha_pure +[gha_wheels]: pages/guides/gha_wheels +[pytest]: pages/guides/pytest +[task runners]: pages/guides/tasks +[right in the guide]: pages/guides/repo_review [cookiecutter]: https://cookiecutter.readthedocs.io [copier]: https://copier.readthedocs.io diff --git a/docs/pages/guides/mypy.md b/docs/pages/guides/mypy.md index 7adb3543..17086f71 100644 --- a/docs/pages/guides/mypy.md +++ b/docs/pages/guides/mypy.md @@ -1,13 +1,7 @@ --- -layout: page title: "Static type checking" -permalink: /guides/mypy/ -nav_order: 9 -parent: Topical Guides --- -{% include toc.html %} - # Static type checking ## Basics @@ -80,7 +74,7 @@ everything. Try to turn on as much as possible, and increase it until you can run with full `strict` checking. See the [style page][] for configuration suggestions. -[style page]: {% link pages/guides/style.md %} +[style page]: pages/guides/style For a library to support typing, it has to a) add types using any of the three methods, and b) add a `py.typed` empty file to indicate that it's okay to look diff --git a/docs/pages/guides/packaging_classic.md b/docs/pages/guides/packaging_classic.md index cbd613fa..d32c258e 100644 --- a/docs/pages/guides/packaging_classic.md +++ b/docs/pages/guides/packaging_classic.md @@ -1,13 +1,7 @@ --- -layout: page title: Classic packaging -permalink: /guides/packaging-classic/ -nav_order: 7 -parent: Topical Guides --- -{% include toc.html %} - # Classic packaging The libraries in the scientific Python ecosytem have a variety of different @@ -18,29 +12,29 @@ outlined as well. There are several popular packaging systems. This guide covers the old configuration style for [Setuptools][]. Unless you really need it, you should be using the modern style described in [Simple -Packaging]({% link pages/guides/packaging_simple.md %}). The modern style is +Packaging](pages/guides/packaging_simple). The modern style is guided by Python Enhancement Proposals (PEPs), and is more stable than the setuptools-specific mechanisms that evolve over the years. This page is kept to help users with legacy code (and hopefully upgrade it). -{: .note } - -> Raw source lives in git and has a `pyproject.toml` and/or a `setup.py`. You -> _can_ install directly from git via pip, but normally users install from -> distributions hosted on PyPI. There are three options: **A)** A source -> package, called an SDist and has a name that ends in `.tar.gz`. This is a copy -> of the GitHub repository, stripped of a few specifics like CI files, and -> possibly with submodules included (if there are any). **B)** A pure python -> wheel, which ends in `.whl`; this is only possible if there are no compiled -> extensions in the library. This does _not_ contain a setup.py, but rather a -> `PKG_INFO` file that is rendered from setup.py (or from another build system). -> **C)** If not pure Python, a collection of wheels for every binary platform, -> generally one per supported Python version and OS as well. -> -> Developer requirements (users of A or git) are generally higher than the -> requirements to use B or C. Poetry and optionally flit create SDists that -> include a `setup.py`, and all alternate packing systems produce "normal" -> wheels. +:::{note} +Raw source lives in git and has a `pyproject.toml` and/or a `setup.py`. You +_can_ install directly from git via pip, but normally users install from +distributions hosted on PyPI. There are three options: **A)** A source +package, called an SDist and has a name that ends in `.tar.gz`. This is a copy +of the GitHub repository, stripped of a few specifics like CI files, and +possibly with submodules included (if there are any). **B)** A pure python +wheel, which ends in `.whl`; this is only possible if there are no compiled +extensions in the library. This does _not_ contain a setup.py, but rather a +`PKG_INFO` file that is rendered from setup.py (or from another build system). +**C)** If not pure Python, a collection of wheels for every binary platform, +generally one per supported Python version and OS as well. + +Developer requirements (users of A or git) are generally higher than the +requirements to use B or C. Poetry and optionally flit create SDists that +include a `setup.py`, and all alternate packing systems produce "normal" +wheels. +::: ## Package structure (medium priority) @@ -85,7 +79,7 @@ these "[hypermodern][]" packaging tools is growing in scientific Python packages. All tools build the same wheels (and they often build setuptools compliant SDists, as well). -{% rr PP003 %} Note that `"wheel"` is never required; it was injected +{rr}`PP003` Note that `"wheel"` is never required; it was injected automatically by setuptools in older versions, and is no longer used at all. ### Special additions: NumPy @@ -108,18 +102,18 @@ developers that tracks the Otherwise, you would have to list the earliest version of NumPy that had support for each Python version here. -{: .note } +:::{note} +Modern versions of NumPy (1.25+) allow you to target older versions when +building, which is _highly_ recommended, and this will become required in +NumPy 2.0. Now you add: + +```cpp +#define NPY_TARGET_VERSION NPY_1_22_API_VERSION +``` -> Modern versions of NumPy (1.25+) allow you to target older versions when -> building, which is _highly_ recommended, and this will become required in -> NumPy 2.0. Now you add: -> -> ```cpp -> #define NPY_TARGET_VERSION NPY_1_22_API_VERSION -> ``` -> -> (Where that number is whatever version you support as a minimum) then make -> sure you build with NumPy 1.25+ (or 2.0+ when it comes out). +(Where that number is whatever version you support as a minimum) then make +sure you build with NumPy 1.25+ (or 2.0+ when it comes out). +::: ## Versioning (medium/high priority) @@ -206,11 +200,11 @@ env: If you fill in the override version setting when triggering a manual workflow run, that version will be forced, otherwise, it works as normal. -{: .note } - -> Make sure you have a good gitignore, probably starting from -> [GitHub's Python one](https://github.com/github/gitignore/blob/main/Python.gitignore) -> or using a [generator site](https://www.toptal.com/developers/gitignore). +:::{note} +Make sure you have a good gitignore, probably starting from +[GitHub's Python one](https://github.com/github/gitignore/blob/main/Python.gitignore) +or using a [generator site](https://www.toptal.com/developers/gitignore). +::: You should also add these two files: @@ -324,7 +318,7 @@ where = src # extern ``` -{% rr SCFG001 %} Note that all keys use underscores; using a dash will cause +{rr}`SCFG001` Note that all keys use underscores; using a dash will cause warnings and eventually failures. And, a possible `setup.py`; though in recent versions of pip, there no longer is diff --git a/docs/pages/guides/packaging_compiled.md b/docs/pages/guides/packaging_compiled.md index 049df61b..02616b3b 100644 --- a/docs/pages/guides/packaging_compiled.md +++ b/docs/pages/guides/packaging_compiled.md @@ -1,13 +1,7 @@ --- -layout: page title: Compiled packaging -permalink: /guides/packaging-compiled/ -nav_order: 6 -parent: Topical Guides --- -{% include toc.html %} - -{: .highlight-title } - -> Quick start -> -> Once you've done this at least once, feel free to use -> [our cookiecutter/copier template](https://github.com/scientific-python/cookie), -> or `uv init` to get started quickly on new packages! +:::{tip} Quick start +Once you've done this at least once, feel free to use +[our cookiecutter/copier template](https://github.com/scientific-python/cookie), +or `uv init` to get started quickly on new packages! +::: # Packaging Compiled Projects @@ -50,37 +42,34 @@ The most exciting developments have been new native build backends: - [enscons][]: Builds C/C++ using SCONs. (Aging now, but this was the first native backend!) -{: .note } - +:::{note} You should be familiar with [packing a pure Python -project]({% link pages/guides/packaging_compiled.md %}) - the metadata +project](pages/guides/packaging_compiled) - the metadata configuration is the same. - +::: There are also classic setuptools plugins: - [scikit-build][]: Builds C/C++/Fortran using CMake. - [setuptools-rust][]: Builds Rust using Cargo. -{: .highlight-title } - -> Selecting a backend -> -> Selecting a backend: If you are using Rust, use maturin. If you are using -> CUDA, use scikit-build-core. If you are using a classic language (C, C++, -> Fortran), then you can use either scikit-build-core or meson-python, depending -> on whether you prefer writing CMake or Meson. Meson is a lot more opinionated; -> it requires you use version control, it requires a README.md and LICENSE file. -> It requires your compiler be properly set up. Etc. While CMake can be as -> elegant as Meson, there are a lot of historical examples of poorly written -> CMake. +:::{tip} Selecting a backend +Selecting a backend: If you are using Rust, use maturin. If you are using +CUDA, use scikit-build-core. If you are using a classic language (C, C++, +Fortran), then you can use either scikit-build-core or meson-python, depending +on whether you prefer writing CMake or Meson. Meson is a lot more opinionated; +it requires you use version control, it requires a README.md and LICENSE file. +It requires your compiler be properly set up. Etc. While CMake can be as +elegant as Meson, there are a lot of historical examples of poorly written +CMake. +::: ## pyproject.toml: build-system -{% rr PY001 %} Packages must have a `pyproject.toml` file {% rr PP001 %} that +{rr}`PY001` Packages must have a `pyproject.toml` file {rr}`PP001` that selects the backend: -{% tabs %} {% tab skbc Scikit-build-core %} - +::::{tab-set} +:::{tab-item} Scikit-build-core - -{% endtab %} {% tab meson Meson-python %} - +::: +:::{tab-item} Meson-python - -{% endtab %} {% tab maturin Maturin %} - +::: +:::{tab-item} Maturin +::: +:::: -{% endtab %} {% endtabs %} - -{% include pyproject.md %} +```{include} ../../_partials/pyproject.md +``` ## Tool section in pyproject.toml These tools all read the project table. They also have extra configuration options in `tool.*` settings. -{% tabs %} {% tab skbc Scikit-build-core %} - +::::{tab-set} +:::{tab-item} Scikit-build-core These options are not required, but can improve your experience. - -{% endtab %} {% tab meson Meson-python %} - +::: +:::{tab-item} Meson-python No `tool.meson-python` configuration required for this example. - -{% endtab %} {% tab maturin Maturin %} - +::: +:::{tab-item} Maturin - -{% endtab %} {% endtabs %} +::: +:::: ## Example compiled file @@ -286,8 +270,8 @@ allows you to easily provide both Python files and compiled extensions, and keeping the details of your compiled extension private. You can select whatever name you wish, though, or even make your compiled extension a top level module. -{% tabs %} {% tab skbc Scikit-build-core %} - +::::{tab-set} +:::{tab-item} Scikit-build-core Example `src/main.cpp` file: - -{% endtab %} {% tab meson Meson-python %} - +::: +:::{tab-item} Meson-python Example `src/main.cpp` file: - -{% endtab %} {% tab maturin Maturin %} - +::: +:::{tab-item} Maturin Example `src/lib.rs` file: - -{% endtab %} {% endtabs %} +::: +:::: ## Package structure @@ -453,11 +435,10 @@ for those versions. If using pybind11, you don't need NumPy at build-time in the first place. -{: .important } - +:::{important} Python 3.13.4 is broken on Windows for compiling code - it always reports that it is free-threaded. 3.13.5 was rushed out to fix it. - +::: [scikit-build-core]: https://scikit-build-core.readthedocs.io @@ -469,8 +450,6 @@ it is free-threaded. 3.13.5 was rushed out to fix it. [scons]: https://scons.org/ [setuptools-rust]: https://setuptools-rust.readthedocs.io/en/latest/ [maturin]: https://www.maturin.rs -[gha_wheels]: {% link pages/guides/gha_wheels.md %} +[gha_wheels]: pages/guides/gha_wheels - - diff --git a/docs/pages/guides/packaging_simple.md b/docs/pages/guides/packaging_simple.md index b22b7766..3e4be541 100644 --- a/docs/pages/guides/packaging_simple.md +++ b/docs/pages/guides/packaging_simple.md @@ -1,20 +1,12 @@ --- -layout: page title: Simple packaging -permalink: /guides/packaging-simple/ -nav_order: 5 -parent: Topical Guides --- -{% include toc.html %} - -{: .highlight-title } - -> Quick start -> -> Once you've done this at least once, feel free to use -> [our cookiecutter/copier template](https://github.com/scientific-python/cookie), -> or `uv init` to get started quickly on new packages! +:::{tip} Quick start +Once you've done this at least once, feel free to use +[our cookiecutter/copier template](https://github.com/scientific-python/cookie), +or `uv init` to get started quickly on new packages! +::: # Simple packaging @@ -30,76 +22,69 @@ and compiled backends (see the next page). Also see the [Python packaging guide][], especially the [Python packaging tutorial][]. -{: .note-title } - -> Classic files -> -> These systems do not use or require `setup.py`, `setup.cfg`, or `MANIFEST.in`. -> Those are for setuptools. Unless you are using setuptools, of course, which -> still uses `MANIFEST.in`. You can convert the old files using -> `pipx run hatch new --init` or with -> [ini2toml](https://ini2toml.readthedocs.io/en/latest/). - -{: .important-title } - -> Selecting a backend -> -> Backends handle metadata the same way, so the choice comes down to how you -> specify what files go into an SDist and extra features, like getting a version -> from VCS. If you don't have an existing preference, hatchling is an excellent -> choice, balancing speed, configurability, and extendability. +:::{note} Classic files +These systems do not use or require `setup.py`, `setup.cfg`, or `MANIFEST.in`. +Those are for setuptools. Unless you are using setuptools, of course, which +still uses `MANIFEST.in`. You can convert the old files using +`pipx run hatch new --init` or with +[ini2toml](https://ini2toml.readthedocs.io/en/latest/). +::: + +:::{important} Selecting a backend +Backends handle metadata the same way, so the choice comes down to how you +specify what files go into an SDist and extra features, like getting a version +from VCS. If you don't have an existing preference, hatchling is an excellent +choice, balancing speed, configurability, and extendability. +::: ## pyproject.toml: build-system -{% rr PY001 %} Packages must have a `pyproject.toml` file {% rr PP001 %} that +{rr}`PY001` Packages must have a `pyproject.toml` file {rr}`PP001` that selects the backend: -{% tabs %} {% tab hatch Hatchling %} - +::::{tab-set} +:::{tab-item} Hatchling ```toml [build-system] requires = ["hatchling"] build-backend = "hatchling.build" ``` - -{% endtab %} {% tab uv uv_build %} - +::: +:::{tab-item} uv_build ```toml [build-system] requires = ["uv_build>=0.7.19"] build-backend = "uv_build" ``` - -{% endtab %} {% tab flit Flit-core %} - +::: +:::{tab-item} Flit-core ```toml [build-system] requires = ["flit_core>=3.12"] build-backend = "flit_core.buildapi" ``` - -{% endtab %} {% tab pdm PDM-backend %} - +::: +:::{tab-item} PDM-backend ```toml [build-system] requires = ["pdm-backend"] build-backend = "pdm.backend" ``` - -{% endtab %} {% tab setuptools Setuptools %} - +::: +:::{tab-item} Setuptools ```toml [build-system] requires = ["setuptools>=61.0"] build-backend = "setuptools.build_meta" ``` +::: +:::: -{% endtab %} {% endtabs %} - -{% include pyproject.md %} +```{include} ../../_partials/pyproject.md +``` For `requires-python`, you should specify the minimum you require, and you -should not put an upper cap on it {% rr PY004 %}, as this field is used to +should not put an upper cap on it {rr}`PY004`, as this field is used to back-solve for old package versions that pass this check, allowing you to safely drop Python versions. @@ -120,9 +105,9 @@ detection. If you don't match your package name and import name (which you should except for very special cases), you will likely need extra configuration here. -You should have a `README` {% rr PY002 %} and a `LICENSE` {% rr PY003 %} file. -You should have a `docs/` folder {% rr PY004 %}. You should have a `/tests` -folder {% rr PY005 %} (recommended) and/or a `src//tests` folder. +You should have a `README` {rr}`PY002` and a `LICENSE` {rr}`PY003` file. +You should have a `docs/` folder {rr}`PY004`. You should have a `/tests` +folder {rr}`PY005` (recommended) and/or a `src//tests` folder. ## Versioning @@ -140,8 +125,7 @@ dynamic = ["version"] Then you'll configure your backend to compute the version. -{% details Hatchling dynamic versioning %} - +:::{dropdown} Hatchling dynamic versioning You can tell hatchling to get the version from VCS. Add `hatch-vcs` to your `build-backend.requires`, then add the following configuration: @@ -178,8 +162,7 @@ And `.gitattributes` (or add this line if you are already using this file): This will allow git archives (including the ones generated from GitHub) to also support versioning. - -{% enddetails %} +::: ## Including/excluding files in the SDist @@ -194,23 +177,22 @@ This is tool specific. - [PDM info here](https://pdm-backend.fming.dev/build_config/#include-or-exclude-files). - Setuptools still uses `MANIFEST.in`. -{: .warning } - -> Flit will not use VCS (like git) to populate the SDist if you use standard -> tooling, even if it can do that using its own tooling. So make sure you list -> explicit include/exclude rules, and test the contents: -> -> ```bash -> # Show SDist contents -> tar -tvf dist/*.tar.gz -> # Show wheel contents -> unzip -l dist/*.whl -> ``` +:::{warning} +Flit will not use VCS (like git) to populate the SDist if you use standard +tooling, even if it can do that using its own tooling. So make sure you list +explicit include/exclude rules, and test the contents: -{: .note-title } +```bash +# Show SDist contents +tar -tvf dist/*.tar.gz +# Show wheel contents +unzip -l dist/*.whl +``` +::: -> Flit _requires_ `license.file` to be set in your `[project]` section to ensure -> it finds the license file. +:::{note} Flit _requires_ `license.file` to be set in your `[project]` section to ensure +it finds the license file. +::: @@ -226,5 +208,3 @@ This is tool specific. [python packaging tutorial]: https://packaging.python.org/tutorials/packaging-projects/ - - diff --git a/docs/pages/guides/pytest.md b/docs/pages/guides/pytest.md index 6c0612de..eb41dbc9 100644 --- a/docs/pages/guides/pytest.md +++ b/docs/pages/guides/pytest.md @@ -1,13 +1,7 @@ --- -layout: page title: "Testing with pytest" -permalink: /guides/pytest/ -nav_order: 2 -parent: Topical Guides --- -{% include toc.html %} - # Testing with pytest Tests are crucial to writing reliable software. A good test suite allows you to: @@ -29,17 +23,15 @@ in using pytest. The goals of writing good tests are: - Reporting: when things break, you should get good information about what broke. -{: .note-title } - -> What about other choices? -> -> The alternative library, `nose`, has been abandoned in favor of `pytest`, -> which can run nose-style tests. The standard library has a test suite as well, -> but it's extremely verbose and complex; and since "developers" run tests, your -> test requirements don't affect users. And `pytest` can run stdlib style -> testing too. So just use `pytest`. All major packages use it too, including -> `NumPy`. Most other choices, like [Hypothesis][], are related to `pytest` and -> just extend it. +:::{note} What about other choices? +The alternative library, `nose`, has been abandoned in favor of `pytest`, +which can run nose-style tests. The standard library has a test suite as well, +but it's extremely verbose and complex; and since "developers" run tests, your +test requirements don't affect users. And `pytest` can run stdlib style +testing too. So just use `pytest`. All major packages use it too, including +`NumPy`. Most other choices, like [Hypothesis][], are related to `pytest` and +just extend it. +::: ### Basic test structure @@ -66,13 +58,13 @@ This looks simple, but it is doing several things: ### Configuring pytest pytest supports configuration in `pytest.ini`, `setup.cfg`, or, since version 6, -`pyproject.toml` {% rr PP301 %}, or, since version 9, `pytest.toml` or +`pyproject.toml` {rr}`PP301`, or, since version 9, `pytest.toml` or `.pytest.toml`. Remember, pytest is a developer requirement, not a user one, so always require 6+ (or 9+) and use `pyproject.toml` or the pytest TOML ones. This is an example configuration: -{% tabs %} {% tab conf-modern Pytest 9+ %} - +::::{tab-set} +:::{tab-item} Pytest 9+ ```toml [tool.pytest] minversion = "9.0" @@ -84,9 +76,8 @@ testpaths = [ "tests", ] ``` - -{% endtab %} {% tab conf-classic Pytest 6+ %} - +::: +:::{tab-item} Pytest 6+ ```toml [tool.pytest.ini_options] minversion = "6.0" @@ -98,25 +89,25 @@ testpaths = [ "tests", ] ``` +::: +:::: -{% endtab %} {% endtabs %} - -{% rr PP302 %} The `minversion` will print a nicer error if your `pytest` is too +{rr}`PP302` The `minversion` will print a nicer error if your `pytest` is too old (though, ironically, it won't read this if the version is too old, so setting "6" or less in `pyproject.toml` is rather pointless, similarly for 9 if using the new config location). The `addopts` setting will add whatever you put -there to the command line when you run; {% rr PP308 %} `-ra` will print a +there to the command line when you run; {rr}`PP308` `-ra` will print a summary "r"eport of "a"ll results, which gives you a quick way to review what tests failed and were skipped, and why. `--showlocals` will print locals in tracebacks - depending on your tests, you might or might not like this one. -{% rr PP307 %} `--strict-markers` will make sure you don't try to use an -unspecified fixture. {% rr PP306 %} And `--strict-config` will error if you make -a mistake in your config. {% rr PP305 %} `xfail_strict` will change the default +{rr}`PP307` `--strict-markers` will make sure you don't try to use an +unspecified fixture. {rr}`PP306` And `--strict-config` will error if you make +a mistake in your config. {rr}`PP305` `xfail_strict` will change the default for `xfail` to fail the tests if it doesn't fail - you can still override -locally in a specific xfail for a flaky failure. {% rr PP309 %} +locally in a specific xfail for a flaky failure. {rr}`PP309` `filter_warnings` will cause all warnings to be errors (you can add allowed -warnings here too, see below). {% rr PP304 %} `log_level` will report `INFO` and -above log messages on a failure. {% rr PP303 %} Finally, `testpaths` will limit +warnings here too, see below). {rr}`PP304` `log_level` will report `INFO` and +above log messages on a failure. {rr}`PP303` Finally, `testpaths` will limit `pytest` to just looking in the folders given - useful if it tries to pick up things that are not tests from other directories. [See the docs](https://docs.pytest.org/en/stable/customize.html) for more @@ -392,5 +383,3 @@ redirects to the standard library [unittest.mock][]. [pytest]: https://docs.pytest.org [pytest-mock]: https://pypi.org/project/pytest-mock/ [unittest.mock]: https://docs.python.org/3/library/unittest.mock.html - - diff --git a/docs/pages/guides/repo_review.md b/docs/pages/guides/repo_review.md index b68913c7..1f3d09f7 100644 --- a/docs/pages/guides/repo_review.md +++ b/docs/pages/guides/repo_review.md @@ -1,9 +1,8 @@ --- -layout: page title: Repo-Review -permalink: /guides/repo-review/ -nav_order: 110 -interactive_repo_review: true +html_head_extra: | + + --- # Repo-Review @@ -25,6 +24,32 @@ pipx run 'sp-repo-review[cli]' --- -{% include interactive_repo_review.html %} +```{raw} html +
Loading (requires javascript and WebAssembly)...
+ + +``` [Open in new page](https://scientific-python.github.io/repo-review/). diff --git a/docs/pages/guides/style.md b/docs/pages/guides/style.md index 9f4878e7..8b137891 100644 --- a/docs/pages/guides/style.md +++ b/docs/pages/guides/style.md @@ -1,1053 +1 @@ ---- -layout: page -title: Style & static checks -permalink: /guides/style/ -nav_order: 8 -parent: Topical Guides -custom_title: Style and static checks ---- -{% include toc.html %} - -# Style and static checks - -## Pre-commit - -{% rr PY006 %} Scientific Python projects often use [pre-commit][] (or [prek][]) -to check code style. The original, `pre-commit`, has support for more languages, -but `prek` is a faster Rust rewrite that supports most real world usage. - -{% tabs runner %} {% tab prek Prek %} - -Prek can be installed through `brew` (macOS) or `pipx/uv` (anywhere). There are -two modes to use it locally; you can check manually with `prek run` (changes -only) or `prek run -a` (all). You can omit the `run`, as well; such as -`prek -a`. You can also run `prek install` to add checks as a git pre-commit -hook. - -{% endtab %} {% tab pre-commit Pre-commit %} - -Pre-commit can be installed through `brew` (macOS) or `pipx/uv` (anywhere). -There are two modes to use it locally; you can check manually with -`pre-commit run` (changes only) or `pre-commit run -a` (all). You can also run -`pre-commit install` to add checks as a git pre-commit hook (which is where it -gets its name). - -Pre-commit's setup is much slower than prek, but you can install `pre-commit-uv` -with pre-commit to speed up the setup time quite a bit. - -{% endtab %} {% endtabs %} - -Local runs (with `-a`) are the standard way to use it. That will run all the -checks in optimized, isolated environments. After the first run, it's quite fast -(either pre-commit or prek). - -It's worth trying the install command, even if you've tried and failed to set up -a custom pre-commit hook before; it's quite elegant and does not add or commit -the changes, it just makes the changes and allows you to check and add them. You -can always override the hook with `-n`. - -{% rr PC100 %} Here is a minimal `.pre-commit-config.yaml` file with some handy -options: - -```yaml -repos: - - repo: https://github.com/pre-commit/pre-commit-hooks - rev: "v6.0.0" - hooks: - - id: check-added-large-files - - id: check-case-conflict - - id: check-merge-conflict - - id: check-symlinks - - id: check-yaml - - id: debug-statements - - id: end-of-file-fixer - - id: mixed-line-ending - - id: name-tests-test - args: ["--pytest-test-first"] - - id: requirements-txt-fixer - - id: trailing-whitespace -``` - -**Helpful tip**: Pre-commit runs top-to-bottom, so put checks that modify -content (like the several of the pre-commit-hooks above, or Black) above checks -that might be more likely to pass after the modification (like flake8). - -**Keeping pinned versions fresh**: You can use `pre-commit autoupdate` to move -your tagged versions forward to the latest tags! Due to the design of -pre-commit's caching system, these _must_ point at fixed tags, never put a -branch here. - -**Checking in CI**: You can have this checked and often automatically corrected -for you using [pre-commit.ci](https://pre-commit.ci). It will even update your -`rev:` versions every week or so if your checks update! - -To use, just go to [pre-commit.ci](https://pre-commit.ci), click "Log in with -GitHub", click "Add an Installation" if adding for the first time for an org or -user, or "Manage repos on GitHub" for an existing installation, then add your -repository from the list in GitHub's interface. - -Now there will be a new check, and pre-commit.ci will commit changes if the -pre-commit check made any changes. Note that there are a couple of missing -features: Docker based checks will not work (pre-commit.ci already runs in -docker), you cannot enable a `--manual` flag, so extra checks will not run, and -jobs should not download packages (use `additional-dependencies:` to add what -you need). - -{% rr PC901 %} {% rr PC902 %} {% rr PC903 %} You can customize the pre-commit -message with: - -```yaml -ci: - autoupdate_commit_msg: "chore(deps): update pre-commit hooks" - autofix_commit_msg: "style: pre-commit fixes" - autoupdate_schedule: "monthly" -``` - -The frequencies can be "weekly" (the default), "monthly", and "quarterly". - -If you prefer, you can use Dependabot, which will also auto-update, though it -doesn't run the checks. The config looks like this: - -```yaml -version: 2 -updates: - - package-ecosystem: "pre-commit" - directory: "/" - schedule: - interval: "monthly" - groups: - pre-commit: - patterns: - - "*" - cooldown: - default-days: 7 -``` - -## Format - -{% rr PC110 %} [Black](https://black.readthedocs.io/en/latest/) is a popular -auto-formatter from the Python Software Foundation. One of the main features of -Black is that it is "opinionated"; that is, it is almost completely -unconfigurable. Instead of allowing you to come up with your own format, it -enforces one on you. While I am quite sure you can come up with a better format, -having a single standard makes it possible to learn to read code very fast - you -can immediately see nested lists, matching brackets, etc. There also is a -faction of developers that dislikes all auto-formatting tools, but inside a -system like pre-commit, auto-formatters are ideal. They also speed up the -writing of code because you can ignore formatting your code when you write it. -By imposing a standard, all scientific Python developers can quickly read any -package's code. - -Also, properly formatted code has other benefits, such as if two developers make -the same change, they get the same formatting, and merge requests are easier. -The style choices in Black were explicitly made to optimize git diffs! - -There are a _few_ options, mostly to enable/disable certain files, remove string -normalization, and to change the line length, and those go in your -`pyproject.toml` file. - -{% tabs formatters %} {% tab ruff Ruff-format %} - -Ruff, the powerful Rust-based linter, has a formatter that is designed with the -help of some of the Black authors to look 99.9% like Black, but run 30x faster. -Here is the snippet to add the formatter to your `.pre-commit-config.yml` -(combine with the Ruff linter below): - -```yaml -- repo: https://github.com/astral-sh/ruff-pre-commit - rev: "v0.15.14" - hooks: - # id: ruff-check would go here if using both - - id: ruff-format -``` - -As you likely will be using Ruff if you follow this guide, the formatter is -recommended as well. - -{% details You can add a Ruff badge to your repo as well %} - -[![Code style: Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/format.json)](https://github.com/astral-sh/ruff) - -```md -[![Code style: Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/format.json)](https://github.com/astral-sh/ruff) -``` - -``` -.. image:: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/format.json - :target: https://github.com/astral-sh/ruff -``` - -{% enddetails %} - -{% endtab %} {% tab black Black %} - -Here is the snippet to add Black to your `.pre-commit-config.yml`: - -```yaml -- repo: https://github.com/psf/black-pre-commit-mirror - rev: "26.5.1" - hooks: - - id: black -``` - -{% details You can add a Black badge to your repo as well %} - -[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) - -```md -[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) -``` - -``` -.. image:: https://img.shields.io/badge/code%20style-black-000000.svg - :target: https://github.com/psf/black -``` - -{% enddetails %} - -{% endtab %} {% endtabs %} - -In _very_ specific situations, like when making a 2D array, you may want to -retain special formatting. After carefully deciding that it is a special use -case, you can use `# fmt: on` and `# fmt: off` around a code block to have it -keep custom formatting. _Always_ consider refactoring before you try this -option! Most of the time, you can find a way to make the Blacked code look -better by rewriting your code; factor out long unreadable portions into a -variable, avoid writing matrices as 1D lists, etc. - -{% details Documentation / README snippets support %} - -{% rr PC111 %} If you want Black used in your documentation, you can use -blacken-docs. This can even catch syntax errors in code snippets! It supports -markdown and restructured text. Note that because black is in -`additional_dependencies`, you'll have to keep it up to date manually. - -```yaml -- repo: https://github.com/adamchainz/blacken-docs - rev: "1.20.0" - hooks: - - id: blacken-docs - additional_dependencies: [black==24.*] -``` - -{% enddetails %} - -## Ruff - -{% rr PC190 %} [Ruff][] [(docs)][ruff docs] is a Python code linter and -autofixer that replaces many other tools in the ecosystem with a ultra-fast -(written in Rust), single zero-dependency package. All plugins are compiled in, -so you can't get new failures from plugins updating without updating your -pre-commit hook. - -[ruff docs]: https://beta.ruff.rs -[ruff]: https://github.com/astral-sh/ruff - -```yaml -- repo: https://github.com/astral-sh/ruff-pre-commit - rev: "v0.15.14" - hooks: - - id: ruff-check - args: ["--fix", "--show-fixes"] -``` - -{% rr PC192 %} The hook is named `ruff-check`. {% rr PC191 %} The `--fix` -argument is optional, but recommended, since you can inspect and undo changes in -git. `--show-fixes` is highly recommended if `--fix` is present, otherwise it -won't tell you what or why it fixed things. - -{% rr RF001 %} Ruff is configured in your `pyproject.toml`. Here's an example: - -{% tabs ruff-config %} {% tab ruff-simple Simple config %} - -```toml -[tool.ruff.lint] -extend-select = [ - "B", # flake8-bugbear - "I", # isort - "RUF", # Ruff-specific - "UP", # pyupgrade -] -``` - -{% endtab %} {% tab ruff-full Full config %} - -```toml -[tool.ruff.lint] -extend-select = [ - "ARG", # flake8-unused-arguments - "B", # flake8-bugbear - "BLE", # flake8-blind-except - "C4", # flake8-comprehensions - "DTZ", # flake8-datetimez - "EM", # flake8-errmsg - "EXE", # flake8-executable - "FA", # flake8-future-annotations - "FLY", # flynt - "FURB", # refurb - "G", # flake8-logging-format - "I", # isort - "ICN", # flake8-import-conventions - "ISC", # flake8-implicit-str-concat - "LOG", # flake8-logging - "NPY", # NumPy specific rules - "PD", # pandas-vet - "PERF", # perflint - "PGH", # pygrep-hooks - "PIE", # flake8-pie - "PL", # pylint - "PT", # flake8-pytest-style - "PTH", # flake8-use-pathlib - "PYI", # flake8-pyi - "Q", # flake8-quotes - "RET", # flake8-return - "RSE", # flake8-raise - "RUF", # Ruff-specific - "SIM", # flake8-simplify - "SLOT", # flake8-slots - "T10", # flake8-debugger - "T20", # flake8-print - "TC", # flake8-type-checking - "TRY", # tryceratops - "UP", # pyupgrade - "YTT", # flake8-2020 -] -ignore = [ - "PLR09", # Too many <...> - "PLR2004", # Magic value used in comparison -] -typing-modules = ["mypackage._compat.typing"] - -[tool.ruff.lint.per-file-ignores] -"tests/**" = ["T20"] -``` - -{% endtab %} {% tab ruff-ignore Ignore-based config %} - -```toml -[tool.ruff.lint] -select = ["ALL"] -ignore = [ - "ANN401", # Disallow Any - "PLC0415", # Import should be at top of file - "PLR09", # Too many ... - "PLR2004", # Magic value used in comparison - "PT013", # It's correct to import classes for typing! - "RUF009", # Too easy to get a false positive (function call in dataclass defaults) - "S", # Some subset is okay - "D", # Too many doc requests - "TD", # Todo format - "FIX", # Hacks and todos - "TID252", # Relative imports are fine - "COM", # Trailing commas teach the formatter - "A", # Okay to shadow builtins - "E501", # Line too long - "C90", # Complexity - "SLF001", # Private members are okay to access - "ERA", # Commented out code -] -typing-modules = ["mypackage._compat.typing"] - -[tool.ruff.lint.per-file-ignores] -"tests/**" = ["T20"] -``` - -{% endtab %} {% endtabs %} - -Ruff [provides dozens of rule sets](https://beta.ruff.rs/docs/rules/); you can -select what you want from these. Like Flake8, plugins match by whole letter -sequences (with the special exception of pylint's "PL" shortcut), then you can -also include leading or whole error codes. Codes starting with 9 must be -selected explicitly, with at least the letters followed by a 9. You can also -ignore certain error codes via `ignore`. You can also set codes per paths to -ignore in `per-file-ignores`. If you don't like certain auto-fixes, you can -disable auto-fixing for specific error codes via `unfixable`. - -There are other configuration options, such as `typing-modules`, which helps -apply typing-specific rules to a re-exported typing module (a common practice -for unifying typing and `typing_extensions` based on Python version). There's -also a file `exclude` set, which you can override if you are running this -entirely from pre-commit (default excludes include "build", so if you have a -`build` module or file named `build.py`, it would get skipped by default without -this). - -The `src` list which tells it where to look for top level packages is no longer -needed if just set to `["src"]` in Ruff 0.6+. {% rr RF003 %} - -{: .warning } - -> If you don't use a `[project]` table (older setuptools or Poetry), then you -> should also set: -> -> ```toml -> target-version = "py39" -> ``` -> -> This selects the minimum version you want to target (primarily for `"UP"` and -> `"I"`) {% rr RF002 %} - -Here are some good error codes to enable on most (but not all!) projects: - -- `E`, `F`, `W`: These are the standard flake8 checks, classic checks that have - stood the test of time. Not required if you use `extend-select` (`W` not - needed if you use a formatter) -- `B`: This finds patterns that are very bug-prone. {% rr RF101 %} -- `I`: This sorts your includes. There are multiple benefits, such as smaller - diffs, fewer conflicts, a way to auto-inject `__future__` imports, and easier - for readers to tell what's built-in, third-party, and local. It has a lot of - configuration options, but defaults to a Black-compatible style. - {% rr RF102 %} -- `ARG`: This looks for unused arguments. You might need to `# noqa: ARG001` - occasionally, but it's overall pretty useful. -- `C4`: This looks for places that could use comprehensions, and can autofix - them. -- `EM`: Very opinionated trick for error messages: it stops you from putting the - error string directly in the exception you are throwing, producing a cleaner - traceback without duplicating the error string. -- `ISC`: Checks for implicit string concatenation, which can help catch mistakes - with missing commas. (May collide with formatter) -- `PGH`: Checks for patterns, such as type ignores or noqa's without a specific - error code. -- `PL`: A set of four code groups that cover some (200 or so out of 600 rules) - of Pylint. -- `PT`: Helps tests follow best pytest practices. A few codes are not ideal, but - many are helpful. -- `PTH`: Want to move to using modern pathlib? This will help. There are some - cases where performance matters, but otherwise, pathlib is easier to read and - use. -- `RUF`: Codes specific to Ruff, including removing noqa's that aren't used. -- `T20`: Disallow `print` in your code (built on the assumption that it's a - common debugging tool). -- `UP`: Upgrade old Python syntax to your `target-version`. {% rr RF103 %} -- `FURB`: From the refurb tool, a collection of helpful cleanups. -- `PYI`: Typing related checks - -A few others small ones are included above, and there are even more available in -Ruff. You can use `ALL` to get them all, then ignore the ones you want to -ignore. New checks go into `--preview` before being activated in a minor -release. - -{% details You can add a Ruff badge to your repo as well %} - -[![Code style: Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json))](https://github.com/astral-sh/ruff) - -```md -[![Code style: Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json))](https://github.com/astral-sh/ruff) -``` - -```rst -.. image:: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json - :target: https://github.com/astral-sh/ruff -``` - -{% enddetails %} - -{% details Separate tools that Ruff replaces %} - -### PyCln - -[PyCln][] will clean up your imports if you have any that are not needed. There -is a Flake8 check for this, but it's usually nicer to automatically do the -cleanup instead of forcing a user to manually delete unneeded imports. If you -use the manual stage, it's opt-in instead of automatic. - -```yaml -- repo: https://github.com/hadialqattan/pycln - rev: "v2.6.0" - hooks: - - id: pycln - args: [--all] - stages: [manual] -``` - -### Flake8 - -[Flake8][] can check a collection of good practices for you, ranging from simple -style to things that might confuse or detract users, such as unused imports, -named values that are never used, mutable default arguments, and more. Unlike -black and some other tools, flake8 does not correct problems, it just reports -them. Some of the checks could have had automated fixes, sadly (which is why -Black is nice). Here is a suggested `.flake8` or `setup.cfg` to enable -compatibility with Black (flake8 does not support pyproject.toml configuration, -sadly): - -```ini -[flake8] -extend-ignore = E203, E501 -``` - -One recommended plugin for flake8 is `flake8-bugbear`, which catches many common -bugs. It is highly opinionated and can be made more so with the `B9` setting. -You can also set a max complexity, which bugs you when you have complex -functions that should be broken up. Here is an opinionated config: - -```ini -[flake8] -max-complexity = 12 -extend-select = B9 -extend-ignore = E203, E501, E722, B950 -``` - -(Error E722 is important, but it is identical to the activated B001.) Here is -the flake8 addition for pre-commit, with the `bugbear` plugin: - -```yaml -- repo: https://github.com/pycqa/flake8 - rev: "7.3.0" - hooks: - - id: flake8 - additional_dependencies: [flake8-bugbear] -``` - -This _will_ be too much at first, so you can disable or enable any test by it's -label. You can also disable a check or a list of checks inline with -`# noqa: X###` (where you list the check label(s)). Over time, you can fix and -enable more checks. A few interesting plugins: - -- [`flake8-bugbear`](https://pypi.org/project/flake8-bugbear/): Fantastic - checker that catches common situations that tend to create bugs. Codes: `B`, - `B9` -- [`flake8-docstrings`](https://pypi.org/project/flake8-docstrings/): Docstring - checker. `--docstring-convention=pep257` is default, `numpy` and `google` also - allowed. -- [`flake8-spellcheck`](https://pypi.org/project/flake8-spellcheck/): Spelling - checker. Code: `SC` -- [`flake8-import-order`](https://pypi.org/project/flake8-import-order/): - Enforces PEP8 grouped imports (you may prefer isort). Code: `I` -- [`pep8-naming`](https://pypi.org/project/pep8-naming/): Enforces PEP8 naming - rules. Code: `N` -- [`flake8-print`](https://pypi.org/project/pep8-naming/): Makes sure you don't - have print statements that sneak in. Code: `T` - -{% details Flake8-print details %} - -Having something verify you don't add a print statement by mistake is _very_ -useful. A common need for the print checker would be to add it to a single -directory (`src` if you are following the convention recommended). You can do -the next best thing by removing directories and file just for this check (`T`) -in your flake8 config: - -```ini -[flake8] -per-file-ignores = - tests/*: T - examples/*: T -``` - -{% enddetails %} - -### YesQA - -Over time, you can end up with extra "noqa" comments that are no longer needed. -This is a flake8 helper that removes those comments when they are no longer -required. - -```yaml -- repo: https://github.com/asottile/yesqa - rev: "v1.5.0" - hooks: - - id: yesqa -``` - -You need to have the same extra dependencies as flake8. In YAML, you can save -the list given to yesqa and repeat it in flake8 using `&flake8-dependencies` and -`*flake8-dependencies` after the colon. - -### isort - -You can have your imports sorted automatically by [isort][]. This will sort your -imports, and is black compatible. One reason to have sorted imports is to reduce -merge conflicts. Another is to clarify where imports come from - standard -library imports are in a group above third party imports, which are above local -imports. All this is configurable, as well. To use isort, the following -pre-commit config will work: - -[isort]: https://pycqa.github.io/isort/ - -```yaml -- repo: https://github.com/PyCQA/isort - rev: "8.0.1" - hooks: - - id: isort -``` - -In order to use it, you need to add some configuration. You can add it to -`pyproject.toml` or classic config files: - -```ini -[tool.isort] -profile = "black" -``` - -[isort]: https://pycqa.github.io/isort/ - -### PyUpgrade - -Another useful tool is [PyUpgrade][], which monitors your codebase for "old" -style syntax. Most useful to keep Python 2 outdated constructs out, it can even -do some code updates for different versions of Python 3, like adding f-strings -when clearly better (please always use them, they are faster) if you set -`--py36-plus` (for example). This is a recommended addition for any project. - -```yaml -- repo: https://github.com/asottile/pyupgrade - rev: "v3.21.2" - hooks: - - id: pyupgrade - args: ["--py39-plus"] -``` - -[pyupgrade]: https://github.com/asottile/pyupgrade - -{: .note } - -> If you set this to at least `--py37-plus`, you can add the annotations import -> by adding the following line to your isort pre-commit hook configuration: -> -> ```yaml -> args: ["-a", "from __future__ import annotations"] -> ``` -> -> Also make sure isort comes before pyupgrade. Now when you run pre-commit, it -> will clean up your annotations to 3.7+ style, too! - -{% enddetails %} - -## Type checking - -{% rr PC140 %} One of the most exciting advancements in Python in the last 10 -years has been static type hints. Scientific Python projects vary in the degree -to which they are type-hint ready. One of the challenges for providing static -type hints is that it was developed in the Python 3 era and it really shines in -a Python 3.7+ codebase (due to `from __future__ import annotations`, which turns -annotations into strings and allows you to use future Python features in Python -3.7+ annotations as long as your type checker supports them). For now, it is -recommended that you make an attempt to support type checking through your -public API in the best way that you can (based on your supported Python -versions). Stub files can be used instead for out-of-line typing. -[MyPy](https://mypy.readthedocs.io/en/stable/) is suggested for type checking, -though there are several other good options to try, as well. If you have -built-in support for type checking, you need to add empty `py.typed` files to -all packages/subpackages to indicate that you support it. - -Read more about type checking on the [dedicated page][mypy page]. - -The MyPy addition for pre-commit: - -```yaml -- repo: https://github.com/pre-commit/mirrors-mypy - rev: "v2.1.0" - hooks: - - id: mypy - files: src - args: [] -``` - -You should always specify args, as the hook's default hides issues - it's -designed to avoid configuration, but you should add configuration. You can also -add items to the virtual environment setup for MyPy by pre-commit, for example: - -```yaml -additional_dependencies: [attrs==23.1.0] -``` - -{% rr MY100 %} MyPy has a config section in `pyproject.toml` that looks like -this: - -```ini -[tool.mypy] -files = "src" -python_version = "3.10" -strict = true -enable_error_code = ["ignore-without-code", "redundant-expr", "truthy-bool"] -warn_unreachable = true - - -# You can disable imports or control per-module/file settings here -[[tool.mypy.overrides]] -module = [ "numpy.*", ] -ignore_missing_imports = true -``` - -There are a lot of options, and you can start with only typing global code and -functions with at least one type annotation (the default) and enable more checks -as you go (possibly by slowly uncommenting items in the list above). You can -ignore missing imports on libraries as shown above, one section each. And you -can disable MyPy on a line with `# type: ignore`. One strategy would be to -enable `check_untyped_defs` first, followed by `disallow_untyped_defs` then -`disallow_incomplete_defs`. You can add these _per file_ by adding a -`# mypy: